The Best Kept Secret in the JDK: VisualVM

It's amazing the things that are right in front of you that you don't realise. VisualVM is probably the best example of this in the Java community. Everytime you go and update your JDK, you go straight to your IDE and capitalise on the new features provided. At least, that's what I thought I was doing.

I first read about VisualVM in an article Geertjan Wielenga wrote back in February of last year when the tool had it's third milestone release.  VisualVM could be the most important tool for Java developers, as well as being the best kept secret.  In a time where we're still trying to prove that Java isn't slow, this is the tool to help us keep the promise of performant Java applications.

VisualVM In A Nutshell 

VisualVM is a visual tool that integrates some of the commandline JDK tools and gives you powerful profiling capabilities. All for free! The commandline tools that it bundles up include jstat, JConsole, jstack, jmap and jinfo, which you get with your standard JDK distribution.

VisualVM will allow you to generate and analyze heap data, track down memory leaks, monitor the garbage collector and perform memory and CPU profiling, as well as being able to browse and operate on MBeans. While VisualVM runs on JDK6 it can monitor applications created since JDK1.4. 

For me, these features are critical. Up to now I've been using commercial tools like YourKit to run performance checks on my applications, but now I can get equivalent functionality from an open source tool. And who knows the JVM better than Sun?

How To Get VisualVM

The best part of VisualVM is that it's in your JDK bin directory already, if you've got any version of JDK1.6 since update 7. To run it, just click on the jvisualvm.exe icon.

All you need to do is click on the jvisualvm.exe and the application starts up. All Java applications running will be displayed on the right hand side navigation bar. Note that there is no need to register your application with VisualVM - it'll appear automatically. You can also view remote applications in this navigator.

That unknown application (pid 5392) is Eclipse. 

Using VisualVM 

The list of features available is pretty impressive. VisualVM allows you to: 

• Monitor application performance and memory consumption
• Monitor application threads
• Profile application performance and analyze memory allocation
• Take thread or heap dumps
• Analyze core dumps 
• Save snapshots to analyze the application offline.

You'll find instructions on how all of these features work in our previous post on the topic, as well as in the VisualVM getting started documentation.

An IDE Comparison with VisualVM 

I wanted to do something different in my use of VisualVM, so I decided that I would do a comparison of the three leading IDEs. I did a fresh install of NetBeans 6.5.1 (JavaSE only version), Eclipse (for Java Developers) and IntelliJ IDEA (8.1.2).

The test is as follows: 

• Run the Memory Profiler in VisualVM
• Create a new project in the IDE
• Create a simple class with a main method printing Hello World
• Save the Memory Profiler results
• Get a Heap Dump of the application
• Get information from the Monitor view in VisualVM
• Restart the IDE and using the CPU Profiler, perform the same steps in the IDE
• Save CPU Profiler results

I realise that both the CPU and Memory Profiling could have been done simultaneously, but for the purpose of this article I wanted to keep them seperate. 

When I ran the IntelliJ tests I had to start it up with my own JDK rather than the IntelliJ one. I'm unfamiliar with the IDE, but I noticed it had to parse the files in the JDK when I created a new project with the JSDK.

Also, it's fair to note that I've just downloaded the default distributions of these IDEs, without trimming them down to the minimum for Java development.

Heap Dump

I found the best means for comparison across the applications was the Heap Dump. This is really easy to do - at any stage during the application run, right click on the application in VisualVM and select Heap Dump.

The criteria I will compare here is the total bytes and total classes.

IntelliJ :

• Total Bytes:   76,582,715   
• Total Classes: 14,368

Eclipse:

• Total Bytes:   62,112,960
• Total Classes: 11,428

NetBeans:

• Total Bytes:   31,576,772
• Total Classes: 10,583

It's clear here the NetBeans is the one that'll use up less of the heap using half the bytes that Eclipse needs. IntelliJ seems to need a lot more.

The Monitor View is also interesting to look at as your application is running. Here I simply looked at the threaded local classes. Eclipse almost hits 25K, IntelliJ gets closer to 27K, while NetBeans stays below 20K

Memory & CPU Profile Results

I noticed that when I started profiling, that I needed to wait for instrumentation to be performed on the classes and methods. For the IDEs this took a significant, but not unreasonable, amount of time.
Memory and CPU profiling isn't that useful for a comparison of products. Where profiling comes in handy is when you need to find where leaks exist in your application. Nevertheless, here are the results of the profiling for the IDEs.

It's interesting to look at the common objects between the IDEs. For example, byte[] is heavily utilized in NetBeans, but to a lesser extent in both Eclipse and IntelliJ. We do see a significant use of char[] on all IDE's

• NetBeans : 19.6% of memory
• Eclipse  : 31.6% of memory 
• IntelliJ : 32.7% of memory

The CPU profiling shows where the majority of time was spent in execution. There's a big difference in the methods  that the IDEs spend there time on here

• NetBeans spends most time on org.openide.util.RequestProcessor$Processor.run() - 58.1%
• Eclipse spends it's time on java.lang.reflect.Method.invoke() - 45.5%
• IntelliJ uses java.net.SocketInputStream.read() 38.7% of the time
  

As a comparison, this tells us very little. But it is interesting to see the difference in how the IDEs work behind the scenes.

I must mention how easy this comparison was to do with VisualVM. I started it up before any of the IDEs and used the same instance all the way through, with the VisualVM registering each instance of the IDE automatically. 

IDE Integration 

Java developers are spoilt for choice when it comes to IDE's. It can be uncomfortable to step out of them to do any programming tasks - we've gotten used to it. Thankfully, there are plugin's available for all the popular IDEs, so you can work with VisualVM from the comfort of your favourite development environment, ignoring the command line. 

NetBeans integration is an interesting topic when it comes to VisualVM. VisualVM is actually the profiler that comes as part of NetBeans by default. In it's standalone format, it is a NetBeans RCP application, which can be further extended with other Netbeans modules. 

Eclipse and IntelliJ IDEA integration is available here. MyEclipse 7.5 also comes bundled with VisualVM plugins. As an Eclipse developer, I was interested to see how this would work out - after all the tool is based on the NetBeans platform. 

Eclipse Integration

First you'll need to download the VisualVM Launcher plugin, and extract it to you Eclipse home directory. When you open the Run Configurations dialog, you will get a choice of application launchers. The default launcher to use can be changed in the Preferences page depending on what you're running.

In my own setup I have chosen to use the VisualVM Launcher for Debug and the standard Eclipse Application Launcher otherwise. You can change these settings for applications, applets or Eclipse applications.

You will also need to specify a path to the VisualVM executable that you wish to use, and a path to the JDK.

That's the point where Eclipse integration ends, as it launches up an instance of VisualVM automatically and you can work from there. Perhaps in the future there will be some way of hooking directly into my IDE instance and profiling from there. 

Extending VisualVM

If all the features listed above aren't enough for your own requirements, you can roll your own VisualVM plugins. There are a number of API entry points available, as illustrated in the diagram below. 

The main entry points available are:

• Tab extension. By default, VisualVM provides tabs with labels such as "Overview", "Monitor", and "Threads". You can create new tabs just like these. Optionally, your own tabs can be extendable so that others can plug subtabs into your tabs. Your tabs can either be available for all data sources in VisualVM or to specific types of data sources.
• Subtab extension. Within tabs such as those listed above, you can provide new subtabs. However, this is only possible if the tab has been defined to be "pluggable", which is explained below. The following tabs are all pluggable: "Overview", "Monitor", "Threads", "Heap Dump", and "Host". When you create a new subtab, you can specify its position in relation to the other subtabs within the tab. Subtabs can either be available for all data sources or for specific data sources.
• Menu extension. A data source, and its subnodes, can have one or more menu items added to their contextual menus.
• Application type extension. By default, all applications visualized in VisualVM are treated equally, that is, they all have the same icons and tabs, except if a plugin is provided to provide additional functionality. You might also want to provide functionality for a specific application, which means that you would need to define a new application type. You can do this to do something as simple as provide a distinguishing icon for a running instance of your application. Alternatively, you might want to provide a lot of functionality, including graphs, to show the processing of your specific application.
• Data source extension. The "Application" data source is only one of the types of data sources that can be monitored and managed in VisualVM. Another of the data sources is the "Host" data source, which lets you monitor and manage local and remote hosts. You might want to create new data sources, in situations where you want to monitor and manage a type of data source that VisualVM does not cater for by default.

There's a detailed page on the VisualVM site showing you how to create your own plugins for VisualVM.  It seems pretty straightforward, especially if you are familiar with the plugin development approaches in NetBeans or Eclipse.  And there are quite a few plugins available already through the VisualVM Plugins Center. To see these go to Tools/Plugins.

Considering how easy it is to get VisualVM (you have it already), and the extensible platform that it's built on, surely this will become one of those tools that you can't do without. I'm impressed with the way that it just picks up the applications that are running, instead of having to register with the profiler. In theory, you could have it running in the background all the time, while you perfect and relaunch your Java application.

It's something that you probably haven't taken much notice of before, but it's well worth taking it for a run and seeing if it helps you spot any weak performance areas in your application.