Java Performance Tuning, Profiling, and Memory Management

Java application performance is an abstract word until you face its real implications. It may vary depending on your interpretation of the word 'performance'. This article is meant to give the developer a perspective of the various aspects of the JVM internals, the controls, and switches that can be altered to optimal effects that suit your application. There is no single size that can fit all. You need to customize to suit your application.

You may be facing one of the issues listed below:

1. The dreaded java.lang.OutOfMemory Error
2. Your application is literally crawling.

Before we take the plunge into solving the issues, we first need to understand some of the theory behind the issues.

Theory

What does the JVM do? The Java Virtual Machine has two primary jobs:

1. Executes Code
2. Manages Memory
   This includes allocating memory from  the OS, managing Java allocation including heap compaction,
   and removal of garbaged objects 

Besides the above, the JVM also does stuff like managing monitors.

Very Basic Java Theory

An object is created in the heap and is garbage-collected after there are no more references to it. Objects cannot be reclaimed or freed by explicit language directives. Objects become garbage when they’re no longer reachable from the root set (e.g static objects)

Objects inside the blue square are reachable from the thread root set, while objects outside the square (in red) are not.

The sequence of the garbage collection process is as follows:

1. Root set tracing and figure out objects that are not referenced at all.

2. Put the garbage objects from above in finalizer Q

3. Run finalize() of each of these instances

4. Free memory

Infant mortality in Java

Most of the objects (80%) in a typical Java application die young. But this may not be  true for your application. Hence there is a need to figure out this rough infant mortality number so that you can tune the JVM accordingly.

Diagnosis:

There are some starting points to diagnose the problem.You may start with the '-verbose:gc' flag on the java command and see the memory footprint as the application progresses, till you find a spike. You may analyze the logs or use a light profiler like JConsole (part of JDK) to check the memory graph. If you need the details of the objects that are occupying the memory at a certain point, then you may use JProfiler or AppPerfect which can provide the details of each object instance and all the in/out bound  references to/from it. This is a memory intensive procedure and not meant for production systems. Depending upon your application, these heavy profilers can slow down the app upto 10 times.

Below are  some of the ways you can zero-in on the issue.

A) GC outputs

-verbose:gc   

This flag starts printing additional lines to the console, like given below

[GC 65620K -> 50747K(138432K), 0.0279446 secs]
[Full GC 46577K -> 18794K(126848K), 0.2040139 secs] 
Combined size of live objects before(young+tenured) GC -> Combined size of live objects(young+tenured)  after GC (Total heap size, not counting the space in the permanent generation
-XX:+PrintHeapAtGC : More details
•-XX:+PrintGCTimeStamps will additionally print a time stamp at the start of each collection.
111.042: [GC 111.042: [DefNew: 8128K->8128K(8128K), 0.0000505 secs]
111.042: [Tenured: 18154K->2311K(24576K), 0.1290354 secs]
26282K->2311K(32704K), 0.1293306 secs]
The collection starts about 111 seconds into the execution of the application. The tenured generation usage was reduced to about 10%
18154K->2311K(24576K)

B) hprof output file

java –Xrunhprof:heap=sites,cpu=samples,depth=10,thread=y,doe=y
The heap=sites tells the profiler to write information about memory utilization on the heap, indicating where it was allocated.
cpu=samples tells the profiler to do statistical sampling to determine CPU use.
depth=10 indicates the depth of the trace for threads.
thread=y tells the profiler to identify the threads in the stack traces.
doe=y tells the profiler to produce dump of profiling data on exit.

C) -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=C:\OOM.txt

Dump the heap on OOM, and then analyze the OOM.txt (Binary file) with jhat tool (bundled with JDK)

The command below will launch http server  @port 7777 . Open a browser with the URL 'http://localhost:7777'  to see the results.

jhat -port 7777 c:\OOM.txt

D) Profiling the app

You can profile the application to figure out Memory Leaks.

Java memory leaks(or what we like to call unintentionally retained objects), are often caused by saving an object reference in a class level collection and forgetting to remove it at the proper time. The collection might be storing 100 objects, out of which 95 might never be used. So in this case those 95 objects are creating the memory leak, since the GC cannot free them as they are referenced by the collection.

There are also other kinds of problems with managing resources that impact performance, such as not closing JDBC Statements/ResultSets in a finally block (many JDBC drivers store a Statement reference in the Connection object).

A java "memory leak" is more like holding a strong reference to an object though it would never be needed anymore. The fact that you hold a strong reference to an object prevents the GC from deallocating it.. Java "memory leaks" are objects that fall into category (2). Objects that are reachable but not "live" can be considered memory leaks.

JVMPI for Profiling applications  give a high level of detailing
Profilers: Hprof, JConsole, JProfiler, AppPerfect, YourKit, Eclipse Profiler, NetBeans Profiler ,JMP, Extensible Java Profiler (EJP), TomcatProbe, Profiler4j

JConsole is good for summary level info, tracking the memory footprint, checking Thread deadlocks etc. It does not provide details of the Heap object. For Heap details you may use AppPerfect (licensed) or JProfiler.

E) For NativeHeap issues.....

JRockit JDK (from BEA) provides better tools than the SUN JDK to peep inside the JNI Heap(atleast on Windows).

JRockt Runtime  Analyzer ...this  is part of the jrockit  install.
jrcmd PSID print_memusage
JRMC.exe  ...launch from /bin and  start  recording.

Try to get some Solution:

Based on the findings from the diagnosis, you may have to take these actions:

1. Code change - For memory leak issues, it has to be a code change.
2. JVM parameters tuning - You need to find the behavior of your app in terms of the ratio of young to old objects, and then tune the JVM accordingly. We ll talk abt when to tune a parameter as we discuss the relevant params below.
   
   Memory parameters:
   
   Memory Size:  overall size, individual region sizes
   
   -ms, -Xms  
   sets the initial heap size (young and tenured generation ONLY, NOT Permanent)
   
   If the app starts with a large memory footprint, then you should set the initial heap to a large value so that the JVM does not consume cycles to keep expanding the heap.
   
   -mx, -Xmx
   sets the maximum heap size(young and tenured gen ONLY,NOT Perm) (default:  64mb)
   
   This is the most frequently tuned parameter to suit the max memory requirements of the app. A low value  overworks the GC so that it frees space for new objects to be  created, and may lead to OOM. A very high value can starve other apps and induce swapping. Hence, Profile the memory requirements to select the right value.
   
   -XX:PermSize=256 -XX:MaxPermSize=256m
   
   MaxPermSize  default value (32mb for -client and 64mb for -server)
   Tune this to increase the Permanent gereration max size.
3. GC parameters:
   
   -Xminf [0-1], -XX:MinHeapFreeRatio [0-100]
   
   sets the percentage of minimum free heap space - controls heap expansion rate
   
   -Xmaxf [0-1], -XX:MaxHeapFreeRatio [0-100]
   
   sets the percentage of maximum free heap space - controls when the VM will return unused heap memory to the OS
   
   -XX:NewRatio
   
   sets the ratio of the old and new generations in the heap. A NewRatio of 5 sets the ratio of new to old at 1:5, making the new generation occupy 1/6th of the overall heap
   defaults: client 8, server 2
   
   -XX:SurvivorRatio
   
   sets the ratio of the survivor space to the eden in the new object area. A SurvivorRatio of 6 sets the ratio of the three spaces to 1:1:6, making each survivor space 1/8th of the new object region

Garbage Collector Tuning:

 Types of GarbageCollectors (not complete list)

1. Throughput collector: (default for Server JVM)
   •parallel version of the young generation collector.
   •-XX:+UseParallelGC
   •The tenured gc is the same as the serial collector (default GC for client JVM).
   •multiple threads to execute a minor collection
   •application has a large number of threads allocating objects / large Eden
   •-XX:+UseParallelOldGC (major also in parallel)
2. Concurrent low pause collector  :
   •collects the tenured generation and does most of the collection concurrently with the execution of the application. Attempts to reduce the pause times needed to collect the tenured generation
   •-Xincgc™ or -XX:+UseConcMarkSweepGC
   •The application is paused for short periods during the collection. A parallel version of the young generation copying collector is used with the concurrent collector.
   •Multiprocessor; apps that have a relatively large set of long-lived data (a large tenured generation;
   •Apps where response time is more important than overall throughput e.g. JAVA_OPTS= -Xms128M -Xmx1024M -XX:NewRatio=1 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:E:\loggc.txt
   
   FlipSide: Synchronization overhead, Fragmentation

Performance Solution

1. Application Software profiling
2. Server and JVM tuning
3. Right Hardware and OS
4. Code improvement as per the Behaviour of your application & profiling results….…. easier said than done
5. Use JVM the right way : optimal JVM params
6. Client / server application
7. •-XX:+UseParallelGC  if u have multiprocessors

Some Tips

• Unless you have problems with pauses, try granting as much memory as possible to the virtual machine
• Setting -Xms and -Xmx to the same value ….but be sure about the application behaviour
• Be sure to increase the memory as you increase the number of processors, since allocation can be parallelized
• Don’t forget to tune the Perm generation
• Minimize the use of synchronization
• Use multithreading only if it benefits. Be aware of the thread overheads. e.g a simple task like counter incrementing from 1 to billion ....use single thread. Multiple threads will ruin to mutiple of 10. I tested it out on dual CPU WinXP with 8 threads.
• Avoid premature object creation. Creation should be as close to the actual place of use as possible. Very basic concept that we  tend to overlook.
• JSPs are generally slower than servlets.
• Too many custom CLs, reflection : increase Perm generation. Don't be PermGen-agnostic.
• Soft References for memory leakages. They enable smart caches and yet do not load memory. GC will flush out SoftReferences automatically if the JVM runs low on memory.
• StringBuffer instead of String concat
• Minimize JNI calls in your code
• XML APIs – be careful …SAX or DOM- make correct choice. Use precompiled xpaths for better performance of the queries.

Conclusion:

There can be various bottlenecks for the entire application, and application JVM may be one of the culprits.There can be various reasons like JVM not  tuned optimally to suit your application, Memory leakages, JNI issues etc. They need to be diagnosed, analyzed and then fixed.