Virtual Threads: A Definite Advantage

In order to prove this use case, we will create one million platform threads and virtual threads. After generating these threads, we will analyze their heap and thread behavior with HeapHero and fastThread tools. Through this exploration, we aim to highlight the distinctions in performance between platform threads and virtual threads.

What Are Virtual Threads(VT) In Java?

Virtual threads are a way of creating threads that are not tied to a specific operating system thread. This can be useful for applications that need to create a large number of threads, as it can reduce the overhead of creating and managing each thread. They are also useful for applications that need to create threads that are not temporary, as they can guarantee that each thread will have a chance to run.

This functionality was introduced in Java 21. Virtual threads are also known as "green threads" or "lightweight threads." It is a software implementation of threads that uses the operating system’s threads to achieve concurrency. They are managed by the Java Virtual Machine (JVM) and are not visible to the programmer.

Why Are Virtual Threads Special?

Virtual threads are a special type of thread that is created by platform threads and will take only a negligible amount of resources upon creation. Because of this functionality, it is possible to generate many virtual threads that can be used for multi-threaded programming.

Since creating a virtual thread is very cheap, it will not throw any error, unlike platform threads. Another advantage of virtual threads is that it is NOT required to pool them like platform threads in Java.

Platform Threads

Basically, platform threads are native threads(java.lang.Thread) that are part of JDK.

We are going to generate one million threads using the thread class in Java. During the course of creating these many huge threads, the operating system will become highly unstable and throw OutOfMemoryError. We are going to experiment with this behavior in Ubuntu Linux. The JDK version for this experiment is 21.

static int cnt = 1; 
public static void main(String[] args) { 
for(int i = 0; i<1000000; i++) {         
        new Thread(                 
              new Runnable() {                     
              @Override                     
              public void run() {                         
                 try {                             
                    TimeUnit.HOURS.sleep(1);                         
                } catch (Exception ex) {}                     }                 
             }         
         ).start();       
        cnt++;     
   } 

}

In the code above, we are creating one million threads inside a for loop and sleeping each thread for up to 1 hour. When a thread is sleeping, all the resources will be cached by the operating system. In this particular case, the OS needs to hold all the resources of each and every thread for a longer period of time, which is a very resource-intensive operation. Remember: creating a thread in Java is a very expensive operation. That is the reason it is required to pool threads during the application startups in the multi-threaded programming paradigm.

Soon the above code will throw an OutOfMemory error. You can see the error in the image below:

Fig 1: Image for OutOfMemoryError for platform threads

Virtual Threads

Now, let’s develop a code for a virtual thread. The use case is the same but we are going to dynamically generate one million virtual threads in a loop.

static int cnt = 1; 
public static void main(String[] args) {   
      var executor = Executors.newVirtualThreadPerTaskExecutor();   
      IntStream.range(1, 1000000).forEach(i ->{       
        Future result =  executor.submit(() ->{           
          try {               
              TimeUnit.HOURS.sleep(1);           
          } catch (InterruptedException e) {               
             throw new RuntimeException(e);           
          }           
          String uuid = UUID.randomUUID().toString();       
       });       
       if(cnt == 999999) {           
             generateHeapDump(2);       
       }       
       cnt++;   
   }); 
}

In the code above, one million virtual threads are created dynamically by calling newVirtualThreadPerTaskExecutor() of the ExecutorService class in the java.util.concurrent package.

During the execution of the code, we are going to take a snapshot of the heap memory using the generateHeapDump() method. Basically, we take the heap dump when the counter reaches 999999. This way we will make sure that maximum data is captured in the heap memory log.

What? No, OOM Error!!

What is an OutOfMemoryError? A system will throw this error when sufficient memory is not available for the application to process the transactions. Then why are we experiencing OutOfMemoryError in the case of platform threads and not in virtual threads?

We will understand more about it with the help of this diagram below:

Fig 2: JVM Memory snapshot

There are three sections of the memory – Heap, Metaspace, and Others. In the case of platform threads, the thread stacks are stored in the Others region.

Each thread has a separate memory and it is stored in the Others region. When a memory is allocated for the thread and after the process is finished, this memory should be released. In the platform thread scenario, the threads are waiting for one hour and the memory associated with them is not released quickly. Also, new threads are generated again and they also are waiting for one hour. This way a lot of memory needs to be allocated in the Others region and the JVM is not able to release the memory quickly. Hence, it throws this error. This video can explain JVM in 10 minutes.

In the case of virtual threads, the VTs are stored in a Heap region, which is controlled by the JVM process because they are considered as objects. When you run the code for the virtual thread scenario, you can see that it will create one million VTs and sleep for one hour. These virtual threads are saved inside the heap memory and the resource required to generate the thread is very minimal. Hence, it will NOT throw OutOfMemoryError in this case.

Note: Sometimes, in the case of virtual threads, it may also throw OutOfMemoryError. This is because the "heap memory" is going to be exhausted when a huge number of virtual threads are created. But in the above case, it will not throw an OOM error because the default memory is more than enough to hold the one million virtual threads!

We will confirm the above theory by analyzing the thread and memory behavior of platform threads and virtual threads.

Comparison of Platform Thread Performance

We are going to do a comparative study of the platform thread performance using fastThread and HeapHero toolsets by doing a thread and heap dump analysis respectively.

Thread Dump Analysis

This is the thread dump report generated by the fastthread.io for platform threads. The report is intelligent enough to provide the likelihood of an OutOfMemoryError.

Fig 3: Thread dump report for platform thread

It says that there are nearly 1,600 threads in the JVM, and these numbers are alarming. The first section of this report provides us with enough information about the status of the application.

Now let us examine the threads with the same stack trace quickly. Below is the diagram for that.

Fig 4: Threads with identical stack trace

This diagram shows the multiple threads with the same stack trace. These threads are in the WAITING stage. This is because the application has created a lot of threads and these are asked to wait for one hour (refer Thread.sleep(..) for platform thread code). There are around 1,600 threads that are asked to wait. Because of this reason, the report displays the stack trace with the same behavior. 

However, it is worth noting in the remaining section of the report. Here’s the link to the detailed report so that you can review and understand it better.

Heap Dump Analysis

Below is the heap dump analysis for the same platform threads using the heaphero.io.

Fig 5: Heap size of heap dump analysis

You can see here the heap size is much smaller. So we can say that in the case of platform threads, these numbers are very high. It is likely to create issues in the application. Here’s the troubleshooting report of the platform threads using the HeapHero toolset.

Comparison of Virtual Threads Performance

We are going to do a comparative study of the Virtual Thread performance using fastThread and HeapHero toolsets by doing a thread and heap dump analysis respectively. 

Thread Dump Analysis

Below is the thread dump analysis for the virtual threads. You can see that the number of threads is around 37. Why is this happening? Why is it not showing all these one million threads in the report?

Fig 6: Thread dump analysis for virtual thread

This is because virtual threads are not considered as threads, so while taking the thread dump, they are not included in the report. This thread dump intelligence report here will tell you what happens on the side is that the heap size would have grown up. 

Heap Dump Analysis

Now let us analyze the report for virtual threads using the HeapHero website. The generated report may be a bit bulky and you need to wait some time before a detailed report.

Fig 7: Heap dump analysis for Virtual Threads

First, take a look at the report and spend some time in it. This report says there are 999999 instances of java.lang.VirtualThreads. All these threads are referenced from one instance of jdk.internal.misc.CarrierThread.

Fig 8: Heap report for Virtual Thread

The interesting part of this report is that the size of the heap is 401 MB. When the code related to the virtual thread is executed, JVM will save all these one million virtual threads' information into the heap area. Hence, the size of the heap is very big in this case. This is the whole point here. This data is definitely eligible for garbage collection as well. Here is the heap analysis report highlighting the same.

Platform vs Virtual Thread Performance Comparison

Now let’s compare the thread counts vs heap size based on the below table:

When the code for platform threads runs, it is generating nearly 1600 threads before it throws OutOfMemoryError. But in that case, the size of the heap is relatively small. This is because, as it is mentioned in the earlier section of this article, the thread stack is saved inside the Others region, not inside the heap. 

In the case of virtual threads, the application creates a relatively very small number of threads, but the size of the heap is very high. This is because the virtual thread uses heap memory.

Conclusion

Virtual threads are a useful tool for creating multi-threaded applications. They can improve the performance of your application by using multiple threads to execute tasks in parallel. Virtual threads can be used the same way as a platform thread is used in a multi-threaded application. There is no overhead in creating and managing each thread, still yields a better result. This is a powerful functionality in the Java language and with the addition of this feature, it is very easy to scale the applications. This is a definite advantage of using virtual threads.