Pitfalls To Avoid When Switching To Virtual Threads

Java virtual thread is a new feature available from JDK 19. It has the potential to improve the application’s availability, throughput, and code quality on top of reducing memory consumption.

Video: To see the visual walk-through of this post, click below:

In this post, let’s learn the pitfalls one should avoid when switching from Java platform threads to virtual threads:

1. Avoid synchronized blocks/methods

2. Avoid Thread pools to limit resource access

3. Reduce ThreadLocal usage

Let’s review these pitfalls in detail.

1. Avoid Synchronized Blocks/Methods

 When a method is synchronized in Java, only one thread would be allowed to enter that method at a time. Let’s consider the below example:

1: public synchronized void getData() { 
2: 
3:  makeDBCall(); 
4: }

In the above code snippet, getData() method is synchronized. Let’s say thread-1 attempts to enter the getData() method first. While this thread-1 is executing the getData() method, thread-2 attempts to execute this method. Since thread-1 is currently executing the getData() method, thread-2 will not be allowed to execute. It will be put in the BLOCKED state. If you are using a virtual thread in this scenario, when the thread is moved to BLOCKED state, ideally, it should relinquish its control of the underlying OS thread and move back to the heap memory. However, due to the limitation of the current virtual thread implementation, when a virtual thread gets BLOCKED because of synchronized method (or block), it will not relinquish its control over the underlying OS thread. 

In this circumstance, you should consider replacing synchronized methods/blocks with ‘ReentrantLock.' The above example code synchronized getData() method can be rewritten like this using ReentrantLock:

1: private ReentrantLock myLock = new ReentrantLock(); 
2:  
3: public void getData() { 
4: 
5:   myLock.lock(); // acquire lock 
6:   try { 
7: 
8:      makeDBCall(); 
9:   } finally { 
10: 
11:      myLock.unlock(); // release lock 
12:   } 
13: }

When you replace the synchronized method with ReentrantLock, the virtual thread will relinquish control of the underlying OS thread, and you can enjoy the benefits of virtual threads.

Note: Virtual thread not releasing the underlying operating system thread when working on a synchronized method is the current limitation in JDK 19. It could be addressed in the future release of Java.

2. Avoid Thread Pools To Limit Resource Access

Sometimes, in our programming constructs, we might have used a thread pool to limit access to certain resources. Say, suppose we want to make only 10 concurrent calls to a backend system; it might have been programmed using a thread pool as shown below:

1: private ExecutorService BACKEND_THREAD_POOL = Executors.newFixedThreadPool(10); 
2:  
3: public <T> Future<T> queryBackend(Callable<T> query) { 
4:  
5:   return BACKEND_THREAD_POOL.submit(query); 
6: }

In line #1, you can notice a BACKEND_THREAD_POOL is created with 10 threads. This thread pool is used in the queryBackend() method to make backend calls. This thread pool will ensure that no more than 10 concurrent calls will be made to the backend system.

At the time of writing this post (Jan’ 2023), there is no API available in JDK to create an Executor (i.e., thread pool) with a fixed number of virtual threads. Here is the list of all the APIs to create virtual threads. When you are using Executor, you can create only an unlimited number of virtual threads. To address this problem, you can consider replacing Executor with Semaphore. In the above example,  queryBackend() method can be rewritten using Semaphore as shown below:

1:   private static Semaphore BACKEND_SEMAPHORE = new Semaphore(10); 
2: 
3:   public static <T> T queryBackend(Callable<T> query) throws Exception { 
4: 
5:      BACKEND_SEMAPHORE.acquire(); // allow only 10 concurrent calls 
6:      try { 
7:          
8:         return query.call(); 
9:      } finally { 
10:          
11:         BACKEND_SEMAPHORE.release(); 
12:      } 
13: }

If you notice in line #1, we are creating a BACKEND_SEMAPHORE with 10 permits. This semaphore will allow only 10 concurrent calls to the backend system. This is a good alternative to the Executor. 

3. Reduce ThreadLocal Usage

Few applications tend to use ThreadLocal variables. In a nutshell, Java ThreadLocal variables are created and stored as variables within the scope of a particular thread alone, and they cannot be accessed by other threads. If your application creates millions of virtual threads, and each virtual thread has its own ThreadLocal variable, then it can quickly consume java heap memory space. Thus, you want to be cautious of the size of data that is stored as ThreadLocal variables.

You might wonder why ThreadLocal variables are not problematic in platform threads. The difference is that in platform threads, we don’t create millions of threads, whereas, in virtual threads, we do. Millions of threads, each with its own copy of ThreadLocal variables, can quickly fill up memory. They say small drops of water make an ocean. It’s very true here.

In general, Java ThreadLocal variables are tricky to manage & maintain. It can also cause nasty production problems. Thus, limiting the scope of use of ThreadLocal variables can benefit your application, especially when using virtual threads.