Java virtual thread is a new type of threading model architecture introduced in JDK 19. Before you try to learn the benefits of Java Virtual Threads, you might want to understand how Java virtual thread works. Here is a post that gives you a quick introduction to Java virtual threads (we highly recommend you to read this introductory article before proceeding further).
Video: To see the visual walk-through of this post, click below:
Java Virtual Threads provide following major advantages:
1. Reduces application memory consumption
2. Improves application availability
3. Improves application throughput
4. Reduces ‘OutOfMemoryError: unable to create new native thread’
5. Improves code quality
6. 100% compatible with Platform Threads
Let’s review these advantages in detail in this post.
1. Reduces application memory consumption
Fig: Typical thread’s life cycle
1. Thread is created in a thread pool
2. Thread waits in the pool for a new request to come
3. Once the new request comes, the thread picks up the request and it makes a backend Database call to service this request.
4. Thread waits for response from the backend Database
5. Once response comes back from the Database, the thread processes it, and sends back the response to customer
6. Thread is returned back to the thread pool
Step #2 to #6 will repeat until the application is shutdown.
If you notice, the thread is actually doing real work only in step #3 and step #5. In all other steps(i.e., step #1, step #2, step #4, step #6), it is basically waiting(or doing nothing). Significant number of threads predominantly waits during most of its lifetime.
Fig: Each Platform Thread is mapped to an Operating System Thread
Whenever a platform thread is created, an operating system thread is allocated to it. Only when the platform thread exits(i.e., dies) the JVM, this operating system thread is free to do other tasks. Until then it cannot do any other tasks. Basically, there is a 1:1 mapping between a platform thread and an operating system thread.
According to this architecture, OS thread will be unnecessarily locked down in step #1, step #2, step #4, step #6 of the thread’s life cycle, even though it’s not doing anything during these steps. Even though OS threads are precious and finite resources, their time is extensively wasted in this platform threads’ architecture. Here is a study to show this memory wastage.
Fig: Virtual Threads are mapped to platform threads when it does real work
On the other hand, when virtual threads are waiting(i.e. step #1, step #2, step #4, step #6 ), they are stored in the java heap memory region just like any other object(aka stack chunk object). This stack chunk object doesn’t occupy much memory like an operating system thread. Only when a virtual thread needs to do real work, it’s tied up to the underlying operating system. Thus, virtual threads comparatively occupy much less memory than platform threads.
2. Improves application availability
Fig: Application server threads gets drained when backend slows down
One of the common problems in most applications(web, mobile, middleware, SOA, microservices, IOT…) is: threads waiting for the backend systems response. Let’s review a typical architecture. Each application has an application server(Apache Tomcat, JBoss, Resin, Weblogic, …) and each application server has a thread pool in the forefront. Whenever a new customer request comes in, a thread from this thread pool is taken out to service the new incoming request. Once the request is serviced, thread is returned back to the thread pool.
All modern applications talk with several backend systems(APIs, Databases, Main Frames, ….). Say suppose any one of the backend systems has an outage and starts to respond slowly, then the application server thread will get stuck waiting for response from the backend system. As more requests start to come to the application, more number of application threads will get stuck. On the other hand, there is only a finite number of threads in the application server thread pool. If all the threads end up stuck waiting for response from the backend system then there won’t be any threads available to service the new incoming request. In such circumstances, the entire application will become unavailable.
If we configure the application server thread pool as Java virtual thread, then we can even create millions of threads without any issue. When a virtual thread gets stuck waiting for a response from the backend system, it gets stored in the Java heap region just like any other application object(which is very lightweight). Thus the application server thread pool can continue to create virtual threads without any major issue until the backend system recovers. Hence, we will never run out of the threads in the application server thread pool. It has considerable potential to improve the application’s availability.
3. Improves application throughput
In most architectures, the number of requests that an application can process is directly proportional to the number of threads that are available in the application server thread pool. Because every single customer request is processed by a single unique thread. Thus, if a smaller number of threads are available, then only a small number of requests can be processed concurrently. This will degrade application’s throughput. On the other hand, if the application server thread pool is configured with Java virtual threads, it can create a significantly higher number(millions) of threads, which will ultimately improve application’s throughput .
Besides that, in certain applications, available threads in the application server thread pool gets saturated first before other compute resources(such as CPU, memory, network, storage) gets saturated. For such virtual threads would be a wonderful enhancement.
4. Reduces ‘OutOfMemoryError: unable to create new native thread’
Applications that run on JVM are prone to ‘java.lang.OutOfMemoryError: unable to create new native thread’. Here is a post that talks more about this type of OutOfMemoryError. This type of OutOfMemoryError happens under two circumstances:
a. When the application creates more threads than RAM capacity of the server(or container).
b. When the application creates more threads than Operating System permissible limit(Note: In OS, there is a kernel limit which dictates number of threads that can be created by an individual process).
Java virtual threads address these two issues very well. Java virtual threads are much light-weight than Platform threads, they will not occupy that much memory. Thus saturating RAM capacity is much harder to do with virtual threads than with platform threads.
Virtual threads don’t need to be assigned to an operating system thread unless they are doing real work. So, the chances for virtual threads application to trip over OS thread limit is much higher with platform threads than with virtual threads.
Note: Java virtual threads will not eliminate ‘java.lang.OutOfMemoryError: unable to create new native thread’. However it can reduce the occurrence of this error because of its inherent architecture.
5. Improves Code Quality
In order to conserve threads, few of the applications have adopted reactive, asynchronous programming models. This type of programming model uses the threads effectively. However, on the other hand this type of programming model complicates the code.
In this type of programming model, when an application needs to make a backend call, it will be made a unique thread. This thread will return immediately once making a backend call. It will not wait for the backend system’s response to come back. Once the backend system responds, a call-back function gets invoked. This call-back function is executed by a different thread and not from the same thread which initiated this operation. Thus code needs to be broken-up into different parts instead of keeping them sequentially. Code to make backend calls is going to be in one part of the application. Code to process the response from the backend is going to be in a different part of the program. Now, if you want to loop or put some conditional operations for the entire backend call it’s going to be tricker. It makes the code maintainability and readability complex. It also makes the debugging & troubleshooting tedious. On the other hand with Java virtual threads, you can create millions of threads (since threads are no longer bottleneck) and implement your entire backend call code sequentially. It will enhance the readability & maintainability of the code.
6. 100% compatible with Platform Threads
Virtual Threads are 100% compatible with Platform Threads APIs. Virtual threads support all features & APIs of Platform threads such as: thread-local variables, synchronized blocks, thread interruption… You only need to change your code(or configuration) on how you are launching the threads (i.e., instead of launching Platform threads, you need to launch Virtual threads). You may read this post ‘APIs to create Java virtual Thread’ post. All other parts of your application code can remain ‘as is’.
Leave a Reply