Recently, I was doing an analysis/tuning on a Java application server installation in order to identify the bottlenecks and fix them. The most common action in such procedure (tuning) is to retrieve many Thread dumps, when system is on load. Please have in mind that heavy load (for some cases) may have side effects that they may lead us to wrong conclusions. So, a more “controlled” load is more preferable than a real heavy load.

When system is on load, you will notice that many Java threads are on RUNNABLE state, but they are not really running. They are waiting for “something“.

The most common reasons that cause threads to wait even they are in RUNNABLE state are the following:

1. Insufficient CPU Resources: When you have more running threads than virtual CPUs, then it is normal to have delays from context switching, kernel, OS jobs and other processes of system.
2. Insufficient RAM: If your RAM is not enough them your system will use swap and this always a problem.
3. I/O: When a thread is in a read() or write() call and waiting for data to write or read, then this Thread is at RUNNABLE state but it is not actually run.
4. Slow Network: This is related to #3, as much slower is a network, it will cause longer delays to the “running” thread(s) that are related to network actions.
5. Process Priority: Processes can have different priorities. If JVM process runs with low priority, then other processes will run more frequently in a CPU. You can this using tools like top (GNU Linux), prstat (Solaris), task manager (Windows).
6. Garbage Collection (GC): When GC is running, there are points (stop-the-world) where all threads of JVM (except GC threads) are freezing. At these points, GC is deleting the useless referenced objects and so freeing the available memory size of heap (but only this). We have to use a such strategy (like CMS or G1) that it will minimize the frequency and duration of stop-the-world points.

The only one reason that is completely caused by JVM is the last one (GC activity). All the other points are mostly depended on OS and hardware. Thus, we must always monitor the system (OS and hardware) too, not only the JVM.

You must have in mind that Java does not use/follow its own threading model. Also current JVM (Hotspot) uses native OS threads and thread scheduling is implemented by underlying OS.

Regards,
Adrianos Dadis.
—
Democracy Requires Free Software