Just in Time Compiler (JIT) in Hotspot

What is JIT Compiler?

The Just In Time Compiler (JIT) concept and more generally adaptiveoptimization is well known concept in many languages besides Java (.Net, Lua,JRuby).

In order to explain what is JIT Compiler I want to start with a definition ofcompiler concept. According to wikipedia compiler is "a computer programthat transforms the source language into another computer language (the targetlanguage)".

We are all familiar with static java compiler (javac) that compileshuman readable .java files to a byte code that can beinterpreted by JVM - .class files. Then what does JIT compile? The answer willgiven a moment later after explanation of what is "Just inTime".

According to most researches, 80% of execution time is spent in executing 20%of code. That would be great if there was a way to determine those 20% of codeand to optimize them. That's exactly what JIT does - during runtime it gathersstatistics, finds the "hot" code compiles it from JVMinterpreted bytecode (that is stored in .class files) to a native code that isexecuted directly by Operating System and heavily optimizes it. Smallest compilation unit is single method. Compilation and statisticsgathering is done in parallel to program execution by special threads. Duringstatistics gathering the compiler makes hypotheses about code function andas the time passes tries to prove or to disprove them. If the hypothesisis dis-proven the code is deoptimized and recompiled again.  

The name "Hotspot" of Sun (Oracle) JVM is chosen because of theability of this Virtual Machine to find "hot" spots in code.

What optimizations does JIT?
Let's look closely at more optimizations done by JIT.

• Inline methods - instead of calling method on an instance of the object it copies the method to caller code. The hot methods should be located as close to the caller as possible to prevent any overhead. 
• Eliminate locks if monitor is not reachable from other threads
• Replace interface with direct method calls for method implemented only once to eliminate calling of virtual functions overhead
• Join adjacent synchronized blocks on the same object
• Eliminate dead code
• Drop memory write for non-volatile variables
• Remove prechecking NullPointerException and IndexOutOfBoundsException 
• Et cetera

When the Java VM invokes a Java method, it uses aninvoker method as specified in the method block of the loaded class object. TheJava VM has several invoker methods, for example, a different invoker is usedif the method is synchronized or if it is a native method.The JIT compiler usesits own invoker. Sun production releases check the method access bit forvalue ACC_MACHINE_COMPILED to notify the interpreter that the codefor this method has already been compiled and stored in the loaded class. JITcompiler compiles the method block into native code for this method and storesthat in the code block for that method. Once the code has been compiledthe ACC_MACHINE_COMPILED bit, which is used on the Sun platform, isset.How do we know what JIT is doing in our program and how can it becontrolled?

First of all to disable JIT Djava.compiler=NONE parameter can beused. 

There are 2 types of JIT compilers in Hotspot - one is used for client programand one for server (-server option in VM parameters). Program, running onserver enjoys usually from more resources than program running on client and toserver program top throughput is usually more important. Hence JIT inserver is more resource consuming and gathering statistics takes more time tomake the statistics more accurate. For client program gathering statics for amethod lasts 1500 method calls, for server 15000. These default values can bechanged by -XX:CompileThreshold=XXX VM parameter. 

In order to find out whether default value is good for you try enabling"XX:+PrintCompilation" and "-XX:-CITime" parameters thatprint JIT statistics and time CPU spent by JIT. 

Benchmarks

Most of the benchmarks show that JITed code runs 10 to 20 times fasterthan interpreted code. There are many benchmarks done. Below given resultgraphs of two of them:





Its worth to mention that programs that run in JIT mode, but are still in"learning mode" run much slower than non JITed programs.


Drawbacks of JIT


JIT Increases level of unpredictability and complexity in Java program. It addsanother layer that developers don't really understand. Example of possible bugs- 'happens before relations" in concurrency. JIT can easily reorder codeif the change is safe for a program running in single thread. To solve thisproblem developers make hints to JIT using "synchronized" word orexplicit locking.
Increases non heap memory footprint - JITed code is stored in "CodeCache" generation.  


Advanced JIT
JIT and garbage collection.

• For GC to occur program must reach safe points. For this purpose JIT injects yieldpoints at regular intervals in native code.
• In addition to scanning of stack to find root references, registers must be scanned as they may hold objects created by JIT

 

Comments are appresiated.

The article can be found also at:artiomg.blogspot.com/2011/10/just-in-time-compiler-jit-in-hotspot.html