How does the JVM decided to JIT-compile a method (categorize a method as “hot”)?

Question

I already worked with -XX:+PrintCompilation, and I know the basic techniques of the JIT-compiler and why JIT-compilation is used.

Yet I still have not found out how the JVM decides to JIT-compile a method, i.e. "when the right time has come to JIT-compile a method".

Am I right in the assumption that every method starts being interpreted, and as long as it is not categorized as "hot method" it will not be compiled? I have something in the back of my head that I read that a method is considered "hot" when it was executed at least 10.000 times (after interpreting the method 10.000 times, it will be compiled), but I have to admit that I am not sure about this or where I've read this.

So to sum up my question:

(1) Is every method interpreted as long as it not has been categorized as "hot" method (and therefore has been compiled) or are there reasons for methods to get compiled even if they are not "hot"?

(2) How does the JVM categorize methods into "non-hot" and "hot" methods? Number of execution? Anything else?

(3) If there are certain thresholds (like number of executions) for "hot" methods, are there Java flags (-XX:...) to set this thresholds?

Answer 1

HotSpot compilation policy is rather complex, especially for Tiered Compilation, which is on by default in Java 8. It's neither a number of executions, nor a matter of CompileThreshold parameter.

The best explanation (apparently, the only reasonable explanation) can be found in HotSpot sources, see advancedThresholdPolicy.hpp.

I'll summarize the main points of this advanced compilation policy:

• Execution starts at tier 0 (interpreter).
• The main triggers for compilation are
  1. method invocation counter i;
  2. backedge counter b. Backward branches typically denote a loop in the code.

• Every time counters reach certain frequency value (TierXInvokeNotifyFreqLog, TierXBackedgeNotifyFreqLog), a compilation policy is called to decide what to do next with currently running method. Depending on the values of i, b and current load of C1 and C2 compiler threads it can be decided to

  • continue execution in interpreter;
  • start profiling in interpreter;
  • compile method with C1 at tier 3 with full profile data required for futher recompilation;
  • compile method with C1 at tier 2 with no profile but with possibility to recompile (unlikely);
  • finally compile method with C1 at tier 1 with no profile or counters (also unlikely).

  Key parameters here are TierXInvocationThreshold and TierXBackEdgeThreshold. Thresholds can be dynamically adjusted for a given method depending on the length of compilation queue.

• Compilation queue is not FIFO, but rather a priority queue.

• C1-compiled code with profile data (tier 3) behave similarly, except that thresholds for switching to the next level (C2, tier 4) are much bigger. E.g. an interpreted method can be compiled at tier 3 after about 200 invocations, while C1-compiled method is subject for recompilation at tier 4 after 5000+ invocations.

• A special policy is used for method inlining. Tiny methods can be inlined into the caller even if they are not "hot". A bit larger methods can be inlined only if they are invoked frequently (InlineFrequencyRatio, InlineFrequencyCount).

Answer 2

The main parameter to control this is -XX:CompileThreshold=10000

Hotspot for Java 8 now uses a tiered compilation by default using a number of stages of compilation from level 1 to 4. I believe 1 is no optimisation. Level 3 is C1 (based on the client client) and Level 4 is C2 (based on the server compiler)

This means that a little optimisation can happen earlier than you might expect and it can keep optimising long after it has reach the 10K threshold. The highest I have seen is escape analysis eliminating a StringBuilder after one million calls.

Note: a loop iterating many times can trigger the compiler. e.g. a loop of 10K times can be enough.

1) Until a method is considered hot enough, it is interpreted. However some JVMs (e.g. Azul Zing) can compile methods on start up and you can force the Hotspot JVM to compile a method via an internal API. Java 9 may also have an AOT (Ahead Of Time) compiler but it is still being researched AFAIK

2) Number of calls, or number of iterations.

3) Yes -XX:CompileThreshold= being the main one.