Java Performance Testing [duplicate]

Answer 1

Japex may be useful to you, either as a way to quickly create benchmarks, or as a way to study benchmarking issues in Java through the source code.

Answer 2

Keep in mind that the resolution of System.currentTimeMillis() varies between different operating systems. I believe Windows is around 15 msec. So if your doSomething() runs faster than the time resolution, you'll get a delta of 0. You could run doSomething() in a loop multiple times, but then the JVM may optimize it.

Answer 3

I'd imagine you'd want to doSomething() before you start timing too, so that the code is JITted and "warmed up".

Answer 4

Have you looked at the profiling tools in netbeans and eclipse. These tools give you a better handle on what is REALLY taking up all the time in your code. I have found problems that I did not realize by using these tools.

Answer 5

The one flaw in that approach is that the "real" time doSomething() takes to execute can vary wildly depending on what other programs are running on the system and what its load is. This makes the performance measurement somewhat imprecise.

One more accurate way of tracking the time it takes to execute code, assuming the code is single-threaded, is to look at the CPU time consumed by the thread during the call. You can do this with the JMX classes; in particular, with ThreadMXBean. You can retrieve an instance of ThreadMXBean from java.lang.management.ManagementFactory, and, if your platform supports it (most do), use the getCurrentThreadCpuTime method in place of System.currentTimeMillis to do a similar test. Bear in mind that getCurrentThreadCpuTime reports time in nanoseconds, not milliseconds.

Here's a sample (Scala) method that could be used to perform a measurement:

def measureCpuTime(f: => Unit): java.time.Duration = {

  import java.lang.management.ManagementFactory.getThreadMXBean
  if (!getThreadMXBean.isThreadCpuTimeSupported)
    throw new UnsupportedOperationException(
      "JVM does not support measuring thread CPU-time")

  var finalCpuTime: Option[Long] = None
  val thread = new Thread {
    override def run(): Unit = {
      f
      finalCpuTime = Some(getThreadMXBean.getThreadCpuTime(
        Thread.currentThread.getId))
    }
  }
  thread.start()

  while (finalCpuTime.isEmpty && thread.isAlive) {
    Thread.sleep(100)
  }

  java.time.Duration.ofNanos(finalCpuTime.getOrElse {
    throw new Exception("Operation never returned, and the thread is dead " +
      "(perhaps an unhandled exception occurred)")
  })
}

(Feel free to translate the above to Java!)

This strategy isn't perfect, but it's less subject to variations in system load.

Answer 6

Using a Java Profiler is the best option and it will give you all the insight that you need into the code. viz Response Times, Thread CallTraces, Memory Utilisations, etc

I will suggest you JENSOR, an open source Java Profiler, for its ease-of-use and no overheads on CPU. You can download it, instrument the code and will get all the info you need about your code.

You can download it from: http://jensor.sourceforge.net/