How to use Caliper benchmark beta snapshot without maven?

问题

I have been asked to use Google's Caliper project to create a few microbenchmarks. I would very much like to use the annotation features of the newest beta snapshot, but aside from a few small examples I am having trouble finding good documentation on how to actually run the thing... There is a video tutorial up which instructs users on the new maven integration feature, which I was also asked NOT to use.

Right now I just have a small example stripped from one of theirs, modified with some other information I gleaned from another SO question:

public class Benchmarks {

    public class Test {
        @Param int size; // set automatically by framework

        private int[] array; // set by us, in setUp()

        @BeforeExperiment void setUp() {
          // @Param values are guaranteed to have been injected by now
          array = new int[size];
        }

        @Benchmark int timeArrayIteration(int reps) {
            int dummy = 0;
            for (int i = 0; i < reps; i++) {
                for (int doNotIgnoreMe : array) {
                    dummy += doNotIgnoreMe;
                }
            }
          return dummy;
        }

    }

    //(Questionable practice here?)
    public static void main (String args[]) {
        CaliperMain.main(Test.class, args); 
    }

}

Running it gives me the message that I did not set a default value for size. I am having trouble tracking down where I should be putting it.

Removing "size" entirely by commenting out the @Param line and giving a hard value to the array declaration in setUp just leads to it deciding that there are "No Experiments to be done," which makes sense, I suppose.

If there are any up-to-date resources or tutorials that could point out what I am doing wrong (probably a whole lot, honestly) I would be very appreciative.

EDIT:

I have updated to this as per some advice:

public class Benchmarks {
      @Param({"1", "10", "1000"}) int size; // set automatically by framework

  private int[] array; // set by us, in setUp()

  @BeforeExperiment void setUp() {
    // @Param values are guaranteed to have been injected by now
    array = new int[size];
  }

  @Benchmark int timeArrayIteration(int reps) {
    int dummy = 0;
    for (int i = 0; i < reps; i++) {
      for (int doNotIgnoreMe : array) {
        dummy += doNotIgnoreMe;
      }
    }
    return dummy;
  }
}

I am running through the beta snapshot and passing in the Benchmarks class as an argument. I receive the following:

Experiment selection: 
  Instruments:   []
  User parameters:   {size=[1, 10, 1000]}
  Virtual machines:  [default]
  Selection type:    Full cartesian product

There were no experiments to be performed for the class Benchmarks using the instruments [allocation, runtime]

It doesn't seem to be detecting any Instruments. I am not passing any in, as it's mentioned in the documentation that it simply uses default allocation, runtime (which is fine for my purposes).

DOUBLE EDIT: Found that problem, stupid mistake. Will do a quick write-up once I confirm it.

回答1:

Running it gives me the message that I did not set a default value for size.

Parameters are set either from default values:

@Param({"1", "10", "1000"}) int size;

Or by passing values via the the -D flag. E.g.: -Dsize=1,10,1000. Enums and booleans get special treatment in that it uses all possible values without having to list them in the annotation.

Removing "size" entirely by commenting out the @Param line and giving a hard value to the array declaration in setUp just leads to it deciding that there are "No Experiments to be done," which makes sense, I suppose.

The issue is likely that that your benchmark is an inner class and needs a reference to the enclosing class (though this should have been an error). Either make your benchmark class a top-level class (recommended) or make it static.

Also, there is no particular need to include the main method. Invoking CaliperMain with your benchmark class as the first parameter is equivalent.

回答2:

Running it gives me the message that I did not set a default value for size.

That's very simple:

@Param({"1", "10", "1000"}) int size;

Removing "size" entirely by commenting out the @Param line and giving a hard value to the array declaration in setUp just leads to it deciding that there are "No Experiments to be done," which makes sense, I suppose.

No, it doesn't. Without any params, each benchmark method is to be run exactly once. See the other answer for the solution.

There's a quite some Javadoc, e.g., on @Param. Actually, not so much has changed. Annotations have replaced conventions (now you don't need the time prefix), params stayed the same, setup uses an annotation instead of inheritance.

来源：https://stackoverflow.com/questions/21193140/how-to-use-caliper-benchmark-beta-snapshot-without-maven

标签

java

microbenchmark

Caliper