how would you benchmark the performance of a function

Question

here's perhaps a more advanced question. if you have two functions that return a value

int F(int input1, int input2)
{
    int output;
    //some algorithm that assigns value to output//
    return output;
}

int D(int input1, int input2)
{
    int output;
    //another algorithm that assigns value to output//
    return output;
}

With the condition that F(a,b) == D(a,b) (both return the same value for the same inputs).

If you'd like to benchmark their performance, how would you do it? More precisely, how would you isolate the time it takes to perform F(a,b) or D(a,b) such that it does not reflect the time it takes for the other secondary operations in the benchmark setup?

Answer 1

One of the best available opensource solutions is google benchmark . You have to create simple wrappers around code you want to benchmark and link either statically or dynamically with the benchmark lib. It is often useful to have such micro benchmarks compiled near with your code. For inspiration see awesome presentation.

static void BM_F(benchmark::State& state) {
  const auto input1 = state.range_x();
  const auto input2 = state.range_y();

  while (state.KeepRunning()) F(input1, input2);
}

static void BM_D(benchmark::State& state) {
  const auto input1 = state.range_x();
  const auto input2 = state.range_y();

  while (state.KeepRunning()) D(input1, input2);
}

BENCHMARK(BM_F)
    ->ArgPair(1, 10)
    ->ArgPair(10, 100)
    ->ArgPair(100, 1000);

BENCHMARK(BM_D)
    ->ArgPair(1, 10)
    ->ArgPair(10, 100)
    ->ArgPair(100, 1000);

If you want to measure raw CPU cycles, then your only choice is to use direct CPU instructions. For x86 you can use Time Stamp Counter. But you should be aware, that such measuring will not resist any context switches performed by OS or jumping on CPUs. Your only choice in such situation will be to use algo with single flow of execution, remember ID of CPU and TSC value before enter to test function, and check ID of CPU after test function. Then calculating difference between TSC values. You may also setup CPU affinity for your process to stick process to specific CPU.

Another Linux specific possible way to benchmark functions is to use perf tool.

But in any way, any measure will add some error level to the result.