Benchmarking code - am I doing it right?

Question

I want to benchmark a C/C++ code. I want to measure cpu time, wall time and cycles/byte. I wrote some mesurement functions but have a problem with cycles/byte.

To ge

Answer 1

Your cyclecount benchmark is flawed as it includes the cost for walltime/cputime function calls. In general though, I strongly urge you to use a proper profiler instead of trying to reinvent the wheel. Especially performance counters will give you numbers that you can rely on. Also note that cycles are very unreliable as the CPU is usually not running at a fixed frequency or the kernel may do a task switch and halt your app for some time.

I personally write benchmarks such that they run a given function N times, for N being large enough such that you get enough samples. Externally then I apply a profiler such as linux perf to get me some hard numbers to reason about. Repeating the benchmark a given time you can then calculate stddev/avg values, which you can do in a script that runs the benchmark a few times and evaluates the output of the profiler.