Best practices for measuring the run-time complexity of a piece of code

Question

I have a gnarly piece of code whose time-efficiency I would like to measure. Since estimating this complexity from the code itself is hard, I want to place it in a loop and time

Answer 1

First things first, I don't know of an accepted, "scientific" way to scale repetitions and problem size to achieve faster, more accurate time-vs-size plots, so I cannot say anything on the matter.

Other than that, for a better measurement of time complexity I would suggest to measure the average execution time for a fixed size and compare it with the average execution time measured in the previous cycle. After that you increase the size of the input data and repeat the measurement.

This is similar to one of the methods used in Numerical Analysis to estimate errors of numerical methods. You just adapt it to estimate the average error in the execution time of the implementation of your algorithm.

So, to cut it short:

1. Pick a size for the input data.
2. Run the implementation of the algorithm.
3. Calculate the average execution time.
4. If you have a previous calculation of the average execution time then compare it with the current execution time, else repeat from point 2.
5. If difference is greater than or equal to a certain (previously defined) threshold quantity then repeat from point 2.
6. Increase the size of the input data.
7. Repeat from point 2.

Let me know if something is unclear.