Benchmarking: using `expression` `quote` or neither

Question

Generally, when I run benchmarks, I wrap my statements in expression. Recently, it was suggested to either (a) not do so or (b) use quote instead

Answer 1

Your issue is that quote does not produce an expression but a call, so within the call to benchmark, there is no expression to evaluate.

If you evaluate the `call it will actually get evaluated, and the timings are reasonable.

class(quot)
[1] "call"
>class(expr)
[1] "expression"


 benchmark(raw=apply(mat, 2, mean), expr, eval(quot))[, -(7:8)]
        test replications elapsed relative user.self sys.self
3 eval(quot)          100    0.76    1.000      0.77        0
2       expr          100    0.83    1.092      0.83        0
1        raw          100    0.78    1.026      0.78        0

In general, I tend to create a function that contains the call / process I wish to benchmark. Note that it is good practice to include things like assigning the result to a value.

eg

 raw <- function() {x <- apply(mat, 2, mean)}

In which case it looks like that there is a slight improvement by eval(quote(...)).

benchmark(raw(), eval(quote(raw()))

                test replications elapsed relative user.self sys.self 
2 eval(quote(raw()))          100    0.76    1.000      0.75     0.01        
1              raw()          100    0.80    1.053      0.80     0.00        

But often these small differences can be due to overheads in functions and may not reflect how the performance scales to larger problems. See the many questions with benchmarkings of data.table solutions, using a small number of replications but big data may better reflect performance.