Suppose I have a function that takes an argument of type T.
It does not mutate it, so I have the choice of passing it by const reference const T&
I believe I would choose to pass by value whenever possible (that is: when the semantics dictate that I do not need the actual object to work on). I would trust the compiler to perform the appropriate moves and copy-elision.
After my code is semantically correct, I would profile it to see if I am making any unnecessary copies; I would modify those accordingly.
I believe that this approach would help me focus on the most important part of my software: correctness. And I would not get on the way of the compiler---interfere; inhibit---to perform optimizations (I know I cannot beat it).
Having said that, nominally references are implemented as pointers. So in a vaccum, without considering semantics, copy-elisions, move semantics, and stuff like that, it would be more "efficient" to pass by pointer/reference anything whose size is larger than the pointer.