As per the comment under this answer, references were introduced primarily to support operator overloading which quotes Bjarne Stroustrup:
References
With the plus operator overloaded for a heavy class, you would have to write either a + b (by-value, inefficient) or &a + &b (ugly and complicated) to add two objects of this class. But with references you get by-ref even when writing a + b.
a + b
&a + &b