C++ Standard 8.3.2/4 says:
There shall be no references to references, no arrays of references, and no pointers to references.
Because indexation into an array is actually defined in terms of an implicit conversion to a pointer, then pointer arithmetic. So to support this, you'd have to also support pointers to references, and define what pointer arithmetic means on them.