I have to compile a program on a current ubuntu (12.04). This program should then run on a cluster using CentOS with an older Kernel (2.6.18). I cannot compile on the cluste
See this answer.
Is this even the correct approach here
No: you can't use mismatched versions of glibc as your link command does. You used crt0.o and ld-linux.so from new (system-installed) libc, but libc.so.6 from an old (copied from cluster) libc. That is just not going to work.
-rpath sets the DT_RPATH tag but doesn't tell the linker to look there for libs, you want -L for that.