The assembler has one source file to look at. It has to build a memory image based on that one source file. So it starts with 0 for the code, and also for 0 with the data. The results go into an object file.
Now, the linker comes along and needs to combine many object files into an executable (or kernel image, or whatever). It can't leave everyone at 0, they would overlap. So it must relocate the addresses to assign each object file's worth of data to a unique range.
Then, then the image is loaded into virtual memory, the operating system may have good reasons to want to use an address other than '0' as the base virtual address. So, at runtime, it wants to relocate everything again.
Finally, if there are shared libraries involved, neither the assembler nor the linker knows where they will be in the running environment, and so the references to them must be relocated.
