Quite often the scrolling can occur before the page has finished loading. If you have images without widths and heights, the page will jump, then load the image and re-layout itself, making the place you previously jumped to seem wrong.
Edit: Anything else that can change page layout should also be considered with suspicion... this include javascript and CSS that's not loaded in the (never mind that all CSS should be loaded in the head; it isn't always).
If the page is bounced through a redirect, I believe IE will scroll the end page but Firefox won't.