As of iOS 5, it appears that AVQueuePlayer no longer pre-buffers. It did pre-buffer the next track in iOS 4.
I'm not sure why there's been a change or if it's even deliberate on Apple's part or simply a bug. In any case, it's a pain and I'll be submitting a bug to them about it.