I am currently building a similar system, and had as much luck as you did with this. Ended up using UIScrollView to provide a panorama-like effect, even though I was not able to wrap it around entirely successfully. I used the WWDC video from session 104 (2010) to help me optimize large image loading.
OpenGL ES may be a more elegant solution if you have the time/skill (I've only dabbled in OpenGL), but I certainly had no luck finding a panorama library anywhere (except for the deprecated panoramagl). It's certainly a doozy.
