I am looking for a way to do a UIViewAnimationTransitionCurlUp or UIViewAnimationTransitionCurlDown transition on the iPhone but instead of top to
I don't think there is a way beyond writing a custom animation.
More importantly you probably shouldn't try to it. The curl up and curl down are part of the user interface grammar that tells the user that a view is being lifted up or put down over the existing view. It's supposed to be like a sticky note being put down and then removed. A left<->right curl will most likely be interpreted as the something like ripping a page out of a book. It will confuse users.
Whenever you find yourself trying to do something in the interface that the standard API doesn't do easily, you should ask yourself whether such a novel method will communicate something important to user and whether it is similar to the existing interface grammar. If not, then you shouldn't bother.
Unusual interfaces have an initial wow factor but they lead to frustration and errors in day-to-day use. They can also cause Apple to refuse your app.