Neither of your requirements are VPS-specific. I use shared hosting from Site5 and currently run two rails apps through the account, both with gems that are not installed on the server by default and sending email. Installing gems does not require root access.
If you want to use a VPS anyway, both Slicehost and Linode are often recommended for Rails apps. A few more are listed under Deploy on the Rails site. I would encourage you to do some research on your choice in either case. Good luck!
