For Gmail or for your specific purpose: Gmail Contacts.
For Yahoo contacts, as far as I know, Contacts seems to be the one that is used. If someone has a better alternative. Please do mention.
For Twitter, I would highly recommend the Twitter gem.
For Facebook, you already got your Facebook gem sorted. However, I personally use FB Graph.
Edit:
Hmm, I tried having a look at the doc. No examples mentioned. Although it does mention:
See sample/authsub.rb for an example
which uses GmailContacts
Which is nowhere to be found. Perhaps a quick email to the author might help?
It also seems that the the Contacts gem is quite well documented online. If you can find a way to work it out with your project requirements. Then you can use this as a universal solution for other email providers.
Alternatively, another way for contacts retrieval, I found this which can be useful.
