It seems that ruby 2.0.0 has added \"default\" gems to the mix and makes them non removable by gem uninstall.
How can you remove all non default gems?
I used this one line script.
for i in `gem list --no-versions`; do gem uninstall -aIx $i; done
It ignores default gem errors and just proceeds. Simple and self-evident.