It seems that ruby 2.0.0 has added \"default\" gems to the mix and makes them non removable by gem uninstall.
How can you remove all non default gems?
I wrote a script in ruby to remove all non default gems.
https://gist.github.com/nixpulvis/5042764
This is needed now because unlike before 2.0.0 some gems are labeled "default" with the installation of ruby, and cannot be uninstalled with gem uninstall. This makes the previously popular methods for deleting all gems not work.
For reference here it is.
gem list | cut -d" " -f1 | xargs gem uninstall