You can rebuild (and restore to a pristine state) all installed gems with:
gem pristine --all
--all --no-extensions will restore gems without extensions, but despite being documented, --extensions appears to have no effect (at least on rubygems 1.8.23 on Ubuntu 12.10).