So I got prompted to download the latest NuGet Package Manager in VS2010.
All previous updates (including this) has failed with:
VSIXInstaller.SignatureMismatchExc
To uninstall NuGet, you must first run Visual Studio as Administrator, then you will see the Uninstall option available.
Restart VS, but this time run it as your normal account. When you install NuGet going forward, you should be able to update to future versions without running as Administrator.
I believe this issue occurs due to NuGet being installed with MVC 3. I hope in the future we no longer bundle NuGet with other software.