I installed Git for Windows, although I am using the shell not the Windows interface.
If I do a git init, and then try and do a
git remote
I simply deleted the origin section in %appdata%/Local/GitHub/PortableGit_.../etc/gitconfig and everything went back to normal - new repos act accordingly when I add remote origin to them, since they have none when they are created.
There might be side effects for this deletion, but so far I haven't ran into any trouble.