On Windows, I normally just use the binary installer, but I would like to install NumPy only in a virtualenv this time, so I created a virtual env:
virtualen
I didn't have to do the trick of unpacking the .exe file. I just activated my virtual environment, and then:
easy_install numpy