While running
./configure --prefix=/mingw
on a MinGW/MSYS system for a library I had previously run
\'./configure --pref
If sudo make uninstall is unavailable:
In a Debian based system, instead of (or after*) doing make install you can run sudo checkinstall to make a .deb file that gets automatically installed. You can then remove it using the system package manager (e.g. apt/synaptic/aptitude/dpkg). Checkinstall also supports creating other types of package, e.g. RPM.
See also http://community.linuxmint.com/tutorial/view/162 and some basic checkinstall usage and debian checkinstall package.
*: If you're reading this after having installed with make install you can still follow the above instructions and do a dpkg -r $PACKAGE_NAME_YOU_CHOSEN afterwards.