While running
./configure --prefix=/mingw
on a MinGW/MSYS system for a library I had previously run
\'./configure --pref
make clean generally only cleans built files in the directory containing the source code itself, and rarely touches any installed software.
Makefiles generally don't contain a target for uninstallation -- you usually have to do that yourself, by removing the files from the directory into which they were installed. For example, if you built a program and installed it (using make install) into /usr/local, you'd want to look through /usr/local/bin, /usr/local/libexec, /usr/local/share/man, etc., and remove the unwanted files. Sometimes a Makefile includes an uninstall target, but not always.
Of course, typically on a Linux system you install software using a package manager, which is capable of uninstalling software "automagically".