Sorry that there are many similar questions, but I do find that Googling for CMake queries always yields similar-but-not-the-same scenarios, conflicting CMake commands and s
It sounds like you did not pass m32 to LFLAGS too, or there are old obj files skulking about. Be sure to clean first.
This question is similar to yours: cmake, gcc, cuda and -m32