I have a project which uses cmake, one target is set to only build with MSVC:
if (MSVC)
add_library(test SHARED source.cpp)
endif()
Just encountered a similar issue myself and discovered CMAKE_VS_PLATFORM_NAME, which is calculated based on the generator and I think it's worth a mention.
I've tried it with cmake 3.16.1 (According to the docs it's been available since 3.1.3) using: -GVisual Studio 15 2017, -GVisual Studio 15 2017 Win64 and -GVisual Studio 15 2017 -Ax64 and it was set to: Win32, x64 and x64 respectively. So for this scenario I think checking against Win32 should work e.g.
if ("${CMAKE_VS_PLATFORM_NAME}" STREQUAL "Win32")
...
endif ()