Put @ConditionalOnProperty in my main source code.
Then define a test bean and add it as a part of the test configuration. Either import that configuration in the main test class or any other way. This is only needed when you need to register a test bean. If you do not need that bean then go to next step.
Add a property in application.properties under src/test/resources to disable the use of configuration class that is present in the main folder.