I have a lot of frameworks in my app. App works fine in adhoc/enterprise release. Only if I submit to the app store for testflight testing I get this error email from apple:
So I struggled on this for two days. What it turned out to be was I had UITests checked in Archive for the Build for the Scheme I was archiving.
After unchecking it from Archive, re-archiving it, validating it (although validating it before always passed), and "Upload to AppStore" I did not get the e-mail from Apple informing me of Invalid Swift Support. Instead I got the e-mail that it'd been processed and is good to go!