After using macdeployqt I sign my application to avoid Gatekeeper problems.
I can use codesign on all the frameworks and everything inside
According to Apple docs you need to sign version folders, not the framework itself. I've tried that and run into two issues. First of all, some of the Info.plist files of Qt frameworks have incorrect executable names (a _debug suffix). After fixing that I've managed to sign all the frameworks the "Apple way". However, after doing that I was unable to sign the main app and got errors about the Qt frameworks I've just signed.
So the weird but working solution is to sign framework folders instead. This works even with incorrect executable names in Info.plist files.