The original .gitmodules file uses the hard coded https urls but for some automated tests I clone from ssh and make the submodule urls
If you want to modify the URL used for a submodule only for the local repo, then don't modify the .gitmodules file, that is only for changes that you want to push.
Instead, first initialize the local submodule config:
git submodule init
Then modify the .git/config file to change the submodule URL as usual. (Again, no need to modify .gitmodules here, and if this is a new clone you probably won't have .git/modules yet.)
As @jthill points out, an easier way to modify the submodule URLs is:
git config submodule.moduleName.url ssh://user@server/path
At this point you don't want to run git submodule sync, because that will overwrite the changes you just made with the values from the .gitmodules file. Instead, go straight to:
git submodule update --recursive --remote