Below is my build script (not using xcodebuild plugin).
This might also be caused by a default timeout on the keychain.
Check out my answer to "User interaction is not allowed" trying to sign an OSX app using codesign