Before Xcode 6, it was sufficient to provide only a *.p12 certificate and *.mobileprovision file to Xcode in order to export an *.ipa file for Ad Hoc builds.
Xcode 6 ope
Download a previous version of Xcode, I just got the version 4.6.3 and it worked fine so far.
https://developer.apple.com/downloads/index.action