I\'m trying to register a custom protocol to an application on the Mac, i.e:
The important part of the linked page in Jay's answer is the entry in the Info.plist.
I think with Launch Services it will automatically open this app if it is the only one that can handle a particular URL scheme, else you'll need to use the trick that Charlie Martin describes.
I'm not sure what the defaults command that needs to be executed is, or if it is a launchctl command.