You're almost there.
You can't target a repo directly but you can target one if its releases:
e.g. For me this worked:
meteor add cordova:https://github.com/EddyVerbruggen/Toast-PhoneGap-Plugin/archive/2.0.1.tar.gz@2.0.1
outputs:
added cordova plugin https://github.com/EddyVer