I have successfully configured my GitHub client to use SSH, and received the confirmatory email from GitHub that a new SSH key was added. I would like to setup IntelliJ to u
GitHub plugin for IntelliJ lets you to save the password, so you don't have to enter it every time.

(adapted from Multiple SSH Keys settings for different github account, thanks to CrazyCoder comment):
Create ssh key pair
$ ssh-keygen -t rsa -C "activehacker@youremail.com"
Add key
$ ~/.ssh/id_rsa_activehacker
Confirm that the key is added
$ ssh-add -l
Modify ~/.ssh/config
Host github.com-activehacker
HostName github.com
User git
IdentityFile ~/.ssh/id_rsa_activehacker
VCS > Checkout from Version Control > Git

Test

As you can see, you will still have to either enter passphrase for the key pair after every IntelliJ relaunch(I believe the passphrase is kept in memory) or let IntelliJ to store it permanently.