From a VM in GCE, I did the following
gcloud auth activate-service-account --key-file
# \"blah\" is a service account key file (JSON) I generate
Step 1 : gcloud init
Step 2 : Select [2] Create a new configuration
Step 3 : Enter configuration name. Names start with a lower case letter and contain only lower case letters a-z, digits 0-9, and hyphens '-': kubernetes-service-account
Step 4 : Choose the account you would like to use to perform operations for this configuration:[2] Log in with a new account
Step 5 : Do you want to continue (Y/n)? y
Step 6 : Copy paste the link to brwoser and login with the ID which is used to create your google Cloud Account
Step 7 : Copy the verification code provided by google after login and paste it in to the console.
Step 8 : Pick cloud project to use:
Step 9: Do you want to configure a default Compute Region and Zone? (Y/n)? y
Step 10 : Please enter numeric choice or text value (must exactly match list item): 8
Your Google Cloud SDK is configured and ready to use!
Once this is done, make sure the service account configured for the VM has permissions to do the required tasks.