From a VM in GCE, I did the following
gcloud auth activate-service-account --key-file # \"blah\" is a service account key file (JSON) I generate
To use the Google Kubernetes Engine API from a GCE virtual machine you need to add the cloud platform scope ("https://www.googleapis.com/auth/cloud-platform") to your VM when it is created.