Access Kubernetes GKE cluster outside of GKE cluster with client-go?

Question

• I have multiple kubernetes clusters running on GKE (let\'s say clusterA and clusterB)
• I want to access both of those clusters from client-go in an app that is

Answer 1

client-go needs to know about:

1. cluster master’s IP address
2. cluster’s CA certificate

(If you're using GKE, you can see these info in $HOME/.kube/config, populated by gcloud container clusters get-credentials command).

I recommend you to either:

1. Have a kubeconfig file that contains these info for clusters A & B
2. Use GKE API to retrieve these info for clusters A & B (example here) (You'll need a service account to do this, explained below.)

Once you can create a *rest.Config object in client-go, client-go will use the auth plugin that's specified in the kubeconfig file (or its in-memory equivalent you constructed). In gcp auth plugin, it knows how to retrieve a token.

Then, Create a Cloud IAM Service Account and give it "Container Developer" role. Download its key.

Now, you have two options:

Option 1: your program uses gcloud

gcloud auth activate-service-account --key-file=key.json
KUBECONFIG=a.yaml gcloud container clusters get-credentials clusterA
KUBECONFIG=b.yaml gcloud container clusters get-credentials clusterB

Then create 2 different *rest.Client objects, one created from a.yaml, another from b.yaml in your program.

Now your program will rely on gcloud binary to retrieve token every time your token expires (every 1 hour).

Option 2: use GOOGLE_APPLICATION_CREDENTIALS

1. Don't install gcloud to your program’s environment.
2. Set your key.json to GOOGLE_APPLICATION_CREDENTIALS environment variable for your program.
3. Figure out a way to get cluster IP/CA (explained above) so you can construct two different *rest.Config objects for cluster A & B.
4. Now your program will use the specified key file to get an access_token to Google API every time it expires (every 1h).

Hope this helps.

P.S. do not forget to import _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" in your Go program. This loads the gcp auth plugin!