How to specify static IP address for Kubernetes load balancer?

Question

I have a Kubernetes cluster running on Google Compute Engine and I would like to assign static IP addresses to my external services (type: LoadBalancer). I am unsure about whether this is possible at the moment or not. I found the following sources on that topic:

• Kubernetes Service Documentation lets you define an external IP address, but it fails with cannot unmarshal object into Go value of type []v1.LoadBalancerIngress
• The publicIPs field seems to let me specify external IPs, but it doesn't seem to work either
• This Github issue states that what I'm trying to do is not supported yet, but will be in Kubernetes v1.1
• The clusterIP field also lets me specify an IP address, but fails with "provided IP is not in the valid range"

I feel like the usage of static IPs is quite important when setting up web services. Am I missing something here? I'd be very grateful if somebody could enlighten me here!

EDIT: For clarification: I am not using Container Engine, I set up a cluster myself using the official installation instructions for Compute Engine. All IP addresses associated with my k8s services are marked as "ephemeral", which means recreating a kubernetes service may lead to a different external IP address (which is why I need them to be static).

Answer 1

Kubernetes v1.1 will make a few changes.

First, all load-balancers in GCE will get static IPs. This allows us to simulate "update" operations that GCE does not support.

Second, https://github.com/kubernetes/kubernetes/pull/13005 proposes a new field to explicitly set the IP of a load balancer.

Note though that your "ephemeral" IP is yours as long as your Service exists. This is roughly akin to what AWS does with ELB names (randomly assigned, yours until you release it).

publicIPs (or deprecatedPublicIPs in v1) will be replaced with externalIPs with very similar semantics. These are "unmanaged" IPs - kubernetes will not establish a load-balancer using them, but it will accept traffic for them.

clusterIP is an in-cluster address and generally is not available outside of the cluster or "project" or VPC (in GCE or AWS terms)

Answer 2

TL;DR Google Container Engine running Kubernetes v1.1 supports loadBalancerIP just mark the auto-assigned IP as static first.

Kubernetes v1.1 supports externalIPs:

apiVersion: v1
kind: Service
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.10.10
  ...

So far there isn't a really good consistent documentation on how to use it on GCE. What is sure is that this IP must first be one of your pre-allocated static IPs.

The cross-region load balancing documentation is mostly for Compute Engine and not Kubernetes/Container Engine, but it's still useful especially the part "Configure the load balancing service".

If you just create a Kubernetes LoadBalancer on GCE, it will create a network Compute Engine > Network > Network load balancing > Forwarding Rule pointing to a target pool made of your machines on your cluster (normally only those running the Pods matching the service selector). It looks like deleting a namespace doesn't nicely clean-up the those created rules.

---

Update

It is actually now supported (even though under documented):

1. Check that you're running Kubernetes 1.1 or later (under GKE edit your cluster and check "Node version")
2. Allocate static IPs under Networking > External IP addresses, either:
   • Deploy once without loadBalancerIP, wait until you've an external IP allocated when you run kubectl get svc, and look up that IP in the list on that page and change those from Ephemeral to Static.
   • Click "Reserver a static address" regional in the region of your cluster, attached to None.
3. Edit your LoadBalancer to have loadBalancerIP=10.10.10.10 as above (adapt to the IP that was given to you by Google).

Now if you delete your LoadBalancer or even your namespace, it'll preserve that IP address upon re-reploying on that cluster.

---

Update 2016-11-14

See also Kubernetes article describing how to set up a static IP for single or multiple domains on Kubernetes.

Answer 3

If you are running on Google Container Engine, and are using type: LoadBalancer, then Google Cloud Platform should have made a Network Load Balancer for you with a static IP address that will route to this service. You don't need to specify any IP addresses.

To find the network load balancer's IP, run:

gcloud compute forwarding-rules list --project "YOUR-PROJECT-ID"

You should also be able to run:

kubectl get services

Which will return both the cluster and external IP and port for your services.