How to read client IP addresses from HTTP requests behind Kubernetes services?

Question

my web application is running as a Kubernetes pod behind an nginx reverse proxy for SSL. Both the proxy and my application use Kubernetes services for load balancing (as des

Answer 1

As of Kubernetes 1.1, there is an iptables-based kube-proxy that fixes this issue in some cases. It's disabled by default; see this post for instructions for how to enable it. In summary, do:

for node in $(kubectl get nodes -o name); do kubectl annotate $node net.beta.kubernetes.io/proxy-mode=iptables; done

In the case of Pod-to-Pod traffic, with the iptables kube-proxy you will now see the true source-IP at the destination pod.

However, if your Service is forwarding traffic from outside the cluster (e.g. a NodePort, LoadBalancer service), then we still have to replace (SNAT) the source-IP. This is because we are doing DNAT on the incoming traffic to route it to the the service Pod (potentially on another Node), so the DNATing Node needs to insert itself in the return path to be able to un-DNAT the response.