How to get kube-dns working in Vagrant cluster using kubeadm and Weave

Question

I deployed a few VMs using Vagrant to test kubernetes:
master: 4 CPUs, 4GB RAM
node-1: 4 CPUs, 8GB RAM
Base image: Centos/7.
Networking: Bridged.
Host OS

Answer 1

Please take a look at David Bainbridge's repository.

Answer 2

Here is the recipe that worked for me (as of March 19th 2017 using Vagrant and VirtualBox). The cluster is made of 3 nodes, 1 Master and 2 Nodes.

1) Make sure you explicitly set the IP of your master node on init

kubeadm init --api-advertise-addresses=10.30.3.41

2) Manually or during provisioning, add to each node's /etc/hosts the exact IP that you are configuring it to have. Here is a line you can add in your Vagrant file (node naming convention I use: k8node-$i) :

config.vm.provision :shell, inline: "sed 's/127\.0\.0\.1.*k8node.*/10.30.3.4#{i} k8node-#{i}/' -i /etc/hosts"

Example:

vagrant@k8node-1:~$ cat /etc/hosts
10.30.3.41 k8node-1
127.0.0.1   localhost

3) Finally, all Nodes will try to use the public IP of the cluster to connect to the master (not sure why this is happening ...). Here is the fix for that.

First, find the public IP by running the following on master.

kubectl get svc
NAME         CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   10.96.0.1    <none>        443/TCP   1h

In each node, make sure that any process using 10.96.0.1 (in my case) is routed to master that is on 10.30.3.41.

So on each Node (you can skip master) use route to set the redirect.

route add 10.96.0.1 gw 10.30.3.41

After that, everything should work ok:

vagrant@k8node-1:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                               READY     STATUS    RESTARTS   AGE
kube-system   dummy-2088944543-rnl2f             1/1       Running   0          1h
kube-system   etcd-k8node-1                      1/1       Running   0          1h
kube-system   kube-apiserver-k8node-1            1/1       Running   0          1h
kube-system   kube-controller-manager-k8node-1   1/1       Running   0          1h
kube-system   kube-discovery-1769846148-g8g85    1/1       Running   0          1h
kube-system   kube-dns-2924299975-7wwm6          4/4       Running   0          1h
kube-system   kube-proxy-9dxsb                   1/1       Running   0          46m
kube-system   kube-proxy-nx63x                   1/1       Running   0          1h
kube-system   kube-proxy-q0466                   1/1       Running   0          1h
kube-system   kube-scheduler-k8node-1            1/1       Running   0          1h
kube-system   weave-net-2nc8d                    2/2       Running   0          46m
kube-system   weave-net-2tphv                    2/2       Running   0          1h
kube-system   weave-net-mp6s0                    2/2       Running   0          1h


vagrant@k8node-1:~$ kubectl get nodes
NAME       STATUS         AGE
k8node-1   Ready,master   1h
k8node-2   Ready          1h
k8node-3   Ready          48m