问题
I am learning about microk8s and how ingress works.
I have a single node microk8s (v1.18.4) with the following add-ons: DNS, ingress, RBAC, storage
I am trying to get it working with the microbot example. I've read (and reread) through the tutorial but, the address, once the ingress manifest is applied results with the microbot service, routed to 127.0.0.1 (and not the internal pod IP).
I am attempting to access the app: http://192.168.91.166/microbot via web external to the vm that it's running inside (and have tried curl while logged into the vm) it would result in an error page being returned. 192.168.91.166 is the vm's ip.
# kubectl get all -A
NAMESPACE NAME READY STATUS RESTARTS AGE
development pod/microbot-7dcc49575f-fqttp 1/1 Running 1 22h
ingress pod/nginx-ingress-microk8s-controller-phjgl 1/1 Running 1 21h
kube-system pod/coredns-588fd544bf-lb8mx 1/1 Running 3 37h
kube-system pod/hostpath-provisioner-75fdc8fccd-4klwh 1/1 Running 5 37h
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 37h
development service/microbot ClusterIP 10.152.183.19 <none> 80/TCP 22h
kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 37h
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
ingress daemonset.apps/nginx-ingress-microk8s-controller 1 1 1 1 1 <none> 21h
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
development deployment.apps/microbot 1/1 1 1 22h
kube-system deployment.apps/coredns 1/1 1 1 37h
kube-system deployment.apps/hostpath-provisioner 1/1 1 1 37h
NAMESPACE NAME DESIRED CURRENT READY AGE
development replicaset.apps/microbot-7dcc49575f 1 1 1 22h
kube-system replicaset.apps/coredns-588fd544bf 1 1 1 37h
kube-system replicaset.apps/hostpath-provisioner-75fdc8fccd 1 1 1 37h
microbot svc description
# kubectl describe svc microbot -n development
Name: microbot
Namespace: development
Labels: <none>
Annotations: Selector: name=microbot
Type: ClusterIP
IP: 10.152.183.19
Port: <unset> 80/TCP
TargetPort: 8080/TCP
Endpoints: 10.1.62.21:8080
Session Affinity: None
Events: <none>
ingress description
# kubectl describe ingress microbot -n development
Name: microbot
Namespace: development
Address: 127.0.0.1
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/microbot microbot:80 (10.1.62.21:8080)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal UPDATE 14s nginx-ingress-controller Ingress development/microbot
ingress pod log
# kubectl logs -n ingress nginx-ingress-microk8s-controller-phjgl
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: 0.33.0
Build: git-589187c35
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.19.0
-------------------------------------------------------------------------------
W0715 13:52:22.191579 6 flags.go:249] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0715 13:52:22.192057 6 client_config.go:552] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0715 13:52:22.194678 6 main.go:218] Creating API client for https://10.152.183.1:443
W0715 13:52:53.301344 6 main.go:259] Initial connection to the Kubernetes API server was retried 1 times.
I0715 13:52:53.301361 6 main.go:262] Running in Kubernetes cluster version v1.18+ (v1.18.4-1+6f17be3f1fd54a) - git (clean) commit 6f17be3f1fd54a88681869d1cf8bedd5a2174504 - platform linux/amd64
I0715 13:52:53.538979 6 main.go:103] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
I0715 13:52:53.539788 6 main.go:111] Enabling new Ingress features available since Kubernetes v1.18
E0715 13:52:53.542526 6 main.go:120] Unexpected error searching IngressClass: ingressclasses.networking.k8s.io "nginx" is forbidden: User "system:serviceaccount:ingress:nginx-ingress-microk8s-serviceaccount" cannot get resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
W0715 13:52:53.542574 6 main.go:123] No IngressClass resource with name nginx found. Only annotation will be used.
I0715 13:52:53.590982 6 nginx.go:263] Starting NGINX Ingress controller
I0715 13:52:53.604026 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-ingress-tcp-microk8s-conf", UID:"5f1b23d0-297c-4b2f-ac43-f3dc7256f684", APIVersion:"v1", ResourceVersion:"14859", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-ingress-tcp-microk8s-conf
I0715 13:52:53.604099 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-ingress-udp-microk8s-conf", UID:"abbfd3c7-c38c-43df-ae0c-7c26abcdb9cc", APIVersion:"v1", ResourceVersion:"14860", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-ingress-udp-microk8s-conf
I0715 13:52:53.604190 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-load-balancer-microk8s-conf", UID:"bc525770-be0a-4303-b26d-7e1c1d424d39", APIVersion:"v1", ResourceVersion:"14858", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-load-balancer-microk8s-conf
I0715 13:52:54.694319 6 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"development", Name:"microbot", UID:"58d26f3e-d19a-4c53-8a47-1223b3ccc300", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"22517", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress development/microbot
I0715 13:52:54.791730 6 nginx.go:307] Starting NGINX process
I0715 13:52:54.791929 6 leaderelection.go:242] attempting to acquire leader lease ingress/ingress-controller-leader-nginx...
I0715 13:52:54.793717 6 controller.go:139] Configuration changes detected, backend reload required.
I0715 13:52:54.805276 6 leaderelection.go:252] successfully acquired lease ingress/ingress-controller-leader-nginx
I0715 13:52:54.805481 6 status.go:86] new leader elected: nginx-ingress-microk8s-controller-phjgl
I0715 13:52:55.235056 6 controller.go:155] Backend successfully reloaded.
I0715 13:52:55.235148 6 controller.go:164] Initial sync, sleeping for 1 second.
2020/07/15 15:49:54 [error] 38#38: *57190 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
2020/07/15 15:49:54 [error] 38#38: *57190 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
2020/07/15 15:49:54 [error] 38#38: *57190 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
127.0.0.1 - - [15/Jul/2020:15:49:54 +0000] "GET /microbot HTTP/1.1" 502 157 "-" "curl/7.68.0" 81 0.000 [development-microbot-80] [] 10.1.62.21:8080, 10.1.62.21:8080, 10.1.62.21:8080 0, 0, 0 0.000, 0.000, 0.000 502, 502, 502 f4a04290c54d7562d01c2b2ef7b68d6b
2020/07/15 15:50:03 [error] 36#36: *57305 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
2020/07/15 15:50:03 [error] 36#36: *57305 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
2020/07/15 15:50:03 [error] 36#36: *57305 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
127.0.0.1 - - [15/Jul/2020:15:50:03 +0000] "GET /microbot HTTP/1.1" 502 157 "-" "curl/7.68.0" 81 0.000 [development-microbot-80] [] 10.1.62.21:8080, 10.1.62.21:8080, 10.1.62.21:8080 0, 0, 0 0.000, 0.000, 0.000 502, 502, 502 58813a2861ae850b96d10ab9482aaaae
2020/07/15 15:54:44 [error] 37#37: *60738 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
2020/07/15 15:54:44 [error] 37#37: *60738 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
2020/07/15 15:54:44 [error] 37#37: *60738 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /microbot HTTP/1.1", upstream: "http://10.1.62.21:8080/", host: "127.0.0.1"
127.0.0.1 - - [15/Jul/2020:15:54:44 +0000] "GET /microbot HTTP/1.1" 502 157 "-" "curl/7.68.0" 81 0.000 [development-microbot-80] [] 10.1.62.21:8080, 10.1.62.21:8080, 10.1.62.21:8080 0, 0, 0 0.000, 0.000, 0.000 502, 502, 502 b2bc837c1eb56d5e598f8160e4f2e3cb
I0715 16:36:08.010736 6 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"development", Name:"microbot", UID:"58d26f3e-d19a-4c53-8a47-1223b3ccc300", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"94444", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress development/microbot
I0715 16:36:08.011329 6 controller.go:139] Configuration changes detected, backend reload required.
I0715 16:36:08.116361 6 controller.go:155] Backend successfully reloaded.
I0715 16:41:19.262503 6 store.go:383] removing ingress microbot based on annotation kubernetes.io/ingress.class
I0715 16:41:19.262648 6 controller.go:139] Configuration changes detected, backend reload required.
I0715 16:41:19.262688 6 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"development", Name:"microbot", UID:"58d26f3e-d19a-4c53-8a47-1223b3ccc300", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"94444", FieldPath:""}): type: 'Normal' reason: 'DELETE' Ingress development/microbot
I0715 16:41:19.343023 6 controller.go:155] Backend successfully reloaded.
I0715 16:42:53.063307 6 store.go:324] ignoring delete for ingress microbot based on annotation kubernetes.io/ingress.class
I0715 16:43:26.671104 6 store.go:349] ignoring add for ingress microbot based on annotation kubernetes.io/ingress.class with value
deployment manifest:
apiVersion: v1
kind: Namespace
metadata:
name: development
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: microbot
namespace: development
spec:
strategy:
type: Recreate
replicas: 1
selector:
matchLabels:
name: microbot
template:
metadata:
labels:
name: microbot
spec:
containers:
- name: microbot
image: dontrebootme/microbot:v1
imagePullPolicy: Always
envFrom:
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: microbot
namespace: development
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
selector:
name: microbot
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: microbot
namespace: development
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /microbot
backend:
serviceName: microbot
servicePort: 80
I'm wondering if there's something I am not doing correctly. Any help would be much appreciated.
回答1:
In microK8s you should be using http://127.0.0.1/microbot to access a pod via ingress from outside the cluster i.e a browser. This is giving you 502 error in nginx ingress controller log. Few things to check
Check the service has got
Endpointsreflecting correct POD IP usingkubectl describe svc microbot -n developmentCheck if container inside pod is listening on port
8080. Maybe it's80or something else.The application running as a container in the pod need to listen on
0.0.0.0instead of127.0.0.1
来源：https://stackoverflow.com/questions/62917095/microk8s-ingress-ingressed-service-always-resolves-to-127-0-0-1-and-not-pod-i