kubernetes Ingress、Ingress controller

前言

拥抱开源，无私分享，共享技术，相互学习，共同进步，分享更多有深度的文章，欢迎转发分享

四层负载均衡器service回顾

使用四层负载均衡调度器service时，当客户端访问kubernetes集群内部的应用时，数据包走向如下面流程所示

client--->nodeip:port--->service ip:port--->podip:port

客户端-->node节点的ip:端口--->service的ip:端口--->pod的ip:端口

1.Ingress Controller

Ingress Controller是一个七层负载均衡调度器，客户端的请求先到达这个七层负载均衡调度器，由七层负载均衡器在反向代理到后端pod，常见的七层负载均衡器有nginx，traefik等，以我们熟悉的nginx为例，假如请求到达nginx，会通过upstream反向代理到后端pod，但是后端pod的ip地址是一直在变化的，因此在后端pod前需要加一个service，这个service只是起到分组的作用，那么我们upstream只需要填写service地址即可

nginx：需要手动加载配置文件

traefik：定期自动加载配置文件，不需要手动干预，在微服务中几乎都会使用这种调度器

2.Ingress

官方：https://kubernetes.io/docs/concepts/services-networking/ingress/

ingress官网定义：ingress可以把进入到集群内部的请求转发到集群中的一些服务上，从而可以把服务暴漏到集群外部。Ingress 能把集群内Service 配置成外网能够访问的 URL，流量负载均衡，提供基于域名访问的虚拟主机等；

ingress是k8s中的资源，当service关联的后端pod ip地址发生变化，就会把这些变化信息保存在ingress中，由ingress注入到七层负载均衡调度器ingress controller中，也就是把信息传入到七层负载均衡调度器的配置文件中，并且重新加载使配置生效，Ingress 可以用来规定 HTTP/S 请求应该被转发到哪个 Service 上，比如根据请求中不同的 Host 和 url 路径让请求落到不同的 Service 上

总结：

Ingress Controller

Ingress Controller 可以理解为控制器，它通过不断的跟 Kubernetes API 交互，实时获取后端Service、Pod的变化，比如新增、删除等，结合Ingress 定义的规则生成配置，然后动态更新上边的 Nginx 或者trafik负载均衡器，并刷新使配置生效，来达到服务自动发现的作用。

Ingress

Ingress 则是定义规则，通过它定义某个域名的请求过来之后转发到集群中指定的 Service。它可以通过 Yaml 文件定义，可以给一个或多个 Service 定义一个或多个 Ingress 规则。

3.七层调度器traefik

如果七层负载均衡调度器使用nginx，那么每次nginx配置文件修改之后，都需要重新reload才能生效，很不方便，如果使用traefik或者Envoy，可以每隔一定时间重新加载配置文件，不需要手动重新加载，对于这种微服务架构来说特别方便

4.使用七层负载均衡调度器的步骤

（1）部署ingress controller，我们ingress controller使用的是traefik

（2）创建service，用来分组pod

（3）创建pod应用，可以通过控制器创建pod

（4）创建ingress http，测试通过http访问

（5）创建ingress https，测试通过https访问

5.客户端通过七层调度器访问后端pod的方式

使用七层负载均衡调度器ingress controller时，当客户端访问kubernetes集群内部的应用时，数据包走向如下面流程所示：

client--->Nodeip:port----->IngressController--->service--->pod

6.生成证书，在k8s集群的master节点操作

（1）生成私钥

openssl genrsa -out tls.key 2048

（2）生成证书

openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/O=DevOps/CN=tomcat.lucky.com

（3）生成secret

kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key

Secret 对象类型用来保存敏感信息，例如密码、OAuth 令牌和 ssh key。将这些信息放在 secret 中比放在 pod 的定义或者 docker 镜像中来说更加安全和灵活

kubectl get secret 显示如下：

7.部署trafik，在k8s集群的master节点操作

（1）生成openssl.cnf文件，这个文件里面把签发证书时需要的参数已经定义好了

mkdir /root/test

echo """

[req]

distinguished_name = req_distinguished_name

prompt = yes

[ req_distinguished_name ]

countryName = Country Name (2 letter code)

countryName_value = CN

stateOrProvinceName = State or Province Name (full name)

stateOrProvinceName_value = Beijing

localityName = Locality Name (eg, city)

localityName_value = Haidian

organizationName = Organization Name (eg, company)

organizationName_value = Channelsoft

organizationalUnitName = Organizational Unit Name (eg, section)

organizationalUnitName_value = R & D Department

commonName = Common Name (eg, your name or your server\'s hostname)

commonName_value = *.multi.io

emailAddress = Email Address

emailAddress_value = xianchao@qq.com

""" > /root/test/openssl.cnf

（2）生成一个证书

openssl req -newkey rsa:4096 -nodes -config /root/test/openssl.cnf -days 3650 -x509 -out /root/test/tls.crt -keyout /root/test/tls.key

（3）创建一个secret对象

kubectl create -n kube-system secret tls ssl --cert /root/test/tls.crt --key /root/test/tls.key

（4）部署traefik

cat traefik.yaml

---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1beta1metadata:  name: traefik-ingress-controllerrules:  - apiGroups:      - ""    resources:      - services      - endpoints      - secrets    verbs:      - get      - list      - watch  - apiGroups:      - extensions    resources:      - ingresses    verbs:      - get      - list      - watch---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1beta1metadata:  name: traefik-ingress-controllerroleRef:  apiGroup: rbac.authorization.k8s.io  kind: ClusterRole  name: traefik-ingress-controllersubjects:- kind: ServiceAccount  name: traefik-ingress-controller  namespace: kube-system---apiVersion: v1kind: ServiceAccountmetadata:  name: traefik-ingress-controller  namespace: kube-system---kind: ConfigMapapiVersion: v1metadata:  name: traefik-conf  namespace: kube-systemdata:  traefik.toml: |    insecureSkipVerify = true    defaultEntryPoints = ["http","https"]    [entryPoints]      [entryPoints.http]      address = ":80"      [entryPoints.https]      address = ":443"        [entryPoints.https.tls]          [[entryPoints.https.tls.certificates]]          CertFile = "/ssl/tls.crt"          KeyFile = "/ssl/tls.key"---kind: DaemonSetapiVersion: apps/v1metadata:  name: traefik-ingress-controller  namespace: kube-system  labels:    k8s-app: traefik-ingress-lbspec:  selector:      matchLabels:        k8s-app: traefik-ingress-lb        name: traefik-ingress-lb  template:    metadata:      labels:        k8s-app: traefik-ingress-lb        name: traefik-ingress-lb    spec:      serviceAccountName: traefik-ingress-controller      tolerations:      - key: node-role.kubernetes.io/master        effect: NoSchedule      terminationGracePeriodSeconds: 60      hostNetwork: true      volumes:      - name: ssl        secret:          secretName: ssl      - name: config        configMap:          name: traefik-conf      containers:      - image: k8s.gcr.io/traefik:1.7.9        name: traefik-ingress-lb        ports:        - name: http          containerPort: 80          hostPort: 80        - name: admin          containerPort: 8080        securityContext:          privileged: true        args:        - --configfile=/config/traefik.toml        - -d        - --web        - --kubernetes        volumeMounts:        - mountPath: "/ssl"          name: "ssl"        - mountPath: "/config"          name: "config"---kind: ServiceapiVersion: v1metadata:  name: traefik-ingress-servicespec:  selector:    k8s-app: traefik-ingress-lb  ports:    - protocol: TCP      port: 80      name: web    - protocol: TCP      port: 8080      name: admin    - protocol: TCP      port: 443      name: https  type: NodePort---apiVersion: v1kind: Servicemetadata:  name: traefik-web-ui  namespace: kube-systemspec:  selector:    k8s-app: traefik-ingress-lb  ports:  - port: 80    targetPort: 8080---apiVersion: extensions/v1beta1kind: Ingressmetadata:  name: traefik-web-ui  namespace: kube-system  annotations:    kubernetes.io/ingress.class: traefikspec:  rules:  - host: ingress.multi.io    http:      paths:      - backend:          serviceName: traefik-web-ui          servicePort: 80

把traefik镜像traefik_1_7_9.tar.gz上传到k8s各个节点，手动解压：

docker load -i traefik_1_7_9.tar.gz

镜像地址：

链接：https://pan.baidu.com/s/1fTRc0J0iL7DeAH_LNUkTQQ
提取码：xg1z

kubectl apply -f traefik.yaml

（5）验证traefik部署是否成功

kubectl get pods -n kube-system 显示如下，说明部署成功

traefik-ingress-controller-g45th 1/1 Running 0 9m2s
traefik-ingress-controller-lj9md 1/1 Running 0 9m2s

8.部署tomcat

cat tomcat-deploy.yaml

apiVersion: v1kind: Servicemetadata:  name: tomcat  namespace: defaultspec:  selector:    app: tomcat    release: canary  ports:  - name: http    targetPort: 8080    port: 80  - name: ajp    targetPort: 8009    port: 8009---apiVersion: apps/v1kind: Deploymentmetadata:  name: tomcat-deploy  namespace: defaultspec:  replicas: 3  selector:    matchLabels:      app: tomcat      release: canary  template:    metadata:      labels:        app: tomcat        release: canary    spec:      containers:      - name: myapp        image: tomcat:8.5-jre8-alpine        ports:        - name: http          containerPort: 8080        - name: ajp          containerPort: 8009

kubectl apply -f tomcat-deploy.yaml

kubectl get pods 显示如下，说明部署成功

tomcat-deploy-59664bcb6f-5z4nn   1/1     Running   0          20s
tomcat-deploy-59664bcb6f-cgjbn   1/1     Running   0          20s
tomcat-deploy-59664bcb6f-n4tqq   1/1     Running   0          20s

9.部署tomcat http

cat ingress-tomcat.yaml

apiVersion: extensions/v1beta1kind: Ingressmetadata:  name: ingress-tomcat  namespace: default  annotations:    kubernetes.io/ingress.class: "traefik"spec:  rules:  - host: tomcat.lucky.com    http:      paths:      - path:        backend:          serviceName: tomcat          servicePort: 80

kubectl apply -f ingress-tomcat.yaml

10.部署tomcat https

vim ingress-tomcat-tls.yaml

apiVersion: extensions/v1beta1kind: Ingressmetadata:  name: ingress-tomcat-tls  namespace: default  annotations:     kubernetes.io/ingress.class: traefikspec:  tls:  - hosts:    - tomcat.lucky.com    secretName: tomcat-ingress-secret  rules:  - host: tomcat.lucky.com    http:      paths:      - path:        backend:          serviceName: tomcat          servicePort: 80

kubectl apply -f ingress-tomcat-tls.yaml

kubectl get ingress

显示如下：

NAME                 HOSTS              ADDRESS   PORTS     AGE
ingress-tomcat       tomcat.lucky.com             80        103s
ingress-tomcat-tls   tomcat.lucky.com             80, 443   8s

11.配置自己电脑的hosts文件

192.168.0.16 tomcat.lucky.com

12.在浏览器访问：

https://tomcat.lucky.com:443

或者http://tomcat.lucky.com

可看到如下界面，说明https配置成功