1. 情况说明
使用一台VMWare Workstation虚拟机，4核8G内存，50G磁盘
已安装KubeSphere 2.1 版本，已经按照官方文档的入门必读，示例一创建好相应的账号信息等
KubeSphere 文档地址：https://kubesphere.com.cn/docs/v2.1/zh-CN/introduction/intro/
2. 实现的效果
kubeapps官方文档地址：https://github.com/kubeapps/kubeapps
想使用kubeapps，通过helm管理部署在k8s集群的应用，部署，升级，回退版本等
同时可以在KubeSphere中查看部署的应用等
3. 步骤
(1).要求
- k8s集群版本：1.8+
- Helm版本：2.14.0+
- 已安装kubectl
KubeSphere 2.1 版本安装的k8s集群是1.15.5版本，helm版本是2.14.3，已安装kubectl，符合上述要求
[root@ks-allinone ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:07:57Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:07:57Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
[root@ks-allinone ~]# helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
(2) 安装kubeapps
# 查看helm仓库
[root@ks-allinone ~]# helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
# 添加helm仓库
[root@ks-allinone ~]# helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories
# 再次查看helm仓库
[root@ks-allinone ~]# helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
bitnami https://charts.bitnami.com/bitnami
# 开始安装，使用单独的命名空间
[root@ks-allinone ~]# helm install --name kubeapps --namespace kubeapps bitnami/kubeapps
NAME: kubeapps
LAST DEPLOYED: Tue Nov 19 15:32:55 2019
NAMESPACE: kubeapps
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
kubeapps-frontend-config 1 5s
kubeapps-internal-dashboard-config 2 5s
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
kubeapps 0/2 2 0 5s
kubeapps-internal-apprepository-controller 0/1 1 0 5s
kubeapps-internal-chartsvc 0/2 2 0 5s
kubeapps-internal-dashboard 0/2 2 0 5s
kubeapps-internal-tiller-proxy 0/2 2 0 4s
kubeapps-mongodb 0/1 1 0 5s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
kubeapps-76bfb9557f-97f2g 0/1 ContainerCreating 0 4s
kubeapps-76bfb9557f-gj8nl 0/1 Pending 0 4s
kubeapps-internal-apprepository-controller-6764b79b6b-tv76f 0/1 ContainerCreating 0 4s
kubeapps-internal-chartsvc-677b745bb9-nqw4j 0/1 ContainerCreating 0 4s
kubeapps-internal-chartsvc-677b745bb9-wwdrl 0/1 ContainerCreating 0 4s
kubeapps-internal-dashboard-87f979fd7-chwsr 0/1 ContainerCreating 0 4s
kubeapps-internal-dashboard-87f979fd7-jl76k 0/1 ContainerCreating 0 4s
kubeapps-internal-tiller-proxy-57c4d9c6dd-2gwwq 0/1 ContainerCreating 0 4s
kubeapps-internal-tiller-proxy-57c4d9c6dd-tg2n8 0/1 ContainerCreating 0 4s
kubeapps-mongodb-788ff89fd9-wp8dq 0/1 ContainerCreating 0 4s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubeapps ClusterIP 10.233.32.3 <none> 80/TCP 5s
kubeapps-internal-chartsvc ClusterIP 10.233.33.159 <none> 8080/TCP 5s
kubeapps-internal-dashboard ClusterIP 10.233.16.70 <none> 8080/TCP 5s
kubeapps-internal-tiller-proxy ClusterIP 10.233.4.189 <none> 8080/TCP 5s
kubeapps-mongodb ClusterIP 10.233.30.14 <none> 27017/TCP 5s
==> v1/ServiceAccount
NAME SECRETS AGE
kubeapps-internal-apprepository-controller 1 5s
kubeapps-internal-tiller-proxy 1 5s
==> v1beta1/Role
NAME AGE
kubeapps-internal-apprepository-controller 5s
kubeapps-internal-tiller-proxy 5s
kubeapps-repositories-read 5s
kubeapps-repositories-write 5s
==> v1beta1/RoleBinding
NAME AGE
kubeapps-internal-apprepository-controller 5s
kubeapps-internal-tiller-proxy 5s
NOTES:
** Please be patient while the chart is being deployed **
Tip:
Watch the deployment status using the command: kubectl get pods -w --namespace kubeapps
Kubeapps can be accessed via port 80 on the following DNS name from within your cluster:
kubeapps.kubeapps.svc.cluster.local
# 注：这一步是配置网络访问的，这些不用操作，在KubeSphere中操作从而实现外部访问
To access Kubeapps from outside your K8s cluster, follow the steps below:
1. Get the Kubeapps URL by running these commands:
echo "Kubeapps URL: http://127.0.0.1:8080"
export POD_NAME=$(kubectl get pods --namespace kubeapps -l "app=kubeapps" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward --namespace kubeapps $POD_NAME 8080:8080
2. Open a browser and access Kubeapps using the obtained URL.
# 查看kubeapps命名空间的pod情况，需要等所有的pod状态是running或Completed才行
[root@ks-allinone ~]# kubectl get pods --namespace kubeapps
NAME READY STATUS RESTARTS AGE
apprepo-sync-bitnami-1574149200-glnhg 0/1 ContainerCreating 0 10s
apprepo-sync-bitnami-fqhnh-jx7g9 1/1 Running 3 3m51s
apprepo-sync-incubator-1574149200-lbdjz 0/1 ContainerCreating 0 10s
apprepo-sync-incubator-5glkx-v8w5l 0/1 Completed 3 3m51s
apprepo-sync-stable-1574149200-bjfnh 0/1 ContainerCreating 0 10s
apprepo-sync-stable-z2vlh-4f2kv 1/1 Running 0 3m51s
apprepo-sync-svc-cat-1574149200-5kmj7 0/1 ContainerCreating 0 10s
apprepo-sync-svc-cat-274bp-5b2zd 0/1 Completed 0 3m51s
kubeapps-76bfb9557f-97f2g 1/1 Running 0 7m20s
kubeapps-76bfb9557f-gj8nl 1/1 Running 0 7m20s
kubeapps-internal-apprepository-controller-6764b79b6b-tv76f 1/1 Running 0 7m20s
kubeapps-internal-chartsvc-677b745bb9-nqw4j 1/1 Running 0 7m20s
kubeapps-internal-chartsvc-677b745bb9-wwdrl 1/1 Running 0 7m20s
kubeapps-internal-dashboard-87f979fd7-chwsr 1/1 Running 0 7m20s
kubeapps-internal-dashboard-87f979fd7-jl76k 1/1 Running 0 7m20s
kubeapps-internal-tiller-proxy-57c4d9c6dd-2gwwq 1/1 Running 0 7m20s
kubeapps-internal-tiller-proxy-57c4d9c6dd-tg2n8 1/1 Running 0 7m20s
kubeapps-mongodb-788ff89fd9-wp8dq 1/1 Running 0 7m20s
(3) 设置kubeapps访问k8s集群权限
[root@ks-allinone ~]# kubectl create serviceaccount kubeapps-operator
serviceaccount/kubeapps-operator created
[root@ks-allinone ~]# kubectl create clusterrolebinding kubeapps-operator --clusterrole=cluster-admin --serviceaccount=default:kubeapps-operator
clusterrolebinding.rbac.authorization.k8s.io/kubeapps-operator created
# 获取访问k8s api的token
[root@ks-allinone ~]# kubectl get secret $(kubectl get serviceaccount kubeapps-operator -o jsonpath='{range .secrets[*]}{.name}{"\n"}{end}' | grep kubeapps-operator-token) -o jsonpath='{.data.token}' -o go-template='{{.data.token | base64decode}}' && echo
# 记住这个token，访问dashboard界面登陆时需要用
eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6Imt1YmVhcHBzLW9wZXJhdG9yLXRva2VuLTZ3aGJnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6Imt1YmVhcHBzLW9wZXJhdG9yIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZjAzYzUxMWYtNThhZi00YTNjLWE5NzctMzE2YmRkMDA1MTZiIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6a3ViZWFwcHMtb3BlcmF0b3IifQ.xmLLkX3x91HKUilTLnVEyblnAQ0HwW-J__6geg0KOrkB35_gRkKMGp8ftl7vT5KEWRyW8fyAIJHc2LEo8uLGSbsQDzvuUxEBnvUzcgsrOR03Idh8cTdsgpynNnyhlLGCqEq-fBn-8HUCap4B4c00NXImD3BhtkJ6PCy0CfrnQyua1qXR6RqqRH0epIedxsNAGGvFgKmeYO2Eq_04BKsmEMRkcBbGXx848O_6izBW7kHc9lfSEUfFwvrCs1YXBWJk7UGrh9lmlt1U-3BjFAXg-7hC74kx-G1Mj_Z2cxhnJ4W6XckphxE5NPBeoDow0GcD7jx4C4QkzX7fQXrKaKyYPQ
(4) 在KubeSphere中设置kubeapps等
- 使用集群管理员账号登陆，在工作台界面会查看到有一个项目，名称是kubeapps，但是没有分配企业空间。
创建名称是kubeapps的企业空间，然后再返回到该界面，点击右边的三个竖点，分配企业空间，把项目kubeapps分配到企业空间kubeapps - 点击项目名称kubeapps，进入项目详情界面。
先在项目设置，高级设置中开启外放访问，使用NodePort。
然后在应用负载的服务界面，点击kubeapps，更多操作，编辑外网访问，访问方式是NodePort。此时右边会出现"点击访问"按钮，点击这个开始访问kubeapps的dashboard web界面，输入上一步的token值即可登陆进去
4. 使用kubeapps
- 可以添加仓库地址，点击"Configuration"中的App Repositories
- 在Applications界面，选择不同的NAMESPACE,可以显示安装的应用
- 选择"Catalog"可以查看仓库中的应用，直接部署到k8s中进行使用，同时可以在KubeSphere中查看管理