Airflow k8s operator xcom - Handshake status 403 Forbidden

问题:

When I run a docker image using KubernetesPodOperator in Airflow version 1.10

Once the pod finishes the task successfullly, airflow tries to get the xcom value by making a connection to the pod via k8s stream client.

Following is the error which I encountered:

[2018-12-18 05:29:02,209] {{models.py:1760}} ERROR - (0) Reason: Handshake status 403 Forbidden Traceback (most recent call last):   File "/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py", line 249, in websocket_call     client = WSClient(configuration, get_websocket_url(url), headers)   File "/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py", line 72, in __init__     self.sock.connect(url, header=header)   File "/usr/local/lib/python3.6/site-packages/websocket/_core.py", line 223, in connect     self.handshake_response = handshake(self.sock, *addrs, **options)   File "/usr/local/lib/python3.6/site-packages/websocket/_handshake.py", line 79, in handshake     status, resp = _get_resp_headers(sock)   File "/usr/local/lib/python3.6/site-packages/websocket/_handshake.py", line 152, in _get_resp_headers     raise WebSocketBadStatusException("Handshake status %d %s", status, status_message) websocket._exceptions.WebSocketBadStatusException: Handshake status 403 Forbidden 

I'm using K8s service account for this

DAG configs

xcom=true,

get_logs=True,

in_cluster=true

回答1:

So we also hit this problem, we had to modify our rbac rules, in particular we had to add the resource "pods/exec" with the verbs "create" and "get"

--- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: airflow-runner rules: - apiGroups: [""]   resources: ["deployments", "pods", "pods/log", "pods/exec", "persistentvolumeclaims"]   verbs: ["*"] - apiGroups: [""]   resources: ["secrets"]   resourceNames: ["singleuser-image-credentials"]   verbs: ["read","list","watch","create","get"] 

文章来源: Airflow k8s operator xcom - Handshake status 403 Forbidden