Kubernetes Pod Warning: 1 node(s) had volume node affinity conflict

Question

I try to set up Kubernetes cluster. I have Persistent Volume, Persistent Volume Claim and Storage class all set-up and running but when I wan to create pod from deployment,

Answer 1

There a few things that can cause this error:

1. Node isn’t labeled properly. I had this issue on AWS when my worker node didn’t have appropriate labels(master had them though) like that:

   failure-domain.beta.kubernetes.io/region=us-east-2

   failure-domain.beta.kubernetes.io/zone=us-east-2c

   After patching the node with the labels, the “1 node(s) had volume node affinity conflict” error was gone, so PV, PVC with a pod were deployed successfully. The value of these labels is cloud provider specific. Basically, it is the job of the cloud provider(with —cloud-provider option defined in cube-controller, API-server, kubelet) to set those labels. If appropriate labels aren’t set, then check that your CloudProvider integration is correct. I used kubeadm, so it is cumbersome to set up but with other tools, kops, for instance, it is working right away.

2. Based on your PV definition and the usage of nodeAffinity field, you are trying to use a local volume, (read here local volume description link, official docs), then make sure that you set "NodeAffinity field" like that(it worked in my case on AWS):

   nodeAffinity:

        required:
         nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - my-node  # it must be the name of your node(kubectl get nodes)
   

So that after creating the resource and running describe on it it will show up there like that:

         Required Terms:  
                    Term 0:  kubernetes.io/hostname in [your node name]

3. StorageClass definition(named local-storage, which is not posted here) must be created with volumeBindingMode set to WaitForFirstConsumer for local storage to work properly. Refer to the example here storage class local description, official doc to understand the reason behind that.