Why did I encounter an “Error syncing pod” with Dataflow pipeline?

Question

I experiment a weird error with my Dataflow pipeline when I want to use specific library from PyPI.

I need jsonschema in a ParDo, so, in my requiremen

Answer 1

When Dataflow workers start, they execute several steps:

1. Install packages from requirements.txt
2. Install packages specified as extra_packages
3. Install workflow tarball and execute actions provided in setup.py.

Error syncing pod with CrashLoopBackOff message can be related to dependency conflict. You need to verify that there are no conflicts with the libraries and versions used for the job. Please refer to the documentation for staging required dependencies of the pipeline.

Also, take a look for preinstalled dependencies and this StackOverflow thread.

What you can try is change the version of jsonschema and try run it again. If it wouldn't help, please provide requirements.txt file.

I hope it will help you.