Issues running pyspark code via Spark.submit but not with Jupyter-notebook

Question

I am facing an issue with my pyspark code. the purpose of the code is to decode hex values and write it into a field. the code runs perfectly fine in jupyter environment. bu