I need to set a custom environment variable in EMR to be available when running a spark application.
I have tried adding this:
.
For me replacing spark-env to yarn-env fixed issue.