How to limit the number of retries on Spark job failure?

Question

We are running a Spark job via spark-submit, and I can see that the job will be re-submitted in the case of failure.

How can I stop it from having attem

Answer 1

Add the property yarn.resourcemanager.am.max-attempts to your yarn-default.xml file. It specifies the maximum number of application attempts.

For more details look into this link