How to execute Spark programs with Dynamic Resource Allocation?

Question

I am using spark-summit command for executing Spark jobs with parameters such as:

spark-submit --master yarn-cluster --driver-cores 2 \\
 --driver-memory 2G          


        

Answer 1

I just did a small demo with Spark's dynamic resource allocation. The code is on my Github. Specifically, the demo is in this release.