How to change job/stage description in web UI?

Question

When I run a job on Apache Spark, the web UI gives a view similar to this:

While this is incredibly useful for me as a developer to see where things are, I

Answer 1

You can use the following API(s) to set and unset the stage names. https://spark.apache.org/docs/2.3.0/api/java/org/apache/spark/SparkContext.html#setCallSite-java.lang.String- https://spark.apache.org/docs/2.3.0/api/java/org/apache/spark/SparkContext.html#clearCallSite--

Also, Spark supports the concept of Job Groups within the application, following API(s) can be used to set and unset the job group names. https://spark.apache.org/docs/2.3.0/api/java/org/apache/spark/SparkContext.html#setJobGroup-java.lang.String-java.lang.String-boolean- https://spark.apache.org/docs/2.3.0/api/java/org/apache/spark/SparkContext.html#clearJobGroup--

The job description within the job group can also be configured using following API. https://spark.apache.org/docs/2.3.0/api/java/org/apache/spark/SparkContext.html#setJobDescription-java.lang.String-