Apache Spark does not delete temporary directories

Question

After a spark program completes, there are 3 temporary directories remain in the temp directory. The directory names are like this: spark-2e389487-40cc-4a82-a5c7-353c0feefbb

Answer 1

You need to call close() on the spark context that you created at the end of the program.