Spark background threads not cleaned up when using embedded Derby metastore

Question

Some background before I launch into the questions:

I\'m involved in a project that uses a bootstrapped Spark application, and as such, we have a lot of logic dedicat