“Bad substitution” when submitting spark job to yarn-cluster

Question

I am doing a smoke test against a yarn cluster using yarn-cluster as the master with the SparkPi example program. Here is the command line:

<

Answer 1

This may be caused by /bin/sh linked to dash, instead of bash, which often happens on Debian based systems.

To fix it, run sudo dpkg-reconfigure dash and select no.