“Bad substitution” when submitting spark job to yarn-cluster

Question

I am doing a smoke test against a yarn cluster using yarn-cluster as the master with the SparkPi example program. Here is the command line:

<

Answer 1

1. Go to the ambari-yarn.

click on Configs->Advanced->Custom yarn-site->Add Property ...

add hdp version as key and value as your HDP version. You will get hdp version along with below command

hdp-select versions

e.g. 2.5.3.0-37

Now add you property as

hdp.version=2.5.3.0-37

2. Otherwise replace ${hdp.version} as your hdp version(2.5.3.0-37) in yarn-site.xml and yarn-env.sh