spark.sql.crossJoin.enabled for Spark 2.x

Question

I am using the \'preview\' Google DataProc Image 1.1 with Spark 2.0.0. To complete one of my operations I have to complete a cartesian product. Since version 2.0.0 there ha

Answer 1

For changing default values of configuration settings in Dataproc, you don't even need an init action, you can use the --properties flag when creating your cluster from the command-line:

gcloud dataproc clusters create --properties spark:spark.sql.crossJoin.enabled=true my-cluster ...