How to add any new library like spark-csv in Apache Spark prebuilt version

Question

I have build the Spark-csv and able to use the same from pyspark shell using the following command

bin/spark-shell --packages com.databricks:spark-csv_2.10:1         


        

Answer 1

Another option is to add the following to your spark-defaults.conf:

spark.jars.packages com.databricks:spark-csv_2.11:1.2.0