Build a SparkSession

感情迁移 提交于 2019-12-04 07:00:57

问题


I have a spark as interpreter in Zeppelin. I'm using a Spark2.0, I built a Session: Create


回答1:


In general you should not initialize SparkSession nor SparkContext in Zeppelin. Zeppelin notebooks are configured to create session for you, and their correct behavior depends on using provided objects.

Initializing your SparkSession will break core Zeppelin functionalities, and multiple SparkContexts will break things completely in the worst case scenario.

Is set spark.driver.allowMultipleContexts to False is best to do a tests ?

You should never use spark.driver.allowMultipleContexts - it not supported, and doesn't guarantee correct results.



来源:https://stackoverflow.com/questions/49804517/build-a-sparksession

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!