Can I have multiple spark versions installed in CDH?

廉价感情. 提交于 2019-12-01 07:14:01

问题


I'm using cdh5.1.0, which already has default spark installed. However, I want to use Spark 1.3. Can I also install this version to cdh5.1.0? How is it possible to set these up? Will the new version of spark also be monitored via Cloudera manager?


回答1:


Yes, you can run any Apache Spark version you like. Just make sure it's built for the version of YARN you have (2.3 for CDH 5.1.0). You can then run your application as a YARN application with spark-submit. (See http://spark.apache.org/docs/latest/running-on-yarn.html.) It will be monitored like any other YARN application.

Spark doesn't need to be "installed" at all, you just run your application.




回答2:


Please execute below commands from your console, with sudo access and password....it will update your spark current version to latest version

sudo yum install spark-core spark-master spark-worker spark-history-server spark-python
sudo yum install spark-core spark-history-server spark-python


来源:https://stackoverflow.com/questions/29691028/can-i-have-multiple-spark-versions-installed-in-cdh

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!