How to check the Spark version

后端 未结 15 1274
时光取名叫无心
时光取名叫无心 2020-12-29 17:29

as titled, how do I know which version of spark has been installed in the CentOS?

The current system has installed cdh5.1.0.

15条回答
  •  借酒劲吻你
    2020-12-29 18:14

    In Spark 2.x program/shell,

    use the

    spark.version  
    

    Where spark variable is of SparkSession object

    Using the console logs at the start of spark-shell

    [root@bdhost001 ~]$ spark-shell
    Setting the default log level to "WARN".
    To adjust logging level use sc.setLogLevel(newLevel).
    Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
          /_/
    

    Without entering into code/shell

    spark-shell --version

    [root@bdhost001 ~]$ spark-shell --version
    Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
          /_/
                            
    Type --help for more information.
    

    spark-submit --version

    [root@bdhost001 ~]$ spark-submit --version
    Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
          /_/
                            
    Type --help for more information.
    

提交回复
热议问题