How to stop INFO messages displaying on spark console?

后端 未结 20 3379
广开言路
广开言路 2020-11-22 13:40

I\'d like to stop various messages that are coming on spark shell.

I tried to edit the log4j.properties file in order to stop these message.

Her

20条回答
  •  眼角桃花
    2020-11-22 14:26

    Right after starting spark-shell type ;

    sc.setLogLevel("ERROR")
    

    In Spark 2.0 (Scala):

    spark = SparkSession.builder.getOrCreate()
    spark.sparkContext.setLogLevel("ERROR")
    

    API Docs : https://spark.apache.org/docs/2.2.0/api/scala/index.html#org.apache.spark.sql.SparkSession

    For Java:

    spark = SparkSession.builder.getOrCreate();
    spark.sparkContext().setLogLevel("ERROR");
    

提交回复
热议问题