Where are logs in Spark on YARN?

前端 未结 4 1498
执念已碎
执念已碎 2020-12-01 06:26

I\'m new to spark. Now I can run spark 0.9.1 on yarn (2.0.0-cdh4.2.1). But there is no log after execution.

The following command is used to run a spark example. But

4条回答
  •  伪装坚强ぢ
    2020-12-01 06:42

    None of the answers make it crystal clear where to look for logs ( although they do in pieces) so I am putting it together.

    If log aggregation is turned on (with the yarn.log-aggregation-enable yarn-site.xml) then do this

    yarn logs -applicationId 
    

    However, if this is not turned on then one needs to go on the Data-Node machine and look at

    $HADOOP_HOME/logs/userlogs/application_1474886780074_XXXX/
    

    application_1474886780074_XXXX is the application id

提交回复
热议问题