I\'m new to spark. Now I can run spark 0.9.1 on yarn (2.0.0-cdh4.2.1). But there is no log after execution.
The following command is used to run a spark example. But
None of the answers make it crystal clear where to look for logs ( although they do in pieces) so I am putting it together.
If log aggregation is turned on (with the yarn.log-aggregation-enable yarn-site.xml) then do this
yarn logs -applicationId
However, if this is not turned on then one needs to go on the Data-Node machine and look at
$HADOOP_HOME/logs/userlogs/application_1474886780074_XXXX/
application_1474886780074_XXXX is the application id