Where does hadoop mapreduce framework send my System.out.print() statements ? (stdout)

前端 未结 4 491
鱼传尺愫
鱼传尺愫 2020-11-30 20:26

I want to debug a mapreduce script, and without going into much trouble tried to put some print statements in my program. But I cant seem to find them in any of the logs.

4条回答
  •  小蘑菇
    小蘑菇 (楼主)
    2020-11-30 21:19

    Another way is through the terminal:

    1) Go into your Hadoop_Installtion directory, then into "logs/userlogs".
    2) Open your job_id directory.
    3) Check directories with _m_ if you want the mapper output or _r_ if you're looking for reducers.

    Example: In Hadoop-20.2.0:

    > ls ~/hadoop-0.20.2/logs/userlogs/attempt_201209031127_0002_m_000000_0/
    log.index   stderr      stdout      syslog
    

    The above means:
    Hadoop_Installation: ~/hadoop-0.20.2
    job_id: job_201209031127_0002
    _m_: map task , "map number": _000000_

    4) open stdout if you used "system.out.println" or stderr if you used "system.err.append".

    PS. other hadoop versions might have a sight different hierarchy but they're all should be under $Hadoop_Installtion/logs/userlogs.

提交回复
热议问题