Logging from a storm bolt - where is it going?

喜夏-厌秋 提交于 2019-12-06 23:22:29

问题


I have several bolts deployed to a topology on a cluster. Each is configured to log via slf4j. On the test machine I get both the stdout and the file appenders working fine.

When I deploy this to the cluster the logging seems to have disappeared. I don't get anything in the storm logs (on the supervisor machines), to /var/log/* or anywhere else as far as I can tell.

Should I be able to use a logging system inside a storm worker? If so, is there a trick to getting the messages?

Machines are all running CentOS 6.6 x64


回答1:


This blog post suggest a method to find location of log files on storm cluster. http://www.saurabhsaxena.net/how-to-find-storm-worker-log-directory/

When topology is deployed on cluster, topology logs are written in worker*.log files.

As mentioned in the blog post

  • First step would be to fine the log directory location. This can be done using 'ps' command. Search for nimbus process $ ps aux|grep nimbus
  • Look for -Dstorm.log.dir in the output.
  • Go to log dir and look into worker*.log file for topology logs.



回答2:


In my case (using the official storm docker image) logs were in the supervisor container:

/logs/workers-artifacts/MyTopology-1-123123123/123/worker.log



回答3:


You can set storm.workers.artifacts.dir parameter in storm.yaml and from now on, workers artifacts (including logs) will be saved in that path in a folder which is named after your topology name.



来源:https://stackoverflow.com/questions/30720472/logging-from-a-storm-bolt-where-is-it-going

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!