apache storm reliablity timeout configuration

旧城冷巷雨未停 提交于 2019-12-11 03:46:20

问题


I have a nodejs->kafka>storm->Mongo deployed in Linux Ubuntu. Everything is normal originally. Then I changed the method in storm worker which makes storm worker process message very slow, around 1 minute per message, I notice the message is sent again and again from storm. I revert back to original method, everything is fine. (original method process time is 90ms per message).

I guess this is Storm reliability come into player. When message is not acknowledged, or time out, it sends message again.

If my guess is right, how to configure this timeout?

If my guess is wrong, why same message is sent twice or three times?


回答1:


You can set the timeout via configuration parameter Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS. See https://storm.apache.org/javadoc/apidocs/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS

The default value is 30 seconds, see defaults.yaml here: https://github.com/apache/storm/blob/master/conf/defaults.yaml

# maximum amount of time a message has to complete before it's considered failed
topology.message.timeout.secs: 30

When a tuple fails, it should show up in Storm UI and should be logged, too (maybe you need to adjust log level). So you can double check if a tuple times out or not.



来源:https://stackoverflow.com/questions/32039802/apache-storm-reliablity-timeout-configuration

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!