Storm UI throwing “Offset lags for kafka not supported for older versions. Please update kafka spout to latest version.”

可紊 提交于 2019-12-24 08:14:10

问题


I have upgraded my hdp cluster to 2.5 and upgraded topology dependencies of storm-core to 1.0.1 and storm-kafka to 1.0.1. After deploying the new topology with new 1.0.1 dependencies everything is working as expected in the back end but storm UI not showing always zero for "Acked","Emitted", "Transferred" etc.

Storm UI shows a message "Offset lags for kafka not supported for older versions. Please update kafka spout to latest version." under "Topology spouts lag error" what does it mean ?


回答1:


Apache Storm in HDP 2.5 is actually not same as official Apache Storm 1.0.1 since it pulled more patches from the next versions (not released yet).

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_release-notes/content/patch_storm.html

The feature "Offset lag for Kafka" is introduced Storm 1.1.0 (not released yet) and requires relevant storm-kafka version (so starting at Storm 1.1.0, too).

You need to set the version of storm-core and storm-kafka to relevant (also might need to set HWX repo to your pom). You can see the version from the UI, or the result of './storm version', or daemons' log.




回答2:


I completely agree with "Jungtaek Lim" (above answer).

Apache Storm in HDP 2.5 is actually not same as official Apache Storm 1.0.1 since it pulled more patches from the next versions (not released yet).

I fixed this problem by pulling HDP 2.5 strom-kafka dependency from HDP repository (http://repo.hortonworks.com/content/repositories/releases/org/apache/storm/storm-kafka/1.0.1.2.5.0.0-1245/).



来源:https://stackoverflow.com/questions/39932441/storm-ui-throwing-offset-lags-for-kafka-not-supported-for-older-versions-pleas

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!