Debezium mongodb connector 0.3.6 automatically stops tailing mongodb , until restarted at-least once

大城市里の小女人 提交于 2020-01-15 04:00:27

问题


I am using debezium mongodb connector 0.3.6 , running inside a docker container. I have been monitoring kafka-connect for some time ,and found that the connector stops tailing mongodb and sending change events to kafka brokers automatically. Upon investigation , I found that sometimes, after some time of inactivity, it's mongo connection is refused, and upon retrial, it connects successfully ,and sends the large number of records that it has not sent during inactive period. But this is not the case everytime, as I have encountered this inactivity for 1 hour straight, and upon restarting the connector deliberately, I can get it to tail the oplog properly, and hence sends an enormous number of records that were pending during inactivity.

I am attaching 2 log files: - 1- consisting logs where the mongo connection is refused, and it automatically retry and connects successfully. (https://ufile.io/c5v5g) 2- consisting logs where for 1 hour, it was not tailing mongodb oplog, and no change events were sent to kafka , even when new events were written in mongodb oplog. (https://ufile.io/9g4gz)

I am not able to find a particular problem here. My kafka-connect container, mongodb host, and kafka brokers are all running on different hosts, and connected by public ip.

来源:https://stackoverflow.com/questions/44628165/debezium-mongodb-connector-0-3-6-automatically-stops-tailing-mongodb-until-res

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!