NiFi blocked/hanging

孤者浪人 提交于 2021-02-08 02:57:43

问题


We're running a NiFi 1.7.1 server that picks up files via 3 GetFile processors and uploads them to the cloud.

When the server starts, it runs fine and chugs through any files that it finds. However, after running for a few days it seems to grind to a halt:

  • The GetFile processors all show 1 thread running, but they don't seem to be doing anything, even though there are files present in their source directories.
  • Nothing is waiting in any queues.
  • No messages appear in the logs.
  • The "top" command shows java using about 3% CPU and 21% mem. This is a 4-CPU server with 8GB of memory.
  • If I try to stop any processors via the web interface, it will become unresponsive. Upon reloading the page, I get the login screen, but after login it hangs on the loading animation, without showing the flow schema.
  • If I restart the NiFi service, it suddenly runs fine: it will pick up all the waiting files and not leave any threads hanging (according to the web interface). This will last another few days...

What is going on here? How can I resolve this?

Edit: The three GetFile processors each read from a different folder, but they all send their files to the same place. They are configured to pick up all files (filepattern .*), poll every 10 seconds, minimum file age 1 minute, don't keep the source file. I didn't touch the scheduling tab so it's just defaults.

来源:https://stackoverflow.com/questions/53434957/nifi-blocked-hanging

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!