how to control retries of invoke HTTP processor in nifi

﹥>﹥吖頭↗ 提交于 2019-12-11 10:38:57

问题


For e.g. invoke HTTP processor, if the retry relationship is connected to itself, how to control the number of retries if 500 related error occurred?

I want to control the number of retries up to some count for e.g. 5 and it much retry at certain time. First retry after 1 min and second after 30 min third after 24 hrs like this I want to do.


回答1:


There is an open Jira case (NIFI-90) discussing automatic penalization and back-off. In the meantime, if you want to implement your own back-off, you would send your retry relationship to a flow that would eventually be routed back to the InvokeHttp processor (or dropped once the retry count reached the max). The back-off cycle could look like this:

InvokeHttp -[retry]-> UpdateAttribute -> RouteOnAttribute -[give up]-> (Drop)
 ^                                               |
 |                                               v
 |------------------------------------------  (Delay)
  1. UpdateAttribute: Sets/increments a "counter/retry" attribute and/or a correlated "delay amount" attribute.
  2. RouteOnAttribute: Checks the counter to see if the max number of retries (5, e.g.) has been reached, and sends the flow file to (Drop) if so, otherwise continue on. The (Drop) processor can be an UpdateAttribute or something that auto-terminates its outgoing relationship, or some error-handling/reporting logic.
  3. (Delay): This could be an ExecuteScript processor that delays the transfer of a flow file based on either the current retry count and/or the delay amount. Alternatively you could use ControlRate, but you'd inverse the "delay amount" to set the attribute for ControlRate accordingly (using the previous UpdateAttribute to increase the Rate Controlled Attribute's value).



回答2:


I really liked this solution, suggested by Alessio Palma (Scroll to find his response). Looks less messy to me. I do wish processors would have retry/timeout option or even something global on a process group level.



来源:https://stackoverflow.com/questions/43843191/how-to-control-retries-of-invoke-http-processor-in-nifi

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!