How to order files in NiFi based on a timestamp in the filename?

混江龙づ霸主 提交于 2019-12-11 18:34:09

问题


I'm listing files from a directory, then parsing a timestamp out of their filename. I then need to process the files in order. I tried to use the EnforceOrder processor, but it's not designed to enforce order when there are large numerical gaps between the elements, like there would be in timestamps. The Priority Attribute Prioritizer has other issues...

  1. It processes higher priorities first, so it would process them in reverse.
  2. I dont see that it has any ability to queue for a certain period of time. So unless files were all dropped at the same time, it couldn't guarantee the order.

Is there a recommended approach for what I'm attempting to do?

Filenames would have a timestamp with some specified format (below is MM-dd-yyyy)

  • MyFile_11-30-2018.txt
  • MyFile_3-28-2018.txt

They need to be processed in order chronologically based on that timestamp. So for those two files, the second one would need to be processed before the first.

来源:https://stackoverflow.com/questions/51898804/how-to-order-files-in-nifi-based-on-a-timestamp-in-the-filename

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!