Merge two JSON flowfile together in NiFi

余生颓废 提交于 2020-01-24 09:55:06

问题


i want to merge two flowfile that contain JSON object by same specified attribute...

flow1:

attribute:    
xuuid = 123456

content:
{
"sname":"jack",
"id":"00001",
"state":"NY"
}

flow2:

attribute:    
xuuid = 123456

content:
{
"country":"US",
"date":"1983"
}

and i expect this form of data in single output flow:

desired_flow:

attribute:    
xuuid = 123456

content:
{
"sname":"jack",
"id":"00001",
"state":"NY",
"country":"US",
"date":"1983"
}

how do i play with this? MergeContent processor or MergeRecord? i think mergerecord can handle it but i confusing on it.


回答1:


What you are asking for is a streaming join and it is not something that NiFi really does, similar question and answer here:

https://stackoverflow.com/a/42909221/5650316

The merge processors are made to merge pieces of data one after another, not to perform a streaming join. For example, if you have many small json messages you would want to use MergeContent or MergeRecord to merge together thousands of them into a single flow file before writing to HDFS.




回答2:


Yes MergeContent can do this for you.

I use EvalJson --> MergeContent --> AttributesToJson

I have posted a template here you can use to play around. Apache NiFi Merge Json Template

The MergeContent must have this settings : "Keep all attributes","2 a num of entires" ,"Delimiters strategy is Text"



来源:https://stackoverflow.com/questions/52137516/merge-two-json-flowfile-together-in-nifi

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!