Merge two schemas into one in Apache nifi

问题

I'm trying to merge two csv files into a json using Apache nifi. Two csv's are persons.csv containing information about people:

Id|Name|Surname
ABC-123|John|Smith
ABC-111|Allan|Wood
ABC-001|Grace|Kelly

And the second csv contains list of events these people have attended:

EId|PId|Date|Desc
1|ABC-123|2017-05-01|"Groove party"
2|ABC-111|2017-06-01|"Snack No. One"
3|ABC-123|2017-06-01|"The night out"

I'm using a flow of (Nifi flow on git hub):

GetFile
UpdateAttribute (schema.name)
Split Records
ExtractText
UpdateAttribute (correlation.id, newschema)
Funnel
MergeRecords / Merge Content
PutFile

Trying to achieve final json:

{
"Person": {
    "Id": "ABC-123",
    "Name": "John",
    "Surname": "Smith",
    "Events": [{
        "Date": "2017-05-01",
        "Name": "Groove party"
    }, {
        "Date": "2017-06-01",
        "Name": "The night out"
    }]
}
}

But I'm not sure how to set up Merge Record, or how to join multiple csv lines after Merge Content into a single json. Is there a way how to do it?

回答1:

You can actually achieve this using

1- ConvertRecord(CSV to JSON) - using Avro Schema in your case

CSVReader

JsonRecordSetWriter

AvroSchemaRegistry

   {
 "name": "person",
 "namespace": "nifi",
 "type": "record",
 "fields": [
 {"name": "Id" , "type" : "string"},
 {"name": "Name" , "type" : "string"},
 {"name": "Surname" , "type" : "string"}
 ]
}

2 - SplitJson
3 - EvaluateJson
4 - LookupAttribute (Get Desc SimpleCsvFileLookupService)
5 - LookupAttribute (Get Date SimpleCsvFileLookupService)
- 6 - AttriutetoJson

来源：https://stackoverflow.com/questions/50183026/merge-two-schemas-into-one-in-apache-nifi

标签

merge

apache-nifi