apache-nifi

Apache Nifi: Replacing values in a column using Update Record Processor

廉价感情. 提交于 2021-01-28 04:21:37
问题 I have a csv, which looks like this: name,code,age Himsara,9877,12 John,9437721,16 Razor,232,45 I have to replace the column code according to some regular expressions. My logic is shown in a Scala code below. if(str.trim.length == 9 && str.startsWith("369")){"PROB"} else if(str.trim.length < 8){"SHORT"} else if(str.trim.startsWith("94")){"LOCAL"} else{"INT"} I used a UpdateRecord Processor to replace the data in the code column. I added a property called /code which contains the value. $

Python ExecuteScript in NiFi: Transform flowfile attributes & content

筅森魡賤 提交于 2021-01-28 02:55:03
问题 I am trying to create a Python script in NiFi that: Reads some attributes from an incoming flowfile Read the json content of the flowfile & extract specific fields Write attributes to outgoing flowfile Overwrite incoming flowfile with new content that is created in the script (e.g. API call that returns new json) and send it to SUCCESS relationship OR remove the old flowfile and create new with desired content What i ve done so far: import json import java.io from org.apache.commons.io import

Apache Nifi decompression

你离开我真会死。 提交于 2021-01-27 21:18:29
问题 I'm new to Apache NIFI and trying to build a flow as a POC. I need your guidance for the same. I have a compressed 'gz' file say 'sample.gz' containing a file say 'sample_file'. I need to decompress the sample.gz file and store 'sample_file' in a hdfs location. I'm using GetFile processor to get the sample.gz file, CompressContent processor in decompress mode to decompress the same file and PutHDFS processor to put the decompressed file in HDFS location. After running the flow, I can find

NiFi UI not accessible when started from docker-compose

隐身守侯 提交于 2021-01-05 05:55:57
问题 For the life of me I can't get to the NiFi Web UI. It makes me hate security. TLDR; I can't find the right way to start NiFi in a docker container and still access the UI. Here's what I've tried (for 8 hours): docker run --name nifi \ -p 8080:8080 \ -d \ apache/nifi:latest I go to localhost:8080/nifi - timeout. Same on 127.0.0.1 . docker inspect nifi - IP Gateway is 172.20.0.1 with actual IP of 172.0.0.2 . Invalid Host Header and timeout, respectively. Start randomly trying stuff: # I tried