apache-nifi

问题 We're running a NiFi 1.7.1 server that picks up files via 3 GetFile processors and uploads them to the cloud. When the server starts, it runs fine and chugs through any files that it finds. However, after running for a few days it seems to grind to a halt: The GetFile processors all show 1 thread running, but they don't seem to be doing anything, even though there are files present in their source directories. Nothing is waiting in any queues. No messages appear in the logs. The "top" command

问题 I am working on a Nifi flow where I am getting a JSON document with multiple key-value pairs. I am using the ExecuteScript processor with python . My goal here is to create various URLS bases on JSON keys . The keys are numerical and they look like this: keys = [10200, 10201, 10202, ...] The URLs I want are of 3 types and they should look like these: http://google.com/10200 http://bing.com/10200 http://yahoo.com/10200 I am trying to loop through my keys[] and create 3 specific urls for each

问题 I am working on a Nifi flow where I am getting a JSON document with multiple key-value pairs. I am using the ExecuteScript processor with python . My goal here is to create various URLS bases on JSON keys . The keys are numerical and they look like this: keys = [10200, 10201, 10202, ...] The URLs I want are of 3 types and they should look like these: http://google.com/10200 http://bing.com/10200 http://yahoo.com/10200 I am trying to loop through my keys[] and create 3 specific urls for each

问题 I know it's possible to use the rest-api to retrieve the values in the variable registry, but is there a way to set the variable values? When I change a variable I see that the following POST gets sent: http://localhost:8080/nifi-api/process-groups/{ID}/variable-registry/update-requests but I don't really understand where the VariableRegistryEntity comes into play or how I might go about manipulating the variables. More broadly, what I'm trying to do is manipulate variables inside a custom

问题 { "alamrs": [ { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ",

问题 I'm struggling with transformation using JOLT in Nifi My input [ { "value0": 0, "value1": 1, "value2": 2 }, { "value0": 3, "value1": 4, "value2": 5 } ] Desired Output: [ {"val" :0 }, {"val" :1 }, {"val" :2 }, {"val" :3 }, {"val" :4 }, {"val" :5 }, ] I almost managed to get it to work. Here is my (wrong) Jolt Spec: [ { "operation": "shift", "spec": { "*": { "value*": "[&1].val" } } } ] Here is my result: [ { "val" : [ 0, 1, 2 ] }, { "val" : [ 3, 4, 5 ] } ] Thanks! 回答1: Try this, [ { "operation

问题 My csv contain date,name,department 2020-2-4,sachith,{dep_name:computer,location:2323,3434} 2020-2-5,nalaka,{dep_name:engineering,location:3343,5454} final csv should be like : date,name,dep_name,lat,lot 2020-2-4,sachith,computer,2323,3434 2020-2-5,nalaka,engineering,3343,5454 here lat,lot are taken from location:3343,5454 data. I have tried to use UpdateRecord processor for this. In it has some ${field.value:join(','):substringAfter('dep_name:')} But its not working. How can I complete this

问题 Using GetFile I’m collecting files that include JSON records. Each record includes a parameter (let's name it Sensor) that have the Value A or B. Based on that value I wish to route each record to a different output port – which module I should use? Do I need to split the records or I can stay at the level of the file? An example of a record that can be found in the file {"EventTime":"2020-12-07 04:49:00", "Sensor":"A", "Keywords":-9223372036854775808, "EventType":"INFO", "SeverityValue":2,

问题 I have a JSON: { "reports": [ { "columnHeader": { "metricHeader": { "metricHeaderEntries": [ { "name": "ga:sessions", "type": "INTEGER" }, { "name": "ga:bounces", "type": "INTEGER" }, { "name": "ga:sessionDuration", "type": "TIME" }, { "name": "ga:pageviews", "type": "INTEGER" } ] } }, "data": { "rows": [ { "metrics": [ { "values": [ "25", "18", "1269.0", "27" ] } ] } ], "totals": [ { "values": [ "25", "18", "1269.0", "27" ] } ], "rowCount": 1, "minimums": [ { "values": [ "25", "18", "1269.0"

问题 Followed guide ( https://michalklempa.com/2019/04/nifi-registry-nginx-proxy-tls-basic-auth/ ) to set up nginx basic auth, however instead of proxy for nifi-registry I set it up for nifi. Auth is working and page is accessible but somehow processor configure window not opening. The issue is due to nginx since direct access to nifi through HTTP exposed ports works ,just not behind nginx proxy. below is the config I am using: server { listen 9988 ssl; root /usr/share/nginx/html; index index.html