apache-nifi

问题 I am running MiNiFi in a windows server which is behind my company's firewall. My NiFi is running on Hortonworks cluster (3 nodes ex:sj46,sj47,sj48). I am trying to send data from the MiNiFi to NiFi. MiNiFi version: 0.4 NiFi version: 1.5 In MiNiFi, i have ListFile processor, FetchFile processor and RemoteProcessGroup. RemoteProcessGroup: URL: https://sjxx46.xxx:6689/nifi,https://sjxx47.xxx:6689/nifi,https://sjxx48.xxx:6689/nifi TransportProtocol: HTTP NiFi.properties: nifi.remote.input.host=

问题 I use MergeContent 1.3.0 in order to merge FlowFiles from 2 sources: 1) from ListenHTTP and 2) from QueryElasticsearchHTTP . The problem is that the merging result is a list of JSON strings. How can I convert them into a single JSON string? {"event-date":"2017-08-08T00:00:00"}{"event-date":"2017-02-23T00:00:00"}{"eid":1,"zid":1,"latitude":38.3,"longitude":2.4} I would to get this result: {"event-date":["2017-08-08T00:00:00","2017-02-23T00:00:00"],"eid":1,"zid":1,"latitude":38.3,"longitude":2

问题 I get a raw csv file which looks like this id,name,star 1,sachith,2 2,nalaka,1 3,abc,3 I want to map star column with another file where it has 1 1S 2 3S 3 5S and finally csv should look like id,name,star,level 1,sachith,2,3S 2,nalaka,1,1S 3,abc,3,5S I have used ReplaceTextWithMapping, but it replaces all the 1,2,3 values including in id column. Here it defines replacing a value, but I want to map and add a new column to the record. Edit: After @Upvote's answer. My ReplaceTextWithMapping conf

问题 I want to extract filename and store the filename in one of the existing column in the CSV file. How to do this? Which processor to use? what configuration? Ex- i have a filename 'FE_CHRGRSIM_20171207150616_CustRec.csv' and i want to extract ''FE_CHRGRSIM_20171207150616' and store this value under an existing column in the Same CSV file. Please help. TIA 回答1: Usually the "real" file name is available as an attribute on the flow file called "filename". You can use UpdateRecord with a

问题 I was exploring NiFi documentation. I must agree that it is one of the well documented open-source projects out there. My understanding is that the processor runs on all nodes of the cluster. However, I was wondering about how the content is distributed among cluster nodes when we use content pulling processors like FetchS3Object, FetchHDFS etc. In processor like FetchHDFS or FetchSFTP, will all nodes make connection to the source? Does it split the content and fetch from multiple nodes or

问题 I want to achieve the below request-response scenario in Nifi. My objective is to use each value coming as an array(object key 1,object key 2,...) in further differt processors. So if I can convert it into multiple JSON, then using split JSON I can use multiple vales later. Please suggest all sorts of solution for this. Input JSON: [ { "ID": "789654", "Date": "29th Feb", "Key" : ["object key 1", "object key 2", "object key 3"....] } ] Output JSON: [ { "ID": "789654", "Date": "29th Feb", "Key1

问题 I am new in niffi i am using getMongo to extract document from mongodb but same result is coming again and again but the result of query is only 2 document the query is {"qty":{$gt:10}} 回答1: There is a similar question regarding this. Let me quote what I had said there: " GetMongo will continue to pull data from MongoDB based on the provided properties such as Query, Projection, Limit. It has no way of tracking the execution process, at least for now. What you can do, however, is changing the

问题 my use case is to query a set of data for bunch of table and insert it into one table. so my nifi processor basically is like this: executeSql(query set of dataset) >>> convertAvrotoJson >>>> convetJSONtoSQL(convert to insert statement) >>>> putSQL (insert statement) >>> executeSQL(delete a row associated with first processor query in set of table) the problem is in the last executeSQL processor, when its not executing my sql, instead it keep trying to run putSQL query. i replace my delete

问题 I'm doing an install of Apache NiFi 1.3.0... It has a new way to install users, it appears to be only via the GUI. I'm curious if there's a way to boostrap a list of users by DN with policies set to them. I know there's the legacy users file... I'm looking at the users.xml file. It requires an identifier which maps to policies found in the authorizations.xml file. Old NiFi would have had me do it here: https://community.hortonworks.com/articles/886/securing-nifi-step-by-step.html 回答1: Here

问题 I want to make new processor which will be the reunion of GetFile and EvaluateXpath there are several subject i am interested in: How can i shrink my nar file now it is more than 20kb and my nifi can't run it? I want to get File from folder read it's data and put it as an atribute inside new flowfile,then rolling back config xml to it's original folder , how can i roll my config file back to folder b code? Here is simple code i use for getting attributes from xml config file : XPathExpression