apache-nifi

问题 I trying to use InvokeHttpProcessor in Apache NiFi to perform POST request with complex JSON body. Accordingly this tutorial: http://www.tomaszezula.com/2016/10/30/nifi-and-http-post-configuration I know how to use UpdateAttribute processor to add name/value pairs and then apply an additional transformation via AttributesToJSON . But how to deal with complex JSON? For example I have to perform request to GoogleAnalytics reporting API, so I need to perform this request: POST https:/

问题 I am following this link to set up Nifi putHDFS to write to Azure Data Lake.Connecting to Azure Data Lake from a NiFi dataflow The Nifi is within HDF 3.1 VM and the Nifi version is 1.5. We got the jar files mentioned in the above link, from a HD Insight(v 3.6, which supports hadoop 2.7) head node, these jars are: adls2-oauth2-token-provider-1.0.jar azure-data-lake-store-sdk-2.1.4.jar hadoop-azure-datalake.jar jackson-core-2.2.3.jar okhttp-2.4.0.jar okio-1.4.0.jar And they are copied to the

问题 I am going to convert a log file events (which is recorded by LogAttribute processor) to JSON. I am using 'ExtractGroke' with this configuration: STACK pattern in pattern file is (?m).* Each log has this format: 2019-11-21 15:26:06,912 INFO [Timer-Driven Process Thread-4] org.apache.nifi.processors.standard.LogAttribute LogAttribute[id=143515f8-1f1d-1032-e7d2-8c07f50d1c5a] logging for flow file StandardFlowFileRecord[uuid=02eb9f21-4587-458b-8cee-ad052cb8e634,claim=StandardContentClaim

问题 I am going to convert a log file events (which is recorded by LogAttribute processor) to JSON. I am using 'ExtractGroke' with this configuration: STACK pattern in pattern file is (?m).* Each log has this format: 2019-11-21 15:26:06,912 INFO [Timer-Driven Process Thread-4] org.apache.nifi.processors.standard.LogAttribute LogAttribute[id=143515f8-1f1d-1032-e7d2-8c07f50d1c5a] logging for flow file StandardFlowFileRecord[uuid=02eb9f21-4587-458b-8cee-ad052cb8e634,claim=StandardContentClaim

问题 Below is my flow: GetFile > ExecuteSparkInteractive > PutFile I want to read files from GetFile processor in ExecuteSparkInteractive processor, apply some transformations and put it in some location. Below is my flow I wrote spark scala code under code section of spark processor: val sc1=sc.textFile("local_path") sc1.foreach(println) There is nothing happening in the flow. So how can I read files in spark processor using GetFile processor. 2nd Part: I tried below flow just for practice:

问题 I need to use PrioritizeAttributePrioritizer in NiFi. i have observed that prioritizers in below reference. https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings if i receive 10 flowfiles then i need to set the priority value for every flow file to be unique. After that specify queue configuration must be PrioritizeAttributePrioritizer. Then processing flowfiles based on priority value. How can i set priority value for seperate flow files or which prioritizer in Nifi to be work

问题 I'm trying to get my head around on extracting attributes from Avro and JSON. I'm able to extract attributes from JSON by using EvaluateJsonPath processor. I'm trying to do the same on Avro, but i'm not sure whether it is achievable. Here is my flow, ExecuteSQL -> SplitAvro -> UpdateAttribute UpdateAttribute is the processor where i want to extract the attributes. Please find below snapshot of UpdateAttribute processor, So, my basic question is, could we extract attributes form Avro? If yes,

问题 I'm new to nifi and i want to connect SQL server database to nifi and create a data flow with the processors. how can I do this, can any one Help me with this clearly. Thanks in Advance sam 回答1: Here are two great articles on getting information in and out of databases with NiFi: http://www.batchiq.com/database-injest-with-nifi.html http://www.batchiq.com/database-extract-with-nifi.html They describe/illustrate how to configure a DBCPConnectionPool service to provide connection(s) to an RDBMS

问题 I am new to Apache NIFI tool. I am trying to import data from mongo db and put that data into the HDFS. I have created 2 processors one for MongoDB and second for HDFS and I configured them correctly. The job is running successfully and storing the data into HDFS but the job should terminate automatically on success. But it is not, and creating too many files in HDFS. I want to know how to make On Demand Job in NIFI and how to determine that a job is successfull. 回答1: GetMongo will continue

问题 I have a processor which depends on some jar as a dependency. However changes are made to the dependency jar quite frequenctly. So I want to be able to specify the jar path as a property of processor and let the nifi load the jar each time I modify the path in the property and restart the processor. This is supposed to be doable using dynamicallyModifiesClasspath as explained in this article. However I am unable to do this. Below is my code of hello world program using