apache-nifi

问题 For instance, in a custom processor I may wish to evaluate simply the expression in the String "${UUID()}" (just as an example). I don't want to expose an attribute to the user, I just want to evaluate the expression. Can I do that? 回答1: in a custom processor (or script processor) import org.apache.nifi.components.PropertyValue; ... String expression = "${UUID()}"; PropertyValue myValue = context.newPropertyValue( expression ); in this case it's enough to call this to evaluate expression

问题 I have two CSV files: Identity(no,name,Age) which has 10 rows Location(Address,no,City) which has 100 rows I need to extract rows and check the no column in the Identity with Location CSV files. Get the single row from Identity CSV file and check Identity.no with Location.no having 100 rows in Location CSV file. If it is matching then combine the name, Age, Address, City in Identity, Location Note: I need to get 1st row from Identity compare it with 100 rows in Location CSV file and then get

问题 I have running nifi instance 1.2.0. I just tried to use "Wait\Notify" Processor with following reference http://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/#why-mergecontent-dont-suffice Drag the template in canvas and i try to running it. It shows following error in "Wait/Notify" processor. Unable to communicate with cache when processing StandardFlowFileRecord[uuid=faab337f-034c-4137-a2f3-abb46f22b474,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id

问题 I'm trying to load the flow files into MySQL database using bulk load option. Below is the query I'm using as part of the UpdateAttribute processor and passing that query to PutSQL after updating the parameters to do bulk load. LOAD DATA INFILE '${absolute.path}${filename}' INTO TABLE ${dest.database}.${db.table.name} FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' When I ran the flow it's failing saying file not found exception. . There were a total of 1 FlowFiles that failed, 0 that

问题 I'm trying to load the flow files into MySQL database using bulk load option. Below is the query I'm using as part of the UpdateAttribute processor and passing that query to PutSQL after updating the parameters to do bulk load. LOAD DATA INFILE '${absolute.path}${filename}' INTO TABLE ${dest.database}.${db.table.name} FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' When I ran the flow it's failing saying file not found exception. . There were a total of 1 FlowFiles that failed, 0 that

问题 This is the follow up of Import Modules in Nifi ExecuteScript I am new to python as well as nifi. I am trying to execute my python script in ExecuteScript processor. I want to access a server. so i used paramiko client. But when i run the processor, it shows "Import error No module named constant_time" at line session.write(). Though i have this constant_time.py under "/usr/local/lib/python2.7/dist-packages/ " I have also the path "/usr/local/lib/python2.7/dist-packages/ " in sys.path. I have

问题 I have two tables one in mysql test.employee and other in hive default.dept I want to pass empid of test.employee table as a parameter to query in hive table and store data into HDFS ExecuteSQL -> select empid from test.employee (gives 10 records) SelectHiveQL -> SELECT * FROM default.dept where empid = ${empid} (should retrieve 10 records) image description here 回答1: You could do the following: ExecuteSQL - to retrieve the employee records ConvertAvroToJson - for later processing of empid

问题 I am testing out NiFi to replace our current ingestion setup which imports data from multiple MySQL shards of a table and store it in HDFS. I am using GenerateTableFetch and ExecuteSQL to achieve this. Each incoming flow file will have a database.name attribute which is being used by DBCPConnectionPoolLookup to select the relevant shard. Issue is that, let's say I have 2 shards to pull data from, shard_1 and shard_2 for table accounts and also I have updated_at as Maximum Value Columns , it

问题 I want to use 'DBCPConnectionPoolLookup' controller service in 'ExecuteGroovyScript' processor. I set ' database.name ' But I get this Error: This is 'ExecuteGroovyScript' configs: I found someone who had same problem and solved it here, but his approach to solve the problem is not complete. I think I should use CTL as described in answer part of this question but I got this Error: This is my config: Please let me know what is the problem? update : I removed try/catch from code and got these

问题 I have Json array like this; [ { "item_id": "", "timestamp": "", "type": "", "value": "" }, { "item_id": "", "timestamp": "", "type": "", "value": "" } ] I want to create row key for HBase as like this : item_id timestamp type But PutHBaseRecord waits one field from me. What can I do? Can I do something like that? For record reader = JsonPathReader And, rowId = concat($item_id, $timestamp, $type) ???? 回答1: Use UpdateRecord processor before PutHbaseRecord processor. Defines Record Reader