Update csv value using executescript processor fails in apache-nifi

问题

I try to read from a flowfile and update a record value using default value in csv. To that I have used ExecuteScript processor with following python code in it.

import sys
import re
import traceback
from org.apache.commons.io import IOUtils
from org.apache.nifi.processor.io import StreamCallback
from org.python.core.util import StringUtil
from java.lang import Class
from java.io import BufferedReader
from java.io import InputStreamReader
from java.io import OutputStreamWriter

flowfile = session.get()
record = flowfile.getAttribute('record_type')

if record == '0':
    flowfile = session.putAttribute(flowfile,'record_type', 'NEW_USER')
    session.transfer(flowFile, REL_SUCCESS)
    session.commit()
elif record == '1':
    flowfile = session.putAttribute(flowfile,'record_type', 'OLD_USER')
    session.transfer(flowFile, REL_SUCCESS)
    session.commit()
else:
    flowfile = session.putAttribute(flowfile,'record_type', 'IGNORE')
    session.transfer(flowFile, REL_SUCCESS)
    session.commit()

writer.flush()
writer.close()
reader.close()

My csv looks like

id,record_type
1,0
2,1
3,2
4,0

Result should be :

id,record_type
1,NEW_USER
2,OLD_USER
3,IGNORE
4,NEW_USER

I get following error :

AttributeError : 'NoneType' object has no attribute 'getAttribute' in script at line number 13

It says record = flowfile.getAttribute('record_type') this is wrong..

I have no idea how to solve this as I am not good with python.

回答1:

that,s not python, but according to comment from author could be groovy.

use ExecuteGroovyScript processor with following code:

def ff=session.get()
if(!ff)return

def map = [
    '0': 'NEW_USER',
    '1': 'OLD_USER',
]

ff.write{rawIn, rawOut->
    rawOut.withWriter("UTF-8"){w->
        rawIn.withReader("UTF-8"){r->
            int rowNum = 0
            //iterate lines from input stream and split each with coma
            r.splitEachLine( ',' ){row->
                if(rowNum>0){
                    //if not a header line then substitute value using map
                    row[1] = map[ row[1] ] ?: 'IGNORE'
                }
                //join and write row to output writer
                w << row.join(',') << '\n'
                rowNum++
            }
        }
    }
}

REL_SUCCESS << ff

来源：https://stackoverflow.com/questions/58850203/update-csv-value-using-executescript-processor-fails-in-apache-nifi

标签

python

csv

apache-nifi