问题
I've written a UDF which takes a constructor parameter.
I've successfully initialized and used it in grunt
as
grunt> register mylib.jar
grunt> define Function com.company.pig.udf.MyFunction('param-value');
But I can't initialize it using a Pig parameter as in
grunt> define Decrypt com.company.pig.udf.MyFunction($secret);
or
grunt> define Decrypt com.company.pig.udf.MyFunction('$secret');
I tried to initialize $secret
using both -param
and -param_file
options.
Are Pig parameters supported as UDF constructor arguments at all?
The function I'm loading is used to decrypt some data and the param-value contains special characters as "%$!" but no quotes. So I'm unsure whether I'm running into an value expansion issue or a parameter there is just not being expanded.
I'm setting the parameter value without quotes in a param file:
secret = My$ecr%t!
回答1:
It is definitely possible to pass parameter values to UDFs. The problem seems to be the $
sign.
This should the correct syntax:
define Decrypt com.company.pig.udf.MyFunction('$secret');
In your parameter file put:
secret=My\$ecr%t!
The -dryrun option can be helpful for this it will create a file called <nameofpigfile>.pig.subsituted
and not run the actual script.
pig -x local -dryrun -f mypigfile.pig -param_file myparameterfile
Also, you probably need pig >= 0.11
https://issues.apache.org/jira/browse/PIG-2931
来源:https://stackoverflow.com/questions/27803282/is-it-possible-to-pass-the-value-of-a-parameter-to-an-udf-constructor