Is it possible to pass the value of a parameter to an UDF constructor?

南楼画角 提交于 2019-12-11 03:39:32

问题


I've written a UDF which takes a constructor parameter.

I've successfully initialized and used it in grunt as

grunt> register mylib.jar
grunt> define Function com.company.pig.udf.MyFunction('param-value');

But I can't initialize it using a Pig parameter as in

grunt> define Decrypt com.company.pig.udf.MyFunction($secret);

or

grunt> define Decrypt com.company.pig.udf.MyFunction('$secret');

I tried to initialize $secret using both -param and -param_file options.

Are Pig parameters supported as UDF constructor arguments at all?

The function I'm loading is used to decrypt some data and the param-value contains special characters as "%$!" but no quotes. So I'm unsure whether I'm running into an value expansion issue or a parameter there is just not being expanded.

I'm setting the parameter value without quotes in a param file:

secret = My$ecr%t!

回答1:


It is definitely possible to pass parameter values to UDFs. The problem seems to be the $ sign.

This should the correct syntax:

define Decrypt com.company.pig.udf.MyFunction('$secret');

In your parameter file put:

secret=My\$ecr%t!

The -dryrun option can be helpful for this it will create a file called <nameofpigfile>.pig.subsituted and not run the actual script.

pig -x local -dryrun -f mypigfile.pig -param_file myparameterfile

Also, you probably need pig >= 0.11

https://issues.apache.org/jira/browse/PIG-2931



来源:https://stackoverflow.com/questions/27803282/is-it-possible-to-pass-the-value-of-a-parameter-to-an-udf-constructor

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!