Passing a list to Javascript UDF in Apache Pig

ぃ、小莉子 提交于 2019-12-11 22:01:39

问题


If I have an array of stuff in Pig, like so:

datas   = load './data.txt' using PigStorage( '\t');
list    = load './frobdata.txt' using PigStorage();

And I want to pass these on to a UDF, like so:

register './enfrobinate.js' using javascript as frob;
frobbed = foreach datas generate flatten( frob.enfrobinate( list, $0 ) );

I cannot seem to find a prototype that works for passing a list to javascript, and the Pig documentation is not real clear on datatypes for Javascript UDFs.

I am aware of cross in Pig. This is not what I need (It gives me a cartesian product. Which is okay, except when we start getting very huge lists. The 'list' in this case is a few thousand items and the datas is many millions of items.)

来源:https://stackoverflow.com/questions/19843452/passing-a-list-to-javascript-udf-in-apache-pig

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!