Using IN clause with PIG FILTER

痞子三分冷 提交于 2019-12-05 00:20:46

问题


Does PIG support IN clause?

filtered = FILTER bba BY reason not in ('a','b','c','d');

or should i split it up into multiple OR's?

Thanks!


回答1:


I didn't find it in any of the samples in the documentation.

You can get by using AND/OR/NOT




回答2:


You can use below udf from Apache DataFu instead. This will help you to avoid writing lot of OR.

https://github.com/linkedin/datafu/blob/master/src/java/datafu/pig/util/InUDF.java




回答3:


Pig 0.12 added In operator http://www.edureka.co/blog/operators-in-apache-pig-diagnostic-operators/ see bottom of page..release notes. Haven't located it in official docs (apart from bare mention in release notes)




回答4:


No, Pig doesn't support IN Clause. I had a similar situation. Though you can use AND operator and filter keyword as a work around. like

A= LOAD 'source.txt' AS (user:chararray, age:chararray);

B= FILTER A BY ($1 matches 'tapan') AND ($1 matches 'superman');

However, if the number of filtering required is huge. Then, probably, you can just create a relation that contains all these keywords and do a join to filter wherever the occurrence matches. Hope this helps.




回答5:


We can use IN clause as follows:

A = FILTER alias_name BY col_name IN (val1, val2,...,valn);

DUMP A;



回答6:


you can do this likes:

X = FILTER bba BY NOT reason IN ('a','b','c','d');

more info



来源:https://stackoverflow.com/questions/7179165/using-in-clause-with-pig-filter

标签

工具导航Map

JSON相关