How do you decode JSON in Pig that comes from a column?

问题:

There are numerous examples of how to use JsonLoader() to load JSON data with a schema from a file, but not from any sort of other output.

回答1:

You are looking for the JsonStringToMap UDF provided in Elephant Bird: https://github.com/kevinweil/elephant-bird/search?q=JsonStringToMap&ref=cmdform

Sample File:

foo     bar     {"version":1, "type":"an event", "count": 1} foo     bar     {"version":1, "type":"another event", "count": 1} 

Pig Script:

REGISTER /path/to/elephant-bird.jar; DEFINE JsonStringToMap com.twitter.elephantbird.pig.piggybank.JsonStringToMap(); raw = LOAD '/tmp/file.tsv' USING PigStorage('\t') AS (col1:chararray,col2:chararray,json_string:chararray); parsed = FOREACH raw GENERATE col1,col2,JsonStringToMap(json_string); ILLUSTRATE parsed; -- Just to show the output 

Pre-processing (JSON as chararray/string):

------------------------------------------------------------------------------------------------------- | raw     | col1:chararray    | col2:chararray    | json_string:chararray                             |  ------------------------------------------------------------------------------------------------------- |         | foo               | bar               | {"version":1, "type":"another event", "count": 1} |  

Post-processing (JSON as map):

---

------------------------------------------------------------------------------------------------- | parsed     | col1:chararray    | col2:chararray    | json:map(:chararray)                     |  ------------------------------------------------------------------------------------------------- |            | foo               | bar               | {count=1, type=another event, version=1} |  ------------------------------------------------------------------------------------------------- 

This question is a duplicate of How to parse a JSON string from a column with Pig

文章来源: How do you decode JSON in Pig that comes from a column?