how to do Transpose in corresponding few columns in pig/hive

后端 未结 2 511
囚心锁ツ
囚心锁ツ 2021-01-19 04:10

I was wondering is it possible to do transposition corresponding few columns in pig/hive.
as dealing with data i got below requirement

id      jan           


        
相关标签:
2条回答
  • 2021-01-19 04:35

    Yes this is definitely possible in Hive using the built-in "stack" UDF and a case statement. Something like this should work:

    select id, value, CASE
    WHEN value like 'j%'
    THEN 'jan'
    WHEN value like 'f%'
    THEN 'feb'
    WHEN value like 'm%'
    THEN 'march'
    ELSE ''
    END as month
    from table
    lateral view stack(3, jan, feb, march) tb as value
    ;
    

    Let me know if this works.

    0 讨论(0)
  • 2021-01-19 04:47

    Pig doesn't have any built-in function to solve your requirement, but you can try the below approach, i guess it will work for you.

    input.txt

    1       j1      f1      m1
    2       j2      f2      m2
    3       j3      f3      m3
    

    PigScript:

    A = LOAD 'input.txt' USING PigStorage() AS (id,month1,month2,month3);
    B = FOREACH A GENERATE FLATTEN(TOBAG(TOTUPLE(id,month1,'jan'),TOTUPLE(id,month2,'feb'),TOTUPLE(id,month3,'mar')));
    DUMP B;
    

    Output:

    (1,j1,jan)
    (1,f1,feb)
    (1,m1,mar)
    (2,j2,jan)
    (2,f2,feb)
    (2,m2,mar)
    (3,j3,jan)
    (3,f3,feb)
    (3,m3,mar)
    
    0 讨论(0)
提交回复
热议问题