Converting columns to rows (UNPIVOT) in hiveql

风流意气都作罢 提交于 2019-12-02 00:31:16

问题


I have a table with a structure like this:

column1, column2, column3, X1, X2, X3, X4
A1,      A2,      A3,      5,  6,  1,  4

I would like to convert this into

column1, column2, column3, Key, Value 
 A1,      A2,      A3,      X1,  5
 A1,      A2,      A3,      X2,  6 
 A1,      A2,      A3,      X3,  1
 A1,      A2,      A3,      X4   4

I was able to do this already using 4 queries stitched together with "UNION ALL", but since the table is huge and each select translates into a lengthy map-reduce, using UNION makes the query takes N times the time it should ideally take. Where N is number of columns to pivot.

I tried exploring the explode() pre-defined UDTF, but I am not able to work it in this example. I tried something like the following, but am not able to make the syntax work.

select column1, column2, column3, explode(Map('X1':X1, 'X2':X2, ..))

Can someone please point out exactly how to make this work?? I am guessing I could roll my own UDTF, but am hoping this is something pretty standard?

EDIT: There is another question on stackoverflow where something similar was asked, but the formulation is convoluted and in my opinion the wrong answer is currently marked as the correct answer. I think this question is more succint and to the point.


回答1:


Whoops, posted this in a hurry it seems. I have the answer. Posting it here for others who might find this useful. Here is the correct syntax to deal with map and explode to achieve this.

select column1, column2, column3, m_key, m_val from
    (select column1, column2, column3, map("X1", X1, "X2", X2, "X3", X3, "X4", X4) as map1
    from table1) as t1
lateral view explode(map1) xyz as m_key, m_val    


来源:https://stackoverflow.com/questions/38064412/converting-columns-to-rows-unpivot-in-hiveql

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!