Hive: Populate other columns based on unique value in a particular column

问题

I have a two tables in hive as mentioned below in Hive

Table 1:

id   name value
1    abc  stack
3    abc  overflow
4    abc  foo
6    abc  bar

Table 2:

id   name value       
5    xyz  overflow       
9    xyz  stackoverflow 
3    xyz  foo
23   xyz  bar

I need to take the count of value column without considering the id and name column.

Expected output is

id name value
1  abc  stack
9  xyz  stackoverflow

I tried this and works in other databases but not in hive

select id,name,value from
 (SELECT id,name,value FROM table1  
   UNION ALL 
  SELECT id,name,value FROM table2) t 
 group by value having count(value) = 1;

Hive expects group by clause like mentioned below.

select id,name,value from
  (SELECT id,name,value FROM table1  
    UNION ALL 
  SELECT id,name,value FROM table2) t 
 group by id,name,value having count(value) = 1;

and gives the output

id   name value
1    abc  stack
3    abc  overflow
4    abc  foo
6    abc  bar
5    xyz  overflow       
9    xyz  stackoverflow 
3    xyz  foo
23   xyz  bar

We will have to give all the columns in group by which we are using in select clause. but when i give it considers all the columns and the result is different than expected.

回答1:

Calculate analytic count(*) over(partition by value). Testing with your data example:

with 

table1 as (
select stack (4,
              1,'abc','stack',
              3,'abc','overflow',
              4,'abc','foo',
              6,'abc','bar'
             ) as (id, name, value)
),

table2 as (
select stack (4,
              5,  'xyz','overflow',      
              9,  'xyz','stackoverflow',
              3,  'xyz','foo',
              23, 'xyz','bar'
             ) as (id, name, value)
)

select id, name, value
from(
select id, name, value, count(*) over(partition by value) value_cnt
 from
(SELECT id,name,value FROM table1  
  UNION ALL 
 SELECT id,name,value FROM table2) s
)s where value_cnt=1;

Result:

OK
id      name    value
1       abc     stack
9       xyz     stackoverflow
Time taken: 55.423 seconds, Fetched: 2 row(s)

回答2:

You can try below -

seELECT id,name,value FROM table1 a left join table2 b on a.value=b.value
where b.value is null
UNION ALL SELECT 
seELECT id,name,value FROM table2 a left join table1 b on a.value=b.value
where b.value is null

来源：https://stackoverflow.com/questions/55156112/hive-populate-other-columns-based-on-unique-value-in-a-particular-column

标签

sql

Hive