How to delete and update a record in Hive

Question

I have installed Hadoop, Hive, Hive JDBC. which are running fine for me. But I still have a problem. How to delete or update a single record using Hive because delete or upd

Answer 1

Once you have installed and configured Hive , create simple table :

hive>create table testTable(id int,name string)row format delimited fields terminated by ',';

Then, try to insert few rowsin test table.

hive>insert into table testTable values (1,'row1'),(2,'row2');

Now try to delete records , you just inserted in table.

hive>delete from testTable where id = 1;

Error! FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations.

By default transactions are configured to be off. It is been said that update is not supported with the delete operation used in the conversion manager. To support update/delete , you must change following configuration.

cd  $HIVE_HOME
vi conf/hive-site.xml

Add below properties to file

  hive.support.concurrency
  true
 
 
  hive.enforce.bucketing
  true
 
 
  hive.exec.dynamic.partition.mode
  nonstrict
 
 
  hive.txn.manager
  org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
 
 
  hive.compactor.initiator.on
  true
 
 
  hive.compactor.worker.threads
  2
 

Restart the service and then try delete command again :

Error!

FAILED: LockException [Error 10280]: Error communicating with the metastore.

There is problem with metastore. In order to use insert/update/delete operation, You need to change following configuration in conf/hive-site.xml as feature is currently in development.

  hive.in.test
  true
 

Restart the service and then delete command again :

hive>delete from testTable where id = 1;

Error!

FAILED: SemanticException [Error 10297]: Attempt to do update or delete on table default.testTable that does not use an AcidOutputFormat or is not bucketed.

Only ORC file format is supported in this first release. The feature has been built such that transactions can be used by any storage format that can determine how updates or deletes apply to base records (basically, that has an explicit or implicit row id), but so far the integration work has only been done for ORC.

Tables must be bucketed to make use of these features. Tables in the same system not using transactions and ACID do not need to be bucketed.

See below built table example with ORCFileformat, bucket enabled and ('transactional'='true').

hive>create table testTableNew(id int ,name string ) clustered by (id) into 2 buckets stored as orc TBLPROPERTIES('transactional'='true');

Insert :

hive>insert into table testTableNew values (1,'row1'),(2,'row2'),(3,'row3');

Update :

hive>update testTableNew set name = 'updateRow2' where id = 2;

Delete :

hive>delete from testTableNew where id = 1;

Test :

hive>select * from testTableNew ;