Skip first line of csv while loading in hive table

问题

Hello Friends,

I created table in hive with help of following command -

CREATE TABLE db.test 
  ( 
     fname STRING, 
     lname STRING, 
     age   STRING, 
     mob   BIGINT 
  ) row format delimited fields terminated BY '\t' stored AS textfile;

Now to load data in table from file, I am using following command -

load data local inpath '/home/cluster/TestHive.csv' into table db.test;

Problem is, all the rows are getting inserted, and I don't want first row because it contains only column names.

Please suggest me a way to skip first line.

Thanks in advance.

回答1:

To get this you can use hive's property which is TBLPROPERTIES ("skip.header.line.count"="1")
you can also refer example -

CREATE TABLE temp 
  ( 
     name STRING, 
     id   INT 
  ) 
row format delimited fields terminated BY '\t' lines terminated BY '\n' 
tblproperties("skip.header.line.count"="1");

回答2:

Just for those who have already created the table with the header. Here is the alter command for the same.

ALTER TABLE tablename SET TBLPROPERTIES ("skip.header.line.count"="1");

回答3:

Just don't mix double quotes and single quotes in the same CLI command:

CREATE TABLE db.test 
  ( 
     fname STRING, 
     lname STRING, 
     age   STRING, 
     mob   BIGINT 
  ) row format delimited fields terminated BY '\t' tblproperties('skip.header.line.count'='1') stored AS textfile;

otherwise hive load NULL values.

来源：https://stackoverflow.com/questions/20813355/skip-first-line-of-csv-while-loading-in-hive-table

标签

Hadoop

Hive

hiveql