Cloudera Hive: Where to add json-serde-1.3.7 jar file

问题

I am using cloudera 5.8.0

First I run this command:

hive> ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar] to class path
Added resources: [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar]

And than I add the json-serde-1.3.7 jar file

hive> ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with- dependencies.jar;
Added [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar] to class path
Added resources: [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar]

But when I try to create this table in hive:

CREATE EXTERNAL TABLE tweets (    
    id BIGINT,
    created_at STRING,
    source STRING,
    favorited BOOLEAN,
    retweet_count INT,
    retweeted_status STRUCT<
    text:STRING,
    `user`:STRUCT<screen_name:STRING,name:STRING>>,
    entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
    text STRING,
    `user` STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
    in_reply_to_screen_name STRING
)
ROW FORMAT SERDE 'com.cloudera.hive.SERDE.JSONSerDe'
LOCATION '/user/cloudera/flume/tweets'

I get this error:

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: com.cloudera.hive.SERDE.JSONSerDe

回答1:

This solved the problem

ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;
ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar;

CREATE EXTERNAL TABLE tweets (
   id BIGINT,
   created_at STRING,
   source STRING,
   favorited BOOLEAN,
   retweet_count INT,
   retweeted_status STRUCT<
      text:STRING,
      user:STRUCT<screen_name:STRING,name:STRING>>,
   entities STRUCT<
      urls:ARRAY<STRUCT<expanded_url:STRING>>,
      user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
      hashtags:ARRAY<STRUCT<text:STRING>>>,
   text STRING,
   user STRUCT<
      screen_name:STRING,
      name:STRING,
      friends_count:INT,
      followers_count:INT,
      statuses_count:INT,
      verified:BOOLEAN,
      utc_offset:INT,
      time_zone:STRING>,
   in_reply_to_screen_name STRING
) 
    ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
    LOCATION '/user/cloudera/flume/tweets';

来源：https://stackoverflow.com/questions/40854177/cloudera-hive-where-to-add-json-serde-1-3-7-jar-file

标签

Hadoop

Hive

Cloudera