how to run hive in debug mode

问题

i took example from cloudera website to write a custom SerDe for parsing a file

http://blog.cloudera.com/blog/2012/12/how-to-use-a-serde-in-apache-hive/

it seems a good example but when i create table with custom serde

ADD JAR <path-to-hive-serdes-jar>;

CREATE EXTERNAL TABLE tweets (
  id BIGINT,
  created_at STRING,
  source STRING,
  favorited BOOLEAN,
  retweeted_status STRUCT<
    text:STRING,
    user:STRUCT<screen_name:STRING,name:STRING>,
    retweet_count:INT>,
  entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
  text STRING,
  user STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
  in_reply_to_screen_name STRING
) 
PARTITIONED BY (datehour INT)
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
LOCATION '/user/flume/tweets';

it executed perfectly fine but when i do

select * from tweets;

i am getting nothing so thats why i wanted to know if i can run hive in debug mode to see where it is getting failed

回答1:

You better start hive shell by switching logger mode to DEBUG as follows, I hope you could find something useful from there.

hive --hiveconf hive.root.logger=DEBUG,console

回答2:

Hive code can be debugged.This link may help you : https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-DebuggingHiveCode

回答3:

Setting hive --hiveconf hive.root.logger=DEBUG,console may not always work because of company specific setup.

I ended up creating a hive-log4j.properties file in my home directory with following settings:

log4j.rootCategory=DEBUG,console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

and started hive shell using CLASSPATH=$HOME hive which adds your home directory having hive-log4j.properties in front of the classpath and so is picked up.

来源：https://stackoverflow.com/questions/27854359/how-to-run-hive-in-debug-mode

标签

java

Hadoop

Hive