Oozie workflow: Hive table not found but it does exist

Question

I got a oozie workflow, running on a CDH4 cluster of 4 machines (one master-for-everything, three "dumb" workers). The hive metastore runs on the master using mysql (driver is present), the oozie server also runs on the master using mysql, too. Using the web interface I can import and query hive as expected, but when I do the same queries within an oozie workflow it fails. Even the addition of the "IF EXISTS" leads to the error below. I tried to add the connection information as properties to the hive job without any success.

Can anybody give me a hint? Did I miss anything? Any further information needed?

This is the output of the job's log:

  Script [drop.sql] content:
  ------------------------
  DROP TABLE IF EXISTS performance_log;

  ------------------------

  Hive command arguments :
  -f
  drop.sql

  =================================================================

  >>> Invoking Hive command line now >>>

  Intercepting System.exit(10001)

  <<< Invocation of Main class completed <<<

  Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10001]

  Oozie Launcher failed, finishing Hadoop job gracefully

And this is the error message:

  FAILED: SemanticException [Error 10001]: Table not found performance_log
  Intercepting System.exit(10001)
  Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10001]

Answer 1

The problem is other nodes don't know where your MYSQL is , so you are getting error table not found.

You need to do 2 things

1. Copy hive-site.xml in the oozie workflow directory
2. In your Hive action tell oozie that use my hive-site.xml

Something like below

action name="hive-node"> <hive xmlns="uri:oozie:hive-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <job-xml>hive-site.xml</job-xml>

This should work.

Thanks

Answer 2

I have experienced same problem, apart from above mention solution about specifying hive-site.xml properly. I would recommend following things as well.

1. check if you have mysql connector jar (in case you are using mysql as metastore) is available in classpath.
2. in case of oozie hive actions just check if you are not adding hive jars multiple times. e.g. it is already present in oozie share lib and you have also copied in workflow/lib