Oozie + Sqoop: JDBC Driver Jar Location

Question

I have a 6 node cloudera based hadoop cluster and I\'m trying to connect to an oracle database from a sqoop action in oozie.

I have copied my ojdbc6.jar into the sqo

Answer 1

I was facing the same issue it was not able to find the mysql jar. I am using cloudera 4.4 in this even oozie admin -oozie http://localhost:11000/oozie -sharelibupdate command will not work

To resolve the issue I had followed the below steps:

1. create a user in Hue with hdfs and provide the admin privileges

2. using Hue UI upload the jar into /user/oozie/share/lib/sqoop hdfs path or you can use below command:

hadoop put /var/lib/sqoop2/mysql-connector-java.jar /user/oozie/share/lib/sqoop

3. Once the jar is placed run the oozie command.

Answer 2

The JDBC driver jar (and any jars it depends on) should go in your Oozie sharelib folder on HDFS. I'm running Hortonworks Data Platform 1.2 instead of Cloudera 4.2 so the details may vary, but my JDBC driver is located in /user/oozie/share/lib/sqoop. This should allow you to run Sqoop with the JDBC via Oozie.

It is not necessary to put to the JDBC driver jar in the sqoop lib on the data nodes. In my setupt I can't run a simple sqoop eval from the command line on my data nodes. I understand the logic for why you thought this would work. The reason the JDBC driver jar needs to be on HDFS is so that all the data nodes have access to it. Your solution should accomplish the same goal. I'm not familiar enough with the inner workings of Oozie to say why using the sharelib works but your solution does not.

Answer 3

In CDH5, you should put the jar to '/user/oozie/share/lib/lib_${timestamp}/sqoop', and after that, you must update the sharelib or restart oozie.

update sharelib:

oozie admin -oozie http://localhost:11000/oozie -sharelibupdate

Answer 4

If you are using CDH-5 the JDBC driver jar (and any jars it depends on) should go in '/user/oozie/share/lib/lib_timestamp/sqoop' folder on HDFS.