问题
I am new in word of hadoop and sqoop. I installed hadoop 2.7.3 (pseudo mode) and its working fine on my system.
I want integration with sqoop. I am using sqoop sqoop-1.99.7-bin-hadoop200.
1) I extract tar file and move extracted content into /usr/local/sqoop
2) Set Sqoop path into .bashrc file.
3) go to /usr/local/sqoop/server/lib/sqoop.sh server start
and get following error message.
hadoop_usr@sawai-Lenovo-G580:/usr/local/sqoop/server/lib$ sqoop.sh server start
Setting conf dir: /usr/local/sqoop/conf
Sqoop home directory: /usr/local/sqoop
Starting the Sqoop2 server...
0 [main] INFO org.apache.sqoop.core.SqoopServer - Initializing Sqoop server.
34 [main] INFO org.apache.sqoop.core.PropertiesConfigurationProvider - Starting config file poller thread
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)
at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:121)
at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:237)
at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:39)
at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98)
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
at org.apache.sqoop.server.SqoopJettyServer.<init>(SqoopJettyServer.java:67)
at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:177)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 more
4) So when i am looking for jars in lib then i found. following jar list.
Please can you tell me which jar is missing now. So am able to run sqoop and start my learning.
Thanks.
回答1:
I got the same issue
to solve this,I have set below variables in seperate terminal and it works
export HADOOP_MAPRED_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/mapreduce
export HADOOP_COMMON_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/common
export HADOOP_HDFS_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/hdfs
export YARN_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/yarn
回答2:
Copy the jar files from below to $SQOOP_HOME/server/lib :
cp -R $HADOOP_HOME/share/hadoop/common/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/common/lib/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/hdfs/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/hdfs/lib/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/mapreduce/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/mapreduce/lib/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/yarn/* $SQOOP_HOME/server/lib/cp -R $HADOOP_HOME/share/hadoop/yarn/lib/* $SQOOP_HOME/server/lib/
And remove duplicate jars with different versions.
Did face the same issue. I have configured the same versions as yours. These steps are working for me, right now.
Good Luck ! We are on the same page !
回答3:
This got solved by setting the following two environment variables in my .bashrc file.
export HADOOP_COMMON_HOME=/usr/local/hadoop/share/hadoop/common
export HADOOP_MAPRED_HOME=/usr/local/hadoop/share/hadoop/mapreduce
basically ensuring that the common libraries mentioned in the above answers are available through these environment variables for sqoop.sh
回答4:
Please make sure if hadoop-common-0.xx.jar in classpath to resolve classnotfound Exception for configuration class.
hadoop-common-0.xx.jar will be available in /lib.
来源：https://stackoverflow.com/questions/41405072/sqoop-integration-with-hadoop-throw-classnotfoundexception