Hadoop HADOOP_CLASSPATH issues

Question

This question doesn\'t refer to distributing jars in the whole cluster for the workers to use them.

It refers to specifying a number of additional libraries on the

Answer 1

Try to add your jar file in default CLASSPATH variable and also append HADOOP_CLASSPATH to it. Then execute your command.

export CLASSPATH=/your/jar/file/myjar.jar:$CLASSPATH:$HADOOP_CLASSPATH /path/to/hadoop/script fs -text /path/in/HDFS/to/my/file

Answer 2

If someone wants to check hadoop classpath, enter hadoop classpath in terminal.
To compile it, use this: javac -cp $(hadoop classpath):path/to/jars/* java_file.java

Answer 3

If you are allowed to set HADOOP_CLASSPATH then

export HADOOP_CLASSPATH=/path/to/jar/myjar.jar:$HADOOP_CLASSPATH; \
    hadoop fs -text /path/in/HDFS/to/my/file

will do the job. Since in your case this variable is overridden in hadoop-env.sh therefore, consider using the -libjars option instead:

hadoop fs -libjars /path/to/jar/myjar.jar -text /path/in/HDFS/to/my/file

Alternatively invoke FsShell manually:

java -cp $HADOOP_HOME/lib/*:/path/to/jar/myjar.jar:$CLASSPATH \
org.apache.hadoop.fs.FsShell -conf $HADOOP_HOME/conf/core-site.xml \
-text /path/in/HDFS/to/my/file