Where is the classpath set for hadoop

问题

Where is the classpath for hadoop set? When I run the below command it gives me the classpath. Where is the classpath set?

  bin/hadoop classpath

I'm using hadoop 2.6.0

回答1:

As said by almas shaikh it's set in hadoop-config.sh, but you could add more jars to it in hadoop-env.sh

Here is a relevant code from hadoop-env.sh which adds additional jars like capacity-scheduler and aws jar's.

export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}

# Extra Java CLASSPATH elements.  Automatically insert capacity-scheduler.
for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do
  if [ "$HADOOP_CLASSPATH" ]; then
    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f
  else
    export HADOOP_CLASSPATH=$f
  fi
done

# ... some other lines omitted

# Add Aws jar
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:share/hadoop/tools/lib/*

回答2:

Open your bash profile (~/.profile or ~/.bash_profile) for editing and add the following:

export HADOOP_HOME="/usr/local/Cellar/hadoop" then Replace with your own path
export HADOOP_CLASSPATH=$(find $HADOOP_HOME -name '*.jar' | xargs echo | tr ' ' ':') Save the changes and reload.
source ~/.profile

回答3:

When you run hadoop command, it sources a file hadoop-config.sh that resides in $HADOOP_HDFS_HOME/libexec which sets your classpath (CLASSPATH) by picking jars residing in various directories viz.

$HADOOP_HDFS_HOME/share/hadoop/mapreduce 
$HADOOP_HDFS_HOME/share/hadoop/common
$HADOOP_HDFS_HOME/share/hadoop/hdfs etc.

回答4:

As per this blog post, it is in an environment variable named HADOOP_CLASSPATH. You can set it as you would any other environment variable, the specifics of which depend on which shell you use. If you use bash, then you can call like export HADOOP_CLASSPATH=/path/to/wherever:/path/to/wherever/else.

来源：https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop

标签

Hadoop

MapReduce

hadoop2