问题
I was recently able to build Apache Hadoop 2.5.1 with native 64 bit support. So, I got rid of the annoying Native Libraries Warning.
I'm trying to configure Apache Spark. When I start spark-shell, the same warning appears:
14/09/14 18:48:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Some tips:
I had to download a pre-built 2.4 version of Spark because there is still no profile for Hadoop 2.5 with Maven.
The following exports were added to spark-env.sh:
export HADOOP_CONF_DIR=/opt/hadoop-2.5.1/etc/hadoop
export SPARK_LIBRARY_PATH=/opt/hadoop-2.5.1/lib/native
Didn't work with spark-shell and spark-submit. My Hadoop local installation is configured as pseudo-distributed (ResourceManager + YARN support).
回答1:
You should add HADOOP_HOME/lib/native to the LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native
来源：https://stackoverflow.com/questions/25835715/apache-spark-native-libraries