Hadoop “Unable to load native-hadoop library for your platform” warning

Question

I\'m currently configuring hadoop on a server running CentOs. When I run start-dfs.sh or stop-dfs.sh, I get the following error:

Answer 1

In addition to @zhutoulala accepted answer, here is an update to make it work with latest stable version to date (2.8) on ARMHF platforms (Raspberry Pi 3 model B). First I can confirm that you must recompile native libraries to 64 bit ARM, other answers here based on setting some environment variables won't work. As indicated in Hadoop documentation, the pre-built native libraries are 32 bit.

High level steps given in the fist link (http://www.ercoppa.org/posts/how-to-compile-apache-hadoop-on-ubuntu-linux.html) are correct. On this url http://www.instructables.com/id/Native-Hadoop-260-Build-on-Pi/ you get more details specific to Raspberry Pi, but not for Hadoop version 2.8.

Here are my indications pour Hadoop 2.8 :

• there is still no protobuf package on latest Raspbian so you must compile it yourself and version must be exactly protobuf 2.5 (https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz)

• CMake file patching method must be changed. Moreovere, files to patch are not the same. Unfortunately, there is no accepted patch on JIRA specific to 2.8. On this URL (https://issues.apache.org/jira/browse/HADOOP-9320) you must copy and paste Andreas Muttscheller proposed patch on your namenode :

  :hadoop-2.8.0-src/hadoop-common-project/hadoop-common $ touch HADOOP-9320-v2.8.patch
  :hadoop-2.8.0-src/hadoop-common-project/hadoop-common $ vim HADOOP-9320-v2.8.patch
  #copy and paste proposed patch given here : https://issues.apache.org/jira/browse/HADOOP-9320?focusedCommentId=16018862&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16018862
  :hadoop-2.8.0-src/hadoop-common-project/hadoop-common $ patch < HADOOP-9320-v2.8.patch
  patching file HadoopCommon.cmake
  patching file HadoopJNI.cmake
  :hadoop-2.8.0-src/hadoop-common-project/hadoop-common $ cd ../..
  :hadoop-2.8.0-src $ sudo mvn package -Pdist,native -DskipTests -Dtar
  

Once build is successful :

    :hadoop-2.8.0-src/hadoop-dist/target/hadoop-2.8.0/lib/native $ tar -cvf nativelibs.tar *

And replace the content of the lib/native directory of your Hadoop install with the content of this archive. Warning message when running Hadoop should disappear.