Hadoop on Windows Building/ Installation Error

Question

I am trying to install Apache Hadoop 2.7.1 on Windows x64 (8.1 and Server 2012 R2), but I am failing at goal:

    [INFO] Apache Hadoop Common ..............         


        

Answer 1

I successfully built Hadoop 3.0.2 with Visual Studio 2017 on Windows 10.

Here is my settings look like:

1. Windows 10
2. Visual Studio 2017 Community Version
3. Maven 3.5.0
4. JDK 1.8.0_144
5. Cmake 3.10.0-rc1
6. ProtocolBuffer 2.5.0
7. Cygwin

First I open these two solution files in VS 2017 and save.

hadoop-3.0.2-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln

and

hadoop-3.0.2-src\hadoop-common-project\hadoop-common\src\main\native\native.sln

This will migrate those two projects from VS 2010 to VS 2017.

Second I need to change line 141 of the pom file located at

D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\pom.xml

to

This will ensure VS 2017 will be used during the building process.

I have also encountered a problem where MSBuild complains the command is long then stops working, I think this is related to the Maven default repository location. To fix this, I added following line

D:/maven_repo

to file

apache-maven-3.5.0\conf\settings.xml

to move the Maven repository to a new location (in this case D:/maven_repo).

Finally I opened the x64 Native Tools Command Prompt for VS 2017, cd to the hadoop source folder and issued following command:

mvn clean package -Pdist,native-win -DskipTests -Dtar -Dmaven.repo.local=D:\maven_repo

I spend more than 10 hours to fix all problems, hopefully my solution will help others as well.