trying to run MR program version(2.7) in windows 7 64 bit in eclipse while running the above exception occurring . I verified that using 64 bit 1.8 java version and observ
I already had %HADOOP_HOME%/bin
in my PATH
and my code had previously run without errors. Restarting my machine made it work again.
After putting haddop.dll
and winutils
in hadoop/bin
folder and adding the folder of hadoop to PATH
, we also need to put hadoop.dll
into the C:\Windows\System32
folder
Yes this issues arose when I was using the PIGUNITS
for automation of PIGSCRIPTS
. Two things in sequence need to be done:
Copy both the files as mentioned about in a location and add it to the environment variables under PATH.
To reflect the change what you have just done, you have to restart your machine to load the file.
Under JUNIT I was getting this error which would help others as well:
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias XXXXX. Backend error : java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING at org.apache.pig.PigServer.openIterator(PigServer.java:925)
This issue occurred to me and the cause was I forgot to append %HADOOP_HOME%/bin
to PATH
in my environment variables.
In addition to other solutions, Please download winutil.exe and hadoop.dll and add to $HADOOP_HOME/bin. It works for me.
https://github.com/steveloughran/winutils/tree/master/hadoop-2.7.1/bin
Note: I'm using hadoop-2.7.3 version
The version mismacth is main cause for this issue. Follow complete hadoop version with java library will solve the issue and if you still face issue and working on hadoop 3.1.x version use this library to download bin
https://github.com/s911415/apache-hadoop-3.1.0-winutils/tree/master/bin