Data Replication error in Hadoop

Question

I am implementing the Hadoop Single Node Cluster on my machine by following Michael Noll\'s tutorial and have come across data replication error:

Here\'s the full error

Answer 1

I had the same problem, I took a look at the datanode logs and there was a warning saying that the dfs.data.dir had incorrect permissions... so I just changed them and everything worked, which is kind of weird.

Specifically, my "dfs.data.dir" was set to "/home/hadoop/hd_tmp", and the error I got was:

...
...
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory in dfs.data.dir: Incorrect permission for /home/hadoop/hd_tmp/dfs/data, expected: rwxr-xr-x, while actual: rwxrwxr-x
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in dfs.data.dir are invalid.
...
...

So I simply executed these commands:

• I stopped all the demons with "bin/stop-all.sh"
• Change the permissions of the directory with "chmod -R 755 /home/hadoop/hd_tmp"
• I gave format again to the namenode with "bin/hadoop namenode -format".
• I re-started the demons "bin/start-all.sh"
• And voilà, the datanode was up and running! (I checked it with the command "jsp", where a process named DataNode was shown).

And then everything worked fine.