Hadoop: …be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation

Question

I\'m getting the following error when attempting to write to HDFS as part of my multi-threaded application

could only be replicated to 0 nodes instead of min         


        

Answer 1

In my case it was a storage policy of output path set to COLD.

How to check settings of your folder:

hdfs storagepolicies -getStoragePolicy -path my_path

In my case it returned

The storage policy of my_path
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE], creationFallbacks=[], replicationFallbacks=[]}   

I dumped the data else where (to HOT storage) and the issue went away.