hadoop mapreduce: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z

Question

I am trying to write a snappy block compressed sequence file from a map-reduce job. I am using hadoop 2.0.0-cdh4.5.0, and snappy-java 1.0.4.1

Here is my code:

Answer 1

check your core-site.xml and mapred-site.xml they should contain correct properties and path of the folder with libraries

core-site.xml

  io.compression.codecs
org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec

mapred-site.xml

 
      mapreduce.map.output.compress
      true
    

    
     mapred.map.output.compress.codec  
     org.apache.hadoop.io.compress.SnappyCodec
    


    
      mapreduce.admin.user.env
      LD_LIBRARY_PATH=/usr/hdp/2.2.0.0-1084/hadoop/lib/native
    

LD_LIBRARY_PATH - has to contain path of libsnappy.so .