How to write and read files in/from Hadoop HDFS using Ruby?

问题

Is there a way to work with HDFS Api using Ruby? As I can understand there is no multilanguage file Api and the only way is to use native Java Api. I tried using JRuby but this solution is to unstable and not very native. Also I looked at HDFS Thrift Api but it's not complete and also lacks many features (like writing to indexed files).

Is there a way to work with HDFS using Ruby besides from using JRuby or Thrift Api?

回答1:

There are two projects in github that fit what you're asking. ruby-hdfs provides native C bindings to HDFS for Ruby. ganapati interfaces with a Thrift server.

You could also make system calls directly to the file system shell. For example:

cmd = "hadoop fs -mkdir #{hdfs_path}"
cmd += " 2> /dev/null"
system(cmd)
if $? == 0
  puts 'ok'
  exit(0)
else
  puts "Error: failed to create hdfs://#{hdfs_path}"
  exit(2)
end

来源：https://stackoverflow.com/questions/3125519/how-to-write-and-read-files-in-from-hadoop-hdfs-using-ruby