Read the data from HDFS using Scala

Question

I am new to Scala. How can I read a file from HDFS using Scala (not using Spark)? When I googled it I only found writing option to HDFS.

import org.apache.ha         


        

Answer 1

One of the ways (kinda in functional style) could be like this:

val hdfs = FileSystem.get(new URI("hdfs://yourUrl:port/"), new Configuration()) 
val path = new Path("/path/to/file/")
val stream = hdfs.open(path)
def readLines = Stream.cons(stream.readLine, Stream.continually( stream.readLine))

//This example checks line for null and prints every existing line consequentally
readLines.takeWhile(_ != null).foreach(line => println(line))

Also you could take a look this article or here and here, these questions look related to yours and contain working (but more Java-like) code examples if you're interested.