Reading HDFS and local files in Java

Question

I want to read file paths irrespective of whether they are HDFS or local. Currently, I pass the local paths with the prefix file:// and HDFS paths with the prefix hdfs:// an

Answer 1

You can get the FileSystem by the following way:

Configuration conf = new Configuration();
Path path = new Path(stringPath);
FileSystem fs = FileSystem.get(path.toUri(), conf);

You do not need to judge if the path starts with hdfs:// or file://. This API will do the work.