Reading HDFS and local files in Java

前端 未结 3 1664
难免孤独
难免孤独 2020-12-08 10:45

I want to read file paths irrespective of whether they are HDFS or local. Currently, I pass the local paths with the prefix file:// and HDFS paths with the prefix hdfs:// an

3条回答
  •  误落风尘
    2020-12-08 11:45

    Does this make sense,

    public static void main(String[] args) throws IOException {
    
        Configuration conf = new Configuration();
        conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));
        conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/hdfs-site.xml"));
    
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        System.out.println("Enter the file path...");
        String filePath = br.readLine();
    
        Path path = new Path(filePath);
        FileSystem fs = path.getFileSystem(conf);
        FSDataInputStream inputStream = fs.open(path);
        System.out.println(inputStream.available());
        fs.close();
    }
    

    You don't have to put that check if you go this way. Get the FileSystem directly from Path and then do whatever you feel like.

提交回复
热议问题