Spark Scala list folders in directory

前端 未结 9 2417
北恋
北恋 2020-12-05 09:41

I want to list all folders within a hdfs directory using Scala/Spark. In Hadoop I can do this by using the command: hadoop fs -ls hdfs://sandbox.hortonworks.com/demo/<

9条回答
  •  無奈伤痛
    2020-12-05 10:20

       val listStatus = org.apache.hadoop.fs.FileSystem.get(new URI(url), sc.hadoopConfiguration)
    .globStatus(new org.apache.hadoop.fs.Path(url))
    
      for (urlStatus <- listStatus) {
        println("urlStatus get Path:" + urlStatus.getPath())
    

    }

提交回复
热议问题