How to list all files in a directory and its subdirectories in hadoop hdfs

后端 未结 9 1073
故里飘歌
故里飘歌 2020-12-01 05:50

I have a folder in hdfs which has two subfolders each one has about 30 subfolders which,finally,each one contains xml files. I want to list all xml files giving only the mai

9条回答
  •  不知归路
    2020-12-01 06:06

    don't use recursive approach (heap issues) :) use a queue

    queue.add(param_dir)
    while (queue is not empty){
    
      directory=  queue.pop
     - get items from current directory
     - if item is file add to a list (final list)
     - if item is directory => queue.push
    }
    

    that was easy, enjoy!

提交回复
热议问题