Spark : Read file only if the path exists

前端 未结 2 1638
無奈伤痛
無奈伤痛 2020-12-16 12:45

I am trying to read the files present at Sequence of Paths in scala. Below is the sample (pseudo) code:

val paths = Seq[String] //Seq of paths
v         


        
2条回答
  •  温柔的废话
    2020-12-16 13:01

    How about filtering the paths firstly`:

    paths.filter(f => new java.io.File(f).exists)
    

    For instance:

    Seq("/tmp", "xx").filter(f => new java.io.File(f).exists)
    // res18: List[String] = List(/tmp)
    

提交回复
热议问题