Reading property file from external path in spark scala throwing error

问题

I am trying to read property file from external path in spark scala like this :

spark-submit --class com.spark.scala.my.class --deploy-mode cluster --master yarn --files /user/mine/dev.properties /path/to/jar/dev-0.0.1-SNAPSHOT-uber.jar 2020-08-19T06:00:00Z 2020-08-20T07:00:00Z

and I am reading like this:

    val props = new Properties()
    val filePath = SparkFiles.get("/user/mine/dev.properties")
    LOGGER.info("Path to file  : "+ filePath)
    val is= Source.fromFile(filePath)
    props.load(is.bufferedReader())

I also tried like this:

val is= Source.fromFile("/user/mine/dev.properties")

But getting file not found exception in both:

diagnostics: User class threw exception: java.io.FileNotFoundException: /hadoopfs/fs1/data/hdfs1/hadoop/yarn/local/usercache/user/appcache/application_1598445422718_5591/spark-78c8ae71-3381-4317-90e1-d5a6da6b2956/userFil9a-b3c7-4b24-8227-441837/user/mine/dev.properties (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at scala.io.Source$.fromFile(Source.scala:91)
        at scala.io.Source$.fromFile(Source.scala:76)
        at scala.io.Source$.fromFile(Source.scala:54)
 

and when i dont use spark.file I get this :

20/11/17 07:57:45 INFO Client:
         client token: Token { kind: YARN_CLIENT_TOKEN, service:  }
         diagnostics: User class threw exception: java.io.FileNotFoundException: /user/mine/dev.properties (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at scala.io.Source$.fromFile(Source.scala:91)
        at scala.io.Source$.fromFile(Source.scala:76)
        at scala.io.Source$.fromFile(Source.scala:54)
 

来源：https://stackoverflow.com/questions/64871515/reading-property-file-from-external-path-in-spark-scala-throwing-error