How to use file from Databricks FileStore

故事扮演 提交于 2019-12-02 01:55:28

问题


Trying to use a .dat file for ip lookup. File is on Databricks file store from Scala code:

  def getCountryCode(ip: String) {
     val filePath = "FileStore/maxmind/GeoIPCountry.dat"
     val ipLookups = new IpLookups(geoFile = Option(new File(filePath)),
  ispFile = None, orgFile = None, domainFile = None, memCache = false, lruCache = 0)

     val location = ipLookups.performLookups(ip)._1.head
     println(location.countryCode)
  }

I am getting an exception:

java.io.FileNotFoundException: FileStore/maxmind/GeoIPCountry.dat (No such file or directory)

Method works on local environment with relative/absolute paths


回答1:


Use fully qualified path to the file on the FUSE mounted file system:

val filePath = "/dbfs/FileStore/maxmind/GeoIPCountry.dat"

Reference: Using Local File I/O APIs.



来源:https://stackoverflow.com/questions/47815546/how-to-use-file-from-databricks-filestore

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!