Spark not working with pureconfig

后端 未结 1 544
猫巷女王i
猫巷女王i 2020-12-21 03:18

I\'m trying to use pureConfig and configFactory for my spark application configuration. here is my code:

import pureconfig.{loadConfigOrThrow}
object Source         


        
相关标签:
1条回答
  • 2020-12-21 04:13

    You also have to shade shapeless inside its own JAR, in addition to pureconfig:

    assemblyShadeRules in assembly := Seq(
      ShadeRule.rename("shapeless.**" -> "shadeshapless.@1")
        .inLibrary("com.chuusai" % "shapeless_2.11" % "2.3.2")
        .inLibrary("com.github.pureconfig" %% "pureconfig" % "0.7.0")
        .inProject
    )
    

    Make sure to add the correct shapeless version.

    0 讨论(0)
提交回复
热议问题