java.lang.NoClassDefFoundError: org/slf4j/Logger class not found with IntelliJ and SBT

这一生的挚爱 提交于 2020-08-08 05:15:34

问题


I am trying to run a spark scala program where in I have used "import org.apache.spark.internal.Logging". Program was running fine in local until I try to build a fat jar and added assembly.sbt using build.sbt

Here's my build.sbt

lazy val root = (project in file(".")).
  settings(
    name := "TestingSparkApplicationProject",
    version := "0.0.1-SNAPSHOT",
    scalaVersion := "2.11.12",
    mainClass in Compile := Some("com.test.spark.job.TestSparkJob.scala")
  )
val sparkVersion = "2.4.3"
libraryDependencies ++= Seq(
  "net.jcazevedo" %% "moultingyaml" % "0.4.2",
  "net.liftweb" %% "lift-json" % "3.4.1",
  "com.github.scopt" %% "scopt" % "4.0.0-RC2",
  "com.google.cloud" % "google-cloud-storage" % "1.23.0",
  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided"
)

scalaSource in Compile := baseDirectory.value / "src"

My assembly.sbt file as:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.10")

Though it has not build successfully and still showing a error message in the right corner of assembly.sbt file

"Expression type def.settings[Seq[ModuleID]] must conform to Dsl entry in sbt file"

I am getting below error when running my main module 'TestSparkJob'

java.lang.NoClassDefFoundError: org/slf4j/Logger
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
    at java.lang.Class.getMethod0(Class.java:3018)
    at java.lang.Class.getMethod(Class.java:1784)
    at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 7 more
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" 

Update- I got rid of assembly.sbt issue when I recreated my build.sbt with the same code. Code is running fine when deployed to dataproc cluster but I am still getting same error when trying to run my code in local environment.

来源:https://stackoverflow.com/questions/63090249/java-lang-noclassdeffounderror-org-slf4j-logger-class-not-found-with-intellij-a

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!