sbt-assembly

问题 I need to exclude spark and test dependencies from my final assembly jar. I tried to use provider but it was not working. libraryDependencies ++= Seq("org.apache.spark" % "spark-core_2.11" % "2.0.1" % "provided") and execute sbt assembly . Please help me resolve this issue. 回答1: Use exclude option of assembly plugin filtering by direct name or with contains assemblyExcludedJars in assembly := { val cp = (fullClasspath in assembly).value cp filter { f => f.data.getName.contains("spark-core") |

问题 I am building a sbt plugin and want to reference assembly task in the sbt-assembly plugin ( to be dependent on my task) to do this i need to reference it as a library ( as opposed to a plugin), and somehow sbt is not able to resolve it as a libraryDepdendencies this is what my sbt looks like sbtPlugin := true name := "my-sbt-plugin" scalaVersion := "2.10.6" sbtVersion := "0.13.0" resolvers ++= Seq(Resolver.sbtPluginRepo("releases"), Resolver.sbtPluginRepo("snapshots")) libraryDependencies ++=

问题 I am building a sbt plugin and want to reference assembly task in the sbt-assembly plugin ( to be dependent on my task) to do this i need to reference it as a library ( as opposed to a plugin), and somehow sbt is not able to resolve it as a libraryDepdendencies this is what my sbt looks like sbtPlugin := true name := "my-sbt-plugin" scalaVersion := "2.10.6" sbtVersion := "0.13.0" resolvers ++= Seq(Resolver.sbtPluginRepo("releases"), Resolver.sbtPluginRepo("snapshots")) libraryDependencies ++=

问题 Our project mainly consists of two parts Build.scala where the root project lies BuildShaded.scala where some external dependencies are shaded with sbt-assembly. The shaded jars will be depended upon by sub projects under the root project through unmanagedJars setting. The question is how to assembly the shaded project before compiling the root project. Otherwise, the root project will fail to compile since those classes in the shaded jars are not available. 回答1: As I said in the comments, I

问题 What is meant under the term classifiers ? Is it comes from Jars? For example in sbt-assembly plugin: artifact in (Compile, assembly) ~= { art => art.copy(`classifier` = Some("assembly")) } 回答1: classifier is defined by Maven as the fifth element of a project coordinate, after groupId , artifactId , version and packaging . More specifically (from the maven documentation, emphasis mine): The classifier allows to distinguish artifacts that were built from the same POM but differ in their

问题 I have a single scala project built with sbt using a build.scala file. I can use sbt-assembly to generate a single executable jar file from this project without problem. Now I want to generate multiple executable jar files, where each file includes a common set of internal and external base dependencies along with different external dependencies . Is this possible with the current version of sbt-assembly? In maven this is easy, as one can define multiple profiles in the pom, each generating a

问题 I have had this problem since the day I started using assembly . sbt assembly (for any project) never completes on my Mac. It just hangs at the last step . For this one reason . I transfer my code to a Linux box and build there . Anyone else has experienced this ? . Any ideas on how I can go about debugging ? . I had Lion with Java 1.6.0_37 . Now upgraded to Mountain Lion 回答1: I had the same problem some time ago. Do you have anti-virus software installed? In my case it was Sophos, that tried

问题 When using -> in Spark Streaming 2.0.0 jobs, or using spark-streaming-kafka-0-8_2.11 v2.0.0, and submitting it with spark-submit , I get the following error: Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 72.0 failed 1 times, most recent failure: Lost task 0.0 in stage 72.0 (TID 37, localhost): java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object; I put a brief illustration of this phenomenon

问题 I have code which essentially looks like this: class FoodTrainer(images: S3Path) { // data is >100GB file living in S3 def train(): FoodClassifier // Very expensive - takes ~5 hours! } class FoodClassifier { // Light-weight API class def isHotDog(input: Image): Boolean } I want to at JAR-assembly ( sbt assembly ) time, invoke val classifier = new FoodTrainer(s3Dir).train() and publish the JAR which has the classifier instance instantly available to downstream library users. What is the

问题 I'm trying to create a fat jar of a combination of spark and restlet framework, but I keep getting this deduplication error. This is the error for running sbt assembly : java.lang.RuntimeException: deduplicate: different file contents found in the following: /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016