sbt assembly fails due to different file contents found

问题

I am trying to build a project in GitLab. In gitlab-ci.yml, I ran sbt assembly and encountered annoying exception.

[error] (soda/*:assembly) deduplicate: different file contents found in the following:
[error] /root/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-codec-http/jars/netty-codec-http-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-transport-native-epoll/jars/netty-transport-native-epoll-4.0.42.Final-linux-x86_64.jar:META-INF/io.netty.versions.properties

I tried to follow the instruction in sbt-assembly: deduplication found error, and it seems like the MergeStrategy is in place, however the exception remain:

[info] Merging files...
[warn] Merging 'NOTICE' with strategy 'rename'
[warn] Merging 'README' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE.txt' with strategy 'rename'
[warn] Merging 'license/NOTICE' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE' with strategy 'rename'
[warn] Merging 'org/xerial/snappy/native/README' with strategy 'rename'
[warn] Merging 'license' with strategy 'rename'
[warn] Merging 'license/LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/license' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE.txt' with strategy 'rename'
[warn] Merging 'LICENSE.txt' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE' with strategy 'rename'
[warn] Merging 'LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/DEPENDENCIES' with strategy 'discard'
[warn] Merging 'META-INF/INDEX.LIST' with strategy 'discard'
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'

I also tried to change the sbt version, but the problem stays.

Any help with how can I resolve this issue will be great.

回答1:

Looked everywhere could not find any resolution. Check the following resolution if that works for you, It did not work for me: https://github.com/sbt/sbt-assembly/issues/297

回答2:

I once had a similarproblem with some Java packages and log4j, What I did was to create a custom merge strategy in which I only choosed the first occurrence. Maybe you can adapt that code to work for you:

assemblyMergeStrategy in assembly := {
//  case PathList("javax", "servlet", xs @ _*)         => MergeStrategy.first
  case PathList("org", "apache", "commons", xs @ _*)        =>
 //   println(s"$xs")
    MergeStrategy.first
  case PathList(ps @ _*) if ps.last endsWith ".html" => MergeStrategy.first
  case "application.conf"                            => MergeStrategy.concat
  case "log4j.properties"                            => MergeStrategy.first
  case "unwanted.txt"                                => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

Maybe if you change

 case PathList("org", "apache", "commons", xs @ _*)        =>
 //   println(s"$xs")
    MergeStrategy.first

to your problematic packages (netty) you manage to solve it:

 case PathList("io", "netty", xs @ _*)        =>
 //   println(s"$xs")
    MergeStrategy.first

回答3:

Eventually I found a workaround for this problem. Since its a play project, I could use activator to wrap it up and create a package. I used the following code in order to create the package: wget https://downloads.typesafe.com/typesafe-activator/1.3.7/typesafe-activator-1.3.7.zip && unzip typesafe-activator-1.3.7.zip 1> /dev/null activator-dist-1.3.7/activator -Dsbt.log.noformat=true -java-home /usr/lib/jvm/java-1.8-openjdk package clean compile test dist

来源：https://stackoverflow.com/questions/55806027/sbt-assembly-fails-due-to-different-file-contents-found

标签

scala

sbt

gitlab

sbt-assembly