1. 技术文章
  2. Maven build [WARNING] we have a duplicate class

Maven build [WARNING] we have a duplicate class

由 戏子无情 提交于 2019-11-27 14:57:04

Take a look at the "Dependency Exclusions" section in the Maven doc.

In your provided example, I'll exclude the commons-logging:commons-logging-api:jar:1.0.4:compile dependency from org.apache.hadoop.hive:hive-common:jar:0.7.1-cdh3u3:compile. In your pom.xml :

    <dependency>
        <groupId>org.apache.hadoop.hive</groupId>
        <artifactId>hive-common:jar</artifactId>
        <version>0.7.1-cdh3u3</version>
        <exclusions>
            <exclusion>
                <groupId>commons-logging</groupId>
                <artifactId>commons-logging-api</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
user1454388

You might also have run into a limitation of maven-shader-plugin. It replaces the default jar artifact (created by maven-jar-plugin). This works fine on a clean build, but on a rebuild where the jar is not regenerated, the shader runs again on the jar it created last time, which already contains copies of all the class dependencies. That produces a lot of warnings about duplicates.

This issue is still unaddressed as of maven-shader-plugin 2.0: https://issues.apache.org/jira/browse/MSHADE-126

One workaround is to add the maven-jar-plugin explicitly to your pom.xml and add the configuration setting <forceCreation>true</forceCreation>.

In my case, my parent pom was including commons-beanutils and my child module (which is the only thing I wanted to compile) was including commons-io.

The shade plug in complained about duplicates since commons-io and commons-beansutil shared some common classes. Note that beansutiul was being included even though it was not needed, and was not used.

I solve this by minimizing the jar by adding this to the configuration:

<minimizeJar>true</minimizeJar>

Now the shade plugin did not add unused resources.

Warning went away.

You can exclude the jar you don't want (the ones that are giving the duplicate warnings using the following tags under the shade plugin - 

    <configuration>
    <artifactSet>
      <excludes>
        <exclude>commons-logging:commons-logging</exclude>
      </excludes>
    </artifactSet>
    <minimizeJar>true</minimizeJar>
    </configuration>

More details can be found at http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html

You have dependencies in your pom which contain duplicate classes but without the appropriate pom i couldn't say a word about it.

I saw this happen in eclipse when I updated my parent project's dependencies.

I deleted all the files in my target directory and it fixed the issues.

All above (about reviewing dependencies tree and excluding) is correct in the most of cases, but in my case (i didn't have overlapping in my dependencies) preliminary clean helped (don't know why though):

mvn clean package

In my case I was relying on a package that also creates a shaded jar.

Shaded jars are meant for deployment, not installing as a dependency.

Creating a reduced dependency POM during the build process of the dependency, instructs maven on which dependencies can be left out.

In the maven-shade-plugin configuration:

<configuration>
  <createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>

For more details see this post:

What is the maven-shade-plugin used for, and why would you want to relocate java packages?

The error I was getting from maven:

WARNING: x.jar, y.jar contain overlapping classes

标签
maven
build
maven-shade-plugin
apache-commons-logging
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!