1. 技术文章
  2. ClassNotFoundException for Zookeeper while building Storm

ClassNotFoundException for Zookeeper while building Storm

由 不羁的心 提交于 2019-12-02 04:41:30

问题


I'm new to java and Storm so please forgive any obvious mistakes. I'm trying to run storm with a flume connector but It crashes with the following error:

java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ExceptionInInitializerError
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.cluster$loading__4784__auto__.invoke(cluster.clj:1)
    at backtype.storm.cluster__init.load(Unknown Source)
    at backtype.storm.cluster__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.daemon.nimbus__init.load(Unknown Source)
    at backtype.storm.daemon.nimbus__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.testing$loading__4784__auto__.invoke(testing.clj:1)
    at backtype.storm.testing__init.load(Unknown Source)
    at backtype.storm.testing__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:605)
    at clojure.core$use.doInvoke(core.clj:5392)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.LocalCluster$loading__4784__auto__.invoke(LocalCluster.clj:1)
    at backtype.storm.LocalCluster__init.load(Unknown Source)
    at backtype.storm.LocalCluster__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.lang.Var.invoke(Var.java:415)
    at backtype.storm.LocalCluster.<clinit>(Unknown Source)
    at com.storm.WordCountTopology.main(WordCountTopology.java:97)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.server.NIOServerCnxn$Factory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)
    at backtype.storm.zookeeper$loading__4784__auto__.invoke(zookeeper.clj:1)
    at backtype.storm.zookeeper__init.load(Unknown Source)
    at backtype.storm.zookeeper__init.<clinit>(Unknown Source)
    ... 96 more

My pom file

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.storm</groupId>
    <artifactId>my-storm</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>
    <name>my-storm</name>
    <url>http://maven.apache.org</url>
    <properties>
        <flume.version>1.3.0-cdh4.2.1</flume.version>
        <hadoop.version>2.0.0-cdh4.2.1</hadoop.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <repositories>
        <repository>
            <id>clojars.org</id>
            <url>http://clojars.org/repo</url>
        </repository>
        <repository>
            <id>cdh-maven-repo</id>
            <name>CDH Maven Repository</name>
            <url>https://repository.cloudera.com/content/repositories/releases/</url>
        </repository>
    </repositories> 
    <dependencies>
        <dependency>
            <groupId>storm</groupId>
            <artifactId>storm</artifactId>
            <version>0.8.2</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-embedded-agent</artifactId>
            <version>${flume.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-core</artifactId>
            <version>${flume.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                        <artifactId>slf4j-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-configuration</artifactId>
            <version>${flume.version}</version>
        </dependency>
            <dependency>
                <groupId>org.apache.flume</groupId>
                <artifactId>flume-ng-node</artifactId>
                <version>${flume.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-common</artifactId>
                <version>${hadoop.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
                <version>3.4.5</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>1.5.8</version>
            </dependency>
        </dependencies>
       <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-release-plugin</artifactId>
                <version>2.0-beta-9</version>
            </plugin>
        </plugins>
    </build>
</project>



 $uname -a :Linux 2.6.32-358.23.2.el6.x86_64 #1 SMP Thu Oct 17 10:27:22 CEST 2013 x86_64 x86_64 x86_64 GNU/Linux
    $locate zookeeper
       .m2/repository/org/apache/zookeeper/zookeeper/3.4.5-cdh4.2.1/zookeeper-3.4.5-cdh4.2.1.pom

Can you please explain what am I missing?


回答1:


It seems that the key is here:

Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.server.NIOServerCnxn$Factory

This seems caused by using the version 3.4.5 of zookeeper. Have you tried a downgrade to, maybe version 3.3.6?

The branch-3.3 contains that Inner Class NIOServerCnxn$Factory but the branch-3.4 has the NIOServerCnxnFactory instead




回答2:


It seems that final jar is missing zookeeper. Try adding this:

   <dependency>
        <groupId>org.apache.flume</groupId>
        <artifactId>flume-ng-core</artifactId>
        <version>${flume.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
            </exclusion>
        </exclusions>
    </dependency>


<dependency>
  <groupId>org.apache.zookeeper</groupId>
  <artifactId>zookeeper</artifactId>
  <version>3.4.5</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-log4j12</artifactId>
  <version>1.5.8</version>
</dependency>

to your pom.xml.

If it doesn't help issue :

mvn dependency:tree

and send me output.




回答3:


The below POM works for me - 

<dependencies>
    <dependency>
        <groupId>org.apache.zookeeper</groupId>
        <artifactId>zookeeper</artifactId>
        <version>3.4.5</version>
        <exclusions>
            <exclusion>
                <groupId>com.sun.jmx</groupId>
                <artifactId>jmxri</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.sun.jdmk</groupId>
                <artifactId>jmxtools</artifactId>
            </exclusion>
            <exclusion>
                <groupId>javax.jms</groupId>
                <artifactId>jms</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>0.9.3</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.10</artifactId>
        <version>0.8.2.0</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-core</artifactId>
        <version>0.9.3</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>com.espertech</groupId>
        <artifactId>esper</artifactId>
        <version>4.11.0</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>net.sf.opencsv</groupId>
        <artifactId>opencsv</artifactId>
        <version>2.3</version>
    </dependency>
</dependencies>



回答4:


I have also faced the same issue, the problem i noticed in my case is: I am using storm-9.0.1 with Zookeeper 3.4.5.

By change my zookeeper to 3.3.6 or by changing the storm to 9.0.2,My problem have solved.

So Storm-9.0.1 is compatibility with Zookeeper 3.3.6 and Storm 9.0.2 is compatibility with Zookeeper 3.4.5




回答5:


After some days of banging my head I decided to change the aproach, aparently Eclipse could run the project as it was without any issue.

Creating an uberjar from eclipse and puting all the referenced jars in the project's lib folder and then running the uberjar in the server using java -jar solved the issue.

Not a good solution and I never understood why maven couldn't run the project but it works.



来源：https://stackoverflow.com/questions/19958635/classnotfoundexception-for-zookeeper-while-building-storm

标签
java
apache-zookeeper
apache-storm
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!