IRichBolt Error when running topology on storm-1.0.0 and pyleus-0.3.0

Question

I'm running storm topology "pyleus --verbose local xyz_topology.jar" using storm-1.0.0, pyleus-0.3.0, centos-6.6 and getting the Error

Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt

Running: java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-1.0.0
-Dstorm.log.dir=/usr/local/apache-storm-1.0.0/logs -Djava.library.path=/usr/local/    
lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-1.0.0/lib/asm-5.0.3.jar:
/usr/local/apache-storm-1.0.0/lib/clojure-1.7.0.jar:/usr/local/apache-storm-1.0.0/lib/disruptor-3.3.2.jar:
/usr/local/apache-storm-1.0.0/lib/kryo-3.0.3.jar:
/usr/local/apache-storm-1.0.0/lib/log4j-api-2.1.jar:/usr/local/apache-storm-1.0.0/lib/log4j-core-2.1.jar    
:/usr/local/apache-storm-1.0.0/lib/log4j-over-slf4j-1.6.6.jar:
/usr/local/apache-storm-1.0.0/lib/log4j-slf4j-impl-2.1.jar:/usr/local/apache-storm-1.0.0/lib/minlog-1.3.0.jar:
/usr/local/apache-storm-1.0.0/lib/objenesis-2.1.jar:/usr/local/apache-storm-1.0.0/lib/reflectasm-1.10.1.jar:
/usr/local/apache-storm-1.0.0/lib/servlet-api-2.5.jar:/usr/local/apache-storm-1.0.0/lib/slf4j-api-1.7.7.jar:
/usr/local/apache-storm-1.0.0/lib/storm-core-1.0.0.jar:
/usr/local/apache-storm-1.0.0/lib/storm-rename-hack-1.0.0.jar:xyzTopology.jar:
/usr/local/storm/conf:/usr/local/apache-storm-1.0.0/bin
-Dstorm.jar=xyz_topology.jar com.yelp.pyleus.PyleusTopologyBuilder --local
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: backtype.storm.topology.IRichBolt
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 7 more
pyleus local: error: [StormError] Storm command failed. Run with --verbose for more info.

The same topology worked well and without any errors when using the old version of storm-0.10.0 and pyleus-0.3.0.

I looked at the storm-0.10.0 and storm-1.0.0 docs and found that in storm-1.0.0

(a) the package structure has been changed and

(b) the IRichBolt class is not presented and instead of that there are two new classes - BaseStatefulBoltExecutor and BasicBoltExecutor - which implement IRichBolt interface.

It seems new storm-1.0.0 release can't work with pyleus-0.3.0 as it was with old storm-0.10.0 version.

What would be the best solution to resolve the issue and allow to run the topology on storm-1.0.0 and pyleus-0.3.0 ?

Answer 1

Storm 1.0.0 includes a major refactoring with regard to package names. All packages backtype/* got replaced by org/apache/storm. Thus, you need to recompile your topology using Storm 1.0.0 (including some code change; ie, using different imports to align to the new package structure).

As an alternative, you can enable backward compatibility in your Storm cluster (see https://storm.apache.org/releases/1.0.0/index.html)

NOTE

In the latest version, the class packages have been changed from "backtype.storm" to "org.apache.storm" so the topology code compiled with older version won't run on the Storm 1.0.0 just like that. Backward compatibility is available through following configuration

client.jartransformer.class: "org.apache.storm.hack.StormShadeTransformer"

You need to add the above config in storm installation if you want to run the code compiled with older versions of storm. The config should be added in the machine you use to submit your topologies.

Refer to https://issues.apache.org/jira/browse/STORM-1202 for more details.