Getting error while running spark programs in Apache Zeppelin in Windows 10 or 7

Question

I am new to Apache Zeppelin. Installed 0.8.0 and using 7000 port to access Zeppelin. Configured few paths as mentioned below.

JAVA_HOME: C:\Program Files\Java\jdk1.8.0_144 HADOOP_HOME: C:\winutils ZEPPELIN_HOME: C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all

all these variables are included in path variable and respective bin folders

Tried to run spark program and getting the below error. Tried with multiple options to fix it, but unable. Please help.

Spark Program: %spark println(sc.appName)

Error: DEBUG [2018-07-29 00:06:05,371] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:153) - callbackServer is serving now INFO [2018-07-29 00:06:05,380] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:190) - Run interpreter process [C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all\bin\interpreter.cmd, -d, C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all/interpreter/spark, -c, 10.120.44.23, -p, 57136, -r, :, -l, C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all/local-repo/spark, -g, spark] DEBUG [2018-07-29 00:06:09,625] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - Warning: Local jar C:\Users\vvellabo\57136 does not exist, skipping. DEBUG [2018-07-29 00:06:09,626] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - Warning: Local jar C:\Users\vvellabo\10.120.44.23 does not exist, skipping. DEBUG [2018-07-29 00:06:09,627] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - java.lang.ClassNotFoundException: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer DEBUG [2018-07-29 00:06:09,628] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.net.URLClassLoader.findClass(URLClassLoader.java:381) DEBUG [2018-07-29 00:06:09,630] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.ClassLoader.loadClass(ClassLoader.java:424) DEBUG [2018-07-29 00:06:09,632] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.ClassLoader.loadClass(ClassLoader.java:357) DEBUG [2018-07-29 00:06:09,633] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.Class.forName0(Native Method) DEBUG [2018-07-29 00:06:09,634] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.Class.forName(Class.java:348) DEBUG [2018-07-29 00:06:09,635] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.util.Utils$.classForName(Utils.scala:238) DEBUG [2018-07-29 00:06:09,641] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851) DEBUG [2018-07-29 00:06:09,642] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198) DEBUG [2018-07-29 00:06:09,644] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228) DEBUG [2018-07-29 00:06:09,645] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137) DEBUG [2018-07-29 00:06:09,647] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) DEBUG [2018-07-29 00:06:09,656] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - 2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Shutdown hook called DEBUG [2018-07-29 00:06:09,659] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - 2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Deleting directory C:\Users\vvellabo\AppData\Local\Temp\spark-427c3202-c243-4761-86ce-ea51a27a881c INFO [2018-07-29 00:06:09,747] ({Exec Default Executor} RemoteInterpreterManagedProcess.java[onProcessFailed]:250) - Interpreter process failed {} org.apache.commons.exec.ExecuteException: Process exited with an error: 101 (Exit value: 101) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) at java.lang.Thread.run(Thread.java:748) ERROR [2018-07-29 00:07:05,382] ({pool-2-thread-2} Job.java[run]:190) - Job failed java.lang.RuntimeException: Warning: Local jar C:\Users\vvellabo\57136 does not exist, skipping. Warning: Local jar C:\Users\vvellabo\10.120.44.23 does not exist, skipping.

Thanks vvell

Answer 1

I faced a similar issue. Kept coming back to this question in case someone solves it.

This link will help you!

If you do not wish to go through the steps, simply delete the SPARK_HOME environment variable. Zeppelin has it's own library of spark jars. Next go to

%Zeppelin_HOME%\conf\

and rename

zeppelin-env.cmd.template to zeppelin-env.cmd

and add following lines:

set JAVA="C:\Program Files\Java\jdk1.8.0_181"
set JAVA_HOME="%JAVA%"

Make sure you enter the correct path to your jdk. Save it and start zepplin with the command bin\zeppelin.cmd
Your zeppelin with Spark is ready! Confirm by running any simple code: For example, sc.version

Hope this helps!