apache-zeppelin

From Zeppelin-0.7, Zeppelin started supporting Helium plugins/packages using Helium Framework. However, I am not able to view any of the plugin on Helium page (localhost:8080/#/helium). As per this JIRA , I placed sample Helium.json (available on s3 ) under /local-repo/helium-registry-cache. However, after that I got NPE while restarting Apache Zeppelin service. I have tried Zeppelin 0.7 as well as Zeppelin 0.8.0 snaptshot versions. In particular, I want to use map Helium package - Helium-Map in Zeppelin note. Can some one point me to any guide or documentation having detailed steps of using

Is it possible to register a UDF (or function) written in Scala to use in PySpark ? E.g.: val mytable = sc.parallelize(1 to 2).toDF("spam") mytable.registerTempTable("mytable") def addOne(m: Integer): Integer = m + 1 // Spam: 1, 2 In Scala, the following is now possible: val UDFaddOne = sqlContext.udf.register("UDFaddOne", addOne _) val mybiggertable = mytable.withColumn("moreSpam", UDFaddOne(mytable("spam"))) // Spam: 1, 2 // moreSpam: 2, 3 I would like to use "UDFaddOne" in PySpark like %pyspark mytable = sqlContext.table("mytable") UDFaddOne = sqlContext.udf("UDFaddOne") # does not work

I have registertemptable in Apache Spark using Zeppelin below: val hvacText = sc.textFile("...") case class Hvac(date: String, time: String, targettemp: Integer, actualtemp: Integer, buildingID: String) val hvac = hvacText.map(s => s.split(",")).filter(s => s(0) != "Date").map( s => Hvac(s(0), s(1), s(2).toInt, s(3).toInt, s(6))).toDF() hvac.registerTempTable("hvac") After I have done with my queries with this temp table, how do I remove it ? I checked all docs and it seems I am getting nowhere. Any guidance ? Spark 2.x For temporary views you can use Catalog.dropTempView : spark.catalog

Installed apache-maven-3.3.3, scala 2.11.6, then ran: $ git clone git://github.com/apache/spark.git -b branch-1.4 $ cd spark $ build/mvn -DskipTests clean package Finally: $ git clone https://github.com/apache/incubator-zeppelin $ cd incubator-zeppelin/ $ mvn install -DskipTests Then ran the server: $ bin/zeppelin-daemon.sh start Running a simple notebook beginning with %pyspark , I got an error about py4j not being found. Just did pip install py4j ( ref ). Now I'm getting this error: pyspark is not responding Traceback (most recent call last): File "/tmp/zeppelin_pyspark.py", line 22, in

If I have a Scala paragraph with a DataFrame, can I share and use that with python. (As I understand it pyspark uses py4j ) I tried this: Scala paragraph: x.printSchema z.put("xtable", x ) Python paragraph: %pyspark import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns the_data = z.get("xtable") print the_data sns.set() g = sns.PairGrid(data=the_data, x_vars=dependent_var, y_vars=sensor_measure_columns_names + operational_settings_columns_names, hue="UnitNumber", size=3, aspect=2.5) g = g.map(plt.plot, alpha=0.5) g = g.set(xlim=(300,0)) g = g.add_legend()