Zeppelin - Cannot query with %sql a table I registered with pyspark

Question

I am new to spark/zeppelin and I wanted to complete a simple exercise, where I will transform a csv file from pandas to Spark data frame and then register the table to query

Answer 1

You didn't say which interpreter group you were using. If it's livy then you can't access tables registered in %livy.pyspark from %livy.sql. I got this from here:

for now %livy.sql can only access tables registered %livy.spark, but not %livy.pyspark and %livy.sparkr.

If you switch to the standard spark interpreter group it should work. I can confirm this for me using Spark 1.6.3 and Zeppelin 0.7.0. Hopefully the people working on the livy interpreter will fix this restriction...