apache-zeppelin

Convert between spark.SQL DataFrame and pandas DataFrame [duplicate]

柔情痞子 提交于 2019-11-26 23:25:32
问题 This question already has an answer here: Requirements for converting Spark dataframe to Pandas/R dataframe 1 answer Is that possible to convert from to pd.DataFrame under %pyspark environment ? 回答1: Try: spark_df.toPandas() toPandas() Returns the contents of this DataFrame as Pandas pandas.DataFrame. This is only available if Pandas is installed and available. And if you want the oposite: spark_df = createDataFrame(pandas_df) 来源: https://stackoverflow.com/questions/41826553/convert-between

Zeppelin: Scala Dataframe to python

与世无争的帅哥 提交于 2019-11-26 22:52:22
问题 If I have a Scala paragraph with a DataFrame, can I share and use that with python. (As I understand it pyspark uses py4j) I tried this: Scala paragraph: x.printSchema z.put("xtable", x ) Python paragraph: %pyspark import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns the_data = z.get("xtable") print the_data sns.set() g = sns.PairGrid(data=the_data, x_vars=dependent_var, y_vars=sensor_measure_columns_names + operational_settings_columns_names, hue=