How does createOrReplaceTempView work in Spark?

前端 未结 3 735
轻奢々
轻奢々 2020-12-02 14:16

I am new to Spark and Spark SQL.

How does createOrReplaceTempView work in Spark?

If we register an RDD of objects as a table will

3条回答
  •  心在旅途
    2020-12-02 14:43

    SparkSQl support writing programs using Dataset and Dataframe API, along with it need to support sql.

    In order to support Sql on DataFrames, first it requires a table definition with column names are required, along with if it creates tables the hive metastore will get lot unnecessary tables, because Spark-Sql natively resides on hive. So it will create a temporary view, which temporarily available in hive for time being and used as any other hive table, once the Spark Context stop it will be removed.

    In order to create the view, developer need an utility called createOrReplaceTempView

提交回复
热议问题