Syntax while setting schema for Pyspark.sql using StructType

前端 未结 2 1121
日久生厌
日久生厌 2021-01-31 17:47

I am new to spark and was playing around with Pyspark.sql. According to the pyspark.sql documentation here, one can go about setting the Spark dataframe and schema like this:

2条回答
  •  面向向阳花
    2021-01-31 18:29

    You can also use a datatype string:

    schema = 'Name STRING, DateTime TIMESTAMP, Age INTEGER'
    

    There's not much documentation on datatype strings, but they mention them in the docs. They're much more compact and readable than StructTypes

提交回复
热议问题