pyspark 操作

1. 日期转时间戳

spark_df_from_csv = spark.read.csv('/data1/AIPlatform/look_order_cross_city_new_deepfm_0116_0130_origin.csv', header=True)

spark_df_from_csv = spark_df_from_csv.withColumn('parsed_log_time', spark_df_from_csv['parsed_log_time'].cast("timestamp"))

spark_df_from_csv = spark_df_from_csv.withColumn('parsed_log_time', F.unix_timestamp(spark_df_from_csv['parsed_log_time']))

 

来源：CSDN

作者：我是女孩

链接：https://blog.csdn.net/u013385018/article/details/104270438