How can I make (Spark1.6) saveAsTextFile to append existing file?

前端 未结 3 428
难免孤独
难免孤独 2020-12-17 04:21

In SparkSQL,I use DF.wirte.mode(SaveMode.Append).json(xxxx),but this method get these files like

the filename is too complex and random,I can\'t use

3条回答
  •  自闭症患者
    2020-12-17 04:54

    worked on Spark 1.5 , I think this is right usage..

    dataframe.write().mode(SaveMode.Append).format(FILE_FORMAT).**partitionBy**("parameter1", "parameter2").save(path);
    

提交回复
热议问题