Insert overwrite vs Delete HDFS partition and write parquet file in spark

馋奶兔 提交于 2019-12-08 01:59:57

问题


Is there any major performance issue for parquet in using

hiveContext.sql("INSERT OVERWRITE TABLE table_name PARTITION(dt='2016-06-08', country) , select x,y,z,country from DFTmpTable");

VS

DeleteHDFSfile(/table/dt='2016-06-08')
DF.write().mode("append").partitionBy("dt","country").parquet("/table/dt='2016-06-08'")

来源:https://stackoverflow.com/questions/39810659/insert-overwrite-vs-delete-hdfs-partition-and-write-parquet-file-in-spark

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!