How to ensure output dataframe is dynamically partitioned such that each partition is around 128mb?

后端 未结 0 469
悲哀的现实
悲哀的现实 2020-12-28 15:40

In Spark, I have a few jobs chained (i.e. output of one will be input to the next). The issue I am facing is, say, my input dataset to first job is 10GB today and I am repar

相关标签:
回答
  • 消灭零回复
提交回复
热议问题