reading too many small files in pyspark taking so much of time

后端 未结 0 1414
深忆病人
深忆病人 2020-12-08 13:33

I have written pyspark job to load files present in s3 bucket . In s3 there are too many small files , I am reading file one by one in spark . I am reading a file one one by

相关标签:
回答
  • 消灭零回复
提交回复
热议问题