Extremely slow S3 write times from EMR/ Spark

Question

I\'m writing to see if anyone knows how to speed up S3 write times from Spark running in EMR?

My Spark Job takes over 4 hours to complete, however the cluster is onl

Answer 1

How large is the file(s) you are writing too? Having one core writing to a very large file is going to be much slower than splitting the file up and have multiple workers write out smaller files.