Spark 3.0.1 (Databricks 7.3) Write DataFrame to S3 Parquet with mode(“overwrite”): FileAlreadyExistsException

Question

I build a dataframe then write it out to s3 like so:

df.write.mode("overwrite").option("compression"