问题
The databricks docs say that you can change zordering of a delta table by doing:
spark.read.table(connRandom)
.write.format("delta").saveAsTable(connZorder)
sql(s"OPTIMIZE $connZorder ZORDER BY (src_ip, src_port, dst_ip, dst_port)")
The problem with this is the switching between the scala and SQL api which is gross. What I want to be able to do is:
spark.read.table(connRandom)
.write.format("delta").saveAsTable(connZorder)
.optimize.zorderBy("src_ip", "src_port", "dst_ip", "dst_port")
but I cant find any documentation that says that this is possible.
Is there a scala api for delta lake optimization commands? If so, how do I replicate the aforementioned logic in scala?
来源:https://stackoverflow.com/questions/65098666/scala-api-for-delta-lake-optimize-command