Understanding spark physical plan

后端 未结 2 1449
悲哀的现实
悲哀的现实 2020-12-08 02:45

I\'m trying to understand physical plans on spark but I\'m not understanding some parts because they seem different from traditional rdbms. For example, in this plan below,

2条回答
  •  悲&欢浪女
    2020-12-08 03:42

    Tungsten is the new memory engine in Spark since 1.4, which manages data outside JVM to save some GC overhead. You can imagine doing that involves copy data from and to JVM. That's it. In Spark 1.5 you can turn Tungsten off through spark.sql.tungsten.enabled then you will see the "old" plan, in Spark 1.6 I think you can't turn it off any more.

提交回复
热议问题