How does Spark calculate how much memory is being used? Why is it easy to run out of memory?

Question

We know that Spark uses a lot of memory, and that memory usage is divided between execution and storage.

Related instructions show that spark can statistics of how ma