Should memory usage increase when using ElementTree.iterparse() when clear()ing trees?

Question

import os
import xml.etree.ElementTree as et

for ev, el in et.iterparse(os.sys.stdin):
    el.clear()

Running the above on the ODP structure RDF d

Answer 1

I ran into the same issue. The documentation doesn't make things very clear. The issue in my case was:

1) Calling clear does release memory for the children nodes. Documentation says that it releases all memory. Clear does not release the memory for which clear is called, because that memory belongs to the parent which allocated it. 2) Calling root.clear(), that depends on what root is. If root is the parent then it would work. Otherwise, it will not free the memory.

The fix was to keep a reference to the parent, and when we no longer need the node, we call parent.remove(child_node). This worked and it kept the memory profile at a few KBs.