when does python delete variables?

Question

I know that python has an automatic garbage collector and so it should automatically delete variables when there are no more reference to them.

My impression is tha

Answer 1

Implementations use reference counting to determine when a variable should be deleted.

After the variable goes out of scope (as in your example) if there are no remaining references to it, then the memory will be freed.

def a():
    x = 5 # x is within scope while the function is being executed
    print x

a()
# x is now out of scope, has no references and can now be deleted

Aside from dictionary keys and elements in lists, there's usually very little reason to manually delete variables in Python.

Though, as said in the answers to this question, using del can be useful to show intent.

Answer 2

It's important to keep two concepts separate: names and values. A variable in Python is a name referring to a value. Names have scope: when you define a local variable (by assigning a value to a name), the variable's scope is the current function. When the function returns, the variable goes away. But that doesn't mean the value goes away.

Values have no scope: they stick around until there are no more names referring to them. You can create a value in a function, and return it from that function, making a name outside the function refer to the value, and the value won't be reclaimed until some future point when all the references to it have gone away.

More detail (including pictures!) is here: Facts and Myths about Python Names and Values.

Answer 3

Write stuff you want to clear from memory in separate functions. In your example, you can do

  def xdef(z):
    x = f(z) # x is a np.array and contains a lot of data 
    x0 = x[0]

  def funz(z):
    xdef(z)
    y = f(z + 1) # y is a np.array and contains a lot of data
    y0 = y[0]  

    return y[0], x[0] 

This will cause an exception

Answer 4

It depends on the implementation and the type of variable. For simple objects like ints there are some optimisations. In CPython, for example, a simple int will reuse the same memory, even after del has been used. You can't count on that, but it does illustrate that things are more complex than they appear.

Remember that when you del you are deleting a name, not necessarily an object.
For example:

# x is a np.array and contains a lot of data

Would be more accurately worded as:

# x references a np.array which contains a lot of data

del will decrement the reference count on that object, but even when it drops to zero it is not guaranteed to be garbage collected any time soon.

Suggest you look at the gc module for an explanation and inspiration. Then think again.

If you are getting "out of memory" then you probably have a fundamental problem with your design. Most likely you are loading too much data in at one go (try using iterators?), or maybe your code need to be structured better.

I just saw your edit. Do you need all of that array in memory at the same time? Could you use a generator?

Another alternative is to use a database like SQLite or maybe a shelve