Spyder does not realease memory for matplotlib plots

问题

I use Spyder 3.3.6, with Pyhton 3.7.3 over IPyhton 7.7.0 cosole.

For different reasons, Spyder has always been my IDE of choice, but now I have decided to drop it altogether and stick to command prompt only.

The reason is that I have seen a serious memory leak while using matplotlib which is caused by Spyder.

Memory increases linearly until all my 128 GB of memory is consumed to the extend that my Ubuntu machine halted completely and I had to hard reset it. I have had this issue in the past with matplotlib and the solution I found, things like using

plt.close('all') or plt.close(fig) or gc.collect()

helped to some extent, but this time none worked (I should add that I am using MNE library to plot a long list of EEG raw files, hence I cannot try every possible solution that exists for such matplotlib memory issues).

But when I ran the same code using both python and ipython in command prompt, a simple

plt.close(fig)

solved the issue and the memory stays in the order of a few GB during the whole run. Hence, the only logical conclusion is that Spyder somehow messes with the memory management.

So, I wrote this question in case somebody knows a solution to this issue or if not, other people who use Spyder and probably have wasted hours trying to find a python solution would know the issue is in Spyder, not python.

回答1:

For me, using gc.collect resolved the issue. I am using: Spyder 3.3.6, Python 3.7.6, Ipython 7.11.1.

Minimal working/crashing example

import numpy as np
import matplotlib.pylab as plt
import os
import psutil
import gc
process = psutil.Process(os.getpid())
def str_mem_usage():
    mu = process.memory_info().rss / 1024**2
    return 'Memory usage: {:.2f} MB'.format(mu)

arrs = []
for ii in range(10):
    print('it', ii, '\t', str_mem_usage())
    n = 10000
    arr = np.random.rand(n**2).reshape(n, n)
    #arrs += [arr]  # activate for memory leak, obviously

    plt.ioff()
    fig, ax = plt.subplots(1, 1)
    ax.imshow(arr)
    #plt.savefig('tmp.pdf')  # irrelevant for memory leak
    plt.close(fig)
    plt.ion()

    gc.collect()  # deactivate for memory leak

This behaviour is the same for execution in spyder or calling python or ipython from the anaconda distribution.

Output (no memory leak)

it 0     Memory usage: 147.35 MB
it 1     Memory usage: 1682.64 MB
it 2     Memory usage: 1682.98 MB
it 3     Memory usage: 1682.99 MB
it 4     Memory usage: 1682.99 MB
it 5     Memory usage: 1682.99 MB
it 6     Memory usage: 1678.63 MB
it 7     Memory usage: 1644.47 MB
it 8     Memory usage: 1633.58 MB
it 9     Memory usage: 1633.56 MB

Output (memory leak)

it 0     Memory usage: 108.89 MB
it 1     Memory usage: 1635.26 MB
it 2     Memory usage: 2393.71 MB
it 3     Memory usage: 3156.51 MB
it 4     Memory usage: 3919.34 MB
it 5     Memory usage: 4681.07 MB
it 6     Memory usage: 5428.71 MB
...
MemoryError: Unable to allocate 763. MiB for an array with shape (100000000,) and data type float64

Maybe this answer helps some people to reproduce / solve this matplotlib memory leak issue. This is probably the 5th time, I step over it.

来源：https://stackoverflow.com/questions/57409470/spyder-does-not-realease-memory-for-matplotlib-plots

标签

python-3.x

matplotlib

memory-leaks

ipython

spyder