Tracking *maximum* memory usage by a Python function

前端 未结 8 1664
轻奢々
轻奢々 2020-12-02 09:07

I want to find out what the maximum amount of RAM allocated during the call to a function is (in Python). There are other questions on SO related to tracking RAM usage:

8条回答
  •  谎友^
    谎友^ (楼主)
    2020-12-02 09:47

    Reading the source of free's information, /proc/meminfo on a linux system:

    ~ head /proc/meminfo
    MemTotal:        4039168 kB
    MemFree:         2567392 kB
    MemAvailable:    3169436 kB
    Buffers:           81756 kB
    Cached:           712808 kB
    SwapCached:            0 kB
    Active:           835276 kB
    Inactive:         457436 kB
    Active(anon):     499080 kB
    Inactive(anon):    17968 kB
    

    I have created a decorator class to measure memory consumption of a function.

    class memoryit:
    
        def FreeMemory():
            with open('/proc/meminfo') as file:
                for line in file:
                    if 'MemFree' in line:
                        free_memKB = line.split()[1]
                        return (float(free_memKB)/(1024*1024))    # returns GBytes float
    
        def __init__(self, function):    # Decorator class to print the memory consumption of a 
            self.function = function     # function/method after calling it a number of iterations
    
        def __call__(self, *args, iterations = 1, **kwargs):
            before = memoryit.FreeMemory()
            for i in range (iterations):
                result = self.function(*args, **kwargs)
            after = memoryit.FreeMemory()
            print ('%r memory used: %2.3f GB' % (self.function.__name__, (before - after) / iterations))
            return result
    

    Function to measure consumption:

    @memoryit
    def MakeMatrix (dim):
        matrix = []   
        for i in range (dim):
            matrix.append([j for j in range (dim)])
        return (matrix)
    

    Usage:

    print ("Starting memory:", memoryit.FreeMemory()) 
    m = MakeMatrix(10000)    
    print ("Ending memory:", memoryit.FreeMemory() )
    

    Printout:

    Starting memory: 10.58599853515625
    'MakeMatrix' memory used: 3.741 GB
    Ending memory: 6.864116668701172
    

提交回复
热议问题