One of my students showed my the following test case that shows an apparent memory leak in NumPy. I'm wondering if the memory profiler is correct here, or what's going on. Here's the test case:
from memory_profiler import profile
import numpy as np
import gc
@profile
def test():
arr = np.ones((10000, 6912))
for i in range(2000):
arr[0:75,:] = np.ones((75, 6912))
del arr
gc.collect()
pass
test()
This produces the following output:
Filename: test.py
Line # Mem usage Increment Occurences Line Contents
============================================================
5 32.9 MiB 32.9 MiB 1 @profile
6 def test():
7 560.3 MiB 527.4 MiB 1 arr = np.ones((10000, 6912))
8 564.2 MiB 0.0 MiB 2001 for i in range(2000):
9 564.2 MiB 3.9 MiB 2000 arr[0:75,:] = np.ones((75, 6912))
10 37.0 MiB -527.3 MiB 1 del arr
11 37.0 MiB -0.0 MiB 1 gc.collect()
12 37.0 MiB 0.0 MiB 1 pass
It looks like the line with np.ones((75, 6912)) is slowly leaking memory (about 4MB here). If we replace this expression with just 1, then the apparent leak disappears.
I've tested this on Python 3.8.10 and 3.9.5 with Numpy versions 1.21.3 (latest at time of writing) and 1.20.3 and memory_profiler version 0.58.0 (latest at time of writing). My operating system is Ubuntu Linux 20.04 LTS; my student demonstrated this on macOS (not sure which version).
What's going on?