Python randomly drops to 0% CPU usage, causing the code to “hang up”, when handling large numpy arrays?

Question

I have been running some code, a part of which loads in a large 1D numpy array from a binary file, and then alters the array using the numpy.where() method.

Here is

Answer 1

np.where is creating a copy there and assigning it back into arr. So, we could optimize on memory there by avoiding a copying step, like so -

vol_avg = (np.sum(arr) - (arr[arr >=  1.0] - 1.0).sum())/(num**3)

We are using boolean-indexing to select the elements that are greater than 1.0 and getting their offsets from 1.0 and summing those up and subtracting from the total sum. Hopefully the number of such exceeding elements are less and as such won't incur anymore noticeable memory requirement. I am assuming this hanging up issue with large arrays is a memory based one.