Why does pickle take so much longer than np.save?

Question

I want to save a dict or arrays.

I try both with np.save and with pickle and see that the former always take much less time.

Answer 1

Because as long as the written object contains no Python data,

• numpy objects are represented in memory in a much simpler way than Python objects
• numpy.save is written in C
• numpy.save writes in a supersimple format that needs minimal processing

meanwhile

• Python objects have a lot of overhead
• pickle is written in Python
• pickle transforms the data considerably from the underlying representation in memory to the bytes being written on the disk

Note that if a numpy array does contain Python objects, then numpy just pickles the array, and all the win goes out the window.