Stream operations will block until done for both - parallel and not parallel implementations.
So what you see is not the "putting data" process is still going on - most likely it's just data corruption, since HashMap is not threadsafe.
Try using ConcurrentHashMap instead.