According to these:
It's a coincidence (not really, rather it has to do with the hashing algorithm).
Try adding
newHashMap.put(-5, "Fifth");
as last.
Output will be
Key: 0
Value: Second
Key: 1
Value: Fourth
Key: 2
Value: First
Key: 3
Value: Third
Key: -5
Value: Fifth
The javadoc specifically says
This class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time.