If key class is your's make sure the hashCode() and equals() methods implemented.
Basically the access to HashMap should be O(1) but with wrong hashCode method implementation it's become O(n), because value with same hash key will stored as Linked list.