I would definitely choose a TreeMap:
- TreeMap automatically sorts new keys on insertion, no sorting afterwards is needed.
- When a key already exists it has the same performance as a HashMap.
A TreeSet internally uses a TreeMap so why not use TreeMap directly.