key-value store suggestion

Question

I need a very basic key-value store for java. I started with a HashMap but it seems that HashMap is somewhat space inefficient (I\'m storing ~20 million records, and seems t

Answer 1

The map is Map, and so I'm considering using GNU Trove TIntObjectHashMap, and storing the map value as an ascii byte array rather than String.

This doesn't entirely make sense because a TIntObjectHashMap is not a Map. However, the approach is sound.

---

Do you know what kind of space savings I can expect over HashMap for Trove?

The best answer is to try it out.

But here are some rough estimates (assuming a 32bit JVM):

• HashMap keys would need to be Integer instances. They will occupy ~18bytes per instance + 4 bytes per reference. Total 24 bytes.

• Trove keys would be 4 byte int values.

• String values would be 20 bytes + 12 bytes + 2 * number of "characters".

• Byte array values would be 12 bytes + 1 * number of "characters".

• I haven't examined the details of the respective hash table internal data structures.

That probably amounts to around 50% memory saving, though it depends critically on the average length of the value "strings". (The longer they are, the more they will dominate the space usage.)

FWIW, Trove publish their own benchmarks here. They don't look very convincing, but you should be able to dig out their benchmark code and modify it to better match your use-case.