Guava ImmutableMap has noticeably slower access than HashMap

Question

While working on a memory benchmark of some high-throughput data structures, I realized I could use an ImmutableMap with only a little refactoring.

Thinking this would be an improvement, I threw it into the mix and was surprised to discover that not only was it slower than HashMap, in a single-threaded environment it appears to be consistently slower even than ConcurrentHashMap!

You can see the full benchmark here: https://bitbucket.org/snippets/dimo414/89K7G

The meat of the test is pretty simple, time how long it takes to get a large number of random strings that may exist in the map.

public static void timeAccess(Map<String,String> map) {
    Random rnd = new Random(seed);
    int foundCount = 0;

    long start = System.nanoTime();

    for(int i = 0; i < loop; i++) {
        String s = map.get(RndString.build(rnd));
        if(s != null)
            foundCount++;
    }

    long stop = System.nanoTime() - start;

    System.out.println("Found "+foundCount+" strings out of "+loop+" attempts - "+
        String.format("%.2f",100.0*foundCount/loop)+" success rate.");
    System.out.println(map.getClass().getSimpleName()+" took "+
        String.format("%.4f", stop/1_000_000_000.0)+" seconds.");
    System.out.println();
}

And running this against a HashMap, a ConcurrentHashMap, and an ImmutableMap, all containing the same values, consistently showed a dramatic slowdown when using ImmutableMap - often upwards of 15% slower. The more sparse the map (i.e. the more often map.get() returned null) the greater the disparity. Here's the result of a sample run:

Found 35312152 strings out of 100000000 attempts - 35.31 success rate.
HashMap took 29.4538 seconds.

Found 35312152 strings out of 100000000 attempts - 35.31 success rate.
ConcurrentHashMap took 32.1465 seconds.

Found 35312152 strings out of 100000000 attempts - 35.31 success rate.
RegularImmutableMap took 37.9709 seconds.

Is this a documented / expected issue? The Guava Docs indicate Immutable*** is more memory efficient, but says nothing about speed. For slowdowns of this magnitude, I'm inclined to deal with the memory costs and avoid Immutable*** when speed is an issue (and when isn't it?!). Am I missing something?

See also: https://groups.google.com/forum/?fromgroups=#!topic/guava-discuss/I7yPpa5Hlpg

Answer 1

As Louis Wasserman said, ImmutableMap is not optimized for objects with slow equals method. I think the main difference is here:

HashMap:

if (e.hash == hash && ((k = e.key) == key || key.equals(k)))
    return e.value;

ImmtubleMap:

if (key.equals(candidateKey)) {
    return entry.getValue();

As you can see, to check for collisions, HashMap first check the hashes. This allows to reject values with different hashes fast. Since String doesn't make this check in its equals method, this makes HashMap faster. ImmutableMap doesn't use this optimization because it would make the test slower when equals is already optimized.

Answer 2

Some possible reasons:

1. Could that depend on your implementation of RndString.build()?

2. And have a look at the get() implementation of both maps: com.google.common.collect.RegularImmutableMap.get(Object) java.util.HashMap.getEntry(Object) java.util.HashMap tries to compare with "==" first. RegularImmutableMap doesn't. That may speed up

3. Could a different load factor be responsible for that? Perhaps the RegularImmutableMap needs more iterations to find the correct entry.