Following up on this question, it seems that a file- or disk-based Map implementation may be the right solution to the problems I mentioned there.
The google-collections library, part of http://code.google.com/p/guava-libraries/, has some really useful Map tools. MapMaker in particular lets you make concurrent HashMaps with timed evictions, soft values that will be swept up by the garbage collector if you're running out of heap, and computing functions.
Map cache = new MapMaker()
.softValues()
.expiration(30, TimeUnit.MINUTES)
.makeComputingMap(new Function() {
@Override
public String apply(String input) {
// Work out what the value should be
return null;
}
});
That will give you a Map cache that will clean up after itself and can work out its values. If you're able to compute values like that then great, otherwise it would map perfectly onto http://redis.io/ which you'd be writing into (to be fair, redis would probably be fast enough on its own!).