Migrating Memcached client from Memcached-Java-Client to Xmemcached

Question

I have an old memcached client implementation that is based of the danga client, but was slightly modified. I think this client is now called Memcached-Java-Client.

The implementation I have at hand is old, unmaintained, and seem to be bogus.

I need to migrate the client to a newer client with the following constraints:

1. I need the new client hashing algorithm to be compatible with the old one. If this constraint is not met, we will have to make the site (at least partially) unavailable for a few hours.
2. Support various serialization methods, and allow extensibility for the serialization method.
3. Improved performance over my existing client.
4. Be compatible with Moxi

The current implementation uses a consistent hashing algorithm which seemed to be compatible to the consistent hashing used by Xmemcached except that I can't get a cache hit for keys stored by the old client and fetched with the Xmemcached client.

I debugged the code, and I get the same hashcode, but I think the servers ring is different, and thus the overall hashing method is different.

This is the code I used for initializing the xmemcached client:

  final MemcachedClientBuilder builder = new XMemcachedClientBuilder(AddrUtil.getAddresses("server:11211 server:11212 server:11213 server:11214"));
  builder.setCommandFactory(new BinaryCommandFactory());
  builder.setSessionLocator(new KetamaMemcachedSessionLocator());
  builder.setTranscoder(new WhalinTranscoder());
  memcachedClient = builder.build();

I am willing to also give spymemcached a try if I can't get xmemcached to work, but I prefer the later.

Edit: I tried spymemcached as well, and can't get it to be compatible with my current client. Here's my spymemcached client initialization code:

final ConnectionFactoryBuilder builder = new ConnectionFactoryBuilder();
builder.setProtocol(Protocol.BINARY);
builder.setTranscoder(WhalinTranscoder());
builder.setHashAlg(HashAlgorithm.KETAMA_HASH);
builder.setLocatorType(Locator.CONSISTENT);
final MemcachedClient client = MemcachedClient(builder.build(), AddrUtil.getAddresses("server:11211 server:11212 server:11213 server:11214"));

Answer 1

The way i understand you will have the same problem as in xmemcached with spymemcached. As i noted before here- http://bugs.membase.org/browse/MB-1484, the consistent hashing you are using now it a weighted consistent hashing as opposed to other 2, and that's why you are missing some of the keys.

Answer 2

It seems like my old danga client is incompatible with the Xmemcached/Spymemcached clients due to a simple reason: The danga client hashes the host:port strings while the later clients hash the InetSocketAddress which is effectively equals to host/IP:port.

Due to this fact I am unable to remain compatible when migrating to a newer client. Well, actually both Xmemcached, and Spymemcached clients are somewhat extensible and do allow some support in making this happen, but doing this will be too 'hacky' to my preferences.

What I decided to do is to migrate my code to use one of Xmemcached/Spymemcached according to benchmarks results. Both are compatible with moxi's hashing algorithm. I may eventually end up using a client side moxi only - depending on the overhead it will add. I like this option best since it makes the application configuration much simpler.