问题
We hava a Redis key. It's a ZSET structure named test_key.
The key is userId like 123,456,789.The score is time stamps like 1474194838, 1474194839. Its length reached fifty million. We want to split it, just like test_key_1, test_key_2, test_key_3.
How to split it that can make the CRUD more easy?
We are java developer. The most frequently used Redis commodity is zadd, zrem, zrange, zrangeByscore, zrangeByscoreWithScores, zcard and so on.
回答1:
If you have to split the data into several Redis instances, then you need a proxy to dispatch the request to one or more Redis instances, and merge the results from these instances. The proxy can be implemented as a library or a service (e.g. a RPC server).
Dispatch request
Say you split the data into 3 parts, and stored in 3 Redis instances. In order to ensure load balance, the proxy can use MurmurHash function to create a hash key for each userId, and dispatches the request to one or more Redis instances based on the hash key. Take zadd as an example: zadd test_key userId score.
- Calculate Redis instance id:
id = MurmurHash(userId) mod 3 - Send
zaddcommand to the corresponding Redis:zadd test_key_id userId score
Merge results
When the proxy gets results from one or more Redis instances, it merges the results and returns to the client. Take zcard as an example: zcard test_key.
- Get results from all instances:
zcard1 = zcard test_key1,zcard2 = zcard test key2,zcard3 = zcard test_key3 - Merge results:
zcard_res = zcard1 + zcard2 + zcard3 - Return the result, i.e.
zcard_res, to client.
Improve performance
In order to improve performance, when the proxy dispatches request to more than one instance, it should dispatch the request in parallel.
来源：https://stackoverflow.com/questions/39556728/split-a-redis-big-zset