redis cluster怎么保证键的均匀分配(crc16算法)

匿名 (未验证) 提交于 2019-12-03 00:44:02

一、虚拟槽分区

Redis Cluser采用虚拟槽分区,所有的键根据哈希函数映射到0~16383个整数槽内,计算公式:slot=CRC16(key)&16383。
由于采用高质量的哈希算法,每个槽所映射的数据通常比较均匀。

二、源码论证

1、keyHashSlot 函数

keyHashSlot 函数用来计算给定键应该被分配到哪个槽

/* --------------cluster.c---------------------------------------------------------------  * Key space handling  * -------------------------------------------------------------------------- */  /* We have 16384 hash slots. The hash slot of a given key is obtained  * as the least significant 14 bits of the crc16 of the key.  *  * However if the key contains the {...} pattern, only the part between  * { and } is hashed. This may be useful in the future to force certain  * keys to be in the same node (assuming no resharding is in progress). */  //我们有16384个哈希槽,将会用crc16得到的最小有效14位作为给定key的哈希槽。  //但是,如果键包含‘{...}’这样的字段,只有在"{“和“}”之间的才会被哈希。这在将来可能有助于将某些键强制放在同一个节点中(假设没有进行重新分片   // 计算给定键应该被分配到哪个槽 unsigned int keyHashSlot(char *key, int keylen) {     int s, e; /* start-end indexes of { and } */      for (s = 0; s < keylen; s++)         if (key[s] == '{') break;      /* No '{' ? Hash the whole key. This is the base case. */     //0x3FFF转化成十进制就是16383     if (s == keylen) return crc16(key,keylen) & 0x3FFF;      /* '{' found? Check if we have the corresponding '}'. */     for (e = s+1; e < keylen; e++)         if (key[e] == '}') break;      /* No '}' or nothing betweeen {} ? Hash the whole key. */     if (e == keylen || e == s+1) return crc16(key,keylen) & 0x3FFF;      /* If we are here there is both a { and a } on its right. Hash      * what is in the middle between { and }. */     return crc16(key+s+1,e-s-1) & 0x3FFF; } 

2、CRC16算法实现(crc16.c)

具体的CRC16算法实现如下:

//crc16tab数组,便于进行“查表法”计算crc结果,提高效率 static const uint16_t crc16tab[256]= {     0x0000,0x1021,0x2042,0x3063,0x4084,0x50a5,0x60c6,0x70e7,     0x8108,0x9129,0xa14a,0xb16b,0xc18c,0xd1ad,0xe1ce,0xf1ef,     0x1231,0x0210,0x3273,0x2252,0x52b5,0x4294,0x72f7,0x62d6,     0x9339,0x8318,0xb37b,0xa35a,0xd3bd,0xc39c,0xf3ff,0xe3de,     0x2462,0x3443,0x0420,0x1401,0x64e6,0x74c7,0x44a4,0x5485,     0xa56a,0xb54b,0x8528,0x9509,0xe5ee,0xf5cf,0xc5ac,0xd58d,     0x3653,0x2672,0x1611,0x0630,0x76d7,0x66f6,0x5695,0x46b4,     0xb75b,0xa77a,0x9719,0x8738,0xf7df,0xe7fe,0xd79d,0xc7bc,     0x48c4,0x58e5,0x6886,0x78a7,0x0840,0x1861,0x2802,0x3823,     0xc9cc,0xd9ed,0xe98e,0xf9af,0x8948,0x9969,0xa90a,0xb92b,     0x5af5,0x4ad4,0x7ab7,0x6a96,0x1a71,0x0a50,0x3a33,0x2a12,     0xdbfd,0xcbdc,0xfbbf,0xeb9e,0x9b79,0x8b58,0xbb3b,0xab1a,     0x6ca6,0x7c87,0x4ce4,0x5cc5,0x2c22,0x3c03,0x0c60,0x1c41,     0xedae,0xfd8f,0xcdec,0xddcd,0xad2a,0xbd0b,0x8d68,0x9d49,     0x7e97,0x6eb6,0x5ed5,0x4ef4,0x3e13,0x2e32,0x1e51,0x0e70,     0xff9f,0xefbe,0xdfdd,0xcffc,0xbf1b,0xaf3a,0x9f59,0x8f78,     0x9188,0x81a9,0xb1ca,0xa1eb,0xd10c,0xc12d,0xf14e,0xe16f,     0x1080,0x00a1,0x30c2,0x20e3,0x5004,0x4025,0x7046,0x6067,     0x83b9,0x9398,0xa3fb,0xb3da,0xc33d,0xd31c,0xe37f,0xf35e,     0x02b1,0x1290,0x22f3,0x32d2,0x4235,0x5214,0x6277,0x7256,     0xb5ea,0xa5cb,0x95a8,0x8589,0xf56e,0xe54f,0xd52c,0xc50d,     0x34e2,0x24c3,0x14a0,0x0481,0x7466,0x6447,0x5424,0x4405,     0xa7db,0xb7fa,0x8799,0x97b8,0xe75f,0xf77e,0xc71d,0xd73c,     0x26d3,0x36f2,0x0691,0x16b0,0x6657,0x7676,0x4615,0x5634,     0xd94c,0xc96d,0xf90e,0xe92f,0x99c8,0x89e9,0xb98a,0xa9ab,     0x5844,0x4865,0x7806,0x6827,0x18c0,0x08e1,0x3882,0x28a3,     0xcb7d,0xdb5c,0xeb3f,0xfb1e,0x8bf9,0x9bd8,0xabbb,0xbb9a,     0x4a75,0x5a54,0x6a37,0x7a16,0x0af1,0x1ad0,0x2ab3,0x3a92,     0xfd2e,0xed0f,0xdd6c,0xcd4d,0xbdaa,0xad8b,0x9de8,0x8dc9,     0x7c26,0x6c07,0x5c64,0x4c45,0x3ca2,0x2c83,0x1ce0,0x0cc1,     0xef1f,0xff3e,0xcf5d,0xdf7c,0xaf9b,0xbfba,0x8fd9,0x9ff8,     0x6e17,0x7e36,0x4e55,0x5e74,0x2e93,0x3eb2,0x0ed1,0x1ef0 };  //具体的crc16算法实现 uint16_t crc16(const char *buf, int len) {     int counter;     uint16_t crc = 0;     for (counter = 0; counter < len; counter++)             crc = (crc<<8) ^ crc16tab[((crc>>8) ^ *buf++)&0x00FF];     return crc; } 

三、结论

观察上面的crc16算法实现,根据前面 keyHashSlot 函数中的 crc16(key,keylen) & 0x3FFF 可以看到这边会计算‘kenlen’次crc运算,其中每次的运算结果又跟‘key’有关系,因为传入的key值和key的长度不一样,所以基本上可以保证键均匀的分配到各个槽位,各个节点上。

具体的CRC循环冗余校验算法原理可以参考下面的文章:
https://www.cnblogs.com/esestt/archive/2007/08/09/848856.html

文章若有表述不当或者欠缺的地方,欢迎指正~~~

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!