Bit shifting in internet checksums

Question

This is almost certainly a very silly question, but for some reason I'm having trouble with internet checksum calculations. All of the algorithms basically look something like this:

WORD chksm(WORD *startpos, WORD checklen){
ulong sum = 0;
WORD answer = 0;

while (checklen > 1)
{
    sum += *startpos++;
    checklen -= 2;
}

if (checklen == 1)
{
    *(BYTE *)(&answer) = *(BYTE *)startpos;
    sum += answer;
}

sum = (sum >> 16) + (sum & 0xffff);
sum += (sum >> 16);
answer = ~sum;

return answer;}

I'm clear on everything except for the line:

sum += (sum >> 16);

It looks like the line immediately before it adds the top 16 bits to the bottom 16 bits, leaving all zeroes in the top 16 bits. If that's the case, then wouldn't sum >> 16 now be equal to zero? And if so, why is that line there?

Or am I (likely) just having a complete mental failure today?

Answer 1

You are almost right.

The high 16 bits could have been 1 due to a carry.

For example, FFFF + FFFF => 1FFFE, or perhaps FFFF + 1 => 10000.

Answer 2

This is part of the definition of a one's complement sum. You take any overflow bits and add them back to the lower 16 bits. Adding them back could cause further overflow, so you repeat this until the upper bits end up all zero. So, conceptually it's this:

while (sum >> 16 != 0) {
    sum = (sum >> 16) + (sum & 0xffff);
}

However this loop will only ever execute at most twice, so an explicit loop is not necessary. After the first addition there may or may not be an overflow with a carry bit ending up in the upper 16 bits. In that case the upper 16 bits will be 0x0001 and you will have to do one more addition to add that carry bit back in.

Imagine the worst case where sum ends up as 0xffffffff after the initial while loop. Then the additions will proceed as follows:

sum = (0xffffffff >> 16) + (0xffffffff & 0xffff)
    = 0xffff + 0xffff
    = 0x1fffe

sum = (0x1fffe >> 16) + (0x1fffe & 0xffff)
    = 0x1 + 0xfffe
    = 0xffff

And there with two additions we are finished as the upper 16 bits are now clear. This is the worst case, so the loop can thus be unrolled to two additions.

(And then after all that you take the one's complement of the final sum, leading to the highly confusing name for this: the one's complement of the one's complement sum. It took me a long time to wrap my head around this the first time I implemented it—in particular that a one's complement sum does not involve the ~ complement operator.)

Answer 3

I thought that a ulong was 32 bits wide which means that:

sum = (sum >> 16) + (sum & 0xffff)
sum += (sum >> 16);

Adds the top sicteen bits and bottom sisteen bits together. And then the next line makes sum the result of the top sixteen bits; which could have a one in it due to a carry operation.