How does comparing the Sign and Overflow Flag determine operand relationships?

Question

Jump\'s based on comparing signed integers use the Zero, Sign, and Overflow flag to determine the relationship between operands. After CMP with two signed opera

Answer 1

Performing the signed subtraction R = Destination - Source yields a signed result.

Suppose there is no overflow - the usual arithmetic laws holds: if R = Destination - Source > 0 then Destination > Source.
Having no overflow means OF = 0 and R > 0 means SF = 0.

Now suppose there is an overflow - let's call O the most significant, non-sign, bit and S the sign bit.
An overflow condition means that either a) Computing the result's O needed a borrow and result's S didn't or b) result's O didn't need a borrow and S did.

In case a) since result's S didn't need a borrow, the two S bits of the operands were either (1, 0) (1, 1) or (0, 0).
Since result's O needed a borrow, and thus flipping the first source S bit, we must exclude the second and third option.
So the operands sign bits were 1 and 0 (thus Destination < Source), the result's sign bit SF = 0 and OF = 1 by hypothesis.

In case b) since result's S did need a borrow, the two S bits of the operands were (0, 1).
Since O didn't need a borrow, the first operand S bit has been not changed and we don't need to consider any further case.
So the operands sign bits were 0 and 1 (thus Destination > Source), the result's sign bit SF = 1 and OF = 1 by hypothesis.

To recap:

• If OF = 0 then Destination > Source => SF = 0.
• If OF = 1 then Destination > Source => SF = 1.

In short OF = SF.

Answer 2

The OF flags tracks signed overflow, i.e. a change in the sign.
The sign flag obviously just tracks whether a number is negative or not.
Both flags monitor the sign or most significant bit (MSB) of the destination operand.

The compare CMP instructions perform a subtract.
If A != B and both operands have the same sign then obviously the following will happen (assume dword operands).

 100 -  200 = -100 (sign change OF=1 + SF=1, ergo A(100) < B(200)).
-100 - -200 =  300 (sign change OF=1 + SF=0, ergo A(-100) > B(-200)).

If A and B have different signs than the following will happen.

-100 - 100 = -200 (no sign change, SF=1, OF=0, A < B)
100 - -100 = 200  (no sign change, SF=0, OF=0, A > B)

That's all possible scenario's with OF+SF covered.
As you can see A > B only when SF <> OF and A < B only when SF = OF.

The only exception is when unsigned overflow occurs.
Let's assume we're comparing byte operands (-128..127).

126 - -126 = -4 (sign change OF=1 + SF=1, ergo A(126) < B(-126)) ***Oops.

However this will trigger the carry flag (CF) to be set, which the non-overflowing operations will not.
These incorrect results only occur when the result of the calculation does not fit inside the operand size, the solution is to keep a close eye on the carry flag and don't assume that OF and SF handle all possible cases.