How cmp assembly instruction sets flags (X86_64 GNU Linux)

Question

Here is a simple C program:

void main()
{
       unsigned char number1 = 4;
       unsigned char number2 = 5;

       if (number1 < number2)
       {
             


        

Answer 1

Operation of CMP
CMP performs a subtraction but does not store the result.
For this reason the effect on the flags is exactly the same between:

cmp eax,ecx
sub eax,ecx

As per the documentation:

Operation
temp ← SRC1 − SignExtend(SRC2);
ModifyStatusFlags; (* Modify status flags in the same manner as the SUB instruction*)
Flags Affected
The CF, OF, SF, ZF, AF, and PF flags are set according to the result.

Effects on the flags
So the following flags are affected like so:

Assume result = op1 - op2

CF - 1 if unsigned op2 > unsigned op1
OF - 1 if sign bit of OP1 != sign bit of result
SF - 1 if MSB (aka sign bit) of result = 1
ZF - 1 if Result = 0 (i.e. op1=op2)
AF - 1 if Carry in the low nibble of result
PF - 1 if Parity of Least significant byte is even

I suggest you read up on the OF and CF here: http://teaching.idallen.com/dat2343/10f/notes/040_overflow.txt

Order of the operands
I see that you like pain and are using the braindead variant of x86 assembly called ATT-syntax.
This being the case you need to take into account that

CMP %EAX, %ECX  =>  result for the flags = ECX - EAX
CMP OP2, OP1    =   flags = OP1 - OP2

Whereas Intel syntax is

CMP ECX, EAX    =>  result for the flags = ECX - EAX
CMP OP1, OP2    =>  flags = OP1 - OP2

You can instruct gdb to show you Intel syntax using: set disassembly-flavor intel