x86 Assembly: INC and DEC instruction and overflow flag

Question

In x86 assembly, the overflow flag is set when an add or sub operation on a signed integer overflows, and the carry flag is set when an operation o

Answer 1

try changing your test to pass in the number rather than hard code it, then have a loop that tries all 256 numbers to find the one if any that affects the flag. Or have the asm perform the loop and exit out when it hits the flag and or when it wraps around to the number it started with (start with something other than 0x00, 0x7f, 0x80, or 0xFF).

EDIT

.globl inc
inc:
    mov $33, %eax

top:
    inc %al
    jo done
    jmp top

done:
    ret

.globl dec
dec:
    mov $33, %eax

topx:
    dec %al
    jo donex
    jmp topx

donex:
    ret

Inc overflows when it goes from 0x7F to 0x80. dec overflows when it goes from 0x80 to 0x7F, I suspect the problem is in the way you are using inline assembler.