Track a packet as it goes through the kernel (linux)

Question

I have two machines which are set up to use Ip-Security and machine A (lets call them A and B) has a socket which is bound to a particular UDP port on the local machine and it p

Answer 1

Please refer to the project named SystemTap. It allows you to insert user-friendly scripts hooking into any kernel code, without recompiling the kernel. For example:

probe function("ip_rcv").call {
    printf("%d:   ->ip_rcv()\n", gettimeofday_ms()) 
}

It will emit a kernel print for every received packet in the network layer. Of course, you would need to read the sources to follow from there deeper into the network stack.

SystemTap is very capable and quite documented about the various hooks that can be inserted.