Determine asymmetric latencies in a network

Question

Imagine you have many clustered servers, across many hosts, in a heterogeneous network environment, such that the connections between servers may have wildly varying latenci

Answer 1

You can measure asymmetric latency on link by sending different sized packets to a port that returns a fixed size packet, like send some udp packets to a port that replies with an icmp error message. The icmp error message is always the same size, but you can adjust the size of the udp packet you're sending.

see http://www.cs.columbia.edu/techreports/cucs-009-99.pdf