receiving UDP packets send to 127.0.0.1 when using SO_REUSEADDR

Question

I am trying to make a set of applications discover each other using UDP and broadcasting messages. The applications will periodically send out an UDP packet saying who they are and what they can do. Initially we only use to broadcast to INADDR_BROADCAST.

All applications share the same port to listen to (hence the SO_REUSEADDR). An event kernel object is attached to the socket so we get notified when we can fetch a new packet and use that in a WaitFor loop. The socket is used async.

Opening the socket:

FBroadcastSocket := socket( AF_INET, SOCK_DGRAM, IPPROTO_UDP );
if FBroadcastSocket = INVALID_SOCKET then Exit;
i := 1;
setsockopt( FBroadcastSocket, SOL_SOCKET, SO_REUSEADDR, Pointer( @i ), sizeof( i ) );
i := 1;
setsockopt( FBroadcastSocket, SOL_SOCKET, SO_BROADCAST, Pointer( @i ), sizeof( i ) );
System.FillChar( A, sizeof( A ), 0 );
A.sin_family      := AF_INET;
A.sin_port        := htons( FBroadcastPort );
A.sin_addr.S_addr := INADDR_ANY;
if bind( FBroadcastSocket, A, sizeof( A ) ) = SOCKET_ERROR then begin
    CloseBroadcastSocket();
    Exit;
end;
WSAEventSelect( FBroadcastSocket, FBroadcastEvent, FD_READ );

Sending out data to a specified list of addresses:

for i := 0 to High( FBroadcastAddr ) do begin
    if sendto( FBroadcastSocket, FBroadcastData[ 0 ], Length( FBroadcastData ), 0, FBroadcastAddr[ i ], sizeof( FBroadcastAddr[ i ] ) ) < 0 then begin
        TLogging.Error( C_S505, [ GetWSAError() ] );
    end;
end;

Receiving packets:

procedure TSocketHandler.DoRecieveBroadcast();
var
    RemoteAddr:    TSockAddrIn;
    i, N:          Integer;
    NetworkEvents: WSANETWORKEVENTS;
    Buffer:        TByteDynArray;
begin
    // Sanity check.
    FillChar( NetworkEvents, sizeof( NetworkEvents ), 0 );
    WSAEnumNetworkEvents( FBroadcastSocket, 0, @NetworkEvents );
    if NetworkEvents.ErrorCode[ FD_READ_BIT ] <> 0 then Exit;

    // Recieve the broadcast buffer
    i := sizeof( RemoteAddr );
    SetLength( Buffer, MaxUDPBufferSize );
    N := recvfrom( FBroadcastSocket, Buffer[ 0 ], Length( Buffer ), 0, RemoteAddr, i );
    if N <= 0 then begin
        N := WSAGetLastError();
        if N = WSAEWOULDBLOCK then Exit;
        if N = WSAEINTR then Exit;
        TLogging.Error( C_S504, [ GetWSAError() ] );
        Exit;
    end;

    DoProcessBroadcastBuffer( Buffer, N, inet_ntoa( RemoteAddr.sin_addr ) );
end;

When we send out the broadcast data using INADDR_BROADCAST, the local broadcast address (192.168.1.255) or the local IP address everything works fine. The moment we use 127.0.0.1 to "broadcast" to, reception is sporadic but generally does not work.

Does anyone have a clue how to solve this (the address list is changeable)? If all else fails I will lookup all the local IP addresses and just replace 127.0.0.1 with that but that leaves problems when IP addresses change.

Update: When you first start App1, App1 will receive packets. Next you start App2. Now App1 will still receive packets but App2 won't. If you stop App1, App2 will start to receive packets. If you start App3, App2 will receive it's packets but App3 won't.

Thus: only one application will receive the packets when 127.0.0.1 is used.

Also setting IPPROTO_IP, IP_MULTICAST_LOOP to one with setsocketopt does not change anything.

Answer 1

It seems like what you want is to hard code the broadcast address without worrying about what actual IP addresses are in use by the machine. Your first problem is that since this is a new application you should be using multicast instead of broadcast. Then you can use a special multicast address which can be the same everywhere, regardless of what address the machine actually has. I assume all these apps are running on the same machine.

Here's an example program written in Perl. You should be able to adapt the code fairly easily. Start a few copies in different windows to see how it works. Basically it forks a sender and receiver and sends the datetime and pid of the sender. You'll need to install the Socket::Multicast package from CPAN in order to run it.

#!/usr/bin/perl -w
# This example is a reimplementation of Steven's sendrecv Multicast example from UNP
use strict;
use diagnostics;
use Socket;
use Socket::Multicast qw(:all); # Has to be installed from CPAN

my $sendSock;

socket($sendSock, PF_INET, SOCK_DGRAM, getprotobyname('udp'))   
    || die "socket: $!";
setsockopt($sendSock, SOL_SOCKET, SO_REUSEADDR, pack("l", 1))   
    || die "setsockopt: $!";
# create socket with ephemeral port for sending $port = 0
bind($sendSock, sockaddr_in(0, INADDR_ANY))  || die "bind: $!"; 

# create socket for multicast receive
my $recvSock;
my $mcastIP = '239.255.1.2';
my $mcastPort = 9999;

socket($recvSock, PF_INET, SOCK_DGRAM, getprotobyname('udp'))   
   || die "socket: $!";
setsockopt($recvSock, SOL_SOCKET, SO_REUSEADDR, pack("l", 1))   
   || die "setsockopt: $!";

# join to specific port and IPV4 address to select mcast interface
my $imr_multicast = inet_aton($mcastIP);
my $imr_interface = INADDR_ANY;
my $ip_mreq = pack_ip_mreq($imr_multicast, $imr_interface);
my $ip = getprotobyname( 'ip' );

setsockopt($recvSock, $ip, IP_ADD_MEMBERSHIP, $ip_mreq)     
    || die "setsockopt IP_ADD_MEMBERSHIP failed: $!";

# bind to multicast address to prevent reception of unicast packets on this port
bind($recvSock, sockaddr_in($mcastPort, inet_aton($mcastIP)))  || die "bind: $!"; 

# disable multicast loopback so I don't get my own packets
# only do this if you're running instances on seperate machines otherwise you won't
# get any packets
# setsockopt( $recvSock, $ip, IP_MULTICAST_LOOP, pack( 'C', $loop ) ) 
  #  || die( "setsockopt IP_MULTICAST_LOOP failed: $!" );

# fork sender and receiver
my $pid = fork();
if ( $pid == 0) {
    mrecv();
} else {
    msend();
}    

sub msend {
    close($recvSock);
    while (1) {
        my $datastring = `date`; chomp($datastring);
        $datastring = "$datastring :: $pid\n";
        my $bytes = send($sendSock, $datastring, 0, 
                         sockaddr_in($mcastPort, inet_aton($mcastIP)));
        if (!defined($bytes)) { 
            print("$!\n"); 
        } else { 
            print("sent $bytes bytes\n"); 
        }
        sleep(2);
    }
}

# just loop forever listening for packets
sub mrecv {
    close($sendSock);
    while (1) {
        my $datastring = '';
        my $hispaddr = recv($recvSock, $datastring, 64, 0); # blocking recv
        if (!defined($hispaddr)) {
            print("recv failed: $!\n");
            next;
        }
        print "$datastring";
    }
}