Twisted: number of client connections to TCP server limited?

Question

I'm writing a chat server and encountered the following problem while unit testing it. In one of my unit tests, I connect many test clients to my server. As the number of connected users get to 511 the server stops responding without any error message. At this stage everything runs locally on a PC.

I have prepared a simple server, test client and unit test code to paste into the forum.

Any idea why the server hangs up? Any help is much appreciated

This code is basicly from the twisted simple chat tutorial. Simple server:

from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor

class Chat(LineReceiver):

    def __init__(self, users, userNum):
        self.users = users
        self.userNum = userNum

    def connectionMade(self):
        print "Connected to user %d" % (self.userNum)
        self.users[self.userNum] = self

    def connectionLost(self, reason):
        print "Connection to user %d lost" % (self.userNum)
        if self.users.has_key(self.userNum):
            del self.users[self.userNum]

    def lineReceived(self, line):
        for user in self.users:
            if user == self.userNum:
                continue
            self.users[user].sendLine("%d - %s" % (self.userNum, line))

class ChatFactory(Factory):

    def __init__(self):
        self.users = {} # maps user names to Chat instances
        self.nUsers = 0

    def buildProtocol(self, addr):
        self.nUsers += 1
        return Chat(self.users, self.nUsers)

    def clientConnectionFailed(self, connector, reason): 
        print 'connection failed:', reason.getErrorMessage() 
        reactor.stop() 

    def clientConnectionLost(self, connector, reason): 
        print 'connection lost:', reason.getErrorMessage() 
        reactor.stop() 

reactor.listenTCP(8123, ChatFactory())
reactor.run()

This is my test client. This client is instantiated by the unit test several times.

import socket

HOST = "localhost"
PORT = 8123

class TestClient:
    def __init__(self):
        self.connected = False
        try:
            self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        except socket.error, msg:
            print("Socket error %s" % msg)

    def connect(self):
        try:
            self.socket.settimeout(10)
            self.socket.connect((HOST, PORT))
            self.connected = True
        except socket.error, msg:
            print("Socket error %s" % msg)
            self.connected = False

    def disconnect(self):
        self.socket.close()

    def connected(self):
        return self.connected

Finally the unit test code file:

import unittest
from TestClient import TestClient

class TestSequenceFunctions(unittest.TestCase):

    def test_manyUsers(self):
        users = []

        number_of_users = 1000

        for u in range(number_of_users):
            # create client
            users.append(TestClient())
            # connect client to server
            users[-1].connect()
            # check connected state
            self.assertTrue(users[-1].connected, "User %d is not connected" % (u))

        # close connection of all users
        for user in users:
            user.disconnect()

if __name__ == '__main__':
    unittest.main()

Answer 1

The number 511 is rather suspicious. It's close enough to a power of two that my initial hunch is an arbitrarily imposed limit or a bug.

Since you mentioned you're on Windows, I think I can say it's an arbitrarily imposed limit with a bit of confidence. The number of sockets supported by select(2) is limited on all platforms, but the limit is even lower than usual on Windows. By default, it's actually 64. However, Python ups this limit to 512 (the limit isn't mutable like that on most platforms, but it is on Windows - at C compile time).

Failing after 511 users sounds like just what would happen if this were the limit on your system - the 512th socket is the one listening for connections.

Most limits like this are hard to find in a general way. Usually you have to dig into what low-level API or system calls are being used and then look up their documentation, or ask and hope someone else who has (unfortunately) memorized all the various limits helps you out. :)

You can avoid this limit by using the IOCP-based reactor on Windows. It's very easy to switch to. Just insert these lines before the first lines in your server:

from twisted.internet import iocpreactor
iocpreactor.install()

Everything else stays the same (and, in particular, your existing reactor import stays the same, and you keep using reactor, you don't switch to using iocpreactor anywhere else in your program).

You can read more about reactor selection in Twisted's online documentation.