Python: SocketServer closes TCP connection unexpectedly

Question

I would like to implement a TCP/IP network client application that sends requests to a Python SocketServer and expects responses in return. I have started out with the offic

Answer 1

The problem here is that in a TCPHandler, a "request" is actually a complete connection, from beginning to end.* Your handler gets called on accept, and when you return from it, the socket gets closed.

If you want to build a request-response-protocol handler on top of that, which processes multiple protocol-level requests on a single socket-level request, you have to do that yourself (or use a higher-level framework). (Subclasses like BaseHTTPServer demonstrate how to do this.)

For example, you can just use a loop within your handle function. Of course you probably want to add an exception handler here and/or deal with EOF from rfile (note self.rfile.readline() will return '' for EOF, but '\n' for a blank line, so you have to check it before calling strip unless you want a blank line to mean "quit" in your protocol). For example:

def handle(self):
    try:
        while True:
            request  = self.rfile.readline()
            if not request:
                break
            request = request.rstrip()
            print "RX [%s]: %s" % (self.client_address[0], request)

            response = self.processRequest(request)

            print "TX [%s]: %s" % (self.client_address[0], response)
            self.wfile.write(response + '\n')
    except Exception as e:
        print "ER [%s]: %r" % (self.client_address[0], e)
    print "DC [%s]: disconnected" % (self.client_address[0])

---

This will often work with your existing client, at least over localhost on an unloaded machine, but it's not actually correct, and "often works" is rarely good enough. See TCP sockets are byte streams, not message streams for a longer discussion, but briefly, you also need to do the stuff mentioned in David Schwarz's answer: append newlines to what you write from the server (which I already did above), and have the client read line by line instead of just trying to read 1024 bytes at a time (which you can do by writing your own buffer-and-split-lines code, or just by using the makefile method, so it can use rfile.readline() just like the server side does.)

Not fixing the client won't cause the problems that answer claims, but it will cause problems like this:

Sent:     request type 01
Received: resp
Sent:     request type 02
Received: onse type 01
response typ

And you can see that in a real program that actually tried to process the responses programmatically, a response like resp or onse type 01\nresponse typ isn't going to be very useful…

---

_{* Note that SocketServer is an ancient design that nobody really loves. There's a reason Python 3 added asyncio, and that people in Python 2 usually use third-party frameworks like Twisted and gevent. They're both simpler for simple tasks, and more flexible/powerful (and a lot more efficient) for complex tasks.}