Flask server sent events socket exception

Question

I am thinking of using SSE to push new data to the client and using Flot(javascript charting library) display "live" updates. My server runs on python Flask framework and I have figured out how to push the data to the client, but the problem occurs as soon as I leave the page:

Exception happened during processing of request from ('127.0.0.1', 38814)
Traceback (most recent call last):
  File "/usr/lib/python2.7/SocketServer.py", line 582, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
    self.finish()
  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
    self.wfile.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe

I understand why the error occurs - the socket is never closed due the infinite loop serving up "live" data. Question is how do I detect the page change and cleanly close the socket? Can I close the connection on the client side? How do I detect the page change then?

This is the server code skeleton, I would of course replace the text message with json containing the list of objects to display:

def event_stream():
    import time
    while True:
        time.sleep(1)
        yield "data: This is a message number X.\n\n"

@app.route('/stream')
def stream():
    return Response(event_stream(), mimetype="text/event-stream")

Answer 1

You could use either onBeforeUnload or jQuery's window.unload() to make an Ajax call to some tear down method that closes the handle. Something like:

$(window).unload(
    function() {
        $.ajax(type: 'POST',
               async: false,
               url: 'foo.com/client_teardown')
    }
}

There are some inconsistencies with how the unload()/onBeforeUnload() are handled, so you may have some more work to do in something like Chrome.

Answer 2

I have no better answer, but I don't think the above ajax request to server is good.

In flask, SSE use streaming in a Response object, if there is a way to detect the disconnect or pipe broken event in Response, that would be better to handle socket events and release other resources allocated.

Answer 3

I found a dirty (includes mokey patching), but working solution.

Because there is an exception in SocketServer.StreamRequestHandler.finish when the connection drops, we can patch it to catch the exception and handle it as we like:

import socket
import SocketServer

def patched_finish(self):
    try:
        if not self.wfile.closed:
            self.wfile.flush()
            self.wfile.close()
    except socket.error:
        # Remove this code, if you don't need access to the Request object
        if _request_ctx_stack.top is not None:
            request = _request_ctx_stack.top.request
            # More cleanup code...
    self.rfile.close()

SocketServer.StreamRequestHandler.finish = patched_finish

If you need access to the corresponding Request object, you additionally need to wrap the event stream with flask.stream_with_context, in my case:

@app.route(url)
def method(host):
    return Response(stream_with_context(event_stream()),
                    mimetype='text/event-stream')

Again, this is a very dirty solution and propably won't work if you don't use the builtin WSGI server.

Answer 4

Do not use Flask inner dev wsgi server in the prod env. Consider about using uwsgi which could handle this socket error elegantly.

Meanwhile think about use python3 which also catch this sockrt broken well.