Display the result on the webpage as soon as the data is available at server

Question

I am writing a cgi page in Python. Let\'s say a client sends request to my cgi page. My cgi page does the calculation and as soon as it has the first output, it sends back t

Answer 1

Sure.

There's traditional server-driven approach, where the script runs just once, but takes a long time to complete, spitting out bits of page as it goes:

import sys, time

sys.stdout.write('Content-Type: text/html;charset=utf-8\r\n\r\n')

print ''
for i in range(10):
    print '
%i
'%i
    sys.stdout.flush()
    time.sleep(1)

When writing an app to WSGI, this is done by having the application return an iterable which outputs each block it wants sent separately one at a time. I'd really recommend writing to WSGI; you can deploy it through CGI now, but in the future when your app needs better performance you can deploy it through a faster server/interface without having to rewrite.

WSGI-over-CGI example:

import time, wsgiref.handlers

class MyApplication(object):
    def __call__(self, environ, start_response):
        start_response('200 OK', [('Content-Type', 'text/html;charset=utf-8')])
        return self.page()

    def page(self):
        yield ''
        for i in range(10):
            yield '
%i
'%i
            time.sleep(1)

application= MyApplication()
if __name__=='__main__':
    wsgiref.handlers.CGIHandler().run(application)

Note that your web server may foil this approach (for CGI or WSGI) by adding buffering of its own. This typically happens if you're using output-transforming filters like mod_deflate to automatically compress webapp output. You'll need to turn compression off for partial-response-generating scripts.

This limits you to rendering the page bit-by-bit as new data comes in. You can make it prettier by having the client-side take care of altering the page as new data comes in, eg.:

def page(self):
    yield (
        '
-
'
        ''
    )
    for i in range(10):
        yield ''%i
        time.sleep(1)

This relies on client-side scripting so it might be a good idea to include backup non-script-based final output at the end.

All the while doing this, the page will appear to be still loading. If you don't want that, then you'd need to split the script into a first request that just spits out the static content, including client-side script that checks back with the server using either one XMLHttpRequest that it polls for new data through, or, for the really long-running cases, many XMLHttpRequests each of which returns the status and any new data. This approach is much more complicated as it means you have to run your work process as a background daemon process apart from the web server, and pass data between the daemon and the front-end CGI/WSGI request using eg. pipes or a database.