Is long polling possible in Google App Engine?

Question

I need to make application that needs to poll server often, but GAE has limitations on requests, so making a lot of requests could be very costly. Is it possible to use long

Answer 1

We've tried implementing a Comet-like long-polling solution on App Engine, with mixed results.

def wait_for_update(request, blob):
    """
    Wait for blob update, if wait option specified in query string.
    Otherwise, return 304 Not Modified.
    """
    wait = request.GET.get('wait', '')
    if not wait.isdigit():
        return blob
    start = time.time()
    deadline = start + int(wait)
    original_sha1 = blob.sha1
    try:
        while time.time() < deadline:
            # Sleep one or two seconds.
            elapsed = time.time() - start
            time.sleep(1 if elapsed < 7 else 2)
            # Try to read updated blob from memcache.
            logging.info("Checking memcache for blob update after %.1fs",
                         elapsed)
            blob = Blob.cache_get_by_key_name(request.key_name)
            # Detect changes.
            if blob is None or blob.sha1 != original_sha1:
                break
    except DeadlineExceededError:
        logging.info("Caught DeadlineExceededError after %.1fs",
                     time.time() - start)
    return blob

The problem I'm seeing is that requests following a long-polling one, are getting serialize (synchronized) behind the long-polling request. I can look at a trace in Chrome and see a timeline like this:

1. Request 1 sent. GET (un-modified) blob (wait until changed).
2. Request 2 sent. Modify the blob.
3. After full time-out, Request 1 returns (data unmodified).
4. Request 2 gets processed on server, and returns success.

I've used wireshark and Chrome/timeline to confirm that I AM sending the modification request to the server on a distinct TCP connection from the long-polling one. So this snychronization must be happing on the App Engine production server. Google doesn't document this detail of the server behavior, as far as I know.

I think waiting for the channel API is the best hope we have of getting good real-time behavior from App Engine.