Flask slow at retrieving post data from request?

Question

I'm writing flask application that accepts POST requests with json data. I noticed huge differences in response time based on data size being passed to application. After debugging I narrowed down issue to the line where I was retrieving json data from request object. It may be important to note that testing was done on flask development server.

start = time.time()
resp = json.dumps(request.json)
return str(time.time() - start)

I timed this line and for data of 1024 (probably not coincidence) and less characters this took 0.002s and for anything over 1024 over 1 second! What is happening here? Is this the limitation of development server?

EDIT: Same thing happens for getting POST data through request.form.get('somedata') with content lenght over 1024

EDIT: I couldn't replicate issue with same example served by Apache

EDIT: I started digging into Werkzeug module and found that slowness occurs when reading response message self._read(to_read) in wsgi.py module which is passed from BaseHTTPRequestHandler. Still don't know why so slow.

---

Here's environment details: Ubuntu - 10.04 Python - 2.6.5 Flask - 0.9 Werkzeug - 0.8.3

Answer 1

The flask development server is expected to be slow. From http://flask.pocoo.org/docs/deploying/:

You can use the builtin server during development, but you should use a full deployment option for production applications. (Do not use the builtin development server in production.)

As Marcus mentioned in the comments, another WSGI server like gunicorn or tornado would be much faster and more reliable, so definitely use one of those for deployment and benchmarking.

If you're worried about working quickly during development, you can use gunicorn in development just like you would in deployment. If you're deploying to heroku, for example, you can run "foreman start" and the gunicorn server will start right up.

Answer 2

I had this problem on a line like this, it was taking about 1.0 second! It's in a flask post handler:

username=request.form.get('username')

I was testing it with curl -F:

curl -F username="x" http://127.0.0.1:5000/func

I just changed -F to -d and it got 0.0004 seconds!!!

curl -d username="x" http://127.0.0.1:5000/func

I think flask has a problem to retrieving "multipart/form-data" content-type.

Answer 3

If you use curl to send a request, Expect: 100-continue might cause the behavior. I met a similar behavior with uwsgi, flask and curl. What happens in my case are the following:

• If the request body size is larger than 1024 byte, curl posts data with Expect: 100-continue header.
• However, uwsgi can't deal with the header. So the uwsgi doesn't respond 100-continue.
• curl waits for the100-continue response until about one second time-out.

When curl sends 100-continue | Georg's Log was useful for me to know the curl behavior.