Home > Example > Simpler long polling with Django and gevent

Simpler long polling with Django and gevent

Recently released Tornado web server includes an example chat application. This post describes a modification of that example that runs on Django and gevent wsgi server. The modified version achieves the same goal while staying within a familiar web framework Django and using simpler concurrency model.

It implements a simple web chat room with instant notifications and does so by using Ajax with long polling. A dynamic web application of this kind is thought of as a better fit for an asynchronous framework like Tornado or Twisted, than for a traditional thread-pool and/or process-pool based approach like apache+mod_wsgi. This is because each user participating in the chat (or simply not closing their browser window) maintains an open connection with the server for message updates and the amount of memory an open connection takes on server is significantly different depending on the server setup: a few KB for async versus a few MB for thread/process.

However, using an asynchronous framework requires twisting your code somewhat: because it’s all running in the same thread, you cannot simply wait in a view handler until a new message is available, then construct a response and return it to the framework. Instead, you usually return a callback that will be called when a new message is posted to generate the response and thus complete the long polling request. Not a huge obstacle but it does obscure the code flow.

If only we had more light-weight units of execution than threads and processes, implementing ajax apps like this chat would be a lot simpler. Turns out we do, and there are options: Stackless Python and greenlet. The latter is an extension module that runs on the stock Python and that’s what gevent currently supports.

The wsgi server bundled with gevent creates a new greenlet for each incoming connection making it’s possible in a request handler to sleep, wait for event and even access network without blocking anyone. Greenlets are cheap, memory-wise, so it’s about as scalable as callback- or Deferred- based solution. The logic, however, becomes much more transparent:

  • When a new message is posted, set the event.
  • When a client requests the updates (and it already has the latest message), wait for the event.

The code: views.py

  1. antiplex
    March 20, 2013 at 12:08 pm
  2. April 8, 2014 at 7:15 pm
  1. October 7, 2010 at 12:53 am
  2. January 6, 2012 at 9:26 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: