Thanks to Galfy Pundee, Alexander Boudkar and Alexey Borzenkov.

Read the (relatively short) changelog entry, download it from PyPI.

But first let’s revisit the original reasons for choosing libevent1.4 over libev:

•  I considered it more popular and more proven
•  It’s packaged in Debian (apt-get install libevent libevent-dev)
•  It has dns, http and bufferevents

However, I no longer consider any of this reasons important.

First of all, libev has been used in lots of projects since then and underwent lots of scrutiny.

Second, most distributions do not package the latest versions of both libraries so the argument that gevent with libevent is easier to install becomes moot. Actually,  libev is really easy to embed and that’s what we’re going to do for the next version: put all the dependencies in the release tarball. Hopefully this will reduce the number of build problems people get. This will also remove the number of possible variations of gevent because of different library versions used. There still will be an option to dynamically link against the dependencies, it just won’t be the default.

The third argument that libevent has built in support for dns and http still holds. And it is part of what makes gevent cool, but now it’s time to move on.

For example, I encountered the following bugs in Libevent’s dns/http implementation:

• [dns] random failures and timeouts
• [dns] forking breaks it completely (issue #2)
• [dns] /etc/resolv.conf is not handled correctly
• [http] data corruption when pipelining (thus pipeline is disabled by default)
• [http] no streaming support
• [http] random failures in client

I have most experience with libevent1.4 but brief tests and gevent’s test suite show that neither libevent-dns nor libevent-http became solid with the release of libevent2.0.10-stable.

Every time I try to fix one of these issues I find the implementation of libevent-dns/libevent-http unreasonably complex for what they do.

Some of the complexity seem to spur from the fact that libevent2 tries to be thread-safe and this affects the control flow: instead of executing a callback directly, libevent2 authors have to use deferred queues of callbacks to avoid deadlock. So thread-safety of libevent2 has cost in complexity but no benefits for Python programs.

Because libev is just an event loop, we need to replace libevent-dns and libevent-http with something. There’s a port of libevent-dns that works with libev, but there’s no particular reason to use it (other than ease of integration since we already have wrappers for it). I’ve done some experiments with c-ares library (used by curl, among others) and really like the results.

The c-ares library is about twice as large as libevent-dns but I find the code to be much easier to follow. Using c-ares + libev fixed all [dns] issues mentioned above. In addition, gevent.socket now also provides “green” versions of getnameinfo and gethostbyaddr.

Neither c-ares nor libev are thread-safe which is good there.

Now, let’s keep aside libevent’s addons and look at the event loops provided by libev and libevent.

1. Design

In libevent there’s a big multi-functional event class, which is used for

• I/O readiness notifications
• timeouts
• signals (by passing signalnum instead of fd and setting the EV_SIGNAL flag)

The signal handling is an obvious afterthought here, but even I/O notifications and timeouts should be separated, as quite often you only need one of them. Note, that libevent does provides macros (“evtimer_” and “evsignal_”) that imitate “orthogonal” interface but I’m talking about the implementation here.

In libev all of the above provided by different, clearly separated “classes” called “watchers”.

• io watcher
• timer watcher
• signal watcher

There’s a minor difference is that libevent’s events are one-shot by default (unless used with EV_PERSIST flag) and libev’s watchers are “persistent” in libevent parlance – they must be explicitly cleared.

There are also watchers without analogs in libevent:

• ev_prepare and ev_check watchers
• ev_child watcher
• ev_idle watcher

a) ev_prepare and ev_check watchers

Those unconditionally run the associated callback before and after “poll” call. This is useful for signal handling.

The way gevent works is by running the event loop forever in a separate greenlet. One downside of that is that Python cannot execute the signal handlers installed with the standard “signal” module. Of course, there’s signal handling integrated with the event loop, that both libev and libevent provide (exposed as gevent.signal) but that is different from the signal module:

• it won’t work if some Python code is busy looping and the event loop is starving, as it gets dispatched by the event loop
• it has different interface and thus not easily monkey patched
• (on Windows) signal module emulates Ctrl-C signal with WINAPI and neither libev nor libevent do that.

In gevent 1.0, I use ev_prepare to check for Python signals and execute the signal handlers. The end result is that the standard “signal” module now “just works” with gevent. And gevent.signal is also there if you need it.

b) ev_child watcher

This will come handy when implementing gevent.os and gevent.subprocess modules.

c) ev_idle watcher

This executes a callback if the event loop is idle. I think it can be used to integrate other event loops into gevent’s loop (for example, a gui app). I don’t think libevent2 has a way to achieve that – embed another event loop.

I also like the names better, that is “loop” rather than “event_base”, “watcher” rather than “event”.

Marc Lehmann, the author of libev also wrote Perl wrappers for libev and a Coroutine library for Perl, which might explain why the libev API is a good fit for wrapping in a high-level language.

2. Implementation

Both libevent and libev support epoll, kqueue, poll, select, solaris event ports.

Where libevent is nicer than libev is Windows support.

On Windows, libevent accepts Windows handles instead of C runtime file descriptors. Python uses Windows handles too, so gevent happily passes socket’s fileno() to libevent. Libev uses C runtime handles, so we need to convert Windows handles into C runtime handles.

Libevent2 also supports IOCP, via  asynchronous bufferevents. However this feature is marked as experimental and neither libevent-http nor libevent-dns use it. Gevent does not wrap bufferevents, so gevent 0.13 is not benefiting from IOCP even if compiled with libevent2.

Thus, I do not expect performance downgrade on Windows when upgrading from gevent 0.13 to gevent 1.0.

Read the discussion on the mailing list that prompted to write this post.

Note, that mere switch from libevent to libev is not the only change in gevent 1.0. Rather, the whole core have been redesigned. I plan to make a blog post some time soon about this.

Thanks to Nicholas Piël for corrections.

Read the changelog, download it from PyPI.

Thanks to Shaun Lindsay, Nick Barkas, Andreas Blixt.

PyCon 2011: An outsider’s look at co-routines.
This is an overview by Peter Portante that compares greenlet-based approaches like gevent with yield-based coroutines and threads.

This release adds a new experimental gevent.httplib module that implements httplib-like interface on top of libevent http client. It also fixes a number of issues. Read the full changelog for details, get it from PyPI.

Thanks to Tommie Gannert, Örjan Persson, Alexey Borzenkov. Ralf Schmitt, Nicholas Piël, Elizabeth Jennifer Myers, Ned Rockson, Jon Aslund.

Among other things, the SFC will accept the tax-deductible donations on behalf of Gevent that will go the project’s fund.

Thanks to Ralf Schmitt who worked with me on this release.

Thanks to the following people who provided patches: David Hain, Teh
Ekik, Dmitry Chechik, Alexey Borzenkov, Antoine Pitrou, Örjan Persson.

Thanks to everyone who reported bugs and participated in the discussion.

Thanks to Leif K-Brooks (Omegle.com) and Spotify who
responded to the sponsorship call and provided funds for gevent
development!

For the past year, gevent was funded from my savings. Currently, I have to do consulting, which takes away time that could be spent on gevent development. I’d rather work on gevent and if you depend on or plan to depend on gevent you probably prefer that I work on it as well.

If you’d like to sponsor gevent, send me an email mentioning how much you’d like to give and we’ll make arrangements. For me as well as for each individual sponsor the whole thing makes more sense if there’s enough money raised to cover a year of full time or at least a year of part time. Thus, before we finalize the agreement, I’ll notify you about the total amount offered and what it covers. If there’s enough for a year, you’ll get a commitment from me not to accept projects/job offers that will interfere with my gevent work for the next year. If there’s not enough, I will tell how many weeks/months I can commit to and it’s up to you whether to proceed.

You can treat me as a contractor: I have a legal entity set up here and will provide invoices and other paperwork necessary so that you can write this off as a business expense. It also does not have to be a one-time payment for the whole year, per quarter or per month payments are perfectly acceptable too.

Funding is more than donating and is closer to commercial support. It cannot change my vision of the project, but you’ll effectively get a developer working on an important part of your software stack while paying only fraction of the cost.

Additionally, here’s what I can do for sponsors:

• priority help with the issues important to your projects
• put your logo/link on gevent.org homepage (for top sponsors)
• review your gevent-using code (actually I can do it for anyone who asks as I love to see what people are doing with gevent).

If you don’t care about the above but would like to donate to the project, I appreciate that too. Currently, there is no “Donate” button on gevent website, as many PayPal-like services don’t work with my country yet. Nonetheless, there are still ways to do it, send me an email and I’ll reply with the details.

I love working on gevent, it’s the most enjoyable work I’ve ever done and I will remain the maintainer. By funding you can help boosting gevent development and increase its chances to become de-facto standard way for writing network apps in Python.

The full list of changes is available in the changelog. The major new additions are:

• gevent.server module to help you implement TCP and SSL servers. Check out the example.
• gevent.pywsgi – an alternative implementation of WSGI server, based on gevent.server. Unlike libevent-based gevent.wsgi it supports streaming, SSL, keep-alive connections.
• greenlet’s sharing of exc_info is worked around in this version.
• gevent.local is now working properly.
• socket.sendall() function no longer wastes memory by creating substrings of data.
• fixed reference leaks in core module.

Thanks to Ralf Schmitt, Ted Suzman, Luigi Pugnetti, Randall Leeds, Daniele Varrazzo, Nicholas Piël, Örjan Persson, Uriel Katz.

I’m going to give a talk on gevent at EuroPython on Thursday 22nd July at 11.00am in Birmingham, United Kingdom.

The contents of the talk is not well defined yet but I’d like to cover the following:

• Coroutine-based approach to network programming and how it solves Python’s concurrency problems.
• Greenlets: what they do and how they do it. Why they are better than the alternatives – generators, threads.
• How gevent and eventlet use greenlet to do what they do, the basic principles they operate on.
• How sockets/events/queues implemented.
• Real-world applications using gevent; future development plans.

If you have any suggestions regarding the contents of the talk, please post a comment or email me.