Home > Advocacy > libev and libevent

libev and libevent

The next version of gevent (1.0) will use libev for the event loop rather than libevent or libevent2. In this post I’ll explain the reasons behind the switch.

But first let’s revisit the original reasons for choosing libevent1.4 over libev:

  •  I considered it more popular and more proven
  •  It’s packaged in Debian (apt-get install libevent libevent-dev)
  •  It has dns, http and bufferevents

However, I no longer consider any of this reasons important.

First of all, libev has been used in lots of projects since then and underwent lots of scrutiny.

Second, most distributions do not package the latest versions of both libraries so the argument that gevent with libevent is easier to install becomes moot. Actually,  libev is really easy to embed and that’s what we’re going to do for the next version: put all the dependencies in the release tarball. Hopefully this will reduce the number of build problems people get. This will also remove the number of possible variations of gevent because of different library versions used. There still will be an option to dynamically link against the dependencies, it just won’t be the default.

The third argument that libevent has built in support for dns and http still holds. And it is part of what makes gevent cool, but now it’s time to move on.

For example, I encountered the following bugs in Libevent’s dns/http implementation:

  • [dns] random failures and timeouts
  • [dns] forking breaks it completely (issue #2)
  • [dns] /etc/resolv.conf is not handled correctly
  • [http] data corruption when pipelining (thus pipeline is disabled by default)
  • [http] no streaming support
  • [http] random failures in client

I have most experience with libevent1.4 but brief tests and gevent’s test suite show that neither libevent-dns nor libevent-http became solid with the release of libevent2.0.10-stable.

Every time I try to fix one of these issues I find the implementation of libevent-dns/libevent-http unreasonably complex for what they do.

Some of the complexity seem to spur from the fact that libevent2 tries to be thread-safe and this affects the control flow: instead of executing a callback directly, libevent2 authors have to use deferred queues of callbacks to avoid deadlock. So thread-safety of libevent2 has cost in complexity but no benefits for Python programs.

Because libev is just an event loop, we need to replace libevent-dns and libevent-http with something. There’s a port of libevent-dns that works with libev, but there’s no particular reason to use it (other than ease of integration since we already have wrappers for it). I’ve done some experiments with c-ares library (used by curl, among others) and really like the results.

The c-ares library is about twice as large as libevent-dns but I find the code to be much easier to follow. Using c-ares + libev fixed all [dns] issues mentioned above. In addition, gevent.socket now also provides “green” versions of getnameinfo and gethostbyaddr.

Neither c-ares nor libev are thread-safe which is good there.

Now, let’s keep aside libevent’s addons and look at the event loops provided by libev and libevent.

1. Design

In libevent there’s a big multi-functional event class, which is used for

  • I/O readiness notifications
  • timeouts
  • signals (by passing signalnum instead of fd and setting the EV_SIGNAL flag)

The signal handling is an obvious afterthought here, but even I/O notifications and timeouts should be separated, as quite often you only need one of them. Note, that libevent does provides macros (“evtimer_” and “evsignal_”) that imitate “orthogonal” interface but I’m talking about the implementation here.

In libev all of the above provided by different, clearly separated “classes” called “watchers”.

There’s a minor difference is that libevent’s events are one-shot by default (unless used with EV_PERSIST flag) and libev’s watchers are “persistent” in libevent parlance – they must be explicitly cleared.

There are also watchers without analogs in libevent:

a) ev_prepare and ev_check watchers

Those unconditionally run the associated callback before and after “poll” call. This is useful for signal handling.

The way gevent works is by running the event loop forever in a separate greenlet. One downside of that is that Python cannot execute the signal handlers installed with the standard “signal” module. Of course, there’s signal handling integrated with the event loop, that both libev and libevent provide (exposed as gevent.signal) but that is different from the signal module:

  • it won’t work if some Python code is busy looping and the event loop is starving, as it gets dispatched by the event loop
  • it has different interface and thus not easily monkey patched
  • (on Windows) signal module emulates Ctrl-C signal with WINAPI and neither libev nor libevent do that.

In gevent 1.0, I use ev_prepare to check for Python signals and execute the signal handlers. The end result is that the standard “signal” module now “just works” with gevent. And gevent.signal is also there if you need it.

b) ev_child watcher

This will come handy when implementing gevent.os and gevent.subprocess modules.

c) ev_idle watcher

This executes a callback if the event loop is idle. I think it can be used to integrate other event loops into gevent’s loop (for example, a gui app). I don’t think libevent2 has a way to achieve that – embed another event loop.

I also like the names better, that is “loop” rather than “event_base”, “watcher” rather than “event”.

Marc Lehmann, the author of libev also wrote Perl wrappers for libev and a Coroutine library for Perl, which might explain why the libev API is a good fit for wrapping in a high-level language.

2. Implementation

Both libevent and libev support epoll, kqueue, poll, select, solaris event ports.

Where libevent is nicer than libev is Windows support.

On Windows, libevent accepts Windows handles instead of C runtime file descriptors. Python uses Windows handles too, so gevent happily passes socket’s fileno() to libevent. Libev uses C runtime handles, so we need to convert Windows handles into C runtime handles.

Libevent2 also supports IOCP, via  asynchronous bufferevents. However this feature is marked as experimental and neither libevent-http nor libevent-dns use it. Gevent does not wrap bufferevents, so gevent 0.13 is not benefiting from IOCP even if compiled with libevent2.

Thus, I do not expect performance downgrade on Windows when upgrading from gevent 0.13 to gevent 1.0.

Read the discussion on the mailing list that prompted to write this post.

Note, that mere switch from libevent to libev is not the only change in gevent 1.0. Rather, the whole core have been redesigned. I plan to make a blog post some time soon about this.

Thanks to Nicholas Piël for corrections.

  1. April 28, 2011 at 9:52 pm

    Kind of a shame, libevent2 is such a nice library — much better than libevent1. NickM is very receptive to patches.

  2. Howe
    April 28, 2011 at 10:04 pm

    Great article, Denis. I was looking forward this information. I still wonder however how they differ performance-wise. Any stats on this ?

    Thanks!

  3. May 7, 2011 at 5:41 am

    Excellent! I was already planning to port gevent to libev because of its broader selection of handled events and cleaner architecture, now I can skip that and move on to subprocess management.

  4. Jason Toffaletti
    May 12, 2011 at 7:39 am

    Hey Denis, I think you’re making a good choice. libev is smaller and fits more closely with the gevent feature set. Also, I’ve been using c-ares in a few projects now instead of libevent-dns and it has been very solid so far.

    Kevin, libevent2-core is well maintained and you’re right about the authors being very receptive to patches, but the dns and http libraries have both suffered a bit without any one maintaining them. For example, besides the bugs mentioned above, the http library isn’t thread safe yet.

  5. June 12, 2011 at 9:47 am

    How hard will it be to integrate libeio too so we can finally have real(as real as it gets on linux with a threadpool) async file IO?

    • June 14, 2011 at 9:32 am

      I don’t think it’s hard at all, but I only had cursory look at libeio.

  6. est
    June 14, 2011 at 8:22 am

    Does c-ares’s gethostbyaddr support specify a DNS server? It would be AWESOME if gevent could support this feature so we don’t have to use python-dns or pydns, etc.

    • June 14, 2011 at 9:29 am

      Yes, it does:

      import gevent.resolver_ares
      resolver = gevent.resolver_ares.Resolver(servers=['8.8.8.8'])
      print resolver.gethostbyname(‘gevent.org’)

      import gevent
      # gevent.get_hub().resolver = resolver # make this resolver also used by gevent.socket

  7. Ryan
    September 15, 2011 at 5:38 pm

    No mention of Node.js? Surely they influenced this decision at least a little. ;)

  8. Ted S
    September 22, 2011 at 6:50 am

    Will there be anything to replace libevent-http?

    The high-performance C-based http server was partly what drew me to gevent.

    A possible candidate: http://tinyclouds.org/libebb/

    It was made by Ryan Dahl (of node.js fame), and the http parser is based on Ragel (a la Mongrel).

    It doesn’t look that maintained, but it’s being actively used by (http://webroar.in/). They submitted a patch for an SSL issue pretty recently, so they’re maintaining it in some sense (or at least, they haven’t had many issues with it). It looks pretty simple; I could fix any issues that happen to come up.

  9. December 9, 2011 at 11:00 am

    You consider libuv – it’s a platform agnostic wrapper around libev, libeio & c-ares. From the official doc:
    “This is the new networking layer for Node. Its purpose is to abstract IOCP on windows and libev on Unix systems. We intend to eventually contain all platform differences in this library.”

    https://github.com/joyent/libuv

    There’s an existing python wrapper: https://github.com/saghul/pyuv

  10. Why Ohwaionly
    December 20, 2011 at 6:47 pm

    c-ares,. at least in the versions i looked at, has the deadly flaw of breaking when the host supports ipv6, which is why curl on debian for example is not compiled with c-ares support.

    if you like small, you could look at http://25thandclement.com/~william/projects/dns.c.html, which is certainly small, but might not be what you want for other reasons of course.

    • January 10, 2012 at 4:12 pm

      In gevent 1.0b1 you also have an option to use a threadpool-based resolver. You can enable it by setting GEVENT_RESOLVER=thread environment variable.

      It is also possible to plug in custom resolver into gevent, so somebody can build a gevent wrapper around other libs.

  11. March 3, 2012 at 3:28 pm

    what time can 1.0 release?

  12. Den
    March 4, 2012 at 10:07 am

    why not libuv?

  13. Malar
    October 25, 2012 at 6:41 pm

    it appears as if libuv combines the benefits of libevent and libev.

    http://blog.nodejs.org/2011/09/23/libuv-status-report

  14. December 13, 2012 at 6:55 pm

    Another +1 for libuv.

    Aside from all the technical reasons (that other have described elsewhere) you also get the marketing benefits: “You know that cool Javascript framework everyone is talking about, Node.js? Well, you can get the same speed from Python with gevent, because it uses that same libraries under the hood”.

  15. February 7, 2013 at 12:51 am

    Denis, can you comment on the several libuv questions?

  16. tobpe
    March 23, 2013 at 10:46 pm

    +1 för libuv

  17. pcunte
    August 18, 2013 at 1:01 am

    Looks like LibUV is going to be a standard for web/socket servers. It can manage threads if needed as well … the best of both worlds will be events in threads.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: