One of the most frequent problems people have with Orbited generally surfaces before they write a single line of application logic — before, in fact, they successfully start the Orbited server for the first time. This problem is pyevent. It’s a brilliant piece of software, but many of us want to build small projects or simply prototype without having to install two sets of additional headers and gcc.

To most of you, I’m sure this doesn’t sound like a great hardship. Comet developers, however, are not most of anything. We’re strange people and we often work under strange conditions. My house loses its internet connection every few minutes; other developers are subject to the irrational decrees of their admins, or the arcane limitations of their indie operating systems. People have literally decided not to use Orbited because they couldn’t compile pyevent properly. We’re modern folks, and the rule is: make it work for everyone.

So Michael Carter and I were talking about the upcoming release of shiny new 0.4, and we decided that this had gone far enough. So we wrote up a little module to import as event anywhere pyevent would normally fit, and it did its job splendidly. That is, if it detected pyevent, it would use it; otherwise, instead of blowing everything up, it used epoll, or poll, or select.

The initial version supported the most widely-used pyevent commands: ‘read’, ‘write’, ‘timeout’, ‘dispatch’, ‘loop’, and ‘abort’, as well as an optional ‘initialize’ for specifying the preferred order of methods to try. This is useful because the best method can vary from job to job. Select, for instance, is pure python and performs as well as epoll for limited levels of concurrency, but will completely explode if the hard-coded connection limit (FD_SETSIZE) of 1024 is exceeded.

This interface was enough for a pyevent-like event notification system to do its job, but at this point rel (Registered Event Listener) was far from a perfect, drop-in substitute for pyevent, whose extended api has several features that I’d never really used.

event.sys is just sys but nevertheless could conceivably be called by some well-meaning coder.

event.init builds/clears the event queue. It also cancels system signal callbacks, but does not reset the default callbacks. This is not clean behavior, and rel corrects it.

event.signal is pyevent’s “simplified event interface.” It is sort of a combination of event.timeout and python’s built-in signal module.

event.event is a verbose, general interface for read, write, signal and timeout events. Various apps, including Orbited 0.3, use it.

So I rewrote rel and tested it against pyevent’s test.py, and at first it seemed to do its job, performing identically to pyevent. Then I took a closer look at pyevent’s test code, and discovered that one of the test methods was disabled (there was no call to event.dispatch, so the code was never executed). I added that line, and immediately got an error — still, mind you, using pyevent. Here’s what the code looked like:

def __signal2_cb(sig):
    ...
...
event.timeout(2, __signal2_cb)

For those of you unfamiliar with pyevent, event.timeout typically looks like this: “event.timeout(seconds to wait, callback function, arguments to pass to the callback function)”. In the code above, there were no arguments specified, so it blew up when it tried to call __signal2_cb with no arguments. I changed “def __signal2_cb(sig)” to “def __signal2_cb(sig=None)” to get past that, which made everything run, with one catch: with this test method enabled, pyevent ran about 5 seconds slower than the other methods. Very strange.

So I set about isolating the anomaly, and put together test2.py (in this egg), which demonstrates the odd behavior. Surely, a kindly reader will have some idea what’s going on. Anyway, I moved on.

The finishing touch (for now): Michael wrote a magical python path override function that essentially replaces sys.modules['event'] with rel. This means that to rel-ize any pyevent app, all you need to do is add

import rel
rel.override()

to the top of your start script. Magic! This will use the default method ordering (pyevent, epoll, poll, select) unless you specify an order of your own with rel.initialize([list of methods]). Either way, rel will emulate the pyevent api with the first method in the list that works.

(Note: To use pyevent, get pyevent + python development headers, libevent headers, gcc. For epoll, install python-epoll. Without these, rel will fall back to poll, which comes standard on most Unix systems, or as a last resort, select, which works pretty much anywhere.)

Anyway, Michael’s magic override script enabled me to test rel with three pyevent-driven servers - Orbited 0.3.1, FAPWS 0.7, and apricot 0.2 - in ipython without changing a line of code. It works without a hitch for these, as well as for the pyevent-based HTTP/1.1 client Ludo.

I’m very happy about this. It means that we can all keep developing for pyevent, and with the mere inclusion of two lines of safety net, deploy our software anywhere. Anyway, give it a try, see what you think, let me know if you can solve that test2.py mystery. And let me know if you think of any other methods that rel should support. (Epoll, poll, and select are currently implemented; Kqueue is on the way).

Also, Michael told me that he discussed the future of pyevent with the current maintainer, Dug Song, and some other interested parties. There is a good chance that libevent’s evhttp and possibly evdns will be exposed in the pyevent 0.4 release. Rest assured that rel will support this functionality as well, and without any external dependencies.

google code: http://code.google.com/p/registeredeventlistener/

This entry was posted on Sunday, February 17th, 2008 at 7:41 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.