The orbited 0.4.0 branch has been under heavy development for the past month and we are finally starting to near an alpha release. While this release won’t be immediately useful to Comet developers, it can give you a good idea of what will be in the final Orbited 0.4.0 release. There’s news on a new website, the ORBIT protocol, the STOMP Protocol, layered communication, CSP, Revolved, deployability, and the goals for the 0.4 branch.

New Website

Das-q and maddiin (Christian and Martin czura) from the #orbited IRC channel on freenode have been hard at work on a new website for Orbited written in DJango. The website features blog integration, a bug tracker, integrated documentation, and an IRC announcement bot which publishes to the channel whenever any website content changes. We hope to use this website as a rallying point towards explaining every facet of the 0.4.0 branch and documenting its usage for all cases.

ORBIT Protocol (OP)

The 0.1-0.3 releases all use the current description of the OP protocol which is very minimalistic. In the 0.4 branch we’ve re-written the protocol. We have an in-progress attempt to document it in the svn repository: OP Documentation. This protocol includes a means for calling back to the application for events such as signon, signoff, delivery success, and delivery failure. The callbacks don’t have to occur over OP; there is an option to have callbacks issued as http requests to the application. That means you can simply implement a url like /orbited/signon in your application, and specify it as the signon callback. Whenever a user connects to orbited, your application will be notified. I think the most useful callback is likely signoff, because its very hard to create independently on the client-side. Additionally, the signoff callback will delivery a list of all messages that failed to be delivered in the case of a browser timeout. One obvious use case would be a dead-simple way to persist messages that failed to be delivered but should be delivered the next time the user signs on.

STOMP Protocol

Marcus Cavanaugh brought the STOMP protocol to our attention as a possible means of replacing OP. The reasoning is that STOMP has client implementations in many languages, whereas OP only currently has implementations in Python, Ruby, PHP, and C#. The new OP protocol has no client-side implementations. The Orbited 0.4 branch will therefore include a STOMP server as a means to interact with the server int he same way an app could over OP. The reason we aren’t just completely getting rid of OP is that its easier for us to think in terms of OP for Orbited, and then map functionality on to STOMP. OP is intended as a perfect fit for the domain, by definition, so its easier to experiment with. We don’t really expect to see OP client implementations in other languages, and as such we won’t worry to much about breaking backwards compatibility for OP.

Layered Communication

I recently participated in an article series discussing Bayeux. I argued that Bayeux lacks many of the facilities you’d expect from a communication protocol in its domain. The core developers are all in agreement that Bayeux is not a good standard for many reasons, but primarily flexibility, scalability, and reliability. Therefore we are implementing three tiers of communication in Orbited. The first is the transport layer, and it is identical to the interaction of Orbited 0.3.x. The Orbited.js file is an implementation of the transport layer of Orbited. It allows for unreliable server->client delivery of messages.

CSP

The Session layer in Orbited 0.4 is CSP. Some information about the specification can be found here, though is highly subject to change even before the alpha release. This layer is used by including /_/CSP.js in the application html document, and then using the CSP object much in the same way that the Orbited object from Orbited.js was previously used. The main difference is that the CSP layer guarantees message delivery except in the face of a browser-side timeout, even over frequent reconnects due to a spotty network. Also, CSP handles pinging the browser to handle connection state. CSP and the new hooks in OP together provide are a powerful unidirectional messanging platform for modern application frameworks like Django, Pylons, Rails, or PHP. The application has only to deal with dispatching messages to the client, and it gets reliability and connection state management for free.

Revolved

Revolved is a bi-directional, publish subscribe, and peer-messaging layer similar in many regards to Bayeux. In the past we’ve stuck with the minimalist approach of only implementing unidirectional messaging for the purposes of scalability. You can find an explanation of why in my conference slides from AjaxExperience. But the primary request by users is for publish subscribe functionality. We want to be able to offer this functionality, but not at the cost of scalability. We experimented with Revolved in the 0.3.x series by creating a Revolved plugin and a corresponding javascript file. This plugin is almost wholly undocumented besides this mailing list exchange, the plugin was basically a success. Instead of including Revolved as a plugin though, we are integrating it with the core of Orbited 0.4. It is built on top of CSP so as to inherit all of the reliability guarantees. The back-end hasn’t been fully fleshed out though. The 0.3.x series included a simple replication mechanism for running multiple Revolved nodes. We don’t want to include this type of functionality in Orbited 0.4.0 directly because the problem of scaling publish/subscribe nodes in a general way is a very tough problem. Therefore, Revolved will include a plugin api itself for implementing various backends for publish/subscribe, such as IRC. Our ultimate goal is to provide a general method of distributing Revolved so as to provide the best possible throughput and latency for publish/subscribe while also keeping good peer messaging performance. We think Revolved in 0.4.0 will be a better bet than Bayeux, but it is still a long shot from being generally scalable to hundreds of thousands of users. I imagine it would work well for tens of thousands of concurrent users though.

Deployability

As mario mentioned, your pyevent worries are over. We have implemented a pure-python library that emulates pyevent’s network api, providing select, poll, and epoll backends. It will also check for pyevent at runtime and use that if it is installed. This means that installing orbited 0.4.0 is a matter of running “easy_install orbited” and thats it.

Addtionally, Orbited is now a fully compatible WSGI server. The implications are that it can act as the deployment server for many python frameworks, such as Django, Turbogears, and Pylons. This won’t help PHP or Rails developers, but it is a step in that general direction. I will note that this type of deployment binds performance of the Comet layer to your application layer, but isn’t necessarily an obstacle towards horizontal scalability.

Conclusion

Orbited 0.4.0 represents many changes from previous versions. We have three particular goals: 1) Deployability, 2) Integration with synchronous frameworks, 3) Better Server <–> Browser communication protocols. The alpha release will provide a good preview of these features. If you have any suggestions or comments, don’t hesitate. Now is the time to voice them. I don’t expect we’ll see a set of changes nearly this radical again. The 0.4.0 series will likely be similar to an eventual 1.0 release.

This entry was posted on Saturday, February 23rd, 2008 at 1:24 am and is filed under Releases. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.