Announcing Orbited 0.2.0

October 26th, 2007 by Jacob Rus

The Orbited core development team is pleased to announce the release of Orbited 0.2.0.

Core Team:

  • Michael Carter
  • Jacob Rus
  • Mario Balibrera

Our Goals:

  • Support all major browsers without loading bars, clicks, or hour glasses
  • Create a standard, interoperable Orbited js library
  • Refactor transport architecture to achieve greater extensibility
  • Create a plugin system
  • Standardize logging / debugging output
  • Standardize the urls
  • Increase the separation of orbited sub modules

Orbited 0.2 is nearly a complete rewrite of most of Orbited’s internal components. It reorganizes architecture while leaving external interfaces nearly unchanged.

There are many great new features that will be documented over the course of the next few weeks via blog posts and updates to the core documentation. In particular, stay posted for information about the new plugin system. We provide a simple admin application as an example of how to create a plugin. Also stay posted for documentation on creating new transports by extending orbited’s Comet transport system.

We would like to thank the following contributors of documentation, patches, tutorials, and ideas for their time (in no particular order, and apologies to anyone we forgot to list):

  • Paul Johnston
  • Matthew Desmarais
  • Mike Zaic
  • Rob Morris
  • Alex Russel
  • Max Nickel
  • Christian Czura
  • Ciarán Walsh

Status of Orbited and AjaxExperience

October 4th, 2007 by Michael Carter

I just received word that my talk titled “Comet for highly scalable applications” has been accepted to AjaxExperience. From the look of it, AjaxExperience will be a much nicer conference than AjaxWorld, at least in how they treat speakers. I’ve had personal assistance with scheduling from the conference organizer and they are paying my airfare. I’ll be giving a similar talk to that which I gave at AjaxWorld. As I’ve mentioned before, you can find those slides here, and the speaking outline here.

Orbited is coming along nicely. I’ve gone over all the patches that were submitted in the last month. I accepted a few directly, but most I had to rewrite. The reason is that we are skipping Orbited 0.1.6 and moving directly to Orbited 0.2.0 as most of the codebase has been changed. There’s been a complete restructuring so it’ll be easier for developers to get involved in specific parts of the project without understanding everything. In particular, creating and understanding transports will be very straightforward and will not require changing the Orbited source. We also moved away from the pipe-based urls like “/location|user,session,transport” in favor of standard query strings: “/location?user=x&session=y&transport=z”. I personally liked the look of the old urls, but the end-user never sees them, so the result was that Orbited sometimes breaks ie with no tangible benefit.

Stay tuned over the next week for Orbited 0.2.0. We’ll need help porting the documentation, though only minor changes will be needed in the javascript and application code.

Good Advice

September 13th, 2007 by Michael Carter

I chatted with Rob Morris this evening via the orbited livehelp demo. He later joined us with a real irc client on freenode (you don’t actually have to use the livehelp app to talk to us. Just join the irc chat.) He had a number of questions that were particularly good, so I asked him to send me all of his thoughts / concerns about orbited in an email. Within an hour he sent me an email with a list of points, all of them very valid. The list is so great that I decided to post it here. To some extent I think you can treat the coming responses as a todo list for the Orbited development team. I’ll answer some of them in my blog, but primarily I want to make the answers more accessible directly from the orbited page.

Note that I’m not suggesting that all of these are open questions — far from it. Rather, not all of these questions have answers that are accessible without reading the mailing list archives, reading this entire blog, or talking with us directly. Without further adieu, here’s his email:



So, here are my thoughts right now. This is basically a brain dump of what my concerns are after a day of looking at and playing with Orbited.

I’m sure I’m missing obvious things, and that some of my concerns aren’t valid/relevant. Take with a large grain of salt!

Documentation Gaps

Lord knows, this is a big one, but here are 4 that jump to mind.

  1. You talk about lots of methods of connecting (iframe, iframe-domain, etc) but no where is there a list, with pros & cons.
  2. The config file is not documented that I could see. That’s kind of a must.
  3. Javascript how-tos related to Orbited would be very nice. Should be possible to build a nice JS library to do simple stuff. Until then, what is the domain of JS problems that a user would need to address to use Orbited in production?
  4. Quick overview of JSON and how you’d use it with Orbited (this is not a universal tech yet, and for those that aren’t familiar, it’s ominous and scary without an introduction).

Support Issues

  1. What browsers fail? (no javascript, no XSS support, IE trusted site issue… others?)
  2. What net connections fail? (bad/old proxies? restrictive firewalls? HTTP 1.0 only nodes in path?)
  3. Other failure cases for users?
  4. What are the server requirements? (run a dedicated orbited daemon, external port, many cheapo hosting plans don’t like this)
  5. How stable is the Orbited protocol? The apis? What’s the roadmap?
  6. How stable is Orbited itself?
  7. What kind of load can it take? Any metrics? X simultaneous connections on Y hardware?

Best Practices

Basically, I’d like to see a map of what a real production Orbited deployment would look like. At the least, you need to address the following:

  1. How best to handle authentication?
  2. Data scrubbing/security: <script>alert(’We have a problem’);</script>
  3. What’s up with the Loading… title and hourglass cursor? How do I avoid/minimize these? Leading to…
  4. Diff connection models, which is preferred? Why? What is required above and beyond the basic model?
  5. Scaling overview: what would this look like for a meebo-sized website? What would be required (broad brush strokes, discussion of issues to be addressed)?
  6. Failure modes: what happens when orbited dies/restarts? Is there a way to rescue the nodes? (ie track all chat messages in DB, resend state as needed, some sort of javascript reset handler for iframe?) What should I do as an orbited dev to minimize fallout from failures?

Other Issues

  1. You need a wiki. :-)
  2. You need info about yourselves, including contact information. Anyone looking to use an open-source app is buying into not only the tech, but the team behind the tech for support, leadership, etc. At least an email address! I had to get yours from the google group.
  3. A list of what to use this stuff for. Chat is obvious, but you can do better than “any app that needs lots of real-time communication”. Give folks a reason to get fired up. Collaborative document editing? Games? Sports scores? LiveBlogging? What are some things you envision to get folks started using the tools?
  4. The Projects link in your site nav bar is broken, goes nowhere.

AjaxWorld Slides

September 11th, 2007 by Michael Carter

As I mentioned in my previous post, Jacob and I are giving presentations at the AjaxWorld conference, September 23rd-26th. The title of my topic is “Comet for Highly Scalable Applications”. Take a look at the slides — They aren’t supposed to stand for themselves; rather, they are a speaking aid. I’ll post my speaking notes to go with them at a later date.

FdAjax Security + Accusations

September 1st, 2007 by Michael Carter

I am responding to a rather confrontational comment post by Grzegorz Daniluk in which re responds to Jacob Rus’s post Why Orbited Doesn’t Suck. Here is the post by Grzegorz:

FdAjax allows to send directly to other users a string or a number. Moreover even this option can be disabled. It is up to a developer what he will do with that string. This is completely different from what Jacob Rus claims about FdAjax here in this post.

On Refwell blog there is example chat application which uses direct user to user communication. Mr Jacob Rus, please provide a proof that you can do what you described in you blog post. Otherwise I’ll have to treat your post simply as FUD.

Grzegorz, The authors of the Orbited blog seek only to disseminate facts. We would never intentionally misinform our readers, and I resent the accusation.

I’ve taken a closer look at Grezgor’s FdAjax blog posts, and it seems that Jacob and I have both had some misconceptions about how FdAjax works. I’ve been thinking in terms of Cometd for so long that when I saw some example code from FdAjax, I misunderstood. Specifically, I looked at this code from the blog post titled FdAjax and Mini-chat:

var opt = {
    onSuccess: function(resp) {
        try { eval(resp.responseText); } catch (e) {}
        setTimeout("fdajax.send_request();", 20);
    },
    onFailure: function(req) {
        setTimeout("fdajax.send_request();", 10000);
    },
    method: 'get',
    parameters: "cmd=wait&user_id=" + fdajax.user_id +
                "&win_id=" + fdajax.win_id + "&types=chat"
};

I noticed the eval on the third line and thought it was handling javascript events sent directly from one browser to another. This is on closer inspection not the case — Jacob’s post was written after a quick survey he took of various comet servers, and rereading it neither of us caught this — and I’m sorry for any misunderstanding that resulted. I’ll look more closely at FdAjax and put together a comprehensive review when I get a chance. In the mean time, we retract any suggestion that FdAjax is inherently insecure.

In the future, please simply point out our mistake. No need to additionally impugn our character; we have no intention of misleading readers, and are happy to make corrections when we have erred.

Orbited at AjaxWorld

August 29th, 2007 by Michael Carter

In line with my past post about moving forward, we are presenting Orbited at its first conference: AjaxWorld. The conference is located in Santa Clara, CA and will be held September 23 — 26. There will be two presentations: “Comet for Highly Scalable Applications” by me, and “Comet for Everyone: Building Simple, Scalable Comet Applications with Orbited” by Jacob. If you weren’t already planning on attending this conference, definitely don’t come on account of our presentations. Unless your company wants to pay. The conference fee is upwards of $1700 — much more than I would pay for such an event.

Strangely, my presentation is listed under the track “Security and Performance.” I suppose you could look at it as a lecture on performance, but it is definitely not about security. Jacob had the good fortune of being listed under “Advanced AJAX” which sounds a bit more like Comet. It seems to me that they need a “Comet” track because other presentations on this topic are listed under such tracks as “Hot Topic”, “Enterprise Ajax”, and “RIAs in Action”. Go figure. At least they have talks on Comet.

I believe they are recording the presentations and possibly even simulcasting both the slides and video feed. Don’t quote me on that. Regardless, we’ll post our presentation slides and speaking notes here on the blog, after the conference.

WordPress migration

August 27th, 2007 by Michael Carter

Please bear with us. We are attempting to migrate the blog to Wordpress, but the script I wrote isn’t working quite as expected. Hopefully in a day or two the blog will be back to normal.

Why the migration? We have a whole slew of announcements to make and we wanted a nice blog for once.

Juggernaut is a Bad Idea

August 27th, 2007 by Michael Carter

I guess I’m a year out of date, but I’m just getting to reading various blogs about comet servers besides orbited. I just finished reading Dion Almaer’s post, Juggernaut: Comet for Rails? and I have a few bones to pick.

Some quick background: Juggernaut uses flash sockets to maintain communication between the browser and the push server. Comet has generally been described as an interaction model of bi-directional asynchronous communication… basically adding http-push to the traditional http-pull methods we all use. Comet has been more specifically defined in terms of long-polling and http streaming, methods that both use javascript and http to accomplish the goal.

Towards the end of the article, Dion lists the “advantages/disadvantages of using a flash socket over other methods,” but what he really does is give a factually incorrect listing of reasons he thinks flash is better than anything else without listing a single disadvantage. I’m not sure if this is something Dion wrote, or something Alex McGraw, the author of Juggernaut, wrote. The exact same list appears on the Juggernaut homepage. I’ll assume for the time being that Dion originated this list.

Dion says, in regards to flash socket’s supriority, “It’s much less of a hack.” Yes, many implementations of comet are considered to be “hacks,” but what about Flash? We are basically sticking a flash runtime, an alien piece of software, directly into a browser and then providing crude interfaces for the two to communicate. I have had no end of trouble with memory leaks because of faulty interaction between the flash runtime and Firefox. Additionally, Flash is really a duplication of many features already present in http/browsers. Instead of building on these features, Flash circumvents them and then is crudely hacked wholesale on top of browsers. Perhaps we need sockets and vector graphics, but that should be defined by web standards rather than adobe. But lets assume that javascript comet is a hack for a moment, and Flash isn’t. Whats the problem? With good apis the end-developer doesn’t really care what happens behind the scenes, only that it works. Its a hack that only a few people have to deal with once, and then our problem is solved forever. And those people are already dealing with it… just look at orbited and cometd.

Dion says, “It doesn’t crash your browser (Comet can do this after a while) ” I can only assume he is referring to what happens with novice javsacript programmers keep creating additional XHR objects when trying to implement long-polling, often resulting in memory over-usage and the occasional browser crash. This has nothing to do with the technology though. Its like blaming c because some programmers forget to deallocate memory. Furthermore, HTTP streaming, another common implementation of comet, doesn’t suffer from this problem at all. I have never seen nor heard of HTTP streaming crashing a browser.

Dion says, “95% of browsers support it (flash 6).” But many processors do not, such as AMD64 and ARM. So that means that this technique fails for all iphones. Iphone users, I imagine, will soon be a respectable chunk of users who would be using fancy web applications in the first place.

Dion says, “It’s much easier to implement” This is true… assuming you drop all of http, which is what makes comet hard to implement in the first palce. If you create your own, non-standard protocol for communication between the browser and push server, then its easier to implement. But, of course, your solution only works with itself. What if you want to swap Juggernaut out for cometd? You can’t hold on to any of your browser code, it all has to go. Wheras, if you were using Bayeux (via cometd or an alternative), then you could swap out the backend at will and the browser code need not be changed in the slightest.

Dion says, “It can use a different port - unlike comet - so you don’t need any custom dispatch servlets for forwarding messages through rails to the push server - it can connect directly.” This is factually incorrect. You can use any port with comet. Orbited includes an enlightening example: a web-based irc chat that uses comet. It has static content served from port 80, and has comet served from port 8000. This works, no problem. Don’t take my word for it, you can even find a running instance of the example.

Additional Disadvantages

  1. Juggernaut internalizes its publish/subscribe architecture much in the same way that fdAjax or Cometd do. Refer to my sixth point in my previous post, The Failure of Cometd

  2. Flash isn’t universal. I mentioned this in earlier refutation, and it is important. There are many users who can use comet but cannot use flash.

  3. Flash sockets don’t use http. This is much more dangerous than the obvious lack of standards… it will fail for many “enterprise” firewalls that don’t just blindly look at port numbers. They examine packets and block traffic that doesn’t appear to be http. Users will visit the page and it just won’t work and there will be no explanation — not even an error in the javascript console. We had our network revolution and http on port 80 is all that survived, best to remember that.

  4. In soviet Russia, you own flash.

Future Direction

August 25th, 2007 by Michael Carter

It seems like I just started Orbited yesterday, though in reality I started prototyping it about a year ago. A lot has happened since then and when I step back and look at the website and the project as a whole, I’m very pleased.

I’ve been sticking with low version numbers because I only want adventurous developers who are interested in pushing the boundaries of the web, but don’t mind tracking down a bug or two. In my mind its a small price to pay. Of course, 95% of developers aren’t interested in the least — They’ll adopt once the technology works flawlessly. At this point, I’m pleased to announce that We’ll soon be switching to 0.2.0 development in the repository. If there’s any new functionality you want, now’s the time to say so.

Before 0.2.0 work commences, I want to catch every type of error possible and provide a meaningful report in the error log. The Catch-all functions fine, but its not nice to see cryptic tracebacks all the time. So hopefully over the next couple weeks you’ll know exactly what goes wrong when something breaks.

I also think that Orbited is stable enough for real use. I’m not going to quite call it production ready, but I personally am confident enough to use it in production. The big problem now is opening Orbited up to a wider array of communities. At the moment we only have a functioning python and ruby Orbited client. We need to actively recruit developers from the following languages:

  • PHP
  • Perl
  • Java
  • C#/.net

If we have simple chat tutorials for all of these major languages, then we’d be ready to start getting some coverage on ajaxian and other similar sites. There’s only so much the core development team can do, particularly because our skill set is slanted towards C, python, and Java. There are so many application developers out there who use PHP that we cannot ignore them and consider ourselves a serious contender in the space of comet servers.

But Orbited itself is ready for the next set of developers, the slightly less adventurous kind. To that end, we are going to start publicizing Orbited on various developer mailing lists and irc channels. I recently submitted an abstract to the AjaxWorld Conference, as did Jacob. If accepted, I will talk on the share-nothing architecture of Orbited, and the difficulties of scaling pubsub Comet architectures. Jacob will present on Orbited from a more practical standpoint — how to use it to easily create real-time applications.

This is an exciting time, and it’s only going to get better. Twenty years from now all of these problems will be gone — boring old news. But right now we are poised to invent solutions and really push the envelope. That’s exciting.

Announcing Orbited 0.1.5

August 25th, 2007 by Michael Carter

Announcing Orbited 0.1.5

  • New website: www.orbited.org (thanks to Jacob Rus!)
  • Live Demo! Get help from out irc channel… in your web browser!
    • www.orbited.org/livehelp (thanks to Jacob Rus!)
  • Major stability fixes
    • No longer crashes with malformed input
    • Catchall error reporting (includes line number)
    • Proxy fix
      • somehow proxy lost HTTP/1.0 support (missing content-length header case)
      • Caused it to not work with Pylons/Paste. now it does
  • New Pylons Demo.. tutorial soon to come (thanks to Matthew/desmaj!)