I’ve been putting out a lot of propaganda about how scalable Orbited is. I think its time I explain why Orbited is scalable, and what the cost of that scalability is.

Let me first direct your attention to memcached, a brilliant distributed caching system from the livejournal guys. If you haven’t heard about it, go read and come back. The short description is that it acts like a distributed hash table. Each piece of data hashes to a specific server, so you always know where to find data, and memcached nodes don’t ever share information.

This is known as share nothing architecture. The purpose behind it is the cut down on intra-node dependance, and ultimately to allow scaling an application laterally. The classic version of this is web application sessions. If you store session data on each server node then as users access other nodes they need to contact the “home” node of that user to ask for session information. While this works, you end up with increasing overhead of each additional server. So adding the 10th node might not actually get you any performance increase because of the overhead in exchanging session state information with the other 9 nodes.

As far as Comet style communication goes, the main problem is maintaining state information about channels in the publish/subscribe paradigm. Cometd uses this pattern. Other examples include IRC and Jabber. The basic idea is that users can subscribe to channels, and publish to channels. Upon publishing an event to a channel, every member of that channel receives the message.

This is seductively trivial to implement on a single server. But what happens when you add the second server? You have to figure out some way to partition your user base. Partitioning by channel wouldn’t make much sense. Let’s say you had users subscribed to channel A connect to server A, and users subscribed to channel B connect to server B. What if all users need to connect to both channels? Then all users connect to both servers, and you’re back to square one.

Okay, then let’s partition based on users. Half the users will connect to server A, and the other half to server B. This works a bit better, but creates an additional problem. The crux of that obstacle is this: If you want to dispatch an event to channel A, to which server do you send that event? The answer, inevitably, is both servers. We’ve solved the public-side of our scaling problem nicely. I mean, the bandwidth coming from the public to us scales linearly. But our back-end suffers the consequences.

Long before we reach a respectably sized server cluster, our intranet will be completely saturated by our webapp dispatching events. On top of that, our webapp will have some serious cpu usage issues what with all the tcp connections and event dispatching to each of the Comet nodes. We could abstract a bit — use a central dispatching server, or put logic on each node in our Comet cluster such that each node will replicate an event to the other nodes. But no matter how you slice it you’ll end up with an unscalable solution, as event data needs to be re-exchanged an increasing amount with each additional node.

Disclaimer: I’m simplifying the explanation of scaling publish/subscribe to some extent. There are all sorts of graph theory experts who know great ways to solve some of these problems. So I’m not saying that we should throw out publish/subscribe. Rather, I’m saying that it’s damn hard to scale. I doubt we’ll see a truly scalable and open source comet server that provides a good publish/subscribe mechanism.

It’s time to see how Orbited fits in to this mess. You can think of Orbited the same way you would think of Memcached. Except, instead of distributing pieces of data across nodes, we’re distributing Comet connections. Each connection has a unique id, and based on that id the browser is assigned to a particular node. Each web app node uses the same hashing algorithm to dispatch events, so they always know where every user is without communicating with other nodes.

Orbited does not handle publish/subscribe whatsoever. There is no way for us to do that and still scale linearly. But as a result, every time we send an event from our application to a user, it takes a single path through our intranet to their browser.

So now the bad news. Most applications benefit from publish/subscribe. Even a simple stock ticker or chat application. And Orbited doesn’t provide that support. Our philosophy is that we aren’t experts at scaling publish/subscribe so we’ll let someone else do it. For example Jabber, or IRC. So if you want to use publish/subscribe, then you need to add an additional layer to your stack. This is going to be frustrating for newcomers who want to create a prototype and leave worrying about all the scaling rigamarole until later, or never. As a result, many applications, even my most basic demo app (UPDATE: this demo app has become cherrychat) have to roll their own subscribe based system.

I don’t want to end with the bad news, so I’m going to point out an academic paper from 2004 titled “Building Content-Based Publish/Subscribe Systems with Distributed Hash Tables” which discusses the future of scaling publish/subscribe on the internet. The authors propose that we use distributed hash tables as the basis of scalable publish/subscribe systems. So, rest assured that by adopting Orbited you are on the right track. It may not be a full stack solution, but it’s fast and scalable and sometimes those concerns trump all else.

This entry was posted on Monday, April 9th, 2007 at 12:10 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.