Well, it's been awhile since we've written anything here, so I figured it was about time to get some more news out. We've been hunkered down working hard on the next revision of WebSync, which is coming soon (don't worry, I'll post about it here so you'll get all the juicy details). In the meantime, however, I thought I'd write down a couple of interesting things that have come up in the past month or so.

The Most Difficult Thing™

Writing a comet server is hard. I don't think anyone would deny that. Implementing a solution that scales vertically, horizontally, is robust and is easy to troubleshoot is no walk in the park. But the real question is: is that the most difficult thing we've had to do?

Well, the answer is probably yes ;). But, there's a *very* close second. And that is creating an API that's simple to use, but also powerful enough to handle all the corner cases that developers want to use it for.

That's been a major focus for us with our upcoming version of WebSync: a clean, easily understood API that anyone can use, but that also can be twisted and manipulated in crazy ways so developers who are pushing WebSync to the max can still do what they want/need with it.

A Rose by Any Other Name

Although Shakespeare's claim that it might smell as sweet might be true, if you called a Rose a Skunkweed, there might be fewer people who would try to smell it. The same principle applies to APIs - if you don't use a name or convention that makes logical sense when naming your methods, properties and events, there will be few people who would try to use it, making the API difficult to use

Not only does a poor API make your software difficult to use, it makes development with it frustrating, which is a terrible thing indeed; nothing will turn developers off your software faster than being given stupid choices that have no obvious distinction. Our goal for the new version of WebSync? Make the software so easy to use that you don't need documentation. (Don't worry - we ramped up the quality of our documentation too, so you can use it, you just are less like to need to do so.) We had many...ahem...vociferous dicussions...about how to name our API to make it the absolute best API around, not just for a comet server, but for any software component available.

The end result is hopefully an API that's so clean and simple to understand, that even if your intellisense breaks, you'll know exactly what calls to make and arguments to use when implementing your awesome solution with WebSync. And if you find the API has problems or is confusing in places - we want to know! You'll find all our contact info here.

Many complications arise when building a real-time web application that employs "comet" or "reverse Ajax" to push data from the server. This is mostly due to the fact that real-time web applications require multi-threading, either directly or indirectly, and many of the same issues that arise in threading scenarios directly parallel the issues present in comet-based websites.

A frequently found issue has to do with the timing of subscribing to a comet channel versus the initial state load of the web page.

For example, consider a simple chat application. When the page loads up, the user expects to see some recent chat messages, as well as start receiving any newly published messages. The initial state load is a distinct operation from the subscription to receive new messages. As such, one must be executed before the other. There are only two possible options:

1. Load state, then subscribe.
2. Subscribe, then load state.

The first option results in the possibility of missing messages. It's feasible (and likely under load) for a user to publish a message after the state has loaded, but before the subscription has completed. This message would not be included in the state load, and it would not be delivered to the user over comet since the subscription was not present at the time of delivery.

The second option results in the possibility of duplicated messages. It's feasible (and again likely under load) for a user to publish a message after the subscription has completed, but before the state has loaded. This message would be delivered to the user over comet since the subscription was present at the time of delivery, and it would be included in the state load.

It has been suggested that the initial state load should be delivered in the response to the subscription request. This is absolutely possible with WebSync, but it does not remove the problem - it just narrows the gap in which the problem can occur. The scenario where messages go missing or are duplicated becomes less likely, but we find ourselves in a dangerous situation when we try to hide bugs instead of fix them. Testing becomes more difficult and we run the risk of unleashing a bug that is more difficult to track down.

So what do we do?

In general, the best thing to do is not lose messages. (As always, there are exceptions, but this is the general case we have found.) So, subscribe first, and load state data afterwards, usually through an Ajax request.

• Page loads and a loading indicator is shown.
• WebSync client is created and subscribes to the channel.
• In the subscribe onSuccess handler, POST a request for initial state with Ajax.
• In the Ajax onSuccess handler, populate the UI and remove the loading indicator.

To get rid of the duplicate messages, set a loaded flag to true once the UI has been populated. If the WebSync client receives a message and the loaded flag is false, push it onto a list. Once the loaded flag has been set to true, read through the list and process new messages, discarding duplicates. Duplicates can be identified by a server-generated ID that is attached to every message publication.

A common misconception about IIS and the .NET framework is that it is unable to scale well to tens of thousands of simultaneous requests. WebSync has proven that this is not the case, so why does that misconception still exist?

Not just a page anymore

In classic ASP, there was no way to make an incoming request go "idle" and just wait for something else to push it along. Instead, requests blocked on the thread pool, eventually saturating it so no further requests could be processed.

ASP.NET takes the same approach with standard requests, but provides another option as well - the IHttpAsyncHandler. This handler is designed to allow multiple requests with long-running executions to take minimal CPU usage and not saturate the thread pool, a perfect solution for building a comet server. It's no surprise, then, that this is the foundation on which WebSync is built.

Beyond the basics

Of course, creating the async handler is pointless if each request just blocks on a thread from the CLR thread pool waiting for events. Even if each request blocked for just a few seconds, you'd quickly exhaust the thread pool under load. The requests have to be offloaded for batch analysis in a separate bounded thread pool. The bounded thread pool can then make use of shared data structures to balance the request load while waiting for events to trigger a response.

To make this work, the CLR thread pool has to have a large number of threads available for the handling of incoming and outgoing requests, and the secondary bounded thread pool has to have just a few threads for the handling of everything in between. The trick is to get incoming requests off the CLR thread pool quickly for long-running processes so the secondary thread pool can bear the weight and keep the CLR thread pool light on it's feet.