by anton.venema
18. November 2009 23:34
A common misconception about IIS and the .NET framework is that it is unable to scale well to tens of thousands of simultaneous requests. WebSync has proven that this is not the case, so why does that misconception still exist?
Not just a page anymore
In classic ASP, there was no way to make an incoming request go "idle" and just wait for something else to push it along. Instead, requests blocked on the thread pool, eventually saturating it so no further requests could be processed.
ASP.NET takes the same approach with standard requests, but provides another option as well - the IHttpAsyncHandler. This handler is designed to allow multiple requests with long-running executions to take minimal CPU usage and not saturate the thread pool, a perfect solution for building a comet server. It's no surprise, then, that this is the foundation on which WebSync is built.
Beyond the basics
Of course, creating the async handler is pointless if each request just blocks on a thread from the CLR thread pool waiting for events. Even if each request blocked for just a few seconds, you'd quickly exhaust the thread pool under load. The requests have to be offloaded for batch analysis in a separate bounded thread pool. The bounded thread pool can then make use of shared data structures to balance the request load while waiting for events to trigger a response.
To make this work, the CLR thread pool has to have a large number of threads available for the handling of incoming and outgoing requests, and the secondary bounded thread pool has to have just a few threads for the handling of everything in between. The trick is to get incoming requests off the CLR thread pool quickly for long-running processes so the secondary thread pool can bear the weight and keep the CLR thread pool light on it's feet.