Previous Table of Contents Next


After an HTTP request has been dequeued by a thread in the pool, this thread performs the necessary computation and file system operations to service the request. The requested data is then transmitted synchronously to the client. After the data transmission completes, the thread returns to the pool and reinvokes the dequeue method to retrieve another HTTP request.

In contrast with the handle-based Thread Pool model, the queue-based Thread Pool design makes it possible to accept (or reject) all incoming connections rapidly and prioritize how each client is processed. The primary drawback stems from the extra context switching and synchronization required to manage the queue in the Web server.

The asynchronous Thread Pool model, shown in Figure 2.16, uses the ACE Proactor, which manages an I/O completion port. An I/O completion port is a thread-safe queue of I/O completion notifications that resides in the OS kernel (in contrast, the queue-based Thread Pool managed the thread in user space). Each I/O operation is initiated and handed off to the kernel, where it runs to completion. Therefore, the initiating thread does not block. When these operations complete asynchronously, the kernel queues the resulting notifications at the appropriate I/O completion port.

As with the synchronous Thread Pool model, the asynchronous Thread Pool model is created during Web server initialization. Unlike in the synchronous model, however, the threads wait on an I/O completion port rather than waiting on accept. The OS queues up results from all asynchronous operations (e.g., asynchronous accepts, reads, writes) on the I/O completion port. The result of each asynchronous operation is handled by a thread selected by the OS from the pool of threads waiting on the completion port. The thread that dequeues the completion notification need not be the same one that initiated the operation.

The asynchronous Thread Pool model is typically less resource intensive and provides more uniform latency under heavy workloads than synchronous Thread Pool models (Hu et al., 1997). It is also more scalable because the same programming model works for a single thread as well as multiple threads. The primary drawback with the asynchronous Thread Pool model is that it is not portable to platforms that lack asynchronous I/O and proactive event dispatching. Windows NT 4.0 is the main contemporary operating system that supports I/O completion ports in its OS API. The ACE Proactor encapsulates the Windows NT 4.0 I/O completion port asynchronous demultiplexing mechanism within a type-safe C++ wrapper.

2.4.4.4. The Thread-per-Session Web Server Model

In the Thread-per-Session model, the newly created handler thread is responsible for the lifetime of the entire client session rather than just a single request from the client. As with the Thread-per-Request model, only one thread blocks on the acceptor socket. This acceptor thread is a factory that creates a new handler thread to interact with each client for the duration of the session. Therefore, the new thread can serve multiple requests from a client before terminating.

The Thread-per-Session model is not appropriate for HTTP 1.0 because the protocol establishes a new connection for each request. Thus, Thread-per-Session is equivalent to Thread-per-Request in HTTP 1.0. The Thread-per-Session model is applicable in HTTP 1.1, however, because it supports persistent connections (Fielding, Gettys, Mogul, Frystyk, & Berners-Lee, 1997; Mogul, 1995). Figure 2.17 illustrates the Thread-per-Session model.


FIGURE 2.17.  The Thread-per-Session Web server model.

The Thread-per-Session model provides good support for prioritization of client requests. For instance, higher-priority clients can be associated with higher-priority threads. Thus, a request from a higher-priority client is served ahead of requests from lower-priority clients because the OS can preempt lower-priority threads. One drawback to Thread-per-Session is that connections receiving considerably more requests than others can become a performance bottleneck. In contrast, the Thread-per-Request and Thread Pool models provide better support for load balancing.


Previous Table of Contents Next