Previous | Table of Contents | Next |
To avoid blocking the server for extended periods of time, each I/O request can be broken into small chunks and sent separately. Therefore, the State pattern is typically used to maintain each HTTP handlers state (e.g., awaiting the GET request, transmitting the nth chunk, closing down). Likewise, the timer queue capabilities of the ACE Reactor can be used to prevent denial of service attacks where erroneous or malicious clients establish connections and consume Web server resources (e.g., socket handles) but never send data to or receive data from the server.
The main advantages of the Single-Threaded Reactive model are its portability and its low overhead for processing very small files. It is not suitable for high-performance Web servers, however, because it does not utilize parallelism effectively. In particular, all HTTP processing is serialized at the OS event demultiplexing level. This prevents Web servers from leveraging the parallelism available in the OS (e.g., asynchronous I/O) and hardware (e.g., DMA to intelligent I/O peripherals).
2.4.4.2. The Thread-per-Request Web Server Model
In the Thread-per-Request model, a new thread is spawned to handle each incoming request. Only one thread blocks on the acceptor socket. This acceptor thread is a factory that creates a new handler thread to process HTTP requests from each client.
The Thread-per-Request model is a widely used model for implementing multithreaded Web servers. This model runs on any OS platform that supports preemptive multithreading. The structure of a Thread-per-Request Web server based on the ACE Reactor and ACE Active Objects is shown in Figure 2.13.
FIGURE 2.13. The Thread-per-Request Web server model.
Figure 2.13 illustrates how the ACE Reactor and HTTP Acceptor components can be reused for the Thread-per-Request model, i.e., the ACE Reactor blocks in the main thread waiting for connection events. When a connection event occurs it notifies the HTTP Acceptor factory, which creates a new HTTP Handler.
The primary difference between the Thread-per-Request model and the Single-Threaded Reactive model is that a new thread is spawned in each HTTP Handler to process every client request concurrently. Thus, the HTTP Handler plays the role of an active objecti.e., the ACE Reactor thread that accepts the connection and invokes the HTTP Handler executes concurrently with the threads that perform HTTP processing. In HTTP 1.0, the life cycle of an HTTP Handler active object is complete when the file transfer operation is finished.
The Thread-per-Request model is useful for handling requests for large files from multiple clients. It is less useful for small files from a single client due to the overhead of creating a new thread for each request. In addition, Thread-per-Request model can consume a large number of OS resources if many clients perform requests simultaneously during periods of peak load.
2.4.4.3. The Thread Pool Web Server Model
In the Thread Pool model, a group of threads is prespawned during Web server initialization. Each thread blocks on the same acceptor socket, waiting for connections to arrive from clients. Prespawning eliminates the overhead of creating a new thread for each request. It also bounds the number of OS resources consumed by a Web server.
The Thread Pool model is generally the most efficient way to implement the event dispatcher in high-performance Web servers (Hu, et al., 1997). This model is most effective on OS platforms (e.g., Windows NT, Solaris 2.6) that permit simultaneous calls to the accept function on the same acceptor socket. On platforms that do not allow this (e.g., most SVR4 implementations of UNIX), it is necessary to explicitly serialize accept with an ACE Mutex synchronization object.
There are several variations of the Thread Pool model. Figures 2.14 and 2.15 illustrate the handle-based and queue-based synchronous Thread Pool models, respectively. Figure 2.16 illustrates the asynchronous Thread Pool model. Each of these variants is outlined in the following text.
FIGURE 2.14. The handle-based synchronous Thread Pool Web server model.
FIGURE 2.15. The queue-based synchronous Thread Pool Web server model.
FIGURE 2.16. The asynchronous Thread Pool Web Server model.
The handle-based synchronous Thread Pool model shown in Figure 2.14 does not use a Reactor. Instead, each thread in the pool directly invokes the handle_input method of the HTTP Acceptor, which blocks awaiting client connections on the acceptor socket handle. When clients connect, the OS selects a thread from the pool of HTTP Acceptors to accept the connection. After a connection is established, the acceptor morphs into an HTTP Handler, which performs a synchronous read on the newly connected handle. After the HTTP request has been read, the thread performs the necessary computation and file system operations to service the request. The requested file is then transmitted synchronously to the client. After the data transmission completes, the thread returns to the pool and reinvokes HTTP Acceptors handle_input method.
Client requests can execute concurrently until the number of simultaneous requests exceeds the number of threads in the pool. At this point, additional requests are queued in the kernels socket listen queue until a thread in the pool finishes its processing and becomes available. To reduce latency, the Thread Pool model can be configured to always have threads available to service new requests. However, the number of threads needed to support this policy can be very high during peak loads as threads block in long-duration synchronous I/O operations.
One drawback with the handle-based Thread Pool model is that the size of the socket listen queue is relatively small (around 8 to 10 connections on most OS platforms). Therefore, high-volume servers that receive hundreds of Web hits per second might not be able to accept connections fast enough to keep the kernel from rejecting clients. Moreover, it is not possible to prioritize which connections are dropped because the kernel does not distinguish among different clients.
The queue-based synchronous Thread Pool shown in Figure 2.15 uses the Half-sync/Half-async pattern, which combines the Reactor and Active Object patterns. In this model, the ACE Reactor thread accepts connections from clients (via the HTTP Acceptor) and manages all the HTTP handlers. When HTTP requests arrive from clients, they are validated briefly by the associated HTTP Handler in the ACE Reactor thread and then are enqueued in the thread-safe ACE message queue that joins the async and sync layers in the Web server. Each active object in the thread pool invokes the dequeue method of the request queue, which blocks awaiting client requests.
Previous | Table of Contents | Next |