The Web and Windows NT are an exciting pair of technologies to be involved with today. With the rapid development of technologies for NT, such as ISAPI from Microsoft, and the strong growth of the Internet and intranets, mastering Windows NT and Web development promises to be an exciting journey.
As you develop ISAPI applications, you will find that it is vital to understand how the Web works. The fact is, the more you understand about Web communications, the more powerful your ISAPI programs will be.
ISAPI extensions and filters effectively become part of the Web server, as illustrated in Figure 2.1, through runtime dynamic linking. That is, at runtime, the server loads your dynamic link library (DLL) through the LoadLibrary and GetProcAddress functions.
At that point, your ISAPI programs have complete access to Web requests as they occur. They canability to supply the results of hypertext transport protocol (HTTP) requests, modify the results from other server extensions or static files, and log specialized information about those requests. In other words, ISAPI is a new and powerful means to an end. The end we want to achieve is efficient and dynamic Web communications.
ISAPI extensions and filters become part of the Web server.
The Web is based on a client-server model. Client-server computing distributes the basic components, user interface, program logic, and data, between the client and server computers.
In a client-server model, the client
In a client-server model, the server
In traditional client-server modeling, clients and servers can be "thin" or "fat." The terms indicate a functional relationship rather than a physical characteristic of the computer. The standard components of an application are a user interface, the program's logic or business rules, and data storage.
Client-server computing is a division of labor that is typically needed by most applications. The server is optimized to provide data to multiple clients, and the client application is optimized to interact with the end-user. The one task that may reside on one or both sides of the model is the functional logic.
Thin clients are generally limited to a user interface. All processing and program logic reside on a fat server. The server is called "fat" because the functional logic resides on the server. This is the most common model on the Web today. This model is illustrated in Figure 2.2.
The client in our case is typically a Web browser such as Microsoft's Internet Explorer. The Web browser is simply a user interface to many kinds of data that are retrieved from the Web server. In our case, the ISAPI applications provide the functional logic to the HTTP server and the client displays the various types of data from a specific URL.
Thin clients work with fat servers.
By moving the program logic from the server to the client, we create a fat-client, thin- server model. Each client accessing the server makes program flow decisions based on the logic or business rules that reside on the client computer. The thin server acts as a gateway to data storage for clients. The fat-client, thin-server model is illustrated in Figure 2.3.
Fat clients work with thin servers.
There are other models such as distributed services in which program functionality is shared between clients and servers.
Clients on the Web are also called user agents. The server in our case is a Web server such as Microsoft Internet Information Server (IIS). In the past, because Web browsers were simply a user interface, the Web model was a fat server, thin client. However, with the innovation of technologies that allow client-side programming, such as Microsoft ActiveX/COM, the browser is now rapidly being enabled to do client-side processing.
User Agents
User agents refer to any client software that initiates a request to a Web server. As the Web increases in complexity, new types of user agents are invented to provide new functionality for Web users.
Web Browsers
The most popular type of user agent is the Web browser. By now, most readers probably have installed countless beta versions of various Web browsers. Because the Web is changing so quickly, developer's often need the latest browser version using the latest HTML tags and features like style sheets, and the latest technologies like ActiveX.
Web browsers are the "human interface" to the Web. They enable us to navigate from Web site to Web site and to bookmark our favorite sites so that we can return later. On the practical side, they allow us to research subjects that interest us, to find old friends, or to read the latest news from around the world.
The first Web browser was created in November 1990 on a NeXT computer by Tim Berners-Lee, a consultant for CERN, the European Laboratory for Particle Physics (http://www.cern.ch). In February 1993 an alpha version of Mosaic for X (http://www.ncsa.uiuc.edu/SDG/Software/Brochure/UNIXSoftDesc.html#XMosaic) was released by the National Center for Supercomputing Applications (NCSA).
The lead programmer on this project at the NCSA was Marc Andreeson. Marc left the NCSA to create a company that we've come to know as Netscape Communications Corporation. Netscape Navigator is currently the most popular browser on the Web.
Microsoft entered into the browser market in early 1995 with Internet Explorer (IE) (http://www.microsoft.com/IE). Microsoft is aggressively marketing IE and many industry analysts expect IE to become the standard Web browser within the next year (see Figure 2.4).
Microsoft Internet Explorer.
Other User Agents
Other user agents have been designed to harness the resources of the Web. For example, some user agents compile information about the contents of a Web site, which are then presented for search and retrieval. Users can instantly search tens of thousands of Web pages for key words, phrases, and even concepts.
You probably know these user agents by names like Excite (http://www.excite.com), AltaVista (http://altavista.digital.com), Lycos (http://www.lycos.com) and InfoSeek (http://www.infoseek.com). Figure 2.4 shows the user-interface that AltaVista provides to their Web index. This technology has been scaled down to the enterprise level in software packages such as Microsoft ActiveX Search, code-named Tripoli.
Another class of user agent has been created to find broken links on Web sites. These agents traverse a chain of document links, requesting each document from the server and ensuring that they are valid. These "spiders" note new URL links in each document that have not been verified and then follow these new chains till site verification is complete.
A new class of intelligent agents is being developed today to do more complicated tasks such as creating custom reports based on information located on the Web. Some of these intelligent agents even negotiate business transactions.
The Web browser will continue to serve as the human interface to the Web, but user agents such as these will help us to find the information that's really important to us.
Altavista: A popular Web crawler.
Web Servers
The other component of the client-server model is, of course, the server. Web servers are the software packages that accept connections from user agents and service the requests from those clients by sending back responses in the form of informational messages and documents.
As we mentioned before, the first Web software was developed on a computer from NeXT in November 1990. Once the Web client-server model was developed, programmers began working on stand-alone servers and rudimentary browsers.
By the end of 1992 there were 26 "reasonably reliable" servers on the Internet, and by January of the next year the number doubled. Near the end of 1993 the number had grown to 200 known Web servers. Three years later, at the end of 1996, there are hundreds of thousands of Web servers on the Internet and an unknown number of servers in use on corporate intranets.
Early HTTP Servers on Windows
One of the earliest Web servers on the Windows platform was named Win-HTTPD by Bob Denny. Bob created a true multithreaded application that runs under Windows 3.1. The server also allowed image mapping, security, and CGI programs. Bob went on to write a Web server known today as O'Reilly Website.
Another early Web server for the Windows platform is the EMWAC HTTP server, designed to run on Windows NT. The authors of this software quit developing the free product and rewrote the server to market it commercially. This product is marketed today under the name of Process Purveyor.
Commercial HTTP Servers
At one time it was almost impossible to find HTTP servers for the Windows platform because most of the available options were focused on UNIX. But the tides have turned and these days UNIX is being dropped by a number of large software companies in favor of the Windows NT platform. In fact, there are so many Web servers available today on the Windows NT platform that we'll look at just a few of them.
Web servers are also commonly called HTTP servers. HTTP is the protocol used by these servers to communicate with Web clients. It is a client-server protocol designed for fast and efficient transfer of hypertext documents.
HTTP is a simple protocol to understand. It is based on a request-response paradigm.
A simple HTTP transaction looks like this:
Figure 2.6 shows a simple but complete HTTP transaction.
A complete HTTP transaction.
HTTP is also a stateless protocol. That is, the protocol does not maintain any information about previous requests to the server: each request is independent of all other requests. Because the protocol is stateless, each request must have complete information about the current request being made.
TCP/IP Connections
Before looking at HTTP requests, we'll cover briefly how Web clients and servers connect. HTTP connections generally use TCP/IP, which is made up of two different protocols.
Before an HTTP connection can be made, the client and server computers must both be connected to the Internet. This is usually done through TCP/IP.
The other element in achieving a connection is the domain name system (DNS). Before a connection can be made with a remote host, the IP address of the host must be determined. We usually know a host's name by something like www.microsoft.com. These English descriptions are used because people can remember www.microsoft.com more easily than they can remember 207.68.137.35.
The DNS system works like this:
This entire process is typically completed within a fraction of a second, although some requests may be longer because of network congestion. Once our Web browser knows the IP address of the HTTP server, it attempts to open a connection to that IP address, typically using port 80, and the HTTP request is submitted to the server.
Using host names like www.microsoft.com also allows the remote DNS server to do some interesting tricks such as load balancing. By rotating the IP address among a number of host computers, the DNS server can manage server loads. You can see this in action by pinging www.microsoft.com a number of times. You will notice that the IP changes each time you ping the host.
Ping is a utility that comes with Windows 95 and Windows NT that can test a network connection or allow you to determine a remote host's IP address. To use ping, open a Command prompt and type PING to receive help using the utility.
![]()
HTTP Requests
The simplest request consists of a request method, a requested URI, and the version of HTTP that the user agent can accept. A URI is a formatted string that identifies a network resource giving a name, location, or other identifying characteristic of the resource. We look at URIs and URLs in a little more detail shortly.
So what happens when you click on a link in your Web browser? If the link is a simple GET request, a request is generated by the browser, and the browser opens a connection to the server and sends the request. The request looks something like this:
GET /default.htm HTTP/1.0
This request tells the server to GET the resource named /default.htm and send it back to the user agent using HTTP version 1.0. The server receives this request and tries to find the file, and then send it back using HTTP 1.0.
If it cannot find the file, an error message is returned. Once the response is completed, the connection to the user agent is closed by the server. The user agent then displays the results of the request on your screen.
Let's look at how the server responds to this GET request in more detail.
HTTP Responses
The Web server's response consists of a status line, response header, entity header, and entity body. Here is an example of how a successful HTTP transaction would look in response to our GET request:
HTTP/1.0 200 OK
Server: Microsoft-IIS/2.0
Date: Fri, 16 Aug 1996 16:53:58 GMT
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Fri, 12 Jul 1996 01:30:00 GMT
Content-Length: 4051
[blank line ended with CRLF]
<!doctype html public "-//IETF//DTD HTML//EN">
<html>
<head>
</head>
<body background="/images/backgrnd.gif">
[other information sent has been removed]
The first item sent back from the server is called the status line. The status line consists of a string identifying the HTTP version and a status code. In this case the status of our request is 200, which equates to the status description of OK. Table 2.1 lists the common HTTP status codes.
Table 2.1 HTTP Status Codes
Status Code | Response Phrase | Definition |
200 | OK | The request has succeeded. The information returned with the response is dependent on the method used in the request, as follows: GET, an entity corresponding to the resource requested is sent in the response; HEAD, the response must only contain the header information and no Entity-Body; POST, an entity describing or containing the result of the action. |
201 | Created | The request has been fulfilled and resulted in a new resource being created. The newly created resource can be referenced by the URI(s) returned in the entity of the response. |
202 | Accepted | The request has been accepted for processing but the processing has not been completed. |
204 | No Content | The server has fulfilled the request but there is no new information to send back. |
300 | Multiple Choices | This response code is not directly used by HTTP/1.0 applications but serves as the default for interpreting the 3xx class of responses. |
301 | Moved Permanently | The resource requested has been assigned a new permanent URL, and any future references to this resource should be made using that URL. |
302 | Moved Temporarily | The resource requested resides temporarily under a different URL. Since the redirection may be altered on occasion, the client should continue to use the Request-URI for future requests. |
304 | Not Modified | The client has made a conditional GET request and access is allowed, but the document has not been modified since the date and time specified in the If-Modified-Since field. The server must respond with this status code and not send an Entity-Body to the client. |
400 | Bad Request | The request could not be understood by the server because of malformed syntax. The client should not repeat the request without changes. |
401 | Unauthorized | The request needs user authentication. The response must include a WWW-Authenticate header field (Section 10.16) containing a challenge applicable to the resource requested. |
403 | Forbidden | The server understood the request but is refusing to fulfill it. Authorization will not help and the request should not be repeated. |
404 | Not Found | The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. |
500 | Internal Server Error | The server encountered an unexpected condition that prevented it from fulfilling the request. |
501 | Not Implemented | The server can't fulfill the request. This is the appropriate response when the server does not recognize the request method and can't use it for any resource. |
502 | Bad Gateway | The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request. |
503 | Service Unavailable | The server can't handle the request because of temporary overloading or maintenance of the server. This is a temporary condition that will be alleviated after some delay. |
The server also sends back additional information called response headers. These normally include the kind of server software being used, and the current server date and time. Also included is an entity header with the date the resource requested was last changed, the type of resource in multipurpose Internet mail extensions (MIME) type/subtype form, and the length of the entity body.
Once these request headers are sent, the server ends the request header with a blank line. The server then sends the entity body, which in our example is 4051 bytes of text/html data.
Request Methods
In the example above, we use the GET request method. GET is designed to retrieve information from the HTTP server. The URI designated in the GET request can be a static HTML file or it can be an ISAPI DLL that, for example, generates a dynamic HTML page from a SQL server database.
GET works fine if we want only to get information from the server. But what happens if we need to send information to the server? Web sites usually need a number of different data entry forms such as sign-up forms or feedback forms.
Forms allow Web browsers to post data to the server.
The POST method allows Web browsers to send data to the Web server. Forms, like the one shown in Figure 2.7, are typically used to post data to a Web server. A POST request with request headers looks like this:
POST /scripts/srch.dll HTTP/1.0
Accept: */*
Referer: http://199.1.154.46/isapi/srch.htm
Accept-Language: en
Content-Type: application/x-www-form-urlencoded
UA-pixels: 1024x768
UA-color: color16
UA-OS: Windows NT
UA-CPU: x86
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0B; Windows NT)
Host: 199.1.154.46
Connection: Keep-Alive
Content-Length: 92
Pragma: No-Cache
[blank line ended with CRLF]
input_text=now+is+the+time+for+all+good+men+%28and+women%21%29+to+come+to+ISAPI+programming.
As you can see, the client begins the POST request with the request method, the URI the data will be posted to, and the HTTP version. At the end of our request headers, you see the standard blank line and then the data we posted to the server.
The data encoding is denoted in the Content-Type subtype. In this case the encoding is x-www-form-urlencoded. This encoding does a number of simple data transformations such as translating spaces to a plus sign.
We have posted the data to the server. In this case the server responds to the Web browser with this:
HTTP/1.0 200 OK
Server: Microsoft-IIS/2.0
Date: Fri, 16 Aug 1996 16:50:02 GMT
Content-Type: text/html
<head><title>Search Results</title></head><BODY
BACKGROUND="/samples/images/backgrnd.gif"><BODY BGCOLOR="FFFFFF"><TABLE><TR><TD>
<IMG SRC="/ images/SPACE.gif"
[other html data deleted]
The Web browser receives the response consisting of test/html data. The browser removes the status line, response, and entity headers and then displays the HTML in the browser's window.
We've looked at the GET and POST request methods. The final method we'll look at is the HEAD request method. The HEAD request method allows user agents to get information about a specific URI entity without getting the entity itself. A HEAD request takes the form of
HEAD /default.htm HTTP/1.0
The server responds by returning the same response headers as it does for a GET request, but the server does not include the entity body:
HTTP/1.0 200 OK
Server: Microsoft-IIS/2.0
Date: Mon, 19 Aug 1996 22:22:23 GMT
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Fri, 12 Jul 1996 01:30:00 GMT
Content-Length: 4051
The HEAD method is useful to find out whether a resource has changed or to find out other characteristics of the resource without actually getting that resource.
Table 2.2 summarizes the common request methods.
Table 2.2 HTTP Request Methods
Method | Definition |
GET | The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI. If the Request-URI refers to a data-producing process, the produced data and not the source text of the process is returned as the entity in the response, unless that text happens to be the output of the process. |
POST | The POST method requests that the destination server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. |
HEAD | The HEAD method is identical to GET except that the server must not return any Entity-Body in the response. The meta-information in the HTTP headers in response to a HEAD request should be identical to the information sent in response to a GET request. |
Request Headers
Request headers provide the user agent and HTTP server with additional information about an HTTP transaction. For example, the Accept header field provides information about what MIME types the client can accept. The User-Agent header field provides the name of the user agent that is requesting the document. Here is our original GET request as Microsoft IE 3.0 would send it:
GET /default.htm HTTP/1.0
Accept: */*
Accept-Language: en
UA-pixels: 1024x768
UA-color: color16
UA-OS: Windows NT
UA-CPU: x86
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0B; Windows NT)
Host: 199.1.154.46
Connection: Keep-Alive
The request is still a GET request for the document /default.htm using HTTP 1.0. But the Accept header tells the server that we can accept any MIME type or subtype, as shown by the */*.
Accept-Language shows that the preferred language of the client is English. The server could tailor the language response by using this header field.
UA-pixels and UA-color give information about the client's video capabilities. The server could select graphics that are created specially for various screen resolutions based on this information. UA-OS and UA-CPU show the type of operating system and hardware the client is running under, and so on.
As you can imagine, a server ISAPI application could use each of these items and tailor the output to give the user-agent the best possible presentation of the information requested.
Table 2.3 lists the common HTTP request headers. Note that some of the header fields listed are from the HTTP 1.1 specification. The reality of Web development is that the technology is changing so quickly that developers may have to use it before the specifications are final.
Table 2.3 Common HTTP Request Headers
Header | Definition |
Accept | The Accept request-header field specifies certain media types that are acceptable for the response. Accept headers can be used to show that the request is limited to a small set of types, as in the case of a request for an inline image. |
Accept-Language | The Accept-Language request-header field is like Accept but restricts the set of natural languages that are preferred as a response to the request. |
Authorization | To authenticate itself with a server-usually after getting a 401 response-a user agent includes an Authorization request-header field with the request. The Authorization field value is the authentication information of the user agent for the realm of the resource requested. |
Cache-Control | The Cache-Control general-header field gives directives that must be obeyed by all caching mechanisms along the request/response chain. The directives are meant to prevent caches from adversely interfering with the request or response. These directives typically override the default caching algorithms. |
Content-Base | The Content-Base entity-header field specifies the base URI for resolving relative URLs within the entity. This header field is described as Base in RFC 1808, which is expected to be revised. |
Content-Encoding | The Content-Encoding entity-header field is a modifier to the media-type. When present, its value shows what additional content coding has been applied to the resource and thus what decoding mechanism must be applied to get the media-type referenced by the Content-Type header field. The Content-Encoding allows a document to be compressed without losing the identity of its underlying media type. |
Content-Length | The Content-Length entity-header field shows the size of the Entity-Body, in a decimal number of octets, sent to the recipient. Or, in the case of the HEAD method, this field shows the size of the Entity-Body that would have been sent had the request been a GET. |
Content-Type | The Content-Type entity-header field shows the media type of the Entity-Body sent to the recipient or, in the case of the HEAD method, the media type that would have been sent had the request been a GET. |
Date | The Date general-header field represents the date and time at which the message was originated and has the same semantics as orig-date in RFC 822. |
Expires | The Expires entity-header field gives the date/time after which the entity should be considered invalid. This allows information providers to suggest the volatility of the resource or a date after which the information may no longer be valid. Applications must not cache this entity beyond the date given.
The presence of an Expires field does not imply that the original resource will change or cease to exist at, before, or after that time. However, information providers who know or even suspect that a resource will change by a certain date should include an Expires header with that date. |
Host | The Host request-header field specifies the Internet host and port number of the resource requested, as obtained from the original URL given by the user or referring resource. The Host field value must represent the network location of the origin server or gateway given by the original URL. |
If-Modified-Since | The If-Modified-Since request-header field is used with the GET method to make it conditional. If the resource requested has not been modified since the time specified in this field, a copy of the resource is not returned from the server. Instead, a 304 (not modified) response is returned without any Entity-Body. |
Last-Modified | The Last-Modified entity-header field shows the date and time at which the sender believes the resource was last modified. The exact semantics of this field are defined in terms of how the recipient should interpret it. If the recipient has a copy of this resource that is older than the date given by the Last-Modified field, that copy is considered invalid. |
Location | The Location response-header field defines the exact location of the resource that was identified by the Request-URI. For 3xx responses, the location must show the server's preferred URL for automatic redirection to the resource. Only one absolute URL is allowed. |
Pragma | The Pragma general-header field includes implementation-specific directives that may apply to any recipient along the request/response chain. All Pragma directives specify optional behavior from the viewpoint of the protocol; however, for some systems, behavior has to be consistent with the directives. |
Referer | The Referer request-header field allows the client to specify, for the server's benefit, the address (URI) of the resource from which the Request-URI was obtained. This allows a server to generate lists of back links to resources, logging, optimized caching, etc. It also allows obsolete or mistyped links to be traced for maintenance. The Referer field must not be sent if the Request-URI was obtained from a source that does not have its own URI, such as input from the user keyboard. |
Server | The Server response-header field holds information about the software used by the originating server to handle the request. The field can have multiple product tokens (Section 3.7) and comments identifying the server and any significant subproducts. By convention, the product tokens are listed in order of their significance for identifying the application. |
User-Agent | The User-Agent request-header field holds information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user-agent limitations. Although it is not needed, user agents should include this field with requests. |
WWW-Authenticate | The WWW-Authenticate response-header field must be included in 401 (unauthorized) response messages. The field value consists of at least one challenge that shows the authentication scheme(s) and parameters applicable to the Request-URI. |
URLs, URIs, and URNs
You'll run into the terms URL and URI often in working on the Web. We'll try to unravel the confusion about the difference between URLs and URIs.
URLs have been called uniform resource locators and universal resource locators. The current specifications use uniform rather than universal.
URLs are a form of URIs. In practice, a URL is a protocol and address that identify an "object." That object could be a document, graphic image, video movie, sound file, or host computer.
The only protocols that can be used, according to the specifications are file://, ftp://, file, http://, mailto://, news://, nntp://, telnet://, wais:// and prospero://.
Uniform resource identifiers (URIs) are a general family of resource identifiers consisting of URLs and URNs.
A uniform resource name (URN) is "any URI that is not a URL."
Figure 2.8 shows the relationship between URIs, URLs and URNs.
The URI family of identifiers.
MIME Types
The MIME specification was created by a need to transfer graphics images and other binary formats in e-mail. The use of the MIME scheme was adopted by the designers of the HTTP protocol to allow these same binary documents to be transferred via the Web.
The MIME naming convention uses a type/subtype scheme in which type represents a major category such as audio, video, image, or text. The subtype specifies the application type such as wav, gif, tif, or jpeg.
You will notice in HTTP response headers that the server sends a Content-Type header denoting the MIME type of the document the GET, HEAD, or POST request results in. In the case of our GET request:
GET /default.htm HTTP/1.0
The beginning of the response header looks like this:
HTTP/1.0 200 OK
Server: Microsoft-IIS/2.0
Date: Fri, 16 Aug 1996 16:53:58 GMT
Content-Type: text/html
Note that the content-type is text/html. If the resource requested is a GIF file, the Content-Type looks like this:
Content-Type: image/gif
Table 2.4 lists some of the common MIME types.
Table 2.4 Common MIME Types
MIME Type | MIME Subtype | Application |
audio | x-wav | WAV audio file |
audio | x-aiff | AIFF audio file |
text | html | HTML file |
text | plain | TXT text file |
image | gif | GIF image |
image | jpeg | JPEG image |
image | tiff | TIF image |
video | mpeg | MPEG video |
video | quicktime | Apple QuickTime video |
video | x-msmovie | Microsoft AVI video |
Testing HTTP Requests
From time to time you will find it useful to directly test a request against a live server. Instead of writing an application to handle this situation, you can use the Telnet application that comes with Windows 95 and Windows NT. To test this, try the following:
You may need to turn logging on (see Figure 2.9) to capture all the information that is returned by the server. To do this, select the Terminal menu and then select Start Logging.
You can also use the Notepad program that comes with Windows NT and Windows 95 to develop complex GET and POST requests. Once you have developed the request, you can copy and paste these requests into the Telnet session.
Logging the Telnet sessions allows you capture the data for analysis later.
HTTP Cookies
We would be remiss not to discuss HTTP cookies. Cookies allow the server to store small amounts of data on the client side and have that information sent back to the server in specified circumstances.
Cookies are at least a partial answer to the problem of the stateless nature of the HTTP protocol. States can be preserved through the use of cookies.
A cookie is sent as part of the HTTP response header from the server. For example, if we want to store a variable named UserAccount on a client's Web browser, we can send a cookie that looks like this:
Set-Cookie: UserAccount=6758; path=/; domain=.abcdef.com
This tells the browser to send a cookie called UserAccount in all future request headers, with a value of 6758, for all documents requested under the path '/', including subdirectories.
Let's look at a few HTTP requests and see how this works. Suppose you've just started your browser and want to look at a document located at http://www.abcdef.com/latest_news.html. You type in the URL, and the browser generates a request that looks like this:
GET /latest_news.html HTTP/1.0
Accept: */*
Accept-Language: en
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0B; Windows NT)
Host: 199.1.154.46
Connection: Keep-Alive
The server at www.abcdef.com finds the /latest_news.html document and responds with this:
HTTP/1.0 200 OK
Server: Microsoft-IIS/2.0
Date: Fri, 09 Aug 1996 10:03:23 GMT
Content-Type: text/html
Set-Cookie: UserAccount=6758; path=/; domain=.abcdef.com
Accept-Ranges: bytes
Last-Modified: Sat, 17 Aug 1996 01:30:00 GMT
Content-Length: 2592
[blank line ended with CRLF]
<!doctype html public "-//IETF//DTD HTML//EN">
<html>
<head>
</head>
<body background="/images/backgrnd.gif">
Now, unseen by the Web user, the browser receives a cookie named UserAccount with a value of 6758. Any cookies that are valid for the given domain, path, and date-time are sent in the HTTP request header until the browser is closed. For example, the next request from the browser might look something like this:
GET /news/world_news.html HTTP/1.0
Accept: */*
Accept-Language: en
Cookie: UserAccount=6758
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0B; Windows NT)
Host: 199.1.154.46
Connection: Keep-Alive
If we want to store this cookie in the client browser's cookie cache file, we can specify the expires attribute and set the expiration date to a time in the future. We could also set additional cookies so that the browser sends back multiple data items when a request is made. In that case, the request from the browser would look like this:
GET /news/world_news.html HTTP/1.0
Accept: */*
Accept-Language: en
Cookie: UserAccount=6758; Hits=3; Level=2
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0B; Windows NT)
Host: 199.1.154.46
Connection: Keep-Alive
As you see, we have cookies set for UserAccount, Hits, and Level. Developers who use cookies should take note that a client can have up to 30 cookies per server or domain set at any one time.
A bug in Netscape Navigator 1.1 and earlier prevents cookies from being set when the path attribute is set to '/' and the expires attribute is not specified. Make sure you specify an expires attribute if you want the cookie saved when the path is set to '/'.
![]()
The current draft specifications for cookies indicate that a maximum of 300 cookies can be stored on a client computer at any given time. Cookies are limited to 4,096 bytes, including the length of the cookie names.
A list of cookie attributes is in Table 2.5.
Table 2.5 Cookie Attributes
Variable | Example | Definition |
name | UserAccount=6758 | This is the name of the cookie you want to work with. The cookie name is the only required attribute of a cookie. |
Expires | expires=Mon, 11-Nov-1996 00:01 GMT | The expires attribute sets the date that the cookie expires. Once the expiration date arrives, the cookie is destroyed by the browser and is no longer sent to an HTTP server. |
Domain | domain=www.abcdef.com | The domain attribute specifies which domains the cookie should be sent to. For example, a cookie with the domain of .abcdef.com is valid within the domain www.abcdef.com and search.abcdef.com. But a cookie with a domain of www.abcdef.com is only valid within that specific domain. |
Path | path=/ | The path attribute shows the subset of URLs for which the cookie is to be used. If the path is set to '/', the cookie is sent to the server for all URLs requested. If the path is set to '/foo', the browser sends the cookie to the server when documents in the /foo directory are requested. Documents requests in the /foobar directory also qualify to receive the cookie. |
Secure | secure | If the secure attribute is present, the cookie is only sent by the browser that is connected through a secure communications connection using Secure Sockets Layer (SSL). If the secure attribute is not present, the cookie is transmitted over secure and non-secure connections. |
Cookies are useful because once a user is identified, the server can essentially have the client's browser identify itself in future HTTP transactions. State information can be passed on each transaction, including record number requests, user account information, and other information that normally would have to be stored in HTML hidden fields.
The server could also encrypt the cookie data using strong encryption such as DES, Blowfish, or a custom encryption scheme. Because the encryption and decryption process is located on the Web server, a developer in the U.S. does not have to worry about export laws.
And because the server simply wants that token data passed back to the server on each request, the Web client does not need any special software to handle encryption or decryption.
![]()
You should not put too much trust in the cookie cache for a number of reasons. First, the cookie file can be erased by the user at any time. If the only source of important data is the cookie cache file, you've just lost that data.
Next, if the browser receives over 300 cookies, it can delete cookies at its own discretion. There aren't any rules in the specifications about which cookies are chosen for deletion.
And finally, the user may switch browsers. The file format for the cache cookies is not specified and varies from vendor to vendor.
HTTP Authentication
Creating secure areas on a Web site is often necessary. For example, you might want to secure an area from all Web users except those who have subscribed to a premium service, or you might want to reserve an area of your site for remote business partners or employees. A few different authentication schemes are available, depending on the Web server you are using. Table 2.6 lists the schemes currently available.
Table 2.6 Authentication Schemes
Name | Details |
Anonymous | The user is not authenticated. A user name or password is not requested from the user agent asking for resources from the server. Information published is available to anyone who has the URL. |
Basic | The user agent must authenticate itself with a user name and password for each realm. Basic authentication encodes the user name and password with base64 encoding. This is considered a non-secure method and should be treated accordingly. |
NT Challenge/Response | The user agent must authenticate itself with a user name and password for each realm. NT Challenge/Response (NTLM) (shown in Figure 2.10) encrypts the user name and password, providing a secure method of transferring the user name and password to the server. When possible, you should use NT Challenge/Response over basic authentication. The only HTTP server supporting NTLM is IIS. |
Microsoft's Internet Service Manager allows you to configure authentication options.
As we saw before, the HTTP protocol is stateless. Therefore, once a client has "logged onto" the HTTP server and an HTTP transaction is completed, the client is immediately "logged off." That is, the server retains no knowledge of the client or its permissions.
As you can see from Table 2.6, the Basic authentication scheme transmits the user name and password for each request. This again is because of the stateless nature of the HTTP protocol.
The server uses the user name and password in an attempt to authenticate the user and allow access to the secured areas. NT Challenge/Response works the same way but transfers the user name and password in encrypted form.
Let's take a look now at how an authenticated HTTP transaction works. We'll look at basic authentication and then at NTLM. First, our user agent requests the default document from the /secure-docs/ directory:
GET /secure-docs/ HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Accept-Language: en
UA-pixels: 1024x768
UA-color: color16
UA-OS: Windows NT
UA-CPU: x86
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0; Windows NT)
Host: 199.1.11.197
Connection: Keep-Alive
The Web server gets the request and determines that the area has been secured. The server does not get a user name and password, so the server responds with a 401 Access Denied response message:
HTTP/1.0 401 Access Denied
WWW-Authenticate: Basic realm="www.abcdef.com"
Content-Length: 24
Content-Type: text/html
Error: Access is Denied.
Because the server has sent a 401 Access Denied message, the Web browser displays a dialog box requesting a user name and password for this secured area. A typical authentication dialog is shown in Figure 2.11.
The 401 Access Denied response instructs the browser to display an authentication dialog box.
Once the user types in a user name and password, the Web client sends another request, as follows:
GET /secure-docs/ HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Accept-Language: en
UA-pixels: 1024x768
UA-color: color16
UA-OS: Windows NT
UA-CPU: x86
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0; Windows NT)
Host: 199.1.11.197
Connection: Keep-Alive
Authorization: Basic eW91IHZhc3RlZDp5b3VyIHRpbWUhDQo=
You can see that the client now includes an Authorization header that gives the realm name Basic, and the user name and password base64 encoded. Unencoded, this would reveal the user name and password separated by a colon.
The server checks its user database, determines the validity of the supplied user name and password, and sends back the requested URL if they are correct. If the user name and password are not correct, another 401 message is sent to the browser and the browser redisplays the authentication dialog box.
This continues until the user supplies a correct user name and password combination, or the user presses the Cancel button, in which case the browser displays an "Error: Access is Denied message on the Web browser screen.
NT challenge/response security works roughly the same way. The initial GET request is made with the Authorization header. The server determines that the URL is in a password- protected area and responds with a 401 error.
Because NTLM authentication is active on the server, the response header includes a WWW-Authenticate: NTLM header informing the client that NTLM authentication is the preferred method.
If the client allows NTLM, it uses NTLM to respond to the request. If it does not allow NTLM, the client falls back on the Basic authentication scheme.
HTTP/1.0 401 Access Denied
WWW-Authenticate: NTLM
Content-Length: 24
Content-Type: text/html
Error: Access is Denied.
Once we enter the information and press Enter, the Web browser reissues the request, as follows:
GET /secure-docs/ HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Accept-Language: en
UA-pixels: 1024x768
UA-color: color16
UA-OS: Windows NT
UA-CPU: x86
User-Agent: Mozilla/2.0 (compatible; MSIE 3.0; Windows NT)
Host: 199.1.11.197
Connection: Keep-Alive
Authorization: NTLM T2ACMM7ARUAABAAAAA7IAAAABgAGAUJASTO5Ww
As you can see, the Authorization header is included in the request header. This shows the server that NTLM will be used to authenticate the user. The server decrypts the user name and password data, compares it to the database, and responds accordingly.
One other interesting thing to note about the encrypted user name and password is that under NTLM they encrypted information changes with each request, whereas the base64'd method leaves the user name and password the same, regardless of how many times it is transferred.
The definitive source of information on Internet protocols and standards is a series of documents called Request For Comments (RFC). Information on everything from HTTP to simple mail transfer protocol (SMTP) to netiquette guidelines can be found in the RFCs. Table 2.7 lists a number of RFCs that ISAPI/Web developers will find useful.
A searchable index of RFCs is available at http://ds.internic.net/ds/dspg1intdoc.html.
Table 2.7 Important Internet RFCs
RFC | Location | Description |
1808 | http://ds.internic.net/rfc/rfc1808.txt | Relative Uniform Resource Locators |
1630 | http://ds.internic.net/rfc/rfc1630.txt | Universal Resource Identifiers in WWW |
1738 | http://ds.internic.net/rfc/rfc1738.txt | Uniform Resource Locators (URL) |
1736 | http://ds.internic.net/rfc/rfc1736.txt | Functional Recommendations for Internet Resource Locators |
1866 | http://ds.internic.net/rfc/rfc1866.txt | Hypertext Markup Language-2.0 |
1980 | http://ds.internic.net/rfc/rfc1980.txt | A Proposed Extension to HTML: Client-Side Image Maps |
1867 | http://ds.internic.net/rfc/rfc1867.txt | Form-Based File Upload in HTML |
1942 | http://ds.internic.net/rfc/rfc1942.txt | HTML Tables |
1945 | http://ds.internic.net/rfc/rfc1945.txt | Hypertext Transfer Protocol-HTTP/1.0 |
1738 | http://ds.internic.net/rfc/rfc1738.txt | Uniform Resource Locators (URLs) |
822 | http://ds.internic.net/rfc/rfc822.txt | Standard for the Format of ARPA Internet Text Messages |
1341 | http://ds.internic.net/rfc/rfc1341.txt | MIME (Multipurpose Internet Mail Extensions) |
1896 | http://ds.internic.net/rfc/rfc1896.txt | The text/enriched MIME Content-type |
1872 | http://ds.internic.net/rfc/rfc1872.txt | The MIME Multipart/Related Content-type |
1180 | http://ds.internic.net/rfc/rfc1180.txt | A TCP/IP Tutorial |
1034 | http://ds.internic.net/rfc/rfc1034.txt | Domain Names-- Concepts and Facilities |
1035 | http://ds.internic.net/rfc/rfc1035.txt | Domain Names-- Implementation and Specification |
1794 | http://ds.internic.net/rfc/rfc1794.txt | DNS Support for Load Balancing |
1983 | http://ds.internic.net/rfc/rfc1983.txt | Internet Users' Glossary |
1796 | http://ds.internic.net/rfc/rfc1796.txt | Not All RFCs are Standards |
Web Resources
There are a number of other resources you should become familiar with. Many of these are located at the home of the Web, the World Wide Web Consortium (W3C). See Table 2.8 for a list of these resources.
Table 2.8 Web Developer Resources
Location | Description |
http://www.w3.org/ | World Wide Web Consortium (W3C) |
http://www.w3.org/pub/WWW/Protocols/ | HTTP Protocol Specifications |
http://www.w3.org/pub/WWW/MarkUp/ | HTML Specifications |
http://www.mcp.com/general/workshop/ | The HTML Workshop |
By now you should have a good idea how the Web works and how the HTTP protocol works. We've seen how browser cookies can help overcome the fact that HTTP is a stateless protocol and we've looked at how the HTTP authentication process takes place.
For more information on topics covered in this chapter: