Click Here!
home account info subscribe login search My ITKnowledge FAQ/help site map contact us


 
Brief Full
 Advanced
      Search
 Search Tips
To access the contents, click the chapter and section titles.

Perl CGl Programming: No experience required.
(Publisher: Sybex, Inc.)
Author(s): Erik Strom
ISBN: 0782121578
Publication Date: 11/01/97

Bookmark It

Search this book:
 
Previous Table of Contents Next


Using Perl to Decode a Server Log

Perl has so many facilities built into it for dealing with text—especially if the text is set up in a consistent way—that there is very little reason to even attempt to take a Web server log file apart in any other language.

First of all, if you know what the format of the log file is (and you certainly should, on your own server!), you can break a line of log entries into individual fields and store them in one line of Perl code. Let’s use the example in the previous section from the Internet Information Server logs:

   152.163.195.39, -, 6/13/97, 21:37:00, W3SVC, OWSLEY, 207.77.84.202,
   ⇒ 328, 60, 29, 304, 0, GET, /wml/homepage.gif, -,

Because the entries are separated by commas, we can use Perl’s split function to put each entry in its own variable.

Remember from the previous section that the entries are:

  IP address of the client
  Client’s user name, which is always empty (-) in IIS
  Date of request
  Time of request
  Service name (always W3SVC in IIS)
  Web server’s name (computer name)
  Server IP address
  Elapsed CPU time (in milliseconds) of the operation
  Number of bytes received
  Number of bytes sent
  Service status code
  Windows NT status code
  Operation requested
  Target of the operation
  Ender (always -)

The most difficult task in creating a Perl program to extract these fields is thinking up variable names for all of them. The critical thing to remember is that the order of the variables in the call to split must match the order of entries in the log file line. These lines of Perl code will do it:

   ($ClientIP, $Dummy, $DateTime, $SvcName, $SrvrName, $SrvrIP, $CPUTime,
           $BytesRecv, $BytesSent, $SvcStatus, $NTStatus, $Operation,
           $Target, $Dummy) = split (/,/, $LogLine);

Remember that split parses the string passed to it for the pattern between the slash characters (//) in its first argument and uses the pattern as a way to break the string down into components separated by the pattern. In this example, we’re looking for the comma as a delimiter in the $LogLine string. Any time that split encounters a comma, it will load the text it has read previously into the next variable in the list on the left side of the equation.

Let’s turn the example into a snippet of executable code by:

1.  Loading the example log file line into a local variable.
2.  Processing the variable with split, extracting each field into its own variable.
3.  Printing the individual values to the screen.

Enter the following code into your text editor:

   #!perl/bin/perl

   # Hard-code an example log line into a local variable.

       $LogLine = "152.163.195.39, -, 6/13/97, 21:37:00, W3SVC"
                   .", OWSLEY, 207.77.84.202, 328, 60, 29, 304, 0"
                   .", GET, /wml/homepage.gif, -";

   # Extract the components using split()

       ($ClientIP, $Dummy, $Date, $Time, $SvcName, $SrvrName, $SrvrIP,
           $CPUTime, $BytesRecv, $BytesSent, $SvcStatus, $NTStatus,
           $Operation, $Target, $Dummy) = split (/,/, $LogLine);

   # Print the values to the screen.
       print "Client’s IP address = $ClientIP\n";
       print "Date of request = $Date\n";
       print "Time of request = $Time\n";
       print "Service name = $SvcName\n";
       print "Server name = $SrvrName\n";
       print "Server’s IP address = $SrvrIP\n";
       print "Processing time = $CPUTime\n";
       print "Received $BytesRecv bytes of data\n";
       print "Sent $BytesSent bytes of data\n";
       print "Server returned status of $SvcStatus\n";
       print "Windows NT returned status code $NTStatus\n";
       print "Operation requested = $Operation\n";
       print "Target of operation = $Target\n";

   #                    End logs.pl

Save this script as logs.pl and run it from the command line. Your output should look similar to Figure 9.5.


Figure 9.5:  An example of a simple extraction from a log file line

Dissecting the Example Code

You probably noticed one aspect of our rudimentary log entry extraction program immediately: it’s ugly, at least in the way it prints the information. Don’t worry: You’ll learn how to pretty it up shortly.

Meanwhile, let’s discuss the program we have so far.

In the code line that calls split, notice that two variables are called $Dummy:

   # Extract the components using split()

       ($ClientIP, $Dummy, $Date, $Time, $SvcName, $SrvrName, $SrvrIP,
           $CPUTime, $BytesRecv, $BytesSent, $SvcStatus, $NTStatus,
           $Operation, $Target, $Dummy) = split (/,/, $LogLine);

Recall from the definition of an IIS log file entry line in the previous section that two fields are always empty and therefore designated with hyphens: the client’s user name and the last entry in the line. The variable $Dummy is actually being filled twice, which means that after split is finished, $Dummy will contain the value of the last field put in it. It really doesn’t matter, because we intend to ignore whatever the variable contains. That’s why it’s called $Dummy.


TIP:  Because of the way split works, the second declaration of $Dummy in the list could have been left out entirely. The processing of the $LogLine string would have simply stopped when split ran out of variables in which to place the extracted fields.

The logs.pl script introduces another new Perl concept, too: the concatenation operator, or dot (.). Notice the line in which $LogFile is given a string value:

   # Hard-code an example log line into a local variable.

       $LogLine = "152.163.195.39, -, 6/13/97, 21:37:00, W3SVC"
   → ", OWSLEY, 207.77.84.202, 328, 60, 29, 304, 0"
   → .", GET, /wml/homepage.gif, -";


Previous Table of Contents Next


Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc.
All rights reserved. Reproduction whole or in part in any form or medium without express written permission of EarthWeb is prohibited.