This article covers some of the basics of log analysis, hitting on what we believe are the key points and techniques, so you too can analyzes your voluminous server logs.

Log Types

Logs fall into a number of different categories, based on their format, source and typical contents. I'm not going to list them all here, it would take up the rest of the article and probably the rest of the year, but we can generalize into a few key types.

• Content: Log content can be information, alerts and warnings, or fatal errors. The access logs in Apache and IIS are examples of informational logs. Alerts, warnings, and fatal errors are typically lumped together into a single 'error' log (which is essentially what the syslog is within Unix), or may be further split into specific types of errors or sources (in the style of the Event Log system in Windows). In some cases, all log information is dumped together into a single file, and it's the file content that helps describe what a particular entry is referring to.

• Source: Logs come from everything — from applications and the system to drivers and libraries. The source is used as a method of classification. For example, the security subsystems may have their own log, or their log can define where and how information is updated. System logs are generated and handled by the operating system; application logs may be with the application, in a central location with the system logs, or in a temporary location.

• Format: Logs can be in either a text or binary format. Not surprisingly, text is the more popular format because from the developer's and reader's perspectives, it is the easiest to work with. Binary is generally impossible to read without some form of processing, but information in a binary log may be better formatted and can use specific and structured data types for elements such as dates, times, and classification. This makes it easier to parse (provided the format is known) because complex assumptions or judgements on what the content might contain don't need to be made. Dates and times are examples of binary friendly data, but in a text file, they must be processed to identify them as recognizable, usable date.

Regardless of the originating format, location, and content, to get any useful information out of the logs they must be processed so each log entry is identifiable as well as each of the entry fields that make up the information.

Often, the log format is predetermined. There are standards for syslog, HTTP logs, and many others. However, if you are lucky, you can also change the format of the output log within certain applications. This lets you customize the contents of the log and the format of the contents, making it easier to post-process the contents. For example, the standard access log format within Apache 2.0 is configured with the following line in the configuration file:

LogFormat "%h %l %u %t \"%r\" %>s %b" common

However, it's completely configurable, so Apache can be set to create XML-like output by changing the above line to read:

LogFormat "<logitem><host>%h</host>
    <ident>%l</ident>
    <user>%u</user>
    <datetime>%t</datetime>
    <url>%r</url>
    <statuscode>%>s</statuscode>
    <bytes>%b</bytes>
    </logitem>" \
common

Note that these lines (and many of the examples throughout this article) have been formatted for clarity and should all be on the same physical line. You can, by the way, use this text to achieve the same result within IIS 6 and Windows Server 2003.