Advanced Logging Techniques With Apache

Logging in Apache

When was the last time you seriously thought about the content of your Web logs — Not processing the logs and producing statistics, but the actual raw content and information the logs store?

Many people will never choose a format beyond the standard common log format most systems use. They will generate logs containing all the requests, errors, and other information. This can be wasteful on resources — both in terms of Apache reporting the information and the disk space required to store it. There are, however, ways in which you can simplify and customize the logging system to suit your needs.

Creating Custom Log Formats

Both versions of Apache allow you to create custom log formats with only the fields you want using the format and layout that you desire.

The custom log format system enables the clear definition of exactly which fields of information to include in the log and the surrounding format. This means you can use spaces, tabs, or other delimiters to specify the format of each line written to the file.

Individual fields within the log format are specified using a special format string. For example, %a is replaced in the actual log with the IP address of the remote host. For a full list of the fields available, see the Apache mod_log_config documentation.

To specify a format, use the LogFormat directive. For example, the Common Log Format would be set using the directive

LogFormat "%h %l %u %t \"%r\" %>s %b" common

The actual format of the information (i.e., the information surrounding the fields you include) is entirely up to you. For example, the following log format generates a log using an XML structure. It is formatted for clarity — the actual definition would have to be within a single line, but the embedded \n character will generate a new line when the log is written.

<programlisting><![CDATA[LogFormat "<logitem><host>%h</host>\n
<ident>%l</ident>\n
<user>%u</user>\n
<datetime>%t</datetime>\n
<url>%r</url>\n
<statuscode>%>s</statuscode>\n
<bytes>%b</bytes>\n</logitem>" commonxml

Formats can be given a name (as they are in both of the examples above), and we can use it when specifying a log for a host or virtual host to request the desired log format:

CustomLog logs/access_log common

It's possible to create a variety of log formats and for the individual logs to contain different information, so we can spread access, errors, scripts, referrer, and other information across a range of different files. We can also be selective about what information appears in each file by using the conditional logging system.

>> Conditional Logging