Advanced Logging Techniques With Apache Page 2

Not all log entries are useful or interesting. For example, you might want to ignore all the log entries the testing department generates but continue recording all user errors. Or, you might want to ignore everything but HTML accesses in the logs, therefore ignoring images and other sundry items that might make up your page.

You can ignore these elements when you are processing the logs, but additional processes are then required. It's also worth remembering that anything recorded and not used carries CPU and disk overhead. For example, a single page with 20 images on it will generate 21 entries in the log — a lot of additional work and storage space for something that will be ignored.

When using conditional logging, the environment variable system sets a variable based on the request. The CustomLog directive accepts an environment condition that will be applied to the CustomLog configured. For example, let's say you wanted to log requests for only .html or .shtml files from within the Web server. First, create an environment rule specifying which files to include in the logs. For example, to set up an environment rule for our html/shtml files:

SetEnvIf Request_URI "(\.html|\.shtml)$" html

Now, apply this environment variable setting to the CustomLog directive:

CustomLog logs/access.log common env=html

To reject some files, for example to include everything except image files, create the rule the same way:

SetEnvIf Request_URI "(\.gif|\.png|\.jpg)$" image

But use an exclamation mark before the environment rule in the CustomLog directive to exclude those matching the environment variable that has been set:

CustomLog logs/access.log common env=!image

The SetEnvIf directive is quite extensive. More information about using the directive is found in the online Apache HTTPD documentation.

The effects achievable with this are quite extensive. On some sites, for example, different departments (as identified by the first directory in the URI) may be written to special directories is available only to each department:

      
SetEnvIf Request_URI "^/weather/" weather_dept
SetEnvIf Request_URI "^/news/" news_dept
CustomLog /groups/weather/www-access_log common env=weather_dept
CustomLog /groups/news/www-access_log common env=news_dept

Conditional logs can restrict or split the information however you want. But keep processing overhead in mind.

>> Subdividing the Logs