Advanced Logging Techniques With Apache Page 2
Download the authoritative guide: Data Center Guide: Optimizing Your Data Center Strategy
Download the authoritative guide: Cloud Computing: Using the Cloud for Competitive Advantage
Conditional Logging
Not all log entries are useful or interesting. For example, you might want to ignore all the log entries the testing department generates but continue recording all user errors. Or, you might want to ignore everything but HTML accesses in the logs, therefore ignoring images and other sundry items that might make up your page.
Contents Logging in Apache Creating Custom Log Formats Conditional Logging Subdividing the Logs Logging Directly to a Database or Application Speeding up the Logging Process |
You can ignore these elements when you are processing the logs, but additional processes are then required. It's also worth remembering that anything recorded and not used carries CPU and disk overhead. For example, a single page with 20 images on it will generate 21 entries in the log a lot of additional work and storage space for something that will be ignored.
When using conditional logging, the environment variable system sets a variable based on the request. The CustomLog directive accepts an environment condition that will be applied to the CustomLog configured. For example, let's say you wanted to log requests for only .html or .shtml files from within the Web server. First, create an environment rule specifying which files to include in the logs. For example, to set up an environment rule for our html/shtml files:
SetEnvIf Request_URI "(\.html|\.shtml)$" html |
Now, apply this environment variable setting to the CustomLog directive:
CustomLog logs/access.log common env=html |
To reject some files, for example to include everything except image files, create the rule the same way:
SetEnvIf Request_URI "(\.gif|\.png|\.jpg)$" image |
But use an exclamation mark before the environment rule in the CustomLog directive to exclude those matching the environment variable that has been set:
CustomLog logs/access.log common env=!image |
The SetEnvIf directive is quite extensive. More information about using the directive is found in the online Apache HTTPD documentation.
The effects achievable with this are quite extensive. On some sites, for example, different departments (as identified by the first directory in the URI) may be written to special directories is available only to each department:
SetEnvIf Request_URI "^/weather/" weather_dept SetEnvIf Request_URI "^/news/" news_dept CustomLog /groups/weather/www-access_log common env=weather_dept CustomLog /groups/news/www-access_log common env=news_dept |
Conditional logs can restrict or split the information however you want. But keep processing overhead in mind.