Advanced Logging Techniques With Apache Page 2
Not all log entries are useful or interesting. For example, you might want to ignore all the log entries the testing department generates but continue recording all user errors. Or, you might want to ignore everything but HTML accesses in the logs, therefore ignoring images and other sundry items that might make up your page.
Logging in Apache
Creating Custom Log Formats
Subdividing the Logs
Logging Directly to a Database or Application
Speeding up the Logging Process
You can ignore these elements when you are processing the logs, but additional processes are then required. It's also worth remembering that anything recorded and not used carries CPU and disk overhead. For example, a single page with 20 images on it will generate 21 entries in the log a lot of additional work and storage space for something that will be ignored.
When using conditional logging, the environment variable system sets a variable based on the request. The CustomLog directive accepts an environment condition that will be applied to the CustomLog configured. For example, let's say you wanted to log requests for only .html or .shtml files from within the Web server. First, create an environment rule specifying which files to include in the logs. For example, to set up an environment rule for our html/shtml files:
SetEnvIf Request_URI "(\.html|\.shtml)$" html
Now, apply this environment variable setting to the CustomLog directive:
CustomLog logs/access.log common env=html
To reject some files, for example to include everything except image files, create the rule the same way:
SetEnvIf Request_URI "(\.gif|\.png|\.jpg)$" image
But use an exclamation mark before the environment rule in the CustomLog directive to exclude those matching the environment variable that has been set:
CustomLog logs/access.log common env=!image
The SetEnvIf directive is quite extensive. More information about using the directive is found in the online Apache HTTPD documentation.
The effects achievable with this are quite extensive. On some sites, for example, different departments (as identified by the first directory in the URI) may be written to special directories is available only to each department:
SetEnvIf Request_URI "^/weather/" weather_dept SetEnvIf Request_URI "^/news/" news_dept CustomLog /groups/weather/www-access_log common env=weather_dept CustomLog /groups/news/www-access_log common env=news_dept
Conditional logs can restrict or split the information however you want. But keep processing overhead in mind.