How not to log certain requests in Apache

By default Apache logs all requests. Sometimes you need to exclude logging of certain requests from Apache‘s access.log. For example some automated requests coming and flood your log distorting your statistics. This short guide shows how you use Apache’s SetEnvIf directive to prevent Apache from logging such requests.

The SetEnvIf directive can be used in the following contexts in your Apache configuration: in the global Apache configuration (if the directive should be valid for the whole server), in vhost configurations (if the directive should be valid only for that specific vhost), between (if the directive should be valid only for a certain directory and its subdirectories), and in .htaccess files (AllowOverride FileInfo must be set).

With SetEnvIf, you can prevent requests from getting logged based on the following criteria:

• Host
• User-Agent
• Referer
• Accept-Language
• Remote_Host: the hostname (if available) of the client making the request.
• Remote_Addr: the IP address of the client making the request.
• Server_Addr: the IP address of the server on which the request was received (only with versions later than 2.0.43).
• Request_Method: the name of the method being used (GET, POST, etc.).
• Request_Protocol: the name and version of the protocol with which the request was made (e.g., “HTTP/0.9”, “HTTP/1.1”, etc.).
• Request_URI: the resource requested on the HTTP request line – generally the portion of the URL following the scheme and host portion without the query string.

The SetEnvIf directive has the following form:

where attribute is one of the criteria I’ve just mentioned, and regex is a Perl compatible regular expression.

Now let’s assume that Monit is requesting the file /monit/token once a minute to check if Apache is still running. Obviously, we don’t want to log these requests because they are not from a real user. Therefore we use the following SetEnvIf directive:

^ means that the Request_URI must begin with /monit/token, $ means that it must also end with /monit/token (so only /monit/token matches this regular expression). If we used “^/monit/token”, any URL beginning with /monit/token would match the regular expression, e.g. /monit/token/example.html; “/monit/token$” would match any URL ending in /monit/token, e.g. /example/monit/token.

Now we have an iframe in /iframe/iframe.html that we don’t want to log either. This is what we’d use:

Now we must tell Apache that it must not log all requests labeled with dontlog. Find the CustomLog directive in your Apache configuration, e.g.

or

and add env=!dontlog to the line:

or

Restart Apache afterwards. Now it won’t log any request anymore that is labelled with dontlog.

To prevent all requests made with a certain browser, e.g. Internet Explorer, from getting logged, you could use:

To not log requests from any client whose hostname ends in bla.example.com, use:

To not log requests from any client whose hostname begins with example, use:

To not log requests from a certain IP address, use something like:

If you don’t want requests of your robots.txt to get logged, use:

Apart from SetEnvIf, which is case-sensitive, you can use SetEnvIfNoCase which is case-insensitive.

For example, in order not to log certain search engine spiders, you could use:

Or to not log certain file extensions, use something like this:

To not log certain referrals (e.g. from your own domain), use something like:

Good luck!