This HOW-TO written by Brian
Clifton with thanks to Karel
Kerezman
Originally written 2000-06-29. Last update 2002-01-04.
Analog is claimed as the most popular web logfile analyser in the world. (Details). Whether running multiple virtual hosts or a single root web server, a useful feature is to run Analog and then roll over both the logfile just analysed and the Analog report file. At the same time, the logfile can be compressed and the Analog report e-mailed to yourself or the virtual host client. This can be achieved quite simply using crontab and logrotate.
This example was developed and tested using a default install of RedHat 6.1 using Apache v1.3.9-8 and Analog v4.11. Also running on RedHat 7.0 using Apache 1.3.14. Please note, this is just one example and is not the only method of achieving the same goal!
Each week (or any set time period):
This results in the following files being created:
By this method combined_log.html has no meaning as upon creation it is immediately rotated. In this example, apache is using the 'combined' log format described in http.conf e.g.
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
#
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined<VirtualHost www.adomain.tld>
...
...
CustomLog /home/httpd/path_to_home_dir/logs_dir/name_of_log_file combined
</VirtualHost>
Each minute, the system crontab checks what jobs require scheduling. Scheduling is set in the etc/crontab file.
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
# column headings - thanks Toby
# mins, hr, date, month, day, command
# run-parts
# Min Hr Date Month Day Owner Command File
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
and what jobs are to be run is described in for example /etc/cron.weekly. In the above example, the directory /etc/cron.weekly is checked at 04:22 every Sunday morning.
My /etc/cron.weekly directory contains a :
logrotate
makewhatis.cron
slocate.cron
tmpwatch
The important file is logrotate:
#!/bin/sh
#added by bc 23/5/00 to rotate apache logs
/usr/bin/analog -G +g/home/httpd/path_to analogue_cfg file/vdomain.cfg
/usr/sbin/logrotate /etc/logrotate.conf
The third line runs Analog for a virtual host. The last line does the rotation. logrotate.conf contains:
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# send errors to root
errors your@emailaddress
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own lastlog or wtmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
rotate 1
}
/var/log/lastlog {
monthly
rotate 1
}
# system-specific logs may be configured here
# Added by BC 22/5/00
# rotate log file:
/home/httpd/company-domains.net/logs/combined_log {
ifempty
copytruncate
rotate 4
weekly
mailfirst
#mail your@emailaddress
errors your@emailaddress
compress
postrotate
/usr/bin/killall -HUP httpd
endscript
}
# rotate Analog report:
/home/httpd/company-domains.net/logs/combined_log.html {
ifempty
copytruncate
rotate 4
weekly
mailfirst
mail your@emailaddress
errors your@emailaddress
nocompress
}
Note the first part of this file (up to # Added by BC 22/5/00) sets default parameters. Below my comment, parameters over-ride the defaults. One caveat of the default parameters is:
# send errors to root
errors your@emailaddress
This does not work as etc/crontab has: MAILTO=root which over-rides that set in logrotate.conf
The part that does the rotating/compressing/e-mailing, follows the comment:
# system-specific logs may be configured here
# Added by BC 22/5/00
Read man logrotate for details concerning what options may be useful to you. Another caveat is that the man file appears to indicate the order of the commands is un-important. For example (RedHat 6.1):
nocompress
for /var/log/news/* appears after endscript. However changing this to compress will not work. It must come above postrotate.