Logrotate tutorial

If you try to generalize this to multiple logs you might need to use the sharedscripts directive to restart particular daemon once. See, for example, default version of /etc/logrotate.d/syslog script

This directive means that the postrotate script will only be run once (after the old logs are processed), not once for each log which is rotated.

Normal shell quoting rules apply within postrotate and endscript brackets, with ', ", and \ characters supported.

Specifying bzip2 compression options

As we mentioned before there is great sense to use bzip2 for compression of large log file. This can be achieved by using statements

compresscmd /bin/bzip2
compressext .bz2>

For example:

/var/log/cron {
   size 10M
   copytruncate
   create
   compress
   compresscmd /bin/bzip2
   compressext .bz2
   rotate 4
}

Examples of complete configuration files

Typically you do not modify "master" configuration file and just add "slices" to the directory logrotate.d.

Here is simple configuration from manpage, which has only historical interest and does not represent "state of the art" of writing logrotate configuration files:

# sample logrotate configuration file
compress

/var/log/messages {
    rotate 5
    weekly
    postrotate
        /usr/bin/killall -HUP syslogd
    endscript
}

"/var/log/httpd/access.log" /var/log/httpd/error.log {
    rotate 5
    mail [email protected]
    size 100k
    sharedscripts
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
}

/var/log/news/* {
    monthly
    rotate 2
    olddir /var/log/news/old
    missingok
    postrotate
        kill -HUP 'cat /var/run/inn.pid'
    endscript
    nocompress
}

The next section of the config files defined how to handle the log file /var/log/messages. The log will go through five weekly rotations before being removed. After the log file has been rotated (but before the old version of the log has been compressed), the command /sbin/killall -HUP syslogd will be executed.

The next section defines the parameters for both /var/log/httpd/access.log and /var/log/httpd/error.log. They are rotated whenever it grows over 100k in size, and the old logs files are mailed (uncompressed) to [email protected] after going through 5 rotations, rather than being removed.

The sharedscripts means that the postrotate script will only be run once (after the old logs have been compressed), not once for each log which is rotated. Note that the double quotes around the first filename at the beginning of this section allows logrotate to rotate logs with spaces in the name. Normal shell quoting rules apply, with ', ", and \ characters supported.

The last section defines the parameters for all of the files in /var/log/news. Each file is rotated on a monthly basis. This is considered a single rotation directive and if errors occur for more than one file, the log files are not compressed.

Please use wildcards with caution. If you specify *, logrotate will rotate all files, including previously rotated ones. A way around this is to use the olddir directive or a more exact wildcard (such as *.log).

See also

• Log rotation in RHEL/Centos/Oracle linux
• SLES Log rotation

Putting it in cron

You need to create a cron job to actually make logrotate work. For example:

00 03 * * * /usr/sbin/logrotate -s /var/tmp/foobar.logrotate.status -f /etc/logrotate/logrotate.conf

In this example the command runs at 3 AM everyday. The -s /var/tmp/foobar.logrotate.status option specifies where logrotate keeps its status information. This file has to be writeable by the user running the cron. The last argument /etc/logrotate/logrotate.conf illustrates the ability of logrotate to use custom location of the configuration file. It can be useful for testing.

Reference

Normally, logrotate is run as a daily cron job. It will not modify a log multiple times in one day unless the criterion for that log is based on the log's size and logrotate is being run multiple times each day, or unless the -f or --force option is used.

Any number of config files may be given on the command line. Later config files may override the options given in earlier files, so the order in which the logrotate config files are listed is important. Normally, a single config file which includes any other config files which are needed should be used. See below for more information on how to use the include directive to accomplish this. If a directory is given on the command line, every file in that directory is used as a config file.

If no command line arguments are given, logrotate will print version and copyright information, along with a short usage summary. If any errors occur while rotating logs, logrotate will exit with non-zero status.

Options

-d, --debug

Turns on debug mode and implies -v. In debug mode, no changes will be made to the logs or to the logrotate state file.

-f, --force

Tells logrotate to force the rotation, even if it doesn't think this is necessary. Sometimes this is useful after adding new entries to a logrotate config file, or if old log files have been removed by hand, as the new files will be created, and logging will continue correctly.

-m, --mail <command>

Tells logrotate which command to use when mailing logs. This command should accept two arguments: 1) the subject of the message, and 2) the recipient. The command must then read a message on standard input and mail it to the recipient. The default mail command is /bin/mail -s.

-s, --state <statefile>

Tells logrotate to use an alternate state file. This is useful if logrotate is being run as a different user for various sets of log files. The default state file is /var/lib/logrotate.status.

--usage

Prints a short usage message.

--?, --help

-Prints help message.

-v, --verbose

Turns on verbose mode.

Configuration File

logrotate reads everything about the log files it should be handling from the series of configuration files specified on the command line. Each configuration file can set global options (local definitions override global ones, and later definitions override earlier ones) and specify logfiles to rotate. A simple configuration file looks like this:

# sample logrotate configuration file
compress

/var/log/messages {
    rotate 5
    weekly
    postrotate
        /usr/bin/killall -HUP syslogd
    endscript
}

"/var/log/httpd/access.log" /var/log/httpd/error.log {
    rotate 5
    mail [email protected]
    size 100k
    sharedscripts
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
}

/var/log/news/* {
    monthly
    rotate 2
    olddir /var/log/news/old
    missingok
    postrotate
        kill -HUP 'cat /var/run/inn.pid'
    endscript
    nocompress
}

The next section of the config files defined how to handle the log file /var/log/messages. The log will go through five weekly rotations before being removed. After the log file has been rotated (but before the old version of the log has been compressed), the command /sbin/killall -HUP syslogd will be executed.

The next section defines the parameters for both /var/log/httpd/access.log and /var/log/httpd/error.log. They are rotated whenever it grows over 100k in size, and the old logs files are mailed (uncompressed) to [email protected] after going through 5 rotations, rather than being removed. The sharedscripts means that the postrotate script will only be run once (after the old logs have been compressed), not once for each log which is rotated. Note that the double quotes around the first filename at the beginning of this section allows logrotate to rotate logs with spaces in the name. Normal shell quoting rules apply, with ', ", and \ characters supported.

The last section defines the parameters for all of the files in /var/log/news. Each file is rotated on a monthly basis. This is considered a single rotation directive and if errors occur for more than one file, the log files are not compressed.

Please use wildcards with caution. If you specify *, logrotate will rotate all files, including previously rotated ones. A way around this is to use the olddir directive or a more exact wildcard (such as *.log).

Here is more information on the directives which may be included in a logrotate configuration file:

compresscmd

Specifies which command to use to compress log files. The default is gzip. See also compress.

uncompresscmd

Specifies which command to use to uncompress log files. The default is gunzip.

compressext

Specifies which extension to use on compressed logfiles, if compression is enabled. The default follows that of the configured compression command.

compressoptions

Command line options may be passed to the compression program, if one is in use. The default, for gzip(1), is "-9" (maximum compression).

copy

Make a copy of the log file, but don't change the original at all. This option can be used, for instance, to make a snapshot of the current log file, or when some other utility needs to truncate or parse the file. When this option is used, the create option will have no effect, as the old log file stays in place.

copytruncate

Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one. It can be used when some program cannot be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.

create mode owner group

Immediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated). mode specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and group specifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreate option.

daily Log files are rotated every day.

dateext

Archive old versions of log files adding a daily extension like YYYYMMDD instead of simply adding a number. The extension may be configured using the dateformat option.

dateformat format_string

Specify the extension for dateext using the notation similar to strftime(3) function. Only %Y %m %d and %s specifiers are allowed. The default value is -%Y%m%d. Note that also the character separating log name from the extension is part of the dateformat string. The system clock must be set past Sep 9th 2001 for %s to work correctly. Note that the datestamps generated by this format must be lexically sortable (i.e., first the year, then the month then the day. e.g., 2001/12/01 is ok, but 01/12/2001 is not, since 01/11/2002 would sort lower while it is later). This is because when using the rotate option, logrotate sorts all rotated filenames to find out which logfiles are older and should be removed.

delaycompress

Postpone compression of the previous log file to the next rotation cycle. This only has effect when used in combination with compress. It can be used when some program cannot be told to close its logfile and thus might continue writing to the previous log file for some time.

extension ext

Log files with ext extension can keep it after the rotation. If compression is used, the compression extension (normally .gz) appears after ext. For example you have a logfile named mylog.foo and want to rotate it to mylog.1.foo.gz instead of mylog.foo.1.gz.

ifempty

Rotate the log file even if it is empty, overriding the notifempty option (ifempty is the default).

include file_or_directory

Reads the file given as an argument as if it was included inline where the include directive appears. If a directory is given, most of the files in that directory are read in alphabetic order before processing of the including file continues. The only files which are ignored are files which are not regular files (such as directories and named pipes) and files whose names end with one of the taboo extensions, as specified by the tabooext directive. The include directive may not appear inside a log file definition.

mail address

When a log is rotated out-of-existence, it is mailed to address. If no mail should be generated by a particular log, the nomail directive may be used.

mailfirst

When using the mail command, mail the just-rotated file, instead of the about-to-expire file.

maillast

When using the mail command, mail the about-to-expire file, instead of the just-rotated file (this is the default).

maxage count

Remove rotated logs older than <count> days. The age is only checked if the logfile is to be rotated. The files are mailed to the configured address if maillast and mail are configured.

minsize size

Log files are rotated when they grow bigger than size bytes, but not before the additionally specified time interval (daily, weekly, monthly, or yearly). The related size option is similar except that it is mutually exclusive with the time interval options, and it causes log files to be rotated without regard for the last rotation time. When minsize is used, both the size and timestamp of a log file are considered.

missingok

If the log file is missing, go on to the next one without issuing an error message. See also nomissingok.

monthly

Log files are rotated the first time logrotate is run in a month (this is normally on the first day of the month).

nocompress

Old versions of log files are not compressed. See also compress.

nocopy Do not copy the original log file and leave it in place. (this overrides the copy option).

nocopytruncate

Do not truncate the original log file in place after creating a copy (this overrides the copytruncate option).

nocreate

New log files are not created (this overrides the create option).

nodelaycompress

Do not postpone compression of the previous log file to the next rotation cycle (this overrides the delaycompress option).

nodateext

Do not archive old versions of log files with date extension (this overrides the dateext option).

nomail Don't mail old log files to any address.

nomissingok

If a log file does not exist, issue an error. This is the default.

noolddir

Logs are rotated in the same directory the log normally resides in (this overrides the olddir option).

nosharedscripts

Run prerotate and postrotate scripts for every log file which is rotated (this is the default, and overrides the sharedscripts option). The absolute path to the log file is passed as first argument to the script. If the scripts exit with error, the remaining actions will not be executed for the affected log only.

noshred

Do not use shred when deleting old log files. See also shred.

notifempty

Do not rotate the log if it is empty (this overrides the ifempty option).

olddir directory

Logs are moved into directory for rotation. The directory must be on the same physical device as the log file being rotated, and is assumed to be relative to the directory holding the log file unless an absolute path name is specified. When this option is used all old versions of the log end up in directory. This option may be overridden by the noolddir option.

postrotate/endscript

The lines between postrotate and endscript (both of which must appear on lines by themselves) are executed (using ) after the log file is rotated. These directives may only appear inside a log file definition. Normally, the absolute path to the log file is passed as first argument to the script. If sharedscripts is specified, whole pattern is passed to the script. See also prerotate. See sharedscripts and nosharedscripts for error handling.

prerotate/endscript

The lines between prerotate and endscript (both of which must appear on lines by themselves) are executed (using ) before the log file is rotated and only if the log will actually be rotated. These directives may only appear inside a log file definition. Normally, the absolute path to the log file is passed as first argument to the script. If sharedscripts is specified, whole pattern is passed to the script. See also postrotate. See sharedscripts and nosharedscripts for error handling.

firstaction/endscript

The lines between firstaction and endscript (both of which must appear on lines by themselves) are executed (using ) once before all log files that match the wildcarded pattern are rotated, before prerotate script is run and only if at least one log will actually be rotated. These directives may only appear inside a log file definition. Whole pattern is passed to the script as first argument. If the script exits with error, no further processing is done. See also lastaction.

lastaction/endscript

The lines between lastaction and endscript (both of which must appear on lines by themselves) are executed (using ) once after all log files that match the wildcarded pattern are rotated, after postrotate script is run and only if at least one log is rotated. These directives may only appear inside a log file definition. Whole pattern is passed to the script as first argument. If the script exits with error, just an error message is shown (as this is the last action). See also firstaction.

rotate count

Log files are rotated count times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather than rotated.

size size

Log files are rotated only if they grow bigger then size bytes. If size is followed by k, the size is assumed to be in kilobytes. If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100Gare all valid.

sharedscripts

Normally, prerotate and postrotate scripts are run for each log which is rotated and the absolute path to the log file is passed as first argument to the script. That means a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscripts is specified, the scripts are only run once, no matter how many logs match the wildcarded pattern, and whole pattern is passed to them. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. If the scripts exit with error, the remaining actions will not be executed for any logs. This option overrides the nosharedscripts option and implies create option.

shred

Delete log files using shred -u instead of unlink(). This should ensure that logs are not readable after their scheduled deletion; this is off by default. See also noshred.

shredcycles count

Asks GNU shred(1) to overwite log files count times before deletion. Without this option, shred's default will be used.

start count

This is the number to use as the base for rotation. For example, if you specify 0, the logs will be created with a .0 extension as they are rotated from the original log files. If you specify 9, log files will be created with a .9, skipping 0-8. Files will still be rotated the number of times specified with the count directive.

tabooext [+] list

The current taboo extension list is changed (see the include directive for information on the taboo extensions). If a + precedes the list of extensions, the current taboo extension list is augmented, otherwise it is replaced. At startup, the taboo extension list contains .rpmorig, .rpmsave, ,v, .swp, .rpmnew, ~, .cfsaved and .rhn-cfg-tmp-*.

weekly

Log files are rotated if the current weekday is less than the weekday of the last rotation or if more than a week has passed since the last rotation. This is normally the same as rotating logs on the first day of the week, but it works better if logrotate is not run every night.

yearly

Log files are rotated if the current year is not the same as the last rotation.

Files

/var/lib/logrotate.status

Default state file.

/etc/logrotate.conf

Configuration options.

NEWS CONTENTS

• 20120207 : Creating logfile archives with logrotate ( Creating logfile archives with logrotate, Feb 07, 2012 )
• 20120207 : Changing Apache log rotation behaviour on CentOS / RHEL ( March 22, 2008 )
• 20120207 : Manage Linux log files with Logrotate by Jim McIntyre ( December 27, 2000 , TechRepublic )
• 20120207 : Rotating Linux Log Files MDLog-sysadmin ( Rotating Linux Log Files MDLog-sysadmin, )
• 20120207 : logrotate notes on usage ( )
• 20120207 : Logrotate -- A Backup Solution by Sean D. Conway ( Sys Admin )

Old News ;-)

[Feb 07, 2012] Creating logfile archives with logrotate

Many of the services installed on Linux machines will produce logfiles which grow, and grow, and grow. If left unchecked you can easily fill a disk with a large collection of logfiles if you're not careful.

The most common method of keeping logfile growth in check is to use logrotate, and many Debian packages are setup to work with this by default.

The most obvious package which uses it is Apache, the webserver, which by default keeps its logfiles in the directory /var/log/apache (or /var/log/apache2).

If you examine this directory you will see that there are a bunch of logfiles which are archived:

root@mystery:~# ls -1 /var/log/apache2/
access.log
access.log.1
access.log.2.gz
access.log.3.gz
access.log.4.gz
access.log.5.gz
error.log
error.log.1
error.log.2.gz
error.log.3.gz
error.log.4.gz
error.log.5.gz

Here the current logfiles access.log, error.log are kept raw as are yesterday's logfiles (access.log.1 and error.log.1). Previous logfiles are compressed with gzip and only kept for five weeks. (I know it's five weeks and not five days because I've looked at the configuration - It's not clear from this output though!)

The process that is in charge of compressing and rotating these logfiles is called logrotate and it is executed once per day upon Debian installations.

As we saw when we were looking at scheduling commands with cron there is a directory called /etc/cron.daily which contains scripts which are executed once per day. Here you will find the logrotate driver script.

Every day this script runs and examines two things:

• The configuration file /etc/logrotate.conf
• The configuration directory /etc/logrotate.d

The latter is where most of our packages are configured. This directory contains configuration files which other packages have installed. For example if you install apache the file /etc/logrotate.d/apache will be installed.

Many servers such as exim the mailserver will install their own configuration file, and you can add your own.

A typical logrotate configuration file looks like this:

/var/log/apache/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if [ -f /var/run/apache.pid ]; then
                        /etc/init.d/apache restart > /dev/null
                fi
        endscript
}

You can see several important things here. The most obvious is the list of files that will be matched by this configuration file:

/var/log/apache/*.log {
...
}

After this we have a collection of configuration terms, a different one on each line. In the example above we have:

• weekly
  • The files should be rotated every week. Opposite: daily
• rotate nn
  • We should keep no more than nn files.
• compress
  • Compress older files with gzip. Opposite: nocompress
• delaycompress
  • Don't compress yesterdays files. Opposite: compress
• notifempty
  • Don't do any rotation if the logfile is empty. Opposite: ifempty
• create xx user group
  • If we have to create the new file give it the given mode, owner, and group.
• sharedscripts
  • Run any given prerotate or postrotate script for each logfile individually. Opposite: nosharedscripts.
• postrotate + endscript
  • Anything between these is executed after the rotation process. Opposite : prerotate

Hopefully that should have made sense!

The upshot of this script is that any file which matches /var/log/apache/*.log is rotated every week, compressed, if it's non-empty. The new file is created with the file mode of 640, and after the rotation has finished the server is restarted.

If we wish to install a local service which creates a logfile we can cause it to be rotated very easily, just by adding a new logrotate configuration file.

Assuming we have a new service "fred" which produces its output in /var/log/fred/output.log we can cause this to be rotated every day with a script like this:

/var/log/fred/*.log {
  daily
  missingok
  rotate 7
  compress
  delaycompress
  create 640 fred fred
  sharedscripts
     /etc/init.d/fred restart
  endscript
}

This will:

• Run daily.
• Keep no more than 7 days worth of logfiles at any one time.
• Not complain if there is a logfile missing.
• Compress the older files, but not yesterdays.
• Create the new logfiles as being owned by the user and group fred.
• Restart the service after rotating the logfiles.

Any of the existing files in the logrotate directory can be examined for more examples - and the manpage documents all the options you may use in a clear manner:

man logrotate

Changing Apache log rotation behaviour on CentOS / RHEL

March 22, 2008

On a default install of CentOS or Red Hat Enterprise Linux, the log rotation script will automatically rotate the Apache log file each day and then reload the httpd service. This post looks at how to prevent this action from occuring automatically, or to change the behaviour to rotate the log files if your naming convention for log files is different from the default.

The cron daemon on a CentOS or Red Hat Enterprise Linux server by default runs the scripts in the directory /etc/cron.daily on a daily basis. This includes running the "logrotate" script which runs /usr/sbin/logrotate passing it the /etc/logrotate.conf configuration file. The default version of this logrotate.conf file is as follows:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4



# create new (empty) log files after rotating old ones
create


# uncomment this if you want your log files compressed
#compress


# RPM packages drop log rotation information into this directory
include /etc/logrotate.d


# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    minsize 1M
    create 0664 root utmp
    rotate 1
}


# system-specific logs may be also be configured here.

These default variables are fairly straight forward and give you control over how frequently the log files are rotated, how many lots of previous log files to keep before deleting them, whether or not to compress them and so on. It also tells logrotate to process all the files in the directory /etc/logrotate.d

The file which controls log rotation of Apache on CentOS and Red Hat Enterprise Linux is named /etc/logrotate.d/httpd, and the default contents of this file are as follows:

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

What this file is saying, is to rotate all files that match the pattern /var/log/httpd/*.log. The default access and error log files in Apache on CentOS and RHEL are named "access_log" and "error_log" so they match this pattern.

If you have changed the default names of the log files and they don't match this pattern, then you can have logrotate rotate your log files by making sure the pattern is in this file in place of the /var/log/httpd/*.log line. For example, if your log files were named error.foo and access.foo then /var/log/httpd/*.foo would match them.

If you do not wish the logrotate script to rotate your Apache log files at all, then you can simply delete this file like so, logged in either as root or using sudo:

rm -f /etc/logrotate.d/httpd

The changes will take effect the next time the logrotate script is run.

Manage Linux log files with Logrotate ; By Jim McIntyre

December 27, 2000 | TechRepublic

Log files are the most valuable tools available for Linux system security. The logrotate program is used to provide the administrator with an up-to-date record of events taking place on the system. The logrotate utility may also be used to back up log files, so copies may be used to establish patterns for system use. In this Daily Drill Down, I'll cover the following topics:
;

• The logrotate configuration
• Setting defaults for logrotate
• Using the include option to read other configuration files
• Setting rotation parameters for specific files
• Using the include option to override defaults

The logrotate program
The logrotate program is a log file manager. It is used to regularly cycle (or rotate) log files by removing the oldest ones from your system and creating new log files. It may be used to rotate based on the age of the file or the file's size, and usually runs automatically through the cron utility. The logrotate program may also be used to compress log files and to configure e-mail to users when they are rotated.

The logrotate configuration
The logrotate program is configured by entering options in the /etc/logrotate.conf file. This is a text file, which may contain any of the configuration options listed in the table below. The options entered in /etc/logrotate.conf may be used to set configuration parameters for any log file on the system. These options may also be used to allow logrotate to read configuration parameters from other log files, by using the include parameter.

;

Options

Function

;
;
compress	This is used to compress the rotated log file with gzip.
nocompress	This is used when you do not want to compress rotated log files.
copytruncate	This is used when processes are still writing information to open log files. This option copies the active log file to a backup and truncates the active log file.
nocopytruncate	This copies the log files to backup, but the open log file is not truncated.
create mode owner group	This rotates the log file and creates a new log file with the specified permissions, owner, and group. The default is to use the same mode, owner, and group as the original file.
nocreate	This prevents the creation of a new log file.
delaycompress	When used with the compress option, the rotated log file is not compressed until the next time it is cycled.
nodelaycompress	This overrides delaycompress. The log file is compressed when it is cycled.
errors address	This mails logrotate errors to an address.
ifempty	With this, the log file is rotated even if it is empty. This is the default for logrotate.
notifempty	This does not rotate the log file if it is empty.
mail address	This mails log files that are cycled to an address. When mail log files are cycled, they are effectively removed from the system.
nomail	When mail log files are cycled, a copy is not mailed.
olddir directory	With this, cycled log files are kept in the specified directory. This directory must be on the same filesystem as the current log files.
noolddir	Cycled log files are kept in the same directory as the current log files.
prerotate/endscript	These are statements that enclose commands to be executed prior to a log file being rotated. The prerotate and endscript keywords must appear on a line by themselves.
postrotate/endscript	These are statements that enclose commands to be executed after a log file has been rotated. The postrotate and endscript keywords must appear on a line by themselves.
daily	This is used to rotate log files daily.
weekly	This is used to rotate log files weekly.
monthly	This is used to rotate log files monthly.
rotate count	This specifies the number of times to rotate a file before it is deleted. A count of 0 (zero) means no copies are retained. A count of 5 means five copies are retained.
tabootext [+] list	This directs logrotate to not rotate files with the specified extension. The default list of extensions is .rpm-orig, .rpmsave, v, and ~.
size size	With this, the log file is rotated when the specified size is reached. Size may be specified in bytes (default), kilobytes (sizek), or megabytes (sizem).

The /etc/logrotate.conf file
The /etc/logrotate.conf file is the default configuration file for logrotate. The default /etc/logrotate.conf file installed with Red Hat Linux is shown below:
# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# send errors to root
errors root
# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress
1
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own lastlog or wtmp --we'll rotate them here
/var/log/wtmp {
; monthly
; create 0664 root utmp
; rotate 1
}

/var/log/lastlog {
; monthly
; rotate 1
}

# system-specific logs may be configured here

Setting defaults for logrotate
Default configuration settings are normally placed close to the beginning of the logrotate.conf file. These settings are usually in effect system-wide. The default settings for logrotate on this system are established in the first 12 lines of the file.

The third line
weekly

specifies that all log files will be rotated weekly.

The fifth line
rotate 4

specifies that four copies of old log files are retained before the files are cycled. Cycling refers to removing the oldest log files and replacing them with new copies.

The seventh line
errors root

sends all logrotate error messages to root.

The ninth line
create

configures logrotate to automatically create new log files. The new log files will have the same permissions, owner, and group as the file being rotated.

The eleventh line
#compress

prevents logrotate from compressing log files when they are rotated. Compression is enabled by removing the comment (#) from this line.

Using the include option
The include option allows the administrator to take log file rotation information, which may be installed in several files, and use it in the main configuration file. When logrotate finds the include option on a line in logrotate.conf, the information in the file specified is read as if it appeared in /etc/logrotate.conf.

Line 13 in /etc/logrotate.conf
include /etc/logrotate.d

tells logrotate to be read in the log rotation parameters, which are stored in the files contained in the /etc/logrotate.d directory. The include option is very useful when RPM packages are installed on a system. RPM packages' log rotation parameters will typically install in the /etc/logrotate.d directory.

The include option is important. Some of the applications that install their log rotation parameters to /etc/logrotate.d by default are apache, linuxconf, samba, cron, and syslog. The include option allows the parameters from each of these files to be read into logrotate.conf.

Using the include option in /etc/logrotate.conf allows the administrator to configure a rotation policy for these packages through a single configuration file.

Using include to override defaults
When a file is read by /etc/logrotate.conf, the rotation parameters specified in the include will override the parameters specified in the logrotate file. An example of /etc/logrotate.conf being overridden is shown below:
#Log rotation parameters for linuxconf
/var/log/htmlaccess.log
{ errors jim
notifempty
nocompress
weekly
prerotate
/usr/bin/chattr -a /var/log/htmlaccess.log
endscript
postrotate
/usr/bin/chattr +a /var/log/htmlaccess.log
endscript
}
/var/log/netconf.log
{ nocompress
monthly
}

In this example, when the /etc/logrotate.d/linuxconf file is read by /etc/logrotate.conf, the following options will override the defaults specified in /etc/logrotate.conf:
Notifempty
errors jim

The nocompress and weekly options do not override any options contained in /etc/logrotate.conf.

Setting parameters for a specific file
Configuration parameters for a specific file are often required. A common example would be to include a section in the /etc/logrotate.conf file to rotate the /var/log/wtmp file once per month and keep only one copy of the log. When configuration is required for a specific file, the following format is used:
#comments
/full/path/to/file
{
option(s)
}

The following entry would cause the /var/log/wtmp file to be rotated once a month, with one backup copy retained:
#Use logrotate to rotate wtmp
/var/log/wtmp
{
monthly
rotate 1
}
Although the opening bracket may appear on a line with other text or commands, the closing bracket must be on a line by itself.
Using the prerotate and postrotate options
The section of code below shows a typical script in /etc/logrotate.d/syslog. This section applies only to /var/log/messages. On a production server, /etc/logrotate.d/syslog would probably contain similar entries.
/var/log/messages
{
prerotate
/usr/bin/chattr -a /var/log/messages
endscript
postrotate
/usr/bin/kill -HUP syslogd
/usr/bin/chattr +a /var/log/messages
endscript
}

The format for this script uses the following methods:
;

• The first line, /var/logmessages, declares the file for which this script will be used.
• The curly braces,{ }, are used to enclose the entire script. All commands contained within these braces will be run on the /var/log/messages file.
• The prerotate command specifies actions to be taken prior to the file being rotated by logrotate.
• The command /usr/bin/chattr -a is run to remove the append-only attribute from /var/log/messages.
• The endscript command marks the end of the prerotate portion of this script.
• The next line, postrotate, specifies the following commands are to be run on /var/log/messages after the file has been rotated by logrotate.
• The command /usr/bin/killall -HUPsyslogd is run to reinitiate the system logging daemon, syslogd.
• The next command, /usr/bin/chattr +a /var/log/messages, reassigns the append-only attribute to the /var/log/messages file. This means the file may only be seen in append mode. This prevents the file from being overridden by any other program or user.
• The endscript command appears on a line by itself and marks the end of the postrotate portion of this script.
• The last curly brace,}, marks the end of commands to be applied to the /var/log/messages file.

Running logrotate
There are three steps involved in running logrotate:
;

1. Identify the log files on your system.
2. Create rotation schedules and parameters for the log files.
3. Run logrotate through the cron daemon.

The code below shows the default cronjob shipped with Red Hat Linux to allow logrotate to run daily:
#/etc/cron.daily/logrotate
#! /bin/sh

/usr/sbin/logrotate /etc/logrotate.conf

This cronjob allows logrotate to run daily with the rotation parameter specified in /etc/logrotate.conf.

Conclusion

Log rotation is the first step in log file management. The logrotate utility provides the Linux administrator with the ability to maintain a log file rotation policy and to retain copies of log files to assist in establishing patterns related to system usage. In this Daily Drill Down, we looked at the installation and configuration of logrotate, used the include option to read configuration files related to RPM packages, and ran logrotate as a cronjob. We also discussed the proper methods for restarting logrotate after the log rotation procedure is completed.

The authors and editors have taken care in preparation of the content contained herein but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for any damages. Always have a verified backup before making any changes.

Rotating Linux Log Files MDLog-sysadmin

Comments Chris Riney - June 7, 2006

I'm glad that he mentioned that this was based on a Debian systems, since the RedHat & SuSE systems that I work with do not have the syslog "Feature" that is described (must be part of syslogNG which I'm not familiar with). Both of these major commercial distributions manage the syslog files via logrotate.

Actually I'm just starting to work with SuSE Enterprise, and I'm noticing the differences in the way RedHat & SuSE implement scheduling Hourly, Daily, Weekly, & Monthly rotations.

A feature of of LogRotate 3.7.x that I found, (and was looking for) is the 'dateext' which appends the rotation date to the end instead of a sequential number. The problem I have with the sequential number, is that logrotate moves the older log's up the sequence, which causes those file to be backed-up multiple times with differential backups.

One item that you didn't cover, is how logrotate knows when it is the proper time to rotate a daily, weekly, or monthly file. Part of that is controlled by the status file (/var/lib/logrotate.status on most Linux system) which tracks when a particular file was last rotated. The other part is that Monthly rotations are done with the first run of logrotate that month, and Weekly's are run if the last run was later in the previous week, or the last rotation was over 7 days prior (Sunday=0 Saturday=6). So if you are running logrotate daily, then it works itself out to the first day of the month, and Sunday's for the weeklies.

logrotate notes on usage

These are my personal notes on how to use `logrotate' on a linux system.
I'm writing these notes to help me make sense of how things work.

2005-5-5: I don't know where the logrotate source went to. All of my links to logrotate are broken. The source in about September 2001 was logrotate-3.3.tar.gz. That probably isn't very useful though.

How logrotate is invoked from cron.

On one of my RedHat 5.2 systems, `crond' is started from the script /etc/rc.d/init.d/crond. The manual `man cron' says that cron

searches for /etc/crontab and the files in the /etc/cron.d/ directory.

My /etc/crontab file contains the following.

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
0  0 1 * * root run-parts /etc/cron.monthly

And the /etc/cron.d directory contains nothing.

So clearly, any logrotate commands are being run out of one or more of the /etc/cron.* files. The manual `man 5 crontab' indicates that the above commands mean that the program `run-parts' will be run as user `root' for each of the scripts.

The command `run-parts' turns out to be the script /usr/bin/run-parts, which is very short, as follows.

#!/bin/bash

# run-parts - concept taken from Debian

# keep going when something fails
set +e

if [ $# -lt 1 ]; then
        echo "Usage: run-parts <dir>"
        exit 1
fi

if [ ! -d $1 ]; then
        echo "Not a directory: $1"
        exit 1
fi

for i in $1/* ; do
        [ -d $i ] && continue
        if [ -x $i ]; then
                $i
        fi
done

exit 0

In essence, this just runs all of the scripts in the specified directory. E.g. each hour, the executable plain files in /etc/cron.hourly are run.

It turns out that the `logrotate' program is invoked from /etc/cron.daily. This is the contents of the file /etc/cron.daily/logrotate.

#!/bin/sh

/usr/sbin/logrotate /etc/logrotate.conf

This means that the only file that `logrotate' gets its instructions from directly is /etc/logrotate.conf, which contains the following lines.

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 8 weeks worth of backlogs
rotate 8

# send errors to root
errors root

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own lastlog or wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    rotate 1
}

# system-specific logs may be configured here

This is all explained in the manual `man logrotate'.

Operation of logrotate.

The directory /etc/logrotate.d contains the following files.

-rw-r--r--   1 root     root          354 Oct 13  1998 apache
-rw-r--r--   1 root     root          108 Aug 28  1999 cron
-rw-r--r--   1 root     root          188 Oct 14  1998 linuxconf
-rw-r--r--   1 root     root          156 Oct 13  1998 mgetty
-rw-r--r--   1 root     root          327 Aug 12  1998 syslog
-rw-r--r--   1 root     root          457 Sep 10  1998 uucp

The file /etc/logrotate.d/apache is the one I'm interested in for this exercise. This file contains the following.

/var/log/httpd/access_log {
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
}

/var/log/httpd/agent_log {
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
}

/var/log/httpd/error_log {
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
}

/var/log/httpd/referer_log {
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
}

When I installed the latest version of Apache to get the PHP3 and PostgreSQL to work (around 20 March 2000) on my web server machine, I installed Apache so that the log files were in /home2/apache/logs instead of /var/log/httpd.

Therefore what I need to do now is to modify the /etc/logrotate.d/apache file so that the files referred to are all in the directory /home2/apache/logs instead. My new /etc/logrotate.d/apache script is as follows, and I saved the old one in directory /etc/logrotate.d/old1.

# The new improved logrotate script for apache on fox.

/home2/apache/logs/*-access_log {
    rotate 9
    monthly
    errors [email protected]
    create
    ifempty
    olddir /home2/apache/logs/oldlogs
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
    }

/home2/apache/logs/*-combref_log {
    rotate 9
    monthly
    errors [email protected]
    create
    ifempty
    olddir /home2/apache/logs/oldlogs
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
    }

/home2/apache/logs/*-error_log {
    rotate 9
    monthly
    errors [email protected]
    create
    ifempty
    olddir /home2/apache/logs/oldlogs
    postrotate
        /usr/bin/killall -HUP httpd
    endscript
    } 

For a log file of an ad-hoc utility I wrote, I have now created this configuration file /etc/logrotate.d/dping for `logrotate'.

# The new improved logrotate script for dping on fox.

/var/log/dping.txt {
    rotate 9
    monthly
    compress
#    delaycompress
    copytruncate
    ifempty
    errors [email protected]
    postrotate
    endscript
    }

Since the creating process cannot be HUPped to make it work with a new log file, I've used `copytruncate' to copy the file and then truncate it. The `delaycompress' is not useful because the original log file is copied rather than moved. Creation of .cvsignore file

Now on Sunday 30 September 2001, it's time to get `logrotate' going on my new SuSE 7.1 web server `dog'. The problem is that SuSE 7.1 does not come with logrotate software on it. So I had to go and look for it. The link at redhat was wrong, but by looking around a bit, I finally found it at ftp://ftp.redhat.com/pub/redhat/linux/code/logrotate/. I downloaded this file: logrotate-3.3.tar.gz.
[2005-5-5: I think it has disappeared again now. But here's a backup copy of logrotate-3.3.tar.gz.]
This can be compiled very simply with `make' and `make install'. There's a manual logrotate.8 with it too. The binary is installed as /usr/sbin/logrotate. So now all I have to do is write a configuration file for the logrotate program and then write a cron script for it.

(Note that since my main initial consideration is to rotate the httpd logs for the beginning of October 2001, I could just use the Apache tool for this purpose, which is /usr/local/apache/bin/rotatelogs on my system. However, I can't understand the documentation for this. So I'm playing safe and using the more flexible redhat `logrotate' tool instead.)

On my SuSE 7.1 machine `dog', the manual says that after reading the per-user crontab files in /var/spool/cron/tabs on start-up, the cron process reads the file /etc/crontab. On my machine as configured, I find the following.

root@dog# more /etc/crontab
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root

#-* * * * *    root  test -x /usr/sbin/atrun && /usr/sbin/atrun
0 21 * * *     root  test -x /usr/sbin/faxqclean && /usr/sbin/faxqclean
5 22 * * *     root  test -x /usr/sbin/texpire && /usr/sbin/texpire
25 23 * * *    root  test -e /usr/sbin/faxcron && sh /usr/sbin/faxcron | mail FaxMaster

#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * *   root  test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons
59 *  * * *     root  rm -f /var/spool/cron/lastrun/cron.hourly
14 0  * * *     root  rm -f /var/spool/cron/lastrun/cron.daily
29 0  * * 6     root  rm -f /var/spool/cron/lastrun/cron.weekly
44 0  1 * *     root  rm -f /var/spool/cron/lastrun/cron.monthly

There's nothing here to help me with initiating my daily script. (By the way, the fax commands are a bit worrying. I'll get rid of those when I understand exactly what they do. They obviously produce many meaningless message which root receives every day!)

So my next step is to look at the files in /etc/cron.d, because the cron manual says that all scripts in this directory are read next. On my machine the only file in /etc/cron.d is a script `seccheck', which produces copious useless messages to root every day and week. (I'll see if I can get rid of that some day too!)

The directory /etc/cron.daily contains a script `aaa_base_rotate_logs' which contains a complex set of rotation rules, but how are the scripts in this directory invoked? Hmmm... Maybe they're invoked from that `/usr/lib/cron/run-crons' script. Yes!! That's where it's invoked from. Yet another big, incomprehensible script. The core of that script is the following Bourne-shell loop.

SPOOL=/var/spool/cron/lastrun
for CRONDIR in /etc/cron.{hourly,daily,weekly,monthly} ; do
    test -d $CRONDIR || continue
    BASE=${CRONDIR##*/}
    test -e $SPOOL/$BASE && {
        case $BASE in
          cron.hourly)  TIME="-cmin  +60 -or -cmin  60" ;;
          cron.daily)   TIME="-ctime +1  -or -ctime 1"  ;;
          cron.weekly)  TIME="-ctime +7  -or -ctime 7"  ;;
          cron.monthly) TIME="-ctime +`date -u +%d`"    ;;
        esac
        eval find $SPOOL/$BASE $TIME | xargs -r rm -f
    }
    if test ! -e $SPOOL/$BASE ; then
        touch $SPOOL/$BASE

        # keep going when something fails
        set +e
        for SCRIPT in $CRONDIR/*[^~,] ; do
            test -d $SCRIPT && continue
            test -x $SCRIPT || continue
           case "$SCRIPT" in
               *.rpm*)         continue ;;
               *.swap)         continue ;;
               *.bak)          continue ;;
               *.orig)         continue ;;
               \#*)            continue ;;
           esac
           /sbin/checkproc $SCRIPT && continue
           nice -15 $SCRIPT
        done
    fi
done

Now what does this mean?
The command `BASE=${CRONDIR##*/}' means that BASE is set to the `longest substring of $CRONDIR which matches pattern "*/"', according to my Bash reference card. This just means that the leading path components are removed. (This is done more simply in C-shell!) In the case of the daily cron job, if there is a file /var/spool/cron/lastrun/cron.daily (which is true), then the following command is run.

eval find /var/spool/cron/lastrun/cron.daily -ctime +1 -or -ctime 1 | xargs -r rm -f

The `xargs' command (which I have never seen before) builds and executes a command line from standard input. Weird! The `xargs' manual says this.

If  the  standard  input  does not contain any non-blanks,
do not run the command.  Normally, the command is run once
even if there is no input.

So in this case, the command `rm -f' is executed for each of the files with modification times within 24 hours of the current time. I don't really understand this.

It looks like the file removal commands in the file /etc/crontab are designed to synchronise the operation of the quarter-hour operations. The commands are only executed if the files in /var/spool/cron/lastrun have been removed. What a convoluted way of achieving a simple objective!!

Next any file in the directory /etc/cron.daily which do not end in the characters `~' or `,' (presumed to be edited files) are executed if they are executable and do not have the endings .rpm, .swap, .bak or .orig as follows.

           /sbin/checkproc $SCRIPT && continue
           nice -15 $SCRIPT

The loop continues is the process is already running. Otherwise it is run with nice level 15.

What this all means finally is that any script in the directory /etc/cron.daily will be run at 00:00 on each day. More to the point, the /etc/cron.monthly scripts are run at 00:00 at the beginning of each month. It all looks a bit dodgy because the `date' test is `date -u', which gives the current UTC day. But this should work, although the motivation for the `-u' is not quite clear.

All I have to do now is write a script and put it in /etc/cron.monthly. No problem!

Logrotate -- A Backup Solution by Sean D. Conway

Sys Admin

After a system failure, tracking down the media, determining the last known good backup, and starting the recovery process can be tricky. I implemented the solution described in this article to reduce the stress of performing backups. Using a program found in Linux called logrotate, I created a backup solution to consolidate and automate the backup process. Logrotate's configuration file contains the details to create the backups, compress the backup files, provide automatic rotation, and remove stale files.

Logrotate is a utility designed to handle the management of log files. Logrotate maintains a series of log files, automatically rotates the files, and removes the files based on the parameters provided in a configuration file. You can specify the name of any file in the logrotate configuration file, it is treated the same as a log file.

I used logrotate when I worked with Red Hat (RH) 6.0. Since then, I have found it to be part of a standard RH install. The current version is available from:

http://www.rpmfind.net/linux/RPM/

The program can be installed with an rpm -Uvh logrotate-3.6.4-1.i386.rpm (current RH 7.3 version) command. Following are a few points of interest in the logrotate tour:

/usr/sbin/logrotate -- Utility residence
/etc/logrotate.conf -- Default configuration file
/etc/logrotate.d -- Default directory for other configuration files
/var/lib/logrotate.status -- Default state file

On RH, logrotate is run from a daily job directory /etc/cron.daily/logrotate. When logrotate is called, it obtains information about which files it is processing by using the default configuration file /etc/logrotate.conf. Alternate configuration files can be used if optioned. The configuration file defines the file to rotate and the parameters to apply. A configuration file entry may look like this:

#daily
"/var/log/logfile.log" {
        daily
        rotate 7
        compress
        }

The man pages for logrotate explain the parameters used to control logrotate. The remainder of this article describes how logrotate can be used to produce a backup schema.

When planning the backup process, I used a variation on the grandfather-father-son backup tape rotation method. The backup set consists of grandfather(monthly), father(weekly), and son(daily) as shown in Figure 1.

Under this grandfather-father-son implementation method, a full backup is performed every day (Monday-Sunday); a full backup of the oldest daily is preserved every Sunday; a full backup of the oldest weekly is preserved the last day of a four-week cycle. This assumes you start the daily cycle on a Monday. After 1 year, 6 dailies, 3 weeklies, and 12 monthly copies are preserved.

Logrotate works similarly to a stack in programming. As each file is rotated, it is pushed onto the stack and forces other files down the stack. The size of the stack is determined by the rotate value in the logrotate configuration file. Once the stack capacity is reached, the first file in gets pushed out. Figure 1 illustrates the idea.

To produce the backup schema shown in Figure 1, create the configuration file /etc/logrotate_backup.conf from the following listing:

##############################
#
#Name:logrotate_backup.conf
#
#Description:script designed to produce backups
#grandfather(monthly),father(weekly),son(daily)
#
#Author: Sean D. Conway
#
#Wed Aug 28 18:12:59 CDT 2002
#
#Revision:
#
#NOTE:
#replace the path to be backed up.
#don't remove "" good for files that have spaces.
#maintain the "." file extensions this will preserve the frequency
################################

#daily (son)
"/home/oracle/backup/database" {
        daily
        rotate 7
        compress
        }

#weekly (father)
"/home/oracle/backup/database.7.gz" {
        weekly
        rotate 4
        }

#monthly (grandfather)
"/home/oracle/backup/database.7.gz.4" {
        monthly
        rotate 12
        }

#daily (son)                         <--comments line
"/home/oracle/backup/database" {     <--file to rotate
        daily                        <--rotation schedule
        rotate 7                     <--rotation interval
        compress                     <--compress the file
        }

The logrotate utility can access the configuration files using two methods:

1. The configuration file can be placed in the default configuration file directory /etc/logrotate.d. This directory is listed in logrotates default configuration file /etc/logrotate.conf. All files in /etc/logrotate.d are additional configuration files that will be processed by the program.

2. An alternate method is to force the logrotate program to use an alternate configuration file from a command line. The following is the cron entry to support logrotate running each day at 07:00 using a configuration file called /etc/logrotate_backup.conf.

00 07 * * * /usr/sbin/logrotate -f /etc/logrotate_backup.conf

The logrotate backup mechanism is used to support three database servers as shown in Figure 2. Each database server produces a backup file and sends it to a server called a tape_gate. On the tape_gate server, logrotate runs daily and takes care of producing the rotation schema. The complete backup series is exported to tape for offsite storage.

Tape_gate also runs a licensed tape client for accessing the tape silo. This configuration allows the administrators to use the client software to schedule and access the backups. The administrator can use the backups stored locally or pull the file back from the tape storage device. All the file creation dates are preserved, and a simple grep on a month name will produce a list of backup files available for that month.

To get support for this backup solution, I had to address some design limitations. This implementation requires a server to process the backups. The horsepower required for the server is minimal. For this installation, I used a reclaimed Pentium desktop with an additional cost of two hard drives. It does provide an alternate point for accessing backups. I find it easier to access a backup from a disk rather than searching a tape library.

Backup files are moved from the source to tape_gate using secure ftp. At selective network slow periods the additional network traffic is acceptable. Secure ftp provides access to a ready backup from tape_gate if needed.

Table 1 shows the complete set of filenames after a one-year cycle. The drive capacity must be sufficient to support the number of system backups being handled. The cost for hard drive capacity today is lower than it was ten years ago. Sixty-gig IDE drives are common. At 14-MB compressed, I was able to get all the backups on a 60-GB drive. The three-database server backups total 500 MB compressed.

The feature that sold this design was cost. A client license for accessing the tape silo costs $1200. Each of the databases runs a different operating system, so each would have needed a different client license. Instead, two drives cost $300 and $1200 for one client license for tape_gate; I saved $2100. The alternative was to create NFS mount on all the servers.

I have been asked to add additional backups to the server. It seems this common approach to backups for a variety of systems can make life easier for administrators. Certainly, one common backup mechanism for all systems is easier to remember than a different backup method for each system.

Sean D. Conway is a former college instructor turned Network System Specialist for a regional telecommunications company in Canada. Working with administrators, they are responsible for the care and feeding of operational support systems inside the telecommunication cloud. He still dabbles in network theory lectures. His formal electronic engineering technology training and education background are valuable tools in achieving a goal -- making learning about computers and networks easier.

Recommended Links

Softpanorama Recommended

Top articles

Sites

• https://fedorahosted.org/logrotate/ the latest version of the logrotate ; package:
• The Tao of Mac - HOWTO Use logrotate
• Logrotate - Linux Config Wiki
• Configuring logrotate Red Hat
• Logrotate - Gentoo Linux Wiki
• logrotate - notes on usage
• Slicehost Articles Understanding logrotate on Ubuntu - part 1
• Quick HOWTO Ch05 Troubleshooting Linux with syslog - Linux Home Networking

Alternatives

logrotate

Custom version of logrotate

Description

logrotate rotates log files into an archive. The name of the archived file is based on the date, so the ssh log would be saved to /archives/2003/ssh/2003-01-01.gz, for example.

logrotate handles multilog directories by concatenating all fully processed logs and saving the combined contents.

After rotating each log, logrotate will normally delete it (you can tell it not to) and may optionally touch the file (useful for Solaris syslog, which expects to log to an existing file!) and/or run a script of your choice (useful to send a HUP signal, for example). logrotate can optionally be told to do something before it starts work and/or do something when it has finished.

Configuration

Edit the logrotate script and set the global variables listed at the top.

archive
Specifies the root for the archived logs.
Default: /archives

firstly
Commands to be executed before doing anything else.
Example: svc -d /service/mysql
Default: <nothing>

finally
Commands to be executed immediately before exiting.
Example: svc -u /service/mysql
Default: <nothing>

verbose
Set this to anything and logrotate will be verbose. You can also use the -v flag when running the script.

Then set the paths to needed tools if necessary. In some cases you may need to set custom paths to the GNU versions of tools.

date
Path to GNU date.
Example: /usr/local/bin/gdate
Default: date

gzip
Path to GNU gzip.
Example: /usr/local/bin/gzip
Default: gzip

Next you need to define which services are going to be rotated. Edit the logs variable for that. Each entry in logs is just a label and can be named anything you like. For example, you might say:

logs="qmail mysql"

This would tell logrotate to expect directives defining how to rotate qmail and MySQL logs.

For each entry in the logs line, you must define two variables: path_xxx and files_xxx. The path variable tells logrotate where the logs are stored. Using the qmail example above, you might write

path_qmail=/var/log/qmail

The files variable lists the files (or multilog directories) which should be rotated. You can put wildcards and other shell stuff in here if you want.

Example:

files_qmail="send smtpd pop3d"

;

logrotate uses the path and files variables together to determine what to rotate. Still with the example above, it would rotate /var/log/qmail/send, /var/log/qmail/smtpd and /var/log/qmail/pop3d.

The archived files would be stored in /archives/year/send/year-month-day.gz etc.

There are some optional variables which you can set for each log target.

keep_xxx
If this is set, logrotate won't delete files after archiving them. In any case, logrotate will never delete files if the archive operation failed.

touch_xxx
If this is set, logrotate will touch files after archiving and (maybe) deleting the originals.

hup_xxx
Commands to be run before rotating files.
Example: hup_apache="killall -1 httpd"

copy_xxx
If this is set, logrotate will copy files and then operate on the copies. This is not the default because it means using up much more disk space but it is useful to rotate logs from a program which keeps filehandles open but which you don't want to HUP.

prefix_xxx
Subdirectory of $archive where logs should be be rotated.
Example: prefix_qmail=qmail
qmail logs would now be archived under /archives/year/qmail.

suffix_xxx
Appended to filename when choosing path for archived log.
Example: suffix_qmail=.log
qmail logs would now be archived under /archives/year/xxx.log.

old_xxx
Suffix appended when logs are kept (keep_xxx is set). Defaults to .old when unset.
Example: old_apache=.yesterday

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019