logrotate: automatic log rotation and archiving

Application logs fill up disk space within a week, and manually running rm *.log is a recipe for trouble. logrotate handles this automatically: it rotates, compresses, and deletes old files on a schedule. Let’s see how it works and how to set it up in five minutes.

How It Works

logrotate runs daily through cron. The default config lives in /etc/logrotate.conf, and additional configs are included from /etc/logrotate.d/. During rotation, the current file gets renamed, a new empty one is created, old copies are compressed and numbered.

The cycle looks like this:

app.log        →  app.log.1      (compressed: app.log.1.gz)
app.log.1.gz   →  app.log.2.gz
...
app.log.5.gz   →  deleted

The mechanism relies on rename or mv, so the process must hold the file descriptor open. If rotation isn’t picked up by the application, you end up with duplication or an empty log.

Configuration Structure

# /etc/logrotate.conf — global settings
weekly          # rotate weekly
rotate 4        # keep 4 copies
compress        # compress old logs
include /etc/logrotate.d/

Files from /etc/logrotate.d/ override global values for specific logs. The format is straightforward:

/path/to/log {
    directive value
    ...
}

Directives inherit from the global config unless overridden. You can specify multiple paths separated by spaces or use a glob pattern — convenient for rotating all application logs in one block.

Key Directives

Basic parameters covering 90% of use cases:

size overrides weekly/monthly/daily when set. Rotation happens when the file reaches the specified size AND the interval has passed. So size 100M with weekly means rotation no earlier than a week and only if the file exceeds 100M.

Example Config for an Application

Assume your myapp service writes to /var/log/myapp/. Config:

/var/log/myapp/*.log {
    daily
    rotate 14
    size 50M
    missingok
    notifempty
    compress
    dateext
    dateformat -%Y%m%d-%s
    sharedscripts
    postrotate
        systemctl reload myapp > /dev/null 2>&1 || true
    endscript
}

sharedscripts ensures postrotate runs once for all rotated files, not once per file. Without it, the script executes as many times as files match the rotation criteria.

For Python applications using standard library rotation:

/var/log/myapp/app.log {
    su root myapp
    daily
    rotate 7
    size 200M
    missingok
    notifempty
    compress
    postrotate
        /usr/bin/pkill -HUP -f "python.*myapp" || true
    endscript
}

su changes the rotation process owner — useful when the application runs under a separate user with restricted log permissions.

Debugging and Manual Execution

Dry-run mode shows what would happen without making changes:

logrotate -d /etc/logrotate.d/myapp

Output contains each decision: which file gets renamed, which gets compressed, which commands run. Look for renaming and running postrotate script lines.

Force rotation bypassing the schedule:

logrotate -f /etc/logrotate.d/myapp

-f ignores the last rotation time and size conditions. Combining with -d is a safe way to verify before production:

logrotate -d -f /etc/logrotate.d/myapp

To rotate a specific log outside the schedule but respecting conditions — use the state file:

# check state
cat /var/lib/logrotate/status

# temporarily shift last rotation time
sed -i 's|/var/log/myapp/app.log.*|/var/log/myapp/app.log 2024-01-01-00:00:00|' /var/lib/logrotate/status
logrotate /etc/logrotate.d/myapp

Syntax check without execution:

logrotate -d /etc/logrotate.conf

If there are no errors — output is empty (without -d) or shows the action plan (with -d).

Default cron setup:

# /etc/cron.daily/logrotate
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

On most distros you don’t need to touch this file. If you need rotation more often than once a day — add to /etc/cron.hourly/ or write a separate cron job.