Logging Part 2: File Logs & Rotation

By the end of this lesson you can:

• Write logs to a file with FileHandler.
• Log to the console and a file at once with multiple handlers.
• Cap log size with RotatingFileHandler (by bytes) or TimedRotatingFileHandler (by day).
• Give each handler its own level — verbose file, quiet console.

A naive file log appends forever:

app.log   →   12 KB   (day 1)
app.log   →   400 MB  (month 3)
app.log   →   14 GB   (month 9 — disk full, server down) 💥

Logging to a file

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s  %(levelname)-8s  %(message)s",
    filename="app.log",      # ← this sends output to a file instead of console
    filemode="a",            # "a" append (default), "w" overwrite each run
)
logging.info("this lands in app.log, not the screen")

One keyword (filename=) redirects everything to a file. But the file still grows forever — and you lose the console. We fix both by building handlers ourselves.

Console AND file at once

import logging

log = logging.getLogger("app")
log.setLevel(logging.DEBUG)

fmt = logging.Formatter("%(asctime)s  %(levelname)-8s  %(message)s",
                        datefmt="%Y-%m-%d %H:%M:%S")

console = logging.StreamHandler()           # → terminal
console.setLevel(logging.INFO)              # quiet: INFO and up
console.setFormatter(fmt)

file = logging.FileHandler("app.log", encoding="utf-8")  # → file
file.setLevel(logging.DEBUG)                # verbose: everything
file.setFormatter(fmt)

log.addHandler(console)
log.addHandler(file)

log.debug("only in the file")
log.info("in both console and file")

Each handler has its own level. A common setup: console shows INFO+ (a tidy live view), the file records DEBUG+ (the full forensic trail). Same log calls, two destinations.

Rotation by size

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    "app.log",
    maxBytes=1_000_000,    # roll over at ~1 MB
    backupCount=5,         # keep app.log.1 … app.log.5, prune older
    encoding="utf-8",
)

When app.log reaches maxBytes, it's renamed app.log.1 (and existing .1 → .2, etc.), a new app.log starts, and anything beyond backupCount is deleted. Maximum disk used ≈ maxBytes × (backupCount + 1) — bounded forever.

Rotation by time

from logging.handlers import TimedRotatingFileHandler

handler = TimedRotatingFileHandler(
    "app.log",
    when="midnight",       # roll over each day at 00:00
    backupCount=14,        # keep 14 days of logs
    encoding="utf-8",
)

Now you get one file per day (app.log.2026-05-27 …), keeping a fortnight. when can be "midnight", "H" (hourly), "W0" (weekly on Monday), and more. Use this when "show me yesterday's log" matters; use size-based when bursts of volume are the worry.

Goal: one reusable setup_logging() you can drop into any automation — console for live INFO, a rotating file for full DEBUG history.

import logging
from logging.handlers import RotatingFileHandler
from pathlib import Path

def setup_logging(name: str = "app", log_dir: str = "logs") -> logging.Logger:
    Path(log_dir).mkdir(parents=True, exist_ok=True)

    log = logging.getLogger(name)
    log.setLevel(logging.DEBUG)
    if log.handlers:                 # already configured — don't double up
        return log

    fmt = logging.Formatter(
        "%(asctime)s  %(levelname)-8s  %(name)s  %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S")

    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    console.setFormatter(fmt)

    fileh = RotatingFileHandler(
        Path(log_dir) / f"{name}.log",
        maxBytes=1_000_000, backupCount=5, encoding="utf-8")
    fileh.setLevel(logging.DEBUG)
    fileh.setFormatter(fmt)

    log.addHandler(console)
    log.addHandler(fileh)
    return log

# usage in any script:
log = setup_logging("backup")
log.info("backup started")
log.debug("detail recorded only in logs/backup.log")
log.warning("disk at 85%%")

# console (INFO and up):
2026-05-28 14:30:01  INFO      backup  backup started
2026-05-28 14:30:01  WARNING   backup  disk at 85%

# logs/backup.log (DEBUG and up) ALSO contains:
2026-05-28 14:30:01  DEBUG     backup  detail recorded only in logs/backup.log

Read the code

This is a setup you'll genuinely reuse. The if log.handlers: return guard makes it idempotent — safe to call from multiple modules. The console stays readable (INFO+) while the rotating file captures everything (DEBUG+) and can never exceed ~6 MB on disk. Drop setup_logging() at the top of any Level 7 project and you instantly have professional observability.

Persist logs by adding a FileHandler — or attach several handlers to log to console and file at once, each with its own level (quiet console, verbose file). To stop logs eating the disk, use RotatingFileHandler (rolls over at maxBytes, keeps backupCount files) or TimedRotatingFileHandler (rolls over by time, e.g. daily). The rotation is invisible to your log.info(...) calls. A reusable setup_logging() with a rotating file gives every automation durable, bounded, professional logs.