Contents

Introduction to logging in Python

A gentle, practical introduction to logging in Python


Why bother with a dedicated logging library?

  • Prints don’t scale. print() is fine during quick experiments, but real programs need a record that can be filtered, rotated, or shipped elsewhere.
  • Separation of concerns. You decide what to log in your code; logging decides where and how to write it (console, file, etc.).
  • Built-in, no extra dependency. The standard library’s logging module is powerful enough for most applications.

Core concepts

ConceptRole in the ecosystemTypical examples
LoggerThe entry point your code calls (logger.info(...)). You can have many, one per module."__main__", "my_package.worker"
HandlerDecides where the record goes.StreamHandler (stdout), FileHandler, TimedRotatingFileHandler, SMTPHandler
FormatterDecides how the record looks.'%(asctime)s - %(levelname)s - %(name)s - %(message)s'

A minimal logger

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(levelname)s | %(message)s"
)

logging.info("Hello, world!")
  • basicConfig is a one-liner good for small scripts.
  • In bigger projects, mixing multiple modules / log files, you’ll want finer control.

Rotating files at midnight

Rotating a log file means creating a new log file after a certain time or size limit is reached. In this case, a new file is created every night at midnight. Only the most recent two log files are kept—yesterday’s and today’s—while older ones are deleted automatically.

  • Keeps log files from growing forever.
  • Eases log shipping/archiving.
  • A single command wipes logs older than n days.

The TimedRotatingFileHandler in the helper below:

ParameterValue
when="midnight"Rotate every day at 00:00 server local time.
interval=1Every 1 unit of when (here: days).
backupCount=1Keep one old file (log_file.log.2025-05-17). Older ones are deleted.
encoding="utf-8"Avoids surprises with non-ASCII characters.

If you set backupCount=30, it will keep:

  • Today’s log file (log_file.log), and
  • The 30 most recent rotated log files (log_file.log.2025-05-17, log_file.log.2025-05-16, …)

Drop-in helper: get_logger

# logger.py
import logging
from pathlib import Path
from logging.handlers import TimedRotatingFileHandler

LOG_FILE = Path("./logs/log_file.log")
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)  # ensure ./logs/

def get_logger(name: str) -> logging.Logger:
    """Return a module-specific logger configured for daily rotation."""
    logger = logging.getLogger(name)
    logger.setLevel(logging.INFO)

    # Prevent adding duplicate handlers if the module is imported repeatedly
    if not logger.handlers:
        handler = TimedRotatingFileHandler(
            filename=LOG_FILE,
            when="midnight",
            interval=1,
            backupCount=1,
            encoding="utf-8",
        )
        formatter = logging.Formatter(
            "%(asctime)s - %(levelname)s - %(name)s - %(message)s",
            datefmt="%Y-%m-%d %H:%M:%S",
        )
        handler.setFormatter(formatter)
        logger.addHandler(handler)

        # Block propagation so the same record is not also printed
        logger.propagate = False

    return logger
  • if not logger.handlers:: Guarantees that multiple imports don’t attach multiple handlers, which would duplicate every line in the output.
  • logger.propagate = False: Stops messages from bubbling up to the root logger, so you don’t accidentally get console spam when your app also configures a root handler.
    • In other words, “Don’t pass my log message to parent loggers. I’ll handle it here.”

Using the helper in your scripts

# worker.py
from logger import get_logger
logger = get_logger(__name__)

def create_job(job_id: str):
    logger.info(f"Create: JobID: {job_id}")

Handling exceptions:

try:
    result = do_something()
except ProcessException as e:
    # Log message plus full traceback
    logger.error("%s: %s", e.code, e.message, exc_info=True)
  • When you’re inside an except block and you want to record not just the error message, but also where exactly the error happened. This is a case where exc_info comes in:
try:
    1 / 0
except ZeroDivisionError as e:
    logger.error("An error occurred!", exc_info=True)

This will produce a log like:

2025-05-17 10:23:00 - ERROR - __main__ - An error occurred!
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    1 / 0
ZeroDivisionError: division by zero

Without it, you would only see

2025-05-17 10:23:00 - ERROR - __main__ - An error occurred!