A correlation ID is an identifier attached to related application events so that a developer can follow a single request, job or operation through multiple Python log records. In a busy application, many tasks run at once, making chronological log order insufficient to reconstruct what happened. A request-specific ID helps connect messages from validation, database access, external services and error handling without confusing one user’s operation with another. This is OSPython.043, continuing after OSPython.042 on capturing warnings in application logs.

Why related log records need an identity

Imagine a web service processing hundreds of requests per second. A warning about a slow query and an error about a failed response might occur a few milliseconds apart but belong to different requests. A timestamp tells when each event occurred; the correlation ID tells which operation each event belongs to. This is a form of contextual logging, where metadata is attached to ordinary log messages.

Correlation IDs should be unique enough for the relevant system and retention window, easy to pass between components and safe to expose to authorized diagnostic systems. They are not authentication tokens or secrets. Do not place passwords, access tokens or personally identifying data in an identifier. The earlier lesson on logging secrets safely explains why logging metadata also needs privacy controls.

Creating a request-scoped ID in Python

The standard library’s uuid module can create random UUID version 4 identifiers. A request handler may generate an ID at the boundary of a new operation, then attach it to subsequent log messages. The example below shows the idea with LoggerAdapter, which supplies extra fields to a logger without requiring every message to repeat them manually.

import logging
from uuid import uuid4

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s request=%(request_id)s %(message)s",
)

logger = logging.getLogger("orders")


def process_order(order_id: int) -> None:
    request_id = str(uuid4())
    log = logging.LoggerAdapter(logger, {"request_id": request_id})
    log.info("order validation started: order=%s", order_id)
    log.info("order validation completed: order=%s", order_id)


process_order(1042)

Both records emitted by the call to process_order have the same ID, while another call receives a different one. The formatting string expects request_id to exist on every record processed by that handler. In a larger program, a missing field can cause a formatting error, so the application should use a consistent adapter, filter or record-factory strategy rather than mixing incompatible records.

The following existing editorial photograph shows a Python programming environment, providing visual context for writing and inspecting diagnostic output.

Maintaining context across concurrent tasks

A single LoggerAdapter works well when the logger is explicitly passed through the relevant call chain. In asynchronous Python, however, many coroutines can interleave on one thread. A mutable global variable is not a safe place to store the current request ID because unrelated operations may overwrite it. Python’s contextvars module provides context-local state that can follow asynchronous task execution under its documented semantics.

The following example attaches a context variable to records using a logging filter. It is deliberately small and uses the standard library only.

import logging
from contextvars import ContextVar
from uuid import uuid4

request_id_var = ContextVar("request_id", default="-")


class RequestIdFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        record.request_id = request_id_var.get()
        return True


handler = logging.StreamHandler()
handler.addFilter(RequestIdFilter())
handler.setFormatter(
    logging.Formatter("%(levelname)s request=%(request_id)s %(message)s")
)

logger = logging.getLogger("service")
logger.setLevel(logging.INFO)
logger.addHandler(handler)
logger.propagate = False


def handle_request() -> None:
    token = request_id_var.set(str(uuid4()))
    try:
        logger.info("request started")
        logger.info("request completed")
    finally:
        request_id_var.reset(token)


handle_request()

The token allows the previous context value to be restored in a finally block, preventing a completed operation’s ID from leaking into later work in the same context. A production application should integrate context creation and cleanup at its request or job boundary. Thread pools, process pools and distributed services may need explicit propagation; do not assume context automatically crosses every boundary. Related lessons include multiprocessing logging and ProcessPoolExecutor logging.

Correlation across service boundaries

When one service calls another, an authorized request identifier can be passed through a designated HTTP header or message field. The receiving service should validate the field’s length and allowed characters, then decide whether to preserve it or generate a local ID according to its trust policy. A client-supplied ID must not be treated as proof of identity or permission.

For more complex distributed systems, the W3C Trace Context specification defines standard tracing headers such as traceparent. A trace ID connects related operations across services, while span IDs identify individual units of work within that trace. An ordinary application correlation ID can be simpler, but it should not be confused with a complete distributed tracing implementation.

The next existing Python developer image illustrates a code review and diagnostic environment. In practice, developers often combine correlation IDs with structured fields such as service name, event type, severity and timestamp.

Working example: investigating an error

Suppose an API request produces three events: INFO request=7fa2 start, WARNING request=7fa2 upstream slow and ERROR request=7fa2 timeout. A separate request produces INFO request=93b1 complete between the warning and error. Searching for request=7fa2 reconstructs the relevant operation without mixing the successful request into the incident timeline. In a real deployment, use a longer, collision-resistant ID rather than these shortened teaching examples.

Correlation does not prove causation: two events sharing an ID may still need timestamps, component details and supporting metrics to establish what failed. The exception logging lesson explains how to preserve tracebacks, while dictConfig covers consistent application-wide logging configuration.

Operational safeguards

High-volume services should avoid unbounded log output and sensitive fields. When a request ID is present in all records, the logging pipeline should still use sensible levels, filters, retention rules and access controls. For slow handlers, QueueHandler and QueueListener can move work off latency-sensitive code paths. If an application captures warnings, its warning records also need a deliberate context strategy; see OSPython.042.

Knowledge check

• Why is a timestamp insufficient to associate all log records with one concurrent request?
• What is the benefit of restoring a ContextVar using its token?
• Does a correlation ID authenticate a user or prove that one event caused another?

Answers: Concurrent operations interleave their events; resetting prevents stale context from contaminating later work; no, it is an observability identifier rather than an authorization or causation mechanism.

References and further reading

Primary references include the Python logging documentation, Python contextvars documentation, Python uuid documentation and the W3C Trace Context standard. For earlier curriculum material, follow the contextual links to LoggerAdapter, asyncio logging and secret redaction.

BitcoinVersus.Tech Editor’s Note: Help keep independent Python education accessible with a donation of 5,000 sats. Scan the QR code below with a compatible Bitcoin wallet to contribute.

Display the QR on another screen and scan it with your Bitcoin wallet.