sys.monitoring -- Execution Event Monitoring in Python

For years, Python developers who needed to build debuggers, profilers, or coverage tools relied on sys.settrace() and sys.setprofile(). These older APIs worked, but they imposed a heavy performance penalty on every traced function call. Introduced in Python 3.12 through PEP 669, sys.monitoring is a fundamentally different approach -- a low-impact event monitoring framework that lets tools subscribe to specific execution events without dragging down the entire interpreter.

If you have ever profiled a Python application and noticed that the act of profiling itself made everything dramatically slower, you already understand the problem that sys.monitoring was designed to solve. The older tracing APIs insert themselves into every function call and line execution, whether the tool actually cares about those events or not. The monitoring framework flips that model entirely: tools declare exactly which events they want to observe, and the interpreter only fires callbacks for those specific events. The result is overhead that, for many use cases, is close to zero when monitoring is inactive and remains far lower than the old approach even when actively watching for events.

What Is sys.monitoring and Why Does It Exist

The sys.monitoring namespace lives inside the sys module. You do not need to import it separately -- a simple import sys gives you access to sys.monitoring and all of its functions and constants. The namespace provides the API that monitoring tools use to register themselves, declare which execution events they want to observe, and supply callback functions that the interpreter will invoke when those events occur.

The design came out of PEP 669, authored by Mark Shannon as part of the Faster CPython initiative. The core insight was that the old tracing mechanism required the interpreter to check whether tracing was active on every single function call and line transition, even when no tool was listening. With sys.monitoring, the interpreter uses a technique called "quickening" to dynamically patch bytecode instructions. When no tool is monitoring a particular event at a particular location, the instruction runs at full speed with no overhead at all. When a tool activates monitoring for an event, the interpreter replaces the relevant bytecode with an instrumented version that fires the callback.

This architecture means that code running under a debugger on Python 3.12 and later can actually outperform code running without a debugger on Python 3.11, because the monitoring framework avoids the constant per-instruction checking that the old tracing system imposed.

Tool Identifiers and Registration

Before a tool can register callbacks or activate events, it needs to claim a tool identifier. Tool IDs are integers in the range 0 through 5, giving the interpreter room for up to six independent tools to operate simultaneously without stepping on each other. The system provides four pre-defined constants to help common tool types coordinate:

import sys

# Pre-defined tool ID constants
sys.monitoring.DEBUGGER_ID   # 0
sys.monitoring.COVERAGE_ID   # 1
sys.monitoring.PROFILER_ID   # 2
sys.monitoring.OPTIMIZER_ID  # 5

You are not required to use these specific IDs for their named purposes. They exist purely as a convention so that, for example, two different debugger packages will both reach for ID 0 and discover the conflict immediately rather than silently interfering with each other.

To register a tool, call use_tool_id() with the desired ID and a descriptive name. When the tool is finished, it should release the ID so other tools can use it:

import sys

# Claim a tool ID
sys.monitoring.use_tool_id(sys.monitoring.PROFILER_ID, "my_profiler")

# Check which tool owns an ID
tool_name = sys.monitoring.get_tool(sys.monitoring.PROFILER_ID)
print(tool_name)  # "my_profiler"

# Release the tool ID when done
sys.monitoring.free_tool_id(sys.monitoring.PROFILER_ID)

Calling use_tool_id() with an ID that is already in use raises a ValueError. The free_tool_id() function automatically calls clear_tool_id() first, which unregisters all events and callbacks associated with that tool before releasing the ID.

Understanding the Event System

The monitoring framework organizes execution events into several categories. Each event is represented as a power-of-2 integer constant within the sys.monitoring.events namespace, which makes it easy to combine multiple events using bitwise OR.

Local Events

Local events are tied to specific locations in the program's execution and can be individually disabled on a per-instruction basis. These are the events you will use in the vast majority of monitoring scenarios:

PY_START fires when a Python function begins execution, immediately after the call. PY_RETURN fires immediately before a Python function returns. PY_YIELD and PY_RESUME handle generator and coroutine suspension and resumption. CALL fires before any call in Python code, including calls to C functions. LINE fires when execution moves to an instruction with a different line number. INSTRUCTION fires before every single VM instruction -- use this sparingly, as it can be expensive. JUMP fires on unconditional jumps in the control flow graph. BRANCH_LEFT and BRANCH_RIGHT fire when a conditional branch goes one direction or the other. STOP_ITERATION fires on artificial StopIteration raises from generators and coroutines.

Ancillary Events

Two events -- C_RETURN and C_RAISE -- are classified as ancillary. They track returns and exceptions from C-level callables (built-in functions, C extension methods, etc.), but they are controlled by the CALL event. You will only receive C_RETURN or C_RAISE callbacks if you are also monitoring CALL.

Other Events

A final group of events is not tied to specific code locations and cannot be individually disabled with the DISABLE sentinel. These include RAISE (an exception is raised), RERAISE (an exception propagates out of a finally block), PY_THROW (a generator or coroutine is resumed via throw()), PY_UNWIND (a function exits during exception unwinding), and EXCEPTION_HANDLED (an exception is caught).

Registering Callbacks and Controlling Events

Monitoring an event requires two separate steps: registering a callback function and activating the event. Neither step alone is sufficient -- you need both.

Setting Events Globally

The simplest approach is to activate events globally, meaning they fire for all code objects across the entire interpreter:

import sys

E = sys.monitoring.events
TOOL_ID = sys.monitoring.PROFILER_ID

sys.monitoring.use_tool_id(TOOL_ID, "simple_profiler")

# Activate PY_START and PY_RETURN events globally
sys.monitoring.set_events(TOOL_ID, E.PY_START | E.PY_RETURN)

# Check which events are active
active = sys.monitoring.get_events(TOOL_ID)
print(active == E.PY_START | E.PY_RETURN)  # True

Setting Events Per Code Object

For more targeted monitoring, you can activate events for a specific code object only. This is powerful for scenarios like setting a breakpoint on a particular function without impacting the rest of the program:

import sys

E = sys.monitoring.events
TOOL_ID = sys.monitoring.DEBUGGER_ID

sys.monitoring.use_tool_id(TOOL_ID, "targeted_debugger")

def function_to_watch():
    x = 10
    y = 20
    return x + y

# Monitor LINE events only inside function_to_watch
sys.monitoring.set_local_events(
    TOOL_ID,
    function_to_watch.__code__,
    E.LINE
)

Registering Callback Functions

Callback functions receive different arguments depending on the event type. The general pattern is that every callback receives a code object (identifying where the event occurred) and an instruction_offset (the bytecode offset within that code object). Some events provide additional arguments:

import sys

E = sys.monitoring.events
TOOL_ID = sys.monitoring.PROFILER_ID

sys.monitoring.use_tool_id(TOOL_ID, "call_tracer")

def on_py_start(code, instruction_offset):
    print(f"Entering: {code.co_qualname}")

def on_py_return(code, instruction_offset, retval):
    print(f"Leaving:  {code.co_qualname} -> {retval}")

def on_call(code, instruction_offset, callable_obj, arg0):
    print(f"Calling:  {callable_obj}")

# Register the callbacks
sys.monitoring.register_callback(TOOL_ID, E.PY_START, on_py_start)
sys.monitoring.register_callback(TOOL_ID, E.PY_RETURN, on_py_return)
sys.monitoring.register_callback(TOOL_ID, E.CALL, on_call)

# Activate the events
sys.monitoring.set_events(TOOL_ID, E.PY_START | E.PY_RETURN | E.CALL)

For CALL events, the callback receives the callable object and its first argument (or sys.monitoring.MISSING if there are no arguments). For instance method calls, the callable is the unbound function from the class, and arg0 is the instance (self).

Disabling Events from Within Callbacks

A powerful feature of the framework is that a callback function can return sys.monitoring.DISABLE to turn off monitoring for that specific event at that specific instruction. This is how tools achieve near-zero overhead: once a callback determines that a particular location is no longer interesting, it disables itself at that point.

import sys

E = sys.monitoring.events
TOOL_ID = sys.monitoring.DEBUGGER_ID

sys.monitoring.use_tool_id(TOOL_ID, "one_shot_debugger")

def on_line(code, line_number):
    if line_number == 42:
        print("Hit line 42!")
        # Keep monitoring this line
        return None
    # Not interested in this line, disable monitoring here
    return sys.monitoring.DISABLE

sys.monitoring.register_callback(TOOL_ID, E.LINE, on_line)
sys.monitoring.set_events(TOOL_ID, E.LINE)

If you need to re-enable events that were previously disabled (for example, when setting new breakpoints), call sys.monitoring.restart_events() to reactivate all disabled local events for a given tool.

Practical Examples

Building a Simple Function Profiler

This example tracks how many times each function is called and measures total execution time per function:

import sys
import time
from collections import defaultdict

E = sys.monitoring.events
TOOL_ID = sys.monitoring.PROFILER_ID

call_counts = defaultdict(int)
call_times = defaultdict(float)
start_times = {}

def on_start(code, offset):
    name = code.co_qualname
    call_counts[name] += 1
    start_times[id(code)] = time.perf_counter()

def on_return(code, offset, retval):
    key = id(code)
    if key in start_times:
        elapsed = time.perf_counter() - start_times.pop(key)
        call_times[code.co_qualname] += elapsed

sys.monitoring.use_tool_id(TOOL_ID, "function_profiler")
sys.monitoring.register_callback(TOOL_ID, E.PY_START, on_start)
sys.monitoring.register_callback(TOOL_ID, E.PY_RETURN, on_return)
sys.monitoring.set_events(TOOL_ID, E.PY_START | E.PY_RETURN)

# --- Code to profile ---
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

result = fibonacci(20)

# --- Print results ---
sys.monitoring.set_events(TOOL_ID, E.NO_EVENTS)
sys.monitoring.free_tool_id(TOOL_ID)

for name in sorted(call_counts, key=lambda n: call_times[n], reverse=True):
    count = call_counts[name]
    total = call_times[name]
    print(f"{name}: {count} calls, {total:.6f}s total")

Spying on Variable Mutations

The INSTRUCTION event can be used to monitor every VM instruction and detect when a value changes unexpectedly. This technique is useful for tracking down hard-to-find mutations in global state:

import sys

E = sys.monitoring.events
TOOL_ID = sys.monitoring.DEBUGGER_ID

watched_list = [1, 2, 3]
last_known_length = len(watched_list)

def on_instruction(code, offset):
    global last_known_length
    current_length = len(watched_list)
    if current_length != last_known_length:
        print(f"List changed at {code.co_filename}:{code.co_qualname}")
        print(f"  Length went from {last_known_length} to {current_length}")
        last_known_length = current_length
        breakpoint()
    return sys.monitoring.DISABLE

sys.monitoring.use_tool_id(TOOL_ID, "mutation_spy")
sys.monitoring.register_callback(TOOL_ID, E.INSTRUCTION, on_instruction)
sys.monitoring.set_events(TOOL_ID, E.INSTRUCTION)

Monitoring Exceptions

Tracking where exceptions are raised throughout a program is a common requirement for observability. The RAISE and EXCEPTION_HANDLED events make this straightforward:

import sys

E = sys.monitoring.events
TOOL_ID = sys.monitoring.PROFILER_ID

sys.monitoring.use_tool_id(TOOL_ID, "exception_tracker")

def on_raise(code, offset, exception):
    print(f"Exception raised: {type(exception).__name__}: {exception}")
    print(f"  in {code.co_qualname} at offset {offset}")

def on_handled(code, offset, exception):
    print(f"Exception handled: {type(exception).__name__}")
    print(f"  in {code.co_qualname}")

sys.monitoring.register_callback(TOOL_ID, E.RAISE, on_raise)
sys.monitoring.register_callback(TOOL_ID, E.EXCEPTION_HANDLED, on_handled)
sys.monitoring.set_events(TOOL_ID, E.RAISE | E.EXCEPTION_HANDLED)

# --- Demo ---
def risky_division(a, b):
    try:
        return a / b
    except ZeroDivisionError:
        return float('inf')

risky_division(10, 0)

sys.monitoring.free_tool_id(TOOL_ID)

What Changed in Python 3.14

Python 3.14, released in October 2025, brought several notable updates to the monitoring framework. The BRANCH event is now deprecated and has been replaced by two separate events: BRANCH_LEFT and BRANCH_RIGHT. This change allows tools to disable each branch direction independently, which provides better performance for coverage tools that only need to track one side of a conditional.

The practical impact is significant for coverage tools in particular. The popular coverage.py library now uses sys.monitoring as its default measurement core when running on Python 3.14 and later, instead of falling back to the older sys.settrace() approach. The sys.monitoring API was not fully capable of supporting branch coverage until the 3.14 changes landed, since the original BRANCH event could not be disabled independently for taken versus not-taken paths.

Additionally, Python 3.14's pdb debugger now supports a sys.monitoring-based backend. When you launch pdb from the command line or call breakpoint(), Python 3.14 uses the monitoring backend by default, resulting in noticeably faster debugging sessions compared to the old sys.settrace()-based approach.

Key Takeaways

1. Low overhead by design: sys.monitoring uses bytecode quickening to ensure that unmonitored code runs at full speed. Unlike sys.settrace(), there is no per-instruction penalty when events are not active.
2. Granular event control: Tools subscribe to only the events they need, and can further narrow monitoring to specific code objects or even disable monitoring at individual instruction offsets from within callbacks.
3. Multi-tool coexistence: Up to six tools can operate simultaneously using distinct tool IDs, allowing a debugger, profiler, and coverage tool to run side-by-side without interfering with each other.
4. Callback-based architecture: Each event type has a well-defined callback signature. Callbacks receive the code object, instruction offset, and event-specific data like return values, exceptions, or callable references.
5. Actively evolving: Python 3.14 deprecated the BRANCH event in favor of BRANCH_LEFT and BRANCH_RIGHT, and tools like coverage.py and pdb are already built on top of the monitoring framework as their default backend.

The sys.monitoring namespace represents a fundamental shift in how Python handles tool instrumentation. Whether you are building a custom profiler, writing a code coverage plugin, or just trying to track down a mysterious mutation in your application's global state, this framework gives you the precision and performance that sys.settrace() never could. As the ecosystem continues adopting it -- with pdb, coverage.py, and other tools leading the way -- sys.monitoring is quickly becoming the standard foundation for all Python execution monitoring.