我在PyCon US上的Talk，pdb的基本机制

码农高天

AI-assisted summary of 我在PyCon US上的Talk，pdb的基本机制 — original source. Check the original for full context.

Source published: 2024-08-27

Key ideas

Python 标准库调试器 pdb 在底层依赖 sys.settrace() 机制，但由于其必须全局分发 call 事件且只能函数级过滤 line 事件，会导致测试中递归代码（如 fib(24)）出现高达 100 到 200 倍的严重性能衰减。 (15:00)
pdb 暂停程序和单步调试并无特殊硬件级黑魔法，本质是在 trace 回调中直接运行类似 REPL 的交互式 input 死循环，并通过对下一行代码事件的比对来决定是否停靠。 (6:00)
Python 3.12 引入的 PEP 669（sys.monitoring）通过以代码对象（code object）为粒度的局部监控与动态关闭非断点代码行事件，成功打破了全局追踪瓶颈，为调试器带来近零开销运行的可能。 (15:00)
尽管 sys.monitoring 因需要主动编排事件和保障向下兼容而在标准库 pdb 中尚未正式合入，但第三方调试工具如 PyCharm (pydevd)、VS Code (debugpy) 以及 coverage.py 均已跟进集成该机制以实现百倍提速。 (20:00)

Detailed notes

## Background 该视频由 YouTube 频道“码农高天”于 2024 年 8 月 27 日发布。内容为其在 PyCon US 2024 上发表的附带双语字幕的技术演讲，题为《What makes a Python debugger possible and how can we make it 100x faster》，旨在向开发者揭秘 pdb 背后的底层工作原理及将调试器速度提升百倍的优化方案。

## TL;DR 本场演讲深入剖析了 Python 进程内调试器的底层实现机理与演进方向。高天首先讲解了 pdb 如何利用 `sys.settrace()` 捕获全局调用与代码行事件，并通过回溯栈帧修补 `f_trace` 以及启动类 REPL 交互循环来挂起程序和检查局部变量。随后，他通过基准测试指出传统跟踪因粗粒度事件派发导致程序面临上百倍性能衰减的致命缺陷。最后，他详细介绍了 Python 3.12 引入的 `sys.monitoring`（PEP 669）机制，展示了如何通过按代码对象按需监听和动态禁用非断点行事件来实现近乎零开销的高性能调试，并展望了该特性在 pydevd、debugpy 等生态工具中的落地应用。

## Atmosphere & style 整场演讲呈现出轻松风趣却极具极客深度的硬核技术分享氛围。高天以幽默的现场举手互动破冰，随后迅速转入底层系统机制，层层剥开看似神秘的调试器“魔法”，用极为平实的逻辑还原 CPython 栈帧与解释器事件派发的本质。讲解节奏严谨利落、直击痛点，既坦率指出了历史遗留设计的性能局限，又洋溢着对 Python 现代监控架构演进的工程热情。

## Bulletin - [15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) Python 标准库调试器 pdb 在底层依赖 sys.settrace() 机制，但由于其必须全局分发 call 事件且只能函数级过滤 line 事件，会导致测试中递归代码（如 fib(24)）出现高达 100 到 200 倍的严重性能衰减。 - [06:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=360s) pdb 暂停程序和单步调试并无特殊硬件级黑魔法，本质是在 trace 回调中直接运行类似 REPL 的交互式 input 死循环，并通过对下一行代码事件的比对来决定是否停靠。 - [15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) Python 3.12 引入的 PEP 669（sys.monitoring）通过以代码对象（code object）为粒度的局部监控与动态关闭非断点代码行事件，成功打破了全局追踪瓶颈，为调试器带来近零开销运行的可能。 - [20:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=1200s) 尽管 sys.monitoring 因需要主动编排事件和保障向下兼容而在标准库 pdb 中尚未正式合入，但第三方调试工具如 PyCharm (pydevd)、VS Code (debugpy) 以及 coverage.py 均已跟进集成该机制以实现百倍提速。

## Key takeaways - [00:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=0s) 进程内调试器（in-process debugger）的核心能力可归纳为任意位置暂停、获取运行时上下文、动态执行求值以及细粒度控制程序执行流四项要求。 - [02:30](https://www.youtube.com/watch?v=kzTDEaQx0us&t=150s) sys.settrace() 是 Python 原生追踪机制的核心，通过注册全局回调监听 call、return、line 和 exception 四类解释器内部事件。 - [02:30](https://www.youtube.com/watch?v=kzTDEaQx0us&t=150s) sys.settrace() 存在全局性与颗粒度割裂的架构缺陷：一旦注册便必须捕获全局所有函数调用，且在函数内部只能全量开启或全量关闭行级事件。 - [06:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=360s) pdb.set_trace() 执行时首先注册全局追踪函数，随后回溯遍历调用栈为所有上层已存在的栈帧修补挂载 f_trace，以便在下一行代码执行前立刻截获控制权。 - [06:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=360s) pdb 挂起程序的本质是在 trace 函数接收到 frame 参数后启动命令行交互死循环（REPL），只有用户输入 continue 或下一步命令后循环才会退出并恢复运行。 - [10:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=600s) 调试器读取调用栈和局部变量依赖 CPython 原生维护的 frame 链表对象，通过 f_back 沿调用栈回溯，并从 f_locals、f_lineno 和 f_code 中提取上下文。 - [10:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=600s) 动态求值依赖 eval() 和 exec()，而在调试中直接修改局部变量曾因 CPython 的 fast locals 内部数组机制面临同步困难，该设计直到 Python 3.13 PEP 667 才被彻底现代化重构。 - [10:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=600s) 单步调试命令（如 step、next、return）本质上不修改任何指令指针，而是通过跟踪函数在每一行事件触发时动态判断是否符合用户指定的停靠条件。 - [15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) 传统追踪开销极其昂贵：在包含紧密循环或密集调用的无关逻辑场景下，哪怕仅在一个无关函数打断点，反复触发的回调也会导致运行时间膨胀一两百倍。 - [15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) Python 3.12 的 sys.monitoring (PEP 669) 实现了针对单个 code object 的事件监控，使无关函数如 fib() 可以完全绕开监控并以原生速度运行。 - [15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) sys.monitoring 允许调试器在运行时动态注销循环体内特定非断点行的事件，从而消除了循环迭代中反复派发 Python 级回调的开销。 - [20:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=1200s) 相比过去被动接收全量事件并在 pdb 内部做过滤，sys.monitoring 要求调试器主动在正确位置编排事件订阅，极大增加了调试器的底层设计复杂度。 - [20:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=1200s) 标准库 pdb 因向下兼容和稳定性考量尚未正式合并 sys.monitoring 后端，但主流生态工具（pydevd、debugpy、coverage.py）已率先采用并证实了百倍加速收益。

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=0s) 演讲开场与 Debugger 的基本需求

在演讲正式开始前，主持人提醒台下听众最后会预留约 5 分钟的 Q&A 提问环节，并指出通道上的立式麦克风位置。随后，主讲人高天登台致谢并开启话题。在正式切入正题前，高天在现场做了一个简单的举手调查：“有多少人用过 Python debugger？又有多少人知道它是如何工作、为什么能工作的？”看到台下热烈的反应，高天开玩笑说大家既然都懂那就讲完了可以散场了，随即言归正传指出，今天的主题正是揭秘 Python debugger（以 pdb 为代表）底层的运作机制，并探讨如何通过系统级优化将其运行速度提升高达 100 倍。

高天引导大家思考：debugger 本质上也只是一个普通的程序，那么要让一个程序成为真正可用的调试器，到底需要具备哪些核心功能？他总结了四个最核心的能力要求： 1. **任意位置暂停**：必须能够让正在运行的目标程序在任意指定的位置中断挂起，从而让开发者能够介入并检查程序状态。 2. **获取运行时信息**：必须能够读取程序当前的上下文信息，例如当前的局部变量（local variables）、调用栈帧（call stacks）等，帮助开发者理解程序正在做什么、当前处于何种状态。 3. **动态执行代码（辅助但强大）**：虽然不是绝对必须的硬指标，但如果支持动态求值（evaluation）就会极具价值——比如执行任意表达式或语句、临时调用某些函数、甚至直接在运行时为局部变量赋新值。 4. **控制程序执行流**：必须能够细粒度控制程序的后续执行节奏，例如单步跳入（step into）、单步跳过/跳出（step out/over）等，“ask your program to run at a pace that you desire. Like, stop wherever you want so that you can, you know, understand what the program is doing.”

```mermaid flowchart TD accTitle: Debugger核心功能架构 accDescr: 展示一个合格的Debugger所必须具备的四个核心能力特征

subgraph CoreCapabilities["Debugger 核心能力要求"] A["1. 任意位置挂起程序 (Pause Execution)"] B["2. 获取运行时上下文 (Inspect Stack & Locals)"] C["3. 动态代码执行与求值 (Dynamic Evaluation)"] D["4. 步进控制执行流 (Step Into / Over / Out)"] end ```

高天特别强调，本次 Talk 讨论的全部范围限定在“进程内调试器”（in-process debugger），即调试器与被调试程序运行在完全相同的同一个 Python 进程内部。虽然业界也存在跨进程的 Out-of-process debugger，但两者的架构和底层原理大不相同，不在本次讨论之列。

---

[02:30](https://www.youtube.com/watch?v=kzTDEaQx0us&t=150s) 从 breakpoint() 到 sys.settrace() 底层机制

在日常开发中，大家的调试习惯各不相同：有些人喜欢使用 VS Code、PyCharm 等 IDE 提供的图形化 GUI 调试器，而有些人则偏爱命令行调试。使用 Python 标准库调试器 pdb 时，现代 Python（3.7+）中最常见的做法是在源码中直接插入内置函数 `breakpoint()`。而对于有较长 Python 使用经验的老程序员来说，大家更习惯的写法是 `pdb.set_trace()`，这两者在底层做的事情几乎完全一致。

“breakpoint” 是全行业所有编程语言通用的标准术语，但在 Python 调试生态中，却有着 `set_trace` 这样一个奇怪的术语。高天解释道，这正是解开所有 Python 调试器魔法的钥匙——它与 Python 底层的跟踪机制 `sys.settrace` 密不可分。`sys.settrace()` 接受一个全局的跟踪函数（trace function），或者说回调函数（callback function）。这个回调函数会在解释器发生特定事件时被触发，主要包括四种核心事件类型： - **`call` 事件**：某个函数即将被调用进入。 - **`return` 事件**：某个函数即将返回。 - **`line` 事件**：某一行代码即将被执行。 - **`exception` 事件**：程序抛出了异常。

```mermaid flowchart TD accTitle: sys.settrace 事件派发与函数粒度控制 accDescr: 展示sys.settrace注册后全局函数调用派发及返回值对单函数跟踪的影响

A["调用 sys.settrace(global_trace_func)"] --> B["全局函数调用触发 'call' 事件"] B --> C{"global_trace_func 返回什么?"} C -->|"返回 local_trace_func"| D["追踪该函数内部后续事件 ('line', 'return', 'exception')"] C -->|"返回 None"| E["完全忽略该函数内部事件 (函数粒度过滤)"] ```

`sys.settrace` 的运作规则十分独特：当通过该机制注册了一个全局 trace 函数后，接下来解释器中**每一个函数的调用**都会强制触发一个 `call` 事件并执行该回调。但这个全局 trace 函数并不能直接监听该函数内部具体的每一行代码；相反，它的职责是返回“另一个局部 trace 函数”。如果它返回了一个新的 trace 函数，解释器后续才会用这个返回的函数去监听该函数体内部的 `line`、`return` 和 `exception` 事件；如果它返回 `None`，则意味着告诉解释器“我对这个特定函数内部发生的一切都不感兴趣”，解释器便会跳过该函数后续所有行级事件。

高天强调了这一设计的两面性： - **全局性**：“that this specific mechanism is a global mechanism. Which means as long as you set_trace, all of the function calls will trigger the call event.” 开发者无法做到仅监听某几个特定函数的调用而不触发其它函数的 `call` 事件，一旦开启就是全局介入。 - **粗粒度与细粒度的割裂**：事件控制的最小颗粒度是“函数级”。对于某一个具体的函数，你只能选择全开（捕获其内部所有事件）或者全关（返回 `None` 彻底不捕获），不能在函数内部仅选择性开启某几行。

此外，高天补充说明，Python 也支持通过给特定的 frame 对象的 `f_trace` 属性手动赋值，将 trace 函数直接挂载到当前已经执行了一半的局部栈帧上，这为接下来剖析 pdb 的实现埋下了伏笔。

---

[06:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=360s) pdb.set_trace() 的执行流程与 REPL 交互本质

弄清了 `sys.settrace` 的原理后，高天揭示了 `pdb.set_trace()` 的内部实现。许多人以为 `pdb.set_trace()` 有某种黑魔法，但它在本质上就是一个极其普通的 Python 函数，并没有任何深奥莫测的特殊语法支持。

当代码中执行到 `pdb.set_trace()` 时，它只完成了几项关键操作： 1. **注册全局 Trace**：调用 `sys.settrace()` 注册自己的跟踪回调函数，确保之后的所有函数调用事件都会通知到 pdb。 2. **回溯并修补上层栈帧**：因为按照标准机制，局部 trace 函数必须在函数刚刚被 `call` 时返回才能捕获行事件，但此时所有处于调用栈上层的函数（caller 以及更外层的 frames）早已经完成调用了。为了解决这个问题，`pdb.set_trace()` 会手动遍历调用栈上的所有外层帧，逐个将它们的 `f_trace` 设置为该跟踪函数。 3. **静候下一次行事件**：完成设置后，`pdb.set_trace()` 函数直接返回。紧接着，被调试程序准备执行下一行代码，此时立刻触发下一个 `line` 事件，调试器的回调函数随之被激活。

```mermaid flowchart TD accTitle: pdb.set_trace 内部初始化流程 accDescr: 展示执行pdb.set_trace时注册全局追踪与追溯上层栈帧的过程

Start["代码执行 pdb.set_trace()"] --> RegGlobal["调用 sys.settrace(pdb_trace) 确保捕获未来所有的 'call' 事件"] RegGlobal --> LoopFrames["回溯遍历调用栈所有上层已存在的帧 (Frame Hierarchy)"] LoopFrames --> PatchFrame["手动将每个栈帧的 f_trace 赋予 trace 函数"] PatchFrame --> Return["pdb.set_trace() 执行完毕返回"] Return --> NextLine["程序准备执行下一行代码 立即触发 'line' 事件"] NextLine --> EnterREPL["pdb 收到当前 frame 参数 启动类似 REPL 的交互循环挂起程序"] ```

当 trace 函数在 `line` 事件中被触发时，解释器会将当前正在执行的栈帧对象（`frame`）作为参数传给它。pdb 在拿到当前 frame 之后，其暂停程序的机制非常直观粗暴：它直接在 trace 函数内部启动了一个类似于交互式终端 REPL 的命令行死循环（`while True` 配合读取标准输入 `input()`）。只要用户在控制台中没有输入 `c`（continue）或者下一步指令，这个死循环就不会退出，整个程序的正常执行流也就被牢牢“卡住”在调试器的这一行中。

一旦用户在控制台输入了 `continue`，交互循环退出，trace 函数返回，解释器才会继续往下运行下一行代码，并在下一个满足条件的事件处再次进入该循环。高天总结道，这就是 pdb 暂停程序以及控制进入交互界面的底层全貌。

至于调试器如何提取运行时的信息（例如调用栈信息和局部变量），高天指出这在 Python 中同样没有任何魔法。因为 CPython 运行时本身就已经通过链表形式把完整的调用栈帧（`frame` 链表，每个 frame 包含 `f_back`、`f_locals` 等属性）组装完毕，调试器只需要直接读取传进来的 `frame` 对象的内置属性，就能轻而易举地拿到当前的全部局部变量和完整的调用链轨迹。

[10:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=600s) Inspecting Execution Frames and Controlling Program Flow

Python debuggers rely directly on the execution frames exposed by the CPython runtime, meaning that most of the foundational machinery for inspection requires no special magic. When a trace function is invoked, it receives the active execution frame as an argument. Through the `f_back` attribute on that frame, a debugger can traverse backward up the call chain, which forms a singly linked list representing the entire call stack. Every single frame in this chain gives direct access to the program's runtime state: local variables are available in `frame.f_locals`, the line currently being executed is tracked in `frame.f_lineno`, and the associated code object resides in `frame.f_code`. From `f_code`, one can retrieve static metadata such as the function name and the filename where the code was defined. As Gao Tian puts it to the audience: “You don't even need a debugger to access these functions; you can just write your own program and, you know, access the frame and all those things.”

Beyond reading variables, a full-featured debugger must allow interactive execution, such as evaluating arbitrary user expressions (`x * 2`) or mutating state inside the suspended frame. This is conceptually implemented by running strings through Python's built-in `eval()` and `exec()` functions. However, writing back mutated local variables into a running frame has historically been tricky: in CPython, function-level locals are stored in an optimized fast array (`fast locals`) rather than a standard dictionary. Modifying `frame.f_locals` required specialized C-level hacks to copy values from the dictionary back into the fast variable array, an area that has undergone major redesign in Python 3.13 via PEP 667.

When it comes to stepping and execution control, debuggers offer commands like `step`, `next`, `return`, and `continue`—vocabulary largely inherited from GDB. Gao Tian explains that behind these varied names, the underlying mechanic is identical: determining where and when to halt execution. Because `sys.settrace()` triggers a line event immediately before any bytecode line runs, the debugger already pauses before every execution step. The sole responsibility of the debugger is deciding whether the upcoming line matches the user's requested stopping condition.

- **Stack traversal**: `frame.f_back` chains execution frames into a linked list from the current execution site all the way to module scope. - **Frame state introspection**: `frame.f_locals` exposes local variables, `frame.f_lineno` provides the active line number, and `frame.f_code` holds file paths and function identifiers. - **Expression evaluation**: Uses Python's native `eval()` and `exec()` execution contexts against the current frame's namespaces. - **Fast locals caveat**: Function locals reside in CPython's fast variable slots rather than normal dictionaries, requiring synchronization tricks until PEP 667 modernized `f_locals`. - **Stepping logic**: Commands like `step`, `next`, and `return` do not manipulate the instruction pointer directly; they evaluate whether the upcoming line event fulfills the stop criteria.

```mermaid flowchart TD accTitle: Trace Hook Frame Inspection and Execution Control accDescr: Diagram showing how a trace hook receives an execution frame, inspects stack frames, and checks break conditions. subgraph StackTraverse["Call Stack (Linked List)"] F1["Current Frame (frame)"] -->|"f_back"| F2["Caller Frame (frame.f_back)"] F2 -->|"f_back"| F3["Root Frame (...)"] end

subgraph ExecControl["Debugger Decision Loop"] Lineno --> Decision{"Is this line / frame a target stop?"} Decision -->|"Yes (step/breakpoint hit)"| Prompt["Prompt PDB REPL (eval/exec)"] Decision -->|"No"| Resume["Resume Execution until next event"] end ```

---

[15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) The Tracing Overhead Crisis and the Promise of `sys.monitoring`

While `sys.settrace()` makes implementing a debugger straightforward, it introduces severe runtime overhead. Because standard tracing operates at a coarse granularity, setting a trace function causes CPython to emit an event for virtually every single line of code and every function call across the entire process. Gao Tian demonstrates this bottleneck with a simple benchmark: a test script defining a trivial `add()` function and a recursive Fibonacci function calculating `fib(24)`. Without any debugger attached, the code finishes almost instantly. However, attaching PDB and setting a single breakpoint inside `add()` inflates runtime to approximately 416 milliseconds—a 100x to 200x slowdown, even though the breakpoint has nothing to do with the heavily executed Fibonacci loop.

This overhead occurs because `sys.settrace()` must fire thousands of call events and line events inside `fib()` just in case the debugger wants to inspect them. While in theory a debugger could return `None` from a call event to disable line tracing inside that specific function frame, PDB does not optimize this aggressively. Furthermore, recursive functions still generate massive volumes of call events regardless, and any function containing a tight loop alongside a breakpoint will fire line events on every single iteration. Gao Tian underscores the core architectural flaw: “The granularity is just too coarse: you can either set all of the call events or not, and in a function, you can either enable all line events or none.”

To resolve this bottleneck, Python 3.12 introduced PEP 669 with the `sys.monitoring` API. Unlike `sys.settrace()`, `sys.monitoring` is an event monitoring system that can register listeners locally per code object. This allows a debugger to enable call events strictly on the specific functions it cares about, leaving functions like `fib()` completely unmonitored. Crucially, `sys.monitoring` also supports selectively disabling events on arbitrary lines at runtime. If a debugger hits a line event inside a loop and determines that line is not a breakpoint target, it can instruct CPython to deactivate event generation on that specific line, ensuring subsequent loop iterations run at full native speed without interrupting Python execution.

- **Performance penalty**: A single breakpoint in an unrelated function caused a recursive `fib(24)` test to slow down by ~100x (taking ~416 ms). - **Trace granularity limitations**: `sys.settrace()` forces global dispatching of call events and blanket subscription to line events across active frames. - **Per-code-object monitoring**: `sys.monitoring` (PEP 669) permits activating callbacks only on targeted functions rather than globally across the interpreter. - **Arbitrary line disabling**: Debuggers can dynamically deregister events on non-breakpoint lines within loops, eliminating repetitive callback overhead.

```mermaid flowchart LR accTitle: Comparison between sys.settrace and sys.monitoring accDescr: Contrasting global event dispatching in sys.settrace with selective code-level dispatching in sys.monitoring. subgraph LegacyTrace["Legacy: sys.settrace()"] C1["Interpreter"] -->|"Fires every call event"| T1["Global Trace Hook"] C1 -->|"Fires every line event"| T1 T1 -->|"PDB filters events in Python"| PDB1["Massive Overhead (100x Slowdown)"] end

---

[20:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=1200s) Implementation Complexities, Backward Compatibility, and Ecosystem Adoption

Combining per-code-object monitoring with selective line disabling unlocks the possibility of near-zero-overhead breakpoints, delivering up to a 100x performance gain for Python debugging. However, realizing this potential presents steep engineering challenges. Under the old `sys.settrace()` model, the engine simply dumped all execution events into PDB, leaving PDB's internal logic to filter what to ignore. With `sys.monitoring`, the debugger must actively orchestrate exactly which events are bound to which code objects and dynamically reconfigure event subscriptions as execution advances. As Gao Tian remarks regarding the balance between speed and architectural difficulty: “It's arguably much more complicated, because before, we got every single line event and we could just decide when to stop inside PDB's logic; now we need to make sure that we set the correct events in the correct place.”

The second major hurdle is backward compatibility within the standard library. Changing PDB's internal backend from `sys.settrace()` to `sys.monitoring` must be invisible to end users, preserving expected command behaviors and third-party integrations across different Python versions. Because of these stability constraints, official PDB adoption in the standard library remains an ongoing effort that has not yet landed in upstream releases.

Nevertheless, third-party developer tools across the Python ecosystem have embraced PEP 669 to accelerate debugging and code coverage: - **PyCharm / pydevd**: JetBrains' debugger backend (`pydevd`) incorporates `sys.monitoring` to eliminate tracing slowdowns. - **VS Code / debugpy**: Microsoft's Python debugging team is actively implementing support for `sys.monitoring` inside `debugpy`. - **Coverage tooling**: Ned Batchelder's `coverage.py` and newer coverage utilities have integrated `sys.monitoring`, documenting substantial runtime speedups in community benchmarks.

Gao Tian closes the presentation by noting that as these tools adopt the new monitoring infrastructure, Python developers will soon have access to debuggers and profiling systems that run orders of magnitude faster than traditional PDB.

## Notable quotes > [15:00](https://www.youtube.com/watch?v=kzTDEaQx0us&t=900s) The granularity is just too coarse: you can either set all of the call events or not, and in a function, you can either enable all line events or none. — 高天

## Mentioned 高天, pdb, Python, CPython, sys.settrace, sys.monitoring, PEP 667, PEP 669, VS Code, PyCharm, GDB, pydevd, debugpy, coverage.py, Ned Batchelder, JetBrains, Microsoft, PyCon US

## Video description 终于把这个东西翻译完了，双语字幕的我今年PyCon US 2024的Talk——What makes a Python debugger possible and how can we make it 100x faster。对pdb背后的机制感兴趣的小伙伴可以看一下！

[Source](https://www.youtube.com/watch?v=kzTDEaQx0us)

中文简报

PyCon US演讲：pdb的基本机制解析本场演讲深入剖析了 Python 进程内调试器的底层实现机理与演进方向。高天首先讲解了 pdb 如何利用 `sys.settrace()` 捕获全局调用与代码行事件，并通过回溯栈帧修补 `f_trace` 以及启动类 REPL 交互循环来挂起程序和检查局部变量。随后，他通过基准测试指出传统跟踪因粗粒度事件派发导致程序面临上百倍性能衰减的致命缺陷。最后，他详细介绍了 Python 3.12 引入的 `sys.monitoring`（PEP 669）机制，展示了如何通过按代码对象按需监听和动态禁用非断点行事件来实现近乎零开销的高性能调试，并展望了该特性在 pydevd、debugpy 等生态工具中的落地应用。 Python 标准库调试器 pdb 在底层依赖 sys.settrace() 机制，但由于其必须全局分发 call 事件且只能函数级过滤 line 事件，会导致测试中递归代码（如 fib(24)）出现高达 100 到 200 倍的严重性能衰减。 (15:00) pdb 暂停程序和单步调试并无特殊硬件级黑魔法，本质是在 trace 回调中直接运行类似 REPL 的交互式 input 死循环，并通过对下一行代码事件的比对来决定是否停靠。 (6:00) Python 3.12 引入的 PEP 669（sys.monitoring）通过以代码对象（code object）为粒度的局部监控与动态关闭非断点代码行事件，成功打破了全局追踪瓶颈，为调试器带来近零开销运行的可能。 (15:00) 尽管 sys.monitoring 因需要主动编排事件和保障向下兼容而在标准库 pdb 中尚未正式合入，但第三方调试工具如 PyCharm (pydevd)、VS Code (debugpy) 以及 coverage.py 均已跟进集成该机制以实现百倍提速。 (20:00)

Reading summary

主讲人高天在 PyCon US 的演讲中指出，Python 标准调试器 pdb 依赖解释器内置的栈帧链表与 `sys.settrace()` 机制实现进程内暂停与求值，但粗粒度事件分发会带来高达 100 倍的性能损耗；而 Python 3.12 引入的 `sys.monitoring` 正推动调试与分析工具进入低开销时代。进程内调试器的四大能力与栈帧自省机制 (0:00) 高天将讨论范围限定在调试器与目标程序位于同一进程的“进程内调试器”。一个合格的调试器需要具备四项基本能力：在任意指定位置挂起目标程序、获取运行时上下文信息、动态执行代码与修改变量，以及细粒度控制程序单步执行节奏。高天强调，调试器本质上只是一个普通程序，获取运行时状态并不依赖黑魔法，而是直接利用 CPython 运行时暴露的原生数据结构。当执行中断时，调试函数会接收当前的执行帧对象 `frame`。CPython 通过单向链表结构以 `frame.f_back` 记录完整的调用栈轨迹，调试器可通过 `frame.f_locals` 读取局部变量、通过 `frame.f_lineno` 确定当前执行行号，并利用 `frame.f_code` 提取函数名与源文件等静态元数据。表达式求值通常依赖内置的 `eval()` 和 `exec()`；而针对修改局部变量的需求，由于 CPython 早期将函数内部变量保存在快速数组而非普通字典中，写回变量长期需要底层特殊处理，直到 Python 3.13 借助 PEP 667 才完成了规范化改造。 sys.settrace 的事件分发与 pdb 交互实现 (2:30) 在日常开发中，无论使用 Python 3.7+ 的 `breakpoint()` 还是传统的 `pdb.set_trace()`，底层都依赖 `sys.settrace()` 注册的回调函数。该机制负责监听四种核心解释器事件：函数调用的 `call`、函数退出的 `return`、行代码即将执行的 `line` 以及异常抛出的 `exception`。该机制具有强烈的全局性，一旦启用，解释器内所有函数调用都会触发 `call` 事件；全局回调必须返回一个局部跟踪函数才能继续监听该函数内部的行事件，若返回 `None` 则忽略该函数内部所有后续事件。调用 `pdb.set_trace()` 时，其内部首先调用 `sys.settrace()` 注册全局回调，接着反向遍历调用栈上已经存在的上层栈帧并逐一对其 `f_trace` 属性赋值，随后函数退出并等待被调试程序在执行下一行代码时立即触发 `line` 事件。在收到当前栈帧后，pdb 会在回调内部启动一个基于死循环与标准输入的 REPL 命令行交互界面，从而将主程序执行流牢牢卡住；诸如 step、next、return 等步进指令，本质上并不是直接控制指令指针，而是由调试器在每次行事件到来前判定该行是否满足预设的挂起条件。全局跟踪的性能瓶颈与 PEP 669 针对性优化 (15:00) 以 `sys.settrace()` 为核心的传统实现存在严重的性能代价。高天在现场测试中演示，在一个包含密集计算 Fibonacci 数列 `fib(24)` 和简单 `add()` 函数的程序中，仅仅在与递归逻辑完全无关的 `add()` 中设置一个断点并挂载 pdb，运行耗时就激增至约 416 毫秒，带来高达 100 至 200 倍的性能减速。这种损耗源于粗颗粒度的事件调度：即使不关注某段代码，解释器依然会针对每一次函数调用派发事件，并在循环体内为每一行字节码反复触发回调通知。为彻底解决这一架构缺陷，Python 3.12 引入了 PEP 669 及其 `sys.monitoring` 接口。与全量捕获的传统机制不同，`sys.monitoring` 允许针对特定的代码对象（Code Object）局部注册监听器，使无关的密集计算函数能够以纯原生速度运行。此外，该接口支持在运行时动态禁用单行代码的事件监听，如果调试器判定循环内部的某一行不是断点目标，即可即时停用该行的事件回调，避免在循环迭代中反复进入调试逻辑。架构重构挑战与第三方工具链的演进 (20:00) 利用 `sys.monitoring` 实现极低开销断点虽可带来百倍级别的性能飞跃，但也大幅提升了工程实现的复杂度。旧机制只需无脑接收所有事件并在内部统一过滤，而现代方案要求调试器主动管理每个代码对象与特定事件的绑定状态，并在代码执行期间动态调整监听范围。与此同时，标准库中的 pdb 必须严格保障向后兼容性与各版本的行为一致性，因此官方 pdb 底层向 PEP 669 的迁移仍在推进之中，尚未正式合并入上游稳定发行版。不过，第三方生态工具已在加速拥抱这一新架构。JetBrains 的调试器内核 `pydevd`（服务于 PyCharm）以及微软面向 VS Code 开发的 `debugpy` 均已推进或落地对 `sys.monitoring` 的适配；Ned Batchelder 维护的 `coverage.py` 等覆盖率统计工具也相继接入了新 API，并在社区基准测试中展现出大幅度的执行提速。随着生态演进，Python 开发者将普遍获得运行开销显著降低的现代调试体验。 Python 调试器从 `sys.settrace()` 到 `sys.monitoring` 的演进，解决了长期存在的全进程跟踪性能拖累；尽管标准库 pdb 的完全现代化改造仍受制于兼容性包袱，但更广泛的 IDE 与代码覆盖率生态已先行兑现百倍提速的性能红利。

Source quotes

[15:00] 高天: The granularity is just too coarse: you can either set all of the call events or not, and in a function, you can either enable all line events or none.

Original transcript

[0:00] Okay, welcome everyone. to the top of what makes a typo bigger and how we can make it 100x faster via CamDom. Cam is at, it's quite about five minutes at the end, for Q&A. So there's -- this microphone is going to be on that standard. We've got to get it out if you have any questions. you can go up to the microphone and ask an idea. So yeah, keep it on your table. for one, my name is Ian. And today I'm going to talk about how to look So, uh, quick survey report. And how many of you has used what type of data report? And how many of you know how it works and why it works? Right, you knew everything, so you're good to go. Just kidding. Today we're going to talk about the mechanics, what makes a high-dose motor possible, and how can we make it? much faster than Thank you. Stop. what makes it the better. I mean, you can think about it, development is just a program, right? And there are different types of things that you might have other. Thank you. Thank you. So in order to meet a developer, You can be able to stop the program at an arbitrary position. and, you know, we can start checking on this workflow premise TV. and they need to be able to get family information. for example, the local variables, the cost facts, understand what your program is doing, what is the current situation. And it's not a must, but it would be nice if you can run some diagramming the way. For example, if you do some evaluation of these questions, or to some statements. Right? Call some functions. or assign a new value to your variables. That would be really a good thing. And last but not least, Uh, you need to be able to control the execution. step into, step outside. basically, ask your program to run at a pace that you desire. Like, stop wherever you want so that you can, you know, understand what the program is doing. And today in our presentation, we are talking about an in-proc debugger or debugger that runs in the same process. That's your program at Vortipati. So the ERP is out of process debuggers, and they are kind of different forms. So remember, in today's implementation, We're talking about the debunker intelligence. in the same process as you were debugging. How to bring that bugger? That's something that we want to talk about. Four. We talked about all of the mechanics and Malcolm Maddox. Some people are using different kind of debuggers, right? Some people use the GUI debugger in VS Code or PyTor. And some people just do it. for people to mount white bugger. And if you use PDB, and which is the EastCardner Library debugger. you probably will do that. something like not in the in your source code. Of course, we can do it in a... within Hawaii, but that's something that's very common. You add it and there's stuff there. But if you are older If you have. long-term experience. With Python, you are probably more familiar with PDB.setphrase. which does almost the exact same thing as Repoint. Print point is a very common term among all the people are nursing all different kinds of languages, right? That's a common thing. But That's crazy. It's a really weird part of it. You don't hear that in other languages. Why, French fleas. that. is related to the magic behind. buying off all the Python connectors. And this is that's that's nice. And in case you notice they have to share the same name, so crazy. So-- What does that step phrase does? is that. that a trigger function or a callback function. that sense. on certain events. Like a called event, a function is being called. A returned event, function is being returned. A lying event. The line of code is about to be executed. or an exception or an exception is raised. Of course, you can deal with possible, but that's kind of the stories we're going to talk about today. and the outcome isn't how it actually works is that After you do, this concept phrase your print function. That's the last line of the-- the last line. I'll put it there. for all the functions [5:00] - doctor. it will trigger a call event. And I think that... calling back it will uh, enough, you work. a case function, you will call your case functions. And your plus function is supposed to replace Yeah. another pre-flexion. That's beautiful. with all the events inside the function So you can return none. That means I don't go to So, I used to this specific function. many events in this function that will take care. Otherwise, you should return a function that deals with the return event, the lie event, the exception event in that specific function. And from the description above, you can realize that this specific mechanism is a global mechanism. Which means As long as you set the price You... all of the functional thoughts will trigger the quality of it. it's impossible to turn object call events on some of the functions and turn it on others. That's implies what was the block loop, so that we can't Okay. meet that granularity is function. You can either turn on all the events Can this be a thumbs up? Or you can turn off all of the events In that function, there is no better granularity. So they're on that path. and on the function level. And one thing to mention is that. It can be set manually for a function, for example, if you are in this. the function ha. I mean, the function must clock and start return, and, you know, your inside function is possible, truth set the crease function manually on the S-Sick Functions so that you can get the one event and the two events after the vaccine. That is something that I mentioned here, because, when we talk about maybe being a little bit more, what PDB.SitFace does. And with the statement, that is the alternative up. that we just mentioned earlier. So pb.setTrace is really just a normal Python function. There's no But magic, isn't it? So things like that, it is called It just of how this stuff separates on the subject of the mention. to set the trace function to set the trace function. or it's basically a callback function. so that every time an event triggers, it can call that just a function. And also, it sets the exact same trace function on all the frames above the color. to all the functions above the column. So, like we said, You are supposed to send a trace bin for a specific function when the function is being called. But all those functions are already caught. So I'm going to get to math. Many waves went through it, but those functions That's what it does. And that's basically Excellent. and then it returns from the object and waits for the next checker. So all the live events, all the events, the return events, All those wins are set in the functions above all the functions before. and just waits for a Normally, the trace function is called before executing the exact next line, because that would be the next event, the next line event. and for the credit function that receives the current running frame as an argument. I think me. which function, it is real, where the There's no lie even when it's about to execute the law. and the PDB in the trace function, Just as far as the commandment is similar to Ripple. Thank you. you must be writing, written in this code before, or like this wildcrawl and includes Get a screen. our brain. because I switched the Nats. Break up Ritaloop or Doodleoop Gap That's exactly what PDV is. and just start something like that and then if you get to use or use the command line continue or Thanks. you will hope. return from the tree function and wait for the tree function. for the next event procedure. So that's-- how is that program there, and how to enter PDB and the-- very basic structure of WebPDB. What about information information? We need to So, you can see the exact information And the most common one is the call stack. or part of the function clause. that we keep this point. and in Python They are just rain offense. They are like Now we're going to have that that CPython already put together for us. and put a lot of people in the game So there's no magic that we need to do there. the current frame. if they're me. [10:00] the frame that triggers the line amount of the frame 30 minutes. It's the argument. that pass into our clear function, so we get that for free. And the color of the curve frame is in. frame.fax. which is also put together by CPipe already. so just not I think we need to do that. And. That's exactly how it is basically just a link list. of French or French. call it the Hispanic List. rates. and we can just access everything. Simple, right? And for each flipping frame, You can access the local variables in frame.aslocal. put together by C-Python. And you can access the current line number, which y is .x cubed in frame.fy number. Yeah. And then there's this code object, which has information like where is this function defined which file. or what is the function name of that function of the information after them to this cell. And you can access that in frame.fco. Those days just there. You don't even need a debunker to access these functions. You can just write your own program and, you know, access the frame and all those things. So, know Dr. Magic Bear. and codex choose him. That's something that we want right to move up to. be able to evaluate an arbitrary expression. For example, if they already have a local variable X, I probably want to know what this X times Q. And it would be nice to be able to achieve some cool find some new variables to the-- assign some new values to the local variable, or, you know, run some function. And you think about that. That is basically just groundbreaking strength. And that's a good goal. afraid of exit. space times space 2 and cover that code and write that. And those functions -- are the definition. of the built-in function email and email. So it's also something that's the Python holiday. Well, let's... The only thing that's a little bit tricky is how to steward a result. into the local area. whatever. familiar with Python in Python functions, the local variables are like fast variables. They are not stored in the dictionary. actually stored in their locals. their sort is new level. So there are some tricks there. to start back. the variables starting the dictionary back to the... of fast miracles. But... That has been changed by 267. And so that's why I'm not going too deep into this. because there's a different story in fact on 20 to 30. Excuse and control. We have different names. or accident control in different language, right? Those common ones are -- that in queue - Thank you. - Yeah. and being Python or in PDB, you'll have different names, like staff, mags, the time. which are mostly borrowed from GDP. Could you really, like, think about this? It's just... stop. at some. Thank you. All those different kinds of cognates, too, It's basically just start that different process. place. uh, of the Eno-Cruber. And the only thing that debugger needs to do is to determine Where is something? And if you remember what the metabolism for monuments before every single line of code x cubed We fire a live event. and we bring up the people. So PB knows, okay, computer and software. that is about rescue. The only people who need to know, people who decide Why do we stop now? Thank you. basically because We've got defense. A big time. a line that's about to execute. can be based on a specific. Not here, not here. And that's how can you be able to all of these experts programs. So if you like. or collection of the think about what I talk about, The replacement is Pretty simply. Like, it's like CPython has already put together everything for us. They provide like this. That's that phrase. for us and it provides all of the information there in the frame. and Thank you. But... What about the overshot? Think about that. It could be possible. Thank you. [15:00] It's bigger. event and be simple line of code. That could be-- that could be a really huge overhead. So we're going to take an example. So I have this piece of code. That I want to... and there is an add function which is trivial, And that was set up before here. And I have plastics. with the NACI. Thank you. which is implemented in recursive style. And in my test function, I've run a good patch of 24 first. and then I run the absolute. So without any debugger. You can run this directly. but I think But if you attach PB and set a single break point in a tab, It takes about 400 and 16 milliseconds. That's about 100x small down. about protection. closer to it. 210. uh What? X-Men, P-C-Y, I mean, everyone can figure it out. It's just because all 3,000 events like I thought, you can You can only suck and Call the medicine. for every single function. pause. Imagine you get lucky. There are a lot of call events that are going to be done. which are not relevant to completely the law. And of course, you have a lot of blind events or alpha-touch functions, that is not them. Um, if you have to listen to it. presentation. really carefully. and realize, wait, You said that the granularity is functioning. Why can't we turn off the events inside flip function because those are not useful with only cloud events on events. That's actually correct. but it's not implemented in PDB yet. So portal class are welcome. But Even if you turn up events in a state function, there will still be call events. And because hypnachia is a recursive function, you will have a lot of compliments. Bye. as you can imagine it If you have a The visual code that inside a single function Thank you. Beautiful. Yeah. It comes in. And the liner you want to break after that, you will still need to trigger out alignment within the form. You cannot turn the law. It's not an option. So-- Even though in this case it can be optimized, there will still be a lot of other cases and issue. and make this you So... is that linearity is just-- true corpse. You can either set all of the call-ins or not. And in a function, you can either set, like, enable audience, You don't get the way we have. and that is slowing down our debugger that makes If otherwise, 3.6. so much slower than the normal production. and Good news is that we have a new tool. and it's called Sys.micro. So Scott Monkron is a low-cost monitoring system that can be set locally on a cool oven. And what that means is that now I can only enable me cause event on certain functions. but not in others. so that in the example of a top, I can only in my book how to then update that comes or maybe not even climate, but they can. and I can disable the parliament. for these big functions. that's something that can reduce the overhead clock. AND, UH, to have on turn can't enable any subset of events. System entry into a lot of a different kind of events, but even to four participants. that crazy. that's like live in the car. Thank you. exception, right? And in the box, you can give them all or to talk about But what's that mantra, you can Fickin' cheese. even though any event status is useful, you have a resource for you, and disable the others. And the really good thing is that it's possible to basically invent on the arbitrary line after event. fight it. So what that means is that if you acquire a vehicle for loop inside a function. and you're configured as one element. and PD weekend. they started like, OK, line is not useful. Soap. Do not send AQMe anymore. like to not request to not fire events. on that line. and it's possible to be sublugged. and it will not bother peeing again. The difference, as we imagine, is that before, stuff. That's that phrase. Bye. the my guns sends all the events to me and You stop on this line, stop on this line. [20:00] No, no, no, no. now. to say, No. and not any more. So they want And all of those combined together that it is Thank you. for us. to receive Almost. zero over track. Great point. if we can implement it correctly, so that we only fire to them. when we cheat. And that means of 100 acts. improvement. Everything sounds quick, right? Fun, theater, song. Issues. Principal. - Cool. It's arguably much more complicated. full capacity I mean, before, we click around every single one event, and, uh, You know, we can just decide when to stop inside D, inside D's logic. because we've got a lot of events. But now we need to make sure that we set the correct events in the correct fire. And, uh, back into that cluster. more work. So to enable the -- in order. to achieve. the minimum overhead. We need to figure it out. - Awesome. And the bigger problem is the backwards compatibility. Because the new mechanism there will be backwards from that different issue. Uh, can make others different. so people don't yo, we can keep pdb exactly-- almost exactly the same. uh, like the TV users feel Bye. That, in the back end, has changed. But it must be a little trick. beating. Facebook, that is a big thing. in your standard library. whether we can make the change or not. And All, of course, would use switching back end. on this big as full as possible, that if we to the moment. Wow. Right. So that's another thing. So that's why we have not I use this. Thank you. PVP yet. We have the Drap into the system, that kind of work. Ah, that is not in the of the community. Negative less. There are tools that author I'm bored. and are using this. of new mechanisms to achieve much better oversight. For example, Pinech Farm uses Pine D and Pine D. that D is fine system operating. And I believe Microsoft is working on it. the FNs for the VS Code, which also uh you can use this is that my training And if you are into coverage, there is a coverage stop height. Try this thing. But there is also another annual coverage tool And I believe they have a plot exception. as well. talking about system microin and how it can improve the speed and accelerate the US data with tools. stuff with that. uh Great news. yet there are others that try this out. so we can have a debug or a tab. much, much faster than PDV. Hopefully. OK. That's about everything in the U.S. right into people. Questions? happening.