"Performance (Really) Matters" with Emery Berger

Emery Berger 谈“性能（确实）至关重要” Emery Berger 解释道，随着登纳德缩放定律和摩尔定律的终结，软件性能无法再依赖硬件带来的被动提升，而微架构布局的脆弱性往往会导致标准性能分析器的数据产生误导。针对这一问题，他介绍了因果性能分析器 Coz，该工具利用“虚拟加速”（人为延迟并发线程）来准确预测优化特定函数将如何影响整体吞吐量和延迟。随后，他展示了专为 Python 设计的性能分析器 Scalene，它能够区分 Python 字节码、底层原生 C/C++ 库以及系统任务的运行时间和内存使用情况，同时追踪复制量等诊断指标。在各项基准测试以及 Ferret 和 Dedupe 等实际软件中，这些工具证明了极少数针对性的代码修改即可带来显著的性能提升。 2010 年左右登纳德缩放定律的终结使得单线程性能无法再自动增长，这让软件层面的优化在编译型语言和高级脚本语言中都变得至关重要。 (9:24) 传统性能分析器往往只报告执行时长而非因果关系，从而经常误导开发者，掩盖了优化某个热点是否真能提升应用程序的吞吐量或降低延迟。 (19:19) Coz 引入了因果分析，通过人为放慢并发线程来模拟“虚拟加速”，从而在修改代码之前准确预测整体性能的提升幅度。 (23:41) Scalene 为 Python 提供了大约 10% 到 30% 开销的性能分析，区分了 Python 字节码执行与原生 C/C++ 扩展，并追踪跨边界的内存复制量。 (43:11)

Computer scientist Emery Berger argues that the end of Dennard scaling has exposed the severe memory and computational overheads of modern scripting languages, while legacy profilers actively misdirect developers tackling concurrency and native runtimes. To address this, Berger presents causal profiling via Coz and fine-grained runtime attribution via Scalene to pinpoint actionable bottlenecks without relying on brittle microarchitectural side effects. The Post-Dennard Performance Deficit (6:53) In the 1980s and 1990s, rapid hardware acceleration under Moore's Law encouraged the emergence of high-level dynamic languages like Python, Perl, and Ruby. Berger characterizes these as 'irrational exuberance languages,' designed under the assumption that hardware gains would outrun structural bloat. For example, Python historically required up to 240 bytes for a basic integer—subsequently reduced to 24 bytes, but still six times larger than C++'s 4-byte baseline—while a naive matrix multiplication loop runs 47 times faster in C, and over 60,000 times faster when leveraging optimized C libraries. Microarchitectural Brittleness and Profiler Pitfalls (12:12) Developers can no longer rely on automatic frequency increases because Dennard scaling collapsed around 2010 due to power and thermal limits. At the same time, optimizing by hand is complicated by microarchitectural sensitivity. Citing research showing that semantically neutral modifications like changing environment variable lengths or link order can cause performance swings of ±40% due to cache-set conflicts, Berger cautions that modest benchmark gains are often layout artifacts rather than genuine improvements. Causal Profiling and Virtual Speedups in Coz (22:11) Conventional profilers track call counts and aggregate execution times, which frequently misleads engineers into optimizing non-critical components, such as UI loading animations. In concurrent, long-running server pipelines, Berger proposes causal profiling implemented in the open-source tool Coz. Rather than attempting to make code faster ahead of time, Coz pauses concurrent tasks to simulate the 'virtual speedup' of a target component, tracking progress toward user-defined latency or throughput endpoints via Little’s Law. Bridging Python and Native Execution with Scalene (36:36) Because data science relies heavily on Python orchestrating native C and C++ extensions, traditional profilers distort measurements when runtime signals are deferred until control returns to the interpreter. Scalene overcomes this by using deferred signal delivery to deduce whether CPU time was spent in Python, native libraries, or system I/O, tracking metrics within 1% of ground truth in calibration tests. Mitigating Bias through Runtime Randomization (55:46) Addressing concerns about measurement distortion, Berger notes that Coz's asynchronous interleaving continually churns thread schedules, smoothing out positional cache bias. This mirrors Berger's earlier work on Stabilizer, which continuously re-randomizes stack, heap, and code locations at runtime. Beyond eliminating layout anomalies for rigorous compiler evaluation, Berger confirms this continuous address disruption acts as a moving-target defense, eliminating the fixed address assumptions required for exploits like return-oriented programming. While tools like Coz and Scalene isolate true bottlenecks, modern memory layout remains so brittle that a single-character shift in user input can alter cache alignment. Berger's group plans to extend Scalene's profiling paradigm to languages like PHP and R, while exploring automated advisory systems that directly identify and recommend native library drop-ins for identified script bottlenecks.

马萨诸塞大学阿默斯特分校教授Emery Berger指出，随着丹纳德缩放定律失效，硬件无法再为软件带来自动提速，而底层微架构的脆弱性与传统性能分析工具的缺陷正误导开发人员；他为此提出了因果分析工具Coz与低开销Python分析器Scalene以实现精准优化。 免费午餐终结与脚本语言开销 (6:53) 在20世纪70至80年代，硬件性能随着摩尔定律持续跃升，开发者即使不修改代码也能享受硬件升级带来的性能红利。这种趋势催生了Python、Ruby、PHP等动态脚本语言，Berger将其称为“非理性繁荣语言”。这些语言优先考虑开发效率，却伴随着沉重的内存与执行开销。例如，C++中占用4字节的整型在Python中曾高达240字节，现今仍需24字节；在三层嵌套矩阵乘法测试中，未经优化的C语言实现比Python快47倍，而经过优化的C库甚至快60,000倍以上。 然而，这种性能红利在2010年左右戛然而止。尽管摩尔定律使晶体管数量持续增加，但由于热设计功耗受限与电压漏电，丹纳德缩放定律彻底失效，处理器的单核主频无法再持续攀升。这意味着自动提升单线程性能的“免费午餐”已经终结，开发者必须依靠严谨的工程手段来优化软件性能。 微架构脆弱性导致测量失真 (12:12) 现代多核处理器高度依赖组相联硬件缓存、分支预测器和TLB等复杂微架构组件，导致软件性能对内存和代码布局表现出极强的脆弱性。Berger引用2009年的研究《Producing Wrong Data Without Doing Anything Obviously Wrong》指出，链接器调整`.o`目标文件顺序或改变环境变量大小等语义中立的改动，即可使性能产生高达正负40%的剧烈波动。 这是因为环境变量大小改变会平移栈基址，而目标文件顺序变动会改变函数在代码段中的位置，这可能导致两条高频执行的指令映射到同一缓存组，从而引发严重的缓存冲突与内存停顿。Berger强调，这种随机布局偏差常常超过现代编译器优化通道带来的收益。在排除了布局干扰后，编译器优化带来的实际性能提升在标准测试集上微乎其微，这使代码调优面临极高的测量假象风险。 因果分析与虚拟加速破解并发瓶颈 (19:19) 传统的性能分析工具源于20世纪80年代，主要统计函数调用次数和总执行时间。在长时间运行、并发和异步的现代应用中，这种方法往往导致开发者把精力浪费在非关键路径上，例如持续运行的UI加载动画。为此，Berger团队研发了因果性能分析器Coz，利用因果图直接预测对某一组件加速后，整个系统吞吐量或延迟的改善程度。 由于在优化前无法凭空使某个模块变快，Coz提出了“虚拟加速”技术：通过人为暂停或延迟与目标代码并发执行的其他所有线程，制造系统局部变慢的相对效果，从而在数学上精准等价于目标模块获得了提速。开发者仅需在代码中插入吞吐量或延迟的“进度点”，Coz就能结合排队论中的利特尔法则计算延迟并进行预测。实际案例表明，Coz指导团队将Ferret图像搜索引擎的排序阶段线程数增加27%，实现了整体验证后的21%吞吐量提升；并帮助Dedupe定位到了将异或误写为加法的哈希函数缺陷，仅改动一个字符就消除了缓存桶冲突，使整体性能提升9%。 Scalene实现Python与本地代码解耦 (36:36) 针对广泛用于数据科学和机器学习的Python语言，Berger团队开发了开源分析器Scalene。传统分析工具无法区分Python解释器开销与底层C/C++原生扩展的执行耗时，且内存分析器可能带来高达150倍至1,000倍的严重减速。Scalene在保持CPU分析仅约10%开销、全功能分析约30%开销的前提下，清晰地将CPU时间拆分为Python解释器时间、原生C/C++代码时间和系统I/O时间。 针对Python在执行原生C扩展时推迟操作系统信号传递的机制，Scalene将这一限制转化为测量手段：若定时器信号准时到达，则计入Python时间；若信号被推迟，则将超出预期的延迟时间全额计入原生代码。此外，Scalene利用紧凑的波形图展示内存使用趋势，并引入“跨边界拷贝量”指标，帮助开发者识别在Python与原生环境之间低效搬运数据的行为。例如，在一个案例中，去除一段对原本已是NumPy数组的数据重复调用`np.array()`的冗余代码后，程序运行时间从6.3秒下降至5.4秒，内存消耗显著减少。 随机化消除布局偏差与安全延伸 (55:46) 在面对关于性能测试随机性与环境噪声的质疑时，Berger阐明了Coz的稳健性机理：Coz在多线程并发执行中随机注入延迟，不断扰动线程调度与内存访问顺序，这种类似“电子云”的动态混合天然平滑了局部的内存布局偏差。这一理念源于其早期的Stabilizer研究，该系统在运行时持续对函数、堆、栈和全局变量进行随机重排，以此消除虚假的对齐假象，评估编译器优化的真实效能。 Berger同时指出，运行时的动态内存重排技术在计算机安全领域具备重大价值，直接构成了“移动目标防御”。正如其研发的DieHarder安全内存管理器一样，通过在内存空间中不可预测地分散对象并辅以保护页，攻击者无法依赖固定的内存地址或结构体偏移，从而使利用缓冲区溢出或释放后引用漏洞实施漏洞利用的可能性基本降为零。 Berger总结认为，尽管Python等高级脚本语言提升了开发生产力，但性能瓶颈往往不可避免，关键在于借助工具精确识别纯Python开销并将其替换为原生库。目前Scalene的核心设计正被拓展至PHP和R语言；而关于能否利用因果分析指引编译器在关键瓶颈处执行深度、耗时的重度优化，仍是兼具潜力与工程挑战的探索方向。

## Background 该视频由国际计算机学会（ACM）于 2020 年 10 月 28 日在 YouTube 平台发布。内容为马萨诸塞大学阿默斯特分校教授 Emery Berger 主讲的技术研讨会，探讨在摩尔定律红利终结的背景下传统性能分析工具的失效，并介绍了其团队研发的两款新型 Profiler——面向并发底层系统的 Coz 与面向 Python 的 Scalene。 ## TL;DR Emery Berger 教授深入剖析了现代系统性能优化的困境：由于登纳德缩放比例定律失效与底层微架构极其脆弱，传统以耗时统计为核心的分析工具极易产生误导。为此，他详细讲解了基于“虚拟加速”理念的因果分析器 Coz，该工具可在不重构代码的前提下精准预测组件加速对多线程系统吞吐量与延迟的影响；同时介绍了专为 Python 打造的 Scalene 分析器，它利用信号延迟投递等巧妙机制，以极低开销精准拆解纯 Python 与 Native 代码的 CPU、内存占用及跨边界数据复制成本。最后，Berger 在问答环节解答了性能测量中的硬件噪声、编译器优化局限以及内存随机化对系统安全防御的深远影响。 ## Atmosphere & style 整场讲座兼具深厚的学术底蕴与轻松风趣的极客幽默，Berger 教授大量穿插《辛普森一家》的经典梗图与生动比喻，将晦涩的微架构冲突、排队论和系统底层机制拆解得妙趣横生。演讲与问答节奏张弛有度，既有对传统优化盲区的辛辣讽刺，又有扎实严密的实证数据支撑，令听众在轻松愉悦的氛围中体会到底层系统设计与性能调优的精妙艺术。 ## Bulletin - [09:24](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=564s) 摩尔定律主频提升与登纳德缩放定律在2010年左右走向终结，现代硬件过热与漏电限制了频率暴涨，迫使开发者告别靠硬件换代白嫖性能的时代。 - [12:12](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=732s) 传统性能分析工具极易产生误导，常将并发系统中的加载等待动画误判为最大优化瓶颈，且环境变量或链接顺序引起的内存布局微变就会引发高达±40%的性能剧烈摆动。 - [22:11](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1331s) 因果性能分析器 Coz 通过“虚拟加速”技术人为放慢并发组件，精准预测针对 C/C++/Rust 等底层系统代码的吞吐量与延迟优化收益，现已收录进 Linux 官方软件源。 - [42:31](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2551s) 针对 Python 开发的高效分析器 Scalene 开销仅为 10% 到 30%，利用信号延迟投递机制成功拆解纯 Python 与底层 Native C 执行耗时，并引入内存复制开销指标根除性能瓶颈。 ## Key takeaways - [04:38](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=278s) 80年代算力受限催生了汇编、Fortran 与 C/C++ 等贴底且零额外开销的“金属语言”，当今移动设备的综合性能较当时提升了约3,000倍。 - [06:54](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=414s) 硬件狂飙期引发了对脚本语言的“非理性繁荣”，导致 Python 在基础数据结构（如 int、list、dict）上的内存开销达到 C++ 的数倍之多。 - [09:24](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=564s) 以矩阵乘法为例，经算法与系统库深度优化后的 C 语言实现比纯 Python 原生循环快达 60,000 倍以上。 - [12:12](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=732s) 链接顺序或环境变量长度导致的微小内存布局改变，会影响缓存组命中、分支预测器和 TLB，其造成的性能波动甚至超过编译器 -O0 到 -O3 的优化幅度。 - [19:19](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1159s) 传统耗时分析器（Profiler）在现代多线程异步架构下失灵，容易将耗时最长但对吞吐量无贡献的代码（如加载等待动画）误判为优化目标。 - [22:11](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1331s) 因果分析器 Coz 运用相对论思想的“虚拟加速”机制，通过在运行时人为注入微延迟放慢并发运行的其他组件，数学等价地推导出目标代码的优化收益曲线。 - [26:01](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1561s) 开发者只需在代码关键业务节点插入轻量级“进度点”，Coz 便可基于利特尔定律推算虚拟加速对端到端延迟与吞吐量的真实因果影响。 - [28:57](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1737s) 在图像检索流水线 Ferret 中，Coz 指导团队重新编排各阶段线程配比，以极简代价获得了 21% 的应用整体吞吐量提升。 - [31:51](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1911s) 在 Dedupe 系统中，Coz 帮助排查出由于累加求和引发严重哈希冲突的致命 Bug，仅将加号改为异或符号就使数据桶利用率从 2% 飙升至 82%。 - [36:14](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2174s) Python 在现代高性能计算中扮演着“胶水语言”角色，核心优化策略是将纯 Python 耗时工作尽可能下沉到底层 C 扩展模块中处理。 - [39:05](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2345s) Scalene 能够精确将 CPU 耗时拆分为纯 Python 时间与底层 Native C 运行时间，并展示行级内存走势的 Edward Tufte 风格火花线图表。 - [39:05](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2345s) 跨越 Python 解释器与底层的“复制开销”（Copy Volume）是严重拖垮性能的隐蔽杀手，例如对 NumPy 数组误调 `np.array()` 会引发巨大的冗余分配与销毁峰值。 - [42:31](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2551s) 相较于传统 Python 分析工具动辄 10 倍至 150 倍以上的性能减速，Scalene 凭借微量采样将运行开销压低在 10% 至 30% 之间。 - [45:03](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2703s) Scalene 巧借 Python 解释器在执行 Native C 代码时延迟投递定时器信号的语言限制，通过比对挂钟时间差精准还原底层 C 库的实际耗时。 - [53:26](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=3206s) 为应对性能测量中的不确定性，Coz 采用无偏实验设计，而 Scalene 内置了置信区间过滤机制，直接隐藏高方差、疑似噪音的数据指标。 - [55:46](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=3346s) Coz 依赖多线程并发执行的高度随机性抵消硬件微架构层面的内存对齐偏差，该思路与团队此前用于评估编译器优化的 Stabilizer 运行时随机化系统相通。 - [58:52](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=3532s) 因果性能分析必须在经充分优化编译的生产环境二进制上执行；同时，动态打乱内存布局的 Stabilizer 与 DieHarder 在安全领域构筑了极难攻破的“移动目标防御”。 ## Detailed walkthrough ### [00:02](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2s) 主持人开场与 ACM 资源介绍 本次讲座由普渡大学（Purdue University）电气与计算机工程学院副教授米林德·库尔卡尼（Milind Kulkarni）主持。库尔卡尼首先向参会者致意，并介绍了国际计算机学会（ACM）的概况。ACM 目前服务于全球近 10 万名计算领域的专业人士和学生群体。库尔卡尼介绍了他个人的学术背景与研究方向，他的研究主要聚焦于开发能够在复杂的计算平台上高效挖掘复杂应用程序局部性与并行性的编程语言、编译器和系统架构；他本人既是 ACM 会员，也是 IEEE 高级会员。 库尔卡尼随后向大家展示了 ACM 为推动计算机专业发展所提供的丰富教育与学术资源。ACM 会员可通过 ACM 学习中心（learning.acm.org）获取前沿技术资源以保持职业竞争力。此外，ACM 作为推动全球计算创新的重要平台，持续出版包括《Communications of the ACM》和《Queue》在内的顶级期刊杂志，并维护着全球最全面的计算机文献数据库——ACM 数字化图书馆（ACM Digital Library）。学会还定期举办汇集前沿专家的国际学术会议，支持课程研发与教师培训，评选包括图灵奖（ACM A.M. Turing Award）和 ACM 计算奖（ACM Prize in Computing）在内的行业顶尖荣誉，并推行《ACM 伦理规范》（ACM Code of Ethics），为计算机从业者在专业实践中提供道德准则。 - ACM 全球会员规模：近 100,000 名计算专业人员与学生。 - 核心学术资源：ACM Digital Library、ACM Learning Center、《Communications of the ACM》、《Queue》。 - 行业荣誉与准则：图灵奖、ACM 计算奖、ACM 伦理规范。 --- ### [01:40](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=100s) 活动须知与参会互动指南 在正式切入技术主题前，主持人库尔卡尼详细说明了本次在线研讨会的平台操作指南与会务注意事项。演示幻灯片在讲座过程中会自动同步翻页，参会者如果遇到画面卡顿或音频中断等播放故障，可在 Windows 系统下按 F5 刷新、在 Mac 系统下按 Command+R 刷新，或者直接在移动设备浏览器中重新加载页面，必要时也可通过关闭并重新打开播放窗口来恢复。 关于提问与互动环节，参会者可随时在下方的 Q&A 对话框中提交技术疑问。库尔卡尼表示他将在演讲过程中实时梳理并整理这些问题，并在演讲结束后留出专门的问答时间请主讲人集中解答。整场讲座均有完整录制与归档，会后将向参会者发送通知邮件，相关内容也会在 ACM 学习中心发布。会议结束时页面将弹出反馈问卷，参会者还可利用社交分享工具通过推特话题标签 `#ACMweb` 进行讨论，未能在此次直播中完全解答的问题将移至 Discourse 社区页面继续深入交流。 库尔卡尼在说明提问机制时向观众明确承诺：“I'll organize the questions as Emery speaks and he'll reserve some time at the end of the presentation to address them.”（“我会在艾默里演讲期间整理大家提出的问题，他将在演讲结束前专门预留时间逐一解答。”） --- ### [02:59](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=179s) 演讲嘉宾介绍：Emery Berger 教授及其系统软件贡献 主持人正式隆重介绍了本场技术讲座的主讲嘉宾——马萨诸塞大学阿默斯特分校（UMass Amherst）信息与计算机科学学院教授艾默里·伯格（Emery Berger）。伯格教授曾先后担任微软研究院（Microsoft Research）以及加泰罗尼亚理工大学与巴塞罗那超级计算中心（UPC/BSC）的访问学者。他的研究横跨编程语言、运行时系统和操作系统，尤其专注于以透明、无侵入的方式全方位提升计算机系统的可靠性、安全性与运行性能。 伯格教授与合作者共同研发了多项具有深远产业影响力的软件系统： - **Hoard**：一种快速且高可扩展的多线程内存管理器，已被英国电信（British Telecom）、思科（Cisco）、瑞士信贷（Crédit Suisse）、路透社（Reuters）、加拿大皇家银行（Royal Bank of Canada）、SAP 和塔塔集团（Tata）等全球知名企业广泛采用，并且是苹果 Mac OS X 操作系统内置内存管理器的核心基石。 - **DieHard**：一种具备容错能力的内存管理器，直接影响了微软 Windows 7 容错堆（Fault-Tolerant Heap）的底层架构设计。 - **DieHarder**：高安全性内存管理器，为 Windows 8 堆内存的一系列安全加固机制提供了直接灵感与技术参考。 伯格教授曾获微软研究学者奖、美国国家科学基金会 CAREER 奖、Lilly 教学学者奖，以及 PLDI、OOPSLA、ASPLOS 等顶级学术会议的“最具影响力论文奖”（Most Influential Paper Award）和最佳论文奖。他于 2019 年当选 ACM Fellow，历任 SIGPLAN 执行委员会委员、ACM TOPLAS 副主编、PLDI 2016 大会程序委员会主席，并担任 ASPLOS 2021 的联合程序委员会主席。在介绍完毕后，库尔卡尼将主讲麦克风正式交予伯格教授。 ```mermaid flowchart TD accTitle: Emery Berger 核心系统软件与其产业应用 accDescr: 展示 Emery Berger 教授主导研发的 Hoard、DieHard 与 DieHarder 三大内存管理系统及其在各大企业与主流操作系统中的应用成果。 subgraph MemorySystems["核心系统软件研发成果"] H["Hoard 内存管理器<br/>（高并发、低碎片）"] DH["DieHard 内存管理器<br/>（容错与抗崩溃）"] DHR["DieHarder 内存管理器<br/>（安全加固）"] end subgraph IndustryAdoption["工业界与操作系统转化"] Apple["Mac OS X 内存管理器底层"] Enterprise["企业级采用：Cisco / SAP /<br/>Credit Suisse / Reuters 等"] Win7["Windows 7 容错堆 (FTH)"] Win8["Windows 8 堆安全加固"] end H --> Apple H --> Enterprise DH --> Win7 DHR --> Win8 ``` --- ### [04:38](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=278s) 硬件演进与“贴近底层”的金属语言时代 伯格教授在接过发言权并顺利共享屏幕后，幽默地用经典动画片《辛普森一家》（The Simpsons）展开了本场讲座的切入点。他展示了辛普森一家在 1980 年代首播时的初代草图画风与当今精致画风的巨大反差，直言这部常青动画在这几十年间发生了翻天覆地的视觉进化。令人惊奇的是，计算机硬件的发展轨迹与这种演进形成了完美的同频共振。 伯格教授对比了现代智能手机与 20 世纪 80 年代的主流计算机：如今人手一部的普通智能手机，通常配备 6 核心、主频超过 2 GHz 的 64 位处理器，以及 4 GB 以上的运行内存；而 80 年代的计算机主频仅以 MHz 计算，CPU 大多是 8 位或至多 16 位，内存容量更是微乎其微。如果将两者的综合计算能力与容量进行除法换算，当今的移动计算设备在各维度上综合性能提升了大约 3,000 倍。 在 80 年代之前，由于硬件速度极其缓慢、资源极度匮乏，程序员别无选择，必须使用没有任何额外开销、完全贴近物理硬件底层的编程语言。伯格教授形象地将这一时期的工具命名为“金属语言”（Metal Languages）——从最初的汇编语言（Assembler），到 Fortran，再到 C 和 C++。这些语言在设计之初就将消除抽象惩罚和零额外开销作为第一准则，因为在当时的硬件算力约束下，任何微小的性能冗余都是无法承受的奢侈。 伯格教授对该时期的语言设计特征总结道：“These languages, by design, have virtually no overhead, and that was really important because computers were just so slow.”（“这些语言在设计上几乎没有任何开销，这在当时至关重要，因为当时的计算机实在是太慢了。”） - 算力跨越：当今智能手机综合性能较 80 年代计算机提升约 3,000 倍（6 核 2+ GHz 64 位 vs. 8/16 位 MHz 处理器）。 - “金属语言”（Metal Languages）：汇编、Fortran、C、C++，核心特征为极致的贴近底层与近乎零运行时开销。 --- ### [06:54](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=414s) 摩尔定律红利与“非理性繁荣”语言的内存膨胀 从 70 年代末到 80 年代初开始，计算机世界步入了一场长达数十年的性能狂欢。摩尔定律（Moore's Law）以每 18 个月芯片晶体管数量翻倍的指数速度向前推进，与此同时处理器的时钟主频也在持续飙升。在晶体管密度暴增与时钟频率跃升的双重推动下，整个计算工业进入了一个奇妙的黄金周期：软件工程师即使不优化代码，只要隔年买一台新计算机，程序性能就会自然而然地成倍加速。 这种硬件算力的指数级爆发，让产业界产生了一种金属语言或许已不再必要的乐观错觉。工程师们认为硬件已经足够快，完全可以容忍在软件设计中牺牲运行效率以换取开发便利。因此，80 年代至 90 年代催生了大量执行效率远低于 C/C++ 的新型脚本语言，包括 Perl、Tcl/Tk、PHP、Ruby 以及 Python。伯格教授借用格林斯潘的经济学热词，将这批语言戏称为“非理性繁荣语言”（Irrational Exuberance Languages）。当时的设计者们普遍误以为算力将永远无限制地变快下去，因而任由显著的系统低效在语言底层肆意蔓延。 伯格教授以荷马·辛普森（Homer Simpson）凸起的啤酒肚作比喻，一针见血地指出这类动态语言在内存空间占用上的严重“肥胖膨胀”（bloated）。以 Python 为例，数据结构的实际内存开销令人触目惊心： - **32 位整数（int）**：在 C++ 中仅占 4 字节，而在 Python 中由于动态对象头与元数据的封装，膨胀到了 24 字节，开销高达 6 倍； - **空列表（empty list）**：在 C++ 中开销为 24 字节，而在 Python 中达到了 56 字节，直接翻倍有余； - **空字典 / 映射（empty dict）**：在 C++ 中只需 24 字节，而在当今的 Python 中则高达 64 字节，接近 C++ 的 3 倍。 伯格教授在剖析这种语言设计思潮时尖锐地指出：“These were languages where people simply thought things were just going to get faster and faster forever, and so it was okay to have pretty substantial inefficiencies.”（“这些语言诞生的时期，人们单纯以为计算机会永远越来越快，因此觉得存在相当大的效率低下也是完全可以接受的。”） | 数据结构类型 | C++ 内存开销 | Python 内存开销 | 膨胀倍数 | |:--- |:--- |:--- |:--- | | **标准整数（int）** | 4 字节 | 24 字节 | 600%（6 倍） | | **空列表（Empty List）** | 24 字节 | 56 字节 | 233%（约 2.3 倍） | | **空字典（Empty Dict）** | 24 字节 | 64 字节 | 267%（约 2.7 倍） | ### [09:24](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=564s) 脚本语言的内存与性能劣势，以及摩尔定律“免费午餐”的终结 Emery Berger 指出，像 Python 这样的脚本语言虽然在近年进行了内存优化，但其内存消耗依然非常庞大。就在一两年前，Python 的一个基础类型甚至需要消耗 240 字节，是 C 语言等底层语言的 10 倍左右。Berger 开玩笑说，这个数字让人联想到动画人物荷马·辛普森（Homer Simpson）的体重。除了内存占用巨大，这类语言在运行效率上的低下也是臭名昭著的。Berger 引用了麻省理工学院 Charles Leiserson 教授给出的矩阵乘法示例：如果以 Python 原生循环实现的矩阵乘法作为性能基准（记为 1 倍速），那么直接用 C 语言重写相同的逻辑就能获得 47 倍的速度提升；若进一步使用 C 语言能采用的各类算法优化及标准 C 函数库，其运行速度能提升超过 60,000 倍。这种性能鸿沟极其惊人。 ```mermaid flowchart LR accTitle: 矩阵乘法在不同语言及优化手段下的执行性能对比 accDescr: 对比原生Python、未经优化的基础C语言，以及充分优化后的C语言库函数的性能倍数差异。 A["Python 原生循环<br/>基准: 1x"] -->|"纯用 C 重写"| B["基础 C 语言<br/>速度: 47x"] B -->|"应用算法与系统库优化"| C["充分优化的 C / 专用库<br/>速度: > 60,000x"] ``` 在过去，很多开发者并不在乎这种性能差距，因为当时硬件发展正处于一段“仿佛永远不会停下的快车道”。然而，这场“免费午餐”在 2010 年左右彻底终结。Berger 解释称，摩尔定律（Moore's law）本身目前尚未完全死亡，芯片上的晶体管数量仍在持续增加，但登纳德缩放比例定律（Dennard scaling，即时钟频率随工艺缩放而自动提升）却彻底停滞了。由于现代芯片面临严重的过热与漏电（voltage leakage）问题，硬件厂商根本无法再大幅拉升时钟频率。Berger 感叹道：“So, things were great in the '80s. Um you know, if you needed to speed up your program, you just didn't do anything. You could literally just go and have a beach vacation uh and wait, and your code would run faster by virtue of the fact that time had elapsed and computers had gotten faster. But today, you have to work. Today, it's really hard.” 如今硬件性能不再自动白送，荷马·辛普森唯一的“达夫啤酒（Duff）”只能靠功能饮料顶上了。 - **原生 Python 对比优化 C 性能倍差**：优化后的 C 语言实现相比纯 Python 循环矩阵乘法快了 60,000 倍以上。 - **历史时间节点**：约在 2010 年，单核主频自动提升的趋势宣告终止。 - **硬件物理瓶颈**：芯片过热和电压泄漏严重限制了时钟频率的继续增长。 --- ### [12:12](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=732s) 脆弱的现代性能评估：“毫无过错却得出错误数据” 当程序员（例如剧中的荷马）想要手动提升软件性能时，传统流程往往会遭遇意料之外的假象。假设荷马重写了部分代码，将原始版本 $A$ 优化为版本 $A'$；基准测试显示 $A$ 耗时 90 秒，而 $A'$ 耗时 87.5 秒，荷马便欣喜地宣布自己取得了 2.8% 的性能提升。然而，Berger 指出这很可能只是一种假象。造成性能变化的真正原因，往往是代码微小的改动与底层硬件错综复杂的微架构交互所产生的纯粹巧合。 Berger 引用了其同事在 2009 年发表的知名论文《Producing Wrong Data Without Doing Anything Obviously Wrong》（在没有做任何明显错事的情况下产生错误的数据）。该论文证明，极其微小的环境调整即可引发剧烈的性能波动： - **链接顺序（Link Order）**：在 Makefile 里仅仅改变几个 `.o` 目标文件的放置顺序，就会直接改变各个函数在可执行文件内存空间中的分布地址。 - **环境变量（Environment Variables）**：在加载程序时，环境变量会被先载入内存，程序栈在其后开始分配。因此环境变量长度或顺序的变化会直接偏移程序栈的基地址，导致数据在内存中的绝对位置发生整体平移。 这种内存布局的微妙变化所导致的性能差异，甚至超过了“从完全无优化（-O0）切换到全优化（-O3）”的效果，引发高达 $\pm 40\%$ 的性能剧烈摆动。 ```mermaid flowchart TD accTitle: 内存布局变化影响缓存命中的偶发机制 accDescr: 展示未优化版本与重构版本如何因内存地址错位导致缓存冲突的偶然消除。 subgraph SG1 ["原程序 A"] A1["热点代码行 1"] --> C1["Cache Set X (发生冲突，性能低)"] A2["热点代码行 2"] --> C1 end subgraph SG2 ["重构版本 A' (纯巧合)"] B1["代码地址改变"] --> C2["Cache Set X (仅映射单一热点)"] B2["数据/冷代码"] --> C3["Cache Set Y (无冲突，速度变快)"] end ``` Berger 举例说明这种缓存冲突机制：在原始程序 $A$ 中，两行频繁执行的热点代码可能碰巧被硬件散列算法映射到芯片缓存的同一个“缓存组（Cache Set）”中，导致组相联容量过载并触发冲突缺失（Conflict Miss），使得 CPU 不得不频繁从缓慢的系统主存中获取数据；当荷马进行微小修改后，热点代码被偶然挤到了不同的缓存组，冲突凭空消失，程序从而意外加速。更糟糕的是，现代 CPU 为了榨干最后一点性能，引入了繁杂的微架构特性，它们都与虚拟或物理内存地址强绑定： - **分支预测器（Branch Predictor）**：预测 `if/else` 跳转方向，其内部查询表由程序计数器（PC）地址直接索引。 - **转译后备缓冲区（TLB）**：用于将虚拟地址映射为物理地址，内存地址偏移可能导致热点数据跨越页面边界，使得原本能装入 TLB 的数据无法装入，造成严重的性能衰退（或反向的偶然提升）。 - **分支目标预测器（Branch Target Predictor）与硬件预取器（Prefetcher）**：同样极易受到代码布局变动的干扰。 Berger 强调编译器层面的优化同样已达瓶颈：“So, performance is really brittle. If you change something you can get dramatic swings in in performance and the performance swings that you get are really vastly larger than the compiler can even produce.” 面对如此脆弱的性能特征和硬件复杂性，现代分析工具却停滞不前，未能为开发者提供足够的指引。 --- ### [17:33](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1053s) 荷马的“猫咪搜索神器” Augle 与多线程架构模型 为了说明传统性能分析工具在何处陷入瘫痪，Berger 虚构了一个发生在 2020 年的商业应用案例：荷马看准了“猫咪与互联网”的热度，决定打造一款名为 **Augle** 的移动应用，号称将彻底颠覆现有的图像搜索行业。Augle 的核心业务流程包括：用户用智能手机拍下自家猫咪的照片并发送至后端服务器，服务器返回互联网上所有与该猫咪高度匹配的“表情包（memes）”。 为了支持该业务，荷马设计的原型服务采用了并发多线程架构： 1. **工作流程一（存储分支）**：创建专门的线程执行图像压缩，将其缩小为缩略图，并将其保存到数据库中以备后续检索（Berger 幽默地在架构图上画了一张服务器专用的 3.5 英寸软盘作为存储媒介）。 2. **工作流程二（特征检索分支）**：在并行推进的另一侧，工作线程对上传的猫咪图像执行特征提取算法，随后在庞大的表情包库中进行相似度搜索，并将搜索结果封装返回给用户客户端。 3. **汇合同步**：两条分支执行完毕后进行同步汇总，最终结束整个处理周期。 Berger 强调，这种由多任务并行与管道阶段构成的软件架构极大地简化了现实中的并发问题；在实际应用中，多线程系统充斥着用于规避数据竞态的锁（locks）机制与复杂的同步原语，而这也正是传统性能分析工具极易发生误判的根源所在。 ### [19:19](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1159s) 传统性能分析工具的困境与“加载动画”谬误 真实世界的应用程序远比玩具模型复杂，为了防止多个操作并发执行造成互相破坏，代码往往充满交织的逻辑。Emery Berger 设定了一个情境：假设主角 Homer 编写并发布了一个名为 Oggle 的应用，用户虽喜爱该产品，但 Oggle 的运行速度极其缓慢。Homer 感到十分沮丧，此时任何程序员的标准做法都是借助软件性能分析工具（Profiler）来排查瓶颈。 主流性能分析工具的核心设计思路基本定型于 20 世纪 80 年代与 90 年代。简而言之，这类工具会统计每个函数的调用次数以及运行耗时。根据传统优化直觉，开发者应当找出执行频率最高、运行时间最长的代码段并施加优化。然而在 Oggle 这类复杂并发系统中，盲从这套逻辑会带来荒谬的结果：Homer 抓取分析数据后全力优化了最耗时的代码，结果发现自己优化的居然是前端的“正在加载（loading blinky）”等待动画。加载动画在整个处理流程中确实从头运行到尾、耗时最长且频繁触发，但无论把它优化得多快，应用的实际吞吐量和响应速度都不会有任何提升。 ```mermaid flowchart LR accTitle: 传统性能分析工具的误导逻辑 accDescr: 展示传统 Profiler 将最耗时但不影响核心处理的加载动画误判为优化目标的逻辑缺陷。 subgraph Trap["传统 Profiler 指标判定"] A["收集函数执行统计数据"] --> B{"寻找最耗时 / 频繁执行的代码"} B -->|"误选"| C["加载动画 / 轮询等待线程"] end subgraph Consequence["优化后果"] C --> D["开发者投入大量时间优化"] D --> E["真实端到端性能提升: 0%"] end ``` Emery Berger 指出，传统分析工具之所以在现代软件面前失灵，是因为它们忽视了现代应用的体系特性：“在现代应用中，我们关心的是延迟等指标；许多程序在概念上是永不终止的，并且高度并行、异步与并发，而现有的分析工具根本没有跟上时代，它们在本质上仍停留在 80 年代（In modern applications, we care about things like latency. We have programs that run forever conceptually. Uh, they're parallel, they're asynchronous, they're concurrent and the existing profilers just have not kept up. They're essentially still stuck back in the '80s.）。”面对结构错综复杂的异步并发程序，开发者亟需一种能够穿透表面执行耗时、直击性能因果关系的新工具。 --- ### [22:11](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1331s) 因果分析图谱与“虚拟加速”的核心构想 如果能有一把“神奇的放大镜”或某种“魔法仙尘”，撒在 Oggle 的源码上便能精确指出 Homer 应该优化的核心模块（例如搜索组件），并给出确定性的加速收益，性能优化将事半功倍。因果性能分析（Causal Profiling）正是将这种构想具象化的解决方案：它不仅告诉开发者优化哪段代码会带来改变，还能明确预测出能带来多大程度的改变。 因果分析器输出的核心产物是“因果分析图（Causal Profile）”。在这张二维坐标图上，X 轴代表某个特定组件单独被优化的百分比（加速比），Y 轴则代表整个程序的性能加速比、吞吐量提升幅度或延迟降低幅度。图表呈现出三类极具指导价值的典型曲线： 1. **收益平截曲线**：对搜索（Search）模块进行优化，在达到特定阈值前整体性能收益随优化力度急剧攀升，但随后进入平台期（Plateau），这清晰地指引开发者何时优化、何时该适可而止； 2. **零收益水平线**：数据压缩（Compression）模块无论被加速多少倍，整体性能都毫无变化，意味着它根本不在关键路径上，优化它纯属浪费时间； 3. **稳步增长曲线**：特征提取或索引（Indexing）模块虽然斜率不如搜索模块陡峭，但优化收益会随着投入稳定持续地向上积累。 ```mermaid flowchart TD accTitle: 因果分析实验机制对比 accDescr: 对比需要魔法的直接加速实验与基于相对论的虚拟加速实验。 subgraph ImpossibleMethod["理想化方法（依赖魔法）"] M1["直接物理加速目标组件 A"] --> M2["观测整体应用加速收益"] M2 --> M3["绘制出单点性能曲线"] end subgraph CausalMethod["Coz 实现方案（虚拟加速）"] V1["保持目标组件 A 运行速率不变"] --> V2["人为注入微延迟，放慢并发运行的其他组件"] V2 --> V3["观测整体应用放缓程度"] V3 --> V4["依据相对性将放缓差值等价映射为 A 的加速收益"] end ``` 要获得这样的图表，理论上必须进行“性能实验”。如果拥有魔法，我们可以将各模块逐一加速不同幅度并观测整体反应，但这在现实中是不可能的。因果分析依靠的核心洞察是“虚拟加速（Virtual Speedup）”。Emery Berger 将其类比为相对论思想：要想在并发环境中观测加速某个组件的效果，无需真正让该组件跑得更快，只需按比例放慢所有与其并发运行的其他模块即可。在操作系统中人为放慢代码轻而易举，放慢其他并发线程所导致的相对性能差异，在数学与因果关系上完全等价于目标模块被真实加速后的表现。通过循环往复地对各个模块执行微小的放慢实验，分析器即可收集到所有散点，从而在不借助任何魔法的前提下绘制出精准的因果分析图。 --- ### [26:01](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1561s) 进度点、利特尔定律与 Coz 的工程落地 对于如 Oggle 这类持续提供服务的网络后端而言，“程序运行完毕”根本不是目标，传统的总体执行耗时指标毫无意义。开发者真正关心的指标只有两项：从发出请求到接收响应之间的耗时——延迟（Latency），以及单位时间内能够处理的请求总量——吞吐量（Throughput）。 为了让因果分析器理解应用程序的业务进度，开发者不需要重写程序，只需在源码中嵌入轻量级的批注，即“进度点（Progress Point）”： - **吞吐量测量**：开发者只需在关键业务闭环处（如成功返回搜索响应）插入一个宏调用。代码每次经过该点便会触发内部计数器累加，分析器在执行虚拟加速实验时便能直接观测该宏的触发速率变化，生成以吞吐量为 Y 轴的因果预测图。 - **延迟测量**：若要测量从发出“猫咪图片”搜索请求到收到结果的端到端响应时间，开发者需在事务起始与结束位置分别标记 `begin` 和 `end` 两类延迟进度点。 在测量延迟时，系统引入了排队论（Queuing Theory）中的经典基石——**利特尔定律（Little's Law）**。利用该定律的重排形式，因果分析器无需针对每个并发请求挂载沉重的端到端计时追踪，只需追踪系统内并发事务的总数量以及基于现有进度点测得的吞吐率，即可严密推导并换算出端到端延迟变化。 该理论最终在名为 **Coz** 的因果分析器中得以完整实现。Coz 专为 Linux 系统打造，现已直接集成到 Debian 和 Ubuntu 的官方软件源中。具备超级用户权限的开发者只需运行简单的包管理命令即可将其安装到系统并即刻投入使用： ```bash sudo apt install coz-profiler ``` - **利特尔定律的核心推导关系**：在平稳系统中，平均在途请求数 $L$ 等于到达率（吞吐量）$\lambda$ 乘以平均响应时间（延迟）$W$；通过追踪并发事务数与吞吐量，系统能够低开销推导出虚拟加速实验对延迟指标的真实影响。 - **极简侵入性**：进度点在底层仅表现为轻量级宏，几乎不破坏业务逻辑原有结构，也不引入显著的运行时性能损耗。 ### [28:57](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1737s) Coz 真实案例分析：优化 Ferret 的多线程流水线 在实际操作中，用户只需运行 `coz run -- <程序名> <参数>` 即可启动因果分析器（Causal Profiler）。Coz 会在程序运行过程中随机注入“虚拟加速”性能实验，观察其对吞吐量或延迟等进度点（Progress Points）的因果影响。为了展示 Coz 在真实复杂系统中的威力和实战表现，Emery Berger 引入了一个假想的以图搜图应用示例“Augle”。尽管该系统并未实际完整构建，但研究团队找齐了其核心底层组件来剖析：用于图像去重与压缩的 Dedupe、用于内容比对与相似检索的 Ferret，以及作为后端存储的 SQLite。 ```mermaid flowchart LR accTitle: Ferret 图像检索流水线 accDescr: 展示 Ferret 从输入经特征提取、分词、索引到排序并最终输出的阶段流水线 subgraph Pipeline ["Ferret 处理流水线"] direction LR IN["输入 (Input)"] --> FE["特征提取"] FE --> SEG["图像分割 (Segmentation)"] SEG --> IDX["索引 (Indexing)"] IDX --> RNK["排序 (Ranking)"] RNK --> OUT["输出 (Output)"] end ``` 在测试 Ferret 时，团队旨在提高图像处理的吞吐量。Ferret 本身是一个典型的管道式（Pipeline）并行架构，各阶段之间通过多线程传递数据。原有实现非常粗暴，给流水线中的每一个阶段都分配了相同数量的线程。当 Coz 生成分析图表后，几处性价比极高（Bang for the buck）的优化点立刻浮现出来：排序（Ranking）阶段收益最大，其次是索引（Indexing），随后是分割（Segmentation），而特征提取等阶段无论怎么加速都对整体吞吐量毫无助益。团队做出了据 Emery 所称“可能是我们做过最简单的一次优化”：单纯根据 Coz 预测的加速收益，重新按比例调整各阶段的线程分配权重。 - **极低代价带来显著加速**：不改写任何核心算法，仅仅重新编排流水线各个阶段的线程配比，就获得了 21% 的吞吐量提升。这种由于流水线瓶颈分配不均引发的问题，在传统耗时分析器（Profiler）中根本无法诊断。 - **高精度的因果预测验证**：团队将排序组件的线程数从 16 个增加到 22 个，相当于投入计算资源增加了 27%，理应带来排序组件自身 27% 的吞吐量增长；而 Coz 事先根据曲线明确预测“排序组件若提升 27% 吞吐量，整体应用将提速 21%”，实际测量结果恰恰也是 21%，“完全符合预期”。 - Emery Berger 赞许道：“So, that's great. Cause is doing what it's supposed to do, and it's guiding us to optimizations that were would basically have been impossible to find otherwise.” --- ### [31:51](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1911s) Dedupe 哈希函数灾难与其它系统实战 紧接着，Emery Berger 剖析了另一个有趣的案例：用于数据去重的工具 Dedupe。Dedupe 的逻辑概念很清晰，它通过切割图像内容，找出两幅相似图片中完全一致的数据块并让它们共享内存指针（例如将 Grumpy Cat 图片切碎并提取重合部分），从而省去重复存储的空间。在 Dedupe 的核心机制中，所有图像块的指针都保存在一个分桶哈希表（Hash Table）中。当系统运行遭遇性能瓶颈时，团队排除了锁竞争（Lock Contention）假说（代码中其实细粒度地使用了极多把锁），原本也怀疑是桶太少导致冲突，但即便调高哈希桶总数，性能依然毫无起色。 ```mermaid flowchart TD accTitle: Dedupe 性能瓶颈排查与修复流程 accDescr: 展示从排查锁与桶大小到最终通过将加号改为异或修复哈希冲突的决策路径 A["Dedupe 性能受阻"] --> B{"排查锁竞争?"} B -->|"排除"| C{"增加哈希桶数量?"} C -->|"无性能提升"| D["检查 Hash 函数代码"] D --> E["发现合并数值使用了 '+' 符号"] E -->|"导致正态分布，桶利用率仅 2%"| F["修改为 '^' (XOR 异或)"] F --> G["桶利用率升至 82%，带来 9% 整体提速"] ``` 团队根据 Coz 指向的具体链表遍历循环进一步深挖，最终震惊地发现底层的哈希函数设计堪称灾难。开发者在计算哈希值时，使用了加号（`+`）去累加图像各部分的数据，由于中心极限定理的作用，哈希值呈现出严重的正态分布（钟形曲线），而非均匀分布（Uniform Distribution）。在测试的 1,024 个桶中，数据疯狂挤占在极少数的桶里，有效利用率仅有 2%，导致每个非空桶平均挤了 76 个块。团队仅仅做了一处极其轻微的源码修改：把加号 `+` 改成了按位异或符号 `^`（XOR）。 - **单字符修改创造奇迹**：哈希函数的利用率直接从 2% 飙升至 82%，单个桶内的平均块数从 76 个骤降至 2 个，链表遍历操作获得了 96% 的提速，进而精确兑现了 Coz 预测的 9% 整体性能提升。 - **SQLite 优化**：在另一个广泛使用的核心数据库 SQLite 上，Coz 同样引导团队完成了传统分析器绝不可能发现的改动，斩获了高达 25% 的性能提升。 - **删除一行代码的极限收益**：在针对流体模拟与聚类程序（如 `fluidanimate` / `streamcluster`）的调优中，原作者自己实现了一套自定义线程屏障（Barrier），效率极低。团队直接删掉自定义屏障、换回系统原生的标准同步原语，“实质上通过删除一行代码获得了 38% 乃至接近 70% 的性能提升”。 --- ### [36:14](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2174s) 面向胶水语言的全新挑战：进入 Scalene 与 Python 调优 Coz 在 C、C++、Rust 等“裸金属级”（Metal）语言以及 Java 领域展现了巨大的威力，但编程语言生态中仍然存在一大块以“臃肿且缓慢”（Fat and slow）著称的广阔领地——以 Python 为代表的高级脚本语言。然而，Python 拥有一项特殊的系统定位：它极度依赖并深嵌于 C 语言生态中。在很多高性能计算与机器学习场景下，Python 本质上是作为一门“胶水语言”（Glue Language）运行的，负责在上层编排和串联起由 C/C++ 等贴底语言编写的高效模块。只要 Python 脚本能最大程度地把计算工作甩给底层 C 扩展库去执行，整体执行速度就能得到大幅保障。 为了解决 Python 开发者难以理清代码耗时归属的痛点，Emery Berger 团队研发了专为脚本语言打造的新一代分析器——**Scalene**。Scalene 的核心使命，就是清晰地告诉程序员哪些代码行被困在缓慢的纯 Python 解释器中，哪些代码行正在高效调用底层 C 代码，从而精确指导他们将哪些部分推入底层“裸金属世界”。 - **极简的使用方式**：用户无需学习复杂的命令行体系，只需将运行命令从 `python script.py` 替换为 `scalene script.py`。 - **丰富的分析输出控制**：支持通过参数指定单项指标（例如通过特定选项仅分析 CPU 耗时，或生成交互式的 HTML 可视化报告），并通过 `--help` 查看完整控制项。 - Emery Berger 在讲座中展示了一个简明而精炼的真实小代码案例，借此演示如何借助 Scalene 在寥寥数秒内定位问题根源：“Scalene is designed to help Python programmers move their code from bloated slow Python into fast and efficient metal languages like C, with basically just telling them which parts of their code need to be moved into the metal land, if you will.” ### [39:05](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2345s) 解读 Scalene 的分析报告：CPU 耗时细分、内存走势与复制开销 Emery Berger 演示了当针对示例代码运行 Scalene 时呈现的丰富分析结果。传统的性能分析工具（Profiler）只会笼统地指出“程序在何处消耗了 CPU 时间”，而 Scalene 与众不同，它能将 CPU 耗时清晰地拆解为**纯 Python 执行时间**与**底层 Native 原生代码（即硬件层面执行的 C/C++）时间**。此外，它还会标出系统态（System）时间（例如 I/O 等待）。Berger 强调区分这些维度的关键价值：“你必须拥有这些信息，才能弄清楚哪些东西是你能优化的，哪些是不能优化的；如果性能瓶颈根本不在 CPU 上，那你无论如何也无法通过优化 CPU 逻辑来提升性能。” 紧接着，Scalene 展示了内存消耗情况：不仅统计消耗了多少内存，还细分出这些内存是由 Python 自身分配的，还是源自底层 Native 代码。在示例的第 6 行代码中，Scalene 报告指出这一行分配了多达 763 MB 内存，且其中 Python 自身的分配比例为 0%，说明分配全部发生在底层 Native 原生代码中。界面的另一大亮点是借鉴了 Edward Tufte 设计理念的火花线（Sparklines）微线图，横轴表示时间，纵轴表示内存消耗量。屏幕上方展示了整个程序生命周期内的全局内存消耗走势，而底部针对第 6 行则显示了专属图表，这一行代码独占了程序 87% 的内存活动。 此外，Scalene 引入了一项全新的关键指标——**复制开销（Copy Volume）**。Berger 解释称，当数据在 Python 运行时与底层 C/Native 代码之间来回传递时就会发生复制；如果频繁穿越该边界，复制本身就会沦为严重的性能瓶颈。理想的设计应当是一次性将数据推入 Native 端，让计算全程在底层跑完后再返回结果，而非频繁反复跳转。 - **第 6 行代码的性能问题汇总**： - 占用了总运行时间中 34% 的 Native 代码执行耗时。 - 单行分配了 763 MB 内存，占据全程序 87% 的内存活动。 - 火花线图呈现出典型的“锯齿波（Sawtooth）”形态，Berger 指出这正是大量发生数据复制与立即释放的铁证。 - 根本原因：程序员错误地写了 `np.array(...)` 试图将输入转为紧凑的 NumPy 原生数组，但其调用的 NumPy 均匀随机数生成器生成的本就已经是 NumPy 数组；该多余调用导致系统先分配新数组、将原数组全量拷贝过去，然后再销毁旧数组，从而产生了锯齿峰值。 ```mermaid flowchart TD accTitle: Python and Native memory copying bottleneck accDescr: Diagram illustrating the redundant copy overhead when calling np.array on an existing NumPy array. subgraph NativeCode["NumPy 底层 (Native C/C++)"] A["numpy.random 生成的原生数组"] B["冗余分配新内存缓冲区 (763 MB)"] C["深拷贝原数据并释放旧内存"] end subgraph PythonBoundary["Python 解释层"] D["第 6 行: 误调 np.array()"] end A --> D D --> B B --> C ``` --- ### [42:31](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2551s) 极简修复与性能对比：低开销与高精度的兼得 针对上述问题，修复方式极为简单：直接删掉无用的 `np.array` 包裹调用即可。由于原有输入本身即为 NumPy 数组，删除该调用在语义上完全等价。修改后效果立竿见影：CPU 占用比例大幅下降；虽然第 6 行内存占比仍显示为 85%，但其基数（分母）已经大为缩减；内存曲线中的锯齿状暴涨彻底消失；总内存消耗从 1 GB 锐减至 860 MB，程序整体运行时间也从 6.3 秒缩短到 5.4 秒。 Berger 随后谈到了分析工具的痛点：如果一个工具极度精准但极其缓慢，根本没人愿意使用。“你的初衷就是因为程序跑得太慢，如果拿一个 Profiler 反而让它慢得像爬行，那过程会极其痛苦。”通过对比图表，Scalene 展示了颠覆性的性能：无论是在仅分析 CPU 的模式还是在包含内存与复制量全开的完整模式下，Scalene 的开销都微乎其微——完美基准为 1.0，而 Scalene 仅在 1.1 至 1.3 之间，即开销仅为 10% 到 30%。 反观业内已有工具，Python 官方自带的标准分析工具 `profile` 会使程序变慢 16 倍，其他工具也普遍导致 10 倍乃至 35 倍的减速。更为致命的是，这些传统工具充其量只能指出某行代码耗时的百分比，既无法区分纯 Python 与 Native C 代码，也完全没有内存分析功能。而当时业内唯一已知的另一款 Python 内存分析器，会导致程序遭遇高达 152 倍的极端减速：原本跑 5 秒的代码在加上该分析器后足足运行了 12 分 38 秒，某些情况下甚至能观察到 1,000 倍的恐怖减速。相比之下，Scalene 在测量精度和低运行开销之间取得了压倒性优势。 | 分析工具类型 | 典型工具 / 模式 | 运行开销 / 减速倍数 | 核心局限 / 特点 | |:--- |:--- |:--- |:--- | | **标准 CPU 分析器** | Python 内置 `profile` | 约 16x 减速 | 仅给出代码行耗时百分比，无法拆分 Python/Native | | **其他流行分析器** | 常见三方 CPU profiler | 10x ~ 35x 减速 | 无 Native 细分，缺失内存与复制监控 | | **传统内存分析器** | 既有 Python 内存分析器 | 152x ~ 1000x 减速 | 速度极其缓慢（5 秒代码跑 12 分 38 秒） | | **Scalene (CPU 模式)** | Scalene CPU only | 约 1.1x（+10% 开销） | 极低开销，毫秒级高精度统计 | | **Scalene (完整模式)** | CPU + 内存 + 复制卷 | 约 1.3x（+30% 开销） | 全功能覆盖，支持行级 Native/Python 隔离与内存趋势 | --- ### [45:03](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=2703s) 逆向思维破局：利用“信号延迟投递”精确拆分 Python 与 C Berger 揭示了 Scalene 背后的核心实现机理。他风趣地指出，Scalene 的核心设计哲学在于“去寻找这些‘非理性繁荣语言’（Irrational Exuberant Languages）中各种奇怪的语言特性与限制，并借力打力，化限制为武器，从而压榨出看似不可能获取到的信息”。 在具体实现上，Berger 深入解析了“延迟信号投递（Deferred Signal Delivery）”机制。作为一款基于采样的分析器（Sampling-based Profiler），Scalene 依靠系统定时器定期中断 Python 程序来探查当前的执行上下文。如果代码完全在 Python 解释器内执行，设定每 0.1 秒发送一次信号，信号就能平稳准时地被处理。然而，Python 解释器在调用诸如 NumPy 等底层的 C/C++ 扩展库时，不会在 C 代码执行中途投递定时器信号，而是会将所有信号挂起，**一直延迟到执行权交还回 Python 解释器的那一刻才统一投递**。 对于普通的采样分析器而言，这一机制是一场灾难：Native 代码执行期间完全没有信号命中，分析器以为“什么都没发生”，导致大量的 C 代码执行时间彻底隐形，甚至误导开发者以为瓶颈在纯 Python 代码上。然而 Berger 团队选择“化柠檬为柠檬水”，将这一缺陷反转为突破口： 1. **真实物理时钟追踪**：每次信号实际被处理时，Scalene 都会记录当前操作系统的挂钟时间（Wall-clock Time）。 2. **时差比对与归因**：当下一次信号到来时，比对实际流逝的时间与预期的采样周期。如果间隔远大于预设的 0.1 秒，就证明中间出现了由于执行 Native C 代码而导致的投递延迟。 3. **双计时器记账**：Scalene 维护独立的 Python 计时器和 Native C 计时器；Python 计时器只累加理论的采样时间片（如 0.1 秒），而超出理论间隔的多余延迟耗时，则全部自动计入该代码行所对应的底层 C 计时器中。 4. **虚拟时间与系统态解析**：配合 CPU 虚拟时间（Virtual Time），Scalene 还能推导出进程在 CPU 实际执行与系统态（I/O 等待或挂起休眠）之间的真实耗时。 Berger 展示了一组验证实验：编写一段由 C 底层计算和纯 Python 解释器各占特定比例（理论值为 C 占 70%，Python 占 30%）的基准测试程序。通过上述延迟信号还原算法，Scalene 测量给出的结果几乎严丝合缝地吻合 70/30 的比例，测试误差仅在 1% 左右，充分证明了该测量方法的高度准确性。最后，Berger 提到团队期望未来将 Scalene 的这套机制推广扩展至更多其他具有类似机制的动态脚本语言之中。 ### [049:27] 总结 Scalene 与 Coz：从脚本语言到底层系统的性能调优 Emery Berger 指出，各类脚本语言之间存在着大量的共性特征。通过利用这些通用底层机制，Scalene 的核心分析方法不仅适用于 Python，还能毫无困难地移植到其他类似语言中。目前，团队正在推进将该技术应用于 PHP，同时美国东北大学（Northeastern University）的合作者也正在着手将其引入 R 语言生态。 总结而言，Scalene 是一款专为“非贴近硬件底层（non-metal）语言”量身打造的性能分析器。它不仅帮助开发者优化 Python 代码本身，更关键的是引导开发者识别哪些 Python 代码片段应当被下沉重写为原生代码（Native code）。它通过将运行时间与内存消耗严格拆分为 Python 部分与原生代码部分，极其精确地标明内存消耗的具体位置，尤其是发生跨边界拷贝（Copying）的节点——这往往意味着在脚本层与底层库之间存在高昂的数据搬运开销。Scalene 运行高效、安装便捷，Python 用户只需运行一条简单的 pip 命令即可安装。Emery Berger 幽默地澄清，自己绝非贬低脚本语言：“Don't get me wrong, I like Python. It's a fine language as long as it's not choking you.（别误会，我喜欢 Python。只要它不把你卡死，它就是门极好的语言。）”最后，对于从事 C/C++、Rust 等底层语言（Metal languages）现代应用开发的程序员，他推荐使用已在 GitHub 开源并提供包管理安装的因果分析器 Coz。 ```mermaid flowchart TD accTitle: Scalene and Coz Language Profiling Strategy accDescr: Diagram illustrating how Scalene targets scripting languages and separates native code, while Coz targets compiled metal languages. subgraph Scripting["高级脚本语言生态 (Scalene)"] direction TB Py["Python / 当前核心"] Ext["扩展计划: PHP, R 语言"] Metrics["核心指标: 原生时间拆分 / 内存消耗 / 跨边界拷贝"] Py --> Metrics Ext --> Metrics end subgraph Metal["贴近硬件底层语言 (Coz)"] direction TB Compiled["C / C++ / Rust"] Causal["因果分析 (Virtual Speedup)"] Compiled --> Causal end subgraph Goal["优化落地目标"] Opt["精准定位真正瓶颈与重构点"] end Metrics --> Opt Causal --> Opt ``` - **Scalene 安装指令**：`pip install -U scalene` - **语言扩展方向**：UMass 团队正在研发 PHP 支持；东北大学团队正在推进 R 语言支持。 - **工具选型分水岭**：Python 等脚本语言使用 Scalene；C/C++/Rust 等编译型底层语言使用 Coz。 --- ### [051:13] 问答开始：程序员生产力与执行性能的平衡 在讲座问答环节，主持人首先回顾了演讲开篇提到的核心概念——底层贴近硬件的语言（Metal languages）与“非理性繁荣语言”（Irrational exuberance languages）之间的对立。主持人提出：人们之所以涌向 Python、Ruby 这类语言，是因为其开发效率远超底层语言，那么开发者究竟该如何在 Python 带来的高开发效率与 C/C++ 才能提供的极致性能之间取得平衡？ Emery Berger 解释道，所谓“非理性繁荣”，其核心谬误在于过去人们总以为性能不再重要，“I used to hear that articulated all the time. Um, computers are just getting faster and faster. It's not a problem. Don't spend all your time on this. We can focus exclusively on productivity.（我过去总听到这种说法：计算机变得越来越快，性能不再是个问题，别把时间浪费在这上面，我们可以全神贯注于生产力。）”然而在现实中，这种语言可能会极其缓慢。由于现代 Python 生态中拥有大量用原生 C/C++ 编写的底层库，生产力与性能之间并不存在不可调和的矛盾。开发者完全可以用 Python 快速编写程序，并通过 Scalene 定位出狭窄的性能瓶颈。如果该瓶颈恰好可以通过现有的 C 库解决，问题便迎刃而解。Emery Berger 进一步展望了未来的研究设想：开发一款能够自动分析代码特征并给出“你应该使用某某开源底层库”建议的分析工具，甚至在理想情况下能够自动将该库应用到项目中，尽管他坦言全自动替换可能“有点步子迈得太大（a bridge too far）”。 --- ### [053:26] 性能测量的不确定性、噪音与置信区间过滤 主持人随后汇总了多个观众关心的核心技术问题：性能剖析时的“测量不确定性”（Uncertainty in measurement）。一方面，分析工具本身是否会产生探针效应（Perturbation），从而扭曲原本的执行表现？另一方面，目标程序自身往往具备大量非确定性（Non-determinism）与环境噪音，分析器应如何在这种波动中给出可靠的诊断？ Emery Berger 明确指出，Coz 与 Scalene 都基于**采样技术（Sampling）**构建。采样意味着分析器仅在周期性唤醒时介入，极大地降低了对被测系统本身的侵入与干扰。针对 Coz，团队在算法设计上极其苛刻，确保其所有的虚拟加速实验完全无偏（Unbiased），不会系统性地向某一方向偏倚结果；而对于脚本层的 Scalene，采样同样能够让偶然的测量扰动在大样本下相互抵消。关于程序本身的随机性与方差，Berger 强调如果某项指标只占运行时间的 3%，但每次运行都有 $\pm 3\%$ 的波动，那该指标在统计学上就处于置信区间之外，绝不应被视作行动依据。Berger 透露，Scalene 内部其实已经建立了置信区间评估机制，只是尚未找到最佳的可视化形式，因此目前的策略是直接在报告中隐藏所有置信度过低的数据：“And I think that's probably a reasonable way forward on these things is just to have profilers that decline to report things as being actionable if they're likely to be noise.（我认为这类工具一个合理的发展方向，就是当某些数据很可能是噪音时，性能分析器直接拒绝将其作为可操作的优化建议上报给用户。）” - **低开销机制**：依靠周期性采样避免插桩带来的性能扭曲。 - **无偏设计**：Coz 确保测量实验在统计学上不产生系统性偏差。 - **内部置信区间过滤**：Scalene 内部计算波动率，主动剔除落在置信度阈值之外的潜在噪音数据。 --- ### [055:46] 规避微架构“硬件噪声”：并发随机性与 Stabilizer 主持人提出了一个深入微架构层面的尖锐问题：演讲前期曾引用 Todd Mytkowicz 等人的研究，指出非常微小的环境差异（如环境变量长度、链接顺序引起的内存布局变动）造成的性能差异，甚至足以掩盖编译器 `-O3` 优化的效果。那么当开发者使用 Coz 调整线程数，或者根据 Scalene 调整代码时，如何确定自己所获得的提速是真实的逻辑优化，而不是偶然撞上了某种硬件微架构的诡异巧合？ Emery Berger 解释，Coz 在机制设计层面上天然免疫这类内存布局效应。在多线程并发程序中，Coz 会以完全随机的间隔在随机选中的线程上暂停随机的时长。这种高度随机化的并发执行，打乱了指令在处理器上的流水线执行顺序以及内存访问时序，其分布就如同一团“电子云”（Electron cloud），随着时间推移自然抵消并抹平了固定内存布局或硬件对齐造成的局部偏差。Berger 还借此补充了团队之前关于该主题的一项成果——**Stabilizer** 系统。当时为了在 `-O2` 与 `-O3` 评估中消除内存布局的影响，他们开发了 Stabilizer，在运行时对单线程顺序程序的函数入口、堆、栈以及全局变量布局进行完全动态的随机重定位。他指出，并发程序的乱序执行在 Coz 中自发起到了类似 Stabilizer 的扰动与平滑作用。随后，主持人引出了下一个关于 Coz 与现有编译器优化参数相互作用的观众提问。 - **Coz 的平滑机制**：通过在多线程间引入随机的实验性虚拟加速与暂停，让底层硬件微架构的偶然对齐效应在时间均值上被冲刷抵消。 - **Stabilizer 工具**：UMass 针对顺序程序研发的运行时随机化工具，对函数、堆、全局变量和栈内存进行持续重排，用以评估脱离布局偏置后的编译器真实效果。 ### [58:52](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=3532s) Coz 与编译器优化、脆弱的内存布局与极端优化构想 观众提问探讨是否可以利用类似 Coz 的因果性能分析工具来辅助指导编译器的优化方向。Emery Berger 明确指出，Coz 的核心定位是找出程序中哪些部分值得优化以提升整体吞吐量或延迟；如果开发者打算使用 Coz，必须在经过生产级优化编译的代码上运行，而绝不能在未优化的代码上测试。他打了一个生动的比方：“这在某种意义上就像你正在猪身上做实验，却非想知道这个实验结果是否适用于树木一样”，开发者必须在真正关心的真实生产目标上采集性能数据。 谈及编译器对内存布局的优化时，Berger 直言这类优化效果参差不齐且极其脆弱。现代计算机体系结构中的内存布局效应高度敏感，甚至仅因微小的运行时环境差异就会失效。比如系统用户名从“Arun”换成“Homer”，仅仅因为多了一个字符，整个栈空间就会发生偏移，从而产生巨大的缓存和性能波动。除非程序每次运行的代码与处理的数据规模完全一致，否则刻意优化的布局往往无法保持稳定。至于编译器与因果分析的结合，Berger 构想了一种极具吸引力但现有编译器出于编译速度考虑而放弃的场景：假如编译器能提供一种“彻底放飞”的模式，让开发者告知系统某些核心代码至关重要，允许编译器在周末花费一整天甚至一周的时间进行极其深度的穷尽优化，这种由性能瓶颈驱动的长耗时优化器将会非常有趣且有价值。 ```mermaid flowchart TD accTitle: Coz 使用原则与编译器极端优化概念 accDescr: 展示 Coz 必须运行在已优化的生产环境代码上，并对比常规编译器与深度优化器设想。 subgraph Coz["Coz 运行原则"] A["生产优化代码 (Production Binary)"] -->|"执行因果性能剖析"| B["精准锁定潜在性能瓶颈"] C["未优化代码 (Debug / Non-optimized)"] -.->|"错误的使用方式 (不可预测/无意义)"| B end subgraph Compiler["编译器优化考量"] D["常规快速编译模式"] -->|"受限于编译速度时间预算"| E["浅层/脆弱的布局优化"] F["周末穷尽优化模式 (假想)"] -->|"愿意耗费数天换取极限性能"| G["深度重构与代码极限加速"] end ``` - Emery Berger 警告不要在调试或未优化的二进制上运行性能实验：“在某种意义上，这就像你在猪身上做实验，却非想看这个实验是否适用于树木一样（in a sense, it's like you're performing an experiment on a pig, and you really want to see if the experiment applies to trees. It's just like don't do that, right? Like you really want to run the experiment on the thing that you actually care about）。” - 内存布局的敏感性甚至能被单个字节打破：用户名仅相差一个字母（如“Homer”与“Arun”），栈偏移的变化就足以彻底颠覆缓存命中率并对性能带来巨大冲击。 - 设想中的极限编译器：许多开发者完全乐意在重要模块上对编译器说：“我知道你平时对我友好和善，但现在你可以彻底放飞，花上一天、两天甚至一周的时间去尽情优化（I know normally you're trying to be nice and friendly to me, but you go crazy and spend, you know, a day, two days, a week running）。” ### [01:01:07](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=3667s) Stabilizer 与内存随机化在安全领域的价值：移动目标防御与 DieHarder 问答环节的最后一个问题围绕 Stabilizer 的运行时随机放置机制是否对系统安全带来影响展开。Emery Berger 证实了两者之间极其紧密的联系。他透露，在撰写 Stabilizer 的论文时，团队曾反复纠结究竟是将其包装为一个纯粹的性能评测系统、一个安全防御系统，还是兼而有之。最终为了行文的清晰度与聚焦，他们选择将 Stabilizer 定位为性能工具发布。然而，Stabilizer 引入了极高强度的运行时再随机化机制，是同类研究中首个在运行时动态对如此多元素进行连续随机化的系统。 这种在运行时打乱代码与数据布局的技术，在安全领域被称为“移动目标防御”（Moving Target Defenses），其核心目标是瓦解利用固定内存结构发起的底层漏洞攻击。Berger 提到了主持人在开场介绍中提及的 DieHarder 系统，该系统就是一个专门针对安全场景设计的随机化内存分配器。DieHarder 通过空间中的随机分配与保护页（Guard Pages）机制，使得攻击者预测内存地址的难度等同于预测高质量伪随机数生成器的输出。一旦内存地址和偏移量彻底无法预测，黑客基于固定偏移或硬编码地址实施可靠利用的成功概率基本降为了零。 - Emery Berger 阐释了随机化如何从根源上摧毁内存攻击：“攻击者能够安全利用基于已知地址或已知偏移的漏洞的可能性，基本上降为了零（And so, the likelihood of an attacker being able to safely exploit something where they have addresses or known offsets basically goes to zero）。” - 移动目标防御（Moving Target Defenses）：通过持续变动运行时的代码位置、堆栈与内存排布，使恶意软件与攻击载荷无法依赖既定的内存拓扑。 - DieHarder 的实现机制：利用严密的空间随机内存分配结合隔离用的 Guard Pages，构筑了高强度的防御屏障，直接影响了 Windows 8 堆内存安全特性的加固演进。 ### [01:02:52](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=3772s) 结语与 ACM 活动信息 主持人 Milan Milenkovic 代表 ACM 对 Emery Berger 的精彩报告以及多项开创性工作表示诚挚感谢，并向所有在线参与讨论、提出高质量问题的参会者致谢。Milenkovic 补充回应了参会者普遍关心的录像问题：本次网络研讨会的完整录播将在几天内上线至 `learning.acm.org` 平台。最后，主持人呼吁观众填写屏幕上弹出的即时问卷以推荐未来的讲座主题或学者，并提醒大家关注 ACM 官方网站（acm.org 及 learning.acm.org）以获取后续技术讲座与学术活动的最新动态。研讨会至此圆满结束。 ## Notable quotes > [12:12](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=732s) So, performance is really brittle. If you change something you can get dramatic swings in in performance and the performance swings that you get are really vastly larger than the compiler can even produce. — Emery Berger > [19:19](https://www.youtube.com/watch?v=7g1Acy5eGbE&t=1159s) In modern applications, we care about things like latency. We have programs that run forever conceptually. Uh, they're parallel, they're asynchronous, they're concurrent and the existing profilers just have not kept up. They're essentially still stuck back in the '80s. — Emery Berger ## Mentioned ACM, Milind Kulkarni, Emery Berger, Purdue University, University of Massachusetts Amherst, Microsoft Research, Universitat Politècnica de Catalunya, Barcelona Supercomputing Center, Hoard, DieHard, DieHarder, Windows 7, Windows 8, Mac OS X, Coz, Scalene, Stabilizer, Python, C++, C, Fortran, Rust, Charles Leiserson, NumPy, Ferret, Dedupe, SQLite, Todd Mytkowicz, Milan Milenkovic ## Video description Title: Performance (Really) Matters Speaker: Emery Berger Date: September 23, 2020 ABSTRACT Performance clearly matters to users. For example, the most common software update on the AppStore is "Bug fixes and performance enhancements." Now that Moore's Law has ended, programmers have to work hard to get high performance for their applications. But why is performance hard to deliver? I will first explain why current approaches to evaluating and optimizing performance don't work. I'll show how complicated performance has become on modern systems, and how compiler optimizations have essentially run out of steam. Next, I'll introduce two radically new performance profilers that guide programmers directly to the code they need to change to improve application performance. The first is Coz, a new "causal profiler" for C/C++/Rust (with a Java version as well) that lets programmers optimize for throughput or latency, and which pinpoints and accurately predicts the impact of optimizations via what we call "virtual speedup" experiments. Coz's approach unlocks previously unknown optimization opportunities. Guided by Coz, we improved the performance of applications by as much as 68%; in most cases, this involved modifying less than 10 lines of code and took under half an hour (without any prior understanding of the programs!). Coz now ships as part of standard Linux distros (apt install coz-profiler). The second is Scalene, a "scripting-language aware" profiler for Python (with other scripting languages to come). Scalene runs orders of magnitude faster than other profilers while delivering far more detailed information -- information that's especially valuable to Python programmers. Via a combination of sampling, inference, and disassembly of byte-codes, Scalene efficiently and precisely attributes execution time and memory usage to Python, which developers can optimize, or library code, which they cannot. Its novel sampling memory allocator efficiently reports line-level memory consumption and trends with low overhead, helping developers reduce footprints and identify leaks. Finally, Scalene reports a new metric, copy volume, that helps developers root out insidious copying costs across the Python/library boundary, which can drastically degrade performance. Scalene is available on PyPi (pip install scalene). SPEAKER Emery Berger Professor, University of Massachusetts Amherst, ACM Fellow Emery Berger is a Professor in the College of Information and Computer Sciences at the University of Massachusetts Amherst, the flagship campus of the UMass system. He graduated with a PhD in Computer Science from the University of Texas at Austin in 2002. Professor Berger has been a Visiting Scientist at Microsoft Research and at the Universitat Politècnica de Catalunya (UPC)/Barcelona Supercomputing Center (BSC). Professor Berger’s research spans programming languages, runtime systems, and operating systems, with a particular focus on systems that transparently improve reliability, security, and performance. He and his collaborators have created a number of influential software systems including Hoard, a fast and scalable memory manager that accelerates multithreaded applications (used by companies including British Telecom, Cisco, Crédit Suisse, Reuters, Royal Bank of Canada, SAP, and Tata, and on which the Mac OS X memory manager is based); DieHard, an error-avoiding memory manager that directly influenced the design of the Windows 7 Fault-Tolerant Heap; and DieHarder, a secure memory manager that was an inspiration for hardening changes made to the Windows 8 heap. His honors include a Microsoft Research Fellowship, an NSF CAREER Award, a Lilly Teaching Fellowship, the Distinguished Artifact Award for PLDI 2014, the Most Influential Paper Award at OOPSLA 2012, the Most Influential Paper Award at PLDI 2016, the ASPLOS 2019 Influential Paper Award, five CACM Research Highlights, a Google Research Award, a Microsoft SEIF Award, and Best Paper Awards at FAST, OOPSLA, and SOSP; he was named an ACM Fellow in 2019. Professor Berger is currently serving his second term as an elected member of the SIGPLAN Executive Committee; he served for a decade (2007-2017) as Associate Editor of the ACM Transactions on Programming Languages and Systems, was Program Chair for PLDI 2016, and is co-Program Chair of ASPLOS 2021. MODERATOR Milind Kulkarni Associate Professor, Purdue University [Source](https://www.youtube.com/watch?v=7g1Acy5eGbE)

[12:12] Emery Berger: So, performance is really brittle. If you change something you can get dramatic swings in in performance and the performance swings that you get are really vastly larger than the compiler can even produce. [19:19] Emery Berger: In modern applications, we care about things like latency. We have programs that run forever conceptually. Uh, they're parallel, they're asynchronous, they're concurrent and the existing profilers just have not kept up. They're essentially still stuck back in the '80s.

[39:05] Emery Berger: You can't optimize CPU performance if the bottleneck is not the CPU. [23:41] Emery Berger: What we’re going to do is we’re going to give the effect of speeding something up by slowing everything else down that’s running concurrently with it.

[0:02] Serving a global membership of nearly [0:04] 100,000 computing professionals and [0:06] students. [0:07] My name is Milind Kulkarni. [0:09] Currently, I'm an associate professor in [0:10] the School of Electrical and Computer [0:12] Engineering at Purdue University. My [0:13] research focuses on developing [0:15] languages, compilers, and systems that [0:16] can efficiently and effectively exploit [0:18] locality and parallelism in complex [0:20] applications on complex computation [0:21] platforms. I'm a member of the ACM and a [0:24] senior member of the IEEE. And for more [0:26] on my background, you can see the bio [0:27] widget on your screen. [0:31] For those of you who may be familiar [0:33] with unfamiliar with ACM or what it has [0:35] to offer, here's more information. [0:37] ACM offers educational and professional [0:39] development resources that bolster [0:40] skills and enhance career opportunities. [0:42] Our members can stay competitive in the [0:44] constantly changing world of computing [0:45] with a range of ACM Learning Center [0:47] resources at learning.acm.org. [0:49] And you can see some of those highlights [0:50] on your screen. [0:52] ACM recognizes the role of computing in [0:54] driving the innovations that sustain [0:55] competitiveness in a global environment. [0:57] ACM provides, among other things, timely [0:59] computing information published by ACM, [1:02] including Communications of the ACM and [1:03] Queue magazines, access to the ACM [1:05] Digital Library, the world's most [1:06] comprehensive database of computing [1:07] literature, [1:09] international conferences that draw [1:11] leading experts on a broad spectrum of [1:12] computing topics, support for education [1:14] and research including curriculum [1:16] development, teacher training, the ACM [1:18] Turing and ACM Prize in Computing [1:20] awards, and the updated ACM Code of [1:22] Ethics, a collection of principles and [1:24] guidelines designed to help computing [1:25] professionals make ethically responsible [1:27] decisions in professional practice. [1:30] ACM enables its members to solve [1:32] critical problems using new technology [1:34] that enriches our lives and advances our [1:36] society in the digital age. [1:40] Before we get started, [1:42] I'd like to quickly mention a few [1:44] housekeeping items as shown on the slide [1:45] in front of you. [1:46] First, the slides will advance [1:47] automatically throughout the event. On [1:49] the bottom panel, you'll find some [1:50] additional widgets and resources. [1:52] If you're experiencing problems with the [1:54] slides or audio, press the F5 key in [1:55] Windows, command R if you're on a Mac, [1:57] or refresh your browser on a mobile [1:59] device. Or you can close and relaunch [2:01] presentation. [2:02] To control volume, adjust the master [2:03] volume on your computer. And if you have [2:05] questions during this talk, please type [2:07] them into the Q&A box at any time and [2:09] click the submit button. [2:10] I'll organize the questions as Emery [2:12] speaks and he'll reserve some time at [2:13] the end of the presentation to address [2:15] them. [2:16] This session is being recorded and will [2:18] be archived. You will receive an [2:19] automatic email notification when it [2:21] becomes available. [2:22] And you can check learning.acm.org for [2:24] updates on this and other upcoming [2:26] webcasts. [2:27] Finally, at the end of the presentation, [2:29] you'll see a survey open on your screen. [2:31] Please take a minute to fill it out to [2:33] help us improve our tech talks. And you [2:34] may also open the link to the survey at [2:36] any time from the resources window. [2:39] You can use the Facebook and Twitter [2:40] widgets on the bottom panel to share the [2:41] presentation link with your friends, as [2:43] well as tweet comments and questions to [2:45] us using the hashtag ACM web. [2:47] We'll be watching for your tweets. And [2:49] we also have a community discourse page [2:51] to continue the discussion after this [2:52] webcast, including questions we won't be [2:55] able to get through during the Q&A [2:56] session. [2:59] Today's presentation is Performance [3:01] Really Matters by Emery Berger. [3:04] Emery Berger is a professor in the [3:06] College of Information and Computer [3:07] Sciences at the University of [3:08] Massachusetts Amherst, the flagship [3:10] campus of the UMass system. [3:12] He's been a visiting scientist at [3:13] Microsoft Research and at the [3:15] Universitat Politècnica de Catalunya [3:17] Barcelona [3:18] and the Barcelona Supercomputing Center. [3:21] His research spans programming [3:22] languages, runtime systems, and [3:23] operating systems with a particular [3:25] focus on systems that transparently [3:27] improve reliability, security, and [3:29] performance. [3:32] He and his collaborators have created a [3:33] number of influential software systems, [3:34] including Hoard, a fast and scalable [3:36] memory manager that accelerates [3:37] multithreaded applications used by [3:39] companies including British Telecom, [3:40] Cisco, Credit Suisse, Reuters, the Royal [3:42] Bank of Canada, SAP, and Tata, and on [3:44] which the Mac OS X memory manager is [3:47] based. [3:48] He's also created DieHard, an [3:49] error-avoiding memory manager that [3:51] directly influenced the design of the [3:52] Windows 7 fault-tolerant heap and Die [3:55] Harder. [3:56] A security memory a secure memory [3:58] management that was an inspiration for [4:00] hardening changes made to the Windows 8 [4:02] heap. [4:03] Emery's numerous honors include a [4:05] Microsoft Research Fellowship, an NSF [4:07] Career Award, a Lilly Teaching [4:09] Fellowship, and several awards from top [4:11] conferences such as PLDI, OOPSLA, [4:13] ASPLOS, FAST, and SOP. [4:15] He was named an ACM Fellow in 2019 and [4:18] is currently serving as an elected [4:19] member of the SIGPLAN Executive [4:21] Committee. [4:22] He served for a decade as associate [4:24] editor of the ACM Transactions on [4:26] Programming Languages and Systems, was [4:27] programming was program chair for PLDI [4:29] 2016, and is co-program chair of ASPLOS [4:32] 2021. [4:34] Emery, without further ado, [4:36] take it away. [4:38] All right, thank you so much, Milind. [4:40] So, let me go ahead and do the screen [4:41] share. [4:48] Get a little inception going. [4:52] Hopefully this is actually working. Yes. [4:59] Okay. [5:05] Great. All right, so I'm happy to be [5:06] here today to talk about why performance [5:08] really matters. [5:10] So, uh before I do, I'm going to talk [5:12] about our favorite family, uh The [5:13] Simpsons. And uh so, uh hopefully most [5:16] of you are familiar with this family. [5:18] This show has been on the air for uh for [5:20] an incredible amount of time. [5:21] Um but things have changed in uh in this [5:25] incredible amount of time. So, this is [5:26] uh The Simpsons today, but um The [5:29] Simpsons actually premiered in the '80s, [5:30] and in the '80s they looked really [5:32] different. [5:33] Um I would say they've improved [5:35] substantially over the years. Um and the [5:38] this sort of evolution turns out to have [5:40] been echoed not only in uh Simpsons, but [5:42] also in our computers. Uh so, uh Uh, if [5:45] you have a smartphone today, that [5:46] smartphone today may have six cores, uh, [5:49] each running over 2 GHz, 64-bit [5:51] processor, 4 gigs or more of RAM. You [5:54] compare this to the world in the '80s, [5:57] we had computers that were vastly less [6:00] advanced. Um, where it was, uh, the [6:02] speed is measured in megahertz, uh, the [6:04] processors were 16-bit at best, [6:06] sometimes even 8-bit, uh, and the amount [6:08] of RAM was much smaller. In fact, if you [6:10] divide these out, if you put one on top [6:12] of the other, uh, you can see, uh, [6:14] roughly speaking, that today's computers [6:17] are about 3,000 times larger, faster, [6:20] better than computers in the past. So, [6:22] before the '80s, when computers were so [6:24] slow, um, [6:26] basically programmers had to resort to [6:28] languages that were designed to have as [6:31] little overhead as possible and to be as [6:33] close to the metal as imaginable. Um, so [6:35] these, uh, these robots are meant to [6:37] represent these various programming [6:38] languages, starting with assembler, [6:41] moving into Fortran and C and C++. Uh, [6:44] and I like to call these the metal [6:45] languages. Uh, these languages, by [6:47] design, have virtually no overhead, and [6:49] that was really important because [6:51] computers were just so slow. [6:53] However, uh, in the intervening years, [6:55] we've been on, uh, this performance [6:57] ride. So, right starting around, uh, the [7:00] late '70s and early '80s, um, computers [7:02] started getting faster and faster. [7:05] And so, what happened was we had these [7:07] twin trends that were occurring. Um, [7:09] one, the number of, uh, transistors was [7:12] going up at a rate of, uh, of roughly [7:14] twice every 18 months, known as Moore's [7:16] Law. Um, I draw your attention to the [7:18] fact that this is a log scale, so it [7:20] looks like it's linear, but it's really, [7:21] really super linear. It's going up by, [7:23] uh, uh, exponentially. Um, and at the [7:26] same time, clock speeds were going up. [7:28] And so, what happened was these chips [7:30] were getting more and more, uh, gates on [7:32] them, more and more transistors, uh, and [7:34] they were getting faster and faster. And [7:36] so, the effect was every year, uh, your [7:39] you could literally go and buy a new [7:40] computer and your computer would just [7:42] run faster. [7:44] So, what the implication was was that [7:47] these metal languages maybe were no [7:49] longer necessary because computers were [7:51] getting faster and faster and we could [7:53] afford to be a little more slack with [7:56] how we built these languages. And so, in [7:58] the '80s and the '90s, there was this [8:00] period of time where a bunch of other [8:02] languages came out that were vastly less [8:04] efficient than C or C++. Um starting [8:08] with Pearl and including languages like [8:10] Tcl/Tk, PHP, Ruby, and Python. Um I like [8:14] to call these the irrational exuberance [8:15] languages. These were languages where [8:18] people simply thought things were just [8:19] going to get faster and faster forever, [8:21] and so it was okay to have pretty [8:23] substantial inefficiencies. [8:25] So, I'm going to focus today on Python [8:27] when I'm talking about these, but all of [8:29] these languages share very similar [8:30] features. [8:32] So, the arrow is pointing to Homer's [8:34] stomach, and that's because [8:36] all of these languages are kind of [8:37] bloated [8:38] in terms of the amount of space they [8:40] consume. So, [8:42] you might think that an int, for [8:44] example, is relatively small amount of [8:47] information. It's a small number of [8:48] bytes. And in C++, the size of an int is [8:51] going to be four. [8:52] However, in Python, that same int is [8:55] actually 24 bytes. So, it's six times [8:57] larger than it needs to be. [9:00] This is true for really it's just true [9:01] across the board, so it's not just [9:03] numbers. So, an empty list in C++ would [9:06] be 24 bytes, and in Python, it turns out [9:09] it's 56. So, more than double. [9:12] And then my favorite example is an empty [9:14] dict, that is to say an associative [9:16] array or a map. [9:17] The size of it in C++ is 24 bytes. In [9:21] Python, today it's 64. [9:24] So, it's nearly three times as much, but [9:27] this is the result of Python putting on [9:29] a little bit of a diet up until [9:32] just a year or so two years ago. It It [9:35] 240 bytes. So, it's 10 times more [9:37] memory. Um and I note this is [9:39] suspiciously close to uh to Homer [9:41] Simpson's own weight. So, definitely uh [9:44] Python and these other languages um are [9:47] inefficient in terms of memory [9:48] consumption, but they're also famously [9:51] inefficient in terms of performance. Uh [9:53] so, they're infamous that they're slow. [9:56] Um and so, just to give you a sense of [9:58] how slow they can be, um here is a [10:01] sample bit of code that does matrix [10:03] matrix multiply. [10:04] Uh so, this is the naive matrix matrix [10:06] multiply just expressed in a loop in [10:07] Python. Um and uh and I I I want to [10:10] thank Charles Leiserson for uh for this [10:12] example. Um here is a graph showing the [10:16] difference in applying different sort of [10:19] uh techniques to the uh this Python [10:22] code. [10:23] I'm sorry, to matrix multiply. So, if [10:25] Python is one, C just writing it in C is [10:28] 47 times faster. And when you apply all [10:31] the optimizations that you can do in C [10:33] and that are included in standard C [10:35] libraries, it's over 60,000 times [10:37] faster. [10:38] So, this is a pretty big gap. Um and you [10:41] know, maybe it seemed to make sense when [10:42] Python uh was going to get faster by [10:45] virtue of the fact that we were on this [10:46] ride, and this ride seemed like it was [10:48] going on forever. But unfortunately, the [10:51] ride did not go on forever. And in fact, [10:53] it stopped. So, we were here, but now [10:57] we're here. Um the ride is over. So, [11:00] starting around 2010, uh the ride just [11:02] ended. And what happened was this trend [11:05] that we hoped and expected was going to [11:08] continue of you know, drawing this line [11:10] out. You can see the dotted lines for [11:12] transistors and clock speed. [11:14] What happened was we still got more [11:16] transistors, right? The number of [11:18] transistors continued. That is to say, [11:20] this is Moore's law, and Moore's law has [11:22] not yet ended. It has to end eventually, [11:25] but has not yet ended. Um however, what [11:27] did end was uh scaling of clock speed. [11:31] Uh and this is referred to as Dennard [11:32] scaling. And Dennard scaling has simply [11:34] run out of steam for a variety of [11:36] reasons. Um I'm not going to go into [11:38] those here, but in in short, the chips [11:40] are basically too hot or they leak too [11:42] much voltage. So, it's not really [11:44] possible to clock these things up in any [11:46] realistic way. [11:48] So, things were great in the '80s. Um [11:51] you know, if you needed to speed up your [11:52] program, you just didn't do anything. [11:54] You could literally just go and have a [11:55] beach vacation uh and wait, and your [11:58] code would run faster by virtue of the [12:00] fact that time had elapsed and computers [12:01] had gotten faster. But today, you have [12:03] to work. Today, it's really hard. Um and [12:06] so, the only opportunity Homer has for a [12:08] Duff is going to be in the form of an [12:10] energy drink at this point. [12:12] So, I'm going to explain right now why [12:14] performance has gotten so complicated uh [12:16] and why we're going to have to do new [12:18] things to get us out of this these [12:20] problems that we in some way created for [12:22] ourselves. [12:23] So, here is the way a typical [12:26] performance evaluation goes, where [12:28] somebody says, "I have an idea, like [12:30] Homer has an idea. I'm going to speed up [12:32] a program." So, Homer goes and rewrites [12:34] some code, uh changes around some [12:36] things, maybe refactors some stuff out, [12:39] and eventually ends up with an optimized [12:40] version. Um at least he hopes it's going [12:42] to be optimized. So, he goes and he runs [12:44] the programs. Uh there's A, the original [12:47] program, and A prime, the optimized [12:49] version. A takes 90 seconds. Great. A [12:52] prime takes 87.5 seconds. So, Homer [12:55] says, "Woo! 2.8% faster." All right? So, [12:58] his genius idea to optimize the code was [13:01] uh was successful. [13:03] Uh or so it seems. [13:05] So, it is faster. And the question is, [13:07] why? Why is it faster? So, of course, it [13:09] could be just a total accident. So, what [13:12] could have happened was this interaction [13:15] between the changes that Homer made and [13:18] the underlying hardware. So, there's a [13:20] lovely paper uh that a few of my [13:22] colleagues uh wrote uh back in 2009. [13:25] Um the the paper has a title, "Producing [13:28] Wrong Data Without Doing Anything [13:30] Obviously Wrong." [13:31] Uh, and it gathered together a bunch of [13:33] information showing that very modest [13:36] changes, uh, things that you would never [13:38] expect would have a big impact, can [13:39] actually have a gigantic impact on [13:41] performance. And so, [13:43] things like changing the link order, [13:46] that is to say, if you have a makefile [13:48] and you have a bunch of .o's, if you [13:50] move those .o's around, that changes [13:52] where the functions end up in memory. [13:54] Likewise, if you change the environment [13:57] variables, the environment variables [13:59] actually are loaded into memory when you [14:02] load your program and then the stack [14:03] starts after it, so it moves the program [14:05] stack, which means the data has moved [14:07] around. Right? So, you might think, [14:09] "Well, okay, these are fine. These seem [14:11] like they shouldn't have any effect." [14:12] But, it turns out they can have a [14:14] gigantic effect. They can have an effect [14:16] that is larger than the impact of of no [14:19] optimization to full optimization, which [14:22] is you can get plus or minus 40% swings. [14:25] So, let me explain what's going on. Uh, [14:28] here's an example. So, suppose in the [14:30] original program A, before Homer made [14:33] his changes, uh, there's two hot lines [14:36] of code and they both mashed mapped to [14:38] the same cache set. [14:40] So, modern hardware, because especially [14:42] because things have run out of steam, [14:44] they've gotten quite complex in an [14:46] effort to uh, extract every bit of [14:48] performance out. And so, there are these [14:50] on-chip caches. The caches are organized [14:52] into sets and uh, they're not like um, [14:55] they don't necessarily take full [14:56] advantage of the whole size of the [14:58] cache. And if you have a bunch of things [15:00] that map to the set, it can overflow it [15:01] and you get a result called a conflict, [15:04] uh, which means that the cache is no [15:05] longer efficient and you have to go [15:07] fetch uh, information from memory, which [15:09] is slow. [15:10] Um, however, when Homer made his changes [15:13] and made his optimized version, it could [15:16] have been the case that just by sheer [15:18] accident, uh, he ended up [15:20] making it so that uh, things that used [15:22] to map to the same set no longer do and [15:25] there's something that's either cold or [15:26] there's just nothing there at all [15:28] resulting in no conflict. [15:30] So, going from a conflict to no conflict [15:33] could mean that your program is faster [15:34] simply because of the cache. [15:37] So, you might think well this is maybe [15:39] the only issue to deal with but it turns [15:41] out the cache is not the only thing to [15:42] blame. There are lots of performance [15:44] sensitive microarchitectural features on [15:47] modern chips that are influenced by [15:49] program addresses or data addresses. [15:51] These include a branch predictor which [15:54] tries to predict whether which way a [15:57] branch is going to go like is it is it [15:59] the if condition or the else condition? [16:01] That's based on tables that are indexed [16:03] by program counter location. [16:05] There's a thing called a TLB which maps [16:08] your virtual addresses to physical [16:09] addresses and if you have [16:12] something that used to fit in the TLB [16:13] which is a kind of cache and now because [16:16] you've moved things around things [16:17] straddle pages and they no longer fit in [16:19] TLB that can seriously degrade [16:20] performance or vice versa you could make [16:23] things accidentally fit and then things [16:25] get better. [16:26] There's a thing called a branch target [16:28] predictor, there's a prefetcher, suffice [16:30] it to say that there's lots of stuff [16:32] going on and it's quite complicated. [16:34] All right? In addition, [16:36] optimizations themselves over time have [16:39] kind of run out of steam. Even so, if [16:42] you factor out all of these layout [16:43] changes [16:45] it turns out that across a vast array of [16:48] these these applications that people use [16:50] to test compilers the speed ups that you [16:52] get that are statistically significant [16:54] are really quite small. So, [16:57] you really can't count on much [16:59] assistance from the compiler to really [17:02] get you out of this problem. All right? [17:04] So, performance is really brittle. [17:06] If you change something you can get [17:09] dramatic swings in in performance and [17:11] the performance swings that you get are [17:13] really vastly larger than the compiler [17:15] can even produce. [17:16] So, Homer needs help. [17:19] Homer can't rely on compiler [17:20] optimizations anymore. Can't rely on [17:22] computers getting faster every year. [17:24] And unfortunately, tools have not kept [17:26] up. So, let me show you an an example of [17:30] where tools fail for Homer. [17:33] So, suppose Homer is in modern day 2020 [17:38] and has an idea for an app. [17:40] And Homer's idea for the app is this [17:42] great idea cuz it involves cats and the [17:45] internet. [17:46] So, the idea is you go on your [17:47] smartphone and you take a picture of a [17:49] cat. And then it gets sent to some [17:53] servers and it comes back and it returns [17:55] all the images that match this cat in [17:56] the form of memes. So, this is great. [17:59] Clearly, it's going to take over the [18:01] internet. He has a great idea for the [18:03] name. He's going to call it Augle. And [18:05] Homer is convinced it's going to totally [18:07] disrupt image search. [18:09] So, so he goes and sets about building [18:10] Augle. Here's the the way the prototype [18:13] is going to work. You take a picture. [18:16] It sends it to the Augle servers. [18:18] And in Homer's genius move, it saves the [18:20] image in the database so it can be [18:22] retrieved later by other Augle users. [18:25] It then goes and finds similar images [18:27] and sends the results back. [18:30] So, if we step back a minute, I'm just [18:32] going to describe the software [18:33] architecture of this of the of Augle. [18:36] So, [18:37] in a sense, when you go to Augle, it's [18:39] going to create two separate threads. [18:41] One of which is going to go and do some [18:43] compression of the image. Maybe shrink [18:46] it down in size to a thumbnail or [18:48] something. It's going to save it to a 3 [18:52] and 1/2 inch floppy disk as servers do. [18:55] And at the same time, in parallel, it [18:58] can actually be extracting the features [18:59] that it's going to use to look up these [19:01] other images, then perform the search, [19:04] and then send them back to the user. [19:06] And of course, this is the whole process [19:08] and they sync at the end. [19:09] So, I've simplified things a lot. Not [19:11] just in the form of putting these in [19:12] term in icons but also it we're talking [19:15] about concurrency. Concurrency involves [19:17] these things like locks to prevent [19:19] things from happening simultaneously and [19:21] collaborate each other. [19:22] So real applications obviously more [19:24] complex. Fine. Let's say Homer goes and [19:27] produces this application, ships it, [19:29] everybody loves Oggle [19:31] but then Oggle ends up being really [19:33] slow. [19:35] So this is really sad. Homer is sad. [19:37] What is Homer to do? [19:39] So what Homer would normally do, what [19:41] any of us normally in principle should [19:42] do is go use a software profiler. [19:45] So a software profiler [19:47] essentially goes and takes your program [19:49] and produces statistics about it and [19:51] those are meant to inform you as to what [19:53] you should do to make the program run [19:55] faster. [19:56] So the big ideas in software profilers [19:58] really came out in the 80s and 90s [20:02] and this is a bit of a caricature but to [20:04] a first approximation this is what [20:06] profilers do. [20:08] They do something like report the number [20:10] of calls that you make to each function. [20:12] And they report the run time for each [20:14] function. So you get these statistics [20:16] and what you're supposed to do with them [20:17] nominally is you find the code that is [20:21] frequently executed and it runs for a [20:23] long time and that's the code where you [20:25] should spend your time, right? That's [20:27] what you optimize and that's going to [20:28] make your program run faster. [20:30] So the question is when we do this in [20:32] Oggle is this actually going to help? [20:34] Would this speed up Oggle? So [20:37] suppose Homer takes the profiler [20:40] gets the information, finds the code [20:42] that lasts the longest, that runs the [20:44] most time [20:46] and he applies it to Oggle [20:48] and after a lot of effort it turns out [20:51] oh no, what he did was he optimized the [20:54] loading blinky part. [20:56] And so that's not really helping Oggle [20:59] go any faster but what he did was he [21:02] found frequently executed code. That [21:04] loading thing is quite frequently [21:06] executed. And it runs for a long time. [21:08] In fact, it runs for the longest time. [21:10] It runs from the beginning to the end of [21:12] the whole augel process. [21:14] Um, so [21:15] optimizing that is clearly a mistake. [21:17] That's not what he should have been [21:18] optimizing, right? And it turns out that [21:20] existing profilers just don't do a great [21:23] job of finding the important code in [21:25] modern applications. In modern [21:26] applications, we care about things like [21:28] latency. We have programs that run [21:30] forever conceptually. Uh, they're [21:33] parallel, they're asynchronous, they're [21:34] concurrent and the existing profilers [21:36] just have not ca- kept up. They're [21:38] essentially still stuck back in the [21:39] '80s. [21:41] So, we need to do better. [21:42] So, [21:43] the question is, what would speed up [21:45] augel, right? So, we would like to know, [21:48] ideally, uh, imagine there was some sort [21:50] of magic pixie dust that we could [21:52] sprinkle over the augel source code and [21:54] it would land on the code where where [21:57] Homer needed to spend his optimization [21:59] efforts, right? So, suppose it was this [22:01] magnifying glass, that is to say, the [22:02] search part. That would be great. I [22:04] would tell Homer, go work on this. It's [22:06] guaranteed to produce a certain result. [22:09] So, what would this information look [22:10] like? [22:11] So, [22:12] maybe not going to look like sparkles on [22:14] your source code. That might not be as [22:15] helpful as you'd want. Um, so, what we [22:17] propose is something we call a causal [22:20] profile. So, a causal profile tells you [22:23] where your optimizations will make a [22:25] difference and tells you how much of a [22:27] difference they'll make. So, suppose [22:29] you've got these components of your [22:30] program [22:31] and you've got a graph. On the x-axis, [22:34] you've got the speed up. Uh, that is the [22:36] speed up for an individual component. [22:38] And on the y-axis, you've got the [22:40] program speed up or the improvement in [22:42] throughput or the reduction in latency. [22:44] And we'll talk more about that in a [22:45] minute. [22:46] The kind of graph that you'd like to see [22:48] is, look, if I spend a lot of time on [22:51] the search part, I'm going to get a bang [22:53] for my buck that's pretty high up until [22:56] a certain point when it plateaus. So, [22:58] I'll know to optimize and I'll know when [23:01] it's time to stop my optimization [23:02] efforts. [23:03] Similarly, I'll find code where I [23:05] shouldn't spend any time. Right, it [23:07] turns out this compression thing maybe [23:08] is never going to be a bottleneck no [23:10] matter how fast I make it. So, this is [23:12] really useful to know. I'm not going to [23:14] work on that at all. And I might find [23:16] other code like the indexing part, uh [23:18] feature extraction, where yes, I don't [23:21] get as big a bang for my buck as with [23:23] the uh the search, but if I keep working [23:25] on it, I'll make the program run faster [23:27] and faster. [23:28] Right? So, So, this is great. Uh it [23:31] tells you really if I speed up a [23:33] particular component by a certain [23:35] amount, I will get a certain speed up. [23:37] And so, I can plan my effort [23:39] accordingly. [23:41] So, this is all terrific. It would be [23:42] wonderful to have. Um the the challenge [23:45] is how do we get this information? So, [23:47] what I want to know is how I do a [23:49] certain change, right? I get this [23:51] effect. Um so, I'm going to describe [23:53] something that's impossible first and [23:55] then show you how to make it possible. [23:57] So, the impossible part is we're going [23:59] to run experiments. Let me call these [24:00] performance experiments. So, these [24:03] experiments depend on magic. [24:06] So, if we could magically speed up every [24:09] component, uh we could speed it up and [24:12] then observe what the effect is, right? [24:14] So, we speed it up by a little bit and [24:16] we get a certain speed up. We speed it [24:18] up a little more, we observe another [24:21] speed up, right? And so on, right? And [24:23] then we just do this for every [24:25] component. We see, well, if I speed that [24:27] up, that has no effect, and so on. [24:30] Right? And then we can use this to build [24:31] our graph. Unfortunately, we're going to [24:34] have to do this without magic because [24:36] magic is not a thing. Also, if we have [24:38] magic, this is what we would do. We'd [24:40] magically optimize the things to zero, [24:43] uh and it would be awesome. [24:45] But, magic doesn't exist. Sorry, Homer. [24:48] Um so, we need another way. So, how can [24:50] we do these performance experiments [24:51] without magic? [24:53] So, the key insight that uh causal [24:55] profiling depends on is using something [24:58] that we call virtual speedups. So, a [25:00] virtual speedup is kind of like a theory [25:02] of relativity thing. [25:04] What we're going to do is we're going to [25:05] give the effect of speeding something up [25:08] by slowing everything else down that's [25:10] running concurrently with it. So, here [25:13] when we go and we speed up virtually the [25:17] the transmission, what we're really [25:19] doing is we're slowing other things [25:20] down. Turns out it's really easy to slow [25:22] things down. Just harder to speed them [25:24] up. So, when we slow those things down, [25:27] we can then observe the resulting [25:28] slowdown, and that slowdown corresponds [25:31] to the speedup [25:33] the results we would have gotten had we [25:35] actually sped that thing up. [25:38] So, we get a point in our causal graph, [25:40] and we can just rinse and repeat. We do [25:42] this over and over again. We observe the [25:44] results, and we build out the graph. [25:47] And we'll do this for component by [25:49] component, just as described before. [25:52] So, here we observe the effects. [25:55] And eventually we get all the results [25:57] for the red one, and finally we do the [25:59] same for everything. All right. So, [26:01] there's our causal profile. So, this is [26:03] great. Now, I've spoken about this in [26:04] terms of program speedup, but this [26:07] doesn't necessarily make sense in the [26:09] context of a program that is not meant [26:11] to ever end. And the Ogle server [26:13] conceptually runs forever, and making [26:15] the Ogle server terminate quickly is not [26:17] a goal, right? That's not what we want. [26:20] So, what we actually care about is we [26:22] care about things like [26:24] how long does it take between sending a [26:27] request to Ogle and getting a response [26:29] back. [26:30] And this is referred to as latency. [26:33] Right? So, the other thing we might care [26:35] about is how fast did the results come [26:37] back? How quickly is are these things [26:39] being served up? [26:40] And this is called throughput. [26:42] So, [26:44] when you're doing causal profiling, you [26:45] don't just run causal profiling with an [26:47] ordinary program without any changes. [26:49] You have to make some slight uh [26:51] annotations to the program. We call [26:54] these annotations progress point. [26:56] Uh so progress point. So uh for example, [26:59] suppose Homer wants the throughput to be [27:01] higher for sending responses back. So to [27:04] optimize this, [27:05] Homer has to mark it as a progress [27:06] point. And in our implementation, this [27:09] is just a macro. [27:10] And so the way this works is every [27:12] single time the code runs, um it [27:14] registers this fact, uh and it keeps a [27:17] counter, and it runs the experiments and [27:18] tries to accelerate the rate at which [27:21] those things are occurring, or it [27:22] observes the impact on those things. [27:24] And so when you have uh the real server [27:26] that's deployed, it's doing this for all [27:28] of them simultaneously, and this one [27:31] progress point that you put in measures [27:33] throughput and gives you a graph where [27:35] the Y axis is throughput. [27:38] So that's with one progress point, [27:39] right? So it says, "If I speed up some [27:41] component, how much faster will I get uh [27:44] that particular thing to run?" [27:47] However, what if Homer wants to minimize [27:48] response time? That is the time between [27:51] starting a request to Oggle, like [27:52] saying, "Here's my kitty cat." until you [27:54] get the pictures back. So here, uh you [27:57] have to add two different kinds of [27:59] progress points. We call them latency [28:01] progress points. There's a begin and an [28:03] end. [28:05] Um and the way it works internally is [28:07] that this begin and end just keeps track [28:09] of the number of things going through [28:11] the system, and it also keeps track of [28:13] the rate. [28:14] And it turns out that we can exploit [28:16] this uh this classical result from the [28:18] field of queuing theory called Little's [28:20] Law. [28:21] Uh and the way that Little's Law works, [28:23] we've reordered it a little bit, but it [28:24] turns out that you can actually get [28:26] latency [28:28] just by keeping track of the number of [28:29] transactions in the system [28:31] and the throughput, and we can measure [28:33] the throughput by using uh the um the [28:36] existing progress point functionality [28:38] that I described before on one of these [28:40] progress points. [28:41] Great. So we built a causal profiler uh [28:44] for Linux. It's called cause. Um and it [28:47] ships with Debian and Ubuntu. It's quite [28:49] easy to install. So, on Linux, as long [28:53] as you have superuser privileges, you [28:54] can install it. [28:56] And then, you run it just by saying [28:57] cause run {dash} {dash} {dash} and your [29:00] program and the arguments. And then, it [29:03] initializes it it starts running these [29:05] random performance experiments on code [29:08] that's running and observes the effect. [29:10] Again, you put in your progress points [29:12] and then it'll produce results that [29:14] either are about latency or about [29:15] throughput. [29:17] So, I'm going to show you [29:19] some like case studies of using causal [29:21] profiling on augle. [29:23] So, we didn't really build augle, but we [29:25] have the components that could make up [29:27] an augle. So, one is the compression [29:30] part. [29:31] It's called dedupe. Another is image [29:33] comparison that actually allows you to [29:35] do this business of looking up images [29:37] and finding similar ones. [29:39] And of course, hooking it up with a [29:40] database like SQLite. [29:43] So, let me talk to you first about [29:45] Ferret. So, here's what happened when we [29:48] ran Ferret with with causal profiling [29:51] where we wanted the throughput to be [29:52] higher. [29:54] And what we discovered was that you can [29:56] see that there's a couple places where [29:58] there's a substantial bang for the buck, [30:00] especially on the top one. And it turned [30:03] out that all of these things turn were [30:05] in separate components of Ferret. So, [30:08] the top one is in ranking, middle [30:10] indexing, the bottom is segmentation. [30:13] And Ferret is organized as a pipeline. [30:16] And so, these are the colors that came [30:18] from the previous graph. [30:21] And you can see it starts with input and [30:22] it sends everything through and then [30:24] produces output. [30:25] And the way that Ferret is built is that [30:27] it assigns a fixed number of threads to [30:29] every stage. But we now know that [30:32] certain stages, speeding them up can [30:34] have a big impact. And some like feature [30:36] extraction don't matter that much at [30:38] all. So, they They don't need as many [30:40] threads. So, what we did, probably one [30:43] of the simplest optimizations we've ever [30:44] done, [30:45] all we did is we reorganized the number [30:47] of threads that were assigned to things [30:49] based on the predicted speed up. [30:52] And in fact, we got a 21% speed up just [30:55] by reordering or reorganizing which [30:57] threads go where. [30:59] So, this is not something that would [31:00] show up in a normal profile at all. [31:03] And and so, this was really great. We [31:05] were able to get this very, very quick [31:06] speed up with minimal effort. [31:08] So, one of the cool things is we're also [31:10] able to validate that Cause really was [31:12] doing what we hoped it would do. [31:14] So, for example, if we focus in on the [31:17] ranking part, we increased it from 16 [31:19] threads to 22 threads. This is a 27% [31:22] increase in the amount of resources [31:24] devoted to ranking, which should lead to [31:26] a 27% increase in ranking throughput. [31:29] And if you look at the graph, a 27% [31:32] increase in ranking throughput leads to [31:34] a 21% improvement according to Cause's [31:37] predictions, which was exactly what we [31:39] observed. [31:40] So, that's great. Cause is doing what [31:42] it's supposed to do, and it's guiding us [31:43] to optimizations that were would [31:46] basically have been impossible to find [31:47] otherwise. I'll walk you through one [31:49] other one real quick cuz it's a lot of [31:50] fun. [31:51] So, we looked next at Dedupe. Dedupe [31:55] does compression by deduplication, [31:57] hence the name. [31:59] And the way it works is really simple, [32:01] in in conceptually simple, I should say. [32:03] You take a picture, [32:05] and you've got another picture which is [32:06] quite similar. [32:07] We're going to compress how much space [32:09] is stored for these pictures by carving [32:12] out the elements that are definitely the [32:14] same, [32:15] and the ones that are separate, and then [32:16] splitting every image instead of being [32:18] one object into a pointer to multiple [32:21] objects that are their components. So, [32:24] the first image is Grumpy Cat 1, and [32:26] it's these three pieces, and the second [32:28] is Fun Is Awful, and it has those other [32:31] pieces in this shared component only [32:33] occupies, [32:34] you know, the memory that the one image [32:36] takes up instead of being duplicated. [32:38] Great. So, the way that Ddup works is [32:41] that there's a hash table. Uh the hash [32:43] table has some hash function that it [32:45] applies to the images. [32:46] Um and then it has buckets, uh and it [32:49] stores them in not not the image itself, [32:51] but a pointer to it into these various [32:53] buckets. All right. So, in theory, these [32:56] things should be spread out pretty [32:58] uniformly across all of the uh the hash [33:00] table, right? So, you should have pieces [33:02] here and pieces there. [33:04] Um it turns out that uh Kostas actually [33:07] noticed that there was a problem in the [33:09] hash table. [33:10] And so, we were surprised by this. [33:13] Uh and we thought maybe the issue was [33:15] that there was one big lock that was [33:16] causing contention, but it turned out [33:18] that was not the issue. [33:20] Um so, in fact, there are many many [33:22] locks. Uh crazy number of locks. So, [33:25] so then we were like, "Oh, hmm, wonder [33:27] what's going on?" So, Kostas [33:29] specifically said that the loop that [33:30] accesses this list is important, and [33:33] we've now ruled out anything having to [33:35] do with locks. [33:37] But, in principle, uh we said, "Well, [33:40] maybe there's just not enough hash [33:41] buckets, and so it's leading to a lot of [33:43] collisions." So, we increased the number [33:45] of hash buckets, and this led to no [33:47] performance improvement whatsoever. So, [33:50] uh really the question that we ended up [33:52] with was what else on earth could [33:54] possibly be causing collisions? So, we [33:56] took a look at the hash function, and it [33:58] turns out that the hash function was a [34:00] catastrophe. So, the hash function um [34:03] this this graph shows uh for all the [34:06] buckets, we had made it 1,024 buckets, [34:09] how many items ended up in each bucket, [34:11] and you can see that there's a ton that [34:13] end up in a very small number of [34:15] buckets. Uh so, the overall utilization [34:17] was just 2%. That is, all of the things [34:20] been uh they hashed to just 2% of the [34:23] bins, which is really no good. [34:26] Um it turns out that the hash function, [34:28] first, you shouldn't roll your own hash [34:29] functions in general unless you're very [34:31] careful. [34:32] Whoever wrote this used a plus sign to [34:35] combine the values in the images. [34:38] And this actually leads to a normal [34:40] distribution, which is not what you [34:42] want. You want a uniform distribution. [34:44] So, by changing the plus sign to the [34:47] XOR, which is the carrot, [34:50] we were able to get 82% bin utilization. [34:53] So, literally changing one character, we [34:55] got a 9% speed up. [34:56] And it turns out that again, we're able [34:59] to actually measure whether Cause [35:01] Cause's predictions were accurate. [35:03] The blocks per bucket on average went [35:06] from 76 to two. This led to a 96% [35:10] reversal speed up. [35:11] And that, according to Cause, was a [35:13] would lead to 9% speed up, which is what [35:16] we got. [35:17] So, we optimized other things. I don't [35:19] have time to talk about these in this [35:21] talk. But we optimized SQLite and sped [35:23] it up by 25%. [35:25] Also, via an optimization that would [35:27] never have been revealed by any other [35:29] kind of profiler. As I mentioned before, [35:31] 9% on D dupe, 21% on Ferret. We've [35:34] applied Cause to a bunch of different [35:36] things. [35:37] We've gotten some substantial [35:39] improvements with very, very small [35:41] amount of programmer effort. So, Cause [35:43] is quite effective at pointing us to [35:45] exactly the lines of code that need to [35:47] be changed. My favorite is probably the [35:50] the ones the fluid animated stream [35:51] cluster [35:53] where the authors wrote their own custom [35:56] barrier to synchronize a bunch of [35:58] parallel threads. And this turned out to [36:00] be a pretty poor implementation, or at [36:02] least one that was not anywhere near as [36:04] good as the system implementation. So, [36:06] we removed the custom one and used the [36:07] system supplied one and got 38 or almost [36:10] 70% speed up by deleting a line in [36:12] effect. [36:14] So, great. So, that's Cause. We've [36:17] basically got effective performance [36:19] profiling now that is good for modern [36:22] applications. [36:23] And it's easy to install and use. And I [36:25] if you if you have any performance [36:27] issues that you're interested in dealing [36:28] with that are in uh C or C++ or Rust or [36:32] Java, then those this is where Cause [36:35] shines. [36:36] So, like I mentioned, [36:38] this is great for your applications [36:40] written in these metal languages, in the [36:42] not-so-metal language of Java, [36:44] uh in the [36:47] maybe scary for metal uh language called [36:49] Rust. [36:50] So, these are fine. Um these are all [36:53] great languages. Um but um [36:56] I think he ran away cuz of the Rust. Um [36:58] so, anyway, [36:59] uh what about going back to these [37:01] languages like uh the slow languages I [37:04] mentioned before, like Python? [37:06] So, so uh I got a little run in there. [37:08] Right? So, Python is still a problem. [37:11] Why is Python a problem? So, Python is a [37:13] problem because it is, as I mentioned, [37:16] really fat and slow. [37:18] Um but Python is an unusual language uh [37:21] in the sense that um well, there's many [37:23] senses in which Python is an unusual [37:24] language, but in particular, Python is [37:27] really um deeply embedded in an [37:29] ecosystem, which is it written in C. Um [37:33] and this means that uh it's it can [37:36] really be used as kind of a glue [37:38] language to take your Python code uh [37:41] that just glues together a bunch of [37:43] components that are written in C, that [37:45] is to say, in metal languages. And to [37:49] the extent that you can do this, this [37:50] will speed up your Python programs a lot [37:52] because really the Python code won't be [37:55] spending very much time uh running and [37:58] just sort of orchestrating things in [38:00] these metal languages, which as we know [38:02] are going to be vastly faster. [38:04] So, to this end, we've built a profiling [38:07] system that we call Scalene, uh and [38:09] Scalene is designed to help Python [38:11] programmers move their code from uh [38:15] bloated slow Python into fast and [38:18] efficient uh metal languages like C, [38:21] um with [38:22] basically just telling them which parts [38:24] of their code need to be uh need to be [38:26] moved into the into metal land, if you [38:28] will. So, Scalene is quite easy to use. [38:31] If you're writing your you've got your [38:32] Python program, instead of invoking it [38:34] with Python, you invoke it with Scalene. [38:36] Uh, there's a number of options. You can [38:38] tell it I only want to profile CPU. Um, [38:41] I would like HTML output. Uh, you can [38:43] get a list of all of these options by [38:45] typing {dash}{dash}help. [38:47] So, I'm going to walk through a simple [38:48] little program. This is a program that [38:50] was uh sent to us. Um, and it turns out [38:54] that this uh this program we were able [38:56] to quickly identify a problem by using [38:59] Scalene and uh and speed it up quite a [39:01] bit. So, it's a it's nice and small and [39:03] fits on a slide. Uh, so I'll show you [39:05] what uh what Scalene produces. [39:08] So, when you run Scalene on this code, [39:10] you get a lot of information, and so I'm [39:12] going to walk you through what it's [39:13] telling you. [39:15] First, it shows the amount of CPU time [39:17] that's being spent, but unlike a a [39:19] normal profiler that just says, "Here's [39:21] where I spend CPU time." Uh, Scalene [39:23] divides it up into the time spent in [39:25] Python and the time spent in native [39:27] code, where native means the metal. Um, [39:31] it also indicates how much time is spent [39:32] in the system. And the system means IO, [39:35] for example. Uh, and that's another [39:37] thing that you might want to be able to [39:38] like you want that information to know [39:40] what you can optimize and what you [39:41] cannot. You can't optimize CPU [39:43] performance if the bottleneck is not the [39:45] CPU. [39:47] Uh, next, it shows how much memory is [39:49] being consumed. [39:50] How much of it is from Python? Uh, so is [39:53] it Python that's allocating memory or is [39:55] it the native code? And where is it [39:57] coming from? So, you can see um, the [40:00] line six says 763 megabytes [40:03] are produced by this one line of code, [40:06] and it's not Python. So, 0% is uh is not [40:09] shown on the screen. It means that it's [40:11] definitely not Python allocating the [40:12] memory. [40:14] It also has this handy-dandy little [40:16] graph that it creates. Uh this is [40:18] inspired by uh uh sparklines uh from [40:21] Tufte. Uh the idea is that you basically [40:24] can see it as a graph over time. Uh so [40:27] the x-axis, if you will, is showing you [40:29] time and the y-axis is showing memory [40:31] consumption. [40:32] At the top, you can see the memory [40:34] consumption over the lifetime of the [40:35] program. And on the bottom, on line six, [40:38] it's the 87% that is just uh for the [40:41] program on that line of code. [40:43] And then finally, it has this new metric [40:46] that we call copy volume. And this is an [40:48] important metric because copying is what [40:51] happens when you go between Python and [40:53] native code or C. And when those results [40:56] come back, and if you do that too often, [40:58] that itself can be a bottleneck. So you [41:00] have to be careful about how much stuff [41:02] you do. It's best if you sort of get all [41:05] the stuff in native land, the stuff runs [41:07] in native side, and then comes back [41:09] instead of constantly going back and [41:11] forth. [41:13] So in sum, you can see that uh for this [41:15] particular line of code is clearly [41:17] problematic. It's spending 34% of the [41:19] total run time in native code. [41:22] Um 763 megabytes are being allocated. [41:25] 87% of the total memory activity is uh [41:28] is happening on this one line of code. [41:31] Um and it's exhibiting this sawtooth [41:32] pattern, which turns out to be uh a very [41:36] clear indicator that copying is [41:37] happening. And this really clinches it. [41:40] It says there's just a ton of copying [41:42] happening. So what's happening is uh [41:45] by accident, the programmer put in this [41:48] call np.array, which is meant to convert [41:52] the input from uh whatever it is, like a [41:54] Python array, into a NumPy array, which [41:57] is much, much smaller and uh and more [41:59] dense, since the metal representation. [42:02] But it turns out this was already a [42:04] NumPy array. Uh NumPy's uniform random [42:07] number generator already creates NumPy [42:09] arrays. So what it was doing was uh [42:11] allocating an a an array to copy it [42:14] into, copying the old array, and then [42:16] deallocating that array. So, that's the [42:18] sawtooth bump. That that peak is when it [42:21] allocated new space to hold the copy in [42:25] addition to the the space that was [42:26] necessary for the original, and then it [42:28] deallocated it. [42:31] So, this turns out to be easy to fix as [42:32] well. [42:33] All we do is we get rid of the np.array [42:36] call. [42:37] So, now instead of saying np.array of [42:39] this thing, we just remove it. That is [42:41] in fact semantically identical. And you [42:43] can see this has a big impact. Suddenly, [42:46] the CPU percentage dropped a lot. [42:49] The memory usage continues to be [42:50] substantial. It's 85%, but it is over a [42:55] smaller denominator, and that sawtooth [42:57] pattern has gone away. You can see that [42:59] it went from 1 GB to 860 MB in total [43:03] memory usage. [43:05] And the runtime also dropped from 6.3 [43:08] seconds to 5.4 seconds. [43:11] So, [43:12] in sum, Scalene gives you these [43:15] benefits. It is precise. It has all this [43:18] information, but it's not just precise. [43:20] If you have a profiler that's really [43:21] precise, but it's dead slow, nobody will [43:24] want to use it, right? Your whole thing [43:26] is you've got a program that's too slow, [43:28] and so if you take a profiler and it [43:29] slows it down even further, that's just [43:32] going to be painful. [43:34] So, Scalene also is fast. This graph [43:37] shows a bunch of previous profilers, and [43:39] on the far right you can see Scalene in [43:42] its CPU only mode or its full mode, [43:44] which does the memory profiling and copy [43:46] volume profiling and everything. [43:48] And you can see that Scalene imposes [43:50] pretty limited overhead. So, 1.0 would [43:53] be perfect. So, 1.1, 1.3 means 10% or [43:57] 30%. [43:58] Some of these existing profilers, [44:00] including profile, which is a standard [44:03] profiler that ships with Python, slows [44:05] down program by 16x, [44:07] uh and others are like almost 10x or [44:10] 35x, which is crazy. It's important to [44:12] note that these things only indicate um [44:16] percentage of time spent um in uh [44:20] like in a particular uh line of code at [44:22] best. [44:24] They don't break it down by which ones [44:25] are Python, which ones are C. And most [44:27] importantly, they don't have any of the [44:29] other information about memory. So, [44:31] there's only one other memory profiler [44:33] that we were aware of, um and it slows [44:36] things down by a factor of 152x. [44:39] So, on this one program that took 5 [44:42] seconds to run, uh running it with a [44:44] profiler took 12 minutes and 38 seconds, [44:47] uh which is crazy. And we've seen um [44:49] we've seen cases where the memory [44:50] profiler uh slows things down by 1,000x. [44:53] So, Scaling is a huge win over these the [44:57] over the CPU profilers in terms of their [44:58] precision and memory profilers simply [45:00] because they're incredibly slow. [45:03] So, you might be asking yourself, "How [45:05] on earth do we do this?" And basically, [45:07] the theme of Scaling is to find uh weird [45:11] like aspects of these uh irrational [45:13] exuberant languages um and leverage [45:16] them. Um so, what we do in all of these [45:19] cases is find things that they're doing [45:21] that seem to be limitations and then [45:24] exploit them or work around them in some [45:26] way that allows us to get more [45:28] information out than seemed to have been [45:30] possible. So, I'm just going to talk [45:32] about one of these today. [45:34] Um it's quite easy to understand and [45:35] it's it's nice and clean. So, most uh of [45:39] these scripting languages, um in fact, [45:41] all of all of them now, um have this [45:44] phenomenon that's called deferred signal [45:46] delivery. So, like some previous [45:49] profilers, Scaling is a sampling-based [45:51] profiler, which means it periodically [45:53] interrupts the Python program and checks [45:56] where it is at that moment, and it does [45:58] this over and over and over again. This [46:00] allows it to operate with low overhead [46:02] while getting pretty precise [46:03] information. [46:04] So, this is fine if you have code that's [46:06] running exclusively in Python. So, if [46:08] you send a signal every 0.1 seconds, for [46:11] example, it will get delivered roughly [46:13] every 0.1 seconds as you would hope. [46:16] However, [46:18] um Python itself actually doesn't just [46:21] deliver the signals that come from uh [46:24] your your timer, um it delays them until [46:28] Python regains control. So, if you [46:30] execute some code that actually runs in [46:32] C++ or C, like NumPy, uh none of those [46:36] signals get delivered until that code [46:38] finishes executing. [46:40] So, this is unfortunate because from the [46:42] perspective of a profiler, nothing has [46:44] happened. It's basically there there'll [46:47] be no information gathered, and it could [46:48] be the bulk of the time that your [46:50] program is actually uh executing, and [46:52] it'll get no information, and worse, [46:55] it'll mislead you. So, you could have a [46:56] program, and the program says, "Oh, I'm [46:58] spending all my time in this stuff," [47:00] where secretly you're actually spending [47:02] your time in uh native code that's not [47:05] even going to show up in a report. So, [47:06] this is This is really bad. This seems [47:09] like a huge problem, but we kind of [47:10] turned lemons into lemonade here. [47:13] And so, here is what we do. [47:15] Every single time we get a signal, we [47:18] record the current time. That is, what [47:20] is the time on the wall? Um and when we [47:23] come back and we get a signal again, we [47:25] compare [47:26] uh and see how much time is elapsed. [47:28] Now, if we're not interrupted, then it [47:30] would be whatever the delay was, the [47:32] actual interval of 0.1 seconds, for [47:34] example. Um however, there could be a [47:36] delay, and if there is a delay, we know [47:40] that that delay is basically because uh [47:43] it was executing native code. So, then [47:46] we can account for it. So, we keep these [47:47] separate counters. Uh we have a Python [47:49] time counter and a C time counter. Uh [47:52] the Python time counter we only [47:53] increment by the actual sampling [47:55] interval, and we attribute the rest of [47:57] the delay to the C time. [48:01] Likewise, we can actually leverage the [48:03] fact that we can measure this thing [48:04] called virtual time, which is the amount [48:06] of time that we have in fact spent [48:08] running on the CPU, and discover how [48:10] much time the process spent in system [48:12] time. That is to say, doing IO. [48:15] Um or sleeping for some other reason. [48:18] So, uh this [48:20] by basically, uh you know, we have this [48:22] deferred signal delivery, which seems to [48:24] be a problem. By leveraging it using [48:26] these other timers, we're actually able [48:28] to recover a lot of information. [48:30] So, it turns out this is quite accurate. [48:32] Um [48:32] here is just just to show you that it's [48:34] accurate. Uh we've got a little program [48:37] that we've designed to run for a certain [48:39] amount of time. We actually measured how [48:41] much time the C takes, and we have uh a [48:43] sleep, which puts the thing to sleep for [48:46] some period of time. Um oh, actually, in [48:48] this particular case, I'm misspeaking. [48:50] We have C code, which actually invokes a [48:52] C method. Um so, it's not about [48:53] sleeping, it's about native code. Uh and [48:55] then Python code, where it's running in [48:57] the interpreter. [48:58] And so, you can see that um [49:01] in principle, the Python code, which is [49:03] shown here, should take 30% of the time, [49:06] and the native code should take 70% of [49:08] the time. And in fact, it comes up with [49:10] nearly exactly 70/30. The error is about [49:14] 1%. So, so Scalene does a really good [49:16] job using these techniques. [49:19] Uh we hope to extend Scalene to some [49:21] other irrational exuberant languages. Um [49:24] basically, all of these languages have [49:27] features that are in common, and by [49:30] using these features, we should be able [49:32] to actually um [49:33] get the same kind of results that we get [49:35] for Scalene, uh and apply them to these [49:37] languages without too much difficulty. [49:40] Um we're currently looking at doing this [49:41] for PHP, uh and colleagues of ours at [49:44] Northeastern are looking into applying [49:46] it for the R programming language. [49:50] So, in to conclude, Scaling is this [49:53] profiler that's designed for handling [49:55] these these non-metal languages and [49:58] making it easier for you to optimize not [50:00] only the Python parts, but really steer [50:03] you to identifying the parts that are [50:06] that are in Python and should be turned [50:08] into native code by splitting out the [50:11] native time and the native memory from [50:13] the results. [50:14] Very precisely indicating exactly where [50:17] memory is being consumed and in [50:19] particular where copying is happening, [50:21] which indicates a problem of moving [50:23] stuff across these boundaries. It's [50:25] fast. It's easy to install and use. [50:29] If you use pip on Python, you can just [50:32] say pip install {dash} big U Scaling [50:35] and that will do it. So, in sum, Scaling [50:39] should help programmers out there who [50:42] are stuck using Python. [50:44] Don't get me wrong, I like Python. [50:46] It's a fine language as long as it's not [50:47] choking you. Making your Python programs [50:49] run faster. And if you're using metal [50:51] languages [50:53] and modern applications, then you should [50:55] use Cause. Also quite easy to install [50:57] and here are the links to the various [51:00] GitHub repositories. [51:01] And with that, I will conclude and I'm [51:03] happy to take questions. [51:13] So, thanks very much for that [51:14] enlightening talk, Emery. And I think [51:17] we've got a ton of questions that have [51:18] come in over the the duration of your [51:20] talk. I'm going to try to sort of hit [51:22] some of the high-level ones so that we [51:24] can kind of get a broad picture of what [51:27] people are thinking about as it relates [51:29] to your talk. I'm going to start with [51:30] something that kind of goes back to one [51:32] of the first set of slides that you [51:33] talked about, this distinction between [51:34] metal languages and irrational [51:36] exuberance languages. [51:38] And I think the the impetus for this [51:40] question is that people came up with [51:41] these irrational exuberance languages [51:42] for a reason, is that they're much [51:43] easier to program in than in languages. [51:45] Yeah. So, how do you balance programmer [51:47] productivity that you get from languages [51:49] like Python and Ruby and stuff like that [51:50] with the performance that you kind of [51:52] have to go to C or C++ to get? [51:55] Yeah, so I mean that's a great [51:56] observation. I mean, I think that the, [51:58] you know, um, [52:01] the irrational exuberance part was that [52:03] performance no longer mattered. Um, I [52:05] used to hear that articulated all the [52:06] time. Um, computers are just getting [52:08] faster and faster. It's not a problem. [52:10] Don't spend all your time on this. We [52:12] can focus exclusively on productivity. [52:14] And I think these languages are great [52:15] for productivity, um, but it they can [52:18] also be really, really slow. And if you [52:21] spend any time using Python, [52:23] that's the one of those languages I [52:25] spent the most time with, um, [52:27] you know, you quickly run into [52:28] performance issues. Sometimes it's [52:29] great, and then when it's not great, [52:31] it's terrible. Um, so I don't think that [52:34] there's [52:35] because Python has all these great [52:36] libraries that are written in native [52:38] code, it's not necessarily, um, a huge [52:41] tension. You can write your program in [52:43] Python, and if you use something like [52:45] Scalene, you find your bottleneck. Uh, [52:47] if your bottleneck turns out to be [52:48] something for which there's an existing [52:50] C library, perfect. Uh, it told you [52:52] essentially what to do. Ideally, we [52:54] actually would like to extend this [52:56] research to essentially tell you, "Here [52:58] is the library you should be using." [53:01] Right? Like we have identified certain [53:02] characteristics of your code. Those [53:05] happen to be satisfied by some kind of [53:06] library. Here is the library [53:08] recommendation that you should use. I [53:10] mean, in an ideal world, we'd also just [53:11] automatically apply the library. Uh, but [53:13] that might be a bridge too far. [53:19] Yeah, that that's that would be an [53:20] amazing uh, tool to to have access to as [53:23] a programmer who works in these [53:24] irrational exuberance languages. [53:26] Um, we had several questions that came [53:27] in that I think were kind of related to [53:29] the same general [53:32] issue, which is, um, [53:34] uncertainty in measurement. Uh, so both [53:36] in terms of when I'm doing this kind of [53:37] profiling, do you worry about the [53:39] profiler perturbing the results that you [53:41] get? but also just more generally the [53:44] the programs that you're profiling might [53:46] have a lot of uncertainty or or [53:48] variability or non-determinism that you [53:50] have to deal with. [53:51] But how do you manage that when you're [53:53] when you're using these kind of tools? [53:54] right. That's a great question. So so [53:56] first, both Cause and Scalene are built [53:59] on sampling. [54:01] So sampling really means that you have [54:03] very little impact from the profiler [54:05] itself. So the profiler is only waking [54:07] up very periodically and doing some sort [54:11] of thing. [54:12] With Cause, we were very careful to make [54:14] sure that whatever it did was not [54:16] biased. That is to say it didn't have an [54:18] effect one way or the other on the [54:20] reported results. [54:22] With Scalene, we couldn't play the same [54:24] tricks cuz you're doing stuff at the [54:26] Python level, but again it's sampling [54:27] based. So it should wash out. [54:30] Variability, [54:32] there's definitely inherent variability [54:34] and noise in lots of programs and not [54:36] just in the systems that they they [54:38] actually execute on. [54:41] And and this is something that [54:43] complicates reasoning about performance [54:45] period. I mean it's important to [54:47] actually understand how much variance [54:49] you have. [54:50] If you get a result where it's like, [54:52] well this thing is spending 3% of its [54:54] time and every time you run your program [54:57] it's plus or minus 3%, then you should [54:59] consider that to not be within kind of a [55:01] confidence interval. [55:02] So [55:03] interestingly, we actually have this [55:05] information for Scalene. I just haven't [55:08] come up with a way of displaying it. So [55:10] we have confidence intervals that we [55:12] build internally and we just don't show [55:14] anything where the confidence is too [55:16] low. [55:17] Visualizing that is tricky. There's [55:19] already a lot of information coming up. [55:22] So the the way we've chosen to do it is [55:24] just to not report things where the [55:25] confidence is low. That this is not just [55:28] due to the kind of variance you would [55:29] get. And I think that's probably a [55:31] reasonable way forward on these things [55:33] is just to have profilers that decline [55:35] to report things as being actionable if [55:38] they're likely to be noise. [55:44] Makes sense. [55:46] Um so we have a a couple of questions [55:47] that I I think tie together uh two [55:49] things uh that might actually also give [55:51] you an opportunity to talk about some of [55:52] your other uh research, which is [55:54] uh you know, you mentioned uh the [55:56] finding of Mitrovic et al. that that [55:57] like really small differences can that [55:59] that are can kind of swamp the effects [56:01] of of even 03 um [56:04] and you're using tools like scaling and [56:06] cause to make performance improvements [56:08] to code or to make changes to code that [56:10] you hope will lead to performance [56:10] improvements. Uh how do you know that [56:12] you're not just hitting the same kinds [56:14] of weird hardware [56:16] uh effects when you, for example, change [56:18] the number of threads in cause? [56:21] That's a great question. So, um so cause [56:24] in particular actually uh turns out to [56:27] uh [56:28] by design have an effect where it's sort [56:30] of immune to this problem. Um [56:33] so the reason is is that if you have a [56:34] multi-threaded program and you're [56:36] randomly interrupting it, um so you're [56:38] randomly picking things that are [56:39] happening and you're randomly doing [56:41] different sizes and different [56:42] everything, everything is unbiased uh [56:44] and you're in effect getting um [56:46] different executions every time because [56:48] of the concurrency. Um so you're [56:51] you know, intuitively you're mixing [56:53] things up constantly. Um so the you [56:57] know, the the whatever code is running [56:58] on the processor or whatever memory is [57:00] currently being accessed, those things [57:02] are changing all the time. So that [57:03] actually has the effect of giving you [57:05] kind of a random, what should I call it? [57:07] It's sort of like an electron cloud in a [57:09] sense of the distribution of everything. [57:12] And so when you get points over time, it [57:15] kind of factors out all of those [57:16] effects. So it's a great question and [57:18] it's uh it's spot-on. So we had a paper [57:22] where we also built the system and I [57:23] didn't talk about it today for lack of [57:25] time, um but the results that we have [57:27] for -02 and -03 [57:30] I I mentioned, you have gone, you know, [57:32] might not have noticed, that I factored [57:34] out the effects of memory layout. So, we [57:37] built this tool called Stabilizer, which [57:38] does this [57:40] by actually randomizing everything at [57:42] run time. So, in a weird way, so [57:45] Stabilizer does it precisely, it does it [57:46] for sequential program, it has a random [57:49] placement of functions, random placement [57:50] of the heap, random placement of [57:52] globals, random placement of the stack, [57:54] and it does it over and over again. [57:56] Um, [57:57] the what happens with Cause is that you [57:59] kind of get the benefit of that just by [58:01] virtue of the fact that you're running [58:02] concurrent code. [58:04] Um, so the concurrent code is causing [58:06] this kind of disruption, whereas [58:07] Stabilizer is going to work for [58:08] sequential program as well. [58:13] Hope that answers the question. [58:14] Thanks, yeah, that that's a great [58:15] answer. Yeah, no, I think I think it's, [58:16] um, you know, the the the the concern is [58:19] a real one, and I think you've you've [58:20] given a good reason why we we might be [58:22] able to, uh, sort of eliminate it. Um, [58:26] there's a few questions that are kind of [58:27] related to, uh, something that I thought [58:28] was interesting, it's something I hadn't [58:29] even thought about until these questions [58:31] came in, uh, about the interaction [58:32] between Cause and existing compiler [58:34] optimizations. Um, [58:36] are there kind of weird interactions [58:37] that happen? Like, if you turn on [58:38] compiler optimizations, will Cause [58:40] either give you [58:41] an incorrect diagnosis of where an issue [58:43] might be coming from, or does it sort of [58:45] affect what looks like the the the [58:47] important parts of code? Uh, and then [58:49] somebody asked a slightly different [58:50] question, which is, could you use [58:52] something like Cause to help target [58:54] compiler optimizations? [58:56] So, let's see. So, I mean, Cause is [58:59] really about identifying, you know, [59:01] you've got a program, and you want to [59:02] know what parts of it you should speed [59:04] up, right? Uh, it [59:06] If you are going to apply Cause, you [59:07] should definitely apply Cause with the [59:09] optimized code that you'd be running in [59:10] production. Um, it doesn't make sense [59:13] to, uh, to run Cause with unoptimized [59:15] code. That It it just, um, [59:18] in a sense, it's like you're performing [59:20] an experiment on, uh, on a on a pig, and [59:24] you really want to see if the [59:25] experiment, uh, applies to trees. [59:27] It's just like don't do that, right? [59:29] Like you really want to run the [59:30] experiment on the thing that you [59:32] actually care about. Um so, [59:35] with respect to compiler optimizations, [59:38] I mean, [59:39] I don't think So, there's the layout [59:41] stuff and compilers actually do take [59:43] into account layout and try to do layout [59:45] optimizations [59:47] um with mixed effects. Um and like I [59:49] mentioned, these things are super [59:50] brittle. So, just the mere fact that, [59:53] you know, uh you come in and your [59:54] username your username is Homer and [59:57] somebody else's username is Arun, it [59:59] differs by one character, uh that one [60:01] character is going to shift the stack [60:02] and it can have a gigantic impact, [60:04] right? So, I feel like that the effort [60:06] that's spent on layout optimizations, um [60:08] people should be careful because unless [60:10] you're running the same code all the [60:11] time with very similar kind of size [60:14] data, um you'll get very different [60:16] effects. Um so, yeah, so I'm not sure [60:18] that cause has anything to say in [60:19] particular about um where compiler [60:21] optimizers should work unless it's one [60:24] of those situations like, "Boy, if I [60:26] spend an hour in the compiler, like [60:29] imagine. Imagine the compiler could just [60:31] go to town for an hour or a day [60:33] optimizing some code and they just don't [60:35] do it cuz nobody wants their compiler to [60:37] be so slow." If you were willing to say, [60:40] "Look, this code really matters to me. [60:42] I'll be gone this weekend. You go [60:44] optimize as much as you can and I'll [60:46] come back." Like that actually would be [60:48] fine. Like I think many people would be [60:50] happy with an optimizer where you could [60:52] be like, "I know you normally you're [60:53] trying to be nice and friendly to me, [60:55] but you go crazy and spend, you know, a [60:58] a day, two days, a week running." [61:01] Um [61:01] that that would be interesting. [61:07] Thanks. Um so, I think we have time for [61:08] one last quick question and I'm choosing [61:09] this one because I think it'll give you [61:11] an opportunity to talk about yet another [61:12] one of your cool pieces of work that's [61:13] related. Uh somebody heard your [61:15] discussion of Stabilizer and [61:17] asked, "Does the randomized placement of [61:19] things you've mentioned have any [61:20] implications for security? [61:23] Right, yes. Uh So, yes. [61:27] So, actually [61:28] when we when we wrote Stabilizer, we [61:31] were trying to decide whether we should [61:33] pitch it as a [61:34] performance thing, which is what it was [61:35] intended for, or a security thing, or [61:38] both. [61:39] And we decided that for cleanliness of [61:42] discussion, we would just focus on the [61:44] one thing. So, we kept it as [61:45] performance. But actually, [61:47] re-randomization and so on, [61:50] I mean, the amount of randomization [61:51] Stabilizer produces is really, really [61:53] high. And it's one of the first it's [61:56] definitely the first system out there to [61:58] randomize things [61:59] at runtime. [62:01] And so many of these things at runtime. [62:03] So, there's a whole line of these things [62:04] that are called moving target defenses. [62:07] And so, moving target defenses are [62:08] exactly intended to thwart certain kinds [62:10] of attacks. [62:12] The Die Harder paper that that Milan [62:15] mentioned at the beginning is about a [62:16] randomizing memory allocator [62:18] specifically designed for security. So, [62:21] we we have random allocation of things [62:23] in space. [62:25] There's guard pages. It's very you can't [62:27] predict [62:28] any more than you can predict a random [62:29] number generator [62:31] it's output. You can't predict what [62:33] addresses things will be at. And so, the [62:34] likelihood of you of an attacker being [62:37] able to safely exploit something where [62:40] they have addresses or known offsets [62:42] basically goes to zero. [62:45] So, yes, it definitely [62:47] lots of security implications for [62:48] randomization. [62:52] Thanks for that answer, Emery. Thanks [62:53] for all of your answers and for for [62:54] talking about your your amazing work and [62:56] and a lot of the other pieces of work [62:57] that you've done. But I'm afraid we've [62:59] run out of time for today. And I'd like [63:01] to thank Emery again for his informative [63:03] presentation and his answers to all of [63:04] the questions. And I'd like to also [63:06] especially thank all of you for both [63:08] asking these great questions and for [63:10] attending the talk and participating. [63:14] So, one set of questions that we got [63:16] that I didn't really get a chance to [63:17] answer is that yes, this talk was [63:19] recorded and will be available online in [63:21] a few days at learning.acm.org. [63:23] And you can find announcements about [63:25] upcoming talks and other ACM activities [63:27] at both learning.acm.org [63:29] and at acm.org. [63:32] Please do remember to fill out our quick [63:35] survey where you can suggest future [63:36] topics or speakers and you should be [63:38] seeing that on your screen shortly. [63:40] On behalf of the ACM, Emery Berger and [63:43] myself, Milan Milenkovic, thanks again [63:44] for joining us and I hope you'll join us [63:46] again in the future. And with that, [63:48] we'll conclude.