Y Combinator Researchers Scale Self-Play and Real-Time RAG to Boost AI Efficiency

Y Combinator研究人员扩展自我博弈与实时RAG以提升AI效率 活动表明，应用自我引导强化学习可使70亿参数模型的性能媲美6700亿参数模型，同时大幅降低推理延迟。 该视频涵盖了探讨AI研究前沿与实际应用的五场核心演讲。讲者们探讨了将数据扩展至数十亿样本如何使ESM-Cambrian模型在无需多序列比对的情况下映射蛋白质生物学，以及自我引导式自我博弈如何防止语言模型在强化学习期间遭遇性能瓶颈。此外，本次活动还展示了多项突破，包括利用StreamRAG降低语音AI延迟、在Lean中原生形式化神经网络验证，以及通过并行化智能体工作流将软件开发重新构想为高APM的即时战略游戏。 Y Combinator Paper Club是一个面向研究人员、工程师和创始人的双周聚会，旨在探讨应用AI研究方向，重点关注记忆与测试时计算解决方案。 (0:00) 当前的AI训练受限于非单调的上下文学习表现以及低秩适应（LoRA）中的瓶颈期，这与人类平滑的学习过程形成鲜明对比，并引发了对SPSA等反向传播替代方案的关注。 (0:00) ESM-Cambrian模型家族将BERT风格的掩码语言建模应用于蛋白质生物学，利用对数线性神经缩放定律直接从氨基酸序列中实现无监督的3D结构理解。 (5:47) 通过将训练数据扩展至28亿个宏基因组样本，ESM-Cambrian模型绕过了导致ESM2等前代模型陷入平台期的生物数据墙。 (14:08) ESM Fold 2消除了AlphaFold所需的耗时且计算成本高昂的多序列比对（MSA）步骤，直接处理单条原始序列，在复杂目标上的得分与AlphaFold 3相差3分以内，并在抗体设计方面表现更优。 (15:33)

Presentations at a Y Combinator AI research meetup demonstrate that modern advances are driven by scaling raw data, structured self-play, formal mathematical verification, streaming retrieval, and high-throughput agent workflows rather than hand-engineered human heuristics. Unsupervised Scaling and Structural Discovery in Biology (14:08) Presenting research on the ESM Cambrian (ESMC) protein language model, Yas Beg argues that Richard Sutton's 'bitter lesson' applies directly to biology. Earlier protein models like ESM2 plateaued at roughly 50 million sequences, but ESMC scaled training to 2.8 billion metagenomic samples pulled directly from environmental DNA. By relying on a simple masked language modeling objective across raw amino acid sequences, the system learns complex 3D protein topology without hand-crafted structural representations. ESMC uses a looped architecture to iteratively refine structural predictions at test time, unlocking inference-time scaling similar to diffusion steps. On protein-protein interaction benchmarks, the alignment-free ESM Fold 2 scored within three points of AlphaFold 3, and surpassed AlphaFold 3 (50 versus 47) on antibody-target design tasks where evolutionary sequence alignments are sparse. Mechanistic interpretability further showed that internal representations spontaneously organize into functional biological features, successfully identifying invariant catalytic domains like the nucleophilic elbow across unrelated proteins. Self-Guided Self-Play to Prevent Reasoning Plateaus (25:06) Post-training compute allocations for reinforcement learning are increasingly rivaling initial pre-training budgets, yet manually assembling verifiable coding and mathematics environments creates a severe data bottleneck. Stanford researcher Luke explains that while asymmetric self-play—splitting a model into a conjecturer that designs tasks and a solver that attempts them—theoretically enables unbounded learning, standard language model self-play rapidly plateaus. On a dataset of 3,000 formal Lean 4 math problems, vanilla self-play stalled near a 60% solve rate, matching standard RL baselines. This stagnation occurs because the conjecturer exploits reward signals by generating artificially convoluted, inelegant statements to keep solver success around 50%. To fix this loophole, Luke and his co-authors introduced Self-Guided Self-Play (SGS). SGS anchors task generation in unsolved target problems and uses a third 'guide' evaluator to score whether newly generated problems remain conceptually relevant and free of spurious complexity. Using SGS and eight times the compute, a 7-billion-parameter model matched the baseline problem-solving capability of an unaugmented 670-billion-parameter model. Dynamic Streaming RAG for Real-Time Voice Agents (43:49) Giga researcher Arnab Matei highlights research from Meta aimed at reducing speech agent latency. Spoken conversational agents make hallucinations harder for users to detect than written text, making Retrieval-Augmented Generation (RAG) critical. However, waiting for a user to finish a full spoken query before triggering retrieval introduces multi-second delays that interrupt conversational cadence. Rather than executing full retrieval passes over fixed audio blocks, the highlighted research fine-tunes models to assess partial, streaming user utterances dynamically. The system predicts intermediate queries, checks whether newly arriving audio introduces actionable semantics, and initiates retrieval before the speaker finishes. In benchmarking, this predictive streaming RAG reduced interaction latency by 0.5 seconds on synthetic voice data and nearly 1.5 seconds on human-spoken speech while maintaining answer accuracy. Provable Correctness and Neural Verification via Lean (56:00) Caltech PhD student Robert George outlines the rise of interactive theorem proving using Lean, emphasizing a transition from informal mathematical reasoning to machine-verified intelligence. Lean enforces strict, step-by-step tactical verification that eliminates logical gaps or unverified assumptions, powering recent automated solutions to Erdős conjecture problems and Putnam competition benchmarks. Extending verification to machine learning engineering, George introduced torch-lean, a native framework that compiles neural network specifications and PyTorch-style tensors into verifiable representations. Using torch-lean, developers can formally prove architectural properties, such as showing FlashAttention is mathematically equivalent to standard attention and proving permutation invariance when positional encodings are omitted. George also formalized GPU CUDA kernels to prove that low-level floating-point non-determinism during batch inference causes subtle variations that flip the final argmax calculation even at zero temperature. Maximizing Parallel Agent Throughput in Engineering (60:39) Channel AI CEO Luke Orthwine argues that working with autonomous coding agents requires flipping conventional software engineering practices. Instead of treating programming as single-threaded, linear design akin to chess, developers should operate like competitive real-time strategy (RTS) players managing parallel production lines. By deploying multiple isolated Git work trees, developers can run dozens of autonomous agents simultaneously across cloud instances. Orthwine emphasizes treating token consumption as cheap while minimizing human interruptions: agents are instructed to proceed autonomously until generating a completed pull request rather than pausing for guidance. To manage this concurrency, Channel AI tracks 'Agent Actions Per Minute' (tool calls per minute) and employs distinct video game audio cues to alert engineers when specific worker sessions require input. Orthwine reports that moving from ad hoc LLM assistance to this parallel, agentic workflow increased monthly pull requests per engineer by an additional 60%. Across domains ranging from structural biology and formal mathematics to voice latency and parallel coding, AI research is converging on scalable, automated verification systems that eliminate human bottlenecks, though researchers caution that biological data walls, post-training sample efficiency cliffs, and synthetic self-play degradation remain open hurdles.

Y Combinator 举办的技术研讨会展示了人工智能在蛋白质生物学、自我博弈强化学习、实时语音检索、形式化数学验证以及高并发智能体软件工程中的前沿探索。多位研究者指出，AI 正从依赖人工先验特征与单线程开发模式，加速转向依赖通用计算扩展、严格形式化约束与高通量自主学习系统的全新范式。 蛋白质生物学扩展与纯序列建模 (14:08) 斯坦福大学博士生 Yas Beg 介绍了 Chan Zuckerberg Biohub 团队关于蛋白质模型 ESM-Cambrian（ESMC）的研究，探讨理查德·萨顿的“苦涩教训”（The Bitter Lesson）在生物学领域的适用性。传统蛋白质结构预测（如 AlphaFold）高度依赖人工构建的多序列比对（MSA），但构建 MSA 耗时巨大且在抗体设计等缺乏同源序列的场景中难以生效。研究团队放弃了结构先验与 MSA，转向对 28 亿个来自土壤、海洋及肠道环境 DNA 的宏基因组样本进行纯掩码语言建模，突破了此前 ESM2 在 5000 万样本下遇到的参数扩展瓶颈。 在下游验证中，采用单序列输入的 ESM Fold 2 引入循环模型结构（looped model）以扩展测试期推理计算，其在常规蛋白质复合物预测上逼近需要人工特征的 AlphaFold 3（相差约 3 个点），并在抗体设计基准测试中以 50 比 47 反超 AlphaFold 3。此外，利用稀疏编码技术分析其激活空间表明，该模型在无监督条件下自主形成了从单氨基酸、催化结构域（如亲核肘）到蛋白质整体功能的层级化生物概念表征。 强化学习自我博弈与引导式探索 (25:06) 斯坦福大学研究员 Luke 针对大语言模型后训练阶段算力占比剧增的趋势，探讨了非对称自我博弈（Asymmetric Self-Play）在自动化生成强化学习任务时的局限。理论上，由推想器（conjecturer）出题与求解器（solver）答题组成的博弈系统应能打破人工标注上限；但在针对 Lean 4 形式化数学数据集的实验中，常规自我博弈的表现迅速停滞在约 60% 的解决率，未能超越标准强化学习基线。Luke 指出，模型在单纯以“50% 求解率”为奖励驱动时，会倾向于利用漏洞生成冗长、晦涩且缺乏启发性的恶劣伪难题。 为破除这一“不优雅陷阱”，研究团队提出了自我引导自我博弈（Self-Guided Self-Play, SGS）算法。该方法强制推想器基于求解器失败的人类目标问题生成相关合成题，并引入第三角色“引导器”（guide）评估题目的相关性与非人为复杂度，将难度奖励与引导评分相乘。在 8 倍后训练算力支持下，装备 SGS 的 70 亿参数模型成功达到了其 6700 亿基准模型的解题性能水平。 实时语音交互的流式低延迟检索 (43:49) 来自 Giga 的研究员 Arnab Matei 分享了 Meta 关于流式检索增强生成（StreamRAG）的研究，直面实时语音交互中高延迟与幻觉不可容忍的工程瓶颈。与文本场景不同，语音用户难以在听觉中即时识别细微幻觉，而标准 RAG 需等待用户完整陈述后再进行多秒文档检索，彻底割裂了对话流畅度。 StreamRAG 探索了在用户发言过程中进行实时检索的方案。研究涵盖两种机制：一是固定时间间隔的流式检索，将音频分块并在接收时即刻执行局部查询与中间结果比对；二是通过微调大语言模型，使其根据未完全表述的内容动态判断是否需要触发检索，或复用已有上下文直接生成回复。实验表明，流式检索架构在合成数据上降低了 0.5 秒延迟，在真实人类语音数据上则缩短了近 1.5 秒的响应等待，同时保持了与最终全量检索相当的准确度。 Lean 形式化证明与神经网络验证 (47:23) 加州理工学院博士生 Robert George 探讨了形式化交互式定理证明器 Lean 4 在科学与人工智能领域的落地。随着重大数学难题（如 Erdős 猜想及国际数学奥林匹克竞赛题目）接连被各大顶尖实验室的形式化系统突破，软件与算法正经历从宽松工程向可严格验证代码（verifiable coding）的演变。 George 研发了统一框架 torch-lean，支持直接在 Lean 语言内部构建、训练并严格验证神经网络属性。该框架实现了类似 PyTorch 的张量系统与中间表示，不仅在规范层面形式化证明了 Flash Attention 与标准注意力的完全等价性以及缺少位置编码时的置换不变性，还进一步深入验证到底层 GPU CUDA 内核。针对零采样温度下大模型推理存在的非确定性问题，该验证从数学上推导并证实，浮点数运算中极微小的累积变化会导致批处理过程中的最终 argmax 索引发生跳变，进而引发生成行为漂移。 即时战略范式与激进代币编程 (58:05) Channel AI 创始人 Luke Orthwine 阐述了将传统软件开发转变为多智能体协同运行的高并发工作流。Orthwine 主张打破单线程人工编程的传统习惯，将开发者角色重新定义为类似即时战略（RTS）游戏玩家的指挥者：核心目标在于维持高频的有效操作（APM），动态调度系统注意力，并最大化智能体代币的消耗预算。 在具体工程实现上，该团队通过“线性工作树”（linear work trees）在云端实例上并行运行多个仓库环境，配置智能体自主执行开发、启动测试服务器并直接提交拉取请求（PR）。为压低人工检索开销，团队将代码语义沉淀为 Wiki 样式的结构化 Markdown 文档，并通过游戏单位音效实时提示智能体阻断事件。Orthwine 提倡践行“满意决策”（satisficing）准则，即允许智能体产出非最优解，依靠极低的纠错成本实现高通量吞吐。该方案使得团队人均月度 PR 交付量在原有大模型增幅基础上再次提升了 60%。 上述研究揭示了前沿 AI 正在走向两个相互交织的方向：一方面，在生物学和自主推理等领域，研究者正通过海量无监督扩展与博弈自学习摆脱对人工规则的依赖，但仍面临合成数据退化与分布偏移的阻碍；另一方面，形式化数学验证与高并发多智能体系统正在确立严格的确定性约束与高通量工程控制。如何平衡无监督探索的泛化能力与可验证系统的严谨性，依然是推动模型走向真实生产与基础科学的核心未决问题。

## Background 该内容由 Y Combinator 于 2026 年 6 月 12 日在 YouTube 平台上发布。视频源自 YC Paper Club 活动，简介指出这是一个每两周在山景城办公室举行的研究交流会，本期核心探讨蛋白质生物学中的规模定律、大模型的自我博弈、实时语音智能体的流式 RAG、Lean 形式化验证，以及如何像打即时战略游戏一样用智能体进行编程。 ## TL;DR 本次分享会聚焦于 AI 研究的最新前沿与应用落地。主讲人们共同探讨了规模定律在蛋白质语言模型（如 ESMC）中的完美映射与湿实验成果，利用自我引导机制（SGS）打破大模型自我博弈的性能停滞，以及通过 Stream RAG 攻克语音智能体交互中的高延迟痛点。同时，会议展现了 Lean 语言在代码和科学验证领域的破圈应用，并分享了将编程重构为多智能体并行调度、如同操作即时战略（RTS）游戏般的极客开发工作流。 ## Atmosphere & style 整场活动洋溢着硬核、前沿且极具极客浪漫主义的活跃氛围，充满了从容的幽默感与思维碰撞的火花。主持人 Francois 严谨的学术洞察与 Lukens 狂热、反传统教条的“Token 最大化”黑客风格形成鲜明而有趣的对比，台下观众互动热烈。听众能切身感受到技术在医疗、数学和软件工程底层带来颠覆性突破的时代兴奋感，并在最后吃着麻薯甜甜圈的轻松茶歇中，体会到一种充满活力的技术社区归属感。 ## Key takeaways - [00:08](https://www.youtube.com/watch?v=3rWSvrFahIY&t=8s) 测试时计算（Test-time compute）与递归自我提升虽然被部分学者寄予厚望，但仍面临在有限时界的马尔可夫决策过程（MDP）中难以有效穷尽未知解空间的底层限制。 - [00:08](https://www.youtube.com/watch?v=3rWSvrFahIY&t=8s) 当前 AI 领域面临两大底层核心挑战：‘单样本智能度’（持续学习中的每样本效率最大化）与‘单瓦特智能度’（能效比与规模的权衡）。 - [00:08](https://www.youtube.com/watch?v=3rWSvrFahIY&t=8s) 上下文学习（ICL）和 LoRA 微调在样本量增加时均会遇到性能瓶颈或非单调波动的‘畸形现状’，这与人类通过持续练习带来线性提升的认知模式截然不同。 - [05:47](https://www.youtube.com/watch?v=3rWSvrFahIY&t=347s) 神经网络的规模定律（Neural Scaling Laws）在蛋白质生物学中完全成立，ESMC 等大模型纯粹通过一维氨基酸序列的掩码预测即可在无监督状态下涌现出对三维空间折叠结构的精密认知。 - [23:26](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1406s) 借助宏基因组学带来的海量自然界进化数据，新一代蛋白质大模型打破了此前的缩放平台期，证明了‘苦涩的教训’（The Bitter Lesson）在生物学领域同样适用。 - [23:26](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1406s) 摒弃传统多序列比对（MSA）人工特征的单序列蛋白质大模型，在通用复合体及抗体设计任务中表现逼近或超越 AlphaFold 3，且具备显著的运行速度优势。 - [23:26](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1406s) 利用稀疏自编码器（SAE）等机械可解释性工具分析发现，纯文本预训练的蛋白质大模型其潜在空间能够自发涌现并解耦出对应真实生物学概念的层级化特征。 - [31:33](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1893s) 传统非对称自我博弈（Asymmetric Self-Play）中，出题者常通过制造冗长、混乱的人造复杂度来降低求解者成功率，导致虚假的高难度，从而使模型性能陷入停滞。 - [34:38](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2078s) 自我引导型自我博弈（SGS）通过锚定人类目标任务分布并引入‘引导者’角色惩罚恶意复杂度，成功使 7B 小模型在同等算力下追平 670B 模型的性能。 - [37:21](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2241s) 语音 AI 时代由于交互的强对话感以及人类对音频幻觉的敏感性，比文本 AI 更迫切地需要检索增强生成（RAG）技术的支持。 - [41:12](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2472s) Stream RAG 通过在用户说话期间对不完整音频文本块进行流式处理与提前检索，巧妙利用时间差对冲了传统 RAG 带来的致命延迟，且保持了极高准确率。 - [47:23](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2843s) 以 Lean 为代表的交互式定理证明器（ITP）结合大语言模型，正将科学与数学 AI 推向完全显式、拒绝模糊证明的‘可验证智能’新时代。 - [53:19](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3199s) 通过 TorchLean 等全栈形式化框架，研究者不仅能在线性规格层面证明 FlashAttention 等算法的等价性，还能深入到 GPU 细微算子级别精准定位硬件底层的非确定性 Bug。 - [58:05](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3485s) 在 AI Agent 辅助编程的背景下，软件工程正从深思熟虑、线性的‘国际象棋模式’转变为高强度、多线并行的‘即时战略游戏（RTS）模式’。 - [01:02:53](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3773s) Agent 时代的开发工作流应极度贬低智能体的劳动与 Token 成本，最大化榨取其并行试错能力，通过‘满意化’（追求足够好而非完美）原则来解放人类工程师的认知带宽。 - [01:07:54](https://www.youtube.com/watch?v=3rWSvrFahIY&t=4074s) 利用诸如 APM（每分钟工具调用数）追踪工具、游戏音效提示以及结构化知识库（Wiki）链接，可以大幅度提升多智能体并行的可视化与管理效率。 ## Detailed walkthrough 以下是根据视频第1部分（00:08 — 14:06）的录音文本整理的详细、忠实的中文全景实录与技术解析。 --- ### 开场致辞与当前AI研究的核心考量 会议由 Francois Chaubard 主持。他首先对到场观众表示了热烈欢迎，并指出根据之前的反馈，本次分享会的内容将大幅向“应用落地”倾斜。紧接着，他依次介绍了本期 Paper Club 的重磅主讲阵容与覆盖的五大前沿领域：首先是由他最喜爱的共同研究者 Yasa Baig 带来的“生物学 AI（AI for Biology）”；随后是来自 Tatsu 实验室的 Luke Bailey，他将探讨类似于 AlphaZero 风格的、针对大语言模型（LLM）的自我博弈（Self-play）机制；第三位是来自 Giga 的研究员 Arnab Maiti，他分享的 Stream RAG 是一项极具差异化的应用，专注于实时语音智能体（Real-time voice agents）的构建；第四位是 Robert George，他将深入探讨使用 Lean 进行科学形式化验证（Formal verification）的激动人心的进展；最后登场的是被戏称为“AI Token 最大化榨取者（AI token maxer）”的 Lukens Orthwein。 在抛砖引玉、呼吁更多人参与未来主题宣讲的同时，Francois 分享了自己近期高度关注的几个技术方向。他指出，在过去一年半的时间里，**内存/记忆机制（Memory）** 绝对是最炙手可热的话题，从 Mem0、递归语言模型（Recursive language models）、其所在实验室开发的 Cartridges，到 H-Net 以及动态分块（Dynamic chunking）等技术，各种新颖的思路层出不穷。 随后，Francois 谈到了他前几周与 Noam Brown 录制并发布的一期播客节目。Noam Brown 至今仍持有这样一种观点：人类生成的解空间子空间（记为 $H$）依然是关键；如果我们在 $H$ 上进行模型训练，那么在推理端，我们完全可以通过**测试时计算（Test-time compute）** 以及**递归自我提升（Recursive self-improvement）** 的方式跨越其边界，一路探索并涌现出超越人类已知范畴的全新解空间（即 $F - H$）。然而，Francois 坦言自己对这一路径的可行性“极其怀疑（really struggle with this）”。他补充道： > “这并不是说它完全不可能，而是说我们通过采样穷尽整个未知空间的可能性微乎其微。如果解空间的正解全集是 $F$，那么即便给予再多可行的测试时计算量或递归自我提升，仅在已知的人类解 $H$ 上进行训练，必然会把你限制在某个典型的人类解集内。在现实中，你根本无法有效地对 $F - H$ 的空间进行充分采样，更不用说穷尽它了。除非你有无穷无尽的递归提升和无限的计算资源，但遗憾的是我们没有。生命本身是一个部分可观测马尔可夫决策过程（POMDP），而我们面临的是一个有限时界的马尔可夫决策过程（Finite horizon MDP）。” 为了攻克这一瓶颈，Francois 认为当前 AI 领域仅剩两大底层核心挑战： 1. **单样本智能度（Intelligence per sample）**：即在持续学习（Continuous learning）中，每获取一个新样本 $N$，如何让模型在以该样本为条件时实现性能的最大化。 2. **单瓦特智能度（Intelligence per watt）**：即能效比与模型规模的权衡。其所在实验室的 Yvonika 和 John（有望在下期带来主题演讲）认为，从单瓦特智能度的视角来看，有时较小的模型反而能带来更好的回报。 在深入拆解“单样本智能度”时，Francois 分享了自己对当前行业通行的**上下文学习（ICL, In-Context Learning）** 与 **LoRA 局部微调** 的对比实验心得。他用 Magnus Carlsen（芒努斯·卡尔森）打比方：人类在遵循“一万小时定律”时，算法机制是不变的，我们下越多盘棋，棋力就会呈单调线性提升。但目前的模型并非如此： * **上下文学习（ICL）的局限**：在小样本阶段，ICL 往往是效果最好的最优解。但随着输入样本数量的增加，模型的性能并不会单调提升，而是开始像“拳击手闪躲（bob and weave）”一样起伏不定，时好时坏。最终，当它撞到模型预训练的上下文长度硬性窗口（Context length cliff）时，性能直接戛然而止，无法无限延伸。 * **LoRA 与全量微调（SFT）的断层**：在样本量极低时，采用较低秩（Rank）的 LoRA 微调表现出乎意料得好，但随着样本量增加，它也会很快触及瓶颈（Peter out）。直到样本量规模极大时，才能最终过渡到端到端全量监督微调（SFT Group O）。 * **现状的畸形**：如果每增加一个新样本（第 $N+1$ 个样本），为了获得那一点点性能微调，我们不得不反复用特定的 Rank 去重新训练一个 LoRA。随着数据流的持续输入，在不同的数据节点上竟然存在完全不同的“最优操作路径”，这与人类的认知模式大相径庭。因此，业界必然需要探寻一种拥有极高单样本智能度的全新学习范式（Learning procedure）。 此外，Francois 还表达了他对**反向传播算法（Backpropagation）替代方案** 的狂热兴趣。从大脑生物学的角度来看，目前几乎没有证据表明人脑在学习时会去计算权重矩阵的转置。他目前对同时扰动随机逼近算法（SPSA）非常看好，并呼吁如果有人知道其他替代反向传播的创新路径，请务必向他推荐。最后，他表达了对生物、机器人、语音等跨界突破以及“疯狂的创始人黑客创新（unhinged founder hacks）”的期待，并提出了通过举办闪电演讲、联合发布 AI 基准测试（AI benchmarks）以及共同开发开源项目等方式，来进一步提升 Paper Club 的影响力和趣味性。 --- ### Yasa Baig：蛋白质生物学的世界模型 演讲由 Stanford 博士二年级学生 Yasa Baig 接棒。他不仅与 Francois 在同一个实验室共事两年，同时还接受 Stanford 生物工程界泰斗、前 Chan Zuckerberg Biohub 负责人 Steve Quake 教授的共同指导。Yasa 幽默地评价 Francois 总能为团队带来极具创意且高度“分布外（out of distribution）”的奇思妙想，虽然有时大家完全猜不透这些灵感从何而来。 本次 Yasa 分享的论文源自加州湾区的 Biohub 团队刚刚于上周发布的研究成果。尽管台下听众大多是硬核的 AI/ML 背景，但 Yasa 明确表示：这并非一堂枯燥的生物课，其核心叙事是“**生物学视角的‘苦涩的教训’（The Bitter Lesson）**”。该研究在底层深度探讨了**规模（Scale）**——这一整个 AI 社区为了让系统变强而笃信的最基础、最原始的假设——在解决生物学尤其是**蛋白质生物学（Protein Biology）** 问题时，究竟是如何发挥作用的。 Yasa 首先帮大家复习了 Richard Sutton 那篇著名的文章《苦涩的教训》。其核心前提是：在过去 70 年的 AI 发展史中，最终胜出的绝非那些由人类手工设计、注入特定领域先验知识的专家系统，而是那些最通用、能完美杠杆化算力与数据规模的底层方法。正如 AlphaGo 和 AlphaGo Zero 一样，它们在早期由于极度缺乏人类经验，表现一度远逊于专家系统，但只要持续堆叠计算量，它们最终会无情地反超并实现指数级碾压。知识系统在短期内能拔得头筹，但从长远来看，更庞大、结构更通用的模型才是终极赢家。 当前生物学 AI 领域的最新豪赌正是：这一规律是否同样适用于蛋白质生物学？研究者们试图借鉴 OpenAI 在语言模型中所揭示的经典神经网络规模定律（Neural Scaling Laws），将其平移到药物设计、细胞机制理解等领域。 * **左侧的已知世界**：在自然语言大模型中，我们拥有非常丝滑、完美的对数线性（Log-linear）规模曲线，模型损失会随着计算量和数据量的提升展现出极强的可预测性下降。 * **右侧的未知豪赌**：对于更广泛的生物学现象，尤其是蛋白质，这种曲线是否同样存在？大语言模型（LLM）的成功配方，在面对生物学这种相对于自然语言而言极度“分布外（out of distribution）”的全新领域时，究竟是会无缝平移，还是会直接崩解？ 为了让全场程序员听懂，Yasa 用了极度精简的类比完成了必要的生物学概念铺垫。在人体内有三大类大分子：脂质、碳水化合物和蛋白质。 * **1D 序列**：蛋白质在本质上就是一条氨基酸链条。大自然中一共存在 20 种不同的氨基酸，它们排列组合组合成了近乎无限的分子可能。你可以将每个蛋白质完全视为由 **20 个字母组成的特殊字母表** 拼成的文本序列。 * **3D 结构与功能**：正是这段一维的字母序列，决定了蛋白质独特的 3D 空间三维折叠结构。而正是由于这些奇特多变的 3D 形状，蛋白质才能在细胞中各司其职，比如催化化学反应、阻挡病原体入侵等。 本次报告的核心模型是 **ESMC**。这是由最初在 Meta、后成立独立公司、现加入 Biohub 的 Evolutionary Scale 团队经过数年迭代推出的第四代最新力作。该模型的训练思路与大语言模型如出一辙：他们完全不向模型提供任何关于蛋白质三维空间几何、电荷或物理化学的先验人类知识，只是从自然界（包括人体、细菌、各种极端环境）中收集了数以亿计、历经数百年进化的蛋白质一维氨基酸序列。 模型采用类似于 BERT 的**掩码语言模型（Masked Language Model）** 训练机制：将这些由 20 种 Token 组成的字符串随机遮蔽掉一部分，然后让一个庞大的 Transformer 架构纯粹基于上下文的共现模式去预测被掩码位置的字母。正所谓自然语言处理（NLP）中的古老谚语——“字义由其上下文决定（You'll know a word by the company it keeps）”，在蛋白质世界里，研究者的核心假设是：**氨基酸的属性同样由其共现的氨基酸决定**。只要规模足够大，模型在被迫学习这套“蛋白质语法”的过程中，就能自然而然地“涌现”出人类真正关心的结构与功能属性。 为了系统性回答“苦涩的教训是否在生物学成立”，Yasa 梳理了一张贯穿后续演讲的“大语言模型 vs 蛋白质语言模型”映射表： | 自然语言概念 (LLM) | 蛋白质语言模型对应物 (Protein LM) | |:--- |:--- | | 文本 Token | 氨基酸 (Amino Acids) | | 整个互联网数据 | 全球进化序列数据库 (Evolutionary Sequence Databases) | | 自回归/掩码预测 | 掩码 Token 预测 (Masked Token Prediction) | | 涌现能力 (如推理) | 涌现的蛋白质三维结构与生物功能理解 | | 机械可解释性 (SAE 稀疏自编码器) | 蛋白质表征的力学可解释性分析 | --- ### 规模定律在蛋白质语境下的完美映射 为了验证模型是否真的随着规模上量而学到了东西，我们需要一个极为严苛的代理指标（Proxy）来探测其对蛋白质结构的理解。论文作者选择的评测指标是：**长距离蛋白质接触预测（Long-distance protein contact prediction）**。 其背后的逻辑非常硬核：蛋白质在输入端只有一维的线性序列，但它在三维空间中会进行极其复杂的卷曲折叠。如果一个模型真的从一维文本中悟出了更高级的三维空间几何，那么它必须有能力准确预测出那些在一维序列上相隔极远、但在最终三维空间折叠后却紧紧挨在一起的两个氨基酸节点（即长距离接触）。相比之下，序列上相邻的氨基酸在空间中挨着是显而易见的，没有含金量。作者将这种在完全无监督状态下、从模型内部表征直接线性读取长距离空间接触的准确率指标命名为 $P@L$（特定长度下的长接触精度）。这是一个不掺杂任何结构训练数据、纯粹依靠掩码语言任务逼出来的“净室（clean unsupervised）”结构知识度量。 Yasa 展示了 Biohub 最新构建的名为 **ESM Cranberry** 的模型家族在 **3亿（300M）、6亿（600M）和60亿（6B）** 三种参数量规模下的表现。令人振奋的是： * **惊人的外推性**：研究团队最初仅基于低算力预算（Low-end training runs）的实验数据拟合了一条“计算最优预测曲线（Predict compute optimality curve）”，当他们真正砸下巨资跑完大规模模型训练后，实际的性能散点几乎严丝合缝地落在了这条预测对数线性曲线上。 * **完美平移**：这铁证如山地回答了第一个疑问：**规模定律在蛋白质生物学中完全成立**。只要你持续追加算力、扩大模型参数、喂入更多进化数据，其定性趋势与 LLM 毫无二致。在没有任何结构先验、没有看过任何 3D 结构图纸的情况下，大模型纯粹靠看一维氨基酸的共现，就完美推导出了大自然精密的立体工程学。 不过，Yasa 在最后也抛出了一个引人深思的戏剧性转折（Interesting twist）…… * **核心核心事实列表**： * **蛋白质基础**：由 20 种不同的氨基酸 Token 组成，一维序列直接决定三维空间折叠形态和最终的细胞生物功能。 * **模型训练细节**：ESMC Cranberry 家族涵盖 300M、600M 和 6B 三种主流参数规模，采用纯文本掩码语言建模。 * **评估核心指标**：$P@L$（长距离蛋白质接触预测精度），用于在不暴露 3D 数据的前提下，无监督读取模型内部涌现的结构认知。 By the way, to unlock the full functionality of all Apps, enable [Gemini Apps Activity](https://myactivity.google.com/product/gemini). ### 蛋白质生物学的“苦涩教训”与数据缩放 在早期的研究中，上一代蛋白质大模型（如 ESM2）在增加参数量时遭遇了瓶颈，收益递减，缩放曲线陷入平台期。然而，新一代的 ESMC（ESM Cambrian）模型打破了这一魔咒。演讲者指出，团队并没有在架构中设计多么精妙的归纳偏置（inductive bias），虽然其中包含高超的工程实现，但核心驱动力纯粹是**数据缩放**。他们将原始 ESM2 的 5000 万个训练样本直接推到了 2.8 亿个，这主要得益于宏基因组学（metagenomic）数据的引入——这些氨基酸或蛋白质序列是从土壤、海洋和人类肠道等自然环境的 DNA 测序中直接获取的，它们来自人类从未培养过、甚至根本不了解的微生物。 演讲者幽默地将此比作生物学版的“语言模型数据墙”讨论。与依赖人类过去 30 年产生的数据不同，生物学中大自然和进化已经持续生成了 40 亿年的训练数据。事实上，人类目前在地球上采样的蛋白质序列多样性还不到已知总量的 1%，更不用说生命诞生以来演化出的所有序列了。这就引出了理查德·萨顿（Richard Sutton）著名的“苦涩的教训”（The Bitter Lesson）在生物学上的应用：一个纯粹基于掩码语言模型（MLM）目标训练的通用模型，能否战胜像 AlphaFold 那样通过人工精细调优、具有强归纳偏置的专家模型？ - **AlphaFold 的优势与局限**：AlphaFold 的强大源于人工构建的特征输入——多序列比对（MSA）。为了预测一个蛋白质的结构，它需要寻找数百个进化“近亲”并将它们堆叠排列，利用家族间的协变模式提取结构信息。这种领域工程非常精美，但也导致模型运行极慢，需要构建庞大的数据库，且在缺乏进化同源序列的场景（如抗体设计）中完全失效。 - **ESMC 的做法**：ESMC 彻底抛弃了 MSA，直接输入单条序列。模型提取出每个残基的嵌入表示（per-residue embeddings）后，直接喂给一个专门的结构预测模块，将其投影到三维坐标空间中，实现“相同输出，零人工特征”。 --- ### 递归模型与推理期计算的缩放 在架构设计上，ESM 团队在将表示转换为结构的投影网络中加入了一个非常有趣的特性。该特性基于实验室校友 Dan Fu 的研究，采用了一种**循环/递归模型（looped model）**。这种参数共享的算法设计近期在机器学习界备受关注，它为模型提供了一个在推理期（test-time）缩放计算量的杠杆。 通过这种设计，模型在预测结构时，可以让表示在一系列层中循环流转，从而在无需重新训练或微调的前提下，不断精细化（refine）其结构预测。这就像扩散模型中的步数，或者大语言模型在推理期的采样一样，为“测试期计算”开辟了空间。 - **性能表现（DOCQ 通过率）**：DOCQ 是衡量结构预测好坏的标准，代表两个紧密结合的蛋白质其预测形状与真实情况足够接近的测试用例比例。 - **与 AlphaFold 3 的对比**：在通用蛋白质复合体预测中，基于单序列且无 MSA 的新 ESM Fold 2 模型，其表现与依赖手工特征的 AlphaFold 3 仅差约 3 个百分点。 - **抗体设计大获全胜**：在对医药和生物技术至关重要的单克隆抗体（MAB）设计任务中，ESM Fold 2 以 50% 对 47% 的成绩击败了 AlphaFold 3。因为在巨大的抗体多样性面前，人类已知的序列变异相对较小。演讲者总结道：“这并不意味着 MSA 已经彻底消亡，而是说明手工特征只有在数据丰富时才有帮助，而当药物设计师真正需要它时，它往往无能为力。”此外，该模型不仅准确，而且由于省去了古典计算生物学中极耗时间的 MSA 构建过程，其运行速度要快得多。 --- ### 可解释性、蛋白质地图与人类福祉的现实意义 随后，演讲者展示了该模型在机械可解释性（mechanistic interpretability）上的惊人发现。团队借用了 Anthropic 等大模型社区常用的稀疏编码分析工具（sparse coding analysis），对模型的激活值进行解耦，试图寻找特征空间中单语义激活的方向。结果表明，完全通过“填空题”预训练的蛋白质大模型，其潜在空间自动分解成了清晰的、对应真实生物学概念的特征。 这些特征在没有受到任何监督的情况下，被大语言模型 Agent 自动标注并组织成了完美的层级结构：底层是单个氨基酸特征，向上依次是结构基序（structural motifs）、完整的蛋白质域（protein domains）、功能位点，顶层则是整个蛋白质的功能角色。 - **亲核肘基序（Nucleophilic Elbow）的例子**：这是一种在多种酶催化中多次独立进化、互不相关的蛋白质中反复出现的催化域。模型成功在四个演化距离极远、结构各异的蛋白质中精准识别出了这一基序，证明它理解了更深层次的生物学直觉，而非死记硬背相似序列。 - **超大蛋白质图谱（Google Maps for Proteins）**：团队利用该模型对数以亿计（乃至多达 70 亿）的蛋白质进行了结构预测和分析，构建了目前世界上最大的蛋白质结构数据库。在 Shape 空间中展开后，它呈现出清晰的家族映射，比如能一眼识别出对生物技术至关重要的 CRISPR-Cas9 酶。演讲者激动地惊叹：“这简直太疯狂了，进化关系和功能关系完全是模型作为副产品白嫖（for free）得来的！” - **湿实验验证与抗癌药物设计**：该模型被应用于逆向设计（inverse design），并在湿实验室（wet lab）中得到了验证。它在完全基于序列空间的层面上，设计出了能与特定治疗靶点结合的蛋白质药物。其中最成功的应用是设计出了 PD-L1 结合剂——这是现代免疫治疗的里程碑，帮助了无数传统手段无法对抗的癌症患者。 演讲者在最后向台下和 YouTube 屏幕前的机器学习与软件人才发出了热情的邀请：“生物学是机器学习的绝佳去处，因为模型还很年轻，而数据量每年都在以指数级速度爆炸，我们根本不受数据限制。即使你对数据缩放不感冒，也请来看看它对全人类带来的巨大福祉。” --- ### Luke Bailey：利用自我引导缩放增强大模型的强化学习自我博弈 在一阵掌声中，第二位演讲者 Luke Bailey 登台。主持人 François 介绍道，Luke 是来自 Tatsuya 和 Tatsunori 实验室的博士二年级学生，刚从英国来到哈佛大学攻读计算机科学，曾主攻对抗鲁棒性（adversarial robustness），现在专注于后训练阶段的自我博弈（post-training self-play），这正是 AlphaZero 精神的延续。 Luke 随后切入正题，分享了他们几个月前发表的论文《Scaling Selfplay with Selfguidance》（通过自我引导缩放自我博弈）。他首先梳理了当前顶尖大语言模型的标准训练管线： 1. **预训练（Pre-training）**：在海量的网页文本上进行粗放的学习。 2. **后训练（Post-training）**：近期业界的一个显著趋势是，人们在后训练阶段投入了极其恐怖的算力，用于进行大规模、长时间的强化学习（RL）。 在这些强化学习运行中，研究人员收集了海量的任务，涵盖编程、数学以及与各种软件交互的场景。大模型 Agent 在这些环境中尝试采取一系列行动，环境返回奖励，模型则通过提高优秀走子（rollouts）的权重、降低糟糕走子的权重来进行训练。Luke 指出，业界如今在 RL 后训练上投入的算力已经正在逼近、甚至超越预训练的算力。虽然其中还夹杂了一些指令微调（instruction tuning）和对齐（alignment），但绝大部分算力都被这些漫长的 RL 运行给吞噬了。当人们在后训练中增加 RL 任务的数量和算力时，下游性能会得到显著提升，正如 Cursor 在其 Composer 2 技术报告中展示的那张极其精美的图表所揭示的规律一样。 By the way, to unlock the full functionality of all Apps, enable [Gemini Apps Activity](https://myactivity.google.com/product/gemini). ### [27:22](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1642s) 强化学习任务与自我博弈（Self-Play）的局限性 在传统的强化学习（RL）任务中，模型通常在预先定义好的单一环境中进行训练。演讲者展示了一组研究数据，其 $x$ 轴代表训练步数（即投入的算力和新的强化学习任务），结果呈现出一条非常平滑且漂亮的上升曲线。这表明随着任务量和算力的同步增加，模型性能会带来可靠的提升。这种方法在下游基准测试（如实际的代码编写任务）中也表现出持续且显著的改进。 然而，这条看似完美的“配方”却潜藏着巨大的现实瓶颈： * **人工收集任务的成本高昂**：图表中的 $x$ 轴采用的是对数级尺度（Log Scale），这意味着若想维持模型性能的持续增长，所需喂入的任务数量将呈指数级爆发，完全依赖人工收集任务是难以为继的。 * **遭遇性能天花板**：我们最终的期望是让模型超越人类所能设计的全部问题。如果完全依赖预设的环境，一旦模型攻克了所有既有任务，或者在某些极难的任务中完全无法获得奖励，它的提升就会彻底停滞。 为了打破这一僵局，研究者引入了“自我博弈”（Self-Play）的概念。其核心问题在于：我们如何让模型自动、源源不断地生成全新的强化学习任务，在这些任务上进行训练，并不断循环往复？在自我博弈的框架下，模型扮演着双重角色：它既需要**生成任务**，又需要**尝试解决任务**，且训练的目标是同时提升这两方面的能力。 --- ### [28:57](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1737s) 对称与非对称自我博弈的演进 为了让大家更好地理解，演讲者回顾了自我博弈的发展历程，并将其划分为两种主要模式：**对称自我博弈**（Symmetric Self-Play）与**非对称自我博弈**（Asymmetric Self-Play）。 * **对称自我博弈**：最典型的例子就是著名的 AlphaGo 和 AlphaZero。在围棋的规则和棋盘这一固定环境里，智能体需要一个对手来相互切磋。于是，系统让“较老版本的自己”充当对手。在这种模式下，模型两次承担了完全相同的角色（都是围棋选手），因此被称为“对称”结构。 * **非对称自我博弈**：近年来在大型语言模型（LLM）领域，非对称自我博弈开始异军突起（尽管它最早可以追溯到更早的控制问题研究中）。在这种架构下，模型内部被赋予了两个不同的身份： 1. **推测者/出题者（Conjecturer）**：负责为求解者凭空创造出整个强化学习环境和任务。例如，它会构思出一个编程难题，并同步生成一堆用于验证的单元测试。 2. **求解者（Solver）**：相当于传统的智能体，在出题者构建好的环境中进行大量的采样和试错（Rollouts），通过获取奖励来训练自己。 从理论上讲，自我博弈的魅力在于其“学习能力无上限”的完美承诺。如果我们仅用人类的演示数据（Demonstrations）来训练模型，模型永远不可能超越这些示例的上限。而自我博弈通过不断催生新的学习信号，理论上可以实现无限自我进化，这在围棋等双人游戏中已经得到了证实——模型早已远远将人类最高水平甩在身后。大家对大语言模型的期许也是如此：先通过人类数据训练达到人类水平，然后开启疯狂的自我博弈，最终跨越人类巅峰去解决极其复杂的科学或数学难题。 > “在理论上，没有什么能限制它的学习。……但遗憾的是，现实并非如此。” 在实际操作中，如果让大语言模型进行长时间的自我博弈，其性能会像传统强化学习一样迅速陷入停滞（Plateau），所谓的“秘密武器”并没有在现实中完美兑现。 --- ### [31:33](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1893s) 虚假的高难度：传统自我博弈的“凑字数”陷阱 为了诊断导致性能停滞的病因，研究人员剖析了最基础的语言模型自我博弈算法： 1. **任务采样**：出题者（Conjecturer）和求解者（Solver）本质上是同一个模型，只是冠以不同名称。出题者首先采样并生成合成任务。 2. **尝试求解与验证**：求解者尝试解决这些任务，并通过某种奖励信号（如出题者同步写好的单元测试）来验证正确性。 3. **模型更新**：求解者基于所有正确的尝试进行更新；而**出题者的更新则完全取决于求解者的成功率**。如果求解者无法解决，出题者的奖励为 0；反之，出题者的奖励则为 $1 - \text{求解者成功率}$。 这种奖励机制的初衷非常直观：逼迫出题者去生产那些处于求解者能力边界（Frontier）上的、极具挑战性的高难度课题。 为了验证这一机制，研究团队引入了 **3000 个形式化数学问题**。这些问题完全用 Lean 4（一种数学证明编程语言）编写，这意味着无论是问题陈述还是证明过程，都可以被计算机自动且绝对准确地验证。然而，实验结果令人大跌眼镜： ``` 问题解决率 (%) 100 | 80 | 60 | ---------------------- (渐近线停滞在约 60%) 40 | 20 | 0 +---------------------- 0 投入的算力 (Compute) ``` 实验表明，随着训练的推进，出题者确实越来越擅长产生让求解者感到棘手的任务，合成任务的数量疯狂飙升。然而，这些任务对提升求解者的实际能力**毫无用处**，常规自我博弈的表现甚至没有跑赢普通的强化学习基准线。 演讲者在现场展示了一个出题者在训练后期生成的 Lean 4 数学题。屏幕上被蓝色高亮标出的结论部分用“惨不忍睹”、“灾难般复杂”来形容。原来，模型找到了作弊的捷径：**奖励机制要求问题必须很难，而让问题变难最简单的方式，不是提高数学深度，而是把题目变得极其冗长、恶心、充满人工雕琢的混乱与不优雅。** 这就像为了考倒学生，老师不去出微积分的思维大题，而是出了一个长达三页纸的高中代数死算题。学生只要在任何一个微小的符号上马虎出错，题目就做不对。这种合成问题虽然成功把求解率压到了 50%，但对于真正数学能力的进化毫无裨益。 --- ### [34:38](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2078s) 自我引导型自我博弈（SGS）的提出 针对上述顽疾，研究团队提出了**自我引导型自我博弈**（Self-Guided Self-Play, SGS）来重塑算法架构。他们针对那 3000 个无法解决的核心难题，实施了两个维度的创新： * **锚定任务分布**：不再让出题者天马行空地瞎编，而是给它一个未能解决的“目标问题”，提示它必须生成一个与之**紧密相关**的衍生合成问题。通过这种方式，将合成数据的分布牢牢锚定在人类认为有价值、有意义的问题空间里。 * **引入“引导者”（Guide）角色**：如果继续沿用旧的奖励机制，模型很快就会抛弃上述约束，重新倒向制造垃圾题的怀抱。因此，模型被赋予了第三种身份——引导者。它专门负责审查合成问题与目标问题之间的关联度，并严厉惩罚那些故意堆砌复杂度的行为。 在 SGS 算法中，出题者面临着**双重奖励（Dual Reward）**的考核：一方面，问题依然要保持适度的挑战性（提供强化学习所需的探索信号）；另一方面，这个难度得分必须乘以引导者的评分（Guide Score）。 由于时间紧迫，演讲者遗憾地略过了他投入大量精力的“学术级大规模强化学习基础设施（RL Infra）”的细节，直接展示了核心成果： 在同等算力下，SGS 方法成功打破了常规自我博弈的停滞僵局。团队使用了一个 **7B（70 亿参数）的小模型**，在花费了大约 8 倍的自我博弈算力后，其展现出的能力成功追平了比它大得多的 **670B（6700 亿参数）的“老大哥”模型**。 不过演讲者也非常坦诚且风趣地自嘲道，图表显示解决率依然没有达到 100%，自我博弈的探索之路还远未结束。但幸运的是，“读个 PhD 的时间非常长”，他有大把的时间继续在未来死磕这个难题。 --- ### [37:21](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2241s) Stream RAG：语音 AI 时代的检索增强生成新挑战 在 luke Bailey 演讲结束后，主持人 François Chaubard 邀请下一位演讲者 **Arnab Maiti** 上台。Arnab 目前在 Giga（YC 旗下发展最快的公司之一，市值已达三四亿美元）担任研究员，他在华盛顿大学取得了博士学位，主要研究方向为老虎机学习（Bandit Learning）。 Arnab 带来的分享是关于 Meta 团队发表的论文 **Stream RAG**。他强调，今天讨论的核心不是去死磕这篇论文的每一个微小算法细节，而是去剖析它所精准捕捉到的、在**语音 AI（Voice AI）**时代正在涌现的全新技术挑战。 回顾 2023 年，大语言模型（LLM）面临最严重的痛点就是“幻觉”（Hallucination），尤其是各种胡编乱造的文献引用。后来，**检索增强生成（RAG）**技术的全面铺开极大地缓解了这一问题。通过将用户的输入查询（Query）送入 RAG 系统，检索出真实可靠的参考文档喂给 LLM，从而让输出变得有据可依。 然而，随着当前大量语音 AI 初创公司的爆发，用户对交互的期待发生了质的改变： * **强烈的对话感（Conversational Nature）**：用户希望像和真人聊天一样，随口问一句“今天天气怎么样？”，智能体就能立刻自然地接话“现在大概 22°C”。 * **对幻觉更加敏感**：在文字交互中，人类很容易一眼扫到并揪出文本里的胡话；但在语音对白中，人类在倾听时极难实时捕捉到细微的幻觉错误。因此，语音 AI 比文本 AI 更需要 RAG 的保驾护航。 --- ### [41:12](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2472s) 打破 10 秒死寂：流式 tool 调用与实时响应 既然语音 AI 迫切需要 RAG，那为什么不能直接把现成的 RAG 管道搬过来用呢？答案是**致命的延迟（Latency）**。 > “如果我问语音智能体一个问题，它需要死机 10 秒钟才能回答我，那这完全不是自然的对话。” 传统的 RAG 必须等到用户把整句话完全说完整、系统将其全部转化为文本后，才慢吞吞地触发检索。为了打破这种尴尬的死寂，Stream RAG 提出了一个极其聪明的核心主张：**不等用户把话说完。在用户声音流不断涌入的同时，系统就开始对已经说出口的词汇进行实时拆解和分析，在用户还在说话的间隙，就把 RAG 给偷偷跑完。** Arnab 举了一个非常生活化的例子：用户可能会说：“嘿，今天天气怎么样啊……我想根据这个决定待会儿要不要出门。” 在这句话中，最核心的检索诉求在前半句（查天气）就已经完全表达结束了，后半句完全是无关的唠叨。如果能有一种机制，灵敏地捕捉到前半句的意图并提前触发 RAG，就能完美利用后半句的说话时间来对冲检索延迟。 这篇论文重点研究了两种流式处理策略，其中第一种非常朴素，被称为**固定间隔流式 RAG（Fixed Interval Streaming RAG）**： * 算法将连续的输入音频切割成若干个固定的数据块（Blocks）。 * 每当一个数据块 $B$ 到达，系统就立刻针对当前已有的片段运行一次 RAG 并产生结果 $R_B$，如此循环直至整句话结束。 这就引出了一个非常核心且活跃的研究问题：**由于不能等全句结束，系统到底该信任并采纳哪一个中间数据块的检索结果？** 论文给出的解决方案巧妙地利用了 RAG 管道内部组件的特性。RAG 包含很多微小的子模块，有些模块（如初筛检索）运行得极快。系统可以迅速为当前的中间查询拉取一组文档，并去跟最终完整查询可能匹配的顶级文档进行实时比对，如果发现两者的重合度和匹配度已经极高，系统就可以果断拍板：“不等了，直接用这个中间查询把整个 RAG 管道全部跑通！” Arnab 再次强调，最值得大家关注和激荡脑力的不是某个具体的权衡算法，而是这种在流式输入中**寻找最完美的时间拐点、果断中止等待并断定‘这个片段对我已经足够核心’**的思考范式。 ### [43:49](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2629s) 优化 Stream RAG 触发机制与效果 在探讨优化流式检索增强生成（Stream RAG）时，演讲者提出了一种更具策略性的方法。如果不想在每个音频文本块（chunk）到来时都盲目调用 RAG 管道，可能会造成极大的计算资源浪费。因此，一种潜在的解决方案是微调一个专用的轻量级模型。该模型的作用是在特定文本块到达时，自主判断其中是否包含“关键的新信息”。如果包含，则触发并生成新的检索查询；如果判断认为基于过去文本块生成的查询已经足够回答问题，则直接生成最终的音频响应。 > "在论文中，他们描述了一种训练后的管道。对于不完整的口头提问，他们首先利用大语言模型（LLM）生成一些伪查询（pseudo queries），接着运行 RAG 并查看检索到的文档。根据这些检索到的文档，系统进而决定这个不完整的查询是带来了全新的内容，还是说我们已经掌握了有用的素材。" 演讲者指出，这篇论文的核心逻辑是基于“目前为止不完整提问的检索质量”来做出决策。但这绝非唯一路径。未来的研究方向完全可以聚焦于直接分析“目前为止不完整提问的语义本身”，即仅凭问题自身的表意，判断是否已经足够回答，从而决定是否有必要启动整个繁琐的 RAG 流程。 - **延迟优化表现**：该论文在一年前基于较小的开源模型进行了测试。结果显示，在合成数据集上，流式 RAG 成功让延迟减少了 0.5 秒；而在人类真实的口头数据集上，延迟缩短了接近 1.5 秒。 - **准确率保持**：对比在最终完整提问后才运行的传统 RAG，流式 RAG 在大幅降低延迟的同时，保持了几乎相同的回答准确率。 - **生产环境价值**：演讲者总结道，尽管如何精准把握用户说话期间的检索时机属于“有趣的小问题”，但只要在生产环境中攻克这些细节，就能带来巨大的性能红利。 --- ### [47:23](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2843s) Lean 语言：验证智能的新时代 接下来登场的是加州理工学院（Caltech）三年级的博士生罗伯特·乔治（Robert George），他的主要研究方向是用于数学和科学的 AI。他带来了一场关于 Lean 证明辅助器的深度分享，并宣告学术与工业界正迈入一个“可验证智能（Verified Intelligence）”的新时代。 乔治首先回顾了过去两周以及前几年数学 AI 领域的爆发式突破。从 2024 年 DeepMind 凭借 AI 在国际奥林匹克数学竞赛（IMO）中斩获银牌级别的表现（对应 2024 年 IMO 的公开进展），到 OpenAI 宣称攻克了拥有 80 年历史的埃尔德什（Erdős）差异问题。在 OpenAI 的宣传视频中，菲尔兹奖得主陶哲轩（Terry Tao）也对这些技术大加赞赏。更近的一周内，DeepMind 再次发布新成果，不仅解决了大批埃尔德什猜想相关的难题，更跨界攻克了其他科学领域的多个开放式奥林匹克问题。这些研究最酷的地方在于，它们都将“形式化验证（Formal Verification）”无缝嵌入到了技术闭环之中。 > "我们都上过高中微积分和大学的数学课，那种非形式化的数学（informal math）是非常灵活的。有时候你的教授写完证明直接写个 QED，或者干脆来个‘威吓证明法’（proof by intimidation），很多关键步骤根本没有完整写下来。但我认为，形式化的世界要求你必须做到完全显式（fully explicit）。" - **自动化与交互式证明器的光谱**：在 1990 年代至 2020 年代，学术界广泛使用基于 SMT 求解器的自动定理证明器（ATP），它们虽不需要多少人类介入，但能表达的数学逻辑非常有限。而以 Lean、Coq、Isabelle 为代表的交互式定理证明器（ITP），基于更严密、表达力更强的依赖类型论（Dependent Type Theory）。尽管它过去需要大量的人力（例如无数研究者耗时十年为 Lean 维护著名的数学库 Mathlib），但如今它与 LLM 的结合正在以“小时”为单位刷新进展。 - **Lean 语言的多重身份**：Lean 不仅是一个定理证明器，它同时也是一个速度极快的函数式编程语言。它支持元编程、宏（Macros）以及自定义自动化，甚至有人用它开发游戏。 - **Mathlib 的生态威力**：作为目前最庞大的形式化数学库，Mathlib 拥有数百万行高质量代码，涵盖了从拓扑学到代数几何的顶尖数学研究。在 Lean 的世界里，研究者可以通过 VS Code 的 Info Goal 视图清晰地看到当前步骤需要证明的“子目标（Sub-goals）”，通过一条条被称为“策略（Tactic）”的代码步步推进。乔治笑称，虽然它的学习曲线很陡峭，但体验比学 C++ 好得多，且成功跑通证明、无法欺骗 Lean 内核的那种成就感是无与伦比的。 --- ### [53:19](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3199s) 形式化验证在代码与科学领域的破圈 乔治随后梳理了形式化突破的编年史。早在 2020 年，OpenAI 的 Ilya 和 Stan 就发布了 PACT 和 GPT-f，这是首个用于自动定理证明 Generative 语言模型，并在 miniF2F（一个奥林匹克级别的竞赛基准）上进行了测试。此后，全球范围内无论是中国的顶尖团队、美国的 Frontier Labs，还是各类初创公司（如 Harmonic AI 在普特南数学竞赛 Putnam 中利用其 Aristotle 系统成功攻克了全部 12 道难题），都在推动这一趋势。一旦有人宣称用 AI 找到了某个数学难题的解，他们现在往往会同时用 Lean 将其形式化证明出来。 除了纯数学，乔治强调形式化验证正在向另外两个巨大的领域“破圈”：代码验证（Program Verification）与科学 AI（AI for Science）。 > "大语言模型确实能写代码，但它们能证明代码是绝对正确的吗？随着生成式代码的规模呈指数级增长，Bug 的规模也在同步膨胀。这是一个价值数万亿美元的工业级痛点。我们不能仅仅追求‘宽泛编码’（wide coding），而应该转向‘可验证编码’（verifiable coding）。" - **程序验证的三要素**：人类心智中的规格说明（Specification）、实际编写的代码（Code）、以及证明代码完全符合规格的证明过程（Proof）。乔治介绍了他们提出的 Bridge 工作，正是利用 Lean 作为函数式语言的特性，引导 LLM 更好地完成代码证明。同时，他推荐大家去关注由斯坦福大学 Clark Barrett 团队发起、DeepMind 等共同参与的 CSLab（计算机科学形式化库）生态。 - **TorchLean 框架**：这是乔治近期推出的首个在 Lean 中完整编写神经网络的统一框架。它拥有类似 PyTorch 的张量系统，一切均可编译至共享的中间表示（IR）。基于此，研究者可以实现验证浮点数运算、进行神经网络的认证鲁棒性验证等。 - **实际应用案例**：在 TorchLean 中，他们成功在规格层面（spec level）证明了 FlashAttention 与标准注意力机制（Standard Attention）的完全等价性，并证明了在不引入位置编码时注意力机制的置换不变性（Permutation Invariance）。乔治甚至完全在 Lean 内部原生训练了一个 Andrej Karpathy 风格的 GPT-2 模型。 - **解决硬件非确定性 Bug**：针对 Thinking Machine Lab 去年指出的大模型在温度（Temperature）为 0 时依然因硬件底层产生非确定性（Non-determinism）的现象，乔治在 TorchLean 中对该系统进行了全栈形式化，一路深入到 GPU 细微的 CUDA 算子级别。证明了正是由于极其微小的浮点数运算差异，在 Batch 批处理中翻转了最终的 argmax 结果。这种将形式化推进到解决现实软件 Bug 的尝试，向人们展示了科学代码完全可信的未来。 --- ### [58:05](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3485s) 创始人 AI 骇客：当编程变成一场即时战略游戏 在全场的掌声中，主持人 Francois Chaubard 引出了本场活动的最后一位演讲嘉宾——Channel AI 的首席执行官卢肯斯·奥特温（Lukens Orthwein）。Francois 调侃卢肯斯是这场极度严谨的形式化验证（Lean）分享的“反面”：他是极度崇尚“Token Maxing（极限消耗 Token）”的代表。作为哈佛大学计算机系的毕业生，卢肯斯曾在 2012 至 2015 年间负责微信（WeChat）海外广告的增长业务，被 Francois 戏称为“香港之狮”，如今他正带领着一家方向激进的消费级娱乐 AI 公司。 卢肯斯登台后，直接抛出了他的核心论点：在 Claude、Agent 辅助编程大行其道的今天，软件工程的底层逻辑需要被彻底重构。过去几十年里沉淀下来的许多关于“什么是好编程”的教条和假设，在今天不仅不再适用，甚至恰恰成了开发者应该避免的误区。 > "国际象棋是过去编程模式的一个不完美比喻。在过去，你希望编程是非常线性的。你需要去预测未来，极其深思熟虑地设计架构，以确保系统足够健壮、能够完美运行且绝对正确。即便你写得很粗糙，它本质上依然是一个单线程的过程，你在特定时刻只需要专注并担心眼前的这一件事。" 然而，在 Agent 触手可及的全新世界里，卢肯斯认为编程已经从传统的、深思熟虑的线性“国际象棋”，蜕变成了一场多线并行、高强度的“即时战略游戏（RTS Game）”。开发者不再需要把大量精力耗费在“预测未来会遇到什么 Bug 并提前设计冗余架构”上，而是应该转变为一个全局的指挥官，同时调度多个 AI Agent 去试错、去冲锋、去探索不同的代码路径。Channel AI 团队正是通过这种在常人看来有些“疯狂”和“不羁”的开发工作流，最大化地榨取 AI 的生产力，从而在竞争激烈的消费级 AI 赛道中保持身位的绝对领先。 ### [01:00:39](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3639s) 编程就像打即时战略游戏（RTS） 演讲者 Lukens Orthwein 分享了他对使用智能体（Agent）系统进行编程的独特见解。他指出，由于自己是即时战略游戏（RTS）的狂热粉丝，在使用智能体系统时，他强烈地感觉到这与打 RTS 游戏几乎一模一样。这种体验与下国际象棋截然不同，尤其是在高水平的对抗中，没有任何一个单一的操作可以让你完美地直接胜出。你必须在同一时间平衡许多不同的事情：经济要持续运转，生产线不能停，各个单位必须在地图上做一些建设性的工作，同时你还要参与各种交战。 这就带来了一个核心问题：如何最大程度地实现系统运作和个人注意力的并行化。在游戏里，随着新地图的开启、新情报的获取，你必须不断加入修正性的反馈。卢肯斯认为，这完美地映射了当今使用智能体进行代码开发的实际现状。 --- > “对我来说，使用智能体进行编程的感觉，和打 RTS 游戏简直一模一样。” ### [01:01:32](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3692s) 构建智能体协调工具：从线性工作树到多智能体并行 为了让大家更有体感，卢肯斯介绍了他和团队围绕这一理念构建的工具。他们开发了一套名为 LW 的系统，即线性工作树（linear work trees）。在过去只有单个开发者时，本地机器上只需要一个代码仓库（repo）。但现在，为了适配智能体的并行开发，工程师的机器上需要同时运行海量的代码仓库，它们各自独立编译，互不干扰。通过将 Git 工作树（git work trees）、任务管理软件以及可移植的工作流相结合，他们成功地将一个或多个自主智能体插入到特定的工作流程中。 在日常开发和发布产品时，卢肯斯通常会使用一个主要由 Claude（有时也用 OpenAI Codex 2）驱动的编排智能体（orchestrator agent）。他的目标是尽量减少从“产生修复想法”到“启动实际工作”之间的按键次数。他把这比作在 RTS 游戏里框选一个单位，然后直接在地图的另一端右击鼠标命令其移动——你不需要在一开始就精细操控，等会儿回过头来再让它高效运作即可。 ### [01:02:53](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3773s) 贯彻 RTS 核心原则：追求宏观经济与极致并行 在系统运作中，编排智能体扮演着监视“小地图”的角色，时刻追踪着被派往各处的数个工作智能体（workers）的状态。团队对这些工作智能体提出了一个非常明确的要求：尽可能走得更远。整个系统的设计逻辑是：**极度贬低智能体的时间和劳动成本，而将人类工程师的时间和精力视为最高溢价**。这意味着，即使智能体可能会犯错，即使它们稍后需要被纠正，也必须鼓励它们在向人类寻求反馈之前，尽可能地向前推进工作，最好是一路推进到提交拉取请求（PR）的阶段。 从单次 Token 消耗的角度来看，这种让大量智能体并行乱撞的做法可能非常浪费，但它能为人类节省极大的时间，允许工程师同时开展多项任务。当工作智能体完成任务后，系统会自动化地将结果和总结反馈回知识库，让系统本身不断学习进化，而不需要人类进行大量的打字或介入。卢肯斯强调，这必须是一个具备高度可见性的系统，而不是盲目地直接催生 20 个智能体，然后一厢情愿地“希望”它们能毫无差错地在生产环境中搞定一切——因为在现实中这根本不可能发生。 * **智能体目标**：不只是写出代码，必须以直接提交 Pull Request 和生成结构化工作总结为终极目标。 * **权限配置**：在开发和运行脚本时，尽可能在“危险地跳过权限确认”（dangerously skip permissions）的模式下运行。如果无法做到，也应立即为其构建沙盒，因为频繁的人工权限确认会极大拖慢整体推进速度。 * **前端自动化**：在进行前端开发时，智能体启动后会自动在后台把本地开发服务器运行起来、执行完测试，并时刻等待。人类过来时只需打开对应的浏览器端口标签页，就能以最少的点击和步骤直接进行预览和测试。 ### [01:06:36](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3996s) 逆转思维：不要让 Claude 预估时间，要把代码视作高昂的真理源 在探索智能体边界的过程中，团队发现了一些 LLM 的固有缺陷并沉淀出了应对策略。例如，Claude 目前非常不擅长预测一项任务需要消耗多长时间。如果你问它搞定某个问题要多久，它常常会根据训练数据中人类的表现回答“大概需要一个工程师两周的工作量”；然而在实际操作中，它可能只需要一条优秀的提示词，在 20 分钟内就能把代码写完。卢肯斯认为，智能体在时间预估上由于缺乏针对性的最新训练数据，未来也可能会持续滞后。因此，他们在系统里内置了硬性规则：永远不要信任智能体自身对时间的评估。 此外，行业普遍认为“代码是唯一的真理源（Source of Truth）”，但对于智能体而言，直接去通读和解析庞大的底层代码以提取上下文，是一件极其昂贵且效率低下的事情。相反，在当前长上下文大模型内存充足的背景下，反向操作往往极其廉价——即激进地为智能体编写结构化的文档。 --- > “代码对智能体来说是一个非常昂贵的上下文来源。在上下文能长久留在内存的今天，积极地去写文档、构建类似 Wiki 相互链接的结构化知识库，反而更加便宜，这能让未来的智能体轻松享受到上下文红利。” ### [01:07:54](https://www.youtube.com/watch?v=3rWSvrFahIY&t=4074s) 宏观重于微观：像拉高 APM 一样提升智能体工具调用率 卢肯斯提出了另一个 RTS 核心原则：“默认宏观，关键时刻再微观”（macro by default, micro when it counts）。在游戏里，哪怕你微操单个单位的走位再厉害，如果你压根没有开矿暴兵，你注定会输掉比赛。编程也是一样，深入某些关键的复杂工单（Tickets）进行微调确实不可避免，但在你陷入微观细节的任何时候，都应该同时思考：我还能不能再催生出一些不需要我消耗过多认知带宽的小任务，让它们同时向前推进？ 为了最大化利用人类的认知容量，团队甚至不在乎某些任务断掉或被挂起。你可以把一个任务搁置 3 天，回过头来直接让 Claude“提醒我之前在这儿干了些啥”，因为这种上下文召回是非常廉价的。相反，真正昂贵、却经常被人们忽略的隐形成本，是“没有同时去推进这十多件事”。 为了像高水平 RTS 选手那样实现高效审计和快速纠偏，他们做了一些非常有意思的硬核极客尝试： * **可视化监控**：不把智能体的运行日志藏在深处，而是设置快捷键，像 RTS 选手切屏一样，在不同的智能体终端流（T-mux Sessions）之间一键反复横跳。 * **Starcraft/Warcraft 音效提示**：由于人类无法时刻盯着所有屏幕，卢肯斯将不同的智能体任务根据工单类型进行颜色编码和主题分类，并将其映射到了《魔兽争霸》（Warcraft）和《星际争霸》（Starcraft）的不同单位上。当某个智能体需要人类介入或完成了任务，音箱里就会播放对应的游戏语音或受击特效（例如“我们的基地正受到攻击”），人类一听就能形成条件反射，立刻切屏过去修正。 * **APM 追踪器（每分钟操作数）**：他们在内部甚至为自己和智能体构建了 APM 追踪工具。在编程领域，智能体的 APM 并不是去统计键盘鼠标的点击次数，而是统计**每分钟内智能体进行了多少次工具调用（Tool Calls）**。通过监控 1 分钟、5 分钟、1 小时、1 天到 7 天的工具调用频率曲线，来量化团队的真实生产力，确保 Claude 购买的 Token 资源每时每刻都在满负荷运转，不造成任何经济浪费。 ### [01:13:54](https://www.youtube.com/watch?v=3rWSvrFahIY&t=4434s) 让智能体自动生成幻灯片与自主提炼工单 在分享的尾声，卢肯斯向大家揭秘，他今天上台演讲所使用的整套 PPT，其实就是用上述这套“智能体打通知识库”的逻辑开发出来的。 首先，他把主持人 Francois 提出的演讲主题和要求直接复制给 Claude，并附带了团队长久以来积累的内部知识库文档，命令它根据里面的方法论和开发哲学组织出一套幻灯片内容。虽然智能体没有做到一键完美成型，但在人类后续进行了大约 15 次细节微调和修改后，就直接达到了可以登台演示的最终效果。最核心的闭环在于，演示完成后，卢肯斯又将这些人类修改后的最终版本、反馈意见以及纠错细节重新喂回了系统的知识库。由于 LLM 遍历和吸收相互链接的结构化文档效率极高，整个系统在未来生成同类商业决策、甚至自动为团队生成和拆解开发工单时，表现得越来越聪明。 ### [01:14:59](https://www.youtube.com/watch?v=3rWSvrFahIY&t=4499s) 像即时战略职业选手一样编程：AI 时代的管理艺术 Lukens Orthwein 进一步阐述了他将 AI 智能体（Agent）引入日常开发后的核心管理哲学。他认为现在的工程师应该时刻保持“全栈（Full Stack）”状态，并且要具备极高的敏锐度和响应速度。在实际操作中，即使 AI 智能体目前在某些任务上的表现比人类更差、更慢，但“让它去干”依然是更优的选择，因为在代码被搞砸后进行修改和重构是非常容易的。他引入了经济学中的一个专业术语——“满意化（Satisficing）”，即追求“足够好”而非“完美”，并强调这是在当前时代推进所有工作的关键原则。 这种工作流的彻底转变带来了惊人的生产力爆发。Lukens 兴奋地分享了他们团队的惊人数据：通过全面应用这套方法，工程师人均每月的 PR（Pull Request）产出已经达到了原来的 3.5 倍。特别是在过去的一个月里，当整个团队全员深度贯彻这一模式后，人均月 PR 产出在原有的高基础上又激增了 60%。他总结道，人的智商是无法在短期内大幅提升的，但可以训练自己的行为模式，去模仿那些擅长多任务并发处理的专家。他号召大家：“我们应该去像即时战略（RTS）职业选手一样编程，学习如何在这套系统里达到操作的最优解。” * **核心理念**：鼓励全栈、高响应性的工作流，践行经济学中的“满意化（Satisficing）”原则，即追求足够好而非完美。 * **量化成果**：团队的工程师人均月 PR 产出实现了 3.5 倍的增长，在全员全面普及该方法的最后一个月，该指标又追加了 60% 的增幅。 * **最终行动倡议**：将编程视作即时战略（RTS）游戏，通过并行调度不同的“任务票据（Tickets）”，实现个人与 AI 协同的效率最大化。 --- ### [01:16:06](https://www.youtube.com/watch?v=3rWSvrFahIY&t=4566s) 纸上得来终觉浅：YC 论文俱乐部圆满落幕与茶歇交流 随着最后一位演讲者的精彩分享结束，主持人 Francois Chaubard 重新接过麦克风，为本次 YC Paper Club 划上圆满的句号。活动随即进入了大家期待已久的茶歇与自由社交环节。Francois 笑着打趣道，现场已经准备好了各式各样的甜点，包括曲奇饼干、冰淇淋、棒冰以及麻薯甜甜圈（Mochi Donuts）。面对现场观众对“麻薯甜甜圈”到底是什么的疑惑，台下有人热情地回应“它超级好吃！”。 在轻松愉快的氛围中，Francois 表达了对所有到场嘉宾和讲者的诚挚谢意。他透露，稍后会向大家发送一份反馈表，希望参与者能不吝赐教、分享真实的想法。同时，他也向在场的创始人和研究员们发出了未来的演讲与选题征集邀请。由于两周后的下一期沙龙名额已经全部报满，目前他们正在积极筹备 7 月份的第一场活动。他热情地呼吁有志于分享前沿 AI 论文的同学们尽快与他联系，并在掌声中结束了本期活动。 * **现场茶歇**：提供曲奇、冰淇淋、棒冰和麻薯甜甜圈，现场互动气氛轻松融洽。 * **下期预告**：两周后的 Paper Club 席位已完全约满，目前正公开招募 7 月份第一期活动的演讲人（Presenters）和论文创意。 * **闭幕致谢**：主持人感谢了所有人的到来，并计划发放反馈表以持续优化活动体验。 ## Notable quotes > [00:00](https://www.youtube.com/watch?v=3rWSvrFahIY&t=0s) 这并不是说它完全不可能，而是说我们通过采样穷尽整个未知空间的可能性微乎其微。如果解空间的正解全集是 $F$，那么即便给予再多可行的测试时计算量或递归自我提升，仅在已知的人类解 $H$ 上进行训练，必然会把你限制在某个典型的人类解集内。在现实中，你根本无法有效地对 $F - H$ 的空间进行充分采样，更不用说穷尽它了。除非你有无穷无尽的递归提升和无限的计算资源，但遗憾的是我们没有。生命本身是一个部分可观测马尔可夫决策过程（POMDP），而我们面临的是一个有限时界的马尔可夫决策过程（Finite horizon MDP）。 — Francois Chaubard > [05:47](https://www.youtube.com/watch?v=3rWSvrFahIY&t=347s) 这并不意味着 MSA 已经彻底消亡，而是说明手工特征只有在数据丰富时才有帮助，而当药物设计师真正需要它时，它往往无能为力。 — Yasa Baig > [05:47](https://www.youtube.com/watch?v=3rWSvrFahIY&t=347s) 这简直太疯狂了，进化关系和功能关系完全是模型作为副产品白嫖（for free）得来的！ — Yasa Baig > [28:57](https://www.youtube.com/watch?v=3rWSvrFahIY&t=1737s) 在理论上，没有什么能限制它的学习。……但遗憾的是，现实并非如此。 — Luke Bailey > [41:12](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2472s) 如果我问语音智能体一个问题，它需要死机 10 秒钟才能回答我，那这完全不是自然的对话。 — Arnab Maiti > [43:49](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2629s) 在论文中，他们描述了一种训练后的管道。对于不完整的口头提问，他们首先利用大语言模型（LLM）生成一些伪查询（pseudo queries），接着运行 RAG 并查看检索到的文档。根据这些检索到的文档，系统进而决定这个不完整的查询是带来了全新的内容，还是说我们已经掌握了有用的素材。 — Arnab Maiti > [47:23](https://www.youtube.com/watch?v=3rWSvrFahIY&t=2843s) 我们都上过高中微积分和大学的数学课，那种非形式化的数学（informal math）是非常灵活的。有时候你的教授写完证明直接写个 QED，或者干脆来个‘威吓证明法’（proof by intimidation），很多关键步骤根本没有完整写下来。但我认为，形式化的世界要求你必须做到完全显式（fully explicit）。 — Robert George > [53:19](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3199s) 大语言模型确实能写代码，但它们能证明代码是绝对正确的吗？随着生成式代码的规模呈指数级增长，Bug 的规模也在同步膨胀。这是一个价值数万亿美元的工业级痛点。我们不能仅仅追求‘宽泛编码’（wide coding），而应该转向‘可验证编码’（verifiable coding）。 — Robert George > [58:05](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3485s) 国际象棋是过去编程模式的一个不完美比喻。在过去，你希望编程是非常线性的。你需要去预测未来，极其深思熟虑地设计架构，以确保系统足够健壮、能够完美运行且绝对正确。即便你写得很粗糙，它本质上依然是一个单线程的过程，你在特定时刻只需要专注并担心眼前的这一件事。 — Lukens Orthwein > [01:00:39](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3639s) 对我来说，使用智能体进行编程的感觉，和打 RTS 游戏简直一模一样。 — Lukens Orthwein > [01:06:36](https://www.youtube.com/watch?v=3rWSvrFahIY&t=3996s) 代码对智能体来说是一个非常昂贵的上下文来源。在上下文能长久留在内存的今天，积极地去写文档、构建类似 Wiki 相互链接的结构化知识库，反而更加便宜，这能让未来的智能体轻松享受到上下文红利。 — Lukens Orthwein ## Mentioned Francois Chaubard, Yasa Baig, Luke Bailey, Arnab Maiti, Robert George, Lukens Orthwein, Y Combinator, Tatsu 实验室, Giga, Lean, Mem0, Cartridges, H-Net, Noam Brown, Magnus Carlsen, Steve Quake, Chan Zuckerberg Biohub, Biohub, Richard Sutton, AlphaGo, AlphaGo Zero, OpenAI, ESMC, Evolutionary Scale, Meta, BERT, ESM Cranberry, ESM2, Dan Fu, AlphaFold, AlphaFold 3, ESM Fold 2, Anthropic, CRISPR-Cas9, Cursor, AlphaZero, Tatsuya 实验室, Tatsunori 实验室, Claude, Harmonic AI, DeepMind, 陶哲轩, Coq, Isabelle, Mathlib, Ilya, Stan, PACT, GPT-f, miniF2F, TorchLean, Andrej Karpathy, GPT-2, Thinking Machine Lab, Channel AI, 微信, OpenAI Codex 2 [Source](https://www.youtube.com/watch?v=3rWSvrFahIY)

[0:00] Francois Chaubard: 这并不是说它完全不可能，而是说我们通过采样穷尽整个未知空间的可能性微乎其微。如果解空间的正解全集是 $F$，那么即便给予再多可行的测试时计算量或递归自我提升，仅在已知的人类解 $H$ 上进行训练，必然会把你限制在某个典型的人类解集内。在现实中，你根本无法有效地对 $F - H$ 的空间进行充分采样，更不用说穷尽它了。除非你有无穷无尽的递归提升和无限的计算资源，但遗憾的是我们没有。生命本身是一个部分可观测马尔可夫决策过程（POMDP），而我们面临的是一个有限时界的马尔可夫决策过程（Finite horizon MDP）。 [5:47] Yasa Baig: 这并不意味着 MSA 已经彻底消亡，而是说明手工特征只有在数据丰富时才有帮助，而当药物设计师真正需要它时，它往往无能为力。 [5:47] Yasa Baig: 这简直太疯狂了，进化关系和功能关系完全是模型作为副产品白嫖（for free）得来的！ [28:57] Luke Bailey: 在理论上，没有什么能限制它的学习。……但遗憾的是，现实并非如此。 [41:12] Arnab Maiti: 如果我问语音智能体一个问题，它需要死机 10 秒钟才能回答我，那这完全不是自然的对话。 [43:49] Arnab Maiti: 在论文中，他们描述了一种训练后的管道。对于不完整的口头提问，他们首先利用大语言模型（LLM）生成一些伪查询（pseudo queries），接着运行 RAG 并查看检索到的文档。根据这些检索到的文档，系统进而决定这个不完整的查询是带来了全新的内容，还是说我们已经掌握了有用的素材。 [47:23] Robert George: 我们都上过高中微积分和大学的数学课，那种非形式化的数学（informal math）是非常灵活的。有时候你的教授写完证明直接写个 QED，或者干脆来个‘威吓证明法’（proof by intimidation），很多关键步骤根本没有完整写下来。但我认为，形式化的世界要求你必须做到完全显式（fully explicit）。 [53:19] Robert George: 大语言模型确实能写代码，但它们能证明代码是绝对正确的吗？随着生成式代码的规模呈指数级增长，Bug 的规模也在同步膨胀。这是一个价值数万亿美元的工业级痛点。我们不能仅仅追求‘宽泛编码’（wide coding），而应该转向‘可验证编码’（verifiable coding）。 [58:05] Lukens Orthwein: 国际象棋是过去编程模式的一个不完美比喻。在过去，你希望编程是非常线性的。你需要去预测未来，极其深思熟虑地设计架构，以确保系统足够健壮、能够完美运行且绝对正确。即便你写得很粗糙，它本质上依然是一个单线程的过程，你在特定时刻只需要专注并担心眼前的这一件事。 [60:39] Lukens Orthwein: 对我来说，使用智能体进行编程的感觉，和打 RTS 游戏简直一模一样。 [66:36] Lukens Orthwein: 代码对智能体来说是一个非常昂贵的上下文来源。在上下文能长久留在内存的今天，积极地去写文档、构建类似 Wiki 相互链接的结构化知识库，反而更加便宜，这能让未来的智能体轻松享受到上下文红利。

[5:47] Yasa Baig: The Bitter Lesson Comes from Biology [5:47] Yasa Baig: you'll know a protein by amino acids it keeps [20:21] Yasa Baig: Google Maps for proteins [27:01] Luke Bailey: conjecturer [27:01] Luke Bailey: solver [47:23] Robert George: new era of verified intelligence [47:23] Robert George: you cannot fool this theorem prover [49:03] Max Tegmark: very coding [58:05] Francois Chaubard: token maxing to the max [58:05] Francois Chaubard: probably the most unhinged technical CEO that I know [60:39] Luke Orthwine: grabbing a unit and just like clicking across the map [65:31] Luke Orthwine: one of like the really obvious things that Claude is super bad at today is predicting how long it'll take to do something [65:31] Luke Orthwine: Macro by Default, Micro When it Counts [68:50] Luke Orthwine: nobody is good who doesn't have high APM [74:59] Luke Orthwine: learn the methods of doing it just like program like an RTS pro

[0:08] Thank you guys so much for coming. This [0:10] one will have much a much more applied [0:11] bent based on the feedback. We have a [0:13] bunch of really cool people that I'll [0:14] introduce in a second, but we're [0:16] covering AI for uh biology by my [0:20] favorite one of my favorite co- [0:22] researchers, Yas Beg. We have Luke um [0:24] out of Tatsu's lab talking about [0:26] selfplay, Alpha Zero style selfplay for [0:28] LLMs. Super excited about that. Arnob [0:31] will be uh presenting he's a researcher [0:33] at Giga on stream rag uh super different [0:36] application you know thinking about uh [0:39] real realtime voice uh agents uh Robert [0:42] George working on lean for science super [0:44] exciting and then the AI token maxer [0:47] himself Luke Worthwine [0:50] cool so I want to introduce some like [0:52] call for presentations you know maybe [0:54] inspire some of my interest and maybe [0:56] inspire some some of you guys to jump up [0:58] and and ask for a presentation on this [1:00] stuff. I think memory has been like the [1:01] hot topic for at least the last year and [1:03] a half. There's been so many papers from [1:05] mem zero to recursive language models [1:07] cartridges out of uh our lab hnet, you [1:10] know, dynamic chunking stuff. There's so [1:12] many different ideas and so I'm [1:14] definitely interested in that area. If [1:15] you guys want to present on that, I did [1:17] this Nome Brown podcast I think a couple [1:19] weeks ago launched and like he's still [1:22] of the view that this human generated [1:24] subspace H is still if we train on that [1:28] we can test time compute our way out of [1:30] it and recursive self-improve out of it [1:33] all the way to get to this F minus H. [1:36] And I just like really struggle with [1:37] this and I really really don't see how [1:40] it's probable. Not that it won't it's [1:42] not possible, but it's just not probable [1:44] that we'll sample all of that. So, I'm [1:45] really interested in that and that's [1:46] definitely in Luke's. We were talking [1:48] about that a bunch and I think that [1:50] basically the left side is alpha go, the [1:52] right side is alpha zero. And I think [1:53] that alpha zero unbiased by um humans [1:58] meandering is uh the way we'll get to [2:01] much more intelligent systems, maybe [2:02] even dare say agi. And the right way to [2:05] say this um would be um I want to be [2:07] very careful. If the full solution space [2:10] f is f, training on known human [2:12] solutions will limit you to some typical [2:14] set h despite any feasible amount of [2:17] test time compute or recursive self um [2:19] improvement. You won't feasibly sample f [2:22] minus h. Um and especially all of it. If [2:26] if it's infinite recursive self [2:28] improvement, infinite test compute [2:29] maybe, but we don't have infinite. So [2:32] life's a pom dp and this is a we're [2:34] finite horizon mdp. [2:37] Intelligence per sample. I think this is [2:38] like the two major problems left in my [2:40] opinion are intelligence per sample, [2:42] intelligence per watt. And intelligence [2:43] per sample. I always think about this [2:44] like as I get one new sample, I do this [2:47] continuous learning. What is the right [2:49] thing to do if I'm trying my goal is to [2:50] maximize performance condition upon that [2:53] n. Most people's answer to this right [2:56] now in practice is ICL. And I actually [2:59] played with this. As you increase the [3:01] number of samples in ICL, it is not [3:03] monotonically improving in performance. [3:05] And so it actually starts to bob and [3:07] weave. It gets worse. It gets better [3:08] sometimes. Um and then it hits a cliff, [3:11] which is the context um uh length, [3:14] context length that the model was [3:15] trained on, and it literally just stops. [3:17] So it clearly can't go on forever. It [3:19] doesn't monotonically improve. And I [3:22] started playing around with Laura. I [3:23] think Laura at higher at lower ranks for [3:26] uh lower amounts of sample size actually [3:28] does impressively well. And then it has [3:30] this kind of arc. They both peter out [3:32] pretty quickly as you increase number of [3:33] samples all the way until you do SFT [3:36] group O all the way at the end. And if [3:38] you look at this, it's kind of weird [3:40] that like you have this like ICL is the [3:42] optimal thing to do in the beginning and [3:43] then training the whole thing. If I get [3:45] one new sample, I want to retrain with [3:48] Laura at some rank on N plus1 samples [3:51] just to get that little bump in [3:52] performance. And there's a different [3:54] optimal thing to do all along the way as [3:57] I stream and I get more and more [3:59] samples. And that's just not how we are. [4:01] So we're kind of like monotonically [4:03] improving. And the more chess games that [4:06] Magnus Carlson plays, he just keeps [4:08] getting better. The, you know, 10,000 [4:10] hour rule, etc., etc., we just keep [4:12] getting better. And it's the same algo. [4:14] And so I think there's just something [4:15] really different happening in us. And so [4:17] there's must exist some learning [4:19] procedure that is has a much higher [4:21] intelligence per sample. And then [4:22] intelligence for Watt out of my lab, [4:24] Ivonica and John who will hopefully come [4:26] to the next one uh and give a talk on on [4:28] this. And I just think it's the right [4:29] way to think about it. Arguing that [4:30] having smaller models um are sometimes [4:32] actually better from an intelligence per [4:34] watt perspective. Alternatives to back [4:36] prop for those that know me, I'm very [4:38] hot on this. Back to the brain um and [4:40] how we learn. There's very little [4:42] evidence that the brain is taking the [4:44] transpose of the weight matrix and there [4:46] must exist some other learning [4:48] procedure. I'm highly interested in [4:49] SPSA, but if there's alternatives that [4:51] I'm not aware of, please like recommend [4:53] them. And I'm really interested in novel [4:55] breakthroughs. Yaso is one of my [4:57] favorite AI researchers, but he's mostly [4:59] focused on bio and he always um sends me [5:01] biopers and it's super interesting. [5:02] Whether it's about how birds navigate [5:06] the world via iron in their liver [5:08] apparently that's how they they actually [5:10] navigate [5:12] crazy robotics, uh speech, other things [5:14] like that as well. And then of course [5:17] unhinge founder hacks very interested in [5:20] that as well. Call for ideas on ways to [5:23] make the club better. So, better ways to [5:26] meet you if it's a lightning round if [5:27] it's not. Um, some people talked about [5:29] some AI benchmarks that we could [5:30] actually launch together. That'd be kind [5:31] of fun. Club challenges to challenge [5:33] each other. And then, uh, any open [5:35] source ideas that you want to hack on [5:37] together for this club or something or [5:39] otherwise. It' be very interesting. All [5:40] right, that's all I got. Thank you so [5:42] much. [5:47] >> Hi. Yeah. Uh, thanks France for that [5:49] introduction. We've been labmates now [5:51] for like two years, something like that. [5:53] Yeah. Um, I think France is a great [5:55] example of someone who brings very [5:57] creative and very out of distribution [5:58] ideas to our group all the time, even if [6:00] I sometimes like have no idea where he [6:03] gets them from. Um, but um, that being [6:06] said, uh, he asked me to give a talk on [6:08] some bioai things and I thought why not? [6:10] Um, so I'll be presenting on this paper [6:12] that came out just last week from Biohub [6:14] folks um, here in California, not too [6:17] far actually in the Bay Area. I think [6:18] they just moved to the city actually. [6:20] Um, so I am a second year PhD student [6:22] with France, but I'm also co-advised by [6:23] Steve Quake over at Stanford. Sort of [6:25] anyone in biology probably will know [6:27] Steve at least tangentially done a lot [6:29] of work in bioengineering and all kinds [6:31] of applications was director of the [6:32] biohub where this work came out um just [6:34] until recently. So uh a lot of uh a lot [6:37] of overlap but uh the high level pitch [6:39] for this work is that I know most of [6:40] this audience is probably more like AI [6:42] ML types. I want to talk a little bit [6:44] about how sort of a lot of these ideas [6:46] from sort of that's motivating a lot of [6:48] progress in language modeling and AI [6:49] very broadly have been sort of recently [6:50] been translating into biology with a [6:52] focus on this recent paper because I [6:53] think it really does a really excellent [6:54] job of interrogating how scale which you [6:57] know at some level has been like the [6:58] fundamental primitive in terms of [6:59] assumptions that we as a community have [7:01] in terms of how to make things better um [7:03] has actually been playing out for a lot [7:04] of these biological problems [7:06] particularly protein biology. So um [7:08] there won't be like much bio in this [7:10] talk. I'll try to focus more on the ML, [7:11] but feel free to ask questions. Um so [7:13] yeah, like I called this talk the uh [7:16] bitter lesson comes from biology. The [7:17] actual paper title is right below that. [7:20] But I mean just a quick refresher. I'm [7:21] sure everyone in this specific audience [7:23] probably read Richard Sutton's famous [7:25] article. You know, basic premise here is [7:28] that like, you know, across the past 70 [7:29] years of AI, methods that win are [7:31] methods that are general that sort of [7:33] exploit really fundamentals of like [7:35] scaling compute and data as opposed to [7:37] methods that sort of handgineer human [7:39] domain, human domain knowledge. And [7:41] Sutton always cites his work in or like [7:43] you know a lot of the work that was [7:44] early in the field. So alpha go alpha go [7:46] zero that's sort of just inordinately [7:48] scaled compute and then for a long time [7:49] they were far worse than sort of expert [7:51] systems until they eventually overtook [7:53] and then exponentially improve past them [7:55] right knowledge systems win at first but [7:58] then eventually sort of these like big [7:59] large dumber models will like um you [8:02] know win in the long run this is sort of [8:04] a new goal in biology is to what extent [8:06] can we study like this is actually also [8:07] true for a lot of these sort of [8:08] biological AI problems right um the bet [8:11] sort of behind this paper and they do a [8:12] really exploring is well the same [8:15] pattern basically also saw protein [8:16] biology right um can we take like you [8:20] know a lot of these ideas in scaling law [8:22] analysis here is from the um uh famous [8:25] you know neural scaling laws paper and [8:26] then translate them for all these [8:27] problems that we care about for say [8:28] designing a drug or like you know trying [8:30] to understand how like a cell works [8:32] right so on the left is something that [8:33] we trust it's like a language model we [8:35] have like this nice smooth log linear [8:37] scaling laws that we see that moss like [8:38] predictively falls as a function of [8:40] compute and data uh the question on the [8:42] right is whether this curve will exist [8:44] for bio spec more generally but proteins [8:46] specifically in this paper you know sort [8:48] of does our LLM recipe transfer or does [8:50] biology really out of distributional [8:51] domain relative to language sort of [8:53] break it right that's the bet and like [8:55] in this talk I'll basically chat about [8:56] like three like vignettes from this [8:58] paper that sort of interrogate to what [8:59] extent this is true [9:02] so this slide is all the biology you'll [9:04] ever need about a quarter of the slide [9:06] at least for this presentation um let's [9:09] talk about proteins so in your body [9:11] there's like broadly three major classes [9:12] of macro molecules. There's lipids, [9:14] carbohydrates, and proteins. Um, a [9:17] protein is just a string of amino acids, [9:18] a special type of biomolelecule. There's [9:20] 20 varieties of amino acids. If you put [9:22] them together into a sequence, you can [9:23] have like a virtually infinite number of [9:25] possible molecules that then fold into. [9:27] So, you can think about it largely as [9:29] just every single protein is this 20 [9:31] letter alphabet. And that string [9:33] specifically determines a unique 3D [9:34] shape. And by virtue of the shape of [9:36] that protein, what job it does in your [9:38] cell like presents catalyzes a reaction, [9:40] keeps pathogens out, etc. Um, The work [9:43] done in this paper is that their goal is [9:45] to train um ESMC sort of their like [9:48] third major or fourth major iteration in [9:50] a long series of models of the group of [9:52] evolutionary scale group originally at [9:53] Meta then their own company now Biohub [9:55] have been training for a few years now [9:57] where the cell is very similar to [9:59] language where we let's just take [10:01] hundreds of millions of years of or sort [10:03] of evolved sequences that we've sort of [10:05] gone out and found across biology both [10:07] in humans but also across bacteria and [10:09] in our environment and just go train a [10:11] big mass language model on them, right? [10:14] So, I mean nowadays we mostly train NTP [10:16] models, but the pitch here is that if I [10:18] take some protein represented as like [10:20] you know these strings of 20 tokens like [10:22] hide a few and then can I train a really [10:24] big BERT style transformer to predict [10:27] mass positions as a function of the [10:28] other ones nearby. And the crucial part [10:30] is that we never tell it anything about [10:32] the protein beyond just the sequence. [10:34] Right? So all this guy's got access to [10:36] is just this string and it's being asked [10:38] to basically learn things about the [10:40] grammar of that protein as a function of [10:42] which protein other amino acids tend to [10:44] co-occur with right so like I think [10:46] there's like this old saying in natural [10:47] language processing it's like you'll [10:48] know a word by the company that it keeps [10:50] and here the idea is that you'll know a [10:52] protein by amino acids it keeps and the [10:54] bet is that if we do this at scale just [10:56] on the simple sequence task we will [10:57] eventually get all these sort of other [10:59] properties of protein say like structure [11:00] that we do care about sort of for [11:04] And uh yeah like I said before there's [11:06] been a lot of prior work on this like [11:08] largely from evolutionary scale but a [11:10] few other or a few other sort of groups [11:12] working largely on this bit. Uh this [11:13] table will be a map for the rest of the [11:14] talk. So like every row is a concept you [11:17] probably already know from natural [11:18] language and then analog onto the [11:19] protein context. So tokens become amino [11:21] acids. The internet becomes sort of all [11:23] evolution sequence databases all the [11:25] proteins we can actually go out and [11:26] measure. Mass token prediction stays as [11:28] mass token prediction and sort of [11:30] emerging capabilities. we talk about [11:31] language model having like become [11:33] emerging structure and function within [11:35] like basically understanding of a [11:36] protein and then there's also sort of [11:38] like this really fun stuff at the [11:40] bottom. So like recently there's been a [11:41] lot of advancements in sort of these [11:42] interpretability toolkits from the mechi [11:44] folks you know things like sparse [11:45] autoenccoders some of the earliest work [11:47] basically in really trying to [11:48] interrogate um using the toolkit the [11:50] language modeling community has built to [11:52] understand language models now in a [11:53] protein language model setting. So I'm [11:55] going to fill in the right hand column [11:57] for the rest of this talk with sort of [11:58] evidence and three questions which is [11:59] that do these models learn with scale? [12:01] Um can they basically substitute for a [12:04] lot of these handbuilt features sort of [12:05] does the bitter lesson hold and like [12:07] what do these representations actually [12:08] encode interpretably? So question one is [12:11] do scaling laws even hold in the protein [12:12] context in the way that we see them in [12:14] the language context. First let me just [12:15] try talk a little bit what we measure [12:17] when we're talking about sort of [12:18] emergent properties sort of how do we [12:19] actually like study the model see if [12:21] it's learning anything right we need a [12:23] proxy for does the model understand [12:25] protein structure for instance right um [12:26] the one the authors use in this paper is [12:28] that they look at the internal or they [12:30] basically take the model representations [12:32] during training and they use this to [12:34] predict um long distance protein [12:36] contacts so the idea here is that [12:37] proteins have a one-dimensional sequence [12:39] but they fold into complex [12:40] threedimensional shapes and if the model [12:43] is sort of understood something complex [12:44] about the protein structure or something [12:46] emerging about the protein structure. It [12:47] should be able to predict um contacts [12:48] that occur over long distances sort of [12:51] nearby contacts are rather kind of [12:52] obvious and this is like a really [12:54] challenging object for it to get just [12:56] sort of denovo purely from sequences [12:57] alone. They called this P at L right um [13:00] sort of a long contact precision at some [13:02] given length and it's just a clean [13:05] unsupervised readout sort of structural [13:06] knowledge inbuilt in the model that's [13:07] learned during this language modeling [13:08] objective. Um on the right I plot the [13:11] performance of this or the authors plot [13:12] the performance of this I should say [13:13] against training compute for the for [13:15] basically this new model family the [13:17] authors have built recently called the [13:18] ESM cranberry at 300 million 600 million [13:20] 6 billion parameter scales. uh [13:22] interestingly and they had this fit line [13:24] which is basically this predict compute [13:25] optimality curve which they um estimated [13:28] just from sort of lowend training runs. [13:30] So relatively low computational budget [13:32] and they find it actually extrapolates [13:34] very cleanly to real model training runs [13:36] meaning so the answer is like do these [13:37] models with scale and this data at least [13:39] suggests that the answer is like yes [13:40] right like you do see this nice log [13:42] linear curve right if you keep investing [13:44] more and more compute you training more [13:45] and more protein data with larger and [13:47] larger models um you see the same exact [13:49] same broad qualitative shape as the LM [13:50] scaling or sort of LM setting and arrest [13:54] retransfers cleanly meaning that without [13:56] any kind of like predisposed part of the [13:58] model that we've taught to look at [13:59] purching structure even didn't get any [14:01] protein structures. It does a good job [14:02] of sort of picking these out just from [14:03] sequence co-occurrence patterns. [14:06] Um there's like one interesting twist [14:08] though is that I said before there's [14:10] been a lot of prior work from this group [14:11] as well as others and trying to answer [14:12] these scale questions. So not the first [14:14] ones to look at this but previous models [14:16] um so the sort of the prior generation [14:17] ESM2 models shown in um purple here had [14:21] actually not shown the same behavior. [14:22] They sort of hit this wall where they [14:23] kept adding more parameters and they got [14:25] diminishing returns and you had this [14:26] sort of flattening out the scaling [14:28] curve. this ESMC or ESM Cambrian model [14:30] sort of the green line keeps climbing [14:31] with no plateau. And their fix for this [14:33] wasn't really like they came up with [14:34] like a really clever inductive bias in [14:36] the architecture. Not to say there isn't [14:37] a lot of excellent engineering work in [14:38] this paper, but really it was just data [14:39] scaling, right? They um had about 50 [14:42] million training samples in their [14:43] original ESM2 paper and here they just [14:44] pushed that to 2.8 billion by pulling [14:47] largely in metagenomic data. So [14:48] essentially amino acids or protein [14:50] sequences that have been found from [14:52] sequencing DNA actually out in like dirt [14:54] and oceans and like human guts from like [14:57] organisms that nobody has like really [14:58] ever cultured or even has really really [15:00] elucidated. And their conclusion is that [15:02] more data ends up being really important [15:04] and keep getting sort of uh are [15:06] basically justifying the cost for [15:08] increasing compute. So it's like the [15:10] protein version of LM data wall [15:11] conversation, right? Like except here in [15:13] biology, evolution has been generating [15:14] this train data for for four billion [15:16] years and not humans in like the past 30 [15:18] or so. And you know compared to tokens [15:20] in natural language like I mean we've [15:22] only sampled like less than 1% of all [15:24] known protein sequence diversity and [15:25] that's like only currently at this [15:27] moment in time let alone like all of the [15:28] sequence diversity evolution has sampled [15:30] since the beginning of life on Earth. [15:33] Um sort of question two in this paper I [15:35] think is interesting is that um it's [15:37] sort of the most bitter lesson part and [15:38] they really try to evaluate to what [15:40] extent their paper can do or how well [15:42] their model trained purely on mass [15:43] language modeling objectives can compete [15:45] against a structure based model with [15:46] sort of handtuned inductive components. [15:48] So I'm sure you're all familiar with [15:49] Alphaold won the Nobel Prize a few years [15:51] ago was sort of a landmark moment in bio [15:53] really show that these computationals [15:55] have a lot of value in the biology [15:56] sphere. Alfold is brilliant but its [15:58] power comes from basically building [16:00] handput in or handbuilt inputs sort of a [16:02] manual feature curation called a [16:04] multiple sequence alignment or an MSA. [16:06] So to fold a protein it goes and finds [16:08] hundreds of evolutionary cousins of that [16:09] protein and stacks them up. Um these [16:12] patterns of sort of coariation across a [16:14] family are essentially this encodesical [16:17] information you need to do to be able to [16:18] get structure. This is like a beautiful [16:20] domain engineering application and it's [16:22] the sort of like really good human [16:23] crafting objective bias that the bitter [16:25] lessons at least claims should [16:26] eventually lose right think like hog [16:28] features in CD and compared to sort of [16:30] things we used to do before this is [16:32] actually like far more bitter lesson [16:33] than like say building a whole physics [16:34] simulator for a protein but it's also [16:36] really slow to do this right we need to [16:38] build this huge databases the sequence [16:39] alignment it takes time right and it's [16:41] absent precisely where you often want it [16:43] for instance the antibody design task we [16:45] come back to at the end um ESM just [16:47] throw this away all it says is it just [16:49] takes input sequence and instead of an [16:50] alignment and it just feeds in the [16:52] model's representations as the input to [16:54] their structure predictor and these are [16:55] just like per residue embeddings. So [16:57] take your input sequence you get a set [16:58] of like per amino acid just like some [17:00] numerical representation and we just [17:02] train the specialized module to do [17:04] predicting the like large protein [17:06] structure right so this folding network [17:07] it's kind of like a projection into uh [17:09] 3D corded space so same target same [17:12] output no handbuilt features and the [17:14] question becomes can this general model [17:15] representation like match the sort of [17:17] specialist model in getting that MSA [17:19] value one interesting architectural note [17:22] though for sort of the more ML folks in [17:23] the crowd um the one there in their [17:25] projection networks for the part that [17:27] converts representation to structure. [17:28] There is actually one really interesting [17:29] feature that actually builds off some of [17:31] the work from our lab alum Dan Fu. Um [17:34] and they have a actually a looped model, [17:36] right? I mean there's a lot of [17:37] excitement about these recently for [17:38] parameter sharing and I just think [17:39] they're cool algorithmically for a [17:40] number of reasons. [17:42] And this is gives them basically a lever [17:44] by which they can scale inference time [17:45] compute, right? So essentially they have [17:47] a model that predicts structures and [17:48] they have a procedure by which [17:49] representations can be fed through a [17:51] series of layers and sort of refine [17:53] their structure predictions without [17:55] necessarily retraining or any kind of [17:56] fine tuning. This is like our test time [17:58] compute access and something like say [17:59] diffusion steps could be or like test [18:00] time sampling from LA lab. I'll keep [18:03] this in mind just for later results. And [18:05] the sort of like headline figure I would [18:07] say pointing out here is that um they [18:10] basically show that yeah their technique [18:11] works really well. Um just a quick [18:13] definitions on the left we have this [18:15] thing called DOCQ pass rate. This is [18:16] just a metric for how good your [18:18] structure prediction was. Essentially [18:19] it's a measure of the fraction of test [18:21] cases where the predicted shape of two [18:23] proteins stick that stick close together [18:24] is close enough to be like really useful [18:26] to realistic settings. And there are two [18:27] groups in each panel. One is for single [18:29] sequence with no MSA and the other is um [18:32] a single sequence plus an optional MSA [18:34] you can also feed to the model or is [18:36] required for competitor model. And when [18:37] we look at the sort of outcomes from [18:39] this, what we see is that for general [18:41] protein protein complexes, ESM fold 2, [18:42] their sort of new projection model from [18:44] a single sequence with no MSA lands [18:46] within about three points of alpha 3 [18:47] which does take these handcrafted [18:48] features. So we get near par without the [18:51] crutch and but if we look at the [18:53] antibody applications which is on the [18:55] right on the left hand side here right [18:57] um the modality but is like you know [18:59] essentially behind like all modern NC or [19:01] MAB based drugs or monol antibodies tons [19:03] and tons of applications in human [19:04] biology and biotech. um we are actually [19:06] winning or are comparably winning or the [19:08] authors are comparably winning. So [19:09] broadly like single sequence ESM fold 2 [19:11] does actually build alpha fold 3 sort of [19:13] 50 versus 47 on this really specific [19:15] design task that people really do care [19:16] about and biologically this makes a lot [19:18] of sense compared to say like um other [19:21] classes of proteins the amount of sort [19:22] of sequence variation that's been [19:24] sampled in the space of all known [19:25] antibodies relative to structure is [19:26] considerably smaller considering their [19:28] enormous diversity. So the headline [19:30] isn't that MSAs are dead yet, right? [19:32] It's that handle features only help [19:33] where it's abundant and basically where [19:36] drug designers really need it often does [19:38] go away. And this general method still [19:40] basically just save a lot from [19:41] pre-treating across all known [19:42] revolutionary contacts. [19:45] So we're not quite there yet. And one [19:47] other thing though worth flagging is [19:48] that a second point says give it MSA. We [19:51] can also scale the amount of test time [19:52] compute. So how many loops we run in [19:54] this recursive model in order to prove [19:55] performance. And we do see um basically [19:57] returns on this meaning that like the [19:58] better loss at least at inference time [20:00] also seems to broadly hold. And it's not [20:03] just accurate just as an aside. This is [20:04] just quick um it's also just much [20:06] faster. MSA construction just takes a [20:07] lot of classical computational biology [20:09] time. So at least if throughput is your [20:11] concern or latency is your concern, you [20:12] can with this single representation like [20:14] get quicker results. Though the wall [20:15] clock times here are like you know well [20:17] within like I would consider to be [20:19] pretty good to start off with. [20:21] Um, and the last bit, I'll get through [20:22] this a little bit quicker, is just they [20:24] did a really interesting analysis of [20:25] like sort of mechanistic [20:26] interpretability, like what are these [20:27] models actually learning and sort of can [20:28] we find features that are interpretable [20:30] as humans in the same way that sort of [20:31] language modeling folks in the Mechai [20:33] community have found in language models [20:35] like anthropic has. Um here they sort of [20:38] just apply the same tool or they borrow [20:40] a lot of the tools for like sparse [20:42] coding analysis here where they look at [20:44] activations from these models and try to [20:45] see if they can decouple them find these [20:47] like mono semantic activating directions [20:49] inside their feature spaces and they ask [20:52] is this also going to be a property in [20:53] protein models and their answer is [20:54] largely yes um right so from like pure [20:58] fill-in-the-blank pre-training the [20:59] model's latent space decomposes into [21:01] clean features that correspond to real [21:02] biological concepts here the these [21:04] concepts have been annotated by LM [21:05] agents And they're organized actually [21:07] quite interestingly in a nice hierarchy. [21:09] So you have like features that [21:10] correspond to say individual amino acids [21:12] at the bottom then like structural [21:14] motifs then like whole protein domains, [21:16] right? So look longer or larger portions [21:17] of the individual protein molecule up to [21:19] like functional sites and whole protein [21:21] roles, right? And none of this was [21:22] supervised, right? The model like [21:23] learned to organize its latent space [21:25] purely just through MLM, which is like [21:27] crazy. Um [21:30] I'll just with one example maybe to [21:32] close things out before I finish [21:33] everything and I think I'm actually have [21:34] one more slide after this. Um this is a [21:37] instance of a feature activation that [21:38] corresponds to a really specific [21:40] well-known protein motif called the [21:42] nucleophilic elbow. This is a type of [21:43] catalytic domain that's used in a lot of [21:45] enzyme catalysis. It's really [21:46] interesting because it's evolved [21:47] multiple times in multiple different [21:49] proteins unrelated to each other. So [21:50] it's a it's a vitif biology keeps coming [21:52] back to and the model has basically [21:54] learned to identify in the four quite [21:56] structurally diverse proteins from like [21:58] both evolutionary distance as well as [21:59] the rest of the protein. So it's like [22:01] found a consistently occurring motif in [22:03] very different backgrounds. So it's like [22:04] it's basically learn to look at the [22:05] right thing not just sort of memorizing [22:07] like you know broad similarly comparable [22:09] sequences. It's like a deeper level of [22:11] intuition. [22:12] And if you look at the sort of the whole [22:14] SE activation space, you can find like [22:16] nice structures that sort of correspond [22:18] to like various known aspects of [22:20] biology, right? This organization isn't [22:21] just local, it scales to all of life, [22:23] right? So they um ended up building [22:25] actually a huge atlas of their pro with [22:27] their model afterwards sort of just [22:28] folding and analyzing um millions of or [22:31] up to I think seven billion proteins. [22:32] This is the largest atlas I think out [22:34] there in alpha protein structure [22:35] databases, more than alpha folds even. [22:37] And they've predicted like you know O of [22:40] a billion of these as I mentioned before [22:41] and laid them out here by the [22:43] representations in SAPE space and you [22:44] get like a really nice interesting like [22:46] protein space family map right you can [22:48] find that there's clear families that'll [22:50] cross clear here are like for instance [22:51] crisper castine enzymes which if you're [22:53] not a biologist maybe you still probably [22:54] have heard of and really important for a [22:56] lot of biotechnology applications it's [22:58] kind of like a Google maps from proteins [22:59] and it's produced all as a byproduct of [23:01] the model right like just naturally it's [23:03] like picked up evolutionary relationship [23:04] as well as functional ones just denovo [23:06] for free which I think is like I don't [23:08] know if you're maybe not a protein nerd [23:10] like me I just think this is like [23:11] utterly crazy right [23:13] um so like just to finish like does a [23:15] bitter lesson scale to biology not [23:17] perfectly yet I mean some of this [23:19] analysis still requires a lot of [23:20] handcrafted features and it's not fully [23:21] competitive but we're getting very close [23:24] um but even if we just don't care about [23:25] one specific downstream the model just [23:27] from a relatively quite small amount of [23:29] or like a relatively quite simple [23:30] pre-training objective and a lot of data [23:31] has like learned an enormous amount of [23:33] bio that we can reverse interrogate [23:34] after the fact um and just for record [23:37] like they found that data scaling does [23:39] keep improving. Um I want to just point [23:41] out you know partially as a process like [23:44] our partially just like try to convert a [23:46] lot of smart people like we have in the [23:47] audience there's lots of folks work on [23:48] ML a lot of applications software um [23:50] biology is a great place to work in ML [23:52] because the models are still really [23:53] young and the other thing is that the [23:55] data is increasing exponentially per [23:56] year and that rate of increase is also [23:58] going up meaning that like we're not [23:59] data limited it's a great time to work [24:01] in this space and we need a lot of these [24:03] tools uh and any audience members [24:05] watching this on YouTube similar pitch [24:07] um just as one last thing um I didn't [24:09] get talking into detail but the one [24:11] application they use for their models [24:12] for inverse design. So they actually [24:13] develop a lot of potential protein drugs [24:16] and they validate a lot of use at least [24:17] in um wet lab settings to show that [24:19] these are potential like proteins that [24:21] you can design using this model purely [24:22] in sequence space for the most part by [24:24] the way um with the exception of like [24:26] one structure head at the end um that [24:28] bind various like known molecules that [24:30] have therapeutic effect right so for [24:32] instance uh this PDL1 binder is [24:34] basically the most or is like basically [24:36] a medication that is now the sort of big [24:38] success of amunotherapy it's helped [24:40] plenty of patients with cancers in ways [24:41] that historically have never been able [24:42] to tackle before, right? And developing [24:45] medications that sort of targeted this [24:46] protein was immensely challenging. And [24:48] like if we can basically reduce the [24:49] costs for developing such future drugs [24:51] for future targets, it would have [24:52] enormous human impact. So like even if [24:54] the data scale doesn't sell you, then [24:55] maybe some of the human impact will. But [24:57] broadly speaking, it's a really exciting [24:58] time and it's wonderful to see that a [25:00] lot of these lessons are at least [25:01] translating and people are really making [25:02] steady progress. [25:06] Okay, next we have Luke. um second year [25:10] PhD out of uh Tatsu and Tangu's lab uh [25:15] fresh from the UK. Then he went to [25:16] Harvard CS uh worked on adversarial [25:20] robustness and now post-training [25:22] selfplay and is directly uh uh in the [25:25] spirit of this um alpha zero kind of [25:28] mindset and so we've been chatting with [25:31] that about that a lot. All right, please [25:33] welcome Luke. [25:38] Okay. Hi everyone. Um, yeah, I'm Luke. [25:41] Um, I guess I'll be presenting on [25:44] this paper we put out uh a few months [25:46] ago called Scaling Selfplay with [25:47] Selfguidance. I guess more generally, [25:49] I'll be talking about selfplay for LMS. [25:52] Um, this work was with some great [25:54] co-authors, Caillou, Kan, and my two [25:56] advisers, Tatu and Tangu. [26:00] Okay, so um, what does the current [26:02] training stack look like for big LMS? [26:06] Two simple parts basically. We pre-train [26:08] the model on web text and then we [26:09] postrain it. And interestingly recently [26:13] the post- trainining we've ended up [26:14] spending you know a huge amount of [26:17] compute on doing large scale long [26:19] reinforcement learning runs. And what [26:21] does that reinforcement learning look [26:22] like? You collect a huge number of tasks [26:25] coding tasks maths tasks tasks [26:27] interacting with different bits of [26:29] software. And you just have the agent [26:32] take a bunch of actions in those [26:33] environments. you get some reward back [26:35] and we train the model on that data [26:37] upwaiting the good rollouts down waiting [26:39] the bad rollouts and like I said the [26:42] interesting change that's happened is [26:43] we're now approaching the amount or even [26:45] surpassing that we're spending on [26:46] pre-training actually on this very long [26:48] running RL post training and I've swept [26:52] some things under the rug that we do at [26:53] post training as well like a bit of [26:55] instruction tuning and and uh alignment [26:57] but really most of the compute spent on [26:58] these long RL runs [27:01] okay so we also know that as we increase [27:04] the number of uh RL tasks during post- [27:07] training and we increase the amount of [27:08] compute we get better downstream [27:10] performance and I think this is best [27:12] illustrated by this like really [27:13] beautiful plot from the composer 2 [27:15] technical report from cursor where what [27:18] they're both basically showing is they [27:19] have loads of RL tasks such that they [27:22] only ever the model only ever sees each [27:23] task once and so on the x-axis scaling [27:27] training step is basically each training [27:28] step I'm putting in some compute and a [27:30] new RL task and what they show is nice [27:33] smooth line as you increase the amount [27:35] of tasks and compute you put in, you get [27:37] this reliable improvement. [27:39] And I guess they had this nice eval set [27:41] on the left, but they also have a [27:42] downstream benchmark on actual coding on [27:44] the right. And that's also like [27:45] increasing reliably in a really nice [27:47] way. This recipe tells us great, just [27:49] collect more and more RL tasks, put them [27:51] in a loop, and model going to keep [27:52] getting better and better. But [27:54] generally, we're going to have to [27:55] collect these RL tasks by hand, which [27:57] might be a problem if you want to keep [27:58] on feeding. You'll notice log scale on [28:00] the x-axis there. And I guess there's [28:02] another problem where you might think [28:04] that um eventually we'd like the model [28:06] to surpass any of the problems we can [28:08] give it. [28:10] So I guess the question that Cell Play [28:12] asks is how can we automatically [28:13] generate new RL task to the model, train [28:15] on those and repeat. [28:18] Okay, so like I said in traditional RL, [28:20] we'll have a predefined task and we [28:22] train the model on that predefined [28:23] environment and task. [28:26] But in selfplay, we do something [28:27] slightly different where the model does [28:29] two things. It's going to generate RL [28:30] tasks and it's going to attempt to solve [28:32] those tasks. And crucially, we train it [28:35] to be better at both of these things. [28:36] So, we train it to be better at in [28:38] virtual commas, we'll go through what it [28:39] means to be better to generate tasks and [28:41] then also to get high reward in those [28:43] tasks. [28:46] So, how do we fit I guess some papers [28:49] we've likely seen from the past into [28:50] this description of selfplay? Because [28:52] you might be thinking, this doesn't look [28:53] exactly like what I thought of when I [28:54] read the alpha go alpha zero paper. So [28:57] those traditional works we'd call [28:58] symmetric selfplay. [29:00] And in this case uh let's say in alpha [29:03] go how you train the model is you have [29:05] the go agent and then you have the rules [29:07] of go and you have the go board. That's [29:08] great but that is nonrl environment I [29:10] can interact with. Like I need an [29:11] opponent to play against. And so this [29:14] generate RL task part. They have an [29:17] older version of the agent take the role [29:18] of the opponent. So in this case [29:20] generating the task because I just put [29:22] an older version of myself in there and [29:24] now I have a nice RL task. It's a go [29:26] board with an opponent. [29:28] So this would traditionally be called [29:29] symmetric selfplay because the model is [29:30] taking on the same ball twice, a go [29:32] player. [29:34] More recently, however, uh in the LM [29:36] space, there's been the rise of [29:38] asymmetric selfplay. This actually hails [29:40] from a lot of older work on control [29:42] problems and things like this. But [29:44] asymmetric selfplay, we instead more [29:46] generally just have a model that I will [29:47] call in this talk a conjecturer that [29:49] will just generate entire RL tasks for [29:52] the solver to then operate in. The [29:54] solver is the equivalent of the agent [29:56] here. So the conjecturer might come up [29:58] with a coding problem and then come up [29:59] with a bunch of unit tests and then [30:01] it'll go into that environment to do a [30:04] bunch of rollouts, get reward and train [30:06] on that. [30:10] Great. So so why do why am I excited [30:12] about selfplay? Why do I think you [30:13] should be excited about selfplay? So I [30:15] guess this first point is [30:17] the first point I have to go go in some [30:19] some depth. So so in principle nothing [30:20] bounds learning. And what do I mean by [30:22] that? So if I take a bunch of [30:23] demonstrations from humans and I train a [30:25] model on that, I think it's clear that [30:26] the model will never get better than [30:28] those demonstrations. [30:29] So the next step is okay, I'm going to [30:31] create a bunch of environments out of [30:33] the model learning those environments. [30:35] That's regular RL. We have two problems [30:37] there. One, if you ace all of the [30:38] environments, you'll never get any [30:40] better. Or the second problem is if I [30:42] can't even get any reward in those [30:43] environments, I will also never get any [30:44] better. So selfplay on the other hand is [30:47] going to say I'm going to keep on [30:48] generating new learning signal with new [30:50] tasks. learn it and just keep on [30:51] improving hopefully forever. [30:54] And indeed, we saw this was the case [30:55] with two-player games like Go. It just [30:58] kept on getting better beyond human uh [31:00] performance and kept improving. So, the [31:02] promise for LM is I can take some I can [31:04] train on a bunch of human data. I get to [31:06] like human level and then I can run [31:08] loads of selfplay and go far beyond that [31:10] and hopefully solve really interesting [31:11] problems with with our models. [31:14] But unfortunately, this is not how it [31:15] works. [31:17] So in practice if I run which we'll get [31:19] into this talk if I run selfplayer for a [31:21] long time it plateaus I the model stops [31:23] improving at some point which is the [31:24] exact same that happens when you run RL [31:26] like as much as I'm trying to tell you [31:28] there's a bunch of secret source going [31:29] on like it doesn't actually play out. [31:33] So basically this paper we try to figure [31:35] out like why is this happening and then [31:37] like do one step to solving the problem [31:39] but by no mean by no means completely [31:41] solve it. Okay. So to begin with we need [31:44] to understand like the baseline LM [31:46] selfplay algorithm pretty simple we're [31:49] going to sample synthetic tasks from the [31:50] conjecturer which is just our model [31:53] conjecture and solver same model just [31:54] given it two different names the model [31:56] will then the solver then attempts them [31:58] and we verify the correctness using [32:01] some reward signal somehow like perhaps [32:02] the conjectur wrote unit tests for us to [32:05] check and then we're going to update the [32:07] solver just on all the correct rollouts [32:10] and then this is the key part the [32:12] conjecturer gets updated ated on this [32:14] reward which is zero if the prover if [32:17] the solver could not solve the problem [32:18] and one minus the solver rate otherwise [32:21] okay what is that actually doing that is [32:23] basically saying all the conjecturer [32:25] must do is produce problems that are [32:26] hard for the solver model and I think in [32:29] principle that makes a lot of sense the [32:31] idea is if the conjecturer can ace this [32:33] I will keep on giving you problems at [32:35] the frontier of your capability you will [32:37] be able to solve them and learn from [32:38] them and we'll just keep on expanding [32:39] and expanding expanding and get better [32:41] and better and [32:44] Okay, so let's see how this recipe does. [32:47] So we take in our paper like 3,000 [32:50] formal math problems. So this is just uh [32:52] in lean for you can write out the [32:54] problem statement in this coding [32:55] language in math. So you write out a [32:57] math problem in this coding language. [32:59] You can write the proof in the coding [33:00] language. Then you can automatically [33:01] verify if it's correct. So we take 3,000 [33:04] problems and we run like our best RL [33:05] baseline on it. And this is the amount [33:07] of compute we put in here. And on the y [33:09] axis we have how many problems you [33:10] solve. And you can see it plateaus out [33:13] and we fit a law and it asmmptotes at [33:15] like 60%. [33:17] And if we on the right hand side we're [33:18] gonna say how much synthetic new task [33:20] did we generate. RL generates no [33:22] synthetic task. So by construction this [33:24] stays at zero. Now I'm going to fill in [33:27] the vanilla selfplayer with that solver [33:28] rate reward. And I'm not going to show [33:30] the left for now. We see as time goes on [33:32] the conjecture gets better and better at [33:34] its job. It keeps on generating more and [33:36] more tasks on the frontier of the [33:37] solver's capabilities which seems really [33:39] good and yet these tasks are completely [33:41] useless. The cell play does no better [33:43] than regular RL. [33:45] So this is not very promising. So now we [33:47] need to understand why. And here what [33:49] I'm visualizing or I'm literally showing [33:51] you is one of the problems the [33:52] conjecture generates late on in [33:55] training. And we don't really understand [33:57] this. I've highlighted in blue that the [33:59] conclusion to the statement in lean. And [34:01] if anyone's using that this is horrific. [34:02] This is an incredibly complicated, [34:04] overly complex disaster of a statement. [34:06] And so what is basically happening is we [34:08] reward the conjecture for producing [34:10] tricky problems. But the easiest way to [34:12] produce tricky problems is produce these [34:14] basically messy, artificially complex [34:16] and elegant problems. It is the exact [34:18] equivalent of if I wanted you to get [34:20] like 50% solve rate problem, I could [34:22] just give you like a three-page long [34:24] high school calculus problem and you [34:25] would make some little mistake [34:26] somewhere. But that was a completely [34:28] useless synthetic problem for like other [34:31] tasks we care about in maths for [34:32] example. [34:34] Great. So how do we fix this in a minute [34:38] because I've been talking too slowly. [34:40] So we've diagnosed this problem. Here is [34:42] like roughly at a high level how we try [34:45] attempt to solve it. So there are two [34:48] parts of our algorithm SGS self-guided [34:49] selfplay. We're going to take the set of [34:52] problems we cannot solve the 3,000 [34:53] problems and we do two things. one for [34:56] each of those problems we cannot solve [34:57] we're going to get the conjecturer to [34:58] produce a related problem to it. So once [35:00] you prompt it to produce a synthetic [35:01] problem that is related. So this way [35:03] we're trying to ground the synthetic [35:05] data distribution in a distribution of [35:07] problems that we think is good at least. [35:08] And next if you still just trained on [35:10] the solvent rate reward you would [35:12] eventually ignore this prior and still [35:14] produce that junk. So we're going to [35:15] introduce a new reward signal which is [35:17] the model takes on a third role and it [35:19] will literally judge it looks at [35:20] synthetic problem and the target problem [35:22] it came from and decide if these two [35:24] things are actually related and not [35:27] overly complex. So we call this third [35:29] component a guide. [35:31] Okay. So the algorithm looks like this. [35:34] It's very similar. We for every target [35:37] problem we haven't solved we'll sample a [35:39] conjecture uh from the conjecture that [35:41] is related to it. We then will attempt [35:42] to solve them. And then what changed [35:44] here is when we update the conjecturer, [35:46] we now have this dual reward. One, we [35:47] still want the problem to be tricky. [35:49] That is important. So we can get RL [35:50] signal on it for the solder. And we'll [35:52] multiply it by this guide score. Great. [35:54] Okay. There's a bunch of kind of [35:55] subtleties we cover in the paper that I [35:57] will skip over because we don't have [35:58] loads of time. If you want to talk about [36:02] I'm going to say largeish scale RL [36:04] infra, the academic size. That's what I [36:06] spent most of my time doing. So I would [36:08] like to talk about that, but there is [36:10] not time. So let's just look at the head [36:13] headline results here. Here is basically [36:16] the same type of plot. I've put the RL [36:18] baseline on here. Recall that like [36:20] standard selfplay is exactly in line [36:22] with that. We've also put parallel [36:23] sampling down here just to show you that [36:25] indeed RL at least gives you a boost. [36:27] And I guess I wouldn't be here unless [36:29] our method works better. So the method [36:31] does work better. Um like ground how [36:34] much better it's doing. We we we were [36:35] using a 7 billion parameter model here [36:37] and this is it like 670B like big [36:40] brother and we spend eight times as much [36:44] compute doing the selfplay at this you [36:47] yeah we do eight times compute the [36:48] selfplay but we get like to the ability [36:50] of that larger model at least it's pass [36:52] up for ability so you spend a lot more [36:54] compute but we are able to get this like [36:55] little 7B guy to do as well as the [36:58] bigger model but very sadly you will [37:01] notice like this is not at 100%. So like [37:03] the work is is by far not done. The [37:04] problems itself plays like you would [37:05] just ate all the problems here and so [37:07] there's a bunch of well there's lots of [37:10] future work but luckily a PhD is very [37:12] long so I'll be able to work on that. [37:15] Um but yeah that's the summary. [37:21] Awesome. Thank you so much Luke Bailey. [37:24] Okay uh next one we have Arnab Matei. Is [37:27] that the right way to say it? Matei. um [37:29] who is a researcher currently at Giga [37:32] one of YC's fastest growing companies I [37:34] think market cap is like 400 million 300 [37:36] million something like that now so [37:38] really fast growing YC company uh PhD [37:41] University of Washington focus on bandit [37:43] learning um yeah please let's tell us [37:46] about stream rag [37:51] >> so um there's a paper by the group at [37:56] meta and I kind of chose chose this [37:59] paper to kind of maybe highlight some of [38:02] the new emerging challenges that are [38:05] coming up especially in a voice AI kind [38:08] of setup. Um my goal with this talk is [38:13] not more like about talking specific [38:16] details about this paper but more like [38:18] to highlight the good problems that they [38:21] have identified and I feel like there's [38:24] a lot of research that is to be done [38:26] here and it also kind of closely mirrors [38:29] what at least I do in my production uh [38:32] setup like I look at these kind of [38:34] problems I do the research and then I [38:36] try to come up with a method that will [38:38] work probably in production. So yeah, [38:41] let's get started. This is a very [38:43] classical setup where uh you probably [38:47] ask a input question to an alm and it [38:49] gives an output. And if you remember [38:51] maybe from 2023 maybe there was a lot of [38:55] hallucinations but especially like say [38:57] around citations and all but over time [39:00] maybe the hallucinations went down and a [39:03] big role [39:05] was uh rag like you kind of give the [39:10] input query to a rag system. it kind of [39:12] goes and figures out relevant [39:14] information that needs to be provided to [39:16] an LLM and then the LM probably gives [39:19] you an output which is hopefully not [39:21] hallucinated. [39:25] Now uh a lot of voice AI uh startups are [39:29] also coming up and [39:32] a natural expectation with the voice AI [39:34] is that okay you're having like a [39:36] conversation like oh you can ask oh [39:38] what's the weather like and the agent [39:41] would reply like hey the weather [39:42] currently is like 22°C [39:45] and so on and maybe you can ask a [39:46] follow-up question so it's more like [39:48] conversational in nature [39:50] and so even here as well you would like [39:53] the output to We like there shouldn't be [39:56] any hallucination. Especially in voice, [39:58] we care about this even more because [40:01] from a human perspective, it's difficult [40:04] to kind of actively catch hallucinations [40:06] when you're listening to it compared to [40:08] like when you're reading it over text. [40:12] So one might ask, okay, what's the issue [40:15] with just using rag here? like can't you [40:17] just take the input query take apply rag [40:21] give the relevant information to the [40:23] voice agent and get the output the issue [40:26] is that rag would add a lot of latency [40:31] um like for example if I ask a voice [40:33] agent some question and the voice agent [40:36] takes 10 seconds to reply that's not at [40:39] all natural especially if you want to [40:40] have some sort of natural conversation [40:42] so that's where this paper kind of looks [40:45] at A very clever idea I would say uh [40:49] which is like instead of like waiting [40:53] for the question to end and then [40:55] activate your rag pipeline you kind of [40:59] start analyzing the words that are being [41:02] spoken by the user and somehow figure [41:05] out a way to [41:08] run the rag system while the question is [41:11] being spoken. Like for example uh like [41:15] you might ask like hey what's the [41:16] weather today like I'm I want to decide [41:19] based on that whether I want to go out [41:21] or not. The main question is in the [41:23] first part. So the second part of your [41:25] question might be irrelevant. So we want [41:29] um some sort of uh mechanism via which [41:32] we can figure out okay uh when to call [41:36] this rack system and appropriately get [41:39] the right uh information. [41:43] So this particular paper focuses on two [41:46] approaches. Uh the first one is fairly [41:48] simple. Um so it's called fixed rag uh [41:52] fixed interval streaming rag. So the [41:56] idea is like you divide the audio into [42:00] certain blocks and after each block [42:04] arrives you can like run rag on every [42:07] block. So [42:10] uh so after when the block B arrives you [42:12] run your rag get the results for the rag [42:15] RB and you keep on doing till this uh [42:18] till the end probably. Now the question [42:23] the main question here is like which [42:25] block to consider because you ideally [42:28] cannot like wait the entire goal was you [42:30] cannot wait till the end and then run [42:33] the rag. So what do you do? So the main [42:37] uh maybe idea here is that rack pipeline [42:41] has lot of mini components. So maybe [42:44] some of the components are like easy to [42:47] run or like more faster to run. So for [42:50] example uh you can kind of get some [42:53] documents very quickly and you can say [42:56] okay for the entire query what were the [42:58] top documents and for the intermediate [43:01] query what were the top documents and [43:02] are they matching or not? This is just [43:04] one of the ideas which is from the [43:06] paper. Uh and then based on that you can [43:09] decide okay should I go ahead with the [43:10] intermediate query and uh just do the [43:13] entire rack pipeline on that. Um so the [43:18] thing I want to stress is not the method [43:20] per se but the point that okay when you [43:23] are getting this uh input in chunks at [43:28] what point can you stop and say that [43:31] okay like this chunk is like super [43:33] relevant for me. Uh so this is like an [43:37] active question I would say like how [43:38] would you do that? This uh paper does it [43:40] in a very simple manner which is just to [43:43] maybe look at the initial path of the [43:44] rack pipeline and if they kind of match [43:48] like if the end path matches the [43:49] intermediate part then you go ahead with [43:51] the intermediate and let the full rack [43:53] pipeline complete. Another approach [43:55] could be like you you can probably [43:57] fine-tune a model to kind of trigger on [44:01] its own like when to call the rack [44:03] because in the previous approach you [44:05] were calling rag on every single chunk. [44:08] So maybe that's computationally [44:10] wasteful. So what you can probably do [44:15] is when a particular chunk arrives you [44:18] can maybe fine-tune some model and ask [44:22] it to decide whether [44:24] uh this chunk uh is like in critical new [44:29] information and you should generate a [44:32] new query or the query that you [44:34] generated based on the past chunks are [44:36] good enough for you to just answer the [44:39] question. And uh based on that you can [44:42] generate the final audio. [44:44] Yeah. So in in the paper they kind of [44:46] describe a post- training pipeline. What [44:48] they do is like they kind of uh for the [44:51] partial uh spoken uh question they kind [44:55] of generate some pseudo queries using [44:57] some LLM and then uh they run a rag on [45:00] that and they look at the retrieved [45:02] documents [45:04] and based on the retrieved documents [45:07] they kind of decide okay is this uh [45:10] partial query like uh something new or [45:15] is it already like we already have the [45:17] useful material. [45:18] So in this okay um in this paper [45:22] essentially they are kind of basing [45:24] their decision based on the retrieval [45:27] quality of the partial question so far. [45:30] That's that would be my takeaway. But [45:32] maybe there are different ways in which [45:34] you can do this assessment. Maybe you [45:35] can look into the semantic of the [45:37] question so far like is the partial [45:39] question so far good enough for me to [45:41] answer this question just by looking at [45:43] the question. No no no need to do this [45:45] entire lag pipeline. So there are my my [45:47] point is like you need not uh this need [45:50] not be the only way there might be so [45:53] many different ways and that's where the [45:55] research maybe uh is required like while [46:00] a user is speaking their question how do [46:03] we like on like why instead of waiting [46:06] till the end how do we like figure out [46:08] okay this part of that question is good [46:11] enough for us to go and do the retrieval [46:13] yeah so that that's what they do [46:15] probably I'll [46:16] quickly give a glimpse of the results [46:18] from the paper. Um [46:21] this paper is a year old. So they were [46:23] like yeah looking at some smaller open [46:26] source models. Um so they were they kind [46:30] of considered the rag benchmark [46:32] converted into audio and uh showed that [46:35] the latency kind of decreases for the [46:38] synthetic data sets by 0.5 seconds and [46:40] for human data sets like human spoken [46:43] data sets by uh almost like 1.5 seconds [46:48] and uh the accuracy [46:51] uh comparison like uh if there was rag [46:55] uh after the final query and streaming [46:57] rags it kind of remains the same like um [47:01] yeah so yeah so that's what the paper is [47:04] about so like the key takeaway is like [47:06] there are some interesting small [47:09] problems here like but if you can crack [47:11] the small problems it can lead to huge [47:13] gains in the production yeah thank you [47:23] okay next up we have Robert George. Um, [47:28] come on up. Uh, thirdyear PhD at [47:30] Caltech. [47:31] >> Yes. [47:31] >> Okay. My brother got his PhD at Caltech. [47:34] >> Um, and, uh, you work on AI for math and [47:37] science. [47:38] >> Yeah. [47:38] >> Um, and what are you going to tell us [47:39] about? [47:40] >> I'm going to tell us about lean. [47:42] Basically, Luke already told a little [47:45] bit, but I want to go more in depth. So, [47:48] I'm going to be talking about lean and [47:50] what I think is this new era of verified [47:53] intelligence. Um so let's get into it. [47:56] So again there's bunch of breakthroughs [47:59] in the past like couple of weeks itself [48:01] like first I want to go back like two [48:03] years before you know like we said that [48:06] IMO open and even deepine actually at [48:10] the 2024 IMO got the gold medal then you [48:13] know there's this very famous problem [48:17] list which is very famous right now [48:18] where people are trying to kind of solve [48:20] new open unsolved odos problems and you [48:23] know you can see that it's keep on keep [48:24] on increasing with the new models from [48:27] like open AAI depend and all. Um just [48:30] two weeks ago OpenAI claimed to solve [48:32] another big breakthrough 80-year-old [48:34] Odosh problems. You know Terry Tower was [48:36] has this promotional video at OpenAI [48:39] which he showed really well about these [48:40] kind of things. And then last week [48:42] Deepmind released something also solving [48:45] bunch of new not only ODOS problems but [48:47] problems in like other different fields [48:49] right but this paper is cool because [48:51] they also use some kind of formal [48:52] verification in the loop. So I want to [48:55] say that you know we all took like high [48:57] school calculus we took undergrad [48:58] college math courses and all this you [49:00] know informal math is very very flexible [49:02] right um your your professor say [49:05] sometimes you know proof by QED like you [49:07] know sometimes it's like proof by [49:09] intimidation or something that right [49:11] there many of the steps are not fully [49:12] written down but this is where I believe [49:14] that you know formal world is like you [49:17] have to be fully explicit right and I'll [49:19] talk and introduce the language lean [49:21] again before lean In past couple of [49:25] hundred years, you know, people have [49:26] been doing formal math a lot, but you [49:28] know, lean has just kind of this really [49:31] good design language just kind of taken [49:33] off, right? So again, first thing is [49:36] it's very easy to check if a proof is [49:38] correct or not. You cannot fool this [49:40] theorem prover. Secondly, uh it's [49:43] scalable. Again, there's bunch of issues [49:45] over there, but I can talk more about [49:47] that soon. So before that I just want to [49:49] give you like a precursor. So people do [49:51] know about like um there was a thing [49:54] previously like in the 1990s even right [49:57] now actually 2020s and all this is [49:59] there's this thing called automatic [50:00] theorem provers which are basically like [50:01] SMT solvers um they are basically um [50:06] minimal effort from humans you know but [50:08] they're very limited expressivity in [50:10] what type of mathematics they can encode [50:12] in some sense right and on the far right [50:14] hand side you can see interactive [50:15] theorem provers like lean rogue Isabel [50:18] which are very have a much more stricter [50:21] like expressive logic system. So it's [50:23] based at least some of them are based on [50:24] dependent type theory but much more [50:27] effort from humans to kind of write down [50:29] these proofs right like if you're [50:30] talking about like 10 years people have [50:32] been contributing to this very famous [50:33] library called math lab in lean um [50:36] there's a lot of human effort to kind of [50:37] pick premises and all this and again we [50:41] all know how good LLMs are right now at [50:43] kind of combining with these kind of [50:45] theorem provers to kind of do proof [50:47] checking for like research level math [50:49] right and it's so much news that I you [50:52] know if If I go on Twitter right now, I [50:54] can open up a bunch of posts saying how [50:56] much progress past couple of hour [50:58] probably in some sense right so first [51:00] thing is I want to introduce why leen [51:02] you know Luke mentioned this formal very [51:04] messy language but I actually think it's [51:06] a very beautiful language again one can [51:08] argue no but um it's a very fast [51:11] language again it's also people think of [51:13] it as only a theorem prover but it's [51:15] actually also a functional programming [51:17] language right you can use it as a [51:19] programming language itself so it's [51:20] compile checking Um it's very good [51:23] unified. So this is what like the proofs [51:25] and programs. Um you can do like meta [51:27] programming, you can do macros, custom [51:29] automation, you know, you can I've seen [51:31] people trying to even create like games [51:34] on with using lean, right? It's actually [51:36] super cool. So lean has something called [51:38] the foreign face interface where you can [51:39] do like external library bindings like [51:41] you can do on the CUDA or something that [51:43] um I want to point out the math liy. I [51:46] think that is the coolest biggest [51:48] formalized math library out there. um I [51:51] forgot how many number of lines probably [51:53] at least in a million or so but all of [51:55] these are really high quality math right [51:57] from like say topology to algebraic [51:59] geometry and all this and again it's an [52:01] interactive theorem prover so you always [52:02] have to you know the human can sometimes [52:04] be in a loop but it is also a very [52:08] scalable language because you know not [52:10] only frontier labs are pushing a lot of [52:12] money into it and also the world is but [52:14] um there's more data being generated [52:17] either through synthetic or like a lot [52:19] of people like even myself I do manual [52:20] formalizations. Um so just very short I [52:23] don't want to take time but this is how [52:24] a simple lean code looks like like in VS [52:26] code you have like an info goal view [52:29] which shows like what are the current [52:30] kind of sub goals. So goal is basically [52:32] like what are you trying to prove at [52:34] this step. So the first theorem is like [52:36] you're basically showing associivity of [52:38] addition of like natural numbers right [52:39] like a plus b plus c is equal to a plus [52:42] c plus b and each line in a proof is [52:44] usually called like a tactic. So usually [52:48] when people talk about like proof search [52:49] they mean like you know you can search [52:51] over this kind of tactic space there are [52:53] methods where you do foolproof [52:54] generation but you know these are the [52:56] two different axis. So this is how lean [52:58] code looks. It's not as bad as it seems. [52:59] It's a steep learning curve. I think [53:01] it's much better than even C++ in some [53:03] sense like learning but um you at least [53:06] get really um at least for me I get very [53:09] happy when I see oh I've fully proven [53:10] this theorem right there's no [53:12] assumptions like I cannot like handwave [53:14] or fool the lean kernel basically like [53:16] you have to be fully 100% sure um now I [53:19] want to talk about the formalization [53:20] breakthroughs right I talked about [53:21] informal but actually the first book was [53:24] actually in 2020 Ilia and uh Stan was [53:27] from open they released something called [53:29] GPDF um this was first generative [53:31] language model for automatic theorem [53:32] proving mini F2F is just like a Olympia [53:36] level kind of competition but you see [53:37] the amount of progress like it's kind of [53:39] exponential right like from open source [53:41] models big players in China in the US [53:44] Canada like across the entire world um [53:47] last year's IMO you know again deep mind [53:50] claimed to not have used lean I if you [53:53] see the open air solutions some kind of [53:54] DSL of lean kind of stuff in the [53:56] solutions um but even Steve prover from [53:58] China also got the IMO gold And then [54:00] obviously there's a bunch of like axi [54:02] improver there's harmonic AI like they [54:04] got recently in the pakam they got all [54:06] the 12 problems solved [54:09] most of the odos problems now when [54:10] people are saying they kind of um claim [54:12] to have a solution using AI they also [54:15] prove it um using like say Aristotle [54:17] from harmonic um and then another [54:20] amazing work was kind of this fields [54:22] metal work from math inc and obviously [54:24] the Google Google deep mind stuff in [54:26] some sense right and again I love the [54:29] fact that you know everyone's is talking [54:30] about math and all but you know for me [54:32] personally there's also these two other [54:34] bubbles right like there's also code now [54:36] one can argue what is program [54:38] verification as well you know bugs are [54:39] really expensive it's like a huge [54:41] trillion dollar industry wide coding is [54:44] all of a sudden really great like [54:45] everyone is generating but we I want [54:47] code that needs guarantees right I think [54:49] that's like something which I'm very [54:51] interested in and also AI for science [54:53] matters like there's uh repro uh [54:55] reproducibility and all this kind of [54:57] stuff so I want to go through this [55:00] really fast but um LMS can write code [55:03] but can they prove it's correct um you [55:05] know there's scale of generated code [55:07] there's that of bugs uh how can you kind [55:10] of capture human intent and the [55:11] verification language and again in short [55:14] I want to talk about like program [55:15] verification is like there's these three [55:17] concepts where humans actually always [55:19] have some kind of like specification [55:21] about like what they want their code to [55:22] do so a proof is basically saying that [55:24] the code kind of satisfies that [55:26] specification um there's this work which [55:28] I introduced called bridge where you can [55:30] use this lean as a functions programming [55:32] language to kind of elicitate the llms [55:35] to kind of prove this kind of code [55:37] better. Um so I like this code from max [55:40] tagm where they say that we should shift [55:42] from actually wide coding to like very [55:44] coding right. Um verifiable coding will [55:47] be like definitely I think a much more [55:49] better way. Um and you should contribute [55:51] to CS lab. This is started from Clark [55:53] Barry's group at Stanford. There's bunch [55:54] of from deep mind and all. But if you [55:56] want to contribute to CS concepts and [55:57] all, you should definitely contribute [55:58] with CSL. Um I want to go through [56:00] quickly just about one last work about [56:03] uh torch which I recently introduced. [56:05] This is the first unified framework for [56:07] actually writing down neural networks in [56:09] lean. So you have this kind of full like [56:11] pytor style like tensor system. [56:14] Everything compiles down to a shared [56:16] intermediate representation. You can [56:17] kind of prove properties of specs like I [56:19] can show you some examples. You have [56:21] like verified floatingoint arithmetic. [56:23] you can kind of do even like neural [56:25] network verification like certified [56:26] robustness kind of stuff right and again [56:28] there's bunch of applications which I [56:29] show but I think one cool thing that [56:31] I'll show this and the next slide is [56:33] that you know you can show that the [56:34] flash attention is equal to like at [56:36] least in the spec level is equal to the [56:39] uh normal standard attention right again [56:41] we don't worry about like IO and all [56:43] this processing also you can a very [56:45] standard fact is like the attention [56:46] mechanism is permutation in if you don't [56:48] have position like curtains so I [56:50] actually kind of trained a GP2 style [56:52] like Karpathi's thing in torching itself [56:54] fully natively in lean right and you can [56:56] kind of prove properties about it and [56:58] all this um one thing I think I can end [57:01] with this slide is that thinking machine [57:03] lab last year released something about [57:05] um this kind of non-determinism even [57:08] when you have like temperature zero um [57:11] when you put it into your LM inference I [57:13] actually kind of formalize this whole [57:15] system in torch lean all the way down to [57:18] like almost a GPU kind of like small [57:20] cuda level kernel verification because [57:22] the whole goal in this blog was saying [57:23] that the tiny floatingoint arithmetics [57:27] can flip the final argmax in the kind of [57:29] the batch thing. So again there's a blog [57:31] you can check it out on my website but [57:33] uh I was very very cool that you can [57:35] kind of do real life software [57:37] verification [57:39] um in some sense and uh again there's a [57:42] bunch of different slides I have but I [57:43] kind of want to end on this note just [57:44] for the sake of time but you know I see [57:47] a future where uh science like even code [57:51] can be formally verified through a lot [57:52] of building blocks which people are [57:54] putting a lot of effort in and this is [57:56] one of the examples that I think is like [57:57] my fuse matter like kind of contribution [57:59] to the ammo wall in some sense. [58:05] >> All right, great job. Okay, [58:09] for our last presentation, it's going to [58:12] be the antithesis of lean [58:17] and token maxing to the max. Um, very [58:21] excited uh to introduce Luke Orthwine, [58:25] his close friend. um we're friends in in [58:28] uh in Woodside together. Um and did his [58:32] uh CS degree at Harvard, then ran growth [58:35] at WeChatad from 2012 till 2015. Uh [58:39] which is why we call him the lion of [58:41] Hong Kong. [58:43] Um and now has been running his startup [58:45] channel AI and is probably the most [58:48] unhinged technical CEO that I know. So [58:52] >> thank you Francois. [58:56] Um yeah so the the idea behind this talk [59:00] is sort of um what we uh at channel have [59:05] done to try to take the the best [59:07] advantage of sort of rethinking how you [59:10] should do software engineering in this [59:12] world of agentic programming assistance [59:15] cla etc. Um and really you know the the [59:20] ways in which uh I think many [59:23] assumptions about what good programming [59:24] is are now sort of the opposite of what [59:26] you should be doing. Uh and these are [59:28] sort of what we have have worked through [59:30] ourselves and found very useful and [59:32] wanted to share with all you guys to [59:33] give some context. channel AI. We're a [59:36] consumer entertainment uh AI business. [59:39] Uh and we're really focused on the [59:41] problem of automating as much as [59:44] possible of not just software [59:46] development but content development. How [59:48] do you really create like an endto-end [59:50] system uh that is pure AI that uh gets [59:53] people to pay you money uh and stay [59:55] engaged etc. Uh we've had pretty solid [59:58] success with that so far. Um, and it's [60:01] inspired us to think in our own [60:02] workflows, how can we just sort of max [60:05] this and and be as far ahead of the [60:07] curve as possible. Um, and chess is an [60:10] imperfect analogy to what programming [60:12] used to be like, but I think the ways [60:13] that it uh is useful is like yeah, maybe [60:18] programming before you wanted to be very [60:19] linear. You wanted to predict the [60:20] future. You wanted to design very [60:22] thoughtfully systems that would be like [60:24] robust and work well uh and and be [60:27] correct. Um, and even if you're trying [60:29] to do something sloppily, it's still [60:31] like a single threaded process where you [60:33] only are worrying at a given moment [60:35] about what's in front of you. Um, and to [60:39] me, I'm a big fan of real-time strategy [60:41] games using Agentic systems. Feels [60:44] exactly like playing real-time strategy [60:46] games to me. Uh, and there are a lot of [60:48] properties of those games that are very [60:49] different from chess. Um, one thing and [60:52] especially if you look at like highle [60:53] play uh there is no single aspect that [60:57] you can do perfectly and like succeed. [60:59] You have to be balancing many different [61:01] things at once. You have to always have [61:03] your economy running, your production [61:05] running, your units doing something [61:06] productive. You need to be engaging. And [61:08] so this notion of like how do you [61:11] maximally parallelize both what your [61:12] systems are doing but also your [61:14] attention so that you are adding the [61:16] corrective [61:17] uh feedback that's necessary as you [61:20] learn new things as the map is exposed [61:21] all this kind of stuff. Um anyway this [61:24] to me feels like exactly what like [61:26] coding with agents is like um and this [61:29] what we'll talk about. Um so in terms of [61:33] like tools we've built just to like [61:35] ground this in a very simple thing. This [61:38] is the LW stuff is just like our linear [61:40] work trees. Um, a lot of people early on [61:42] started using realizing how useful git [61:44] work trees are when you do coding [61:46] development. Having separate uh I assume [61:50] everybody kind of knows where they are, [61:51] but in case not like you know it was [61:54] fine to have one repo on your machine [61:55] when you were the only one doing [61:56] development. Now you need to have like [61:58] lots and lots of repos on your machine [62:00] all doing development in parallel. Uh [62:02] all compiling separately and like not [62:04] stepping on each other's toes. Um and so [62:08] the combination of like uh using work [62:10] trees, using task management software, [62:13] uh having the actual work itself be [62:16] portable, um which is what the team bit [62:19] comes in, and then like sticking in [62:21] autonomous agents, one or many different [62:23] ones on a given workflow. Um the way [62:26] that we basically ship stuff, the way I [62:28] ship stuff, uh is I have an orchestrator [62:31] agent that's run by Claude usually, but [62:33] could be codeex 2. Uh, I try to have as [62:37] minimal a number of keystrokes as [62:39] possible to go from like here's an idea [62:41] of something that needs to be fixed to [62:43] work being started on it because I can [62:45] course correct that work later. Think [62:47] like grabbing a unit and just like [62:48] clicking across the map and you'll come [62:50] back later to like make it work [62:52] effectively. Um, status tracking, [62:56] watching your mini map, it's the RTS [62:57] equivalent uh from the orchestrator of [63:00] all the different uh spawned workers [63:02] that you have working. Uh, and then all [63:04] those workers being instructed basically [63:06] to try to go as far as they can, really [63:09] put like a really low premium on their [63:11] time and effort and a high premium on [63:13] yours. So even if they're going to be [63:16] wrong, even if they're going to need to [63:18] be corrected later, it's better for them [63:19] to push as far as they can before they [63:21] ask for feedback. Uh, so that you can [63:25] just have a lot of them running in [63:26] parallel, even if it's wasteful from [63:28] like a per per token standpoint. it's [63:30] like saving you a lot of time or letting [63:32] you do more things at once. Anyway, so [63:34] they try and take everything all the way [63:36] to a PR uh not just a PR but also like a [63:40] summary that's well I'll get into that [63:41] later anyway. So uh uh and then like how [63:44] do you take each the results of every [63:46] worker who completes something and like [63:48] feed it back into the system so that the [63:49] system learns and becomes better again [63:51] like without the human having to type a [63:53] lot of things or doing minimal work so [63:54] they can do a lot of these things at [63:56] once. Uh and then other pieces like how [63:58] do you tag in other teammates? we'll [63:59] also get into. Um, but anyway, this is [64:02] very much like an RTS where you're like [64:04] producing units, trying to move them [64:05] around, trying to constantly adapt to [64:07] stuff, but also with really high [64:08] visibility, not just like spawning 20 [64:11] agents and like hoping that you'll, you [64:13] know, solve this problem for me, make no [64:15] mistakes, and it'll just work in the [64:16] end, cuz that doesn't actually happen in [64:18] production. [64:19] Um, so like some general guidelines or [64:24] or or practices uh that that that we use [64:28] that I use uh at least um but but that [64:31] we've we've uh spread through our team [64:34] is like trying to run almost everything [64:36] including scripts that you run because [64:38] sometimes scripts are a lot better and [64:39] save on context space than than just [64:41] like doing everything by the LLM [64:43] obviously but running everything from [64:45] the cloud instances always like never [64:47] typing anything outside of it if you can [64:49] avoid it. Uh having this portability [64:52] because a lot of times you start work on [64:53] a ticket, you start work on something [64:54] and actually the reason you're stuck on [64:56] it is cuz someone else on your team or [64:58] even maybe another machine. Maybe you're [65:00] running it locally on your computer and [65:01] then you're like, "Oh like I got [65:02] to go home now, but I want this to run [65:04] overnight and I make it really easy to [65:05] move it elsewhere uh and let other [65:07] people pick it up. Uh maybe it needs [65:09] more compute to do something. Whatever. [65:11] It needs more memory." Um, and uh, and [65:14] then also just like always running in [65:15] dangerously skip permissions mode like [65:17] whenever possible. Uh, if you can't be [65:19] running in dangerously skip permissions [65:21] mode, do what you need to do to like [65:22] make a sandbox so you can, but if you're [65:25] having to give feedback at any regular [65:26] pace, like you're going to go really [65:28] slow. Uh, and then like so what yeah, [65:31] what do the workers do? As I mentioned [65:32] before, they're always trying to go to [65:34] PR. Uh, they are not rigorously adhering [65:38] to like the given spec you do. they're [65:40] trying to learn and adapt to it as they [65:41] go because your specs will be wrong. Uh, [65:43] and it's okay for them to make [65:45] assumptions because you can correct them [65:47] uh as you catch them. Um, and then, you [65:50] know, for like, for example, front-end [65:52] development doing every everything is [65:54] like pre-baked into the worker spawn. So [65:56] boot the local dev server, run tests [65:59] yourself on it, have it ready and [66:00] waiting so that the human can just come [66:02] and open a browser tab pointing to the [66:04] right port and they can just test the [66:06] thing as quickly as possible. Minimizing [66:08] the number of human steps that need to [66:09] be taken and like clicks to just move [66:12] something forward to the next step uh [66:14] step. Um and also just like lots of [66:18] things baked in that are like what are [66:20] things that we know really reliably? the [66:22] agent's going to be bad about how do we [66:24] learn about those things, bake them in, [66:25] put them in uh to not just like the [66:28] cloud MD file, but also like broader [66:31] reaching graphs that you have of MD [66:33] files uh which I'll get to later uh to [66:36] make those things [66:38] less of a problem. So, for example, one [66:39] of like the really obvious things that [66:41] Claude is super bad at today is [66:43] predicting how long it'll take to do [66:44] something. If you ask it like how long [66:46] is it going to take to solve this [66:47] problem be like a maybe like two weeks [66:49] of like you know one engineer's work and [66:51] in practice it takes like one prompt and [66:53] it can do it in 20 minutes cuz it's [66:55] trained on what it would have taken a [66:56] human to do those things that's all it's [66:58] like basis for training data the these [67:00] systems haven't been around long enough [67:02] for that to be updated and I think [67:03] they'll like always be behind anyway so [67:05] you can take all these things and be [67:07] like no no never trust yourself in these [67:09] ways uh and uh and then also like you [67:14] people think a lot and a lot of times [67:15] it's kind of true that like the code is [67:16] the source of truth but the code is [67:18] often like a really expensive source of [67:20] truth for the agents to pull context out [67:22] of and it's actually really cheap [67:24] especially when you have all the context [67:26] loaded in memory to like aggressively [67:28] document things in a way that benefit [67:31] future agents. So uh not just like [67:34] writing comments in the code but also [67:36] structured linked uh um sort of wiki [67:40] style knowledge knowledgebased files [67:42] that will make future agents have an [67:45] easy time um basically take advantage of [67:48] the context as much as you can uh and [67:49] also helps the visibility of humans and [67:51] and audit auditability of what you do. [67:54] Uh, so macro by default, micro win it [67:57] counts is another RTS principle. Like [68:00] you can't win a game of RT uh like RTS [68:02] game usually if you're just really good [68:04] at moving your individual units because [68:06] if you didn't make any units, you're [68:07] just going to lose. Uh so yes, it's [68:11] important to like deep dive and tunnel [68:12] vision into certain things that are [68:14] really critical. Some tickets for sure [68:15] take a long time, but anytime you're [68:18] like tunnel visioned into something, you [68:20] should always be thinking, how do I [68:21] spawn as many other little things that [68:23] don't take my cognitive bandwidth as [68:25] much and just like move those things [68:27] forward? Um, so that always you're [68:30] basically like maxing out your cognitive [68:32] capacity. Um, and again, like things can [68:35] wait. You can come back to them like 3 [68:36] days later. It's not that expensive and [68:38] you can just ask Claude like remind me [68:39] what the hell I was doing with this [68:40] thing. All this stuff is really cheap. [68:42] what's expensive but doesn't feel [68:44] expensive is like not doing these things [68:45] at the same time. Um anyway, so macro [68:50] necessary, micro useful, but you can win [68:53] honestly in RTS games and I think in a [68:56] lot of things, including in programming, [68:57] if you just macro enough, if you just do [68:59] enough things, you'll kind of uh [69:03] stupidly adjust your way towards [69:04] something that's good if you're just [69:05] always really quickly identifying [69:07] problems and solving them. Um and yeah, [69:10] this is gets back to like the high [69:11] visibility thing. So, one of the things [69:13] that I really like about you like how I [69:16] set things up is it's not like a lot of [69:19] agents that are kind of tucked away and [69:21] that you have to like dig in hard to [69:22] actually read what their ongoing stream [69:25] is and what they're actually doing like [69:27] like in an RTS game like you click [69:29] buttons to immediately jump to different [69:32] key points in the map so you can always [69:34] be auditing stuff and always like catch [69:35] it and correct it quickly if it's a [69:37] critical thing. That true I that too I [69:39] find is like super useful in [69:41] programming. Uh because again like [69:43] they're going to make mistakes all the [69:44] time. They're going to like go in wrong [69:46] directions and you definitely save time [69:47] and value if you catch them early, fix [69:49] them, course correct. Uh so you should [69:51] be kind of like looking around between [69:52] your different agents, monitoring them [69:54] while you are also trying to have as [69:56] many as you can. Um another thing to [70:00] this point that I personally like a lot [70:02] uh and is like a big thing in RTS games [70:04] is audio. So, like the only way that you [70:07] can manage a big army across the whole [70:08] map is to have lots of audio cues where [70:11] it's like your base is under attack or [70:13] you know this guy's moving or whatever [70:15] thing is happening. You don't have to be [70:17] looking at you can hear and it's like [70:19] okay I need to put my attention to [70:21] this thing and you know based on like a [70:23] lot of variety these audio cues that you [70:25] can learn and they're good like [70:26] pneumatic devices. Uh what's important? [70:29] What do I need to act on right away? [70:30] What don't I? So, like the way I run my [70:32] personal setup is I actually have all of [70:35] my individual agent uh like T-M sessions [70:39] mapped to different Warcraft and [70:41] Starcraft units uh that are colorcoded [70:44] and themed based on the type of ticket [70:46] it is. And then they play the actual [70:48] sound effects from Warcraft and [70:49] Starcraft units. So, I immediately know [70:52] and like visually identify. I don't even [70:54] have to read like this tab needs my [70:56] attention. This thing's going on. [70:58] Anyway, like to me it just seems like a [70:59] natural way of like take advantage of [71:01] these and and again like Cludes made all [71:03] these things for me really quickly as [71:04] like a side ticket that I was working on [71:06] over time while I worked on eight other [71:08] things. So it's like why not do these [71:10] things and these these devices pneumatic [71:12] devices uh or or whatever like cues for [71:15] people are really optimized in gaming [71:17] and they like know what good sound [71:18] design is to like be memorable and [71:20] otherwise catch your attention in [71:22] different ways. Um, yeah, and like cult [71:25] use of color, icons, anything that's [71:28] just like quicker to read and process [71:30] because I actually do think like these [71:31] things matter a lot, especially if [71:33] you're trying to uh really aggressively [71:36] get a lot of stuff done and the sky is [71:37] kind of the limit in how you can do that [71:39] stuff. Another thing we built internally [71:41] is like an APM tracker. Uh, and I'll [71:43] just show quickly here. Um, [71:53] so and this this is Warcraft 3, which is [71:55] like one of the lower APM requiring [71:56] professional RTS games, but this is what [71:59] it looks like to actually play this game [72:01] well uh at the at the top level. And one [72:04] of the things that you'll notice is like [72:06] no APM is not the uh the thing that like [72:11] if you max it, you're the best player in [72:13] the world, but nobody is good who [72:15] doesn't have high APM. And so you can [72:17] just kind of take that as a mental [72:19] rubric like if I'm like thinking [72:21] and like typing slowly and like am I if [72:25] this was a competition, would I really [72:26] be the best? Like do I really need to [72:28] take that much time in everything I'm [72:30] doing? and how much can I just take like [72:32] lots of little micro decisions and you [72:35] know fall toward the right uh the right [72:38] goal or toward making things better. Um [72:40] anyway, so this is just something like [72:43] we we you know each of us run like [72:45] personally on our computers and keep an [72:46] eye on and it's just like just keep [72:49] track of like are things moving and this [72:52] this APM is not like clicks you have [72:54] because I don't think that's like a [72:55] great tracker for for for agent use. We [72:58] use tool you tool calls. It's like how [73:00] many tool calls are your agents doing [73:02] per minute? Uh this minute, this five [73:05] minutes, this hour, this day, this seven [73:08] days, like how do you max all those [73:10] things and have high numbers. Um and [73:12] again, it's like it's it's one metric [73:14] among many, but it's how are you [73:16] actually being really productive or are [73:18] you really doing the most you could be [73:19] doing if you have a low APM? Uh probably [73:22] not. So otherwise like things probably a [73:24] lot of people know uh easy way to to use [73:28] tokens more effectively is just like do [73:30] a lot of things in parallel do different [73:31] things with the same agent do different [73:33] agents in parallel it will uh invariably [73:37] like for complex tasks usually give you [73:39] a better outcome than if you did it by [73:41] yourself and just like in an RTS like [73:43] you should be spending your resources [73:44] you should never have your claude tokens [73:46] like sitting unused that's really [73:48] inefficient economy like use them all [73:50] every hour period that you man. Um, [73:54] knowledge base. This is like a really [73:55] big thing that that for us I think is [73:56] still like somewhat early on. But, uh, [73:59] this whole presentation I made and [74:02] started the exact same way that, uh, I'm [74:04] just describing how I do tickets, which [74:06] is I went to Claude, I took what France [74:09] asked me to talk about, I pasted it in, [74:11] I said, "Look at our knowledge base and [74:12] how we do stuff." And put together a [74:14] PowerPoint presentation based on the [74:16] philosophies embedded in there and like [74:17] what I've told you before. and he didn't [74:19] like oneshot it, but it's like [74:22] I maybe did like 15 edits to it, you [74:24] know, and and got to this presentation. [74:26] Uh, and then I refed it all back into [74:28] the knowledge base and said like learn [74:29] everything that I've said and all the [74:31] the the advice I've given and [74:33] corrections I've given and like make [74:34] those better instilled in the knowledge [74:36] base. And this knowledge base is [74:37] basically just because like linked docs [74:39] are much faster diverse by LLMs. And so [74:43] uh and you can encode everything [74:45] including business knowledge and indeed [74:47] like Claude and and Codex are really [74:48] good at coming up with features and [74:49] stuff if they have enough knowledge [74:50] about your business. Uh so trying to [74:52] build this up in an automated way is [74:53] super useful. People come up with their [74:55] own tickets. Uh because if you have [74:57] something you could do everybody you [74:59] should just like do it. Everybody should [75:00] be full stack all the time. Uh be [75:03] reactive. Uh and uh even if agent does [75:08] it way worse than you or slower than [75:09] you, it's still better to have it do it. [75:11] And uh it's easy to change things when [75:13] they're screwed up. Satisficing is a [75:15] word from economics is like do things [75:17] satisfi like enough but not perfect. Uh [75:21] really really key principle for like [75:23] everything. Uh mix different ticket [75:25] sizes at the same time. Uh you know in [75:28] like we we've three and a halfx our [75:31] output uh PRs per engineer per month. uh [75:35] both because LM have made ourselves [75:36] better, but like when we like really [75:37] adopted this stuff broadly with everyone [75:39] on the team this last month, we grew [75:41] another 60% in our PRs per engineer per [75:43] month. So like you're not going to get a [75:45] lot smarter, but the thing you can train [75:47] on yourself is like how do I act like [75:50] people who are good at doing these kinds [75:52] of things really well like RTS pro [75:54] players? What does it look like to be [75:55] like optimal in this and how can I learn [75:57] the methods of doing it just like [75:59] program like an RTS pro? Thank you. [76:06] Okay, I think that's all we have. Um, [76:10] now I think Vikica, we have cookies, ice [76:13] cream, and popsicles and mochi donuts. [76:17] Okay, what is a mochi donut? It's [76:20] delicious. Okay. Um, yeah. So, thank you [76:22] guys so much for coming. It was a lot of [76:24] fun. Uh, I will send out a feedback [76:26] form. Please review it and give me give [76:29] me back your thoughts. um think about [76:31] those uh call for presentations and [76:34] calls for ideas. If you guys have ideas, [76:36] let's let's definitely hear them. Um and [76:38] uh looking for more papers coming up [76:40] probably in in two weeks. I think we're [76:42] already fully slated. Um and then [76:44] basically the first one in July uh you [76:47] know, we're looking to fill out as well. [76:48] So if you wanted to present, please let [76:49] me know. That's all I got. Thank you [76:52] everyone.