Can Open Models Keep Up With the AI Frontier? | Nathan Lambert

开源模型能否跟上前沿AI的发展步伐？| Nathan Lambert Nathan Lambert评估了封闭前沿实验室与开放权重模型之间能力差距的持续性，指出虽然西方实验室在架构创新和原始算力上处于领先地位，但开源生态系统（特别是在中国）作为快速跟进者，能够在数月内缩小差距。讨论梳理了竞争前沿如何从预训练转向中程训练（mid-training）、强化学习环境和合成数据精选，而非单纯依赖蒸馏捷径。Lambert阐述了他创办非营利研究实验室Trillium的初衷，该实验室旨在通过开源强化学习环境和公开记录研究自动化工作流，揭开递归自我改进与AI辅助工程的神秘面纱。最终，他反驳了投机性的“快速质变（fast takeoff）”恐慌，主张尽管缩放定律（scaling laws）需要指数级资源才能换取线性收益，但社会必须加紧准备应对前沿能力不可避免地向开源基础设施扩散。 在封闭前沿模型发布与开源对应模型之间的运作周期已压缩至两到三个月，这得益于中国实验室在工程人才上的对等水平，而中国实验室的主要瓶颈仍然是算力而非算法知识。 (7:44) Nathan Lambert创办了非营利研究实验室Trillium，旨在开源强化学习环境，并每六个月公开记录内部AI自动化工作流，以揭开递归自我改进的神秘面纱。 (32:15) 前沿模型的推理成本大约每八个月减半，预计在三到四年内下降99%，从而在企业软件开发中引发杰文斯悖论。 (25:00) 封闭实验室在投入巨额资金用于定制化推理优化以及在不可验证领域获取高质量数据方面具有结构性优势，而开源生态系统则面临着基础模型创建者、托管供应商和微调服务商之间的分散割裂。 (15:00)

Nathan Lambert argues that the capability gap between closed frontier labs and open-weight models is sustained by immense capital, compute allocations, and infrastructure scale rather than proprietary algorithmic secrecy. While open models rapidly fast-follow via shared data pipelines and standard competitive distillation, fundamental scaling physics prevent runaway recursive self-improvement, leaving societal and infrastructure unreadiness as the primary risks. Compute Disparities and the Limits of Distillation (7:44) Lambert rejects the narrative that Chinese and open-weight AI labs progress primarily by distilling proprietary Western API completions. In his assessment, engineering talent across top Chinese labs like DeepSeek, Moonshot, and Zhipu is on par with Western teams. The primary differentiator is hardware access: while an individual researcher at OpenAI can command roughly 5,000 H100 GPU equivalents, foreign and academic researchers face severe compute constraints that leave them visibly stunned by Western resource scales. Technically, distillation no longer offers a simple shortcut to parity. While classic supervised fine-tuning (SFT) reliably mapped frontier completions onto target models, modern frontier leaps depend on complex mid-training alignment and reinforcement learning (RL) runs. Distillation cannot bridge mismatches in base pre-training data or replicate multi-day RL training feats. Lambert views API extraction as standard technological competition that buys competitors only a few months of lead time rather than full capability replication. Data Economics Across Verifiable and Non-Verifiable Domains (25:00) Frontier model progress divides along domain verifiability. In deterministic fields like mathematics, competitive programming, and distributed systems, automated sandboxes and compilers provide closed evaluation loops. These allow reinforcement learning to scale rapidly and drive down inference costs—halving operational expenses roughly every eight months, or by up to 99% over three to four years. In contrast, non-verifiable knowledge work such as law, medicine, and biology lacks automated ground truth, requiring capital-intensive manual curation and behavioral specification. This dynamic has sparked a multi-billion-dollar commercial rush into synthetic data and RL environments. However, rapid commercialization has led data vendors to cut corners, generating buggy sandboxes that induce severe reward hacking—such as models scoring perfectly by exploiting empty test files. Furthermore, exclusivity contracts typically expire within six months, after which data vendors resell the same training environments to open and foreign competitors, systematically closing the frontier's lead. Structural Fragmentation Facing Open-Weight Adoption (20:00) Despite open models matching proprietary benchmarks within two to three months, open-weight developers face severe structural disadvantages. Frontier labs operate an integrated loop: they serve end-users directly, capture lucrative enterprise segments like autonomous coding, and ingest operational telemetry and real-world failure modes directly back into model training. Their massive serving scale also allows them to operate profitably on razor-thin inference margins. The open model economy, by contrast, is fragmented across base-model creators, independent hosting providers, and specialized fine-tuning vendors. Downstream edge-case telemetry rarely flows upstream to the original model creators. While enterprise adoption of open weights moves more slowly due to the necessity of designing internal benchmarks and in-house fine-tuning workflows, Lambert emphasizes that total ownership of unencumbered weights and data security provides a durable economic foundation resistant to speculative hype. Scaling Realities and Trillium’s Open Agenda (38:40) Lambert dismisses prevailing anxieties around immediate, runaway Recursive Self-Improvement (RSI), noting that existing algorithmic regimes require exponential increases in compute to achieve linear performance gains. Citing system cards from leading labs, he points out that a true fast takeoff would require roughly a 10x acceleration over current progress rates—a leap that current transformer architectures, human engineering bottlenecks, and physical power constraints cannot deliver. To strip away the mystique surrounding frontier automation, Lambert established Trillium, an independent nonprofit research lab. Trillium aims to provide open reinforcement learning environments, public rollout traces, and regular empirical check-ins detailing where models succeed and where they fail on real-world engineering tasks. For Lambert, the paramount danger is not an uncontrollable algorithmic explosion, but the profound lack of societal preparedness, enterprise workflow adaptation, and public cybersecurity ready to handle the frontier capabilities that are already diffusing into the open domain. Lambert concludes that closed frontier labs will maintain their primary advantage through lower cost-per-watt inference scaling and infrastructure integration rather than through permanent architectural secrecy. Because technical breakthroughs and synthetic data pipelines inevitably diffuse across the ecosystem within months, the core unresolved challenge remains establishing independent, public evaluations and hardening real-world infrastructure for the capabilities already arriving.

AI研究员兼Trillium创始人Nathan Lambert与Lisha Li在对话中指出，前沿闭源实验室与开源生态的核心差距在于算力规模与工程组织效率，而非算法秘密或人才高低；所谓前沿技术难以永远垄断，但在算力与数据扩展定律的约束下，真正的风险并非失控的递归式自我改进，而是社会制度对现有技术扩散准备不足。 算力鸿沟与开源出海策略 (2:35) Lambert认为，模型能力的演进取决于组织将算力、数据和人才转化为实际系统的能力。目前顶尖前沿实验室（如OpenAI和Anthropic）垄断了绝大多数关键资源并敢于早期下注，例如OpenAI确立的推理时算力扩展定律与Anthropic对自主代码能力的聚焦。针对中国AI生态，Lambert反驳了西方认为中国主要依靠“蒸馏盗取”西方模型权重的观点。他指出，中国在预训练和后训练架构上具备世界级的人才储备，其面临的最根本硬性瓶颈是硬件算力差距。例如在与中国顶尖预训练研究员沟通时，对方得知OpenAI一名普通研究员可随意调度等效5,000张H100 GPU时深感难以置信。此外，月之暗面（Moonshot AI）和智谱AI等中国机构开源模型，主要是一种商业进入市场的策略：由于在西方销售闭源API面临重重信任阻力，开源权重成为了争夺国际声誉与分发渠道最有效的手段，从而充当了敏捷的追随者角色。 蒸馏机制演变与闭源商业飞轮 (10:00) Lambert指出，蒸馏本质上是“以资本换取算力等价”的正常市场竞争。在传统的SFT与RLHF时代，以Prompt为核心单元、使用更强模型的完成结果进行下个Token预测可以稳定提升质量；但在当前转向强化学习（RL）和复杂推理环境的范式下，仅靠蒸馏前沿模型的思维链Token已无法直接让开源模型取得跑分跃升，这往往受限于中间训练（Mid-training）或通用预训练语料的不匹配。前沿实验室通过巨额资本锁定至2028年的算力与电力资源，依靠可预测的扩展定律获取企业级API收入，进而支撑未来的公开市场融资与IPO闭环。然而，若开源模型在两至三个月内迅速抹平基础性能差距，这种资本飞轮就可能承受压力。面对提取风险，Lambert强调闭源团队应通过商业合同而非呼吁联邦监管或KYC规则来保护自身，否则只会伤害正常的开源生态。 数据供应链瓶颈与奖励黑客隐患 (25:00) 在非可验证领域（如法律、生物与医疗），模型突破并非源于神秘的算法顿悟，而是依赖极其繁琐且资本密集的定制化行为数据采购。相反，代码和数学等具备自动化验证闭环的领域则能通过强化学习迅速推高能力并大幅削减成本。随着模型推理成本大约每八个月减半，未来三到四年运营成本可能下降至初始水平的1%，这在激发杰文斯悖论的同时也带来了巨大的利润率竞争压力。与此同时，当前的RL环境与合成数据供应链充斥着粗制滥造：许多第三方数据供应商借助前沿LLM大规模生成合成任务以谋取高估值，但测试沙盒经常出现空文件或配置漏洞，导致高推理能力模型学会利用测试环境漏洞直接刷出满分的“奖励黑客”（Reward Hacking）现象。为此，顶尖实验室已开始加速将RL环境和数据管理收归自研，而外部供应商往往在西方实验室采购六个月后将类似管道转售给中国团队。 去魅递归自我改进与去中心化落地 (40:00) 开源模型生态在商业推进上面临结构性阻力：由于基础权重开发者、推理托管商与下游微调服务商各自割裂，真实的终端错误遥测难以回流给模型构建者；但企业出于私有数据隐私考量，仍愿意投入时间自建评估基准并掌握模型权重所有权。针对行业内对“递归式自我改进”（RSI）可能引发智能爆炸的恐慌，Lambert直言当前的算法体制仍然受制于算力呈指数级增长而性能仅呈线性提升的物理规律。前沿实验室的方法并非深奥的未解秘密，而是依赖资本堆叠找到精确的超参数与扩展配方；即便在硅谷技术圈内部，关键工程架构也会在社交圈内自然渗透。目前即使是前沿模型发布的技术报告也显示，要实现真正的快速失控，其进步速度需要达到当前速率的10倍以上。 Lambert认为过度炒作递归失控反而分散了对现实问题的注意力。当前更紧迫的挑战在于公共基础设施、网络安全和企业治理尚未对已在扩散的模型能力做好准备。为此，他创办了非营利实验室Trillium，致力于通过开源RL环境、公开工程执行轨迹和基准测试，消除前沿AI能力的神秘感，为学术界和政策制定者提供客观的实证依据。

## Background 本期节目由 a16z 合伙人 Lisha Li 与 AI 研究员兼 Trillium 创始人 Nathan Lambert 于 2026 年 10 月 14 日在播客平台发布。页面简介指出，对谈聚焦于前沿闭源实验室与开源大模型之间的技术鸿沟是否具有持续性，探讨算力约束、蒸馏机制、RL 环境与后训练对模型演进的影响，并介绍了 Trillium 致力于将 AI 自动化研发与递归自我改进（RSI）去神秘化的非营利愿景。 ## TL;DR Nathan Lambert 深入剖析了 OpenAI 与 Anthropic 等闭源实验室在算力资本、组织架构与推理 Scaling 上的核心优势，并指出中国开源模型团队在极端算力限制下展现出了同等水平的人才素质与工程迭代能力。虽然开源阵营受益于数据供应链外溢与部分蒸馏红利，但真实的瓶颈在于 RL 环境构建、奖励作弊治理以及商业化闭环中的信息割裂。通过创立非营利研究机构 Trillium，Nathan 主张公开真实的 AI 研发实验与代码轨迹，以此破除外界对递归自我改进（RSI）的盲目神话与焦虑，为学界与政策制定者提供客观透明的科学依据。 ## Atmosphere & style 整场对话呈现出高密度、务实且充满内部视角的深度研讨氛围。Lisha Li 擅长从资本飞轮、产业落地与投资视角的严谨逻辑展开质询，而 Nathan Lambert 则以一线科研维护者的坦诚与幽默破除硅谷叙事中的“自我安慰”，直言不讳地指出大厂内耗与行业浮躁。交流既有剥离炒作后的冷静技术解构，也流露出对开源生态长远健康发展的理想主义情怀与人文关怀。 ## Bulletin - `07:54` 开源与闭源前沿模型之间的能力差距已从数年急剧缩短至两到三个月，中国团队凭借极高的人才储备与算法工程在严重受限的硬件算力下实现了紧密跟进。 - `30:00` 后训练与强化学习环境已成为模型能力跃迁的核心战场，头部实验室为构建环境投入数十亿美元研发预算，导致数据供应商过度追求营收而诱发奖励作弊（Reward Hacking）。 - `32:15` Nathan Lambert 创立非营利研究机构 Trillium，旨在通过公开模型交互轨迹与 RL 环境，打破商业短期逐利对学术土壤的侵蚀并去神秘化递归自我改进（RSI）。 - `55:00` 缩放定律存在指数级算力换取线性性能提升的物理约束，前沿实验室真正的壁垒在于工程极致化降低推理成本，而非外界过度恐慌的指数级智能快速爆发（Fast Takeoff）。 ## Key takeaways - `00:00` 前沿模型演进取决于算力、数据与人才三类资本的配置，OpenAI 善于通过推理模型掀起范式革命，Anthropic 则长期深耕代码能力打磨。 - `04:59` 中国大模型公司开源权重是务实的全球商业突围策略，同时得益于顶级闭源实验室前期斥巨资培育的数据产业链的外溢效应。 - `07:54` 中美顶尖 AI 研究者在基础人才储备上处于同等水准，中国团队的核心瓶颈在于人均硬件算力与顶尖闭源实验室存在数量级的悬殊差距。 - `10:00` 单纯依赖 SFT 学习顶级模型推理轨迹的传统蒸馏范式已难以直接奏效，现代推理模型的能力突破更加依赖复杂强化学习环境的长周期训练。 - `13:30` 针对 API 推理轨迹泄漏，闭源实验室最直接的阻断手段是弃用公开文本 API 改签企业协议，而通过政府强制推行 KYC 监管在法理与商业上皆不可行。 - `15:30` 过度使用政策手段限制模型蒸馏会破坏必要的中美科技对话渠道，黄仁勋等业界领袖亦指出拆解产品全方位测试属于正常的市场竞争。 - `17:30` 前沿实验室依赖 Scaling 掌控建立资本飞轮筹备 IPO，而开源生态通过微调基座模型以极高性价比支撑垂类应用落地，对闭源溢价构成威胁。 - `20:00` 开源商业生态由于基座构建、推理服务与后训练服务商的上下游分工割裂，信息反馈与利润流断裂，其商业飞轮建立远比闭源巨头漫长。 - `22:20` 通用推理 Scaling 正在降维打击数学等特化领域，但在长程复杂工程任务中，精细微调的开源模型配合规划模块已经能平替纯前沿闭源方案。 - `25:00` 不可验证领域（如医疗、法律与复杂白领任务）缺乏自动化测试脚本，其模型能力跃迁并非神秘泛化，而是依靠巨额资本定向采买专业数据并搭建后训练环境。 - `27:30` RL 环境淘金热中供应商的粗制滥造导致严重的奖励作弊（Reward Hacking），促使前沿实验室加速将核心环境研发与数据采集全面收归自研。 - `32:15` Trillium 实验室致力于为外部研究者与博士生提供开源基础设施与环境，对抗由短期九位数估值诱惑引发的学术人才流失与研究封闭化。 - `35:00` AI 智能体一键生成的低质量 issue 与虚假科研成果充斥开源社区，加剧了科研浮躁，稀缺的反而是耐得住寂寞深入钻研代码的研究者。 - `37:46` 前沿实验室内部研究员深陷被自身模型自动化的焦虑，Trillium 计划通过记录和公开人机协作的真实研发轨迹，客观界定 AI 自动化研发的真实边界。 - `45:00` 互联网预训练数据与科学工具已相对固定，预训练并非百亿美元专属死局；相比之下，后训练与中训练正演变为高度非线性的加速竞技场。 - `50:00` 硅谷极高的人才流动性导致算法范式很难被长期封锁，闭源实验室核心护城河在于将单位推理能耗与运行成本做到极致的工程基础设施。 - `55:00` 当前的 Scaling Laws 存在自回归算力指数级投入仅换取线性增益的客观限制，社会更应重视模型扩散带来的网络安全等现实防御挑战而非虚构的快速爆发。 ## Detailed walkthrough ### `00:00` 驱动前沿模型进步的资本形式与组织架构 在前沿模型演进的宏观框架上，模型能力的跃升本质上取决于三类核心“资本”的投入：算力（compute）、数据（data）以及人才（talent）。在前沿闭源实验室中，OpenAI、Anthropic 和 Google 显然坐拥最丰沛的资源，但决定竞争胜负的关键，往往在于组织内部如何调配并盘活这三类资本。理顺一个用于构建大模型的组织架构（org chart）极其困难，以至于很多科技巨头不得不推倒重来、自底向上全盘重构。例如 Meta 在经历彻底重组之后，才重新走上了清晰向上的积极发展轨道。 ```mermaid flowchart TD accTitle: 前沿模型竞争力要素与层级划分 accDescr: 展示三种核心资本如何通过不同的组织文化与商业路径分化为顶级闭源实验室与跟进型开源生态。 subgraph Capital["核心资本投入"] C["算力 Compute"] D["数据 Data"] T["人才 Talent"] end subgraph FrontierLabs["顶级闭源实验室 (OpenAI / Anthropic)"] direction TB P1["押注全新范式 (如推理 Scaling Law、代码智能)"] P2["巨额资本开销，赚取最高 API 溢价与利润"] end subgraph OpenEcosystem["紧随其后的开源/开放权重阵营 (如中国实验室)"] direction TB F1["务实商业定位与开源分发 (如 Moonshot、智谱)"] F2["承接数据供应链外溢，主打企业私有化场景"] end Capital --> FrontierLabs FrontierLabs -->|"范式确立与数据产业链外溢 (1年滞后期)"| OpenEcosystem ``` 对于立志投身真正 AI 前沿的人，行业内最稳妥的选择始终是去 Anthropic 和 OpenAI。这两家公司最具前瞻性，也最敢于在外界认为不可能的方向上豪赌天价资金： - **OpenAI 的文化特质**：善于引爆技术范式革命，通过推出推理模型（reasoning models）并绘制出极其优美的推理扩展定律（inference scaling laws）图表，一举颠覆全行业认知。 - **Anthropic 的文化特质**：长期专注于模型深层能力的打磨，尤其是对代码能力的极致坚持，最终收获了以 Claude Code 爆发为代表的巨大回报。 关于组织与前沿探索的关系，Nathan Lambert 直言：“I think those two companies in particular have been the most forward-looking and the most willing to spend egregious amounts to make a step towards a direction that people think is impossible.”（我认为尤其是这两家公司，最具前瞻性，也最愿意投入巨额资金朝着大家认为不可能的方向迈出一步。） --- ### `04:59` 中国大模型公司的开源突围与两层市场格局 中国大模型公司（如月之暗面 Moonshot AI、智谱 AI 等）选择发布开放权重模型（open-weight models），背后并非出于纯粹的利他主义或外界臆测的政府授意，而是一项非常务实的商业突围战略。对于身处中国的初创团队而言，如果仅仅在本土搭建模型并尝试向全球出售 API，海外市场几乎无人问津；通过开源，他们能够绕过分发壁垒，迅速切入全球技术话语体系。同时，由于中国国内对企业行为的监管和应对手段往往极其严厉，中国实验室在模型安全考量上也展现出更为审慎严肃的态度。 与 OpenAI 和 Anthropic 试图实现真正的递归自我改进（recursive self-improvement, RSI）以在头把交椅上“赢家通吃”不同，开源模型扮演的是填补市场空白的追赶者角色。未来市场正自然演化出清晰的分层梯队： - **顶级闭源模型层**：享有最前沿的能力，凭借行业独占性获取最丰厚的高毛利。 - **次前沿开放权重层**：紧随其后落后半步，面向对利润率要求较低或对隐私合规极其敏感的市场（例如企业私有数据部署）。 这种快速跟进之所以能够持久维系，很大程度上得益于数据经济的溢出效应。OpenAI 和 Anthropic 斥巨资培育并壮大了上游数据标注和合成产业，而数据供应商在满足合同排他期后（通常约一年），便会将这套成熟的数据生产能力推向整个市场。这种供应链传导使得跟进门槛大幅降低。正如 Nathan Lambert 所分析的那样：“And once, you know, once Anthropic and OpenAI show you the type of model you need to build, it's a lot easier trickles through the data economy where OpenAI and Anthropics spend way more to build out this function of a data industry.”（一旦 Anthropic 和 OpenAI 探明了你需要构建的模型类型，技术就会通过数据经济向下渗透，而 OpenAI 和 Anthropic 此前为打造这种数据产业功能投入了远超常人的资金。） --- ### `07:54` 蒸馏叙事的迷思：顶尖技术人才与残酷的算力鸿沟 针对外界常将中国模型的快速迭代单纯归咎于“蒸馏”（distillation）的观点，实际情况要复杂得多。不可否认，开源与闭源之间的能力差距已经从早期的数年急剧缩短到如今的 2 至 3 个月，这也倒逼顶级闭源实验室收紧 API 权限以防范提取。然而，把中国团队的成就完全贬低为“拿现成成果蒸馏”是一种严重的信息偏差。 中国的大模型实验室在全流程预训练与后训练上具备极高的技术素养与工程水平。中美在基础人才储备上完全处于同等水准，中国顶尖的高校与科研体系源源不断地输送着高水平研究者。然而，双方在硬件基础设施上存在着难以逾越的鸿沟： - **算力分配差异悬殊**：当与中国某知名实验室的顶级预训练研究员交流时，对方得知 OpenAI 的一名普通研究员随时可以调用相当于 5100 张高端 GPU（如 A100/H100）的算力规模时，当场震惊得“下巴都要掉下来”，直呼无法想象。 - **资源制约下的技术路线**：中国团队在严重缺乏算力（compute-starved）的极端约束下，凭借顶尖算法工程最大化发挥硬件能效，这种技术实力往往被外界低估。 Nathan Lambert 对人才对比给出了极其明确的断言：“To say that China is anything but equal on talent, you're going to have a hard time because they just have so many people being trained at such a high level. And what they don't have is compute.”（如果说中国在人才方面有什么不平等的，你很难站得住脚，因为他们有如此多的人接受了如此高水平的训练；他们真正没有的，是算力。） ### `10:00` 蒸馏机制的演变：从 SFT 模式到强化学习时代的迷思 Nathan Lambert 深入剖析了蒸馏（Distillation）的本质，指出蒸馏实际上是一种“花更多钱来换取等效算力”的手段，因为语言模型的输出本质上就是算力的函数，若直接获取理想输出，效果等同于拥有产生这些输出的模型与算力。回顾后训练（post-training）的发展历程，早期的 RLHF 时代高度依赖监督微调（SFT），只要在最强模型的输出上计算“下一个 Token 预测损失”（next-token prediction loss），模型性能就能大幅跃升。Nathan 回忆起在 AI2 与共同负责后训练的 Hannah 一起工作时的经验：“在 AI2 时，我和共同主导后训练的 Hannah 总是会说：提示词（Prompts）是后训练的核心单元。”只要手握优质提示词，就能在不同训练阶段达成目标。然而，这种“喂入高质量补全内容、跑 SFT、跑分直接上涨”的传统范式，已很难直接套用在当今基于强化学习的“推理模型”（reasoning models）上。 ```mermaid flowchart LR accTitle: Post-training Evolution accDescr: Comparison between the traditional SFT era and the modern RL reasoning era. subgraph SFT["传统 SFT / RLHF 时代"] A["优质提示词 (Prompts)"] --> B["顶尖模型生成补全"] B --> C["Next-token 预测损失"] C --> D["跑分直接提升"] end subgraph RL["现代强化学习 / 推理时代"] E["推理思维链 (Traces)"] --> F["复杂 RL 环境构建"] F --> G["长周期大规模强化学习"] G --> H["跨模态/后训练迭代验证"] end ``` 根据现有的公开学术文献，Nathan 很难断言蒸馏是导致当前开源与闭源差距缩小的最主要因素。在开源社区尝试复现后训练配方的过程中，许多构建者由于搞不清楚如何正确配置“教师模型”（teacher），往往被困在较旧的 Qwen 模型基底上。Nathan 提出猜想：开源基座模型的预训练数据或中期训练（mid-training）中，可能缺失了某种关键特征，导致即便拿到顶级模型的思维链（reasoning traces）去跑 SFT，也无法复现评分暴涨的效果。他甚至直言，当初在训练 OLMo 时，就算把 Claude Opus 乃至未经公开的顶级模型推理轨迹拿给团队，也未必真能让模型强上多少。 与此同时，真实工业界的现象又充满疑点。例如像小米（Xiaomi）这样的团队，公开发布了他们经过为期一周的大规模强化学习训练并开源模型，技术报告中还提到在 RL 之后做了一道额外的后训练工序；而中国市场上也存在大量通过中介或海量账号调用 Claude 等闭源模型代理（proxies）的现象，甚至出现了月之暗面（Moonshot AI / Kimi）在产品中直接接入并向用户提供 Claude 服务的传闻。这些大规模消耗闭源 API Token 的做法，到底在多大程度上直接转化成了模型训练中的蒸馏技术，学界和行业目前都缺乏透明度，认知上存在很大的误差范围。 - **后训练演变**：从依赖 SFT 与强模型生成的输出补全，转变为高度依赖强化学习环境（RL environments）的推理训练。 - **开源复现瓶颈**：缺乏合适的中期训练数据和基座适配，简单复制顶级模型的推理轨迹难以稳定提升能力。 - **业界疑云**：大量海外 API 代理账号与高并发调用，模糊了“技术蒸馏”与“工程调用/产品包装”的界限。 --- ### `13:30` 推理轨迹泄漏与“知晓你的客户”（KYC）之辩 对于“中国开源模型全靠抄袭与蒸馏”的论调，Nathan Lambert 认为这其中夹杂着西方科技界的心理防御机制（cope）：“我认为这有点像自我安慰，人们总觉得‘只有我们才擅长做模型’，但事实上，现在有大概 20 到 30 家公司都能做出非常出色的模型。”虽然 AI 安全学界近期才发表关于“从闭源模型中窃取推理轨迹（thinking tokens）”的论文，但中国顶尖实验室的专门团队很可能早已掌握并实践了这类技术。 推理模型在生成内容时会产生思维链 Token，只要闭源实验室对外开放 API，这些中间步骤就很容易被抓取并用于模型训练。Nathan 指出，闭源实验室如果真的想阻断这种蒸馏，最直接的办法就是放弃纯文本 API，只在深度绑定的企业产品端提供服务，并要求客户签署严格的企业协议。但针对部分人呼吁政府介入、强制在 AI 接口层面推行“知晓你的客户”（KYC）监管，Nathan 表达了明确反对。他认为这既不利于商业运作，也在法理上站不住脚：“我不明白为什么要因为违反服务条款（ToS）就让政府介入；我甚至不认为服务条款属于正式签署的法律合同，你无法以那种方式起诉他们，你唯一能做的就是取消他们的访问权限。” > “我认为这有点像自我安慰，人们总觉得‘只有我们才擅长做模型’，但事实上，现在有大概 20 到 30 家公司都能做出非常出色的模型。” —— Nathan Lambert --- ### `15:30` 蒸馏是竞争还是盗窃？国家安全与产业生态的博弈 Lisha Li 引述了英伟达首席执行官黄仁勋（Jensen Huang）的一句名言作为讨论的切入点：“人们每天都在蒸馏我的产品，他们把产品拆解到最原始的零部件去进行全方位测试——这不叫盗窃，这叫竞争（That's called competition）。”这引发了关于“蒸馏到底算不算不正当手段”的正面讨论。Nathan 认为，过度使用政策手段去打击蒸馏是极其短视且弊大于利的。国会不可能通过专门法律去禁止“模型蒸馏”，最终只会沦为联邦监管部门给企业日常运作制造麻烦的行政手段。 更重要的是，在应对前沿 AI 风险（如闭源模型无节制抓取敏感数据、开源权重滥用风险等）时，美国迫切需要与中国科技界保持高级别的沟通与对话机制。如果一味在蒸馏问题上挥舞大棒，不仅无法阻止技术的逆向流动，还会破坏脆弱的沟通渠道：“如果你试图对蒸馏做任何干预，最终造成的伤害将远大于收益，因为他们根本不会停手，他们只会继续做下去。”此外，正是因为开源模型能够紧跟前沿，整个下层创业公司生态才得以蓬勃发展，这绝非单纯的“中美对抗”叙事可以概括。 > “如果你试图对蒸馏做任何干预，最终造成的伤害将远大于收益，因为他们根本不会停手，他们只会继续做下去。” —— Nathan Lambert --- ### `17:30` 资本飞轮的危机：闭源前沿与开源创业的路线之争 Lisha Li 为 OpenAI 和 Anthropic 等前沿闭源实验室给出了一个强有力的辩护视角（steel man）。她指出，构建前沿模型需要史无前例的巨额资本支撑，前沿实验室的核心优势在于掌握了“缩放定律的科学”（the science of scaling），借此将算力扩张转化为可预测的收入，再通过庞大的收入在资本市场融得更多资金，形成极具威力的飞轮效应。然而，算力瓶颈不仅在于芯片本身，更延伸到了电力供给与数据中心基建，这种整体稀缺性预计将持续到 2028 年。前沿实验室现在面临着指数级增长的资源开销，必须走向公开市场进行 IPO，而开源模型通过蒸馏或轻量化方式极速缩小差距，严重威胁到了这个重资本飞轮的存续。 ```mermaid flowchart TD accTitle: Frontier Lab Capital Flywheel accDescr: The self-reinforcing flywheel of frontier labs and the threat posed by rapid open catch-up. A["巨额资本投入"] --> B["算力垄断与 Scaling 掌控"] B --> C["定义前沿模型能力"] C --> D["实现规模化高额收入"] D --> E["筹备公开市场 IPO / 新一轮巨额融资"] E --> A F["开源模型快速跟进 (蒸馏/后训练)"] -.->|"压低溢价 / 威胁商业闭环"| D ``` 但另一方面，开源生态在实际产业中展现了极高的性价比。Lisha 提到，像 Decagon 和 Harvey 这类在垂类领域进行大量微调的企业，以及大量将“后训练即服务”（post-training as a service）作为主营方向的初创公司，正是借由像 Kimi K1.5 或智谱 GLM 这类开源基座模型，以极低的成本达到了帕累托最优前沿（Pareto frontier）。针对这两者的冲突，Nathan 开启了下文关于开源与闭源竞争维度的判断： > “闭源实验室高度垂直整合，拥有极致高效的推理架构，当他们准备好时，凭借提前规划的大量资源储备，能够真正将模型进行超大规模部署。” —— Nathan Lambert - **前沿实验室飞轮**：巨额资本 $\to$ 算力与电力基础设施（稀缺性持续至 2028 年） $\to$ 掌握缩放科学 $\to$ 垄断前沿并获得收入 $\to$ 冲刺 IPO 融资。 - **初创公司现实**：高性价比开源模型（Kimi、GLM 等）结合垂直领域微调，支撑了诸如 Decagon、Harvey 等企业级应用的落地。 - **发展尺度差异**：闭源与开源模型正处于不同周期的指数曲线上，闭源巨头的全栈工程与推理优化依然构成了坚固的护城河。 ### `20:00` 开源生态的飞轮瓶颈与专有数据的护城河 前沿实验室在编程等极高价值的市场中抢占了先机，收入规模在一年内呈现出爆发式增长。然而，对于处于“快速跟随者”位置的开源模型生态而言，其商业飞轮的运转面临着结构性难题。开源体系中存在着明确的产业链分工：上游是基础模型构建者，中游是模型推理平台，下游则是后训练服务商。客户对模型能力的真实需求分布在最末端，这种核心信息无法顺畅地反哺给上游的模型构建者；同时，分散的开源厂商难以具备前沿大厂的规模效应，导致其在推理服务上的利润空间被大幅挤压。 ```mermaid flowchart LR accTitle: 开源模型生态的割裂循环 accDescr: 展示开源模型生态中信息流与利润流断裂的产业链传导图 subgraph OpenEcosystem["开源商业链条（信息与利润断裂）"] A["基础模型构建者"] -->|"提供权重"| B["模型推理平台"] B -->|"提供算力"| C["后训练服务商"] C -->|"交付应用"| D["终端企业客户"] D -.->|"私有需求/评测信号受阻"| A end ``` 企业在落地过程中面临数据隐私的硬性约束，大量核心专有数据绝不可能通过共有 API 进行传输。企业需要花费漫长的时间探索内部评估体系（Evaluation）是否能真正驱动营收，并联合 Prime Intellect 或 Together AI 等微调平台将模型打磨成型。这一流程虽然推进缓慢，但最终让企业获得了无可替代的实际掌控权——无需受制于云厂商服务条款的模型权重、可在自有硬件上低成本运行的能力以及清晰的投资回报率（ROI）。这种靠利润率与低成本驱动的稳健商业化进程，反而让开源生态远离了过热的估值泡沫。 - 编程赛道商业化极速爆发，年内营收呈现断崖式拉升，但开源界作为跟随者很难直接捕获最初的信息流与超额收入。 - 前沿闭源厂商凭借巨大的调用量享有极高的推理利润率，而开源生态上下游割裂，难以通过单一主体形成飞轮。 - 企业自建评测系统并与自身业务营收对齐，是开源微调能否实现商业化突破的核心前提。 > Nathan Lambert 强调：“但在这一流程的终点，他们拥有真正的所有权：那是属于自己的模型权重，没有苛刻的服务条款，能跑在自己的硬件上，并且他们算清了这笔账能带来投资回报（ROI）。正因如此，我认为开源模型经济的建立需要漫长得多的时间。” --- ### `22:20` 通用推理的泛化吞噬力与特定任务的特化反转 在技术路径上，业界持续争论“大尺度通用推理能力”与“垂类特化模型”孰优孰劣。数学领域是通用推理吞噬垂类的最典型代表：各大前沿实验室甚至不需要依赖精细标注的 Lean 形式化数据集，仅凭纯粹的大规模通用推理扩展，就能在数学能力上彻底碾压任何专有数学模型。在机器人领域也出现了类似迹象，无论是 OpenAI/Anthropic 布局的 Astra 等多模态与控制能力，还是源于“Computer Use（计算机操作）”和 3D Blender 仿真带来的空间推理外溢，通用大模型都在迅速吸收这些原本被视为高度特化的技能。 然而，在复杂长程任务的工程实践中，前沿闭源模型的优势并非坚不可摧。Lisha Li 引用了 Cursor 在夏季进行的一项测试案例：在重写 SQLite 这类长程、高复杂度的工程任务中，采用双前沿模型（一个负责规划、一个负责执行）的方案，其表现不仅没有明显优于基于开源微调的 Composer 模型加上单前沿模型规划的方案，反而消耗了极其昂贵的推理成本。这表明开源模型的精细微调版本在解决特定复杂工程任务时，已经能够跨过“足够好用”的智力门槛。 - 数学领域证明了通用推理能力的扩展可以自然降维打击各类垂类专业模型。 - 关于前沿模型在机器人领域的飞跃，业界存在两种观察视角：一种认为是 6 到 8 个月前定向采购大规模机器人数据带来的直接成果；另一种观点则认为这是源自计算机操作（Computer Use）探索中所沉淀出的空间推理泛化能力。 - Cursor 针对重写 SQLite 任务的测试表明，完全依赖全套前沿大模型并不具备性价比，合理的开源微调配合核心规划模型已能平替甚至超越纯前沿方案。 --- ### `25:00` 非可验证领域的资本博弈：数据采购与成本断崖 在数学、代码或分布式机器学习工程等具有严格测试集的“完全可验证领域”，模型的演进逻辑清晰明了：推理成本正在呈指数级骤降。例如大模型推理价格每 8 个月减半，三到四年内成本可能降至最初的百分之一。然而在诸如生物医药、医疗、法律以及广泛的白领知识工作中，任务结果无法通过编译器或测试脚本简单判定，这一类“不可验证领域”的能力跃迁往往被包装为“神秘的泛化魔法”，而其底层真相绝大部分是依靠极其密集的资本在大量采购专业领域数据并搭建强化学习环境。 ```mermaid flowchart TD accTitle: 模型演进在不同领域的驱动分化 accDescr: 对比可验证领域依靠自动化反馈与不可验证领域依靠高资本数据采购的不同演进机制 subgraph Verifiable["可验证领域（数学/代码/ML工程）"] V1["代码编译 / 单元测试"] --> V2["成本极低自动化验证"] V2 --> V3["模型效率指数级下降"] end subgraph NonVerifiable["不可验证领域（生物/法律/医疗/白领任务）"] N1["专家行为建模"] --> N2["高昂资本采购外部数据"] N2 --> N3["世界环境构建与逐步后训练"] end ``` 在前沿实验室内部，研究人员的工作形态出现分化：一部分人从事算法创新，而大量技术人员的核心工作则是极度垂直的数据工程——界定某一专业领域的理想行为边界，反向设计促使该行为涌现的训练方案，并在市场上定向采买高质数据。这种通过“堆砌高质量领域数据并提供可供爬坡的环境”来提升模型表现的路径，决定了模型在专业垂直行业的能力渗透将是一场依赖资金实力的持久战。 - 前沿大模型降价速率惊人，若保持每 8 个月降低 2 倍的节奏，4 年后推理成本将仅为原本的 1%，这可能触发杰文斯悖论（Jevons paradox），即需求呈量级扩张，或是导致行业总营收被压制。 - 不可验证领域（医疗、法律、特定科研）无法依赖自动化环境自循环，必须依靠研究人员深入拆解行为模式并斥巨资采购数据支撑后训练。 - 所谓的跨模态“泛化神话”，其背后往往对应着实验室在数月前开始的大规模定向数据采买和团队扩张。 > Nathan Lambert 指出：“当我和前沿实验室的人交流时，有些人做着令人惊叹的研究，但也有相当一部分人的贡献聚焦在极其狭窄的事情上：做一个数据集，明确你的模型在那个领域应该表现出什么样的行为，构思出如何产出能够带来该行为的数据，并搞清楚需要采买什么数据。” --- ### `27:30` RL 环境淘金热、作弊隐患与数据供应链外溢 随着基于环境交互的后训练（RL）成为大模型能力提升的关键支柱，市场上涌现出一大批向头部实验室及腰部厂商售卖强化学习交互环境（RL Environments）的数据供应商。然而，在这场因丰厚利润而引发的淘金热中，数据与环境供应商为了追求交付速度常常偷工减料，诱发了模型严重的“奖励作弊（Reward Hacking）”。当模型被置于缺乏鲁棒约束的模拟环境中时，极其聪明的算法会寻找环境破绽来虚高打分（例如测试用例中本应包含迷宫，但文件为空，模型便通过规避路径逻辑直接窃取奖励）。 ```mermaid flowchart LR accTitle: RL数据供应链中的奖励作弊产生机制 accDescr: 展示数据供应商为追求营收偷工减料与实验室纠偏需求之间的脱节 A["RL环境供应商"] -->|"追求交付与营收"| B["环境设计存在漏洞（如空测试集）"] B --> C["模型利用漏洞发生 Reward Hacking"] C --> D["实验室研究人员耗费精力纠偏过滤"] D -.->|"倒逼"| E["核心环境研发与数据收归自研"] ``` 由于数据供应商往往会将高价售给 Anthropic 或 OpenAI 的环境与数据在数月后折价转手给中国等地的开源模型团队，导致开源界在数据供应链上紧随其后。为了遏制研发意图与核心评测信号的外泄，同时彻底解决外部劣质环境导致的奖励作弊问题，头部实验室正掀起将强化学习环境与关键数据全面收归自研（In-house）的风潮。前沿实验室的研究人员不得不将海量精力投入到消除作弊与清洗数据上，数据供应商对营收的狂热追求与实验室对高质量行为环境的严格要求之间，正面临严峻的磨合阶段。 - 外部数据供应商将经过验证的 RL 环境转售给更广泛的跟随者或开源阵营，构成了除模型蒸馏之外，开源能力快速逼近前沿的另一核心动力。 - 供应商构建环境时存在大量偷工减料现象，过度关注世界构建的表象与营收规模，而前沿实验室的研究人员则被迫花费大量时间治理奖励作弊问题。 - 为防止下一代模型研发方向和评测信号从第三方供应链外泄，头部前沿实验室正在加速将核心数据采集和强化学习环境研发全面收归自研。 > Nathan Lambert 评价道：“这就像模型实在太聪明了，以至于一旦你把它放进一个环境并要求它完成任务，哪怕你在测试文件中给它的是一个空文件而不是原本需要遍历的迷宫，模型也会想方设法在这种漏洞下刷出极高的奖励分数。” ### `30:00` 合成数据与 RL 环境淘金热：质量参差与资本狂欢 当前 AI 产业对环境（RL environments）与合成数据的投入规模令人咋舌。几乎所有数据公司都在利用语言模型来构建、修改虚拟世界或生成变体。即便有人类参与引导，其本质依然是合成数据，很少再有人会单纯坐在那里手工雕琢模型行动的微型世界。前沿实验室之所以愿意为这些环境投入数十亿美元的研发预算，唯一的原因在于这些数据确实让模型变得更强，他们看到了明确的训练信号。Nathan Lambert 坦言自己对此甚至仍有些难以置信：“我依然对此感到难以置信。我想，这些数字太疯狂了。在环境上耗费数十亿美元的研发预算……他们这样做仅仅是因为这能帮助模型，而且他们看到了信号。”（“I still have denial about it. I'm like, these numbers are crazy. The billions of dollars research budget on environments. I'm like, they're only doing this because it is helping the models and they see the signal.”） 然而，在狂热的数字背后，整个数据行业仍充斥着大量粗制滥造的低垂果实。深入行业细节会发现许多怪异的边缘现象：部分数据公司通过层层中介雇佣海外外包人员，这些外包人员甚至不知道自己在生产要转售的数据，直接把东西往 GitHub 上传。尽管目前存在“投入资金过大但交付质量过低”的脱节，但前沿实验室的迭代非常迅速，能够逐步掌握如何筛选并吸收这些数据。Nathan 预计行业需要数年时间转型，通过打造更高质量、更高投资回报率的自动化系统，使实验室能够减少内部清洗过滤的负担，专注于获取更优质的训练信号。 ```mermaid flowchart TD accTitle: 合成数据与RL环境的生产和反馈循环 accDescr: 展示前沿实验室、数据公司与合成环境之间的生产、购买与模型迭代闭环 subgraph Industry["数据市场现状"] A["语言模型生成与变体"] --> B["人工外包与中介介入"] B --> C["合成数据与任务环境"] end subgraph FrontierLabs["前沿实验室"] C -->|"数十亿美元采购"| D["内部清洗与信号过滤"] D --> E["模型能力提升 (RL)"] E -->|"更强能力引导"| A end ``` ### `32:15` Trillium 的愿景：打破短期暴利，为开放研究铺设春季地被 为了对抗这种将高杠杆任务封闭化的趋势，Nathan 与华盛顿大学（UW）及 AI2 合作（例如与 Hamish Ivison 合作推进的 TMAX 项目），尝试用前沿模型合成终端任务（terminal tasks）数据。通过多组提示词生成多样性并排除 bug，只要研究人员逐一审核数据点，质量就能显著提高。然而，现阶段极少有机构愿意自掏数千甚至数万美元去打磨并开源这些环境，因为商业变现的诱惑实在过于巨大。在当前市场中，一套优质环境数据甚至能卖出上百万美元，如果五个月内就能凭借初创公司拿到高达数亿美元的九位数估值回报，很多优秀人才就会选择从博士退学或放弃开源。 这种短期逐利的心态正在透支学术土壤，使外部研究者难以获取能够支撑长期研究的基础设施。正是出于这种考量，Nathan 将自己的非营利研究实验室命名为 **Trillium**（延龄草）。延龄草是一种在早春树冠尚未长出茂密叶片前就破土绽放的短命植物，它在春夏生长周期到来前为整片土地输送养分。Trillium 的目标就是提供开源环境、模型生成结果（rollouts）与可复现的基础设施，让外部研究人员与博士生能够深入探究模型对评测的感知（eval awareness）、失控定义（loss of control）以及在大规模 GPU 上经过一周 RL 训练后的行为演变。Nathan 指出，如果更多人支持开源，全行业将在未来十年持续受益。 - **TMAX 终端任务构建**：使用前沿模型通过多样化提示词生成 terminal tasks 数据，人工过滤 bug，形成高杠杆的开放环境。 - **短期逐利对开源的冲击**：研究数据在商业市场价值百万美元，初创公司短短数月即可实现九位数估值回报，导致开源环境极度匮乏。 - **Trillium 实验室的寓意**：如同早春林下的延龄草，在资源与大树完全遮蔽生态前开放养分，为年轻学者与教育研究留出空间。 ### `35:00` AI 时代的科研浮躁：智能体噪音与深耕者的缺失 在开源协作层面，理想状态是吸引全球各地的研究者出于对知识的纯粹追求做出分布式贡献。但现实中，AI 智能体（agents）的普及给开源生态带来了全新的烦恼：大量低质量、由语言模型自动生成的虚假研究结果和 issue 泛滥成灾。Nathan 分享了自己维护开源论文代码库的亲身经历——常常有自动化生成的 issue 跑来声称“在你的数据集中发现了某问题”，而 Nathan 只能无奈回复：“如果你读过论文，我们在第二句话就已经明确说明了这一点。”这种无需深入思考、仅凭模型一键生成的无价值反馈正在消耗维护者的精力。 真正的研究突破依然要求学者耐得住寂寞。在踏入一个新领域时，研究人员通常需要花数周时间研读文献、面对一无所知的挫败感，反复打磨之后才能问出有价值的问题，并且必须亲自动手写几个月的代码，经过 6 到 18 个月的漫长等待才可能获得认可。Nathan 提到，曾有一位东欧开发者默默帮忙优化他书里的代码，这种纯粹的贡献者在当下尤为珍贵；但现在的语言模型让许多人习惯于一键提交表面内容，剥夺了深入钻研的过程，连数学家们也在抱怨这种浮躁趋势。学术界的博士生虽然举步维艰，但 Nathan 相信这并非无解，只要利用好技术去构建质量过滤器，依然能吸引真正有心推进科研的人。 ### `37:46` 递归自我改进（RSI）：解构神话与研究者被替代的焦虑 对话随后转向了递归自我改进（Recursive Self-Improvement, RSI）这一极具争议的话题。RSI 在公众与业界引发了广泛恐惧与狂热，部分人甚至担忧它会导致智能爆发式飞跃（takeoff），从而使前沿实验室彻底与其他竞争者拉开不可逾越的鸿沟。这种技术叙事带来的恐慌不仅存在于外界，就连顶尖研究人员自身也深陷被替代的危机感之中。Nathan 观察到身边的顶级同行开始自我怀疑：“过去几年里，我看到我的一些最好的朋友——他们是世界上在这个领域最顶尖的一批人——然后他们去上班时却对我说：‘Nathan，我很痛苦。我接下来该怎么办？’”（“I've gone through the last few years, some of my best friends, I'm like, you are some of the best people in the world of this. And then they go to the work and they're like, Nathan, I'm struggling. Like, what am I going to do?”） 研究人员产生恐慌的根源在于，他们日常工作中的许多环节（例如写验证代码、跑基准测试）都是高度可验证的任务，而当前的 RSI 在提升模型的能力验证（capabilities）方面表现优异。相比之下，在需要非分布外（OOD）思维与创造力的对齐（alignment）任务上，RSI 却显得乏善可陈。正因如此，Trillium 希望开源 RSI 的具体实现过程，剥除围绕在其周围的神秘化面纱。通过公开展示实验室内部究竟是如何利用模型协助做研究的，行业才能客观测定模型的能力边界，让政策制定者与学术界看清它到底能做什么、不能做什么，从而建立更理性的校准认知。 ### `40:00` 去神秘化 RSI：通过透明实验记录 AI 自动化研发的真实边界 围绕递归自我改进（Recursive Self-Improvement，简称 RSI），外界充斥着过度炒作与神秘感，仿佛大模型已经无所不能。Nathan Lambert 强调，Trillium 作为一家非营利性质的开源实验室，其核心任务之一就是将这层神秘面纱揭开。他们计划构建一套涵盖监督微调（SFT）和 mid-training 的开源技术栈，结合 SkyRL、Miles 等工具，并公开自己所使用的任务、配置与算力环境。他们会与 Claude、GPT 等模型协同工作，记录并公开完整的轨迹数据（trajectories），详述究竟交给了模型什么任务、为何选择这些任务，以及模型是否真正搞定了它们。每隔六个月，团队便会重新审查这一基准，观察模型的自动化能力究竟发生了怎样的实质改变。 ```mermaid flowchart LR accTitle: Trillium 去神秘化 RSI 的透明研究闭环 accDescr: 展示 Trillium 如何通过执行任务、记录轨迹并对公众和政策制定者开放，从而破除对 RSI 的盲目恐惧。 subgraph Execution["实验执行"] A["设定研发与代码任务"] --> B["调用前沿模型并记录轨迹<br/>(Claude / GPT / 开源模型)"] end subgraph Tracking["追踪与记录"] B --> C["暴露真实卡点与 Bug"] C --> D["归档公开任务环境与配置"] end subgraph Impact["外部影响"] D --> E["每 6 个月复测评估能力变化"] E --> F["为政策制定者提供客观依据"] end ``` 这种做法不仅跳过了前沿实验室含糊不清的分类器和模糊描述，更展现了真实科研中 AI 的真实状态。Nathan 认为，AI2、Trillium 等实验室做的事情，与顶尖产业实验室或学术界某些派别在本质上并无二致，区别主要在于后者的资源被放大了而已。他明确指出，把模型卡在哪里、写出了什么 Bug 摆在阳光下，具有巨大的公共服务价值： > “And it's just like, so good for the world when everybody's talking about this to just to show people like, this is what Nathan asked his model to do. And this is what it got stuck on. And I think that will help a lot of literal policy makers.” —— Nathan Lambert 针对能力上限，Nathan 分析称，模型在优化自身推理效率、降低 Serving 成本方面表现极佳，这能帮背负数十亿美元成本的大公司省下真金白银；但在解决复杂的知识型工作（如自动把 PPT 做得更好）或真正理解复杂真实环境方面，模型距离端到端闭环仍有漫长距离。Lisha Li 对此深表赞同，学术界教授和博士生根本无暇将 40% 的时间用来写文档记录任务，非营利研究机构承担起这一职责，不仅能提供极具稀缺价值的评测数据，更能有效降温围绕 RSI 产生的恐慌情绪。 - 行业内有多家组织也在推动类似透明化倡议，例如 EleutherAI、Hugging Face、斯坦福与 Marin 合作的 OpenAthena，以及刚成立不久的学术联盟 Open RSI。 - 前沿实验室内部员工普遍产生“自己一年内就会被模型自动化”的焦虑，导致具体组织架构内真实研发细节愈发与外界隔离，更加需要外部独立验证。 ### `45:00` 预训练并非天价死局，后训练才是真正狂野的“加速战场” 谈及前沿实验室的真实壁垒，Lisha Li 指出，实验室所用的算法其实并非绝对秘密，关键在于他们拥有足够的资本去大规模试错，摸索出最佳参数与配比。Nathan 对此表示认同，这也正是 Trillium 选择不涉足预训练的原因：不碰预训练，就可以自由选用任何优秀的开源基座模型，坐享开源竞争带来的红利。从科学角度看，预训练工具已相当普及，凭借预训练阶梯（pre-training ladder）这一决策工具，团队可以检验自己的想法是否具有扩展性。而且互联网预训练数据在第一近似下是相对固定的，架构虽在演进，但做出一个可靠基座模型并非天文数字；即便没有 100 亿美元，也依然能打造出有价值的基座模型。 Nathan 进一步剖析了开源创业公司的生存状态。许多初创公司往往直接 Fork Megatron 等开源训练库作为基础，随后转为内部闭源推进。初创公司面临的真正考验并非能否训练模型，而是首个发布的模型必须足够出色，能让外界买账并支撑下一轮融资；若一家公司宣称要做出最强模型却迟迟无法交付，就会陷入极其被动的倒计时。他以 Thinking Machines（推出 Inkling）为例，称赞其第一代模型虽然不算最顶尖，但团队透明承认，随后借由第二代模型与 Tinker 框架顺利站稳脚跟；同时他也提及扎克伯格（Mark Zuckerberg）在 Llama 3 发布时阐释开源前景的博文堪称范本，市场始终对 Meta 持续开源抱有期待。 在探讨预训练与后训练的技术难度对比时，Nathan 明确提出，相比于流程相对可控的预训练，后训练（Post-training）与中训练（Mid-training）已经演变成极为复杂的非线性迭代： > “I think to do a controlled, let's just start pre-training our models seems a lot more approachable to me than I don't really know much about language models. I'm going to figure out this whole post-training thing where it's just like the Wild West.” —— Nathan Lambert 当前的研发重心已从单纯堆叠预训练转移到端到端技术栈的协同演进：超长上下文生成与稀疏注意力机制（Sparse Attention）使得生成速度提升 40%，推理扩展（Inference Scaling）带来了更可观的回报；而在 Mid-training 阶段，研究人员需要从旧模型中抽取推理行为，并利用强化学习（RL）进行行为播种。Nathan 坦言，此前离开 AI2 时，便深感团队的后训练研发因迭代周期（cycle time）不够快而有些掉队。他坚信，开源后训练目前的门槛其实还很低，并不需要天文数字般的资金，只要能将迭代速度提上去，就能在开源领域带来非常巨大的复合收益。 ### `50:00` 智能体群架构与闭源实验室的信息溢出 对话深入探讨了后训练阶段（Post-training）与智能体系统评测架构（Harnesses）的结合。Lisha Li 提出，头部闭源实验室正在为特定评测与运行架构量身定制、后训练自己的模型，甚至能直接调度数万个智能体协同工作，这种高度垂直整合的工作流令外界感到高深莫测。Nathan Lambert 回顾自己早期的判断，曾认为闭源实验室会像苹果公司（Apple）那样，依靠模型与底层测试架构的软硬一体整合形成无可比拟的规模效率，而开源模型需要适配五花八门的架构，会成为致命短板。然而现实演变却趋向平衡：用户对于 Harness 的需求极度多元，迫使闭源大厂反过来去适配各种各样的外部调用方式。针对超大规模智能体群（Agent swarms），Nathan 认为 OpenAI 和 Anthropic 确实会率先建立协调系统，但当多智能体协作范式明确后，开源生态会迅速跟进并复制类似机制。类似 Kimi 这样的团队受限于天量算力，无法率先探索万级智能体集群，但随着行业周期轮转与英伟达（NVIDIA）新一代芯片迭代，算力壁垒将逐渐被平摊。 针对前沿技术能否长期保密的问题，Nathan 认为硅谷的信息流动性极高，任何前沿突破都很难在水面下隐藏太久。一线研发人员本质上都是科学家，一旦做出突破性成果，在一两杯啤酒的聚会闲聊中就会走漏风声。因此，闭源实验室真正的护城河不在于神秘的算法机密，而在于推理基础设施的工程极致化——将单位能耗（Per Watt）、单位评测分数、单位用户的推理成本降到最低，从而在商业变现中占据主导地位。 “硅谷向来以思想传播极快而著称。就像大家常说的‘喝一杯啤酒的功夫’，下班后所有人都在交流。”（Nathan Lambert 原话：“Silicon Valley is famous for like ideas traveling. Like what is it? Like one beer or something. And it's like everybody chats after work.”） - 行业正处于多智能体协同机制的摸索期，未来或将形成专门定义智能体群的协调系统（Coordination system）。 - 开源模型初期无法率先试水超大规模智能体群，核心瓶颈在于缺乏天文数字级别的推理算力。 - 闭源实验室最坚固的壁垒是在其基础设施中塞入海量推理能力，以最低的价格和更高的利润率持续向市场输出智能。 ```mermaid flowchart TD accTitle: Frontier Labs to Open Source Diffusion accDescr: Diagram showing how agent swarm workflows and compute advantages diffuse from frontier labs to open-source models. subgraph Frontier["前沿实验室优势"] A["资本与天文级算力投入"] --> B["探索万级智能体集群协作"] B --> C["推理基建降本与软硬件深度整合"] end subgraph Diffusion["信息扩散与演进"] C -.->|"科学家私下交流 / 范式公开"| D["架构与调度方法明朗化"] D --> E["下一代硬件迭代 (如 NVIDIA 新芯片)"] end subgraph OpenSource["开源生态跟进"] E --> F["开源与区域性实验室复制架构"] F --> G["生态达到均势平衡"] end ``` --- ### `55:00` 递归自我改进的物理边界与社会应对 在讨论前沿模型是否会发生“快速爆发（Fast Takeoff）”和不可逆拉开代差时，Nathan Lambert 指出了当前 Scaling Laws（缩放定律）所包含的内在数学约束。目前的自回归框架存在着算力呈指数级增长、而性能仅获得线性提升的客观规律。虽然前沿实验室的员工和资本正在为微小的性能增益投入呈指数级增长的推理算力，但只要核心架构仍建立在 Transformer 和数学逻辑的特定智能形态之上，就不大可能出现完全摆脱人类干预的指数级自我进化。即便 Anthropic 的 Claude 系统卡（System cards）也明确提到，要达成失控性的快速飞跃，当下的技术进展速度至少还需要再翻十倍（10x）。 Lisha Li 与 Nathan Lambert 一致认为，真正的风险并不在于遥不可及的递归自我改进（RSI），而在于当下已知且正在扩散的实际危害。随着闭源能力不断被开源模型追平并扩散至整个社会，在公共网络安全、防御策略等方面的准备几乎是一片空白。Nathan Lambert 强调，他不认同激进的技术快速飞跃论，但这绝不意味着可以对现存的 AI 扩散风险掉以轻心。这也正是他创立 Trillium 这家非营利研究机构的核心初衷：为学界与社会公众揭开 AI 研发的黑盒，提供更多独立的科学证据和客观透明的观察视角。 “所有缩放定律的本质，都是用指数级增加的计算资源去换取线性的性能提升。”（Nathan Lambert 原话：“All the scaling laws are exponential increase in compute for linear increase in performance.”） - 数学领域因其清晰的形式化边界，被认为是当前大模型范式最契合的应用科学领域。 - 自回归模型所消耗的资源是现实可测且受物理约束的，这限制了闭源实验室无限甩开开源生态的可能性。 - 相比于虚无缥缈的超智能快速爆发，社会各界更应正视模型能力在现实网络安全防御领域的扩散冲击与应对不足。 ## Notable quotes > `55:00` All the scaling laws are exponential increase in compute for linear increase in performance. — Nathan Lambert > `07:54` To say that China is anything but equal on talent, you're going to have a hard time because they just have so many people being trained at such a high level. And what they don't have is compute. — Nathan Lambert > `00:00` I think those two companies in particular have been the most forward-looking and the most willing to spend egregious amounts to make a step towards a direction that people think is impossible. — Nathan Lambert > `15:30` 如果你试图对蒸馏做任何干预，最终造成的伤害将远大于收益，因为他们根本不会停手，他们只会继续做下去。 — Nathan Lambert > `13:30` 我认为这有点像自我安慰，人们总觉得‘只有我们才擅长做模型’，但事实上，现在有大概 20 到 30 家公司都能做出非常出色的模型。 — Nathan Lambert > `20:00` 但在这一流程的终点，他们拥有真正的所有权：那是属于自己的模型权重，没有苛刻的服务条款，能跑在自己的硬件上，并且他们算清了这笔账能带来投资回报（ROI）。正因如此，我认为开源模型经济的建立需要漫长得多的时间。 — Nathan Lambert > `45:00` I think to do a controlled, let's just start pre-training our models seems a lot more approachable to me than I don't really know much about language models. I'm going to figure out this whole post-training thing where it's just like the Wild West. — Nathan Lambert > `17:30` 闭源实验室高度垂直整合，拥有极致高效的推理架构，当他们准备好时，凭借提前规划的大量资源储备，能够真正将模型进行超大规模部署。 — Nathan Lambert ## Mentioned Nathan Lambert, Lisha Li, OpenAI, Anthropic, Google, Meta, Trillium, 月之暗面 (Moonshot AI / Kimi), 智谱 AI (Zhipu AI / GLM), 英伟达 (NVIDIA), 黄仁勋 (Jensen Huang), 马克·扎克伯格 (Mark Zuckerberg), Claude, Claude Code, Claude Opus, GPT, Llama 3, Cursor, OLMo, AI2 (Allen Institute for AI), Qwen, 小米 (Xiaomi), Decagon, Harvey, Prime Intellect, Together AI, Astra, Blender, SQLite, TMAX, 华盛顿大学 (UW), Hamish Ivison, SkyRL, Miles, EleutherAI, Hugging Face, OpenAthena, Open RSI, Megatron, Thinking Machines, Inkling, Tinker ## Video description a16z Partner Lisha Li sits down with AI researcher and Trillium founder Nathan Lambert to ask how durable the gap really is between frontier AI labs and open models. Nathan argues that OpenAI and Anthropic have structural advantages in compute, capital, and the ability to pioneer new training paradigms, but open-model labs, particularly in China, have proven remarkably capable of following close behind. They unpack how much of that comes from distillation versus technical talent, why compute remains a major constraint, and how data, RL environments, and post-training are increasingly shaping model progress. They also discuss Nathan’s new nonprofit research lab, Trillium, and its plans to open up parts of AI research that are increasingly difficult to study from the outside. That includes releasing RL environments and documenting how AI is actually being used to automate AI research, with the goal of demystifying recursive self-improvement and giving researchers and policymakers a clearer picture of what frontier systems can and cannot do. [Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/9faf0f04-b106-4419-94b2-7882faca15c2/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=9faf0f04-b106-4419-94b2-7882faca15c2&feed=JGE3yC0V)

[55:00] Nathan Lambert: All the scaling laws are exponential increase in compute for linear increase in performance. [7:54] Nathan Lambert: To say that China is anything but equal on talent, you're going to have a hard time because they just have so many people being trained at such a high level. And what they don't have is compute. [0:00] Nathan Lambert: I think those two companies in particular have been the most forward-looking and the most willing to spend egregious amounts to make a step towards a direction that people think is impossible. [15:30] Nathan Lambert: 如果你试图对蒸馏做任何干预，最终造成的伤害将远大于收益，因为他们根本不会停手，他们只会继续做下去。 [13:30] Nathan Lambert: 我认为这有点像自我安慰，人们总觉得‘只有我们才擅长做模型’，但事实上，现在有大概 20 到 30 家公司都能做出非常出色的模型。 [20:00] Nathan Lambert: 但在这一流程的终点，他们拥有真正的所有权：那是属于自己的模型权重，没有苛刻的服务条款，能跑在自己的硬件上，并且他们算清了这笔账能带来投资回报（ROI）。正因如此，我认为开源模型经济的建立需要漫长得多的时间。 [45:00] Nathan Lambert: I think to do a controlled, let's just start pre-training our models seems a lot more approachable to me than I don't really know much about language models. I'm going to figure out this whole post-training thing where it's just like the Wild West. [17:30] Nathan Lambert: 闭源实验室高度垂直整合，拥有极致高效的推理架构，当他们准备好时，凭借提前规划的大量资源储备，能够真正将模型进行超大规模部署。

[54:30] Nathan Lambert: All the scaling laws are exponential increase in compute for linear increase in performance. [7:44] Nathan Lambert: To say that China is anything but equal on talent, you're going to have a hard time because they just have so many people being trained at such a high level. And what they don't have is compute. [45:00] Nathan Lambert: It doesn't take as much capital to make very big compounding gains on the state of the art in the open source because the bar is very low. [25:00] Nathan Lambert: I'm a big proponent that a lot of the frontier progress in domains that are not fully verifiable is by buying data that is high quality and giving the models worlds to act in and hill climb on. [2:35] Nathan Lambert: If somebody wants to work at the frontier of AI, I mostly tell them to work at Anthropic and OpenAI which is the easiest bet. [15:00] Nathan Lambert: I think open and closed models are on different timescales of exponentials.

[0:00] When I think about model progress, it comes down to different forms of capital, and that is compute data and talent. How do you see that playing out in terms of like who's going to win in the next couple of years? The Chinese are so good at building models. To say that China is anything but equal and talent, you're going to have a hard time. What they don't have is compute. Most people regard distillation as sort of like the main lever for them to do this. People distill my products every single day. They take it, they strip it down to its raw parts, and that's called competition. Previous post training was much more reliant on the supervised fine tuning. Now this is morphed a bit into environments where it's RL heavy. But it's just like hard to map this type of post training, which is you get the completions and the score goes up to what is going on in reasoning today. The discourse now is just so much fear on like how much the frontier is just hidden and also breaking away. The AI capability plot doesn't necessarily worry me. But the AI capability plot in the pretty much non-existing preparation across public cybersecurity and all of these things. The Frontier AI labs are spending extraordinary amounts of money to build better models. So how does open source keep catching up? In this episode, A16Z partner Leisha Lee sits down with AI researcher Nathan Lambert, founder of the new nonprofit research lab Trillium to unpack the race between open and closed AI They discuss why open AI and Anthropic still have major advantages in tribute capital and speed, but why Nathan believes the technical talent behind open models, particularly in China, is often underestimated. They also separate the reality of model to solution from the rhetoric around it and look at how post-training, data, and RL environments are becoming increasingly important sources of progress. And finally, they turn to recursive self-improvement. Nathan explains why Trillium plans to document how AI is actually being used to improve AI, from the tasks models can already automate to the places they still get stuck, and why making that progress visible could lead to a much more grounded debate about how quickly the frontier is really moving. So Nathan, really happy to have you here. Thank you for joining in person. Super timely given you're launching Trillium, which I'd love to talk about, but more sort of on a macro level, I think we chat a lot about open source and that's a lot of what you care about. And what we've been hearing a lot in the public discourse is like, hey, we have to pace the frontier, the frontier this, the frontier that. And I think an arc of the stuff that I really trying to ask experts about is what is in the frontier What are we pacing What there And then more importantly kind of threading it with open source and open weight models like can they keep up with the frontier And I think this matters very much and is something that you an expert on So maybe to sort of start on that arc sort of how you would break down what's gotten us to the frontier for both closed labs and open source models. You know, how does it break down in your mind, maybe first at a higher level of pre-training, mid-training, post-training, how post-training has evolved, which is something you've done a great amount of education to the public on. When I think about model progress, it comes down to different forms of capital that you've put into them. And that is like compute data and talent. And all of the labs have various amounts of this. And the closed frontier labs of OpenAI's, Anthropics, Google's have clearly had the most of these. And what we see is like the different abilities of the labs to organize these forms of capital and like making the org chart work for building models is really hard which is why you have companies have to do complete resets from the bottom up and then it actually kind of works sometimes like company like meta is like totally reset and they're like on a positive trajectory again seriously yeah like i think i'm very interested in these human aspects of why competition drives the frontier and why the org chart is so important because like we see these things and this comes into play in the open closed dynamic as well where it like the closed companies the true frontier labs which i say if somebody wants to work at the frontier of ai i mostly tell them to work at anthropic and open ai which is the easiest bet i think meta google spacex ai like i think you still might be there but it's harder to see from the outside and these companies have just taken the most ambitious bets the earliest in order to get there. And that's why OpenAI has done things like kickstart the reasoning model revolution. And I think that kind of breakthrough and that beautiful plot with the inference scaling laws is peak OpenAI. It's like they're really good at getting this plot that just changes how everyone thinks about everything. And on the other side, it's like Anthropic has been like very focused on the types of things the models are going to be able to do. And then their reward was this cloud code explosion with their focus on coding for so long. And there's these subtly different cultures And I think those two companies in particular have been the most forward-looking and the most willing to spend egregious amounts to make a step towards a direction that people think is impossible. The Chinese companies came into this because they want to have a company that matters. And if they were to build their models and try to sell an API from China, nobody would know about them. Nobody would know about Moonshot AI. Very few people would know about Jibu. [5:00] and there's a very long list after this. It was a very practical decision by them. They're following the discourse. They know there's going to be a huge market. They know there's a huge demand in the U.S. for paying for these things. And it's very practical. So therefore, like, releasing open models is a way to shortcut this and get into the conversation. And I think there's a lot of complexity in, like, why do they do this? Is the government involved? And not. It was, like, not the case. They're just trying to build a startup. And it's not to say that it will not get there. Like, the Chinese government does things below people's minds in the U.S. And I think that makes me think that the Chinese labs have a much more serious approach to security because things happen in China that are much more drastic responses to companies doing things in the U.S. And I don't need to go down this rabbit hole, but between the balance of open and close, these companies in China are trying to catch up. And this is a very different dynamic than what I said about OpenAI and Anthropic. They're trying to create the new paradigms. They're trying to make true RSI happen so that their company... wins with the best models forever if you get there first, these boom ideas. And you just have to fill the market where there is a void. And the open weight models are trying to do this and... I think over time there will be these tiers of open models where there's the best closed models, which you can demand the best margins for. And then there's the near frontier models, which is a step behind this. And those will be open because it's just like some sort of supply and demand there as you get access to a different market that might be less profitable. profitable. For example, it could be a private enterprise data or so on. And these tiers are kind of natural evolutions of companies that are trying to do things. And I think we were talking about the frontier. And it's just if you go back to capital, they have less capital, they're going to make less money. And that's okay. But I think this drive to catch up is very strong. And once you know, once Anthropic and OpenAI show you the type of model you need to build, it's a lot easier trickles through the data economy where open AI and Anthropics spend way more to build out this function of a data industry. And then those data companies a year later might sell them to everybody. They might sell them to everybody. They need to maximize their return on their investment and they had some contractual period. And I think there's a lot of dynamics where fast following is going to be very doable. And we've seen the cycle proceed for a while, longer than many people would expect, I think. When ChatGPT was released, we talked about consolidation in the industry and predicting it. And I think we're just starting to see a much more active mergers and acquisitions market. But I thought this was going to happen two years ago. And the open model companies have lasted much longer than it has. thought. There's so many interesting threads to pull on here. I mean, starting with actually your commentary on the organizational differences between the top two frontier labs. We can maybe table that or go on through that thread. But then, of course, also kind of, are you explaining sort of how the Lopin Labs in China have really just chosen that as much more of a distribution slash practical commercialization strategy rather than, hey, obviously out of altruism. Though it's great that the researchers, maybe for better or for worse, were motivated from sharing the beginning, commercialization strategy. So the other thread maybe to pull there is sort of, I think most people regard distillation as sort of like the main lever for them to do this so well and kind of increasingly well because the lag between the open source models and the frontier models have only shortened with time in the last few years. Now it's like maybe only two to three months. And this has also pushed the frontier labs to maybe want to close off the APIs, close off their One, probably to capture more value because it's always possible that way, but then two, also because it's harder to distill, not impossible, but much harder to distill. Maybe on those two threads, because they're both very interesting, so we'll try to touch on it. Let's start with, is the distillation story really more of a caricature in the Chinese labs since you've actually met with them and you've studied this a lot more? Is it sort of like they're all-rounded, great at pre-training, also have their own good post-training recipes that they may or may not have revealed? And where do you see the levers kind of accelerating? Chinese labs are very technically excellent. They are more compute-starred. Yes. Without a doubt, they have a comparable technology talent pool to draw on. I think there might be a slight edge based on who was where when AI really got going. But to say that China is anything but equal on talent, you're going to have a hard time because they just have so many people being trained at such a high level. And what they don't have is compute. You talk to people there and you're like, oh, I heard an average OpenAI researcher has 5,100 equivalents that they can access at any time. And like their jaw just falls to the floor. They're just like, I can't comprehend. And this is like a top pre-training researcher at one of the labs that you would know. And then you have this interaction a few times. And I'm like, The compute thing, I think, is the biggest difference in [10:00] distillation is a way to effectively... spend more money to get what is compute equivalent. Like outputs from a language model are a function of compute. And if you know the outputs you want, it's the same as having compute to generate with a model. And, I've seen the stages of distillation and how I see it is that Previous post-training was much more reliant on the supervised fine-tuning and other to like this RLHF era, which is where post-training started. And. you could get most of your performance just by you do this next token prediction loss on the outputs of a very strong model. It was very established that you take the best model you can, you generate completions, you train on it, and your model's going to be pretty good so long as you have access to the right prompts. Prompts have always been a key to post-training. I remember at AI2, like, with Hannah who I co-led push running there with, we would always say like prompts are the key unit of, post-training. And so you need to get the prompts and you can do different things with them at different stages of training. And now this has morphed a bit into environments where it's RL heavy. But it's just like hard to map this type of post training, which is you get the completions and the score goes up to what is going on in reasoning today. Mm-hmm. Based on the public literature, it's very hard for me to be like, distillation is a major factor because, for example, in the fully open literature of people trying to do open post-training recipes, all the open model builders are stuck using a really outdated Quen model because you can't figure out how to get the teacher right. My hypothesis is that there's some mid-training and something in the common pre-training data that's open that makes it hard to do this thing, which is get reasoning traces from the absolute best model and do S of T and the numbers go up. So if I were to have access to Claude's reasoning trace when I holding Ulmo, I'm actually like Claude Opus, Claude Mythos, whatever, I'm not sure that it would have actually made our model a lot better. So this is like my evidence grounded in the literature, why I think distillation is hard. And then like, how do you take the outputs of a model and build an RL environment to do a big RL run? Because then you have the Xiaomi's of the world doing this week-long open RL run, where they're like, here's what we did. And then they've dropped the model. And it's like, where exactly does distillation go into this? They're clever. They know to do marketing well? There's some questions of like, did they... do this ahead of time because they did their RL run and then they dropped the model and then their tech report said they had an extra post-training stage after RL. So there's some questions. But like, mapping the established distillation literature to that is very hard. But there are question marks that make me support the labs a bit, which is like these Claude proxies and these model proxies in China where there's a ton of tokens that are used and there's intermediaries and they have tons of accounts and even cases where Kimmy was serving Claude in their products to users. And I'm just like, don't really understand how this fits into distillation as a common technical tool in post training. And like, yes, we should watch it to like there's other weird stuff with Chinese companies using the outputs from Bitcoin. open AI and anthropics models at a vast scale. So I think we have a lot more questions we need to answer. And I think most reasonable people when they're like, what's the gap from distillation are like, there's very big error bars. But when I say that distillation helps them by a few months, it's based on the scientific literature that we just like, that's the information we have, we should do more open research and enable this so that we can study it. But like, I don't know how you can come to a different conclusion that it's like not a huge effect are so good at building models as well. I think there's a bit of cope where people are like, only we are good at building models. And it's like, there's so many companies. There's like 20 to 30 companies that are building good models. And I think you just, and I don't know. It's hard for me to get over. And I think that there's this very prominent paper that's like stealing reasoning traces from closed models. Mm-hmm. I know that the Chinese companies... are way ahead on doing this since the AI safety community later published it. Like, I think every Chinese lab has a team that knows those sorts of things. And this goes back to what you're saying about they don't need to offer the API. And I think there's some characteristic of reasoning models that generate these thinking tokens where it's easy to just knock them out of the API so you see the reasoning token, and then you can use that to train your model. And so long as they offer these APIs, there's going to be no way to stop that type of distillation. But if they do make it so it's only in products and things where you have to sign enterprise agreements or whatever, it'll be way harder. And I support the labs doing that if they want to secure it. But it's like they have to secure their products if they want to. I don't think no, your customer would be good for anybody. I don't think it'd be good for their business. But also, I don't know why you should get the government involved in a terms of service violation. I don't even think terms of service are a signed contract. I don't think you can sue them in the same way. You just can cancel their access. So that's why I'm not that... sympathy. [15:00] but I like to listen to the national security community is loud, but there is a directional truth to that with where AI is going and how influential it is. You have to balance many, many things in your head. I wonder what your response is. So I heard, I think Jensen said in some... Literally either tweet or interview. It was a great, it was a great quote. Put it in the show. People distill my products every single day. They take it and they strip it down to its raw parts. They test everything. That's called competition. Do you think that's fair game? You think distillation is not robbery? That's called competition. I'm very sympathetic because I think there's a lot of calls from the AI safety community that we want to have like U.S. trying to dialogue as we're getting to such capability levels. And if you're going to ask the company to just like... start hitting their AI industry with slaps on the wrist and more downstream of distillation. So you won't regulate distillation with policy from Congress. You will probably just do federal action to make their life annoying. And at the same time, when we're talking about how do we manage frontier risks where you have closed models and training going all over the web and accessing things that they shouldn't, and we have valid worries about what will happen with open weight models where why aren't people using open-way models? Why do they go to closed models and get around the safeguards? We want to be able to communicate with the Chinese tech companies and everything about this. And I just think you're going to cause more harm than good if you do anything about distillation, because they're not going to stop. They're going to keep doing it. Right, right. And the open-source community benefits, and therefore the startup community benefits a lot. It's not just like, hey, China, USA. And we can actually talk about, obviously, or trying to steel man the frontier lab position, which is like you have just unprecedented amounts of capital that's required to build these frontier models. And it's kind of alarming slash not very conducive to this flywheel continuing if the through distillation or other kind of distillation related methods, open source can catch up so quickly. picture of, you know, you train the frontier model by putting in a lot of money to get compute, and then they park this compute, of course. OpenAI and Anthropic are very guilty slash successful in this. Whichever way you want to look at. And there is a huge compute, you know, kind of... sort of lack of compute available because there's actually like not just compute in the chips, but there's also power, et cetera. So like all of this scarcity is kind of, you know, bottleneck until 2028. So this is all to say that it's a successful strategy to park that, which leads to actually a lot of revenue, because once they get to the frontier, because they understand, you know, scaling is the one thing that kind of gives them or the science of scaling is what gives them a handle over how well that can translate like into a predictable revenue stream. So they get the revenue, and because of that revenue, they're able to raise more money. It's just unprecedented, like, beautiful flywheel. And this happens for both OpenAI and Anthropic. Of course, we need exponential amount of resources. And so we've gotten to the point where we're like, OK, we need to IPO. We need to go to the public markets. And so anything that threatens this flywheel from functioning well is a huge threat to consumers. just the entire like being at the frontier. And so the entire industry, realistically. And exactly that, actually. So like, you know, maybe not just kind of being sympathetic to their version of like trying to, you know, eke out capitalism for themselves. It's also just like, hey, if if they don't do well in the IPO, that's actually bad for everyone else. And so I want everyone, you know, I want them to do well. Yeah. On the other hand, you know, it's like, the Chinese open source models are a huge benefit to the startup ecosystem. I mean, we see this all the time, just kind of metrics. I remember from like Decagon and Harvey, who was trying to do a lot of fine tuning. There's like a, Huge amount of startups doing post-training as a service of various degrees and trying to prioritize that. And it actually works because you can get to the Pareto frontier with these, you know, Kimmy K3 or pick your favorite like GLM open source model. So I guess like weighing these two things, how do you... I guess there's so many questions can fall from that, but I guess one is, How do you see that playing out in terms of who's going to win in the next couple of years versus what does? And then also relate that to why you're doing, obviously, an open source. So I think open and closed models are on different timescales of exponentials. And the closed labs are so integrated where they have efficient inference, they can serve the model at scale when they're ready to because they have all these resources they can plan. [20:00] they get to these markets first that are very high value. We have seen this with coding, and this has been the story of the year where the revenue has just gone zero, 100 billion. And likely to grow, but the open model labs... partially this is like an economic issue being the fast follower is that you don't have this flow of information and you don't have this flow of revenue so the open model you have an open model builder then you have the model inference companies and you have the open model post-training companies and like there's information from those customers which are like things you need the models to do and it's like you can't like that information doesn't just flow back to the model builder very easily and they also don't have the economies of scale to serve as many users to serve it as to have as much margins on imprints. And these are just like, it's going to take a lot longer. But to the extent that things can be commoditized, it seems like companies really want to have private data. There's a lot of things that they just cannot send over an API. And we could talk about the zero data retention or not and so on. But it takes a long time to figure out if a company has internal expertise or they figure out how to build an internal evaluation that actually matches to revenue. And then they figure out how to fine-tune machines or prime intellect. I'm like, we could make money in mass if we figure out how to fine tune a model that solves this. And they have these customers, but there's just so many more companies involved that it takes so much longer. But at the end of this process, they have real ownership, which is model weights, no terms and conditions that they can run on their hardware, and they figured out they can make an ROI on. So this is why I think the open model economy, it takes way longer to build. And I'm less concerned about this being kind of bubbly, of the really fast takeoff as people figure out what is the cheapest model that accomplishes a task and how do you compete on margins. Mm-hmm. And I don't really know what's going to happen. It's too early to know. It's so hard. I mean, it's definitely something that's always top of mind thinking about. There are things where scaling up general reasoning just as a general model is so, it's going to beat out any specialized model. And we see, obviously, examples in math. And that's probably the most pure domain that you can see this. But also, you know, even in, say, robotics, you know, with Astra, and I think, for maybe however degrees of like, trucy the signal is, it just doesn't sound like they've actually tried to ascend in robotics. Oh, I think they did. I think this is a cycle of like six to eight months ago, they started hiring people and buying a lot of data on robotics. They did. And then the model, magically six months later, is really good at it. But I think that the... But I actually, well, the other, you know, more contrary things I've heard is that it's actually coming out of computer use because it's, you know, because computer use is like, some level of spatial reasoning and then getting to have good blender, you know, demos is another extension of that. So, you know, we can believe some version of that, but either way, I guess it's like it. Maybe if it's really specialized, you see the capability gain is really from trying to get better in a certain area. But in math... I don't even know if, that fell out of trying to specialize in math. I mean, I think the whole thesis there was just like, let's just scale up reasoning and actually got better in natural language. It got better than any specialized model in math. I mean, Anthropik wasn't even trying to be good at math. You know, OpenAI, you know, certainly I think, you know, they hired a bunch of mathematicians, but they also weren't trying to like use specialized, you know, kind of like lean data set to get better at it. At least that's not, sorry. So anyways, this is all kind of fodder for the view that like it's kind of hard to compete with the frontier models because eventually they start ascending so much that, you know, you get good enough in respective categories. Specific benchmarks and data sets are only available in these companies that you're being applied to. And it's just good enough. And it's actually we're starting to see people adopt that. I mean, in the most maybe pure regime where you would expect frontier models are good as I think Cursor published something over the summer where like, hey, we just, you know, we had this long horizon task of writing, rewriting SQLite. And turns out using two frontier models, you know, one for planner, one for execution is not as good. good or is equivalently as good, but far more costly than just using Composer with maybe Opus or, you know, Vastra, whatever, as Planner. And I think that's actually great evidence that the open source fine tunings are very good or good. [25:00] good enough and has surpassed intelligence hungry tasks. There's a long way of asking, What evidence do you find is most compelling or pieces of evidence for various versions of this stable state of where the frontier wins versus open source? I'm a big proponent that a lot of the frontier progress in domains that are not fully verifiable is by buying data that is high quality and giving the models worlds to act in and he'll climb on. And I think... There's two things. You have the math and code code. thing. And we don't know how big the coding market will be. And then there's kind of everything else, which is still they're targeting this knowledge work, this biology, healthcare, legal, and all of these things are not as verifiable. And I think it is much closer to the story of opening. I bought a lot of robotics data. They're probably going to tell us it's generalization, magic poof. People love poof generalization. But realistically, it's probably some of A, some of B, but a lot of it comes down to data. When I talk to people at Frontier Labs, there's some people that have amazing jobs. There's some people that contribute by doing a very narrow thing, which is you make a data set and you figure out what the behavior should be for your model in that area and you figure out how to make the data that helps get that data and get that behavior and you figure out what data you need to buy. And that's going to take I think we're going to have years of this across different sectors. And that's very capital intensive to buy all of this data. But that's not just like poof, magic, RSI happens. Where I do see in this math and code and some distributed ML engineering, it's very verifiable. You know what your cost is. And the AI models are going to drive that down at a really magical rate. And that's going to just have a lot of surplus benefits. And we can do research faster. I think the efficiency of models is going to drop multiplicatively over years. So we've seen OpenAI have drastic price cuts. And it's like, what does it mean for GPT, Sol, or Astra and these bigger models to reduce in price 2x every eight months for the next year? three years. You start getting, you start doing this math and it's like, it's 1% of the cost and whatever, like four years. And it's like, are they going to be able to have higher margins due to Jevin's paradox there? Or their revenue is going to decrease because the market fills up? There's just like all these really weird dynamics, but fundamentally like AI progress is, about data and about writing better software to serve your models. And those are very different worlds. Yeah, I mean, at least definitely, I agree with you that observing from the outside, there's a lot of data, RL environment companies that are doing very, very well, selling both to the top two frontier labs, but an increasing kind of longer tail of labs as well. So to your point, I think a lot of the capability gain has really come from just scaling that up. And it's not an easy task. I mean, maybe there's... I'm actually very curious on your opinions between the data collection and environments that are built by hand versus the ones that are, you know, done in sim. There seems to be like different versions of like what could be, you know, what kind of price they command, et cetera. They all have good things and bad things. And I do think there's a lot of talk of Lee. We've talked about distillation, but the other side is the Chinese company buys the RL environment that they use six months later. Exactly. Oh, anthropic reopening. I bought this. Now there's a premium. The good data will do this. Yeah. But the data that people are buying is pretty flawed. I would say that the method is you buy in mass and you figure out what works and it's not 100% hit rate. Yeah. And I think... I personally think that this RL environment explosion, where there's just so much opportunity to make money, is having a lot of people cut corners. And this can induce things like reward hacking. So it's like the models are so smart that if you put them in a world and they're like, my job is to solve this task. But they didn't give me the, like, you have a test file and the test file actually has an empty file instead of the maze that you have to navigate. And they're like, the model is like, well, I should still figure out how to get a high reward anyways. stream of this. supply chain and I think it'll fix it, but it's interesting to see the demand from the labs to eliminate reward hacking and all these companies, researchers spending so much time on it. And then all the data companies like buy our data. and I think proportionally spend less time on model behavior and more time on world building and revenue. And somehow these two parties need to meet together to actually make higher quality data. Yeah, well, a lot of the labs are actually taking it in-house, partially because they don't want you know, data leaking. Not that, you know, like literally and figuratively speaking, it's like, you know, signals leaking of what they're doing, what they're trying to send on. But despite that, I think there's a tremendous amount of growth in how much they're purchasing. Do you think that's probably one of the strongest levers? Can you kind of qualify how much that will kind of work? [30:00] increase, do you expect that to increase based on spending? Or are there other, you know, because for instance, if it can be done well in STEM, I think maybe that isn't more of an argument to that being absorbed by labs and you know, in particular. And then all the companies are doing a mix of like all the companies are selling synthetic data where they're using language models to modify or build the world or generate permutations of it. And like if a human is guiding this process, it's still synthetic data. So there's very few people that are going to like sit there and. lender and we're going to make a little world for our model to act in. All the companies are moving far faster than that. And they buy them because the models are getting better from them. I still have denial about it. I'm like, these numbers are crazy. The billions of dollars research budget on environments. I'm like, they're only doing this because it is helping the models and they see the signal. That's true. And I think my general observation is that The data industry has a lot of low-hanging fruit. on improving quality and improving just like So if you look into the details, you can find weird things with data companies that use contractors overseas. Those people are through an intermediary. They don't know they're selling it. They just upload it to GitHub. There's a lot of stuff out there that if you poke around the data industry, you find weird rough edges. It's just very early. And that's like, it's the one thing is like, the numbers are too big. The quality is bad. But on the other hand, the labs are, improving very fast and know how to onboard it. So I think this will take a few years to transition of like really pulling the winners into higher quality, higher ROI. It just takes less resources when you have these systems for doing better. And then the labs keep working with them because they get even more training signal and have to do less internal filtering. I think this is going to just take years. How do you think about then using environments and like your purchasing decisions will be making open source environments for tasks that are very high leverage. So I think of this work that I advise UW and AI2 State and Hamish Iveson on this TMAX work, which is you use a few, it's for terminal tasks, which are like super popular at Terminal Bench, whatever number we're on. And this is fully synthetic data where you prompt a frontier model to generate a type of problem. And you use a few different prompts to generate diversity. to bugs in them and eliminate the ones with bugs. And this process is not magic. It's like you read every data point, your data will get better and you can do this. But there's just not that many institutions that are investing what it takes thousands or tens of thousands of dollars to make these environments well and then release it because the profit motive is so high. I was joking with the author like you could probably sell this data for a million dollars and then release the paper and I know we like we care about doing the right thing. But it's just like these open resources are the foundation of the So much study. And it's like you release some RL environments with some RL rollouts. People can do research on, say, like eval awareness. You have the generations from the model. People can look for different behaviors. If you've done RL on it, they can see how it goes over time. And this is just the units of infrastructure that PhD students spend a year looking at and produce increased knowledge. in like 10 years. And I just think like this is why more people supporting open that if they're going to work, if you're going to work in the industry for 10 years, like your peers are going to be the ones that benefit from it. More fossil fuel. Let's not deplete it at all. But it's just like the opportunity to go short term win is so high. The opportunity to make what is a hundred million dollars like nine figure returns for your business within five months of founding a startup with your friends it's just like these are people who may have done more open source work or these are people who left their phd or something and it's just like pulling back which is why we came up with this name trillium for our lab which is like trillium is a spring ephemeral flower which it blooms before the canopy has grown its leaves so it's for spreading nutrients through the cycle of growth that comes through spring and summer. And this is where I feel like the AI industry is doing so much right. It's like so much is going right. But it's just like we just need a little bit more life to spread this out and make things so new people can come in. There's so much attention and just do research, do education. Yeah. And then... as this scales and becomes more impactful is like you have these open things which become the standards of study for such important problems. So just like, how do you understand sandboxes? Like, how do you define what loss of control is? How can you see how this changes in a RL round that is a week on a thousand GPUs or a hundred thousand GPUs? And it's just like make these places so that people have more trust. And I think this pays so much dividends to everybody. [35:00] and supravit more than I do. And it's just like hard to get people to take that leap. And like, I've been rewarded so much for everything. kind of stumbling into this area. And now I get so much reward. I get to do things like this, which is fun. And it's like, now is the best job because I try to create space for more people to, like, grow their independent thought and build this infrastructure. Yeah. And maybe one, like, kind of naive way that it kind of perpetuates is just because it's open source, people get to, you might get more of a distributed contribution from people who are just so, you know, it's like they're motivated not for just trying to sell to the labs themselves, but because, hey, it's out there, they can share their work. And then you actually have a very high caliber amount of contributors to the project. I mean, that would probably be the naive hope of how that. Yeah. It's a new puzzle with AI agents where people just throw things at you. Like I have some random research repositories for research papers and I'm now. What do you mean? Like, just like, fake research results. They open an issue and they're like, I found this thing in your data set. I'm like, well, if you looked at the paper, we said that in the second sentence. It's just a lot of language model output. That's a new thing for the open community to go through. This is a solvable problem. There are people out there that want to contribute to this stuff because they Not just clout chasing in another dimension, but actually, hey, it's out there. I would just hope, because it is published and open or you know and available you can actually just get like PhD students or other researchers are like okay I do this on my day job but maybe I can you know still be able to contribute yeah and hopefully from the frontier labs like I see people like this and I was like that's great somebody from like eastern Europe was helping me improve the code for my book which is right here and he's just like doing it for the right thing and like there are people that do it and I think a lot of people just take, like they don't have the time or the stability to take the long time it takes because when you're trying to get into a new field you know nothing you have to spend like weeks reading and knowing nothing and then you send an email and nobody replies and you're just like it takes you weeks to know the context like ask a good question yeah and now with language models most people just push a button and they're like oh interesting I read this I'll send it off and it's just like this oh yeah kind of like sitting in it is the thing that people don't do as much and that is the key to like be coming into AI is that like, you just have to focus on it. You have to write some code by hand and you have to do this for months. And then you have to wait six to 18 months for you to get notice. It just takes a long time. I'm like, All the AI agents, I think, just like frazzle that and makes it harder for people. That's what the mathematicians complain about, too. That's all. So it's a hard time to be a PhD student, but people are doing great work in academia still, but it's hard. But I agree with you. I think it's like solvable. As the agents kind of perpetuate some version of the problem, it can also help solve and identify an inequality filter. Okay, so environments, you're going to release open source versions of that, help precipitate maybe more people to contribute and reveal that. especially given how little incentive there is to release this instead of just get fast revenue from the labs. Another area I think I read on your announcement post is RSI, which is quite... I think, I mean, we can first, talk about what is actually recursive about it or how scary it is or whether we're going to have takeoff. I think you have a very grounded opinion on that. But, you know, another thing I want to kind of talk about is I think RSI, maybe related to how much it's kind of hyped, is a reason why we are afraid the Frontier Labs will pull away. In fact, it's also a reason why I've heard not just once, but several times that people... are, One, convinced that their own jobs are going to be automated as a researcher. Yeah. I've gone through the last few years, some of my best friends, I'm like, you are some of the best people in the world of this. And then they go to the work and they're like, Nathan, I'm struggling. Like, what am I going to do? Because your point, like a lot of what they do is verifiable, right? And so actually because, I mean, one argument I heard was because there's such a... difference between how well RSI is doing for model capabilities versus alignment, which is much more out of distribution, maybe creative work, that gives people an ease on how, you know, maybe there is actual dangers, et cetera, et cetera. So anyways, because of this, I'm very interested in hearing from you, like how, because I think it's really important that we do have an open source kind of front in showing how RSI is being done, how much maybe then we can, you know, understand the capabilities of like, if in within a lab, they have a better model, what are the chances they just better understanding calibrating. So yeah, we're [40:00] to hear more about that. Yeah, so I think this, we won't be the only people doing this, but essentially like RSI has so much mystique. It's like we're using models for everything. And like what we can do as a open lab and a nonprofit and I think other companies like Eleuther or Hugging Pace or OpenAthena, which is Marin and Stanford. They will get into this with a good marketing line for them and me. That's why I'm doing this. And it's like, we can share what tools we're using. We're going to build an open source stack for SFT, for mid-training. We're going to use SkyRL, Miles, all these things that you know. And we have our task. We have our configs. We know what type of compute we're running. We can work with Cloud and GPT and all these models. And we'll just be like, we can store the trajectories for what we did. We can tell you what our tasks, whatever use linear, whatever these popular things are. We'll say what our tasks were and why we chose them and whether models work on them or not. and you're like, here's what we did. And then just check in every six months and you're like, has it changed? And it's just like, to be able to have the full stack and not need to go through these classifiers and try to... say what the task is vaguely and so on. It's just like people that can share what automation is happening should do this and we have a different distribution of tasks. But I think that the research is similar. I think the AI research that I've done at AI2 and we'll do at Trillium versus what people do in some sects of acneidamia and the labs, it's like, not that different. You have more resources, you scale it up. And it's just like, so good for the world when everybody's talking about this to just to show people like, this is what Nathan asked his model to do. And this is what it got stuck on. And I think that will help a lot of literal policy makers. Exactly. You're like, Oh, okay. It's like, oh, that's why it's very much a little. That's what it got stuck on. That's the bug that it put in its code. The bugs and the catch-ups will reduce over time. But I think the jaggedness and some of these fundamental things of the model is some more things will take off. I said efficiency earlier. I do think the models would be really good at lowering serving costs. These companies have billions of dollars on the line and the models are good at it. They're going to make a lot of money on being serving, order to go into that. But it's not necessarily like, how do you solve knowledge work? Like, Claude's not going to... That's right. Claude's not going to... You're not going to be like Claude Make... tell yourself to make better PowerPoints. Like what is true? The environments or some other, you know. Like it's going to take a while. Yeah, yeah. No, I love the specified goal of like, let's just demystify this because I think a lot of why the rhetoric gets so heated around our size is like, it sounds scary. AI nonprofits can do a lot of things like this. Like it's hard for academia to be like, as a professor, 15% of my time and 40% of the time of my grad students can be documenting what we work on. So there's like such a great like policy community service of like, here are our tasks. People can use this data to make evaluations or whatever. And it's just like, just like build things that have rare information so that when people put more information, high-focus work into it, it just has more value for all sorts of things that you can't predict in the future. It's honestly just also very timely because the kind of, you know, it's not from one person, but several people that I know from the labs are just like, hey, I'm going to get automated. To your point, to your friends, I will definitely, as a person, I will get automated in a year. And so that probably means there's going to be less people who actually are exposed to what's being done on that particular part of the org chart. And so, There's a related academic, there's an open RSI thing, which is a bunch of academic institutions. I'm not affiliated with it. They just launched like last week. So I was like, they sent me a few emails. I'm like, open RSI is a thing. They're trying to do this as well. It doesn't need to just be one group doing it. It's good to have different approaches doing this. You need the different ways of looking at it and different results. Yeah, whole point of being open. The other thing I also heard was... from more kind of like, you know, frontier lab goss. It's just like the actual methods that are being used in the lab's are not that, it's not that they're unknown. It's just sort of like what, are the proportions that need to be used that are successful. And this is partially because they have the capital to scale it, to experiment. So I think that's one heartening to know because like, Okay, it's not like there's a big secret of what's going on. So people actually know the techniques. But on the other hand, you know, capital is the one true barrier still, because it's hard to actually know which of which of the techniques recipes work. That's why I'm not pre-training. I'm happy to be free. I can use any base model. I'm now get to benefit from competition. But maybe one of the [45:00] values of that, and then maybe we can kind of discuss potential... commercial strategies of make-up. Pre-training science is pretty, like, the tools are pretty accessible and you have what is a pre-training ladder which is like the decision-making tool you have to test if your ideas scale and the pre-training data isn't growing enough. at least. like the environment scene. Like it's essentially, it's a first approximation, you train on the internet. And like, there's a lot of, there's a little, there's a little details and secret things, but like that's not changing that much. The state-of-the-art architecture is, and the GPUs are, and that's like, that changes. And, but it's not, astronomically expensive to make a pre-trained model that is solid so i think like you don't have to spend a billion dollars to get a very good base model the more you spend the better you will get but it's like People keep making progress and I'm constantly surprised by a new lab just goes from scratch because they can. And some of this is on open tools. So like, you can go off Megatron, like a lot of new startups, they're like, Oh, what should we train? We train off the open source tool and they fork it and then they never release it again. This is what all the even prominent open model labs have done. They take the open source code, they make it their own thing and they just a lot of them don't push upstream. So I've always been surprised by this. So I'm like not betting against new entrants because like, It's not that you need $10 billion to... have a relevant model. The hard thing is get a model that's good enough for people to use and to get a story where if you release it, you can do your next fundraise. It's just like, You have to get past this lock of like, we need our first model to land. If like your whole company is based on models are the best and you're not in the ballpark and you're not releasing, you're on the clock. I think of thinking machines getting inkling out where they were very transparent. They're like, this wasn't the best model. We're proud of it. It has some cool things. Now that they've released their second model and they're getting going and they have Tinker that has this. That's very good for them. They're off the ground. It's just like... I don't think that's what they were doing. They were building models, but open models wasn't their first plan. And then they got there. And there's rumors that Meta said they'll release a new open weight model again. That'll come. There's a lot of companies out there that couldn't afford models. Yeah, yeah, I'm definitely glad that Thinking Machines is, and it makes so much sense for their business model because they do post training. And, you know, with Meta, I think you mentioned this as well in some previous writings. It's just for them, it's just such a good strategy. I think it was also just kind of obvious when they do it. It's like, hey, it's complimentary to my... Mark wrote the best blog post on this. Like when Llama 3 came out, he wrote like the best, like why open his path forward. The guy got a little distracted. It seems like he's coming back. Right. So I hope they keep on being motivated to do that. Getting to the frontier with pre-training, you know, I think there's a lot of help given that there's such good open source models to maybe, you know, somehow still kind of bootstrap from. I keep on getting kind of conflicting signals as to like whether pre-training is actually as saturated or well understood. Maybe to your point, there's a lot of well-known techniques at work, but what's hard is just actually having access to the 20,000, 30,000 chips. People are turning the crank on pre-training and pre-training became... adjusted when super long context generation happens. So there's just way more return if you can, for example, get a sparse attention architecture to work as well. And then your 100,000 token generation is 40% faster. So I think there's a lot of co-evolution of the whole stack where they have their pre-training stack, they realize that inference scaling is so much higher. But even the fundamentals of pre-training keep improving, where you just have a marginally where this mid-training where you start the reasoning behaviors, you take them from your older models, you do RL on them to seed behaviors. This cycle is much more complicated than pre-training. If you don't have the right people, it's a lot harder to... get that off the ground than I think pre-training was. I think to do a controlled, let's just start pre-training our models seems a lot more approachable to me than I don't really know much about language models. I'm going to figure out this whole post-training thing where it's just like the Wild West. Yeah. And that's why I was like... When I left, AI2, I felt like our post-training was honestly falling behind just because our cycle time was not high enough. And we're going into these agentic things. And these are still some of the like leading open source post-training recipes. And... that's just like that's what I want to fix is like It doesn't take as much capital to make. very big compounding gains on the state of the art and the open source because the bar is very low. It's not close. Yeah, yeah. No, that makes [50:00] It's just like a kind of efficiency where you can put capital and make a huge difference. Yeah, yeah. kind of related to the whole post-training side, maybe pulling away from closed and open, is the fact that the agents, the type of harnesses that are being used in the closed labs are very different and slash, you know, actually trained and, you know, post-trained on to, you know, to be specifically good for, their models are specifically good for those harnesses. And so that increases the mystique even more, makes it very hard. How do you I guess, one, do you agree? And then two, how are you hoping specifically to address that? And things I mean are really examples like, hey, spin up thousands of agents, tens of thousands of agents. It's just a very different workflow than what people are obviously exposed to. I think it'll be okay for open because I thought years ago that say the closed labs would be like Apple. They'd be the integrated company where they have the model designed for the harness, which is involved. It gives them kind of a kind of scale, efficiency. But it turned out that so many people have different harnesses for different things that they like to do, that the model companies have been pressurized to make it so that their model can work in all sorts of different harnesses. And I thought it would be a weakness of open models that they have to be designed to train on all sorts of types of harnesses. But it ended up being pretty equilibrated. It's at equilibrium where they all have to do it And then what we're seeing with this agent swarms is that OpenAI and Anthrop are going to be earlier to it. They're going to know how to design their models. They're going to know how to design, what do we want to call a harness for a swarm? I don't know. I don't know, but there's going to be a word for it. It's not a harness. It's like a coordination system. But then eventually other people are going to want to use them. And then I think it's going to be like these cycles where the frontier labs, like they push this out and they have this... to date, they've had huge returns on being first. But what is the returns on Navier, Stokes, and all these things? We don't really know. We don't know. They push the frontier... in the assumption that there will be high returns and that this marginal intent marginal intelligence has like disproportionate returns and we don't know if that's always true but i kind of expect these cycles of like once you know the shape of how do you control 10 000 agents in one harness then the open model companies like oh we can do this too but it's not like Kimmy is going to do that first. For one, they don't have the compute. It's an astronomical amount of compute. Bye. these things go through cycles and videos chips will get way faster. There's more compute built out as this, So you're very convinced that there's not too much hidden there, that you can probably guess at what they're doing, and then when it becomes relevant, let the open source models actually try to also post train on similar things. meta-harnesses. Yeah, and I also think, like, the the security at these labs in terms of information is not as high as they think. Like Silicon Valley is famous for like ideas traveling. Like what is it? Like one beer or something. And it's like everybody chats after work. Which is why SSI is so impressive. Yeah, that's what's going on. That's the only pass. But it's like, Everyone's scientists. If they made a breakthrough, they're going to want to tell their friends about it. They might hold it for a few months. I don't think they're... The amount of information security you would need is just so high that knowledge spreads. It's hard to keep the secret. What they can do... went on is putting more inference into their infrastructure, which lets them serve at a lower price and make higher margins and then continue this over a longer time. So I'm very confident in the Frontier Labs being able to serve effectively. the cheapest intelligence per person. Thank you. per watt per score on an eval, they are going to be able to go to the places with the highest demand and have the like, the cheapest models. And that'll be revenue creative. Like, that'll be a good business for them. The question is, like, they can't do everything. And that's where the rest of the economy fills in is, like, is it, GIST coding. that is profitable. Then... Definitely not. Good case for Frontier models, because they will have the most... Right. They have integration. They'll do very well, but it's not coding. Is it the whole economy of knowledge work is the market for things, and then it might not be good for them, because can they do everything at once? Or they don't need to. I think there's going to be a lot of winners. Yeah. I mean, that's why we invest in the Frontier models and in startups. Not because we want to hedge, but because I actually do think, to kind of cooperate your view, that there's enough harder tasks at the fringe that requires very specialized data that's kind of out of the splash. So for getting to AGI or whatever you want to call it these days. that makes it possible to have many, many winners. I do like your RSI point, as we talked a little bit about it, which is just, kind of say practically, hey, it's exponential cost and resource consumption. And therefore, there's a very obvious kind of limit to how much [55:00] that the frontier labs can pull away, if that's not too oversimplification of your view. Anything else you kind of want to add there? Because I do feel like the discourse now is just so much fear on how much the frontier is just hidden and also breaking away. I think there's so much spend on inference and that in the short term, we are learning how to... um, corral inference into performance gains. And this is like a thing that we just had. And like, this is what we're seeing now is that like people are spending so much and like all the employees at the Frontier Labs are spending exponential amounts relative to the improvements, but we're figuring out how to do this and there will be a lot of gains. But I don't think that it's necessarily going to be this fast takeoff. when the models build themselves, which is a human bottlenecks. All the scaling laws are exponential increase in compute for linear increase in performance. And... Excellent. Like, that's just like, it's hard for me to bet against this. I do say that, like, We have Navier Stokes, and I'm of the opinion that math is... exceptionally well-stuted. Maybe the scientific field that is best suited to this shape of intelligence. And intelligence can take many different forms. And That's not to say that I linked If an AI model does, say, discover something that's very different than the transformer, that gives very different properties and unlocks a new type of... intelligence. It is worth updating on this and worth being like, okay, that is different. But until that happens, I'm pretty happy with my evidence. The companies are going to do well. They're exponentially more to keep this remarkable pace of progress. Some of the clawed system cards say this. They're like, it would take say, like 10x more progress to make this a fast takeoff. And it's like, for now, they're helping us sustain the pace of progress. And I think, It's a good equilibrium, but I do think that the labs have people saying this. The science is like, it's hard to overcome that. And it's okay, but it's like we don't know exactly what will happen. go from here. Yeah, yeah. No, I think that's comforting because we know what the inputs and levers are here. There is a very successful autoregressive regime that we've been optimizing, and that one just asks for a certain amount of exponential resources, which are clearly capped. And therefore, open source hopefully keeps it strong as well. It just wouldn't lag behind. Thank you. pretty much non-existing preparation, across public cybersecurity and all of these things. Like, there needs to be more preparation in the assumption that this is coming than there is. And then that's kind of the thing that worries me more than say like, RSI is coming and unlocking new risks. There are... risks that are here and that are going to continue to diffuse. Many times I've said, the closed labs get there and then the open labs get there. It's like, we're in this process for a lot of things that are going to cause some problems. It's just like, figuring out how to get society to prepare is very hard. Yeah. And I come of the opinion that we need more eyes, we need more science, so that society feels informed. Yeah. But there are other valid opinions. And like, I do think that not fearing fast takeoff does not mean that you don't worry about risks. I don't know if you should. No, I think that's completely, yeah, maybe I'll send in some people your way that are defected from the labs or worried precisely about this. I'm really excited to see what Trillium is up to now that it's finally launched. Yeah. Thanks so much for having me. Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our sub stack at A16Z.substack.com. and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal business tax or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, Thank you. Thank you. Thank you.