Mercor Scales AI Data Pipeline to $2 Billion Run Rate on Expert RL Environments

Mercor依托专家RL环境将AI数据流水线年化运行率推至20亿美元 由人类领域专家构建的强化学习环境正在取代众包标注，成为前沿AI模型的主要竞争壁垒。 Mercor首席执行官Brendan Foody详述了AI训练数据从众包标注向专家构建的强化学习（RL）环境的演进，该环境由世界（worlds）、应用（apps）和任务验证器（task verifiers）构成。他强调，在自动化反馈不足的领域，来自律师、工程师和医生的深厚专业知识对于衡量前沿模型性能至关重要。通过展示与顶尖律师共同构建的真实法律RL环境，Foody呈现了针对性后训练如何在公司法等特定领域基准上带来显著的能力提升。最后，他概述了数据商业化模式，并预见未来将以超长周期任务和虚拟同事为核心。 Mercor 在 4 个月内年化收入从 10 亿美元激增至 20 亿美元，印证了 AI 训练数据正从低门槛众包标注全面转向由顶级专业人才构建的强化学习（RL）环境。 (0:00) 真实世界的 RL 环境由“世界”（项目背景、邮件与文档）、“应用”（高保真软件克隆）和“任务”（提示词与基于规则的验证器）三大要素构成，其中构建能兼顾创新解法与准确度的验证器是最大技术难点。 (2:29) 人类领域专家在衡量前沿模型能力上依然不可或缺，因为模型在法律分析等缺乏确定性反馈的主观任务中无法可靠评估自身表现。 (3:57) 在包含 1,800 个任务的 Apex Agents 环境中投入约 50 万美元算力后训练 GLM-47，模型在公司法测试中的准确率从 4.7% 跃升至 26.6%，展现出垂直领域环境驱动的巨大效能跃迁。 (9:45)

Mercor CEO Brendan Foody argues that frontier AI training has transitioned from low-skilled crowdsourcing to domain-expert Reinforcement Learning (RL) environments that simulate full workplace software ecosystems. In Foody's view, human domain experts remain essential because foundation models cannot reliably measure their own frontier performance or independently author their own evaluation verifiers. Architectural Components of Modern RL Environments (2:29) Foody traces the evolution of AI training from 2020's low-skilled crowdsourcing—focused on supervised fine-tuning and basic preference selection—to the modern agentic era driven by Reinforcement Learning with Verifiable Rewards (RLVR). Enterprise-grade RL environments simulate real-world work through three core layers: "worlds" containing authentic project artifacts such as emails, documents, and spreadsheets; "apps" that clone standard software like Salesforce, Microsoft 365, and Google Workspace via interfaces like Model Context Protocols and Computer-Use Agents; and "tasks" pairing realistic prompts with rubrics or unit tests. Human Verification as the Primary Bottleneck (3:57) Constructing accurate verifiers represents the most difficult technical challenge in environment design. Foody states that models cannot grade their own frontier work without introducing excessive noise, comparing unassisted model grading to a student grading their own homework. While models can act as copilots to generate background files or test trajectories, verifiers must be authored and calibrated by elite human practitioners to score diverse valid strategies without penalizing unexpected solutions. Post-Training Capability Gains and Diagnostic Metrics (9:45) To demonstrate the real-world impact of RL environments, Foody cites Mercor's Apex Agents dataset of 1,800 tasks. Post-training GLM-47 using approximately $500,000 in compute increased its corporate law benchmark accuracy from 4.7% to 26.6%, while also improving performance on external benchmarks like GDPval. To diagnose whether a base model can learn from a domain dataset, Mercor evaluates rollout distributions between Pass@1 and Pass@16: a model that scores zero across 16 rollouts lacks the baseline capabilities to generate a learning gradient, whereas a model that fails Pass@1 but succeeds once or twice in 16 attempts provides the optimal gradient for reinforcement learning. Commercial Models and Emerging Horizons (11:17) Enterprises acquire RL training data through three primary avenues: task-based custom environments priced between $50 and $10,000 per task (with frontier labs purchasing up to 50,000 tasks monthly), licensed off-the-shelf datasets, and hourly expert provisioning. Looking forward, Foody identifies two key horizon shifts: expanding task duration from under 10 hours to ultra-long workflows spanning 100 to 1,000 hours, and embedding simulated virtual co-workers into environments to address the fact that 60% to 70% of human work involves interpersonal interaction, compared to roughly 1% of current AI evaluations. Foody anticipates that specialized RL environments will enable vertical software companies to reach frontier intelligence by post-training open-weight models, bypassing the need to build foundational models from scratch. However, the model's ongoing expansion depends on whether providers can economically scale high-skill human labor to build verifiers across all 205 Bureau of Labor Statistics GDP categories.

Mercor 首席执行官 Brendan Foody 认为，人工智能数据训练已进入由跨领域专家构建强化学习（RL）环境的“智能体时代”，高保真应用环境与基于精准评分规则的验证器取代了低技能众包标注，成为决定模型后训练上限与企业建立竞争壁垒的核心要素。 数据范式跃迁：众包标注到智能体环境 (0:47) Foody 回顾指出，AI 数据市场在 2020 年主要依赖低技能众包进行行为克隆，工作重心是监督微调（SFT）和人类反馈强化学习（RLHF），即标注员仅在多个选项中选择偏好回应，这奠定了早期 GPT-3 和 GPT-4 的突破。然而迈入 2024 年后，行业迅速转向“智能体数据时代”，前沿实验室不再需要普通的众包评判员，转而寻求软件工程师、律师、医生和投资银行家等高技能专业团队，协作开发前沿评估基准与强化学习环境。 真实世界的强化学习环境由三大核心架构构成：“世界”（企业的上下文资产，包括即时通讯、演示文稿、文档、表格和电子邮件）、“应用”（通过 MCP、CLI 或计算机使用智能体 CUA 进行交互的高保真软件克隆，如 Salesforce、Google Workspace）以及“任务”（由提示词与评分规则等验证器组合而成）。数据环境也从最初以网页搜索为主要工具的深度研究模式，演进为覆盖本地代码库与完整桌面端应用的高保真模拟，以真实拟合实际办公分布。 人类专业能力不可替代性与验证器构建 (3:57) Foody 强调，在缺乏确定性程序反馈的领域（如幻灯片设计或法律定性），模型无法衡量自身在前沿水准上的表现；让模型评判自身生成的幻灯片无异于“让学生自己批改作业”，会导致评估信号充斥噪声。为了覆盖美国劳工统计局（BLS）GDP 分类中的全部 205 个专业领域，仅在第二季度，Mercor 的专业人才网络就投入了 250 万个专家工时。 构建高精度验证器是当前技术难度最高的一环。以 Mercor 联合 Latham & Watkins 等顶级律所律师构建的开源法律环境为例，资深律师需撰写真实并购交易场景并设计完整数据室架构，专家借助模型在环（model-in-the-loop）技术高密度填充文档，再将其渲染至应用克隆中。面对复杂的综合诉求（如评估《油污法》下的责任限额），系统通过专业评分细则防止模型“奖励作弊”（reward hacking），并结合轨迹分析校准自动化打分器。而在网络安全等具备客观可证明性的技术领域，专家主要负责前期搭建多维网络环境，后续验证则可通过红蓝对抗智能体实现自动化评分。 诊断指标与模型后训练的效能跃升 (9:45) 在评估基座模型是否具备强化学习可训练性时，Foody 指出模型参数量虽有影响，但核心诊断指标在于 Pass@1 与 Pass@16 之间的表现差距。如果模型在 16 次轨迹采样中全部失败（Pass@16 = 0），意味着基座模型尚不具备从该环境中吸收经验的底座能力；最理想的训练窗口出现在模型 Pass@1 失败、但在 16 次尝试中能够成功 1 至 2 次（Pass@16）的区间，这能为强化学习算法提供优化所需的精确梯度信号。 实证数据显示了强化学习环境训练的显著效果：在包含 1,800 个任务的 Apex Agents 数据集上，对 GLM-47 投入约 50 万美元算力进行后训练后，模型在公司法基准测试上的准确率从 4.7% 跃升至 26.6%，并在缺乏数据室上下文的外部基准（如 GDPval 和 Apex V1）上展现出跨领域泛化收益。Foody 观察到，随着 GLM-52 和 Kimi K3 等开放模型表现提升，垂直应用层公司无需从零构建基座大模型，便能在专有业务领域内达成顶尖智能水平。 商业交付模式与超长周期智能体的演进 (11:17) 在商业合作维度，Foody 总结了三种主流采购方式：按任务定制环境（单价在 50 美元至 10,000 美元之间不等，例如复杂法律任务约为 2,000 美元，前沿实验室每月采购规模可达 50,000 个任务）、现成多租户数据集授权，以及初期探索时按小时采购专家工时。定价模式主要基于下游商业价值倒推（如前沿开源模型所代表的市场溢价）或人力成本加成。针对行业关于“合成数据”的讨论，Foody 澄清带有可验证奖励的强化学习（RLVR）本质上正是对合成数据的系统化应用：通过模型自主生成海量轨迹并由验证器打分，模型在人类专家设立的环境框架内实现了从自我生成中迭代进化。 关于未来演变，Foody 预测智能体环境将经历两大范式转变：一是任务时长从现存的 10 小时以内扩展至 100 到 1,000 小时的超长工作流；二是引入“虚拟同事”互动机制。在现实工作中，人际沟通协同占据了 60% 至 70% 的精力，而现有主流基准测试（如 Tau-bench）对社交动态的考察仅占约 1%，填补这一真实性缺口将是智能体训练环境的下一个核心课题。 Foody 预测，未来 12 个月内将有数十家垂直领域应用公司通过掌控专有数据环境复制 Cursor 的成功。然而，将环境任务扩展至数百小时并引入复杂的协同交互，对环境仿真的保真度以及评分细则在多路径解答下的鲁棒性提出了更高要求；如何在非确定性领域持续保持低噪声、抗作弊的自动化验证，仍是规模化落地过程中有待持续验证的技术挑战。

## Background 本视频由红杉资本（Sequoia Capital）于 2026 年 8 月 18 日发布在 YouTube 平台上。内容记录了在红杉资本举办的“Own Your Intelligence”活动中，Mercor 首席执行官 Brendan Foody 围绕当前 AI 训练前沿——强化学习环境（RL Environments）所作的主题演讲及深度问答，详细拆解了其三大构成要素、顶尖律所真实落地案例以及企业构建自主智能的核心策略。 ## TL;DR 演讲系统梳理了 AI 数据市场从低门槛众包标注向专家共建“强化学习环境”的重大演进，阐述了由真实业务世界（Worlds）、高拟真应用克隆（Apps）与带验证机制的任务（Tasks）组成的三大技术支柱。Brendan 强调，在缺乏确定性沙盒的现实工作场景中，唯有人类专家制定的评分细则才能防止模型奖励作弊并准确度量前沿极限，而案例表明仅用 1,800 个法律任务和 50 万美元算力就能促成领域能力的显著跃迁。此外，分享还深度剖析了按任务定制等三种数据商业模式、Pass@1 与 Pass@16 的可训练性启发式判据，并预告了超长时序任务与虚拟同事协作等下一代技术跃迁。 ## Atmosphere & style 整场演讲与互动节奏紧凑、干货极度密集，充盈着前沿工程实战与商业洞察并重的先锋科技感。主讲人语言干练且逻辑缜密，既有大所律师虚拟数据室的真实生动还原，又有对奖励破解、合成数据本质的深刻技术剖析。后半程问答环节观众与讲者高频切磋，直击定价逻辑、攻防博弈及自建还是采买的商业抉择，传递出一种正在合力拆解大模型后训练黑盒、探寻下一代自主智能体底座的务实与兴奋感。 ## Bulletin - [00:00](https://www.youtube.com/watch?v=a00xIn5kwhM&t=0s) Mercor 业务呈现爆炸式增长，年化运行收入在约 4 个月内从 10 亿美元翻倍至 20 亿美元，单季度专家贡献时间已达 250 万小时。 - [00:47](https://www.youtube.com/watch?v=a00xIn5kwhM&t=47s) 数据市场完成从低门槛众包标注（SFT/RLHF）向顶尖行业专家构建 RL 环境与前沿评测的结构性跨越，成为训练自主智能体（Agent）的核心底座。 - [09:45](https://www.youtube.com/watch?v=a00xIn5kwhM&t=585s) 在开源模型底座上，仅用 1,800 个法律任务和约 50 万美元算力进行后训练，便让模型在公司法专项任务上的得分率从 4.7% 飙升至 26.6%，并展现出跨域泛化能力。 - [18:05](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1085s) 未来强化学习环境正迎来两大范式转变：从 10 小时以内扩展至 100 到 1,000 小时的超长时序任务，以及引入能够模拟真实团队协作的虚拟同事（Virtual Coworkers）。 ## Key takeaways - [02:29](https://www.youtube.com/watch?v=a00xIn5kwhM&t=149s) 完备的强化学习环境由三大支柱构成：包含项目真实业务上下文的“世界”（Worlds）、高拟真克隆 SaaS 工具的“应用程序”（Apps），以及成对绑定的指令与细则验证器“任务”（Tasks）。 - [03:57](https://www.youtube.com/watch?v=a00xIn5kwhM&t=237s) 真实世界绝大多数工作没有确定性代码沙盒，让模型自我评估等同于“让学生给自己批改作业”，只有超越模型认知极限的顶尖人类专家才能可靠衡量能力前沿。 - [05:35](https://www.youtube.com/watch?v=a00xIn5kwhM&t=335s) 构建验证器（Verifiers）的核心难点在于既要兼容高维解空间中十数种不同解决路径，又要防止模型在海量探索轨迹中进行“奖励作弊（Reward Hacking）”。 - [06:44](https://www.youtube.com/watch?v=a00xIn5kwhM&t=404s) 顶尖律所律师深度参与构建的真实法律环境，将全套虚拟数据室挂载至 Google Workspace 克隆系统，通过严苛 Rubric 迫使模型在复杂长文本中进行精确法律推演。 - [08:18](https://www.youtube.com/watch?v=a00xIn5kwhM&t=498s) 高性能开源权重模型的爆发改变了行业竞争格局，应用层企业无须从头做昂贵的预训练，依托专有数据后训练即可在垂直细分赛道建立差异化模型护城河。 - [10:15](https://www.youtube.com/watch?v=a00xIn5kwhM&t=615s) 高质量强化学习数据市场已形成三种成熟商业模式：按任务定制（By-Task）、开箱即用的多客户复用现成数据（Off-the-Shelf），以及传统的专家工时租赁（Hourly Experts）。 - [12:36](https://www.youtube.com/watch?v=a00xIn5kwhM&t=756s) 数据定价主要采用由客户模型提升目标倒推（Outcome-based）和专家工时成本加成（Cost-plus）两种维度，单个前沿任务价格在 50 美元至 10,000 美元之间不等。 - [14:05](https://www.youtube.com/watch?v=a00xIn5kwhM&t=845s) RL 环境中的数据质量被严格拆解为环境真实性（Realism）与验证器准确性（Accuracy），需借助轨迹分析与专家在环质检确保机器评分与专家人工判断高度一致。 - [16:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=990s) RLVR 本质是对合成数据的一场巨大押注，但纯模型自动生成评分准则会引入约 50% 的致命训练噪声，必须依赖“AI Copilot + 人类专家”的协同工作流把控边界。 - [20:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1230s) 现实工作中高达 60% 至 70% 的时间在与人沟通协作，但现有基准评测仅有约 1% 的内容涉及协同互动，构建虚拟同事填补社交鸿沟是智能体落地的关键方向。 - [22:47](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1367s) 在网络安全等天然具备对抗博弈属性的“可证明领域”，可通过红蓝智能体攻防对抗实现闭环自动化验证，人类专家的角色转向搭建逼真的靶场环境架构师。 - [23:46](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1426s) 判断基础模型是否能从数据集有效学习的关键指标是 Pass@1 与 Pass@16 的差距：展开 16 次轨迹采样全错则无法形成有效梯度，而只要偶然命中 1 到 2 次即可快速通过强化学习掌握技能。 - [25:19](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1519s) 自建全球专家网络与企业软件克隆生态具有极高门槛与规模经济效应，企业通过专属隔离团队采购定制数据，比从零造轮子能更高效地筑造专有护城河。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=a00xIn5kwhM&t=0s) 开场：Mercor 的爆发式增长与强化学习环境的兴起 在红杉资本（Sequoia Capital）举办的“Own Your Intelligence”活动开场时，主持人透露了 Mercor 令人咋舌的增长速度：在短短 4 个月左右的时间里，Mercor 的年化运行收入（Revenue Run Rate）从 10 亿美元翻倍飙升到了 20 亿美元。这一火箭般的崛起，使 Mercor 成为当前各家企业思考如何对自有模型进行后训练（Post-training）、构建自有核心智能时的绝对焦点。本场分享包含约 15 分钟的 Brendan Foody 主题演讲，聚焦强化学习环境（RL Environments）这一崭新的前沿课题，随后预留 15 分钟用于观众问答互动。 ### [00:47](https://www.youtube.com/watch?v=a00xIn5kwhM&t=47s) 数据市场演变史：从低门槛众包到前沿智能体数据 Brendan Foody 首先梳理了 AI 数据市场的演进历程，并阐释了该脉络如何催生出 Mercor 的创立。数据生态大致经历了两个截然不同的时代： 1. **2020 年代的众包行为克隆时代（Crowdsourcing Era）**： 这个时期主要依赖监督微调（SFT）的输入输出对，以及人类反馈强化学习（RLHF）。其标注形式相对基础——标注员在几个模型生成的回答中选出更偏好的一个。整个行业正是通过这种众包标注的 SFT 和偏好数据，推动了 GPT-3 的微调，并最终催化出 ChatGPT 和 GPT-4 的突破。 2. **2024 年以来的智能体数据时代（Agentic Era）**： 市场发生了一场结构性巨变，彻底脱离了低技能要求的行为克隆众包模式，迈向需要顶尖领域专家的协作式开发。模型若要走向下一代自主智能体（Agent），就必须由软件工程师、律师、医生、投资银行家等行业专家组成团队，协同构建前沿基准评测（Frontier Evals）与 RL 环境。 Mercor 的首个大型项目始于深层研究类智能体（Deep Research），随着各大前沿实验室纷纷扩大强化学习智能体（RL Agent）的训练规模，Mercor 迅速成长为领先实验室的主要智能体数据提供商，其客户群体也进一步横跨至各类顶级应用层公司，涵盖 Harvey、Cognition 以及 Ramp 等。在过去 12 个月中，基于可验证奖励的强化学习（RLVR，Reinforcement Learning with Verifiable Rewards）范式进一步扩展，进化出了包含高拟真软件生态与庞大项目背景的完整 RL 环境，训练智能体熟练掌握人类在日常笔记本电脑上操作的所有专业生产力工具。 ### [02:29](https://www.youtube.com/watch?v=a00xIn5kwhM&t=149s) 什么是强化学习环境：Worlds、Apps 与 Tasks ```mermaid flowchart TD accTitle: RL Environment Components accDescr: The three core building blocks of a real-world Reinforcement Learning environment: Worlds, Apps, and Tasks. subgraph ENV["RL 环境三要素 (RL Environment)"] direction TB subgraph W["1. 世界 (Worlds)"] W1["真实业务上下文与项目背景"] W2["Slack消息 / 邮件 / 幻灯片 / 文档 / 表格"] end subgraph A["2. 应用程序 (Apps)"] A1["高拟真克隆环境 (Google Workspace, Salesforce, ServiceNow)"] A2["交互接口 (MCP, CLI, KUA)"] end subgraph T["3. 任务与验证 (Tasks)"] T1["真实复杂场景 Prompts"] T2["细颗粒度评分细则 (Rubrics) / 单元测试 (Unit Tests)"] end W --> A A --> T end ``` 在 Brendan 的定义中，一个完备的 RL 环境由三大核心支柱交织而成： - **世界（Worlds）**：对应真实项目或企业运营中的一切信息背景，包括往来邮件、即时通讯消息、幻灯片演示文稿（Slides）、技术文档以及数据表格（Sheets）等综合材料。 - **应用程序（Apps）**：对 Salesforce、ServiceNow、Microsoft 365 等主流企业级 SaaS 平台进行高保真克隆，智能体可通过模型上下文协议（MCP）、命令行接口（CLI）或图形交互界面与其无缝交互。 - **任务（Tasks）**：将具体的指令（Prompts）与严谨的验证器（Verifiers）成对绑定。验证器可以是以规则为基础的细则（Rubrics），也可以是确定性的单元测试（Unit Tests），既用于基准评测，也用于强化学习的策略奖励训练。 前沿实验室若要让类似 Claude 这样的模型完全自动化人类在笔记本电脑上完成的所有脑力劳动，最大的壁垒就在于如何覆盖全美乃至全球经济中全部 Worlds、Apps 以及 Tasks 的完整分布。为了支撑这一目标，数据生产规模呈现爆炸式增长。Mercor 专家人才网络的人力产出图谱显示出一条陡峭的上升曲线：仅在最近的一个季度（第二季度），专家贡献时间就高达 250 万小时，且总体增速仍在持续加速。 ### [03:57](https://www.youtube.com/watch?v=a00xIn5kwhM&t=237s) 为什么衡量前沿能力必须依赖人类专家？ 要将环境分布拓展到现实经济的每一个角落是一项巨大的工程。美国劳工统计局（BLS）归纳的细分职业跨越了 205 个专业领域（如 GDP-Bench/GDP-Eval 所涵盖的范围），而每个职业背后都嵌套着海量的行业专用软件、业务分支和具体任务。 在绝大多数真实领域，唯有人类才能准确衡量模型是否抵达了能力前沿。纯粹依靠模型自我评估存在根本性缺陷： - **可验证沙盒的罕见性**：除了数学计算或纯代码这类极少数能够通过确定性编译、计算器获得清晰正负反馈的“纯净模拟环境”外，绝大多数真实世界的工作（例如撰写一份用于汇报的商业幻灯片）都没有单一标准答案。 - **“让学生给自己打分”的困境**：当模型生成了一份幻灯片时，它很难可靠地识别出自身在逻辑推理、视觉排版或商业常识上的隐性错误。Brendan 形象地打比方道，直接让模型判断好坏“就如同让学生给自己批改作业一样不可靠”。 - **导师评估机制的复刻**：人类的认知成长大部分来自教授批改论文、助教评估汇报所给予的外在反馈，而非自说自话。因此，必须由顶尖行业专家制定结构化的评分细则（Rubrics），以客观标尺捕捉模型推理路径中的细微疏漏。 ### [05:35](https://www.youtube.com/watch?v=a00xIn5kwhM&t=335s) 构建验证器：捕捉高维解空间与防范奖励破解 构建高质量的验证器（Verifiers）极其困难。以制作一份商业幻灯片为例，解题的合理路径往往有十多种完全不同的结构；与此同时，可能踩中的逻辑陷阱、格式缺陷和事实错误更有几十处之多。验证器必须既能包容所有有效的解决方案，又能精准拦截各类边缘错误。 如果验证细则设计不够完备，模型在强化学习过程中极易学会“奖励作弊（Reward Hacking）”——即找到评分逻辑上的漏洞，给出表面高分实则低劣的无用输出。为了确保一个任务在生成 100 条不同的探索轨迹（Rollout Trajectories）时，每一条轨迹都能被精准打分，团队必须在智能体质量控制、专家在环（Human-in-the-loop）以及针对数据的后训练流程中投入极为庞大的研究精力。 ### [06:44](https://www.youtube.com/watch?v=a00xIn5kwhM&t=404s) 案例拆解：顶尖律所专家参与构建的法律 RL 环境 为直观展示这套机制，Brendan 拆解了一个由 Mercor 联合前沿实验室开发并已公开发布的法律 RL 环境： ```mermaid flowchart LR accTitle: Legal RL Environment Architecture accDescr: Step-by-step pipeline of a real legal reinforcement learning environment from scenario design to leaderboard evaluation. subgraph BUILD["环境构建阶段"] direction TB E["大所律师设计真实案情"] DR["构建完整拟真数据虚拟数据室 (Data Room)"] CL["渲染至 Google Workspace 克隆系统"] E --> DR --> CL end subgraph ROLLOUT["执行与评测阶段"] direction TB P["法律 Prompts 注入 (如 OPA 责任上限判定)"] TR["模型轨迹探索 (Trajectories)"] R["抗奖励破解 Rubric 验证器判定"] P --> TR --> R end BUILD --> ROLLOUT ``` - **真实大所背景输入**：聘请来自盛德（Sidley Austin）或瑞生（Latham & Watkins）等顶级律所的执业律师，复刻其在真实“Big Law”重大案件中的复杂背景。 - **全要素数据室（Data Room）**：专家协同 AI 智能体编织一整套逼真的企业诉讼数据室，内含纷繁复杂的邮件沟通记录、往来信件、附件、扫描件及其对应的真实文件规格体系。 - **拟真软件交互与长文本推理**：该数据室被原汁原味地挂载到高保真克隆的 Google Workspace 中。任务要求模型在海量上下文中进行严谨法律推演，例如：“依据《石油污染法》（Oil and Petroleum Act），在充分审酌数据室中所有既有上下文的前提下，评估星际油轮国际有限公司（Star Tankers International Limited）相较于库珀·杰弗里斯能源公司（Cooper Jefferies Energy Corporation）所面临的最大综合责任上限”。 - **抗作弊的 Rubric 验证矩阵**：由教授级专业法律专家制定详尽的判分维度，通过多维指标交叉约束，彻底阻断模型利用提示词捷径博取高分的可能，最终沉淀出高置信度的领域排行榜。 ### [08:18](https://www.youtube.com/watch?v=a00xIn5kwhM&t=498s) 开源权重冲击下的模型演进与企业智能护城河 这一套曾由前沿闭源实验室垄断的强化学习训练方法论，正在加速下沉并普惠至垂直应用层公司。随着 GLM-52、Chimera K3 等高性能开源模型在法律、金融排行榜上占据前列，垂直软件公司获得了在专属领域超越通用前沿模型的底座基础。 Brendan 指出，应用层企业构建自主智能的战略支柱正聚焦于三要素： 1. **算力（Compute）**； 2. **算法与研究员（Algorithms / Researchers）**； 3. **专有数据集（Data Sets）**。 在这三者中，数据正成为最关键的护城河与核心差异化壁垒。以 Mercor 的真实实验为例：团队在公开的 **Apex Agents** 数据集上仅选取了 **1,800 个法律任务**，利用约 **50 万美元（500k）的算力**对模型进行微调后训练，结果迎来了惊人的性能飞跃： - 模型在公司法专项任务上的得分率从 **4.7% 暴涨至 26.6%**； - 这种后训练展现出极强的泛化性，不仅显著提振了包含数据室的测试集，而且在完全不含数据室环境的泛化基准（如 GDP-Eval 和 Apex V1）上同样实现了优异的跨域迁移与指标提升。 这印证了垂直领域企业不必耗费数亿美元从头预训练，而是能像 Cursor 依靠精炼数据构建行业领先的代码模型一样，借由如 Harvey 等伙伴深度共建专有 RL 环境，在法律、金融等细分纵深赛道筑造专有的模型前沿壁垒。 ### [10:15](https://www.youtube.com/watch?v=a00xIn5kwhM&t=615s) 企业购买与构建高质量数据的三种商业模式 在过去的实践中，为客户打造专属智能已展现出惊人的价值。未来一年内，类似由企业自主掌控专属模型智能、并以此构建核心商业壁垒的案例将会成倍涌现。关于高质量强化学习（RL）数据集的获取与构建，行业内主要呈现出三种成熟的采买方式： 第一种也是最主流的方式是**按任务定制（By-Task）**。客户往往会锁定特定领域的环境形态（例如法律环境），并为大规模扩充任务支付单价，比如“单个任务支付 2,000 美元”。部分头部的前沿实验室（frontier labs）单月就会向 Mercor 采购高达 50,000 个任务，整体采购规模相当庞大。这类任务的复杂度差异极大，有的仅需数小时即可完成，有的甚至需要顶级人类专家耗费长达一个月的心血。 第二种是**开箱即用型现成数据（Off-the-Shelf Data）**。Mercor 已经投入了数亿美元自主构建覆盖广泛领域的专属数据集，并将其授权提供给多家客户。对于当下涌现的大量新型实验室（Neo Labs）而言，这种现成数据正变得极具吸引力，因为十家不同的实验室完全没有必要各自重复构建一套一模一样的数据集，“构建一次，普惠全行业”在经济效益上更为划算。 第三种则是**专家工时租赁（Hourly Expert Model）**。这种模式只向客户输出专家人才，由客户自行组织和管理专家团队开展工作。虽然像法律 AI 独角兽 Harvey 早期与 Mercor 合作时就是从雇佣法律专家起步，但随着行业发展，纯工时租赁已经不再是公司的核心业务重心，客户正不可逆转地转向前两种规模化、产品化的数据交付模式。过去仅仅掌握在前沿实验室手中的 RL 环境技术，如今正快速下放到所有应用层公司手中。 ```mermaid flowchart TD accTitle: 三种高质量数据采买模式 accDescr: 展示按任务定制、现成数据集与专家工时租赁三种模式的对比与流向 subgraph ByTask["1. 按任务定制 (By-Task)"] A1["客户指定领域环境与规格"] --> A2["按任务定价 (如 $2,000/task)"] A2 --> A3["前沿实验室月度万级规模采购"] end subgraph OffTheShelf["2. 现成数据集 (Off-the-Shelf)"] B1["投入数亿美元自主预先构建"] --> B2["标准化、高质量多领域覆盖"] B2 --> B3["多客户复用，Neo Labs 首选"] end subgraph ExpertHourly["3. 专家工时租赁 (Hourly Experts)"] C1["仅提供专业人才匹配"] --> C2["客户自行管理标注流程"] C2 --> C3["早期试点形态，正向标准化产品演进"] end ``` ### [12:36](https://www.youtube.com/watch?v=a00xIn5kwhM&t=756s) 问答环节：数据究竟该如何定价与估值？ 来自 Astro Guide 的 Ali 提出了一个直截了当的核心问题：“你们究竟如何为数据定价？数据的价值该如何衡量？” 数据的定价和估值通常从两个核心维度来切入。首要且最自然的考量是**从客户期望的模型提升倒推（Outcome-based Pricing）**。客户采购数据的终极目标是推动模型能力跃迁，在特定基准评测或排行榜上登顶前沿。因此，Mercor 会从该目标对客户的商业价值出发，逆向推导达成这一目标大约需要多少量级的高质量任务，从而敲定任务单价。以 Nvidia 这样的巨头为例，若他们愿意投入“十亿美元去打造一个处于前沿地位的开源基座模型”，那么为此配套的数据要素定价空间和运作逻辑就会完全不同。 另一个常规的定价维度则是**成本加成（Cost Basis & Margin）**。当一个复杂任务需要耗费专家 10 个小时的人工时间，且专家的时薪高达 150 美元时，该任务的基础人工成本便达到 1,500 美元。在此基础上，Mercor 会根据该任务的独特性、前沿门槛以及技术差异化程度，叠加不同比例的利润率。整体来看，平台上的任务单价跨度极为巨大，便宜的基础任务低至 50 美元，而极度复杂前沿的高难度任务单价则可飙升至 10,000 美元。 - **模型成果倒推法定价**：以客户的排行榜冲顶目标和模型提升意愿为锚点，评估达成目标所需的数据量并反向定价。 - **专家工时成本加成**：按顶尖专家时薪（如 $150/小时）乘以耗时（如 10 小时），在 $1,500 成本底线上叠加技术溢价。 - **单任务价格区间**：从 $50 到 $10,000 不等，完全取决于任务的复杂度与稀缺性。 ### [14:05](https://www.youtube.com/watch?v=a00xIn5kwhM&t=845s) 问答环节：解构“数据质量”与验证器的轨迹分析 当业内人士频繁讨论“数据质量”时，常常混淆了不同的概念。在 RL 环境中，数据质量其实严格对应着两个独立但同等重要的维度：**环境真实性（Realism）**与**验证器准确性（Accuracy of Verifiers）**。 真实性意味着环境必须精准复刻现实商业运转的分布。如果目标是让 AI 自动化企业法律业务，那么系统就必须还原一个真实企业律师工作环境中的全部信息分布。顶尖专家需要介入制定细化大纲，梳理覆盖整个行业分布的任务分类法（Taxonomy），确保环境中的应用、文件、文档与日常业务毫无二致。 而另一个决定模型训练成败的瓶颈在于自动评分器与验证器的准确性。在训练智能体时，系统可能会针对同一个任务生成例如 Kimi K3 等模型的 100 条行动轨迹（Trajectories），随后完全依赖评分量表（Rubric）对这 100 条路径进行自动化打分。由于智能体在复杂环境中探索出的行为路径千奇百怪，自动评分器的打分逻辑必须与真实人类顶尖专家人工排序的结果高度一致。 为了确保验证器不发生漂移，Mercor 采用了一种被称为**轨迹分析（Trajectory Analysis）**的质量控制机制： - 首先在待提升的模型上展开大约 10 条真实行动轨迹并由系统自动打分； - 随后结合智能体质检系统（Agentic QC Systems）与顶尖人类专家的交叉复核，检验机器评分与专家人工判断的一致性； - 此外，引入人类反馈评估（Human Feedback Evals）与偏好标签（Preference Labels），将其反向作为针对自动评分器本身的评测基准，从而彻底杜绝自动评分漏洞。 ### [16:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=990s) 问答环节：破除合成数据误解与人类专家的不可替代性 在讨论合成数据（Synthetic Data）与数据室（Data Rooms）构建时，行业内存在着显著的认知误区。许多人将强化学习可验证奖励（RLVR）与合成数据割裂开来，但实际上“RLVR 本质上就是对合成数据的一场巨大押注”。它的核心运作方式，正是放弃让昂贵的人类逐字手写监督微调（SFT）示范，转而让模型自主生成海量的合成轨迹，再通过验证器打分筛选，让模型在这些合成经验中完成强化学习演进。 不仅如此，AI 模型在构建环境和设计任务环节本身就扮演着不可或缺的角色。就像现实中起草法律备忘录的律师必然会使用 Claude 或 ChatGPT 来提升工作效率一样，构建庞大数据室的专家也理应借助顶级大模型来加速内容生成。然而，这绝不意味着可以完全摒弃人类走全自动合成路线。 人类专家在全流程中之所以具备不可替代的核心壁垒，是因为**在定义上，你必须依赖更具认知能力的人类，去度量那些超越模型前沿极限（Beyond the Frontier）的能力**。如果简单粗暴地指令模型“自主构建一个法律环境，并自行判断生成的多份法律备忘录孰优孰劣”，产出的结果往往极度嘈杂、充满幻觉且缺乏明确的有效信号。只有当评估者的能力凌驾于被测模型的前沿边界之上时，才能提供稳定可靠的度量与监督。 ### [18:05](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1085s) 问答环节：从搜索工具到 2025 应用爆发，以及接下来的两场范式转移 RL 环境在当下的集中爆发并非偶然，而是经历了一段清晰的技术演进历程。最早期的探索源于深度研究（Deep Research）范式。当时智能体唯一的外部工具就是搜索（Search），专家们只需围绕搜索结果制定评估量表，属于一种较为轻量级的原始 RL 环境。 到了 2025 年，多应用（Apps）环境之所以成为绝对的行业主流，是因为业界普遍意识到，制约模型落地实用的核心瓶颈在于：模型能否驾驭真实代码库中的全部上下文，以及能否熟练调用人类笔记本电脑上的各种桌面与企业级工具。为了让模型在用户的实际使用分布中表现出色，就必须先让模型在包含同等应用生态的数据分布中完成强化训练。 面向未来，数据生态在继续扩充真实世界应用多样性的同时，正在迎来两场决定性的范式转移： 1. **超长时序任务（Ultra-Long Horizon）**：目前绝大多数智能体训练任务的执行跨度极少超过 10 小时。行业亟需构建出能够对应人类专家 100 小时、甚至 1,000 小时连续工作负荷的超级长程任务环境。 2. **引入虚拟同事（Virtual Coworkers）**：长程真实任务从不是单打独斗完成的。在现实工作中，绝大多数任务都需要与团队中不同角色频繁协同与信息交互，未来的 RL 环境必须引入具备多智能体交互特性的虚拟同事机制，以模拟高度真实的组织协作生态。 ### [20:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1230s) 评估中的现实鸿沟：社交互动与虚拟协作者 在探讨智能体进入真实工作流时，现场讨论触及了一个被严重低估的维度：职场中的社交互动。当被问及一个人的日常工作中有多少比例是在与他人沟通协作时，大多数人的答案往往落在 60% 到 70% 之间，甚至有人认为这一比例要高得多。然而，反观现有的评测基准，用来衡量模型与他人协作能力的测试占比却微乎其微——大约只有区区 1%，即便是像 Tau-bench 这样相对前沿的基准测试，也仅仅触及了一点皮毛。 这种落差导致了巨大的“现实鸿沟”（realism gap）。现实中的智能体不能只是孤立地处理离线文档，它们必须学会在复杂的工作环境中与不同的人类同事以及其他软件智能体共同协作、沟通与博弈。如果评测体系无法准确衡量这种深层次的社交互动能力，构建真正可用、能融入真实团队的智能体就无从谈起。 - **职场社交时间占比**：日常工作中与他人沟通协作的时间普遍高达 60% 至 70%。 - **现有评测覆盖率**：主流 Eval 中专门测试人际交互与协作能力的题目仅占约 1% 左右（如 Tau-bench 涉及少量该类场景）。 --- ### [21:08](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1268s) 评分细则（Rubric）瓶颈与 AI Copilot 辅助生成 现场观众随后抛出了一个核心瓶颈问题：在强化学习环境中，任务评测依赖于量身定制的评分准则（Rubric-based Verifier），而这往往极度依赖各领域的人类专家；那么，是否有办法利用模型或启发式方法来实现评分准则的自动化与规模化扩展？Brendan Foody 指出，目前行之有效的解法并不是让模型完全取代人类，而是引入“AI Copilot”协助专家共同构建任务与验证器。在协同工作流中，专家可以通过 Copilot 与智能体的执行轨迹（Trajectory）展开对话，快速厘清模型究竟在哪个环节出错，进而显著提升编写效率。 然而，如果完全依赖模型自我生成评分规则，技术上面临着致命的噪声困境。如果一个模型本身能力不足以解决任务，它就根本无法可靠地写出指出自身错误的评判准则——其判断往往是“一半对、一半错”，而这种程度的噪声从训练的角度来看是“完全无法接受的”（unworkable from a training standpoint）。因此，在整个构建链条中，最需要人类深度介入的环节恰恰是任务和评分标准的制定。虽然底层环境的许多文档和数据可以利用合成技术生成，且只需人类把关以确保其符合真实分布，但唯独任务与判定规则必须由人类专家牢牢锚定。唯一的例外在于知识蒸馏：如果使用更强的高阶模型为弱模型生成任务，较弱的模型确实能够从中有效学习。 ```mermaid flowchart TD accTitle: AI Copilot 协同任务与评分准则构建流程 accDescr: 展示人类专家与 AI Copilot 协同分析智能体轨迹并生成低噪声评分细则的流程 subgraph EvaluationLoop ["轨迹审查与协同"] A["智能体执行轨迹 (Trajectory)"] --> B["AI Copilot 辅助分析"] B <--> C["领域人类专家 (Domain Expert)"] end subgraph RubricGeneration ["细则输出与校验"] C --> D["精准评分准则 (Rubric Criteria)"] D --> E["高保真验证器 (Reliable Verifier)"] E --> F["低噪声 RL 训练数据"] end ``` - **纯模型自评的困境**：能力不足的模型自行生成判断规则时错误率高达约 50%，高噪声直接摧毁训练效果。 - **合成数据与任务的边界**：环境背景资料（如虚构系统与文档）可高度依赖合成与人类框架，但评判任务（Task & Rubric）必须由人类专家把关。 - **蒸馏特例**：若采用强模型（如顶级前沿模型）为弱模型生成任务与评分准则，仍能实现有效的知识蒸馏。 --- ### [22:47](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1367s) 网络安全与可证明领域的自动化验证优势 来自 Cribl 的观众 Nikhil 提出了针对特定“可证明领域”（provable domains）的疑问：例如在网络安全防御或应急响应中，智能体的目标是在既有系统中寻找漏洞并排查故障，在这些场景下，人类专家是仅用于设计搭建环境，还是同样需要参与评分？这类领域是否有更高的扩展空间？ Brendan Foody 表达了非常积极的看法，指出网络安全正是那种“不一定始终需要人类来编写验证器”的典型领域。其核心原因在于安全攻防具备天然的博弈属性，可以通过构建攻防对抗智能体（Attacker vs. Defender Agents）来实现自动化校验。在这种场景下，人类的角色发生了转变：专家不再需要为每一个细分动作编写判定规则，而是退居幕后担任环境架构师，负责搭建高度逼真、具备多样性与代表性的网络靶场与系统环境，后期的攻防胜负与漏洞触发则可交由系统机制自动化判定。 - **攻防对抗替代人工打分**：利用红蓝对抗智能体（Attacker / Defender）可实现闭环判定，大幅降低对人类验证器的依赖。 - **安全领域的人力侧重**：人力重心从“逐题评分”转向“靶场架构”（设计逼真且具多样性的安全与系统环境）。 --- ### [23:46](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1426s) 如何判断基础模型是否触及瓶颈：Pass@1 与 Pass@16 启发式 现场观众提出技术层面的疑问：在给定相同高质量后训练数据集的情况下，不同基座模型的最终表现往往会趋同收敛；如果选用较小的模型，其最终表现可能会大打折扣，这究竟是单纯受限于参数量，还是有更本质的判断准则？企业如何断定自己是否受制于基础模型本身的上限？ Brendan Foody 解释道，参数量确实会在模型的可训练性（Trainability）上发挥作用，但判断基础模型是否能从特定数据集中有效学习的关键启发式指标，在于观察其 **Pass@1 与 Pass@16 之间的差距**。如果为一个任务采样展开 16 条轨迹，模型在 16 次尝试中全部彻底失败（Pass@16 为 0），那么指望模型在此类任务中通过强化学习自发学会基本是徒劳的（hopeless）。理想的学习状态是：模型在单次尝试（Pass@1）时失败，但在展开 16 次轨迹采样时能够偶然蒙对一到两次（Pass@16 命中 1~2 次）。只要存在这一两次正确的微弱信号，模型就能在强化学习的引导下快速捕捉规律并高效泛化。 - **绝望区间（Hopeless）**：Pass@16 = 0，展开 16 次轨迹全错，几乎无法形成有效梯度。 - **理想训练区间（Ideal Learning）**：Pass@1 失败，但 Pass@16 成功 1~2 次，模型能沿着有效轨迹迅速强化掌握技能。 --- ### [25:19](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1519s) 自建数据还是外部合作？规模经济与定制化壁垒 针对企业在 RL 后训练阶段应当“自建数据体系”还是“选择外部伙伴”的提问，Brendan Foody 分享了 Mercor 的核心商业定位与数据战略。他透露，Mercor 目前有很大一部分业务是提供“定制化专属数据”（Custom Data）。为了保护客户的长期护城河，他们会组建与外界完全隔离的专属团队，专门为核心大客户量身定制顶级数据集，且数据资产完全归客户所有。这种机制不仅让客户能够牢固维持自身专属领域的竞争优势，同时又能坐享专业数据平台所沉淀的基础设施红利。 Brendan Foody 观察到，市场上总有一些公司试图在内部从零搭建属于自己的全套专家人才网络和标注工程架构，但前沿顶尖实验室的发展轨迹已经充分证明了这一模式的高昂代价。构建与维护全球顶尖专家网络、高保真企业软件克隆生态以及验证器底层架构，本身具有极其显著的规模经济（Economies of scale）。与其花费数年去重复造轮子，与拥有规模化平台与人才储备的专业伙伴深度绑定，才是当前模型与应用公司在激烈的 AI 竞速中快速取得领先的最优解。 - **专属隔离团队**：为关键客户配置独立团队构建专有数据集，产权全权归客户所有，构筑独家模型壁垒。 - **复用规模红利**：前沿实验室的实践表明，自建全套人才网络极其沉重，借助成熟数据伙伴的规模经济与平台基建效率更高。 ## Notable quotes > [03:57](https://www.youtube.com/watch?v=a00xIn5kwhM&t=237s) 就如同让学生给自己批改作业一样不可靠 — Brendan Foody > [10:15](https://www.youtube.com/watch?v=a00xIn5kwhM&t=615s) 构建一次，普惠全行业 — Brendan Foody > [12:36](https://www.youtube.com/watch?v=a00xIn5kwhM&t=756s) 十亿美元去打造一个处于前沿地位的开源基座模型 — Brendan Foody > [16:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=990s) RLVR 本质上就是对合成数据的一场巨大押注 — Brendan Foody > [16:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=990s) 在定义上，你必须依赖更具认知能力的人类，去度量那些超越模型前沿极限的能力 — Brendan Foody > [16:30](https://www.youtube.com/watch?v=a00xIn5kwhM&t=990s) 自主构建一个法律环境，并自行判断生成的多份法律备忘录孰优孰劣 — Brendan Foody > [21:08](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1268s) 一半对、一半错 — Brendan Foody > [21:08](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1268s) 完全无法接受的 — Brendan Foody > [22:47](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1367s) 不一定始终需要人类来编写验证器 — Brendan Foody > [23:46](https://www.youtube.com/watch?v=a00xIn5kwhM&t=1426s) Pass@1 与 Pass@16 之间的差距 — Brendan Foody ## Mentioned Mercor, Brendan Foody, 红杉资本 (Sequoia Capital), GPT-3, ChatGPT, GPT-4, Claude, Harvey, Cognition, Ramp, Salesforce, ServiceNow, Google Workspace, Microsoft 365, 美国劳工统计局 (BLS), GDP-Bench, GDP-Eval, 盛德律师事务所 (Sidley Austin), 瑞生律师事务所 (Latham & Watkins), 石油污染法 (Oil and Petroleum Act), 星际油轮国际有限公司 (Star Tankers International Limited), 库珀·杰弗里斯能源公司 (Cooper Jefferies Energy Corporation), GLM-52, Chimera K3, Apex Agents, Apex V1, Cursor, Nvidia, Kimi K3, Tau-bench, Cribl, Ali, Nikhil ## Video description RL environments have become the hottest topic in AI training data. But what are they, exactly? At Sequoia Capital’s Own Your Intelligence event, Mercor CEO Brendan Foody breaks down the three components: worlds (the messages, docs, and files of a real project), apps (high-fidelity clones of tools like Salesforce and Google Workspace), and tasks (prompts paired with rubric-based verifiers). He walks through a real legal environment built with lawyers from top firms, and shares post-training results showing dramatic gains on domain-specific tasks from modest compute. Brendan also covers why humans remain essential for measuring the frontier, how Mercor prices data, the shift from crowdsourced labeling to expert-built environments, and what's next: ultra-long-horizon tasks and virtual coworkers. He argues that technology once limited to frontier labs is now reaching application companies, and the datasets they build are becoming their moat. 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner? [Source](https://www.youtube.com/watch?v=a00xIn5kwhM)

[3:57] Brendan Foody: 就如同让学生给自己批改作业一样不可靠 [10:15] Brendan Foody: 构建一次，普惠全行业 [12:36] Brendan Foody: 十亿美元去打造一个处于前沿地位的开源基座模型 [16:30] Brendan Foody: RLVR 本质上就是对合成数据的一场巨大押注 [16:30] Brendan Foody: 在定义上，你必须依赖更具认知能力的人类，去度量那些超越模型前沿极限的能力 [16:30] Brendan Foody: 自主构建一个法律环境，并自行判断生成的多份法律备忘录孰优孰劣 [21:08] Brendan Foody: 一半对、一半错 [21:08] Brendan Foody: 完全无法接受的 [22:47] Brendan Foody: 不一定始终需要人类来编写验证器 [23:46] Brendan Foody: Pass@1 与 Pass@16 之间的差距

[3:57] Brendan Foody: like asking a human to grade their own homework. [5:35] Brendan Foody: Data is increasingly recognized as the single most differentiating moat among the three pillars of an AI strategy: compute, algorithms/researchers, and training datasets. [6:44] Brendan Foody: assessing the maximum total liability for Star Tankers International Limited versus Cooper Jefferies Energy Corporation under the Oil Pollution Act, accounting for every document inside the data room. [9:45] Brendan Foody: Model accuracy jumped from 4.7% to 26.6%. [12:49] Brendan Foody: Individual task prices range dramatically from $50 to $10,000 per task. [14:17] Brendan Foody: Ensuring that rubric-based auto-graders score model outputs identically to how a panel of human experts would rank them. [16:42] Brendan Foody: RLVR itself is fundamentally a bet on synthetic data: instead of paying humans to write static Supervised Fine-Tuning (SFT) responses, models generate thousands of synthetic trajectories that are scored by verifiers so the model can learn iteratively from its own generations. [17:16] Brendan Foody: You cannot simply instruct a model to generate a complex legal environment and then accurately distinguish between high- and low-quality output; the resulting feedback is far too noisy and lacks clear evaluation signals. [18:05] Brendan Foody: Introducing virtual co-workers into environments will bridge this critical realism gap. [23:46] Brendan Foody: If a model attempts 16 trajectory rollouts on a task and fails every single one, it lacks the foundational capabilities needed to learn from that dataset.

[0:00] Um, Record, I think you guys grew from a [0:01] 1 to a 2 billion dollar revenue run rate [0:03] in the last 4 months or so. [0:06] Um, so this company's off to the races [0:08] and I think you were just so front and [0:09] center to how companies are thinking [0:11] about uh post training their own models, [0:13] uh building their own intelligence. So, [0:15] thank you for joining us for this [0:16] conversation. Um, format-wise what we're [0:19] going to do is we've 15 minutes or so of [0:22] content from Brendan. He's going to talk [0:23] about uh RL environments in particular, [0:26] which I think is a, you know, new [0:28] frontier topic. It'll be fun to fun to [0:30] explore. And then we're going to leave [0:32] 15 minutes or so at the end for Q&A [0:34] again. So, uh please keep please keep [0:37] questions back pocket. I will turn it [0:38] over to you, Brendan. [0:39] >> Sweet. So, I'll be talking about RL [0:41] environments. Starting out, I figured [0:43] it's helpful to give a little bit of the [0:45] background on the history of the data [0:48] market and how that history ties into [0:51] Record's origin story. Where things [0:53] really started in 2020 in the era of [0:56] crowdsourcing data for behavior cloning. [0:58] So, this was mainly supervised [1:00] fine-tuning data, inputs and outputs, [1:02] and RLHF data where you would have a [1:06] annotator select from a couple of model [1:08] responses which they preferred. And we [1:10] were able to make all this progress in [1:13] fine-tuning GPT-3, making progress [1:16] towards ChatGPT and GPT-4 in the [1:18] crowdsourcing era of agentic data. But, [1:21] what we saw changing in the market, [1:24] especially as we uh headed into 2024, [1:27] was this giant transition away from the [1:30] low-skilled crowdsourcing era of [1:33] behavior cloning data and moving towards [1:36] the agentic era of data. Of how do we [1:38] find the highest-skilled experts in the [1:40] world that can work collaboratively in [1:42] teams to build frontier evals and RL [1:44] environments for the next generation of [1:46] models models. All the software [1:48] engineers, lawyers, doctors, bankers, et [1:50] cetera that could measure the frontier [1:53] of intelligence and help to use that to [1:57] improve model capabilities. And so, [2:00] Mercor grew up with our first big [2:02] project being deep research. I guess the [2:04] first prominent RL agent [2:07] scaling up dramatically with all of the [2:09] frontier labs to become the primary [2:11] agentic data vendor to [2:14] all of the leading labs [2:16] and also all of the leading application [2:17] layer companies ranging from Harvey, [2:19] Cera, Cognition to Ramp. And what's been [2:22] really exciting over the last 12 months [2:26] especially is how RLVR within the [2:29] agentic data paradigm has evolved to [2:32] also include RL environments with these [2:35] rich apps and worlds that teach agents [2:38] how to use all of the tools on [2:41] our laptops that we use every day. So, [2:43] I'll be talking about that [2:45] and of course how this technology that [2:48] started in the frontier labs is now [2:51] getting disseminated to the application [2:53] layer and all of the products that all [2:55] of you are building [2:57] as you [2:58] work on your company. So, high level on [3:00] what an RL environment is is that it [3:02] includes three parts. The first part is [3:04] the worlds. So, this includes all of the [3:06] messages, slides, docs, sheets, etc. [3:09] that correspond to everything you would [3:11] have in a real project or company that [3:14] you're working on. The second part is [3:16] the apps which is high fidelity clones [3:18] of popular applications, Salesforce, [3:21] ServiceNow, Microsoft 365, etc. that [3:24] agents can interact with via MCP, CLI, [3:26] or Kua. And then the third part is the [3:28] tasks where we have prompts and [3:30] verifiers. Verifiers could be rubrics or [3:33] unit tests that can be used either for [3:35] eval or training. And the barrier for [3:37] frontier labs to automate everything [3:40] that you can do on your laptop using [3:42] Claude is how do they cover the full [3:44] distribution of all of the worlds, all [3:47] of the apps, and all of the tasks in the [3:49] economy. And so, there's been this [3:51] enormous scale out in order to do that. [3:54] Um, where humans have been really [3:57] central to how we build these [3:58] environments, obviously with models in [4:00] the loop meaningfully. And so I put a [4:02] graph here of the amount of expert hours [4:05] that um, [4:07] of throughput in from our talent network [4:09] over the last 24 months. Um, [4:12] and it's a a pretty crazy trajectory [4:15] with respect to um, 2.5 million hours [4:19] um, in Q2 alone with growth sort of [4:21] accelerating on the amount of expert [4:23] time uh, used to build out all of these [4:27] environments. The reason being of course [4:29] as I mentioned we need to scale out the [4:31] environment distribution across every [4:34] category in the economy. Many of you [4:35] might know GDP val where there's 205 [4:39] domains in the Bureau of Labor [4:40] Statistics across all the different [4:42] jobs, but then you have to think through [4:44] how do we have all of the apps [4:45] corresponding to all of those jobs, all [4:47] the different scenarios, all the tasks. [4:49] Is this enormous build out. Only humans [4:52] can measure the frontier in most [4:54] domains, not every domain. There are [4:55] rare exceptions like math where you have [4:58] a really clean simulation environment [4:59] and so uh, the model's able to learn [5:02] from whether it got the right answer, [5:03] but in most domains like building a [5:05] slide deck uh, the model has an [5:07] incredibly hard time identifying [5:09] reliably where it made its own mistake. [5:11] It's as if you would be asking a human [5:14] to grade their own homework. And so [5:16] that's why it's really valuable to have [5:18] a human create a rubric similar to how a [5:22] professor would create a rubric to grade [5:23] an essay or a TA would grade that slide [5:26] deck. Similar to the way that a lot of [5:27] us learn it's in large part from the [5:30] feedback we got from those around us [5:31] rather than [5:33] uh, purely plugging things into a [5:35] calculator or clean simulation. Um, [5:38] and then building these verifiers is [5:39] hard cuz anytime you're building the [5:41] slide deck you need to understand the [5:44] full problem space of what are the 10 [5:46] different slide decks that, you know, [5:48] could be a good path to go down? What [5:50] are the dozens of mistakes you could [5:52] possibly make? And how do you build a [5:53] comprehensive verifier that captures [5:56] this full solution area of what's [5:57] possible? And so, what I'll walk through [5:59] is a sample RL environment. [6:01] Excuse me, to also break this down for [6:03] all of you. Um and part of the reason [6:06] that this is so cool, which I'll get to [6:07] in a moment, is that we developed a lot [6:10] of this technology in collaboration with [6:13] the labs. These are, of course, ones [6:15] that we have open-sourced and published [6:17] to the world, but now that's all [6:19] starting to get disseminated to the [6:21] application layer companies that are [6:24] building and owning their own [6:25] intelligence. As they realize that the [6:27] three core pillars of their AI strategy [6:30] are their compute, their algorithms or [6:32] researchers, and the data sets they [6:34] build. And data's often the most [6:36] differentiating factor. And so, this is [6:38] one that we published, um [6:40] as a legal environment, where we have [6:42] lawyers from top law firms like Latham [6:45] and Watkins write out a scenario of a [6:48] real project that they worked on in [6:50] their big law job. And then they create [6:52] a full outline for a data room that [6:55] corresponds to all of the different uh [6:57] messages, emails, files, size of files. [7:01] I cut off the full data room cuz it's [7:03] it's very extensive. Um and of course, [7:06] there's a lot of model in the loop with [7:07] how they effectively populate this. [7:09] Similar to how a software engineer now [7:11] should not be coding by hand entirely [7:13] themselves, they should probably be [7:15] orchestrating agents in how to do this [7:17] very productively. Um [7:19] and then we render that data room into [7:21] the apps, the clones of Google Workspace [7:23] you can see in this scenario, and have [7:25] prompts uh to roll out model [7:27] trajectories against this. And so, in [7:29] this one, it's evaluating the maximum [7:32] total liability for Star Tanker uh [7:34] Tankers International Limited compared [7:37] to Cooper Jefferies Energy Corporation [7:38] under the Oil and Petroleum Act, uh [7:41] considering all of the context uh from [7:43] this real scenario in the data room. And [7:44] then as I mentioned, similar to how a [7:46] professor would create a rubric to grade [7:48] an essay, they have these key rubric [7:50] criteria that correspond to what are the [7:53] characteristics of a of a accurate model [7:56] response. And making sure that these [7:58] rubric criteria avoid reward hacking and [8:01] effectively align with the uh when you [8:05] roll out 100 trajectories, making sure [8:06] all of those scores are accurate is [8:09] incredibly technically challenging. And [8:11] so there's an enormous amount of [8:13] research, agenda quality control, [8:15] training on the data, etc. that goes [8:17] into how you solve that problem and then [8:18] ultimately produce these high-quality [8:20] verifiers and leaderboards um that give [8:23] you an aggregate model score across how [8:25] well all of the different models are um [8:27] doing on a particular domain. And as we [8:30] can see, one of the big changes over the [8:32] last few months is that GLM 52 and [8:35] Chimera K3 are on the leaderboard. And [8:37] so that is a huge opportunity for all of [8:40] you because that gives us the foundation [8:44] to actually achieve frontier [8:46] intelligence and all of the specific [8:48] applications uh and verticals that [8:51] you're focusing on um that's not too far [8:53] away. And so to give a little bit of [8:56] context on what that looks like, um I'll [8:58] share an example of post-training on [9:00] Apex Agents, which is the data set that [9:03] I uh or the sample I just showed before, [9:06] where we have 1,800 tasks in this [9:08] example. This was post-training run of [9:11] GLM 47, but we're redoing a lot of them [9:13] for Chimera K3, so we'll have updated [9:15] results for you all soon. Where you can [9:17] see the jumps just on 1,800 tasks with [9:20] about 500k in compute are pretty [9:23] dramatic. Um corporate law going from [9:26] 4.7% to 26.6%, [9:29] um but notice that this is just Apex [9:32] Agents data set we gave it, and it [9:34] actually generalized incredibly well to [9:37] GDP valve and uh Apex V1, which doesn't [9:40] have these data rooms. [9:42] Even just seeing nominal gains and some [9:45] other benchmarks [9:47] as well. And so, we're doing a lot of [9:50] this work of working with customers like [9:52] Harvey, who I know will present on stuff [9:54] later to help build out the [9:58] environments corresponding to their [10:00] specific domain so that they can build [10:02] frontier intelligence within that. And I [10:05] think Andrew talked about how Cursor was [10:07] a great first example of how an [10:10] application layer company could build a [10:13] industry-leading model that, you know, [10:15] built an enormous amount of value for [10:16] their customers. And I believe that over [10:18] the next 12 months, there is going to be [10:21] dozens of examples just like that where [10:23] companies own their own intelligence and [10:25] that is the key source of the modes that [10:28] they're building. And actually, Josh and [10:29] I talked about this the other day as [10:31] well. Um [10:33] a couple of examples of ways to curate [10:35] high-quality data sets. [10:37] The general three that we see most that [10:40] I'm happy to [10:42] talk about and send people links to is [10:44] first by task is the most common. [10:47] Where people would say, "I really like [10:48] this data shape of environments in law [10:53] and we will pay $2,000 per task to scale [10:56] this up." And [10:58] as an example, certain frontier labs [11:00] might buy 50,000 tasks a month from us. [11:03] And so, it tends to be pretty dramatic [11:06] scale. [11:07] And these tasks would generally be very [11:09] complex. Some would even take humans up [11:11] to a month to complete that given task. [11:14] Sometimes it would take just a few [11:16] hours. And these would be sort of custom [11:18] per task pricing. Second is [11:20] off-the-shelf data [11:22] where we have we've invested hundreds of [11:24] millions of dollars in building our own [11:26] data sets that we sell to multiple [11:28] customers. All these new neo labs are [11:30] generally airing more towards [11:32] off-the-shelf data because it doesn't [11:34] make sense for 10 different labs to all [11:36] be building their own uh data sets. Um [11:38] there's a lot of value to building [11:40] something once that can uh then be [11:42] applied to everyone. And then the final [11:44] which um we see a little bit of, but is [11:46] uh less of our focus anymore, is just [11:49] providing um the experts so that [11:52] customers are able to [11:54] uh or organize the experts on their own [11:57] um [11:58] in just an hourly model. Um so we we do [12:00] a little bit of that when people like [12:02] that was how Harvey got started with us [12:04] hiring [12:05] uh some lawyers, um but it generally [12:07] moves towards more of these uh scaled [12:09] offerings of data over time. So, that's [12:12] a little bit of the background of how to [12:14] build RL environments, what they are, [12:16] and I'm really excited about all this [12:18] technology that we have uh [12:21] that has previously been limited to the [12:23] frontier labs all making its way to all [12:25] of you. And so, happy to answer any [12:27] questions um [12:28] about that. [12:34] Sweet. [12:35] Go ahead. [12:37] >> Hey, uh [12:39] um I'm Ali from Astro Guide. My question [12:41] is it's kind of open-ended, but simple. [12:43] How do you price data? [12:45] Like how do you value data? [12:47] >> So, there's a so many different ways. I [12:50] mean, [12:51] the most natural would be our customers [12:53] care about model improvement, right? And [12:56] so, our customers have a given goal of [12:58] they want to you know, be at the [13:00] frontier on a given leaderboard. And so, [13:03] we're able to work backwards from how [13:05] much is that worth to them and how much [13:07] should we charge uh per task, how many [13:09] tasks do we think would get them to that [13:11] goal. And so, when we think about uh a [13:13] company like Nvidia, they're probably [13:15] willing to pay, you know, a billion [13:17] dollars to have a frontier open-source [13:19] model. And so, there's a lot of [13:21] complexity of like how do we you know, [13:23] price all the different ingredients that [13:24] go into um making that happen. [13:28] The other way that we price [13:31] when it lens we look at it through is [13:33] also our cost structure to make them [13:36] where of course [13:39] when we have a task that takes 10 hours [13:42] of human time and [13:44] we're paying the human $150 an hour [13:47] there might be a $1500 cost basis and so [13:50] then it becomes a question of what [13:52] margin do we want to run on top of that [13:54] based on how differentiated and frontier [13:56] that specific task is. [13:59] But it's it's super wide range. Like we [14:00] have tasks that range from $50 to [14:03] $10,000. So. [14:05] >> How do you think about data quality? You [14:08] mentioned like you know utilize human [14:10] expert to label data and how do you how [14:12] do you compare the human label data and [14:14] you know the frontier lab you know [14:16] frontier [14:17] alliance the judgment data? How do you [14:20] compare them to from your opinion? [14:23] >> So first question was how do we think [14:25] about quality? Second one was sort of [14:27] how do we compare the like judgment of [14:30] preference labels to the auto graders? [14:33] >> Yeah. [14:33] >> So the core spot which is generally when [14:36] people say data quality they're [14:38] referring to two things. First is [14:41] realism and secondly is accuracy of [14:44] verifiers. [14:46] On realism it's just like they want to [14:48] automate everything in the economy that [14:50] corresponds to corporate law in this [14:53] case right? And so it's like how do we [14:54] make sure that this actually reflects [14:57] the real distribution of what we would [14:59] see in a real lawyer's environment. And [15:02] that's one of the reasons that experts [15:04] create outlines and help to guide all [15:06] the processes of the data curation. [15:08] Realism of the environment the apps the [15:11] tasks everything is incredibly important [15:14] and and also granularly understanding [15:16] the taxonomy that drives that realism [15:18] across the entire distribution that [15:20] you're looking for. The second part of [15:23] it relates to the other way people think [15:25] about quality, which is the accuracy of [15:27] the verifiers. Cuz the way that you [15:29] would train one of these models is you [15:30] might roll out 100 trajectories of Kimi [15:33] K3 and then use this rubric to score all [15:36] of those trajectories. And as you can [15:38] imagine, there's like so many different [15:41] paths that a model can go down. And so [15:43] you want to make sure that the way this [15:45] rubric is doing the scoring is the same [15:48] as if we were to just have human stack [15:49] rank those 100 trajectories. And so what [15:51] we do for that is a process called [15:53] trajectory analysis, where we roll out [15:56] 10 trajectories of the model that we're [15:58] focused on improving, [16:00] um, and then score all of those, and [16:02] have some combination of agentic quality [16:05] control systems and some human review go [16:08] through to make sure that, um, all of [16:12] the scores align with, uh, the goals. [16:15] Um, [16:16] and sometimes you can also use, uh, [16:18] human feedback evals or preference [16:20] labels to as a eval for your auto [16:22] grader, um, is the other way related to [16:25] that that you're able to solve for it. [16:30] Go ahead. [16:30] >> Um, how much do you think like synthetic [16:32] data generation plays into all of this, [16:34] especially like creating these large [16:36] data rooms? [16:38] >> So the fascinating thing is I think that [16:39] there's been a lot of misinterpretation [16:42] of what people mean when they say [16:44] synthetic data because, like, RLVR is a [16:46] bet on synthetic data. It's basically [16:49] let's roll out a bunch of synthetic [16:50] model trajectories rather than having [16:52] the humans write the SFT, and then let's [16:54] score all of them, and let the models [16:56] learn from all of these like, uh, [16:59] synthetic model trajectories. So I think [17:01] that's the first way that synthetics get [17:03] used. The second way is that models play [17:07] giant role in the way that we populate [17:11] environments and create tasks in the [17:13] same way that, um, a lawyer that is [17:16] writing a legal memo should definitely [17:18] be using [17:20] Claude or ChatGPT to do that. The [17:23] experts that are building out these data [17:24] rooms should definitely be using Claude, [17:26] ChatGPT, or whatever model [17:29] to help them do that. And there's a lot [17:31] of ways that the model can make them [17:32] more efficient. But the reason that [17:34] humans are still an essential component [17:36] of the process that's incredibly [17:38] differentiated is that you need humans [17:41] almost definitionally to measure what is [17:43] beyond the frontier of the model [17:45] capabilities. [17:46] Like the models you you can't just tell [17:48] the model like come up with the legal [17:49] environment and then like tell me which [17:51] of your legal memos are like good and [17:53] bad. It's super noisy and there's not [17:55] like clear signals associated with that. [17:57] You need [17:58] something that has capabilities beyond [18:00] the frontier of that model to do so [18:02] reliably. [18:05] >> Thank you for doing this. Um my question [18:08] is RL environments seem like they're all [18:10] the rage now and maybe have been for [18:11] about a year. I had I hadn't really been [18:14] hearing about them prior to that and it [18:15] was all [18:17] human expert labeling. And so [18:19] kind of why why why is it all about RL [18:21] environments now? Is that the most [18:23] relevant thing for application companies [18:25] to be thinking about? And is there any [18:26] something after RL environments? [18:29] >> Um so I'll I'll start with why it's [18:32] become the rage and maybe some of the [18:35] differences also between the deep [18:36] research paradigm and the like envi- the [18:39] sort of environments paradigm as we saw [18:41] it in 2025. And then I'll talk about [18:43] looking forward what we see evolving in [18:45] the data landscape. Specifically, I [18:47] think that the reason the deep research [18:50] environments were the first was because [18:51] like deep research had tool use with [18:53] search. So search was the tool in the [18:55] environment that the model [18:58] would work in. But the experts would not [19:00] necessarily be populating apps. So it [19:02] was sort of a lighter version of an RL [19:04] environment where they would just like [19:06] create rubrics corresponding to this. [19:08] And again, like I only talk about this [19:10] stuff cuz it's a couple of years old at [19:11] this point, so it's no longer [19:13] uh super confidential. And then for the [19:16] like trend of apps in 2025, I think that [19:20] really became giant because people [19:23] realized that the primary bottleneck to [19:26] making the models useful was how they [19:27] started to use both all the context in [19:30] the code base and all of the tools on [19:32] our laptops, right? And so if we want [19:34] this in the user distribution of usage, [19:37] then we need to get it in the data [19:39] distribution that the models are [19:41] learning from. [19:42] Uh and so there's going to continue be [19:45] to be this giant scale up of diversity [19:48] across all three of these categories on [19:50] a going forward basis, but there's going [19:52] to be some changes [19:53] um to maybe name two of those changes [19:56] that we're thinking about the most. The [19:58] first one is ultra long horizon. Like [20:01] right now agents mostly aren't trained [20:03] to do things that are over 10 hours, and [20:06] we need to start building tasks for [20:07] things that might take a human 100 hours [20:10] or even 1,000 hours to do. And so that's [20:12] going to be a giant shift. And then the [20:14] other large shift that we're seeing is [20:17] introducing virtual co-workers, which [20:19] corresponds to that. Um like one my [20:22] favorite questions to ask people when [20:24] they're thinking about their data [20:25] distribution is what percentage of tasks [20:27] that they do in their job require [20:30] interacting with other people. [20:32] Uh and most people would say like 60% or [20:34] 70%. Some people say a lot more, some [20:36] people say a little bit less. Um but [20:38] then if you map that on to what [20:41] percentage of evals measure how well the [20:44] models can interact with other people, [20:46] it's like 1%, maybe Tau bench has a [20:48] little bit of this. [20:50] Um [20:51] and so there's this giant realism gap [20:53] associated with how you actually measure [20:57] how well agents engage in social [20:59] interaction um throughout uh all of the [21:02] different people and other agents that [21:04] they need to work with uh in their jobs. [21:07] Good. [21:08] >> You talked about rubric generation, [21:10] which is like is it like a bespoke [21:12] access of verify that you put task? And [21:14] from what I understood, that's like [21:16] bottleneck by experts. Have you found [21:17] any success of like being able to scale [21:20] that up with your models giving you like [21:22] some heuristic or even like post any [21:24] models for [21:26] >> We have found that you can make it a lot [21:29] more efficient if you have a like AI [21:32] copilot that's able to work with the [21:34] expert in creating the task and the [21:36] verifier. [21:37] Um so, the expert can talk to the [21:40] trajectory and understand exactly what's [21:42] happening and where it's going wrong. [21:44] The challenge is just that if you're [21:46] trying to improve FABLE, [21:48] FABLE cannot reliably write out the like [21:53] rubric criteria for where it's making [21:54] mistakes. It might get like half of them [21:57] right and half of them wrong, and that [22:00] amount of noise is unworkable from a [22:03] training standpoint. Um and so, that's [22:06] the reason that the the process that [22:08] requires humans the most is the task [22:10] creation. Uh like a lot of the [22:12] environments, we can get [22:14] like use a lot of synthetic. Uh [22:17] it's helpful to have humans write the [22:18] outlines they're familiar with the [22:19] environment and and uh [22:22] and they're grounded in reality, a [22:23] realistic distribution, but with the [22:26] task that's those tend to really require [22:29] humans. Um [22:30] with with rare exceptions, um [22:33] in code or if you're sort of distilling [22:37] from like if you have a model that's [22:38] worse than Kimikaze 3, then it can [22:41] definitely learn from tasks Kimikaze 3 [22:42] is creating. So, there are there are [22:43] some exceptions if you're doing it that [22:45] way. [22:47] >> Hey, this is uh Nikhil from uh Cribl. [22:50] So, when we think about RL environments [22:52] for certain provable domains like cyber [22:54] defense or incident response, where [22:57] there's the model is or the agent is [22:59] trying to find a flaw in an existing [23:01] system, [23:02] Do you use uh [23:04] humans for just authoring that [23:05] environment or setting it up or do you [23:07] also use that for grading? Is there Is [23:09] there a way to scale that up? [23:11] >> I I actually think cyber is one where [23:14] you don't necessarily always need humans [23:16] for the verifiers cuz you can have an [23:18] attacker and a defender agent. [23:20] Um [23:20] and uh I think you're right in saying [23:23] that for cyber you can have humans more [23:26] so or or architect what is a realistic [23:29] like environment um and sort of set up [23:32] the environment um cuz you do need a lot [23:35] of diversity. [23:36] Uh but then it's less human intensive [23:39] with respect to uh building verifiers. [23:43] >> Thanks. [23:46] >> How can you tell when you're limited by [23:47] the base model? [23:50] >> What do you mean by that? [23:51] >> Well, ostensibly you're using the same [23:53] data set for all these models here and [23:55] they somewhat land around the same final [23:57] performance on this list here. But maybe [24:00] if you try to smaller model, which is [24:01] maybe a good place to start, it would [24:03] land much lower. Um what is the cause [24:05] there? Is it just parameter count or [24:08] >> So [24:09] the parameter count will definitely play [24:12] a role in [24:14] how effectively the model does insofar [24:18] as how trainable it is. I think that um [24:22] the main thing to look at is generally [24:24] the gap between the like [24:28] pass at 16 and the pass at one. If you [24:31] have a model where you roll out 16 [24:33] trajectories and it gets all of them [24:35] totally wrong, the it's sort of like [24:38] hopeless that the model is going to [24:40] learn from that um for the most part. [24:42] Maybe you roll out another 100 [24:44] trajectories and it gets one of them [24:45] right. Um [24:47] versus if you have um [24:49] the ideal case is that you have [24:52] um [24:52] pass at one it fails, but then pass at [24:55] 16 when you roll out 16 trajectories it [24:57] gets it right once or twice, and then [24:59] the model is able to learn very [25:00] effectively from that. So, that's [25:02] generally the heuristic we would use for [25:06] how strong the base model needs to be to [25:09] effectively learn from a given data set. [25:14] Cool. Maybe [25:15] Oh, [25:16] final question really quick. [25:19] >> What advice do you have for [25:21] companies as they partner with you [25:23] on data that they should rely on [25:25] themselves as [25:27] part of their RL post-training versus [25:29] relying [25:30] What's the complementary [25:32] >> Well, I think this is why a giant [25:34] portion of our business is custom data, [25:36] where it's like we have teams that are [25:39] siloed and fully exclusive to [25:42] critical customers to make sure that we [25:45] build out the best data sets in the [25:46] world that they own. [25:48] And that allows them to maintain their [25:51] competitive advantage associated with [25:52] this, while also benefiting from all of [25:54] the infrastructure that we've built. And [25:56] I think there are some companies that [25:57] try to like build out all of the talent [26:00] network and infrastructure in-house, but [26:02] I think if you look at what the frontier [26:03] labs do and the best models do, it's a [26:05] pretty good indication that there's so [26:07] many economies of scale from working [26:09] with a partner that has all of these [26:11] economies of scale, the platform, the [26:13] talent network, etc. So, anyways, thanks [26:16] for all having me. [26:18] >> [applause]