Anthropic, OpenAI Pivot From Model Scale to Compute Cost Efficiency

Anthropic与OpenAI转向：从模型规模转向计算成本效率 企业部署限制正迫使人工智能行业放弃纯粹的规模竞赛，转向集成系统架构以及消除逐词元串行解码的轻量级模型。 专家小组探讨了在Claude Opus 5.5和GPT-6 Sol等近期前沿发布中，人工智能行业从纯粹的模型规模向推理成本、词元效率和集成系统架构的决定性转变。他们深入剖析了TypeSafe走红的Jev AI，拆解其快速的“系统1”分类器架构如何用适用于软件流水线的直接、校准数值决策取代逐词元解码循环。随后，对话转向IBM与NASA合作构建的月球基础模型，重点探讨共享表征如何帮助科学家通过陨石坑计数确定太阳系年龄并寻找地表冰。本期节目最后向即将离任的主持人Tim Hwang表达了由衷的敬意，这也是他最后一次作为该节目的首席主播亮相。 前沿人工智能发布已将焦点转向计算和词元效率，Claude Opus 5.5和GPT-6 Sol等发布在大幅减少词元使用量和运营开销的同时，达到了早期重型模型的性能水平。 (0:00) TypeSafe推出了Jev，这是一款专用的“系统1”模型，它绕过了串行文本生成，直接输出类型化数值和结构化概率，从而消除了软件分类任务中的KV缓存瓶颈。 (10:48) Jev采用针对校准决策的强化学习（RLCD），而非标准的RLHF或RLVR，将统计校准和原始得分精度置于逐步推理链之上。 (23:45) IBM与NASA合作推出了基于月球表面数据训练的开源视觉基础模型，用于绘制陨石坑地图、计算行星年龄并识别潜在的水冰沉积物。 (30:00)

Recent artificial intelligence development has shifted from raw parameter scaling to compute and token efficiency, system-level agent integration, and specialized decision models that bypass generative text altogether. Efficiency Pressures and the Shift to System Architectures (0:00) The latest wave of frontier models—including xAI's Grok 4.7, Anthropic's Claude Opus 5.5, and OpenAI's GPT-6 Sol and Luna—emphasizes preserving high capability while reducing token expenditure and compute overhead compared to predecessors like Astra and Fable 5.1. Kaoutar El Maghraoui notes that the industry's focus has evolved from monolithic benchmark chasing to system intelligence, where a foundation model acts as an internal reasoning component integrated with long-term memory, execution sandboxes, web browsing, and enterprise infrastructure. Gabe Goodhart observes that tone changes, such as Claude Opus 5.5 adopting a terse, pragmatic style, serve practical economic goals by cutting down token consumption. However, Goodhart cautions that declining token pricing for end users does not necessarily prove underlying inference compute breakthroughs. Because model providers keep active parameter counts and FLOP costs secret, lower fees may reflect venture-backed subsidies aimed at winning enterprise market share rather than true algorithmic efficiency. TypeSafe's Jev and Non-Generative System 1 Models (10:48) TypeSafe's Jev model, introduced by former OpenAI researcher Diogo Almeida, illustrates a departure from the standard generative paradigm by positioning itself as a fast 'System 1' classifier. Standard large language models rely on sequential decode loops—generating one token at a time, reading the key-value cache, and performing matrix operations in repeated cycles—which forces software engineers to parse unstructured text with brittle regex rules for simple enterprise tasks. Jev instead outputs typed values and structured probability vectors in parallel, which better matches GPU hardware strengths and cuts memory bandwidth bottlenecks. Martin Keen contrasts this approach with the industry's reliance on Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from Verifiable Rewards (RLVR). Jev uses Reinforcement Learning for Calibrated Decisions (RLCD) to optimize directly for score accuracy rather than explanatory text. In benchmark demonstrations, Jev matched frontier models while executing sub-second paths in the 'Wikipedia Game' and making tactical navigation choices in Doom, though it trades away the interpretability of an explicit reasoning chain. Deterministic Orchestration and the Limits of Model Confidence (16:00) The panel emphasizes that replacing generative pipelines with classifiers introduces calibration challenges. David Zax and Kaoutar El Maghraoui note that an autonomous model claiming high confidence can introduce serious failure modes if its probabilities are uncalibrated. Goodhart points out that neural outputs represent heuristic scores rather than rigorous mathematical probabilities. Nevertheless, enterprise pipelines rarely require absolute probabilistic precision; they need stable relative ordering to route workflows and triage actions. To manage non-deterministic behavior, IBM uses internal programming models like Malaya. Rather than prompting an open-ended language model to run an entire workflow via natural language, this approach maintains a rigid, deterministic code skeleton and delegates only targeted, non-deterministic decision points to compact classification models. Scientific Vision Foundation Models in Lunar Exploration (30:00) Shifting away from commercial chat interfaces, a joint initiative between IBM and NASA demonstrates the utility of domain-specific computer vision foundation models trained on lunar surface imagery. The models are designed to map surface topography, calculate terrain risks, and locate potential water ice deposits. David Zax notes that automated crater counting provides a method to measure surface impact history and determine planetary chronology. Kaoutar El Maghraoui highlights that the principal engineering achievement is sensor alignment and data reuse rather than expanding model scale. By converting multi-resolution, variable-lighting sensor feeds into unified vector representations, a single pre-trained base model can support multiple downstream scientific inquiries without requiring researchers to train bespoke models from scratch. While architectural shifts toward structured System 1 models and hardware-software co-design deliver measurable efficiency gains, enterprise adoption remains tied to unresolved questions around probability calibration, the lack of interpretable reasoning trails, and the unknown gap between commercial API pricing and actual underlying compute costs.

IBM播客团队指出，前沿人工智能竞争正从单纯追求模型参数规模与基准跑分，转向以计算成本收益为核心的系统智能；从降低Token消耗的高效前沿大模型，到TypeSafe开发的Jev等针对确定性软件决策的系统1分类模型，再到IBM与NASA合作的月球专用视觉模型，行业重心正加速向全栈效率与专业化落地倾斜。 前沿模型转向效率与系统智能 (0:00) 当前以xAI的Grok 4.7、Anthropic的Claude Opus 5.5以及OpenAI的GPT-6 Sol和Luna为代表的新一批前沿模型，已不再片面追求参数膨胀，而是聚焦于在保留前沿性能的同时降低Token消耗与算力成本。Martin Keen举例称，Claude Opus 5.5能够以显著低于早期旗舰模型Fable 5.1的Token开销处理视频编辑等复杂任务，而GPT-6 Sol同样相比前代大幅提升了算力效率。Kaoutar El Maghraoui进一步指出，业界的关注焦点已从孤立的模型评测转向集成化的系统智能。单一基础模型正在转变为代理系统的内部推理引擎，与持久化内存、网络浏览、代码执行环境及企业后端深度融合，这使得跨组件的协同优化和软硬件协同设计比追求微小跑分提升更具实际价值。 Token精简表象与真实算力成本 (5:07) 在模型交互风格方面，Claude Opus 5.5摒弃了早期被部分用户诟病的冗长与人格化表达，转向更为直接、简明的行文风格。Gabe Goodhart分析认为，这一转变既符合Anthropic侧重企业生产力的品牌定位，也是控制Token开销的直接手段，因为减少冗余的人格化用词能够从物理层面节省Token支出。然而，Goodhart发出警示，强调不应将面向消费者的Token节省与底层的实际推理算力效率混为一谈。各大模型实验室对其真实活跃参数量、私有架构设计以及推理时的FLOP成本依然保持高度保密，当前价格下降究竟源自真实的算法与硬件架构突破，还是依靠风险投资补贴以争夺市场份额，目前尚无定论。 Jev AI与系统1结构化决策重构 (10:48) 由前OpenAI研究员Diogo Almeida创立的TypeSafe推出了名为Jev的“系统1”模型，该模型迅速在开发者群体中引发关注。该模型专门面向软件工程场景，彻底打破了传统大语言模型按Token逐字顺序生成的解码机制，转为直接输出预定义决策的数值概率与类型化结果。El Maghraoui从硬件层面分析指出，传统生成式模型受制于逐字解码与KV缓存往返的内存带宽瓶颈，而Jev通过直接预测并行结构化结果，大幅契合了GPU与AI加速器的并行计算优势。该模型摒弃了早期依赖人类反馈的强化学习（RLHF）及针对推理链的可验证奖励强化学习（RLVR），转而采用针对校准决策的强化学习（RLCD）。在实际验证中，Jev在复古游戏《Doom》的实时移动射击以及“维基百科跳转游戏”等路径规划测试中，均在亚秒级时间内做出了与耗时数秒的链式思考大模型相当的最优决策。 企业工作流中校准度与确定性编排 (16:00) 对于Jev这类以分类和打分为核心的模型，企业落地中的关键风险在于概率校准而非单纯的置信度高低。David Zax提出，如果系统在90%的高置信度下实际正确率极低，将造成严重的过度自信灾难；因此模型输出的概率必须与现实统计真实性高度吻合。Goodhart指出，企业代理系统的下游服务往往不需要绝对纯粹的数学概率，而依赖可靠的相对排序来支撑业务路由（如红黄绿状态分流）。为解决传统LLM生成冗长非结构化文本带来的正则解析负担，IBM内部研发了名为Malaya的编程模型，该模型将整体流程构建为严格的确定性代码骨干，仅在关键的不确定性决策分叉点调用轻量级模型，从而实现高效率与高可靠性的结合。 IBM与NASA合作月球科学视觉模型 (30:00) 在垂直领域落地方面，IBM与NASA展开合作，发布了基于月球表面遥感影像训练的开源计算机视觉基础模型。该模型致力于通过分析撞击坑密度来测定天体表面的相对年龄，并协助勘测对未来深空探测至关重要的月球水冰沉积物。El Maghraoui强调，该项目的核心突破在于跨传感器分辨率与多角度光照条件下的严谨数据工程，使得一套预训练表示能够跨多项科学任务复用，而非依靠单纯扩大模型参数；Keen则指出，基于Transformer机制的特征向量化能够有效剔除低角度阳光阴影等视觉干扰，凸显了基础模型在传统科研领域的工程复用价值。 本期节目梳理了AI从单体规模堆叠向算力经济效益、系统级代理与垂直领域科学落地演进的行业趋势。然而，前沿大模型实际计算成本的真实透明度、轻量化分类器在自动化系统中的绝对校准可靠性，以及高昂算力补贴退坡后的商业可持续性，仍是全行业有待解答的核心问题；此外，本期录制亦标志着联合主持人Tim Hwang正式卸任该播客的主持工作。

## Background 本期内容由 IBM 于 2026 年 9 月 25 日在播客平台发布，为《Mixture of Experts》第 126 期。页面简介指出，本期节目由 Tim Hwang 与 David Zax 联合主持，邀请了 Kaoutar El Maghraoui、Gabe Goodhart 与 Martin Keen 三位专家，重点探讨了近期密集发布的大模型（如 Claude Opus 5.5 与 GPT-6 等）背后的效率趋势、TypeSafe 推出的 Jev AI 模型，以及 IBM 与 NASA 合作的月球探测项目。 ## TL;DR 本期节目聚焦 AI 产业正从追求单纯参数规模转向单位算力效率与“系统智能”的底层拐点，深入分析了 Claude Opus 5.5 和 GPT-6 Sol 等前沿模型在降本与 Token 经济学层面的考量及真实算力成本疑虑。嘉宾们详细解构了 TypeSafe 推出的 Jev“系统 1 模型”，探讨其如何抛弃低效的自回归文本生成、通过直接输出类型化分类向量来颠覆工程决策落地。随后，讨论转向 IBM 与 NASA 联合研发的月球计算机视觉基础模型，展现了通用多模态表征在数陨石坑与探测水冰等硬核基础科学场景中的复用价值。最后，全场向即将卸任主持职务的 Tim Hwang 致以温馨而风趣的告别致敬。 ## Atmosphere & style 整场对话在极客式的高密度技术推演与轻松幽默的演播室氛围之间自如切换，嘉宾们既有对矩阵计算、KV Cache 和校准决策等底层硬核细节的犀利洞察，又不时穿插着自嘲与行业调侃。讨论节奏明快且极富化学反应，既严肃审视了风投资本补贴与大模型工程落地的痛点，又洋溢着将 AI 重归硬核科学探索的极客兴奋感。节目尾声从严密的学术探讨温情过渡为对离任主持人的真挚致敬，在充满欢声笑语与惜别情怀的温润基调中圆满收官。 ## Bulletin - `00:00` 大模型行业竞争重心正在从“模型智能”转向“系统智能”，Claude Opus 5.5 与 GPT-6 Sol 等新模型均强调通过收缩参数与精简表达来大幅压低 Token 消耗与推理成本。 - `11:15` 前 OpenAI 研究员创办的 TypeSafe 发布首个“系统 1”模型 JEV，抛弃自回归文本生成与串行解码循环，直接并行输出结构化类型值与置信度向量以终结解析噩梦。 - `23:45` JEV 采用“校准决策强化学习”（RLCD）训练，专攻离散决策并在《Doom》与维基百科跳转游戏中展示出媲美大模型且达到亚秒级响应的性能。 - `28:40` IBM 与 NASA 联合发布月球探测专用多模态视觉模型，通过多源传感器数据对齐与单次基座预训练，高效支持撞击坑计数及水冰资源识别等深空科研任务。 ## Key takeaways - `00:00` 前沿模型进入密集发布的“黄金档期”，核心主线从盲目追求参数规模和高耗能转向效率工程与单位算力成本控制。 - `03:32` 行业评估大模型的标准正从基准测试跑分结构性转向长期任务稳定性、工具调用能力以及软硬件协同设计的系统级智能。 - `05:00` Claude Opus 5.5 收敛拟人化闲聊与个性化输出，旨在直奔主题以降低单次任务消耗的 Token 总量，契合企业级生产力需求。 - `07:23` API 调用价格暴跌掩盖了真实的推理成本，外界质疑低价究竟源于底层技术突破还是风投资本对云端算力的高额补贴。 - `10:00` 量化、稀疏化、KV Cache 管理与专用数据流加速器等软硬件联合优化，已成为决定 AI 产品在商业上能否实际落地的决定性因素。 - `11:15` 传统大模型逐字生成散文文本再交由软件做后处理解析，在工程上极其低效脆弱；JEV 转型为直接输出类型值的“专用模型”。 - `13:14` 从硬件计算机制来看，JEV 彻底消除了自回归串行解码循环的高频 KV Cache 读写瓶颈，充分发挥了 AI 加速器的并行吞吐特长。 - `16:08` 模型在工业落地中最需要的是严格校准的置信度，使自动化下游系统能够依据可靠的统计分布设定阈值并执行离散分支决策。 - `20:00` 针对确定性业务工作流，IBM Malaya 等架构主张使用代码框架固化流程，仅在关键决策分支点引入类似 JEV 的快速预测模型。 - `23:45` 大模型强行用“系统 2”思维链解决简单的分类或二选一问题存在巨大浪费，JEV 证明了直觉式“系统 1”决策模型具备高能效优势。 - `30:00` IBM 与 NASA 的合作表明基础模型绝非仅限于聊天机器人，其核心价值在于海量异构科学数据的表征对齐与多任务复用性。 - `35:00` 空间图像的多模态处理表明，只要物理信息被正确向量化，底层通用数学架构就能跨越文本生成用于天文物理学等前沿硬核科学。 ## Detailed walkthrough ### `00:00` 新一波前沿模型密集发布：核心关键词是“效率” 在《Mixture of Experts》播客第 126 期中，主持人 Tim Hwang 与联合主持人 David Zax 邀请了三位嘉宾：AI Native Systems 首席研究科学家 Kaoutar El Maghraoui、AI Foundations 首席架构师 Gabe Goodhart 以及 IBM Master Inventor Martin Keen。本期节目探讨了三大话题：大热的 TypeSafe Jev AI 模型、IBM 与 NASA 的前沿合作，以及近期席卷整个行业的大模型“扎堆发布潮”。最近几周仿佛进入了大模型发布的“黄金档期”（sweeps season），包括 Grok 4.7、Claude Opus 5.5 以及 OpenAI 的 GPT-6 Sol 与 Luna 接连亮相，且调用价格一路走低。 面对这波眼花缭乱的新模型，Tim Hwang 提出疑问：这些究竟只是小修小补的渐进式改进，还是在宏观上预示了更深层次的转变？Martin Keen 指出，本周所有发布的共同主题都是“效率”。此前类似 Astra 和 Fable 这样聪明绝顶的前沿大模型虽然性能强悍，但“吃 Token 就像发疯一样”（eat tokens like crazy），无论是按月订阅还是调用 API，背后消耗的算力都高得惊人。本周发布的新模型通过略微缩减参数规模，在几乎不损失智能水平的前提下大幅提升了工作效率。例如 Claude Opus 5.5 的能力已经足以匹敌庞大的 Fable 5.1，且 Token 消耗量大幅降低；OpenAI 推出的 GPT-6 Sol 也是基于前代 5.6 打造的超高效版本。 ```mermaid flowchart TD accTitle: AI 模型迭代重点从纯参数量向系统效率演进 accDescr: 展示了从高耗能、大参数模型向高效紧凑模型及系统级智能 Agent 的演化路线 subgraph Phase1["前期：重算力与绝对智能"] A["巨型前沿模型 (如 Astra / Fable 5.1)"] -->|"消耗巨量算力与 Token"| B["高推理成本 / 高 API 账单"] end subgraph Phase2["当前：效率优先与精简推理"] C["高效型模型 (如 Claude Opus 5.5 / GPT-6 Sol)"] -->|"参数量微调缩减 / 紧凑表达"| D["大幅降低 Token 成本与单次任务开销"] end subgraph Phase3["未来：系统智能 (System Intelligence)"] D --> E["模型作为核心组件"] E --> F["集成记忆、工具调用、浏览器与代码执行"] F --> G["软硬件协同设计与底层基础设施优化"] end ``` - 本期登场模型与对比：Grok 4.7、Claude Opus 5.5（能力对标先前的 Fable 5.1，推测后续还将有 Fable 5.5）、GPT-6 Sol 与 Luna（对比上一代 5.6 更高效）。 - 算力困境：模型研发机构和企业用户都在面临算力满负荷与成本激增的压力，促使研发重心从片面追求参数规模转向效率工程。 --- ### `03:32` 从“模型智能”到“系统智能”的根本转向 针对这波由算力瓶颈倒逼的效率革命，Kaoutar El Maghraoui 从底层硬件与系统基础设施的视角进行了剖析。过去每发布一款前沿模型都是轰动业界的重大事件，而现在每隔几周就有新模型问世，模型名称本身的重要性正在被稀释。对于企业和开发者而言，评估标准已经变成了实际落地效益：模型能否稳定执行更长时间的任务？能否可靠地调用工具？能否帮助削减基础设施支出？是否采用了新的算法优化或更好的软硬件协同设计（hardware-software co-design）技术？ Kaoutar 强调，整个行业的竞争重心正在经历结构性迁移：“现在的竞争正从模型智能转向系统智能（The competition is moving from model intelligence to system intelligence）。” 在这一新范式下，基础模型不再是孤立的庞然大物，而是退居为智能体（Agent）内部的一个核心组件，与记忆模块、工具接口、网络浏览器、代码执行环境以及企业内部系统紧密协同。相比在公开基准测试（benchmarks）上多刷几个百分点，如何对这套复杂的完整系统进行全局联合优化，才是眼下最关键的胜负手。 - 评估模型价值的新核心维度：长期任务稳定性、工具调用可靠度、推理与部署成本降幅、软硬件协同设计深度。 - 智能体系统架构组件：底座大模型、外部记忆模块、Tooling 扩展、内置浏览器、安全代码执行沙盒、企业数据与系统接入端。 --- ### `05:00` Claude Opus 5.5 的克制克己：人格化与 Token 经济学 随着各类桌面客户端的模型选择菜单变得愈发眼花缭乱（动辄堆砌十多个新旧版本，还要用户手动选择“思考努力程度/Effort”），用户体验变得日益复杂。Tim Hwang 将视线引向 Claude Opus 5.5 的具体表现。他观察到，此前不少人抱怨 Claude 的“爹味”或“自身特质”（Claude-ish）过于浓厚，模型个性太强；而在 5.5 版本中，Anthropic 明显对其进行了收敛约束，输出风格变得干练直接、直奔主题。这引发了关于模型到底该保留多少拟人化个性的争论。 Gabe Goodhart 对此回应，这种转变兼具品牌定位与经济效用的双重驱动。一方面，Claude 始终定位于“专注搞定工作”（get work done）的企业级品牌，而 ChatGPT 更多偏向通过聊天界面建立人际互动感，因此 Anthropic 收紧风格符合其商务定位。更根本的原因在于效率：如果一个模型不需要用废话和客套来模仿人类情感，就能用更少的词句完成同样的任务，消耗的 Token 就会显著减少。对此，Gabe 还不忘幽默地调侃 Martin：“你身后的那些机器绝对没有在本地运行模型，它们只是在运行一个与云端模型通信的智能体（Agent）——你知道什么更省钱吗？是真正能跑在你身后机器上的模型！” - 风格转变与品牌定位：Claude 强化严肃、精炼的办公伙伴形象，弱化闲聊社交属性；OpenAI 则更强调交互感。 - Token 压缩与信息密度：降低不必要的“口癖”和客套话，直接在输出端降低消耗的 Token 数量，兼顾用户体验与推理账单。 --- ### `07:23` 降价潮背后的悬念：算力成本真降了，还是资本在贴钱补贴？ 在深入分析“Token 效率”时，Gabe Goodhart 提出了一个关键的技术与商业拷问：对外宣传的“高 Token 效率”（用户完成一个目标所需的 Token 数量减少）并不等同于底层的“高算力效率”（处理单个 Token 所需的物理算力减少）。目前各厂商公布的数据中，真正缺失的是模型在推理阶段消耗的具体参数规模与算力开销。对于消费端而言，更低的标价和更少的 Token 需求确实带来了实打实的好处，但外界并不清楚厂商运行每个 Token 的真实内部成本倍率。 Gabe 犀利地指出，行业目前尚未厘清当前的低价究竟源于真正的技术突破，还是依靠风投资本支持的市场争夺策略：“长期以来大家一直担心风险投资在为这些公司提供补贴，一旦它们不再是私营企业并走向公开市场，算力成本的清算终将来临（Once they are no longer privately held firms and they're out there in the public, that that is going to have a cost reckoning）。” 各厂商可能只是在利用外部资金争取时间，通过低价抢占市场份额，等待未来的技术追赶成本。Tim 对此补充道，这种“以最少 Token 完成任务”的导向，酷似当年 Google 强调的“减少用户在搜索结果页停留时间以证明搜索精准度”。Kaoutar 总结称，模型架构、推理加速软件与专用 AI 加速芯片设计正以前所未有的速度在“效率”这一原点上加速收敛，下一阶段的软硬件协同演进将充满变数。 | 观察维度 | 表面现象（消费端 / API 层） | 潜在现实（基础设施 / 商业层） | |:--- |:--- |:--- | | **Token 使用量** | 单次任务消耗的 Prompt 与 Output Token 减少 | 紧凑表达减少了推理吞吐压力，但真实计算矩阵仍未披露 | | **定价与收费** | 调用单价暴跌，呈现“向底线赛跑（race to the bottom）” | 可能存在风投资金高额补贴，真实推理硬件成本仍高企 | | **技术指标** | 盲目刷榜跑分降温，注重工具集成与端到端任务完成 | 模型架构、推理框架引擎、底层加速芯片协同向功耗比收敛 | ### `10:00` 效率之战与硬件加速：AI 从“追求规模”转向“单位算力成本” Kaoutar El Maghraoui 深入剖析了当前大模型部署的底层技术演进。她指出，诸如量化（quantization）、稀疏化（sparsity）、KV Cache 管理、推测解码（speculative decoding）、更高效的批处理（batching），以及专用数据流加速器（dataflow accelerators）等硬件优化手段，早已不再是模型训练完成后的附加修补工作。Kaoutar 强调：“These things are not just optimizations after the model is built. They increasingly also determine whether the product is economically deployable at all.”（这些技术已不再仅仅是模型构建后的优化，它们越来越决定了一个产品在经济上是否具备落地的可行性。） 回顾近年的发展主线，Kaoutar 指出： - **2023 至 2024 年**：行业焦点主要集中在追求参数规模，模型越做越大； - **2025 年**：核心竞争点转移到了增强推理能力（reasoning）； - **2026 年（当前）**：竞争则全面聚焦在“单位算力成本下的推理能力”（better reasoning per unit of compute cost）。 尽管全行业对“效率至上”达成了共识，但在硬件加速器设计领域，如何真正达成预期的能效指标仍存在巨大挑战。对谈中嘉宾们打趣道，虽然现在订阅顶级模型的费用高得令人咂舌（eye-watering），但用户实际上依然享受着科技巨头的巨额补贴。 --- ### `11:15` TypeSafe 发布 JEV：颠覆文本生成的“系统 1”架构 对话随即切入近期在业内引发巨大轰动的新发布——由 TypeSafe 公司推出的 JEV 模型。TypeSafe 宣称他们构建了一种全新的模型品类，即所谓的“系统 1（System 1）模型”，而 JEV 则是该系列首个面向代码领域的公开发布版本。主持人 Tim Hwang 补充了关键的背景信息：TypeSafe 的联合创始人是前 OpenAI 研究员 Diogo Almeida，他曾是早期参与 ChatGPT 项目的核心成员，也是里程碑式论文 InstructGPT 的共同作者之一。尽管 Almeida 长期以来行事极其低调，在 X 平台（原 Twitter）上仅有约 1500 名关注者，但他发布 JEV 的单条推文瞬间斩获了超过 4000 万次浏览，彻底引爆了 AI 圈。 ```mermaid flowchart TD accTitle: 传统 LLM 与 TypeSafe JEV 模型的架构与硬件工作负载对比 accDescr: 对比传统大语言模型通过自回归循环生成自由文本与 JEV 模型直接并行输出结构化类型值的工作流与硬件特征。 subgraph Traditional["传统大语言模型 (LLM)"] A["输入任务 / Prompt"] --> B["逐 Token 串行自回归解码"] B --> C["高频读写 KV Cache & 往返加速器"] C --> D["生成非结构化散文 / 自由文本"] D --> E["后处理解析为程序决策 (容易出错)"] end subgraph JEV["TypeSafe JEV (System 1 模型)"] F["输入结构化决策任务"] --> G["消除自回归循环 / 并行计算分支概率"] G --> H["极低 KV Cache 流量 / 硬件友好"] H --> I["直接输出类型值 (Typed Values) 与决策得分"] end ``` David Zax 探讨了 JEV 背后关于“专用模型（fit-for-purpose model）”的颠覆性主张： - **传统 LLM 的错配**：当今的大型语言模型本质上像“小说家”，擅长为人类的可读性逐字逐句生成流畅的散文文本。然而在企业级实际应用中，工程师们却不得不削足适履，将这些长篇大论后处理并强制约束为软件系统底层的简单离散选择（如多选分支、布尔值判断等）。 - **JEV 的解决方案**：完全放弃自由文本生成，在软件层面直接输出“类型值（typed values）”，大幅削减冗余环节，从源头上重塑执行效率。 --- ### `13:14` 硬件视角的底层变革：消除串行解码循环 针对 JEV 的宣传噱头，Kaoutar 从硬件架构专家的视角给出了深入拆解。她认为，在 Agent（智能体）工作流等企业场景中，系统往往只需要一个明确的分数、决策或“是/否”结果，根本不需要模型耗费时间写出整段话再去让下游代码进行文本解析。 从硬件计算的物理本质来看，JEV 的核心优势在于它彻底重构了工作负载： 1. **打破串行解码瓶颈**：普通语言模型一旦启动生成，必须依赖串行解码循环（sequential decode loop）——生成一个 token 后，回写状态，再触发下一次矩阵运算读写 KV Cache，循环往复。 2. **充分发挥并行硬件特长**：JEV 放弃了生成式长文本，转而在底层直接并行评估不同决策分支的概率分布。GPU 与 AI 加速器天生极其擅长处理高度并行的计算任务。 3. **性能提升的本质**：Kaoutar 犀利地指出，JEV 的效率飞跃“并不是他们发现了某种让矩阵乘法变快的神奇魔法，而是他们改变了向硬件索取的工作形式（they changed what they're asking the hardware to do）”。 尽管如此，Kaoutar 也明确表达了自己的保留意见。她警告称，JEV 依然依赖模型来生成决策概率与评分，其声称的“校准度（calibration）”和预测正确性仍需保持警惕，在自动化系统落地中，决策的真实可靠度甚至远比其宣称的超低延迟更为关键。 --- ### `16:08` 概率 vs 置信度：模型校准在企业决策中的关键作用 David Zax 随后以非技术视角切入，探讨了模型在工业界落地时最核心的技术痛点之一——**校准（Calibration）**。他用生动的例子打比方：模型不仅有对错之分，更有“盲目自信（overconfident）”与“过度自卑（underconfident）”之别。如果自动驾驶系统把前方物体识别为停步标志（stop sign）并给出了 90% 的置信度，但实际抽检发现它在这些场景下的正确率只有 10%，这就是致命的“过度自信”。多年来，行业深受大模型“以无比自信的口吻胡说八道（spout utter nonsense）”的幻觉之苦。JEV 声称消除幻觉的核心，正是其将优化目标聚焦于极度精准的模型校准：模型可能不一定是识别停步标志能力最强的，但如果它标注自身有 60% 的把握，它在统计学意义上的准确率就必须严丝合缝地落在 60%。 作为老牌软件工程师，Gabe Goodhart 对此产生了极其强烈的共鸣。他回忆起在前生成式 AI 时代（pre-generative AI days），工程团队在构建分类器并为其编写 REST API 时，曾对返回字段命名产生过经典争论： - **概率（Probability）**：在数学上有严格的分布定义与统计约束，不透明的黑盒近似算法往往无法给出严谨的数学概率。 - **置信度（Confidence）**：本质上是工程妥协产物，往往只是“在模型输出上套一个 Sigmoid 函数”，使其在输入分布上看起来呈现平滑的钟形曲线。 Gabe 总结道，对于 JEV 乃至下游的企业决策系统而言，采用黑盒运算去强求严格的概率学定义确实面临底层数学挑战；但从实际落地的决策角度来看，置信度已经足够胜任。下游消费系统只需根据 JEV 给出的相对置信度分布设定阈值，即可干脆利落地做出自动化分支决策：“下游系统只需要拿到这个置信度分布并决定：向左还是向右？红灯、黄灯还是绿灯？” ### `20:00` 分类模型的演进、LLM 的工程噩梦与 Jev AI 的定位 针对校准（calibration）与置信度得分，嘉宾们指出用户往往必须凭经验来选取阈值。尽管很难在严格数学意义上证明模型输出的是绝对概率分布，但至少能够保证置信度得分在彼此相对关系上是可靠的。Gabe Goodhart 随后抢过话头，试图将 Jev 放置在整个 AI 模型发展谱系中进行横向审视。他指出，Jev 本质上是一个分布预测器（distribution predictor），也就是经典分类模型。分类是自然语言处理乃至整个 AI 领域最古老的基础问题：输入一个特征向量，然后尝试根据一组可选标签对其进行分类。 过去大家之所以放弃自己训练分类器——哪怕这种模型在尺寸上能比大语言模型小好几个数量级，在运行速度和算力需求上也高效得多——是因为在过去监督学习的时代，必须投入巨大精力去清洗标注数据并训练模型。随着大模型的出现，大家开始依赖少样本提示（few-shot prompting），仅凭几个示例去微调激活相关权重区域，让模型给出分类。然而正如 Kaoutar El Maghraoui 此前指出的，让大语言模型逐字逐句生成类似“我认为这是一个停车标志，理由是……”这样的文本输出，完全是一场软件工程的灾难。工程师不得不去编写复杂的解析代码，处理大小写敏感度、多余的空格以及一堆不知所谓的废话。相比之下，Jev 这类模型直接输出一个数字向量（如位置一是停车标志、位置二是让行标志），无论是在推理算力消耗上还是在提取结果的工程代价上，都具有压倒性优势。 ```mermaid flowchart TD accTitle: LLM 生成与 Jev 向量输出的工程对比 accDescr: 对比传统大语言模型生成冗长文本解析与 Jev 直接输出分类向量的效率差异 subgraph LLMApproach["传统大语言模型方式"] A["输入特征/Prompt"] --> B["逐 Token 自回归生成"] B --> C["输出长文本: '我认为这是停车标志，因为...'"] C --> D["工程清洗与解析 (正则/大小写/空格处理)"] end subgraph JevApproach["Jev 向量预测方式"] E["输入特征"] --> F["分布预测 / 确定性判断"] F --> G["直接输出概率/置信度向量: [0.98, 0.01, 0.01]"] end ``` 针对这种在工作流中做出精准决策的需求，Gabe 提到了 IBM 内部研发的一套名为 Malaya 的编程模型（programming model，而非 AI 模型）。许多现有的 AI 工作方式是在纯文本中描述一套流程，然后赌模型能老老实实按步骤执行；而 Malaya 则反其道而行之，让开发者构建出具备良好定义的确定性工作流框架，仅在关键决策分叉点引入像 Jev 这样的模型处理非确定性输入。Gabe 强调了这一结合的威力： > “So that's the real power of a model like this is being able to take something that you know is a well-defined path through the actions and make good decisions with non-deterministic inputs at the important junctures of that thing so that you can walk the right path.” —— Gabe Goodhart ### `23:45` 系统 1 与系统 2 之争：校准决策强化学习与实测基准 Martin Keen 接着对 Jev 的设计理念和训练机制展开剖析。Jev 自称为“系统 1 模型”（System 1 model），这直接借鉴了丹尼尔·卡尼曼（Daniel Kahneman）在《思考，快与慢》（Thinking, Fast and Slow）中提出的理论——系统 1 代表无意识的、直觉式的快速自动反应，而系统 2 代表深思熟虑、按部就班的理性推理。现场还开玩笑称 Martin 掏出实体书是在推销他的带货链接分成。Martin 指出，当前很多大模型都在追求思维链（Chain of Thought）式的推理，强行用昂贵的“系统 2”去解决本该属于“系统 1”的简单分类问题：如果答案只是二选一的“是/否”或十五个选项里选一个，调用庞大的 LLM 是极度浪费的。 在训练机制上，Jev 也彻底脱离了传统框架。通常大语言模型早期依赖人类反馈强化学习（RLHF）来迎合人类偏好，后来面向智能体（Agent）引入可验证奖励强化学习（RLVR，即完成特定操作后给予奖励）。而 Jev 采用的是**校准决策强化学习**（Reinforcement Learning for Calibrated Decisions，RLCD）。它不讨好人类，也不以达成特定外部任务目标为转移，只专注于输出的判断在统计意义上是否准确、概率校准是否真实。针对 David 提出的“无推理链如何验真”的问题，Martin 承认 Jev 不会给出任何解释性理由，它只返回一个数字；但团队提供的基准测试表明，面对 Astra 或 Fable 等前沿大模型，Jev 能在极短的时间和极低的 Token 成本下得出完全相同的结论。 - **Doom（毁灭战士）演示**：Jev 被部署在经典游戏《Doom》中，实时执行极速的二元与多元判断（向左走、向右走、是否开火），在三维空间中灵敏巡航并精准猎杀敌人。 - **维基百科跳转游戏（Wikipedia Game）**：挑战者需要从一个冷门词条（如“英国中世纪”）通过页面超链接跳转到毫不相干的目标（如“飞行汽车”，类似六度分隔理论）。顶尖 LLM 通常需要花费数秒的思维链计算在 5 次点击内完成，而 Jev 同样只用 5 次点击，但耗时处于亚秒级（sub-second）。 Martin 总结说，起初大家可能觉得这不过是旧时代分类模型的换壳，但了解其底层的校准决策强化学习以及比肩前沿大模型的实测表现后，会发现这确实是一项极具潜力、值得长期关注的技术。 > “We've been using system two when we really only needed system one for things that are just a classification problem, as Gabe was mentioning.” —— Martin Keen ### `28:40` IBM 与 NASA 合作：登月探索专用视觉模型 Tim Hwang 对 Jev AI 的深度讨论进行了收尾，称赞这期解析成功穿透了业界的炒作滤镜，看清了背后的工程与学术本质。随后，他将讨论引向当天节目的最后一个重磅话题——IBM 与美国国家航空航天局（NASA）的全新合作成果。 这项合作发布了一套专为月球探测打造的专用模型组合。这些模型主要是高精度的计算机视觉模型（Computer Vision models），专门针对月球表面的各类卫星及探测器图像进行优化，能够协助科研人员极速识别、分割和分析月表的地貌特征（如撞击坑、岩石结构、潜在着陆点及地质资源等），为重返月球的深空科研任务提供底层 AI 支持。 ### `30:00` NASA 与 IBM 合作月球模型：跳出聊天机器人，回归极客的硬核科学 谈及 NASA 与 IBM 合作的月球模型时，David Zax 开玩笑地表示，作为一名阿波罗登月狂热粉丝，他最初读到这项研究时差点“无聊到哭出来”，因为表面上看它只是在帮人“更快地数陨石坑”。他原本纳闷数陨石坑到底有什么意义、人类怎么会干这种世上最悲惨的工作；但深入了解后才发现，计算月球特定区域的陨石坑数量其实是推断太阳系演化与年龄的关键指标——陨石坑如同宇宙的时钟，记录了外来碎片撞击月球的频率。此外，该模型还能协同解决水冰识别这一高维多参数难题，对未来月球探索与基地建设意义重大。 Gabe Goodhart 对此极为兴奋，盛赞这让 AI 重回“极客专属（AI for nerds）”的黄金时代。他指出，当前 AI 正在从纯粹引人入胜的数学构想演化成沉重的社会技术系统（socio-technical problem），过去习惯在终端里折腾数学与代码的极客们，如今不得不面对复杂而令人不安的人类现实问题；因此看到 AI 被重新投入到纯粹的基础科学中令人精神一振。他强调，基础模型绝不仅限于聊天机器人或代码助手，这种核心技术能够极大地拓宽人类的认知边界，其深远价值远胜于优化几个 Token 来将博文转成连环画。 Kaoutar El Maghraoui 补充指出，该项目的精髓在于“复用性（reuse）”与“数据工程（data engineering）”： - **单次预训练与多下游任务适配**：针对海量月球探测数据仅进行一次基座预训练，便能将学到的通用表征迁移适配到多种不同科学问题上，彻底摆脱了每做一个科学实验就要从零造轮子的困境。 - **异构传感器对齐**：数据工程在本项目中比盲目堆砌模型参数量更重要。月球探测传感器涵盖了截然不同的分辨率、成像角度和观测环境；将这些多源观测数据对齐，并把数据采集时的物理元信息输入模型，是突破科学计算瓶颈的核心。 ```mermaid flowchart LR accTitle: 月球基础模型数据处理与多任务应用流程 accDescr: 展示月球多源传感器数据经过数据工程对齐与基础模型预训练后，复用于多项月球科学任务的流程。 subgraph Input ["多源传感器数据"] direction TB A["不同分辨率遥感图像"] B["不同光照/太阳入射角"] C["物理观测元数据"] end subgraph Core ["基础模型工程"] direction TB D["数据工程与向量对齐"] E["月球多模态基础模型"] end subgraph Tasks ["科学探索下游任务"] direction TB F["撞击坑自动计数与年龄推断"] G["月球水冰与资源分布识别"] H["未来月球殖民与基地选址"] end Input --> D D --> E E --> Tasks ``` - Gabe Goodhart 总结道：“AI for nerds again, which is awesome. Nerds and scientists and geeks and people that like to think about esoteric problems that most people would look at and say, why the hell are we counting craters on the moon?” - Gabe 进一步反思当前 AI 产业：“We live in an age where AI is starting to become not just a fascinating technology, but a hugely important socio-technical problem.” - Kaoutar El Maghraoui 强调架构复用的重要性：“What I really like a lot about this project is that really it shows that a foundation model doesn't have to be a chatbot. And the value here is reuse.” --- ### `35:00` 多模态泛化本质与主持人 Tim Hwang 的告别致敬 Martin Keen 从模型底层机制切入剖析，指出大家通常习惯将基础模型默认为“输入文本、输出文本”的 LLM，但只要非文本信息能够被向量化并表达为一系列数值，就能由同一种数学逻辑驱动。空间拍摄的图像进入多模态基础模型后，模型能够理解并补偿因太阳入射角变化而在陨石坑上投下的阴影与遮挡。“底层运行的是相同的数学原理与推理逻辑，就像文本大模型用来写诗一样，”Martin 总结道，这生动诠释了通用技术架构跨界应用于硬核物理科学的巨大魅力。 随后，主持人 Tim Hwang 正式宣布本期是他主持《Mixture of Experts》的最后一期节目。接棒主持的 David Zax 开玩笑表示，不管这段会不会被剪掉，他都要跨越“审查”为 Tim 送上隆重的告别致辞。David 拿自己两岁的女儿打趣，形容女儿总喜欢偷穿父母巨大的鞋子在公寓里东倒西歪，而自己接替 Tim 的岗位就像“穿上了一双巨鞋（stepping into some very large shoes）”，只希望自己别摔得太惨。 演播室随后洋溢着温馨而幽默的赞誉氛围： - **举重若轻的转场功底**：Gabe Goodhart 表示自己每次录完都要回听，因为 Tim 在各话题转场时总能抛出一针见血的引子（nugget of leading insight），自然而然地搭建好整场节目的对话流。 - **让嘉宾免于尴尬的提问智慧**：Martin Keen 感慨节目从不预设台本，普通主持人很容易在抛问题时把嘉宾逼到墙角，但 Tim 的提问方式总是让嘉宾瞬间顿悟、行云流水地切入核心。 - **两年来保持敏锐与临场感**：Kaoutar El Maghraoui 感谢 Tim 两年来始终让每位嘉宾保持全神贯注（keeping us on our toes），把节目的专业度与听众黏性做到了极致。 - **意想不到的天使投资邮件**：Gabe 幽默爆料，前几天他居然收到了一封冷邮件（cold email），对方听了他在过去一年节目里的发言，以为他是圈内大佬并邀请他做天使投资人；Gabe 调侃自己当场回复“我钱包里只有一张 5 美元钞票，你们要吗？”。 - 最终，大家一致赞扬 Tim 为节目带来了绝佳的格调（panache），使之成为 IBM 在面向大众的传播中最亮眼的一道风景线。 - Martin Keen 评价模型底层机制：“It's able to do clever things like look at a particular image that was taken where the sun is at a particular angle that's kind of obscuring a little bit how the craters are visible and to be able to adjust to that, all just using the same sort of math in these models.” - David Zax 对 Tim 致敬：“I think, Tim, you've done a splendid job making technical topics really approachable... my two-year-old daughter is obsessed with wearing her parents' shoes and stumbles around in my apartment, and I'm stepping into some very large shoes here. May I fall on my face less often than my two-year-old daughter.” - Gabe Goodhart 盛赞 Tim 的个人魅力：“You've brought a level of panache to this show that really has made it stand out as a rare light of consumer-facing value out of IBM.” ## Notable quotes > `03:32` The competition is moving from model intelligence to system intelligence — Kaoutar El Maghraoui > `30:00` We live in an age where AI is starting to become not just a fascinating technology, but a hugely important socio-technical problem. — Gabe Goodhart > `23:45` We've been using system two when we really only needed system one for things that are just a classification problem, as Gabe was mentioning. — Martin Keen ## Mentioned Tim Hwang, David Zax, Kaoutar El Maghraoui, Gabe Goodhart, Martin Keen, Diogo Almeida, Daniel Kahneman, IBM, NASA, Anthropic, OpenAI, TypeSafe, Google, Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna, Grok 4.7, Astra, Fable 5.1, ChatGPT, InstructGPT, JEV, Malaya, Doom, Thinking, Fast and Slow ## Video description Visit Mixture of Experts podcast page to get more AI content → https://www.ibm.com/think/podcasts/mixture-of-experts It's been a wild week in AI, and on episode 126 of Mixture of Experts, host Tim Hwang and co-host David Zax talk to panelists Kaoutar El Maghraoui, Gabe Goodhart and Martin Keen to discuss what all these releases have in common: efficiency. We start with the model release pile-up. Anthropic shipped Claude Opus 5.5, and OpenAI launched GPT-6 Sol and Luna. Opus 5.5 is 20%... [Source](https://cdnapisec.kaltura.com/p/1773841/sp/177384100/playManifest/entryId/1_qdiqjb79/format/download/protocol/https/flavorParamIds/0/name/media.mp3)

[3:32] Kaoutar El Maghraoui: The competition is moving from model intelligence to system intelligence [30:00] Gabe Goodhart: We live in an age where AI is starting to become not just a fascinating technology, but a hugely important socio-technical problem. [23:45] Martin Keen: We've been using system two when we really only needed system one for things that are just a classification problem, as Gabe was mentioning.

[3:40] Kaoutar El Maghraoui: The model is becoming one component inside an agent that has memory, tools, a browser, code execution, access to enterprise systems. [23:45] Martin Keen: We've been using System 2 when we really only needed System 1 for things that are just a classification problem

[0:00] This is becoming a big efficiency problem. And now because we're reaching the point where the model names matter less, a new frontier model used to feel like a major event. Now we're getting them every few weeks. What I really care about is what's changed. All that and more on this week's mixture of experts. I'm Tim Huang, and welcome to Mixture of Experts. Each week, MOE brings together a panel of the sharpest and most good-looking minds working at the frontiers of artificial intelligence to lead you through the week's news. On this week's episode, we've got Kauter L. McRowy, Principal Research Scientist, AI Native Systems, Gabe Goodhart, Chief Architect, AI Foundations, and Martin Keen, Master Inventor. And as always, I'm joined by my co-host, David Zacks, three big stories today. We're going to talk about the widely hyped JEV model. We're going to talk about a really interesting collaboration between IBM and NASA. But first, I really want to start by talking about just the welter of model releases in the last few weeks. Once again, we are in like sweeps season for model releases. Grok 4.7 is out. Claude Opus 5.5 is out. GPT-6 Sol Luna is out. Everybody is releasing models and the prices are getting lower and lower and lower. And I guess maybe, Martin, I'll start with you. I mean, the question I have always, this is the perennial question with these model launches at this point is, does this matter? What's different here? Are these just incrementally better models? you know, in the meta across all of these model releases? Maybe that's the more interesting question. Yeah, the model releases this week have all been around a theme of efficiency. So we've seen the big model releases of like Astra and Fable. And these are, you know, frontier models, super, super clever models, the best really we have. But they also eat tokens like crazy. And anybody with a subscription to one of these, or if you're paying API costs, It just takes so much compute for these to run. So the models that came out this week were all addressing that efficiency in that they are a bit smaller, I believe in parameter count, but they can get a lot more done with hopefully not too much less intelligence. And in fact, in the case of the Claude model, Claude 5.5 for Opus. That has come out and is maybe as good as the model, the big model of Fable 5.1, which goes to show there's probably a Fable 5.5 coming down the road at some point. But it can do basically everything that those big models could do at a much, much reduced token cost. And the machines behind me right now are running that model, doing a bunch of video editing tasks, they are. And the same thing can be said with the new release of Sol as well for ChutGPT 6.0. That is basically a much more efficient version of the previous model, which was 5.6. So what we're really seeing today, this week, is just much more efficient models that are really as good as the biggest models that these labs have. Yeah, for sure. And I guess, Kato, the question I had for you, because that was what I picked up as well as kind of like everybody's about efficiency now. who works a lot in hardware and you think about the hardware side of this a lot is it sure seems to me that, you know, in some ways it's like these companies have to do this because otherwise they're also just maxing out their compute as well. And so I'm kind of curious about like how you read. this sort of efficiency push weighed against the massive cycles that the companies themselves are burning on their own compute. Yeah, definitely. This is becoming a big efficiency problem. And now, because we're reaching the point where the model names matter less, a new frontier model used to feel like a major event. Now we're getting them every few weeks. What I really care about is what's changed. Can it work longer? Can it use tools reliably? Can it help me reduce my costs? Is it cheaper? Is it efficient? What's the impact on the infrastructure or what do I need? Can I reduce basically costs? Does it have maybe new algorithms or better hardware software co-design techniques and things that will help it basically drive that efficiency? And now the competition is moving from mobile intelligence to system intelligence. the model is becoming one component inside an agent that has memory, tools, a browser, code execution, access to enterprise systems. So that's a much bigger shift than just another few points on a benchmark. So I think the system as a whole and how do we optimize all of these. [5:00] things together is becoming really important. Yeah, for sure. I feel like the little model selector in all of the desktop versions of these apps is becoming increasingly confusing. It's like, here are the 10 models you could use, some legacy, and you can also choose effort, which is definitely becoming very complicated. Gabe, if you're game, I mean, I think, just to kind of exercise some editorial here, I thought 5.5 Opus was maybe the most interesting you want to take a walk with me, I would love to talk a little bit about Opus 5.5. And, you know, the first thing I want to talk a little bit about is like people were kind of actually getting annoyed at Claude in my world, where it's like, oh, man, there's like almost like too much Claude-ish, right? There's like too much personality on these models. And in comparison, a lot of people have been commenting like for 5.5, basically Anthropics seems to have buttoned up the model a little bit. It's a lot more terse. It's a lot more to the point. And that's kind of interesting because I think there's always been a debate over how much personality people out of these models. And I guess, Gabe, I'm curious about how you take that. Is it just that they over-indexed or does it turn out that people really don't want a high-personality model? I think... A little bit of both. I do think... There's a bit of a sort of corporate branding delta here. So I think Claude very much attempts to be the get work done corporate branding. And I think OpenAI, especially powered through the chat interface or chat GPT, aims to be a bit more interpersonal. So I think it is in keeping with corporate branding for Claude to try to button it up a little bit and keep it straight and to the point. The other important thing to the earlier topic is efficiency, right? If you use fewer words to sound human, you can still get the job done and you can't cost fewer tokens. So it's actually probably also in service of that. And that was the big claim that I saw about all of these models was that they were more token efficient, which is an interesting difference, not necessarily correlated with compute efficient. So I do want to quibble a little bit with Martin here. Those machines behind you are absolutely not running the model. They are running an agent that is talking to the model. So you know what's cheaper a model that actually runs on the machines back behind you. So that's an interesting thing that is not disclosed about all of these new models is – actually how big they are and how much compute is required at inference time. So this is the biggest question that I had in this wave of models is they're clearly aiming for consumer efficiency by keeping the price low and the number of tokens that a given consumer needs to accomplish a task as low as possible. And that is genuinely goodness because that also correlates with the overall token efficiency on the inference side because they don't have to run as many tokens to serve the users that they need. But what we don't know is what is the multiplier there? How much does each one of those tokens actually cost them to run? And we have some intuitions about how much it costs to train these models. And I think there's rumors floating around about how big they are and what their architectures are. But a lot of those details are still hidden. So the part that I am still really curious about, and I imagine we'll get more details in the coming days and weeks, is how much this race to the bottom in terms of token cost is correlated with a race to the bottom in terms of the actual compute costs to run these things, or whether we're still waiting for that final shoe to drop of the actual difference between what it costs the companies to run the models versus what they're charging us. I think there's been a longstanding worry that backing these firms. And once they are no longer privately held firms and they're out there in the public, that that is going to have a cost reckoning. And so I don't think we have an answer to that with this round. I think what we have an answer to is that from a consumer standpoint, there is clearly downward pressure on the pricing. So I don't know whether the technology they have is actually backing that up or whether they're just leaning on the extra time by pushing out their and get more market capture. So we're all going to try to get that low cost market on our models. So we'll see what happens with that. But I'm really curious what happens in that gap between the two. Yeah, the efficiency of the token, I think, is really it almost reminds me of back in the day, you know, Google always used to say, well, we actually want minimal time on site because it means that you found what you wanted and you left. And I guess I'm kind of curious if those kind of same KPIs will start to play out in the AI space where it's like, look, the minimal number of tokens actually is best because Yeah, I think also from a hardware perspective, this is, I think, why I think the next phase is going to be very interesting. Things like model architecture, inference software, and accelerator design are also converging around efficiency. [10:00] to see things like quantization, sparties, the KV cache management, the speculative decoding, the better batching, and also the data flow accelerators, the specialized accelerators, they're no longer just, you know, these things are not just optimizations after the model is built. They increasingly also determine whether the product is economically deployable at all. And if we see the trends that we've been seeing, like between 2023 and 2024, it was mostly about bigger and bigger models. 2025, I thought the better reasoning was kind of the headlights there. And this year, it's more focused on better reasoning per unit of compute cost. So I echo what everybody's saying. Efficiency is becoming very important. And there is some conversions in the market. hardware accelerator design space, there's still a lot of things that we still need to work out in terms of really reaching that. really efficiency targets. It's still very challenging. Yeah, absolutely. Yeah, I feel like I'm paying a eye-watering amount for the top tier, but I'm still getting a good deal out of the company. I'm still being subsidized. I'm going to move us on to our next topic. We're going to stop talking about the latest generation of model releases to talk about a model release. The next topic I really wanted to cover was JEV, which has been getting a lot of press and I think a lot of hype recently. The background is a company called TypeSafe, which has announced that they have built what they call a new class of models called System 1 models. And JEV is really their kind of first public release of this for coding. And David, there's a bunch of really interesting attributes for what they appear to be doing here. I'm sort of a former open AI engineer or researcher named Diogo Almeida. He was sort of early at chat GPT or at open AI, rather. He wrote the co-wrote the kind of foundational instructor GPT paper. That said, he was kind of I hadn't heard of him. He was kind of under the radar from my point of view. And then he released this model and it just sort of blew up. Right. He had a X account with 1500 followers. His post got 40 million views, I think. about JEV to people, I think it does speak to this theme of efficiency that we're talking about, right? I think there's sort of one or two value props that I want to talk about with respect to JEV, but the first is to do with kind of a fit-for-purpose model, you could say, right? So I think the contention of JEV is, hey, we are using LLMs, which are basically like novelists, right? They can fluently generate word after word of prose for human readability. And then, in the enterprise use case, we're then kind of, you know, retrofitting that and forcing that into making these simple structured decisions for sort of often the multiple choice questions that that kind of underlie software and processes. values of typed values is the term in software, right? It's just going to output those typed values directly, and it's going to be much more efficient in the process. Kaltar, did I get that right? Have I understood this correctly? And secondly, are you buying the hype here? When you looked at the Jeff release, are you as excited as everyone else is? Yeah, I think it's a very interesting direction that they're taking. And of course, the claim that they have is we don't need maybe decisions or scoring or yes and no answer. And those especially will be very useful when the software is using these LLMs in the agentic workflows, et cetera, where I don't need to read the whole sentence, then try to interpret what this is saying and then turn it into decisions or some score, et cetera. And if I look at the hardware angle here, what I find interesting about Jeff from a hardware perspective is that they're changing the workload itself With a normal language model, once you start generating an answer, you generate one token, then the next token, then the next, etc. And that creates that sequential decode loop. So every step needs to go back to the accelerator, reach the model state, the KV cache, does the computation, and write things back and then repeats. option should I choose or what is the probability of each outcome? So Jeff gives us this freeform text generation for those workloads. And instead, it kind of produces the structured decisions that and these probabilities in parallel. So from a hardware side, I think that is attractive [15:00] to because Parallel computation is exactly what GPUs and AI accelerators are really good at. So you have less sequential decoding here, much less output generation, and potentially much less KV cache traffic. So I think part of their efficiency gain is not necessarily that they found a magical way to make matrix multiplication faster, but they change what they're asking the hardware to do. But I still have some reservations about, you know, some of their claims, in terms especially of the correctness, because they still rely on models to generate some of their scores and some of their claims. So I think that part, you know, we need to be careful about, especially the calibration that they mentioned. changes. So for automation, I think that it becomes very important and maybe even more important than maybe the latency numbers that they're claiming. I want to chime in there and I'm always a little interested in catching up the people in the back of the class like me who are non-technical. And I want to talk a little bit about what I learned this week about this theme of calibration. You mentioned confidence scores and calibration. And broadly, again, my understanding It can be right or wrong, but almost just as important, it can be overconfident or underconfident in its prediction, right? So we can say, that's a stop sign, and I'm 90% sure. But if, lo and behold, when it makes that guess and that confidence score, and it's only right 10% of the time, you have a major overconfidence problem. And so that is what we call, I believe, calibration, right? actually is. And for years now, we've been so frustrated with hallucinating models that very confidently spout utter nonsense, right? And so they claim that this model can't hallucinate, which maybe is true or not. But more to the point, I think, if I understand correctly, what they've optimized for is really getting that calibration, right? So their model kind of lives or dies by getting the confidence score right. And it may not be, if I understand correctly again, the best at guessing whether that's a stop sign or not. But if it tells you it's 60% sure, it better darn well be 60% sure, not more and not less. So first of all, Gabe, did I get that right, my understanding of calibration? And secondly, If I did, why is this so important? Like, what becomes unlocked, especially in enterprise use cases, once you can be confident of the model's confidence, once it builds the trust of its confidence score. Okay, yeah. So on the confidence score, you did get that right. And one of the things I loved about this announcement was it I think basically hit on some critical moment in every single aspect of my historical journey as a software engineer. So I have so many reactions to this model. But on the confidence score, it reminds me very, very... directly of problems we had in the pre-generative AI days, building classifiers and creating REST APIs for said classifiers and deciding what was the name of the field that we used to indicate the quality of the output. Was it probability or was it confidence? We always chose confidence and not probability because probability has a mathematical definition. It is a distributional statement. confidence is just a number. Confidence is just we made it up. We slapped a sigmoid on there and it looks like a nice bell curve when we look at it over the distribution of inputs. And sure, this one's better than that one. And this one's got a higher number than that one. So trust it. So trust. This you're exactly right to pick on this point because, uh, This gets into some real challenges of using this type of opaque approximate math to do things that actually have rigorous mathematical definitions like probability. Um, The... end goal of all of this however is useful decision making which is where a model like jev or even our pre-generative ai classifiers typically tended to say well actually confidence is fine because the only thing you really need to be able to do is take the scores relative to one another and make a decision based on those scores so the downstream consumer of a jev output is simply going to take that confidence distribution and say, okay, where's my threshold that I say left or right? Where's my threshold that I say red, yellow, or green? [20:00] Like, I just have to pick a number and I have to do that empirically. That's that's left as an exercise to the user. So the calibration is important. And I'm not sure whether they are claiming that it is a proper probability distribution. That would be a hard mathematical claim to back up. But They are probably at least claiming that the confidence scores are true relative to one another. I want to pick on a very different aspect of this model. I'm just going to steal the floor while I have it because I think it's really important to couch this model relative to all the other models and where it fits. So we've talked about this as essentially a distribution predictor. And in other words, it's a classification model. Classification is a problem that has been observed. It's in many ways the oldest problem. in NLP or even not even NLP in in AI in general. You get a feature vector in and you try to classify that feature vector based on some set of labels that could apply to it. The reason that Many, many, many people have moved away from training their own classifier models, which can be orders of magnitude smaller than LLMs, to be clear, smaller and more efficient, both in terms of runtime and compute power needed. The reason everyone has moved away from that is that in the bad old days, you had to train your own classifier in a supervised learning manner. classification problem. So you give it a few examples, which essentially tickle the weights area that is relevant to your classification problem. And you say, use those examples to form your model's intuition and then splat me out a classification for this thing, please. However, as Kautor accurately pointed out, generating tokens by tokens that says, I believe this is a stop sign period. I think so because it has, and then you just have to write that says, okay, I got a big pile of gobbledygook. I got the word stop sign in there. Like, you know, do I have to check for case insensitivity? Do I have to look for weird white space in between? Like, you know, it's just a software engineering nightmare. So getting out a vector of numbers that says, you know, entry one is stop sign entry two is you know yield sign etc like that's way way way more efficient both in terms of inference compute and in terms of the engineering effort to extract the output so in that sense this is great however llms are used for so so so so so much more than classification so um What it comes down to is that this model is targeting a very, very specific and important problem that is a subset of the problems that LLMs can tackle. Now, I want to... point out that this area of making deterministic decisions, or at least close to deterministic decisions, is one that we at IBM have been tackling with a program programming organization. model, not a AI model, but a programming model called Malaya. So this is essentially a way of inverting what many people are doing, which is describing a program in plain text and hoping that the model will actually execute the steps that you say. Those steps in the plain text description are perfect for a model like JEV. So a programming model like Malaya can take a model like JEV, use it where there is a clear decision point to be made in a workflow and actually create that has small non-deterministic elements where you have to make a decision. So that's the real power of a model like this is being able to take something that you know is a well-defined path through the, actions and make good decisions with non-deterministic inputs at the important junctures of that thing so that you can walk the right path yeah the thing that really interested me two things one the name this is a system one model what on earth does that mean well apparently it's called a system one model as reference to the system one and system two thinking from danny kahneman's thinking fast and slow book which i was delighted to see that because i have the book with me i like i read this He gets a cut for every purchase made off of this MOE episode. Oh, absolutely. Click on my affiliate link in the description. What that was saying is, you know, system one thinking is kind of the automatic thinking. It just kind of happens. We don't have to consciously think about it. And system two thinking is thinking more through step by step. And if we think about that in terms of AI models, Large language models today, thinking models in particular, they do this chain of thought reasoning, this kind of system to thinking. And one of the arguments for JEV is to say, look, for a lot of these decisions, we don't need the system to step by step thinking. We've been using system to when we really only needed [25:00] system one for things that are just a classification problem, as Gabe was mentioning. So if the answer is yes or no, or the answer is pick one of 15 options, then a large language model is a huge overhead to go through, to come up to that answer, to drive to that answer. So it was interesting to hear how JEV was trained. That was the other thing that really got me interested, because large language models, they're typically trained to be primarily chatbots, certainly initially. And that uses reinforcement learning from human feedback. So we're trying to get an answer that is most agreeable to a human. And then these models, as they've become thinking models and are now used more for AI agentic purposes as well, there's reinforcement learning from verifiable rewards, which is to say, if I ask for you to do this thing, did you actually do this thing? And the model is rewarded when it does. but JEV and the system one models are built on a different reinforcement learning which is reinforcement learning for calibrated decisions so this is not trying to come up with the answer that is pleasing to a person you know it's structured in that way and it not trying to come up with an answer that that comes up to a reward it just trying to say is this answer right or not Now David you talked about calibration and this is a timed model It gives you back a number basically as to say what the answer is. And of course, the question is, is the answer right? And it's not coming back with any supporting information. So if I ask a large language model a question, it will give me an answer, but it might also actually tell me why it thinks that is the answer. Now, that reasoning chain, how it got to that answer may be entirely or at least partially hallucinated, but at least it gives you an answer. With this model, it's just going to give you a number. So how do you know it's right? Well, the benchmarks that the team behind Jev have offered up show that it is able to get to the same decision as the top frontier models. So if you ask the same question to Astra or to Fable, most often Jev would also come up with the same answer. but much quicker and with fewer token costs. So it's able to reach an answer that a large language model will come up with, but it hasn't been tested against actual sources of truth. I mean, it could be that the large language model is also wrong. So it will be interesting to see more benchmarks where it actually tested against things in real life That said some of the demos they provided so there a demo of of jeff basically playing doom where it figures out how to move around in space and to shoot the bad guys and so forth um and you know that is making constant decisions these these yes or no decisions or where should i move to should i move left should i move right yes or no and so forth and it kind of works in that that perspective so it was pretty accurate at hunting out the bad guys. And then I saw another demo where they kind of played the Wikipedia game, which we've all played this, right? Where you start at one page, like, I don't know, the English medieval period, and then you've got to get to another completely unrelated page, like flying cars. Like Kevin Bacon. How many, yeah, exactly. How many clicks does it take to get there, right? And it turned out that the best LLMs could often get to, from topic A to topic B, like within normally five clicks, which is pretty impressive. But then Jeff could do it in five clicks as well, but it could do it in sub-second periods of time, whereas a large language model would have to go through all of its reasoning chain of thought to get there and take multiple seconds to do it. So there are some early indications that not only is it coming up with the same answers as large language models but also that those answers are generally going to be right But there's so much more to see with this. So I think with Jeff, when it first came out, everyone's first thought, or at least my first thought was, oh, another classification model, that's wonderful. We've had these for a long time. But seeing the training behind it and this idea of reinforcement learning for calibrated decisions and seeing how well it stacks up to the very top frontier models with large language models. I think that's why it's this kind of such an exciting technology to keep an eye on. Well, this is all great. Awesome discussion. And I'm glad we got the explainer here because I think, you know, again, I think this is why I'm always for in a lot of ways to be able to cut through some of the hype and get behind the scenes on what's going on. We're going to stop talking about a model to talk about another model. I think the last one I really wanted to cover today is one that caught my eye. It's a nice collaboration between IBM and NASA, specifically the release of a set of models that are kind of specific for lunar exploration. And so these are effectively kind of computer vision models that are really good at kind of picking up on sort of lunar surface images and be able to identify features for researchers. [30:00] And, um, David, I like this one a lot because I think in some ways, this is sort of a really interesting story of essentially the way that machine learning can be used to extract more knowledge out of the data we already have, which I think is a thing that we don't talk about so much, but I think is really interesting as part of the story here. Yeah. At first, I mean, sort of like Martin, my first reaction, I'm kind of like a moon and like Apollo fanboy, but yeah. At first I read about this, I kind of wanted to cry tears of boredom because it was like, it helps you count craters quicker. or something like that. And I was like, why does this matter? Are humans counting craters at all? That sounds like the saddest job in the world. But then I learned that actually, this is very important, that if you can count the number of craters in a certain area of the moon, you can infer certain things about the age of the solar system and how often, because actually those craters are sort of like a clock for the universe, right? Because they represent the rate at which kind of external debris kind of hits the moon. And then you can infer things about the age of So I became very excited. learning about how this model can accelerate the science. Beginning with you, Gabe, what got you excited about this model? I saw there was also it can help identify ice, which is apparently a very difficult kind of multi-parameter problem to solve and could have implications for lunar exploration and colonization. So I think what got me excited about this model was it's AI for nerds again, which is awesome. nerds and scientists and geeks and people that like to think about esoteric problems that most people would look at and say why the hell are we counting craters on the moon and then as you accurately pointed out or at least i hope is accurate because i am not a moon nerd uh apparently craters on the moon means a whole lot to the age of the universe and and we can learn tons from that and that's where geeks live that's where nerds live like so um We live in an age where AI is starting to become not just a fascinating technology, but a hugely important socio-technical problem. And those of us that have lived in this from the interesting math and the interesting geeking out about what we could get our terminals to do are starting to find ourselves wrestling with very human problems. And that's important, but it's also uncomfortable. science uh problem here so i think that got me excited um You know, I also love seeing AI applied in a domain that is not a chatbot or a coding agent, simply because it's a good reminder that this foundational technology applies much more broadly than the consumer use cases that we're all consuming. And I know. That conversation is often lost when we get into the latest releases from the frontier labs. But there's some real benefits to the overall knowledge and state of humanity that we can glean out of the core technology behind all of this that probably have nothing to do with how efficient, how token efficient we can generate, you know, comics from our favorite blog posts. Yeah, I think I agree with you, Gabe. And I, What I really like a lot about this project is that really it shows that, that a foundation model doesn't have to be a chatbot. And the value here is reuse. And so you train a large body of lunar data ones. And then you can adapt to, the representations to really several scientific questions, instead of just starting from zero every time. And that is of huge value. And also what I think the most important work here is also the data engineering, not just the model size. The sensors here have different resolutions, different observation conditions, etc. And aligning all of those measurements and giving the model information about how the data was captured can also be just as important as adding more parameters. There is a lot of interesting engineering data problems here that we can learn tons from. And, of course, just, you know, being able to answer these big scientific questions, you know, has huge value to humanity, of course. I think the other thing that makes it super interesting is, you know, When we think of foundation models, the first most obvious thing is to think of a text-based foundation model. I put some text in, it generates text out. But of course, we have multimodal models now that work natively with other modalities other than text, right? So we're all familiar with image generation models, where an image can go in, an image can come out. And the way that works is just basically, as long as the information can be readied, [35:00] as vectors, as a series of numbers. then it can be used in a foundation model. So it's nice to see that expanded to things like, you know, pictures taken in space and to be able to represent that information again as vectors and then to be able to use a foundation model, really the same sort of foundation model that's dealing with text, but now it's able to do clever things like look at a particular image that was taken where the sun is at a particular angle. That's kind of obscuring a little bit how the craters are visible and to be able to adjust to that. all just using the same sort of math in these models, the same sort of reasoning that is also being used to write us a poem in a text-based large language model as well. So a really cool use of a foundation model just applied to something completely different. That's great, and a great note to end on. Well, that's all the time that we have for today. Martin, Gabe, Kautar, always a pleasure to have you on the show. And thanks for joining all your listeners. This is actually going to be my last episode hosting Mixture of Experts, on the show. And if you join what you heard. You can get us on Apple Podcasts, Spotify, and podcast platforms everywhere. And we'll see you all next week on Mixture of Experts. I, whether this gets edited out or not, want to do a little bit of a bigger send-off for Tim. And we can see whether this passes the censors. But I will just say, I've been a fan of the show for a long time. I think, Tim, you've done a splendid job making technical topics really approachable. my daughter my two year old daughter is obsessed with wearing her parents shoes and stumbles around in my apartment and I'm stepping into some very large shoes here may I fall on my face less often than my two year old daughter but yeah you we this is quite a responsibility to step into so thank you so much for what you've done. Yeah, thank you. It's very generous and Godspeed. Echoing that, Tim, I feel like one of my favorite things actually is to always listen to the episode myself after we have recorded it, because I feel like when we're recording, I'm in the mode of like, OK, I got to think of what my insight is going to be about this topic. And then when I listen to it again, I realize that like your transitions between topics always have this like nugget of leading insight that clearly like cues it up so nicely. The way you have done that, it just orchestrates so nicely this flow. So I've always appreciated that. Thank you. Yeah, totally. Because it would be very easy, I think, when you are asking the question to the next person, you kind of easily put the person on the spot. It's like, oh, I wasn't expecting that one. Because we don't have like a script knowing what's coming in. We just have the topics. But you always manage to sort of move the conversation to the next person in a way that makes it just like blindingly obvious to me what I need to say. Which is like, that's a real talent. Yeah. I echo what everything was said. tremendous job and amazing job here you know to keeping the show so exciting even for us you know as we participate we're always at the edge what's going to happen next you know what's going to be the the next question how do I get prepared for this so and that really makes it interesting and so naturally flowing and I think we can see the the engagement also from the listeners so thank you so much Tim. Thank you. Yeah, I mean, it's a great compliment of like two years of just keeping us on our toes. So I really appreciate that. My testament to this is that I got a cold email the other day asking me to be an angel investor in a startup because they heard my quotes from like the last year on Nickster of Experts and thought I was somebody that would be ready to be an angel investor. And I was like, well, yes, how much do you need? Well, yes, I have I have a five dollar bill in my wallet. Would you like that? You've brought a level of panache to this show that really has made it stand out as a rare light of consumer-facing value out of IDM. Let's put it that way.