OpenAI Unveils GPT-5.6 as Developers Pivot to Output Metrics

OpenAI发布GPT-5.6，开发者重心转向产出指标 放弃单纯追求Token消耗量，转而采用针对性的模型规格与算法工具沙箱化策略，可在加速任务完成的同时降低企业AI部署成本。 本次分享介绍了从“tokenmaxxing”（盲目追求原始Token消耗量）向“valuemaxxing”（最大化每个Token所带来的具体业务成果）的转变。OpenAI的Charlie Guo详细剖析了GPT-5.6模型系列——Sol、Terra和Luna，以及编程式工具调用、提示词缓存和上下文压缩等实用策略。随后，初创公司Ploy的特邀嘉宾Bryant Chou与Lorenzo Gentile分享了实际工程实践，展示了如何通过优化KV缓存断点、批量处理工具调用以及转向确定性逻辑，以更低成本实现更快的构建。 AI开发文化正从“tokenmaxxing”（关注原始Token体量与活跃度）转向“valuemaxxing”（衡量每项任务的具体业务成果与性价比）。 (2:50) GPT-5.6模型家族包含针对不同工作负载优化的三个层级：GPT-5.6 Sol（前沿逻辑与复杂任务）、GPT-5.6 Terra（均衡的主力模型）以及GPT-5.6 Luna（高吞吐、低延迟且低成本）。 (7:11) 编程式工具调用可在隔离的JavaScript沙箱内执行代码，省去了中间的模型推理轮次，并减少了网络往返次数。 (15:16) 采用静态内容在前、动态上下文在后的方式组织提示词结构，能够保障键值（KV）缓存的连续性，从而大幅削减成本。 (16:04) 上下文压缩与持久化推理能够在长Agent线程中保持回答的连贯性，同时显著降低输入Token开销。 (17:04)

During an OpenAI Build Hour, presenters outlined an operational transition from raw token consumption to 'valuemaxxing'—evaluating AI deployments by business outcomes, task completion, and engineering efficiency across the GPT-5.6 model lineup. Shifting Focus from Token Volume to Business Outcomes (2:50) Charlie Guo from OpenAI's DevEx team argued that early industry practices of measuring progress through 'tokenmaxxing'—such as tracking raw token counts or creating internal usage leaderboards—led organizations to exhaust annual AI budgets within months. The alternative framework, valuemaxxing, focuses on measurable returns and workflow leverage, evaluating whether doubled token spend yields commensurate value. Guo noted that valuemaxxing does not inherently require spending fewer tokens. In high-stakes tasks or critical migrations, increasing token volume—through escalated reasoning depth, multi-model consensus judges, or high-speed execution modes—can reduce calendar time from months to weeks and mitigate delivery risks. Model Tiering and Reasoning Configurations in GPT-5.6 (7:11) The GPT-5.6 family spans three tiers: Sol (flagship reasoning and complex coding), Terra (balanced daily driver), and Luna (low-latency, high-volume tasks). SWE-bench evaluations indicated that lower-tier models paired with longer reasoning chains can match the single-shot performance of larger configurations, making total cost per completed task more relevant than individual token pricing. For Codex workflows, Guo recommended starting with GPT-5.6 Sol on medium reasoning, reserving top-tier settings for complex logic while dropping to light reasoning or Terra for routine tasks. Operating modes also trade tokens for velocity: Fast Mode increases execution speed by 1.5×, whereas Pro Mode (accessed via API parameter) and Max Reasoning Effort unlock unrestrained reasoning depth. Ultra Mode layers on Max reasoning to spawn specialized sub-agents, which the speakers advised against using for trivial workflows. Prompt Caching and Harness Optimizations (15:16) Because prompt cache systems are strictly append-only, any early modifications to the context window—such as prepending dynamic timestamps—break cache continuity and increase token costs over multi-step agent loops. Granular caching of unchanging system blocks reduced input expenses by 90% across sequential API tests. To maintain long threads, context compaction trimmed input overhead by 82% (from 24,000 to roughly 4,000 tokens) in a 72-turn test, accepting a one-time 30-second delay for lower ongoing spend. Programmatic tool calling further reduces overhead by executing code directly inside an isolated JavaScript sandbox. Shifting multi-step tasks into local execution loops reduced input tokens by 24% and lowered output usage by eliminating the model's step-by-step intermediate verbal reasoning. Production Agent Efficiency at Ploy (23:00) Bryant Chou and Lorenzo Gentile shared production patterns from their startup, Ploy, which converts websites into automated growth engines. By setting a static cache breakpoint after the system prompt and tool definitions, Ploy achieved an 89% token reduction on the initial turn of new chats and a 5% drop across all production token spend. A second cache breakpoint applied after persistent workspace memory preserved cache hits across consecutive sessions. Ploy reduced overall tool schema sizes by 45% and total benchmark costs by 33% by loading core tools (used in >70% of runs) permanently while dynamically appending niche schemas to the end of the context. Batching independent file reads and edits into combined, multi-action tool calls cut costs by 14% without affecting evaluation pass rates. Enabling Exa's search highlights compressed tool output size by 70%, projecting $37,000 in annual token savings, while converting non-reasoning loops into deterministic code paths bypassed entire model round trips to cut workflow token spend by 54%. Multi-Agent Delegation Depth and Thread Management (45:54) Addressing architectural trade-offs, the panel warned against deep delegation hierarchies. Guo observed that agent hierarchies extending beyond two layers generally suffer from degraded intent and context loss across hops. Gentile highlighted that Ploy's earlier trials with multi-agent handoffs performed worse in speed, cost, and accuracy than single context windows, warning that chaining mixed models like Sol, Terra, and Luna risks compounding context decay. Thread hygiene remains an operational challenge. While Guo stated that modern context compaction allows single Codex threads to remain viable for weeks, heavy default environments with numerous active plugins still justify starting fresh chats. Chou and Gentile noted that user tendencies to run unrelated tasks inside persistent workspace threads create context bloat, prompting ongoing UX and orchestration experiments to automate thread resets. Speakers emphasized that architectural improvements—such as pruning unused context, batching tool calls, and shifting deterministic logic to code—should precede caching and compaction strategies. The team announced that the next Build Hour on July 29th will cover ImageGen 2.

OpenAI 在 Build Hour 研讨会中提出从单纯度量消耗的“Token 狂热”（Tokenmaxxing）转向关注实际业务成效的“价值最大化”（Valuemaxxing）范式。OpenAI 与初创企业 Ploy 的技术专家指出，通过精细化匹配 GPT-5.6 模型层级、利用沙盒与批处理重构工具调用、维持静态键值（KV）缓存以及将非必要推理移入确定性代码，开发团队能够在控制预算的同时大幅提升研发与生产效率。 业务成效导向的价值最大化范式 (2:50) OpenAI 开发者体验团队的 Charlie Guo 指出，AI 应用早期普遍存在以 Token 消耗总量、Prompt 计数或并行代理数作为内部进展指标的“Token 狂热”，导致部分企业在数月内意外耗尽全年 AI 预算。相比之下，“价值最大化”主张以业务产出质量和时间投入回报为核心，要求工程团队明确 AI 介入的具体杠杆环节，并通过严格的评估套件（evals）衡量任务完成度，核心诊断指标转变为“如果将 Token 预算翻倍，团队能否明确验证其业务回报”。 Charlie 强调，价值最大化并不意味着一味缩减 Token。在多项关键场景中，增加 Token 投入反而能创造更高净值：例如为高风险任务调用更大模型或提升推理深度以确保输出无误；通过高强度 Token 消耗将耗时数月的代码迁移压缩至数周完成；以及在核心应用中运行多个不同的大语言模型作为评判者（LLM-as-a-judge）来捕获边缘故障并防范系统性风险。 GPT-5.6 模型层级与推理模式选型 (7:11) GPT-5.6 系列划分为三个梯度：用于复杂编程与深度逻辑的前沿旗舰 Sol、平衡智能与延迟成本的主力日常模型 Terra，以及适用于高吞吐量常规任务的低延迟模型 Luna。SWE-bench 等基准测试表明，低阶模型配合更长的思考链能够达到大模型单次生成的表现，因此“单项任务完成总成本”比单 Token 单价更具指导意义。在 Codex 开发实践中，Charlie 建议以“GPT-5.6 Sol 搭配中等推理”为默认基准，避免在常规任务中滥用最高推理档位，并在开发者时间最为紧迫时启用可提速 1.5 倍的 Fast Mode。 在推理模式层面，GPT-5.6 引入了完全放开思考 Token 数量限制的 Max 模式、通过 API 参数 `reasoning_mode` 开启的 Pro 模式，以及由原生系统提示词驱动任务拆解与子代理派发的 Ultra 模式。Charlie 提醒，Ultra 模式对于常规任务存在明显性能过剩。而在多代理架构设计上，Charlie 与 Ploy 工程师 Lorenzo Gentile 均警告，多层级子代理（超过两层调用）在实践中极易发生意图漂移与上下文丢失，更适用于无共享状态、无代码冲突的独立并行任务。 静态 KV 缓存与上下文压缩机制 (16:04) 在多步骤代理执行过程中，上下文呈二次方增长，有效利用键值（KV）缓存可降低约 10 倍的上下文处理成本。然而，模型供应商的缓存机制依赖严格的“仅追加”（append-only）特性，在提示词开头修改动态变量（如插入当前日期或动态工具定义）会破坏后续所有调用的缓存连贯性。OpenAI 实验表明，将系统提示词前置固定后，5,500 Token 提示词在连续请求中实现了 90% 的输入成本削减；Ploy 在系统提示词及工具定义后设置首个缓存断点，使新会话首条消息的 Token 消耗下降 89%，并在跨会话的工作区持久记忆后设置第二断点以进一步锁定命中率。 针对超长对话，OpenAI 的上下文压缩（Compaction）接口可在经历 72 次交接的调用中将输入 Token 削减 82%（从 24,000 降至约 4,000），且输出结果一致，但单次压缩需要承担近 30 秒的单次延迟。Lorenzo 警告，开发者应首先排查代理日志以清除多余往返与臃肿数据，缓存是性能优化手段而非掩盖混乱上下文设计的补丁。 沙盒程序化调用与确定性代码分流 (15:16) GPT-5.6 推出的程序化工具调用允许模型在隔离的 JavaScript 沙盒中编写并运行代码，使计算、数据结构转换和连续工具调用在本地沙盒内完成。该机制消除了模型在各步骤间冗长的言语推理与多次网络往返，在官方测试中减少了整整一轮模型交互，并直接降低了 24% 的输入 Token。 在企业级建站自动化平台 Ploy 的实践中，工程团队落地了多项工具精简策略：将高频工具（覆盖超 70% 运行）常驻系统提示词，低频工具动态追加至上下文末尾，使工具模式体积缩减 45%、整体评估成本下降 33%；提示词引导代理将独立工具调用合并为单步批量执行，削减了 14% 的成本并降低了响应延迟；将搜索工具 Exa 配置为仅返回精简要点（highlights），使网页搜索上下文体积缩减 70%，年均预计节省约 37,000 美元；团队更进一步将无需大模型参与的逻辑重构为硬编码的确定性程序代码，直接消除了一整轮代理交互，在该特定流程中削减了 54% 的 Token 支出。 从消耗导向转向价值最大化，要求工程团队破除对通用基准测试与纯粹模型推理的依赖。Ploy 联合创始人 Bryant Chou 认为，构建持久竞争优势取决于针对真实生产场景建立自研评估套件并敏捷重构架构；同时，团队如何在超长会话中平衡上下文连续性与 Token 消耗，以及如何在多代理自主协作中避免语义损耗，仍是高阶 AI 系统工程中需要持续权衡的课题。

## Background 本期视频由 OpenAI 官方频道于 2026 年 8 月 3 日发布在 YouTube 平台。根据页面简介，本期 Build Hour 聚焦如何利用全新发布的 GPT-5.6 从每个 Token 中获取更高价值，涵盖模型选型、评测适配、Agent 脚手架优化，并特邀初创团队 Ploy 分享生产级迁移至 GPT-5.6 的实战经验与现场演示答疑。 ## TL;DR 活动首先提出了从盲目消耗 Token（Tokenmaxxing）转向关注真实业务产出（Valuemaxxing）的范式演变，并系统拆解了 GPT-5.6 家族 Sol、Terra 和 Luna 三款模型的特性与选型权衡。随后，OpenAI 工程师通过“价值最大化实验室”实测演示了提示词缓存（Prompt Caching）、编程式工具调用（Programmatic Tool Calling）及上下文压缩（Compaction）等核心技术在削减成本和降低延迟上的巨大收益。最后，Ploy 创始人 Bryant Chou 与工程师 Lorenzo Gentile 深度剖析了如何解决多步智能体中的二次方级成本陷阱，结合纯追加式缓存架构与按需工具设计，展示了如何在生产环境中实现性能提升 2.2 倍、成本削减 27% 的落地经验。 ## Atmosphere & style 整场研讨会充满极客务实风范与高质量的工程幽默，开场以《办公室》热梗破冰，迅速定调了打破盲目烧 Token 迷信的清醒氛围。讲师与特邀嘉宾之间的互动默契自如，从现场实时运行的“水上摩托熊猫”SVG 对比实验，到 Lorenzo 充满激情地拆解第一性原理与二次方成本黑洞，节奏紧凑且干货密集。问答环节更是坦诚务实，直面多层子代理失控与长上下文污染等血泪教训，让观众在轻松真实的共鸣中感受到一线工程落地的严谨与硬核魅力。 ## Bulletin - [02:50](https://www.youtube.com/watch?v=jyuyY86GJnA&t=170s) OpenAI 举办 GPT-5.6 专题 Build Hour，倡导开发者从盲目消耗 Token 的“Tokenmaxxing”转向注重实际业务产出的“Valuemaxxing”。 - [07:11](https://www.youtube.com/watch?v=jyuyY86GJnA&t=431s) GPT-5.6 家族细分为前沿旗舰 Sol、全能主力 Terra 及低延迟高吞吐 Luna 三款模型，核心评估指标从单 Token 标价转为任务端到端交付总成本。 - [19:12](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1152s) 利用 Programmatic Tool Calling 的 JS 沙盒执行与 Prompt Caching，实测可分别削减 24% 输入 Token 并降低 90% 输入成本，长会话上下文压缩率更达 82%。 - [23:00](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1380s) 初创企业 Ploy 分享生产迁移经验，通过保持上下文“纯追加”、按需加载工具 Schema 及工具批处理，成功将 Agent 构建速度提升 2.2 倍，综合成本降低 27%。 ## Key takeaways - [02:50](https://www.youtube.com/watch?v=jyuyY86GJnA&t=170s) 行业正经历从单纯攀比 Token 消耗量的“Tokenmaxxing”到强调实际交付成果与 ROI 的“Valuemaxxing”范式转变。 - [04:31](https://www.youtube.com/watch?v=jyuyY86GJnA&t=271s) 追求 Valuemaxxing 并非单纯省钱，在提升推理深度、换取挂钟时间、规模化验证流程及安全裁判等关键节点应主动加注 Token。 - [04:31](https://www.youtube.com/watch?v=jyuyY86GJnA&t=271s) 评测体系（Evals）是衡量业务价值的核心抓手，其本质是为具体工作流清晰定义端到端“好”的标准。 - [07:11](https://www.youtube.com/watch?v=jyuyY86GJnA&t=431s) GPT-5.6 矩阵覆盖 Sol（攻坚深层推理）、Terra（平衡性价比）与 Luna（高吞吐低延迟），模型选择需综合平衡智力、成本与延迟三维坐标。 - [12:24](https://www.youtube.com/watch?v=jyuyY86GJnA&t=744s) 日常使用 Codex 推荐默认采用 GPT-5.6 Sol 的 medium 推理档位，非必要无需拉满至 extra high，并推荐开启 Auto Approval 保持开发心流。 - [15:16](https://www.youtube.com/watch?v=jyuyY86GJnA&t=916s) Programmatic Tool Calling 允许模型在 JavaScript 沙箱中直接运行计算与工具调用，规避中间思维链膨胀并减少网络往返。 - [15:16](https://www.youtube.com/watch?v=jyuyY86GJnA&t=916s) 提示词缓存（Prompt Caching）对位置极度敏感，时间戳等动态变量必须置于末尾，确保前置长静态 System Prompt 长期稳定命中缓存。 - [21:27](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1287s) 长多轮对话场景下，上下文压缩（Context Compaction）可在消耗极短后台处理时间的前提下，将输入体积大幅压减 80% 以上并保持回答精度。 - [28:11](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1691s) 多步 Agent 循环若无精准缓存控制，每次动作重新处理单调递增的历史会导致整体 Token 处理量呈现 $O(N^2)$ 级爆炸式增长。 - [30:15](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1815s) 为保护脆弱的 KV 缓存，上下文必须保持“纯追加（Append-only）”；动态修改顶部工具 Schema 会打碎缓存并导致成本飙升 10 倍。 - [32:05](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1925s) 采用“核心工具常驻 + 边缘工具按需追加”架构，配合跨会话工作区记忆断点缓存，可大幅缩减 Schema 体积与首字成本。 - [35:58](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2158s) 通过 Prompt 引导合并决策输出，或设计支持复数参数的复合工具（如批量读文件），能显著降低中间思考 Token 并提升执行效率。 - [42:38](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2558s) GPT-5.6 中的 Pro 转化为统一的 reasoning_mode 参数，而 Ultra 模式则是叠加在 Max 推理之上、用于极端重度工程的主动分发子代理机制。 - [45:52](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2752s) 子代理层级超过两层往往伴随严重的语义丢失与任务漂移，子代理更适合高度扁平、无状态竞争的横向并行任务，嵌套调度需极为谨慎。 - [50:31](https://www.youtube.com/watch?v=jyuyY86GJnA&t=3031s) 缓存优化应作为后置手段建立在链路 Traces 审计之上，优先消除冗余 Tool 输出并将确定性逻辑下沉为代码，才能最大化工程收益。 ## Detailed walkthrough ### [00:01](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1s) 开场介绍与 GPT-5.6 发布回顾 活动由 OpenAI 初创团队营销负责人 Christine 主持，开发者体验团队（DevEx）的 Charlie 共同主讲。Christine 介绍道，Build Hour 系列活动的核心目标是为开发者和初创团队提供前沿的最佳实践、工具链与 AI 专长，帮助大家基于 OpenAI 的 API 和模型高效扩展业务。本次活动不仅在 OpenAI 主页保持更新，也会在当天同步上线 YouTube，方便开发者随时回看复习。 秉承 Build Hour“以梗图（Meme）破冰”的传统，本次开场晒出了一张《办公室》（*The Office*）风格的趣味表情包，主题直指“如何从每个 Token 中榨取更多价值”。这是 OpenAI 正式发布 GPT-5.6 后的首期 Build Hour。Christine 回忆，模型发布后团队在社交平台 X 上向社区征集使用反馈，收到的热情回复令人备受鼓舞。大家普遍反馈的一个核心亮点是该模型的 Token 使用效率极高，真正做到了“把每一分钱和每一个 Token 都花在刀刃上”（get the most bang per token）。 ```mermaid flowchart LR accTitle: Build Hour 活动核心议程流转 accDescr: 展示从理念演进到工具实践，再到客户迁移案例与现场问答的完整分享流程。 subgraph S1["理论与演进"] A["Tokenmaxxing 转向 Valuemaxxing"] end subgraph S2["工具与工程优化"] B["Codex 价值极大化"] C["Agent Harness 运行时调优"] end subgraph S3["实践与互动"] D["Ploy 生产迁移案例"] E["实时问答与互动"] end A --> B B --> C C --> D D --> E ``` 本期内容主要涵盖四个核心板块： - **范式转变**：从单纯追求调用量的“Tokenmaxxing”转向注重实际产出的“Valuemaxxing”； - **编码与框架实操**：在 Codex 中践行价值极大化，并针对 AI Agent 的脚手架（Harness）进行优化； - **生产级迁移案例**：明星初创企业 Ploy 团队分享如何将生产环境中的 AI Agent 迁移至 GPT-5.6； - **现场问答互动**：由 OpenAI 团队和 Ploy 嘉宾共同解答关于模型选型、延迟优化和工具调用的疑问。 --- ### [02:50](https://www.youtube.com/watch?v=jyuyY86GJnA&t=170s) 范式转变：从“Tokenmaxxing”到“Valuemaxxing” Charlie 接过话茬，直奔主题：究竟什么是“价值极大化”（Valuemaxxing）？他认为，要理解这个概念，必须先回看今年早些时候社区中盛行的“Token 极大化”（Tokenmaxxing）。在当时，许多团队习惯用“消耗了多少 AI”来衡量技术进展——比如每天烧了多少 Token、发起了多少次 Prompt 交互、并发调度了多少个智能体。甚至有企业设立了内部积分榜（Leaderboard），专门横向比较各个员工或小组每天、每周消耗的 Token 数量，把资源消耗本身当成了政绩。 然而，这种狂热在下半年迎来了急剧反转。不少企业惊愕地发现，原本规划支撑全年的 AI 预算在短短几个月内就被彻底烧光。许多企业管理层不得不紧急收紧审批、强行给 AI 支出降速（Throttling AI spend）。多位 AI 领域的 CEO 也公开反思，指出盲目攀比 Token 消耗的积分榜已经本末倒置。行业的共识开始发生清醒的转向：企业考核员工的指标应当是实际的产出与创造的业务价值，绝非无意义地烧 Token。 正是在这一背景下，Valuemaxxing 的理念应运而生。它要求团队将衡量进展的标尺彻底掉头——不再看你用了多少 AI，而是看 AI 实际帮你完成了什么工作：交付了多少业务成果？为团队节省了多少工时？代码库和工程制品的综合质量得到了多大程度的飞跃？Charlie 提出了一个值得所有技术决策者自省的问题：“如果你明天将 Token 预算翻倍，你如何证明这笔钱花得值？”（"If you doubled your token spend tomorrow, how would you know if it was worth it?"）绝不能再天真地假设“花销翻倍产出自然就会变好”，必须建立从 Token 消耗到具体业务成果的明确因果链条。 --- ### [04:31](https://www.youtube.com/watch?v=jyuyY86GJnA&t=271s) 价值衡量的关键问题与评测体系（Evals） 为了帮助开发者和企业理清这套价值链条，Charlie 提供了一套层层递进的思考框架：首先明确企业想要提升的核心成果是什么；其次剖析通向高质量成果的具体工作流；进而定位在工作流的哪些关键节点注入智能与 Token 才能真正加速流程或拉升质量；最后，拿出确凿的数据与证据，证明系统不仅可用，而且质量确实得到了提升。这一切的核心抓手，正是当前 AI 研究与工程中最受关注的评测体系（Evals）。 如果脱离复杂的学术定义将 Evals 进行极简概括，它的本质就是一句话：“你如何定义什么叫做‘好’？”（"How do you define what good looks like?"）。从过去单纯用 Chatbot 处理琐碎任务，演进到如今让自主 Agent 承载端到端的完整业务结果，清晰界定“优秀的标准”变得前所未有地关键。 Charlie 特别指出，追求 Valuemaxxing 并不意味着单纯“少花钱、砍 Token”。在某些关键场景下，**主动增加 Token 消耗反而是实现更大价值的最佳手段**： - **提高推理深度与模型规模**：针对容错率极低的核心业务，调高模型的 Reasoning Level 或升级为更强模型，以充沛的 Token 换取不可妥协的高质量； - **以 Token 换取现实挂钟时间（Wall-clock time）**：例如面对大规模代码跨语言重构迁移，原本需要数月的人工协作，若利用并发 Agent 大规模消耗 Token，可以在两周之内密集完成交付； - **业务验证后的规模化放量**：一旦某套 Agent 工作流被严格证明有效且具备正向 ROI，就应该果断加注 Token 预算成倍放大产出； - **风险控制与自动化裁判**：引入基于大模型的裁判系统（LLM-as-a-judge），甚至调用多款异构模型组成评审矩阵，通过消耗更多推理 Token 穷尽各种边缘情况（Edge cases），构筑坚固的安全底线。 --- ### [07:11](https://www.youtube.com/watch?v=jyuyY86GJnA&t=431s) Codex 中的价值极大化与 GPT-5.6 家族矩阵 落地 Valuemaxxing 可以从两个维度切入：一是开发者在日常编码中使用 Codex 的个体视角；二是企业在 OpenAI API 之上架构智能体系统的组织视角。Charlie 首先从个体开发与 Codex 切入，而这一切的基础正是全新的 GPT-5.6 系列模型。该家族根据不同应用场景划分为三款差异化模型： | 模型名称 | 定位与适用场景 | 核心权衡特征 | |:--- |:--- |:--- | | **GPT-5.6 Sol** | 前沿旗舰模型（Frontier / Flagship），用于最复杂的编码挑战与高难度专业任务 | 最高智力水平，适合深层推理与攻坚 | | **GPT-5.6 Terra** | 全能主力模型（Daily Driver），适用于绝大多数常规开发工作 | 智能、调用成本与响应延迟之间的黄金平衡 | | **GPT-5.6 Luna** | 高吞吐量场景模型（High-volume workloads），面向对成本与延迟极度敏感的大批量任务 | 极低延迟与极优单价，应对吞吐密集型任务 | 针对 SWE-bench 等行业主流编码评测基准的横向对比显示，GPT-5.6 家族展现出了极为出色的 Token 效率。Charlie 分析指出，评估模型时切忌仅看“单次直出（One-shot）的原始智力”或表面上的“单 Token 单价（Cost per token）”。工程实践表明，给一个单次推理能力中等的模型分配更充裕的思考 Token 和推理预算，其最终任务完成质量完全可以比肩顶级模型。 因此，现代 AI 工程最关键的指标不是单纯的 Token 标价，而是**任务端到端交付的总体成本（Cost per overall task to completion）**。在这一评估维度下，开发者可以根据具体的任务复杂度在推理深度、响应延迟与调用成本构成的多维坐标系中灵活配置最适模型。在梳理完理论背景后，Charlie 随即切换界面，准备展示具体的对比演示。 ### [09:18](https://www.youtube.com/watch?v=jyuyY86GJnA&t=558s) 价值最大化实验室（Valuemaxxing Lab）与 SVG 生成矩阵演示 Charlie 演示了他在 GPT-5.6 和 Codex 辅助下构建的“价值最大化实验室”（Valuemaxxing Lab），该项目的代码将在 Build Hour 活动结束后完全开源。实验室首页展示了一个由 GPT-5.6 构建的三维交互式图表，直观对比了 Artificial Analysis 智能指数中主流大模型在“智能程度”（Intelligence）、“推理成本”（Cost）以及“生成速度”（Speed / Latency）三大核心维度上的权衡关系。 为了让不同模型及其推理等级（Reasoning Levels）的表现更加直观，Charlie 打造了一个视觉矩阵功能（Visual Matrix）。用户只需填入 API Key，系统就会并发调用多个模型和推理档位，生成对应的矢量图形（SVG）。Charlie 此前测试了“骑自行车的鹈鹕”（pelican riding a bicycle），展示了随着推理档位提高，画面从最初 Luna 模型的粗糙简陋，逐步提升到高推理等级下极具细节感的成图；而在 GPT-5.6 Sol 这一层级，起点就已经是高精细度输出。紧接着，Charlie 在现场实时发起了一项新测试——让所有模型生成“骑水上摩托的大熊猫”（a panda bear riding a jet ski）。测试结果实时渲染出来，可以清楚地观察到推理等级提升带来的质变：低档位几乎没有任何阴影渲染，而高推理档位则加入了海岛绿植、细致逼真的波浪浪花，以及背景中太阳与云朵的层级光影。 ```mermaid flowchart LR accTitle: 模型与推理等级的 SVG 生成效果对比 accDescr: 展示从低推理档位到高推理档位在视觉细节上的逐步演进过程。 subgraph LowTier["低阶 / 低推理档位"] A["基础轮廓\n线条粗糙"] B["缺乏光影\n扁平化配色"] end subgraph HighTier["高阶 / 高推理档位"] C["环境渲染\n岛屿绿植与精致海浪"] D["复杂光影\n太阳云层立体阴影"] end LowTier -->|"提高 Reasoning 级别 / 升级到 Sol"| HighTier ``` - **开源承诺**：整个 Valuemaxxing Lab 代码仓库已托管至 GitHub，供开发者自行克隆与本地运行。 - **三维分析维度**：基于 Artificial Analysis 数据，同时衡量智能水平、单 token 成本与响应生成延迟。 - **视觉矩阵设计**：通过统一的 SVG 生成 Prompt 橫向比对 Luna、Terra、Sol 等模型在不同推理等级下的表达上限。 --- ### [12:24](https://www.youtube.com/watch?v=jyuyY86GJnA&t=744s) Codex 使用准则：分级选型与 token 换时间 针对日常开发中如何榨取 Codex 的最大价值，Charlie 分享了三条实用策略。首先是**核心模型选型准则**：默认推荐从 **GPT-5.6 Sol 的 medium 推理档位**开始。他特别澄清了一个普遍误区——许多开发者习惯开箱即用直接拉满到“extra high”，但实际上除了架构设计、超复杂工程等高难度任务外，大部分日常编码需求在 Sol medium 下就能以极佳的性价比完美解决；只有在感觉模型表现不够深入时再向上调高档位，反之若非代码密集型任务，则可调低至 light 甚至切换为 Terra。 其次是**善用“以 Token 换时间”的效率特性**： 1. **Fast Mode**：可开启 1.5 倍速极速输出，适合追求敏捷反馈但愿意消耗更多额度的场景。 2. **Auto Approval（自动审批）**：强烈建议设为默认操作模式。Charlie 明确指出“目前不建议在 Codex 中直接使用 full access”，但每一步操作都人工弹框授权又极度打断心流，因此使用另一个模型在后台自动审计并放行安全操作的 Auto Approval 是最佳平衡点。 3. **Chronicle 功能**：Codex 最新的屏幕录制与记忆沉淀特性。开启后，它会消耗一定 token 来记录用户的开发流程并形成工作记忆，对重复性高频任务会越用越敏捷。 第三是**主动瘦身 `AGENTS.md` 与 Skill 指令**。随着 GPT-5.6 的基础指令遵循与上下文理解能力大幅跃升，半年前或一年前写在 Agent 配置里的大量繁琐、冗余兜底说明现在已成浪费。开发者应定期回溯审计，尽可能删繁就简，精简输入端 prompt 以节省上下文 token。 --- ### [15:16](https://www.youtube.com/watch?v=jyuyY86GJnA&t=916s) 框架（Harness）端价值最大化：四大核心 API 新特性 当讨论转向基于 OpenAI API 构建自主 Agent Harness 或生产级应用时，Charlie 强调了四项能够直接降低延迟、削减成本并提升稳定性的核心 API 特性： 1. **编程式工具调用（Programmatic Tool Calling）**：随 GPT-5.6 推出的一大突破。平台为模型配备了一个专用的 JavaScript 沙箱环境，模型可以直接编写并在沙箱中执行 JS 代码，用于多工具链式调用或复杂本地计算。这一机制直接把繁琐的数学运算和逻辑推演从模型的思维链（Chain of Thought）中卸离至沙箱执行，既省去了大量推理 token，又避免了在模型与终端用户之间来回往返（round trip）的网络延迟。 2. **提示词缓存（Prompt Caching）与结构规划**：开发者可以显式指定 prompt 缓存部分。Charlie 提醒在组织结构时务必将静态内容前置、动态内容后置。例如涉及“当前日期”（如 July 23rd）时，若放在 10,000 token 的 prompt 开头会导致每次日期变更整段缓存失效，而将其追加到末尾即可确保前置的庞大静态主体长期稳定命中缓存。 3. **持久化推理（Persistent Reasoning）**：API 允许开发者跨对话轮次直接保留并复用先前的推理思考过程，而不再局限于只拼接最终的 Assistant 文本输出或摘要。这不仅增强了复杂长任务的回答连贯性，还显著提高了后续请求的缓存命中效率。 4. **上下文压缩（Compaction）**：针对历史对话爆炸、堆积大量探索性工具调用废弃信息的长会话场景，可开启 Auto Compaction 或通过 API 显式触发压缩，大幅精简每次重新发送的输入体积。 ```mermaid flowchart TD accTitle: 编程式工具调用工作流 accDescr: 展示模型如何通过 JavaScript 沙箱直接运行代码并调用工具，从而避免思维链膨胀和多次网络往返。 subgraph LLM["GPT-5.6 核心"] P["接收输入任务"] Code["生成执行代码\n(JS Script)"] end subgraph Sandbox["JavaScript 沙箱环境"] Exec["沙箱安全执行"] Math["高密集度本地计算\n(无需消耗推理 Token)"] TCall["直接并发调用外部 Tools\n(免去逐次 Round-trip)"] end P --> Code Code -->|"代码注入"| Exec Exec --> Math Exec --> TCall Exec -->|"合并结果返回"| LLM ``` - **Programmatic Tool Calling 核心收益**：将计算与工具编排转移至沙箱，规避了思维链中的 token 浪费和高延迟轮询。 - **缓存位置敏感性**：高频变动变量（如时间戳）必须置于提示词末尾，避免长达数千 token 的静态前缀频繁爆仓失效。 - **Persistent Reasoning 机制**：将完整的 reasoning turns 沉淀并跨请求继承，兼顾模型上下文延续性与底层缓存命中率。 --- ### [18:24](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1104s) 优化实验室功能演示：Prompt Caching A/B 测试 在介绍完理论特性后，Charlie 切回实验室演示界面，准备针对刚才梳理的 API 功能开展现场 A/B 测试对比。在视觉矩阵后台，刚才提交的“水上摩托熊猫”已全数跑完，展现出了不同模型对 Prompt 解析的逼真度层次。随后 Charlie 切换到测试实验室的第三个功能标签页，准备发起针对 Prompt Caching 的实际调用，后台通过注入一段约 5,500 token 左右的预置长提示词，来实测命中前后在延迟与计费成本上的直观差距。 ### [19:12](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1152s) Harness 价值最大化实测：Prompt Caching 与编程式工具调用 在深入企业落地案例之前，Charlie Guo 通过代码实操展示了智能体 Harness 中几项关键技术的量化收益。首先测试的是 Prompt Caching（提示词缓存）。在设置了一个数千 Token 的大型 Prompt 后，系统先后发起了两次相同的 API 请求来对比缓存命中前后的差异。结果显示，启用缓存后，输入成本直接下降了 90%。尽管首屏或偶尔请求的等待时间因缓存建立稍微慢了两三秒，但 Charlie 指出，如果业务场景不追求极致的单次毫秒级延迟，且存在大量可复用的长上下文（例如极其庞大的系统提示词 System Prompt），那么利用提示词缓存是削减模型成本最有效、立竿见影的手段。 紧接着是编程式工具调用（Programmatic Tool Calling）的对比演示。Charlie 构建了一个模拟业务场景的真实任务链：模型需要列出所有账户、追踪各账户的使用量，并自动创建一张工单。在常规工具调用链路中，模型需要发起工具调用请求，等待外界执行并将结果返回给上下文，模型理解后再发起下一次调用，产生多次往返交互。而在编程式工具调用模式下，GPT-5.6 可以直接向 JavaScript 沙盒中注入代码，由沙盒在本地密集执行工具调用、统计汇总数据，并将最终清洗好的结果一次性返回给模型，最后由模型给出答复。 实测数据显示，使用编程式工具调用使得输入 Token 消耗减少了 24%，并直接省下了一整轮（turn）的模型推理交互。这种节省不仅体现在输入端，更波及输出端——因为数据计算与合并都在沙盒内部完成，模型无需在输出中进行冗长的数据对齐与中间推理。而且经过两端答案比对，优化后的链路产出了完全一致的目标结果。 ```mermaid flowchart TD accTitle: Conventional vs Programmatic Tool Calling accDescr: Comparison of conventional multi-turn tool calling and single-turn programmatic JavaScript sandbox tool calling. subgraph Conventional["传统工具调用流程（多轮往返）"] direction TB C1["模型思考"] --> C2["调用工具 1"] C2 --> C3["上下文回传并思考"] C3 --> C4["调用工具 2 & 统计"] C4 --> C5["生成最终回复"] end subgraph Programmatic["编程式工具调用（沙盒执行）"] direction TB P1["模型生成 JS 代码"] --> P2["注入 JavaScript 沙盒"] P2 --> P3["沙盒直接调工具并计算"] P3 --> P4["返回紧凑数据至模型并作答"] end ``` - **Prompt Caching 成本削减**：输入 Token 成本立减 90%，非常适合庞大且复用率高的系统提示词。 - **编程式工具调用收益**：减少 24% 的输入 Token，节省 1 轮完整模型轮次交互，同时降低模型输出端用于中间推理的 Token 损耗。 --- ### [21:27](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1287s) 上下文压缩实测：以极低 Token 维护长轮次历史 第三个现场实验展示了 Context Compaction（上下文压缩）的威力。在此案例中，智能体经历了一个包含多达 72 轮对话交互及任务交接（Hand-offs）的超长上下文历史。Charlie 针对这一长历史对比了两种方式：一种是直接带着全部历史向模型提问，另一种则是先触发一次上下文压缩机制（Compaction），随后再请求答案。 测试结果极为震撼：经过压缩后，处理相同任务所需的输入 Token 剧降了 82%。在未压缩状态下，该请求每次需要向 API 发送多达 24,000 个输入 Token；而在触发压缩后，输入体积瞬间压减到了略高于 4,000 个 Token。虽然执行压缩本身在后台耗费了不到 30 秒的处理时间，但在完成单次压缩后，后续所有的问答推理速度均有所提升，并且最终输出与未压缩前的答案丝毫不差。Charlie 提醒开发者，在自身业务面临大量推理 Token 压力时，可以通过这类压缩调优来判断速度、耗时与成本的最佳平衡点。 - **压缩效果**：输入 Token 从 24,000 个缩减至 4,000 余个，压缩率达到 82%。 - **时延代价**：进行上下文压缩需要一次性耗费约 30 秒的后台压缩时间，但能极大地解放后续请求的吞吐量与成本。 --- ### [23:00](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1380s) 客户聚焦：Ploy 创始人 Bryant Chou 亮相 在讲完理论与基础演示后，Charlie 强调不仅要看 OpenAI 的实验室指标，更要检验模型在实际生产级系统中的表现，随后将舞台交给了本场的重磅嘉宾——来自 Ploy 的创始人 Bryant Chou 和 AI 工程师 Lorenzo Gentile。Ploy 前不久在 Hacker News 首页发布了一篇关于将生产级智能体全面迁移至 GPT-5.6 Sol 的深度技术实录，引起了开发者社区的轰动。 Bryant 拥有深厚的产品与架构背景，他曾是知名无代码建站巨头 Webflow 的联合创始人兼 CTO，并在那里工作了 12 年半。8 到 9 个月前，随着 AI 与 Agent 技术的爆发，Bryant 意识到网站不该仅仅停留在静态设计，而是应当为企业增长提供更强大的原生动能，于是创立了 Ploy。Ploy 的愿景是将营销网站转变成企业“最勤奋的员工”——不仅提供网站托管、SEO 与 AEO（AI 搜索引擎优化），还能通过智能体自动识别匿名访客、匹配目标画像（ICP）、调取 CRM、撰写并发送跟进邮件，形成一条闭环的自主营销漏斗。 更为关键的是，Ploy 打造了独有的“Slurp”引擎：用户只需输入网址，系统能在 60 秒内完整爬取网站的所有 CSS 及各个响应式断点，以确定性算法直接将其逆向重构为一个设计系统，使 AI 生成的页面具备极高水准的设计品味，消除俗气的“AI 味”。在刚结束的上一期 Y Combinator 训练营中，已有约 13% 的 YC 初创公司成为了 Ploy 的客户。 - **Bryant 履历**：Webflow 联合创始人兼前 CTO，拥有 12.5 年建站产品技术经验。 - **Ploy 核心功能**：60 秒内将任意网站逆向提炼为设计系统（Slurp 机制），实现 SEO/AEO 优化、匿名访客画像去匿名化以及自动化营销转化。 - **市场渗透**：上一期 YC 创业营中有 13% 的项目正在生产环境中使用 Ploy。 --- ### [28:11](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1691s) Lorenzo 登场：拆解多步智能体中的 $O(N^2)$ 成本黑洞 紧接着，Ploy 的核心 AI 工程师 Lorenzo Gentile 接过麦克风，开启了关于底层工程实战的“价值最大化”拆解。Lorenzo 自称在加入 Ploy 后彻底迷上了 AI 工程，并极度痴迷于系统调优与成本削减。他表示，早在决定将生产环境迁移至 GPT-5.6 之前，Ploy 就已经对智能体循环做了大量的优化积累，而这些前期工程资产在迁移至 GPT-5.6 时发挥了放大器般的威力。 在前面 Charlie 提到的上下文压缩、高效工具调用、编程式调用以及缓存这四种核心技术中，Lorenzo 认为 Prompt Caching（KV 缓存）不仅最为重要，而且最容易被开发者低估和忽视。他从第一性原理分析了标准 Agent 循环的致命软肋：每当智能体决定下一步动作时，它都必须完整处理之前积累的所有上下文 Token。如果一个复杂的智能体任务跨越 10 个步骤、调用了 10 次工具，随着上下文历史在每一步单调递增，整个交互过程中所处理和计费的 Token 总量实际上会呈现出极具破坏性的 $O(N^2)$（二次方）级别爆炸增长。如果没有精准的缓存控制，多步执行的生产级智能体将迅速演化成吞噬预算的成本黑洞。 - **Lorenzo 背景**：Ploy AI 工程师，专注于极致的系统性能提升与模型成本缩减。 - **多步智能体机制陷阱**：在缺乏优化的情况下，智能体随执行轮次递增不断重新读取全部前序历史，导致整体 Token 处理量呈现 $O(N^2)$ 级累加。 ### [30:15](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1815s) 提示词缓存与保持上下文“纯追加”的机制 启用提示词缓存（Prompt Caching / KV Cache）能够带来革命性的成本效益：对于已经被模型处理过的输入 Token，由于 OpenAI 等模型提供商保存了这些 Token 对应的 KV 向量缓存，调用成本可直接实现约 10 倍（10x）的断崖式降低。在超长上下文场景下，缓存不仅大幅缩减开销，还能极大改善模型的首字与整体响应速度。 然而，缓存机制极其脆弱。为了让缓存能够逐层累积并长期生效，整个上下文窗口（Context Window）在逻辑上必须严格保持“仅追加”（Append-only）。很多团队在尝试“自作聪明”地节省 Token 时反而踩入致命误区——比如动态按需调整工具列表。因为工具 Schema 通常位于上下文窗口的最顶端，如果随着 Agent 循环轮次的推进动态增删或修改工具定义，整个缓存链条就会在最前端被彻底打碎（Cache Break），迫使后续所有重复内容重新计费。正如 Charlie 之前所强调的，若在 System Prompt 中动态插入当前时间戳（Current Time）等易变信息，也会立即破坏 KV Cache。 ```mermaid flowchart TD accTitle: Context Window Cache Preservation accDescr: Diagram illustrating how append-only structure protects prompt caching while dynamic updates break it. subgraph BrokenPattern["破坏缓存的错误做法"] B1["可变的 System Prompt / 动态工具<br/>（时间戳、即时变更的定义）"] -.->|"头部被修改<br/>Cache Miss"| B2["整个后续上下文必须全部重新计算<br/>（成本暴增 10x）"] end subgraph OptimalPattern["纯追加式的缓存保护策略"] O1["静态 System Prompt 与 Core Tools<br/>（设置缓存断点，保持完全不可变）"] -->|"稳定命中缓存"| O2["历史对话与追加内容"] O2 --> O3["末尾动态挂载按需工具<br/>（Append-only，绝不污染前序缓存）"] end ``` --- ### [32:05](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1925s) 按需工具模式与断点缓存工程实践 为了在不破坏 KV 缓存的前提下动态收紧上下文，Ploy 团队设计了一套“核心工具 + 按需工具（On-demand Tools）”的加载架构： 1. **核心工具（Core Tools）**：根据生产环境的使用频率统计，将出现概率超过 70% 的高频基础工具固定在上下文最顶部，作为始终加载的不可变底座。 2. **按需工具（On-demand Tools）**：对于使用频率较低的特定领域小众工具，仅声明一个名为 `on-demand tools` 的调度元工具。当 Agent 判断需要使用特定小众能力时，动态调用该元工具，把对应工具的完整 Schema 格式化并**追加到上下文末尾**。这样既能按需赋予 Agent 新能力，又完全不会影响前半段累积的大规模缓存。 在实际评测中，这套策略将工具 Schema 占用的 Token 总体缩减了 45%，并在 Eval 体系中直接转化为 33% 的端到端成本降低。 除了工具解耦，利用各厂商提供的断点（Breakpoints）显式指定缓存节点也是拉高命中率的核心利器。当同一个用户在平台内开辟多个不同的会话窗口（Chat）时，通常都共享相同的 System Prompt 和核心工具定义。在此处设置缓存断点后，跨会话无需再重复计算这部分基座上下文。 此外，Ploy Agent 设计了在跨聊天会话间保持持久化的“工作区记忆”（Workspace Memory）。团队在工作区记忆的末尾设置了第二道缓存断点：在多个会话共享同一项目记忆且未发生更新时，该断点之上的所有内容都能直接享受缓存命中。该机制使得新会话第一条消息的输入 Token 成本骤降 89%，在 Ploy 的生产环境中带来了整体 5% 的净支出缩减。 --- ### [35:58](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2158s) 工具调用的批处理与精简设计 针对代码生成类 Agent（Coding Agent）频繁发起工具调用的场景，合并与批处理工具调用是一个收益极高却常被忽视的抓手。假设 Agent 需要独立执行三个操作：“编辑文件 A”、“读取文件 B”、“读取文件 C”。如果拆分成三次串行调用，模型不仅要在三个轮次中重复处理整个上下文输入 Token，还会额外消耗三次用于决策的思考 Token（Thinking Tokens）。 优化方案分为提示词引导与工具接口改造两个层面： - **Prompt 引导批量输出**：在 Prompt 中明确要求 Agent 在单步决策内合并输出互不依赖的多个工具调用，一次性返回并执行，消除冗余的思考步数与上下文回放。 - **复合工具设计（Multi-action Tools）**：将高频的单一原子操作工具重构为支持复数参数的接口，例如直接调用支持一次性读取多个文件的接口（如 `read_files([B, C])`），从而进一步缩减输出 Token。 在保持相同评估通过率的前提下，仅批处理优化就带来了 14% 的成本节省，并伴随端到端执行延迟的显著下降。 在工具数据层面的工程瘦身同样立竿见影： - **搜索摘要（Highlights）**：针对网页检索依赖的 Exa 搜索接口，开启 Highlights 特性，仅截取核心要点输出，使搜索工具的响应体数据量瞬间降低 70%，预估每年可为 Ploy 节省 37,000 美元的 Token 支出。 - **确定性逻辑下沉**：严格梳理 Agent 的职责边界，将原本需要模型介入二次编排的中间逻辑转化为代码层面的确定性流水线（Deterministic），省去工具到 Agent 再到下一个工具之间的长链路往返（Round-trip），在特定场景下实现了 54% 的 Token 降幅。 --- ### [39:40](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2380s) 创始人视角：二次创业者面对模型迭代的认知迭代与 Q&A 演讲结束后，OpenAI 团队重新接入画面并进入现场问答环节。主持人 Charlie 首先将问题抛给 Ploy 创始人 Bryant Chou：作为一名成功的二次创业者，在面对底层大模型日新月异的技术浪潮时，是如何思考全面迁移至 GPT-5.6 的决策逻辑与权衡的？ Bryant 坦言，初创公司永远在寻找某种“Alpha”（超额优势与壁垒）。但在 AI 工具的开发世界里，唯一的生存法则就是亲自去跑、去死磕市面上的每一个全新模型。Ploy 花费了大量心血构建能够高度贴合真实商业场景的 Eval 评估体系——从复杂的营销策略梳理、长距离指令遵循，到整体网页重构与导入整个 WordPress 站点的极端复杂用例，所有能力都被严密压测。 Bryant 笑着自嘲道，身为“二次创业者”在当前的 AI 时代甚至可能是一种“劣势”：因为技术演进实在太快，过去积累下来的大量经验和思维定势反而必须被主动“忘掉”（Unlearn），必须彻底放空心态去迎面接受“海量知识的冲击灌溉”（Drinking from the firehose），而这种高烈度的认知刷新过程其实充满了乐趣。 --- ### [41:36](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2496s) 解构 GPT-5.6 的推理级别：Max、Pro 与 Ultra 的本质差异 在进入技术提问环节后，观众抛出了一个高度关注的问题：在将任务委派给子代理（Sub-agent）并使用 GPT-5.6 的 Ultra Effort 模式时，Token 消耗似乎极高。子代理到底是仅接收当前任务所必需的局部上下文，还是保留并继承了当前对话与工作区的完整上下文？ OpenAI 团队成员对这个问题表示极度赞赏，指出当前社区对 GPT-5.6 推理努力等级（Reasoning Effort）中的 Max、Pro 以及 Ultra 存在大量混淆。为澄清概念，团队详细拆解了这些档位的设计定位： | 推理模式 / 档位 | 特征与核心机制 | 适用定位与资源调度 | |:--- |:--- |:--- | | **Max** | GPT-5.6 引入的最新推理努力级别，突破了原有的 Extra High 上限。 | 彻底放开模型的内部推理约束，相当于“解开缰绳”（Takes the leash off），允许模型根据问题复杂度自由消耗尽可能多的思考 Token。 | | **Pro** | 具备结构化调控特征的专用推理模式，与开放式的 Max 机制形成鲜明对比。 | 针对深度专业化计算任务提供受控的高强度推理支持。 | | **Ultra** | 结合了大规模上下文继承与极端深度思考的高阶执行模式。 | 在多 Agent 委派任务时若未做上下文隔离，极易引发上下文与思考 Token 的双重膨胀。 | ### [42:38](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2558s) GPT-5.6 Pro 参数迁移与 Ultra 模式的子代理机制 在回答关于模型版本与推理模式的提问时，Charlie Guo 澄清了历史遗留命名与 API 架构的新调整。以往用户在 ChatGPT 中经常能看到“Pro”模式选项，而在 Codex 或部分开发者接口中却未必明显。很多重度 API 开发者可能已经留意到：API 列表中根本找不到类似 `gpt-5.6-pro` 这种专属的模型 Slug。这并不是模型被取消了，而是 OpenAI 将其重构成了一个独立的 `reasoning_mode` 参数。开发者可以选用任何一款 GPT-5.6 模型（如 Sol、Terra、Luna），并在其推理模式参数中选择 `standard` 或 `pro`。选择 Pro 模式后，能获得与网页端 Pro 完全相同的效果，但其代价也相当直观——模型会消耗显著更多的推理 Token，生成完整答案的耗时也会大幅拉长。 紧接着，Charlie 深入拆解了位于 Max 推理之上的全新模式——Ultra。严格来说，Ultra 并不是一个全新的推理等级，而是一个叠加在 Max reasoning 之上的运行模式。在 Codex 或 ChatGPT 工作流中，Max 和 Ultra 默认均处于未开启状态。OpenAI 刻意没有“开箱即用”地提供它们，主要是为了提醒用户时刻关注自己的 Token 账单与开销；如果想要使用，必须在设置菜单中手动开启。Ultra 的运作机制在于两点结合：它首先继承了 Max reasoning 允许按需消耗海量推理 Token 的特性，同时进一步修改底层 System Prompt，极具侵略性地驱使模型去派发更多子代理（subagents）。GPT-5.6 是 OpenAI 训练出的第一款原生具备自主评估并派发子代理能力的模型。以往在 Codex 中，用户必须硬编码或在 Prompt 中显式指令“对于这个任务请使用子代理”，它才会照做；而 GPT-5.6 则能够自主判断当前任务是否需要分发子代理。Ultra 模式则是将这种分发倾向推向极致。 Charlie 特别形象地打了个比方：Ultra 模式绝对是极度消耗 Token 的“重炮手”，必须留给真正极其繁重的复杂工程场景。如果你仅仅是用它来查收邮件或写个日常通知，那无异于“拿着火箭筒去打小刀战”（bringing a bazooka to a knife fight），火力严重过剩且毫无必要。至于子代理与上下文隔离的问题，Charlie 明确表示，主代理分发任务时仅会向子代理注入其执行任务所必需的上下文，任务完成后再返回结果；这与 Codex 里的“side chat”（侧边对话）完全不同，侧边对话会继承并携带当前主聊天的全量上下文。随后 Lorenzo Gentile 从工程师实现视角补充道，让子代理拥有全量上下文在工程上类似“分叉上下文窗口”（forking the context window），但在绝大多数落地场景中，让编排代理（orchestrator agent）提炼出所需背景的简要 Summary 并传递给子代理，是目前更经济、高效的方案。 ```mermaid flowchart TD accTitle: 子代理调度与上下文隔离流转图 accDescr: 展示主代理如何基于 Ultra 模式调度子代理，以及与全量上下文 side chat 的机制对比。 subgraph Orchestration ["GPT-5.6 编排层"] A["用户复杂任务需求"] --> B["GPT-5.6 主编排代理 (Orchestrator)"] B -- "开启 Ultra 模式" --> C{"自主评估任务可拆解性"} end subgraph Subagents ["任务执行层 (最小必要上下文)"] C -- "分发子任务 A (精简 Summary / 最小上下文)" --> D["子代理 A"] C -- "分发子任务 B (精简 Summary / 最小上下文)" --> E["子代理 B"] D --> F["返回结果 / Patch"] E --> F end subgraph SideChat ["对照机制 (全量上下文)"] B -. "Codex 侧边对话 (Side Chat)".-> G["携带当前主会话完整上下文"] end F --> H["合并汇总并生成最终解答"] ``` ### [45:52](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2752s) 代理深度权衡：纵向多层派发 vs. 横向扁平并行 针对“代理委托深度（子代理继续拉起子代理）对性能有何影响、何时该深何时该浅”的问题，Charlie 和 Lorenzo 分享了各自在工程实践中的经验。Charlie 直言，从个人经验看，只要委托层级超过两层，效能通常就会显著衰退。因为在层层向下转述与委托的过程中，原始上下文与约束会发生严重的语义丢失。因此，子代理最能发挥威力的场景是任务能够高度扁平化、高度并行化的架构——各个子任务之间无需共享复杂状态，不会同时修改同一批文件，彼此之间拥有清晰独立的执行泳道。 Lorenzo 对此深表赞同，并给出了 Ploy 在生产系统中的血泪教训。Ploy 团队探索子代理已有相当一段时间，但就在三个月前，结合当时主流模型的表现，他们甚至得出过一个极端结论：在成本、性能与响应时间的综合权衡下，完全不使用子代理反而表现更好。其根源在于“把任务从一个 LLM 委托给另一个 LLM 本身就是一个极其损耗信息（lossy）的过程”。理想情况下，最上层的任务拆解与方向把控应该尽可能由人类专家规则或强专家模型主导。虽然 GPT-5.6 Sol 的编排能力相比旧模型已经有了质的飞跃，但多层深度的风险依然很大。如果调度链路是“Sol 调度 Sol，再调度 Sol”，或许还能维系基本逻辑；但如果链路变成了“Sol 调度 Terra，Terra 再调度 Luna”，在模型能力断崖下，深层子代理的编排与执行效率极易崩溃。Lorenzo 最终总结道，对于多层代理嵌套，“务必谨慎行事（proceed with caution）”。 - **代理层级衰减律**：超过两层代理委托，上下文损耗急剧增加，任务漂移风险显著上升。 - **理想并行场景**：无共享状态（stateless）、无重叠文件修改冲突、具备独立隔离执行泳道的任务最适合横向子代理。 - **异构模型嵌套风险**：高阶模型向下委托给低阶轻量模型时，若轻量模型自身还需承担复杂二级编排，极易出现任务崩溃。 ### [48:41](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2921s) 新开会话 vs. 上下文压缩（Compaction）：长上下文的工程取舍 关于“在大型项目或代码仓库中，究竟是开启新会话更好，还是依赖系统的上下文压缩机制（Compaction）”，Charlie 提出了一个稍微带有一点“争议性”（spicy take）的观点。放眼一年前，他是坚定的“每做一件新事就必开新会话”派，以此竭力避免上下文污染；但随着模型理解力和压缩机制的演进，他现在已经不再频繁纠结于压缩问题，在 Codex 中甚至常常让一个会话线程持续运行数天乃至数周。然而，在一种特殊情况下开启全新会话仍然至关重要：当开发者的个人配置本身就极为沉重时——例如一开始就加载了庞大的背景规范、挂载了十几个插件、配置了大量技能或附加了多个参考文件。当系统必须让模型在每一次输出时都死死记住这些全量前置信息时，上下文污染的几率依然存在，此时重新开辟干净会话远比强行压缩更有保障。 Lorenzo 则从产品与用户体验（UX）的切身观察切入探讨了这个痛点。在 Ploy 的实际监控中，团队经常发现有用户在单个长会话中持续工作，甚至在改完网页代码后，紧接着就在同一个会话里抛出一个风马牛不相及的日常搜索提问。这种在长文本历史下进行无关查询的行为完全是在白白浪费 Token。这由此演化成了一个全新的 AI UX 难题：系统究竟应该把重置对话的决定权完全交给用户，还是应该由后台 LLM 静默检测意图漂移并自动切换会话？这依然是 Ploy 团队正在深入攻关的设计课题。 ### [50:31](https://www.youtube.com/watch?v=jyuyY86GJnA&t=3031s) 缓存依赖的反思与 Compact 机制的底层统一性 在问答环节的尾声，观众抛出了一个深度架构问题：本次分享充分展示了 Prompt Caching（提示词缓存）带来的巨大红利，那么过度依赖缓存是否存在负面陷阱？Lorenzo 对此做出了客观剖析。他强调，缓存在价值最大化（Valuemaxxing）的体系中属于后置手段，必须建立在充分的 Traces（追踪日志）审计之上。在盲目给系统套上缓存之前，开发者首先应该审视： 1. 某些工具调用（Tool Call）是否输出了超大规模的不必要 Token？ 2. 上下文窗口中是否存在大量重复、冗余的数据输出？ 3. 是否逼着 Agent 去做那些原本写几行确定性原生代码就能解决的事情？ 只有把这些基础工程治理好，解决掉前 80% 的问题，引入缓存才能真正放大价值。Charlie 进一步补充道，缓存机制虽然强大，但过早且盲目地介入反而会使整个架构变得极其脆弱（brittle），导致开发人员在调整业务逻辑时束手束脚、受制于缓存键的设计。引入缓存的黄金时机，是在你对自己应用中哪些路径是确定性受益点有了清晰透彻的掌控之时。 最后，针对“Compact 压缩功能在不同模型或不同推理等级下是否运行机制不同”的技术细节，Charlie 透露，其底层的 API Endpoint 逻辑是完全统一的。虽然部分核心技术细节属于内部“独家秘方”（secret sauce）无法完全公开，但其本质原理都是一致的：系统提取当前的对话历史，将其重新编码并映射为一种紧凑表征（representation），使底层模型在后续调用中依然能够精准理解并复用该上下文。尽管在不同模型与推理等级下，最终被编码保留的信息颗粒度可能会根据对话的具体特征产生细微差异，但从底层运行机制与接口契约来看，它在所有环境下的表现是完全标准化且一致的。 ### [53:05](https://www.youtube.com/watch?v=jyuyY86GJnA&t=3185s) 总结致谢与核心开发资源盘点 在问答环节接近尾声时，讲师 Charlie Guo 对来自 Ploy 的联合创始人 Bryant Chou 与 AI 工程师 Lorenzo Gentile 的深度分享表达了由衷感谢。他指出，两位嘉宾结合实际业务踩坑与优化经验所带来的见解极具启发性，特别是对正在探索如何高效落地大语言模型的初创团队创始人及一线开发者而言，提供了极具价值的实战参考。 为了帮助开发者在会后能够快速落地并动手实践，Charlie 系统梳理了本次 Build Hour 提及的关键学习资源与开发文档。针对观众在聊天区频繁询问能否亲自运行本次演示 Demo 的问题，Charlie 给出了明确肯定的答复：Build Hour 中使用的全部代码均已开源在官方代码仓库中。此外，官方还会提供完整的录播视频点播，并通过邮件将所有文档与代码链接汇总分发给参会者，无需担心遗漏细节。 - **GPT-5.6 发布博客**：深入解析 GPT-5.6 系列模型（Sol、Terra、Luna）的架构亮点、智能水平以及性价比优势。 - **编程式工具调用指南（Programmatic Tool-Calling Guide）**：指导开发者如何在 Harness 和 Agent 工作流中更高效、精准地对接外部工具与结构化输出。 - **模型对比工具（Compare Models）**：用于在不同任务场景中权衡智能水准、延迟表现及成本消耗。 - **Ploy 生产级智能体迁移实战博文**：详尽记录 Ploy 如何完成向 GPT-5.6 的生产级迁移，实现构建速度提升 2.2 倍、成本降低 27% 的真实案例分析。 - **Build Hour 官方开源代码库**：包含本场演示所使用的全部配套代码与 Agent 测试脚本，方便开发者直接克隆并本地调试。 ### [54:00](https://www.youtube.com/watch?v=jyuyY86GJnA&t=3240s) 下期预告与活动收尾 在活动临近尾声时，Charlie 预告了下一期 Build Hour 的筹备安排。下期活动将聚焦于图像生成领域的前沿进展（围绕 ImageGen 2 展开），帮助开发者掌握最新视觉生成模型的特性与工程化集成技巧。最后，团队鼓励广大开发者持续关注活动页面、订阅后续技术排期，并积极反馈希望在未来活动中探讨的技术议题与实战痛点。伴随着全体嘉宾的致谢与道别，本期关于 GPT-5.6 价值最大化的 Build Hour 圆满落下帷幕。 ## Notable quotes > [00:01](https://www.youtube.com/watch?v=jyuyY86GJnA&t=1s) 把每一分钱和每一个 Token 都花在刀刃上 — Christine > [02:50](https://www.youtube.com/watch?v=jyuyY86GJnA&t=170s) If you doubled your token spend tomorrow, how would you know if it was worth it? — Charlie Guo > [04:31](https://www.youtube.com/watch?v=jyuyY86GJnA&t=271s) How do you define what good looks like? — Charlie Guo > [12:24](https://www.youtube.com/watch?v=jyuyY86GJnA&t=744s) 目前不建议在 Codex 中直接使用 full access — Charlie Guo > [42:38](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2558s) 拿着火箭筒去打小刀战 — Charlie Guo > [39:40](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2380s) Drinking from the firehose — Bryant Chou > [45:52](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2752s) 把任务从一个 LLM 委托给另一个 LLM 本身就是一个极其损耗信息（lossy）的过程 — Lorenzo Gentile > [45:52](https://www.youtube.com/watch?v=jyuyY86GJnA&t=2752s) 务必谨慎行事（proceed with caution） — Lorenzo Gentile ## Mentioned OpenAI, GPT-5.6, GPT-5.6 Sol, GPT-5.6 Terra, GPT-5.6 Luna, Codex, Charlie Guo, Christine, Bryant Chou, Lorenzo Gentile, Ploy, Webflow, Y Combinator, SWE-bench, Artificial Analysis, Exa, WordPress, ImageGen 2, YouTube, Hacker News, GitHub ## Video description Get more useful work from every token with GPT‑5.6. In this Build Hour, you’ll learn how to choose the right model for your workload—and migrate production agents while improving speed, cost, and performance. Charlie Guo (DevEx), Bryant Chou (Founder, Ploy) and Lorenzo Gentile (AI Engineer, Ploy) will cover how to: • Choose between GPT‑5.6 Sol, Terra, and Luna based on intelligence, latency, and cost • Measure and optimize cost-performance across real workloads • Adapt evals, agent harnesses, tool schemas, and prompt caching when migrating models • Move production agents to GPT‑5.6, featuring a customer spotlight on how Ploy achieved 2.2× faster builds at 27% lower cost • Apply these lessons through live demos and Q&A 👉 GPT-5.6 blog: https://openai.com/index//gpt-5-6 👉 Programmatic tool-calling: https://developers.openai.com/api/docs/guides/tools-programmatic-tool-calling 👉 Compare models: https://developers.openai.com/api/docs/models/compare 👉 Migrating a production AI agent: https://ploy.ai/blog/migrating-a-production-ai-agent-to-gpt-5-6 👉 Build Hour repo: https://github.com/openai/build-hours 00:00 Introduction 02:50 Tokenmaxxing vs. valuemaxxing 07:11 Valuemaxxing Codex 15:16 Valuemaxxing your harness 23:48 Customer spotlight: Ploy’s migration to GPT-5.6 39:57 Q&A 53:13 Resources and closing [Source](https://www.youtube.com/watch?v=jyuyY86GJnA)

[0:01] Christine: 把每一分钱和每一个 Token 都花在刀刃上 [2:50] Charlie Guo: If you doubled your token spend tomorrow, how would you know if it was worth it? [4:31] Charlie Guo: How do you define what good looks like? [12:24] Charlie Guo: 目前不建议在 Codex 中直接使用 full access [42:38] Charlie Guo: 拿着火箭筒去打小刀战 [39:40] Bryant Chou: Drinking from the firehose [45:52] Lorenzo Gentile: 把任务从一个 LLM 委托给另一个 LLM 本身就是一个极其损耗信息（lossy）的过程 [45:52] Lorenzo Gentile: 务必谨慎行事（proceed with caution）

[2:50] Charlie Guo: If you doubled your token spend tomorrow, how would you know if it was worth it? [2:50] Charlie Guo: accidentally burning through their entire annual budget for AI in a handful of months [41:40] Charlie Guo: fully takes the leash off [41:40] Charlie Guo: a little bit like bringing a bazooka to a knife fight [35:59] Lorenzo Gentile: win across the board [39:57] Bryant Chou: drink from the firehose [48:41] Charlie Guo: spicy take [53:05] Charlie Guo: super insightful [54:00] Charlie Guo: Thanks, everybody

[0:01] Hi everyone, welcome back to OpenAI [0:03] Build Hours. I'm Christine on the [0:04] startup marketing team and today I'm [0:06] here with Charlie. [0:07] >> Hi, I'm Charlie from the developer [0:08] experience team. [0:10] >> So today our topic is going to be about [0:12] value maxing with GPT-3.5. [0:15] So next slide. If this is your first [0:17] time with us, uh the whole purpose of [0:18] Build Hours is to empower you with the [0:20] best practices, tools, and AI expertise [0:22] to scale your company using OpenAI APIs [0:24] and models. Uh down below is our [0:27] homepage. So we're constantly posting [0:28] new sessions as they come up. Feel free [0:30] to check back and you can also catch up [0:32] on demand. We also post all of these [0:34] sessions on YouTube um on the day of. So [0:37] check back later in case you want to [0:39] review anything that we're chatting [0:40] about today. [0:44] So we've traditionally started every [0:46] Build Hour with a meme and we're back. [0:50] Uh so if we have any Office fans around, [0:52] um this is really about just getting [0:55] more value per token um and Charlie's [0:59] going to walk you through uh some tips [1:00] and tricks on how to do this. [1:03] So this is our first Build Hour after [1:06] GPT-3.5 release. Um so right after we [1:10] released this model, we asked everyone [1:13] on X kind of why they love this model um [1:16] and we were really just like humbled and [1:18] and also like taken away by all the [1:20] reasons that people submitted. And I [1:22] think a common theme that we saw was [1:24] just how token efficient this model is [1:27] particularly in just having you guys get [1:29] the most bang for per token. Um and [1:33] recently actually, I think this is two [1:35] days ago on X, uh we reached a new [1:38] milestone. Um so we're we're really [1:40] happy to see that this is resonating [1:41] with people um and that this new model [1:44] um [1:44] is is letting people do more with with [1:47] less tokens. [1:50] So to give you a snapshot of what we're [1:51] talking about today, the first is we're [1:54] going to explain this shift that's gone [1:55] from token maxing to value maxing. [1:58] Um we're going to show you value maxing [2:00] in Code X and then also value maxing [2:02] your harness. And then I'm really [2:04] excited that we will have Floy, um one [2:06] of our favorite startups uh join today [2:09] to chat about how they've actually [2:10] migrated their AI agent over. [2:13] And last but not least is Q&A. Um so on [2:15] the right side of your screen, if you [2:17] toggle over to the Q&A button, you can [2:19] actually submit questions and our team [2:22] will be in the room um answering them on [2:24] chat and then also selecting a few to [2:26] answer live towards the end. And we'll [2:28] have our friends from Floy also join. So [2:31] feel free to ask any questions um [2:33] and we're really excited to kick this [2:34] off. So over to you, Charlie. [2:37] >> Thanks so much, Christine. Um thanks [2:38] everybody for joining. So excited to be [2:40] here. Uh [2:41] I want to talk about value maxing, but [2:43] I'm sure there's you know a lot of [2:44] questions about what what is that, [2:46] right? And I think you can't quite [2:47] explain what it is without first [2:48] starting with token maxing. Um and I [2:51] think token maxing is a term that that [2:52] emerged, you know, earlier in this year. [2:54] Um and to kind of encapsulate it, right? [2:57] It was essentially this this idea of [2:59] measuring progress by how much AI you're [3:02] using. I think that might be how many [3:04] tokens you're burning, how many prompts [3:05] you're sending, how many agents you're [3:06] managing at the same time. And I think [3:09] we even saw companies that had stood up [3:11] internal leaderboards to just track, you [3:13] know, how many tokens their employees [3:14] were using day over day or week over [3:16] week. [3:18] Later on in the year though, I think [3:19] some of that sentiment started to [3:21] change, right? Uh we started to see uh [3:24] you know, companies that were [3:25] accidentally burning through their [3:26] entire annual budget for AI in a handful [3:29] of months, right? And we started to see [3:31] a lot of enterprises start to pull back [3:33] and say, "Well, maybe we should be [3:34] throttling our AI spend." Um or even [3:36] some AI CEOs saying that, you know, some [3:39] companies may have gotten carried away [3:40] with their with their token [3:41] leaderboards. Um and importantly that we [3:43] should be measuring employees on the [3:45] outputs instead, right? and the value [3:47] that they're generating. [3:49] So that brings us to value maxing. In [3:51] contrast with token maxing, it's [3:53] measuring progress not by how much AI [3:56] you're using, but what AI is actually [3:58] helping you to accomplish, right? How [4:00] much work you're getting done, how much [4:02] time are you saving, how much, you know, [4:04] quality are you improving in the code [4:06] and the artifacts that you're making. [4:10] I think a question that I like to come [4:12] back to when I think about, you know, [4:13] this framing of value [4:15] is something like if you doubled your [4:17] token spend tomorrow, how would you know [4:19] if it was worth it, right? You know, [4:21] there is perhaps a [4:23] naive assumption of like, yeah, like we [4:24] spend more on tokens, it's going to be [4:26] better, but, you know, how does that [4:28] work back to to concrete outcomes? [4:31] And I think there are a few questions [4:32] that you can start to ask yourself as [4:34] you're thinking through this for your [4:35] own processes or your own company. [4:37] Things like what are the outcomes that [4:40] you are trying to improve? [4:42] With those outcomes, what are the [4:44] workflows that lead to, you know, good [4:47] quality outputs? Where in those [4:49] workflows can you add intelligence, can [4:51] you add AI, can you add tokens, right? [4:53] And maybe accelerate or improve quality. [4:56] And, you know, what evidence do you have [4:59] that you are improving the quality or [5:00] that you are, you know, using, building [5:02] a working system here, right? A lot of [5:05] this, I think when it when folks work [5:06] with AI, you've probably heard the term [5:08] evals, and I think a lot of AI research [5:10] and engineering these days is driven by [5:12] having good evals, which to boil it [5:14] down, to oversimplify it, basically [5:16] means [5:17] how do you define what good looks like? [5:19] And I think as we move from just having [5:21] to AI, you know, having AI chatbots [5:23] perform simple tasks to having AI agents [5:26] be responsible for entire outcomes, that [5:28] question is still very valuable. What [5:30] does good look like and how do you [5:31] define that and track that within your [5:33] company or your role? [5:37] And I think I think I I I do want to say [5:38] too that, you know, in some cases it [5:40] might seem like the answer here is, [5:42] well, we should spend fewer tokens, [5:43] right? Like we should um our models or [5:46] cut spend, and I'm sure in some cases [5:48] that's that's absolutely true. Um I did [5:50] also want to point out there might be [5:51] some cases where you want to spend more [5:53] tokens and that's actually still driving [5:55] more value, right? Um you may decide, [5:58] "Hey, we we actually want to ensure that [6:00] we have a high-quality result, so we're [6:01] going to bump the reasoning level on the [6:04] model um or switch to a bigger model." [6:06] You might say, um "It's important that [6:08] this gets done faster, so we are going [6:10] to spend more on tokens now in order to [6:13] save uh you know, real-world all clock [6:15] time." Um that might be the case. You [6:17] maybe you've got a big migration and you [6:19] want to migrate something into a [6:20] different programming language. Um maybe [6:22] you could do it, you know, in a couple [6:23] of months with fewer tokens or, you [6:25] know, maybe you can do it in a couple of [6:26] weeks and spend some more. Uh similarly, [6:29] if you know your workflows are working [6:31] well and you want to scale them, then [6:32] you can decide to spend more uh in order [6:34] to just scale those outputs if they're [6:35] working well. And last but not least, [6:37] you know, if you want to manage risk, um [6:39] you might have a system where it's [6:40] really important that the output is is [6:43] high quality and, you know, and [6:44] bulletproof. Um and in that case, you [6:46] may want to do something like use LLMs [6:48] as a judge, um and you may want to use [6:50] more or use a diverse range of them, [6:52] right? In order to ensure that it's [6:53] catching uh all sorts of different edge [6:55] cases. [6:58] Okay. So, with that in mind, I want to [7:00] talk about two ways of thinking about [7:02] value maxing here. I think one is as an [7:04] individual developer while you're using [7:06] Codex. Uh and I think another is uh as [7:09] an organization or as a company that's [7:11] building on top of OpenAI's APIs. [7:13] We're going to start with Codex. [7:17] Um [7:17] like Christine mentioned, I think this [7:19] the story of of value maxing in Codex uh [7:21] really starts with the GPT-3.5 family, [7:24] right? Um if you're not familiar, we've [7:26] got three new models in this family: [7:28] Soul, Terra, and Luna. Uh Soul is, you [7:32] know, our frontier model, our our um [7:34] flagship model for the most complex [7:36] coding and professional tasks. Terra is [7:39] a really great daily driver um and I [7:41] think it is, you know, good for uh for [7:43] more balanced work where intelligence [7:45] and cost or latency both matter. [7:48] And Luna, you know, is we really like [7:50] and we've gotten a lot of positive [7:51] feedback around it for high-volume [7:53] workloads where, you know, the the [7:55] intelligence level is not as demanding, [7:57] but you really do care about cost and [7:59] latency. [8:02] And I think we've we've put a lot of [8:04] work Open AI has put a lot of work into [8:05] making these models, especially 5.6, [8:08] token efficient. [8:10] If you kind of look at this is Deep [8:13] Suey, a benchmark popular coding [8:15] benchmark that's out there. And if you [8:17] take a look at, you know, the [8:18] performance on the benchmark relative to [8:21] how many tokens you're spending, you can [8:24] see that like the most bang for your [8:26] token, as Christine mentioned, is is [8:28] coming from the 5.6 family. And it's a [8:31] mix of different models at different [8:33] reasoning levels. So, I think as as as [8:36] we've gotten better models and as we've [8:38] as we've gotten them to run for longer, [8:40] one of the most important things to [8:41] think about is not just how many tokens [8:44] does it spit out or even like what is [8:46] necessarily the the highest amount of [8:48] intelligence that it has. I think we're [8:50] finding that you can take a model that [8:52] is maybe not as smart at one shotting [8:54] something, and if you're willing to [8:55] spend more tokens on it, you can get [8:57] similar performance. So, I think as you [8:58] evaluate some of this type of work, it's [9:01] useful to take a look at not just cost [9:03] per token, but cost per overall task to [9:06] completion, right? And [9:09] you can you can start to like fiddle [9:10] with these and and see how different [9:12] things compare along different axes. [9:15] Speaking of comparing along different [9:16] axes, I want to jump into a demo to [9:18] explore a little bit around these models [9:21] and their reasoning levels to kind of [9:22] get an an idea of you know, what we're [9:25] doing. [9:30] So, this is a value maxing lab that I [9:31] built with the help of GPT 5.6 and [9:34] Codex. We're going to make sure this is [9:36] open source after the build hour today, [9:38] so everybody's going to have access to [9:39] this code and you can run this yourself. [9:41] Um, [9:42] here just on this first page, uh [9:45] I had 5.6 build a three-dimensional [9:48] chart comparing, you know, a lot of the [9:50] leading models in the artificial [9:52] analysis intelligence index. Um, [9:55] and you can take a look at how they kind [9:57] of compare along these different axes of [9:59] intelligence, um, cost, and speed, [10:02] right? And how often how long it [10:04] actually takes to to generate these [10:05] outputs. [10:07] Um, but I think the more interesting [10:09] thing here is this visual matrix, uh [10:12] where um [10:14] if you supply an API key, like I've done [10:16] here, um [10:17] I actually I'm going to render a number [10:20] of SVGs along the different models and [10:22] the different reasoning modes, so we can [10:24] kind of see what what the output looks [10:26] like. So, um [10:28] let's just say instead of a pelican [10:30] riding a bicycle, I want a [10:33] panda bear, [10:34] uh [10:36] riding a jet ski. [10:39] And I've selected all of them. Let's go [10:41] ahead and run [10:43] the matrix. [10:44] Um, it's going to work in the [10:46] background. I think to start with [10:47] though, I do want to show everybody, um, [10:49] I previously ran this with the pelican [10:51] on a bicycle and you can start to see I [10:54] think the difference in sophistication, [10:58] um, as the models move up, right? And [11:00] even if you take a single model, um, I [11:01] think you can see the difference in [11:03] sophistication, um, along, you know, if [11:06] you just take Luna for example, [11:08] this first model is, I think, you know, [11:11] to be fair is a little bit crude, right? [11:13] Um, [11:14] but as you introduce more reasoning, uh [11:17] the final result is is actually pretty [11:19] good, right? Um, it looks like we're [11:21] getting some results in here, um, but [11:23] you can see like for example Sol starts [11:24] out pretty high fidelity, um, and then [11:27] as it goes up, it just gets even more [11:29] more powerful, right? Obviously, you [11:32] want your own benchmarks, you want your [11:33] own evals to to measure, and I think a [11:35] great product will have this sort of [11:37] thing coming in. Um but uh this is just [11:40] a a nice little demo to kind of out of [11:41] the box take a look at um what is it [11:44] like how does it compare across multiple [11:46] different uh reasoning levels, right? [11:49] And so we can see um [11:52] we can see the run that I just started, [11:53] right? It's coming in and you know [11:55] already uh [11:57] you know even just on a few examples [11:59] here, you can see there's like not much [12:00] shading going on in this first one uh [12:03] versus now we've added like green for [12:05] the islands, uh bigger like more [12:07] detailed waves, uh shading on the sun [12:10] and the clouds in the background, um [12:13] which I think is pretty cool just to be [12:14] able to visually see everything that's [12:15] going on here. [12:22] >> Cool. [12:23] >> Um [12:24] I think some more practical tips and [12:26] tricks when it comes to uh using Codex [12:28] and trying to get the most value out of [12:30] it. Uh [12:31] you know, we saw a little bit of the the [12:33] grid there in terms of how the reasoning [12:34] levels differ, but my rule of thumb here [12:37] is to start with Soul uh and [12:39] specifically start with Soul on medium. [12:41] Um I think it's a really great model, [12:43] and I do want to kind of dispel the [12:45] notion here. I see a lot of developers [12:47] who just assume I have to be on extra [12:51] high for everything out of the box, [12:53] right? Um and I think that is definitely [12:55] true for for sophisticated workloads, [12:57] for complex projects that you're [12:59] tackling, but a lot of day-to-day stuff [13:02] is actually going to be really well [13:03] served by 5.6 Soul on medium. Um I think [13:07] if you're finding it's not quite as [13:09] intelligent as you need, or it's not [13:10] quite as thorough as you were hoping [13:11] for, um you can tune up. Uh and in some [13:14] cases, you might even decide to to tune [13:16] down and to swap to uh the light [13:18] reasoning or even to to 5.6 Tera um if [13:21] the work you're doing is not very [13:22] coding, you know, intensive. [13:25] Um I think the second tip here is to [13:27] consider when you might want to trade [13:29] tokens for time. I think there's like uh [13:31] a number of ways where uh you you might [13:34] just want something to go faster and you [13:36] might want to save your own time in in [13:37] exchange for more tokens. Fast mode is [13:39] the obvious um use case here. You can [13:41] turn on [13:42] fast mode [13:43] uh to run your outputs at 1.5 x speed. [13:47] Though, it will consume your usage [13:48] limits a bit faster. Um, you we highly [13:51] recommend auto approval as the default [13:53] mode um for a number of reasons. I think [13:55] um you know, we we don't recommend using [13:58] full access in Codex at this point. Um, [14:01] and I would you know, I know personally [14:03] and from hearing from a lot of [14:04] developers uh requesting permissions on [14:07] every single action is a little bit [14:09] frustrating. So, um auto approval I [14:12] think is a great sweet spot uh to have [14:14] another model review the uh the outputs [14:17] and make sure that they're safe. [14:20] Um, and Chronicle is a a relatively new [14:22] feature in Codex. Um, and if you turn [14:23] that on, that will allow you to I think [14:25] record your screen and it basically [14:27] develops memory for the task that you're [14:29] doing. So, as you start to do similar [14:32] tasks over and over again, um you can [14:34] you know, use you'll use more tokens to [14:36] to remember what's happening, but it'll [14:38] get smarter about the type of work that [14:39] you do every day. [14:42] Uh the third thing here is um take a [14:44] look at your agent's MD and your skills. [14:46] I think models have gotten um you know, [14:49] very much more sophisticated, much more [14:51] intelligent, and I think a lot of the [14:54] things that we had in our agent's MD and [14:55] our skill instructions 6 or 12 months [14:58] ago um might be unnecessary now. I think [15:00] when it comes to instruction following [15:02] and persistence, um you know, we can go [15:04] back and audit and say, "Hey, do we [15:06] really need to be explicit about every [15:07] single thing in here?" Or is there a way [15:09] to make this concise and save on our [15:11] input tokens? [15:16] So, now I want to switch to talking [15:17] about value maxing your harness, right? [15:19] Or you know, your your product if you're [15:21] building on top of uh our APIs. [15:23] Um there are a handful of I think key [15:26] API features here that um [15:29] we we've recently introduced or or [15:31] somewhat recently introduced to try to [15:33] help uh developers get more value. Um [15:36] and I'm going to talk through a few of [15:37] them here. I think programmatic tool [15:39] calling is a big one. Uh we recently [15:41] introduced this with the launch of 5.6. [15:44] And this is a really cool new feature [15:46] where um to oversimplify a little bit, [15:49] we give the model access to a sandbox, a [15:51] JavaScript sandbox, and it can write [15:54] code to execute in that sandbox. Um that [15:57] code can do things like call tools or [15:58] run computations. Um and you're moving [16:01] all of that, you know, all of that work [16:04] out of the model's chain of thought, out [16:06] of its reasoning, um and into the [16:07] sandbox. And so, you're both saving [16:10] tokens because it's not, you know, [16:11] itself reasoning through the math or the [16:13] logic of something, and you're saving [16:14] time potentially because it can make a [16:16] bunch of tool calls um without having to [16:18] round trip all the way through the model [16:20] and then all the way back uh to the end [16:22] user. [16:23] Um we also have had prompt caching. I [16:26] think that's changed a little bit [16:26] recently with the introduction of 5.6. [16:29] Um but now uh we Yeah, we we have the [16:32] ability for you to specify, you know, [16:34] which parts of your prompts you want to [16:35] cache. Um [16:37] this can be a really great way to to [16:39] save both time and tokens as well. Um I [16:42] think you can do some things um You can [16:44] be also a little bit smart about how [16:45] you're using your prompts. I think one [16:47] example here is [16:49] let's say you have a product where it's [16:51] important to know what date it is, for [16:53] example, right? Um you would want to [16:55] actually append that information to the [16:57] end of your prompt. Um and if you may [17:00] maybe you've got like a massive a 10,000 [17:02] token prompt, and then you want to [17:03] append at the end, oh, and by the way, [17:04] today's date is, you know, July 23rd, [17:07] right? Um and by doing so, like if you [17:09] did it at the beginning, uh it would [17:10] make your life a lot harder when it [17:12] comes to trying to cache that prompt. Um [17:14] and if you do it at the end, you can [17:15] take all of that, you know, chunks of [17:17] your prompt that don't change and save a [17:19] lot of money by caching them. Uh [17:21] persistent reasoning is another new [17:23] feature. Um basically, we've exposed the [17:26] ability in the API to uh previously, it [17:29] would we would have like reasoning [17:31] summaries or you would just chain the [17:32] the final assistant messages together. [17:34] But now with persistent reasoning, you [17:36] can save the reasoning turns across and [17:39] we do find this um this improves uh [17:42] continuity of answers. So, it is a boost [17:44] to performance and capability, but also [17:46] cache efficiency, right? If you do want [17:47] to just say, "Okay, we're we're just [17:49] going to keep sending all of these [17:50] through and and we can get better on our [17:51] caching." Um and last but not least, [17:53] compaction. Uh this one has been around [17:55] for a little while, but definitely want [17:57] to call this one out because um if you [17:59] are finding that your use case has a lot [18:01] of user prompts that get very long, uh [18:04] very heavy, lots of tool calls, lots of [18:06] things that were exploratory, but maybe [18:08] not useful in the long run, um you can [18:11] uh turn on auto compaction um or you can [18:13] decide if you want to explicitly trigger [18:16] compaction yourself through the API, uh [18:18] and then, you know, just reduce the [18:19] amount of inputs that you're sending [18:20] every time. [18:24] Cool. So, if we go back to uh the demo, [18:30] uh we can see I think a bunch of these [18:32] pandas finished up, right? Um again, I [18:35] think some of them to varying degrees of [18:38] realism and and accuracy here. Um but I [18:42] want to go to this third tab here um in [18:44] our uh [18:46] optimization lab, right? And I want to [18:48] share maybe I need to refresh this real [18:50] quick. [18:52] Um [18:53] I think I want to share uh some of these [18:56] outputs um because we're going to do a [18:58] little bit of AB testing across some of [19:00] the features that I just talked about, [19:01] right? Um this first one is prompt [19:04] caching. And so, in this demo, uh behind [19:07] the scenes, I'm making an API request um [19:09] with a I think it's like a 5 and 1/2 [19:12] thousand token prompt. Um, and we're [19:14] going to go ahead and cache it and then [19:15] we're going to do two API requests to [19:17] see um, how how the results ended up [19:21] coming out with and without caching, [19:22] right? [19:24] And so we can see here that um, [19:26] the input costs were 90% lower um, when [19:30] it comes to to using the cache, right? [19:32] Um, and uh, [19:34] if you have things it looks like the the [19:36] wait time slowed down by a couple of [19:38] seconds. Um, but if you're if you don't [19:41] need, you know, like the absolute most [19:43] latency and you know you're going to [19:44] reuse a bunch of prompts, um, if you've [19:46] got like a massive, you know, system [19:48] prompt that you want to cache, uh, this [19:50] is uh, a really great way to uh, to save [19:53] on cost. [19:55] Um, the second here is going to be uh, [19:58] programmatic tool calling. [20:01] And so uh, behind the scenes I've I've [20:03] crafted a prompt um, that's basically [20:05] asking for a specific task. Uh, it needs [20:08] to call a number of tools, it needs to [20:09] list accounts, track their usage, um, [20:11] and then, you know, create a support [20:13] ticket. Um, and normally with uh, [20:16] without programmatic tool calling, um, [20:19] we would have the model, it would call [20:20] tools, those results would be returned [20:22] to the model, it would decide what to do [20:24] next, maybe call more tools, and then [20:25] generate the final answer. Um, with [20:28] programmatic tool calling though, the [20:29] model, like I said, has this JavaScript [20:31] sandbox. It's writing code to just [20:34] inject into that sandbox, and the [20:35] sandbox is going to call the tools, it's [20:37] going to crunch the numbers, uh, and [20:38] then return the final results, and then [20:40] the model is going to decide to answer, [20:41] right? [20:43] And so we can see here um, that yeah, we [20:45] ended up with 24% fewer tokens, uh, [20:48] input tokens by using programmatic tool [20:50] calling. Um, we saved I think an entire [20:52] model turn, uh, by using the sandbox. [20:55] Um, and not only I think is the, you [20:58] know, this is a little bit of a toy [20:59] example, right? Um, so I think, you [21:01] know, if you want to test this yourself, [21:02] definitely use a real world use case. [21:04] Um, but not only did we save on input [21:06] tokens, um, we actually save on output [21:08] tokens cuz the model had to do less [21:10] reasoning about the final answer um, and [21:12] then figure out how it's going to like [21:14] return that to the user. Um, and in this [21:16] demo too, we can just check um, you [21:18] know, if you if you behind the scenes [21:19] change the prompt, you can check like [21:20] did it actually get to the same uh, [21:22] output, right? Which in this case it did [21:24] and that's what we want to see. [21:27] Um, in this one uh, this is a also a [21:29] quick context compaction one, right? So, [21:32] um, basically, you know, there's a uh, [21:35] pretty big history. I think there was [21:36] like 72 prior turns or hand-offs in this [21:40] uh, in this use case. Um, and I want to [21:42] see how it compares when we um, [21:45] ask a a question and ask for the answer [21:48] versus if we trigger a compaction first [21:50] uh, and then ask for the answer, right? [21:52] And what I'm expecting to see um, is [21:54] hopefully the same answer, but also um, [21:56] I think significantly fewer uh, input [21:59] tokens when when it comes to uh, [22:02] comes to our costs, right? Yeah, and so [22:04] I think this is you know, uh, quite a [22:06] dramatic output. We've got 82% fewer [22:09] input tokens for the same task. Um, [22:11] without compaction, we were sending [22:13] 24,000 input tokens and then after it [22:16] was, you know, a little over 4,000. Um, [22:20] in this case it's just asking for a [22:21] single answer, so there was not a ton of [22:24] of output or reasoning. Um, but I think [22:26] again a useful experiment for your own [22:27] use cases. If you think you will need a [22:29] lot of reasoning tokens, can we tune [22:31] this this prompt or this scenario to [22:33] figure out how it's going to go, right? [22:35] Um, [22:36] I think in this case it was a little bit [22:37] faster um, but uh, you did ultimately [22:40] take a hit on the compaction time. It [22:42] looks like it took uh, a little under 30 [22:44] seconds to to compact. Um, and then [22:47] yeah, we expect to see the exact same [22:49] answer coming out uh, which is what we [22:52] see here. [22:53] >> [clears throat] [22:58] >> Cool. Um, and so I've talked a lot about [23:00] you know, uh, theoreticals and in in how [23:02] OpenAI sees stuff, but I also want to [23:04] make sure we're we're talking about some [23:05] real-world use cases. And for that, [23:07] we're going to turn it over to Ploy. [23:11] >> Awesome. Thanks so much. [23:13] Um [23:14] Let's please welcome Ploy on stage. [23:22] We'll just give them a minute. [23:25] Ryan, Lorenzo. [23:26] >> Yeah. Can you guys hear me? [23:29] >> Yes, you're good. [23:31] Hey Ryan. Hey Lorenzo. [23:33] Okay. Thanks for joining us today. Um [23:36] we saw on the front page of Hacker News, [23:39] you guys put out a great guide on [23:40] migrating a production agent over um to [23:44] 5.6 Soul. Um so really excited for you [23:46] guys to be here with us live chat [23:48] through. Um first, a little bit about [23:50] what what you guys are building over at [23:51] Ploy. Um and then also just kind of a [23:54] step-by-step guide um through through [23:56] this um [23:57] through how you guys migrate your agent. [23:59] So feel free to take it away. [24:01] >> Amazing. Um hi everyone. Uh I'm just [24:04] trying to share my screen. It's [24:05] disabled. [24:06] >> We'll stop sharing screen on our side. [24:09] >> Okay, great. [24:15] Amazing. Um all right. Welcome everyone. [24:19] Um [24:19] my name is Bryant and I'm excited to [24:23] uh share a little bit about what we're [24:25] building here at Ploy. And uh today [24:29] uh I have my AI engineer extraordinaire [24:32] Lorenzo joining me. [24:33] Um so I was previously the co-founder [24:37] and CTO of Webflow. So I spent about 12 [24:39] and 1/2 years there. And [24:43] about [24:44] eight or nine months ago, [24:46] you know, AI happened, [24:48] agents happened, and I'm like, well, [24:50] what can the website actually do to help [24:52] a business grow? You know, I made a tool [24:54] that made it easy for folks to design, [24:57] but I just had this incredible itch to [24:59] take the website much further. [25:02] So, that's what we're building at Play. [25:04] So, what Play is, it's an entire [25:06] marketing platform that's starting with [25:09] your website. It gives you the ability [25:11] to run SEO, AEO optimization, [25:15] helps you improve your conversions. It's [25:17] also a web host. But, then on top of [25:19] that, it can also power your ads, it can [25:22] help you find customers, it can [25:24] de-anonymize the visitors coming to your [25:26] website, all with the purpose to help [25:29] turn your website into your hardest [25:31] working employee. And by that, I mean [25:33] help you drive revenue as automatically [25:36] as possible. [25:39] So, how does it work? So, very simple, [25:41] you sign up and within 60 seconds, you [25:45] have a Slurp'd website. So, what that [25:47] means is is that we're crawling your [25:50] sites, we're taking all of the CSS, [25:53] we're looking at every single responsive [25:55] breakpoint, and we're actually [25:56] deterministically [25:58] converting your existing homepage into a [26:01] design system that Play can then use to [26:04] build subsequent pages or optimize your [26:06] existing ones. [26:09] And if you actually want to design from [26:11] scratch, uh you can also use Play to do [26:13] that, too. So, [26:16] really sort of proud of the fact that [26:18] Play's platform doesn't have any AI [26:19] tells or as few of them as possible. [26:22] Every single site is actually incredibly [26:24] bespoke. [26:26] So, there's a lot of different things [26:28] that Play can do for you. So, it can [26:30] design with human taste, it can drive [26:33] revenue outcomes, it can help you [26:35] measure your entire sort of marketing [26:37] funnel with built-in analytics. And [26:39] then, we also have these expert [26:41] Playbooks that you can run, and you can [26:43] think of them as marketing skills. [26:46] Just to drive home this sort of point a [26:48] little bit. So, this is something that [26:49] people can actually do with Play, which [26:51] is they can actually create this whole [26:54] sort of marketing funnel that's [26:56] autonomous. So, someone comes to your [26:58] site, we can deanonymize that person, we [27:01] can help qualify that person to see if [27:04] they're your ICP, and we can activate [27:07] that lead. We can sync it to your CRM, [27:09] we can draft the email and send it out. [27:12] These are just some of the ways that we [27:14] can really turn your website into a [27:15] growth engine. [27:17] And you can do a bunch of things, too. [27:19] So, we actually have these playbooks [27:21] that, you know, really well-known growth [27:23] and marketing leaders throughout the [27:25] tech startup scene have created for [27:27] Play. So, to help you create AEO [27:29] comparison pages, to help you build a [27:31] content strategy, [27:33] or even email your ICP website visitors. [27:38] And just to wrap it up, [27:40] it's really for a lot of different [27:43] audiences. So, large startups, large [27:46] enterprises, [clears throat] agencies, [27:48] and [27:49] a place where I have a lot of affinity [27:52] for, which is YC startups. And I was [27:54] part of the last YC batch, and really [27:57] proud to say that around 13% of the [27:59] startups in the last batch are also [28:01] using Play. [28:03] So, I'll just hand it over to let [28:04] Lorenzo to actually talk about the true [28:06] value maxing part [28:08] that we spent a lot of time on here at [28:09] Play. [28:11] >> For sure. [28:12] Um [28:13] let me get my presentation up. [28:20] All right. What up, fam? [28:22] My name's Lorenzo, and I'm an AI [28:25] engineer. I [28:27] just I actually started AI engineering [28:30] when I joined Play, and I've just been [28:33] obsessed and having so much fun. Um I [28:36] really love optimizing things and saving [28:39] costs. So, this is actually something [28:41] we've been working on for a long time [28:43] with our agent even [28:45] before we switched to GPT 5.6. [28:48] Um but all the work that we've done [28:50] prior carries over to GPT 5.6 and I [28:54] figured that was probably the most [28:56] valuable thing to talk about in this [28:58] presentation. [28:59] So, this [29:01] dovetails really nicely off of [29:04] um Charlie's talk about, you know, [29:07] compaction, efficient tool calling, uh [29:11] programmatic tool calling, [29:13] um caching. [29:16] I I of those four, [29:18] I would say caching is kind of the the [29:21] most [29:24] important and sort of easily overlooked. [29:27] And there's a lot of details, so I [29:28] figured it's worth diving in. [29:30] So, just a quick explainer on [29:34] how exactly [29:36] KV caching or prompt caching works [29:39] um in your typical agent loop. [29:42] When you Every time the agent [29:45] try decides what to do next, it has to [29:48] process all of the prior input tokens. [29:51] So, if you have an agent loop that [29:54] calls, you know, 10 tools across 10 [29:56] different steps, there's actually a n [29:59] squared amount of like tokens that get [30:01] processed because um the context [30:04] accumulates and every time you call the [30:08] next tool call, [30:10] you have to process all the previous [30:12] tokens [30:13] and that's just, you know, building up [30:15] is kind of what you're seeing here. [30:17] So, with caching enabled, what that [30:21] allows is uh basically a 10x reduction [30:24] in cost for all of the tokens that have [30:27] already been processed uh cuz OpenAI and [30:30] other AI providers store the embeddings [30:33] from all those tokens. [30:35] And so, you know, this is just massive [30:39] for [30:40] cost efficiency [30:41] and um it also helps with um [30:45] responsiveness in [30:48] generally with [30:50] with really long contexts. [30:53] Um but even if you have [30:57] uh caching enabled [30:59] it's really easy for this [31:02] cache to break because this context [31:06] window needs to be append only in order [31:09] for the [31:11] um for the caching to accumulate and [31:15] persist. So this example here, if you're [31:18] trying to be smart about saving tokens [31:19] and say like dynamically loading tools, [31:23] the tools actually live at the very [31:24] beginning of the uh context window. So [31:28] if you are modifying the tools as you [31:32] are going through different steps uh in [31:35] the agent loop, that breaks your cache [31:38] and you're going to end up with a ton of [31:40] next cost um like we talked about on the [31:42] previous slide. [31:44] Um as Charlie mentioned too, like if [31:47] you're updating any information in the [31:49] system prompt like uh the current time, [31:52] that will break the system [31:55] uh the KB cache as well [31:58] um which really hurts your uh [32:00] cost. So um [32:05] there's a few [32:06] tricks that we've implemented [32:09] um in order to [32:11] first load tools on demand without [32:14] breaking the cache. So all we've really [32:17] done here is [32:18] we decided what tools need to be always [32:21] on. [32:22] And those are like the core tools that [32:25] we see [32:26] um but based on usage patterns they are [32:29] used in like you know 70% or more of [32:32] sessions. And then we have a lot of [32:34] tools that are more niche and we call [32:36] these on-demand tools. [32:38] And so when the agent needs to use one [32:41] of these on-demand tools, it simply [32:43] loads this tool schema. It gets appended [32:46] to the end of the uh context window. So, [32:50] you don't break any of the cash up here. [32:53] And then the agent is is able to use the [32:57] these on-demand tools perfectly fine [32:58] using the on-demand tools tool. [33:02] And um yeah, in our testing we saw a 45% [33:07] We're able to reduce the the tokens of [33:09] our tool schemas by 45%. [33:12] Um and in evals that showed like a 33% [33:15] cost reduction, which was really [33:18] awesome. [33:20] Um [33:21] and and and quick side note is like if [33:23] we had broken the cash every time we're [33:25] like [33:26] updating the tool schema, this would [33:28] actually end up being more expensive. [33:29] So, it's a it's a really great uh [33:32] strategy, but you need to make sure you [33:34] do it right to not break the KV cache. [33:37] Um [33:38] another thing that is really helpful for [33:42] uh maximizing the cache hit rate is [33:46] using breakpoints. [33:47] And this is something that I went over [33:49] in uh my GPT-5.6 [33:52] implementation guide. Um [33:55] this is slightly different between AI [33:57] providers. We've we've used both uh [34:00] Claude, Gemini, and GPT models. And um [34:05] this is this is a part where there's a [34:07] little bit of variation between [34:09] providers. Um but the the core concept [34:12] is the same. [34:14] Uh which is you [34:17] if if if you have a user that is um [34:20] creating multiple chats within your [34:22] platform, [34:23] you will likely have a set of tools and [34:25] a system prompt that is the same for [34:28] every chat. [34:30] So, [34:31] this like we were talking about before, [34:34] you don't need to process the system [34:36] prompt and the tools every single time. [34:39] So, you if you if you set a a breakpoint [34:42] to cache the system prompt and the [34:45] tools, this now gets saved for every [34:49] single chat afterwards. And [34:52] so, you can save a lot of tokens this [34:55] way. Um and then on top of that, there [34:59] there's like [35:00] um a feature that we have in the Play [35:03] Agent is working memory [35:05] or the workspace memory. So, this is [35:07] persistent across chats. [35:10] And this is unique to each workspace. [35:13] And this may change between chats, but [35:16] it may not. So, we set another [35:19] breakpoint after the workspace memory. [35:22] So, if there's multiple chats where the [35:26] workspace memory does not change, then [35:28] we're able to cache that as well and uh [35:31] increase the cache hit rate um [35:34] by And yeah, and the results of this was [35:38] the first message, if you're able to [35:40] cache the system prompt, uh and the tool [35:42] calls, which is really large for us, we [35:44] saw an 89% reduction in the first [35:48] message in a new chat. [35:51] And um yeah, this this reduced our token [35:53] spend by 5% overall in production. [35:58] And um [35:59] this is a trick I love because it's so [36:02] simple um [36:04] and [36:05] kind of like right under your nose sort [36:09] of thing. So, if you have a an agent [36:11] that's making a lot of tool calls, for [36:13] example, a coding agent, like uh the [36:16] Play Agent, [36:17] there's going to be a lot of token uh a [36:20] lot of tool calls within a single [36:23] uh session. [36:24] So, [36:26] and and batching these together makes a [36:28] huge difference. So, this example is [36:32] three tool calls: edit file A, read file [36:34] B, read file C. They're all independent. [36:38] Um but if the agent is doing these in [36:41] three separate steps, it has to re [36:44] uh process all of the input tokens and [36:47] the whole context window, and it [36:50] actually does a little bit of thinking [36:52] for each tool call. [36:54] Um but this is all this can be saved if [36:57] you just prompt the agent to try to [37:00] batch together tool calls into a single [37:03] step. [37:04] Um that looks like this, and you save [37:08] all of these um cached input tokens and [37:11] the extra thinking tokens here. [37:13] Um lastly, you can also design your [37:17] tools to perform multiple actions in one [37:20] tool call. So, that's what we've done [37:22] here, where you say read file B and file [37:25] C in one tool call, and that saves um [37:30] output tokens. And so, when we we [37:33] implemented this, ran some evals, we saw [37:35] a 14% cost reduction with the same pass [37:38] rate, and and I think it was actually a [37:40] little bit faster, too. So, win across [37:42] the board. [37:44] Um lastly, [37:46] I just There's a lot more I didn't cover [37:48] here, but as kind of a general [37:53] tip or or area of focus for me is really [37:57] optimizing your tools. So, Charlie, [38:00] again, mentioned programmatic tool [38:01] calling. [38:02] There's so much you can do just if you [38:05] or just design your tools in an [38:08] intelligent way. [38:09] Um [38:10] one really easy win for us, we use Exa [38:14] as our web search provider, and they [38:16] have a feature called highlights, which [38:19] slims down the output to just the most [38:22] important sort of like bullet points. [38:25] Immediate 70% reduction [38:28] in the web [38:30] tool response size, and that's estimated [38:33] to save us $37,000 a year in tokens. Um [38:37] and the same quality on our evals. [38:41] Um another thing here, we have [38:45] put a lot of thought into [38:47] thinking about what needs to be done by [38:49] the agent versus what can we just make [38:51] deterministic. Um there's one tool where [38:54] we um were able to save like a huge [38:58] round trip from the tool to the agent to [39:03] the next tool call. [39:04] Um and yeah, that was like a 54% [39:08] reduction in token spend. [39:11] So, yeah, that's pretty much it. [39:14] Uh lot more I could cover, and then as [39:17] Christine mentioned, we have a blog post [39:19] here on actually migrating to GPT-5.6. [39:23] There were for the most part, like I [39:25] said, everything carried over really [39:27] easily from the previous agent in terms [39:30] of our caching stuff, but there were [39:31] some details on the [39:33] breakpoints and stuff that we cover in [39:35] that blog. [39:36] Uh but yeah, that's it for me. [39:40] >> Awesome. Thanks so much for joining us. [39:43] Um we'll go back and share screen. [39:46] Thanks everyone who submitted so many [39:47] questions. We answered as many as we [39:49] could. We had chat, there's a couple [39:51] that we flagged to answer live and chat [39:54] go through them, especially with with [39:55] Brian Lorenzo here with us to kind of [39:57] share this actual real-life example. [40:00] Um so, the first one was probably the [40:02] most important. [40:05] Brian, [40:06] >> [snorts] [40:06] >> Troy really loved it. [40:08] I think it's awesome for you to be here, [40:10] especially as a founder. We have a ton [40:11] of startup founders who are building [40:13] really on early on in their journey. [40:15] Would love to hear from you, just like [40:17] kind of a quick update on like why you [40:20] chose to migrate and some of the [40:22] trade-offs, especially as a second-time [40:24] founder. [40:26] >> Yeah. [40:28] I think, you know, as a startup, we're [40:30] constantly looking for some type of [40:32] alpha, right? Some type of advantage. [40:34] But then, honestly, with with building [40:38] AI tools, you you really just have to [40:41] try every single model that's out there, [40:43] right? And [40:45] we have spent a lot of time making sure [40:48] that our evals reflect real-world use [40:51] cases, and that whenever a new model [40:54] comes out, many companies do this, [40:56] obviously, but we really want to stress [40:59] test, you know, whether or not they fit [41:01] Poe's use cases. So, you know, [41:03] everything from marketing strategy to [41:05] following really long instructions to [41:08] redesigning a whole page to importing a [41:10] WordPress site. You know, these are all [41:12] things that are [41:14] our agents uh need to be really good at. [41:16] So, um as a second-time founder, I don't [41:19] know, maybe [41:20] there's potentially even a disadvantage [41:22] in this in the sense that AI is just so [41:25] uh quick that a lot of my learned [41:27] experiences may need to be unlearned, [41:29] and I have to really drink from the [41:31] firehose now. So, that's um that's been [41:34] really, really fun. [41:36] >> Love that. Thanks for sharing. [41:38] Um okay, let's dive into some of the the [41:40] technical questions. Um this first one, [41:43] when using GPT-5 uh 0.6 with Ultra [41:45] Effort, it seems extremely token [41:47] intensive when delegating work to sub [41:48] agents. Are sub agents given only the [41:50] specific context needed for their task, [41:52] or do they receive and maintain the full [41:55] context of the current chat or [41:56] workspace? [41:57] >> Um I love that somebody asked this. [41:59] Thank you for asking this question, cuz [42:00] I think there's currently a lot of [42:01] confusion out there around the [42:03] differences between Ultra and Max and [42:07] Pro. Um [42:08] and so, I think to to maybe start, like, [42:10] I want to explain the difference between [42:13] um Max and Pro, and then we'll get to [42:14] Ultra. So, uh Max is a new reasoning [42:18] effort level above extra high. Prior to [42:20] GPT 5.6, uh the highest you could go was [42:23] extra high. Um and then now with 5.6, [42:26] there is there is Max, which essentially [42:28] just like, you know, fully takes the [42:30] leash off and and tells the model like, [42:31] "Use as much reasoning as as you think [42:33] you might need to solve this problem." [42:35] Um that is in contrast to Pro, which [42:38] historically is, you know, been [42:39] available as an option in ChatGPT and [42:41] not Codex. Um [42:43] as well as through the API. Uh you may [42:45] have noticed if you're a heavy API user, [42:46] you may have noticed that there's no GPT [42:49] 5.6 Pro uh model slug in the API. Um and [42:53] that's because we've actually moved it [42:54] into a new reasoning mode uh parameter. [42:58] And so, you can take uh any of the new [42:59] 5.6 models in reasoning mode, you know, [43:02] there's like standard and Pro. Um and [43:04] that'll essentially give you the same uh [43:07] the same effect as using Pro in chat. Um [43:10] but it'll spend significantly more [43:11] tokens, will take significantly longer [43:13] to come back to you with an answer. [43:15] Um [43:16] Okay. So, now that we've talked about [43:17] those two, let's talk about Ultra. Um [43:19] Ultra is not quite a new reasoning [43:21] level, but it's it's a little bit of a a [43:23] new mode that exists on top of Max [43:26] reasoning. And so, in Codex or in [43:28] ChatGPT work, um if you select Ultra [43:31] mode, and by the way, to get to Max and [43:34] Ultra, um I believe they're not enabled [43:37] out of the box uh because, you know, we [43:39] we want folks to be mindful about their [43:40] token spend. Um and so, if you go into [43:42] settings, you can enable those two [43:43] options. But, Ultra takes Max reasoning, [43:47] um which already is like uses many [43:49] reasoning tokens as you need, um and [43:51] then also modifies the underlying system [43:53] prompt to um [43:55] push it in the direction of using more [43:57] subagents, right? And 5.6 is the first [43:59] model that we've trained that's like [44:01] natively um meant to use subagents. I [44:03] think in previous models in Codex, you [44:05] could tell it, "Hey, use subagents for [44:06] this task." and it would do so. But, 5.6 [44:09] is the first model where [44:10] it is making calls for itself and like, [44:12] "Okay, here's where I should probably [44:13] use a sub agent or not." And and Ultra [44:16] basically turns that behavior also to [44:18] the max, right? [44:19] So, if it seems extremely token [44:21] intensive, it 100% is extremely token [44:23] intensive. And we do recommend it for [44:26] like the, you know, the really heavy [44:28] hitter use cases where, [44:30] just, you know, you need to spin up sub [44:32] agents to to accomplish a ton of work. [44:34] Otherwise, you know, if you're using it [44:36] to check your email, it's a little bit [44:38] like bringing a bazooka to a knife [44:40] fight, right? Like, you probably don't [44:41] need that much firepower. [44:43] Um, [44:44] but let's get back to the to the meat of [44:45] the question here. You know, are sub [44:47] agents given only the specific context [44:49] needed or do they receive the full [44:51] context? [44:52] They're given the context that they [44:54] need, right? And so, the the [44:57] main agent will dispatch [45:00] a sub agent and that can go and and [45:02] perform tasks and then return. That's [45:04] different to say like a side chat in [45:06] Codex, which is going to get the full [45:08] context of the current chat as it works. [45:13] >> Thanks for sharing that, Lorenzo. [45:14] Anything on your end that you wanted to [45:15] add to this if you've you've tried it? [45:19] >> Um, [45:20] this [clears throat] Yeah, my first [45:22] thought when I saw this question was you [45:24] could actually, if from a developer's [45:26] standpoint, you could implement it [45:27] either way. I haven't tried this, but I [45:30] have thought about um, [45:32] this [clears throat] this idea of giving [45:34] a sub agent the full context. It would [45:36] almost be like forking the context [45:38] window. [45:39] Um, I haven't tried this. I think most [45:42] implementations just have the [45:44] orchestrator agent write up a summary of [45:47] whatever the sub agent needs. That's [45:49] probably more efficient in most cases. [45:52] >> Awesome, thanks for sharing. Okay, next [45:55] question. How does agent depth, meaning [45:57] the number of delegation layers where [45:58] sub agents can spin up their own sub [46:00] agents, affect performance? What types [46:02] of tasks benefit from greater depth, and [46:04] when is a wider but shallower agent [46:06] structure more effective. [46:08] >> Um I think the first question is that's [46:10] a really good question. I I don't know [46:11] that I that I have all the data to [46:13] recommend something on here, right? In [46:14] terms of performance. Um I think I what [46:16] I can say about the types of tasks is [46:19] uh you know I think it's [46:20] you know, I think it's it's I personally [46:23] in the work that I do, I think I [46:24] personally um find it less effective [46:27] once you get past maybe two layers of [46:30] sub agents because I think you're losing [46:32] a lot of context um as you continue to [46:34] delegate and again that's that's totally [46:36] anecdotal. Um and in general I think I [46:39] find that agents like sub agents in [46:41] general are more useful when the work [46:44] can be like parallelized pretty um [46:47] effectively, right? And you're not super [46:50] worried about like having to share state [46:52] or having to like touch the exact same [46:54] files as you go um and you're able to [46:56] like, you know, [46:57] carve out independent lanes for each sub [46:59] agent to work with, but again, Lorenzo, [47:01] happy to to hear your take. [47:05] >> Yeah, the we we've been experimenting [47:08] with sub agents for a while in Poe and [47:12] we found with, you know, the models that [47:14] were around 3 months ago [47:16] we're actually way better off just not [47:18] using sub agents at all in terms of [47:21] costs, uh performance, and time. [47:25] Um I think the the meat of this question [47:28] really comes down to which model are you [47:30] using and how good are the orchestration [47:32] abilities of that model cuz [47:36] sub agents, like delegating a task from [47:38] one LLM to another LLM, [47:41] is is a bit lossy. [47:43] Um [47:44] you [47:45] want these, [47:47] you know, tasks or the direction to be [47:51] as human-driven as possible or [47:53] expert-driven. Um Um, that is changing [47:56] with recent models. Um, like G G5.6 [48:00] Souls orchestration abilities are much [48:02] better. Um, if you want to do multiple [48:05] layers, [48:07] the question is [48:08] the the if you're having a an [48:12] orchestrator agent delegate to a sub [48:14] agent and then that sub agent delegates [48:16] to another sub agent, is that second [48:18] level sub agent good at orchestrating [48:21] sub agent tasks? If you're doing Soul [48:23] Soul Soul, maybe, but if you're doing [48:26] Soul [48:27] Terra Luna, you might be losing uh [48:31] some efficiency there, but overall, I'd [48:34] just say proceed with caution and [48:37] yeah. [laughter] [48:39] >> Love that. Great advice. [48:41] Um, okay. Next question. Are there [48:44] instances where starting a new chat in a [48:46] project such repo is better than [48:48] compaction? [48:49] >> Um, yeah, I don't know if this is a [48:51] little bit of a spicy take, but I think [48:53] there there probably still are some some [48:55] instances. Um, and [48:57] by and large, I I don't think about [49:00] compaction anymore. I think when I work [49:01] in Codex, I'm happy to work in the same [49:03] thread for for days if not weeks and and [49:06] I, you know, just keep going. I think I [49:07] used to be a year ago I used to be one [49:09] of those people that was like every [49:10] single new thing is like new thread, new [49:12] chat, right? To avoid, you know, context [49:14] pollution. Um, and now I don't worry [49:16] about that as much. I think the [49:17] instances where it is so useful is if [49:19] your uh personal setup is like very [49:22] heavily loaded with like a ton of [49:24] context out of the box, um, and you are [49:26] enabling like a ton of plugins or skills [49:29] or, you know, attaching files, um, [49:32] because that is going to be stuff that [49:33] you want to, you know, presumably [49:34] they're all relevant, but that is stuff [49:36] that you're going to want to be bringing [49:37] in. Um, and if it's, you know, if it's [49:39] stuff that the model has to keep in mind [49:40] for every answer, then in that case, uh [49:43] it probably makes sense to start a new [49:44] chat rather than relying on compaction. [49:46] >> Yeah. [49:47] All right, so are you team new chat, [49:50] same chat? Thoughts? [49:53] >> Yeah, we've seen users have super long [49:57] chats in work sessions in play and then [50:01] just use that for everything. And it's [50:04] like if you're working on your website [50:07] and then you want to [50:08] do some random search after, you're just [50:11] wasting tokens if it's a a completely [50:13] separate task. Uh so this is something [50:16] that is kind of a UX [50:18] new UX problem of like when do you [50:21] switch to a new chat? Do you let users [50:23] decide? Do you use LLMs to decide? And [50:26] that's still an area that we're thinking [50:28] about. [50:30] >> Yeah, good point of view. [50:31] Okay, we have two more questions left. [50:33] Uh the presentation covers the benefits [50:35] of caching well. Can you please touch on [50:37] the downsides to overly relying on [50:40] caching for value maxing? [50:43] >> Uh [50:44] >> I I can I can say a few words. [50:47] >> Um [50:48] yeah, like [50:50] we like I think Charlie's portion was a [50:53] great overview of like [50:55] all the tools. [50:57] Um I definitely did more of a deep dive [50:59] into caching. So this is assuming that [51:02] you're like [51:04] you've you've already done a little bit [51:05] of an audit of all your traces and say [51:08] like oh, this tool call is outputting [51:12] way too many tokens. Um or you're like [51:15] duplicating [51:17] um outputs in the in the context window [51:21] or you are [51:23] making the agent do stuff that it [51:25] doesn't need to do, that you could just [51:27] write code for. [51:29] Um so that all comes down to just [51:33] basics of um like auditing the traces of [51:37] what your agent is doing. Um and then [51:41] once you've done, you know, uh [51:44] the 80% of the job there, then you can [51:47] you can definitely get a lot of value [51:49] from the caching side. [51:55] >> Nice. [51:55] >> Yeah, I think the only thing I'd add is [51:57] is, you know, [51:58] um [51:59] caching as a general concept, right? I [52:01] think it like it is often very [52:02] beneficial, but um you want to introduce [52:05] it when you're ready for it. I think [52:07] when you feel like you have a good grip [52:08] on like the the parts of the application [52:10] that will benefit, otherwise you can get [52:13] brittle, right? And you can you can feel [52:14] a little bit hamstrung. [52:17] >> Awesome. Thanks. Okay, last question [52:19] before we wrap up. Does Compact run [52:22] differently on different models and [52:25] reasoning levels? [52:27] >> Uh the underlying endpoint should work [52:29] should work the same, right? And I think [52:30] like with different models, um you know, [52:32] essentially uh there's some secret [52:34] sauce, so I can't talk about everything [52:36] here, but uh essentially we're taking [52:38] the um [52:40] the history of the conversation and [52:42] turning it into like uh a representation [52:44] that the model can can understand and [52:46] reuse in the future. Um and uh [52:49] that, you know, depending on the model [52:51] and the reasoning level, um [52:53] you know, [52:54] what gets encoded maybe maybe slightly [52:57] different, but that also depends on the [52:58] chat, right? And so I think like uh you [53:00] would expect um [53:02] as a mechanism, it it runs the the same, [53:05] but, you know, per chat the the outputs [53:07] might be a little bit different. [53:11] >> Okay, awesome. Lorenzo, Bryant, thanks [53:13] so much for joining us, um especially on [53:15] the Q&A, super insightful, and I think a [53:17] lot of um the startup founders and and [53:19] developers um really appreciated it. [53:22] So, to wrap up, here are some resources. [53:25] Um first and foremost, GPT 5.6 blog, um [53:29] programmatic tool calling guide, um [53:32] compare models, um and Cohere's uh [53:35] migrating a production AI agent um [53:37] uh blog. And then a couple of people [53:40] asked in the post actually about your [53:42] demo and if you can try it themselves. [53:45] So the answer is yes. Our all the code [53:47] that we use for build hours is shared on [53:49] our build our repo. So feel free to [53:51] check it out. And as I mentioned [53:53] earlier, we're going to post all of this [53:54] on demand, send out an email with all of [53:56] these links. So don't worry too much [53:57] about copying all of this down. [54:00] Next we have our upcoming build hour on [54:03] July 29th. This one's going to be on [54:05] ImageGen 2. So you'll learn the latest [54:08] on that. And as always, copy down this [54:10] link. We're constantly posting new [54:11] sessions. Feel free also to reach out if [54:13] you have any ideas on what you want to [54:15] hear next. And with that, thanks so [54:17] much. And we'll see you next time. [54:19] >> Thanks, everybody.