Anthropic Debuts Developer Tools as Enterprise AI Agents Face Context Blowout

企业级AI智能体面临上下文暴增，Anthropic推出开发者工具 部署高性价比的生产级智能体，需要将自动化提示词缓存与混合顾问策略相结合，以防止庞大的企业数据集挤爆上下文窗口。 本次演讲概述了用于克服运行 AI 智能体所面临的高成本、高延迟和上下文限制的生产级实用技术。内容介绍了可直接用于节省成本并提升响应速度的提示词缓存技术，以及上下文工程的三大支柱：工具搜索、程序化工具调用和压缩。最后，演讲展示了“顾问策略”，该策略将 Claude Sonnet 等低成本执行模型与 Claude Opus 等高智能顾问模型配对，从而在不造成过度支出的情况下捕捉关键细节。 Prompt 缓存能将输入 Token 成本降低 90% 并大幅缩减延迟，且缓存 Token 不计入 API 速率限制，最多可将限制额度提升 10 倍。 (0:00) Anthropic 提出上下文工程（Context Engineering）以解决长程智能体的上下文膨胀问题，强调避免使用过度抽象的框架，精准掌控注入提示词的内容。 (10:00) 通过 Tool Search 延迟加载工具定义、利用 Programmatic Tool Calling 在内存中执行代码提取数据，以及 Compaction 定期压缩陈旧对话，可大幅精简上下文占用。 (10:00) “顾问策略（Advisor Strategy）”通过让 Sonnet 或 Haiku 负责大部分常规工具调用，仅在异常或高风险决策时调用 Opus，以更低成本实现旗舰级推理能力。 (20:00) 平台推出工作负载身份联合（WIF）以消除静态 API 密钥风险，并发布 Ant CLI 工具，支持开发者借助 Claude Code 自主管理平台资源。 (25:00)

Anthropic's Brad Abrams argues that deploying production-grade AI agents requires moving beyond naive prompt loops by pairing prompt caching, proactive context engineering, and a multi-model advisor architecture to control runaway costs and latency. Prompt Caching to Reduce Cost and Latency (0:00) In long-running agent workflows, prompt histories continuously expand as tool calls and responses accumulate, forcing models to reprocess the entire transcript on each turn. Abrams identifies prompt caching as the baseline intervention for production deployments, providing a 90% discount on input tokens and significantly lowering time-to-first-token latency. Because cached tokens do not count against standard API rate limits, implementations can effectively scale throughput by up to tenfold. By default, key-value caches persist for five minutes, allowing multi-turn loops to repeatedly hit pre-computed states. Engineering teams at Cursor, Replit, and Perplexity have reported prompt cache hit rates exceeding 90% using this approach. Context Engineering Pillars to Stop Context Bloat (10:00) Even extended context windows, such as Claude Opus's one-million-token capacity, can be rapidly exhausted when agents ingest raw enterprise data from sources like Slack, Gong, and Salesforce. Abrams cautions against frameworks that obscure prompt construction, advocating for context engineering across three primary mechanisms: Tool Search, Programmatic Tool Calling, and Compaction. Tool Search defers detailed schema loading until an agent dynamically queries for a specific capability, an approach that Abrams notes delivered a 10% token reduction for software platform Lovable while reducing cognitive distraction for the model. To prevent context pollution from massive payload returns, Programmatic Tool Calling prompts the model to generate short, in-memory Python scripts. Rather than dumping entire datasets into the transcript, the script extracts only essential data points—such as a specific metric or an isolated 2,500-character excerpt—before feeding the result back into the prompt, a technique utilized by Quora for HTML parsing. Finally, when sessions approach capacity thresholds (illustrated at 500,000 tokens), Compaction pauses the main loop and routes the transcript to a separate call that replaces stale turns with dense, structured summaries, mirroring production pipelines deployed at Hex. The Advisor Architecture for Tiered Intelligence (20:00) Relying exclusively on flagship models like Claude 3 Opus for end-to-end execution generates prohibitive operational costs, reaching approximately $10 per workflow load in Anthropic's demonstration. To achieve high reasoning quality at manageable expenses, Abrams proposes an advisor strategy modeled on software engineering mentorship. A lighter executor model, such as Claude 3 Sonnet or Haiku, executes routine tool interactions and code execution, while Opus serves strictly as an on-demand advisor for edge cases or high-stakes decisions, a design currently used by Bolt. Anthropic demonstrated this division of labor through a contract renewal scenario: Sonnet processed meeting transcripts and initially reported that an enterprise deal was progressing normally. When the system triggered an Opus review over the same text, the larger model identified an overlooked, mandatory requirement for a fictional material ('cryothane'), preventing contract failure. This hybrid routing confines Opus-level billing strictly to targeted, high-complexity evaluations. Platform Tooling and Credential Management (25:00) To streamline operational maintenance and secure enterprise integrations, Anthropic introduced Workload Identity Federation (WIF) alongside a native command-line interface, the Ant CLI. Workload Identity Federation eliminates the requirement to store static, long-lived API keys within production environments. The Ant CLI replicates Console functionality within terminal environments, enabling coding assistants like Claude Code to inspect platform metrics, adjust cache-control markers, and configure infrastructure without manual dashboard intervention. Abrams maintains that reliable production agents depend on direct visibility into prompt composition rather than framework abstraction layers. While prompt caching and context filtering mitigate immediate memory limits, balancing reliability against unit economics ultimately requires dynamic routing between fast executor models and specialized reasoning advisors.

Anthropic 的 Claude 平台产品负责人 Brad Abrams 指出，将 AI 智能体投入实际生产时，企业普遍面临成本高昂、高延迟和上下文窗口溢出等核心障碍；通过部署提示词缓存、细粒度上下文工程以及主从“顾问策略”，开发者能够在大规模降低 API 支出的同时确保系统的智能水平与可靠性。 提示词缓存降低推断开销与延迟 (0:00) 在长周期的智能体运行流程中，工具调用记录与返回结果会不断追加到输入端，导致上下文体量持续膨胀。Abrams 表示，若缺乏优化机制，模型在每一个交互轮次中都必须重新计算整段历史记录。通过在提示词的静态和常用片段中嵌入缓存控制标记，系统能够将键值（KV）数据提前计算并持久化存储（默认保存时长为五分钟），从而在后续循环中跳过初始推断阶段。这一机制为输入 Token 提供了 90% 的价格折扣，大幅削减了通常占据 API 账单最大份额的开销，并显著降低了首字生成延迟（TTFT）。此外，缓存命中的 Token 不计入 API 速率限制配额，使系统的实际并发承载能力理论上提升达 10 倍之多。 为推动工程化落地，Abrams 指出 Cursor、Replit 和 Perplexity 等工程团队已将提示词缓存命中率优化至 90% 以上。为了降低普通开发者的调优门槛，Anthropic 在 Claude 控制台推出了分析仪表盘以监控生产环境下的缓存表现，同时在 Claude Code 中集成了自动化配置能力。开发者只需输入特定指令，系统便能分析代码库结构、重排提示词层级并自动插入缓存标记，在 HeroCorp 原型演示中，这套机制成功将最初为零的缓存命中率提升为持续写入与命中。 上下文工程的三大支柱破除容量限制 (10:00) Abrams 警告称，许多开发人员倾向于使用高级框架抽象，但这些中间层往往会掩盖实际输入模型的原始上下文细节。在企业级业务中，直接接入企业软件（如 Slack、Salesforce 和 Gong）所产生的庞大数据流，即使在 Claude Opus 的百万 Token 窗口下，也极易在处理早期任务时迅速触顶。为了解决上下文膨胀，他提出了“上下文工程”的三个支柱技术：工具搜索（Tool Search）、程序化工具调用（Programmatic Tool Calling）以及上下文压缩（Compaction）。 工具搜索技术旨在避免预先在系统提示中加载数百个工具定义所造成的“工具臃肿”与模型注意力分散，系统仅声明工具目录，只在模型发起请求时按需实时注入特定工具模式。Abrams 引用开发平台 Lovable 的实践说明，该方法减少了 10% 的 Token 消耗并提升了推理专注度。第二项支柱是程序化工具调用，针对工具返回的大规模数据转储，该方案让模型编写并执行轻量 Python 脚本，在内存中直接清洗和解析 JSON，仅提取目标指标注入上下文，Quora 等团队借此显著控制了成本并优化了 HTML 解析准确度。第三项则是被称为“重锤”的压缩技术，当上下文达到预设阈值（例如演示中的 50 万 Token）时，系统自动挂起循环，通过独立的后台模型调用将历史转储提炼为结构化摘要，清除陈旧交互轮次；Hex 公司目前已在生产中采用这一方案。 执行者与顾问模型协同平衡成本与智商 (20:00) 单纯依赖顶尖旗舰模型运行端到端工作流往往伴随着难以持续的成本支出。在 HeroCorp 演示中，完全基于 Claude 3 Opus 驱动的智能体每次加载需要消耗约 10 美元。为了在维持 Opus 级判断力的同时压缩预算，团队提出了模拟软件工程中“初级开发者配合资深架构师”的顾问策略（Advisor Strategy）。在该架构中，成本效益更高的模型（如 Sonnet 或 Haiku）承担“执行者”职责，负责处理日常工具调用、常规代码执行和大部分标准任务；仅当执行者遇到异常数据或面临高风险决策节点时，系统才会唤醒 Opus 作为“顾问”介入审查。 这一混合架构的实际效能在一场模拟商务谈判审核中得到了验证。在针对虚拟客户的合同续约审查中，Sonnet 遍历了大量的会议记录与客户档案，并初步得出合作进展顺利的结论；然而，由于该业务涉及关键商业利益，系统触发了 Opus 顾问调用。Opus 仔细复核了完全相同的对话记录，敏锐识别出市长在发言深处提出的硬性前置物料需求（虚构材料 Cryothane），这一被 Sonnet 忽略的细节促使系统立即将合同风险等级上调为红色预警。Bolt 等企业已采用类似架构，通过非对称的模型调用分配，在规避昂贵全量推理的前提下保障了关键业务逻辑的完备性。 底层身份凭证集成与命令行自动化演进 (25:00) 除了算法与上下文层面的编排优化，Abrams 和演示团队还阐述了生产级环境所必需的基础设施增强。为了根除传统 API 密钥在代码库或客户端环境中意外泄露的安全风险，平台引入了工作负载身份联合（Workload Identity Federation, WIF），允许系统通过标准身份联合机制安全访问模型资源，避免了对静态凭证的硬编码依赖。 在开发工具链方面，针对 Claude 擅长理解和调用命令行界面的特性，平台推出了 Ant CLI 工具。该命令行接口完整镜像了开发者控制台的管理功能，能够与 Claude Code 紧密联动，让智能体不仅能编写应用层业务逻辑，还能自主执行环境配置、资源管理与测试巡检。Abrams 强调，将系统深度架构在平台原语之上，能够确保智能体随底层基础设施的迭代而实现性能与安全能力的被动自动升级。 Abrams 总结认为，开发高可用智能体的关键路径在于优先落地提示词缓存，继而通过严格限制进入窗口的原始数据并结合模型分层策略来控制开销。尽管演示证明了多项技术叠加能够大幅压降运行成本，但在复杂的实际生产场景中，按需动态检索工具的边界设定、沙箱代码执行的安全隔离以及调用顶尖顾问模型的触发阈值，依然需要开发者根据具体延迟要求与业务容错率进行针对性权衡。

[0:00] Brad Abrams: the most important session of the day [0:00] Brad Abrams: If you're sitting next to one of these guys, you should just talk to them after and get the real tips. [0:00] Claude Code: improve my cache hit rate [5:00] Brad Abrams: Wait, Ben, is this the demo we agreed? We're at Code with Claude. This looks like a late 90s SharePoint UI. [5:00] Brad Abrams: Ben, look, Ben, I gave you a million tokens of context... and that's not enough, apparently. [10:00] Brad Abrams: One mistake I see developers doing is using abstractions over top of the platform that obscure what's in the context [10:00] Brad Abrams: First off, don't you love these animations? Can you tell that I have free tokens from Claude to be able to build these animations? [20:00] Brad Abrams: Opus-level intelligence but at Haiku-level costs or Sonnet-level costs [25:00] Brad Abrams: ignore the rest of the talk and go do that [25:00] Brad Abrams: betting on the platform means that the platform is going to keep getting better

[0:00] Thank you. so please welcome to the stage product management lead clawed platform of anthropic brad abrams Thank you. Well, good afternoon. You've almost made it to the end of Code with Claude. Thank you for hanging in there with us. But I got to tell you you are in the most important session of the day so you made a good choice being here Thank you This is an important session because we going to talk about putting agents into production the real world case of agents in production. And, you know, just to get us started, how many of you already have an agent in production, not some demo proof of concept, but actually in production? Who's got? Okay. Now keep your hands up. Keep your hands up. if you're happy with the cost, reliability, latency. Yes, okay, there's still a few. If you can put your hands down, but if you're sitting next to one of these guys, you should just talk to them after and get the real tips. But we're gonna talk through a few things in this session that will help you manage things like cost and latency and reliability, a few techniques. So let's drill in here, And it turns out the most important technique to think about is prompt caching. I caught a couple of sessions. A lot of people are mentioning it. But if you're not already doing prompt caching, then you absolutely should. And I'm going to tell you, I can look at the analytics, and I know a few of you are not doing prompt caching. So it's absolutely worth mentioning this. With long agents prompt caching is very important because the context continues to grow over tool call after tool call You have the tool call tool result and then the tool call and tool result again, and all of that gets appended to the prompt. And so there's a lot of common elements in that prompt that unless you do prompt caching, we're reprocessing that every time. With prompt caching, if you mark which sections are common in your prompt, then we're able to compute the KV values, essentially pre-cache the models, part of the inputs to the models in KVs and save those. And that saves a lot of latency when that happens and a lot of processing time. We skip the whole first part of inference when you do that. And so it's a big cost savings. In fact, it's a 90% discount. So if you're doing a long-running agent, you're not doing prompt caching, you're missing out on a 90% discount on input tokens, which is the largest share for most customers. You also get faster response time, especially time to first token. And then a little known fact is that it also prompt cash tokens don count against your API rate limits So if you have a rate limit and you want to manage that rate limit as well as you can the cache tokens don't count against that. So we almost 10x those rate limits today, but still, if you still want to manage those, you can with this. And some of our customers have done a really amazing job with prompt caching. I just want to highlight a couple of these. Cursor, Replit, Perplexity are all well into the 90s. And they have done significant engineering effort. I've sat with some of these engineering teams, and I can tell you they have done a lot of effort to get prompt caching into the 90s. But I'm going to tell you a hint. So we have a couple of tools that are available to you right now that you can use that make it way easier. The first is on the screen here. you can go and do some deep inspection of what's happening with prompt caching in your app. So I encourage you, feel free, if you would like to pop open your laptop, go to the Claude console and check out the new prompt cache dashboard under analytics and you can actually see what your production agent is doing in terms of prompt caching. And if it's not in the 90s, [5:00] Some work to do. So that's the first thing. The second thing is we've recently launched a skill for Claude Code that's an expert at prompt caching. And it's installed by default with Claude Code. So all you need to do is go to Claude Code and say, improve my cache hit rate. And Claude will walk you through the process of adding the cache control markers in there, maybe reorganizing some of the prompt, just walk you through that process so you can get a very high cache hit rate. So that's an absolute no-brainer. I think you should take a look at doing. So let's take a look at a demo of how prompt caching looks. So I'm going to invite Ben on stage. Ben, come on out here, and we're going to take a look at this demo. So Ben and I worked on this demo. It's an executive So let's say you're the CEO of a company and you have all these objectives and we're running a... Wait, Ben, is this the demo we agreed? We're at Code with Claude. This looks like a late 90s SharePoint UI. I don know What do you have Claude code Okay pop open Claude code Let see what we can do Okay Ben got Claude code and it attached He got the source code for this and we going to see if we can improve this theme. So how many people want a better theme? Okay, there we go. Okay, so a better theme is this like a little bit more appropriate for the venue that we're in. we are now not some boring 90s CEO. We are the CEO of HeroCorp AI. And what HeroCorp does is rinse out superheroes to battle villains, protect Metropolis, come to your child's birthday party, whatever they do. And then we're seeing the objectives. This is objective one. I'm told that retention of superheroes is a very important thing. and so objective one is around retaining them. And Ben, I don't know, maybe we're not paying superheroes enough. It looks like they're a little low here. And you can see some updates from each of the superheroes, and then a CEO, some tasks that we can go do. So, Ben, do you know, what's the cash hit rate on this? Do you know? No, no, okay. You don know what the cash hit rate is Okay okay So first off you got to know what your cache hit rate is So what we done is we implemented a dev console for this little demo So slide open the dev console and let's take a look. So in this little dev console, what you see here is our context usage, tool calls, and then there's this agentic transcript that's happening. So don't you wish all your apps came with this beautiful dev dashboard? But I notice in this dashboard, one thing that's standing out to me immediately is the cache hit rate is like zero. I mean, I don't know. So Ben, is there something we can do to improve the cache hit rate? So he's going to open up Claude code, go back to that and just improve the cache hit rate. And now we'll rerun that. And notice when we're rerunning it, we're hitting all those same tool calls again. But this time in the agentic transcript, you're seeing cache writes and cache hits. So the first time the inference system sees a prompt segment, it writes it to the cache. So we store those KV values for five minutes by default You can extend that with some options And then the next time that a loop comes around that becomes a cache hit So in a normal agentic loop you'll see some cache hits and some cache reads. So we're doing a little better here, and I think you'll watch over the course of the demo, you'll see that cache hit rate get better. Okay, so that's prompt caching, but if you scroll down a little bit, let's look at some of these other objectives. Ben, look, Ben, I gave you a million tokens of context. One million tokens in this Opus 4.7 million tokens, and that's not enough, apparently. So with all the tool calls that we're doing to get information from Slack, from Gong transcripts, from Salesforce, all of that, in fact, why don't you pop open one of those and just show there's just an enormous amount of data that's getting flooded into the context. So even at a million token context, we're running out before we even get through objective one. So we should think about how we want to handle this. As you might guess, I have a couple of techniques for handling this. Let's switch back to the slides, and let me describe some techniques for context engineering. Okay, so context engineering is really a discipline. [10:00] It's the discipline of deciding what belongs in Claude's context. One mistake I see developers doing is using abstractions over top of the platform that obscure what's in the context. And then as a developer, you don't really know what Claude's seeing in its context. And that makes it difficult for you to optimize. It makes it difficult for you to be a context engineer. So I encourage you to really pay a lot of attention to look at the full transcript that Claude has access to, that Claude's using, because it'll be very insightful. And the discipline is you making a proactive decision to decide what should be in. And I'm going to talk here about three different tools that we have available today in production for you to use to help manage the context. The first one is about reducing the tool declarations that's in the platform, that's in your context. The second is about reducing the tool results that pollute the context. And then finally, compaction reduces all those stale turns that are no longer needed. So let drill into each of these and see how they look So with tool search tool we have many customers that have tens or even hundreds of tools loaded If you have a long running multi agent oftentimes it does need many tools to get its job done. That's one of the beauties of LLMs is their general purpose and can do lots of different things. So we want to encourage you to have a lot of tools. But if If you look at this without case, if you load all those tools in up front in the system prompt, then that leaves very little space to do your actual work. So if we look at without, sorry, with tool search tool, what we're doing is defer loading all those tools. You still declare them up front, but we don't put them in the prompt yet. We defer loading them, and then we load them just in time, just as the model needs them. So if the model, if in a particular agentic trajectory, the model never needed that tool, then it never gets loaded. And that really optimizes the context pretty well. And we see customers like Lovable have reduced their token uses by 10%. And actually, not only does it save you money and latency to do that, but what Lovable saw and I think many people will see it actually increases the intelligence of the model to be more careful about what goes in the context So that tool search tool The next one to look at is programmatic tool calling. So first off, don't you love these animations? Can you tell that I have free tokens from Claude to be able to build these animations? So what programmatic tool calling does is it solves the problem of tools that return too much data. And many of the tools that we just showed you return huge amounts of text that just gets stuffed in the prompt. And, you know, that works fine. If you're building a little demo, that works fine. But here in this session, we're talking about going to production, not building a demo. So to go into production, you need to be a little bit more thoughtful about this. And you can try to mess with the tools to have them return less, but oftentimes what happens when that is you miss a case. There is some case where the model needed that data and then you've removed it and now the model doesn't have it. So the insight that we had here is that the model is actually pretty good at writing Python code. I don know if you noticed that but the models are actually very very good at writing Python So what we do is we expose the model we say here are the set of tools that are available right now in context And it writes Python code You'll notice the first time it writes code to inspect the schema of what's returned. And then the second time, like you're seeing here, it knows the schema. So the tool returns all of its data. It stays in memory. and then the model writes code to pull out just the little bits, just one byte here, a few words there, and it uses just that in its context. So the model is deciding what it needs in its own context. And with this technique, Quora is saving a lot of money and seeing increased intelligence for Python, a lot of HTML parsing that they're doing. And then finally, I call this one the sledgehammer technique, because even if you do a great job at managing your tool declarations and the tool results, if you have a long enough running agent, then you will hit the context window limits. And so what compaction does is it removes all those unused turns, all those turns of the conversation that are not needed [15:00] It just compacts those down to a short summary, and that summary is important. And in fact, there's a lot of intelligence that goes in to create that summary so that the model can continue on. It doesn't lose the thread. It kind of knows what it needs to keep doing with compaction. And we see Hex is actually using that in production now. Okay, so let's, I know you're dying to see, Let's switch back to the demo and see our superhero, our HeroCorp agent. And let's see. So we should add tool search tool. You know what? Let's just add them all. Just add all of them. So we're going to add tool search tool, programmatic tool calling, and compaction in one go. So now when we reload the page, watch the context bar as it goes up there. Notice firstly, it's moving way slower than it did before. Remember before it loaded, in the first objective it went all the way to a million tokens. We're calling the exact same tools that are returning the exact same data, but we being smart We using context engineering to understand what goes in So with much less context we able to load the entire page And I know some of you bean counters have already noticed that the cost went way down when we did that as well So, but let's walk through these one at a time. Just make sure we follow what exactly each of them do. So let's start with, you see, tool search tool in here. So here is what the tool search tool returns. So again, the model looked at the problem we gave it and said, oh, I'm going to need a tool that does this thing, that does hero retention metrics. And then our system went through all of the hundreds of tools that were there and picked out, I don't know, three or four and returned those. So rather than having 100 in context, we only have three or four. And so something like, I don't know, like hero retention metrics gets called. And then I guess later we see it. Do we see it? Which one do we see? Do you see? Oh, yeah, there it is. There it is. Sorry, I missed it. Hero retention metrics is right there. So we dynamically added this tool just in time. And then as soon as we added it the model turned around and called that And we returned the full data from that Okay so that was tool search tool The next one we talked about was programmatic tool calling So in this one, we flag it as code execution in this view. And look at, just examine the code that the model writes. And we, this is not some special thing I have. We give you exactly the code the model writes so you can understand what's actually happening. You see, it's calling those methods. If you look at the results equals async gather, it's calling each of those same tools, the same tools we saw before. It's calling those with the same parameters. But this time, we're not loading all of that into context. The model is loading them into a JSON object and is printing out very well documented. You see contact, pipeline, gong, exactly where it came from. And then you see that colon 2500. The model knows exactly which object that it needs. So it doesn like all of that crap we loaded in the first turn It doesn need that it just needs this one little bit So it pulling that out So it printing it And then that is what gets loaded into the model context Okay so that tool search tool And then finally the last one we talked about was compaction So actually for compaction, why don't we look and see, it gets called a couple of times in here. So what I did just for demo reasons, we set the compaction threshold pretty small, like 500K, something like that. And that is a common technique. You don't have to go, I launched the Million Token Context. I love the Million Token Context, but you may not need all of that. It may save you some cost, and it may save you some latency, and it may give you more intelligent to keep the model at a lower threshold. So that's what we did here. We kept it at a lower threshold. And so as the context grew to that threshold, the model, we paused execution, sent the entire transcript to another model call, which summarized it and gave this summary. So go ahead and expand the summary. And you can see this is the summary of everything that went on. So it took hundreds of tool calls with all their results [20:00] and everything that was going on and just compacted it down to like here's the few things you need to know to keep going so the model could could keep going okay so yeah we hit I think we hit all the things but I notice it's still costing ten dollars to run this thing I mean I got to tell you the Here, if you're selling tokens is a great business. Second to that is selling superheroes. That's also a really good business, but still $10 per load. I mean, we're gonna get in trouble for this thing. So I notice that you're using Opus. Is this, yeah, where does it say this? Yep, yep, so the model that's being used is Opus. Again, Opus 4.7, absolutely great model, but it's expensive. and it may be better to use a smaller model. So it turns out a small model like Sonnet can do tool calling just as well, write code just as well, but there's just a few things it doesn't do as well. And we talk about how to take advantage of that So let switch back to slides and we talk about the advisor strategy So what the problem we trying to solve with advisor is we want opus level intelligence but at haiku level costs or sonnet level costs. That sounds great, right? So the insight behind here really came from engineering teams. So you know, if you've worked with engineering teams, that you can pair a junior engineer with a senior engineer, and that junior engineer will get a lot better, right? Because the senior engineer won't be doing all the work. They won't be hands-on keyboard doing the work for them, but they will do code reviews. They'll look at design documents. They'll do coaching. And that's exactly true with models as well. What we see is that if we give Haiku a way it can call Opus and ask for help, then an Opus can scan over the transcript and see what's happening, then it can give some advice back to Haiku, which really helps a lot. And so you see, again, another beautiful animation here. You see that the executor with Haiku knows every shape It knows exactly what to do with each shape except this oddball shape And the oddball shape it has to go ask the advisor the advisor knows and then it puts uh the the advisor tells the executor what to do with that oddball shape and we're seeing customers like bolt are are using this to help help manage their costs as well okay so let's switch back to the demo and we'll take a look at this step. So yeah, why don't we go ahead and add advisor to this. So what we're going to do with adding advisor is going to switch the model from, it was Opus, now you see that model line, it says Sonnet 4.6 plus Opus as advisor. So immediately, again, I saw the accountant's eyes light up because the price went down significantly, immediately, because we're doing all the tool calling and Python code, whatever. All that's being done with Sonnet, which is way cheaper than Opus. So you get immediate savings. But the question is, are you getting the intelligence as well? So there's one really important objective that we have here, and that is the Metropolis renewal This is a contract I been very worried about for HeroCorp they have to win the Metropolis Renewal And if we look at this advisor call the way it works is Sonnet went and looked at all of the Gong transcripts, all the data about the Metropolis Renewal, and Sonnet said, great, this looks good, this is on track. But then it said, oh, maybe I'll just call the advisor and just make sure because this is a high impact deal. And Opus reviewed the same transcript. It reviewed that same transcript and it said, oh, Sonnet, you missed a thing. Buried deep in the transcript, we see that cryothane is actually needed, that the mayor actually wants cryothane. And this is like way detailed and Sonnet just missed it. And so Sonnet said on track, but Opus caught it. So now you see the advantage. Not only you're using Sonnet for it being cheap. Oh, wait, the marketing team told me not to say it's cheap. It's not cheap, it's inexpensive. So Sonnet is inexpensive. And so we can use that, which is great, but you still get the intelligence [25:00] but just on demand, just exactly what is needed. And it catches this cryo thing. Okay, so we caught that. And so now you can see it says in red, it's not good. We need to do something. And I don't know about you, but I have some tension about this. I want to make sure that we can win the Metropolis contract. So if you scroll down a little bit, there's an actionable thing that the CEO can do. and that is to lock in cryothane. I also love these superhero names. It turns out the legal team wouldn't let me use any real superhero names, so we have to use cryothane. We can lock this in. So should we save the contracts? Who's in favor of saving the contract here? Yes? Okay, great. Thank you. You're with me. Okay, let's click that and lock in cryothane. We have it. Okay, so this is in the agentic world. This is all you do as a CEO is just click the button and you're good. Everything's saved. Okay, we've got just one minute. Thank you, Ben. Let me wrap us up. So back to slides. What we saw here is we saw it's very important to do prompt caching. If you're not doing that, ignore the rest of the talk and go do that. If you are doing that, really pay attention to context engineering. Make sure you are aware of what's in the context and then optimize that for what's actually needed. And finally we talked about using the advisor strategy So it really on intelligence But that not all Amazing stuff that we shipped in the platform just in the last few months So you want to call out my favorite things this Workload Identity Federation So if your security team is concerned about losing API keys, you need to worry no longer. Whiff is a great answer for that. And then I just love this Ant CLI command line tool. you can do in the console just about is available via command line tool. And the best thing about a command line tool is Claude just loves command line tools. So Claude, if you use it via Claude code, just tell it about the Ant CLI tool and it will go and manage everything for you and do just an amazing job at that. So the thing to really take away here is that betting on the platform means that the platform is going to keep getting better and your agents are going to keep getting better as new things keep coming out in the platform. Okay, so that's all. Thank you very much. I appreciate it. Outro Music