Anthropic Debuts Claude Effort Controls to Balance Token Costs

Anthropic 推出 Claude 努力程度控制以平衡 Token 成本 以较低努力程度运行较大模型，相比用尽较小模型的测试时计算，能带来更高的智能水平和更快的完成速度。 该视频介绍了测试时计算（test-time compute）的概念，并通过一个逼真的交通模拟示例，展示了调节 Claude 的努力程度如何影响输出质量、Token 消耗量以及执行时间。视频详细阐述了推理从死板结构向自适应思考的演进过程，在此过程中模型能够动态管理思考、工具调用和文本 Token。此外，视频还就如何利用努力程度旋钮与任务预算提供了实用指导，确立了在延迟和成本约束下，于低努力程度大模型与较小模型之间进行权衡选择的经验法则。 测试时计算（Test-time compute）让 Claude 能在推理阶段通过投入更多时间和 token 来解决复杂问题；在 Opus 4.7 交通模拟示例中，调高努力程度（Effort dial）直接带来了更逼真的逻辑与更高质量的输出。 (2:18) 推断 token 主要动态分配给思考（内部独白）、工具调用（环境交互）和文本生成（用户沟通）三个维度；最新的自适应思考（Adaptive thinking）允许 Claude 在每个环节自由决定何时以及进行多少思考，而无需外部路由介入。 (10:07) 开发者可以通过努力程度旋钮和任务预算（Task budgets）精准把控质量、延迟与成本的权衡；Opus 4.7 引入的“超高”（Extra high）档位在智能水平与开销之间取得了极佳平衡，已成为 Claude Code 和 Claude.ai 的默认选项。 (18:19) 模型选型经验建议在追求快速生成全部结果且兼顾高智力时采用“大模型+低努力程度”，而在重视首字延迟和批量任务时选用小模型；Anthropic 的最终愿景是让 Claude 仅凭质量目标和预算上限即可完全自主分配计算资源。 (19:41)

Anthropic product manager Matt Bleifer argues that scaling test-time compute enables reasoning models to solve increasingly complex engineering tasks, requiring users to govern performance, latency, and cost through dynamic effort dials and task budgets rather than binary thinking toggles. Token Buckets and Adaptive Thinking (5:31) Bleifer defines test-time compute as any expenditure of inference tokens to solve a problem, categorized into three functional groups: thinking tokens for chain-of-thought planning and internal scratchpad reflection, tool-calling tokens for environmental interactions like searching or editing files, and text tokens for user communication. While earlier reasoning models relied on rigid sequential execution or interleaved thinking strictly between tool invocations, Anthropic transitioned its evaluation benchmarks to 'adaptive thinking' starting with Opus 4.6. Adaptive thinking grants the model unconstrained autonomy to decide when, how much, or whether to think at any stage of execution. Rather than relying on external classification routers to toggle reasoning on or off based on perceived task difficulty, the model evaluates the necessity of reasoning per step, allowing it to bypass thinking entirely for simple prompts while preserving internal monologue for complex steps. Effort Dials Versus Binary Thinking Toggles (13:20) Anthropic contends that traditional binary toggles that turn thinking on or off are an ineffective proxy for effort. Bleifer compares this to managing a human colleague or using search tools: developers do not dictate an agent's internal monologue or command it to always or never search, but rather specify the desired rigor and let the model coordinate its own thinking, tool calls, and text outputs. To manage real-world execution costs and waiting times, Anthropic introduced effort dials and task budgets. The effort dial instructs Claude how to trade off speed, expenditure, and quality across all token types collectively, while task budgets enforce upper bounds on token consumption (such as a 100,000-token check-in limit), with future roadmaps targeting explicit runtime and financial limits. Scaling Outputs and Observed Task Shortcuts (2:18) In a benchmark traffic light simulation prompt run on Opus 4.7, scaling the effort dial produced distinct qualitative jumps. A low-effort run took approximately 50 seconds and consumed 4,600 output tokens, yielding a functional simulation that erroneously placed the light in the middle of the road. High effort doubled token consumption and runtime to add realistic multi-vehicle dynamics and reposition the light, while maximum effort used tenfold time and tokens to maximize visual polish. However, Bleifer notes that lower effort settings do not simply degrade reasoning; they can trigger emergent, resource-minimizing shortcuts. In Anthropic's 'Claude Plays Pokémon' evaluation, low effort induced speedrun behavior where the model actively minimized token spend by skipping optional trainer battles, relying on inventory items to avoid backtracking to healing centers, and deploying repels to bypass cave encounters. Strategic Tier Selection and Model Architecture (18:19) Because maximum effort frequently encounters diminishing returns on intelligence-bound tasks, Bleifer recommends empirical evaluations to plot performance against cost curves rather than defaulting to maximum settings. For standard coding and agent workflows, Anthropic set 'extra high' as the default tier in Opus 4.7, positioning high, medium, and low effort as graduated steps for increasingly cost- and latency-sensitive deployments. When choosing between model scale and effort levels, Bleifer suggests pairing larger models with lower effort settings for intelligence-sensitive workflows prioritizing rapid completion. In tests, Opus 4.7 at low effort consumed roughly identical tokens and runtime as Haiku 4.5 at maximum effort while achieving superior results. Conversely, smaller architectures remain optimal for high-volume classification, basic extraction, and scenarios demanding an immediate time to first token. Anthropic frames current effort controls as intermediate steps toward an ultimate goal where Claude autonomously manages its compute allocation across days or months based solely on user-defined quality thresholds and top-line budgets. Until fully autonomous allocation matures, Anthropic advises developers to keep thinking enabled, benchmark specific workloads across empirical effort curves, and review transcripts directly to detect unexpected shortcuts.

Anthropic研究团队产品经理Matt Bleifer指出，扩展测试时计算（Test-Time Compute）正成为大语言模型性能提升的关键杠杆。通过自适应思考、努力程度旋钮（Effort Dial）与任务预算机制，开发者能够在推理成本、延迟与求解质量之间实现动态平衡，而无须依赖粗暴的二进制思考开关。 测试时计算扩展与Token分配结构 (0:11) Bleifer解释称，与通过增大模型规模、增加数据和延长训练周期的“训练时扩展”类似，测试时计算允许模型在推理阶段投入更多时间与计算资源来解决复杂工程难题。Anthropic将测试时计算定义为模型在推理阶段消耗Token以解决问题的行为，并将其细分为三类：用于内在思维链推演和步骤规划的“思考Token”、用于读写文件及外部检索等环境交互的“工具调用Token”，以及用于向用户汇报进度和交付成果的“文本Token”。 以Opus 4.7生成的单向街道交通仿真实验为例，Anthropic展示了努力程度对输出质量的直接影响：低努力模式耗时约50秒并消耗约4,600个输出Token，生成了逻辑基础且将红绿灯错误置于路中央的仿真；高努力模式的耗时与Token消耗翻倍，模型纠正了信号灯位置并引入了智能驾驶模型；当设置为最大努力时，时间和Token消耗达到低努力模式的10倍，换取了高度真实的行车动力学表现与图形细节。Bleifer预测，未来模型运行时间将从数分钟扩展至数周甚至数年以攻克人类重大挑战。 思考机制演进：从交错思考到自适应思考 (10:07) Bleifer梳理了推理模型消耗Token方式的演进过程。早期推理模型遵循严格的线性次序：先消耗思考Token，再调用工具，最后输出文本；随后Anthropic引入了“交织思考（Interleaved Thinking）”，允许模型在调用工具并获取结果后重新进行思考，再决定下一步动作。最新推出的“自适应思考（Adaptive Thinking）”则解除了所有时序与配额约束，允许Claude在整个任务执行流程的任意步骤根据需要自主决定思考的时机与体量。 Bleifer明确指出，自适应思考并非依据输入难易度进行模型切换的路由分流器或外部自动化开关，而是将思考的主动权完全赋予模型单步决策过程；遇到简单任务时，模型可自主选择完全不消耗思考Token。基准测试表明，从Opus 4.6起，Anthropic已全量采用自适应思考，在维持或超越交织思考性能的同时，提供了更自然的交互体验。 努力程度旋钮与传统思考开关的差异 (13:20) 针对以往在用户界面中手动启闭“思考开关”的模式，Bleifer认为这是控制模型努力程度的拙劣手段。强行关闭思考剥夺了模型的核心认知能力，如同强行要求人类协作伙伴关闭内心独白，或硬性规定模型何时调用搜索工具。相比之下，Anthropic推出的“努力旋钮（Effort Dial）”能够驱动思考、工具调用与文本输出协同增减；配合新上线的“任务预算（Task Budgets）”功能，开发者可以设定Token消耗上限（例如达到100,000个Token时暂停并汇报），未来还可扩展至时间或财务成本预算。 在降低努力程度以节约开销时，模型可能会采取出人意料的捷径。Bleifer分享了在《宝可梦·红》评测中观察到的现象：在低努力设置下，Claude展现出类似游戏“竞速通关（Speedrun）”的资源节约策略，包括跳过非必要战斗、使用背包伤药替代折返宝可梦中心、频繁使用喷雾避开野生宝可梦。Bleifer指出，这种低努力表现并非智力退化，而是展示了模型在严格限制下达成目标的高效规划能力。 档位选择策略与模型规模权衡 (19:41) Bleifer提醒开发者，最大努力档位（Max Effort）往往伴随边际收益递减，并不一定代表最高性能或最佳性价比，通常仅建议在最极端复杂的智力任务中试用。为此，Anthropic在Opus 4.7中新增了“超高努力（Extra High）”档位并设为默认项，旨在不造成浪费的前提下最大化输出智能，成为多数编程任务的首选。常规分级中，高努力适合智能敏感型任务，中等努力适合平衡成本与速度，低努力则适合短周期、低延迟工作。 在“大模型配低努力”还是“选用小模型”的取舍上，Bleifer指出，对于既要求基础智力又对速度敏感的任务，在更大模型上运行低努力档位往往更具性价比。例如低努力的Opus 4.7与满负荷运行的Haiku 4.5消耗相当的Token与时间，但生成质量显著优于后者。Bleifer总结了一条经验法则：若系统追求极短的首字生成延迟（TTFT），小模型表现更优且适合批量文本抽取或分类；若追求以较快速度交付完整的复杂成果（TTLT），则应优先采用大模型搭配低努力设置。 Bleifer指出，Anthropic的技术终极愿景（North Star）是实现模型对计算资源的完全自主优化分配，未来开发者只需定义质量底线与成本预算，其余均由模型动态规划。尽管努力旋钮与自适应思考奠定了基础，但团队强调这仅是开端；在实际工程实践中，开发者仍须运行严格的基准评测并深入查阅模型交互日志，而在缺乏评测的编码场景下，建议直接选用超高努力配置。

[2:18] Matt Bleifer: Claude will eventually work on problems not just for seconds or minutes, but for "days, weeks, months, even years, spending tokens to try to solve some of humanity's toughest challenges." [5:31] Matt Bleifer: any form of Claude spending tokens at inference time in order to solve your problem. [10:07] Matt Bleifer: Claude is free to think whenever appropriate. [13:20] Matt Bleifer: poor proxy for an effort dial [13:20] Matt Bleifer: We don't tell Claude to always search or never search. [14:57] Matt Bleifer: Claude Plays Pokémon [18:19] Matt Bleifer: without kind of going overboard [19:41] Matt Bleifer: use small models for a fast time to first token, use bigger models at lower effort for a fast time to last token. [21:56] Matt Bleifer: space and that inner monologue to be able to work through your problem as efficiently as possible, [21:56] Matt Bleifer: if you have evals, use them. [21:56] Matt Bleifer: go with extra high [21:56] Matt Bleifer: they're really just the beginning and there's a lot more to come.

[0:11] All right. [0:12] Hello everyone and welcome. My name is [0:16] Matt Bleifer. I am a product manager on [0:19] Anthropic's research team. And today I [0:22] will be sharing a little bit how [0:25] Claude leverages compute at inference [0:27] time, otherwise known as test time [0:30] compute, in order to break down and [0:32] solve some of your hardest software [0:34] engineering challenges. [0:37] Along the way, I'll share a little bit [0:38] about what levers you have at your [0:40] disposal in order to influence how [0:42] Claude spends tokens, and I will also [0:45] share some best practices to help you be [0:47] able to get the most out of it. [0:53] So one of the key developments in large [0:55] language models over the last couple of [0:57] years has been the scaling of test time [0:59] compute, creating something that we've [1:01] all come to know as reasoning models. [1:04] Similar to how we can scale compute at [1:07] training time by training bigger models [1:09] over longer time horizons using more [1:12] data, we can also scale compute at test [1:15] time by allowing those models to spend [1:17] more time working on a problem. [1:20] So if you look at this graph on the [1:22] left, you can see that when we move from [1:25] Haiku to Sonnet to Opus, as the model [1:28] gets more intelligent, it's able to get [1:30] a better score on our agentic coding [1:33] evaluation. [1:35] And then similarly in the graph on the [1:37] right, as that same model Opus actually [1:40] just spends more time working on a [1:42] problem, it's able to correspondingly [1:44] get better and better scores. This is [1:47] what we mean by scaling test time [1:48] compute. [1:54] Now this This true just of software [1:56] engineering, It's really true of a whole [1:58] variety of knowledge work domains, [2:01] whether it's agentic search, computer [2:03] use, or PhD level academic reasoning. If [2:07] we can allow models to spend more time [2:10] working on a problem, they can achieve [2:12] better and better results. [2:18] So, looking at a bunch of charts and [2:19] graphs is always great for understanding [2:21] the data and the different correlations, [2:24] but nothing really beats seeing a [2:26] tangible example of what this looks like [2:28] in practice. And so, what I went ahead [2:31] and did is I ran Opus 4.7 on a few [2:35] different effort levels, scaling the [2:37] amount of time that it works on a given [2:39] prompt. [2:40] Where in this case, I asked it to create [2:42] a realistic simulation of cars going [2:45] down a one-way street at a traffic [2:47] light. [2:51] So, the first result we have here is [2:54] Opus 4.7 running on low effort. [2:57] You can see in the simulation, it took [2:59] about 50 seconds to produce a result for [3:02] us and worked for about 4,600 output [3:05] tokens. [3:07] And I'd say it accomplished something [3:08] that was fairly reasonable. We do in [3:11] fact have cars going down a one-way [3:13] street. [3:14] Uh they are stopping at the traffic [3:16] light. [3:17] But overall, it's a pretty basic [3:18] simulation. The traffic flow is fairly [3:20] basic. [3:22] Uh the graphics are limited. And for [3:24] some reason, Claude thought it would be [3:26] a great idea to put the traffic light [3:27] right in the middle of the road, which [3:29] maybe wasn't the best design decision, [3:31] but we will still call it functionally [3:33] passing. [3:38] So, the next thing I did here is I went [3:40] ahead and I cranked that effort dial up [3:42] a bit. So, when I moved effort up to [3:44] high for Opus 4.7, [3:47] it took about twice the time working on [3:49] our traffic simulation and double the [3:51] output tokens. But as you can see, it [3:54] was able to achieve a better result. [3:57] It has cars of different types. It [3:59] smartly moved the traffic light over to [4:02] the side of the road. And Opus told me [4:04] that it even implemented what it called [4:06] an intelligent driver model where every [4:09] car would more uniquely respond to the [4:12] dynamics of the car around it, doing a [4:14] better job simulating simulating a [4:17] realistic traffic pattern. So again, [4:20] twice the amount of time, better [4:21] results. [4:26] Now, the last thing that I did here is I [4:28] cranked that effort dial all the way up [4:30] to max. [4:32] And in this setting, Opus 4.7 took 10x [4:35] the amount of time that it did when [4:38] executing this same prompt on low effort [4:41] and 10x amount the amount of tokens. [4:43] But as you can see, it was able to [4:45] achieve the best results yet. We have [4:48] the best graphics, my favorite traffic [4:50] light of all of them, and really [4:52] realistic driving patterns. [4:55] This is all an example of how when you [4:56] allow Claude, even on the same model, to [4:59] just spend more time working on a [5:01] problem, it can get better results. [5:08] As we continue to scale test time [5:10] compute further and further, Claude [5:12] isn't just going to work for seconds or [5:15] minutes or hours on a problem. It's [5:17] going to work for days, weeks, months, [5:20] even years, spending tokens to try to [5:23] solve some of humanity's toughest [5:25] challenges. [5:31] So when I talk about test time compute, [5:33] I really mean any form of Claude [5:35] spending tokens at inference time in [5:38] order to solve your problem. [5:40] However, we can break these token types [5:43] down into three a of distinct buckets. [5:50] The first bucket that we have is [5:52] thinking tokens. This is the classic [5:54] form of tokens that underlying [5:56] underlies what we know as reasoning [5:58] models. [6:00] Thinking tokens represent Claude's [6:02] internal monologue. It's Claude's space [6:04] to reason step-by-step, to consider [6:07] different potential options, do some [6:09] chain of thought reasoning, create a [6:11] scratch pad where it needs to work [6:12] through a problem, and ultimately spend [6:15] time thinking through what it needs to [6:17] do in order to take the best actions and [6:19] deliver the best results. [6:24] The second form of tokens that Claude [6:25] can spend when taking on a task is tool [6:28] calling tokens. [6:30] Tool calling is Claude's way of [6:32] interfacing with the rest of the world. [6:35] Whether it's using tool calls to execute [6:38] a search in this example, giving me more [6:40] information about the Code with Claude [6:42] conference, [6:44] or reading and writing files in order to [6:46] build out software engineering projects. [6:49] There's really millions of different [6:51] tools that Claude can call, but in all [6:53] of these scenarios, tool calling tokens [6:55] are Claude's way of interfacing with its [6:57] environment. [7:02] The last type of tokens that Claude can [7:04] spend is text. And this is Claude's way [7:06] of interacting with you. [7:08] Whether it needs to give you updates as [7:11] it's working on a really, really tough [7:12] problem and let you know how it's [7:14] progressing, [7:15] give you a summary at the end to explain [7:17] all of the things that it did in [7:19] response to the tough task that you gave [7:21] it, or simply just responding to a [7:24] simple question that you have, text [7:26] tokens are Claude's way of communicating [7:28] with an end user. [7:30] So again, we have three different types [7:31] of tokens: thinking, tool calling, and [7:34] text. And all three of these we think of [7:36] as really fundamental to the way in [7:39] which Claude works and the way in which [7:41] Claude responds to problems. [7:47] But all of these tokens that we're [7:48] spending have really direct costs to [7:51] users in the form of both practical [7:53] token costs that we pay for, [7:56] as well as waiting time when Claude [7:58] spends more tokens, it means that we as [8:00] users have to wait longer for our [8:02] result. [8:03] And so, we think it's really important [8:05] that we give users the ability to [8:08] influence or constrain how Claude spends [8:10] tokens. [8:12] Using Claude, [8:14] users can express their preferences and [8:16] constraints in a couple ways. [8:18] The first way is with that effort dial [8:20] that I talked about. [8:22] Effort is a way for you to tell Claude [8:24] how you want it to trade off time, cost, [8:28] and quality when responding to your [8:30] task. [8:32] Should Claude spend more time in order [8:34] to get a better answer? [8:36] Should it spend less time in order to [8:38] get a faster answer? These are all [8:40] preferences that you can give to Claude [8:42] as a user that it will take into account [8:45] when it goes and it spends these tokens. [8:49] Another form of constraints that users [8:51] can provide is in the form of budgets. [8:54] Recently, we launched a feature that we [8:56] call task budgets, which allows you to [8:58] tell Claude an upper bound of the amount [9:00] of tokens that it will spend when [9:02] working on a task. So, you might say, [9:05] "Hey, I want you to build out this [9:07] particular software engineering feature [9:09] for me, but I don't want you to spend [9:11] more than 100,000 tokens before you stop [9:14] and check in with me as a user." [9:17] Budgets could come in the form of [9:19] tokens, [9:20] but they could also come in the form of [9:22] time or cost. And I think this will get [9:25] increasingly important as we continue to [9:27] move up that exponential and Claude is [9:29] working for days, weeks, months, or more [9:32] to be able to give some guidelines about [9:33] how long it should work on a particular [9:35] problem before it stops to check in. [9:39] Now, given all of these preferences and [9:41] constraints, [9:43] it's really up to Claude to figure out [9:45] the best way to spend those tokens in [9:47] order to maximize outcome. So, given the [9:51] user's effort settings, given a [9:52] potential budget setting, how does [9:54] Claude allocate tokens across thinking, [9:57] tool calling, and text in order to [9:59] maximize its performance and its user [10:01] experience? [10:07] When reasoning models were first [10:08] introduced, they followed a really [10:10] specific pattern in terms of how to [10:12] spend these tokens. [10:14] The first thing they would do is they [10:15] would think, and they would spend those [10:16] thinking tokens to work through a [10:18] problem, and then after that, they would [10:20] move on to tool calling, and then [10:21] lastly, they would move to text. [10:25] We improved on this when we introduced [10:27] interleaved thinking, which allowed [10:29] Claude to actually use thinking and [10:31] reasoning in between tool calls. [10:34] In this mode, Claude could call a tool, [10:38] get a result, think about that result, [10:40] then determine what it wants to call [10:42] next, so on and so forth, all the way [10:45] until it decides to give a final answer. [10:52] Recently, we launched adaptive thinking, [10:55] and adaptive thinking is the next [10:56] evolution on top of interleaved [10:58] thinking. [10:59] In this new paradigm, [11:01] Claude is free to think whenever [11:03] appropriate. There's really no [11:04] constraint on when Claude needs to [11:07] think, how much it needs to think, or in [11:09] what order it spends any of these [11:10] tokens. It can leverage thinking, tool [11:13] use, and text in whatever order is [11:16] needed in order to best meet the [11:18] requirements of your task. [11:21] Claude could choose to start with a text [11:23] response in order to acknowledge the [11:25] user request. It could stop to call a [11:27] tool. It could then think about that [11:29] tool, respond to the user to give an [11:32] update, continue to call tools, so on [11:34] and so forth, all the way until it [11:36] provides that final answer of the work [11:38] that it did. [11:40] Claude could also choose not to think at [11:42] all for simple queries that don't [11:44] require it. Now, in practice, Claude [11:47] will typically think more often and [11:49] longer in response to higher effort [11:52] levels, but everything is really prompt [11:54] dependent. You can imagine that if I [11:56] were to survey someone in the in the [11:57] crowd here and say, "What is 2 + 2?" and [12:01] I ask them to spend a little bit amount [12:03] of time on the problem versus a lot of [12:04] time on the problem, you're roughly [12:06] going to spend the equivalent amount of [12:08] time working on that problem. [12:10] However, the story would change a lot if [12:13] I asked you to conduct a really [12:14] sophisticated research task. The [12:16] difference between your thinking on [12:18] low-effort and high-effort there could [12:20] be quite dramatic. [12:26] Now, adaptive thinking is not a model [12:28] router, and it's not an automated [12:30] thinking toggle. So, it's not taking [12:32] your query, classifying it based off of [12:35] difficulty, and figuring out whether it [12:37] should use a thinking version of the [12:39] model or a non-thinking version of the [12:41] model. [12:42] Rather, it's the difference between [12:44] telling Claude, "You must spend at least [12:46] one thinking token at the start of this [12:48] response." and telling Claude, "You can [12:50] spend thinking tokens whenever and [12:53] however needed in order to solve this [12:55] problem." [12:56] It's really about Claude having the [12:58] option to think at every single step of [13:00] the process. [13:03] We run all of our benchmarks on adaptive [13:05] thinking since Opus 4.6, and it's really [13:08] our intelligence maximizing setting that [13:11] shows performance parity or better with [13:13] inner leaf thinking while delivering a [13:14] better user experience. [13:20] So, I want to dig a little bit more into [13:21] effort and contrast it to the ways in [13:23] which we've used thinking in the past. [13:26] Historically, users have used thinking [13:29] toggles a lot like an effort dial. So, [13:32] if you wanted Claude to spend more time [13:34] working on a problem, you might turn on [13:36] thinking inside of Claude.ai or Claude [13:39] Code, and you would expect that it would [13:41] spend more time working in order to give [13:43] you a better result. [13:46] That's a pretty reasonable instinct. [13:47] However, thinking toggles are kind of a [13:49] poor proxy for an effort dial. [13:53] Rather than expressing how hard you want [13:55] Claude to work, what you're really doing [13:57] is you are turning on and off a really [13:59] core capability of the model. You're [14:01] constraining how it's allowed to work, [14:03] not how hard you want it to work. [14:07] An effort dial is a much better [14:08] expression of the idea of spend more [14:11] tokens in order to get a better answer. [14:14] It moves thinking, tool use, and output [14:17] text all together instead of just [14:19] toggling one of them. [14:21] As an analogy with tool use, we don't [14:23] tell Claude to always search or never [14:26] search. We tell Claude to figure out [14:28] when it should search based off of the [14:30] problem at hand, and that's really what [14:32] allows Claude to be agentic in response [14:35] to your query. [14:37] In a similar way, when we work with all [14:39] of our teammates, uh we don't ask them [14:41] to turn their inner monologue on and off [14:43] in response to a question. Uh we ask [14:45] them how hard to try, and then our [14:47] teammates decide how hard they are going [14:49] to think on that problem and what [14:51] actions that they will take in response. [14:57] So, I wanted to dig in a little bit more [14:58] on effort and give you some practical [15:00] guidance of how you should think about [15:02] setting these effort levels depending on [15:04] your use case at hand. [15:07] First, whenever possible, it's always [15:09] best to run evals and then chart [15:11] performance where you compare on your [15:13] x-axis here something like total tokens, [15:16] time, or cost, and on your y-axis [15:19] performance. [15:20] This allows you to create an effort [15:22] curve like this and get a better idea of [15:25] what trade-offs you might make by [15:27] selecting a given effort level. [15:30] Higher effort is going to improve [15:31] performance on most intelligence-bound [15:33] tasks, [15:34] but it also may show diminishing [15:36] returns. For your use case, you might [15:39] look at a graph like this and say, "I [15:41] will spend whatever tokens I need in [15:43] order to get the best intelligence." Or [15:45] you might say, "The relative improvement [15:47] in performance between extra high and [15:49] max is not worth the difference in [15:52] tokens that I will spend, and so extra [15:53] high is a better setting for my use [15:55] case." [15:58] Low effort can instead help accomplish a [16:00] task much quicker and save you a lot of [16:02] tokens, but it's also going to limit how [16:04] thorough Claude is in accomplishing the [16:07] task at hand. [16:08] As a quick tip, my advice is that when [16:11] using low effort, Claude is really [16:13] trying to save tokens as much as [16:14] possible, and so sometimes you may catch [16:16] it taking unexpected shortcuts that you [16:18] didn't expect it to. And so, in addition [16:21] to looking at evals, we always think [16:23] it's a best practice to spend time [16:25] reading your transcripts and better [16:26] understand exactly how Claude is [16:29] responding at a given effort level for [16:31] the thing that you're asking it to do. [16:37] On the flip side, uh low efforts have [16:39] also surprised us in some really [16:41] interesting ways. So, one of my favorite [16:43] evaluations that we've created is called [16:45] Claude Plays Pokémon, where uh Claude [16:48] gets the opportunity to work its way [16:50] through the original Pokémon Red game [16:52] that many of us uh grew up knowing and [16:54] loving. [16:56] When we ran Claude Plays Pokémon on low [16:59] effort, something really interesting [17:01] happened, and it actually ended up [17:03] treating the game much like a speedrun. [17:05] So, it would skip trainer battles in [17:07] order to save itself some time. It would [17:10] use healing items that it stocked up on [17:12] instead of wasting time going back to [17:14] Pokémon Healing Centers, and it would [17:17] spam an item called a repel that would [17:19] limit disruptive encounters with other [17:21] Pokémon, making it through caves much [17:23] more quickly. [17:25] And what I find most interesting about [17:27] this is often times we might correlate [17:29] low effort with lower intelligence, but [17:31] for any of us that grew up playing this [17:33] game, what you really realize is this is [17:35] a super clever strategy. It takes a [17:37] certain amount of intelligence to figure [17:39] out how you might minimize token spend [17:42] in order to get through these levels as [17:43] fast as possible. And so it was [17:45] interesting to see how Claude's [17:47] interpretation of low effort translated [17:49] to beat the game as fast as possible [17:51] employing actually very clever [17:53] strategies along the way. [17:59] So Evals are always ideal. I'll always [18:01] champion them anytime that you have [18:03] them. But I wanted to just give you some [18:04] quick rules of thumb on how you might [18:06] select an effort setting in the absence [18:09] of Evals or even if you have them. This [18:11] is just a good way to think about [18:12] things. [18:14] First, max effort, no surprise, can [18:17] deliver gains on your hardest tasks. But [18:19] as I mentioned before, it can sometimes [18:21] show signs of diminishing returns. [18:24] I recommend testing it for your most [18:26] intelligence demanding use cases, but [18:28] don't always assume that this is going [18:30] to be either the ceiling on performance [18:33] or really the best bang for your buck. [18:35] It could be the case that a level down [18:37] is going to give you roughly equivalent [18:39] performance at a real fraction of the [18:40] cost. [18:43] Extra high effort is a new setting that [18:45] we introduced with Claude Opus 4.7, and [18:48] we found this to be the best setting for [18:50] most coding and agentic use cases. [18:53] This is currently our default in Claude [18:55] Code and Claude.ai for Opus 4.7. And [18:58] like I said, it really does a good job [18:59] maximizing intelligence without kind of [19:01] going overboard. [19:04] High effort is a great setting if you're [19:05] trying to balance token usage and [19:07] intelligence, and this is probably the [19:09] value that I would recommend for any [19:11] intelligence sensitive use case. High is [19:14] a good place to start and test up from [19:16] there. [19:17] Medium is good for cost-sensitive use [19:19] cases where you're willing to trade a [19:21] little bit of intelligence [19:22] in order to get a much faster result. [19:25] And then low is good for reserving for [19:27] kind of your short-scope tasks and [19:29] latency-sensitive workloads. [19:31] However, as I mentioned before, it's [19:32] always good to just put it in practice [19:34] and see what actually happens because it [19:36] might surprise you. [19:41] I mentioned at the start of the talk [19:42] that test time compute is a second way [19:45] of scaling intelligence as compared to [19:47] training time compute. [19:50] So, it kind of begs the question if both [19:51] give similar tradeoffs with respect to [19:53] performance, speed, and cost, [19:56] when should I use a smaller model or [19:59] when should I use a lower effort level [20:01] on a bigger model? [20:03] As some quick guidelines, I would say [20:05] first, low effort on a bigger model is [20:08] good for an intelligence-demanding use [20:10] case where you're trying to optimize for [20:11] speed. [20:13] Going back to our example here of our [20:15] traffic light simulation, you can see [20:17] that Opus 4.7 on low effort spent about [20:20] the same amount of output tokens and [20:22] only took a little bit longer than Haiku [20:24] 4.5 on max effort, but I would say it [20:27] achieved a much better result. So, often [20:29] the low effort on the larger more [20:32] intelligent model can give you a better [20:33] bang for your buck when trading off [20:35] speed versus intelligence on [20:38] an intelligence-demanding use case. [20:41] On the flip side, [20:43] smaller models can be really good if [20:45] you're trying to optimize cost and your [20:47] use case is not too [20:48] intelligence-demanding. [20:50] So, if you have some simpler LLM tasks, [20:52] especially if you need to do them in [20:54] bulk, something like classification, [20:56] information extraction, basic [20:58] summarization, that's where small models [21:00] are going to come in handy and they're [21:02] going to be able to save you a lot of [21:03] cost when you don't need peak [21:05] intelligence. [21:07] Another case where small models are [21:08] really useful is if your application [21:11] demands a really low time to first [21:12] token. So, if you want Claude to give [21:15] responses as fast as possible in [21:17] response to a user query, the nature of [21:19] the smaller models means that they will [21:21] produce tokens often times much sooner [21:24] sooner and give you a better time to [21:26] first token. The way that I think about [21:28] this is use small models for a fast time [21:30] to first token, use bigger models at [21:32] lower effort for a fast time to last [21:34] token. [21:37] Wherever possible, as I said before, I [21:39] recommend evaluating both. It's good to [21:41] build these eval curves across a few [21:43] different model types and across various [21:46] effort levels and then look at what the [21:48] tradeoffs give you for your use case [21:50] that you're trying to optimize. [21:56] All right. So, before closing the talk, [21:57] I wanted to just summarize three key [21:59] actionable items that I hope that you [22:01] take away from this. [22:03] One, enable thinking whenever possible [22:06] in order to give Claude that space to [22:08] reason. Thinking, like I said, is really [22:10] core to how Claude works and gives it [22:12] that space and that inner monologue to [22:14] be able to work through your problem as [22:15] efficiently as possible. [22:17] If you want to modulate the amount of [22:18] time that Claude is spending thinking on [22:20] your problem, I recommend using effort [22:22] levels or budgets as your way of [22:24] influencing Claude's behavior. [22:27] Second, I might sound like a broken [22:29] record here, but if you have evals, use [22:31] them. [22:32] Use that to find your ideal balance. [22:34] Chart your curves, test on different [22:36] effort settings, test with different [22:37] budgets, test with different models, [22:40] look at what the performance gives you [22:42] and decide what makes sense for your use [22:43] case without forgetting to always dig in [22:46] and read those transcripts. [22:48] And lastly, if you're not going to do [22:50] any of that and you just need to make a [22:51] choice and you're working on anything [22:53] coding and software engineering related, [22:55] my advice would be go with extra high. [22:58] It's a pretty good setting and gives a [22:59] great bang for your buck while [23:00] delivering great intelligence. [23:06] Our North Star for Claude overall is [23:07] that it allocates compute incredibly [23:10] well when asked for it, and that you can [23:12] set a quality bar and a budget, and [23:14] Claude will just go ahead and figure out [23:16] the rest and give you the best [23:18] performance for your use case. [23:21] Adaptive thinking and effort levels and [23:23] budgets are all a step in this [23:25] direction, but they're really just the [23:27] beginning and there's a lot more to [23:28] come. I'm excited to share more with you [23:30] in the future, so stay tuned and thanks [23:32] so much for taking the time. [23:34] If you want to chat more about this, [23:36] I'll be around the conference in the [23:38] audience. I'm always happy to nerd out [23:40] about these things, so thank you.