Cloudflare Releases AI Gateway Auto Router in Public Beta

Cloudflare Releases AI Gateway Auto Router in Public Beta Cloudflare has introduced Auto Router in public beta through AI Gateway, enabling organizations to automatically route queries to suitably capable AI models to cut spending without requiring manual model selection. Setting the model to 'cloudflare/auto' routes requests based on task difficulty, reducing costs by up to 30% internally compared to using only frontier models. In Cloudflare's internal 97-task benchmark, Auto Router achieved an 86.6% success rate at $0.0084 per success, compared to 84.2% for GPT-6 Sol and 96.6% for Claude Opus 5.5. The routing system evaluates tasks using edge-based classification across 14 categories and applies context-aware cache penalties before switching models. Automated Cost Reduction and Benchmarks Many users manually select models that exceed the needs of simpler tasks like email summarization. Cloudflare positions Auto Router as an automated decision-maker within AI Gateway, ensuring users only deploy frontier models when necessary. In general knowledge work benchmarks covering email, Slack, calendars, and finance tools, cloudflare/auto achieved an 86.6% success rate at $2.10 total cost ($0.0084 per success), delivering performance at 80% the cost of GPT-6 Sol and 35% the cost of Claude Opus 5.5. Two-Stage Routing Architecture When a request is submitted, AI Gateway first filters eligible models based on configuration, supported features, spend limits, and upstream health status. A multi-head classifier deployed on Workers AI then evaluates the latest conversation turns, outputting probabilities for 14 task categories alongside ratings from one to five for complexity, ambiguity, stakes, and context dependence. A scoring matrix blends these signals with benchmark data and token prices to select the model with the highest estimated utility. Cache Management and Architecture Advantages For long agentic workflows, model switching can discard context caches or reasoning tokens, requiring expensive context rewrites. Auto Router factors in cache read and write costs, imposing a switching penalty that increases with context depth to discourage unnecessary switches across turns. Furthermore, the modular two-stage design provides interpretable routing choices and allows new models to be integrated into the scoring matrix without model retraining.

Cloudflare 推出 AI Gateway Auto Router 公测版以降低 AI 成本 Cloudflare 宣布在 AI Gateway 中公测上线 Auto Router（通过设置模型为 cloudflare/auto）。该功能可自动根据任务复杂度将请求路由至兼顾质量与成本的合适模型，避免用户因全部默认选用前沿模型而产生浪费，目前在测试阶段免费提供。 将模型指定为 cloudflare/auto 即可启动自动路由，内部测试显示相比仅用前沿模型最高可节省 30% 成本。 在通用知识工作基准测试中，其成功率为 86.6%，成本仅为 OpenAI GPT-6 Sol 的 80% 以及 Anthropic Claude Opus 5.5 的 35%。 采用两阶段决策架构：先利用部署在边缘 GPU 上的分类器评估 14 个任务类别与 4 个维度，再结合基准矩阵和价格计算效用选择模型。 将 Prompt 缓存读写成本与模型切换惩罚纳入考量，避免长对话中频繁更换模型导致重复写入上下文开销。 背景与基准表现 许多组织在日常工作中往往直接选用 Opus 等顶级前沿模型处理简单任务，造成不必要的支出。Cloudflare 在模拟包含邮件、日历和财务等 97 项通用知识任务的基准测试中对比发现，cloudflare/auto 的成功率达到 86.6%（252/291），总成本仅为 2.10 美元（每次成功成本 0.0084 美元），相较 Claude Opus 5.5（96.6% 成功率，5.91 美元）大幅降低，并接近 GPT-6 Sol（84.2% 成功率，2.64 美元）的水准。 路由架构与决策机制 当请求到达时，网关会先根据格式、凭据、访问策略及健康状态筛选可用模型池。随后，运行在边缘网络 GPU 上的分类模型会分析近期对话，输出 14 类任务概率及复杂度、歧义度等 4 个维度的评分。评分矩阵进一步结合基准表现与 Token 价格计算综合效用：简单任务中价格权重更高以倾向小模型，高难度任务则放宽成本约束以保障质量。上线新模型时只需调整基准权重，无需重新训练分类器。 缓存成本与模型切换考量 在长代码编写或调试会话中，成本主要由缓存读取驱动，盲目切换模型会丢弃现有缓存并引发重新写入全量上下文的高额支出。Auto Router 引入了随上下文 Token 量递增的模型切换惩罚，单轮对话倾向于沿用持有热缓存的原模型，跨轮次切换则要求新模型能通过更低的读取费用或更少的 Token 消耗弥补重写成本。此外，未来计划更倾向于在同模型家族内切换，以防因跨家族丢弃推理 Token 而增加输出费用。

From our conversations with companies at every stage of their AI adoption journey, we've seen some common patterns. First, there is an exploration period as you bring on every new tool, dole out API keys freely, and let the tokens flow. Then, you converge on the canonical tools for your organization for agentic coding, for non-technical workflows, for running and deploying agents. As companies formalize their AI adoption, they want to manage and oversee token spend for users, but budgets and rules only go so far. The best savings are the ones users never notice. Today, we are releasing Cloudflare's Auto Router in public beta, available through AI Gateway. Set your model to `cloudflare/auto` and the Auto Router will automatically route each request to a model that is capable enough for the task, without requiring an end user to think about model selection. Our early results using the Auto Router internally through our OpenCode harness show a cost savings of up to 30% when compared to using only frontier models like OpenAI Sol and Anthropic Claude Opus. From our own experience tracking AI spend at Cloudflare, we’ve learned managing costs requires a multipronged approach. Previously, we talked about [how to set budgets](https://blog.cloudflare.com/ai-gateway-spend-limits) and limits around AI spend, and [how to see](https://blog.cloudflare.com/identity-aware-ai-gateway/) who is spending across your organization by linking employees to their AI usage. In many harnesses, including OpenCode, Claude Code, and Codex, individual users still select models manually. Of course, not all tasks are created equal, and often individuals end up using models that are overkill for their work. For example, you don't need Opus-level intelligence if you're looking to summarize an email or chat threads. However, you wouldn't want to block that model completely from your security engineering team. Our goal is for AI Gateway to be the control plane for organizations deploying AI internally. Because every request from every user, agent, and tool already flows through it, AI Gateway is in a unique position to do more than observe and enforce. Budgets, spend limits, and identity-aware analytics give organizations visibility and guardrails, but they still rely on individuals to make cost-conscious choices request by request. The next step is for the gateway itself to make intelligent decisions on a user's behalf: sending each request to a model that is capable enough for the task. That way, organizations reduce spend automatically, while users keep access to the most capable models when their work actually needs them. We use Auto Router internally at Cloudflare within our OpenCode deployment and within [Cloudflare OS](https://blog.cloudflare.com/cloudflare-os/), our custom agent harness. In our internal usage, we’ve seen results comparable with frontier models for coding tasks. Auto Router does best when used across a wide range of knowledge-work tasks, like those typically found in a large organization with work spanning both technical and non-technical teams. We evaluated `cloudflare/auto` against OpenAI’s GPT-6 Sol and Anthropic’s Claude Opus 5.5 on our internal general knowledge work benchmark. The benchmark uses simulated workspace tools and covers common day-to-day workflows across email, calendars, Slack, files, travel and finance. Each task requires the model to use these tools to produce a verifiable answer or complete an action. | Model | Successful Trials | Success Rate | Total Cost | Cost per success | | cloudflare/auto | 252/291 | 86.6% (+6.2/−6.9 pp) | $2.10 | $0.0084 | | Anthropic Claude Opus 5.5 | 281/291 | 96.6% (+2.7/−3.8 pp) | $5.91 | $0.0210 | | OpenAI GPT-6 Sol | 245/291 | 84.2% (+6.5/−6.9 pp) | $2.64 | $0.0108 | *97 tasks with three samples per model per task. Parenthetical values show 95% confidence intervals estimated from 10,000 task-level bootstrap resamples, preserving all three repetitions within each task. “pp” indicates percentage points.* Our Auto Router delivered similar performance to other state-of-the-art daily-driver models, coming in at 80% the cost of Sol and 35% the cost of Opus. While that may initially seem surprising, one way to frame the problem a model router solves is through the “jagged frontier” across models. The ability to solve a problem often exists somewhere in this portfolio of models; the router’s job is to choose the right model for each task while balancing quality and price. Savings come from not paying frontier rates for non-frontier work, and they grow with how much of that work you have. Another insight is that lower token prices do not always produce lower-cost outcomes. A model that looks cheaper on paper may end up using disproportionately more tokens to solve a problem. A router should minimize predicted trajectory cost, not just load-balance by dollars per million tokens. This is already useful today, but it’s only the beginning of what the Auto Router can learn from Cloudflare’s position in the inference path. When you send a request to `cloudflare/auto`, AI Gateway first builds the [pool of models](https://developers.cloudflare.com/ai-gateway/features/auto-router/#default-models) that can actually serve it. It filters out models that do not support the request format or execution mode, and accounts for the credentials, billing configuration, access control policies, and spend limits attached to the gateway. It will also filter out unhealthy upstream providers or models during downtime and automatically bring them back into the pool after an outage. For the remaining candidates, the router looks at a compact view of the conversation. It considers the most recent messages, prioritizing the newest turns. The conversation is then sent to a multi-head classification model running on Workers AI and deployed on GPUs across our edge network. The classifier produces two sets of signals. First, it assigns probabilities across 14 task categories (like coding, planning, research, data analysis). It then rates the request across four dimensions on a scale from one to five: complexity, ambiguity, stakes, and dependence on earlier context. A separate scoring matrix combines those signals with model benchmark results to estimate how well each model fits the request. To calibrate the scoring matrix, we defined the preferred model for a set of example task and difficulty profiles, then adjusted the weights to produce those choices. Finally, the router combines expected quality with each model's input and output token prices. On straightforward requests, price carries more weight, so a smaller model can win when it is capable enough. As difficulty rises, the cost penalty falls and stronger models have more room to win. In simplified terms, `cloudflare/auto` selects the model with the highest utility as defined by: For long agentic sessions like debugging or coding, cost is less driven by the model’s list price than by the cost of cache reads, which grows with session length. Switching models throws the cache away and forces a new model to write the whole context again. This can be worth it, as a model with a cheaper cache-read and cache-write prices can pay back the rewrite quickly. Rather than completely avoiding model switching, the Auto Router accounts for the cost of cache reads and writes. Within a turn (one user input loop), the cache is hot and switching rarely pays off, so it’s better to keep using the same model. Across turns, the Auto Router applies a switching penalty that grows with the number of tokens already in context. A model that still holds a live cache for the session is priced at its cheaper cache-read rate. Every other candidate is priced at the full cost of rewriting the context, so the deeper the conversation, the more a switch has to earn back, through higher quality results that use fewer tokens overall or cheaper cache rereads. Switching models has another cost: most models can't read another model's reasoning tokens, so a model switch that drops reasoning tokens means that the new model may have to redo it at output prices. In the future, we want to account for this by having the router prefer to stay within the same model family when it switches. From there, the router returns a ranked list. AI Gateway attempts the winner first and can move to another eligible model if that provider cannot serve the request. This overall design has several benefits. The two-stage architecture (task and dimensions classifier to scoring matrix) means that routing decisions are legible because you can inspect each task’s predicted category and complexity to see how it translated into the model choice. Adjusting the router when a new model is released also does not require retraining — we only add its benchmark-derived weights to the scoring matrix. The same classifier can also support different routing profiles. For example, in addition to `cloudflare/auto`, we plan to release other routers in the future, including `cloudflare/auto-best`, which uses the same classification and model pool, but selects the highest expected quality without applying the cost tradeoff. Our release today is only the starting point, and we’re continuing to invest in research and new routing strategies. In the near term, we want to: The Auto Router is free while in beta. Read more in our [developer documentation](http://developers.cloudflare.com/ai-gateway/features/auto-router/). *Acknowledgements: This project was also made possible by the efforts of Mats Dodd, Sam Scott, Oliver Yu, and Jeff Rafter.*