Cloudflare Releases Open-Source Clef Decision Models and RL Fine-Tuning Service

Cloudflare Releases Open-Source Clef Decision Models and RL Fine-Tuning Service Cloudflare has introduced Clef and Clef-flash, open-source decision models hosted on Workers AI designed to deliver structured, non-autoregressive classifications for agentic workflows, alongside a reinforcement learning fine-tuning platform. Clef and Clef-flash are open-sourced under an Apache 2.0 license and provide Jev-compatible, strictly typed probability outputs for routing and automated decisions. Unlike Jev, Clef includes a vision encoder to handle visual content and supports a 64k context window compared to Jev's 32k. The models rely on frozen Qwen backbones with low-rank adapters and non-autoregressive scoring, bypassing token-by-token generation for reduced latency. Cloudflare is offering fine-tuning via its forward-deployed engineer team, with plans for a self-serve RL platform built on AI Gateway, Containers, and Workers AI. Model Features and Capabilities Decision models produce bounded, structured outputs with probabilities to assist programmatic agent routing without requiring humans in the loop. Clef incorporates a vision encoder for visual classifications and expands context length to 64k tokens. It provides fully compatible APIs with Jev, allowing users to swap models easily, and runs at Cloudflare's edge on Workers AI or locally via weights published on Hugging Face. Architecture and Training Methodology Clef is built using frozen Qwen backbones—Qwen3.8-27B for Clef and Qwen3.5-9B for Clef-flash—combined with rank-256 low-rank adapters and an optimized routing head. Instead of generating intermediate text autoregressively, the model uses a prefill-only pass and a two-stage attention routing process to score valid schema options directly from internal backbone representations. Post-training applies label-smoothed cross-entropy, Brier loss for probability calibration, synthetic permutation datasets, and Reinforcement Learning for Calibrated Decisions (RLCD). Benchmark Results and Latency In benchmark testing, Clef led across several evaluations on the Jev Decision Index, including BFCL (98.47 for Clef, 98.76 for Clef-flash) and BANKING77 (94.20 macro-F1). Across 43 eval benchmarks, Clef models demonstrated lower median latency than Jev, logging 209.3 ms for Clef and 38.8 ms for Clef-flash against Jev's 524.1 ms. An internal domain classification test by Cloudflare's Threat Intelligence team completed in 2.2 seconds with Clef, compared to 4.7 seconds using a general LLM. RL Fine-Tuning Service and Platform To address domain-specific use cases such as bot detection and support ticket triage, Cloudflare is rolling out fine-tuning services. The initiative starts with hands-on assistance from Cloudflare's forward-deployed engineer (FDE) team, aiming to evolve into a self-serve platform. The infrastructure integrates existing Cloudflare AI platform components, including AI Gateway for request capture, Containers for RL sandboxes, and Workers AI model deployment features.

Cloudflare推出开源决策模型Clef及强化学习微调服务 Cloudflare正式发布自研开源决策模型Clef与Clef-flash，支持视觉输入及64k上下文，并在Workers AI托管与Hugging Face开源，同时公布了基于强化学习的定制微调服务规划。 Clef与Clef-flash以Apache 2.0协议开源，兼容Jev API，并集成视觉编码器与64k上下文窗口。 采用基于Qwen的非自回归架构，省去逐字生成阶段，实现更低时延与结构化概率输出。 在多项基准与延迟评估中表现优异，Clef-flash中位延迟仅38.8毫秒。 Cloudflare正通过前线部署工程师（FDE）团队为客户提供Clef定制微调，后续将推来自助RL平台。 决策模型概念与Clef核心特性 决策模型旨在帮助AI智能体根据输入概率输出受限的结构化结果，无需反复微调即可处理新增分类。Clef与Clef-flash模型以Apache 2.0协议在Hugging Face开源，并托管于Workers AI。与仅支持文本的Jev相比，Clef具备视觉编码器且拥有64k上下文窗口（Jev为32k），同时全面兼容Jev API。 性能表现与基准测试 在Jev Decision Index与各类工作流基准中，Clef在BFCL、ToolRet等测试中展现出高准确率，且在Typesafe评测的4个场景中有3项领先Jev。在43项评估延迟测试中，Clef-flash的中位延迟达到38.8毫秒、p95延迟为122.4毫秒，显著快于Jev的524.1毫秒中位延迟与536毫秒p95延迟。 技术架构与训练方法 Clef放弃逐token生成的自回归流程，改以冻结的Qwen3.8-27B（Clef）和Qwen3.5-9B（Clef-flash）为底模，仅执行prefill阶段并通过两阶段注意力路由直接从内部表征评分。训练过程中联合优化路由头与rank-256 LoRA，采用标签平滑交叉熵和Brier损失，并开发针对校准决策的强化学习（RLCD）算法以防分布偏移。 内部应用与RL微调平台规划 Cloudflare已在威胁情报（域名快速分类）、安全审核与Bot检测中验证Clef，分类延迟相比通用LLM减少逾50%。针对特定业务需求，官方先期通过FDE团队协助客户微调模型，未来将整合AI Gateway、容器沙箱及Workers AI能力，打造全流程自助强化学习微调平台。

Over the last few weeks, there has been lots of buzz around decision models such as [Typesafe AI’s Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) System One model. While classifier models have been around for some time, Jev introduces a new decision model concept into the world of AI — a model that produces bounded structured outputs cheaply, quickly and consistently that can be added into a workflow when a decision is required. These models are capable enough to work over any set of inputs without constantly retraining the model to incorporate new classification categories. This contrasts with the world of Large Language Models (LLMs), which are largely non-deterministic, but are open-ended enough to reason and generate text and tool calls for agentic workloads. Today, we’re releasing two Cloudflare-trained decision models, Clef and Clef-flash, [hosted on Workers AI](https://developers.cloudflare.com/workers-ai/models/clef). Clef is currently the leader when evaluated against the [Jev Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index), you can view full results on the [live benchmark demo site](https://clef-evals.workers-ai-mle.workers.dev). These models are smarter, faster, and fully Jev-API compatible, so you can experiment with these hosted models easily. We’re fully open-sourcing these [models on Hugging Face](https://huggingface.co/Cloudflare/clef) under an Apache 2.0 license for you to run locally and experiment with yourselves. Lastly, we’re excited to debut our new reinforcement learning (RL) product, which allows customers to fine-tune Clef to suit their use cases as well. A decision model makes classifications to help agents decide how to act, based on certain probabilities. For example, you can pass in a customer support message (inputs) and ask if it is urgent and which team should handle it. A decision model will return typed answers with probabilities (outputs), which your code can use to route the ticket, trigger an escalation, or defer to a human. This means that a human does not necessarily need to be in the loop for agentic decisions anymore — agents can programmatically gather context, make decisions, and take actions on tasks, or defer to a human when needed. Specifically at Cloudflare, we’ve been testing our new Clef model on our Threat Intelligence team to help us classify website domains. By giving a domain to Clef (with Browser Run) it can quickly identify categories that the domain falls under — for example, it might classify a domain with a 95% chance it is a fashion website, 85% ecommerce, \<1% phishing, etc. This classification took our Clef model 2.2s to fetch, render, and classify the website. In contrast, our fastest general LLM gpt-oss-120b took 4.7s in the same workflow, and only returned two classifications. As a user, you can imagine how a 2x savings in latency and results can help us improve our threat intelligence workflows and be faster in identifying malicious or legitimate domains. Generalize this to any use case where you need to make quick programmatic decisions, and you unlock powerful agentic workflows that are able to autonomously decide, reason, and execute. In music theory, a clef is a symbol placed at the beginning of a musical staff that assigns specific pitch names to the lines and spaces. A decision model is analogous to a music clef because it helps define the domain of the context and the subsequent notes (actions) that follow it. We chose Clef as the name of our family of decision models, as it serves similar purposes, and the CF hearkens to Cloudflare. Although the market is getting increasingly saturated with decision models, Clef has some unique properties that make us excited to release it to the public. First, it has a vision encoder so it’s able to take in images and classify visual content. This is different from Jev, which only does text classification today. Secondly, our model has a 64k context window (compared to Jev’s 32k), which allows users to squeeze more input state for the model to classify against. Third, our model is accurate and powerful, scoring competitively against other decision models on the market across various quality benchmarks. We shortlisted some evaluations below that are important for decision-making as defined by the [Jev Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index) and scored some of the more popular models on the market for it. Check out the table below for benchmarks, or view the scores on our [live decision index demo site](https://clef-evals.workers-ai-mle.workers.dev): | **Benchmark** | **Clef** | **Clef-flash** | **Jev** | **DiffusionGemma Jev** | **Kev 9B** | **Laya** | | BFCL · case exact | 98.47 | **98.76** | 95.75 | 96.52 | 94.51 | 38.13 | | ToolRet · nDCG@10 | **69.19** | 66.43 | 65.28 | 61.21 | 64.26 | 12.69 | | API-Bank · accuracy | 91.93 | **93.11** | 88.19 | 83.66 | 56.30 | 11.41 | | Home appliances · case exact | 82.95 | **97.73** | 52.27 | 42.05 | 25.00 | 0.00 | | When2Call · accuracy | 72.37 | 65.58 | **80.97** | 75.44 | 49.62 | 11.94 | | BANKING77 · macro-F1 | **94.20** | 90.93 | 79.74 | 74.28 | 84.83 | 14.29 | | CLINC150+OOS · macro-F1 | **97.43** | 66.77 | 89.27 | 83.49 | 79.03 | 3.19 | | BRIGHT · nDCG@10 | 45.91 | 39.26 | **47.52** | 42.94 | 38.53 | 19.90 | | Amazon ESCI · macro-F1 | **57.48** | 57.39 | 55.21 | 53.37 | 49.22 | 24.40 | | PhishNChips · accuracy | 79.60 | 75.05 | 62.55 | **85.35** | 50.75 | 50.15 | We also ran benchmarks across [Typesafe’s own eval suite](https://huggingface.co/collections/typesafe/workflowevals) and our Clef models fared well, beating Jev in 3 out of 4 areas. Notably, our Clef-flash performs exceptionally well, given how much faster it is. | **Workflow** | **Clef** | **Clef-flash** | **Jev** | | Invoice processing | **64.7** | 57.1 | 61.8 | | Customer service | 76.3 | **77** | 76.0 | | Security incidents | **62.9** | 61.7 | 61.7 | | Agent trace observability | 68.5 | 69.8 | **71.6** | Across the 43 eval benchmarks that we ran, our Clef models beat the decision models on latency (except for Laya which is very fast but trades off quality in the benchmarks above): | **Benchmark** | **Clef** | **Clef-flash** | **Jev** | **DiffusionGemma Jev** | **Kev-9B** | **Laya** | | Median latency · ms | 209.3 | 38.8 | 524.1 | 84.4 | 51.4 | **5.8** | | p95 latency · ms | 238.6 | **122.4** | 536.0 | 211.2 | 187.9 | 222.5 | On top of the latency benefits from the model itself, our Clef models are hosted on Workers AI. Because they are hosted on Cloudflare’s infrastructure, we’re able to take advantage of our GPUs at the edge, leading to low network latency and faster decisions. This means that you could put Clef into the hot path for agents to make decisions and combine that with one of our LLMs on Workers AI to take action. ``` curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef \\ -X POST \\ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \\ -d '{ "model": "clef", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?" }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades" } }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"] } } }' ``` Clef also produces strictly typed outputs similar to Jev and is fully API-compatible, so you can make the swap extremely easily. The larger Clef model is your more powerful precision model, while the Clef-Flash model is great for latency-critical decisions. The models are enterprise-ready with our guarantee that we don’t read, store, or train on your requests or responses (unless you want to use our fine-tuning product, which we go into below). You can get started with the Clef models today, starting with our [developer documentation](https://developers.cloudflare.com/workers-ai/models/clef) or play around with the [open-source model on the Hugging Face repo.](https://huggingface.co/Cloudflare/clef) If you’d like help tuning Clef for a specific workload, we are also offering fine-tuning services — first as a hands-on partner with our forward-deployed engineer (FDE) team, and then later as a self-serve fine-tuning platform for customers to train and redeploy the model onto Cloudflare. In the same week that Jev came out, we [posted about some experiments](https://x.com/michellechen/status/2101091012559151480) we had with our own homegrown decision model. Our demo goes into how we adapted the DiffusionGemma model to output deterministic probabilities by exposing the logprobs that are generated by a large language model. Our initial approach built upon independent research by [Matt Mastracci](https://x.com/mmastrac), who has been active in the machine learning (ML) community with sharing new ideas and [pull requests to vLLM](https://github.com/vllm-project/vllm/pull/57250) inference engine to make DiffusionGemma support stronger. Clef builds upon this concept, but uses a different base model as the backbone. We currently use Qwen as the base model and post-trained it to suit decision model use cases. During inference, Clef uses Qwen for a prefill-only pass, then scores the valid schema choices in parallel. The decision step is non-autoregressive, so there’s no intermediate text to generate token by token, making Clef significantly faster than autoregressive LLMs. Rather than generating intermediate text to produce structured answers, Clef and Clef-flash derive schema choices directly from internal backbone representations. This approach relies on a specialized two-stage attention routing process: every valid choice extracts context relevant to the prompt, allowing individual field parameters to cross-attend with other fields and back to the original payload prior to scoring. By leveraging a lexical prior, the model preserves semantic intent across options. Ultimately, the architecture unites option-specific evidence routing, joint cross-field attention, and schema-bound scoring. By freezing Qwen3.8-27B for Clef and Qwen3.5-9B for Clef-flash, we jointly optimized the routing head alongside rank-256 low-rank adapters. Our post-training utilizes label-smoothed cross-entropy for valid schema outputs paired with a Brier loss to refine probability calibration. This training leverages our own internal synthetic datasets permutating field orders, prompts, and schema structures. We also developed Reinforcement Learning for Calibrated Decisions (RLCD) to serve as a secondary optimization target, granting partial credit to adjacent ordinal choices, rewarding fully precise record outputs, and applying a reference penalty to prevent distribution shift, giving us better accuracy and generalization. This means that we were able to achieve a few novel things with Clef: we improved accuracy of the model in classification, constrained it to output only probabilities instead of text generation, and made it faster than Jev and the base Qwen models. We heard a lot of internal use cases that required fine-tuning our Clef model to be built into our agentic workflows at Cloudflare. For example, internal teams want a classifier model to be able to evaluate Trust & Safety submissions, help us triage Cloudflare Support requests, or even to be built-in to our Bot products to decide if a crawler is a good bot or bad bot. These use cases are incredibly specific and we have had many years of labelled decisions that we could use to train a specific classifier. When you fine-tune a model, you may give up some general purpose performance in exchange for higher accuracy in a specific domain.. Because Cloudflare has more than 15 years of network data across different domains, we can fine-tune a model to fit these specific use cases which is more accurate and faster than our generic Clef model. We’re working with internal teams already to figure out how we can post-train Clef to create powerful ML models that boost our impact and improve workflows across Cloudflare. These internal teams and use cases are the next remit of our new FDE fine-tuning team and basis for our reinforcement learning (RL) product. We are offering a service to help customers fine-tune Clef to suit their workloads with our hands-on FDE team. From that, we’ll learn from our hands-on experiences to build a self-serve platform that customers can use to capture data, fine-tune, and redeploy the model, all on Cloudflare. This has actually been a long time coming — we’ve been building our AI platform to have the right primitives where we could be building a custom RL product. The interest in Jev shows the need for a fast, small, specific, classifier model, and we chose this to be our niche to start experimenting with RL environments. To do this, we leverage the primitives that we already have built on our Cloudflare platform: This combines a few work-in-progress pieces of the AI Platform that we’ve been working on, including AI Gateway that captures your AI traffic so you can leverage your own request/response data, Containers for RL Sandboxes, and Workers AI’s Bring Your Own Model (Cog) work that has been progressing since our acquisition of Replicate. We’re excited to launch our first Cloudflare-trained ML model from the Workers AI team today. We’re still early here and have a lot more improvements in store, but it is a wonderful first showcase of the hard work we’ve been doing on the AI Platform team. We believe that Clef has the ability to disrupt the way we use agents, which fits naturally into Cloudflare’s mission of being the agent cloud. If you have specific use cases and are already customers of these products — [we’d love to chat with you and be design partners as we experiment in this space.](https://www.cloudflare.com/resource/clef-rl-interest) Try out the Clef models hosted on Workers AI, download the [weights on Hugging Face](https://huggingface.co/Cloudflare/clef) if you’d like to explore for yourself, and reach out if you have fine-tuning use cases you’d like us to help with. Our ML team has been growing in impact, from model optimizations to model training research. If you’re interested in joining our mission, [check out our open roles](https://www.cloudflare.com/careers/).