Cloudflare Tests WAF Resilience Against Frontier AI Models and Strengthens Managed Detections

Cloudflare Tests WAF Resilience Against Frontier AI Models and Strengthens Managed Detections Cloudflare evaluated its Web Application Firewall by using frontier AI models to dynamically mutate exploits against an authorized customer staging environment. After triaging 1,107 automated attempts, human review yielded 49 legitimate findings—predominantly in SSRF and command injection—which led to targeted improvements in Cloudflare's Managed Ruleset. Frontier LLMs were leveraged in a black-box dynamic testing loop without access to source code or internal WAF rules. Out of 1,107 recorded mutation attempts across 45 scenarios, 607 survived triage, including 558 blocked requests and 49 actionable findings. 48 of the 49 investigated findings fell into the Server-Side Request Forgery (SSRF) and Command Injection (CMDi) categories. Human triage guided by five strict questions filtered out malformed, harmless, ambiguous, or out-of-scope bypass leads. The findings resulted in three Cloudflare Managed Ruleset updates, addressing obfuscated host representations, restricted protocols, and cloud metadata protections. Dynamic Testing Harness and LLM Feedback Loop Cloudflare designed a Python-based dynamic testing harness that paired an LLM in two alternating roles: a proposal call to suggest mutated payloads and a review call to evaluate HTTP response data. The models operated without access to WAF rule expressions, rule IDs, or Attack Score internals. Application code strictly governed execution by checking an allowlist, disabling redirects, limiting attempts, and treating response text as untrusted input across 45 testing scenarios covering six primary attack classes. Test Results and Human Triage Process The test recorded 1,107 mutation attempts against a staging environment configured with OWASP Paranoia Level 3, all Cloudflare Managed Rules, and WAF Attack Score blocking at 30 or below. The post-triage dataset contained 607 requests, encompassing 558 blocked attacks and 49 leads worth investigating. Triage engineers applied a five-question framework to exclude invalid requests, ambiguous outcomes, payloads stripped of malicious utility, and non-WAF network issues, ultimately isolating 48 issues in CMDi and SSRF while XSS, SQLi, LFI, and Log4j showed near-complete baseline coverage. Remediation and Ruleset Updates Investigated leads highlighted gaps in normalization and rule breadth, notably where non-standard host formats, like trailing dots or numeric representations, bypassed standard destination checks. Cloudflare validated candidate rules against live traffic to assess false-positive risks, resulting in two new detections—'SSRF - Obfuscated Host' and 'SSRF - Restricted Protocol'—alongside enhancements to the existing 'SSRF - Cloud' rule. Operational Takeaways and Deployment Guidance Cloudflare observed that increasing attempts within a single scenario often yielded redundant patterns, whereas testing diverse entry points and categories proved far more effective. In addition, automated AI outputs consistently required human validation to establish true security findings. Cloudflare advises customers to enforce defense-in-depth—combining Managed Rules, positive security models, and Bot detection—while noting that prompt software patching remains critical, as bypassing a WAF still demands a vulnerable origin application to execute an exploit.

Cloudflare利用前沿AI模型动态测试自身WAF：方法、发现与防护强化 Cloudflare利用前沿大模型模拟黑客对自身WAF开展黑盒动态变异测试，在1,107次攻击尝试中筛选出49项值得排查的绕过线索（绝大多数集中于命令注入和SSRF），并据此优化了托管规则集，强化整体安全防护能力。 构建基于Python的双大模型调用循环测试系统，在无WAF内部规则信息的情况下动态变异攻击载荷。 在覆盖6个攻击类别的45个测试场景中，WAF拦截了绝大多数攻击，人工筛选后确认49项有效发现，其中48项属于CMDi和SSRF。 通过五步筛选机制过滤无效与良性变异，推动Cloudflare托管规则集新增及改进了多项针对混淆主机和受限协议的SSRF规则。 测试证实模型仅作为载荷生成工具，人工复现审查必不可少，且企业需结合正向安全控制与及时打补丁落实纵深防御。 黑盒动态测试系统设计 为评估WAF抵御AI变异攻击的能力，Cloudflare使用Python构建了自动化测试框架，模拟攻击者在不知晓源码和WAF内部规则的情况下动态调整载荷。系统在每个场景中运行双大模型调用：先由提议调用根据上下文和历史记录生成变异载荷，再由审查调用分析响应状态、选定标头及正文，并严格限制尝试次数及目标白名单，将非拦截请求作为人工审查的线索。 测试规模与攻击绕过分布 测试覆盖跨站脚本、SQL注入、命令注入（CMDi）、服务端请求伪造（SSRF）、本地文件包含及Log4j等六大攻击类别，WAF配置了攻击评分拦截与OWASP规则集。在总共1,107次变异尝试中，XSS、LFI、SQLi和Log4j基本实现全覆盖拦截；经过排查掉畸形、良性及未到达目标的请求后，共沉淀出607个有效结果，其中558次被直接阻断，49项需深入分析的绕过线索中有48项落在CMDi和SSRF分类中。 人工分类筛选与规则更新落地 为避免模型产生的噪声干扰，团队依据请求有效性、非拦截明确性、恶意性保留、WAF职责归属及安全可复现性等五大维度进行严格筛选。最终将相关发现归类为四组候选规则并在实时流量中评估误报风险，直接推动Cloudflare托管规则集的三项改进：新增针对混淆主机和受限协议的SSRF检测，并优化了云环境SSRF规则，修复了以内置非标准数值编码解析绕过等问题。 经验总结与纵深防御建议 测试表明，横向扩展起始请求与攻击类型比单场景持续迭代更有效，且由于模型输出存在噪声，人工复现审查至关重要。Cloudflare强调WAF只是防护的一环，建议用户结合正向安全控制（定义预期流量模型）、API安全与机器人检测构建纵深防御，并在启用新规则时先以日志模式运行观察，同时务必保持底层应用及时打补丁以彻底消除隐患。

“Is your WAF ready for frontier AI models?” We keep hearing this question from our customers, so we decided to find out. When it comes to exploiting applications, what LLMs are really good at is iterating and mutating attack payloads faster than any human hacker could do. LLMs can use real-time responses to iterate and change their techniques by, for example, testing different encodings, sending the payload in a different part of the HTTP request, or moving to the next vulnerability to test. Even before LLMs were around, security engineers used two common approaches to test applications: static and dynamic application security testing. The former analyzes code without executing it to identify vulnerabilities, while the latter probes running applications to find runtime flaws. There are plenty of works scanning code with frontier AI models, including details on [how to build your own harness.](https://blog.cloudflare.com/build-your-own-vulnerability-harness/) For the project described in this blog post, we took a dynamic approach: making the LLM act as if it was a hacker to evaluate whether a WAF is doing its job. The LLM had no visibility into source code, no view of the WAF's rules, and could only see selected HTTP response data. We built a WAF tester that starts from known exploits and then iterates by changing how it is encoded or delivered, sends it again, and uses the response to choose the next variation. A request that was not blocked became a lead for human review, not a confirmed exploit. We ran the tester against an authorized customer staging environment across six attack categories and recorded 1,107 attempts. After reviewing the non-blocked requests and removing malformed, benign, duplicate, and out-of-scope observations, the vast majority of the attacks were blocked by the Cloudflare WAF. The requests that got through helped us create new detections to harden our security to benefit all Cloudflare customers. Here we will explain how we set up the system, the types of attacks we tested, which attack vectors bypassed the WAF more easily, and how we fixed it. Most importantly, we share what we learned from this process and how this exercise is becoming a foundational building block of our WAF development lifecycle. Finally, we offer guidance to help you correctly deploy your WAF in front of your application and, most importantly, patch your software. A payload that bypasses the WAF still needs an exploitable application to succeed, so keeping your stack up-to-date remains one of the strongest defenses against attackers. To test our WAF with frontier models, we built a system that iterates over multiple scenarios. A scenario means choosing one attack category, placing the input in a specific part of the request, starting with a version the WAF already blocked, and giving the tester a fixed number of attempts to try other variations. The loop runs LLM models twice: the first is the proposal call, the second is the review call. The first call receives the starting request, the context, a short history of earlier results, and suggests the next variation, then the code builds and sends the request. The review call receives the request context, response status, selected headers, and the response body. The loop stops when mutations stop producing useful variations or when a hard coded attempt limit has been reached. Both model calls work without access to WAF internal information. Neither receives rule expressions, rule IDs, WAF Attack Score details, or the identity of the security layer that acted. We implemented the system in Python rather than wrapping an existing penetration-testing tool. It handles HTTP replay, scenario orchestration, state tracking, and result collection. In the current implementation, the models do not send requests directly — code controls what happens at each step. Before each request, it checks the target hostname against an allowlist, disables redirects, records the attempt, and enforces the attempt limit. After each request, it records the response and uses the model's review to choose the next predefined step. Response text may appear in a later prompt, so the tester treats it as untrusted input. Neither model call can deploy a rule nor change enforcement. The system records structured evidence for each attempt. The main run targeted an authorized customer staging environment protected by Cloudflare’s WAF. We used an allowlisted test User-Agent so the customer’s automated-traffic controls would not stop the test before requests reached the WAF. We ran 45 scenarios. For each, we looked for ways to deliver the same attack differently: different encoding, different part of the request, or the same destination written another way. Of these, 44 covered six attack categories: [cross-site scripting (XSS)](https://www.cloudflare.com/learning/security/threats/cross-site-scripting/), [SQL injection (SQLi)](https://www.cloudflare.com/learning/security/threats/sql-injection/), [command injection (CMDi)](https://community.owasp.org/attacks/Command_Injection), [server-side request forgery (SSRF)](https://en.wikipedia.org/wiki/Server-side_request_forgery), path traversal or local file inclusion (LFI), and Log4j. The remaining scenario covered log injection, reported separately. The WAF in the test zone was configured as follows: [WAF Attack Score](https://developers.cloudflare.com/waf/detections/attack-score/) blocking scores of 30 or below, all [Cloudflare Managed Ruleset](https://developers.cloudflare.com/waf/managed-rules/reference/cloudflare-managed-ruleset/) enabled, and [OWASP Core Ruleset](https://developers.cloudflare.com/waf/managed-rules/reference/owasp-core-ruleset/) with Paranoia Level 3. For the headline measurement, we recorded whether the WAF blocked each request or not. The results describe the configured WAF boundary as a whole, not the performance of any individual rule or detection mechanism. Here is an example of how the LLM adapts a Server-Side Request Forgery (SSRF) attack during the test. Cloud metadata services can expose temporary credentials to workloads. An SSRF vulnerability can let an application fetch that data on an attacker's behalf. A WAF can help stop the malicious request before it reaches the application, but it is only one layer of protection. In this SSRF scenario, the tester sent the same cloud metadata address in different forms (such as integer, octal, and trailing-dot representations of the same IP) and placed it in different parts of the request. The WAF blocked all of them except one. At attempt 18, the model kept the same request structure as the previous blocked attempt and switched to the trailing-dot form. The client encountered a redirect rather than a WAF block. The table below shows selected moments from the session. The hypothesis column summarizes what the model said it was trying before each move. It is not a verbatim transcript, and it is not proof that the explanation was correct. Attempts 17 and 18 are an interesting pair: same request structure, different host representation. One was blocked, one was not. That gave us a specific question: does the trailing dot change how the WAF reads the destination? It was a lead to investigate, but not proof that metadata was accessed. This was one selected trajectory among 45 scenarios. The next section shows how we counted and triaged the full run. Our tester generated 1,107 attempts and the overall result was strong with XSS, LFI, SQLi, and Log4j having near full coverage. While the run produced useful findings, it also produced noise. After human review, we were left with 49 findings worth investigating, 48 of them belonging to CMDi and SSRF. Here is how they break down: | **Metric** | **Value** | **What it means** | |---|---|---| | Recorded mutation attempts | 1,107 | Model iterations across 45 active scenarios; not all produced a usable result | | Post-triage result set | 607 | The 558 blocked requests plus 49 documented WAF-relevant findings | | Blocked requests | 558 | The WAF stopped these before they reached the application | | WAF-relevant findings | 49 | Documented for remediation analysis after human review | The rest did not produce a result worth counting as the model failed to generate a usable HTTP request, some failed before reaching the target, or the payload generated was benign. When a request was not blocked, we worked through five questions before counting it as a finding: | **Question** | **Why it matters** | |---|---| | Did the tester actually send a valid request? | If the model failed or the request never reached the target, the result tells us nothing about the WAF. | | Was the request clearly not blocked? | An ambiguous response is not enough to count. | | Was the request still malicious? | Changing a request to get it past the WAF can also make it harmless. | | Did the behavior belong to the WAF? | Some attacks only work through DNS or network paths the WAF cannot stop at request time. | | Could engineers reproduce it safely? | A fix needs a stable test case with a clear expected result. | We removed anything that failed those checks and combined duplicate cases. What remained became the input for rule, normalization, and mitigation work. Not every finding needed a new rule. Some pointed to gaps in existing [Managed Rules](https://developers.cloudflare.com/waf/managed-rules/) coverage. Others pointed to how the WAF normalized the request or belonged to another security control. We replayed each case and decided where the change should happen. We grouped related findings into four sets of candidate rules, validated each finding, and tested candidates against live traffic before any rule could protect customer traffic. Before a new or updated rule can protect customer traffic, we check its impact on legitimate traffic and assess false-positive risk. Some of the issues we find when evaluating a new rule candidate include: | **Issue** | **Next step** | |---|---| | Missing or narrow detection | Review whether existing rules cover the finding | | Equivalent inputs interpreted differently | Engine or normalization review | | False-positive risk is too high | Revise or reject the candidate | This work contributed to three changes in Cloudflare's Managed Ruleset: new detections for [SSRF - Obfuscated Host](https://developers.cloudflare.com/waf/change-log/changelog/#2026-07-21) and [SSRF - Restricted Protocol](https://developers.cloudflare.com/waf/change-log/changelog/#2026-07-21) in the July 21 release, and improvement of the existing [SSRF - Cloud](https://developers.cloudflare.com/waf/change-log/changelog/#2026-08-04) rule. The SSRF - Obfuscated Host detection came directly from requests that encoded internal addresses in non-standard numeric forms. The model was only one part of the test. We ran the same scenarios with two versions of the same model family. They produced different variations – and the same underlying issues appeared in both. Because request replay and evidence capture stayed consistent, we could compare the runs without treating either model's output as ground truth. More attempts within one scenario did not always find more. Some scenarios started repeating earlier ideas near the end of the 25-attempt limit. We got broader coverage by testing more starting requests, attack categories, and input locations instead of extending one sequence. The model generated requests. We decided which ones mattered. A request that was not blocked still needed replay and human review before it could become a finding, a mitigation, or a regression test. Without that review, there were no findings. WAF is just one layer of detections you can deploy. When you deploy all available protections you increase the effectiveness of your overall stack. First of all, check that Managed Rules, WAF Attack Score are [set up correctly](https://developers.cloudflare.com/waf/get-started/) in front of your application. Other tools you can deploy include API Security, Bots and Fraud detection, and Threat Intelligence to strengthen your posture even further. For example, [positive security controls](http://blog.cloudflare.com/application-profiles) add a different layer: instead of looking only for known attack patterns, they define the request shapes an application expects and identify inputs outside that contract. This drastically reduces your attack surface area. Customers do not need to reproduce this experiment. To maximize the number of rules deployed in front of your application, we recommend running Managed Rules in log first, review matching requests in [Security Events](https://developers.cloudflare.com/waf/analytics/security-events/), and confirm legitimate traffic is unaffected before moving a rule to Block. Alternatively, customers can reach out to their account team to get [Attack Signature Detection](https://blog.cloudflare.com/attack-signature-detection/) turned on, on their zones. This new feature simplifies how to review matched traffic and how to deploy signature detections. If you already perform application security testing, run those tests against a staging hostname protected by the same Cloudflare controls as production. By combining adaptive AI-driven testing with human triage and validation, we found detection gaps that fixed tests might miss and turned those findings into stronger WAF protections, improving our block rate. In a future post, we will share results from further testing using a white-box approach, where the model knows both the application’s vulnerabilities and the WAF rules protecting it.