NVIDIA and CrowdStrike Demonstrate Adaptive Agentic Cybersecurity Loop with Nemotron Models

NVIDIA and CrowdStrike Demonstrate Adaptive Agentic Cybersecurity Loop with Nemotron Models NVIDIA and CrowdStrike evaluated an agentic cybersecurity framework that connects offensive and defensive AI agents into an automated learning loop, using NVIDIA Nemotron models to accelerate detection generation and validation at machine speed. CrowdStrike reported its Nemotron-based Blue Solano defensive model was more accurate than a leading proprietary frontier model at 99% lower cost in internal evaluations. The agentic system pairs Nemotron 3 Ultra for high-level defensive orchestration with a customized Nemotron 3 Super dedicated to generating and repairing detection queries. A specialized defensive harness uses schema tools, telemetry grounding, artifact linting, replay testing, and independent review to enforce behavioral detection rules. In live-fire evaluation against eight unseen attacks, the optimized Nemotron pipeline yielded a 45% generalization rate and three 'gold' detections covering all attacks, whereas the frontier baseline yielded zero gold detections. Study limitations include evaluating only a single scenario family, small detection sample sizes, limited benign noise testing, and harness failures in three of eight live-fire runs. Continuous Closed-Loop Agentic Testing Traditional red and blue team exercises are constrained by manual handoffs during attack execution, telemetry review, detection writing, and retesting. To accelerate this cycle, NVIDIA and CrowdStrike deployed an offensive-defensive loop inside an isolated representative environment based on NVIDIA computing infrastructure and CrowdStrike Falcon sensors. The process spans four stages: the red agent executes an attack path and captures telemetry; the blue agent reconstructs the sequence and identifies detection gaps; a validation harness generates, tests, and backtests candidate rules; and an independently seeded red agent retests the objective, forcing the offense to find alternative evasion paths until no viable attack vectors remain. Defensive Harness Specialization To prevent models from hallucinating queries or writing brittle indicators, the defensive agent harness integrates six specialized validation mechanisms. A schema knowledge base restricts generated queries to supported Falcon fields, while telemetry grounding anchors workflows strictly in observed events. Artifact linting rejects environment-specific strings such as individual IP addresses or hostnames, redirecting logic toward behavioral signals. Candidate detections must also successfully replay against recorded attack telemetry and pass evaluation by an independent judge model with a fresh context window to confirm behavioral alignment. Model Customization and Verifiable Reward Training The defensive stack separates orchestration from detection generation by assigning high-level task planning to Nemotron 3 Ultra and bounded query authoring to a customized Nemotron 3 Super (CrowdStrike's NL2LogScale model). The specialized expert underwent continual pretraining, supervised fine-tuning on 9,349 detection generation and repair examples across 59 error categories, and reinforcement learning. Using NVIDIA NeMo Gym to run generated queries against Falcon LogScale and NVIDIA NeMo RL for group relative policy optimization, training rewarded models based on verifiable F1 event overlap against reference queries rather than mere syntactic correctness. Backtest and Live-Fire Evaluation Outcomes System testing evaluated both recorded attack backtesting and generalization across eight unseen live-fire attacks within the same scenario family. In backtesting, augmenting Nemotron 3 Ultra with the specialized harness and fine-tuned Nemotron 3 Super raised attack detection rates from 16.5% to 41.9%, a 2.5x gain over the default harness. During live-fire testing, five of 11 passing open-model detections (45%) identified at least one attack, compared to 10 of 35 (29%) for the frontier system. When subjected to noise tests on background traffic and independent review for behavioral robustness, three open-model detections qualified as 'gold' status—covering all eight attacks—while zero frontier detections passed all quality gates. Limitations and System Design Principles The authors emphasize that the evaluation serves as a directional, system-level case study rather than a broad industry benchmark. Generalization across distinct scenario families was not evaluated, limited benign traffic prevented production-grade false positive assessments, and three of eight live-fire runs encountered harness failures despite logging complete telemetry. From the findings, four architectural principles were highlighted for specialized agents: defining distinct operational roles per model, training bounded models with verifiable rewards, enforcing strict deterministic validation and independent review, and conducting expert-supervised evaluations under representative operating conditions.

NVIDIA 与 CrowdStrike 合作利用 Nemotron 模型构建自适应攻防智能体网络安全系统 NVIDIA 与 CrowdStrike 在模拟加速计算环境中评估了一套自主攻防智能体系统。系统结合 Nemotron 3 Ultra 编排与经过专门微调的 Nemotron 3 Super 模型，通过红蓝对抗闭环与专用防御 Harness 自动生成和验证检测规则，在实战测试中展现出优于前沿专有模型的泛化能力与成本效益。 传统红蓝对抗依赖耗时的人工交接，该系统通过“执行捕获、处理重建、生成验证、重测适应”四步实现机器速度的自适应闭环。 系统将任务解耦：Nemotron 3 Ultra 负责全局编排与工具调用，定制微调的 Nemotron 3 Super 作为专用专家模型负责生成与修复检测规则。 防御 Harness 集成了模式知识库、遥测锚定、规则 Linting 检查、回放测试及独立评审六大机制以消除幻觉和环境特异性硬编码。 在针对未见攻击的实战测试中，优化后的开源 Nemotron 管线生成的检测规则中有 45% 触发检出（前沿系统为 29%），且唯一产出达到全部 Gold 标准的规则。 闭环自适应攻防工作流 传统安全演练受限于各环节人工交接，难以快速测试大量攻击变种。该系统在隔离的受控环境中建立连续闭环：红方智能体基于威胁目标执行攻击路径并由 Falcon 传感器收集遥测数据；蓝方智能体接收痕迹并重建攻击序列，识别现有防御缺口；随后蓝方生成候选检测规则并通过回测验证后部署；最后红方在新检测规则生效后重新探索规避路径，直至环境中无可行攻击路径。 专用防御 Harness 的六项机制 为确保生成的检测规则具备实战价值，防御 Harness 结合了六项保障机制：通过模式知识库查询 Falcon 支持的字段与语法以防臆造查询；将分析锚定于实际攻击遥测数据以减少幻觉；隔离检测编写任务至专用模型；通过 Linting 自动拒绝语法错误、不支持字段以及写死 IP、主机等环境特异性字符串；将候选规则回放至捕获的攻击流量中验证匹配有效性；最后交由独立裁判评估行为对齐度与鲁棒性，未通过检查时提供结构化反馈供循环修正。 Nemotron 模型的专业化定制与训练 架构将通用编排与特定任务严格拆分。Nemotron 3 Ultra 承担攻击序列重建与步骤规划，而 CrowdStrike 基于 Nemotron 3 Super 构建了 NL2LogScale 模型。该模型历经领域知识持续预训练、涵盖 9,349 个检测生成与多步修复样本的有监督微调，并结合了真实引擎错误与推理轨迹。在强化学习阶段，利用 NVIDIA NeMo Gym 执行查询验证，以生成的查询结果与基准查询在合成日志中的 F1 重叠度作为奖励，并通过 NeMo RL 的 GRPO 算法更新模型权重。 回测与实战评估表现 在针对已记录攻击的回测中，优化后的开源管线将平均检出率从默认 Harness 搭配 Ultra 的 16.5% 提升至 41.9%（2.5 倍增长）。在随后针对 8 种未见攻击的实战测试中，部署的 11 条开源管线规则中有 5 条触发有效检出（45%），而前沿专有系统的 35 条规则中仅 10 条触发（29%）。在进一步要求对良性测试流量保持静默并经专家审查的 Gold 标准筛选下，开源管线产出 3 条合格规则并完全覆盖全部 8 场攻击，前沿系统则无规则符合 Gold 标准。 测试局限性与架构设计原则 评估结果属于方向性的系统级案例，而非通用基准：测试仅涵盖单一攻击场景家族，良性流量有限因而无法完全代表生产环境误报率，且有 3 次实战运行出现 Harness 故障（但保留了完整遥测）。该实践提炼出构建专业化智能体的四大原则：明确可量化任务并合理分配模型职责；使用领域数据与可验证奖励后训练模型；基于权威上下文锚定并结合确定性检查与独立评审；在安全专家控制的真实条件下对完整工作流进行端到端评估。

AI is changing the pace of cybersecurity. Agentic systems can coordinate work and pursue complex objectives over long horizons. Security teams are beginning to apply agents across security operations, but many implementations remain anchored to existing alerts, predefined workflows, and known attack behaviors. The harder problem is identifying what defenses miss and turning those gaps into reliable coverage. This requires repeatedly testing adaptive attacks in an organization’s unique environment and validating candidate detections against normal enterprise activity. Continuous offense-defense testing creates this feedback loop. Controlled attacks produce the telemetry and ground truth defensive agents need to expose gaps, improve coverage, and retest. However, the end-to-end cycle still requires significant manual effort. Could red and blue agents powered by open models and specialized harnesses operate this loop at machine speed and scale? NVIDIA and CrowdStrike evaluated an agentic attack-defense system in an isolated environment modeled on NVIDIA accelerated computing infrastructure. On the defensive side, NVIDIA Nemotron models customized for cybersecurity operate within CrowdStrike SafeMind, its agentic cybersecurity system. CrowdStrike reports that its Blue Solano defensive model is more accurate than the leading proprietary frontier model tested, at 99% lower cost, in CrowdStrike internal evaluations. For this evaluation, the optimized open-model configuration paired NVIDIA Nemotron 3 Ultra for defensive orchestration with a fine-tuned Nemotron 3 Super for detection generation. This post shares how Nemotron models and specialized agent harnesses work together with CrowdStrike’s agentic system, how it was deployed and evaluated, and how the resulting detections performed across independently seeded attack runs. Turning offense and defense into a continuous learning loop Traditional red-and-blue team exercises depend on manual handoffs: the red team executes an attack, the blue team reviews the resulting telemetry, detection engineers develop or update detections, and the red team retests them. Each handoff takes time, limiting the number of iterations and attack variations teams can evaluate. An offensive-defensive agentic system connects these activities into a repeatable closed loop that can operate at machine speed. Within an isolated representative environment, each run produces attack traces and sensor telemetry used to create or refine detections. The resulting context returns to the red-agent harness, which adapts and tests alternative attack or evasion paths until it finds no further viable route within the representative environment. Figure 1. Closed-loop agentic system for cybersecurity The workflow has four connected stages: Execute and capture. Starting with a threat-informed objective, the red-agent harness selected and executed an attack path inside the representative environment. Its action trace recorded each step, while CrowdStrike Falcon endpoint sensors captured the corresponding telemetry. Process and reconstruct. The blue-agent harness then received the action trace, sensor telemetry, and broader attack context. Using information about the available data sources and CrowdStrike detection-engineering expertise as grounding context, it determined which parts of the event sequence could be reconstructed, which existing detections triggered, and where visibility or detection gaps remained. Generate and validate. Based on this analysis, the blue-agent harness generated candidate detections. The validation harness checked each candidate, backtested it against the captured telemetry, returned failures for correction, and sent validated detections to the detection engine. Retest, adapt, and repeat. After a validated detection was deployed, an independently seeded attack retested the same objective. Detection and alert context returned to the red-agent harness, which adapted and explored alternative attack or evasion paths, producing new traces and telemetry for the blue-agent harness. Each cycle is designed to strengthen defensive coverage while forcing the offense to find a harder path with full knowledge of the defense. The loop continues until no further viable path remains within the modeled environment. Building a representative test environment To enable safe, realistic testing, NVIDIA provided a sanitized, natural-language specification representing its accelerated computing infrastructure. An agent-assisted workflow translated the specification into an isolated target cyber agent environment instrumented with Falcon platform sensors. The evaluation used attack paths and observable milestones to measure progress from action traces and sensor telemetry rather than the agent’s own claims. NVIDIA security experts reviewed the environment and threat paths for realism. The same reviewed environment supported every attack run, detection test, and evaluation metric, enabling consistent comparisons across configurations. Specializing the defensive harness The next challenge was turning the resulting attack traces and telemetry into detection rules that could pass technical and behavioral validation. Open agent harnesses already provide capabilities such as planning, tool use, context management, and iterative correction. The Open Secure AI Alliance is helping expand the broader ecosystem of open models, harnesses, and tools for cybersecurity. Figure 2. Specializing a default agent harness for detection engineering The defensive harness combined six mechanisms: Schema knowledge base. A tool allowed agents to enumerate supported Falcon sensor schemas, fields, and query syntax, preventing invented fields and invalid queries. Telemetry grounding. Red-agent traces, Falcon telemetry, and broader attack context anchored the workflow in observed events and relationships, reducing unsupported or hallucinated connections. Specialized detection authoring. A customized Nemotron 3 Super served as a bounded expert for generating and repairing detections, separating this specialized task from the longer orchestration context. Artifact linting. Automated checks rejected syntax errors, unsupported fields, and detections tied to specific IP addresses, hosts, users, or subnets. Failures returned guidance for rewriting the detection around behavioral signals instead of environment-specific strings. Detection replay. Each candidate was replayed against captured attack telemetry. Detections that produced no match were rejected and returned for correction, catching candidates that appeared valid but failed to detect the recorded activity. Independent review. A separate judge with a fresh context evaluated each detection for behavioral alignment, robustness, and appropriate use of multiple signals, catching quality gaps missed by linting and replay. Failed checks returned structured feedback to the blue-agent workflow for correction and another attempt. Together, these mechanisms encoded practices normally applied through manual detection-engineering review, making detection generation grounded, testable, and correctable rather than merely plausible. Customizing Nemotron for defensive orchestration and detection generation Open models such as Nemotron can be post-trained with domain data, deployed with proprietary context in controlled environments, and optimized for cost and scale. In the evaluated open-model configuration, Nemotron 3 Ultra reconstructed attack sequences, planned detection-engineering steps, and invoked tools. When a detection required writing or repair, a customized Nemotron 3 Super served as a bounded expert. This separation kept workflow orchestration distinct from specialized detection authoring. Figure 3. Separating defensive orchestration from specialized detection writing and repair To create this specialized expert, CrowdStrike used Nemotron 3 Super as the base for its NL2LogScale model, then applied continual pretraining on cybersecurity knowledge, supervised fine-tuning, and reinforcement learning with verifiable rewards. Fine-tuning used 9,349 detection-generation and multistep repair examples spanning 59 programmatically generated error types. The training data combined request rephrasings from Nemotron 3 Super, real Falcon LogScale execution errors, and quality-reviewed reasoning traces from Nemotron 3 Ultra. Figure 4. Post-training specialized Nemotron 3 Super for detection, writing, and repai r For reinforcement learning, the training workflow used NVIDIA NeMo Gym to validate and execute generated queries in Falcon LogScale. Invalid queries received real engine errors and up to five repair attempts, with unresolved attempts receiving zero reward. Valid generated and reference queries ran against the same synthetic logs, and the F1 overlap between their returned events supplied the reward. NVIDIA NeMo RL supported group relative policy optimization to update the model. For reinforcement learning, the workflow used NeMo Gym to validate generated queries and NeMo RL to update the model based on verifiable results. Figure 5. RLVR rewarded mat ching query results , not merely valid syntax Evaluating the complete agent system from backtest to live fire A detection may detect the recorded attack used to create it, but fail on a new execution of the same behavior. Therefore, we evaluated each system in two stages: backtesting against the recorded attack, followed by live-fire testing against eight new attacks from the same scenario family. Every passing detection was deployed verbatim to the live detection engine. An independent third-party model judged whether each match represented the intended attack behavior, and results were averaged across independently seeded authoring sessions. Backtesting against the recorded attack With Nemotron 3 Ultra and the default harness, an average of 16.5% of generated detections detected the recorded attack across eight independently seeded sessions. Retaining Ultra while adding the tuned harness, customized Nemotron 3 Super, domain context, tools, and validation raised the mean to 41.9% across six sessions, a 2.5x improvement. Because the optimized configuration changed both the harness and model stack, the gain reflects the complete open pipeline rather than an isolated model ablation. Figure 6. Mean backtest detection across independently seeded authoring sessions Evaluating generalization through live-fire testing During live-fire testing, 11 backtest-passing detections from the optimized open pipeline powered by Nemotron and 35 from the complete frontier system were deployed against eight unseen attacks. Five of the 11 open detections (45%) detected at least one attack, compared with 10 frontier detections (29%). The open pipeline averaged 2.6 detections per detection, compared with 1.1 for the frontier system. Detection alone, however, was only the first quality gate. To qualify as gold, detections also had to remain quiet on available test traffic and pass an independent review for behavioral grounding, multiple signals, and no environment-specific strings. Four of the five firing open detections and nine of 10 frontier detections that fired stayed quiet, covering eight of eight attacks and seven of eight, respectively. After review, three open detections and no frontier detections qualified as gold. The three open detections still covered all eight attacks. Figure 7. Live-fire validation funnel Interpreting the results The frontier system produced more backtest-passing detections, but the optimized Nemotron open-model pipeline yielded a higher share that generalized, averaged more detections per detection, and was the only system to produce gold detections. The evaluation covered one scenario family and small detection sets, so cross-scenario generalization remains untested. Limited benign traffic also means the noise test doesn’t represent production false-positive performance. Three of eight live-fire runs experienced harness failures but produced complete telemetry and were retained. The findings are a directional system-level case study, not a general benchmark. Applying the pattern to specialized agents The evaluation shows a broader pattern for building specialized agents with NVIDIA Nemotron : a reasoning model orchestrates the defensive workflow, a post-trained open model handles a bounded expert task, and an agent harness manages context, tools, and validation. Four design principles emerge: Define measurable tasks and assign clear responsibilities to each model. Post-train specialized models with domain data and verifiable rewards. Ground outputs in authoritative context and validate them through deterministic checks, realistic replay, and independent review. Evaluate the complete workflow in realistic conditions, with security experts controlling scenarios, guardrails, and use of validated outputs. CrowdStrike is advancing this approach through SafeMind, its agentic cybersecurity system, which brings offensive and defensive AI together in a continuous coevolution loop. Explore NVIDIA Nemotron 3 , NeMo Megatron Bridge , NeMo Gym , and NeMo RL to customize and evaluate specialized agents for other bounded domains.