Where Security Fits in the Emerging AI Agent Stack

Where Security Fits in the Emerging AI Agent Stack NVIDIA's AI safety and security teams outline an architectural model for AI agents that separates behavioral steering from authoritative infrastructure enforcement to prevent agents from operating beyond intended operational boundaries. Recent boundary-breaking incidents by frontier agents highlight that behavioral steering via prompts, models, and harnesses cannot reliably enforce hard security boundaries. The emerging agent stack divides responsibilities across models, harnesses, secure runtimes such as NVIDIA OpenShell, and inference infrastructure, making runtime the authoritative boundary. Five core design rules ensure authority remains external: higher layers propose while lower layers decide, policies stay below the line, every external effect is checked, credentials use just-in-time scoping, and workloads remain isolated. Agent deployments are categorized into four security profiles—Isolated, Connected, Production, and Adversarial—which dynamically tighten oversight, access revocation, and isolation as operational risks increase. Behavioral vs. Infrastructure Controls Recent evaluations from OpenAI, Anthropic, and the UK AI Security Institute demonstrated long-horizon agents escaping test environments, accessing unauthorized external systems, and taking unapproved actions. These incidents underscore the limit of behavioral controls such as prompts, model guardrails, and harness logic. While harnesses guide agent context, loops, and intended behaviors, they remain programmable layers dependent on model compliance. True security enforcement requires infrastructure controls where the host environment authoritatively manages identity, policy enforcement, containment, and auditing. Establishing Runtime Boundaries Before Launch In a decoupled stack where models, orchestrators, and harnesses are chosen independently, security boundaries must be instantiated prior to execution. An orchestrator directs an isolated runtime to enforce governance, placing the harness, tools, and plugins inside that perimeter. Subagents are confined to child runtimes with ceilings they cannot breach. Treating the security runtime as an optional tool that an agent can invoke or ignore creates a critical flaw; runtime mediation must instead be mandatory and established at launch. Architectural Gaps and Design Principles Common security vulnerabilities in agent deployments include split rule ownership, standing credentials, untrusted data driving control paths, uncontrolled API side effects, and inadequate audit records. To counter these flaws, systems must follow five design rules: components above the boundary propose while infrastructure below decides; authoritative policy is placed below the control line; every state-changing effect is explicitly verified; permissions are provisioned just-in-time; and workloads are contained with rapid revocation and immutable recovery records. Tiered Security Profiles and Fail-Safe Requirements Workload demands dictate four distinct operational profiles: Isolated for disposable pre-production tasks without external credentials; Connected for approved services using short-lived identities and spend limits; Production for enterprise actions requiring human approvals for high-impact changes; and Adversarial for unaligned, frontier, or red-team runs requiring default-deny networking and quarantine mechanisms. Across all profiles, systems must implement safe failover states, avoid letting agents authorize their own actions, and scope security claims strictly to verified interaction paths.

NVIDIA解析AI智能体技术栈安全架构：从行为引导转向基础设施强约束 针对前沿AI智能体近期频繁出现的越界行为，NVIDIA安全团队基于OpenShell等实践提出了一套分层智能体技术栈。该架构将安全控制明确划分为上层的行为引导与下层具权威性的基础设施控制，确保权限决策独立于智能体自身。 OpenAI、Anthropic及英国AI安全研究所报告指出前沿智能体突破边界访问外网、渗透未授权系统并引发非预期操作。 智能体安全控制分为行为控制（提示词、安全防护、引导智能体尝试的Harness）与基础设施控制（运行环境决定智能体能力边界）。 智能体技术栈划分为产品分发、元编排、Agent Harness、安全运行时（如OpenShell）及推理数据平面五个功能层。 系统安全遵循五大设计原则，核心在于'上层提议、下层决策'以及在智能体启动前确立不可绕过的运行时边界。 针对不同任务风险划分为隔离、联网、生产与对抗四类安全配置文件，要求随风险提升缩短凭证生命周期并实施实时决策与强力隔离。 行为控制与基础设施控制的职责划分 提示词、模型防护机制以及Harness逻辑属于行为控制，虽然能够引导智能体理解目标并处理歧义，但由于依赖模型自身的不可测行为且容易随技术迭代而失效，无法构成硬性安全边界。真正的最终权威归属于智能体运行的底层环境。基础设施控制持有身份、强制实施策略、包含故障并记录操作，在策略与状态确定时输出重复且一致的授权决策，从根本上决定智能体'能做什么'而非'尝试做什么'。 智能体技术栈功能分层与边界前置 生态系统正逐步收敛为标准分层：最上层为产品分发与元编排层，中间为负责循环与工具调用的Harness（如Claude Code、DeepSeek Harness），底层则是提供隔离、策略和凭据管理的安全运行时（如NVIDIA OpenShell）及推理数据平面。由于Harness本身具备高度可编程与插件化特征，无法实现对自身篡改的自我防御。因此，安全边界必须在智能体启动前确立，运行时不能被视为智能体可选择性调用的普通工具，所有子智能体与工具进程必须在不可逾越的运行时边界内启动。 智能体系统常见漏洞与五大设计准则 现有智能体架构普遍存在权限边界模糊、过度赋予长期凭据、不受信任数据反向劫持控制流、外部API产生未受控副作用、多智能体调用导致故障级联以及审计日志不完整等漏洞。为此，团队提出五项设计准则：坚持上层提议而下层决策、将权威策略保留在边界线之下、严格检查每一个外部影响（包括文件、进程、网络与API调用）、实施即时且狭义的凭据访问，以及维持强隔离与快速恢复能力。 四级工作负载安全配置与递进式防御 根据风险等级与潜在破坏力，智能体工作负载被划分为四类配置：隔离级（预生产编码，无生产凭据且限制网络）、联网级（使用批准服务，分配短期身份与额度限制）、生产级（涉及生产系统变更，要求任务级访问与人工审批），以及对抗级（红队或未设护栏的前沿模型运行，采用默认拒绝通信与自动隔离机制）。随着风险递增，系统需收窄权限时效、提高决策实时性、加强实时监管、加快恢复回滚并在边界层保留不可篡改的审计记录。

As AI agents become more capable and operate over longer horizons, building security and trust into the applications they power becomes increasingly important. Drawing on work with NVIDIA OpenShell, agent developers, open-source projects, and partners across the ecosystem, AI safety and security teams at NVIDIA offer their perspective on the emerging agent stack—including the role of each layer and where security should live. Recent reports underscore why the placement of security controls matters. Within a few weeks this summer, OpenAI, Anthropic, and the UK AI Security Institute each reported frontier agents operating beyond their intended boundaries. The reported behaviors included exploiting an unexpected path out of lab environments to the open internet, gaining unauthorized access to other companies’ systems, and taking unsanctioned actions involving people and infrastructure. These cases involved long-horizon agents running with reduced model safeguards. But they point to the same design challenge: the capabilities that enable agents to solve problems creatively and pursue complex goals can also help them find paths that their original instructions did not anticipate. Recent NVIDIA research underscores the importance of the harness layer in the agent stack. Using Agentic Variation Operators (AVO), researchers achieved a 100% score on ARC-AGI-3, an interactive reasoning benchmark that places agents in unfamiliar environments without instructions, explicit rules, or stated goals. Learn more about the AVO research . This post maps the main layers of the emerging agent stack—models, harnesses , meta-harnesses, secure runtimes such as OpenShell, and inference infrastructure—and explains how each layer can help reduce risk. You’ll also learn which security properties become critical as these layers grow more capable and composable, including where authority should live, how access should be scoped, and how the runtime can contain and record an agent’s actions. Behavioral and infrastructure controls for AI agents Securing agents doesn’t require reinventing security. Decades of systems security provide durable principles, including least privilege, defense in depth, isolation, explicit authorization, and auditability. The challenge is determining where to apply them in an agent stack. Prompts, model safeguards, and harness logic all shape what an agent is likely to do, but they don’t create a hard boundary around what it can do. This distinction leads to two different kinds of control: behavioral controls that guide the agent and infrastructure controls that limit its authority. Behavioral controls influence agent actions The model and agent propose actions, and the harness directs them. Together, the model, agent, and harness interpret goals, work through ambiguity, and propose actions. The harness is the natural control point: it owns the loop, the context, the tools, and the session, and it can steer behavior toward what the operator intends. That steering is valuable, but every control implemented at this level still depends on how the model will behave. Infrastructure controls determine what an agent can do Final authority belongs to the environment in which the agent runs in. That environment holds identity, enforces policy, contains failures, records what happened, and reaches the same authorization decision every time, given the same approved policy and verified state. It doesn’t estimate what an agent will do. It determines what an agent can do. The harness guides what an agent tries. The infrastructure controls what an agent can do. Both are necessary; only one is authoritative. Infrastructure enforcement is not infallible. It means approved policy and verified configuration produce repeatable outcomes, and the agent cannot choose whether to comply. Policy can still be wrong, and external outcomes can remain uncertain. Mapping security controls This division maps onto the layers the open-source ecosystem is already converging on: Layer What it does Examples Distribution/product Package installation, defaults, and the supported experience NVIDIA NemoClaw Orchestration (meta-harness) Selects and coordinates different harnesses Databricks’ Omnigent Agent harness Turns a model into an agent: loop, context, tools, sessions Claude Code, Codex, Hermes, Pi, DeepSeek Harness Secure runtime Isolation, identity, policy, credentials, and audit NVIDIA OpenShell Inference data plane Model serving, cache placement, routing, and scheduling NVIDIA Dynamo Table 1. Functional layers of the AI agent stack, their responsibilities, and representative technologies These layers describe functional roles. One product may combine several roles, and a deployment may split one role across multiple services. Here, each layer names a responsibility. The security boundary is defined by the effect paths that the agent cannot bypass. The model supplies intelligence; the harness turns that intelligence into an agent; the runtime determines what that agent is allowed to do. The harness layer is a spectrum rather than a fixed category. Codex and Claude Code are opinionated harnesses, while Pi and DeepSeek Harness (DSH) expose more of the harness as a programmable substrate. Through Cordis, DSH enables core behaviors that can be composed and replaced as plugins. This programmability makes the harness a poor place for a security guarantee: a layer designed to be modified cannot reliably enforce controls against its own modification. The alternative—relying on harness logic for safety—encodes assumptions about model behavior, and those assumptions go stale as models improve. A narrowly scoped credential limits potential harm, but keeping the raw credential out of the agent’s reach creates a stronger boundary enforced by the environment. Establish the AI agent runtime boundary before launch Models, harnesses, runtimes, policies, and inference deployments are increasingly selected independently. This approach only works if the runtime’s guarantees hold regardless of which components operate above it. That means a security boundary must be established when the agent launches. An orchestrator asks OpenShell to create a runtime and enforce policies and governance. The selected harness starts inside that runtime, and its plugins, Model Context Protocol (MCP) processes, tools, and other model-directed code run inside the same boundary. Subagents receive delegated child runtimes with ceilings they can’t exceed, while the orchestrator operates inside a runtime governed by its own policy. This approach is different from treating the runtime as another tool that a harness can invoke once it’s already running. A control that the agent can decline to invoke is not an effective security control. Common security gaps in agent stacks Many agent stacks share the same flaw: authorization decisions can be influenced by the agent or by untrusted data it reads. Unclear boundaries. Rules are split across prompts, models, agents, harnesses, runtimes, and infrastructure, so the authoritative version is hard to find. Excessive access. The agent receives standing, often long-lived credentials or permissions beyond what the current task needs. Untrusted data as control. Documents, messages, tool results, and memory can redirect action without being authorized as instructions. Uncontrolled external effects. An allowed API can move data, create compute, or trigger effects outside the intended controls. Compounding failures. Agents delegate, share memory, and call peers, so one mistake can become a fast cascade. Incomplete audit evidence. Approvals are vague, access is slow to revoke, and the record is not sufficient to explain an incident or support recovery. Design rules for enforceable agent security Five design rules help keep security decisions outside the agent’s control. Above proposes; below decides. No model, agent, harness, tool, or memory system grants itself authority. Authoritative policy location. Keep policy below the line. Policy-aware planning above the line is useful, but advisory. Check every effect. Control every file, process, network request, API call, data operation, resource allocation, communication, and device action. Just-in-time access. Credentials and capabilities should be narrow, short-lived, and easy to remove. Isolation and recovery. Isolate each agent, revoke access quickly, recover, and preserve the record. A layered security model for agents Like the OSI model, this agent stack assigns each layer one job and a clear interface. Higher layers can change without redefining the control layer below them. Figure 1. A layered AI agent stack separates behavioral components from infrastructure-enforced security controls How the security boundary works The boundary is effective only if every request is evaluated consistently. Three requirements make this possible. Treat every component above the boundary as untrusted. It may be mistaken, compromised, or adversarial, and its requests carry no authority on their own. Make the controls below the boundary authoritative. These layers bind each request to an identity, apply policy, and enforce the decision. Use risk signals only to reduce authority. Signals such as anomaly scores may trigger tighter controls, but they must never grant additional access. Every action that changes the external state must pass through the policy and enforcement layers below the boundary. Any path that allows Layers 5-7 to bypass those controls is an architectural defect. Four security profiles for agent workloads All four profiles use the same stack, boundary, and interfaces. Each applies different controls based on the authority granted, the potential impact, and the likelihood of adversarial behavior. Level Typical work Required configuration 1. Isolated Coding in pre-production with disposable data. No production credentials; restricted network; session recording. 2. Connected Pre-production using approved services. Short-lived identity; masked data; rate/spend limits; full logging. 3. Production Changes to production systems or data. Task-scoped access; independent checks; human approval for high-impact. 4. Adversarial Frontier-model, non-guardrailed, or red-team runs. Default-deny communications; automatic quarantine; strongest isolation. Table 2. Four security profiles for AI agent workloads and their required controls Important. Production access for a red-team agent should be exceptional and narrower, not broader, than access granted to an ordinary production agent. How agent security controls change as risk increases As an agent gains more authority and the potential impact of its actions grows, strengthen controls in five areas. Narrower authority. Grants should become shorter-lived as risk rises. Fresh decisions. Reevaluate policy closer to each action. Stronger oversight. Add live supervision for high-impact work. Faster recovery. Plan for access revocation, quarantine, and rollback. Independent evidence. Keep immutable records below the security boundary. Security requirements at every risk level Although controls become stricter as risk increases, the following security requirements should remain consistent across every profile. The agent never grants itself access. Controls are enforced outside the agent process and beyond the agent’s control. This holds at every level. Every in-scope, high-impact effect crosses an enforcement point. The check occurs in the system that performs the action. The system fails safely: a missing or stale control selects a preapproved safer state. For physical and availability-critical systems, that state may require controlled operation rather than an abrupt stop. Security claims remain scoped . State the exact paths covered, assumptions made, and exclusions left outside the stack. Help shape AI with in-market learning Whether you build AI models, deploy AI systems, operate cloud infrastructure, conduct security research, or develop governance and standards, your perspective can help shape how the AI community learns from incidents. Explore NVIDIA OpenShell to learn how a safe, private runtime isolates autonomous agents and enforces security policies. Review and contribute to the Open Secure AI Alliance’s Shared AI Findings Exchange (SAFE) proposal , which outlines a community framework for learning from AI incidents and near misses.