AWS Agent Head Replaces Prompting With Hard Controls to Curb AI Errors

@Scale

Production agent misbehavior stems from a controls deficit rather than flawed prompts, making deterministic in-loop steering and out-of-loop policy engines necessary to stop unintended executions.

AI-assisted summary of Bridging the Intent Gap in Agentic Systems | Anoop Deoras from AWS (Live) — original source. Check the original for full context.

Source published: 2026-06-23

Key ideas

Production agent misbehavior stems fundamentally from a controls problem rather than a prompting issue, meaning natural language system prompts fail to enforce runtime invariants. (0:00)
AWS structures agent governance across two axes: control location (in-loop vs. out-of-loop) and corrective action (guiding/nudging with retries vs. hard deterministic blocking). (3:45)
In-loop steering hooks inspect execution ledgers to enforce deterministic sequencing rules in standard Python code, enabling local unit testing of agent constraints. (6:30)
Hard authorization and security boundaries require external policy engines like Cedar at the gateway to prevent adversarial users from negotiating around in-loop reasoning defenses. (10:00)

Detailed notes

## Background Published on YouTube by @Scale on June 23, 2026, this presentation features Anoop Deoras, Director of AWS Agentic AI, delivering a talk titled "Bridging the Intent Gap in Agentic Systems." The session explores why autonomous AI agents fail when moving from local development to production, presenting architectural strategies to enforce reliable agent behavior.

## TL;DR Anoop Deoras argues that production agent misbehavior is fundamentally an architectural controls issue rather than a prompting problem. He breaks down the disconnect into two challenges: the operator specification gap between builder intent and execution, and the user understanding gap caused by ambiguous natural language. To resolve these issues, Deoras presents a two-axis control framework combining in-loop steering hooks that inspect execution ledgers to nudge the agent with out-of-loop policy engines like Cedar that deterministically block unauthorized actions. He concludes that creating trustworthy agentic systems requires moving away from the "prompting treadmill" toward a layered defense calibrated by reversibility, blast radius, and explicit user-in-the-loop validation.

## Atmosphere & style The presentation carries a crisp, pragmatic tone tailored directly to the lived frustrations of AI practitioners and systems engineers. Deoras speaks with calm, authoritative clarity from the podium, puncturing generative AI hype by acknowledging the exhausting reality of prompt engineering whack-a-mole. The delivery feels methodically structured and reassuringly grounded, replacing probabilistic uncertainty with disciplined software engineering patterns and structured architectural diagrams.

## Bulletin - [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) Production agent misbehavior stems fundamentally from a controls problem rather than a prompting issue, meaning natural language system prompts fail to enforce runtime invariants. - [03:45](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=225s) AWS structures agent governance across two axes: control location (in-loop vs. out-of-loop) and corrective action (guiding/nudging with retries vs. hard deterministic blocking). - [06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) In-loop steering hooks inspect execution ledgers to enforce deterministic sequencing rules in standard Python code, enabling local unit testing of agent constraints. - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) Hard authorization and security boundaries require external policy engines like Cedar at the gateway to prevent adversarial users from negotiating around in-loop reasoning defenses.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) Agent failures in production represent an intent gap that cannot be solved by prompt engineering alone; it is fundamentally a systems controls problem. - [01:04](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=64s) The intent gap splits into two distinct challenges: the specification gap (builder intent vs. execution) and the user understanding gap (what the user said vs. what they meant). - [01:04](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=64s) Codifying business rules purely in system prompts fails because natural language instructions cannot guarantee runtime execution logic or prerequisite tool verification. - [03:45](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=225s) Agent behavioral controls map along two main axes: whether the control is inside or outside the execution loop, and whether the response is to nudge or hard block. - [03:45](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=225s) Prompts and standard operating procedures (SOPs) are static build-time levers, whereas steering hooks and policy engines operate dynamically at runtime. - [06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) In-loop steering hooks intercept execution before tool calls and after response generation to evaluate state transitions deterministically. - [06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) Execution ledgers track all prior tool calls and I/O payloads, allowing deterministic Python handlers to verify preconditions and inject targeted corrective nudges. - [06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) In-loop steering introduces hot-path latency and token overhead, while subjective constraints like tone require probabilistic LLM-as-a-judge evaluators. - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) Security enforcement and access boundaries must sit outside the agent's reasoning loop because any logic inside the loop can be debated, negotiated, and bypassed by users. - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) High-level natural language builder intent can be compiled into deterministic authorization policies using Cedar, structuring rules into principals, actions, resources, and constraints. - [12:35](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=755s) The choice between guiding (nudging) and blocking depends on three critical factors: action reversibility, adversarial threat level, and blast radius. - [12:35](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=755s) A layered architecture routes constraints dynamically: policy engines block hard security risks, while steering handlers guide tool sequencing and parameter corrections. - [16:09](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=969s) The user understanding gap cannot be fully eliminated by technology, requiring interaction patterns like plan reflection, proactive fork clarification, and explicit confirmation for high-stakes actions. - [16:09](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=969s) Hyper-personalization across interaction sessions helps bridge human ambiguity by incrementally retaining user preferences over time.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) The Production Intent Gap: A Controls Problem

Opening his presentation at the @Scale podium, Anoop Deoras reflects on the inspiring history of talks at the conference, highlighting a session earlier that morning by Boris on coding agents. Addressing the audience of practitioners and AI builders, he touches upon a shared, frustratingly common experience: an engineer designs an agent locally on a laptop, thoroughly tests its behavior, deploys it to production, and suddenly watches it execute actions that were never intended or planned.

The central mission of the talk is to dissect why this divergence occurs and lay out the architectural tools required to solve it. Deoras emphasizes a foundational premise that challenges conventional agent engineering wisdom: this breakdown is fundamentally not a prompting problem, but a controls problem.

```mermaid flowchart TD accTitle: The Intent Gap in Agent Deployment accDescr: Diagram showing the breakdown between local testing and production deployment where an intent gap emerges. subgraph Local["Local Development"] A["Agent Built on Laptop"] --> B["Local Behavior Testing"] B --> C["Deployment to Production"] end subgraph Prod["Production Reality"] C --> D["Unplanned Actions & Misbehavior"] D --> E["The Intent Gap: Controls vs. Prompting Issue"] end ```

[01:04](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=64s) The Library Agent Failure: Specification vs. Understanding Gaps

To illustrate this failure mode concretely, Deoras walks through a case study of building a library management agent. The agent is designed to manage user profiles, understand preferences, and issue reminders when books are due. The builder establishes a critical business rule: *if a book has been recalled, it must not be renewed*. Under standard development practices, the builder embeds this constraint as a natural language instruction directly into the agent's system prompt. Local testing appears successful, but once deployed to production, the agent routinely renews recalled books anyway.

The breakdown occurs because a rule codified merely in natural language does not guarantee execution logic; the agent never invoked a tool to verify the book's status prior to issuing the renewal. This disconnect highlights what Deoras clarifies are actually two distinct intent gaps:

1. **The Operator Gap (Specification Gap):** The distance between what the agent builder intends for the agent to do and what the agent actually executes in production. 2. **The Understanding Gap (User Understanding Gap):** The distance between what the human user explicitly says and what they actually mean.

While the user understanding gap is significantly more challenging to solve, Deoras focuses the core of his talk on systematically closing the specification gap through deterministic controls, while later addressing mitigation strategies for user ambiguity.

- **Failure Cause:** Natural language system prompts fail to force runtime tool validation (e.g., executing a status check before taking a mutating action). - **Primary Focus:** Closing the specification gap using deterministic, unit-testable runtime controls rather than relying purely on probabilistic prompt adherence.

[03:45](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=225s) The Spectrum of Agent Behavior: Two Key Axes of Control

Guiding agent behavior at scale spans an entire spectrum of mechanisms, ranging from natural language prompts, standard operating procedures (SOPs), and steering hooks, to rigid workflows and external policy engines. Deoras collapses these various levers onto two defining structural axes:

1. **Location of Control:** Does the control mechanism live *inside* the agent execution loop, or does it sit *outside* the loop? 2. **Action on Misbehavior:** Upon detecting an impending misbehavior, does the system *course-correct and nudge* the model to give it another opportunity to do the right thing, or does it issue a *hard block*?

```mermaid flowchart LR accTitle: Two Axes of Agent Behavioral Control accDescr: Two by two grid comparing inside versus outside loop controls against nudging versus blocking actions. subgraph Inside["Inside Agent Loop"] A["Steering Hooks<br/>(Nudge / Retry / Guide)"] end subgraph Outside["Outside Agent Loop"] B["External Policy Engine<br/>(Hard Block / No Retry)"] end subgraph BuildTime["Build-Time Levers (Excluded)"] C["System Prompts"] D["Standard Operating Procedures (SOPs)"] end ```

Deoras distinguishes runtime controls from build-time configurations. Prompts and SOPs are build-time artifacts that define intended behavior upfront, whereas runtime governance requires active mechanisms: in-loop steering and out-of-loop external policies. In-loop steering operates via guidance and retries, giving the model a chance to recover upon misbehavior. Conversely, external policy engines sitting outside the loop offer no negotiation or retry mechanism; they enforce strict, non-negotiable boundaries and block non-compliant actions entirely.

[06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) Implementing In-Loop Steering Hooks

Diving into concrete implementation, Deoras details how runtime steering hooks operate within the execution cycle. In their architecture, steering hooks trigger at two critical junctures: directly before a tool invocation, and directly after the model generates a response.

Returning to the library agent bug—where recalled books were being renewed without a status check—Deoras contrasts traditional prompt patching with steering hooks. In a naive setup, an engineer attempts to fix the bug by updating the system prompt with emphatic phrasing (such as adding instructions in ALL CAPS, "hoping that your agent is going to listen to you"). In contrast, a pre-tool steering hook executes a deterministic handler that inspects the complete execution ledger. This ledger contains every tool executed up to that point, alongside all inputs and outputs.

```mermaid flowchart TD accTitle: Steering Hook Execution Flow accDescr: Workflow showing a steering hook inspecting the ledger before a tool call and nudging the agent if prerequisites are missing. A["Model Proposes Tool Call: RenewBook"] --> B{"Pre-Tool Steering Hook:<br/>Inspect Execution Ledger"} B -->|"getStatus() in Ledger"| C["Allow Tool Execution"] B -->|"getStatus() NOT in Ledger"| D["Inject Steering Nudge:<br/>'Check book status first'"] D --> E["Model Re-evaluates & Executes getStatus()"] ```

When the model attempts to invoke `RenewBook`, the steering handler checks whether `getStatus` exists in the execution history: - If `getStatus` has not been called, the handler intercepts the call and injects a targeted nudge guiding the model to check the book's status before proceeding. - The defining philosophy of this mechanism is captured by the verb **"guide"**, giving the model an opportunity to self-correct dynamically.

Deoras acknowledges that steering is not without trade-offs. Injecting corrective loops introduces additional tokens and latency directly onto the production hot path. Furthermore, non-codifiable requirements (such as adjusting tone or subjective formatting) rely on LLM-as-a-judge evaluators, which cannot provide 100% deterministic guarantees. However, for codifiable rules, steering handlers are written as standard, deterministic Python code. This allows engineers to unit-test agent constraints locally with absolute certainty that the rules will hold identically in production without surprises.

- **Hook Interception Points:** Pre-tool call (inspecting parameters and prerequisite tool calls) and post-response generation. - **Ledger Inspection:** Handlers inspect historical tool traces and input/output payloads to evaluate state transitions deterministically. - **Trade-offs:** Introduces inference latency and token overhead on the hot path; subjective constraints require heuristic LLM judges, whereas logical constraints benefit from deterministic unit testing.

[10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) Out-of-Loop Policy Enforcement and Cedar

To execute a tool call safely, an agent request must pass through an external gateway. Unlike soft steering mechanisms, the gateway acts deterministically: it either allows the execution or denies it outright. There is no conversational feedback loop, no negotiation, and no retry granted to the model. While developers might question why an external, hard blocker is necessary when elegant in-loop reasoning logic exists, the rationale comes down to security boundaries. Because in-loop steering operates entirely through the model's reasoning loop, anything within that loop can inherently be reasoned with. When enforcement logic is placed inside a conversational layer where it can be debated, an adversarial user will inevitably debate and persuade it. For critical security boundaries and authorization checks, having an unyielding component outside the reasoning loop is non-negotiable.

```mermaid flowchart TD accTitle: Agent Control Boundaries Architecture accDescr: Diagram contrasting out-of-loop deterministic policy enforcement with in-loop reasoning and steering.

subgraph ExternalBoundary["Deterministic Gateway (Out-of-Loop)"] GW["Security Gateway"] Cedar["Cedar Policy Engine\n(Allow / Deny)"] GW --> Cedar end

subgraph AgentLoop["Agentic Reasoning Loop (In-Loop)"] LLM["Agent / LLM"] Handler["Steering Handler"] Ledger["Execution Ledger"] LLM <--> Handler Handler <--> Ledger end

TargetTool["External Tool / Action"]

To translate high-level operator intent into deterministic, verifiable enforcement, systems require structured authorization languages. At AWS, this is implemented using Cedar, an open-source authorization policy language, although other policy engines can serve the same purpose. The core challenge lies in mapping an operator's intent—such as requiring that an agent only schedule appointment slots between 9:00 AM and 9:00 PM—into an unambiguous, mathematically verifiable policy. Although natural language is inherently noisy and ambiguous, using it as an authoring interface works here because this is a one-time setup specifically designed for the agent builder persona. Expressing the initial rule in natural language keeps the authoring process approachable; the system then compiles that natural language into Cedar's formal primitives:

- **Principals**: The actors initiating the operation. - **Actions**: The specific tool or operational call being requested. - **Resources**: The targeted entities or slots being modified. - **Constraints**: Logical guards such as `when` clauses that enforce explicit conditions (e.g., temporal windows).

Because compiled Cedar policies are deterministic, builders can inspect and verify the exact constraints to guarantee their intent was preserved before deploying the policy into production.

---

[12:35](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=755s) Guiding vs. Blocking: The Dilemma of Verbs and Layered Controls

Enforcing agent constraints involves choosing between two very different operational verbs: guiding or blocking. Applying the wrong verb constitutes a major failure mode in production agent design. If a system blocks an agent for a trivial, reversible formatting slip, it terminates and wastes an entire multi-step reasoning run. Conversely, if a system merely attempts to gently guide an agent during an irreversible delete operation or a financial transaction, it risks committing a catastrophic, permanent mistake. Selecting the correct verb depends on three key criteria: how reversible the target action is, how adversarial the operational environment is, and how large the potential blast radius could be.

Because modern enterprise agent workflows are intricate and multi-staged, developers do not need to choose exclusively between guiding or blocking. Instead, systems can implement a layered defense that routes constraints dynamically depending on the scenario:

---

[16:09](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=969s) Closing the User Intent Gap and Building Trustworthy Agents

While the operator specification gap can be bridged with formal policies and structured steering, the user intent gap—the understanding distance between what an end user literally says and what they actually mean—remains inherently harder to solve. Because human language is intrinsically incomplete and ambiguous, this gap cannot be eliminated entirely through technology alone. Instead, production systems must adopt operational best practices to manage it:

- **Reading the plan back**: Having the agent reflect its interpretation and execution plan back to the user for explicit confirmation before beginning execution. - **Requiring confirmation for high-stakes actions**: Pausing to solicit explicit approval whenever the agent is about to execute a costly, high-blast-radius, or irreversible action. - **Clarifying at forks**: Forcing the agent to proactively query the user for clarification whenever it hits ambiguity or branching decision points rather than guessing. - **Hyper-personalization**: Incrementally capturing implicit and explicit user preferences across sessions so the agent understands recurring habits without needing repeated elicitation.

Reliability and predictability in agentic systems cannot be achieved by extending prompt lengths or engaging in endless "whack-a-mole on a prompting treadmill." Prompt engineering alone will not solve alignment; robust agent architectures require a structured, layered control strategy. Builders must carefully choose their enforcement verbs based on reversibility, adversarial threat, and blast radius, while remaining clear-eyed and honest about how to manage the user understanding gaps that cannot be completely closed. By mastering when to steer an agent within its reasoning loop and when to block it with hard external policies, teams can transition agentic AI from merely "impressive" experiments into predictable, trustworthy engines of enterprise productivity.

## Notable quotes > [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) this breakdown is fundamentally not a prompting problem, but a controls problem. — Anoop Deoras

> [01:04](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=64s) if a book has been recalled, it must not be renewed — Anoop Deoras

> [06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) hoping that your agent is going to listen to you — Anoop Deoras

> [06:30](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=390s) guide — Anoop Deoras

> [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) Because in-loop steering operates entirely through the model's reasoning loop, anything within that loop can inherently be reasoned with. When enforcement logic is placed inside a conversational layer where it can be debated, an adversarial user will inevitably debate and persuade it. — Anoop Deoras

> [16:09](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=969s) whack-a-mole on a prompting treadmill — Anoop Deoras

> [16:09](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=969s) Reading the plan back — Anoop Deoras

> [16:09](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=969s) Clarifying at forks — Anoop Deoras

## Mentioned Anoop Deoras, AWS, @Scale, Boris, Cedar, Python

## Video description Join the Director of AWS Agentic AI, Anoop Deoras, for his presentation of "Bridging the Intent Gap in Agentic Systems".

Want to learn more about the @Scale conferences? Visit our website: atscaleconference.com

[Source](https://www.youtube.com/watch?v=Vvhe5qG8IpQ)

中文简报

AWS Agent负责人主张用硬性控制取代提示词以遏制AI错误生产环境中Agent的异常行为源于控制缺失而非提示词缺陷，因此需要确定性的循环内引导与循环外策略引擎来制止非预期执行。 Anoop Deoras认为，生产环境中Agent的异常行为从根本上讲是架构控制问题，而非提示词问题。他将这种脱节归纳为两项挑战：构建者意图与执行之间的操作员规范差距，以及自然语言模糊性导致的用户理解差距。为解决这些问题，Deoras提出了一套双轴控制框架，将通过检查执行账本以微调引导Agent的循环内引导钩子，与像Cedar这样确定性拦截未授权操作的循环外策略引擎相结合。他总结指出，构建值得信赖的Agent系统需要摆脱“提示词跑步机”，转向依据可逆性、爆炸半径和明确的用户在环验证来权衡的分层防御。生产环境中Agent的异常行为根本上源于控制问题而非提示词问题，这意味着自然语言系统提示词无法有效强制执行运行时不变量。 (0:00) AWS从两个维度构建Agent治理体系：控制位置（循环内对比循环外）以及纠正动作（通过重试进行引导/微调对比硬性确定性拦截）。 (3:45) 循环内引导钩子检查执行账本，以标准Python代码强制执行确定性时序规则，从而实现Agent约束条件的本地单元测试。 (6:30) 硬性授权与安全边界需要在网关处引入如Cedar等外部策略引擎，以防止对抗性用户绕过循环内推理防御进行交涉渗透。 (10:00)

Reading summary

Anoop Deoras from AWS argues that the failure of AI agents in production is fundamentally a controls problem rather than a prompting issue, stemming from distinct gaps in operator specification and user understanding. Differentiating Specification and Understanding Gaps (1:04) Deoras identifies two core failure modes when deploying agents from local setups to production environments. The operator specification gap occurs when the agent deviates from the developer's operational rules, such as a library management agent renewing recalled books because natural language prompts fail to mandate prerequisite tool checks like status verifications. The user understanding gap arises from the inherent ambiguity between what a human user explicitly requests and what they actually intend. While prompt engineering often attempts to patch these discrepancies with emphatic phrasing, Deoras contends that natural language cannot guarantee execution logic. Closing the specification gap requires deterministic runtime controls rather than probabilistic prompt adherence, whereas user ambiguity demands conversational clarification patterns. In-Loop Steering Hooks for Dynamic Course Correction (6:30) To address behavioral sequencing and parameter validation, Deoras outlines in-loop steering hooks that trigger directly before tool calls or after response generation. By inspecting an execution ledger that logs all preceding tools and input-output payloads, a deterministic handler written in standard Python can intercept improper calls. For example, if an agent calls an action without executing the prerequisite check, the hook halts the call and injects a targeted nudge guiding the model to self-correct. Deoras notes that in-loop steering introduces trade-offs, including added token consumption and latency directly on the execution hot path. Furthermore, while deterministic code can strictly enforce tool dependencies, non-codifiable requirements like tone or formatting must rely on heuristic LLM-as-a-judge evaluators, which cannot provide absolute guarantees. Out-of-Loop Policy Enforcement with Cedar (10:00) For critical security boundaries and access management, Deoras argues that enforcement must reside entirely outside the model's reasoning loop. Because conversational layers can be persuaded or manipulated by adversarial users through prompt injection, external security gateways must enforce hard allow-or-deny decisions without offering feedback, retries, or negotiation to the model. To implement verifiable constraints, AWS compiles natural language operator rules into formal primitives using the Cedar policy language. By structuring rules around principals, actions, resources, and explicit conditions—such as restricting scheduling operations to between 9:00 AM and 9:00 PM—builders can deterministically verify and unit-test authorization guards before deployment. Balancing Layered Controls and Managing User Intent (12:35) Choosing whether to guide or block an agent depends on three criteria: action reversibility, adversarial risk, and blast radius. Deoras warns that hard-blocking minor formatting errors terminates productive reasoning runs, whereas merely nudging irreversible actions or financial transactions creates severe operational risks. Robust architectures layer both mechanisms: external policy engines block authorization breaches, deterministic handlers guide or block based on input severity, and steering handlers guide tool sequencing. Because technology alone cannot resolve human ambiguity, Deoras highlights operational mitigations for the user intent gap. These practices include prompting the agent to reflect its execution plan back to the user, requiring explicit human confirmation before executing high-stakes actions, clarifying at branching decision forks, and building hyper-personalized profiles over time to capture implicit preferences. Deoras concludes that escaping the recurring cycle of prompt adjustments requires implementing structured, layered control architectures. While deterministic in-loop steering and out-of-loop policy engines resolve operator specification gaps, user intent ambiguity cannot be completely eliminated and must be continuously managed through explicit conversational checkpoints.

Detailed notes — zh

## Background 该视频由 YouTube 频道 @Scale 于 2026 年 6 月 23 日发布。内容为 AWS Agentic AI 总监 Anoop Deoras 在 @Scale 大会上的现场演讲，题为《弥合智能体系统中的意图鸿沟》（Bridging the Intent Gap in Agentic Systems），旨在探讨如何解决 Agent 在生产环境中偏离预期行为的核心控制难题。 ## TL;DR 演讲指出 Agent 在生产环境中失效的本质并非提示词编写问题，而是一个系统性的控制问题，并剖析了由规范鸿沟与用户理解鸿沟构成的“意图鸿沟”。Anoop 提出了双维控制光谱：在 Agent 循环内部采用转向钩子（Steering Hooks）通过账本历史温和引导模型自我纠错，在网关处则使用开源 Cedar 策略引擎执行不可协商的硬性拦截，以防对抗性攻击攻破安全底线。针对不同场景，开发者应依据动作可逆性与潜在爆炸半径灵活构建分层控制体系。最终，配合计划复述与主动澄清等交互实践，才能让智能体从“令人惊艳的玩具”演进为真正可信赖、可规模化落地的生产力系统。 ## Atmosphere & style 整场演讲呈现出浓厚而严谨的系统工程与分布式架构风格，务实、敏锐且直击开发者痛点。讲者开篇以生动的图书馆 Agent 反例切入，迅速唤起所有经历过生产事故的开发者的强烈共鸣，随后层层抽丝剥茧，将玄学般的 Prompt 调试拉回到清晰可验证的形式化策略与控制论语境中。演讲不仅逻辑层层递进、论证极具说服力，更带着一种告别“提示词跑步机打地鼠”的工程师清醒感与行业前瞻洞察。 ## Bulletin - [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) 智能体系统在生产环境中偏离预期的核心症结是系统级控制失效，而非 Prompt 编写缺陷，它源于“规范鸿沟”与“用户理解鸿沟”的叠加。 - [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 智能体行为控制机制可解构为控制位置（循环内 vs 循环外）与处置动作（温和引导 vs 硬性拦截）两个核心维度。 - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) 任何处于模型推理循环内的安全规则都可以被对抗性提示词辩驳或说服，因此核心安全鉴权必须移至推理循环外的网关层硬性拦截。 - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) AWS 团队采用开源 Cedar 策略语言将开发者的模糊意图编译为具备形式化验证能力的确定性策略代码，实现零协商拦截。 - [15:58](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=958s) 弥合意图鸿沟需根据任务可逆性与爆炸半径建立分层控制架构，并结合复述计划、主动澄清等工程交互兜底，而非陷入提示词打地鼠的怪圈。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) 智能体从本地测试走向生产环境后的行为失控，本质上是一个系统级控制问题，单纯修改自然语言 Prompt 无法根本解决。 - [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) “意图鸿沟”分为两类：一是开发者预期与生产执行行为脱节的“规范鸿沟”，二是用户口头表述与真实意图偏差的“用户理解鸿沟”。 - [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) 在图书馆智能体案例中，仅在 System Prompt 声明规则无法阻止违规续借，因为 Agent 在决策前根本未调用工具查询状态。 - [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 行为控制的光谱两极对应两套机制：循环内（In-loop）通过转向钩子进行温和纠偏，循环外（Out-of-loop）通过网关策略引擎执行硬性阻断。 - [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 转向钩子利用执行账本追踪历史调用，在工具执行前或响应生成后给模型重新推理和自我修正的机会，但会带来额外 Token 与延迟成本。 - [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 对于语气与主观判断等难以确定性编码的属性需依赖 LLM-as-a-judge，无法提供 100% 确定性；形式化业务逻辑则应编写确定性 Python 规则验证。 - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) 任何位于模型内部推理循环的事物都能被“讲道理”或对抗性攻破，因此鉴权和高危权限必须剥离出推理循环，交由网关绝对拦截。 - [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) 开源策略语言 Cedar 允许将自然语言需求转化为包含主体、操作、资源与约束条件的形式化代码，提供数学层面的可验证性。 - [13:17](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=797s) 选择“引导”还是“拦截”应依据动作可逆性、环境对抗程度及潜在爆炸半径三大维度进行动态分层权衡。 - [13:17](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=797s) 对低风险的格式或顺序偏差采用硬性拦截会白白浪费整个运行周期，而对高风险操作仅用引导则可能造成灾难性后果。 - [15:58](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=958s) 弥合更难解的“用户理解鸿沟”需依赖复述执行计划、高危动作人工确认、关键分支主动澄清与极致个性化记忆等工程交互设计。 - [15:58](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=958s) 在提示词跑步机上玩打地鼠无法解决系统控制缺失，唯有分层控制架构才能推动智能体从“令人印象深刻”蜕变为“值得信赖”。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) 图书馆智能体案例与意图鸿沟的本质在很多智能体（Agent）开发者的日常中，都经历过相同的困境：在本地笔记本电脑上构建了一个 Agent，经过反复测试觉得表现符合预期，随后推向生产环境，结果 Agent 却执行了开发者从未预想过的行为。这并不是一个单纯的 Prompt 编写问题，其核心本质是一个**控制（Controls）问题**。以构建一个“图书管理智能体”（Library Agent）为例： - 该 Agent 需要识别用户身份、理解用户画像与偏好，并在图书到期时发送归还提醒。 - 业务中存在一条明确规则：“已被召回的图书（recalled books）不得续借”。 - 开发者通常会将这条规则以自然语言的形式直接写进 System Prompt 中，在本地测试看似运转正常，便部署到生产环境。 - 然而在生产环境中，Agent 依然频繁为已被召回的图书办理续借。出现这一故障的根本原因在于，规则仅仅作为自然语言写在 System Prompt 里，Agent 在做续借决策前压根就没有调用工具去查询该图书的实际状态。开发者期望 Agent 执行的操作，与 Agent 在生产环境中真实执行的操作之间，出现了巨大的断层。这种“意图鸿沟”（Intent Gap）实际上由两个独立的鸿沟构成： 1. **操作者鸿沟 / 规范鸿沟（Operator Gap / Specification Gap）**：开发者期望 Agent 展现的行为，与 Agent 在生产环境中实际执行的行为之间的差距。 2. **用户理解鸿沟（User Understanding Gap）**：用户口头表达的指令，与其内心真实意图之间的差距。后者的弥合难度要高得多。 ```mermaid flowchart LR accTitle: Intent Gap Types accDescr: Specification Gap versus User Understanding Gap in Agent systems subgraph SpecGap["规范鸿沟 (Specification Gap)"] A["开发者意图"] -->|"控制与对齐偏差"| B["Agent 生产行为"] end subgraph UnderstandGap["理解鸿沟 (Understanding Gap)"] C["用户口头表达"] -->|"语义与意图偏差"| D["用户真实意图"] end ``` ### [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 行为控制的两大维度与转向钩子机制在大规模场景下引导 Agent 的行为存在一个连续的光谱：一端是 Prompt，接着是标准作业程序（SOP）、转向钩子（Steering Hooks）、严格的工作流（Rigid Workflows），另一端则是外部策略引擎（External Policy）。如果将这些机制进行抽象，可以收敛到两个核心控制维度： - **控制逻辑的位置**：控制是发生在 Agent 循环内部（Inside the loop），还是位于循环外部（Outside the loop）？ - **发现异常时的动作**：当检测到 Agent 即将发生违规行为时，是进行**温和纠偏（Nudge/Course-correct）**让模型重试，还是直接**硬性拦截（Hard Block）**？ Prompt 和 SOP 主要是在构建期（Build time）塑造模型行为，而 Steering Hooks 与外部策略引擎则是在运行期（Runtime）实时控制行为。 | 控制机制 | 所在位置 | 应对策略 | 容错机制 | 核心哲学 | |:--- |:--- |:--- |:--- |:--- | | **转向钩子 (Steering Hooks)** | Agent 循环内部 (In-loop) | 温和微调 / 引导 (Nudge) | 允许重试 (Retry/Redo) | 引导 (Guide) | | **策略引擎 (Policy Engine)** | Agent 循环外部 / 网关处 (Out-of-loop) | 强行阻断 (Block) | 零协商、不可重试 | 拦截 (Enforce) | 在转向钩子（Steering Hook）的具体实现中，钩子通常在两个关键节点触发：一是**工具调用前（Pre-tool call）**，二是**模型生成响应后（Post-response）**。回到图书馆 Agent 的违规案例： - **传统非转向做法**：开发者只能去改 System Prompt，甚至加上全大写的单词（如“YOU SHOULD NOT”），寄希望于模型能遵守全大写的指令。 - **转向机制做法**：处理程序（Handler）拥有完整的执行账本（Ledger），能查看此前调用的所有工具及其输入和输出。在执行 `RenewBook` 工具前，Handler 会自动检查 `getStatus` 是否被调用过。若未调用，系统不会直接报错中断，而是向模型发出温和的提示或建议，提醒它必须先检查图书状态，给模型一个自我修正并做正确决策的机会。这种机制并非毫无代价（"There is no free lunch"）。引导模型重新推理会消耗额外的 Token，并增加生产环境关键路径（Hotpath）的推理延迟。此外，对于语气或输出格式等难以确定性编码的属性，通常需要依赖 LLM-as-a-judge，这无法提供 100% 的确定性保证；而针对可形式化编码的业务逻辑，则可以通过编写确定性的 Python 规则并进行单元测试，从而确保本地测试的行为在生产环境中绝对可靠，消除不确定性。而在光谱的另一端，外部策略引擎则直接驻留在网关（Gateway）层，执行不可协商的强制拦截。 ### [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) 硬性拦截与 Cedar 策略：为何安全边界必须独立于推理循环之外在智能体（Agent）调用外部工具时，所有请求都必须经过网关（Gateway）。网关的处理逻辑非黑即白：要么允许（Allow），要么拒绝（Deny），其间没有任何反馈循环，不提供重试机会，更不存在任何协商妥协的余地。针对这种机制，开发者往往会产生疑问：既然在之前的引导机制（Steering）中已经具备了优雅的“循环内（In-loop）推理逻辑”，为什么还要在智能体推理循环之外设立一道生硬的硬性阻断器？其核心本质在于：“引导机制依赖模型的推理循环运行，而任何在模型推理循环中运行的事物，都是可以被辩论和说服的（anything that can run through model's reasoning loop can be reasoned with）”。一旦将安全执行逻辑置于一个可以被讨价还价的组件中，对抗性恶意用户就能利用这一点与模型反复拉扯并最终攻破防线。在涉及安全和授权（Authorization）的场景下，这绝非系统设计者所能承受的风险。为了以确定且可验证的方式实现这些策略，AWS 团队采用了开源授权策略语言 Cedar（语音转录误作 seder/SEDR）。这项功能面向的目标群体是智能体构建者（Agent Builder）。构建者心中通常持有明确的业务意图，例如“该智能体仅允许预约上午 9:00 至晚上 9:00 之间的时段”。将这种模糊的自然语言意图转换为确定性规则的关键，在于将自然语言请求编译为可形式化分析的策略语言。尽管自然语言本身存在信息损耗和歧义，但这套转换机制是专门为构建者打造的一次性操作（one-time exercise）。自然语言接口大幅降低了构建者的表达门槛，而生成的 Cedar 规则则提供了包含主体（Principal）、操作（Action）、资源（Resource）以及 `when` 约束条件的确定性代码，构建者可以在上线前逐项核验其正确性。 ```mermaid flowchart TD accTitle: 智能体策略执行双轨制架构 accDescr: 展示模型内部引导机制与外部网关硬性策略拦截的区别，区分安全鉴权与常规推理 subgraph External["外部网关拦截 (Gateway)"] A["Agent 工具调用请求"] --> B{"Cedar 策略引擎"} B -->|"符合策略"| C["允许 (Allow) 执行"] B -->|"违规操作"| D["绝对拒绝 (Deny) 无协商"] end subgraph Internal["循环内引导机制 (In-loop Steering)"] E["格式/参数轻微偏差"] --> F{"引导处理器 (Handler)"} F -->|"账本历史 + 提示修正"| G["模型自主纠偏与重试"] end ``` - **外部阻断的必要性**：任何处于大模型认知链路内的规则都能被“讲道理”，绝不能将关键的安全授权防线暴露在对抗性提示词攻击（Adversarial Attacks）之下。 - **Cedar 策略核心元素**：包含主体（Principal）、操作（Action）、资源（Resource）以及约束条件（如 `when` 语句），提供严密的形式化数学验证能力。 - **动词选型的代价**：在策略约束中选择“引导（Guide）”还是“拦截（Block）”至关重要。若对可逆的格式轻微偏差施加拦截，会彻底浪费整个 Agent 运行周期；而若对资金划转或不可逆删除这类高危操作仅仅施加引导，则可能直接导致灾难性的不可逆后果。 --- ### [13:17](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=797s) 分层控制策略：根据任务可逆性与爆炸半径权衡“引导”与“拦截” 选择“引导”还是“拦截”，取决于三个核心维度：动作的可逆性（Reversibility）、环境的对抗程度（Adversarial Level），以及潜在的爆炸半径（Blast Radius）。这两者并不是非此即彼的对立关系，在复杂的真实业务流中，完全可以通过分层机制（Layered Mechanism）将它们组合起来。针对严格的安全或权限边界，没有任何商量余地，必须直接交由策略引擎（Policy Engine）执行硬拦截；而对于工具所需的形式化参数检查，则需视爆炸半径的大小动态决定——轻微偏差通过引导轻推（Nudge）模型自我修正，高危调用则直接就地阻断。这种分层设计在控制工具调用顺序（Tool Ordering）时同样发挥着巨大威力。引导处理器能够访问整个交互账本（Ledger），并将这一历史上下文反馈给模型。一旦模型在完成任务时选错了工具调用顺序，账本信息可以指引智能体理解正确的时序依赖并自主纠偏。然而，当边界涉及到语言基调（Tone）、排版格式或主观价值判断（Judgment）时，业内目前缺乏绝对确定性的保障手段，通常必须求助于大模型作为裁判（LLM-as-a-judge）。这种方式虽然是目前针对主观维度的最优选择，但客观上也无法提供 100% 的绝对确定性保证。 | 控制层级 | 适用场景 | 核心机制 | 应对动作与后果 | |:--- |:--- |:--- |:--- | | **硬性安全策略层** | 权限鉴权、资金划转、数据硬删除 | Cedar 策略引擎、API 网关 | **绝对拦截（Block）**：无上下文协商，直接终止违规调用 | | **工具与参数验证层** | 参数类型错误、工具调用顺序错误 | 账本检查（Ledger）、规则校验器 | **视爆炸半径动态切换**：大爆炸半径直接拦截，低风险则反馈纠偏 | | **认知与表述层** | 输出格式微调、语调匹配、主观评估 | LLM-as-a-judge、提示词微调 | **循环内引导（Steering）**：允许模型根据反馈反思并重试 | --- ### [15:58](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=958s) 弥合用户理解鸿沟与迈向值得信赖的智能体在厘清了智能体构建者侧的规范鸿沟之后，目光必须转向更难解决的“用户意图鸿沟（User Intent Gap）”——即用户口头所说与其实际真实意图之间的理解差距。弥合这一鸿沟需要依赖一套被实战检验的工程最佳实践： 1. **向用户复述执行计划**：在智能体开始执行前，将拆解出的任务计划反向同步给用户确认； 2. **高危动作主动确认**：在即将触发不可逆或高成本操作时，必须设置人工确认断点； 3. **关键分支主动澄清**：遇到语义模糊或执行路径出现分叉（Fork）时，主动发起询问而非自行脑补； 4. **极致个性化（Hyper-personalization）**：主动捕捉并积累用户的显式与隐式偏好，使智能体深度熟悉用户的重复业务习惯，无需在每次交互时重新探查意图。智能体的可靠性与可预测性绝不是靠堆砌更长提示词所能解决的工程问题。在提示词跑步机上玩“打地鼠（Whack-a-mole on a prompting treadmill）”不仅徒劳无功，更掩盖了底层的控制缺陷。智能体系统本质上是一个系统级控制问题（Control Problem），唯有建立起结构化、分层的控制架构，在不同层级准确匹配“引导”与“拦截”动词，并坦诚接受那些无法被彻底消除的理解盲区，才能真正实现跨越。当智能体完成了从“令人印象深刻（Impressive）”到“值得信赖（Trustworthy）”的蜕变，其释放出的生产力将是真正颠覆性的。 - **告别提示词打地鼠**：不要寄希望于通过无限拉长 Prompt 来消除不可预测性，长提示词无法解决系统层面的控制缺失。 - **管理不可弥合的鸿沟**：必须清醒认识到用户理解鸿沟具有内在不确定性，需要通过明确的交互设计（澄清、复述、确认）予以工程化兜底与管理。 - **生产力爆发的前提**：只有可扩展且值得信赖的智能体，才能真正武装每一个开发者、团队和组织，彻底释放 Agentic AI 的规模化效能。 ## Notable quotes > [00:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=0s) 已被召回的图书（recalled books）不得续借 — Anoop Deoras > [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 引导 (Guide) — Anoop Deoras > [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) 拦截 (Enforce) — Anoop Deoras > [05:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=300s) There is no free lunch — Anoop Deoras > [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) anything that can run through model's reasoning loop can be reasoned with — Anoop Deoras > [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) 该智能体仅允许预约上午 9:00 至晚上 9:00 之间的时段 — Anoop Deoras > [10:00](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=600s) one-time exercise — Anoop Deoras > [15:58](https://www.youtube.com/watch?v=Vvhe5qG8IpQ&t=958s) Whack-a-mole on a prompting treadmill — Anoop Deoras ## Mentioned Anoop Deoras, AWS, @Scale, Cedar, LLM-as-a-judge, Python ## Video description Join the Director of AWS Agentic AI, Anoop Deoras, for his presentation of "Bridging the Intent Gap in Agentic Systems". Want to learn more about the @Scale conferences? Visit our website: atscaleconference.com [Source](https://www.youtube.com/watch?v=Vvhe5qG8IpQ)

Source quotes

[0:00] Anoop Deoras: 已被召回的图书（recalled books）不得续借 [5:00] Anoop Deoras: 引导 (Guide) [5:00] Anoop Deoras: 拦截 (Enforce) [5:00] Anoop Deoras: There is no free lunch [10:00] Anoop Deoras: anything that can run through model's reasoning loop can be reasoned with [10:00] Anoop Deoras: 该智能体仅允许预约上午 9:00 至晚上 9:00 之间的时段 [10:00] Anoop Deoras: one-time exercise [15:58] Anoop Deoras: Whack-a-mole on a prompting treadmill

Source quotes

[0:00] Anoop Deoras: this breakdown is fundamentally not a prompting problem, but a controls problem. [1:04] Anoop Deoras: if a book has been recalled, it must not be renewed [6:30] Anoop Deoras: hoping that your agent is going to listen to you [6:30] Anoop Deoras: guide [10:00] Anoop Deoras: Because in-loop steering operates entirely through the model's reasoning loop, anything within that loop can inherently be reasoned with. When enforcement logic is placed inside a conversational layer where it can be debated, an adversarial user will inevitably debate and persuade it. [16:09] Anoop Deoras: whack-a-mole on a prompting treadmill [16:09] Anoop Deoras: Reading the plan back [16:09] Anoop Deoras: Clarifying at forks

Original transcript

[0:00] Hey everyone. It's a real pleasure to be here today. Over the past few years, I have had the chance to watch numerous fascinating talks right here at the at scale podium. And just this morning... listening to Boris talking about coding agents, Such a treat. It's a real humbling experience to be here talking in front of you all today. What I plan to talk today is about Agents, of course. and the intent gap and how to close that. Now, I'm sure most, if not all, in the audience has had this experience. We built an agent. on a laptop. We tested for its behavior, We launched it in the production. And then we saw that the agent did something that we never planned for the agent to do. Why does that happen? That's precisely what I plan to talk today about. and the series of steps and tools that can actually fix the problem. And... I want to convince you by the end of the talk that this is really not a prompting problem. This is really a controls problem. All right, let's dive right into it. So let's take a particular failure case. Let's say we are interested in building a library agent. Now this agent needs to know who you are as a user, needs to understand your profiles, your preferences, needs to send you reminders whenever you are due to, submit the book and all the good things. that you want your library agent to do, you have built it. Now let's say there are some rules that you want your agent to adhere to. One such rule is that If the books have been recalled, they should not be renewed. So what do you do as an agent builder? You go back to the system prompt of your agent, and you write this natural language rule, telling the agent what it should do under what circumstances. Now you test the behavior of your agent on your local setup. you're convinced that your agent is doing the right thing And then you deploy the agent in the production. What you see is that oftentimes your agent is renewing the books that have been recalled. Now, why does that happen? Well, it happens because the rule that you intended your agent to stick to, was written as a natural language prompt in the system prompt. Your agent never executed a tool to check the status of the book. And so this is the gap that we are talking about, is the gap between what the agent builder intended for the agent to do versus what the agent actually executed in the production. Now, the title of my talk says the intent cap, singular. Well, I lied. There are actually two gaps. The first gap is the operator gap. or the specification gap. is the distance between what I intend the agent to do and what the agent does in the production. The second gap is the understanding gap. is the user understanding gap. It's the distance between what the user says and what they actually mean. Now in all honesty, the understanding gap is a significantly harder gap to close. So what I plan to do today for the remainder of the talk, I'm going to be closing on the specification gap. However, at the end, I will come back and talk about some of the mitigation strategies that will mitigate or close the user understanding gap to a great extent if not fully. No How do you guide the behavior of the agent at scale? It's an entire spectrum. On one end of the spectrum, you have prompts. And we saw in our library agent, the rules were codified as a natural language instruction in the system prompt. Then you have standard operating procedures. You have steering hooks, something that I plan to talk about today. you have rigid workflows, and then on the other extreme end, you have external policy. So these are different levers that shape the behavior of the agent. What I plan to do is take all these behavior traits, and then collapse them onto two axes. And these access matter more than anything else. The first dimension in this axis is Where does the controls live? Does it live inside the agent loop? or does it live outside? and the second one and this is something that i want to i want you to hold on to for the remainder of [5:00] Talk is, Upon detecting that your agent is about to misbehave, What is it that you would like to do? Would you like to course correct, nudge the model, so that it has another opportunity to do the right thing? Or do you block the operation? Now, The two corrective Behaviors, they sit at the opposite end of the spectrum. Steering is within loop, Nudging behavior. you are basically giving the model a chance to retry. and redo upon understanding that the agent is misbehaving. On the diametrically opposite end of the behavior spectrum, you have policy engine. In policy engine, there is no room for negotiation. There is no room for retry. It's a hard block, whether you agree that the agent should execute or you block that action. Now, you may be wondering why I do not have SOPs or standard operating procedures and prompts on those axes. The reason for that is that SOPs and prompts They shape the behavior of the agent during build time. While what I'm interested in is shaping the behavior of the agent during runtime. And that is where steering and external policy come into the picture. For the remainder of the talk, I will dive deep into one way of implementing some of these steering hooks, and one way of implementing the external policy. Now, Let's look at a particular implementation of how you can steer the model. Now, the steering hook in our implementation, it fires at two places. The first one is right before the tool call, And the second is right after the model has given a response. So now let's go back to our bug when we were building our library agent. The bug that we saw, we were noticing in our production, was that the agent was renewing the books, that were recalled. This is not the behavior we want our agent to adhere to. So now, when we observe this behavior, In In the non-steering world, what you would do is you would go back to the system prompt and you would add couple notes saying, hey, agent, you should not, or you should, all in capitals, hoping that your agent is going to listen to you, especially when the instructions are written in all caps. versus In the steering mechanism, what you have is that the handler has access to the entire ledger. It has access to all the tools that have been executed till that point. and all the input and the corresponding output of all the tools. What that means is that right before calling a RenewBook tool, the handler is gonna look at whether get status was ever called. If not, then in the counterfactual world, in the non-stating world, you would have to go and edit the prompt. But in the steering world, you are going to guide the model. you are going to nudge the model. You are going to suggest to the model that, hey, it needs to check the status of the book before executing this tool. Now, if there is one verb that comes to my mind to define staring, that verb is guide. And that's the entire philosophy. You are giving model another chance to recover. You are giving model gentle nudges. so that it has another chance to do the right thing. Now, there is no free lunch. And this is true even here. Any time you are guiding the model, you have tokens and inferency on the production hotpath. Similarly, for things that can't be codified, such as, say, tone or format, for that you really don't have a deterministic way of writing a code. You have to use a Elelem as a judge. And unfortunately, LLM as a judge does not give you 100% guarantee. Versus that for things that can be codified, you can actually unit test them. What I showed in the previous slide is a deterministic piece of Python code. You can unit test that, which means that all the rules that you are testing in your local setup is guaranteed to work in production. There are no surprises. Now, let's talk about policy engine. The other extreme where you have to take a bit more stricter stance on what you want to do when you deem your agent is misbehaving. Now, in our implementation, Policy Engine sits at the gateway. So anytime the agent has [10:00] To make a call to the tool, It has to go through the gateway. And so your gateway is either going to allow or it's going to deny. There is going to be no feedback. There is going to be no opportunity for retry, no negotiations. Now let's take a pause here. You may wonder, why would anybody want to have a hard blocker outside the agentic reasoning loop? when we have a very elegant in-loop reasoning logic that we just saw in the staring slides. Well, The reason for that is that your steering runs through model's reasoning loop. And anything that can run through model's reasoning loop can be reasoned with. So if your enforcement logic sits in a component that can be argued with, your adversarial user gets to argue with it. And that's precisely what you don't want. for use cases that involve security, and authorization. Now, you don't... The way you have to implement these policies, how do you achieve that? There are a variety of ways you can achieve writing policies that determine whether an agent should execute a certain task or not. The way we have implemented this is we are using an authorization language called seder. It doesn't have to be SEDR. It's just one way of implementing it. No. The operator, remember the persona for whom we are solving this problem, we are building these tools, is the agent builder persona. So they have certain intent in their mind. They want the agent to adhere to certain specifications. So, Let's say for a particular agent, the intent that we want the agent to adhere to is, that the agent should allow appointment slots between 9:00 AM and 9:00 PM. How do you go from this intent to a policy that is deterministic and verifiable? Now, the way we have done this is translate the queries the natural language query, into a language that you can analyze. Now, you may be wondering, well, isn't natural language lossy and noisy? Wasn't that the problem to begin with? The whole problem of user intent is that natural language is ambiguous. And therefore, it's very hard to close the intent gap, especially the user understanding gap. So then how is it that we are solving the problem of closing the gap by going back to natural language? You should remember that, again, I have to remind that this is being built for the agent builder persona, meaning this is a one time exercise, if you will. The beauty of giving the natural language lever to the user is that it's less overwhelming to them. They can express the intent in the national language, but the verifiable language is something that they can, is deterministic and then they can go through and check for the correctness. It's a one-time exercise. So particularly in the case of CEDR, this language has principles, it has actions, it has resources, And it has constraints such as when. So when you take a natural language intent and you translate that into a certain language, you can actually go through the constraints and make sure that the intent that you had in mind is getting translated correctly into the policy. Now, Between guiding and blocking is the same constraint But there are two different verbs. And picking the wrong verb is a failure mode in itself. So you block. a reversible formatting slip of an agent And you have wasted the entire agent run. versus if you guide a money movement or an irreversible delete operation, then you are committing to an irreversible operation. So which verb to pick is really a function of how reversible my action is. how adversarial my task is, my environment is. and how big of the blast radius I'm talking about. Now, the interesting thing is that you don't have to pick one over the other. you can actually route these constraints in a layered mechanism. The typical agentic workflows that we are talking about are complex, and they are huge, and the complexity level is high, meaning we have an opportunity to layer these constraints depending upon the scenario. So for example, if you have a hard security or authorization boundary, No question, sir. [15:00] we go with the policy engine. versus if you have deterministic parameter check required by the tools, Depending upon the blast radius, one may choose to have steering, meaning giving nudge to the agent so that it course corrects. Or if the blast radius is big, then we can block that operation. Similarly, tool ordering. Now, in steering, Because the A handler has access to the entire ledger Giving access of that ledger to the model allows the model to course correct. If there is a certain sequence of tools that you want your agent to adhere to, Giving agents the access to the ledger allows the agent to course correct and understand what is the right tool ordering. if it has made a mistake in picking the right tools. for the right task. Similarly, for tone and format and judgment, steering is the way to go. However, for things such as tone and judgment, there is no deterministic way of ensuring that the agent will do the right thing. You have to resort to LLM as a judge, which unfortunately, does not give you 100% guarantee. Now, at the beginning of the talk, I said there are two gaps. The first one is the specification gap or the operator gap. And I spent majority of the time talking about what that gap is, and what are some of the mechanisms that can close the gap. And I said, I'll come back and talk about the user intent cap. So let's talk about that. So as a reminder, user intent gap is the understanding gap. is the distance between what the user said and what they actually meant. it's a much harder gap to close because, again, you know, it's... trying to understand what the user is trying to say to the agent. But there are some best practices that, at least in our work, have proven very useful. I'm just going to walk you through some of those best practices. The first one is reading the plan back to the user. So if the user is interacting with the agent, it's always a good idea to give the plan back to the user for confirmation. Now, whenever you anticipate that your agent is about to make an irreversible or a costly operation, It's always a good idea to get confirmation from the user. Whenever there is ambiguity, whenever the agent is at a fork, and then there are multiple possibilities, Get the clarification from the user. And the best of all, just know your user. capture the preferences both explicit and implicit. So if your agent becomes a hyper-personalized agent, it doesn't have to re-elicit the intent. It already knows who the user is and what are the repeated use cases the user has with the agent. To close, I want to... I'll leave you with a couple thoughts. The first one is that reliability and the predictability of agent is not a longer prompt problem. you don't want to do whack-a-mole on a prompting treadmill. It's a control problem. And I hope you are convinced that longer prompt is not going to solve this problem. you need to have very structured layered control strategy to build these reliable agents. Second, Choosing which verb to use at which layer is really a function of how reversible your task is. and how adversarial your environment is and how big of the blast radius you really are talking about. And the third one, for the gaps that you really can't close, which user understanding being one of them, you need to be honest about how to manage them. At the end, I want to say that learning how to steer the model, learning when to block the model, we actually have a huge opportunity in front of us to enable the agents to take the step from being impressive to trustworthy. Now, imagine If you have Scalable agents that are trustworthy, the type of productivity that they are going to unleash is going to be magical. And I'm really looking forward to that future where every developer, every team, every organization becomes highly productive virtue of these trustworthy scalable agents. Thank you so much for your attention.