AWS Adds Formal Verification to Bedrock to Prevent Rogue AI Actions

From Probabilistic to Provable: Deploying AI Agents You Can Trust presented by Erin Kraemer, AWS — Ai4

Ai4

Empirical testing fails to guarantee autonomous AI safety across combinatorial execution paths, making mathematical proof outside the model's reasoning loop the only way to scale enterprise agent deployments.

AI-assisted summary of From Probabilistic to Provable: Deploying AI Agents You Can Trust presented by Erin Kraemer, AWS — original source. Check the original for full context.

Source published: 2026-08-23

Detailed notes

## Background Presented by Erin Kraemer, Sr. Principal Technical Product Manager at AWS, and published on YouTube by Ai4 on August 23, 2026, this talk addresses the operational challenges of deploying autonomous AI agents at scale. The session outlines how a unified governance enforcement layer and neurosymbolic automated reasoning pair LLMs with mathematical logic to provide provable behavioral guarantees instead of probabilistic assumptions.

## TL;DR Traditional authorization architectures and empirical testing fail when applied to autonomous agents whose operational risks fluctuate dynamically based on context. To bridge this governance gap, AWS introduces the five pillars of Amazon Bedrock Agent Core—Identity, Contextual Policy, Registries, Observability, and Continuous Evaluation—alongside temporal policy enforcement via the open-source Dogwood language. By applying neurosymbolic automated reasoning derived from proven AWS infrastructure tools, enterprises can mathematically verify every possible execution path and response before runtime. This provable approach resolves the tension between operational velocity and safety, allowing organizations to grant agents full autonomy within rigorously proven boundaries.

## Atmosphere & style The presentation carries an authoritative, pragmatic, and relatable tone, opening with interactive audience polling and nostalgic callbacks to late-night software deployment babysitting at Amazon. Kraemer demystifies dense mathematical verification concepts with intuitive real-world analogies, grounding theoretical automated reasoning into actionable production realities. The talk creates an empowering, solutions-focused atmosphere that guides engineering leaders away from the paralysis of pilot purgatory and towards confident, mathematically backed agent autonomy.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) Enterprises face a severe confidence gap where agents are being built faster than organizations can verify what they do unattended in production. - [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) Unlike deterministic traditional software deployments, autonomous agents dynamically select tools and execute subjective decisions, breaking legacy operational and monitoring paradigms. - [02:13](https://www.youtube.com/watch?v=8uJez6QAYRk&t=133s) Enterprise AI initiatives repeatedly stall in pilot purgatory due to a lack of operational visibility, fragmented guardrails, and an inability to systematically prove safety to security teams. - [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) Static, flat permissions fail for agents because operational risk fluctuates dynamically with context even when using the exact same API. - [07:08](https://www.youtube.com/watch?v=8uJez6QAYRk&t=428s) Agent governance unifies safety, security, and compliance into a single enforcement layer structured around five core pillars: Identity, Contextual Policy, Registries, Observability, and Evaluation. - [07:08](https://www.youtube.com/watch?v=8uJez6QAYRk&t=428s) Deterministic policy gates must execute outside the agent's internal reasoning loop to prevent prompt injection or internal rationalization from bypassing rules. - [11:02](https://www.youtube.com/watch?v=8uJez6QAYRk&t=662s) Bedrock Agent Core introduces stateful temporal policies and the open-source Dogwood language to govern multi-step action sequences across entire interaction sessions. - [11:02](https://www.youtube.com/watch?v=8uJez6QAYRk&t=662s) Empirical testing alone cannot guarantee safety across infinite potential execution paths, necessitating neurosymbolic AI that combines LLMs with mathematical logic. - [13:09](https://www.youtube.com/watch?v=8uJez6QAYRk&t=789s) Automated reasoning provides formal mathematical proofs that organizational invariants and boundaries cannot be breached under any possible execution path. - [14:48](https://www.youtube.com/watch?v=8uJez6QAYRk&t=888s) Automated reasoning in AI builds upon battle-tested AWS infrastructure security mechanisms, including IAM Access Analyzer, S3 Block Public Access, and VPC Reachability Analyzer. - [14:48](https://www.youtube.com/watch?v=8uJez6QAYRk&t=888s) Neurosymbolic verification shifts safety left to the specification stage (using tools like Kiro) to eliminate ambiguities and contradictions before code is generated. - [16:27](https://www.youtube.com/watch?v=8uJez6QAYRk&t=987s) Provable governance resolves the false trade-off between human-in-the-loop bottlenecks and blind production risk, enabling faster deployment within verified boundaries.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) The Confidence Gap: From Babysitting Builds to Governing Agents

Erin Kraemer opens her presentation with a live, interactive question for the audience: how many people in the room currently have AI agents running in production? A fair number of hands go up. She then delivers the catch: keep your hand raised only if you can confidently state everything those agents did in the last 24 hours—every single action taken, every piece of data touched, and every decision made. Almost every hand in the room drops immediately, save for one audience member sitting right up front whom Kraemer playfully congratulates. This stark drop-off encapsulates the current dilemma in autonomous AI: organizations are building agents far faster than they can establish the confidence to actually let them run unattended.

For seasoned technologists, this dynamic is familiar. Kraemer recalls her start at Amazon a quarter of a century ago, when engineering teams routinely stayed up late into the night babysitting software builds and manually monitoring deployments to ensure nothing broke in production. Over time, Amazon engineered rigorous mechanisms to eliminate that manual babysitting: canaries, automated rollbacks, and rapid-response alarms that detected anomalies faster than any human could. These systems allowed engineers to finally sleep at night because correctness was backed by reliable mechanisms. Yet, that operational model relied on a fundamental property: traditional software deployments are deterministic. Even at Amazon's massive website scale, deployments followed known, repeatable execution steps across defined systems with strict, enforceable boundaries.

Autonomous agents shatter that paradigm. Unlike traditional code paths, agents select their tools dynamically on the fly and exercise subjective judgment. While an engineering team can manually observe a few agents executing narrow tasks, that manual oversight falls apart as agents gain autonomy and multiply across an enterprise. At scale, the operational question permanently shifts from "does it work?" to "can I prove that it stays in bounds when I'm not watching?" Today, most enterprises cannot answer that question affirmatively.

[02:13](https://www.youtube.com/watch?v=8uJez6QAYRk&t=133s) The Enterprise AI Dilemma: Trapped in Pilot

Over the past year, Kraemer consulted with enterprise leaders deploying AI agents across diverse sectors, including financial services, healthcare, manufacturing, and the public sector. Regardless of the domain, these organizations run into the exact same three systemic roadblocks: 1. *Lack of operational visibility:* Teams have deployed agents into production or testing, but they do not truly know what those agents are doing across their decision paths. 2. *Fragmented and inconsistent guardrails:* Individual teams build bespoke, ad hoc guardrails that fail to conform to any centralized standard across the enterprise. 3. *The pilot purgatory trap:* Initiatives remain indefinitely stuck in pilot phases because teams cannot mathematically or systematically prove to internal security teams—or even to themselves—that the agents are safe to ship.

Governance, boundary verification, and runtime validation have simply failed to keep pace with model capabilities. This widening governance gap has become the single biggest bottleneck holding back enterprise AI adoption. Bridging this divide demands an approach beyond post-hoc detection: it requires upfront mathematical proof that critical operational boundaries cannot be breached before an action ever executes.

[03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) Flat Permissions vs. Dynamic Risk: The Email Agent Problem

To illustrate why legacy access controls fail with agentic systems, Kraemer presents a deceptively simple scenario: granting an agent permission to manage an email inbox. The agent connects via standard email APIs, allowing it to read incoming messages, draft responses, and send emails. It operates under standard corporate baseline policies, such as prohibiting anti-competitive phrasing and preventing external leaks of customer data. When the agent drafts a routine message to a direct report or a daily collaborator, the stakes are low; a quick glance suffices, and the user trusts the agent’s judgment.

The operational calculus completely changes when the recipient is a Vice President or the CEO. In that high-stakes context, the agent must perform rigorous background work: grounding every single figure against authoritative sources, cross-checking every metric, flagging points of uncertainty, and enforcing a mandatory human-in-the-loop review before hitting send—because, as Kraemer quips, "if that email goes wrong, I look really stupid." Although the underlying tool, the agent, and the API permissions remain identical, the real-world operational risk swings wildly based purely on context.

Traditional authorization architectures cannot comprehend this distinction. Standard Identity and Access Management (IAM) policies, service control policies (SCPs), and data access control lists operate on binary, static logic: is the entity permitted to execute `SendEmail`—yes or no? They possess no native mechanism to evaluate multi-dimensional conditions such as recipient seniority, data sensitivity, or mandatory procedural prerequisites. Because authorization permissions are flat while operational risk is highly dynamic, teams find themselves trapped between two equally unacceptable extremes: either lock the agent down with exhaustive manual approval gates until it becomes functionally useless, or grant blanket permissions and blindly "trust and hope" that the probabilistic model never errs.

[07:08](https://www.youtube.com/watch?v=8uJez6QAYRk&t=428s) The Five Pillars of Bedrock Agent Core

Historically, enterprises treated governance, safety, and security as three independent disciplines maintained by distinct organizational teams. Autonomous agents collapse these silos into a single problem space because a single erroneous or hijacked action can trigger compliance failures, safety breaches, and severe security compromises simultaneously. Governing autonomous agents at scale requires a unified enforcement foundation built upon five fundamental pillars:

```mermaid flowchart TD accTitle: The Five Pillars of Amazon Bedrock Agent Core subgraph AgentBoundary["Agent Reasoning & Execution"] Agent["Autonomous Agent (LLM)"] ActionReq["Action Request / Tool Invocation"] Agent --> ActionReq end

subgraph AgentCore["Amazon Bedrock Agent Core Enforcement Layer"] Registry["1. Agent Registry (Discovery, Cataloging & Standards)"] Identity["2. Dynamic Identity (Least Privilege & Delegated Scopes)"] Policy["3. Contextual Policy (Deterministic Runtime Rules)"] Obs["4. Unified Observability (Full Replay & Decision Tracing)"] Eval["5. Continuous Evaluation (Simulations & Drift Regressions)"]

ActionReq --> Identity Identity --> Policy Policy --> Obs Registry -. Catalogs.- Agent Eval -. Monitors.- Obs end

subgraph Target["External Environment"] APIs["Enterprise APIs, DBs & Services"] Policy -- "Enforced at Gateway" --> APIs end ```

- **Identity:** Agents must be treated as first-class operational actors. They require dedicated machine identities, scoped role delegations, fine-grained access tokens, and strict adherence to the principle of least privilege, explicitly defining whose authority they act under. - **Contextual Policy:** Unlike static IAM checks that merely assess structural API capabilities, modern policy engines must evaluate live environmental context—analyzing who the recipient is, what sensitivity level the data holds, and what downstream risks exist. Crucially, these deterministic policy gates execute entirely outside the agent's internal reasoning loop, ensuring the model cannot use prompt injection or internal rationalizations to bypass rules. - **Registries:** Enterprises cannot govern shadow AI. As agent deployments scale from five to five hundred across disparate departments, a centralized registry provides a unified catalog to discover, audit, curate, and publish approved agents, internal tools, and accessible resources. - **Observability:** Centralizing all agent traffic through a single governance layer exposes the end-to-end operational path: every foundation model call, tool invocation, intermediate routing decision, and policy evaluation. This unified audit trail enables full execution replay to confirm whether mandatory steps—such as grounding checks prior to sending an executive email—actually occurred. - **Evaluation:** Offline pre-deployment evaluations stress-test agent behavior inside sandbox simulations against adversarial edge cases and compliance vulnerabilities. Crucially, evaluation must remain an active, continuous runtime loop post-launch to catch regressions, drift caused by upstream model weight changes, evolving tool definitions, and long-term memory accumulation.

By assembling these five capabilities, Amazon Bedrock Agent Core provides the underlying harness so developers can deploy fully governed agents in minutes without manually stitching together fragmented compliance tooling.

[11:02](https://www.youtube.com/watch?v=8uJez6QAYRk&t=662s) Session-Wide Enforcement with Temporal Policies and Neurosymbolic AI

Kraemer announces new capabilities within Amazon Bedrock Agent Core designed to govern agent behavior across entire multi-turn interaction sessions. Traditional firewalls and guardrails evaluate each API request in isolation. However, sophisticated risks often emerge from sequences of individual actions that seem benign on their own but become critical violations when linked together.

To solve this, Agent Core introduces *temporal policies* enforced deterministically at the governance gateway outside the agent's boundary. These policies maintain a stateful history of everything the agent has executed previously across the session before granting permission for the next step. To support this, AWS is releasing **Dogwood**, a new open-source policy language purpose-built for expressing time-aware, multi-step agent governance rules that models cannot circumvent regardless of their prompts or toolchains.

Yet even complete runtime observability and temporal policy enforcement hit an unavoidable theoretical ceiling: empirical testing alone cannot prove safety across infinite operational paths. A team might run an agent through a hundred test scenarios and watch its policies successfully enforce grounding and human approvals every time. While valuable, empirical success only demonstrates that an issue *has not yet been observed*—it does not prove that an unintended execution sequence *cannot happen*.

Bridging this divide requires neurosymbolic AI, a paradigm pairing the generative, probabilistic flexibility of large language models with the deterministic, mathematical rigor of automated reasoning: - **Large Language Models (LLMs):** Handle fluid natural language comprehension, dynamic adaptability, and contextual problem-solving across messy, real-world data. - **Automated Reasoning (Symbolic AI):** Delivers absolute mathematical proof upfront that hard organizational invariants, regulatory constraints, and security boundaries can never be breached under any possible execution sequence.

[13:09](https://www.youtube.com/watch?v=8uJez6QAYRk&t=789s) Scaling Beyond Human Cognition: Automated Reasoning and Formal Proofs

Mathematical logic fundamentally alters safety by mathematically proving that a specific property holds true across every conceivable scenario—not merely the narrow slice of paths covered in integration tests, but the millions of untested edge cases. While a human engineer can easily reason through a simple toy example featuring a single agent, one tool, a couple of recipient tiers, and basic conditional logic, real-world enterprise deployments break this mental model. Multiplying the architecture to fifty agents, each wired to a dozen tools with distinct organizational roles, sensitivity classifications, and cross-channel output routing creates thousands of potential execution paths. In such dense combinatorial networks, a multi-step chain of individually permitted actions can quietly combine into an unauthorized data exposure that nobody ever explicitly approved.

```mermaid flowchart TD accTitle: Multi-Agent Policy Explosion and Provable Verification subgraph AgentEcosystem["Multi-Agent Mesh: Combinatorial Explosion"] A["50+ Autonomous Agents"] --> B["Dozens of Tools & APIs"] B --> C["Complex Role Matrices"] C --> D["Multi-Channel Outputs"] end

subgraph VerificationLayer["Automated Reasoning & Guardrails"] D --> E["Automated Reasoning Engine"] E --> F["Action Space Proof: Verify zero unauthorized data paths"] E --> G["Amazon Bedrock Guardrails: Verify response alignment deterministically"] end

subgraph Outcome["Guaranteed Execution"] F --> H["Provably Safe Deployment"] G --> H end ```

Automated reasoning provides mathematical certainty where traditional testing falls short. While testing only verifies the specific scenarios an engineer anticipated, formal proof exhaustively evaluates every route, settling role permissions, data categories, and channel paths before an agent ever touches a production request. This exact verification applies equally to the output layer: automated reasoning checks within Amazon Bedrock Guardrails mathematically verify whether a model's generated response complies with predefined organizational policies, establishing identical rigor across both the actions an agent takes and the answers it delivers.

- **Definitive formal guarantees**: Automated reasoning can mathematically prove statements such as: *"Under this condition, no agent under any role has an authorization path that lets sensitive data reach a channel without the required clearance."* - **Bidirectional protection**: Formal verification covers both agent action execution paths (input/tool selection) and content generation (output verification via Amazon Bedrock Guardrails).

---

[14:48](https://www.youtube.com/watch?v=8uJez6QAYRk&t=888s) Proven Infrastructure Heritage and Spec-Time Verification with Kiro

This automated reasoning approach is not an unproven academic experiment; it represents the exact core technology that has secured AWS cloud infrastructure for over a decade. Systems such as AWS Identity and Access Management (IAM) Access Analyzer, Amazon S3 Block Public Access, and VPC Reachability Analyzer evaluate billions of queries every single day, systematically proving whether IAM policies allow unintended data access or mathematically verifying network reachability paths across virtual private clouds. The technology is thoroughly proven; the shift is bringing these formal capabilities directly into the AI agent domain.

```mermaid flowchart LR accTitle: Shift-Left Verification Pipeline with Kiro subgraph SpecPhase["Specification Stage"] A["Natural Language Spec"] --> B["Kiro Neurosymbolic Engine"] B --> C["Detect Ambiguities & Contradictions"] C --> D["Resolved, Precise Spec"] end

subgraph BuildPhase["Code & Runtime Verification"] D --> E["Code Generation"] D --> F["Property-Based Testing"] E --> G["Formal Proof at Scale"] F --> G end ```

Crucially, formal verification can begin before a single line of software is written. Development environments like Kiro apply neurosymbolic techniques during the specification phase to flag contradictions, structural gaps, and ambiguities before they become hard-coded bugs. When a specification ambiguously states that *"the agent requires approval for high-stakes emails,"* the tool immediately surfaces the ambiguity—forcing the author to define whether "high stakes" hinges on recipient seniority, email content sensitivity, or both. Once code is generated, the platform produces property-based tests from those verified specifications to probe hundreds of edge cases, creating a clear dividing line between testing and proof:

---

[16:27](https://www.youtube.com/watch?v=8uJez6QAYRk&t=987s) The False Dichotomy: Moving Faster by Proving Boundaries

Shifting from probabilistic testing to provable guarantees resolves the persistent tension between operational velocity and organizational trust. Today, enterprises routinely resolve this dilemma through two equally flawed approaches: they either smother AI speed by layering on heavy human reviews for every single action, or they deploy unquantified risk into production and *"hope that that phone stays quiet."* Unifying governance and automated proof introduces a viable third path: define the precise operating boundary, mathematically prove that the boundary cannot be breached, and then grant the agent full autonomy to execute freely within those verified parameters.

```mermaid flowchart TD accTitle: Operating Paradigms for Autonomous Agents subgraph FlawedApproaches["Traditional Zero-Sum Approaches"] A["Heavy Human Review"] --> B["Total Loss of AI Velocity"] C["Unconstrained Deployment"] --> D["Unmeasured Risk & Outages"] end

subgraph ProvableApproach["The Third Option: Provable Governance"] E["Define Exact Policy Boundary"] --> F["Mathematically Prove Boundary Holds"] F --> G["Release Agent Autonomy Inside Boundary"] end ```

In the email agent scenario, organizations no longer need to hobble workflow efficiency by forcing human approval onto routine messages. By formally proving that policy rules strictly govern recipient routing and content sensitivity under all conditions, the agent remains completely autonomous for everyday tasks while high-stakes actions receive mandatory oversight. Trust originates directly from mathematical proof rather than artificial capability restrictions. Just as software engineering moved away from manually babysitting production deployments after building dependable automated delivery mechanisms, enterprise AI governance must transition to formal, provable architectures that allow teams to step away with confidence and sleep at night.

- **Speed without reckless exposure**: Mathematical proof eliminates the need to choose between suffocating human-in-the-loop bottlenecks and blind production risk. - **Autonomy as a function of governance**: Governance is not an operational brake that slows autonomous systems down; deterministic boundaries provide the institutional confidence required to unleash full agent capabilities.

## Notable quotes > Too much constraint defeats the purpose; too little creates unacceptable risk. — Erin Kraemer

> Most approaches address governance, safety, and security as separate disciplines. They aren't. — Erin Kraemer

> organizations can confidently move faster because correctness is proven, not assumed — Erin Kraemer

> [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) can I prove that it stays in bounds when I'm not watching? — Erin Kraemer

> [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) if that email goes wrong, I look really stupid. — Erin Kraemer

> [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) trust and hope — Erin Kraemer

> [13:09](https://www.youtube.com/watch?v=8uJez6QAYRk&t=789s) Under this condition, no agent under any role has an authorization path that lets sensitive data reach a channel without the required clearance. — Erin Kraemer

> [14:48](https://www.youtube.com/watch?v=8uJez6QAYRk&t=888s) the agent requires approval for high-stakes emails — Erin Kraemer

> [16:27](https://www.youtube.com/watch?v=8uJez6QAYRk&t=987s) hope that that phone stays quiet. — Erin Kraemer

## Mentioned Erin Kraemer, AWS, Amazon, Amazon Bedrock Agent Core, Amazon Bedrock Guardrails, Dogwood, Kiro, AWS Identity and Access Management (IAM) Access Analyzer, Amazon S3 Block Public Access, VPC Reachability Analyzer, Ai4

## Video description Erin Kraemer - Sr. Principal Technical Product Manager, AWS

As AI agents gain autonomy, governance becomes the central challenge: how do you give agents enough freedom to be useful while ensuring they stay within the right boundaries? Too much constraint defeats the purpose; too little creates unacceptable risk. Most approaches address governance, safety, and security as separate disciplines. They aren't. They require one enforcement layer that can observe every action, enforce constraints deterministically, and intervene when something goes wrong. This talk covers what it takes to govern autonomous agents at production scale and how neurosymbolic AI raises the bar, pairing the creative power of LLMs with the rigor of mathematical logic to deliver provable guarantees about agent behavior. The result: organizations can confidently move faster because correctness is proven, not assumed

[Source](https://www.youtube.com/watch?v=8uJez6QAYRk)

中文简报

AWS 为 Bedrock 引入形式化验证以防范 AI 失控行为经验测试无法保障自主 AI 在组合式执行路径中的安全性，这使得模型推理循环之外的数学证明成为规模化部署企业级 Agent 的唯一途径。面对运行风险随上下文动态波动的自主 Agent，传统的授权架构与经验测试均告失效。为弥合这一治理鸿沟，AWS 推出了 Amazon Bedrock Agent Core 的五大支柱——身份、上下文策略、注册表、可观测性与持续评估，并借助开源 Dogwood 语言实现时序策略执行。通过应用源自成熟 AWS 基础设施工具的神经符号自动推理，企业可以在运行时之前对所有可能的执行路径及响应进行数学验证。这种可证明的方法化解了业务推进速度与安全性之间的矛盾，使组织能够在经过严格证明的边界内赋予 Agent 充分的自主权。企业在构建自主 AI Agent 的速度已远超验证其生产表现的能力，这打破了传统的确定性部署模式，使大量 AI 项目因缺乏可观测性和碎片化护栏而陷入试点泥潭。 (0:04) 静态平面的 IAM 权限无法应对 Agent 随上下文动态波动的操作风险，治理必须将安全与合规整合为涵盖身份、上下文策略、注册表、可观测性和持续评估的统一执行层。 (3:34) AWS 推出 Bedrock Agent Core，引入开源策略语言 Dogwood 与有状态时序策略，在 Agent 推理回路之外实施确定性治理，有效防止提示注入或跨交互步骤的违规链。 (11:02) 仅靠经验测试无法穷尽 Agent 复杂的执行路径，神经符号 AI 将大语言模型与自动化推理相结合，借助数学逻辑事前证明操作边界绝不被逾越，让企业在确保安全的同时释放 Agent 自主性。 (13:09)

Reading summary

Erin Kraemer of AWS argues that enterprise adoption of autonomous AI agents remains stalled in pilot phases because static access models cannot handle dynamic, probabilistic decision-making. She contends that deploying reliable agents requires combining runtime governance with neurosymbolic automated reasoning to mathematically prove operational boundaries hold under all conditions. Contextual Risk and Static Access Control Failures (3:34) Enterprise agent initiatives frequently stall due to fragmented guardrails, poor operational visibility, and security skepticism. Traditional access architectures like standard Identity and Access Management (IAM) enforce flat, binary permissions—such as whether an entity can invoke an email API or submit an order. However, Kraemer illustrates that real-world operational risk is highly contextual: an agent drafting a message to a direct report carries low risk, but emailing a CEO requires verified data grounding and mandatory human review. Similar discrepancies exist across healthcare claim overrides, escalated financial trades, and large capital purchases. Because legacy permissions cannot assess environmental dimensions like recipient seniority or financial scale, enterprises face a dilemma. Teams either impose exhaustive manual approvals that eliminate the productivity benefits of automation, or grant blanket permissions and rely on the hope that a probabilistic model does not fail. Unified Runtime Governance and Stateful Policy Enforcement (7:08) To manage autonomous systems, Kraemer outlines Amazon Bedrock Agent Core, which unifies safety, security, and compliance across five foundational pillars: dynamic machine identities, contextual policy enforcement, centralized registries, full observability replay trails, and continuous evaluation against model drift. Crucially, these deterministic policy gates execute at the gateway outside the agent's internal reasoning loop, preventing prompt injections or model rationalizations from bypassing operational rules. Because sophisticated risks often develop over a sequence of individually permissible actions rather than a single API call, Agent Core adds temporal policies to evaluate stateful session histories before granting subsequent execution steps. AWS introduced Dogwood, an open-source policy language designed specifically to express time-aware, multi-step governance rules that models cannot circumvent. Formal Verification and Neurosymbolic Automated Reasoning (13:09) Kraemer emphasizes that empirical sandbox testing alone cannot prove safety across complex agent architectures. As enterprises expand to dozens of agents interacting across multiple tools, roles, and output channels, the combinatorial explosion of potential execution paths makes exhaustive empirical testing impossible. Integration tests only demonstrate that evaluated cases have passed; they cannot prove that an unauthorized data exposure path does not exist. To bridge this gap, she advocates neurosymbolic AI, combining probabilistic large language models for fluid reasoning with symbolic automated reasoning for mathematical proof. Borrowing methods from AWS infrastructure security tools—such as IAM Access Analyzer, S3 Block Public Access, and VPC Reachability Analyzer—automated reasoning verifies that operational invariants cannot be violated. Through platforms like Kiro, formal verification can also shift left into the specification stage to resolve ambiguities and generate property-based tests before deployment. Overcoming the Dilemma Between Speed and Governance (16:27) Organizations frequently view governance as an impediment to velocity, vacillating between suffocating human-in-the-loop bottlenecks and deploying unmeasured production risk. Kraemer frames this conflict as a false dichotomy. By formally proving that an agent cannot exceed defined operational boundaries, organizations can automate routine actions entirely while reserving human oversight exclusively for high-stakes decisions. Just as traditional software development moved away from manual deployment monitoring through automated canaries and rollbacks, agent governance must transition toward mathematical verification. In Kraemer's framework, rigid boundaries do not restrict autonomy; rather, provable guarantees provide the operational confidence required to release agents into production unattended. Kraemer concludes that enterprise AI must transition from probabilistic trust to formal mathematical verification to unlock widespread production deployment. While automated reasoning provides provable boundaries, organizations must still define precise specifications upfront and maintain continuous evaluation to monitor model drift and changing external tool dependencies.

精炼阅读摘要

Erin Kraemer（AWS）在 Ai4 的演讲中指出，企业部署自主 AI 智能体时面临动态风险与经验性测试失灵的瓶颈；唯有将大语言模型与基于数学逻辑的自动推理相结合，企业才能在事前证明关键操作边界不被突破，从而摆脱人工监护并安全释放智能体的自主生产力。动态风险与传统权限的脱节 (3:34) Kraemer 指出，尽管许多企业已在测试或生产中运行智能体，但多数项目因无法向安全团队证明系统的确定性边界而停留在“概念验证泥潭”（Pilot Purgatory）。传统访问控制机制（如 IAM 策略和服务控制策略 SCP）基于静态的二元判断（即是否允许调用某项 API），无法应对智能体高度动态的上下文风险。例如，智能体处理日常邮件时风险较低，但向副总裁或首席执行官发送包含关键财务指标的邮件时，则必须引入严格的事实验证与人工审核；传统权限体系无法识别接收方职级或数据敏感度等高阶维度，迫使团队在“施加繁琐的人工审批导致效率归零”与“盲目信任概率模型并承担合规风险”之间走极端。此外，这种风险脱节在医疗报销审核、大额金融交易以及制造业采购等跨行业场景中同样普遍。单纯依靠集成测试或经验性测试无法解决问题，因为针对部分样本用例的测试通过，仅能表明尚未观测到漏洞，却无法在数学上证明系统绝不会发生未经许可的越界行为。独立于推理循环的五维治理 (7:08) 针对以往企业将治理、安全与合规划分为孤立部门的弊端，Kraemer 提出通过 Amazon Bedrock Agent Core 建立集中的执行基石，涵盖五大支柱：赋予智能体独立机器身份与最小权限的动态身份体系；评估实时环境上下文的确定性策略；防止影子 AI 泛滥并实现工具资产标准化的集中注册表；支持全链路审计与决策回放的端到端可观测性；以及防范模型权重变化、工具定义调整和长期记忆累积导致漂移的持续评估机制。 Kraemer 特别强调，上下文策略与安全门禁必须完全运行在智能体内部推理循环之外的治理网关层。这一隔离机制确保了模型无法利用提示注入（Prompt Injection）攻击或自身的内部推理来绕过既定业务规则，从而为多智能体环境构筑了确定性的硬隔离屏障。会话时序策略与神经符号验证 (11:02) 在真实业务中，孤立审视某一次 API 调用往往看似合法，但在多轮会话中串联起来的动作序列却可能演变成严重的数据违规。为此，Agent Core 在网关层引入了具备状态感知的时序策略，通过跟踪全会话的执行历史记录来决定是否放行下一步操作。AWS 同步开源了专用策略语言 Dogwood，允许开发者编写时间感知的多步治理规则，彻底杜绝单步检查的盲区。然而，面对无限的运行路径，即使完全具备运行时可观测性与时序策略，经验性测试依然存在不可逾越的理论上限。为此，系统引入了神经符号 AI（Neurosymbolic AI）架构：利用大语言模型提供灵活的自然语言理解与复杂情境适应能力，同时利用自动推理（符号 AI）在事前提供绝对的数学证明，确保企业核心不变式、合规底线与安全防线在任何执行序列下均不会被攻破。形式化证明工程化与信任重构 (14:48) Kraemer 解释称，形式化验证并非理论设想，而是 AWS 保护云基础设施十余年的核心技术成果（例如每天处理数十亿次查询的 IAM Access Analyzer、S3 Block Public Access 以及 VPC Reachability Analyzer）。如今这一技术被拓展至 AI 智能体治理：面对 50 个以上智能体连接数十种工具所产生的组合爆炸，自动推理引擎能穷尽验证所有状态空间，在输入端证明敏感数据绝不存在未授权流转路径，并在输出端结合 Amazon Bedrock Guardrails 确定性验证生成内容的合规性。这种验证还可以进一步“左移”至代码编写前的规范阶段。开发工具 Kiro 能够在自然语言规范定义阶段自动发现歧义与矛盾，并生成基于属性的测试用例。Kraemer 强调，治理绝非减缓系统创新的减速带；只有通过形式化证明建立可信边界，企业才能免去人工监护的沉重负担，在边界内赋予智能体充分的自主行动权，实现更高的业务运转速度。 Kraemer 总结认为，企业对 AI 智能体的信任机制必须从依靠概率的经验性测试转向基于数学的形式化证明，以此重现传统软件工程摆脱手动盯盘、实现自动化交付的历史进程。不过，该演说主要依托 AWS 生态的技术方案进行阐述，形式化验证在跨多厂商异构模型集成时的工程落地成本及通用性界限，仍有待企业在复杂业务中持续验证。

Detailed notes — zh

## Background 本视频由 Ai4 频道于 2026 年 8 月 23 日发布在 YouTube 平台，由 AWS 高级首席技术产品经理 Erin Kraemer 带来主题演讲。内容聚焦于生产级自主 AI 智能体（Agents）的规模化治理挑战，阐述了如何结合大语言模型与神经符号 AI 中的自动推理技术，为智能体行为提供数学级的确定性证明。 ## TL;DR 随着 AI 智能体自主权不断扩大，传统静态权限与事后检测无法应对动态风险与跨步骤级联越权。AWS 提出了覆盖身份、策略、注册表、可观测性和评估的 Agent Core 统一治理底座，并推出时序策略语言 Dogwood。通过复用在 AWS 云基础设施中深耕十年的自动化推理与形式化证明技术，企业得以从“有限测试”跃迁至“全路径数学可证明性”，在确保安全边界坚不可摧的前提下赋予智能体最大的自主创新速度。 ## Atmosphere & style 整场演讲风格严谨务实且富有极强的行业洞察力与技术说服力。开篇通过生动的现场举手互动，精准击中企业在智能体落地过程中“不敢放手”的普遍焦虑；随后借由亚马逊 25 年前软件发布机制的历史回顾，将前沿的神经符号验证理念转化为直观易懂的工程实践对比。叙事层层递进、逻辑闭环严密，带给观众从面对概率性失控的忐忑到重获数学级掌控力的踏实与笃定感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) 智能体落地的核心瓶颈在于信任而非构建速度：当自主智能体数量激增且动态选择工具时，人工盯防彻底失效，核心诉求已转变为在无人值守时证明其绝不越界。 - [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) 传统基于 IAM 或 SCP 的权限控制是静态且二元的，只能判定 API 调用的有无，无法表达基于上下文、数据敏感度与接收者层级的动态风险逻辑。 - [07:08](https://www.youtube.com/watch?v=8uJez6QAYRk&t=428s) 自主智能体打破了治理、安全与网络防护的传统界限，必须依靠包含身份、策略、注册表、可观测性与评估在内的统一治理底座（如 Amazon Bedrock Agent Core）进行协同管控。 - [11:02](https://www.youtube.com/watch?v=8uJez6QAYRk&t=662s) 针对多步跨会话级联调用的复合风险，需在智能体推理循环之外的网关层强制执行时序策略（Temporal Policies），并依托专用开源策略语言 Dogwood 阻断绕过攻击。 - [11:02](https://www.youtube.com/watch?v=8uJez6QAYRk&t=662s) 神经符号 AI（Neurosymbolic AI）将大语言模型的自然语言理解与自动推理（Automated Reasoning）的数学严谨性结合，实现从有限采样测试到数学全覆盖形式化证明的跨越。 - [13:09](https://www.youtube.com/watch?v=8uJez6QAYRk&t=789s) 当企业级系统出现数十个智能体与海量工具连接时，组合路径爆炸导致单点合规操作可能产生级联越权泄露，必须依托形式化证明穷尽验证所有未测试路径。 - [14:48](https://www.youtube.com/watch?v=8uJez6QAYRk&t=888s) 用于智能体治理的自动化推理技术并非学术玩具，而是已在 AWS 核心底层基础设施（如 IAM Access Analyzer、S3 Block Public Access、VPC Reachability Analyzer）实战检验十余年的成熟工程能力。 - [15:24](https://www.youtube.com/watch?v=8uJez6QAYRk&t=924s) 形式化验证前置介入软件规格（Spec）设计阶段，在代码生成前消除定义歧义，并驱动基于属性的测试（Property-based Testing）自动生成全面探针。 - [16:27](https://www.youtube.com/watch?v=8uJez6QAYRk&t=987s) 终结速度与安全信任的零和博弈：通过数学证明构建坚不可摧的安全边界，让智能体在封闭边界内免人工干预极速运行，仅在高危场景精准触发严密审查。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) 智能体落地的信任困境与历史重演演讲伊始，Erin Kraemer 向现场观众提出了一个互动问题：“今天在座的各位中，有多少人已经在生产环境中部署了 AI 智能体（Agents）？”台下举起了相当数量的手。随后她紧接着问：“如果能自信地说出这些智能体在过去 24 小时内所做的每一次操作、触碰的每一条数据以及做出的每一个决策，请继续举着手。”此时全场的手几乎全部放下，只有前排的一位观众依然举着。这精准地反映了当前 Agentic AI 的核心痛点：构建智能体的速度远远快于人们对让其自主运行所建立的信心。这种技术演进的模式在历史上曾反复上演。Kraemer 回忆起 25 年前刚加入亚马逊时的经历：工程师们常常要在深夜值班守着发布流程，肉眼盯着系统部署上线以确保没有东西被搞崩溃。随着时间推移，团队构建了金丝雀发布（Canaries）、自动回滚和自动化告警等机制，使得系统能比人工更快地捕捉异常，工程师们这才获得了放手睡觉的底气。过去那套机制之所以奏效，是因为部署过程是确定性的（Deterministic），即使在亚马逊海量业务规模下极其复杂，但其步骤已知、系统已知且边界可控。然而，智能体完全不同：它们实时自主选择工具并进行判断。面对执行单一窄任务的少量智能体，团队尚能人工盯防；但随着智能体能力越来越强、数量越来越多，核心问题已从“它能否工作”转变为“当无人值守时，我能否证明它绝不越界？” ```mermaid flowchart TD accTitle: 软件交付与 AI 智能体确定性对比 subgraph Traditional["传统软件部署 (确定性)"] direction TB A["已知系统与固定步骤"] --> B["金丝雀与自动回滚机制"] B --> C["事前与事中确定性保障"] end subgraph Agentic["AI 智能体系统 (概率性与动态性)"] direction TB D["自主判断与动态调用工具"] --> E["上下文复杂且行为不可预测"] E --> F["无法单纯依赖人工盯防"] end ``` - 跨行业领导者（金融服务、医疗保健、制造业与政府部门）在过去一年中普遍反馈三大痛点： - 已经部署了智能体，但实际上根本不知道它们在具体做什么； - 每个团队各自构建防御护栏（Guardrails），彼此标准极不一致； - 项目长期停滞在试点（Pilot）阶段，因为无法向安全团队甚至向自己证明系统具备上线安全性。 - 传统的“事后检测”已无法支撑规模化信任，企业亟需能够在事前提供数学证明的边界约束能力。 --- ### [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) 扁平化授权 vs. 动态风险：传统权限控制的局限为了直观展现传统权限控制在智能体面前的失效，Kraemer 举了一个给智能体赋予邮件发送权限的简单例子。通常情况下，只需连接邮件工具并配置基础策略（如避免反垄断竞争言论、严禁对外泄露客户数据等）。如果智能体只是向平级同事或下属发送关于日常事务的邮件，低风险场景下做个粗略浏览即可放行；但如果收件人是副总裁（VP）甚至首席执行官（CEO），整个操作的风险便急剧飙升——此时必须要求智能体核实每个数据来源（Grounding）、对不确定的信息打标预警，并且在正式发送前必须交由人工复核批准。然而，传统的授权体系（如 IAM 策略、服务控制策略 SCP、数据权限）本质上是静态且二元的，只能判定“该智能体是否有权发送邮件：Yes or No”，完全无法表达“仅在满足特定条件、经过特定步骤时，允许向特定对象发送邮件”的细粒度上下文逻辑。“权限是扁平的，但风险从来不是扁平的。”这使得工程团队陷入两难抉择：要么过度限制智能体，要求所有动作都必须人工审批，从而彻底失去自动化价值；要么赋予完全不受限的发送权限，仅凭运气祈祷大模型做出正确决策。 | 行业场景 | 智能体动作 | 常见低风险操作（平级/常规） | 高风险触发操作（需额外校验与审批） | |:--- |:--- |:--- |:--- | | **通用办公** | 发送电子邮件 | 给日常业务对接的平级同事或下属发邮件 | 给 VP 或 CEO 发送包含关键业务数据的邮件 | | **医疗健康** | 提交报销与理赔 | 常规医疗费用报销处理 | 涉及重大医疗决策的临床例外审批（Clinical Override） | | **金融服务** | 执行证券交易 | 500 美元的小额常规交易 | 50,000 美元的大额交易审核流 | | **工业制造** | 供应链零部件采购 | 常规消耗性零部件例行补货 | 六位数金额（数十万美元级）的资本设备采购 | --- ### [07:08](https://www.youtube.com/watch?v=8uJez6QAYRk&t=428s) 统一治理底座：Amazon Bedrock Agent Core 的五大支柱智能体模糊了治理（Governance）、安全（Safety）和网络安全防护（Security）原本由不同团队割裂管理的界限，单一行为可能同时引发这三类风险。为了在动作发生的瞬间结合完整上下文做出动态判定，必须建立一个统一的治理底座。Kraemer 系统阐述了实现自主智能体治理所需的五大核心能力： ```mermaid flowchart LR accTitle: 智能体统一治理底座的五大支柱 subgraph GovernanceFoundation["Amazon Bedrock Agent Core 统一治理架构"] direction TB A["1. 身份 (Identity) 最小权限与委托凭证"] B["2. 策略 (Policy) 确定性脱离推理环执行"] C["3. 注册表 (Registry) 资产集中发现与编目"] D["4. 可观测性 (Observability) 全链路追踪与回放"] E["5. 评估 (Evaluation) 前置仿真与线上持续监测"] end ``` - **身份（Identity）**：智能体是独立的执行实体，必须具备独立的机器身份、作用域权限（Scoped Permissions）、委托凭据与最小权限原则，以便明确指出其作为委托助手的具体授权范围。 - **策略（Policy）**：超越“能否调用该 API”的传统结构化判定，能够深入识别动作上下文（收件人是谁、涉及什么数据、潜在代价有多大），并且该策略检查必须**确定性地运行在智能体推理循环之外**，防止大模型通过 Prompt 绕过约束。 - **注册表（Registry）**：当企业智能体数量从 5 个暴增到 500 个时，注册表提供统一发布、编目和查找所有智能体、工具及资源的中心节点。 - **可观测性（Observability）**：统一截获模型调用、工具执行、路由决策与策略判定，支持完整决策链的链路追踪（Trace）、回溯重放（Replay）与合规审计。 - **评估（Evaluation）**：在上线前针对对抗性输入、极端边缘案例和合规场景进行模拟评测；上线后持续监测模型升级、长期记忆上下文累积及工具变更带来的行为漂移（Drift），形成闭环。 --- ### [11:02](https://www.youtube.com/watch?v=8uJez6QAYRk&t=662s) 时序策略、Dogwood 语言与神经符号 AI 针对跨步骤的复合型风险，Kraemer 正式发布了 Agent Core 的全新能力——**时序策略（Temporal Policies）**，并开源了专为智能体生命周期治理设计的策略语言 **Dogwood**。很多安全风险无法在单一离散动作中暴露，只有将跨会话的多步连续动作串联起来时才会显现。时序策略部署在智能体边界外的网关层强制执行，无论智能体被注入何种 Prompt 或调用何种工具，都无法突破这一硬性防线。 ```mermaid flowchart TD accTitle: 神经符号 AI 架构示意图 subgraph NeurosymbolicAI["神经符号 AI (Neurosymbolic AI)"] direction LR subgraph Probabilistic["大语言模型 LLM (概率层)"] LLM["自然语言理解 灵活推理 处理复杂模糊上下文"] end subgraph Deterministic["自动推理 Automated Reasoning (符号层)"] AR["数学逻辑严谨性 全空间形式化验证 提供不可违背的 Provable 保证"] end end LLM <--> AR ``` 在探讨如何构建绝对信任时，Kraemer 指出了传统测试和检测的局限：即便配置了针对高管邮件的上下文策略并顺利通过了 100 种测试场景，也只能说明“我们尚未见过它违规”，而无法得出“它在数学上绝不可能违规”的结论。为了消除“未曾发生”与“绝不可能发生”之间的鸿沟，**神经符号 AI（Neurosymbolic AI）**将大语言模型（LLM）的创造力、自然语言理解力与自动推理（Automated Reasoning）的数学严谨性深度结合：由 LLM 负责处理非结构化的复杂语境，由自动推理在底层提供数学上可证明的确定性安全保障。 - **Dogwood 开源策略语言**：专为智能体多步会话和时序动作治理打造，在网关层脱离模型推理循环独立生效。 - **核心理念跃迁**：从“基于采样的有限测试（Testing）”迈向“数学逻辑上的全覆盖可证明性（Provable Guarantees）”。 ### [13:09](https://www.youtube.com/watch?v=8uJez6QAYRk&t=789s) 复杂度爆发与形式化证明：从测试已知到证明未知当系统的规模扩大时，人类的直觉管理便会彻底失效。如果只有一个智能体、一款工具、几类收件人层级和几项简单条件，人类大脑尚能梳理清楚所有的交互逻辑；但若将场景扩展到企业级规模——50 个智能体各自连接数十种工具、涉及不同的业务角色、数据敏感度层级以及流向各异的输出渠道，政策之间可能产生的路径组合将瞬间飙升至数千条。此时，单凭人力审查根本无法完全掌控全局。系统内部极易出现隐蔽的动作调用链：链条中的每一个单点动作看似都符合局部权限，但级联叠加后却会导致未经审批的数据泄露或越权行为。这正是时态策略（Temporal Policy）与形式化证明发挥决定性作用的时刻。常规测试手段只能覆盖工程师预先设想到的路径，而数学证明能够穷尽所有可能——包括被测试过的场景，以及数以百万计未被测试到的盲区。借助自动化推理（Automated Reasoning），系统能够从数学上严格确证一条断言："在当前条件下，任何角色下的任何智能体，都不存在允许敏感数据在未经所需授权许可的情况下流入目标渠道的授权路径。"（"Under this condition, no agent under any role has an authorization path that lets sensitive data reach a channel without the required clearance."）这种形式化约束在智能体处理任何真实请求之前，就已对所有角色、数据分类和路由组合完成了确定性推导。同样，这一机制不仅作用于动作执行端，也深入到输出生成端——通过 Amazon Bedrock Guardrails 中的自动化推理检查，系统可以严密证明大语言模型的每一次回复是否完全契合预设的安全与合规政策，在动作执行与内容输出两侧实现同等严格的数学级保障。 ```mermaid flowchart TD accTitle: Testing vs Formal Proof Comparison accDescription: Comparison showing how traditional testing only checks known paths while automated reasoning proves boundaries across all paths. subgraph Trad["传统测试方法（概率性验证）"] T1["人工预想场景 / 单元测试"] --> T2["覆盖有限路径"] T2 --> T3["盲区与级联调用风险遗留"] end subgraph Form["形式化证明与自动化推理（确定性验证）"] F1["数学逻辑形式化建模"] --> F2["穷尽数百万种边界与路径组合"] F2 --> F3["证明不存在越权/泄露路径"] F3 --> F4["智能体在既定安全边界内自主运行"] end ``` --- ### [14:48](https://www.youtube.com/watch?v=8uJez6QAYRk&t=888s) 十年云基础设施沉淀：自动化推理在智能体中的工程化演进这项技术并非停留在学术象牙塔中的实验原型，而是已经在 AWS 底层核心基础设施中实战检验了十余年的成熟能力。在底层云平台中，每天处理数十亿次查询的关键安全功能——例如用于检测非预期访问权限的 IAM Access Analyzer、防止数据意外公开的 S3 Block Public Access，以及用于验证复杂网络拓扑可达性的 VPC Reachability Analyzer——背后全部依托自动化推理技术来证明策略正确性与网络路径连通性。当前的重大转变并非底层数学证明技术本身，而是 AWS 正在将这一套经过十年大规模工业级验证的确定性推理能力，完整迁移并应用到 AI 智能体的治理与执行控制当中。 - **IAM Access Analyzer**：通过形式化逻辑证明资源策略是否会引发未授权的外部跨账户访问。 - **S3 Block Public Access**：在存储桶级别通过数学逻辑确保静态与动态配置绝不向公网暴露数据。 - **VPC Reachability Analyzer**：在无需发送实际数据包的情况下，形式化推导网络配置并证明路径连通性与隔离性。 --- ### [15:24](https://www.youtube.com/watch?v=8uJez6QAYRk&t=924s) 编码前的规范校验与基于属性的测试：走向确定性证明形式化验证的介入节点并不局限于运行期（Runtime），而是在开发者写下第一行代码之前就已启动。在应用与智能体开发环境（如 Qira / 前期规格设计环境）中，神经符号（Neurosymbolic）技术被直接引入到软件规格说明书（Spec）的静态检查中，在代码生成前识别逻辑矛盾、覆盖漏洞与语义歧义。例如，当规格说明中写着"智能体在发送高风险邮件时需要审批"时，系统会立即识别出"高风险"这一描述的模糊性：它究竟是指收件人的职级头脑、邮件内容的敏感度，还是二者兼有？系统会在模糊定义固化为具体代码之前强制开发者作出明确裁决。一旦规格明确并生成代码，系统会基于这些严格规格自动派生出基于属性的测试（Property-based Testing），对数百种场景进行自动化探针探测。正如演讲中所强调的核心认知跃迁："测试告诉你，我们检查了这些用例，没有发生崩溃；而证明则告诉你，对于每一个可能的场景（包括我们从未想过要检查的场景），该属性都恒成立。"（"Testing tells you we check these cases and nothing broke. Proof tells you this holds for every case, including the ones that we never thought to check."）这就是从概率性（Probabilistic）向可证明性（Provable）的根本跨越，且可证明的保障边界仍在持续扩大。 --- ### [16:27](https://www.youtube.com/watch?v=8uJez6QAYRk&t=987s) 终结速度与信任的博弈：让治理成为智能体放手奔跑的底气长期以来，企业在部署 AI 时往往被迫在"创新速度"与"安全信任"之间做零和博弈：要么堆叠繁琐的人工审批流程，将 AI 带来的提速优势完全抹杀；要么为了抢进度而带着无法度量和控制的风险仓促上线，提心吊胆地祈祷警报电话不会在半夜响起。将治理、构建与形式化证明统一起来，为组织提供了破局的"第三种选择"：划定绝对安全的边界，通过数学证明确保边界坚不可摧，然后让智能体在边界内部完全自由地奔跑。开发者不再是依靠侥幸心理"希望"智能体遵守规矩，而是已经从数学上"证明"了它必定会遵守规矩。 ```mermaid flowchart LR accTitle: Unified Governance Paradigm accDescription: Paradigm showing how formal proof enables autonomous agent execution. subgraph Boundary["1. 形式化定义与证明"] B1["定义策略与安全边界"] --> B2["自动化推理证明边界封闭"] end subgraph Autonomy["2. 边界内完全自主"] B2 --> A1["常规任务：免人工干预极速执行"] B2 --> A2["高危操作：精准拦截并触发严密审查"] end ``` 再次回到邮件智能体的案例：最佳实践绝不是为每一封外发邮件都套上人工审批的枷锁从而废掉智能体，而是证明策略能够在正确的条件下、针对正确的收件人精准执行正确的审批步骤。智能体能够自主高效地处理日常琐碎事务，而真正涉及高风险的操作则必然受到应有的严密审查。真正的信任源于严谨的证明，而非对智能体能力的盲目阉割。过去，云工程师不得不通宵守着软件部署发布，直到构建出完善的自动化与防护机制才敢放手安心入睡；如今智能体领域正在经历完全相同的演进历程。底层证明机制与架构已经就绪，治理不再是拖慢智能体步伐的绊脚石，反而是赋予组织信心、让智能体真正释放潜能、并让工程师夜夜安枕的核心基石。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) 今天在座的各位中，有多少人已经在生产环境中部署了 AI 智能体（Agents）？ — Erin Kraemer > [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) 如果能自信地说出这些智能体在过去 24 小时内所做的每一次操作、触碰的每一条数据以及做出的每一个决策，请继续举着手。 — Erin Kraemer > [00:04](https://www.youtube.com/watch?v=8uJez6QAYRk&t=4s) 当无人值守时，我能否证明它绝不越界？ — Erin Kraemer > [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) 权限是扁平的，但风险从来不是扁平的。 — Erin Kraemer > [03:34](https://www.youtube.com/watch?v=8uJez6QAYRk&t=214s) 仅在满足特定条件、经过特定步骤时，允许向特定对象发送邮件 — Erin Kraemer > [13:09](https://www.youtube.com/watch?v=8uJez6QAYRk&t=789s) Under this condition, no agent under any role has an authorization path that lets sensitive data reach a channel without the required clearance. — Erin Kraemer > [15:24](https://www.youtube.com/watch?v=8uJez6QAYRk&t=924s) 智能体在发送高风险邮件时需要审批 — Erin Kraemer > [15:24](https://www.youtube.com/watch?v=8uJez6QAYRk&t=924s) Testing tells you we check these cases and nothing broke. Proof tells you this holds for every case, including the ones that we never thought to check. — Erin Kraemer > [16:27](https://www.youtube.com/watch?v=8uJez6QAYRk&t=987s) 第三种选择 — Erin Kraemer ## Mentioned Erin Kraemer, AWS, Ai4, Amazon Bedrock, Amazon Bedrock Agent Core, Amazon Bedrock Guardrails, Dogwood, IAM Access Analyzer, S3 Block Public Access, VPC Reachability Analyzer, Qira ## Video description Erin Kraemer - Sr. Principal Technical Product Manager, AWS As AI agents gain autonomy, governance becomes the central challenge: how do you give agents enough freedom to be useful while ensuring they stay within the right boundaries? Too much constraint defeats the purpose; too little creates unacceptable risk. Most approaches address governance, safety, and security as separate disciplines. They aren't. They require one enforcement layer that can observe every action, enforce constraints deterministically, and intervene when something goes wrong. This talk covers what it takes to govern autonomous agents at production scale and how neurosymbolic AI raises the bar, pairing the creative power of LLMs with the rigor of mathematical logic to deliver provable guarantees about agent behavior. The result: organizations can confidently move faster because correctness is proven, not assumed [Source](https://www.youtube.com/watch?v=8uJez6QAYRk)

Source quotes

[0:04] Erin Kraemer: 今天在座的各位中，有多少人已经在生产环境中部署了 AI 智能体（Agents）？ [0:04] Erin Kraemer: 如果能自信地说出这些智能体在过去 24 小时内所做的每一次操作、触碰的每一条数据以及做出的每一个决策，请继续举着手。 [0:04] Erin Kraemer: 当无人值守时，我能否证明它绝不越界？ [3:34] Erin Kraemer: 权限是扁平的，但风险从来不是扁平的。 [3:34] Erin Kraemer: 仅在满足特定条件、经过特定步骤时，允许向特定对象发送邮件 [13:09] Erin Kraemer: Under this condition, no agent under any role has an authorization path that lets sensitive data reach a channel without the required clearance. [15:24] Erin Kraemer: 智能体在发送高风险邮件时需要审批 [15:24] Erin Kraemer: Testing tells you we check these cases and nothing broke. Proof tells you this holds for every case, including the ones that we never thought to check. [16:27] Erin Kraemer: 第三种选择

Source quotes

Erin Kraemer: Too much constraint defeats the purpose; too little creates unacceptable risk. Erin Kraemer: Most approaches address governance, safety, and security as separate disciplines. They aren't. Erin Kraemer: organizations can confidently move faster because correctness is proven, not assumed [0:04] Erin Kraemer: can I prove that it stays in bounds when I'm not watching? [3:34] Erin Kraemer: if that email goes wrong, I look really stupid. [3:34] Erin Kraemer: trust and hope [13:09] Erin Kraemer: Under this condition, no agent under any role has an authorization path that lets sensitive data reach a channel without the required clearance. [14:48] Erin Kraemer: the agent requires approval for high-stakes emails [16:27] Erin Kraemer: hope that that phone stays quiet.

Original transcript

[0:04] So I'm absolutely delighted to be with you here today. So I wanted to start with a question and [0:10] this is interactive. How many of you have agents in production today? Raise your hands. All right. [0:19] Fair number of you. Okay. Now I want to keep your hand raised. If you can tell me confidently what [0:27] those agents did in the last 24 hours, every action, every piece of data, every decision [0:34] that they touch, there are very, very few. I see a friend up front who pulled it off. Good job. So, [0:42] this is the state of agentic AI right now. We can build agents faster that we can have [0:49] the confidence to let them run. For those of you who have been in technology for a while, [0:54] we've seen this pattern before. So when I started at Amazon a quarter of a century ago, [1:01] we used to stay up late at night to babysit the builds, basically watching the deployments go [1:06] online. And we would basically try to verify that nothing broke. Over time, [1:13] we built mechanisms so that we didn't have to do that. things like canaries, automatic rollbacks, [1:18] alarms that could catch problems faster than we ever could. We earned the confidence to [1:24] actually step back and sleep at night because we could prove that the system would catch what we [1:31] wouldn't. And that worked a little bit because deployments were deterministic, although at the [1:36] Amazon website scale very complexely so, but they were known steps, known systems, [1:43] and boundaries that we could draw on. enforce. But agents are different. And the reason is they pick [1:50] tools on the fly. They exercise judgment. And a few agents that are working on a few neuro tasks, [1:57] you can watch those. But as they get more capable and as you run more of them, the question changes. [2:06] It stops being does they work becomes, can I prove that they stay in bounds when I'm not watching? [2:13] And most organizations can't answer that question yet. And this isn't just you. I've spent the past [2:21] year talking to leaders deploying agents across all sorts of industries, financial services, [2:28] healthcare, manufacturing, and government. And regardless of the industry, I keep hearing the [2:34] same three things. We deployed agents, but we don't actually know what they're doing. [2:43] Every team built their own guardrails and none of them are consistent and we're stuck in pilot [2:50] because we can't prove to our security team much less ourselves that this is safe to ship. [2:58] So the ability to prove and to govern agents and verify they're doing what they're doing proving [3:05] that they stay within bounds that hasn't kept at pace. And the wider the gap gets the more it [3:11] becomes the thing that's holding everything else back. And closing that is going to take something [3:17] that most governance can't do yet. mathematical proof upfront that a boundary cannot be crossed [3:25] to trust these agents at scale. We need proof and not just detection after the fact. So let me make [3:34] this concrete with something fairly simple. So say I want to give my agent permission to send [3:39] email. I connect it to my email tool and now it can read my inbox. It can draft messages and it [3:45] can send them. And that's easy enough to set up with the permissions that I've always used. [3:51] And I've also got basic standing policies, things like avoid anti-competitive language, [3:57] don't share customer data externally, and you know, things that are pretty straightforward [4:03] guardrails. If that agent is emailing one of my direct reports or a peer that I talk to every day, [4:12] I'll do a quick once over and I'll let it sit. The stakes are pretty low. I trust his judgment. [4:17] I'm debating routine stuff. But then it starts to get tricky. What if it's going to my VP or to [4:25] my CEO? Now, I want the agent to do more work. I wanted to ground every number. I want to check it [4:32] against the source. I want to flag anything it's unsure about. And then it must bring it back to me [4:38] before anything sends because if that email goes wrong, I look really stupid, right? Same agent, [4:46] same permissions, same tool. The only thing that changed is the context of who's on the other end. [4:55] All right, so here's my problem. My authorization system only sees one thing. Is this agent allowed [5:01] to send email? Yes or no? It has no way to say allowed to send email to this person under these [5:08] conditions with these extra steps. The permission is flat, right? The risk is not flat. So, what do [5:15] I do? I either lock the agent down so tight that it can't send anything without my approvals, [5:22] which basically defeats the point of having it, or I basically hand it blanket send access or maybe [5:29] allow allow allow allow allow and hope it makes the great call on its own. And that is basically [5:37] the trade-off that most teams are stuck in, right? Throttle it until it's useless or trust and hope. [5:43] And the tools that we've got right now, they are not built to deal with this. IM policies, [5:48] service control policies, data permissions, they answer that binary question of permitted [5:53] or denied. None of them can weigh that kind of complex context at the moment that the action is [6:00] happening. And this patterns everywhere you look. In healthcare, an agent that files claims, but a [6:09] routine reimbursement and a clinical override are not the same event. In financial services, [6:15] an agent that can execute trades, but a $500 trade and a $50,000 trade should not clear the same way. [6:23] And in manufacturing, an agent that places orders, but a routine parts reorder and a six-f figureure [6:31] capital purchase should not take the same approval paths. So our permissions they stay fixed but the [6:39] governance that needs to move with context and your existing controls can't express that. So [6:46] if you ask your IM policy can this agent send email and it says yes. What it doesn't tell you [6:52] is whether this agent should send this email to this person right now with nothing else required. [6:58] And that's the gap, right? Every time you give an agent a new capability, that gap gets wider. [7:08] So answering that question takes a governance foundation that reads the full context of every [7:14] action. Governance, safety, and security used to be three separate problems owned by three separate [7:20] teams. Agents, they collapse these all together. A single action can raise all three of these risks [7:29] at once. So you need one foundation that can handle all of them. So the familiar building [7:35] blocks like observability, policy enforcement, and identity management, they need to be adapted [7:41] for agents that reason and act on their own. So let's walk through what that takes. So first off, [7:47] identity. Before a policy can weigh context, you have to know who's acting. Agents need their own [7:55] identities. They need scope permissions. They need delegated credentials. They need lease privilege. [8:00] They're actors in their own right and they need to be governed that way. So, back to my email [8:05] example. Identity is what tells the governance layer, this agent is acting as my assistant under [8:11] my delegation in these specific scopes. Policy. Now, this is the one I think matters most. So old [8:21] style policy answers structural questions like can this agent call this API? Yes or no. But you [8:27] need more than that. You need policy that reads an action in context. Who's the recipient? What data [8:34] is in play? What's at stake? In the email example, policy is the thing that says executive emails [8:40] require grounding across check and approvals before they can send. And that check runs [8:46] deterministically outside the agent's reasoning loop. So the agent can't talk its way around it. [8:56] Next up, registries. You cannot govern what you can't see. And when you go [9:00] from five agents to 500 and they spread across every team, without a registry, [9:06] you're trying to enforce standards on agents you don't even know exist. An agent register gives [9:13] you one place to publish to curate and find every agent tool and resource across the organization [9:23] observability. So by routing an agent's actions through one governance layer, [9:29] you can finally see all of it. Every model call, every tool invocation, every routing decision, [9:35] every policy check. Did that email agent actually ground the data and cross-check the numbers before [9:42] it emailed the CEO? With real observability, you could trace it, you can replay it, and you [9:48] can audit the whole decision chain. And finally, evaluation. You can test your agents in simulation [9:57] first against adversarial inputs, edge cases, compliance scenarios before these ever touch real [10:04] users. But evaluations keep going after launch because agents drift as models update, memory and [10:12] context accumulates and tools change. You get to watch live behavior and crouch regressions as they [10:18] show up. It's a loop that you run continuously, not a gate that you clear once. And this is what [10:25] Amazon Bedrock Agent Core is bringing together. Identity, policy, registry, observability, [10:32] and evaluations. So instead of every team bolting together your own controls, you get one foundation [10:38] and the same standards apply no matter which team built the agent. So when you unify governance, you [10:45] can finally answer the question your old controls couldn't. Should this agent take this action in [10:51] this context right now under these conditions? And you don't have to wire all of this together [10:56] yourself. Agent core harness handles that wiring so that you have a gudge governed agent running in [11:02] minutes. Good news. And today I'm really excited to announce that we're adding new capabilities to [11:08] agent core that govern what an agent does across the whole session. So temporal policies weigh what [11:16] an agent has done already before allowing them to move on to the next step. So a policy can [11:23] catch what no single action reveals. where every action in it discreetly might be fine but you've [11:30] got a problem when all of them come together. So this is enforced at the gateway outside the [11:36] agent's boundary. So an action regardless of how the agent is prompted what tools it used the agent [11:44] can't cross that boundary. So this is written in a language called dogwood. This is a new open-source [11:50] policy language that we're releasing today built for governing agents over time. So even with full [11:59] observability, even when you can see everything an agent is doing and why it did it, there's still [12:06] one question detection alone can't answer. So take that contextual policy again. Executive emails [12:12] require grounding, a cross check, and approval. You can enforce it at runtime. You can test [12:18] it against a hundred scenarios and watch it hold every time and that is absolutely worth something. [12:25] But can you prove this for every case, every recipient, every possible email, every sequence [12:31] of a actions that the agent might string together, including the ones that never occurred to you to [12:36] test. There is a world of difference between we haven't seen it happen and it can't happen. [12:47] So that's the difference of what neurosymbolic AI does. It pairs the creative power of LLMs with a [12:55] mathematical rigor of automated reasoning. The LLM does what it's great at. Natural language, [13:02] flexibility, reasoning across messy context. Automated reasoning does what testing can't. [13:09] It uses mathematical logic to prove that a property holds across every possible scenario, [13:16] including the ones that you tested, but also the millions that you didn't. So, [13:22] let's go back to the email example. One agent, one tool, a few recipient tiers, [13:26] a couple of conditions. You can hold all that in your head, but now multiply it. 50 agents each [13:33] word to a dozen tools, different roles, different data, different sensitivity levels. the outputs [13:39] all routing to different channels. So the number of possible paths through your policies, they run [13:44] into the thousands and now no one can hold that in their head. Somewhere there's a chain of actions, [13:53] each one that's allowed on its own that adds up to an exposure that no one signed off on. And that's [13:59] where temporal policy comes into play. So testing, it covers the past that you thought of. Proof, it [14:06] covers all of them. the ones that you thought of and the ones that you missed. Automated reasoning [14:11] can prove a statement like this. Under this condition, no agent under any role has author an [14:17] authorization path that lets sensitive data reach a channel without the required clearance. Every [14:23] role, every category, every route accommodation settled before an agent handles a single request. [14:30] And it works on the output side too. All auto innovated reasoning checks in Amazon better [14:35] guard guardrails prove whether a response lines up with the policies that you defined. So this is the [14:41] same rigor both on actions the agents take and the answers that they give and none of this is [14:48] a lab experiment. So this technology has been securing AWS infrastructure for over a decade. [14:55] IM access analyzer, S3 block public access, VPC reachability. They used to do things that handle [15:02] billions of queries every day. Doing things like proving whether a policy allows unintended access [15:09] or determining whether or not certain network paths can be reached. So for us, what's changed is [15:16] not the technology, but the fact that we're build bringing this capability to agents. And to add to [15:24] that, the proof just doesn't happen at runtime. It can actually start before you've written a line of [15:30] code. So, Kira, our development environment for building applications, agentic ones included, [15:37] uses the same neuro symbolic techniques to check that your specs are right before a single line of [15:44] code gets generated. It catches contradictions, gaps, and ambiguities. So, say the spec reads, [15:50] "The agent requires approval for high stakes emails. What counts as high stakes, right? The [15:56] seniority of the recipient, the content, both." Cira flags that and makes you decide before the [16:03] ambiguity hardens into code. Then once the code exists, it generates property based [16:09] tests from those same specs, probing hundreds of use cases. Testing tells you we check these [16:15] cases and nothing broke. Proof tells you this holds for every case, including the ones that [16:21] we never thought to check. And that is the shift from probabilistic to provable. And the surface [16:27] area of what we can prove is only growing. So what does this mean for you? You stop choosing between [16:34] speed and trust. Today, most organizations settle that fight by starving one side or another. They [16:40] pile on human reviews. They kill even this all the speed that they got from deploying AI. Or [16:47] they ship with risk that they can't measure or control and hope that that phone stays quiet. [16:53] When you unify governance and build and proof, you get a third option. You draw the boundary, [16:59] prove it holds, and then let the agent run free inside of it. You're not hoping that it's going [17:04] to behave. You've proven that it will. Yeah. So, back to my email example one last time. [17:10] You don't hobble it with an approval on every message. You prove that the policy enforces the [17:16] right steps to the right recipients under the right conditions. The agent stays useful for [17:22] the routine work and you stay confident that the high stakes actions get the scrutiny that they [17:27] deserve. The trust comes from the proof, not from limiting the agent. So remember where I started, [17:35] we used to babysit deployments when we until we built mechanisms that let us actually step away [17:41] and sleep at night. The same thing is happening for our agents. The mechanisms are built. The [17:47] architecture is designed to prove. The only open question is whether your governance is [17:52] ready to take that step. So many of us probably walked in here thinking that governance is what [17:58] slows an agent down. But what I want to assert is that governance is actually what's lets you [18:04] let them loose. It's what earns you the confidence to give them more power. When you build governance [18:11] that keeps pace with what your agents do, that lets you sleep at night. Thank you so much.