Retool Advocate Unveils Multi-Agent AI Framework to Curb Project Shutdowns

Who's Watching the Agents? A Practical Framework for Multi-Agent Governance With Retool, Inc. — Ai4

Ai4

Centralized governance platforms with built-in permission scoping and human approval gates prevent the security failures that Gartner predicts will shut down 40% of AI projects by 2027.

AI-assisted summary of Who's Watching the Agents? A Practical Framework for Multi-Agent Governance With Retool, Inc. — original source. Check the original for full context.

Source published: 2026-08-15

Detailed notes

## Background Published by Ai4 on YouTube on August 15, 2026, this presentation features Angelik Laboy Torres, Senior Developer Advocate at Retool, Inc. According to the page description, the talk addresses the growing governance gap in enterprise multi-agent systems, exploring common production failure modes and demonstrating how to build a governed workflow with strict permissions and audit trails.

## TL;DR Angelik Laboy Torres warns that 40% of AI projects are projected to be shut down by 2027 due to legal and security governance failures as teams expand from single agents to complex fleets. She outlines three primary production failure modes: Permission Bleed, the Black Box Problem, and Escalation Gaps. To address these issues, she presents a governed multi-agent architecture using an on-call triage demo where specialized agents operate under strict domain scoping and human-in-the-loop approval gates. Finally, she provides a 5-point governance checklist to help teams ensure data boundaries, unified audit trails, and inherited enterprise permissions from day one.

## Atmosphere & style The presentation is energetic, pragmatic, and highly engaging, opening with an interactive show of hands that immediately exposes the audience's underlying anxiety about agent governance. Torres delivers her talk with clear developer-advocate authority, balancing alarming industry warnings with structured, actionable solutions. The tone transitions seamlessly from an urgent wake-up call about non-linear agent complexity to a clear, technical walkthrough of a live solution.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) Angelik Laboy Torres highlights a severe governance gap in production AI agents between expected behavior and actual actions. - [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) Gartner predicts 40% of AI projects will be shut down by the end of 2027 due to security and legal interventions. - [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) Transitioning from a single agent to a multi-agent fleet shifts the deployment from a simple experiment to a major corporate liability. - [01:55](https://www.youtube.com/watch?v=_VtrDFPjDag&t=115s) Unmanaged agent sprawl across business units creates ungovernable messes without clear cross-agent accountability. - [01:55](https://www.youtube.com/watch?v=_VtrDFPjDag&t=115s) System complexity and interaction relationships grow non-linearly as the number of specialized agents increases. - [01:55](https://www.youtube.com/watch?v=_VtrDFPjDag&t=115s) Core multi-agent capabilities like delegation, autonomy, and specialization inherently increase security risks and attack surface area. - [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) Permission Bleed occurs when agents delegate tasks and secondary agents inherit unintended resource access. - [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) The Black Box Problem makes auditability difficult when teams rely on fragmented raw logs instead of a unified record. - [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) Escalation Gaps leave systems vulnerable when human-in-the-loop pause mechanisms are not programmatically built for high-stakes actions. - [07:24](https://www.youtube.com/watch?v=_VtrDFPjDag&t=444s) A governed multi-agent architecture uses specialized sub-agents with strictly bounded domain permissions to prevent cross-contamination. - [07:24](https://www.youtube.com/watch?v=_VtrDFPjDag&t=444s) Implementing human-in-the-loop approval gates allows human operators to pause execution, grant temporary permissions, or approve sensitive actions. - [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) Centralized platforms ensure new agents automatically inherit existing enterprise RBAC and governance policies from day one. - [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) A 5-question governance checklist evaluates data boundaries, approval gates, audit trails, inherited RBAC, and day-one integration prior to production release.

## Detailed walkthrough Here is a detailed, faithful walkthrough based on the transcript provided:

[00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) The Multi-Agent Governance Gap and the Gartner Warning

The session opens with Angelik Laboy Torres, Senior Developer Advocate at Retool, conducting a quick show of hands with the audience. She asks who currently has AI agents deployed in production, and then asks those with hands raised to keep them up if they feel confident knowing what their agents are doing every second—specifically what data they are touching and whom they are talking to. Observing the immediate drop in hands, she highlights that this discrepancy represents a severe governance gap between what teams think is happening and what agents are actually doing in production.

To emphasize the stakes, she references a Gartner report predicting that by the end of 2027, 40% of all AI projects will be shut down—not because the underlying technology fails, but because internal security and legal teams will cut them loose. She cautions that while a single AI agent is like a "science experiment," a fleet of agents quickly becomes a major corporate liability. Most teams operate in a gray zone, hoping nobody asks the right questions so they can keep experimenting in isolation. To solve this, she outlines her agenda: introducing a 5-question framework, evaluating common production failure modes, and demonstrating a governed multi-agent architecture built to prevent these failures.

- Gartner prediction: 40% of AI projects shut down by the end of 2027 due to security and legal intervention. - The single vs. fleet dynamic: One agent is an experiment; a fleet is a security liability.

---

[01:55](https://www.youtube.com/watch?v=_VtrDFPjDag&t=115s) Agent Sprawl and the Math of Compounding Risk

Addressing the issue of "agent sprawl," Torres describes a typical enterprise scenario where individual business units independently deploy specialized agents: marketing creates one for content, finance builds another for triaging expenses, and ops runs autonomous workflows. While each initiative seems like a good idea on its own, no one sets out to build an "ungovernable mess"—it simply creates itself over time. This unmanaged proliferation leaves executive leadership unable to answer critical operational questions regarding data boundaries, system access, cross-agent interactions, and accountability when autonomous systems act.

She explains the fundamental difference between single-agent systems and multi-agent fleets. A single agent consists of "one brain with one tool" and a single set of permissions, making it easy to mentally model, trace, and assign accountability when failures occur. In contrast, multi-agent systems divide complex tasks among specialized agents (e.g., researching, checking, drafting, approving) managed by an orchestrator. However, the benefits of delegation, autonomy, and specialization directly create severe security vulnerabilities. Delegation translates to autonomous permission passing, autonomy means decisions occur without humans in the room, and specialization dramatically expands the attack surface area. Furthermore, system complexity grows non-linearly: two agents create 1 relationship, 5 agents create 10 relationships, and 10 agents create 45 potential interaction relationships—even before factoring in external databases, APIs, and third-party tools.

- Relationship formula/scaling: 2 agents = 1 connection; 5 agents = 10 connections; 10 agents = 45 connections. - Key security tension: Features that make multi-agent systems powerful (delegation, autonomy, specialization) are the exact features that make them dangerous to govern.

---

[05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) The Three Major Production Governance Failures

Torres identifies the three primary governance failure modes that repeatedly break multi-agent deployments in enterprise environments:

1. **Permission Bleed:** Occurs when an agent is granted access to a database for a specific task but subsequently delegates work to another agent. The secondary agent effectively inherits permissions to resources it was never explicitly signed off to access, causing permissions to spread "like a virus" across the network. 2. **The Black Box Problem:** Arises when auditors request the underlying reasoning behind an agent's action, and the team can only offer unstructured raw log files or fragmented reasoning chains scattered across multiple tools. Lacking a coherent chronological record of inputs, outputs, data viewed, and approvals, teams face an "archaeology problem" rather than a functional audit trail. 3. **Escalation Gaps:** Stems from failing to operationalize human-in-the-loop (HITL) safeguards. While teams agree that sensitive actions (such as high-value financial payments or modifying customer data) require human approval, they often fail to programmatically build the pause mechanism, leaving agents to either execute unauthorized actions or stall indefinitely.

- The 3 Failure Modes: Permission Bleed, Black Box Problem, and Escalation Gaps. - Audit reality: Disjointed log files fail compliance standards when reconstructing agent decisions.

---

[07:24](https://www.youtube.com/watch?v=_VtrDFPjDag&t=444s) Architectural Walkthrough: A Governed On-Call Workflow

To illustrate how to mitigate these failures, Torres presents a live demo of a governed multi-agent workflow designed for an on-call engineering use case—a scenario traditionally requiring an exhausted engineer to manually triage alerts across DataDog, PagerDuty, Slack, and Linear at 3:00 a.m. The system is split across four distinct agents: a top-level orchestrator that evaluates incoming alerts (such as a spike in a checkout API) and delegates tasks, alongside three specialized agents: a Coordinator, an Investigator, and a Communicator.

She details how this architecture directly solves the three main failure modes: - **Comprehensive Audit Trail (Fixing the Black Box):** Every tool call, input, output, and decision is timestamped and recorded centrally. Auditors can trace the orchestrator's explicit reasoning chain, including why specific sub-agents were activated and what outputs were generated, eliminating the need to dig through separate tool logs. - **Strict Permission Scoping (Fixing Permission Bleed):** Each agent is strictly restricted to its specific domain. The Investigator can read DataDog metrics and PagerDuty events but cannot touch Slack or Linear. Conversely, the Coordinator can post to Slack and file Linear tickets but cannot query DataDog. Permissions remain tightly bounded to prevent cross-contamination. - **Human-In-The-Loop Approval Gates (Fixing Escalation Gaps):** Within 30 seconds of an incident trigger, the system posts a summary to Slack containing its hypothesis, confidence score, and proposed remediation steps (e.g., rolling back a deployment or scaling resources). Crucially, high-stakes actions are offered as proposed recommendations rather than executed automatically. The platform allows human operators to interact with the agent, pause execution, grant temporary permissions, or approve actions directly.

- System components: Orchestrator agent plus three specialized sub-agents (Investigator, Coordinator, Communicator). - Integrated tools: DataDog, PagerDuty, Slack, and Linear. - Operational speed: Alert triage, initial hypothesis, and proposed action delivered to Slack within 30 seconds.

---

[12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) Centralized Platforms and the 5-Point Governance Checklist

Torres advocates for using centralized platforms rather than stitching together disparate, custom-built micro-frameworks for agents, workflows, permissions, and front-end apps. When security guardrails and RBAC permissions are managed centrally, every new agent automatically inherits existing organizational governance policies day one, transforming the internal security team from a project blocker into a key champion.

To conclude, she provides a 5-question governance checklist for teams to evaluate their multi-agent systems before deploying to production:

1. **Defined Data Boundaries:** Does every agent have explicitly defined data boundaries? 2. **Built Approval Gates:** Is there an actual human approval gate programmatically built for high-stakes actions? 3. **On-Demand Audit Trail:** Can you produce a complete, unified audit trail on demand? 4. **Inherited RBAC:** Do your agents inherit your existing enterprise permission systems, or are you maintaining a separate security model? 5. **Day-One Governance:** Is governance built into the system architecture from day one, or is it bolted on after the fact?

- Core recommendation: Use centralized platforms to ensure automatic policy inheritance and avoid governance gaps at integration seams. - Strategic takeaway: Answering "yes" to all five checklist items aligns development with security requirements, preventing project cancellation.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) by the end of 2027, 40% of all AI projects will be shut down—not because the underlying technology fails, but because internal security and legal teams will cut them loose. — Angelik Laboy Torres

> [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) while a single AI agent is like a "science experiment," a fleet of agents quickly becomes a major corporate liability. — Angelik Laboy Torres

> [01:55](https://www.youtube.com/watch?v=_VtrDFPjDag&t=115s) one brain with one tool — Angelik Laboy Torres

> [01:55](https://www.youtube.com/watch?v=_VtrDFPjDag&t=115s) Features that make multi-agent systems powerful (delegation, autonomy, specialization) are the exact features that make them dangerous to govern. — Angelik Laboy Torres

> [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) Permissions spread "like a virus" across the network. — Angelik Laboy Torres

> [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) Teams face an "archaeology problem" rather than a functional audit trail. — Angelik Laboy Torres

> [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) While teams agree that sensitive actions... require human approval, they often fail to programmatically build the pause mechanism, leaving agents to either execute unauthorized actions or stall indefinitely. — Angelik Laboy Torres

> [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) When security guardrails and RBAC permissions are managed centrally, every new agent automatically inherits existing organizational governance policies day one — Angelik Laboy Torres

## Mentioned Angelik Laboy Torres, Retool, Inc., Retool, Gartner, DataDog, PagerDuty, Slack, Linear, Ai4, YouTube

## Video description Angelik Laboy Torres - Sr. Developer Advocate, Retool, Inc.

Who's actually watching your agents and what happens when one of them goes rogue?

As enterprises move from single-agent experiments to full agent fleets spanning operations, finance, and compliance, the governance gap has become the #1 reason AI projects get shut down by security and legal teams. The talk will walk you through why multi-agent systems break traditional permission models, where the three most common governance failures happen in production, and how to architect agent systems that your security team will actually trust.

Using a live demo, I'll show how to build a governed multi-agent workflow in Retool, where each specialized agent inherits your existing data permissions, RBAC, and human-in-the-loop approval gates from a single centralized platform, with a full audit trail on every decision. Want to leave with a governance checklist you can apply to your agent system today? Come to this talk!

[Source](https://www.youtube.com/watch?v=_VtrDFPjDag)

中文简报

Retool 布道师推出多智能体 AI 框架以遏制项目关停具备内置权限范围限定与人工审批关卡的集中治理平台，可预防 Gartner 预测将在 2027 年前导致 40% 的 AI 项目关停的安全故障。 Angelik Laboy Torres 警告称，随着团队从单一智能体扩展到复杂的智能体集群，预计到 2027 年将有 40% 的 AI 项目因法律和安全治理失败而被关停。她指出了生产环境中的三种主要故障模式：权限溢出（Permission Bleed）、黑盒问题（the Black Box Problem）以及升级断层（Escalation Gaps）。为应对这些问题，她借助值班分诊演示介绍了一种受治理的多智能体架构，使专用智能体在严格的领域范围限定和人机协同审批关卡下运行。最后，她提供了一份包含 5 个要点的治理清单，帮助团队从第一天起就确保数据边界、统一审计跟踪并继承企业既有权限。 Gartner 预测到 2027 年底将有 40% 的 AI 项目因安全与法律团队干预而被迫关停；多 Agent 协同虽然提升了自主性，但也导致系统复杂度非线性增长（10 个 Agent 可产生 45 种交互关系），使单 Agent 实验演变为企业级安全隐患。 (0:04) 企业在生产环境中主要面临三大治理失效模式：Agent 间委派任务引发的权限外溢（Permission Bleed）、碎片化日志导致的黑盒审查难题（The Black Box Problem），以及高风险动作缺乏人工介入暂停机制的升级断层（Escalation Gaps）。 (5:33) Retool 演示了受治理的多 Agent 值班架构，由顶级编排器配合调查员、协调员和沟通员 Agent，在 30 秒内完成报警分诊并生成修复建议，同时对 DataDog、PagerDuty 等工具实施严格的权限隔离，确保高危操作必须经由人工批准。 (7:24) 建议企业依托集中式平台让 Agent 原生继承既有的 RBAC 权限体系，并对照包含明确数据边界、人工审批门禁、全量审计追踪等在内的 5 项治理清单，在第一天就把控好安全合规风险。 (12:16)

Reading summary

Retool's Angelik Laboy Torres argues that unchecked multi-agent expansion creates compounding enterprise liabilities, warning that deployments risk being shut down by security and legal teams unless governed by strict architectural boundaries. Agent Sprawl and Compounding Relational Risk (0:04) When enterprise business units independently deploy specialized agents for tasks like content generation, expense triaging, and workflow automation, unmanaged proliferation quickly creates an ungovernable operational environment. Torres cites a Gartner forecast predicting that by the end of 2027, internal security and legal teams will shut down 40% of all AI projects—not due to technological failure, but over governance concerns. While a single agent operates as a manageable unit with one tool and one set of permissions, multi-agent systems divide complex tasks across autonomous orchestrators and specialized workers. This structural shift introduces non-linear risk. The potential interaction relationships scale sharply—2 agents create 1 connection, 5 agents create 10, and 10 agents produce 45 potential interaction paths, excluding external databases and APIs. Consequently, the core benefits of multi-agent architectures—delegation, autonomy, and specialization—directly expand attack surfaces and permit autonomous decisions without human oversight. Three Primary Production Failure Modes (5:33) Torres identifies three governance failures that routinely disrupt enterprise multi-agent deployments: permission bleed, the black box problem, and escalation gaps. Permission bleed occurs when an agent delegated a database task passes work to a secondary agent, causing permissions to spread across systems without explicit authorization. The black box problem arises when compliance teams or auditors attempt to inspect agent reasoning. Rather than a coherent chronological record of inputs, outputs, accessed data, and decisions, organizations are left with fragmented logs scattered across disparate tools. Finally, escalation gaps emerge when teams fail to program human-in-the-loop safeguards. Without automated pause mechanisms for sensitive operations such as high-value payments or customer data updates, agents either stall indefinitely or execute unauthorized actions. Architectural Controls for Multi-Agent Workflows (7:24) To counter these vulnerabilities, Torres presents a governed on-call engineering workflow triaging alerts across DataDog, PagerDuty, Slack, and Linear. The architecture divides responsibilities among an orchestrator and three specialized sub-agents: an Investigator, a Coordinator, and a Communicator. Within 30 seconds of an incident trigger, the system provides Slack with an alert summary, hypothesis, confidence score, and proposed remediations. Each component in this architecture directly targets a core failure mode. Domain-scoped permissions prevent permission bleed: the Investigator accesses DataDog and PagerDuty but cannot interact with Slack or Linear, while the Coordinator files tickets and posts updates without database query access. A centralized audit trail records every tool call, input, and decision timestamp to eliminate black-box log hunting. Escalation gaps are closed by requiring human-in-the-loop approvals before high-stakes remediation actions, such as rolling back deployments, can execute. Centralized RBAC and the Five-Question Checklist (12:16) Rather than piecing together disparate micro-frameworks for workflows, user interfaces, and agents, Torres advocates for centralized platforms where agents inherit established role-based access control (RBAC) policies from day one. This structural alignment allows security teams to support rather than restrict agent initiatives. To ensure operational safety prior to production deployment, Torres outlines a five-question governance checklist: verifying explicit data boundaries for every agent; confirming programmatic approval gates for high-stakes actions; maintaining on-demand, unified audit trails; ensuring agents inherit existing enterprise permissions; and embedding governance directly into system architectures from inception rather than retrofitting it later. Torres concludes that answering all five governance criteria aligns engineering speed with corporate compliance standards, though the model presumes organizations possess the tooling and unified infrastructure necessary to enforce cross-agent policy inheritance.

精炼阅读摘要

Retool 高级开发者布道师 Angelik Laboy Torres 指出，多智能体系统的自主性与委派机制极易引发复合安全隐患与治理真空；企业若不从底层架构建立数据边界、审计追踪和人工介入机制，大量 AI 项目或将面临被内控安全与法务部门关停的风险。智能体泛滥与非线性膨胀的安全风险 (1:55) Torres 提出，单一智能体通常仅具备单一工具与明确权限，属于容易追踪与定责的“科学实验”；然而当市场、财务和运营等业务部门各自独立部署专用智能体时，企业将面临“智能体泛滥”（Agent Sprawl）带来的复合安全隐患。多智能体系统依赖任务委派、自主运行和高度专业化，但这三大特性直接导致了安全脆弱性：委派意味着自主传递权限，自主运行排除了人类实时监督，而专业化则大幅扩大了潜在攻击面。此外，系统内部的潜在交互复杂度呈非线性增长，2 个智能体对应 1 组交互关系，5 个智能体增至 10 组，而 10 个智能体在未计入外部数据库、API 及第三方工具的情况下就会产生 45 组连接关系。Gartner 预测，到 2027 年底将有 40% 的 AI 项目被关停，其根本原因不在于技术故障，而在于内部安全与合规部门因无法穿透管控而被迫终止项目。生产部署中常见的三大治理失效模式 (5:33) 在企业落地多智能体系统时，Torres 归纳出三类最容易破坏生产稳定性的治理缺陷。第一类是“权限蔓延”（Permission Bleed），当下游智能体为了完成委派任务而间接调用上游资源时，权限可能像病毒一样在网络中扩散，导致次级智能体获得未经授权的数据访问权。第二类是“黑盒难题”（The Black Box Problem），当审计人员需要了解决策动因时，运维团队往往只能提供散落在多个工具中的原始非结构化日志或碎片化推理链，面临像“考古”一样难以重构输入、输出与批准过程的合规困境。第三类是“升级断层”（Escalation Gaps），团队往往认同大额支付或修改客户信息等高敏感操作必须有人工参与（Human-In-The-Loop, HITL），但在代码层缺乏程序化的暂停与确认机制，导致智能体要么直接越权执行，要么因逻辑缺失而无限期挂起。运维值班场景下的受控架构设计验证 (7:24) 为针对性解决上述失效模式，Torres 演示了一套用于工程值班调度的受控架构，用以自动化处理原本需要工程师凌晨在 DataDog、PagerDuty、Slack 和 Linear 间手动切换的警报排查。该系统由一个负责接收警报（如结账 API 异常）与分发任务的编排智能体（Orchestrator），以及负责具体执行的调查者（Investigator）、协调者（Coordinator）和沟通者（Communicator）三个专业智能体组成。在解决黑盒问题上，系统集中记录带有时戳的工具调用、输入输出与编排推理链，形成统一审计追踪；在权限管理上，系统执行严格的作用域隔离，例如调查者可读取 DataDog 和 PagerDuty 指标但无法触碰 Slack 或 Linear，协调者则反之，严防交叉污染。面对升级断层，系统在触发警报 30 秒内向 Slack 输出包含故障假设、置信度分数及建议修复方案的摘要，高风险动作（如回滚部署或资源扩容）必须由人类操作员确认或临时授权后方可执行。集中式平台治理与上线前五项评估清单 (12:16) 针对分散拼凑自定义微框架所引发的合规缺口，Torres 建议企业采用集中式平台来统一部署智能体、工作流、权限与前端界面。当安全护栏与基于角色的访问控制（RBAC）在中央层面进行配置时，新部署的每个智能体均可自第一天起自动继承现有的企业安全策略，使安全合规部门从业务阻碍者转变为支持者。此外，她提出了团队在将多智能体投入生产前必须通过的五项治理核查清单：智能体是否具备明确界定的数据边界；高风险行为是否内建了程序化的人工审批关卡；是否能按需提供完整的端到端审计日志；智能体是否直接继承了既有的企业 RBAC 权限体系，以及治理架构是否在项目伊始就已内生设计而非事后补丁。 Torres 强调，满足五项治理清单是防止多智能体项目因合规审查被关停的关键前提。不过在复杂的企业IT环境中，如何平衡统一安全约束与各部门开发智能体时的业务敏捷性，仍是技术团队需要持续权衡的实际挑战。

Detailed notes — zh

## Background 该视频由 Ai4 于 2026 年 8 月 15 日在 YouTube 平台发布，主讲人为 Retool 资深开发者布道师 Angelik Laboy Torres。内容聚焦于企业从单 Agent 实验迈向多 Agent 机队时面临的治理鸿沟，深入剖析安全与法务团队关停 AI 项目的核心原因，并通过 Retool 演示如何构建具备原生 RBAC、人机审批门禁与完整审计追踪的受控多 Agent 工作流。 ## TL;DR 演讲指出多 Agent 系统在自主委派和连接呈组合式爆发的过程中，极易引发权限渗漏（Permission Bleed）、审计黑盒（Blackbox Problem）与升级断层（Escalation Gaps）三大治理故障，甚至导致高达 40% 的企业 AI 项目被安全合规部门强制叫停。为化解多 Agent 蔓延带来的合规危机，讲者提出必须依靠集中式平台构建受控架构，为各专职 Agent 设定严格的最小权限范围，并在关键操作前工程化落地 Slack 人机审批门禁（HITL）。通过将全链路决策日志打上时间戳并自动归责，企业能够建立安全团队认可的审计体系，并通过 5 项核心自查清单确保 Agent 治理的原生内置。 ## Atmosphere & style 整场演讲干练务实且极具警示意味，讲者开场便通过犀利的破冰提问直击现场开发者‘只见 Agent 效率、不见后台失控’的认知盲区。叙事结构从剖析多 Agent 拓扑爆炸的严峻风险平滑过渡到清晰的架构拆解与实战演示，褪去了 AI 领域常见的浮夸宣传，展现出强烈的工程严谨性。讲解节奏紧凑有力，将原本繁复枯燥的合规审计问题转化为清晰直观的操作规范，给技术决策者带来兼具紧迫感与落地确定性的专业启发。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) Gartner 预测到 2027 年底将有 40% 的企业 AI 项目被安全与法务部门主动关停，治理与合规鸿沟成为落地的首要阻碍而非技术缺陷。 - [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) 单个 AI Agent 类似边界清晰的科学实验，但缺乏管控的多 Agent 机队（Fleet）快速蔓延会演变成重大的企业安全负债。 - [02:02](https://www.youtube.com/watch?v=_VtrDFPjDag&t=122s) 多 Agent 系统的交互复杂度呈组合级数增长（关系数为 n(n-1)/2），自主委派机制实质上导致权限在无人类监管的情况下私自流转。 - [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) 权限渗漏（Permission Bleed）是多 Agent 治理的核心故障之一，Agent 易在委派子任务时将核心数据访问权违规转交扩散。 - [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) 黑盒与日志分散导致系统难以还原 Agent 读取的数据与推理逻辑，无法满足合规与安全团队对全链路决策审计的要求。 - [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) 升级断层（Escalation Gaps）使人类介入机制（HITL）停留在口头共识，缺乏工程化门禁会导致 Agent 擅自做出越权破坏操作或无声停滞。 - [07:24](https://www.youtube.com/watch?v=_VtrDFPjDag&t=444s) 多 Agent 架构应对每个专职 Agent 实施精细权限范围划分（Permission Scoping），严格隔绝数据读取与操作权限的跨 Agent 传递。 - [07:24](https://www.youtube.com/watch?v=_VtrDFPjDag&t=444s) 高风险操作必须工程化落地为阻塞式审批门禁，以结构化卡片形式呈现置信度与建议操作，经人工授权后方可执行并自动归责。 - [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) 相较于多系统拼凑带来的缝隙风险，集中式治理平台能让新建 Agent 从第一天起原生继承成熟的企业级 RBAC、数据权限与审计管道。 - [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) 多 Agent 治理自查需围绕数据边界、工程化审批门禁、全链路审计还原、现有权限体系继承及原生内置治理能力五个维度展开。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) 破冰互动与 Gartner 预测：AI Agent 落地中的治理鸿沟在演讲开场，Angelik Laboy Torres 通过现场举手互动切入主题，询问在座有多少人已经在生产环境中部署了 AI Agent（智能体）。紧接着，她提出了一个直击痛点的问题：“如果你们确切知道自己的 Agent 每一秒在接触什么数据、正在和谁对话，请继续举手。”面对台下稀疏的反应，她直言这正是当前企业普遍面临的治理困境——人们自以为 Agent 在做的事，与 Agent 实际在后台执行的操作之间存在着巨大的认知鸿沟。为了说明失控的严重性，她引用了 Gartner 的最新预测报告：到 2027 年底，将有高达 40% 的 AI 项目被强制关停。令人深思的是，叫停这些项目的并非技术本身不够成熟，而是企业内部的安全与法务团队出于合规与风险考量直接切断了项目支持。Angelik 提出了一个核心观点：“单个 AI Agent 更像是一个科学实验，而庞大的 Agent 机队（Fleet）则是一项负债。”许多团队在不断生成新 Agent 时，往往心存侥幸地希望安全团队不要提出尖锐问题，但这绝非长久之计。 ```mermaid flowchart LR accTitle: Agent 数量增加带来的治理风险演变 subgraph S1["单 Agent 实验阶段"] A["单脑 / 单工具"] --> B["权限边界清晰"] B --> C["可追溯责任人"] end subgraph S2["多 Agent 机队阶段"] D["多智能体网络"] --> E["权限自主传递 / 交叉连接"] E --> F["安全与合规风险激增"] end S1 -.->|"规模化扩张"| S2 ``` - **Gartner 风险预测**：到 2027 年底，40% 的企业级 AI 项目将被安全与法务部门主动关停。 - **治理鸿沟的本质**：技术实验的快速推进与企业安全管控、审计可见性之间的严重脱节。 --- ### [02:02](https://www.youtube.com/watch?v=_VtrDFPjDag&t=122s) Agent 蔓延危机与单 Agent 到多 Agent 系统的质变当企业内的 Agent 数量从 1 个蔓延到 5 个、10 个时，“Agent 蔓延”（Agent Sprawl）危机便悄然爆发。市场团队可能构建了内容生成 Agent，财务团队搭建了费用报销分类 Agent，运营团队则部署了自动化工作流 Agent。单看每个用例都极具业务价值，但当它们交织在一起时，却无人负责地演变成了一个“无法治理的混乱僵局”。企业 CISO（首席信息安全官）整夜难眠的核心痛点在于：当 Agent 具备自主行动能力并造成破坏时，究竟由谁来承担责任？是开发者、Agent 本身，还是企业？答案毫无疑问永远是企业。从单 Agent 演进到多 Agent，系统属性发生了根本性质变。单 Agent 就像“一个大脑配合一个工具”，拥有单一的权限集，其逻辑完全可以装进开发者的脑海中，一旦出错很容易定位原因；然而，当多个 Agent 协同工作、互相委派任务时，系统的连接数并不是线性增长，而是呈组合式爆发。 ```mermaid flowchart TD accTitle: 多 Agent 协同网络与连接复杂度 subgraph Fleet["多 Agent 协同网络 (n=5 产生 10 条连接, n=10 产生 45 条连接)"] Ag1["Agent A (研究)"] <--> Ag2["Agent B (审查)"] Ag1 <--> Ag3["Agent C (起草)"] Ag2 <--> Ag4["Agent D (审批)"] Ag3 <--> Ag4 Ag1 <--> Ag5["Agent E (执行)"] Ag4 <--> Ag5 end Fleet --> Tool["外部数据库 / APIs / SaaS 工具"] ``` Angelik 指出，2 个 Agent 之间只有 1 条交互关系，5 个 Agent 会产生 10 条关系，而 10 个 Agent 则会迅速膨胀到 45 条关系（计算公式为 $\frac{n(n-1)}{2}$）。这甚至还没算上每个 Agent 连接的数据库、API 和第三方 SaaS 工具。原本为了提升效率而采取的“专业化分工”与“自主委派”，在安全视角下变成了攻击面的几何级数放大： - **自主委派的实质**：等同于权限在无人类干预的情况下在 Agent 间私自流转。 - **专业化拆分的代价**：产生更多的交互接口与安全盲区，最终演变成无人监控的复杂网络。 --- ### [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) 生产环境中多 Agent 系统的三大治理故障在实际生产环境中，多 Agent 系统最常出现的治理崩溃主要集中在三个维度： ```mermaid flowchart TD accTitle: 生产环境中三大 Agent 治理故障 subgraph Failures["三大典型治理故障"] F1["1. 权限渗漏 (Permission Bleed) 权限像病毒一样沿委派链路扩散"] F2["2. 黑盒问题 (Blackbox Problem) 日志分散，决策链无法审计还原"] F3["3. 升级断层 (Escalation Gaps) 缺乏有效的人类介入机制与停机策略"] end ``` 1. **权限渗漏（Permission Bleed）**：某个 Agent 为了完成单一任务获得了对核心数据库的访问权，随后该 Agent 将子任务委派给另一个未经授权的 Agent，导致数据访问权限被悄然转交。“权限在系统中像病毒一样扩散开来”，而这一切从未经过安全团队的正式审批。 2. **黑盒与审计黑洞（The Blackbox Problem）**：当审计员介入调查某项异常决策的推理逻辑时，开发者往往只能提供分散在各个系统中的杂乱日志。如果无法清晰重构 Agent 究竟读取了什么数据、基于什么逻辑做出判断、以及由谁批准执行，这就不是合规审计，而是一场“系统考古挖掘”，审计团队绝不会接受这种结果。 3. **升级断层（Escalation Gaps）**：业内普遍认同在涉及大额支付或敏感客户数据时必须引入“人类在环”（Human-in-the-Loop），但绝大多数团队停留在口头共识，根本没有落地工程实现。结果导致 Agent 在遇到高危操作时，要么擅自越权做出破坏性决策，要么在后台陷入无声的挂起与停滞。 --- ### [07:24](https://www.youtube.com/watch?v=_VtrDFPjDag&t=444s) 实战案例：基于 Retool 构建安全受控的多 Agent 运维系统针对上述痛点，Angelik 展示了一个基于 Retool 构建的多 Agent 故障响应系统。该场景模拟了值班工程师在凌晨 3 点被报警唤醒、疲于排查 Datadog、PagerDuty、Slack 和 Linear 等多套工具的经典痛点。系统通过架构设计严密防范了三大治理风险： ```mermaid flowchart TD accTitle: 基于 Retool 的受控多 Agent 架构 Alert["PagerDuty 警报触发 (如结账 API 激增)"] --> Orch["编排 Agent (Orchestrator) 负责任务拆解与分发，记录全局思考链路"] subgraph Agents["受控专职 Agent (继承严格 RBAC 与范围限定)"] Investigator["调查 Agent (Investigator) 仅限: 读取 Datadog 指标/事件 禁止: 访问 Slack / Linear"] Coordinator["协调 Agent (Coordinator) 仅限: 创建 Linear 工单 / 发送 Slack 禁止: 查询 Datadog 底层指标"] end Orch --> Investigator Orch --> Coordinator Investigator -.->|"输出分析假设与置信度"| Orch Coordinator --> SlackMsg["Slack 人机审批卡片 (HITL Gate) 展示置信度、推荐操作 (如回滚) 与审计文档链接"] SlackMsg -->|"工程师点击审批 / 授权"| Action["执行回滚操作并在 Postmortem 中自动归责标记"] ``` 在系统运转细节与治理方案上，该架构实现了以下防护： - **全程审计追踪破解黑盒**：编排 Agent 记录并打上时间戳存储每一次工具调用、输入与输出。审计员可以清晰查阅思考过程：“由于 XYZ 原因激活调查 Agent”、“通知协调 Agent 在 Linear 生成事后分析报告并在 Slack 发起审批”。 - **精细权限范围（Permission Scoping）遏制权限渗漏**：每个 Agent 仅赋予完成任务所需的最小权限。调查 Agent 只能读取监控指标与告警，严禁触碰 Slack 或 Linear；协调 Agent 只能起草工单和推送通知，严禁跨权限查询底层数据库，彻底隔绝权限传递。 - **工程化人类介入（Human-in-the-Loop）弥合升级断层**：在 PagerDuty 触发告警 30 秒内，Slack 会收到结构化卡片，明确呈现故障假设、置信度分数以及建议操作（如执行 Rollback 还是继续上报），并附带全流程审计文档链接。高危操作仅作为“提议”呈现，必须等待值班工程师点击确认才会执行。 - **自动归责与状态记录**：工程师在 Slack 中完成确认后，系统会自动在复盘报告中记录操作责任人、响应耗时与处理步骤，从根本上解决了“谁该负责”的合规问责问题。 --- ### [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) 集中式治理平台的优势与五大核心自查清单 Angelik 总结道，要避免在每个独立 Agent 上重复造轮子去构建护栏，最有效的方法是采用**集中式平台（Centralized Platform）**。如果企业选择将工作流、权限控制、应用界面与 Agent 割裂拼凑在不同系统中，每一个拼接缝隙都将成为治理失效的重灾区；而在集中式平台中，所有新增的 Agent 都会自动继承现有的企业级 RBAC（基于角色的访问控制）、数据权限与审计管道。这种架构能将安全团队从“业务绊脚石”转变为多 Agent 架构的“最大推动者与拥护者”。 ```mermaid flowchart TD accTitle: 集中式平台治理继承优势 subgraph Fragmented["拼凑式架构 (容易产生治理断层)"] F_Ag["独立 Agent"] -.->|"缝隙风险"| F_Wf["工作流系统"] F_Wf -.->|"缝隙风险"| F_Perm["分散权限体系"] F_Perm -.->|"缝隙风险"| F_App["业务前端应用"] end subgraph Centralized["集中式平台架构 (Retool 一体化治理)"] Core["统一集中管控平台 (中央 RBAC / 审计日志 / 统一数据源)"] Core --> AgA["新建 Agent A"] Core --> AgB["新建 Agent B"] Core --> AgC["新建 Agent C"] style Core fill:#e1f5fe,stroke:#0288d1,stroke-width:2px end ``` 在演讲尾声，Angelik 留给所有开发者和技术管理层一份包含 **5 个核心问题的多 Agent 治理自查清单**： | 序号 | 治理自查问题 | 风险与考量说明 | |:--- |:--- |:--- | | **1** | 每个 Agent 是否都具备清晰明确的数据边界？ | 如果无法准确划定数据访问红线，系统极大概率正存在隐秘的权限渗漏（Permission Bleed）。 | | **2** | 高风险操作是否已工程化落地“人类审批门禁”？ | 不能仅停留在“我们应该有人工审核”的口头意向，必须在代码与交互层构建真正的阻塞式审批流。 | | **3** | 是否能够按需随时提供完整的全链路审计日志？ | 必须能随时向合规/审计人员还原 Agent 的完整决策树、输入输出及审批凭证。 | | **4** | Agent 是继承现有权限系统，还是需要另建一套安全模型？ | 避免为 Agent 单独维护平行的第二套权限，必须直接继承现有的成熟企业 RBAC 体系。 | | **5** | 治理能力是“原生内置”（Day-1 Built-in）还是“事后打补丁”（Bolted-on）？ | 事后强行拼凑附加的治理规则往往漏洞百出，是数据泄露与越权的主要策源地。 | ## Notable quotes > [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) 如果你们确切知道自己的 Agent 每一秒在接触什么数据、正在和谁对话，请继续举手。 — Angelik Laboy Torres > [00:04](https://www.youtube.com/watch?v=_VtrDFPjDag&t=4s) 单个 AI Agent 更像是一个科学实验，而庞大的 Agent 机队则是一项负债。 — Angelik Laboy Torres > [02:02](https://www.youtube.com/watch?v=_VtrDFPjDag&t=122s) 当 Agent 具备自主行动能力并造成破坏时，究竟由谁来承担责任？是开发者、Agent 本身，还是企业？答案毫无疑问永远是企业。 — Angelik Laboy Torres > [02:02](https://www.youtube.com/watch?v=_VtrDFPjDag&t=122s) 自主委派的实质，等同于权限在无人类干预的情况下在 Agent 间私自流转。 — Angelik Laboy Torres > [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) 权限在系统中像病毒一样扩散开来，而这一切从未经过安全团队的正式审批。 — Angelik Laboy Torres > [05:33](https://www.youtube.com/watch?v=_VtrDFPjDag&t=333s) 如果无法清晰重构 Agent 究竟读取了什么数据、基于什么逻辑做出判断、以及由谁批准执行，这就不是合规审计，而是一场系统考古挖掘。 — Angelik Laboy Torres > [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) 如果企业选择将工作流、权限控制、应用界面与 Agent 割裂拼凑在不同系统中，每一个拼接缝隙都将成为治理失效的重灾区。 — Angelik Laboy Torres > [12:16](https://www.youtube.com/watch?v=_VtrDFPjDag&t=736s) 事后强行拼凑附加的治理规则往往漏洞百出，是数据泄露与越权的主要策源地。 — Angelik Laboy Torres ## Mentioned Angelik Laboy Torres, Retool, Gartner, Datadog, PagerDuty, Slack, Linear, RBAC, Human-in-the-Loop, Ai4 ## Video description Angelik Laboy Torres - Sr. Developer Advocate, Retool, Inc. Who's actually watching your agents and what happens when one of them goes rogue? As enterprises move from single-agent experiments to full agent fleets spanning operations, finance, and compliance, the governance gap has become the #1 reason AI projects get shut down by security and legal teams. The talk will walk you through why multi-agent systems break traditional permission models, where the three most common governance failures happen in production, and how to architect agent systems that your security team will actually trust. Using a live demo, I'll show how to build a governed multi-agent workflow in Retool, where each specialized agent inherits your existing data permissions, RBAC, and human-in-the-loop approval gates from a single centralized platform, with a full audit trail on every decision. Want to leave with a governance checklist you can apply to your agent system today? Come to this talk! [Source](https://www.youtube.com/watch?v=_VtrDFPjDag)

Source quotes

[0:04] Angelik Laboy Torres: 如果你们确切知道自己的 Agent 每一秒在接触什么数据、正在和谁对话，请继续举手。 [0:04] Angelik Laboy Torres: 单个 AI Agent 更像是一个科学实验，而庞大的 Agent 机队则是一项负债。 [2:02] Angelik Laboy Torres: 当 Agent 具备自主行动能力并造成破坏时，究竟由谁来承担责任？是开发者、Agent 本身，还是企业？答案毫无疑问永远是企业。 [2:02] Angelik Laboy Torres: 自主委派的实质，等同于权限在无人类干预的情况下在 Agent 间私自流转。 [5:33] Angelik Laboy Torres: 权限在系统中像病毒一样扩散开来，而这一切从未经过安全团队的正式审批。 [5:33] Angelik Laboy Torres: 如果无法清晰重构 Agent 究竟读取了什么数据、基于什么逻辑做出判断、以及由谁批准执行，这就不是合规审计，而是一场系统考古挖掘。 [12:16] Angelik Laboy Torres: 如果企业选择将工作流、权限控制、应用界面与 Agent 割裂拼凑在不同系统中，每一个拼接缝隙都将成为治理失效的重灾区。 [12:16] Angelik Laboy Torres: 事后强行拼凑附加的治理规则往往漏洞百出，是数据泄露与越权的主要策源地。

Source quotes

[0:04] Angelik Laboy Torres: by the end of 2027, 40% of all AI projects will be shut down—not because the underlying technology fails, but because internal security and legal teams will cut them loose. [0:04] Angelik Laboy Torres: while a single AI agent is like a "science experiment," a fleet of agents quickly becomes a major corporate liability. [1:55] Angelik Laboy Torres: one brain with one tool [1:55] Angelik Laboy Torres: Features that make multi-agent systems powerful (delegation, autonomy, specialization) are the exact features that make them dangerous to govern. [5:33] Angelik Laboy Torres: Permissions spread "like a virus" across the network. [5:33] Angelik Laboy Torres: Teams face an "archaeology problem" rather than a functional audit trail. [5:33] Angelik Laboy Torres: While teams agree that sensitive actions... require human approval, they often fail to programmatically build the pause mechanism, leaving agents to either execute unauthorized actions or stall indefinitely. [12:16] Angelik Laboy Torres: When security guardrails and RBAC permissions are managed centrally, every new agent automatically inherits existing organizational governance policies day one

Original transcript

[0:04] Well, hello everybody. Uh, thank you for being here. I actually [0:08] maybe you've done it before in the other sections, but I want to do a quick show of hands [0:12] and I was wondering if you guys have any AI agents right now in production. [0:17] Yes. Okay. And then now I want you to keep it up if you feel that you know what is happening [0:23] at every single second like what data is it touching and then also who is it talking to? [0:30] maybe. Okay. [clears throat] Yes. Um, what I thought is that this is more of a governance [0:36] issue. What we're having right now is that there is a gap between what you think is happening and [0:42] what is actually uh doing. Now, I'm not here to scare you, but I was reading the Gartner uh report [0:48] and it was saying that by the end of 2027, 40% of all AI projects will be shut down. And it's not [0:55] because the technology is bad. is more really because your own security and legal teams are [1:02] cutting it loose. Well, cutting it down. Um, and the reason is that [1:06] I like to think that one AI agent is more of like a science experiment. A fleet of agents is [1:12] a liability. And the reason is that it is more experimental than people think. So [1:18] I think most people can be more in between of like hoping that nobody's asking the right questions so [1:24] they can keep like experimenting by their owns as they're spawning new agents. So by the end of [1:29] this talk I want to give you more of a framework like five questions that you can take back and [1:34] answer for yourself if you think that your multi- aent systems are being governed. I also want to [1:40] evaluate what are the main failures that we're seeing over and over again. and then also a [1:44] mini uh multi- aent system that I built and how it is secure and combats those failures that it has. [1:55] Well, my name is Angelica Torres. I'm a senior developer advocate and to get started I have the [2:02] agent sprawl problem and maybe this version might be happening in your company. You got one agent, [2:09] you got five, you got 10. Now your marketing team maybe spun up something for content finance maybe [2:16] did another one for triaging expenses then you have ops running workflows and individually it [2:23] all seems like a great idea right but once you look at it nobody actually decided to be like oh I [2:29] want to build a ungovernable mess it just created itself now I want you to look at these questions [2:39] what data is each agent touching that may be one of the first ones. The other one is [2:43] which systems can it actually reach. The third one would be what happens when two of them start [2:48] talking to one another. And then if you can't really come up with something for that, your sisle [2:55] is being kept up all night with the following question of when agents act autonomously, [3:01] who is actually held accountable? Is it the developer? No. Is it the agent? I mean, maybe [3:08] he didn't even know what it was doing. Now the company yes most of the time every single time. [3:15] So if I were to talk what makes the difference here is that first of all you have a single [3:19] agent. A single agent is something that is very simple. It's one brain with one tool. [3:24] It's also one set of permissions and in your imagination you can hold it up in your brain. [3:30] If something goes wrong you know exactly where to look and who is accountable for that [3:35] because again there's only one thing that you're pointing at. When you have a fleet of agents [3:39] though entirely different animal multiple of a a multitude of agents that is working simultaneously [3:44] talking to each other delegating works to one another and the moment that you cross from one to [3:49] many is when you build a system and systems behave in a much different way than they do individually. [3:57] But why do we do this? Well, okay, first of all, you have a generalist that is trying to do [4:02] something okay, right? When you want to build something more powerful, you start subdividing the [4:09] task and you start building specialists. And so each agent gets one job. It could be researching, [4:14] checking, drafting, approving, and it gets really really good at that. And then an orchestrator [4:19] comes in and breaks down the compass tax and it starts assigning it to all the different agents [4:24] that is best suited for. Now there's a catch. The same thing that makes them extremely powerful, [4:31] it is what makes them hard to govern. And the reason being that if we flip the coin delegation [4:37] well that's just permissions passing between agents autonomously that is decisions happening [4:43] without you in the room specialization that is more agents more connections more surface area [4:49] so every single thing we just celebrated from your securities POV is a brand new set of problems [4:57] and it's not a linear growth it compounds if we do the math for a second two agents would [5:02] be one relationship. Five agents would be 10 and then 10 agents would be 45 relationships. [5:10] So every single agent that you add doesn't just add a single one. It adds multiple of connections [5:15] that has already in this system. And that is before you even account for databases, your [5:21] APIs and your third party tools. So by the time that somebody noticed that there was an issue, [5:26] it was not 10 agents. It was a network and a network that nobody was watching really. So [5:33] where does it actually break? I like to think of these three problems that come up over and over. [5:39] The first one is the permission bleed. So an agent gets access to a database to do one job. Fine. [5:45] But now that it has access to everything that that data source or that resource that you gave, [5:52] it can delegate that task to another agent. And just like that, your permission moved from one [5:57] agent to another one where it actually hasn't been signed off. So permissions start spreading [6:01] like a virus in a system and nobody granted them on purpose. It just sort of leaked. [6:09] The second one would be the blackbox problem. So your agent makes a decision and then your [6:14] auditor's asking, "Hey, where can I understand the reasoning behind this?" And then your answer [6:19] might be like, "Well, I have a pile of logs if you have them." And it could also be like, [6:23] okay, maybe I want a reasoning chain scatter cross for tools. If you can't reconstruct [6:29] why an agent did what it did, what data it saw, what it decided, and who approved it, [6:34] you don't have an audit trail. You have more of like an archaeology problem trying to find out. [6:39] And then your auditor would be like, that's not an acceptable answer at the end of the day. [6:45] The third one would be the escalation gaps. And not every decision should be autonomous. I think [6:49] we all agree on this, especially when we're having to approve like big payments or you have any [6:55] sensitive customer data, those need a human. But here's the problem that I see constantly is that [7:01] everybody agrees that there should be a human in the loop, but nobody's actually built it. [7:06] So, does the agent stop to think, I think this is a reasonable time for me to stop all of my [7:11] poweress, or who does it escalate to, and what happens while it waits? And if you can't answer [7:18] that, your agents are either making calls that it shouldn't be or is silently stalling. [7:24] So, let's look at a multi- aent system that is governed. In this example, I created the classic [7:30] on call engineer who is having to wake up at 3:00 a.m. and is extremely exhausted [7:35] and they have the responsibility to shift through data dogs, pedagra duty, slack, linear. [7:41] And I divided that work into four agents. I have an orchestrator that at the top decides [7:47] who is going to do what work. Then I specialized I created three specialized agents. There is [7:52] the one that we're looking at right now is the coordinator. I also have an investigator and a [7:57] one of the agents that communicates. Each one of them has its own scope, its own tools, [8:01] its own permissions. There's no share credential. There is no share context that they did not earn. [8:06] And this is really how the agents was built with this tools. Now I'm running an example of I don't [8:12] know like a checkout API that spiked and then I have paged your duty raise an alert. In the real [8:18] world the human will wake up and take action on that right but here I have an orchestrator taking [8:23] the further step and analyzing hey this should be I don't know maybe like a tool that we call to [8:31] instantiate AWS or page you duty. Now one of the first uh solutions for the audit like the [8:38] blackbox problem would be having an audit trail. So we begin by having every tool every input every [8:46] output timestamped and recorded. And if my auditor is asking next month, hey why did this incident [8:52] went the way that it did? I don't have to dig through four different tools. I have an audit [8:58] log that happens at each level of the agent. So or the orchestrator you I can see the thought process [9:04] going. I'm going to activate this agent for XYZ reason and I'm going to wait for that to finish or [9:08] I'm going to go to my communicator and I'm going to say like hey make a postmortem for [9:14] linear and then also make a human in the loop for Slack. We'll we'll see that in a second. [9:20] So yes, the orchestrator is delegating all this very specific work and doesn't skip it and you [9:25] can just see exactly what each thinking is. The next solution would be having a permission scoping [9:33] and this can help with the permission bleed that I was talking at the beginning. As you can see here, [9:38] you can notice that each agent has an ability of what it can do and what it cannot do. [9:43] So the investigator in this case has the ability to read the metrics and events. [9:47] It cannot touch luck. It cannot touch uh linear. And then in here we have that [9:53] you're scoping it down to what it can actually search for. So this is the tool for pager duty [9:59] and then if we go back we will see that the coordinator is actually having the ability [10:03] or the tool for slack and to file postmortem but it cannot query data do so. So your permissions [10:09] don't start bleeding between agent and each one of them holds what it needs as long as it needs it. [10:16] The third one would be the human in the loop. I and if you guys were here for [10:20] the last talk, uh they basically covered a lot about it and this helps with the escalation gap [10:25] and really is that within 30 seconds of pedag duty firing and the orchestrator deciding that [10:32] it should go to this agent to that agent I get a message on my Slack saying what happened. and I [10:37] have the hypothesis, the confidence, um also the recommended action, right? So, this is a very [10:43] sensitive step and it could lead to some potential destruction. I feel like a lot of us have been [10:50] reading a lot of agents going rogue in many ways. So, instead of having that happen, I say, "Hey, [10:56] um I recommend that you surface the investigation points. I recommend that maybe you scale back or [11:02] roll back or if you want to escalate this. Oh, by the way, here is the link where we documented all [11:08] this process be happening. It is proposed rather than just automatically done for you. Another way [11:14] is that uh within the platform that I was using uh if you actually chat with the agent in its process [11:20] whenever it's thinking that it could be something that you should be involved it actually pauses and [11:25] it ask if it should proceed or if we're granting it like new permissions that it should have. [11:34] And so another cool thing is like for this agent this multi- aent system that I built, [11:38] I actually had it um create almost like a report on my Slack. So whenever I like did a roll back or [11:45] I escalated, it made like a comment saying like I was responsible because I was the one on call [11:51] and these are the steps that were taken. So then we're solving the [11:54] questions of who is responsible for this, who is answering for all that and these are just, [11:59] you know, how long did it take? So that 3:00 a.m. engineer doesn't have to be like, oh my god, [12:05] I don't know where to start. I don't know what top to open. and they could just [12:08] be more actionable with um recommended recommendations that the AI agents are giving in. [12:16] So I would just name this um in case maybe people are questioning like how do I get to [12:22] this point? I think the answer really here is centralized platforms and that is because [12:26] whenever I'm building agent per agent I have to build the guardrails for each one of them and in [12:31] this case by having a centralized platform I make sure that it is all inherited so it I don't have [12:37] to rebuild anything alternatively you would just have to stitch all this together you will have [12:41] the agents in one area you will have the workflows in another one you have permissions uh over there [12:47] and then apps for a fourth one so every seam of where those connections happen that is actually [12:53] where the governance can actually start falling through and uh yeah that doesn't mean that you [12:58] have to be rebuilding the guardrails from scratch and which again can be another contributing factor [13:03] so centralization flips that if I have a new agent it is the same framework but automatically [13:08] and my favorite part is that your security team doesn't become your biggest blocker [13:12] they become your biggest champion just because you're helping them out you know [13:19] so now the final checklist that I promised said at the beginning it is five questions [13:23] that I would ask if you're thinking of building your agent systems or have them but don't know [13:29] how secure they are. The first one would be does every agent have defined data boundaries? [13:35] If you can't draw the line then maybe you have permission bleed. The second one is [13:40] a human approval gate for high stakes action. Uh not we shall have one but is it built? [13:47] Can you produce a full audit trail on demand? [13:52] Do your agents inherit your existing permission systems or do you have to build a second security [13:56] model just for them? And then the last one is governance built in from day one or bolted on [14:02] after because bolted on systems is where maybe things can start leaking as well. [14:07] If you answer yes to all this then yeah I think your security team loves you to be honest. [14:12] Um, if you're having, I don't know, some doubts, I can recommend that maybe tomorrow [14:17] or whenever you're done after this conference, you sit down with your teams and ask yourself, [14:23] is this really good or where can we do better? So, thank you. If you guys have any questions, [14:29] um, I'll be around. If not, then we have a booth that you guys can ask us.