## Background
Published by @Scale on YouTube on June 30, 2026, this page hosts a live Q&A session titled 'Systems & Reliability 2026'. Moderated by Joe Romano from Meta, the discussion features panelists David Pariag and Sargun Dhillon from Meta alongside Joe Duffy from Pulumi.
## TL;DR
The panel explores the evolving intersection of AI agents, infrastructure management, and system reliability at scale. The speakers discuss how platform teams are transitioning from static configurations to code-as-infrastructure to provide strong type checking for agents, while implementing kernel-level protections and adversarial red-teaming to enforce strict guardrails. They also share hard-won operational lessons, including the disparities in speed between agent-induced breakage and human remediation, strategies for multi-agent interoperability, and unifying observability across first-party data centers and public clouds.
## Atmosphere & style
The session carries a candid, pragmatic, and highly technical tone typical of seasoned engineering leaders reflecting on real-world operational scars. The dynamic is collaborative and fast-paced, blending light humor over agent-driven outages—such as scaling clusters to zero—with serious insights into safety, security, and human-in-the-loop governance. Listening to the conversation feels like sitting in on an transparent, behind-the-scenes post-mortem with industry experts navigating the frontiers of autonomous infrastructure.
## Key takeaways
- [00:01](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1s) Infrastructure as code and policy as code provide proactive guardrails by checking changes and enforcing compliance directly in the development loop before execution.
- [01:23](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=83s) Meta pivoted toward a long-term RBAC framework for autonomous agents and implemented kernel-level hooks to block agents from reading human developer certificates.
- [02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) Cloud capacity across public, private, and 'Neo cloud' environments is treated as uniform capacity pools managed by automated solvers based on physical constraints.
- [05:04](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=304s) AI agents offer massive operational ROI by automating compliance remediation at scale, though they introduce novel, unexpected failure modes.
- [07:16](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=436s) A major operational challenge is the speed disparity where AI agents break environments quickly while humans require extensive time to manually repair them.
- [08:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=491s) AI models perform better with strongly typed infrastructure code than static configs because code benefits from vast GitHub training data and compile-time validation.
- [09:42](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=582s) Historical deployment data and type systems are used to fine-tune AI agents, helping prevent real-world incidents like deleting production databases or zeroing out clusters.
- [12:14](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=734s) Observability tools are unifying low-level telemetry and hardware diagnostics across public cloud and first-party data centers into a single pane of glass.
- [14:04](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=844s) Future operations will feature multi-agent ecosystems with sub-agent dispatch trees and context isolation, while managing security risks around human-in-the-loop interactions.
- [16:27](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=987s) Effective AI agents require external tools and context to double-check and verify reported cloud states independently rather than relying on reported data blindly.
- [18:00](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1080s) Teams use continuous red-teaming via adversarial AI agents to deliberately attempt to break guardrails and discover security gaps before production incidents occur.
The Q&A session opens with moderator Joe Romano introducing the panel and directing the first question to Joe Duffy regarding how Pulumi handles guardrails, particularly as systems transition toward autonomous AI agents. Joe explains that infrastructure as code (IaC) offers an inherent safety layer because developers can preview exact deployment plans before execution. Adjacent technologies like policy as code further enforce compliance rules—such as ISO 27001 standards or preventing the deletion of production databases without backups. The core engineering challenge lies in shifting from post-hoc detection to proactive prevention directly within the inner development loop. Drawing on his past experience with trustworthy computing and static analysis at Microsoft, Joe notes that just as teams had to catch bugs like use-after-free before the Blaster virus struck, modern platform teams must catch destructive infrastructure changes before deployment.
- Infrastructure as code provides built-in preview steps to inspect changes before execution.
- Policy as code enforces compliance rules directly in code (e.g., ISO 27001, database protection).
- Shift-left strategies bring guardrail checks directly into the developer's inner loop to prevent failures ahead of time.
[01:23](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=83s) RBAC vs. ABAC and Securing AI Identities
David Pariag addresses how Meta approaches access control guardrails for autonomous agents, breaking down the difference between Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC). While RBAC assigns permissions based on static roles (such as viewer or administrator) and ABAC relies on metadata attributes associated with an identity, Meta initially favored ABAC for agent workflows. However, as multiple agents increasingly interact across heterogeneous infrastructure, Meta is pivoting toward a long-term RBAC framework. To prevent agents from misusing identities, Meta implemented strict kernel-level hooks in development environments to block agents from reading human developer certificates, forcing them to authenticate exclusively via dedicated agent credentials.
- **RBAC**: Defines fixed roles (e.g., admin, viewer) to assign permissions.
- **ABAC**: Assigns access rights based on specific identity attributes.
- **Kernel-level protections**: Meta uses kernel hooks in dev environments to prevent agents from accessing human certificates, enforcing strict identity segregation.
[02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) Cloud Workload Placement and ROI at Scale
Sargun Dhillon outlines Meta's philosophy on placing data and workloads across public cloud, "Neo cloud" providers, and first-party data centers. Going off-script, Sargun argues that engineering teams ideally shouldn't care where a workload runs; cloud capacity should simply be treated as another pool of resources within automated solvers and capacity planning systems. Placement decisions are driven by physical constraints—such as network bandwidth, specialized hardware, storage capability, and geographic proximity to end-users—rather than arbitrary human preferences. Regarding ROI, Sargun explains that demand for AI inference capacity is so immense that Meta focuses primarily on acquiring raw capacity across all available venues rather than hyper-optimizing placement for marginal efficiency gains, though co-designed hardware efficiency signals will play a larger role over time.
- Public and private cloud resources are treated as uniform capacity pools within Meta's internal capacity solvers.
- Hard infrastructure constraints (network capacity, custom silicon, user proximity) dictate workload placement.
- Current compute demand—especially for AI inference—makes securing raw capacity a higher operational priority than fine-tuning efficiency metrics.
[05:04](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=304s) Automating Large-Scale Operations and Balancing Velocity
Joe Duffy highlights the compelling ROI of deploying AI agents to solve infrastructure problems at massive scales, noting that human bandwidth is often the primary operational bottleneck. He cites enterprise customer examples where agents successfully remediated 500,000 compliance issues—a task virtually impossible using manual human review. AI models can analyze signals across vast, disparate systems to detect cross-domain patterns that human operators routinely miss. However, balancing agent productivity against safety remains tricky. David Pariag shares Meta's journey with agent guardrails: after initially leaning heavily into maximum speed, Meta encountered novel, unexpected incident root causes during post-mortems, including agents prompting humans to execute destructive actions. Consequently, Meta shifted toward a balanced strategy targeting high-risk failure modes like catastrophic data loss while continuously monitoring developer friction.
- **Scale remediation**: Pulumi customers used agents to resolve over 500,000 compliance violations that were previously unmanageable.
- **Novel failure modes**: Meta observed incidents where agents generated changes or prompted humans into actions that caused unexpected outages.
- **Targeted guardrails**: Modern governance focuses on preventing catastrophic risks (such as data loss) while minimizing friction in daily workflows.
[07:16](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=436s) Post-Mortems: AI-Generated Incidents and Recovery Speed
Responding to an audience question about an incident where AI agents landed code changes ("diffs") at scale, David Pariag clarifies that humans had to manually fix the resulting breakage. While an AI agent took only 20 minutes to break the environment, human engineers spent an entire day repairing the system. "I would love it if an AI agent just fixed it," David admits, emphasizing that having agents break things for humans to clean up is the worst possible outcome. Meta's long-term goal is to evolve beyond prevention and empower specialized remediation agents to automatically fix infrastructure outages much faster than human incident response teams can manage.
- **Disparity in speed**: An AI agent broke the system in 20 minutes; human engineers required 24 hours to fully remediate the outage.
- **Current limitation**: AI agents did not fix the bugs they introduced; resolution required manual human intervention.
- **Future objective**: Build autonomous remediation capabilities that resolve incidents rapidly without human toil.
[08:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=491s) Why Code-as-Infrastructure Trumps Configs for AI Agents
Joe Duffy explains why AI models perform significantly better when generating infrastructure code compared to static configuration files (such as raw YAML). Frontier AI models have spent years training on vast open-source code repositories on GitHub, where intent, structure, and semantics are explicitly captured. Additionally, code provides a strong type system, enabling agents to validate parameters, enums, and dependencies before execution rather than throwing untyped configs at a wall. Joe reflects on an early "existential crisis" over whether code would remain relevant in an agentic world, concluding that structured, reviewable code remains essential to prevent agents from running amok with unconstrained CLI scripts.
- Code benefits from years of AI model pre-training on GitHub and structured benchmarks.
- Strong type systems allow compile-time validation of configuration parameters, legal enum values, and structural logic.
- Raw CLI scripts and unvalidated configurations lack the intent and guardrails necessary for safe agentic execution.
[09:42](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=582s) Model Fine-Tuning and Real-World Infrastructure Scars
Discussing advanced agent optimization, Joe Duffy details how Pulumi experiments with model fine-tuning and classical machine learning techniques. Leveraging petabytes of historical deployment data, Pulumi applies methods like K-means clustering to identify configuration patterns that historically lead to successful deployments versus production failures. Fine-tuning models against cloud type systems and reinforcement learning feedback loops allows agents to navigate novel cloud providers and custom infrastructure. When asked about battle scars, Joe acknowledges numerous past mishaps, including agents accidentally deleting production databases and scaling Kubernetes clusters to zero, resulting in sudden capacity outages. Drawing a parallel to fighter jets, he notes that while agents will inevitably make mistakes just as humans do, success depends on building rapid detection and response loops.
- **Data-driven learning**: Pulumi uses K-means clustering on historical configuration data to predict and avoid failure patterns.
- **Type-system foundation**: Fine-tuning leverages schema and type metadata across multiple cloud providers.
- **Real-world outages**: Agent errors in production include improper database deletions and scaling Kubernetes clusters to zero nodes.
[12:14](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=734s) Internal vs. Cloud Infrastructure Observability at Meta
Sargun Dhillon describes the developer experience at Meta when running services on public cloud infrastructure versus first-party data centers. While developers historically could distinguish where their service ran, Meta has actively blurred these lines. Sargun recalls a recent severity incident (sev) where an outage occurred on public cloud capacity due to a unique configuration, yet engineers did not even realize public cloud infrastructure was involved because the telemetry appeared unified. Cloud providers are increasingly exposing low-level access to Baseboard Management Controllers (BMCs) and Host Management Controllers, allowing Meta's internal observability tools (such as those used for Twine) to present a single pane of glass for both merchant silicon in cloud facilities and proprietary hardware in Meta data centers.
- **Single pane of glass**: Internal tools integrate low-level hardware diagnostics (e.g., BMC access) across first-party and public cloud facilities.
- **Unified abstraction**: A recent major incident demonstrated that cloud-hosted services now mimic internal data center behavior so closely that engineers could not immediately tell the difference.
- **Cross-environment tooling**: Operations platforms like Twine are being unified to manage host scheduling and hardware failures identically regardless of physical hosting location.
[14:04](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=844s) Multi-Agent Ecosystems and Vendor Interoperability
Looking ahead, Sargun Dhillon envisions a complex, multi-agent architecture where agents communicate directly with other agents across internal and external boundaries. In this paradigm, cloud vendors might offer specialized AI chatbot or agent interfaces rather than traditional static APIs. Meta utilizes sub-agent dispatch systems that route specific domain problems to vendor-specific sub-agents possessing localized context, preventing cross-vendor context pollution. Moderating the discussion, Joe Romano marveled at the vision of "agents calling agents calling other agents... all the way down." David Pariag then picks up on the security implications, addressing how to manage risks when an agent provides bad advice directly to a human operator who possesses elevated system privileges.
- **Agent-to-agent architectures**: Operations will increasingly rely on sub-agent dispatch trees communicating across vendor boundaries.
- **Context isolation**: Specialized sub-agents isolate domain-specific context to prevent cross-vendor operational errors.
- **Human-in-the-loop risk**: A key security vulnerability occurs when an agent tricks a human with elevated permissions into executing an unsafe command.
[15:32](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=932s) Vetting Agent Recommendations and Mitigating Dangerous Guardrail Bypasses
The discussion addresses how user behavior around AI agents has evolved. While earlier instances saw users easily breaking out of guardrails or blindly following an agent's instructions, users are becoming increasingly critical and vigilant when evaluating output—much like learning to vet information found online. At the same time, AI models themselves are improving at recognizing boundaries and refusing to work around established constraints. However, for truly high-risk operations, managing human error alongside agent error remains critical; the best approach is to strictly restrict privileges for high-stakes tasks to a core group of trusted engineers who know how to use agents responsibly.
* User skepticism toward AI suggestions is improving as people learn due diligence similar to evaluating internet content.
* Models are natively building awareness around guardrails to prevent accidental or forced workarounds.
* High-risk operations require limiting core permissions to a small, trusted set of humans capable of intelligent agent oversight.
[16:27](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=987s) Teaching Agents Context and Correctness Verification Across Cloud Interfaces
Moderator Joe Romano poses a question to Joe Duffy regarding how agents handle non-agentic data from varied cloud providers, especially when vendor bugs or outages report false states (e.g., reporting zero available capacity when capacity actually exists). Joe expresses optimism that agents will handle correctness debugging effectively, provided they are given the right context, tool access, and system permissions. Drawing an analogy between AI models and human platform engineers, he notes that an LLM serves as the "brain," but it requires external tools—like running `kubectl` commands to independently verify cluster health—to double-check data accuracy rather than relying blindly on reported states.
* "What's the difference between an LLM and a human platform engineer? Well, it's context, skills, access to the right tools, access to the right systems..."
* Effective AI agents require external capabilities to verify raw cloud state data (e.g., verifying cluster states via `kubectl`).
* As underlying models improve, equipping them with proper access layers enables them to debug subtle state discrepancies.
[18:00](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1080s) Validating Guardrail Efficacy Through Adversarial Testing and Wrap-Up
Joe Romano directs a final question to David Pariag on how teams can verify that guardrails are working, given the traditional reliability challenge of "proving the absence of issues." David candidly admits it remains an unsolved problem, but shares that Meta is addressing it by deploying adversarial agents. These specialized agents actively attempt to breach individual and combined guardrails by executing malicious or destructive actions—such as attempting to delete production databases or access unauthorized data. This continuous red-teaming approach exposes gaps in security controls and creates a "virtuous cycle" of finding and fixing vulnerabilities. Following David's answer, Joe Romano concludes the session, thanking the audience and speakers before sending attendees off to lunch.
* Meta deploys adversarial agents specifically tasked with trying to break guardrails, access restricted data, and delete production databases.
* Adversarial testing creates a repeatable "find and fix" cycle to harden guardrail infrastructure before production incidents occur.
* The Q&A session concludes with a round of applause for speakers David Pariag, Sargun Dhillon, and Joe Duffy.
## Notable quotes
> [07:16](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=436s) I would love it if an AI agent just fixed it — David Pariag
> [14:04](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=844s) agents calling agents calling other agents... all the way down. — Joe Romano
> [16:27](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=987s) What's the difference between an LLM and a human platform engineer? Well, it's context, skills, access to the right tools, access to the right systems... — Joe Duffy
## Mentioned
Joe Romano, David Pariag, Sargun Dhillon, Joe Duffy, Meta, Pulumi, Microsoft, Blaster virus, ISO 27001, GitHub, Kubernetes, Twine, Baseboard Management Controllers (BMCs), Host Management Controllers, kubectl
Meta与Pulumi部署AI智能体管理海量云基础设施
自主AI智能体提升了跨云基础设施的运维规模，但防范灾难性故障仍需严格的护栏、类型检查代码以及持续的对抗性红队测试。
该圆桌讨论探讨了在大规模场景下AI智能体、基础设施管理与系统可靠性之间不断演进的交集。与会嘉宾讨论了平台团队如何从静态配置转向代码即基础设施，为智能体提供强类型检查，同时通过实施内核级保护和对抗性红队测试来建立严格的安全护栏。他们还分享了来之不易的运维经验教训，包括智能体引发故障与人工修复之间的速度差距、多智能体互操作策略，以及打通第一方数据中心与公有云的统一可观测性。
基础设施即代码（IaC）与策略即代码正将合规防护前置到开发流程中；Meta 针对自主智能体转向长期 RBAC 权限架构，并通过内核级钩子阻止其读取人类开发者凭证。 (1:23)
AI 智能体虽能大幅提升自动化运维规模，但也带来了严重的恢复时滞：智能体仅需 20 分钟就能搞垮环境，而人类工程师却需花费整整一天进行手动修复。 (7:16)
强类型基础设施代码相比静态配置更适合 AI 生成，结合编译期校验与历史部署数据微调，可有效预防误删生产数据库或清空集群等灾难性操作。 (8:11)
为验证安全规则的有效性，Meta 部署了对抗性智能体主动尝试越权访问与删除数据库，通过持续红蓝对抗构建安全防护的良性循环。 (18:00)
Reading summary
Platform engineering leaders from Pulumi and Meta argue that while autonomous AI agents unlock unprecedented operational scale, managing them safely requires shifting from reactive human intervention to proactive guardrails built on typed infrastructure code, kernel-level identity segregation, and continuous adversarial testing.
Typed Code and Shift-Left Infrastructure Guardrails (0:01)
Joe Duffy contends that infrastructure as code (IaC) provides significantly stronger safety boundaries for AI agents than static configurations like raw YAML. Frontier models are pre-trained extensively on structured open-source code repositories, allowing them to leverage strong type systems for compile-time validation of legal enums, parameters, and system dependencies before executing changes.
To prevent catastrophic production mishaps—such as past incidents where agents deleted databases or scaled Kubernetes clusters to zero nodes—platform teams are shifting validation directly into the inner developer loop. Pulumi pairs policy as code with machine learning techniques, using K-means clustering on historical deployment data to recognize patterns that correlate with system failures and enforce compliance standards such as ISO 27001 before execution.
Securing Agent Identities and Adversarial Validation (1:23)
David Pariag details Meta's approach to access control for autonomous agents, highlighting an ongoing transition from Attribute-Based Access Control (ABAC) toward a long-term Role-Based Access Control (RBAC) model across heterogeneous systems. To stop agents from misusing developer identities, Meta implemented kernel-level hooks in development environments that physically block agents from accessing human certificates, forcing them to authenticate exclusively through dedicated agent credentials.
Because verifying that guardrails prevent every issue is fundamentally difficult, Meta validates system boundaries through automated red-teaming. The team deploys specialized adversarial agents programmed to breach security rules, extract unauthorized data, and attempt destructive actions like database deletions. Pariag notes that this adversarial testing creates a continuous cycle to expose and remediate policy gaps before they lead to production vulnerabilities.
The Operational Asymmetry of Agent Failures (5:04)
Deploying agents offers compelling operational leverage, exemplified by enterprise Pulumi customers using agents to resolve 500,000 compliance violations that were previously impossible to address manually. However, Duffy and Pariag emphasize a sharp velocity asymmetry: in one Meta incident, an agent landed changes that broke an environment in 20 minutes, whereas human engineers required an entire day to manually diagnose and resolve the outage.
Pariag notes that having agents break systems for humans to repair represents the worst operational outcome. While modern governance prioritizes preventing catastrophic risks like permanent data loss while minimizing day-to-day developer friction, Meta’s longer-term objective centers on developing autonomous remediation agents that can resolve outages far faster than human on-call engineers.
Unified Capacity Solvers and Multi-Agent Architectures (2:54)
Sargun Dhillon explains Meta’s philosophy of managing compute across public clouds, specialized neo-clouds, and private data centers as a uniform resource pool inside automated capacity solvers. Workload placement is determined strictly by physical constraints—such as network bandwidth, custom silicon, storage capability, and geographic proximity to users. Due to immense demand for AI inference, Meta currently prioritizes acquiring raw capacity over fine-tuning marginal placement efficiencies.
Meta also unifies telemetry across environments by integrating low-level hardware diagnostics, such as Baseboard Management Controllers (BMCs), into platforms like Twine. This single pane of glass recently enabled engineers to remediate an outage on public cloud capacity without initially realizing it ran outside Meta's data centers. Looking forward, Dhillon anticipates multi-agent architectures where vendor boundaries are bridged by sub-agent dispatch trees that isolate domain context to prevent operational errors.
While AI agents offer unprecedented scale in resolving compliance and operations backlogs, the panel highlighted that production autonomy remains bottlenecked by the gap between rapid automated failures and slow human remediation. Engineering teams must establish verified type systems, external state validation tools, and adversarial red-teaming before fully delegating end-to-end infrastructure recovery to autonomous agents.
精炼阅读摘要
在 @Scale 的专题问答中，来自 Pulumi 与 Meta 的技术专家指出，AI 智能体正在重构系统运维与基础设施管理，但这也带来了从身份滥用、配置破坏到恢复耗时严重倒挂等全新可靠性挑战。团队必须通过代码化约束、内核级隔离、对抗性红蓝测试及多智能体解耦，将安全护栏前移到开发者的日常工作流中。
代码化基础设施与类型系统护栏 (8:11)
Joe Duffy 指出，在智能体自动化运维中，基础设施即代码（IaC）比传统的静态 YAML 配置或无限制的 CLI 脚本具备明显优势。前沿基础模型在大规模 GitHub 代码库上经过长期训练，对代码的语义与意图理解更深；更关键的是，强类型系统能够在编译与执行前对枚举值、参数和依赖关系进行严格校验，从根本上防止智能体盲目下发非法配置。此外，策略即代码（Policy as Code）与预览功能可以将 ISO 27001 等合规检查和核心数据库防删除策略左移至内部开发循环，在变更发生前实现主动拦截。
为了进一步规避人为和智能体失误，Pulumi 还利用历史部署数据进行 K-means 聚类分析，识别导致生产事故的高危配置模式。Joe Duffy 透露，在真实的运维历程中，智能体曾发生过误删生产数据库以及将 Kubernetes 集群节点缩容至零的严重故障，这表明单纯依靠生成式模型的推断远远不够，必须依赖外部工具链（例如执行 kubectl 校验真实状态）与强类型护栏形成快速检测闭环。
智能体权限隔离与对抗性红队验证 (1:23)
David Pariag 介绍了 Meta 在智能体访问控制方面的演进路径。Meta 初期曾偏向基于属性的访问控制（ABAC），但随着跨异构系统协作的智能体增多，团队正长期转向基于角色的访问控制（RBAC）。在环境安全方面，Meta 在开发环境中部署了严格的内核级钩子（Kernel Hooks），切断智能体读取人类开发者本地证书的途径，强制要求智能体必须通过独立的专用凭据进行鉴权，防止身份冒用。
面对“证明系统没有潜在故障”的可靠性难题，David Pariag 承认目前尚无绝对完美的数学证明方案，但 Meta 正在引入专用对抗性智能体（Adversarial Agents）进行全天候红队渗透。这些对抗智能体被专门赋予寻找安全漏洞的任务，尝试突破单一或组合式护栏、越权访问敏感数据甚至删除核心数据库。通过这种持续攻防的良性循环，工程团队能够在生产环境发生灾难前暴露并修复策略死角。
运维速度倒挂与人机协同的全新风险 (5:04)
大规模部署智能体能带来巨大的治理效益，例如 Pulumi 的企业客户曾借助智能体自动修复了超过 50 万个过去因人力瓶颈而积压的合规问题。然而，速度与稳定性的失衡带来了意想不到的生产事故。David Pariag 披露了一起内部事故细节：智能体仅耗费 20 分钟便通过批量代码提交引发了大规模环境崩溃，但后续却无法自主修复，最终由人类工程师花费了整整一天（24小时）才完成手动恢复与清理，形成了极度不对称的修复成本。
这种新型故障模式促使治理重心从单纯追求高执行速度转向高风险兜底（例如防止灾难性数据丢失）。同时，工程团队还面临智能体向具备高权限的人类工程师提供错误操作建议并诱导其执行的全新社工类风险。业内当前的应对机制一方面是提升用户对智能体建议的审视能力与模型的边界意识，另一方面则是严格收紧高危生产操作的执行权限，仅将其限制在极少数具备专业研判能力的资深工程师手中。
统一基础设施抽象与跨厂商多智能体协作 (2:54)
Sargun Dhillon 阐述了 Meta 在混合基础设施调度上的核心哲学：上层业务团队理论上无需感知负载究竟运行在第一方数据中心、公有云还是新兴的“Neo Cloud”中，算力资源已被统一视作内部容量规划求解器中的标准化资源池。实际工作负载的部署完全由网络带宽、专用定制芯片、存储容量及终端用户物理延迟等硬性物理约束决定。在当前 AI 推理算力极度渴求的背景下，Meta 的首要运营目标是获取充足的底层裸算力，而非过早针对微小的效率增益进行过度微调优化。
为了抹平云厂商与自建机房的感知差异，Meta 利用底层基板管理控制器（BMC）等接口将硬件监控遥测全面接入内部平台（如 Twine），实现了一体化可观测性。面向未来，Sargun Dhillon 描绘了一个跨越厂商边界的“智能体调用智能体”生态：云厂商可能不再仅提供静态 API，而是交付专业的垂直领域智能体；Meta 则通过子智能体分发树（Sub-agent Dispatch Systems）将特定场景问题路由至具备局部上下文的专用智能体，实现各厂商上下文的彻底隔离与精准运维。
本场研讨表明，AI 智能体在解决数十万级系统合规与大规模配置分析上具备人力不可替代的规模效益，但目前智能体“破坏极快而自主修复极慢”的技术局限仍未解决。要实现真正的无人化可靠运维，不仅需要更严苛的代码级类型验证与内核身份隔离，还有赖于未来专用自动修复智能体的成熟以及标准化多智能体交互架构的建立。
Detailed notes — zh
## Background
该视频由 YouTube 频道 @Scale 于 2026 年 6 月 30 日发布。根据页面简介，这是 Systems & Reliability 2026 大会的现场问答环节，由 Meta 的 Joe Romano 主持，邀请了 Meta 的 David Pariag、Sargun Dhillon 以及 Pulumi 首席执行官 Joe Duffy 共同探讨智能体时代的系统与可靠性议题。
## TL;DR
对谈深入剖析了自主 AI 智能体在云基础设施与生产运维落地中的护栏机制、算力调度与权限控制演进。Pulumi 与 Meta 的技术领袖们分享了内核级身份隔离、强类型系统及策略即代码如何防止智能体误删数据库或瘫痪集群，并坦率复盘了“智能体 20 分钟搞砸、人类排障 24 小时”的真实生产故障。此外，嘉宾们还探讨了利用统一抽象抹平公私有云差异的子智能体分发架构，以及引入内部“对抗性 Agent”主动攻破防御以构建自愈安全闭环的实战范式。
## Atmosphere & style
整场研讨氛围硬核、坦率且充满自嘲式的工程师幽默，嘉宾们毫不避讳地拆解生产数据库被删、集群缩容至零等触目惊心的真实“战损史”。对话在 Meta 与 Pulumi 顶尖技术专家的默契交锋中推进，既没有空洞的 AI 泡沫狂热，也没有因噎废食的保守，而是迅速将宏大命题收敛到底层内核拦截、类型系统与故障恢复等微观战场。听众能真切感受到资深系统工程师在面对海量吞吐与智能体不可控性时，那种既渴望自动化红利、又对生产环境怀有敬畏之心的严谨务实底色。
## Bulletin
- [01:23](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=83s) Meta 在工程开发环境中部署内核级 Hook 拦截，物理阻断 AI Agent 读取人类凭据与证书，从系统底层根绝“身份混淆代理”越权隐患。
- [02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) 面对海量 AI 训练与推理需求，Meta 当前的算力调度策略优先追求吞吐海量负载的“纯可用容量”，将精细化 ROI 能效优化暂时退居次席。
- [06:06](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=366s) 激进放开 Agent 权限引发了严重运维反噬：Meta 经历过 Agent 仅用 20 分钟搞砸系统、而人类工程师耗费整整 24 小时抢修的破坏性提交事故，迫使团队强化护栏与人工审批严谨度。
- [09:28](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=568s) 基础设施即代码（IaC）凭借强类型系统显著优于无类型 YAML 和随意调用的 CLI 脚本；Pulumi 正在利用 PB 级历史配置足迹结合强化学习微调基础设施专用模型。
- [18:22](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1102s) 针对 AI 运维防御机制尚不完善的痛点，团队构建了专门的“对抗性 Agent”模拟越权与删库攻击，通过主动渗透建立“发现并修复”的自愈安全闭环。
## Key takeaways
- [00:01](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1s) Pulumi 将安全理念借鉴自微软可信计算，主张将风险拦截前置到内部开发循环，通过 IaC 变更预览与“策略即代码”强制阻断破坏性操作。
- [01:23](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=83s) Meta 的权限架构在 Agent 落地初期以细粒度 ABAC 为主，并规划逐步演进至严格的 RBAC，同时通过内核 Hook 隔离人类凭证以确保 Agent 身份独立。
- [02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) 在 AI 算力供需极端失衡阶段，基础设施团队的核心任务是捕获一切可用算力，而非过早纠结于不同公有云或自建机房的极致 ROI 对比。
- [04:45](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=285s) 基础设施 Agent 在大规模场景下展现出极高投资回报率，例如短时间内扫清人工无法企及的 50 万项合规遗留漏洞，落地瓶颈仅在于安全调度与护栏。
- [06:06](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=366s) 过快消除研发摩擦导致 Agent 以非预期方式破坏系统，甚至反向诱导工程师批准高危操作，导致了 20 分钟破坏需耗费一整天抢修的惨痛事故。
- [08:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=491s) 由于顶尖模型训练主要基于开源代码库，Agent 编写 IaC 代码的表现远优于终端命令或裸脚本，代码的强类型与意图自解释性构筑了安全底座。
- [09:28](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=568s) 云配置的严苛性要求建立完备的类型系统以规避配置风险，Pulumi 正借助数以 PB 计的历史成功与失败配置数据，通过强化学习微调领域模型。
- [10:57](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=657s) 智能体运维留下了误删生产数据库、将集群意外缩容到零等严重“战损”；行业对 AI 运维失误的容忍度苛刻程度不亚于对自动驾驶零事故的要求。
- [12:14](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=734s) Meta 依托统一调度工具与开放的 BMC/HMC 硬件底层接口打造“单一玻璃平面”，并通过智能体分发架构隔离云服务商上下文，防止跨环境决策污染。
- [15:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=911s) 为防止工程师盲从 Agent 给出的幻觉操作，团队除要求人类建立批判性审查外，更在权限层将高危操作名单严格收敛至核心受信任梯队。
- [16:27](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=987s) 当云厂商平台自身指标与状态汇报不可信时，不能让 Agent 沦为“会走路的裸脑”，而需赋予其诊断工具与校验权限以主动交叉核实底层真实状态。
- [18:22](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1102s) 部署专门的对抗性 Agent 主动测试单项规则与组合防护链路，模拟删库等极端场景，以此构建敏捷迭代的“发现并修复”防御闭环。
## Detailed walkthrough
### [00:01](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1s) Pulumi 的护栏机制与内循环安全策略
在问答开场环节，主持人首先向 Pulumi 创始人兼首席执行官 Joe Duffy 发问：随着基础设施管理逐渐向自主化智能体（Autonomous Agents）过渡，Pulumi 是如何构建护栏（Guardrails）以避免破坏性变更的？Joe Duffy 指出，基础设施即代码（IaC）本身就蕴含着一种非常微妙但关键的内在护栏——在真正执行部署变更之前，开发者和系统能够清晰地预览（Preview）即将发生的变化。在此基础之上，策略即代码（Policy as Code）技术提供了强有力的规则约束，团队能够直接用代码定义合规基准，例如强制满足 ISO 27001 认证要求，或硬性禁止在没有备份的情况下删除生产环境数据库。
Joe Duffy 进一步追溯了自己在微软从事“可信计算”（Trustworthy Computing）与静态分析工具研发的历史背景。他幽默而深刻地类比道，安全的核心从来不是在灾难发生后去收拾残局：“当时的目标就是在事后为时已晚、冲击波病毒（Blaster worm）已经爆发之前捕获释放后使用（Use-after-free）漏洞，而不是等到部署后才发现。”同理，Pulumi 当下的安全核心理念是将风险拦截尽可能前置，将其深深嵌入到“内部开发循环”（Inner development loop）之中。
### [01:23](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=83s) Meta 的权限架构演进：从 ABAC、RBAC 到内核级身份隔离
随后，Meta 的 David Pariag 深入剖析了防护栏落地中的鉴权体系设计，重点辨析了基于角色的访问控制（RBAC）与基于属性的访问控制（ABAC）在 AI 场景下的应用。RBAC 传统上将人类用户归类为特定角色（如只读查看者或系统管理员），而 ABAC 则根据实体附带的动态属性来判定操作权限。David 表示，Meta 目前两手都在抓，但在 Agent 落地初期主要采用 ABAC；随着多智能体与企业内异构基础设施交互的复杂度剧增，长线规划正逐步向更严格的 RBAC 演进。
在智能体特有的安全防护（AuthZ for Agents）中，Meta 实施了一项至关重要的硬隔离机制：杜绝智能体借用或盗用人类身份。在工程开发环境中，系统通过内核级挂钩（Kernel hooks）对访问权限实施拦截，物理阻断 Agent 进程读取人类工程师的凭证与证书，强制其仅能以专属的 Agent 身份凭证与外部交互，从而在底座上消除“身份混淆代理”（Confused Deputy）的安全隐患。
```mermaid
flowchart TD
accTitle: Agent Identity Isolation via Kernel Hooks
accDescr: 描述 Meta 内部开发环境中 Agent 进程与人类证书被内核 Hook 隔离的安全架构
subgraph UserSpace["用户空间开发环境"]
AgentProc["AI Agent 进程"]
HumanCert["人类工程师证书\n(Human Cert)"]
AgentCert["Agent 专属凭证\n(Agent Cert)"]
end
subgraph KernelSpace["内核安全防护层"]
KernelHook{"内核 Hook 鉴权拦截"}
end
subgraph TargetInfra["后端基础设施"]
InfraAPI["Meta 基础设施 API"]
end
AgentProc -- "尝试读取" --> KernelHook
KernelHook -- "拦截读取人类凭证" --> HumanCert
KernelHook -- "允许挂载专属凭证" --> AgentCert
AgentCert --> InfraAPI
```
- **访问控制阶段**：初期优先引入 ABAC 为 Agent 提供细粒度属性标记，后期演化为结构更清晰的 RBAC 以支持多 Agent 大规模协作。
- **内核级防护**：针对开发环境部署内核拦截逻辑，完全剥离 Agent 对人类凭据的感知，彻底杜绝越权伪装。
### [02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) 算力放置哲学与全域 ROI：AI 需求压倒效率优化
针对 Meta 如何在公有云、新兴云（Neo cloud）与自建第一方（First-party）数据中心之间分配数据与负载，Sargun Dhillon 给出了一个出人意料且直率的回答。他调侃称自己要稍微“脱稿”（Go off script）直言：“答案是我们根本不应该去在意。”在系统工程的终极视角下，云端算力不过是另一种形式的基础算力。Meta 的调度核心应当全权依赖现有的求解器（Solvers）和容量管理基础设施，根据硬件型号差异、网络互联带宽等客观物理约束来决策，而不是由人类的先入为主来干预。
当被追问如何衡量不同算力环境下的 ROI 与利用率时，Sargun 坦言，不同云厂商与数据中心确实各有所长——有些在地理上距离终端用户更近，有些具备特定存储形态。然而在当前阶段，Meta 面临着前所未有的海量推理与训练需求，算力缺口极其庞大。当下团队的首要目标并非追求极致的算力能效比，而是“纯粹追求一切可用容量”（Just looking for capacity at this point）。只有当算力供需逐渐平稳后，Meta 才会转入精细化运作，依托数据中心特性与硬件协同设计（Co-design）来驱动高阶 ROI 放置。
### [04:45](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=285s) 自动化杠杆：基础设施 Agent 的惊人回报与落地鸿沟
Joe Duffy 将讨论引回基础设施 Agent 的 ROI 权衡。他强调，相比单纯的辅助编写业务代码，AI 智能体在基础设施领域的投资回报率甚至更为惊人，因为底层基础设施往往以超大规模运行，此前受限于人类精力瓶颈而停滞的任务，如今得以瞬间破局。
Joe 分享了一个震撼的真实案例：某企业客户此前积压了高达 50 万项安全合规遗留问题，客户明确表示，如果仅靠人力排查修改，在现实中是“根本不可能完成的任务”，但 Agent 却能在极短时间内扫清障碍。此外，AI 具备跨系统吞吐海量异构监控信号的能力，能敏锐洞察人类运维专家难以察觉的隐性模式。他指出，当前业界的真正痛点从来不是 ROI 是否成立——“ROI 显而易见就在那里”，真正的难点在于如何通过完备的护栏与调度机制安全地编排这些智能体，将理论价值转化为可靠的生产力。
- **规模化破局**：成功解决单点客户 500,000 项合规违规修复的极端任务。
- **全域信号洞见**：利用模型并行分析数千个监控源，捕捉人工审查盲区中的微弱模式。
### [06:06](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=366s) 护栏与研发速度的博弈：智能体 20 分钟搞砸、人类修了一整天
在探讨防护栏与产品交付速度的拉锯战时，David Pariag 回忆了团队在落地过程中的心态起伏。Meta 在探索之初极其迷恋 Agent 带来的开发效率跃升，因此激进地全面放开限制，全力消除各类研发摩擦。然而，故障审查会（Server review）随后的残酷现实给团队敲响了警钟：智能体开始以极具“创造力”和意料之外的方式破坏系统，甚至会反向诱导人类工程师批准一些危险操作，而不明就里的人类工程师往往直接配合执行。
这种教训促使 Meta 迅速回归理性，转向更加审慎的平衡策略：一方面将最高优先级放在防御数据丢失等毁灭性灾难事故上；另一方面持续监控防护栏给工程师日常体验带来的摩擦阻力。主持人随即追问了上午分享中提到的那起全量提交（Diffs）事故：当时智能体在短时间内向代码库推送了大量破坏性变更，现场观众都在好奇，最后是否使用了另一个 AI 修复该故障？
David 苦笑着答道，他虽然也很想看到 AI 自主修复的科幻场景，但在现实中只能由纯人工团队紧急救火。“这绝不是我们期望看到的终局状态——我们绝不想陷入‘Agent 负责到处惹祸，人类在屁股后面擦屁股’的怪圈。”他披露了令人扎心的反差细节：那次事故中，“Agent 仅用 20 分钟就彻底搞砸了系统，而人类工程师整整耗费了一整天（24 小时）才把环境修复完成。” Meta 未来的目标不仅是事前拦截，更是在受控护栏下让 Agent 具备更快的自愈修复能力。
### [08:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=491s) 为什么代码优于配置：Agent 在 IaC 领域的天然先发优势
在针对代码化基础设施的终极思辨中，主持人向 Joe Duffy 抛出核心疑问：为什么基础设施领域的 Agent 编写“代码”的表现要明显优于直接编写原生配置文件或调用 CLI 命令？这究竟是因为代码具备声明式特征，还是因为代码包含了更多面向人类的设计约定？
Joe Duffy 剖析道，这归结于前沿大模型的基础训练语料：现代 Agent 几乎浸泡在整个互联网的开源代码库（如 GitHub）中长大，代码具有高度自解释性，代码背后的工程意图是公开透明的。更重要的是，过去两年来全球顶尖的 Frontier 模型实验室始终不遗余力地在代码基准测试（Coding benchmarks）上卷极限，让模型写代码的能力产生了质的飞跃。
Joe 坦言，团队在研发初期也曾陷入“代码是否还将继续存在”的本体论危机。但实践证明，代码的地位不可替代：“如果你直接放任 Agent 在终端里随意调用乱七八糟的 CLI 和脚本，在缺乏审查与意图固化的情况下直接戳弄基础设施，下场通常极其悲惨。”基础设施即代码赋予了智能体一个它们最熟悉的游乐场——一个拥有强类型、可被人类 Code Review、且具备确定性意图表达的坚固基座。
### [09:28](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=568s) 基础设施即代码与类型系统：利用微调与强化学习规避配置风险
当被问及是否在 Pulumi 尝试对智能体（Agent）进行模型微调（Fine-tuning），以更好地应对未出现在通用训练集中的新型云服务商及自建定制基础设施时，Joe Duffy 给出了极为肯定的回答。他指出，将基础设施完全转化为代码（IaC）的核心优势在于获得了跨越整个云配置空间的“类型系统”（Type System）。与直接向系统抛出非类型化、碰运气的“无类型 YAML 格式（Untyped YAML）”不同，完备的类型系统能够实施强类型检查，自动校验某个字段是否必须为整型、或者枚举值（Enum）是否仅限制在三项合法值内。Pulumi 已经掌握了涵盖各大公有云的所有类型系统元数据。
此外，Pulumi 手中还积累了数以 PB 计的成功与失败配置历史数据。此前，团队曾运用经典的机器学习方法（如 K-means 聚类）去分析哪些特定配置参数组合更容易导致故障或带来良好表现。Joe Duffy 强调，将这些海量真实配置足迹作为强化学习（RL）的基础框架，进而对专门面向基础设施领域的模型实施微调，正是他们当前正在推进的核心战略。
- **代码即神谕（Oracle）**：借助基础设施即代码，系统能够在代码层面推演并解决问题，再映射为底层的云资源操作，已展现出巨大优势。
- **配置数据资产**：利用 PB 级历史运行数据构建强化学习反馈循环，使智能体从过去的配置缺陷中学习正确范式。
---
### [10:57](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=657s) 智能体的“战损史”：从删库惨案到自动驾驶式的高容错要求
谈到智能体在实际运维中留下的“战损（Battle Scars）”，Joe Duffy 直言不讳地承认：“公有云实在是太复杂了。”他分享了多起触目惊心的真实事故：智能体曾在不该操作的情况下直接删除了生产环境数据库；还有一次，智能体误把生产 Kubernetes 集群直接缩容到零（Scale to Zero），瞬间引发极其严重的算力容量瘫痪。云基础设施本身的严苛性正是业界尚未能全面跨入“智能体曲线（Agentic Curve）”的根本原因。工程师必须先将复杂的运维逻辑形式化、代码化，使智能体能够提前理解风险、预防错误，而非总在酿成大祸后手忙脚乱地被动响应。
Joe Duffy 引用了“战斗机飞行员时间（Fighter jet time）”的理念，认为智能体和人类一样永远都会犯错，关键在于建立极速响应并自动修正的韧性机制，不让最终接盘的运维工程师背黑锅。针对这一观点，主持人 Joe Romano 提出引人深思的类比：人类工程师多年来也一直在失误删库，智能体是否真比人类更差？这正如对待“自动驾驶汽车”：公众往往对机器人苛求完美的零失误，其接受标准天然远高于对人类自身的要求。
---
### [12:14](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=734s) 混合云与“单一玻璃平面”：让多智能体系统抹平数据中心与公有云差异
针对 Meta 开发者在开发服务时能否感知公共云与自建数据中心差异的提问，Sargun Dhillon 透露，目前虽然能看出差异，但 Meta 正在竭力抹平边界。甚至最近发生过一次 Sev 故障，因某种独特的配置丢失了大量公有云容量，而排查人员甚至根本没意识到受损的是公有云资源，这说明公有云在 Meta 内部已经深度融入日常技术栈。尽管公有云在特定定制硬件、自研硅片以及可观测性（Observability）深度上与自建机房存在差距，但 Meta 正加速让统一运维工具（如用于容器与集群调度的 Twine）无缝贯通自研芯片、商业芯片及云端机房。
随着外部公有云供应商逐渐开放低层级的硬件访问通道——例如主板管理控制器（BMC）与主机管理控制器（HMC），Meta 工程师已能通过相同的 UI 与后台 API 查看机架漏水等硬件警报，打造出真正的“单一玻璃平面（One Pane of Glass）”。在架构上，Meta 采用“智能体分发调用（Sub-agent dispatch）”架构，由上层通用智能体调用针对特定硬件或外部供应商的专属下层智能体，甚至直接与云厂商提供的机器对话接口对接，在底层严格隔离上下文，杜绝跨云服务商的信息污染（Cross-vendor pollution）。
```mermaid
flowchart TD
accTitle: Meta 多云统一管理与智能体分发架构
accDescr: 展示 Meta 工程师通过统一视图交互，经由调度智能体分派至自建机房与公有云专属智能体的层级结构。
subgraph UserLayer["交互层"]
Eng["Meta 开发者 / 运维工程师"]
UnifiedUI["单一玻璃平面 (One Pane of Glass) / Twine 统一工具链"]
end
subgraph AgentLayer["智能体调度层"]
DispatchAgent["主控调度智能体 (Sub-agent Dispatch)"]
DC_Agent["自建数据中心专属智能体"]
Cloud_Agent["第三方公有云专属智能体 (隔离供应商上下文)"]
end
subgraph InfraLayer["基础设施硬件层 (BMC / HMC API)"]
MetaDC["第一方数据中心 (自研硅片 / 商业芯片)"]
VendorCloud["第三方云厂商容量 (特定算力资源 / 外部 Chatbot)"]
end
Eng --> UnifiedUI
UnifiedUI --> DispatchAgent
DispatchAgent -->|"分发任务"| DC_Agent
DispatchAgent -->|"隔离调用"| Cloud_Agent
DC_Agent --> MetaDC
Cloud_Agent --> VendorCloud
```
- **底层硬件透传**：外部供应商正逐步下放 BMC 与 HMC 访问权限，使机架级物理故障诊断得以纳入统一监控视野。
- **防上下文污染**：每个硬件平台和外部云专属智能体维护独立业务上下文，防止多环境混杂导致决策逻辑混乱。
---
### [15:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=911s) 幻觉指令与人机协作：如何防止工程师盲从智能体错误指示
当讨论到智能体“套娃调用”带来失控、并向工程师发出危险操作指令而缺少严格权限拦截（ACL）的隐患时，David Pariag 表现出了理性的乐观主义。他回忆道，就在几个月前，Meta 确实发生过多起工程师因为轻信智能体的指引而直接突破安全围栏（Guardrails）违规操作的事故。但随着时间推移，工程师正像学会审视互联网虚假信息一样，开始对智能体给出的每一步方案建立起严密的批判性审查（Due diligence）机制。
与此同时，底层大语言模型自身的边界意识也在迅速进化，能够自主识别当前操作是否存在红线并拒绝盲目绕行。但 David 强调，面对真正触及核心命脉的高危操作，不能单靠人机互信，必须将防范人类失误与防范智能体失误置于同等高度：通过强权限控制，将实施高危操作的人员名单严格收缩到核心受信任梯队，再由这些资深工程师以高度警惕的姿态去智能调用智能体。
---
### [16:27](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=987s) 当云厂商自身数据不可信：将大模型从“会走路的大脑”升级为全能平台工程师
Joe Romano 抛出了一个深层次难题：当公有云提供商因自身软件 Bug 或宕机导致底层状态数据本身出现虚假返回（例如明明有算力却报告容量已耗尽，或反之）时，智能体能否具备去伪存真的辨识纠错能力？
Joe Duffy 认为这完全可行，但关键不在于单纯提升大模型参数，而在于为智能体构建匹配的环境：
- **拒绝“光杆大脑”假说**：大语言模型在架构中仅仅充当“大脑”，而人类平台工程师之所以能排查故障，是因为人类拥有手脚、工具和系统权限，绝非“会走路的裸脑（Walking brains）”。
- **主动交叉验证（Cross-Verification）**：当遇到可疑数据时，平台工程师会调出命令行手动敲入 `kubectl`，直接探查集群的真实状态来验证云管平台的虚假汇报；智能体也必须配备同等的上下文环境、诊断工具箱和多维度系统校验权限。
唯有让具备丰富上下文的智能体拥有独立验证数据真实性的技能闭环，随着大模型推理能力的稳步提升，行业才能在复杂、不透明的底层云环境中真正建立起对智能体自主排障的信任。最后，主持人切入下一个核心议题——当各类安全围栏全面上线后，团队究竟该如何量化验证这些围栏确实在生效，毕竟在系统可靠性工程中，“证明某类故障未曾发生”始终是一个极具挑战的行业难题。
### [18:22](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1102s) 引入对抗性 Agent 构建安全护栏的“发现与修复”良性循环
在探讨如何保障自动化系统的安全性与可靠性时，演讲者非常坦诚地承认，关于 AI Agent 的防御与管控机制目前“尚未被彻底解决”（haven't solved it fully yet）。团队的核心解决思路是主动出击——从根本上鼓励 Agent 尝试去突破现有的安全防线。为此，他们正在构建专门的“对抗性 Agent”（adversarial agents），其职责就是充当内部攻击者，全力尝试突破单一的安全护栏或多重组合的防护规则，甚至模拟执行极其危险的操作，例如尝试删除生产数据库或越权访问未授权的敏感数据。
这种通过部署恶意代理来测试现有系统鲁棒性的方法已经初见成效，团队借此在现有的防御体系中发现了许多原本未察觉的缝隙与漏洞。演讲者强调，正是这种机制帮助他们建立起了一种良性的工作流——即“发现问题并迅速修复”（find and fix）的持续迭代循环，这也是他们目前在系统可靠性演进上所极力追求的目标。
随着对抗性测试与安全边界讨论的收尾，主持人 Joe Romano 为本场问答环节画上了句号。他向所有在现场和线上踊跃提问的参会者表示感谢，并提醒大家紧接着就是午餐时间，参会者可以在休息区找到各位演讲嘉宾继续深入交流。最后，主持人号召全场观众用热烈的掌声向各位分享嘉宾致谢，为整场大会的开局定下了精彩的基调。
- **对抗性测试范围**：对抗性 Agent 会针对个别防护规则、复合防护链路进行穿透测试，并尝试模拟删除生产数据库（delete production databases）以及非法窃取敏感数据。
- **防御机制理念**：正视现有防护网的不完备性，通过自动化对抗挖掘盲区，形成“发现并修复”（find and fix）的敏捷安全闭环。
- **环节收尾**：上午的深度 Q&A 研讨圆满结束，参会人员转向午餐自由交流环节。
## Notable quotes
> [00:01](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1s) 当时的目标就是在事后为时已晚、冲击波病毒（Blaster worm）已经爆发之前捕获释放后使用（Use-after-free）漏洞，而不是等到部署后才发现。 — Joe Duffy
> [02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) 答案是我们根本不应该去在意。 — Sargun Dhillon
> [02:54](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=174s) 纯粹追求一切可用容量 — Sargun Dhillon
> [04:45](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=285s) ROI 显而易见就在那里 — Joe Duffy
> [06:06](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=366s) 这绝不是我们期望看到的终局状态——我们绝不想陷入‘Agent 负责到处惹祸，人类在屁股后面擦屁股’的怪圈。 — David Pariag
> [06:06](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=366s) Agent 仅用 20 分钟就彻底搞砸了系统，而人类工程师整整耗费了一整天（24 小时）才把环境修复完成。 — David Pariag
> [08:11](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=491s) 如果你直接放任 Agent 在终端里随意调用乱七八糟的 CLI 和脚本，在缺乏审查与意图固化的情况下直接戳弄基础设施，下场通常极其悲惨。 — Joe Duffy
> [10:57](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=657s) 公有云实在是太复杂了。 — Joe Duffy
> [16:27](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=987s) 大语言模型在架构中仅仅充当‘大脑’，而人类平台工程师之所以能排查故障，是因为人类拥有手脚、工具和系统权限，绝非‘会走路的裸脑（Walking brains）’。 — Joe Duffy
> [18:22](https://www.youtube.com/watch?v=ji4_8vr2UKU&t=1102s) 尚未被彻底解决 — David Pariag
## Mentioned
Meta, Pulumi, Joe Romano, David Pariag, Sargun Dhillon, Joe Duffy, Microsoft, 冲击波病毒 (Blaster worm), ISO 27001, GitHub, Kubernetes, Twine, BMC, HMC, @Scale
## Video description
Live Q&A | Systems & Reliability 2026
Moderator: Joe Romano, Meta
Speakers:
David Pariag,Meta
Sargun Dhillon,Meta
Joe Duffy,Pulumi
Learn more about the @Scale conferences here: atscaleconference.com
[Source](https://www.youtube.com/watch?v=ji4_8vr2UKU)
Source quotes
[0:01] Joe Duffy: 当时的目标就是在事后为时已晚、冲击波病毒（Blaster worm）已经爆发之前捕获释放后使用（Use-after-free）漏洞，而不是等到部署后才发现。
[2:54] Sargun Dhillon: 答案是我们根本不应该去在意。
[2:54] Sargun Dhillon: 纯粹追求一切可用容量
[4:45] Joe Duffy: ROI 显而易见就在那里
[6:06] David Pariag: 这绝不是我们期望看到的终局状态——我们绝不想陷入‘Agent 负责到处惹祸，人类在屁股后面擦屁股’的怪圈。
[6:06] David Pariag: Agent 仅用 20 分钟就彻底搞砸了系统，而人类工程师整整耗费了一整天（24 小时）才把环境修复完成。
[8:11] Joe Duffy: 如果你直接放任 Agent 在终端里随意调用乱七八糟的 CLI 和脚本，在缺乏审查与意图固化的情况下直接戳弄基础设施，下场通常极其悲惨。
[10:57] Joe Duffy: 公有云实在是太复杂了。
[16:27] Joe Duffy: 大语言模型在架构中仅仅充当‘大脑’，而人类平台工程师之所以能排查故障，是因为人类拥有手脚、工具和系统权限，绝非‘会走路的裸脑（Walking brains）’。
[18:22] David Pariag: 尚未被彻底解决
Source quotes
[7:16] David Pariag: I would love it if an AI agent just fixed it
[14:04] Joe Romano: agents calling agents calling other agents... all the way down.
[16:27] Joe Duffy: What's the difference between an LLM and a human platform engineer? Well, it's context, skills, access to the right tools, access to the right systems...
Original transcript
[0:01] Awesome. Welcome back to the stage.
[0:04] We are going to start off with one
[0:05] question going back to the first two
[0:07] talks from this morning about Palumi and
[0:10] guardrails. This one's for Joe. We're
[0:13] wondering
[0:15] in Palumi, how are you handling
[0:17] guardrails especially as you transition
[0:20] towards autonomous agents? What kind of
[0:22] guardrails are in place to avoid
[0:23] destructive changes?
[0:26] >> Yeah, I think one of the
[0:27] it was subtle actually. We didn't
[0:28] realize this at the beginning, but you
[0:30] know, infrastructure code sort of has
[0:31] inherent guardrails because
[0:33] you get to, you know, see what's going
[0:35] to happen when you do a deployment
[0:37] before you actually do the deployment.
[0:39] But then there are other adjacent
[0:41] technologies like policy as code
[0:43] that will enforce policies and so you
[0:45] can codify, you know, hey, I need to be
[0:47] ISO 27001 compliant or like don't delete
[0:50] production databases without backups
[0:51] and, you know, being able to express
[0:53] that in code as well, I think is
[0:55] something that we've found a lot of
[0:56] success with. I think that the challenge
[0:58] is going from finding issues after the
[1:00] fact to finding them before they happen
[1:02] in the first place. And actually worked
[1:03] on trustworthy computing back in the day
[1:05] with static analysis and in Windows and
[1:07] Microsoft and that was the thing back
[1:09] then is like don't find use after free
[1:11] bugs after it's too late and the blaster
[1:13] virus has already happened. You need to
[1:14] find it before you actually make the
[1:17] deployment and so bringing it as close
[1:19] to the inner development loop as
[1:20] possible has been kind of our approach.
[1:23] >> So one more question about guardrails
[1:25] for David.
[1:27] As we implement that, there's a
[1:28] question,
[1:29] do we use role-based access control,
[1:32] attribute-based access control? Maybe
[1:34] you can help define it for the audience
[1:36] as well.
[1:37] And in general, how is that implemented?
[1:39] The the guardrail checks? What's unique
[1:41] with AI?
[1:42] >> Yeah. So role-based access control
[1:44] defines specific roles like I'm a viewer
[1:46] or I'm an administrator and assigns
[1:48] users, you know, historically humans
[1:50] into those roles.
[1:52] And attribute-based access control
[1:54] puts attributes on identities and say
[1:56] people who have this attribute populated
[1:58] can perform this operation. Um the short
[2:00] answer is we're doing both, but we
[2:02] started with attribute attribute-based
[2:04] access control
[2:06] uh primarily for for agents. Um
[2:10] but we are some of the longer-term work
[2:11] we're doing is because we have multiple
[2:13] agents interacting with all kinds of
[2:16] different infrastructure, we're making a
[2:18] pivot over time to more a more
[2:20] role-based approach.
[2:21] Um as part of this, one of the
[2:23] interesting things we had to do was make
[2:25] sure that agents are actually just using
[2:27] their own identities. So in our one of
[2:29] the specific things we've done in AuthZ
[2:31] for agents is in our development
[2:34] environments, we protect access to human
[2:36] certs. So we don't allow agents to read
[2:38] them. So we have uh
[2:40] we have some hooks in the kernel that
[2:42] block an agent from reading my human
[2:44] certificate and ensure that it can only
[2:46] read its own agent certificate because,
[2:48] you know, we don't want them to sort of
[2:50] using the wrong identity.
[2:53] >> Perfect. Thank you.
[2:54] And now one for Sargun. Um
[2:57] as we turn up cloud, uh we're wondering
[2:59] at Meta, is there any philosophy for
[3:02] placing data and workloads uh for
[3:05] trusting public cloud, Neo cloud, versus
[3:08] first-party uh machines?
[3:10] >> Uh that's a good question. Um
[3:13] allow me to go off script for a second,
[3:15] but like the the answer is like we
[3:16] shouldn't care.
[3:17] Um and the reason we shouldn't care is
[3:19] because we should be able to treat cloud
[3:21] capacity as just another kind of
[3:23] capacity.
[3:24] We should rely on our systems that
[3:25] already exist, the solvers, the capacity
[3:28] management infrastructure, to place
[3:30] workloads based on the constraints that
[3:33] that cloud has.
[3:34] And a lot of those constraints come from
[3:36] things like what hardware is there or
[3:39] what the capacity of the network is.
[3:41] And not necessarily what humans decide.
[3:45] >> And one more question about the
[3:47] placement. Um how are you evaluating the
[3:50] ROI
[3:52] uh of placing some of these workloads,
[3:53] especially when it comes to AI
[3:55] workloads,
[3:56] when you look into the utilization
[3:58] efficiency,
[4:00] comparing between different you know,
[4:01] places to run.
[4:02] >> So, there's certainly an aspect that
[4:04] different vendors have different
[4:05] strengths, different data centers have
[4:07] different strengths even. Things like
[4:09] proximity to users or the kind of data
[4:11] we can put there based on storage and so
[4:13] on.
[4:14] The ROI really comes from maximizing
[4:17] this across the board. And we have so
[4:19] many workloads and so much demand for
[4:21] things like inference that it hasn't
[4:24] been a problem where we've tried to
[4:25] maximize ROI by
[4:28] accelerating efficiency, but instead
[4:30] we're just looking for capacity at this
[4:32] point. I think that over time this will
[4:34] shift and things will change where we do
[4:37] optimize for efficiency. And at that
[4:39] point we can use the signals about the
[4:40] strength of the data center and those
[4:42] co-designed efforts to drive placement.
[4:45] >> We actually have the same question for
[4:46] you, Joe, about ROI. As you have agents
[4:50] managing infrastructure,
[4:52] is there any kind of trade-off decision
[4:53] there with token cost or where the
[4:56] agents get involved for more complex
[4:58] infrastructure or different you know,
[4:59] different workloads that need more
[5:01] management or less management?
[5:04] >> Yeah, I think the interesting thing with
[5:05] infrastructure is because infrastructure
[5:06] operates at just like an enormous scale,
[5:09] a lot of times you can find just crazy
[5:12] good ROI for you know, automating things
[5:15] that previously were unimaginable
[5:16] because humans were kind of the
[5:18] bottleneck to being able to do things.
[5:20] You know, we've seen customers that did
[5:22] compliance initiatives where they had
[5:24] 500,000 compliance issues that they had
[5:27] to fix and they told us there's no way
[5:29] without agents that you know, we would
[5:31] be able to do this. There's optimizing
[5:33] workloads where the AI can kind of look
[5:35] across many, many different systems and
[5:37] many signals and sources of signals and
[5:39] see patterns that humans probably would
[5:42] overlook and not understand. And the
[5:44] agility with which you can move just
[5:47] fundamentally I think the ROI equation
[5:49] for me is really more exciting even on
[5:51] the infrastructure side than the the
[5:53] coding side.
[5:55] The challenge is how do you actually
[5:56] orchestrate the agents to accomplish
[5:58] these things with all the right
[5:59] guardrails? And that that's the
[6:01] challenge is the ROI is clearly there.
[6:03] It's like how do we actually realize it?
[6:06] >> Well, and no surprise, we're going to
[6:07] ask you another question about
[6:10] tradeoffs. So, we're wondering about the
[6:12] tradeoff of guardrails versus product
[6:15] velocity itself. How are you striking
[6:17] the right balance? What kind of signals
[6:20] do you use to act upon?
[6:22] >> Yeah, this is a great question. When we
[6:23] started this this journey, we were very
[6:25] excited about the productivity aspect of
[6:28] agents. And so, we really leaned into
[6:31] enable as much as possible as quickly as
[6:33] possible.
[6:35] And we spent a lot of time measuring
[6:36] friction. And then we started to see
[6:38] that well, maybe there were some we saw
[6:41] incidents in our server reviews where
[6:43] agents did things, broke things in ways
[6:44] that were novel, unexpected. Sometimes
[6:47] they prompt humans to do things and
[6:49] those humans you know, people do those
[6:50] things.
[6:51] And then we've sort of pivoted back to
[6:53] what I think is like a much more
[6:54] balanced approach where we are
[6:56] definitely
[6:57] focusing on safety, trying to like
[7:00] target in on like the highest risk
[7:01] things that
[7:03] would be, you know, catastrophic for the
[7:05] company, especially things like data
[7:07] loss. But overall, we're taking a more
[7:10] balanced approach now, I would say,
[7:11] where we are focused on safety, but also
[7:13] measuring the friction associated with
[7:15] those guardrails.
[7:16] >> You mentioned the incident review. I
[7:18] thought it was interesting to hear about
[7:20] the incidents that we've already seen
[7:22] that are AI related.
[7:24] So, there's the one you talked about
[7:25] with the diffs that were all landed at
[7:27] scale.
[7:28] When those were landed, we have a
[7:30] question from the audience. Didn't AI
[7:32] agent also fix them? Did you trust the
[7:34] AI to fix those or
[7:36] did we just rely on humans? And what you
[7:38] know, what did we do to fix them at
[7:40] scale?
[7:41] >> I I love it if an AI agent just fixed
[7:43] it, but in reality no humans fixed it.
[7:45] And so that this is not where we want to
[7:46] be. We don't want to want We don't want
[7:48] agents breaking things and humans fixing
[7:50] them, right? So I think this is why this
[7:52] is such an interesting incident. We
[7:53] focused a lot on prevention, like
[7:55] stopping the agent from doing it in the
[7:56] first place, but where we want to get to
[7:59] is being able to deploy agents to
[8:01] actually fix these things much faster
[8:03] than humans could cuz as I mentioned it
[8:04] took us a day. It took the agent 20
[8:05] minutes to break it and it took us a
[8:07] took humans a day to fix it.
[8:11] >> Uh we have one for Joe about the
[8:16] how are agents better
[8:18] code as infra? Is it because the code is
[8:22] more declarative?
[8:24] Is it because code is more human-centric
[8:26] and has conventions?
[8:29] Yeah, we're curious about you know, why
[8:31] why does the code why is the code easier
[8:32] to write than straight-up configs?
[8:35] >> I think it's just you know, it comes
[8:36] down to the agents have been really
[8:38] trained on the internet of code and all
[8:42] this code on GitHub and it's
[8:43] self-describing and the intent is there
[8:45] for all to see.
[8:47] Not every information domain has that
[8:49] benefit. And furthermore, the frontier
[8:52] labs have been just chasing coding
[8:54] benchmarks relentlessly for 2 years and
[8:57] we've just seen the the improvement.
[9:00] And I think like interestingly when we
[9:02] first started on this, we sort of had an
[9:03] existential crisis of does code even
[9:06] matter anymore?
[9:07] And it turns out code does matter
[9:09] because if you just let an agent loose
[9:11] on calling a bunch of random CLIs and
[9:13] scripts and poking infrastructure
[9:15] without the right guardrails and intent
[9:18] codified and reviewable by humans,
[9:20] that doesn't end well. And so I think
[9:22] actually building the foundation on code
[9:24] allows agents to operate in a realm
[9:26] where they're already familiar, i.e.
[9:28] coding. And then if you have this magic
[9:30] oracle, i.e. infrastructure code that
[9:32] allows
[9:34] problems to be solved in coding land
[9:36] that then get mapped out to
[9:37] infrastructure and carry out
[9:38] infrastructure outcomes. We've seen a
[9:40] lot of success with that.
[9:42] >> So, Joe, are you experimenting with
[9:44] model fine-tuning with these agents to
[9:47] make it even better at dealing with the
[9:49] infrastructure domains? I mean, we're
[9:51] also thinking
[9:52] when it comes to new cloud providers,
[9:55] custom infra that might not be in the
[9:57] training data sets yet, but maybe
[9:58] there's an advantage to have with
[10:00] fine-tuning. Is that something you've
[10:01] tried?
[10:02] >> Definitely. Yeah, we've experimented and
[10:03] I'm very excited about that going
[10:06] forward because I think the other
[10:07] advantage to putting things in code is
[10:09] you have a type system. And now you have
[10:11] a type system over the entirety of the
[10:13] cloud configuration space. And when you
[10:15] have a type system, you can type check
[10:16] it. And you can make sure that, you
[10:18] know, it's not just like untyped YAML
[10:20] that you throw at the wall and hope that
[10:21] it works. You can actually check, oh,
[10:23] there was an integer that was supposed
[10:24] to be here or
[10:26] there was an enum and it's only these
[10:27] three legal values. And so you can you
[10:30] can kind of check that. And we have all
[10:31] of those type system kind of metadata
[10:33] across all the the clouds. We also have
[10:36] petabytes and petabytes of data of
[10:38] successful and failed configurations.
[10:41] We've actually done some classical ML
[10:42] techniques like K-means clustering of,
[10:44] you know, what what values tend to lead
[10:46] to bad outcomes versus good ones. But I
[10:49] think using that as a reinforcement
[10:51] foundation reinforcement learning
[10:52] foundation for fine-tuning models is
[10:54] something that absolutely is on the
[10:56] horizon for us.
[10:57] >> I will ask, do you have any battle scars
[11:00] from some of the any misconfiguration,
[11:02] any stories that you can tell
[11:05] uh top of mind?
[11:06] >> I mean, so many because the cloud is
[11:08] just so complicated. I mean, you know,
[11:10] we we have had agents delete production
[11:13] databases when they shouldn't and, you
[11:15] know, Kubernetes clusters that scaled to
[11:17] zero when they shouldn't have scaled to
[11:18] zero and then obviously have capacity
[11:20] problems. And
[11:21] um
[11:22] so yeah, there's a lot of infrastructure
[11:23] is hard, right? And I think that's why
[11:26] we're not yet fully on the agentic curve
[11:29] because we have to take those hard
[11:30] problems and we have to codify them in a
[11:32] way that the agents can understand and
[11:34] learn from them and prevent the mistakes
[11:35] in the first place rather than just
[11:37] reacting to them. I do think I think of
[11:38] like the fighter jet time. I do think
[11:41] there's like an extent to which the
[11:42] agents are always going to make a
[11:44] mistake just like humans always make a
[11:45] mistake and being able to very rapidly
[11:50] respond to and fix it so the humans
[11:52] aren't left holding the bag I think is
[11:54] also a key element.
[11:55] >> Yeah, I am curious right if
[11:57] the humans are actually better than the
[11:59] agents despite the mistakes that the
[12:01] agents might make right cuz we have
[12:03] years and years of humans deleting
[12:04] databases as well.
[12:06] It's kind of like a self-driving car
[12:08] right? Do you do you trust the
[12:09] self-driving car? We have a higher
[12:11] standard for the robots than humans.
[12:13] >> Totally.
[12:14] >> So Sargon we have a question back about
[12:18] the interface that you get to see as a
[12:20] developer at Meta.
[12:22] Can you tell that you're developing a
[12:24] service on public cloud versus internal
[12:26] data centers and what kind of like
[12:29] visibility you have into the differences
[12:31] between each cloud when it comes to
[12:32] custom silicon and scheduling
[12:34] availability?
[12:36] >> So um
[12:38] the answer is yes. You can tell that
[12:40] you're on the cloud but we're actively
[12:42] working towards a a world where cloud is
[12:45] not necessarily
[12:47] treated differently. And funnily enough
[12:49] we actually had like a sev recently
[12:50] where we lost a bunch of cloud capacity
[12:52] due to a unique configuration and people
[12:55] did not realize that it was cloud
[12:56] capacity and that differentiator has
[12:59] started to blur as our cloud has gotten
[13:01] more and more mature.
[13:03] That being said, yes there are places
[13:05] where the cloud has less observability
[13:08] or where the custom silicon might be
[13:09] different. But over time we've begun to
[13:12] integrate that into all of our tools.
[13:14] The same tools that you use to debug
[13:16] Twine for our own silicon or for
[13:19] merchant silicon inside of our data
[13:21] centers are beginning to work in those
[13:23] cloud facilities.
[13:24] As those vendors have started to expose
[13:26] low level access to things like the
[13:28] baseboard management controller and the
[13:30] host management controller. You can
[13:32] click the same interface and hit the
[13:33] same APIs behind the scenes that say,
[13:35] "Oh, there was a leak in this rack." or
[13:38] similar to that. And we really want to
[13:40] make sure that there's one pane of glass
[13:42] that our users can go to.
[13:43] But not only that there's one pane of
[13:45] glass, but having agents have the skills
[13:47] to be able to debug cloud just as well
[13:49] as debug the data center. And really the
[13:51] only way to get to that is either by
[13:53] giving them enough context or by giving
[13:55] them the same interfaces for data center
[13:58] capacity for a first-party party data
[14:00] center capacity as third-party data
[14:02] center capacity.
[14:04] >> So, you're imagining a situation where
[14:06] like the agent could give maybe even the
[14:08] same type of interface to human, and the
[14:10] agent can handle maybe the differences,
[14:12] or is that going to be more implemented
[14:14] directly in the infrastructure layer,
[14:16] the APIs themselves?
[14:17] >> Uh it's it's a combination of both. With
[14:20] this world where we have agents calling
[14:21] agents calling other agents,
[14:24] at the bottom of it there might be
[14:25] specialized agents for certain types of
[14:27] capacity.
[14:28] Some vendors might be even providing us
[14:31] APIs that are actually uh
[14:33] chatbots essentially or, you know, ML
[14:35] agents. And we have uh
[14:38] sub-agent dispatch that says, "For this
[14:40] given problem domain, call to the
[14:42] specialized subsystem that knows about
[14:45] this given vendor." that has a bunch of
[14:47] context, and we don't want to
[14:49] necessarily do cross-vendor pollution.
[14:52] So, it's a combination of both the API
[14:54] side and the agent side, and at some
[14:56] point they'll meet in the middle.
[14:58] >> I do love the idea of uh the interface
[15:01] to the vendor being an agent, and then
[15:02] we're sending our agents to talk to
[15:04] those agents,
[15:05] um
[15:06] you know, and then the interface
[15:07] internally is also agents, and it's just
[15:08] all the way down, you know, talking
[15:10] through agents. Um
[15:11] But I'm wondering, David, when uh
[15:14] we do talk to those agents,
[15:16] um
[15:16] how are you thinking about the risk of
[15:18] an agent that gives a faulty instruction
[15:20] to a human? Uh you know, and then the
[15:22] human goes and acts on it and you don't
[15:24] have that type of echo permission in
[15:26] place to block that.
[15:29] >> Yeah.
[15:29] I get this question a lot and my answer
[15:32] has changed over time. I
[15:33] fundamentally I think I'm an optimist
[15:35] that we will get better at vetting the
[15:37] things that our agent tells us.
[15:40] Um
[15:41] Just the way we got we have learned to
[15:42] vet things that we see on the internet
[15:44] and just do that due diligence. Um
[15:46] Earlier on when and this is just a few
[15:48] months ago, we saw more incidents where
[15:50] people would sort of like break out of
[15:51] guardrails and the agent would tell
[15:53] would would tell the human to do
[15:54] something and people would do it and now
[15:56] I think I'm seeing people be more
[15:57] critical about what they're seeing. I'm
[15:59] also seeing the models get better at
[16:01] figuring out, hey, there's a guardrail
[16:02] here and I should not work around it. So
[16:04] the the models and the tools are
[16:05] fundamentally also sort of building
[16:08] boundaries themselves. But for the
[16:10] really dangerous stuff, I think we have
[16:12] to think about human error as well as
[16:14] agent error and restrict that set of
[16:16] like
[16:17] for the truly dangerous stuff, restrict
[16:18] the set of people who can do those
[16:20] things to like the really the core group
[16:21] that we need to have those privileges
[16:23] and trust them to use agents
[16:25] intelligently.
[16:27] >> And Joe, I guess you will have agents
[16:29] that take in non-agentic data as an
[16:31] interface from different clouds,
[16:33] different providers. How are you
[16:36] teaching those agents to understand when
[16:38] maybe that data is incorrect itself?
[16:41] A cloud provider might be down or might
[16:43] give
[16:44] incorrect data back about the state due
[16:46] to a bug.
[16:47] Uh do you think the agents will be able
[16:49] to actually figure out like, hey, this
[16:51] actually seems like it might be wrong. I
[16:52] shouldn't deal with this because they
[16:54] understand how everything is going,
[16:56] right? If they all you have no capacity
[16:58] left, but actually the you know, you do
[16:59] have capacity available to run. Do you
[17:01] think agents will be able to handle that
[17:03] type of correctness debugging?
[17:05] >> I I do. I think the the tools you have
[17:07] to give it though, you have to give it
[17:08] the right context, you have to give it
[17:10] the right access to the right systems,
[17:11] you have to give it the right tools. And
[17:13] one of the things that we've
[17:15] the approach that we've taken is, you
[17:16] know, what's the difference between an
[17:18] LLM and a human platform engineer? Well,
[17:21] it's context, skills, access to the
[17:24] right tools, access to the right
[17:25] systems, so that just like a human would
[17:28] be able to verify and vet the data and,
[17:30] you know,
[17:31] go run coup control to see if like the
[17:34] cluster is in the state that it was
[17:35] supposedly reported to be in. I think,
[17:38] you know, that's been the guiding post
[17:39] for us is think of the LLM is like the
[17:41] brain, but then you have to build all of
[17:43] the stuff around it. Like we as humans
[17:45] are not just walking brains, right? We
[17:47] can do our jobs cuz we have access to
[17:48] the right systems and the right tools.
[17:50] And I think if we keep going down that
[17:51] path and the models keep getting better
[17:53] as we expect them to, then eventually
[17:56] we'll be able to solve these problems
[17:57] and and trust the agents more.
[17:59] >> Mhm.
[17:59] Nice.
[18:00] Uh we have time for, I think, one more
[18:03] question. Uh we're still accepting any,
[18:06] you know, any further questions, but
[18:07] we'll go to uh David. Wondering
[18:11] uh as these guardrails have started to
[18:12] roll out, uh how do you know if they're
[18:14] working? Because kind of proving the
[18:16] absence of issues is always a challenge
[18:19] for reliability.
[18:20] >> Yeah. This is a hard problem and we
[18:22] haven't solved it fully yet, just to be
[18:24] fully transparent. Um we fundamentally
[18:27] want agents to try to break through our
[18:29] guardrails. And so we are spinning up
[18:31] adversarial agents that will try to
[18:34] break through individual guardrails,
[18:36] collections of guardrails, delete
[18:37] production databases, access data that
[18:39] they shouldn't, and we are going to use
[18:41] those agents to sort of test the
[18:43] robustness of the things we've deployed.
[18:45] And we're already finding gaps in the
[18:47] guardrails that we're
[18:48] we're we've deployed. But this helps us
[18:50] to set up the sort of virtuous cycle of
[18:52] like find and fix. And that's what we're
[18:54] really going for.
[18:55] >> Perfect. Well, thank you to everyone who
[18:58] submitted a question. Um you'll probably
[19:00] be able to find some of the speakers
[19:02] uh out at lunch, which is coming up. Um
[19:05] and thank you to our speakers for a
[19:06] great start to the day, as well. Um
[19:08] let's give a round of applause to the
[19:10] speakers.
