Meta Deploys AI Agents to Guard Network Against Capacity Bottlenecks

Meta 部署 AI 智能体防范网络容量瓶颈 Meta 利用自主智能体对流量激增进行分流筛查并模拟快速干预方案，从而使容量规划能够抵御不可预测的需求冲击，而非依赖精准的长期预测。 Ben Vallis 解释称，由于全球并发流量事件等不可预测的需求激增，长期网络预测本身存在固有缺陷。Meta 没有盲目追求不可能实现的预测精度，而是利用 AI 智能体来优化容量布局并降低风险。该演讲概述了部署 AI 的三大核心支柱：将数千起流量异常筛查提炼为可落地的行动项、通过严格的评估闭环建立高度可信的数据，以及辅助工程人员做出决策。最终，Meta 通过跟踪规划人员的手动干预，构建起一个让人类判断力随时间推移沉淀为历史成功概率的系统。 长期容量预测存在显著的误差不对称性：过度扩容仅意味着提前投入，而建设不足则会严重制约业务，使容量规划成为典型的受限优化问题。 (0:04) 由于突发事件引发的数亿用户并发访问无法精准预测，Meta 将策略重心从追求预测精度转向战略性容量配置，以确保网络在各种极端情况下保持主动。 (0:04) 自主 AI Agent 承担了流量分诊工作，每周从数千个网络异常中筛选出极少数需要人工介入的事件，并将复杂网络调整方案的评估时间从数周压缩至数小时。 (5:26) 通过引入评估回路与上下文丰富机制，Meta 将 Agent 跨系统决策的准确率从 30% 提升至 80% 以上，并通过追踪人工决策记录，使团队经验得以长期复利沉淀。 (5:26)

Meta network planner Ben Vallis argues that because long-range demand forecasting is inherently uncertain, infrastructure strategy must pivot from pursuing predictive precision to using AI for resilient capacity positioning. Strategic Positioning Over Demand Forecasting (0:04) Forecasting network capacity years into the future involves structural error. However, Vallis explains that error directions are asymmetrical: overbuilding capacity in a high-growth business merely makes infrastructure early, whereas underbuilding creates an immediate bottleneck when traffic surges. Planners are particularly vulnerable to sudden concurrency spikes, such as the simultaneous global media consumption across the United Kingdom, Europe, Canada, and Australia following the passing of Queen Elizabeth II. Because anticipating specific geopolitical triggers is impossible, Vallis reframes capacity planning away from incremental forecast accuracy and toward strategic positioning. The objective is to distribute capacity so the network can absorb wide outcome distributions without degrading user experience. Given that component shortages and supply chain limits prevent simply building to the highest possible uncertainty thresholds, positioning becomes a constrained optimization challenge well-matched to AI. Automated Triage and Contextual Data Hygiene Meta applies AI to compress operational response timelines and resolve underlying data fragmentation. On any given week, hundreds or thousands of localized traffic shifts emerge from code updates, internet routing anomalies, or news events. Autonomous triage agents adopt workflows that screen these noisy patterns down to the handful genuinely requiring engineering oversight, allowing teams to intervene before capacity bottlenecks materialize. Deploying AI also forced Meta to address data hygiene, as divergent internal systems initially gave conflicting reports on available capacity. When reasoning across disparate data sets, early agent consistency hovered around 30%. Meta improved this to over 80% by establishing evaluation loops: testing models against realistic operational queries with known answers, standardizing definitions, and supplying richer context. Compounding Human Intuition Through Decision Traces When network events occur, engineers select from a complex slate of trade-offs, such as pulling warehouse hardware, resequencing routing paths, shaping demand, or relaxing Service Level Objectives (SLOs). AI models condense the process of costing and ranking these alternatives from several weeks into hours. When planners reject model recommendations, Meta treats the override as unencoded human judgment rather than a system defect. To preserve this institutional knowledge, Meta assigns unique tracking identifiers to planner decisions. By monitoring the long-term outcomes of human overrides, the system aims to compound institutional memory over time, eventually showing future operators the empirical historical success probabilities associated with each corrective action. Vallis concludes that while AI cannot act as a predictive crystal ball for unforeseen geopolitical events, combining rapid anomaly triage, verified data foundations, and empirical decision tracing enables the network to withstand whatever traffic demands arrive.

Meta 网络规划负责人 Ben Vallis 指出，长期容量预测必然面临不可避免的偏差，且容量短缺对业务造成的瓶颈风险远高于建设过剩；Meta 因而将策略从追求预测精度转向利用 AI 进行容量战略定位，以在供应链受限的前提下抵御极端突发流量。 容量预测的误差不对称与战略定位转型 (0:04) Ben Vallis 直言，多年期的网络容量规划每天都面临预测出错的现实，因此规划团队必须关注误差发生的方向。在高速增长的业务中，若预测过高导致容量过度建设，仅意味着投入节奏提前；然而，若预测过低导致容量不足，一旦实际需求飙升，就会成为限制整个业务发展的关键瓶颈。特别是在重大突发事件引发数亿用户同时在线并发访问（例如英国女王伊丽莎白二世逝世时，英国、欧洲部分地区、加拿大及澳大利亚等多地民众同时获取相关媒体内容）的情况下，需求激增极具破坏力。 Vallis 认为试图精准预测具体的地缘政治或现实事件是不切实际的，因此团队将核心思路从微调预测精度转向战略定位。其目标是让容量部署能够承受尽可能广泛的结果区间，避免网络陷入服务降级的被动境地。虽然理论上将容量采购拉满到不确定性区间的上限可以解决问题，但现实中面临硬件供应链约束和组件短缺的限制，这使容量定位转变为一个极其适合利用 AI 求解的受约束优化问题。 利用智能体过滤海量流量杂讯以迅速响应 在 Meta 的网络环境中，每周都会涌现数百乃至数千种由新用户群体、本地新闻、代码变更、互联网路由变动或随机噪声引发的异常流量模式。人工逐一排查这些模式耗时极长，无法满足狭窄决策窗口的时效要求。Vallis 将其定义为天然适合 AI 智能体处理的场景。 部署上线后，自主智能体展现出了类似人类工程师的排查行为，自发建立了观察清单与优先级分流流程。这套系统成功将数以千计的嘈杂网络流量模式，自动筛选过滤为仅剩极少数真正需要工程团队介入的关键事件，大幅压缩了异常响应时间，使团队得以在极紧迫的窗口内采取干预措施。 建立评估闭环以解决底层数据信任危机 引入 AI 迫使团队直面并解决基础数据治理问题。此前，向不同系统查询现有网络容量可能得到三种相互矛盾的答案。更艰巨的挑战在于，当智能体拼接不同来源的数据进行综合推理时，最初的决策一致性仅有约 30%，这种可靠度完全无法用于实际运维决策。 为建立对数据的信任，团队构建了严密的评估反馈闭环：首先利用答案已知的不同难度真实业务问题对智能体进行基准测试；随后通过标准化定义、详尽描述和扩充上下文来丰富系统认知。通过持续迭代这一评估流程，智能体处理复杂数据推理的决策准确率从初始的 30% 显著提升至 80% 以上。 加速决策推演并将人类判断转化为可复用资产 面对网络突发事件，工程师通常拥有一整套应对方案，包括从仓库调拨硬件备件、进行需求整形、放宽服务等级目标（SLO）或重新调整在途流量路由。以往人工权衡这些动作的成本、风险与收益往往需要耗费数周，而 AI 工具已将该综合分析与排序过程缩短至数小时之内。 更为关键的是，当工程师否决 AI 推荐的方案时，Vallis 并不将其视作算法失败，而是视为尚未被编码的宝贵人类经验与判断。Meta 正在通过为规划师的每项决策标注唯一标识符来追踪决策链路与长期结果。该构想旨在让工程师的判断力随时间累积沉淀为组织资产，避免因人员离职而重置；未来的方案清单将直接展示历史成功概率，例如明确提示在过往类似场景下选择某一应对路径曾达到 70% 的成功率。 Ben Vallis 总结强调，未来世界的突发需求依然无法精准预知，但通过将 AI 定位为优化执行效率、夯实数据一致性并复利沉淀人类经验的工具，工程团队能够在供应链受限的现实条件下，使网络具备应对任何未知波动的弹性与定位能力。

## Background 该视频由技术分享频道 @Scale 于 2026 年 9 月 3 日发布在 YouTube 平台上，内容为 Meta 网络规划资深专家 Ben Vallis 的闪电演讲。视频简介指出，面对长期网络容量预测必然出错且伴随供应链冲击的工程现实，演讲探讨了如何利用 AI Agent 转变思路——不再苦苦追求边际预测精度，而是通过狭窄窗口快速分诊、构建可信数据闭环以及权衡多重决策杠杆，打造一套让组织判断力持续复利的自学习容量规划系统。 ## TL;DR Meta 网络容量规划团队彻底颠覆了传统预测思路，将核心战略从“追求预测准确”转向“优化容量定位”，以确保在遭遇极端流量并发冲击与供应链短缺时永远不被逼入绝境。针对该受约束优化问题，团队通过 AI Agent 在三个维度实现了能力跃迁：首先是自主建立流量异动分诊机制，将海量异常排查耗时从数周压缩至数分钟，抓住狭窄的调整窗口；其次是通过“基线-富化-重测”的 Eval 闭环，将 Agent 跨异构多数据源推理的准确率从初期的 30% 提升至 80% 以上；最后是自动生成覆盖需求整形、SLO 放宽等复合手段的成本化决策菜单，并通过对人类决策轨迹插桩，使工程专家直觉沉淀为系统可复利的胜率模型。 ## Atmosphere & style 整场演讲干练紧凑、务实清醒，展现出大型基础设施架构师独有的工程底气——开篇便直率断言“规划者每天醒来都心知注定会预测错误”，瞬间击碎对 AI“全知水晶球”的虚幻期待。Ben Vallis 毫不避讳系统最初跨源推理仅有 30% 准确率以及指标定义打架的狼狈底细，以一种坦率且极度理性的叙事节奏抽丝剥茧。从承认预测宿命的现实无奈，到用 Eval 闭环与决策插桩反戈一击，整场分享在极度受约束的物理现实与高度自适应的智能进化之间找到了精妙的平衡，传递出浓厚的硬核工程哲学与从容克制的科技掌控感。 ## Bulletin - [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 容量规划核心理念发生质变：长远预测注定会错，团队不再追求微弱的预测精度提升，而是转变为受约束优化下的容量“定位”（Positioning），确保在遭遇突发高并发冲击时永不被逼入绝境。 - [03:26](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=206s) 引入 AI Agent 自动分诊异常流量：通过自主构建监控列表与排查工作流，将每周数百起异动收敛至极少数核心问题，将研判响应周期从数周大幅压缩至分钟级。 - [04:30](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=270s) 通过 Eval 评估循环攻克数据信任危机：面对早期跨源推理仅 30% 的准确率以及定义口径冲突，团队借助语义富化与评测闭环，将生产级推理准确率提升至 80% 以上。 - [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 建立决策插桩机制沉淀人类判断力：AI 将多维杠杆权衡推演时间从数周缩减至数小时，并通过唯一标识符记录专家的真实选择与长期成效，让决策经验在系统中实现复利积累。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 网络容量规划因涉及数年前瞻预测而注定存在数值偏差，盲目追求预测精度是徒劳的，必须学会接受大幅度误差的现实。 - [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 容量规划对向上突增的意外极度敏感：预测偏高仅代表投资过早，而预测偏低且需求暴涨则会直接成为业务发展的致命瓶颈。 - [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 突发事件引发的高并发流量冲击（如女王逝世）几乎无法通过算法提前预测，属于不可预知的现实扰动。 - [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 受制于供应链瓶颈与硬件交付周期，团队无法盲目按预测上限采购，容量配置本质上属于强约束条件下的优化问题。 - [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) Meta 团队将核心哲学转向容量“定位”（Positioning），核心目标是覆盖最广泛的容错区间，避免因容量配置失误将业务逼入死角。 - [03:26](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=206s) 面对代码发布、路由波动或突发事件等多源流量异动，AI Agent 自发形成监控观察列表与分诊逻辑，极大地释放了资深工程师的研判精力。 - [04:30](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=270s) 数据标准化与定义一致性是自动化决策的前提，早期由于口径冲突，针对同一网络库存指标系统曾返回 3 种互斥答案。 - [04:30](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=270s) 跨异构数据源执行端到端推理时，初始 Agent 的答题准确率仅为 30% 左右，完全无法胜任重资产投资决策。 - [04:30](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=270s) 通过引入端到端 Eval 评估循环，配合上下文语义富化与统一定义重测，系统复杂推理准确率跃升至 80% 以上的可用水准。 - [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 容量调控拥有多种权衡杠杆，包括调集备件库存、产品端需求整形（Demand shaping）、放宽服务等级目标（SLO）以及重排在途路由交付顺序。 - [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) AI 能在数小时内生成标明成本、收益与风险的可选方案菜单，取代过去耗时长达数周的人工推演计算。 - [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 人类工程师未采纳 AI 首选推荐方案并非算法失败，而是反映了尚未被系统编码提炼的专家隐性经验。 - [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 通过为每次人工决策注入唯一标识符并持续跟踪其实际影响，决策轨迹得以系统化沉淀为可复利的统计胜率认知。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 预测的宿命与容量定位哲学：从“准确预测”转向“保持抗脆弱” 网络容量规划（Network Capacity Planning）中最艰难的现实在于：规划者每天清晨醒来都心知肚明自己注定会预测错误。因为这项工作必须针对未来数年进行长远预测，但决定容量今天到底部署在何处却刻不容缓。既然未来本质上充满不确定性，规划者势必会在数值上大幅偏离实际情况。多年以来，规划团队必须学会接受并习惯这种“大幅度的错误”，但随之而来的关键问题变成了：“究竟朝哪个方向犯错？” 如果建立了一个极高的需求预测，并据此构建了充裕的容量，但实际需求远低于预测，在一家高增长的企业中这其实并非世界末日——它充其量意味着“我们来早了”（容量提前就位）。然而，如果预测偏低、容量建设不足，而实际需求激增反超容量上限，情况就会急转直下，甚至彻底扼死业务的发展瓶颈。因此，容量规划在实际操作中对向上突增的意外情况（Upside Surprises）极度敏感。这种意外的核心往往是高并发（Concurrency）：当数千万甚至数亿用户在同一时刻涌入网络并消费相似的内容时，流量会形成恐怖的峰值。以英女王伊丽莎白二世逝世（The Queen's passing）为例，英国全境、欧洲部分地区、加拿大、澳大利亚等多国用户几乎在同一时刻连线查看相关多媒体资讯。预测这类地缘政治或突发事件无异于“愚人追梦”（Fool's errand）。因此，Meta 团队选择将问题彻底颠倒过来：不再苦苦追求边际预测精度的微弱提升，而是转变为“定位”（Positioning）——既然明知预测必错，如何部署容量才能覆盖最广泛的容错范围，抵御最大范围的不确定性结果，核心目标就是“永不被逼入绝境”（Never get cornered），绝不因容量定位失误而被迫降低用户体验或业务成果。 面对不确定性锥形图（Cone of uncertainty），最直接的直觉或许是“直接按照上限无脑顶格采购”。但这在现实中根本不可行：任何经历过供应链危机或元器件短缺的工程师都知道，现实中永远无法随心所欲拿到所需硬件。因此，容量配置本质上是一个受约束的优化问题（Constrained optimization problem）。AI 在该领域的价值，并非提供一个能精准预见未来的“水晶球”，而是在约束条件下显著强化团队的容量定位能力。具体体现在三个维度：及时行动（Act in time）、修复并建立对数据的信任（Trust our data）、以及增强人类决策判断力（Augment judgment）。 ```mermaid flowchart TD accTitle: Capacity Planning Decision Mindset Shift accDescr: 比较传统预测模式与定位策略在面对未来不确定性时的不同结果与应对机制 subgraph Trap["传统思路：追求预测精度"] A["多年前瞻性预测"] --> B{"预测偏差方向"} B -- "预测过高" --> C["容量闲置 / 仅为提前部署 (低风险)"] B -- "预测过低" --> D["高并发冲击 (如女王逝世)"] D --> E["业务瓶颈 / 体验降级 / 陷入绝境"] end subgraph Strategy["定位策略：AI 驱动的受约束优化"] F["承认预测必错"] --> G["定位 (Positioning) 容量布局"] G --> H["追求抵御最广结果范围 (Robustness)"] H --> I["保持机动性，避免逼入死角"] end ``` - **演讲者背景**：Ben Vallis，在 Meta 从事网络建设与容量规划工作已有近十年（the better part of 10 years）。 - **核心风险逻辑**：预测过高只是“过早投资”；预测过低则直接“锁死业务”（Bottleneck the business）。 - **供应链现实**：硬件短缺与交付周期限制决定了无法直接“买到预测上限”，必须在物理与预算约束下寻找最优解。 --- ### [03:26](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=206s) 维度一：在窄时间窗口内及时行动，Agent 化繁为简的流量排查 AI 帮助改进容量定位的第一种方式，是确保团队能够在狭窄的决策窗口内“及时行动”（Act in time）。在 Meta 的网络上，每周都会冒出成百上千种前所未见的异常流量模式（Traffic patterns）。针对每一个异动，规划团队都必须深入调查并理解其背后的根因：它究竟代表着某一新用户群体的崛起，某个地区性突发新闻事件，某次代码部署变更，公网上的随机路由异常，抑或仅仅是一场毫无意义的随机噪点？判断这些异常极其耗费时间，但又不得不逐一排查。 这恰恰是一个与 AI Agent 特性完美契合的问题（A perfectly agent-shaped problem）。当把 Agent 投入到这一排查流程中时，其行为表现出惊人的人性化特质：Agent 开始自主建立监控观察列表（Watch lists），形成一套自己偏好的分诊与排查逻辑流程。这一机制将每周成百上千个需要工程师介入调查的流量模式，自动精简并聚合到仅仅少数几个真正需要资深工程师投入精力关注的核心异动。通过这种方式，团队的研判周期从以往的数周大幅压缩，从而使网络工程团队有能力在极其严苛的时间窗口内作出容量调整。 - **排查复杂度**：流量异动诱因极其多维（新用户群、代码发布、公网路由扰动、突发事件、随机噪点）。 - **Agent 行为演变**：AI Agent 自发学会了建立 Watch list，形成了类似人类工程师的分诊（Triage）工作流。 - **业务收益**：滤除海量背景噪点，将数以千计的潜在告警收敛至关键少数，赋予团队在狭窄买入/调整窗口内果断出击的响应能力。 --- ### [04:30](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=270s) 维度二：攻克数据信任危机，通过 Eval 闭环将推理准确率从 30% 提升至 80%+ AI 赋能的第二项关键变化，在于它倒逼团队正视并清理工作流中最不光鲜却最承重的部分——数据质量与定义一致性。就在数月前，如果向不同的 Agent 提问 Meta 当前拥有多少网络库存或可用容量，系统可能会给出三种截然不同的答案。虽然清理这些基础的承重资产（Loadbearing data）如今已成为团队必须达标的基本盘，但更深层次的挑战在于：不仅底层数据需要标准化，在这些数据之上进行逻辑推理的 Agent 是否值得信赖？ 真正的痛点发生在 Agent 需要跨越多个不同数据源进行拼接并执行端到端推理之时。最初测试时团队震惊地发现，Agent 能够始终如一推导得出正确答案的比例仅仅只有大约 30%。对于一个必须依赖其结果进行重资产投资决策的基础设施系统而言，30% 的准确率是完全无法接受的。为了彻底扭转这一局面，团队构建了一套严密的评测评估循环（Eval loops）：首先拿出团队已知确切答案、涵盖不同复杂度维度的真实业务问题对 Agent 进行基线打标（Baselined）；随后进行数据与语义富化（Enrich），补充详细的上下文、字段描述以及全团队通用的共享定义；然后重新测试、验证。通过这一轮又一轮的“循环往复”（Rinse and repeat），系统在复杂跨源推理问题上的准确率在极短时间内从最初的 30% 飙升至 80% 以上。 ```mermaid flowchart LR accTitle: Eval Loop for Data Reasoning Accuracy accDescr: 展示将 Agent 推理准确率从 30% 提升到 80% 以上的评估与语义富化循环流程 A["已知答案真实业务考题<br/>(涵盖多级复杂度)"] --> B["基线性能测评<br/>(初始正确率 ~30%)"] B --> C["上下文语义富化<br/>(补充字段描述/共享统一定义)"] C --> D["重新评测与校验<br/>(Eval 循环往复)"] D --> E["高置信推理达成<br/>(准确率提升至 80%+)"] E -. "持续迭代".-> A ``` - **初始混乱状态**：早期询问同一库存/容量指标，由于定义不统一，系统会返回多达 3 种互相冲突的口径。 - **多源推理痛点**：跨异构系统拼接上下文时，原始基准下 Agent 仅有约 30% 的时间能给出正确分析。 - **治理方法论**：构建端到端 Eval Loop，通过注入上下文、标准化定义与严谨复测，成功将可靠性推至 80% 以上的生产级可用水平。 --- ### [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 维度三：生成决策菜单，沉淀决策轨迹实现人类判断力的复合增值 第三个关键维度在于利用 AI 协助团队挑选最正确的应对动作。即便有了精准的预警和及时行动的窗口，具体采取什么应对手段依然是极度复杂的战略权衡。在任何一个具体的容量紧平衡决策节点上，团队手中的工具杠杆（Levers）不止一种：可以从备件仓库调集硬件，可以通过产品端手段进行流量整形（Shape demand），可以主动放宽某项网络服务等级目标（Relax SLO），也可以对正在运输交付途中的网络资源进行重排或路径重新路由。这些动作各具不同的收益、财务成本与潜在风险。如果完全依靠人工手动推演各方案的性价比，往往需要耗费数周时间；而在 AI 辅助下，几小时内就能生成一份权衡完备的可选行动方案菜单（Menu of options）。 更具战略意义的突破发生在这个菜单呈现之后。实践中，人类工程师并不总是完全采纳 AI 排名推荐的第一选项。Ben Vallis 指出，这绝非算法的缺陷，而是一份“珍贵的礼物”——这代表着人类专家掌握着尚未被系统编码提炼的隐性认知，是真正的“人类判断力”（Judgment）。为了不让这些无形资产随个别工程师的更迭而流失，Meta 开始为工程师做出的每一项决策注入唯一的决策标识符（Identifiers），全面对决策轨迹（Decision traces）进行插桩追踪，记录当时选了什么、背后的上下文是什么、以及长期产生的实际影响。这种机制使分散的人类智慧得以在系统内不断复利累加（Compounds）。其最终构想是：在未来面对同样的决策菜单时，每个门背后都会标注经过历史验证的胜率概率（例如：“如果选择 2 号门方案，历史回溯表明其在 70% 的情境下是最优解”）。 面对未知的未来，长程网络规划依然无法摆脱必然犯错的宿命，但随着 AI 将每一次犯错和纠偏沉淀为可学习的系统认知，容量布局将真正具备自适应进化的生命力。 - **多维干预手段**：仓库调配、需求整形（Demand shaping）、放宽服务等级目标（SLO relaxation）、重排在途路由与交付顺序。 - **时效对比**：方案推演成本从过去的手工“数周”直接压缩至“数小时”。 - **判断力插桩**：通过标识符系统化追踪工程师决策与长期真实效果，将不可言传的工程直觉转化成可复利的统计胜率看板。 ## Notable quotes > Network capacity planning is high-stakes and unforgiving: capacity is committed years ahead, every long-range forecast is guaranteed to be wrong, and the costliest surprises are correlated demand shocks that defeat traditional risk-pooling. — Ben Vallis > Much of the pipeline is already automated — but the decisions that take real judgment have stayed manual, and those are the ones that decide whether a wrong forecast corners you. — Ben Vallis > The real frontier is a plan that learns — agents that remember every decision and how it turned out, so human judgment compounds over time instead of resetting with each planner. — Ben Vallis > [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 网络容量规划中最艰难的现实在于：规划者每天清晨醒来都心知肚明自己注定会预测错误。 — Ben Vallis > [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) 不再苦苦追求边际预测精度的微弱提升，而是转变为“定位”——既然明知预测必错，如何部署容量才能覆盖最广泛的容错范围，抵御最大范围的不确定性结果，核心目标就是“永不被逼入绝境”。 — Ben Vallis > [00:04](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=4s) AI 在该领域的价值，并非提供一个能精准预见未来的“水晶球”，而是在约束条件下显著强化团队的容量定位能力。 — Ben Vallis > [03:26](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=206s) 这恰恰是一个与 AI Agent 特性完美契合的问题。 — Ben Vallis > [04:30](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=270s) 系统在复杂跨源推理问题上的准确率在极短时间内从最初的 30% 飙升至 80% 以上。 — Ben Vallis > [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 人类工程师并不总是完全采纳 AI 排名推荐的第一选项。Ben Vallis 指出，这绝非算法的缺陷，而是一份“珍贵的礼物”——这代表着人类专家掌握着尚未被系统编码提炼的隐性认知，是真正的“人类判断力”。 — Ben Vallis > [05:57](https://www.youtube.com/watch?v=2jwKkYLbCIo&t=357s) 如果选择 2 号门方案，历史回溯表明其在 70% 的情境下是最优解。 — Ben Vallis ## Mentioned Ben Vallis, Meta, @Scale, 伊丽莎白二世 (Queen Elizabeth II), AI Agent, SLO, Eval Loop ## Video description Network capacity planning is high-stakes and unforgiving: capacity is committed years ahead, every long-range forecast is guaranteed to be wrong, and the costliest surprises are correlated demand shocks that defeat traditional risk-pooling. Much of the pipeline is already automated — but the decisions that take real judgment have stayed manual, and those are the ones that decide whether a wrong forecast corners you. This talk is about applying AI to exactly those decisions — not to chase a more accurate forecast, but to position capacity so it keeps our options open no matter how demand breaks. We show how agents are beginning to crack the three that matter most: acting inside narrow buy windows (triaging signals continuously to surface the handful of truly novel sites, weeks down to minutes), trusting data across systems that disagree on even basic definitions (a measure-enrich-regrade loop), and choosing among coupled, timing-sensitive levers (a ranked, costed menu of moves with the reasoning attached). The real frontier is a plan that learns — agents that remember every decision and how it turned out, so human judgment compounds over time instead of resetting with each planner. Learn more about the @Scale conference here: https://atscaleconference.com/ [Source](https://www.youtube.com/watch?v=2jwKkYLbCIo)

Ben Vallis: Network capacity planning is high-stakes and unforgiving: capacity is committed years ahead, every long-range forecast is guaranteed to be wrong, and the costliest surprises are correlated demand shocks that defeat traditional risk-pooling. Ben Vallis: Much of the pipeline is already automated — but the decisions that take real judgment have stayed manual, and those are the ones that decide whether a wrong forecast corners you. Ben Vallis: The real frontier is a plan that learns — agents that remember every decision and how it turned out, so human judgment compounds over time instead of resetting with each planner. [0:04] Ben Vallis: 网络容量规划中最艰难的现实在于：规划者每天清晨醒来都心知肚明自己注定会预测错误。 [0:04] Ben Vallis: 不再苦苦追求边际预测精度的微弱提升，而是转变为“定位”——既然明知预测必错，如何部署容量才能覆盖最广泛的容错范围，抵御最大范围的不确定性结果，核心目标就是“永不被逼入绝境”。 [0:04] Ben Vallis: AI 在该领域的价值，并非提供一个能精准预见未来的“水晶球”，而是在约束条件下显著强化团队的容量定位能力。 [3:26] Ben Vallis: 这恰恰是一个与 AI Agent 特性完美契合的问题。 [4:30] Ben Vallis: 系统在复杂跨源推理问题上的准确率在极短时间内从最初的 30% 飙升至 80% 以上。 [5:57] Ben Vallis: 人类工程师并不总是完全采纳 AI 排名推荐的第一选项。Ben Vallis 指出，这绝非算法的缺陷，而是一份“珍贵的礼物”——这代表着人类专家掌握着尚未被系统编码提炼的隐性认知，是真正的“人类判断力”。 [5:57] Ben Vallis: 如果选择 2 号门方案，历史回溯表明其在 70% 的情境下是最优解。

[0:04] Ben Vallis: I have to wake up every morning and I know that I'm going to be wrong. [0:04] Ben Vallis: fool's errand [5:26] Ben Vallis: perfectly agent-shaped problem [5:26] Ben Vallis: information we have not yet been able to encode—and it's really truly judgment

[0:04] Let's talk about the plan that learns. [0:07] One of the hardest parts about my job is [0:08] that I have to wake up every morning and [0:10] I know that I'm going to be wrong. And [0:12] why is it that I know that I'm going to [0:13] be wrong? Well, it's because as part of [0:16] planning the network, I have to forecast [0:17] many, many years into the future. Yet, I [0:20] still have to make decisions about how [0:21] we're going to place capacity today. And [0:23] inherently, the future is very [0:25] uncertain. And inherently, I'm going to [0:27] just be very wrong. So over the years [0:29] I've had to get comfortable with being [0:30] very wrong on this. And the real [0:33] question for me then becomes wrong in [0:35] which direction. So think with me about [0:37] this for a second. If I were to build a [0:39] forecast that is very very high and I [0:42] build capacity to that forecast but [0:44] demand comes in below it. Well, it's [0:46] really not the end of the world, [0:48] especially in a high growth business. It [0:50] really just means I'm early. But if I [0:52] build a low forecast and I build [0:53] capacity low and demand materializes [0:56] above it, oh man, that's a real problem, [0:59] right? I can bottleneck the business. [1:02] And so what that really means in [1:03] practice is that we become very [1:05] sensitive to surprises. [1:07] So what are those surprises? What are [1:09] the things that can catch us to the [1:10] upside? In a word, it's really [1:12] concurrency. It's when tens of millions [1:14] or even hundreds of millions get on of [1:16] people get online at the same time and [1:18] do the same thing. A great example of [1:21] this would be the queen's passing. This [1:23] is when all of the United Kingdom, parts [1:25] of Europe, Canada, Australia, number of [1:28] other countries all get online at the [1:29] same time and all look at similar types [1:31] of media. And over the years, I've had [1:34] to realize that it's a bit of a fool's [1:36] errand to try and predict these [1:37] geopolitical events. Instead, I've [1:40] learned to do something else, and that's [1:43] positioning. [1:45] Really, what this means is I flip the [1:46] question on its head. I say not how do I [1:49] get better at predicting, how do I [1:50] become incrementally better at [1:52] forecasting as noble of a goal as that [1:54] is. I say instead given that I'm going [1:56] to be wrong, how do I place my capacity [1:58] such that I can cover the widest range [2:00] of wrongness? Said differently here, [2:03] that means hey, how do I position [2:05] capacity such that I can withstand or be [2:07] robust to the largest range of outcomes? [2:10] And really, the goal of all of this is [2:12] to just never get cornered. I never want [2:14] to be in a spot where I have to degrade [2:16] user outcomes or user experience because [2:18] I failed to position my capacity [2:19] correctly. Now, at this point, you might [2:22] say, "Hey, Ben, guess what? I know how [2:24] to solve your problem. That code of [2:26] uncertainty you just showed me a couple [2:27] slides ago, just buy a bunch of stuff. [2:29] Buy to the top of that." And I'm with [2:31] you. I agree. I think it's a great [2:32] strategy, except for in reality, it's [2:35] not feasible. Any of you that have dealt [2:37] with supply chain issues or component [2:39] shortages realize you can never get [2:41] exactly what you want. And so the [2:43] problem space here really is a [2:44] constrained optimization problem. And [2:47] it's on that where I'm seeing a ton of [2:49] benefit from AI. [2:51] And so that really forms the balance of [2:53] my talk here today. I'm not going to [2:54] talk about how my crystal ball is [2:56] becoming less cloudy because of AI or I [2:58] can forecast better because of it. That [2:59] may be possible. But really where I'm [3:01] seeing purchase now, where I'm seeing [3:03] lots of benefits now is from AI helping [3:06] us position better. And it's helping us [3:08] position better in three specific ways. [3:10] One, it's helping us to act in time. [3:12] Two, it's forcing us to fix up and trust [3:14] our data more. And three, it's [3:16] augmenting our judgment. Real quickly [3:19] here, I'm Ben Val, and I've been [3:21] building our network and planning it for [3:22] the better part of 10 years. [3:26] So, the first way that AI is helping us [3:28] position our capacity better, well, it's [3:30] helping us to act in time. What do I [3:32] mean by this? Any given week, we have [3:34] hundreds, if not thousands of traffic [3:36] patterns emerging on our network. And [3:38] for each one of these, we need to [3:40] investigate and understand why. And the [3:42] reasons could be many. It could be a new [3:45] user cohort is showing up. It could be [3:48] that there's some local news event. It [3:50] could be a code change. It could be some [3:52] random routing thing on the internet. Or [3:54] it could just be random. It's hard to [3:56] tell, but each one requires [3:57] investigation, and it takes time. [4:00] This turns out to be a perfectly [4:02] agent-shaped problem. We let the agents [4:05] loose on this, and it's fun to watch [4:06] what they do. they actually act very [4:08] human. They start creating watch lists. [4:10] They start creating preferred ways of [4:11] triaging and investigating things. And [4:14] what it's led to for us is it's [4:15] consolidated those thousands or hundreds [4:17] of traffic patterns that need [4:19] investigation down to just the handful [4:21] that really require the time of our [4:23] engineers. And by virtue of doing this, [4:25] it enables us to act within very narrow [4:27] decision windows. [4:30] The second way that AI is helping us [4:32] position our capacity better is it's [4:34] forcing us to deal with some of the [4:35] unglamorous parts of our workflows, [4:37] specifically around data. A few months [4:40] ago, if I had asked an agent how much [4:42] inventory or how much capacity we have, [4:44] I might have gotten three different [4:46] answers. And now cleaning up these sort [4:49] of loadbearing things, it's just table [4:51] stakes for us. It's wonderful that we're [4:53] getting it done. But the problem in my [4:56] mind transcends this. [4:58] The real problem comes down to not just [5:01] having good base data to work on, but [5:03] can I trust the agents that are [5:05] reasoning over those data? And so when I [5:07] ask a question to an agent, the real [5:09] hard thing comes into play here is when [5:12] the agent has to stitch together a bunch [5:13] of different data sources and start [5:15] reasoning. Can it do it consistently? [5:17] Well, what we found when we started this [5:19] out was that about 30% of the time the [5:22] agents were able to consistently get to [5:24] the right answer, which is unacceptable [5:26] for building something you need to trust [5:28] and act on. So, we dealt with this by [5:31] building eval loops. We asked it real [5:33] questions of varying degrees of [5:35] complexity to which we knew the answers. [5:37] We baselined. We then enriched by adding [5:39] context and descriptions and shared [5:41] definitions. We rinsed and repeated. We [5:44] did the cycle over and over and over. [5:46] We've been able to go from 30% accuracy [5:49] up to 80% plus accuracy very quickly [5:53] depending on the complexity of the [5:54] question. [5:57] The third way AI is helping us to [5:59] position our capacity better is it's [6:01] helping us to pick the right move. You [6:03] remember a couple slides ago I talked [6:04] about acting in time. But that still [6:06] means we have to take the right action. [6:08] And what action is correct? Well, that's [6:10] a hard question. At any given point or [6:13] any given decision point, we could do a [6:15] number of things. We could pull supply [6:16] from the warehouse. We could shape our [6:18] demand. We could relax a service level [6:21] objective. We could resequence or [6:23] reroute things that are in flight. All [6:26] of these have different benefits, costs, [6:27] and risks. And it's very tricky to [6:29] understand which one is right. And [6:32] today, it would take weeks by hand to do [6:34] this. With AI, it enables us to do this [6:36] in a matter of hours. But it's good. And [6:40] I'll be honest about this. It's not [6:41] perfect. And here's why. [6:45] When that menu of options shows up, what [6:48] we're seeing is that our engineers are [6:50] not always taking the AI recommended [6:52] option. And to me, I don't see that as a [6:55] problem. I see it as a gift. I see it as [6:58] information we have not yet been able to [7:00] encode. And it's really truly judgment. [7:03] And so what we're doing is we are [7:04] instrumenting decision traces. We're [7:07] putting identifiers on every single [7:08] decision our engineers are making so [7:10] that we can understand the impact of [7:12] those over time. And as we rinse and [7:15] repeat and do it over and over, that [7:17] judgment compounds. Our hope is that a [7:20] year from now, we can sit down and look [7:22] at that same menu of options. And [7:24] attached to each one of those will be a [7:26] percentage. And that percentage will [7:28] tell you that, hey, you know what? If [7:30] you go through door number two, 70% of [7:33] time that was the right choice in the [7:35] past. If this is something that's [7:37] interesting to you, is a very tricky [7:39] problem, come talk to me. [7:42] So, I still wake up in the morning. I'm [7:46] still a little bit bummed that I'm going [7:47] to be wrong. I just know I'm going to be [7:49] wrong. The future is as uncertain as [7:51] it's ever going to be. And I still think [7:53] positioning positioning ourselves [7:55] against the widest range of outcomes is [7:57] the right approach here. But what I'm [7:59] really excited about is how AI is [8:01] helping us to position for whatever [8:03] comes next. Thank you.