Building the Cloud for an Agentic World | AWS CEO Matt Garman

为智能体世界构建云基础设施 | AWS首席执行官马特·加曼 AWS首席执行官马特·加曼阐述了云基础设施如何不断演进，以适应自主软件智能体而非人类开发者，这推动了快速账户开通、Firecracker微虚拟机执行沙箱以及上下文数据层等创新。在2026年史无前例的2200亿美元资本支出计划背景下，AWS在平衡电网、先进封装和芯片等物理供应链瓶颈的同时，特意为早期初创公司保留了稀缺的英伟达GPU以及定制Trainium加速器算力。加曼还概述了AWS如何通过Bedrock和Continuum等服务，以机器速度解决企业采用智能体时面临的数据主权和安全障碍，并分享了亚马逊内部团队如何采用智能体优先的工程工作流，从而大幅缩减核心团队规模。 AWS计划在2026年进行约2200亿美元的资本支出，并拟在未来几年采购200万颗英伟达GPU，以支持生成式人工智能和智能体云的需求。 (0:00) 为防止被超大规模企业垄断，AWS特意避免向前沿实验室全额分配GPU，而是专门为初创公司保留了英伟达和Trainium算力的专用资源池。 (16:00) AWS推出了一种无摩擦的开户工作流程，支持通过Gmail身份验证、自动化默认配置且无需预先绑定信用卡，在30秒内即可开通完整的生产账户。 (10:00) AWS排名前100的客户中，有超过90%目前在定制的Graviton芯片上运行工作负载；与标准的x86替代方案相比，该芯片能降低20%的成本并提升20%的性能。 (35:00) 在内部，AWS已转向智能体优先的软件工程模式；在人类管理下由智能体生成代码，将构建每个服务所需的团队规模从10人减少至3到4名工程师。 (52:49)

AWS CEO Matt Garman argues that the rise of autonomous software agents requires a fundamental re-architecture of cloud computing, spanning low-latency developer environments, custom silicon, and massive physical infrastructure investments. Adapting Cloud Primitives for Agentic Workflows (6:15) As software agents replace human developers as primary infrastructure operators, AWS is re-engineering its setup and runtime environments. To remove initial friction, AWS introduced an onboarding workflow allowing users to register via Gmail without a credit card and have virtual private clouds and IAM roles pre-configured in under 30 seconds. Because agents execute cascading tool calls where tail latency causes bottlenecks, Garman highlights the need to eliminate P99.9 latency spikes and previewed AWS Context, an abstraction layer designed to let agents query data across Amazon S3 data lakes and Aurora databases seamlessly. Agent-driven infrastructure also challenges traditional assumptions around permanence. While human engineers build for multi-Availability Zone durability, agents frequently require transient, disposable resources, such as spinning up an analytical database for several minutes before destroying it. To isolate untrusted agent execution contexts quickly without the overhead of standard virtual machines, AWS relies heavily on its Firecracker microVM technology, which boots in milliseconds while preserving hardware-level isolation. Capital Deployment and Physical Bottlenecks (16:00) To satisfy generative AI and agent demand, Amazon plans $220 billion in capital expenditures for 2026 and aims to acquire two million NVIDIA GPUs over the coming years. Garman notes that the unprecedented demand temporarily disrupted AWS's historical model of maintaining elastic capacity buffers ahead of demand. He rejects concerns about an AI bubble damaging AWS, citing single-digit maximum customer concentration and the shift of enterprise spending into high-ROI production inference workloads. Garman explains that physical scaling constraints shift continuously across power availability, TSMC fabrication capacity, High Bandwidth Memory, and data center facilities. Meeting these constraints has forced AWS into long-term infrastructure forecasting through 2028 and beyond. AWS now invests directly in nuclear and solar energy projects—using both behind-the-meter generation and grid-interconnection credit models—while using predominantly free-air cooling to reduce water consumption in local communities. Custom Silicon and the Economics of Compute (32:20) AWS’s custom hardware roadmap originated over a decade ago with the acquisition of Annapurna Labs to eliminate the host CPU 'virtualization tax.' This led to the Nitro System, which offloads networking, storage, and management virtualization to dedicated cards, freeing 100% of host compute for customers. The ARM engineering developed for Nitro subsequently yielded the Graviton server CPU, which now accounts for more deployed units annually than any other processor family in AWS data centers, delivering an estimated 20% lower cost and 20% higher performance compared to x86 alternatives. For machine learning, AWS developed its Trainium and Inferentia architectures. Trainium capacity is currently committed through late next year, with the chip handling the majority of inference traffic on Amazon Bedrock due to the large memory and interconnect configurations demanded by frontier models. To sustain broader ecosystem growth, Garman notes that AWS intentionally avoids selling all accelerator capacity to hyperscale tech firms, instead reserving dedicated allocations of Trainium and NVIDIA GPUs for early-stage AI startups. Enterprise Trust, Governance, and Organizational Redesign (40:00) Enterprises remain cautious about autonomous agents due to security risks like unintended production data deletion, data leakage, and external code injection. In response, AWS emphasizes Bedrock's VPC boundary guarantees, preventing enterprise data from leaving the customer's perimeter or being used to train third-party foundation models. To help organizations establish evaluation pipelines, prevent drift, and systematically label data, AWS deploys Forward Deployed Engineering teams under a 45-day enablement model designed to transfer full operational ownership to the customer. Internally, AWS has deployed its Amazon Q agent across all corporate employees, allowing non-technical staff in finance and resource planning to bypass engineering backlogs and automate multi-jurisdictional compliance tasks. Within AWS's core software engineering teams, adoption has evolved into an 'agent-first' approach where autonomous agents generate code under the supervision of human engineers. Garman reports that feature teams historically requiring ten engineers are launching comparable systems with three to four, creating new management challenges around fluidly reassigning staff once builds are complete. While agentic systems offer dramatic productivity gains and operational speed, their broad deployment remains constrained by enterprise trust, rigorous evaluation standards, and hardware supply ceilings across global energy grids and silicon fabrication.

AWS 首席执行官 Matt Garman 指出，云计算正经历从面向人类开发者向支撑自主智能体（Agent）的历史性架构转变。为此，亚马逊正推进规模空前的资本开支（计划 2026 年达 2200 亿美元），并通过底层瞬态虚拟化、自研芯片矩阵与严格的数据治理护栏，全面重构云基础设施。 智能体云架构：瞬态资源与低延迟交互 (6:15) 传统云基础设施主要围绕高可用性、多可用区持久性和人类工程师的工作节奏构建，而自主智能体更倾向于频繁调用即用即弃的瞬态资源，例如启动数据库分析数分钟后立即销毁。Garman 表示，AWS 正在重塑架构以承接此类工作负载，利用 Firecracker 微虚拟机提供毫秒级冷启动与轻量级安全沙箱隔离，并大力优化后端 P99.9 尾延迟，防止智能体在执行多步骤串联调用时陷入停滞。此外，AWS 正在测试名为“AWS Context”的上下文抽象层，使智能体能够跨 S3 数据湖与 Aurora 数据库无缝检索与合成企业数据。 为降低自主系统与早期开发者的上手门槛，AWS 推出了可在 30 秒内完成配置且免绑信用卡的极简注册流程，在后台自动预置 VPC 与 IAM 策略等安全默认项。Garman 强调，该轻量化入口绝非受限的测试沙箱，其底层是完整的生产级 AWS 账户，便于初创团队与智能体工作流在业务扩张时平滑加载企业级安全与组织管控功能，避免了后续数据迁移的摩擦。 2200亿资本支出与动态供应链约束 (16:00) 面对生成式 AI 与上云需求的激增，亚马逊计划在 2026 年投入高达 2200 亿美元的资本开支，并计划在未来几年内采购 200 万张英伟达 GPU。Garman 坦言，算力抢购热潮曾一度使 AWS 暂时失去了传统云服务即时弹性的优势。为构建健康的创新生态，AWS 坚持主动调控配额，拒绝将全部算力垄断性出售给资金充裕的前沿大模型实验室，而是专门为 AI 初创企业保留专有算力池，目前大约能满足 60% 的新进算力申请。 在物理设施层面，算力扩张受制于多维度的动态连锁瓶颈。Garman 引用《目标》中的“约束理论”指出，供应链堵点持续在电网接入、台积电晶圆制造、高带宽内存（HBM）、先进封装及专用建筑工人之间流动，解决一个环节往往会迅速暴露下一个限制。为此，AWS 已将硬件依赖追踪延伸至供应链第 4 至第 5 层级，并深入涉足 20 年期的核能、太阳能等直接电力开发。Garman 认为，替企业客户吸收这种极其繁重的能源与硬件规划复杂度，构成了现代云平台的核心价值之一。 自研芯片矩阵：从Nitro到Trainium (32:20) AWS 的自研芯片体系源于消除传统“虚拟化税”的技术探索。通过收购 Annapurna Labs 打造出 Nitro 系统，AWS 将网络、存储与系统管理任务完全卸载至专用硬件卡上，不仅向客户释放了 100% 的主机算力，还实现了硬件级别的强安全隔离。随后，工程团队将 Nitro 中的 ARM 核心外溢为通用处理器 Graviton。该芯片在过去数年内稳定提供比同等 x86 实例降低 20% 成本且提升 20% 性能的表现，目前 Graviton 的年部署量已高居 AWS 全球算力集群首位，被超过 90% 的前百大客户在生产环境中采用。 在专用 AI 算力领域，AWS 正在推出第三代 AI 处理器 Trainium3，并已预告 Trainium4，其可用配额已被预订至次年年底。Garman 解释称，尽管命名带有“训练（Train）”，但由于超大模型在推理时极度依赖分布式网络互连与大内存吞吐，Trainium 在运行大模型推理时展现出了卓越的性价比。目前 Amazon Bedrock 上的绝大多数基础模型推理负载均运行在 Trainium 芯片上，使其成为兼顾大规模预训练与高效推理的双重主力架构。 企业可信落地：45天交付机制与安全防御 (40:00) 企业在落地智能体时普遍面临工作流重构与系统信任障碍。Garman 观察到，当前生产环境中的智能体大多局限在有人监督的单线性流程中；要释放更大价值，企业必须摆脱简单复刻人类操作的惯性，转向让智能体以高度并行的方式同时探索数十条路径。针对核心数据外泄与误删生产数据库的担忧，Amazon Bedrock 在网络架构上保证企业 Prompt 与私有数据完全隔离在客户自身 VPC 内部，绝不回传给模型提供商；同时 SageMaker 出现显著回流，被企业广泛用于在其专有数据上后训练与蒸馏高性价比的小型开源模型。 为了解决企业在评估系统（Eval）、数据标注与模型漂移检测上的专业断层，AWS 建立了为期 45 天的前置部署工程（FDE）模式，通过联合攻关将评测机制内化至客户团队，期满即主动撤出，避免传统集成商旷日持久的咨询绑定。在安全保障方面，AWS 推出 Continuum 服务，将自身防护超大规模基础设施的内部 AI 工具产品化，基于拓扑与权限上下文以“机器速度”自动化处置威胁。在内部运营中，亚马逊全员普及 Amazon Q，推动产品研发向智能体完全代写代码的模式跃迁，使部分功能团队的人力需求精简过半。 尽管 AI 初创企业可能因循风险投资的幂次法则出现淘汰与分化，但 Garman 坚信基础设施的真实效用不会逆转。当前智能体落地的重心已从单纯的模型能力狂热，转向跨越数十年的电力规划、底层低延迟架构重构、严密的机器级权限沙箱，以及推动企业业务从串行逻辑向大规模并行自主计算的系统性工程转型。

## Background 本期访谈由 Raghu Raghuram 与 Matt Garman 于 2026 年 10 月 8 日在播客及网络平台发布。根据页面简介，a16z 的 Raghu Raghuram 与 AWS 首席执行官 Matt Garman 展开深度对谈，探讨了 AI 如何重塑云计算、云基础设施向面向智能体（Agent）的设计演进、亚马逊数千亿美元的资本开支，以及 AWS 团队自身的智能体实践与技术布局。 ## TL;DR 访谈围绕 AI 时代下 AWS 的底层架构重塑与商业战略展开。Matt Garman 阐述了云基础设施如何从服务人类工程师转向适配具有瞬态计算、极速编排和严苛长尾延迟需求的自主智能体，并介绍了 AWS Context 与轻量虚拟化沙箱等新一代构建块。面对高达 2200 亿美元的资本开支扩张，Garman 分享了在电力接入、TSMC 产能与供应链瓶颈中穿透式保供的挑战，以及为初创企业保留稀缺 GPU 算力的配额策略。此外，两人深入复盘了 Nitro、Graviton 与 Trainium 等自研芯片的演进历程，并探讨了企业在智能体落地中的安全护栏构建，以及 AWS 内部研发向“智能体优先”转型对工程组织形态带来的深刻变革。 ## Atmosphere & style 整场对话呈现出一种兼具底层工程硬核感与高管战略定力的务实基调。前 VMware 掌舵人 Raghu Raghuram 与从 EC2 首任 GM 一路走到 CEO 的 Matt Garman 棋逢对手，两人像是技术老兵围炉复盘，既有相视一笑的行业幽默与自我调侃，又有直面“AI 泡沫质疑”和“电力短缺难题”时的坦诚与笃定。听众能够清晰感受到那种面对人类历史上最庞大资本基建狂潮时的兴奋感，以及对未来组织将被 Agent 彻底颠覆的敬畏与冷静思考。 ## Bulletin - `00:00` 亚马逊计划在2026年投入约2200亿美元资本开支，并计划采购200万张英伟达GPU，同时坚持对宝贵算力进行人工配额保护，专门为初创企业保留专用容量。 - `00:54` AWS年化营收运行率已达1690亿至1700亿美元并维持高速增长，其中约30%至40%的收入来自于曾经由AWS陪伴孵化成长的初创公司。 - `06:08` 面向AI智能体的高并发与低延迟需求，AWS推出公测服务AWS Context打通S3与Aurora等数据源，并优化S3的P99.9长尾延迟以防智能体链式推理阻塞。 - `35:00` 自研芯片Graviton已成为AWS每年上架部署量最大的计算芯片，Top 100客户中超90%已采用；而专为张量计算设计的Trainium第三代产能极度紧缺，并意外成为市场上性价比最高的推理芯片之一。 - `52:15` AWS内部前沿研发团队已全面转向“智能体优先（Agent-first）”开发模式，传统需10人负责的功能现缩减至3到4人配合智能体集群即可高速交付。 ## Key takeaways - `00:00` 云基础设施的核心服务对象正在从“人类工程师”向“自主智能体”快速转变，倒逼云计算在沙箱隔离、网关协议和权限控制上重构底层架构。 - `00:54` 全球范围内绝大多数企业级工作负载仍停留在本地数据中心，云计算不仅远未见顶，更同时受到企业上云与AI爆发双重红利的推动。 - `03:09` 初创企业构成了AWS的技术与业务护城河，他们往往提前5年触碰大型企业未来才会遇到的极端技术瓶颈，倒逼云平台持续演进。 - `05:00` AI初创公司虽在早期依赖提供裸算力的新兴云（NeoClouds），但随着团队扩大，对多租户隔离、企业级IAM权限及合规性的刚需促使其最终回流到AWS等综合云巨头。 - `06:08` 智能体对基础设施的敏感度与人类截然不同：多步链式推理使其对P99.9长尾延迟极度敏感，微小的S3响应停顿就会造成自动化工作流彻底中断。 - `10:00` AWS重构了账户开通流程，支持Gmail注册并在30秒内自动托管底层VPC和网络权限，既降低早期摩擦与适配智能体调用，又避免后期二次迁移的阵痛。 - `12:35` 智能体即用即抛的瞬态资源需求与传统云服务（如Aurora数据库）追求99.999%持久性的架构哲学产生冲突，促使轻量级微虚拟机Firecracker与动态时限权限成为关键构建块。 - `15:00` 面对巨头前沿实验室对GPU的疯狂扫货，AWS选择硬性保留部分宝贵算力专属供给初创企业，通过维护生态多样性以追求长期战略利益。 - `20:00` 不同于部分新兴云将过半算力绑定在少数大客户身上，AWS单一客户算力占比仅为个位数，且用量高度依赖具备正向ROI的真实生产与推理负载，抗泡沫周期能力极强。 - `24:12` 数据中心能源规划逻辑发生根本改变，AWS不得不将电网资本与能源谈判窗口拉长至10到20年，并采取“自带电源（核能、太阳能）”与表后直连模式保障供电。 - `26:40` 基建扩张的瓶颈遵循《目标》中的约束理论不断动态转移，从电力、TSMC晶圆制造、HBM显存到微型连接器交替卡脖子，促使AWS建立穿透至四五级供应商的供应链管理体系。 - `32:38` 自研芯片战略发端于消除“虚拟化税”的Nitro架构，通过收购Annapurna Labs剥离网络与存储虚拟化，不仅释放了裸金属性能，还确立了底层硬件级安全隔离。 - `37:15` 由于大语言模型体积膨胀导致推理底座与训练集群日趋一致，原本定位于训练的Trainium凭借内存与互联优势意外成为高性价比推理核心芯片。 - `40:00` 企业规模化落地自主智能体的核心障碍在于思维方式与信任：必须从机械复制人工流程转向多路径并行探索，并通过精准权限沙箱防范灾难性误操作。 - `42:47` 针对企业在评测（evals）与数据标注上的普遍技术空白，AWS推行“45天赋能并离场”的前线工程（FTE）模式，协助客户建立自运维能力而非绑定外部咨询。 - `44:56` Amazon Bedrock依托VPC级数据隐私隔离保证客户提示词绝不回流模型方，而SageMaker则因开源大模型微调、后训练及专用模型蒸馏热潮迎来了“第二春”。 - `50:00` 网络安全必须从人类速度迈向“机器速度”，AWS推出基于前沿模型的Continuum服务以实现漏洞秒级评估与风险定级，并在全员日常业务中深度普及Amazon Q助手。 - `52:15` 智能体优先的研发范式正在重塑组织形态，人类工程师全面转型为调度管理智能体团队的管理者，显著提升交付效率并对传统功能团队架构带来深刻变革。 ## Detailed walkthrough ### `00:00` 序幕与背景：AI 时代的算力扩张与基础设施重塑 访谈开篇提到了当前云计算与人工智能交叉领域的几个核心矛盾与重大动态。一方面，处于最前沿的人工智能实验室正在以前所未有的速度吞噬市场上几乎所有可用的 GPU 算力；另一方面，云计算平台必须在这些巨头与代表未来企业希望的初创公司之间取得平衡。AWS 正在积极预留稀缺算力，并宣布计划在未来几年内采购 200 万张 NVIDIA GPU。与此同时，母公司亚马逊预计在 2026 年投入高达 2200 亿美元的资本支出（Capex），以支撑这场历史上规模空前的数据中心与基础设施大扩张，且这一投资势头毫无放缓迹象。 在技术架构层面，云基础设施的核心服务对象正在从“人类工程师”向“自主智能体（AI Agents）”转变。无论是计算沙箱（compute sandboxes）、网关协议，还是面向智能体而非人类的权限隔离体系，都在倒逼云厂商重构底层能力。AWS 首席执行官 Matt Garman 指出，智能体的工作流程在底层系统性能上面临着完全不同的挑战，必须从根本上提供一套适配自主软件运行的云体系。正如 Matt Garman 所断言： > “Agentic workflows tend to perform better on AWS than anywhere else.”（智能体工作流在 AWS 上的表现往往比在任何其他地方都要好。） - **资本支出规模**：亚马逊规划在 2026 年投入约 2200 亿美元的资本开支，以应对持续井喷的算力需求。 - **算力采购与分配**：计划在未来数年采购 200 万张 NVIDIA GPU，并在头部前沿实验室的争夺中，极具针对性地为初创企业保留专用容量。 - **自研芯片与团队演进**：除采购商业 GPU 外，AWS 正在大力推进 Trainium 等自研芯片布局；而在内部研发中，工程师的角色正逐步转向“管理智能体团队”。 --- ### `00:54` 从 EC2 的第一美元到 1700 亿美元体量与上云长跑 a16z 的 Raghu Raghuram 与 Matt Garman 共同回顾了 AWS 的创立历程。Matt Garman 早在 2006 年便担任了 Amazon EC2 的首任总经理（GM）。从当年的零起点起步，如今 AWS 的年化营收规模已达到约 1690 亿至 1700 亿美元区间，且在如此庞大的基数下依然保持着惊人的高速增长。Raghu Raghuram 曾长期在 VMware 推动本地私有云基础设施建设，他打趣称自己过去一直在努力把企业工作负载留在本地数据中心（on-prem），而现在整个行业都在经历向云端的大迁移。 Matt Garman 强调，尽管当前的营收规模惊人，但从长远潜力和客户机会来看，整个行业仍处于极其早期的发展阶段。当前全球范围内绝大多数企业级工作负载依然运行在传统的本地服务器上。随着日常计算需求呈指数级递增，再加上人工智能浪潮与传统上云迁移的双重顺风驱动，云业务的上升空间依然十分广阔。谈及这一增长态势与时代机遇，Matt Garman 总结道： > “And we don't anticipate slowing down anytime soon because the demand is just massive.”（我们预计短期内不会放缓步伐，因为需求实在是太庞大了。） - **营收规模与增速**：AWS 当前营收运行率达到 1690 亿至 1700 亿美元，在大体量下仍保持约 37% 的高位增速。 - **工作负载现状**：全球依然有巨量关键工作负载停留在本地数据中心（on-premise），云计算的渗透率远未见顶。 - **双重增长引擎**：业务同时受益于“传统企业上云（Cloud Migration）”与“AI 爆发式算力需求（AI Tailwind）”两大驱动力。 --- ### `03:09` 初创企业DNA：从2005年实习项目到40%营收来源 谈到 AWS 的起步，Matt Garman 分享了一段轶事：2005 年他在商学院就读期间作为实习生加入亚马逊，当时 AWS 还是一个内部孵化项目。他当时的实习课题就是分析“谁会对 AWS 最感兴趣”，最终得出的结论正是“初创公司（startups）”。自那之后，初创企业就成为 AWS 核心业务的命脉。AWS 为初创公司提供的不仅是开箱即用的基础设施，还包括系统架构设计、扩展性咨询以及全套扶持，帮助他们从“车库里的两个人”起步直至长成参天大树。 对于 AWS 而言，深度扶持初创企业不仅是一种商业共赢，更是业务发展的重要护城河。Matt Garman 透露，据粗略估算，目前 AWS 约有 30% 到 40% 的收入来自在 AWS 陪伴下成长起来的曾经的初创公司。更重要的是，初创公司是站在技术最前沿的颠覆者，他们往往比大型银行或政府机构更早触碰技术极限，倒逼云服务不断进化。Matt Garman 这样解释这种共生关系： > “From the very beginning of when we launched AWS, startups have been the lifeblood of the core of what we do.”（从我们推出 AWS 的最开始，初创公司就是我们所做事业的核心命脉。） - **业务营收贡献**：估算有 30% 至 40% 的 AWS 当前总营收来自于曾经由 AWS 孵化并陪伴壮大的初创企业。 - **产品创新飞轮**：初创团队往往提前 5 年提出大型银行或传统机构未来才会需要的极端技术需求，成为推动 AWS 架构演进的关键动力。 --- ### `05:00` 时代之变：AI原生初创的超级规模与架构坚守 对比十多年前与当下的初创企业生态，初创公司的起点和野心发生了质的飞跃。Raghu 和 Matt 观察到，过去初创公司需要漫长的打磨，而现在的 AI 初创公司往往仅凭一个想法就能直接跨入数亿美元甚至十亿美元估值的门槛，资金和资源消耗极其巨大，尤其是模型训练动辄需要千万乃至上亿美元的资本支持。但 Matt 明确指出，尽管规模膨胀极快，底层工程的核心诉求却始终如一：无论规模增长多快，企业依然必须解决架构稳定性、数据安全、网络性能以及严格的 IAM（身份与访问管理）权限控制。 这也是为什么许多 AI 初创公司在度过最原始的单机训练阶段后，依然坚定地从单纯提供算力的“新兴云（NeoClouds）”迁移到 AWS 上的原因。随着团队规模从几个人扩大到数十人，多租户隔离、合规性与企业级权限系统便成为刚需。面对资本密集与技术迭代，初创公司对底层基础设施的综合能力提出了更为苛刻的标准。 ```mermaid flowchart LR accTitle: Startup Evolution on Cloud Infrastructure accDescr: Comparison of early stage compute needs versus scale stage enterprise cloud needs. subgraph Early["初期探索阶段"] A["高额资本投入 / 密集训练"] --> B["获取裸算力 (GPU / NeoCloud)"] end subgraph Scaled["规模化与企业化阶段"] B --> C["高并发与低延迟架构"] C --> D["深层 IAM 权限与合规隔离"] D --> E["AWS 全栈云服务支撑"] end ``` --- ### `06:08` 智能体时代的云重构：AWS Context、P999 延迟与极速数据库 在谈到云平台的演进趋势时，Matt Garman 指出当下的开发者群体出现了一个重大转变：越来越多的研发团队要求云平台能够原生适配“AI 智能体（Agents）”的高频调用，而不仅仅是供人类工程师在控制台上操作。智能体需要极速的资源调配能力（例如在 3 秒钟内启动一个完整的数据库），并且高度依赖规范清晰、结构明确的 API 接口。 为此，AWS 采取了双轨策略：一方面推出专门用于构建智能体的 Bedrock 和 Agent Core 等核心托管服务；另一方面对 S3 等传统基础存储层进行了深度扩展。AWS 正在公测名为 **AWS Context** 的全新上下文层，旨在打通企业存储在 S3、Aurora 等各类数据湖与数据库中的异构数据，让自主智能体能够一站式无缝检索并穿透整个数据生态。 更为关键的技术差异在于“长尾延迟（Tail Latency）”。人类用户对于网页或请求的 P99.9 延迟感知并不明显，但一个正在执行多步复杂推理、链式调用数十次 API 的自主智能体，只要遇到一次极端的 P99.9 S3 延迟就会直接陷入阻塞（Blocked）。Matt Garman 阐述了智能体对长尾延迟的严苛要求： > “Where people are, they don't always care about the P999 S3 latency, like agents do care and get blocked by that.”（对于 S3 的 P99.9 延迟，人类可能并不总是在意，但智能体非常在意，并且会被其彻底卡住。） 针对智能体直接编写代码、自动选择数据库和邮件服务并自主部署的新趋势，Matt Garman 认为 AWS 的底层核心构建块已经具备极佳的弹性和完备性，目前的演进重心在于构建更加轻量、易用且适合自动化工具调用的接口适配层，使智能体无论是在底层原生调用还是通过生态伙伴工具部署时，都能实现近乎零门槛的顺畅交互。 - **新型数据上下文层（AWS Context）**：目前处于 Beta/Preview 阶段，帮助智能体跨越 S3、Aurora 等多数据源建立统一上下文视图。 - **延迟指标转变**：智能体多步并发调用对 P99.9（千分之九百九十九）长尾延迟极度敏感，微小的阻塞都会导致自动化工作流停滞。 - **极速基础设施**：为智能体优化资源编排速度，实现数据库等复杂服务秒级（如 3 秒内）拉起。 ### `10:00` 告别繁琐配置：专为智能体与初创企业打造的 30 秒无感开箱体验 在过去的二十年里，开通一个全新的 AWS 账户向来是一件门槛极高的事情。对于刚接触该平台的新用户而言，如果之前从未配置过 AWS，他们必须手动定义 VPC（虚拟私有云）、设置复杂的 IAM（身份与访问管理）角色与权限。尽管这些架构设计在企业业务扩张后至关重要，但对早期初创公司或轻量级开发者来说，这却成了一种痛苦的折中方案：他们明知几个月甚至几年后自己会需要这些严谨的企业级功能，但在起步的当下，他们只想立刻调取云服务，完全不想被底层繁琐的合规与网络配置所束缚。 为了彻底打碎这一壁垒，AWS 正在逐步推行全新的账户创建流程。如今，用户注册 AWS 不再需要繁琐的绑卡核验或强制的前置架构规划，仅凭一个 Gmail 账户即可完成注册。底层所有的网络和权限配置均由系统默认在后台代为托管，用户在不到 30 秒内就能拉起并运行一个功能完整的真实 AWS 账户。Matt Garman 指出，这种极速体验不仅极大降低了开发者的摩擦成本，更是 AI 智能体（Agents）所迫切需要的交互模式——因为自主智能体在执行任务时，根本不想被迫处理一套复杂的 VPC 初始化流程。更关键的是，这并不是一个被阉割的功能子集沙盒：当业务规模扩大需要引入组织管理或精细化调优时，用户本身就处于一个真实的 AWS 账户内，后续可以直接无缝调用底层所有深度功能，免去了后期数据与服务迁移的阵痛。 - **极速账户初始化**：使用 Gmail 即可注册，无需预先绑定信用卡，后台默认处理底层网络与权限，30 秒内进入全功能运行状态。 - **无迁移负担（No Migration）**：避免了“先用简易小平台再迁移到大云”的断代缺陷，随业务增长可随时就地解锁 AWS 企业级深度特性。 --- ### `12:35` 智能体时代的架构悖论：瞬态资源与企业级“持久性”的冲突与重构 随着开发主力军从人类工程师转向智能体、乃至于底层基础架构运维团队也正被智能体全面替代，AWS 面临着全新的范式转移。Matt Garman 强调，尽管 AWS 既有的底层架构具备极佳的弹性承载能力，但在具体设计哲学上，部分系统正暴露出针对智能体场景“过度设计”（Over-engineered）的矛盾。典型场景如智能体在执行复杂工作流时，往往只需要临时生成一个数据库，执行几秒钟的数据处理后便直接销毁；而 AWS 原生打造的诸如 Amazon Aurora 等数据库，核心指标是面向生产环境的“五个九”（99.999%）数据持久性与极高可用性。这种为持久生产系统打磨的高可靠性架构，对于高度瞬态（Transient）、即用即抛的智能体需求而言，在资源消耗与启动延迟上显得极为沉重。 AWS 并不打算简单妥协推行一种“不可靠、易损坏”的廉价方案，因为系统无法在创建伊始就预判某个数据库最终是转瞬即逝，还是会被保留下来逐渐演变为核心生产库。因此，工程团队正在攻克两全之策：既要支持毫秒级、免除资源浪费的极速构建与丢弃，又要保证其在被保留时能即刻具备顶级的持久性。此外，智能体的爆发式介入催生了对全新底层构建块（Building Blocks）的刚性需求，包括计算沙盒（Compute Sandboxes）、专用网关，以及与人类或服务角色完全不同的动态权限体系。传统的角色赋权往往过于宽泛，而针对智能体，必须引入按任务限定时限（Time-boxed）的极短权限，甚至在特定沙盒内对其调用工具的细分 API 行为进行像素级的颗粒度限制。 ```mermaid flowchart TD accTitle: Agent Infrastructure Challenges and Paradigms accDescr: 展现从传统企业级持久性到智能体轻量级、瞬态基础设施的范式转移，包含资源生命周期与细粒度权限控制。 subgraph Legacy["传统企业级架构范式"] A["高持久性服务 (如 Aurora 99.999%)"] B["静态服务角色 & IAM 权限"] C["重型虚拟化与长期资源配置"] end subgraph AgentNeeds["智能体基础设施核心痛点"] D["瞬态计算需求<br/>(即开即用、随用随销)"] E["高频度、低延迟资源调度"] F["严格执行边界与短期任务沙盒"] end subgraph NewBlocks["AWS 新一代智能体构建块"] G["Firecracker 轻量级微型虚拟机 (MicroVM)"] H["动态时限凭证 (Time-boxed Permissions)"] I["弹性双模资源<br/>(极速丢弃 vs 无缝转生产持久化)"] end Legacy -.->|"范式重构与冲突"| AgentNeeds AgentNeeds -->|"驱动底层工程创新"| NewBlocks ``` 针对计算隔离，AWS 在近十年前研发的开源微型虚拟机 Firecracker 正在展现出巨大威能。市面上绝大多数 AI 沙盒初创公司，其底层技术栈无一例外都建立在 Firecracker 之上。Firecracker 原本并非专门为智能体设计，但其启动飞快、具备严密硬件级安全边界、且完全消除了传统大型虚拟机虚拟化开销的特性，天然契合智能体对安全执行沙盒的严苛要求。 > “Many agents want to create a database that, do a little bit of work and then have the database go away. You really need that database to have five nines of durability.” —— Matt Garman > “And so there's no migration or move later about, which is how do we make it not a choice for the customer, but an easy on-ramp into the depth of features that we know customers and startups are going to want when they start scaling.” —— Matt Garman --- ### `15:00` 2200 亿美元资本支出下的算力战局：初创企业的 GPU 保卫战 Raghu Raghuram 抛出了当前所有初创企业最紧迫的焦虑：前沿顶尖大模型实验室（Frontier Labs）正以吞噬一切的姿态扫荡市面上所有可用的 GPU 算力，而那些没有深厚信用记录、缺乏巨额资金但代表未来潜力的年轻初创公司，该如何在这一轮算力荒中存活？Matt Garman 直言，当前整个行业的资本开支（CapEx）规模是史无前例的，仅针对 2026 年的规划周期，市场预测与内部规划的资本投入就可能高达 2200 亿美元规模，堪称人类商业史上单阶段最庞大的基础设施支出。 然而，单纯拥有资本并不等于能立刻产出算力。算力供应链正受到多维物理极限的连环卡脖子： 1. **电力供应与数据中心土建**：电网负荷接入延期，以及建筑工人和电力工程施工人员的极端短缺。 2. **高阶硬件供应链**：HBM 高带宽内存、先进制程芯片等核心硬件的产能爬坡速度。 面对这种持续的供给赤字，AWS 并没有把所有产能打包卖给能一口气包圆的前沿实验室。Matt Garman 强调，AWS 采取了极为果断的人工配额保护策略（Allocation Strategy）： - **扶持初创生态**：AWS 坚持把极其宝贵的 GPU/加速器算力硬性保留一部分专属供给初创企业，哪怕这意味着牺牲短期内将资源一次性倾倒给巨头的机会，因为维护一个健康的多元生态对 AWS 的长期未来至关重要。 - **履约转化率**：目前面对各类客户提出的苛刻算力请求，AWS 最终能以某种形式（包括提供替代网络拓扑或机型配置）满足约 60% 的需求。 - **巨量采购规划**：AWS 已正式公布在未来几年内采购投放多达 200 万张 NVIDIA GPU 的宏大蓝图，同时持续推进自研 Trainium 芯片与英伟达平台的双轨部署，全力缓解各层级客户的算力渴求。 > “You're right, we could sell every single GPU or AI accelerator we had to probably just the big Frontier Labs and call it a day. We choose not to do that because we actually want to keep growing the full ecosystem.” —— Matt Garman > “We recently announced we're going to be buying, you know, 2 million NVIDIA GPUs over the next couple of years.” —— Matt Garman ### `20:00` 2200 亿美元资本支出、AI 泡沫质疑与分散化客户底气 当 Raghu Raghuram 问及 AWS 的资本支出（CapEx）究竟是何时跨过 10 亿甚至 100 亿美元大关时，Matt Garman 笑着坦承自己记不清确切的历史节点，但他指出，眼下亚马逊整体的资本支出规模已达到惊人的 2200 亿美元，过去三到四年的 AI 基础设施建设彻底将这笔开支推上了加速轨道。在云计算发展的早期，AWS 习惯于走在需求曲线的前面提前投入；但过去几年 GPU 算力的暴增打破了这种常态，甚至在某种程度上破坏了云原本引以为傲的“弹性”。虽然核心计算和存储仍然保持着随需调度的弹性，但在稀缺的加速算力面前，行业不得不重新适应全新的供需现实。 面对外界频繁抛出的“是否会担心 AI 泡沫”的疑问，Matt Garman 显得十分从容。他对比了 AWS 与一些新兴“新云”（NeoClouds）或其他云厂商的商业结构：某些云厂商将高达 30% 到 60% 的产能高度集中在一两家大客户身上，一旦客户违约或转向，风险将是致命的；而 AWS 采取彻底的分散化策略，单一客户在总产能中的占比最高也仅为个位数，绝大多数甚至远低于此。更关键的是，客户使用 AWS 是为了运行真正的生产环境业务，底层核心计算、存储以及与应用深绑的推理工作负载构成了收入的绝大部分。Matt Garman 直言：“And so those are the workloads that I think just aren't going to go away. Because we see enterprises getting positive ROI.” 无论是向企业高管核实当下能力与成本的投资回报率，大家几乎异口同声给出肯定答复，这与互联网泡沫时期仅凭概念炒作截然不同。 即使部分拿到巨额融资的独角兽最终会失败，这在 Matt Garman 看来也是遵循了风投行业运作了五十年的“幂律定律”（Power Law）。从十家公司里赌赢一家并覆盖全部成本，这是技术浪潮的常态。就像互联网泡沫破裂淘汰了一大批公司，却孕育出了 Google 和亚马逊这样持久繁荣的巨头一样，AWS 坚信当下的投资是经得起周期考验的。正如亚马逊现任 CEO Andy Jassy 所公开强调的，AWS 面向未来十年的潜在增长空间依然极其庞大，面对这种量级的历史机遇，企业唯一正确的做法就是倾尽全力进行前瞻性布局。 - **资本支出规模**：亚马逊当前的整体资本支出规模已达到约 2200 亿美元，AI 算力基建是近三四年最主要的驱动引擎。 - **客户集中度对比**：部分 NeoClouds 将 30%–60% 算力绑定在一两家大客户上，而 AWS 单一客户最高占比仅为个位数，抗周期波动能力极强。 - **真实生产驱动**：AWS 的主要用量集中于核心计算、存储和推理工作负载，客户均看到了切实的正面投资回报（positive ROI）。 --- ### `24:12` 超前十至二十年的能源规划：从随用随取到自带核电与光伏 随着资本支出从数亿美元跃升至单笔“200 亿或 500 亿美元的大支票”，AWS 的底层规划逻辑发生了颠覆性变革。Raghu Raghuram 追问这种天文数字的规模如何改变了内部的流程，Matt Garman 感慨，现在的团队甚至要坐在会议室里精确预测 2028 年的算力需求。过去，电力对于数据中心来说更像是一种唾手可得的公用事业服务，“当年我们只需要给电力公司打个电话，说再给我们划拨几兆瓦、几十兆瓦电，他们就会直接批复，因为几十兆瓦根本不算什么，足够支撑我们跑好久”；然而在当下的 AI 时代，AWS 必须“自带电源（Bring your own power）”，直接成为能源项目的主导开发者和资助方。 在能源结构上，AWS 已经连续多年稳居全球最大的可再生能源企业采购方之一。面对电力管网的严峻负荷，他们的打法变得极其复合：既包括为大型太阳能或核能项目直接出资建设、将清洁电力注入公用电网并获取信用额度，也包括在数据中心现场采用“表后（Behind the meter）”直连发电设施的物理接入模式。由于输电网络（Transmission lines）的扩建与审批周期漫长，AWS 不得不将能源和电网资本的规划窗口一口气拉长到 10 年乃至 20 年之后。 这种超前规划的能力，反过来成为了公有云对抗客户私建数据中心最深的护城河。Matt Garman 指出，一般的企业客户根本不可能建立一个专门的能源团队去谈判二十年后的核电协议，也不可能在 2024 年就测算清楚自己在 2028 年到底需要多少显存与内存容量。将纷繁复杂的供电协议、供应链锁定和物理基建剥离出去，正是公有云为全球开发者与企业客户所创造的最核心价值。 ```mermaid flowchart TD accTitle: AWS超前电力与基建规划体系 accDescr: 展示AWS如何通过表后直连与电网回馈相结合的方式锁定未来10至20年的能源供给 subgraph PowerSourcing["能源投资与获取方式"] direction TB A1["核能与先进核反应堆项目"] A2["大型太阳能与风电可再生能源"] end subgraph Integration["能源接入与调度架构"] direction TB B1["表后直连模式 (Behind-the-Meter)<br/>现场机房直供电"] B2["公用电网注入模式 (Into Grid)<br/>出资并网换取清洁电力额度"] end subgraph DataCenterOps["基础设施与交付保障"] direction TB C["2026-2028+ 年长期服务器与内存容量部署"] D["企业级生产环境免受能源瓶颈影响"] end PowerSourcing --> Integration B1 --> C B2 --> C C --> D ``` --- ### `26:38` 《目标》与无尽的瓶颈环游：从 TSMC 产能到四五级供应链穿透 当 Raghu Raghuram 抛出“在 2027 至 2028 年整个行业最严重的瓶颈究竟会在哪里”时，Matt Garman 并没有简单押注于某个单一零部件，而是讲起了自己大学本科时读过的经典管理学名著《目标》（*The Goal*，高德拉特著）。他感叹当年完全没想到这本讲“约束理论”的书在几十年后会成为指导云原生基建的核心哲学：“It turns out there's never one constraint. There's always just the latest constraint.” 系统的瓶颈永远处于动态转移中：这个月如果解决了电力，下个月可能立刻卡在台积电（TSMC）的晶圆制造产能上；晶圆搞定了，高带宽内存（HBM）或网络交换机芯片可能又告急；即便高精尖芯片全部就位，供应链深处某个不起眼的连接器或光模块出现供货故障，整条产线依然会彻底停摆。 除了零部件种类的制约，地理位置上的“不可替代性（Non-fungible）”也是巨大的挑战。Matt Garman 举例说明，AWS 可能在印度尼西亚拥有极其充裕的电力和机房空间，但这并不能解决德国法兰克福电力短缺导致的算力饥渴。不仅如此，机械硬盘（HDD）、固态硬盘（SSD）以及各种微型电子元器件的供应链都极其脆弱。为此，AWS 追踪着数十万个独立零部件的动态，其中大量核心元器件直接由自身团队绕过组装厂进行穿透式垂直管理。 Matt Garman 回忆道，早在十多年前发生泰国洪灾导致全球硬盘断供、以及随后的内存危机爆发时，AWS 就警醒并建立了业界领先的供应链管理团队。从那时起，AWS 不再只盯着机架或服务器整机，而是将供应链的可视化与掌控能力直接向下穿透了四层到五层（Tier-4 / Tier-5 suppliers）。虽然没有任何系统能够做到百分之百免疫于突发冲击，但这种提前数年锁定底层关键元器件产能、并分散多地制造的机制，成为了 AWS 支撑起全球最大规模云基础设施运转的基石。在节目的最后，话题自然引向了当前社会各界对数据中心高耗能的激烈争议，行业该如何向公众讲清数据中心对经济与国家竞争力的真实价值，成为了他们正在内部深思的全新议题。 - **瓶颈转移论**：制约数据中心扩张的因素并非固定不变，电力、TSMC 晶圆、HBM、高速网络设备乃至基础连接器会在不同时期交替成为最大短板。 - **地域不可替代性**：算力与能源的分布存在强烈的地理限制，一地（如印尼）的能源冗余无法直接填补另一地（如德国）的供给缺口。 - **穿透式供应链管理**：吸取了当年泰国洪灾硬盘危机与多次内存短缺的教训，AWS 追踪数十万个零部件，对关键链条向下穿透四至五级供应商进行直接储备与保供。 ### `30:00` 数据中心的社区价值与企业责任 关于数据中心建设在各地引发的争议，Matt Garman 直言 AWS 必须更加主动、更具发声意识。他指出，数据中心运营商的形象往往被少数不良参与者连累，那些公司不尊重当地法规、盲目快速上马项目，导致整个行业被扣上负面帽子。相比之下，AWS 深入考量运营所在社区的长期福祉：不仅大力引进可再生能源，还通过以自由空气冷却（free air cooling）为主的方案，将水资源消耗降至极低水平，致力于实现“水资源正效益”（water positive），并为当地创造高薪就业岗位。 针对社区居民对数据中心的感知偏差，Matt Garman 强调了沟通与透明度对于消除误解的重要性。很多居民一方面享受着 Netflix 等云计算支撑的数字生活，另一方面却对家门口的数据中心心存芥虑。事实上，数据中心带来的财税贡献非常惊人。Matt Garman 分享了一个具体数据：在他最近看到的一份报告中，AWS 运营所在的某县，每位居民每年的税负因为 AWS 带来的税收贡献而足足减少了 5,000 美元。然而，由于这些利好缺乏宣传，民众往往毫无察觉。 > “And I think we just need to be more clear about those benefits that we bring. Because I think if you told the communities, by the way, your tax bill is $5,000 less than it would otherwise be if we weren't here, they might have a little bit of different thought about the building that's over there.” —— Matt Garman - **环境考量**：积极引入可再生能源；广泛采用自然风冷技术，耗水量极少，目标是实现水资源正效益。 - **税收红利**：在部分运营县，AWS 带来的税收使每位居民年均少交 5,000 美元税款。 - **行业呼吁**：呼吁提升数据中心规划的透明度，将遵纪守法的良性企业与违规抢建的不良参与者清晰区分开来。 --- ### `32:38` 自研芯片的源头：从 Nitro 卸载卡到 Annapurna Labs 谈到 AWS 自研芯片的历史渊源，Raghu Raghuram 提到自己团队很久以前就是 Nitro 的首批合作伙伴之一。Matt Garman 回顾称，这一征程大约始于十三四年前，完全是由客户需求和工程痛点层层递推出来的，是 AWS 持续迭代创新的典型代表。当时，服务器资源面临相当严重的“虚拟化税”（virtualization tax），消耗了大量宝贵的算力。客户普遍反馈希望获得与全套物理服务器对等的“裸金属”（bare metal）性能。 AWS 迈出的第一步，是引入网络卸载卡，将所有网络虚拟化逻辑从主机 CPU 剥离到独立卡上运行。在尝到甜头后，工程团队希望进一步将存储虚拟化也卸载出去，但市面上的主流网络卸载卡均无法支持这一需求。随后，AWS 发现了一家在卸载卡上搭载了 ARM 核心的小公司——Annapurna Labs。AWS 团队与极具创新精神和使命感的 Annapurna 团队深度打磨方案，最终果断将其收购。通过定制开发更大的加速卡，AWS 成功将所有网络虚拟化、存储虚拟化乃至整机 VM 管理全面剥离，仅通过卡上的 API 对外暴露接口。 ```mermaid flowchart TD accTitle: AWS Nitro架构演进历程 accDescr: 展示从传统虚拟化税痛点，到网络存储卸载，再到收购Annapurna构建Nitro裸金属安全架构的演变过程。 subgraph Problem["痛点识别"] P1["传统虚拟化税 (Virtualization Tax)"] --> P2["客户诉求：裸金属级别性能与资源利用率"] end subgraph OffloadProgression["分阶段硬件卸载"] P2 --> O1["第一阶段：网络卸载卡<br/>(网络虚拟化剥离)"] O1 --> O2["第二阶段：寻找支持存储虚拟化的硬件方案"] O2 --> O3["发现 Annapurna Labs<br/>(板载 ARM 核心卸载卡)"] end subgraph NitroPlatform["Nitro 平台成型"] O3 --> N1["收购 Annapurna Labs 团队"] N1 --> N2["构建专属 Nitro 架构卡<br/>(管理网络/存储/虚拟机)"] N2 --> R1["计算节点达到纯裸金属性能"] N2 --> R2["硬件级安全隔离：AWS 无权访问客户虚拟机"] end ``` 这种软硬件协同的设计架构最终演变为著名的 AWS Nitro 架构。它不仅彻底消除了虚拟化带来的性能损耗、大幅提升了资源利用率，更在底层架构上实现了坚固的硬件级安全隔离。 > “And we tell people, you know, could then legitimately tell people, we have no access to any of your VMs that are running there. And this has been a huge benefit for us for the last decade, where, frankly, like we have been leading, and others have been kind of slow to do this, that people can do.” —— Matt Garman --- ### `35:00` Graviton 的爆发与 Trainium 的 AI 算力布局 在 Nitro 架构取得突破后，AWS 团队注意到卸载卡上驱动系统的 ARM 核心具备巨大的演进潜力，于是萌生了“为何不把 ARM 核心直接做成通用服务器芯片”的构想。第一代 Graviton 诞生时性能相对孱弱，但客户对 ARM 架构服务器的反响异常热烈。通过追踪 ARM 核心在算力跃升与能效曲线（power utilization）上的斜率，AWS 敏锐地判断出其性价比必将反超传统 x86 架构，随即下注培育整个生态。如今，Graviton 已经成为现象级的爆款硬件，每年 AWS 上架部署的芯片中，Graviton 的数量超过了其他任何芯片类型。 Graviton 能够稳定提供“价格便宜 20%、性能提升 20%”的双重优势，成为企业降低云开销最立竿见影的手段；AWS 排名前 100 的客户中有超过 90% 都在其集群中使用 Graviton，许多客户在迁移后将服务器规模直接砍半。大约五六年前，AWS 预见到 AI 计算的崛起，启动了 AI 芯片 Trainium 的研发。如今 Trainium 已经演进至第三代（Trainium 3），产能异常紧张，订单排期已逼近年底。AWS 正全力统筹资源，为初创公司保留紧缺的 Trainium 算力；目前 Amazon Bedrock 上的绝大多数流量均由 Trainium 承载，同时 AWS 还与 Anthropic、OpenAI 以及众多 AI 原生初创公司达成了基于 Trainium 的深度合作。 | 芯片系列 | 核心设计驱动与架构优势 | 主要客户价值与业务现状 | |:--- |:--- |:--- | | **Graviton** | 基于 ARM 架构，聚焦高能效比与高吞吐通用计算 | 价格便宜 20%，性能高 20%；超 90% 的 AWS Top 100 客户采用；AWS 每年部署量最大的计算芯片 | | **Trainium** | 专为大规模张量计算与高扩展性互联集群设计，覆盖训练与超大模型推理 | 支撑 Bedrock 绝大部分流量；与 Anthropic、OpenAI 达成重大合作；算力供不应求，排期至明年年底 | --- ### `37:15` 命名背后的反差：Trainium 为何成为顶尖推理芯片 针对芯片的命名，Raghu 调侃道：“从名字来看它是个训练芯片，但大家似乎都在把它用于推理？”Matt Garman 幽默地开启了自我批评，坦言 AWS 在产品命名上向来缺乏天赋。最初的设想是 Inferentia 专攻推理、Trainium 专攻训练，但随着大语言模型的体量急剧膨胀，运行推理所依赖的系统底座与训练集群越来越接近，Trainium 凭借其独特的内存与互联架构以及极低的成本，意外成为了当今市场上综合性能与性价比最出色的推理芯片之一。 除了承揽 Bedrock 的推理负荷，Trainium 在大规模训练集群中的吸引力也与日俱增。随着 Trainium 3 的落地和 Trainium 4 的正式公布，越来越多顶尖模型开发机构开始将其架构视为构建下一代超大规模训练集群的基石。 > “Originally, we had a chip called Inferentia for inference and Tranium for training. And then as the models got bigger and bigger, and it turns out you actually want to run the inference on these really large systems, it turns out that Tranium is actually maybe the best inference chip on the market right now from an absolute performance and cost performance point of view.” —— Matt Garman 对话转向大中型企业对智能体（Agent）的实际落地进展时，Matt Garman 指出，虽然企业普遍从现有探索中获得了实打实的价值，但现阶段大多数企业落地构建的智能体仍处于相对简单、直接的初期形态。 ### `40:00` 智能体规模化落地的两大阻碍：重构工作流与构建信任机制 当前大多数企业部署的智能体仍然是非自主的（non-autonomous），依然保留着人在回路（human-in-the-loop）的机制。尽管如此，客户已经从中获取了巨大价值，并开始积极探索如何在确保安全的前提下赋予智能体自主性。Matt Garman 指出，阻碍企业全面规模化采用智能体的主要有两个核心难题： 第一是思维方式的转变。很多企业最初的做法只是机械地把现有工作流照搬给智能体，让它按部就班地复制人工流程。对此，Matt Garman 强调必须引导客户从“全新设计”（greenfield approach）的白板视角来思考问题： > “你想要退一步思考：如果我想达成某个目标，智能体怎样以完全不同的方式来完成？它可以采用大规模并行的方式，尝试 50 种不同途径最终达成目标。” 初创公司往往擅长运行成群结队的智能体集群（agent fleets or swarms），但对于像保险公司这样相对保守的传统企业，如何借鉴初创公司的思路并运用到核保审批等日常业务流程中，仍需要大量引导。 第二是信任与安全机制。企业最大的顾虑在于无法确信智能体是否会做出灾难性决策。Matt Garman 直言： > “你不会真的想让一个智能体随意乱跑，然后不小心删掉生产数据库，那会非常糟糕。” 这种谨慎在当下是完全合理的。AWS 正在与客户共同探讨系统架构，并投入研发全新技术，以确保智能体具备精准的权限边界（permissions）、数据访问护栏（guardrails），不会破坏生产环境。 ```mermaid flowchart LR accTitle: 从机械复制到并行探索的工作流演进 accDescr: 对比传统人类串行工作流与智能体高并发、多路径探索的绿地架构演进流程。 subgraph Legacy["传统模式：机械复制人类流程"] direction TB L1["步骤 1~5 顺序执行"] --> L2["人类最终逐项复核"] end subgraph Agentic["智能体原生模式：Greenfield 并行探索"] direction TB A1["设定目标与安全护栏"] --> A2["多智能体并行尝试 50 种路径"] A2 --> A3["自主收敛产出最佳结果"] end Legacy -.->|"思维重构"| Agentic ``` --- ### `42:47` 评测体系的困局与 45 天前线工程（FTE）赋能模式 当 Raghu Raghuram 问及企业是否在学习搭建模型评估（evals）系统以规范智能体行为时，Matt Garman 直言企业目前在这方面极其欠缺能力。无论是搭建闭环测试环境、合理设计目标导向（goal-seeking）、高质量标注数据以贴合生产环境，还是在生产中持续监测并回测模型漂移（drift），业内几乎没有人能够轻松解决。 正是由于企业面临这一痛点，前线工程（FTE, Forward-Deployed Engineering）团队开始在全行业涌现，AWS 及其合作伙伴也重兵投入 FTE 模式。但 Matt Garman 明确指出，AWS 并不想将其做成无休止赚取咨询费的人力外包生意： > “我们对 FTE 的运作理念是：走进一家准备好接手并承担后续运维的客户那里，在 45 天内并肩工作，教他们如何搭建评测体系、如何规范地标注数据。在 45 天结束时，我们离场，客户已经完全掌握技能、能够独立运转。” - 企业目前在评测（evals）、生产监测与数据标注方面面临广泛技术空白，甚至全行业都缺乏标准解法。 - 客户普遍不希望在未来五年里被外部顾问团队绑定，他们需要的是快速掌握自主运维能力。 - AWS 的 FTE 策略严格以“45 天培训赋能并离场”为目标，协助客户完成知识转移。 --- ### `44:56` Amazon Bedrock 的数据安全底线与 SageMaker 迎来的“第二春” 针对行业内关于“企业是否应该避免将数据送往头部前沿大模型、转而使用开源模型进行私有化适配”的讨论，Matt Garman 表达了明确的赞同。他重申，企业数据是客户最宝贵的资产，这也是 AWS 最初设计 Amazon Bedrock 的核心逻辑。 Matt Garman 回忆道，三年前 AWS 因为没有急于推出大模型服务而承受了巨大的舆论压力，但他强调构建底层安全基础才是正确的道路： > “如果我们运行在 Bedrock 内部，你的数据绝不会回流给模型提供方，他们永远看不到你的 Prompt，数据会始终停留在你自己的受信任环境中。” 如今，随着企业工作负载从概念验证（PoC）走向生产，绝大多数客户选择落在 AWS 的 Bedrock 上，正是因为其在 VPC 级别对数据隐私的坚实保护，以及对 Gemini 等各类自研模型与开源模型的全面支持。 与此同时，越来越多的客户希望利用自有的高质量私有数据，对开源权重模型进行微调（fine-tuning）与后训练（post-training），以蒸馏出更便宜但性能更优的专用模型。这一趋势让 Amazon SageMaker 迎来了“第二春”： - SageMaker 起初是作为模型构建平台而生，如今已成为企业进行开源大模型后训练、微调评测与自建推理托管的核心阵地。 - 客户通过在 SageMaker 上做实验，对比不同开源权重模型的微调表现，加速了定制化模型的落地。 --- ### `48:38` 智能体安全边界与新兴攻击面 针对科技界关于 AI 存在性风险（existential risks）以及 Hugging Face 等平台遭遇安全漏洞的讨论，企业级 CEO 最关心的议题依旧非常接地气：当他们在企业内部拉起智能体时，究竟如何确保安全可信？ Matt Garman 指出，AWS 正在重兵研发智能体安全部署的关键能力，重点覆盖四个防御维度： 1. **严格的权限控制（Permissions）**：明确界定智能体能够调用的工具和接口，防止其越权操作。 2. **沙箱隔离（Sandboxing）**：在隔离环境中执行不可信代码或敏感流程，避免波及生产环境。 3. **动态护栏（Guardrails）**：设定红线，防止模型产生幻觉或执行违规指令。 4. **决策是否保留人在回路**：根据业务重要程度，决定关键节点是否由人工最终签批。 此外，企业高管也开始对超强前沿模型本身可能沦为新型网络攻击面表达出浓厚担忧，AWS 正在这一前沿安全领域持续投入应对方案。 ### `50:00` 机器速度的安全防护与 Amazon Q 的全员落地 谈及安全领域的变革，Matt Garman 指出，AI 对客户环境而言既是真正的风险，也是巨大的机遇。为此，AWS 最近推出了名为 Continuum 的新服务，利用强大的前沿模型来协助客户保护自身环境。Continuum 会全面扫描客户的基础设施以查找漏洞，并凭借对环境上下文、权限配置以及现有补偿性控制措施的深度理解，智能评估风险等级并给出优先级建议。Matt Garman 强调，未来的安全防护绝不能再依赖人类的迟缓反应，正如他所断言：“因为听着，在未来的某个时刻，客户将需要以机器的速度实现安全防护，而不是以人类的速度，也不是等警报响了才让人进去排查。”（“Because, look, at some point, customers are going to need security at machine speed, not at human speed, not at, like, an alarm. Someone goes in and looks at it.”） AWS 自身也在践行这套理念。Continuum 本质上就是把 AWS 内部成熟的安全实践封装并开放给外部客户。在亚马逊内部，AI 与 Agent 的使用已经渗透到了日常运营的方方面面。AWS 向每一位亚马逊员工全员部署了 Amazon Q。借助这一工具，以往需要耗费数周的团队规划与资源管理工作，现在单兵作战仅需数小时即可完成。财务团队甚至自己构建了 Agent 来抓取各地税法并验证合规性。业务线员工不再被软件工程团队排期所卡住，能够自主解决阻碍并加速创新。目前，无论是初创企业还是跨国巨头，都在全量推行 Amazon Q，通过无缝访问企业数据和应用 Agent 来提升全流程效率。 ```mermaid flowchart LR accTitle: Amazon Q 赋能跨职能团队的工作流 accDescr: 展示企业数据通过 Amazon Q 驱动开发、安全、财务等不同部门自主构建 Agent。 subgraph Input["企业底层基石"] Data["企业内部多模态数据 / 权限上下文"] end subgraph Platform["智能底座"] Q["Amazon Q 平台"] end subgraph Teams["业务赋能场景"] Dev["软件工程: Agent 驱动全量编码"] Sec["安全运营: 机器速度风险优先级研判"] Fin["财务合规: 跨区域税规抓取与核验"] HR["组织管理: HR 政策制定与团队资源规划"] end Data --> Q Q --> Dev Q --> Sec Q --> Fin Q --> HR ``` ### `52:15` Agent 优先的研发革命与组织架构的重塑难题 Raghu Raghuram 提到亚马逊一向以严谨度量一切指标而闻名，并追问在广泛采用 Agent 后，收益最显著的领域到底在哪里。Matt Garman 毫不犹豫地指出，最大的飞跃发生在软件开发以及整个产品研发体系上。在过去一年左右的时间里，AWS 内部被称为“前沿团队”（Frontier Teams）的研发小组彻底转向了“Agent 优先”（Agent-first）的工作范式，这已经完全脱离了单纯代码补全的范畴，而是由 Agent 编写全部代码，人类工程师转型为管理 Agent 团队的调度者。AWS 一向以快速迭代功能著称，而 Agent 的加入让发布速度得到了涡轮增压般的飞跃，以满足客户几乎永无止境的新功能渴求。 这种开发范式的剧变直接对传统组织架构提出了挑战。Raghu 询问未来组织应如何进化，Matt Garman 坦言行业目前尚未有标准答案，但 AWS 正在进行激进的内部实验。过去一个项目往往需要由 10 个人组成的团队长期负责特定功能；而在 Agent 辅助下，同样的工作仅需 3 到 4 个人就能以惊人的速度交付。由于产品产出过快，管理者反而需要考虑如何迅速将工程师调配到新的挑战中去，同时在庞大体量下兼顾既有系统的运维。Matt Garman 概括了这一探索期的状态：“人类管理 Agent，Agent 也会管理人类；在很长一段时间内依然会有大量人类参与。我确实认为组织形式会发生改变，我目前还没有神奇的现成答案，但我们正在积极思考这件事。”（“Agents manage people, people manage agents. There's going to be lots of people for a long period of time. I do think organizations will change. I don't know the magic answer yet. But we're actively thinking about it.”） - 组织敏捷性剧变：传统需要 10 人长期驻留的功能团队，现在精简至 3 至 4 人即可在极短时间内完成构建。 - 研发流程颠覆：工程师职责从逐行手写代码，升级为统筹、管理多个专注特定任务的 Agent 集群。 - 运营挑战：如何在快速抽调人员应对新项目的同时，保障庞大基础设施既有资产的长期可靠性与维护。 ### `54:40` 结语与合规声明 对话进入尾声，Raghu Raghuram 与 Matt Garman 共同回顾了这场深刻的产业变革。Garman 对 a16z 及其生态中的众多初创企业客户表达了诚挚感谢，并指出与这些站在技术最前沿的初创团队切磋交流是 AWS 不断进化与学习的核心源泉。Raghu 对 Garman 的倾囊相授致谢，强调关于云端 Agent 架构的探讨未来还将持续深入。节目的最后，主播提醒听众关注 a16z 官方 Substack 邮件通讯，并播报了标准免责声明，重申节目内容仅供参考，不构成任何法律、商业、税务或投资建议。 ## Notable quotes > `26:38` It turns out there's never one constraint. There's always just the latest constraint. — Matt Garman > `15:00` You're right, we could sell every single GPU or AI accelerator we had to probably just the big Frontier Labs and call it a day. We choose not to do that because we actually want to keep growing the full ecosystem. — Matt Garman > `52:15` Agents manage people, people manage agents. There's going to be lots of people for a long period of time. I do think organizations will change. I don't know the magic answer yet. But we're actively thinking about it. — Matt Garman > `03:09` From the very beginning of when we launched AWS, startups have been the lifeblood of the core of what we do. — Matt Garman > `50:00` Because, look, at some point, customers are going to need security at machine speed, not at human speed, not at, like, an alarm. Someone goes in and looks at it. — Matt Garman > `00:00` Agentic workflows tend to perform better on AWS than anywhere else. — Matt Garman ## Mentioned AWS, Amazon, Matt Garman, Raghu Raghuram, a16z, NVIDIA, Trainium, Graviton, Inferentia, Amazon EC2, Amazon S3, Amazon Aurora, AWS Context, Amazon Bedrock, Agent Core, Firecracker, AWS Nitro, Annapurna Labs, Amazon SageMaker, Continuum, Amazon Q, Andy Jassy, Anthropic, OpenAI, Gemini, TSMC, Hugging Face, 《目标》（The Goal） ## Video description a16z’s Raghu Raghuram sits down with AWS CEO Matt Garman to discuss how AI is reshaping the cloud, from the needs of AI-native startups to infrastructure increasingly designed for agents. Matt explains how AWS is adapting as agents write code and manage infrastructure, why it’s reserving scarce GPU capacity for startups, and where custom chips like Trainium and Graviton fit into the AI stack. They also discuss Amazon's $220 billion capital investment, the shifting bottlenecks in the infrastructure buildout, what enterprises need to trust autonomous agents, and how AWS's own teams are building with agents. [Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/e1b4fcb1-9600-4c7b-a712-186a9c63f2fd/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=e1b4fcb1-9600-4c7b-a712-186a9c63f2fd&feed=JGE3yC0V)

[26:38] Matt Garman: It turns out there's never one constraint. There's always just the latest constraint. [15:00] Matt Garman: You're right, we could sell every single GPU or AI accelerator we had to probably just the big Frontier Labs and call it a day. We choose not to do that because we actually want to keep growing the full ecosystem. [52:15] Matt Garman: Agents manage people, people manage agents. There's going to be lots of people for a long period of time. I do think organizations will change. I don't know the magic answer yet. But we're actively thinking about it. [3:09] Matt Garman: From the very beginning of when we launched AWS, startups have been the lifeblood of the core of what we do. [50:00] Matt Garman: Because, look, at some point, customers are going to need security at machine speed, not at human speed, not at, like, an alarm. Someone goes in and looks at it. [0:00] Matt Garman: Agentic workflows tend to perform better on AWS than anywhere else.

[50:00] Matt Garman: At some point, customers are going to need security at machine speed, not at human speed, not at, like, an alarm. [52:49] Matt Garman: The agents write all of the code, you're just managing a team of agents and driving that. [26:14] Matt Garman: There's never one constraint. There's always just the latest constraint. [45:00] Matt Garman: If you're running inside of Bedrock, your data doesn't go back to the model provider. They never see your prompts. It stays inside of your own trusted environment.

[0:00] Agentic workflows tend to perform better on AWS than anywhere else. Compute sandboxes, gateways, agent permissions versus people. A lot of those are things that we have built and are building and thinking actively about. The top frontier labs gobble up every available GPUs. At the same time, you want to promote newer companies that are going to become the enterprises of tomorrow. How are you thinking about balancing? From the very beginning of when we launched AWS, startups have been the lifeblood of the core of what we do. what's possible. We're very intentional about keeping capacity available for the startups. We recently announced we're going to be buying 2 million NVIDIA GPUs over the next couple of years. Your capex this year is what, $200 or something? $220 billion for 26. And we don't anticipate slowing down anytime soon because the demand is just massive. With all the debate around AI extension risk and the hugging face attack, what are CEOs asking you about all these things? Amazon plans to spend $220 billion in capital this year. Thank you. and AWS CEO Matt Garvin says they don't anticipate slowing down anytime soon. In this episode, E16Z's Raguraguram sits down with Matt to unpack what's driving one of the largest infrastructure buildouts in history, and how AI is changing the cloud itself. They discuss how AWS decides who gets scarce GPU capacity, the shifting bottlenecks from power in ships to memory in construction, and the company's bet on custom silicon like Tranium. They also get into what changes when agents increasingly write code and manage infrastructure. And that shares what AWS is seeing inside its own teams, where agents now write code and engineers increasingly manage teams of agents, accelerating how quickly new products can be built. Welcome to the pod, Matt. What a time we're living in. So I have a lot of topics to talk to you about. Awesome. Thanks for having me. I'm excited. Yeah, absolutely. So let's start actually right from the beginning. So you are the First GM for EC2, and that was 2006, right? And today you guys are what, 160, 170 billion in revenue? Yeah, about 169, 170 billion, yeah. They are growing... 37% at 169. Crazy. There's a ton of, it's interesting to think about it from day one, the first dollar revenue. But yeah, the interesting thing is we're still at the early stages of what the business can be and what the opportunity is for customers. Most workloads, there's a huge amount of workloads still that live on-prem today. And the amount of compute that people are doing every single day is more than it was the day before. And so you see the tailwind from AI, you see the tailwind from migration into the cloud. And the business has grown really fast. It's been a super fun thing to be a part of. Yeah, it is. It is. I mean, they'll be writing history books about this and business books about this for a long time to come. I want to touch on the on-prem. That sort of boggles my mind. I obviously did my best to keep them there for a long time. You built a lot of stuff on-prem back in the day. Yeah, yeah. Trying to now get all of that to move into AWS. Yeah, so we can talk about that later. But so if you think about it. EC2 in the early days, and you got your start with obviously selling to startups. And today, the AWS cloud service, I mean, as you reflect on the evolution, what stands out? Yeah. Part one, and then part two, we'll talk about how the nature of how you serve startups has changed. Sure. Well, like you said, actually, it's funny. So I actually interned for AWS in 2005 when we were first kind of was an internal project. It was my business school internship. And my project was actually to come up with an analysis of who we thought AWS would be most interesting to. And the answer was startups, like surprisingly. And so from the very beginning of when we launched AWS, startups have been the lifeblood of the core of what we do for a number of reasons. One is the value proposition is just so attractive, what AWS provides to startups. sure that we're great partners to the startups, helping them not just provide infrastructure, but also advice on how to get your company up and running and how to think about your architecture so it'll scale eventually and a whole bunch of things that we do for the startups. But we also think that for us, it's just good business because the startups today are the enterprises of tomorrow. And it's an imprecise number, but today we estimate that maybe 30, 40 percent of AWS revenue comes from companies that was a one-time startup in AWS's lifetime. So it's fun to see for us to over time and to get bigger and become enterprises effectively. And so that's why we invest so much to startups and why we pay so much attention to the brand new two people in a garage type startups. And not just because of business outcomes, but also because that's who we learn from. These are the innovators that are at the edge of technology, understanding what's possible, pushing our services to say what they'd want more of, what can help them go faster, what can help them achieve their outcomes. And a lot of times they're pushing more than the banks or the governments, the start [5:00] which are the ones that are pushing that envelope. And it really helps us to be better and make sure that we're ahead of that wave where the banks are going to want some capability five years down the road that startups want today. Yeah, yeah. And compared from then to now, how have the startups changed and what they want from you? Obviously, they all want a lot of GPUs and we can talk about that. But besides that? teaming an idea and all of a sudden they're worth a billion dollars. Just go from our offices to your offices. That's right. That's right. And like the size and scale and ambition of ideas, I think, requires more capital. They're bigger to start with. They're also more expensive, too, right, to go after kind of a training a model or doing something that a lot of the folks are doing today. So that's number one, which is just the size that they start from is really big. But I would say number two is some things haven't changed, right? They actually are still thinking about, okay, once I scale, how do I think about an architecture? How do I think about security? How do I think about performance? How do I think about having all the capabilities that I need? How do I think about setting up my IAM setup so that when I have more than three employees, this thing is going to work and scale? Which is a lot of times why they like AWS as opposed to just going to a NeoCloud or something like that. from training. And so I think that's something that hasn't changed and it's exactly the same. I think the scale at which the ramp up is definitely different today. I also think increasingly we're seeing Teams that want a cloud that is great to work with agents and not just with people. And I think that's where we've spent a lot of time. How do you think about, exactly, how do you think about broad scale, how do you think about performance, how do you think about an interface that is a well-defined API interface that agents can actually easily traverse and work across. And so that's also something that we both, I think, were naturally set up to do well, but that we've actually doubled down on to ensure things like how can you start a database in three seconds and really get to some capabilities that agents are excited about. Have you introduced any specific new services that are explicitly targeted at agents? Are people building agents? I'd say what we've done is we've optimized some existing services so that they can both work for people and agents. And so the answer is we definitely have some services designed for building agents. So we have things like Agent Core and Bedrock and pieces like that. But when we look at just taking an underlying component like S3, which is where... the vast majority of companies store their data and have their data lakes. It turns out like a lot of the use cases for people and agents are similar, but You want to have, and one of the things that we have in preview right now, or in beta, is, It's called AWS Context that allows you to build a context layer so that agents can actually more easily find all of the data they want across your various data lakes. And so whether you have your data stored in Aurora or you have it in S3 or you have it in somewhere else in AWS, you can kind of build this context layer where when people aren't necessarily going to access data in that way, but agents actually are happy to go across lots of those different things. And so there are some services that we're building like that. But for the most part, you also care about what does latency look like? What does throughput look like? fast, scalable, They actually care a lot about tail latencies, which is interesting. Yeah. Where people are, they don't always care about the P999. S3 latency, like agents do care and get blocked by that. And so that's the thing that we've cared about for a long time. And from a performance perspective, it's one of the things that really pop. That's why agentic workflows tend to perform better on AWS than anywhere else. Yeah. Obviously, we see a lot of companies starting out, and the very common refrain, of course, is that agents are writing all the code for them, right? And agents are selecting the databases and the email servers and everything that you can name it, right? Yeah. And AWS... like your, feels funny to say, quote unquote, legacy services that have been around for a decade on how do you re-architect them? I think, so yes, I think there are some things that we've thought about. Actually, the core underlying building blocks, I think, are in really good shape. I think there's a usability layer that we're thinking about of how we make easier to use for, we'll call it the very simplest, the use case you call out where somebody is just coding up an app really quickly and they ask it to deploy. codecs, they'll say, I want to build on AWS. Here's my credentials. Here's my stuff. This is how I want to deploy it. And then agents are great and they go do it. There are use cases, though, where you're just you're already not on a cloud. You're just trying something. You say deploy. And frankly, a lot of times that'll go to some of our partners that have an easier to use layer on top. True love, by the way, too. And we love our partners on those fronts, too. But I do think that there are some things that we're doing. [10:00] Where if you're brand new, you don't already have an AWS, you already haven't set up your IAM. A couple of months ago, it was much harder to start up an account, right? It's been like this for 20 years. But you have to define your VPCs, you have to define your IAM roles, all these kind of things, which are actually super important things once you get to be large. And what we hear from customers is like it was a hard tradeoff because they know that they're going to want those. in months or years down the line, but for now, they just kind of want to use the services, not worry about that. And so what we've actually launched is if you go create a new AWS account today, And we're slowly rolling this out. I don't know if it's actually full all the way out yet now. But you don't have to do those things. You don't have to give it a credit card. You can sign up with your Gmail account. You get to just, all of those things are handled in default behind the scenes. And within less than 30 seconds, you're up and running and can be operating an AWS, full AWS account, which is, you know, for those type of systems, much more what the agents want to be able to do. Because they don't want to have to go through all of that setting up your VPCs and pieces like that. of use to that, which we're quite excited about. And I've seen some really positive feedback from customers on it. And we'll keep doing more things like that over time. The great part about that, though, is it's not like a simplistic account and then you have to migrate. If you basically say, OK, now I actually do want to scale into the, I want to develop an organization. I want to go and kind of fine tune some of the things. You can easily come just, you're already in a real AWS account and you can actually then go and do all of those things later when you need them. And so there's no migration or move later. about, which is how do we make it not a choice for the customer, but an easy on-ramp into the depth of features that we know customers and startups are going to want when they start scaling. Yeah, yeah. Yeah. and um So, What has been... some of the hardest things to accommodate as agents have taken over. I mean, you made a name. The way you guys became, the phenomenon that you became was serving developers, right? And then InfraTeams. Now developers are substituted by agents, and pretty soon infra teams are being substituted by agents. What have been some of the hardest things for you? you as the largest service provider in the world to handle that transition? Well, you know, look, I think, like I said, much of our infrastructure is... is actually pretty well set up to handle the scale, which is good. I think there's been a couple of things which are interesting, which are, I think, interesting paradigm shifts where you could argue that some of our systems are... Thank you. I wouldn't say over-engineered, but as an example, many agents want to create a database that, do a little bit of work and then have the database go away. You really need that database to have five nines of durability. Exactly. Right. And so there's some things that we rethink there where when you're creating an Aurora database, like does your production database, like you do want five nines of data, you know, you need durability, you want availability, you want all of those things. And for the agent use case, that's arguably, or maybe not arguably, like overengineered for what we need. And so, you know, it's hard, we don't, you know, And we kind of have this... belief that we don't really want to have a non-durable option that's going to cause problems either. Because you never really quite know if the database that's created wants to stay around for a long time or a short amount of time. And so we're trying to do the hard work to think about how do you accomplish both of those things where you can create it quickly and throw it away and you don't really waste a lot of resources. But if you do want it and it can be durable and stay for a long time and actually grow into a big production database. So those are some trade-offs that we traditional kind of, this is going to be my production system kind of capabilities. match with some of the more transient nature of the infrastructure that agents want to use. So that's one, I think. But there's a bunch. I think the scale and speed and latency of creation of durable resources is another one that's interesting. I think the other one that we are there new building blocks that agents are going to want that we just didn't really need before? And so compute sandboxes, gateways, agent permissions versus people or service role permissions. A lot of those are things that we have built and are building and thinking actively about because they are just brand new building blocks. It's not using the existing building blocks differently, but it's brand new ones where pretty clearly, you want a different permissions for agents. You don't just want to give it. Yeah, exactly. And let it go do whatever you can do. You actually want, you know, very time box, short-term permissions to just go do a task. You actually may not want to give permissions to a whole tool at all. You may want to get very fine-grained commissions of what an agent's able to do from a sandbox, right? You actually want a sandbox that you want to… [15:00] The organization is having its act, too. That's right. That's right. Exactly. And they're different. It's similar. It's the same idea, but it's, you know, how do you have that lightweight and think about. And fortunately, we've done a lot of work with Firecracker, our kind of micro VMs. In fact, a lot of the sandbox companies, startups use Firecracker. They all use Firecracker, which we invented 10 years ago, maybe, or something like that. And it's really fascinating. purpose-built for, I mean, it wasn't purpose-built for agents, but it's actually quite good for agents because you can spin them up really rapidly. They have a great security boundary, and you don't have a lot of that virtualization overhead that you have from a traditional large VM. So, you know, I think those are some new building blocks that we're thinking about, and more emerge every single day, you know. So, I've not told enough agents, I'll come back to it later, but it's a fascinating topic. the. every one of our startups face, and we get asked most frequently, which is, How can we get GPOs, right? Yeah. I mean, obviously, you guys are running... massive GPU farms that are increasing every day. But the top frontier labs gobble up every available GPUs, more power to that. How do you deal with the internal, at the same time, you want to promote newer companies that are going to become the enterprises of tomorrow, like you said. How are you thinking about balancing the capacity needs in these small companies that don't have a lot of credit and whatnot? Well, yeah, it's a great question. And a couple of these things it more challenging too. One is the CapEx expense needed to go deploy kind of all of the compute that everyone needs right now is massive, right? And so we are, I think... So your CapEx is what, $200 or something? $220 billion for 26. At one point I saw that it's, you know, it's a pretty, I mean, that is a larger expense than we've ever had, maybe any company has ever had in a single soon because the demand is just massive. And so at some point, you're limited by how fast can you build data centers, how fast can you deploy capital, how fast can you get memory and chips and all those kind of things. And so all of those are different points, constraints that were, you know, whether it's Power, data centers, capital, memory, chips, they're all kind of constraints at various times. Construction people that build buildings, like, you know, construction people are at a premium today. And so all of those, we, you know, we work really hard to make sure to happen. And then with the capacity that we are able to deploy, which is still a huge amount and not enough, we know, we really think intentionally about what is that allocation strategy. customers. And so those folks are really good customers of ours. Yep. And we want to make sure that we invest in them. We have large enterprise customers, whether it's Salesforce's or JPMC's or, you know, other large companies that have demand for fewer numbers of GPUs or accelerators. Sometimes they want Traniums, sometimes they want NVIDIA GPUs. But we want to make sure that we can support them as well. And we're very intentional about how we make sure that we have capacity for startups. And so what we do is we actually do allocate and we basically say, OK, we're going to keep... You're right, we could sell every single... GPU or AI accelerator we had to probably just the big Frontier Labs and Cloud a Day. We choose not to do that because we actually want to keep growing the full ecosystem. They get a large number, but we want to keep supporting a broader set of customers because we actually think that both the whole ecosystem will be more healthy. For us, there's some depreciation, but it's also just we know these are going to be big companies over time. And so we try to support them. I saw recently that we say yes in some way, shape, or form to something like 60% of the requests we eventually get. Sometimes it's Sometimes it's a slightly different configuration than the customers are looking for. But we really try to lean in and try to allocate as much as we can. And every single startup that you have wants more. Yeah, exactly. And so we're working hard to try to make sure that we have all of those. But, you know, it's hard. Like, that's, you know, like, this is, as a lot of people say, it's a good problem to have. Yes, it is. But it's a problem nonetheless. And we're, you know, we continue to look at, you know, we recently announced we're going to be buying, you know, 2 million NVIDIA GPUs over the next couple of years. Like, we're landing a massive amount of capacity. 2 million. 2 million. So, you know, it's a lot. And it's over the next couple of years. But, you know. And who knows if that's enough? I have no idea, but at some point, again, we're limited by other components as well. But it is. We're very intentional about keeping capacity available for the startups. And I know it's painful to not have enough, but we keep pushing. When did the [20:00] your CapEx... Cross... I mean, pure AWS. I mean, Amazon was a bigger company. Yeah. cross like, let's call it even 1 billion or 10 billion a year? Oh, I don't know. I'd have to go back and look. I'm not sure about that. But it's definitely scaled up over the last couple of years in a pretty meaningful way. Yeah, the AI build out has definitely ramped our CapEx spending. And so I'd say the last, you know, call three or four years, our CapEx has definitely accelerated pretty, pretty meaningfully. I don't know when we crossed a billion, but given we're 220 now, it's probably a while. I mean, we've been spending CapEx for a while. And the company about funding the, and obviously the AWS business is a good one that we like to invest in. Yeah, yeah, of course. And, um... And, you know, I think we were, for the longest time, actually, we, you know, we were investing ahead of where the demand was. And, you know, I think one of the most painful things is that with the real ramp of GPUs, like a lot of the elasticity has unfortunately kind of gone away. And so hopefully we'll get back to it. And in our core compute and storage and things, that elasticity is still there. that we're making now, which is, you know, I think I get lots of questions sometimes about how you feel about that spend and like, are you nervous about a bubble and other things like that? And I will say, you know, we... because of the position we have, like one, we do take this diversified approach. And so not all of our capacity is bundled up in one customer. And I think you go to some of these, whether they're NeoClouds or some of the other providers out there, and you'll see sometimes concentrations of 30, 40, 50, 60% with one or two customers. We're nowhere near that, obviously. We're single-digit percentages at the highest, and usually it's less than that. For one, I think we have a lot less risk on one particular customer. But also, because we have that rich set of services, like, AWS is where people really are coming to launch their production workloads. And so the majority of our usage today actually is either core compute and storage and inference, which is part of that application. And so those are the workloads that I think just aren't going to go away. Because we see enterprises getting positive ROI. You go talk to the customers and you say, at the capability today and the cost today, are you seeing positive returns to your business? And almost to a person, they'll say like, oh, yeah. And so you're like, well, that's not going to go away. depending on that. And so, you know, and you know this, like the VC model, like is every billion dollar startup going to make it? No, they won't. But, you know, that's kind of the game. And that's been true for 50 years. They haven't always been billion-dollar stars. The numbers have changed, but the principles haven't changed. But the principles are the same, right? You bet on 10, and one makes it and pays for the other 10, or whatever the percentage is. Hopefully it's higher than one out of 10. The famous power law. Yeah, right? And so, you saw that with the internet, where there was a bubble and a bunch of internet companies didn't make it, and the internet still thinks. And a lot of the companies that had durable businesses, the Google, the Amazons, the others, like that, they... did pretty well. And so for us, we think we'll feel really good about that investment and the continued investment going forward. You guys have a view of the demand that's unparalleled, right? Because you're seeing across the globe, you're seeing across every segment, enterprise and the big labs and the AI native companies and so on and so forth. So if anybody should call it, you should be able to call it first. I hope so. Yeah. That's our plan. And honestly, we spend a lot of time thinking about it. We're very intentional about how we spend our shareholder capital. And we think that we're making great investments. We have a lot of good protections of how we intentionally invest that money. But yeah, we're very bullish. And I think Andy's been public about saying this. The potential for AWS is really, really large. And over the next decade, the potential is there. And anytime you have an opportunity that's that big, you want to invest to go after it. As the scale of these numbers go larger and larger, right? Has your planning and process dramatically changed in terms of, I mean, you're not writing billion-dollar checks. You're writing like $50 billion checks or $20 billion checks or whatever it is. Yeah. Yes and no. I mean, I think, look, a lot of times we're still very bullish about the investments and lean forward. But the process, so there's a lot of things that have completely changed. How do you even estimate 2028 demand? And there's things that we have to think about now that we just never had to think about. power company to give us another... couple of megawatts or whatever it was, right? Tens of megawatts. And they would just give it to us because 10 megawatts wasn't that much. And that was plenty for us to keep growing. Now we have to bring our own, we bring our own power. And so we pay for power projects, we pay for renewable projects. We're one of the biggest renewable power purchasers each year for the last, tenure [25:00] And so we're regularly bringing on new solar projects, new nuclear projects. So are you talking about behind the meter or are you talking about working with the operator? We'll do both. And so both of those things. And so oftentimes with these power projects that we bring on, we'll pay the capital and pay for the project. And then it'll go into the grid and then we'll get credit for that. So we'll bring those on. Sometimes we'll do behind the meter too. It's a mix. At the scale that we're doing, you have to think about all of those things. you you know, 20 years out of how you're going to think about, um, power, how you're going to think about transmission, how you're going to think about that capital, and that's stuff we've never had to think about before. That's planning we just never had to do. But, you know, we always had to, it's, you know, the other things is when we used to think about server demand that we needed, we'd, you know, we'd have... multiple quarter demand things and we talk with our suppliers and things. Now we work multiple years out just because the size is so much that we have to think about what do we need for 26, what do we need for 27, what do we need for 28. But it's also one of the value that we bring to customers. That's a thing that legitimately customers can't do themselves. They're not going to do power planning. They're not going to plan their memory footprint in 2028. They can't do that. And so that's one of the real values broad set of customers is just, that's just a whole set of things that you don't have to worry about. And then that we spend a huge amount of time thinking about. Yeah. Yeah. You guys are, Thank you. Woo-hoo. generally, I think for the record, the largest buyers of practically every component of a server, correct? I don't know that. I am sure that we're one of the bigger... purchasers of components out there for sure. And who knows about everyone? And that kind of depends on how you think about them and how you measure. So that I was leading to, where do you see the... the constraints being most severe, let's call it 27, 28, and where do you see the constraints easing up? Yeah, it's funny. So I'll answer this in a roundabout way, but I remember that – in undergrad, we actually, this was a long time ago, and I never thought this would be like a useful book that I read, but we read the goal. Oh, of course. And so it turns out there's never one constraint. There's always just the latest constraint. And so you have to think about all of them. And as soon as you hit one, there's another one. Right. And so, you know, what do you just like, what is the constraint that's going to happen in 20 and 28? Like, I don't think there will be one. I think it's like every month for us, it's do you have enough power? And then as soon as power is no longer the constraint, you know, it might be memory. It might be. TSMC capacity. It might be, you know, HBM. It might be networking components. It could be, you know, there's a blip somewhere in the supply chain and like, you know, connectors or whatever it is, right? At some point, you have to think about all of those pieces. And it's... It's not also the... kind of where they happen matters too. You know, so it may be like, hey, we have a ton of power in... Indonesia, but we don't have enough in Germany. You think about where in the world you want that capacity too, because it turns out that not everything is totally fungible. Some is and some's not, but it might be disk drives, it might be SSDs. We think about every single component and we have, you know, I don't know. tens, hundreds of thousands of components, all that we track and think about. And some we rely on our suppliers to manage. Some we direct, much of it we directly manage. And, yeah, we have a whole team that does that, and they're fantastic. They're industry-leading. And we saw this problem coming probably a decade ago and really started not just thinking about, okay, how many servers do we need to track, but just thinking all the way through the supply chain, you know, four tiers, five tiers down. What is the component that could cause an issue for us and making sure that we had kind of guaranteed supply on that. If you think, if you remember, gosh, I don't remember when this was. and there was like the floods in Thailand and no one had disk drives anymore. And so that was... There was disk drive crisis, and then there was memory crisis. Exactly. And so, you know, it's like, I think we think through all of those things. And so we also think about where is there a first occasion in manufacturing, like all of those kind of pieces we try to work through. And we're never going to be perfect on it, but there's always a different supply constraint. Yeah. So, um, Thank you. Obviously, there's a lot of wide-ranging debate about data centers, right? And – It's clear that folks like us, where we stand, but do you think as an industry, we have not done a good job of explaining... why data centers are good for America. generally in the world, but... Yeah. And how do you... What's the internal talk... [30:00] amongst Andy's team on how do we deal with this? Well, look, I think you'll hear more from us over this, and I agree. I think we need to be more vocal and be more upfront, because we actually do a ton. That's really beneficial, both for communities we operate in. We think a ton about how do we bring renewable energy to these data centers? How do we think about being water positive? Actually, our data centers use a really, really small amount of water. We mostly use free air cooling. How do we think about where we are and how we bring high-paying jobs to the communities we operate in. And not all data center operators do that. I think there are some well-chronicled examples of others out there that are not great at that, and they just don't really pay attention to regulations. They think that the rules don't apply, or they just launch really quickly without thinking about those. And I think that it causes a problem for the whole industry because everybody kind of gets lumped into that. And so, look, I think we'll be, I think you're right, vocal about. the benefits that we do bring and think about additional ways that we can help communities understand the benefits that we bring to them, both for the services they use, right? If you usually go to a community and say, well, do you not want to use Netflix? And they'll be like, no, no, I still want Netflix. And it's important for us to think about and highlight the benefits that we bring where I recently saw a report where one of the communities that we operate in, the, the, Everybody in that county pays $5,000 a year less in taxes because of the taxes that we bring to that. And we don't tell them. They don't even know it, right? It's just invisible to them. And so I think we just need to be more... clear about those benefits that we bring. Because I think if you told the communities, by the way, your tax bill is $5,000 less than it would otherwise be if we weren't here, they might have a little bit of different thought about the building that's over there. But not everyone does that. Not everyone kind of... Presumably there's no transparency that's needed on the plans. I think much of the data center community, not just us, is actually pretty good actors. And there's just a few that aren't... that kind of, I think, have caused some of the angst recently. And I think we just need to be a better job of highlighting, you know, who's being good citizens and who's not. Yeah. And before we leave the hardware topic, I want to touch upon Tranium and your whole history with building your own chips. We were one of your first partners using Nitro a long time ago. And since then, Graviton made tremendous progress. So what was the thinking that led to saying, look, we're going to do our own thing? Yeah, it's actually a fascinating story, and I think it's a great example of where – Amazon AWS will innovate. and will iterate over time and continue to think bigger about what we can do, but kind of prove our way there as opposed to, you know, and so as an example, um, It's probably about 13, 14 years ago. we were seeing that there was a pretty significant virtualization tax on the overall number of resources. And... And we were kind of thinking about how do we, our customers were telling us, I want bare metal performance. And they, you know, they're comparing having all the resources of a server. And so the first thing that we did is that we took a network offload card and virtualized all of our network virtualization and pulled it off into an offload card. So that network virtualization got closer to bare metal performance. And back then it was not quite bare metal, but it was closer. And then we... got really excited about that. And we said, okay, what if we could move storage virtualization off as well? Right. And, and, um, and none of the network offload cards could do that. And then we, we found this one company who had some armed cores on an offload card and they were doing it for other reasons. I can't remember their original purpose, but we're like, could you use those to do storage virtualization and some of these other functions? And we're like, maybe. And so we really iterated with them. This was the Annapurna team. Yeah, this is the Annapurna team. And, um, and just love that team, like really innovative, really mission driven, really wanting to solve problems. And so we acquired them and we said, "Look, could you build us a Slightly bigger car that actually could take all the network virtualization off and basically give us a bare metal server?" that has no virtualization on it, no VM virtualization, everything is through APIs on the card. And because we had this view that, one, performance would be much better, resource utilization would be better. And security isolation would be much, much better. And we tell people, you know, could then legitimately tell people, we have no access to any of your VMs that are running there. And this has been a huge benefit for us for the last decade, where, frankly, like we have been leading, and others have been kind of slow to do this, that people can do. But, um, So we got to that and we basically said, look, we're making a decision. [35:00] a lot of progress here. What if we take, you know, and there's a bunch of ARM cores that run this offload card and we said, what if we turn that into a server? And we did that first with Graviton. It was a very underpowered... very small server that we launched. And customers were excited. They're like, I'd love to have an ARM server. This is super interesting. And so we went down the path and Graviton, you know, and part of what we did is we looked and saw that there's the slope of, you know, arm cores were getting faster and where you saw the power utilization and the graph. And you knew the intercept was going to happen for where this architecture was going to be really good for parts. And they just needed somebody to drive the ecosystem and get some of the pieces in place. So we did that with Graviton. And Graviton has been a runaway hit at this point. Have you been public about what percent of your fleet is Graviton? We land more Graviton ships every year than Graviton. than any other type. So it's very popular. And it's, you know, look, there are 20%. cheaper at a 20% better performance and have been like that for the last five, six years. So that's an easy value proposition. And I think the vast majority, something like 90 plus percent of our top 100 customers all use Graviton in some way, shape or form across their fleets. And so That's been a huge win for us and for customers. It's been the single easiest way that customers lower their bill is to move to Graviton. We have examples where people have moved their whole fleets and cut the number of servers they had in half. performance is so much better. It's amazing. And so half his number of servers, each server costs less. Like it's a big win. And so then about five, six years ago, we said, you know, we saw the rise of AI compute happening. Not nearly expecting what it was today, but still saw it was going to be a big material mover. And so we went in and built our first chip in Tranium. And we're now in market with the third generation in Tranium 3. And I've seen fantastic results. So it's, you know, we're out... for capacity through probably towards the end of next year or something like that. And we're trying to, again, we're trying to figure out how we can save some capacity and get startups to be able to use some of the capacity because it's, we see great results. So most of, you know, the majority of traffic on Bedrock all runs on Tranium. And we have great deals with both Anthropic and OpenAI to build on top of Tranium, as well as a set of smaller startups. building on top of Tranium now too. And so now, The name suggests it's a training chip. Yeah, we're bad at it. But everybody's using it for inference, too. So where is the lean architecturally, and where is it going? It's a good point. Look, the vocally self-critical, we're terrible at naming, and so it's not our strength. Originally, we had a chip called Inferentia for inference and Tranium for training. And then as the models got bigger and bigger, and it turns out you actually want to run the inference on these really large systems, it turns out that Tranium is actually maybe the best inference chip on the market right now. from an absolute performance and cost performance point of view. And so... Is it a better memory, memory bandwidth? Where is the... It's just architecture is a little bit different than... than others and it's much cheaper. And so from a cost performance perspective and absolute performance perspective, training was great. And so we're, you know, we use it a ton for, for inference. And as I said, bedrock, it drives much of the bedrock inference today. And, but it's also a good training chip too. And it's, you know, I think we're, we, we, you know, a lot of our, broad set of customers usage is not in training models, but is in using it. And so that's where a lot of people get to use them to the covers. And that's where we're excited about it. But, um, but a lot of the big customers are interested in it for training clusters as well. And particularly as you get to training three and four, um, um, which we've announced, we haven't launched training before yet, but announced it, folks have kind of looked at that architecture and said, yeah, that's the future of where all my training clusters to be as well. So we're quite excited about the future where that goes for these really broad-scale training clusters also. But yeah, it's both. Now, let's get back to talking about agents, but from a perspective of large enterprises or large medium enterprises. Yeah. Where are they in their adoption? And have they, what sort of benefits are you seeing them reap already? And what is the roadmap for them as far as you can tell from your vantage point? Yeah, it's a really good question. I think it's one that... that we've spent a lot of time thinking about. And when I talk to customers all out there, um, today, they view, you know, they're getting a lot of value out of what they've done today. And I would say the agents that most enterprises have built are relatively simple and straightforward. [40:00] and they're starting to think about, and they're mostly non-autonomous, right? They're still kind of people in the loop, if you will. And so I think we're, and by the way, there's, customers are still getting lots of value out of that today. And so they're, really thinking about how do I have these be autonomous, but in a safe way. And I think there's two things that I think hold customers back today from just continuing to scale. And it's already a pretty big business today, but I think it has a massive opportunity to really change the, every single customer out there and every single workflow and really thinking about it. And so, number one is just how to think about it. I think what we originally saw was that enterprises had a workflow and they're saying, great, I would have an agent go do the same workflow. And what we encourage them to really do is think not just replicate, you know, Bob does step one, two, three, four, five. So agent is going to do one step one, two, three, four, five, and then Bob's going to check it at the end. That's not really the model you want. You want to actually step back and say, if I want to accomplish something, how can an agent do it differently? It can do it in a massively parallelized way. It can try 50 different things and get to that. And how do you help it get to that right? outcome and rethink how a computer would solve a problem versus a human solving a problem. And so one of the things is us just helping customers understand how to think about that and really kind of have that blank slate, because that's where you really get value is not just replicating what you're doing today, but thinking from a greenfield approach about how you go solve a problem completely differently. And I'm sure that's how many of your startups are thinking about this too. How do you help customers greenfield solve a problem, not replicate the thing that happens today. That is number one. People running fleets of agents and swarms, whatever you want to call them. Very common these days. And you just want to think about it. And so enterprises are not as again, this is one where you learn from the startups and you try to, how do you apply that to an enterprise world where an insurance company is not necessarily as forward-leaning, but they would love to figure out how they can have a better approval workflow or something like that. So that's number one. But then the second one is this workflows and how do you actually trust the agents? And so we're spending a lot of time thinking about how do we build services to help enterprises feel like they're systems are secure and that they can trust an agent to make a decision, that they can have the right guardrails, that they can have the right permissions on their data, that it's not going to delete production systems, that it's not going to make tragic mistakes. And right now, I think that nervousness, It's probably holding people back. Maybe appropriately, by the way, is holding enterprises back from just saying, okay, go nuts. Like you don't actually want an agent to just go crazy and accidentally delete a production database. That's going to be pretty bad. And so that's how we're kind of actively working through this with customers on how do we both help them architect and frankly invent new technologies and capabilities that are going to help them solve that problem. And so that is one of the areas where I think we'll continue to innovate and we'll get there. good ideas and some good technologies brewing that I think can really help. So are enterprises learning how to do eval systems and so on and so forth to keep the agents straight in the world? They need help. Honestly, both evals, how do you have a constant loop of testing? How do you think about goal-seeking in a reasonable way? How do you have your data labeled in such a way that it actually even makes sense that the eval can actually kind of... approximate what you're going to be doing in production, how do you measure in production, and backtest it so you're not seeing drift. All of those things are problems that enterprises don't know how to solve today. Actually, I don't know if anyone really is great at solving these today. It's why you've seen so many FTEs Teams kind of spin up and AWS and our partners are really leaning into the FDE motion to go and help. And this is the single biggest area where customers need help. And when we think about how you... My view is we want to train our customers to be able to go and do this themselves, right? This is not the traditional motion where I want to have a, people-driven business that goes on forever where you just keep paying consultants over and over and over again. Our view on how FTEs should work is we want to go into a customer who's ready to accept the, you know, to really kind of accept owning this when we're done. And in 45 days, do work where we can teach them how to make an eval, teach them how to get their data in a labeled way, do work alongside with them. And then at the end of 40 to five days, you leave and that customer is, is good and ready to go and trained up. And that's what our customers tell us they want. They don't want to be beholden to an external workforce for the next five years. But they need help today. Yeah, you've made a massive investment in FTEs. So taking that even one step further, right, some of your industry peers have said, look, you can't have all of your data collected going into a big frontier model, what enterprises should really do is to take an open source model, [45:00] and then posturing on your own data and workflows and traces and whatnot. Where do you stand on that? Are you seeing customers actually trying to do that? Or do you guys, how do you think about that? It's great. The first point, I wholeheartedly agree on that first point. Like the customers, like enterprise data is their most valuable asset. And so from the very beginning, it's why we built Bedrock like we did. We have a guarantee that your data never leaves us. your VPC. And so if you're running inside of Bedrock, Your data doesn't go back to the model provider. They never see your prompts. It stays inside of your own trusted environment. And so that is why enterprises kind of run, they prefer to run on top of Bedrock. And it's why you see that business growing massively, like hundreds and hundreds. I mean, it's every month we see that just business, just every week we see that business exploding. whether you're using open models or, um, or closed frontier models. I think Bedrock is a great solution that our customers told us, by the way. If you remember, three years ago, I got a lot of heat. Speaking of bad names, that's a good name, though. Yeah, Bedrock is a good name. That's good. But we got a lot of heat, actually, for being slow to the AI world because we actually built... the foundations of this where we said, look, we're not just going to rush out a service. We really want to think about how do we make sure that we protect our customers' data and build a service that we think is going to be durable for the use cases that we knew about. And if you remember, we got a lot of heat and we said, look, we're going to go build the right thing. And now as people move from proof of concepts to production, vast majority of them are landing in AWS on Bedrock for much. One of the reasons is because of this. It's also because of the set of services that we have. We also offer open models. We offer proprietary models. offer a whole set of capabilities around those. Agent core, we build these building blocks so it's easier to build agents with any of the models that you want, whether they're in bedrock or out of bedrock. For that matter, you can use Gemini or other things for it. But I think it's a differentiating piece for us and it's a super important thing to think about because having that data go back into the model provider I think is a dangerous thing. You talk about open weights models though. I do think that there's a and scenario that I'm excited about. Um, uh, we're, we're really ramping up our support of open weights models and trying to build a good, um, environment. And frankly, this is where today, I think, and I think it's true, a lot of customers believe that they have meaningful proprietary data, that if they could mix in, you know, do some post-training, do some fine-tuning to an open-weights model that they could distill down, they could actually get a better-performing model at a lower price. There's a bunch of pieces here that have to work out well. They actually have a good eval to actually prove that that's true. Most people are doing that in SageMaker today. I think there's more that we can do to make that easier. But actually, like, if you go look at where people doing that, they actually do it in SageMaker on AWS today. They actually host the inference on Via SageMaker. SageMaker's getting a new lease of life. I mean, it is. It was always kind of a model building platform, right? And now, you know, there's... If you think about what enterprises are doing in this world, that's what they're doing is they're basically effectively building their own custom models. And so that is, SageMaker is a great place of doing that. And I think there's some things that we need to keep building on to make that easier and easier to do and to test across different open-wide models, things like that. But that's a, it's an evolving space. I think it's a super interesting one. And it's one we want to make sure that we have all the right things for customers to be able to do if they have the right data and expertise to actually go down that path. debate around AI extension risks and this that and the other, and the security issues. vulnerabilities and so on and so forth in the hugging phase attack. However, enterprise What are CEOs asking you about all these things? Yeah, there's a bunch. And they look, they mostly want to say like, and it goes back to this, like when I launch agents, how can I trust that they're going to do what I want them to do? And so we're increasingly, we have been for a while. Like we're basically, we're, heavily investing in building capabilities that allow people to deploy agents safely into their environment and think about those controls. And some of those are, how do you make sure the agents have the right permissions? How do they have the right sandboxing? How do you make sure that you have the right set of guardrails? How do you really intentionally think about what you want the agents to do and not do? Is there a human in the loop or not? And so we spend a lot of time with our customers thinking about how do you think about safe agent deployment and get better over deploy agents safely into their environment. So there's a lot there. The other angle on that, which a lot of people are worried about, which is just... are some of these really powerful models going to be attack surfaces and, and kind of mythosphere. And so I kind of have a view on their, their, yes, that is a, [50:00] real risk, I think, to customer environments. But it's also a real opportunity. And so we recently launched a service called Continuum, that uses these powerful models to help customers go secure their environment. And so we'll look across their environment, look for vulnerabilities with them. We'll use some of these powerful models and help customers find vulnerabilities they haven't found before. And most importantly, by the way, prioritize which ones, because we know context about their environment, how it's set up, where their permissions are, where they may have compensating controls that make it harder or easier to do. And so Continuum is incredibly popular with customers, actually. really bullish about what's possible from AI to help with AI-powered security. Because, look, at some point, customers are going to need security at machine speed, not at human speed, not at, like, an alarm. Someone goes in and looks at it. And so that, you know, we're running fast to go help build that for customers to help them protect their environments. And I'm very excited about what the Continuum team is building on that front, too. So with the AWS itself, Mm-hmm. Thank you. like how, what's the state of adult usage of... Agents in the broad. Continuum is basically us trying to expose what we do internally. And so we use AI extensively for our own security. We use AI extensively for our own software development. We use agents. Actually, one of the things that's really cool is we use agents across our entire business. And so we rolled out Amazon Quick to every single Amazon employee. to do that a single person can now do in a couple of hours to think about kind of team planning and resource management. I have finance teams that are building projects um, agents to, to go think about how do they go pull tax rules from everywhere and ensure that we have compliance on a bunch of different pieces. And, um, super cool to see that things that used to be blocked by software developers, actually the, the, the line of business folks are able to go and unblock themselves and, and innovate more quickly. Um, and so quick has been a, an enormous blow and, and that has, has grown like wildfire. We see, um, Customers like small startups using it all the way to the largest enterprises in the world, rolling it out to their entire customer base to get the benefit of kind of being able to access all of your enterprise data and easily apply agents and capabilities to help you accelerate your jobs. And so we use it across everything from software development to security to driving HR policies. I mean, you guys are notorious for measuring everything about your operation. Where have you seen the biggest gains? Yeah, you know the answer, software development. That's the real answer. But the derivatives of that. But yeah, the speed of software development has been, and really product development as a whole, not just coding. So you've seen actuals pick up the velocity of new products. deploying new products is massively different than it has been in the past. I think you can see this where AWS has always been known for rolling out features really quickly. And we've seen a turbo boost on that in the last year or so, as we call them frontier teams, as they think about agentic development as opposed to kind of traditional development. And it's not code completion. It really is... agent first, the agents write all of the code, you're just managing a team of agents and driving that. And it's been fun to see the pace at which innovations for customers has been massive. And it has to be because that's the... That's a promise. Our customers out there have a almost insatiable appetite for new capabilities. And that's what we've got to do. Yeah. So organizationally, do you have any insights on how organizations should change? I don't. I'll say that. Agents manage people, people manage agents. There's going to be lots of people for a long period of time. I do think organizations will change. I don't know the magic answer yet. But we're actively thinking about it. Inside you're just trying various experiments? organization, you used to have a team that would own a particular kind of capability for a long time. And you might have 10 people working on that thing. Well, today you can innovate so rapidly that One doesn't have to be 10 people. It can be three to four people. And they build something so fast, you actually want to move them to different projects and problems. And so thinking about how do you both operate and maintain the things that you built while being agile and flexible to move around on an organization that's as big as AWS is active things that we're experimenting with and playing with. But it's it's fun and it's enabling for our employees. They actually love it because they can build faster and do more. But but there's real work there. Yeah, it's a fascinating time. And so thank you very much for your time. We could be talking about this for hours together, but thanks for all you. [55:00] Yeah, thank you for having me and thank you. We love having all your companies as customers and we love learning from them and appreciate having me here. Yeah, we'll keep sending them your way. Excellent. Thank you. Thanks. at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments including a link to our investments. Please see a16z.com forward slash disclosures. Thank you.