AWS Deploys In-House Silicon to Break Proprietary Hardware Lock-In

AWS部署自研芯片打破专有硬件锁定 利用自研ASIC和定制交换机软件替代传统供应商设备，使云运营商能够大幅降低延迟，并突破传统企业数据中心的扩展极限。 James Hamilton详细介绍了支撑AWS的庞大物理、电力及架构规模，重点阐述了定制芯片卸载、改良型开关设备固件、跨太平洋海底光缆以及定制空气动力学服务器机架设计。演讲还探讨了传统基础设施现代化，Infosys展示了向AWS迁移的大型机自动化流水线。NASA JPL的Tom Soderstrom透露了云弹性如何支持深空通信、火星车自主规划和系外行星计算。最后，Matt Wood博士与Hamilton探讨了由P2 GPU实例和Apache MXNet驱动的深度学习工作负载的爆发式增长，并以AWS对公用事业规模可再生能源项目的长期承诺作结。 AWS每天新增的计算容量相当于一家财富500强企业整体IT机构的规模，以跟上需求增长。 (0:01) AWS将新建物理数据中心设施的容量限制在25至32兆瓦之间（约8万台服务器），以在趋于平缓的成本效益与状态自愈网络风暴的波及范围之间取得平衡。 (20:41) 部署在AWS基础设施中的每台服务器均包含至少一颗自研的数十亿晶体管专有ASIC，用于卸载网络数据包处理和虚拟机监控程序任务。 (30:23) AWS签约采购了970兆瓦的公用事业规模太阳能和风力发电，每年输送260万兆瓦时电力，推动其全球可再生能源占比在2016年达到45%。 (91:10)

At AWS re:Invent 2016, Distinguished Engineer James Hamilton and guest presenters argued that hyper-scale cloud operations require end-to-end infrastructure redesign—from subsea cables and custom silicon to disaggregated networking, distributed machine learning, and utility-scale renewable power. Disaggregating Hardware, Custom Silicon, and Capped Failure Domains (20:41) AWS abandoned vertically integrated networking vendors in favor of merchant silicon, custom top-of-rack switch specifications, and in-house routing protocols. Hamilton stated that commercial networking firmware bloated by multi-enterprise feature sets caused avoidable crashes, whereas writing proprietary software tailored only to internal protocols yielded superior uptime. To recover host CPU cycles tied down by software-defined networking, AWS deployed custom network interface cards engineered with Annapurna Labs silicon. Offloading network virtualization, encapsulation, and traffic shaping to an isolated real-time OS on the NIC reduced latency, power, and cost tenfold while preventing hypervisor security breaches from reaching the physical fabric. At the data center level, AWS intentionally caps single-facility power to between 25 and 32 megawatts (roughly 80,000 servers), despite the engineering feasibility of 250-megawatt builds. Hamilton explained that while capital and operating efficiencies flatten logarithmically as facilities grow, the blast radius of an unexpected outage expands linearly. AWS also rewrote commercial switchgear firmware to override ground-fault locks, forcing backup diesel generators—which reach 1,800 RPM in 7 to 8 seconds—to accept load through utility disturbances rather than stranding data center capacity. High-Density Metro Optics and Resilient Subsea Cabling (11:14) AWS inter-region traffic routes entirely over a Amazon-controlled private backbone composed of redundant 100 GbE links, insulating data from public transit carrier peering disputes. At sea, the 14,000-kilometer Hawaiki transpacific cable operates at depths of up to 6,000 meters. Its repeaters are powered by a copper sheath carrying positive and negative 10,000-volt direct current; if a fishing trawler or anchor grounds one conductor, the system shorts to sea water and maintains uninterrupted operations across the opposing line. Within metro regions, AWS deployed high-density optical cables packing 3,456 fibers into a standard 2-inch conduit by folding optical ribbons into structural V-grooves. Hamilton noted that while long-haul subsea transit justifies expensive Dense Wavelength Division Multiplexing (DWDM) transceivers to squeeze 100 wavelengths onto rare glass pairs, short metro runs make pulling dense unamplified fiber far more economical. True Availability Zones are physically isolated data centers separated across flood plains and utility feeds, connected by 126 redundant fiber spans and dual Transit Centers. Enterprise Legacy Migrations and Planetary Science at Scale (46:42) Navin Budhiraja of Infosys announced a dedicated AWS mainframe migration practice, estimating that global enterprises spend over $100 billion annually maintaining 30 million active MIPS on aging architectures. With the COBOL programming workforce declining, Infosys deployed an automated platform to support three migration patterns: emulation rehosting, automated refactoring to Java or C#, and microservice re-architecture. Demonstrating public-sector scale, Tom Soderstrom of NASA's Jet Propulsion Laboratory detailed how upcoming earth-science missions like SWOT and NISAR generate 100 terabytes per day with continuous 100 Gbps ingestion rates, far exceeding NASA’s on-premises facilities. JPL leveraged Amazon EC2 Spot instances to cut processing costs and migrated Mars 2020 path-planning simulations to AWS, reducing daily command sequence computation from 20 minutes down to 5 minutes to prevent the rover from losing operational Martian sols. Deep Learning Acceleration and the Adoption of Apache MXNet (74:40) Dr. Matt Wood of AWS framed artificial intelligence momentum around a convergence of algorithms, broad cloud data, and on-demand GPU infrastructure like EC2 P2 instances holding up to 40,000 CUDA cores. Addressing the programming complexity of neural networks with thousands of layers, AWS selected Apache MXNet as its primary deep learning framework. MXNet unifies imperative code execution with declarative symbolic graphs, optimizing memory across Python, Scala, Julia, Go, and C++. Wood shared scaling benchmarks showing that MXNet achieves 91% linear efficiency across 16 GPUs within a single instance, and retains 88% efficiency when scaling out across a 16-node cluster containing 256 GPUs. While AWS committed engineering resources to MXNet open-source tooling and selected it as the foundational engine for managed AI services, Wood affirmed that AWS will preserve platform flexibility by supporting competing frameworks such as TensorFlow and Caffe in a manner analogous to Amazon RDS database engines. Renewable Energy Integration and Grid Capacity Expansion (88:00) Addressing operational sustainability, Hamilton reported that AWS increased its renewable energy share from 25% to 45% by year-end 2016, despite doubling its global infrastructure footprint over the same period. While AWS could theoretically claim 100% renewable power by concentrating workloads in regions like Oregon that run entirely on clean energy, latency demands force the company to build data centers in regions where local utility grids remain reliant on fossil fuels. To bridge this gap without restricting customer deployment options, AWS contracted 970 megawatts of new generation across six solar and wind installations, supplying 2.6 million megawatt-hours annually to the broader grid. Hamilton likened this total clean capacity to the 1,300-megawatt output of the Bonneville Dam, framing utility-scale additions as the primary mechanism to achieve the company's long-term 100% clean energy mandate. AWS's hyper-scale strategy relies on replacing proprietary hardware and third-party software with internally designed vertical stacks. However, this model ties reliability to Amazon's internal execution across custom silicon fabrication, firmware development, and rapid renewable generation builds that must continuously outpace fleet growth.

AWS副总裁兼杰出工程师James Hamilton在re:Invent 2016上指出，公有云的根本壁垒在于通过端到端的全栈软硬件定制、严格的物理故障域隔离以及极致的供应链整合重塑计算经济学。多位演讲者展示了AWS底层基础设施如何推动传统企业脱离遗留大型机，并为深空探测、分布式深度学习与可再生能源电网提供支撑。 超大规模弹性、物理隔离与专用网络 (2:06) Hamilton指出，AWS每天新增的硬件规模保守估计相当于吞吐一家财富500强企业的整体IT计算容量。以2016年Prime Day为例，流量峰值要求系统弹性扩容数万台虚拟服务器，其计算体量相当于一座建造成本在1.5亿至2.5亿美元之间的中型数据中心，传统私有IT架构因漫长的采购部署周期不得不为偶发峰值常年闲置资产，而云端在流量激增前上线并在回落后即刻释放。在物理布局上，AWS当时已运营14个全球区域并规划扩建4个，拥有68个边缘节点（PoPs）。Hamilton强调，真实的可用区（AZ）由具备独立电力、冷却与防洪平原的分离实体建筑构成，而非竞争对手通过单间仓库隔板划分的形式主义。 设施规模权衡与自研硬件垂直解耦 (20:41) 针对单体数据中心规模，AWS将新建站点容量严格限制在25至32兆瓦（约80,000台服务器）。Hamilton解释，虽然建设更大容量在工程上可行且初期具有对数级成本优势，但随着规模递增其边际收益平缓，而单点故障引发的爆炸半径（Blast Radius）与流量自愈冲击却呈线性上升。为解决商用网络设备的成本与稳定性瓶颈，AWS打破了传统厂商软硬一体的垄断模式，采用博通Tomahawk等商用芯片（Merchant Silicon）配合自研协议栈与定制交换机规格，不仅规避了商用固件冗余代码带来的系统崩溃，还通过单波长25 GbE光学技术大幅削减了光模块成本。 在服务器架构端，AWS自2011年起逐步将软件定义网络（SDN）的虚拟化封包处理任务从宿主机CPU剥离至定制硬件网卡。借助Annapurna Labs研发的定制芯片与专用实时操作系统（RTOS），网络虚拟化运行延迟、功耗及成本均降至原有软件路径的约十分之一，彻底释放了宿主机计算核心并构建了底层安全隔离边界。此外，AWS为解决商业电气开关柜在电网抖动时因误判接地故障而闭锁备用发电机的缺陷，自行重写了底层固件，确保柴油发电机能在7至8秒内升至1,800转并在15至20秒内强行带载。 遗留大型机替代与深空科学计算上云 (44:32) 在传统企业IT现代化方面，Hamilton认为30年前基于发射极耦合逻辑（ECL）和水冷散热的IBM 3090等大型机时代已经彻底终结。Infosys高级副总裁Navin Budhiraja披露，全球企业界仍有约3,000万MIPS的大型机算力在运转，年均运维支出超1,000亿美元；加之COBOL维护人才加速断层与数字化实时交易冲击，Infosys联合AWS推出了涵盖重托管（Rehosting）、重构平台与深度重构的三层自动化迁移框架。 深空探索领域同样因数据爆发倒逼计算架构迁移。NASA喷气推进实验室（JPL）首席技术与创新官Tom Soderstrom介绍，即将发射的SWOT与NISAR雷达对地观测任务每天将产生100 TB数据，带来持续100 Gbps的摄取流量，已彻底超出NASA本地数据中心的承载极限，必须全面依托AWS及EC2竞价实例进行低成本弹性处理。同时，借助云端仿真与机器学习，火星2020漫游车的每日路径规划窗口得以从20分钟压缩至5分钟，有效避免了探测器在火星表面因计算延迟而整日停滞。 深度学习分布式扩展与清洁能源承诺 AWS人工智能总经理Matt Wood博士阐述了驱动现代AI的“算法、数据与实用计算”三要素，并指出P2 GPU实例等高密度算力正在解决数千层复杂神经网络的训练瓶颈。AWS宣布正式将开源框架Apache MXNet选为官方主推的深度学习引擎，同时保持对其他框架的开放兼容。基准测试显示，MXNet在单台P2实例跨多GPU扩展时保持了91%的理论线性效率，在扩展至包含16台实例（256块GPU）的分布式集群时仍维持88%的高效率，已在斯坦福大学糖尿病视网膜病变自动筛查与图森未来（TuSimple）自动驾驶厘米级三维视觉定位中落地。 针对基础设施能耗，Hamilton重申了AWS最终实现100%使用可再生能源的长期承诺。尽管AWS全球计算容量年增长近一倍，其可再生能源供电比例在2016年底已从上一年的25%提升至45%，超过了此前设定的40%目标，并规划在2017年底达到50%。AWS已在风能与太阳能领域签约采购累计970兆瓦的新增装机容量，每年输送清洁电力260万兆瓦时，其峰值发电规模已逼近装机容量1,300兆瓦的美国邦纳维尔水坝（Bonneville Dam）。 该报告系统论证了AWS如何通过底层全栈定制重塑数据中心基础设施，证明云厂商已从计算资源的倒买倒卖者转变为横跨芯片、光通信与电网工程的基础系统发明者。然而，随着单体数据中心能耗逼近公用事业级规模，如何在满足全球客户低延迟就近部署需求的同时，在依赖化石能源的区域电网中完全兑现100%可再生能源承诺，依然是AWS长期扩张所面临的实体工程约束。

## Background 该视频由亚马逊云科技（Amazon Web Services）于 2016 年 11 月 30 日发布在 YouTube 平台上。页面描述显示，本场演讲为 2016 年 re:Invent 大会的周二技术之夜（Tuesday Night Live），由 AWS 副总裁兼杰出工程师 James Hamilton 登台剖析 AWS 的超大规模基础设施创新，并邀请 NASA 喷气推进实验室（JPL）的 Tom Soderstrom 讲解如何借助 AWS 探索宇宙前沿命题。 ## TL;DR James Hamilton 深入揭秘了支撑 AWS 飞速扩张的底层重型基础设施与自研硬件演进，详述了全网 100G 自研私有骨干网、Hawaiki 跨洋海缆的电力工程、32 兆瓦数据中心的甜点容量权衡、定制交换芯片以及逆向重写电力开关柜固件等底层实践。活动中，Infosys 阐述了将企业遗留大型机向云端迁移的实战路径，NASA JPL 展现了如何借助 AWS 弹性算力支撑好奇号火星车降落、海量对地观测数据吞吐与行星轨道模拟。最后，AWS 团队详解了推动现代 AI 爆发的三位一体支柱并宣布 MXNet 为官方首选深度学习框架，同时公布了装机总容量达 970 兆瓦的可再生能源清洁电网布局。 ## Atmosphere & style 现场弥漫着纯粹的硬核极客工程气质与对未知探索的宏大激情。James Hamilton 展现出极具感染力的工程师风采，既能手举冰啤酒大呼为“大型机之死”干杯，又能条理分明地拆解两万伏特海底直流输电与柴油发电机并网电路的物理极限；随着 NASA 专家带入火星着陆“恐怖七分钟”的深空壮举与现代机器学习浪潮，整场演讲在对严苛工程细节的掌控感与人类探索宇宙的浪漫情怀之间完成了绝妙共振。 ## Bulletin - [02:06](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=126s) AWS 每日硬件交付规模保守估计相当于一家财富 500 强企业的整体 IT 体量，其全球区域与可用区均通过自研 100 GbE 专属私网骨干互联，杜绝跨运营商公网中继摩擦。 - [20:41](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1241s) AWS 数据中心单体规模被严格限制在 25-32 兆瓦（约 8 万台服务器），通过控制故障爆炸半径平衡规模经济与自愈恢复风险。 - [31:01](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1861s) AWS 全面推进自研网络芯片与硬件卸载（Hardware Offloading），使智能网卡接管虚拟化逻辑，实现约十分之一的延迟、能耗与硬件成本。 - [01:26:47](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5207s) AWS 正式宣布 MXNet 为首选深度学习框架，其在 16 台 P2 实例跨节点分布式训练中实现了高达 88% 的线性扩展效率。 - [01:28:13](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5293s) AWS 可再生能源使用比例超额达到 45% 并设定次年 50% 的新目标，旗下 970 兆瓦风电与光伏项目年发电量达 260 万兆瓦时。 ## Key takeaways - [03:48](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=228s) 传统自建数据中心因响应需求信号迟缓而被迫过度配置数亿美元资产，而云端弹性伸缩将基础设施转化为按需释放的运营成本，其核心价值在于业务敏捷度与顶线增长。 - [07:21](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=441s) AWS 的可用区（AZ）均为物理隔离的独立建筑群，配备独立的电力、散热与路由，绝非竞品在同一机房内部加设防火墙的文字游戏。 - [08:29](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=509s) AWS 坚持自建全球私有骨干网并全网普及 100 GbE，遵循“数据包经手方越少越可靠”的原则，消除第三方运营商跨网挤压与协议谈判摩擦。 - [10:21](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=621s) Hawaiki 跨太平洋海缆采用 ±10,000V DC 高压直流供电中继器，并在单极接地故障时可切换为利用海水形成回路的 20,000V 容灾运行模式。 - [17:51](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1071s) 数据中心之间通过定制折叠堆叠光缆在 2 英寸管道内容纳 3,456 芯光纤，短距城域网络采用多芯单模直连比昂贵的波分复用（WDM）模块更具成本优势。 - [20:41](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1241s) 单体数据中心建造规模存在边际成本递减收益，AWS 将机房限制在 32 兆瓦以防止数十万台服务器同时崩溃产生灾难性的网络自愈风暴。 - [22:58](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1378s) 网络设备解耦打破了传统垂直整合厂商的溢价与代码臃肿，AWS 自研交换机与精简协议栈从上线首日起可靠性便超越第三方商业设备。 - [26:42](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1602s) 25GbE 凭借单波光学特性，在成本仅略高于 10GbE 的情况下彻底击溃了由 4 个 10G 拼凑而成的高成本 40GbE 方案。 - [31:01](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1861s) 将网络协议与虚拟化逻辑卸载至搭载自研定制芯片的智能网卡，可释放主机 CPU 算力、杜绝微扰动并实现微秒纳秒级的性能提升。 - [35:05](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2105s) AWS 全面重写商业配电开关柜固件，打破厂商在疑似短路时断开发电机的传统保守策略，坚持优先供电并依靠分支断路器与发电机冗余吸收风险。 - [42:00](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2520s) AWS 优质数据中心 PUE 控制在 1.12 至 1.15，专用电源与板载调压模块效率均超 90%，在数亿美元电费基数下极微小的能效优化即可产生巨额节约。 - [46:42](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2802s) 全球企业仍运行约 3,000 万 MIPS 的大型机算力，每年产生超 1,000 亿美元开销，正通过仿真迁移、局部微服务重构与全面重写向 AWS 演进。 - [57:24](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3444s) NASA JPL 新一代对地观测卫星（SWOT/NISAR）日均产生 100 TB 数据并需 100 Gbps 传输，借助 AWS 云计算与竞价实例突破了传统物理机房瓶颈。 - [01:05:46](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3946s) Mars 2020 任务通过引入 AWS 云计算与机器学习，将每日行动规划时间从 20 分钟极限压缩至 5 分钟，避免火星车因超时通信原地空转停工。 - [01:13:26](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4406s) 现代 AI 的爆发源于算法演进、云端多样化海量数据汇聚以及按需弹性 GPU 效用计算构成的“三位一体”推动。 - [01:20:25](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4825s) MXNet 结合了命令式脚本编程与声明式符号执行器的优势，以不足 4 GB 的低内存占用支撑千层网络，并跨 16 台 P2 节点保持 88% 的高扩展效率。 - [01:31:10](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5470s) AWS 正在为电网新增 970 兆瓦风电与光伏装机容量，规模逼近 1300 兆瓦的邦纳维尔水坝，折损后年产 260 万兆瓦时清洁电能以推进 50% 绿色能源目标。 ## Detailed walkthrough ### [00:01](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1s) 重返 re:Invent：从六千到三万两千人的云计算浪潮 AWS 副总裁兼杰出工程师詹姆斯·汉密尔顿（James Hamilton）在全场热烈的掌声与动感音乐中登台，拉开了 2016 年 re:Invent 盛会的序幕。看着眼前座无虚席的庞大场面，汉密尔顿不禁感慨万千。他回忆起自己参加的第一届 re:Invent，当时现场只有 6,000 人，但客户利用 AWS 产品所构建的应用和展现出的创造力就已经彻底震撼了他。而今天，现场参会人数已经暴增至 32,000 人。面对这种爆发式增长，汉密尔顿在台上幽默地调侃道：“大家觉得‘云’这玩意儿真的会成气候吗？看起来动静可真不小。” 汉密尔顿由衷地向现场参会者表达了感谢，感谢大家在百忙之中抽身离开繁忙的工作岗位与家庭，来到拉斯维加斯与 AWS 团队共度这一周。他强调，AWS 的职责就是在本周输出足够硬核、高价值的内容，让每一位工程师和架构师在回到公司后都能显著提升生产力、助力业务成功，并满怀期待地明年再次参会。汉密尔顿随后交代了当晚的分享框架：他将首先深入讲解底层基础设施，这一方面是因为这是他日常专注研发的领域，另一方面则是因为底层架构正是 AWS 拥有独特技术优势的根基；随后会邀请多位重磅嘉宾登台，分享包括 NASA 喷气推进实验室（JPL）在内的不同视角案例；最后，他还会详细披露 AWS 在可持续发展与可再生能源领域的重大举措。 ### [02:06](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=126s) 每天部署一家“财富500强”：不可思议的供应链与基础设施扩张 在进入具体技术架构前，汉密尔顿展示了一张他个人极其钟爱的数据幻灯片。他直言，就算这张幻灯片保持三年前的数据不更新，其背后的工程体量也同样令人瞠目结舌。这背后代表着一个高度精密且近乎奇迹的全球供应链运作链条：上游元器件厂商必须按时向服务器制造商交货，服务器制造商负责制造主板、组装整机、严苛测试并打包推入物流网络；随后，庞大的物流车队精准将机架运送到指定的数据中心门口；与此同时，土建工程必须提前就绪，机房空间、高压配电、冷冻散热与网络光纤必须全部配齐，现场技术人员随后进场上架安装，并在极短时间内完成系统引导与全自动上线。 最令汉密尔顿感到震撼的并不是完成这一整套复杂动作，而是 AWS “每天都在重复这一过程”。他坦言，如果多年前刚加入 AWS 时有人告诉他这套体系可以如此运转，他一定会觉得这听起来就像阿波罗登月计划一样天方夜谭。为了让观众直观理解每天交付的服务器数量究竟有多恐怖，汉密尔顿做了一个保守的对比推算：“哪怕我们保守估计，将亚马逊每天上线消耗的服务器资源等同于一家财富 500 强公司的整体 IT 规模，这也意味着我们每天都在将一整家财富 500 强公司的硬件体量接入网络。”这种规模不仅前所未有，而且还在全速推进中。 ### [03:48](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=228s) 弹性与敏捷：Prime Day 虚拟服务器伸缩与传统自建机房的代价 汉密尔顿首次对外公开了一张源自亚马逊杰出工程师布拉德·波特（Brad Porter）内部技术演讲的弹性扩展监控图。图表横轴记录了时间轴上的关键事件，包括 2015 年 Prime Day、第四季度购物狂欢季以及业务量激增的 2016 年 Prime Day。虽然内部出于商业机密未标注纵轴的具体实例绝对值，但汉密尔顿给出了一个震撼的量级参照：在 2016 年 Prime Day 到来前夕，亚马逊在短短数天内上线了数以万计的虚拟服务器，在精准承受住了峰值流量洪峰后，又在活动结束的瞬间将这些算力彻底释放下线。 这种数以万计服务器的瞬间伸缩，规模相当于一座耗资 1.5 亿至 2.5 亿美元的中型实体数据中心“平地而起，存活数周，随后直接原地解散”。汉密尔顿以此尖锐地指出了传统企业自建数据中心的致命死穴：传统 IT 采购周期漫长，服务器从立项、采购、运输到测试耗时数月，因此对需求信号响应极其迟缓；为了不让业务崩溃，企业只能按照不可预测的最高峰值进行过度配置。汉密尔顿直言：“如果你对需求信号的响应非常缓慢，你能做什么？你只能过度配置，事情就是这么简单。”结果就是数亿美元的资产被套牢三年，大部分时间都在空转吃灰。他进一步强调，虽然云原生弹性节省了海量成本，但吸引数万人涌入云计算领域的真正核心驱动力是“业务敏捷度与顶线增长（Topline Growth）”——无需等待数月即可低成本试错，一旦商业验证成功，立即全速加倍投入。 ```mermaid flowchart TD accTitle: 传统自建机房与 AWS 云端弹性对比 accDescr: 对比传统数据中心因响应慢导致的过度配置与 AWS 敏捷弹性伸缩架构 subgraph TRAD ["传统本地部署（On-Premises）"] T1["漫长硬件采购周期（数月）"] --> T2["按极端业务峰值过度配置"] T2 --> T3["耗资 1.5亿~2.5亿美元建中型机房"] T3 --> T4["非高峰期资产闲置空转三年"] end subgraph CLOUD ["AWS 云原生弹性架构"] C1["实时监控业务需求信号"] --> C2["峰值前秒级/分钟级起服数万节点"] C2 --> C3["承载 Prime Day 海量业务洪峰"] C3 --> C4["峰值过后立即释放注销实例"] end ``` - 传统模式痛点：不仅资产利用率极低，且因为无法快速调整资源，技术团队往往在红线之下胆战心惊，迫使资本开支无度扩大。 - 云端敏捷价值：将算力伸缩转化为纯粹的运营成本，让业务创新可以随时测试并在成功后以小时为单位迅速推向全球。 ### [07:21](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=441s) 真正的区域与可用区：遍布全球的基础设施布局 在讨论全球骨干覆盖时，汉密尔顿展示了 AWS 的全球区域分布图。当时 AWS 已在全球设立了 14 个区域（Regions），并计划在次年新设 4 个区域，使总数达到 18 个，外加覆盖全球各主要城市的 68 个边缘节点（Points of Presence, PoP）。汉密尔顿在此特别澄清了云计算行业关于“区域（Region）”与“可用区（Availability Zone, AZ）”的营销话术与技术实质。 他尖锐地抨击了某些竞品厂商的“文字游戏”：有些厂商在同一个数据中心大楼的两端各放两个机架，中间砌一道防火墙，就声称这两组机架相对独立、不会同时着火或被水淹，从而将其包装成两个所谓的“可用区”。汉密尔顿郑重声明，AWS 的可用区绝非这种敷衍的产物。AWS 的每一个可用区都是彼此物理隔离的独立建筑群，具有独立的电力供应、冷却系统与网络路由，真正具备抵御重大局部灾难与物理故障的冗余容灾能力。 ### [08:29](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=509s) 自研全球私有骨干网：掌控每一跳与全网 100G 互联 汉密尔顿随后揭晓了一张从未向外界公开过的核心网络架构图：AWS 自主构建的全球私有专用网络骨干。针对业界长期以来“亚马逊是否会铺设私有光纤网络”的疑问，他给出了斩钉截铁的答案：跨越各大洲与区域之间的所有 AWS 流量，100% 运行在亚马逊自主建设、完全可控的私有基础设施上，根本不需要借道公共互联网或第三方公用中继服务。 谈到为何坚持投入重金打造全封闭私网时，汉密尔顿分享了他在分布式网络通信中的核心哲学。汉密尔顿直言：“我的准则是，如果有一个数据包，经手它的人越多，它被成功送达的可能性就越低，事情就是这么简单。”单个管理域（Administrative Domain）的可靠性远胜于跨越多个运营商网络的层层中继。在公共互联网环境中，网络服务商之间往往存在商业利益博弈与合同谈判摩擦，甚至会为了施压催签合同而恶意挤压跨网带宽；而 AWS 的私有骨干网则消除了所有中间商博弈，完全置于统一运维管控之下。更关键的是，整套网络从底层就按全冗余高可用标准设计，单条链路故障对终端客户完全无感。在带宽规格上，汉密尔顿自豪地强调，这绝非由传统的 10G 链路拼凑而成，而是全网无死角普及 100 GbE，并在流量密集的核心区域广泛部署了多路并行的 100G 密集光纤阵列。 - 全球私网管控优势：统一管理域带来端到端的服务质量保证（QoS），彻底杜绝跨运营商公网链路的丢包抖动与带宽卡脖子。 - 硬件规格指标：全网所有长途互联链路标配 100 Gbps 物理端口，以密集并行架构实现跨洋与跨大区级超大容量通信。 ### [10:21](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=621s) 铺设海缆与骨干网：Hawaiki 跨太平洋海缆的工程奇迹 James Hamilton 坦言，当团队刚开始推进构建专属私有网络时，他内心其实相当担忧，因为这类基础设施投入“极其昂贵”。但网络工程团队百分之百确信这是必须做的事情，从服务质量（QoS）的角度来看也是绝对正确的抉择。在资产获取上，团队极其注重成本效益：无论是短期租赁、长期租赁、通过不可撤销使用权（IRU）点亮暗光纤（dark fiber），还是在某些场景下直接出资铺设专属线缆，AWS 从不拘泥于某一种单一方式。正如 James Hamilton 所说：“Because we're not religious about there's one true way and we have to it, it's we get good value.” 为了具象化展示这种工程规模，James 介绍了当时刚刚在纽西兰破土动工的最新项目——Hawaiki 跨太平洋海底光缆。这条海缆全长 14,000 公里，最深处位于海平面以下 6,000 米（约 3 英里）。深入技术细节后，他感叹道：“Every time you get involved with technology you learn it's always harder than it looks.” 在看似简单的跨洋光纤背后，隐藏着极其苛刻的物理与电气挑战： - **长途信号中继**：受信噪比限制，每隔 60 至 80 公里就必须部署一个光信号中继放大器（repeater）。 - **极端运行环境**：这些中继器不仅要承受水下 3 英里的巨大水压，还必须在无需任何人工维护的情况下稳定运行 20 年。 - **高压直流供电**：中继器需要电力驱动，但无法铺设巨型粗线。工程上采用了类似陆地长途输电的方案——提高电压、降低电流，从而缩小导体截面积。光纤外层包裹着铜皮导体，运行着高达 10,000V 的正高压直流电（+10,000V DC）和 10,000V 的负高压直流电（-10,000V DC），两极间形成 20,000V 的电位差。 - **海水回路单极冗余**：如果渔船拖网或锚泊意外切断或造成其中一根导体对海短路接地（电位漂移至 0V），系统会将另一根导体的电压直接提升至 20,000V，利用海水作为导电回路（第三根导线）。这样中继器两端仍能保持 20,000V 的工作电位差，维持海缆正常通信，直至故障维修完成再恢复双极供电。 ```mermaid flowchart LR accTitle: Hawaiki 海底光缆供电与海水回路冗余设计 accDescr: 展示光纤外层正负高压供电及单极故障时利用海水构成回路的切换机制。 subgraph Normal["正常运行状态 (20,000V 电位差)"] direction TB P1["正极导体 (+10,000V DC)"] --> R1["光中继放大器 (每 60-80 km)"] N1["负极导体 (-10,000V DC)"] --> R1 end subgraph Fault["单极故障状态 (导体对海短路)"] direction TB P2["升压导体 (+20,000V DC)"] --> R2["光中继放大器 (保持正常运行)"] Sea["海水回路接地 (0V 虚拟导线)"] --> R2 end Normal -.->|"导体接地故障切换"| Fault ``` ### [15:08](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=908s) AWS 区域剖析：5 个 AZ、双中继中心与海量冗余互联 回到 AWS 全球网络视图，James Hamilton 选择剖析一个发展完全的真实 AWS 区域（Region）。他强调这绝非概念图或艺术构想，而是生产环境中真实落地的硬件拓扑。当时 AWS 在全球拥有 14 个区域，另有 4 个在建（规划达到 18 个）。每个区域至少包含 2 个可用区（AZ），大部分新建区域标准配置为 3 个可用区，而他所展示的这一核心区域拥有多达 5 个可用区。 每个区域都标配两个中继中心（Transit Centers），负责将该区域与外部世界互联。无论是 AWS 全球骨干专网、Direct Connect 客户连接（通过 PoP 点接入）、所有对等互联（Peering）伙伴，还是外部 Transit 运营商，全部通过这两座中继中心接入。在物理布线上，每个 AZ 内部全部由光纤打通，各 AZ 之间相互交织互联，同时每个中继中心与每一个可用区之间都铺设了独立光纤。这种高度冗余设计确保了即使施工挖机在错误位置挖断线路，整体服务也能平稳运行而不被外界感知。 - **126 条独立光纤跨段（Unique Spans）**：构成了该区域复杂的网状物理连接。 - **242,472 芯光纤（Fiber Strands）**：分布在这 126 条链路中，提供了惊人的总带宽潜力。 ### [17:51](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1071s) 管道创新与高密度光纤：3,456 芯缆线与波分复用取舍 面对数据中心之间海量的光纤敷设需求，AWS 遇到了严苛的物理限制：行业通用的地下管道直径仅为 2 英寸。重新开挖路面铺设新管道成本极其高昂，但细小的玻璃光纤在拉扯穿管时若强度不足极易折断，必须同时具备坚固的结构内芯和抵御地下恶劣环境的装甲保护层。 AWS 与合作伙伴联合研发并首创部署了一种超高密度光缆技术：常规做法是将光纤排成扁平带状（ribbon cables），而新技术将这些光纤带折叠成“V”形并在横截面的四个象限中多层紧密堆叠。通过这种极致利用空间的物理堆叠设计，成功在 2 英寸的标准管道中塞入了整整 3,456 芯光纤。James 兴奋地指出：“We're the first company to deploy this technology. We absolutely love it, it saved us a ton of money.” 针对网络界常有的疑问——为何不在区域内采用波分复用（WDM）技术来减少光纤数量？James 算了一笔清晰的经济账： - **跨洋长途海缆**：例如 Hawaiki 海缆，每对光纤承载 100 个 100Gbps 载波（波长），整条 6 纤对海缆可达 30 Tbps 容量。长距离敷设时光纤本身成本和放大站成本极高，多波长复用绝对占优。 - **城域短距互联**：当前技术条件下，DWDM（密集波分复用）和 CWDM（粗波分复用）光收发模块的溢价显著高于直接拉独立暗光纤。因此在区域内短途距离上，多芯独立单模光纤是性价比最高的选择。 - **未来演进**：硅光子（Silicon Photonics）技术成熟后可能会重塑成本结构，届时数据中心互联最终会向多波长演进，但在当下，绝大部分链路依旧坚持单光纤直连策略。随后，James 宣布继续深入可用区架构，揭示 AZ 背后由多个独立数据中心构成的真实面貌。 ### [20:41](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1241s) 可用区规模与数据中心的“甜点容量”权衡 James Hamilton 接着展示了 AWS 基础设施令人惊叹的规模数据。AWS 在全球拥有 14 个 Region（区域），每个区域内至少包含两个高可用区（AZ），彼此之间拥有完全冗余的网络链路相连。令 Hamilton 自己都感到震撼的是，在若干个独立的单个 AZ 内部，服务器部署量竟然已经突破了 300,000 台。 然而，在数据中心单体设施的规模上，AWS 却反常地采取了“克制”甚至看起来“倒退”的策略。以往单体数据中心普遍规划在 25 至 30 兆瓦（MW），而目前 AWS 的绝大多数新建机房严格控制在 25 到 32 兆瓦（对应约 80,000 台服务器）。Hamilton 直言，工程上建造 60 兆瓦甚至 250 兆瓦的单体机房毫无技术难度，但 AWS 之所以不建超级大机房，纯粹是完全依赖数据驱动的风险与成本权衡： “The reason is the same reason we do everything: it's data. We just use the data.” ```mermaid flowchart TD accTitle: 数据中心规模效应与故障爆炸半径权衡 accDescr: 对比数据中心规模扩张时边际成本收益与故障爆炸半径的非对称关系 subgraph CostCurve["成本效益 (对数增长)"] A["极小规模机房"] -->|"规模快速扩张"| B["成本大幅降低"] B -->|"规模继续翻倍至 32MW 以上"| C["收益曲线走平 (边际收益极小)"] end subgraph RiskCurve["故障爆炸半径 (线性恶化)"] D["32MW / 8万台服务器宕机"] -->|"AWS 整体冗余可平滑吸收"| E["客户无感知自愈"] F["160MW+ / 50万台服务器宕机"] -->|"瞬时全网流量洪峰"| G["网络自愈风暴 / 灾难性瘫痪"] end ``` 规模经济呈对数曲线发展：当设施从极小扩大到一定体量时，单位成本下降极为显著；但一旦达到 32 兆瓦之后，边际成本曲线迅速变平，继续扩建带来的成本红利微乎其微。相反，风险带来的负面效应却是完全线性的。当一个 32 兆瓦、容纳 8 万台服务器的机房发生故障时，以 AWS 当下的网络冗余规模尚可完全吸收，系统能够平稳完成自愈而不让外部客户感知；但如果建造成 16 万、30 万乃至 50 万台服务器的超大型机房，一旦单点崩溃，为了治愈系统而瞬间激增的东西向跨机房重平衡网络流量，将形成可怕的风暴灾难。因此，虽然多造中等规模机房会产生略微高一点的开销，但这才是对客户最为负责的架构方案。 - 全球拥有 14 个 Region，部分单 AZ 承载超过 300,000 台服务器。 - 新建机房标配为 32 兆瓦（MW），对应约 80,000 台服务器，避免故障爆炸半径（blast radius）失控。 --- ### [22:58](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1378s) 抨击传统垂直整合网络：自研硬件与协议栈的可靠性奇迹 谈到网络工程，Hamilton 再次情绪饱满地抨击起“老派”网络设备厂商，他认为过去垂直整合的网络架构严重阻碍了整个科技行业的创新步伐。在那种传统模式下，特定厂商一手包办了底层 ASIC、硬件电路板以及上层网络协议栈，这与数十年前大型主机（Mainframe）垄断服务器行业的封闭生态毫无二致。随着服务器体系在过去二三十年间彻底解耦，芯片、整机与系统软件厂商在开放标准下激烈竞争、共同演进，服务器才迎来了真正的繁荣；如今网络世界也终于步入同样的解耦阶段。 由于商用网络设备过去被人为维持在高昂的溢价水平，数据中心常常陷入计算资源因网络带宽不足而“搁浅”（Stranded）的窘境。为此，AWS 全面转向自研定制路由器（包括 Top-of-Rack 柜顶交换机），自行定义硬件规范，并组建了自己的网络协议研发团队。Hamilton 坦言，当初决定自己做网络设备时，最核心的驱动力确实是降低成本，但他随后见证了自研带来的更大惊喜——极高的系统可靠性： “It turns out as big as the cost gain is—and oh my God, it is a big cost gain—the biggest gain is actually reliability.” 商业网络设备之所以极不可靠，原因在于各大客户的架构师都会向厂商提出极度冷门且复杂的专属需求；厂商为了成单把功能全塞进固件，导致代码库臃肿到根本无法维护，新版本发布时甚至连厂商自己都不测试这些边缘特性，因为根本没人真正使用。而 AWS 自研路由器只有一个需求来源，那就是 AWS 自身业务。开发团队拥有极高的克制力，为了可用性坚决砍掉一切花哨特性。“手机和寻呼机响起来的是我们自己的工程师”，这种运维责任倒逼他们保持极简设计。原本 Hamilton 曾做好心理准备，认为自研初期稳定性肯定会下降并为此准备了借口，结果现实证明从上线第一天起，自研系统的可靠性就彻底碾压了第三方商业设备。 此外，当极端事故触发寻呼机报警时，AWS 自己的工程师直接手握底层源码和机房物理权限，可以就地排查并火速热修复。如果是向传统商业网络设备供应商报障，厂商必须在他们的实验室中复现故障，而全地球根本没有任何一家设备厂商拥有足以复现 AWS 这种规模并发的测试环境；一次深层故障排查往往需要消耗长达六个月之久，对云服务商而言无异于一场灾难。 --- ### [26:42](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1602s) 孤注一掷 25GbE：击破光模块物理成本的经济学 在网络演进的十字路口，AWS 做出了一项当时在外界看来极其疯狂且充满风险的决策：跳过主流路线，全面拥抱 25GbE（25G 以太网）。当时行业普遍通行的标准是 10G 和 40G，25G 属于刚冒头的新生事物，甚至市场上还面临着严重的 25G 光模块短缺风险。绝大多数无法解决光学供应链难题的公司都被迫选了 40G。作为核心决策人之一，Hamilton 深入剖析了这背后的光学底层经济学。 数据中心网络的核心物料成本主要由两大部分构成：ASIC 交换芯片以及光模块（Optics），其余硬件开销在庞大的规模面前几乎都可以忽略不计。理解网络速率的成本，本质是看光纤中的单波（Wave）传输能力： | 规格类型 | 波长通道数（Waves） | 物理实现机制与成本特征 | |:--- |:--- |:--- | | **10 GbE** | 1 波（Single wave） | 基准单波传输，历史行业标准，光模块成本为基准 1x | | **40 GbE** | 4 波（Four waves） | 本质是将 4 个 10G 波长通道拼在一起，光模块成本接近 10G 的 4 倍 | | **25 GbE** | 1 波（Single wave） | 高频单波传输，光模块物理成本与 10G 几乎持平，仅略微高出一点 | | **50 GbE** | 2 波（Two waves） | 仅需 2 个 25G 通道即可实现 50G 带宽，成本与复杂度远低于 40G | 40G 方案在物理上是“硬凑”了 4 个 10G 光波通道，导致其光纤收发器成本直接翻了近四倍，极为低效。而 25G 则是真正的单波技术，其光模块成本相较 10G 只有极小的增幅。采用 25G 技术架构意味着 AWS 可以仅用 2 条光波就组装出 50Gbps 的链路，以远低于 40G 的成本换取更大的有效带宽。AWS 依靠无可匹敌的采购体量扫清了供应商的供货障碍，坚信这种符合物理经济学逻辑的最优解必将成为全行业的未来标准。 --- ### [28:28](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1708s) 3.2T 交换芯片、生态繁荣与硬件卸载的开端 在展示正在服役的自研路由器内部结构时，Hamilton 重点介绍了其核心驱动心脏——Broadcom Tomahawk（博通战斧）ASIC 交换芯片。在发布之初，按晶体管数量计算，它是整个人类科技史上集成度最高、最为复杂的专用集成电路（ASIC）。这颗巨兽级芯片具备高达 3.2 Tbps 的全双工无阻塞交换吞吐量，意味着它可以支持 128 个 25GbE 端口在全线速满载并发的状态下运行而绝不发生拥塞阻断。 ```mermaid flowchart LR accTitle: 3.2Tbps 交换芯片与无阻塞网络架构 accDescr: 展示 Broadcom Tomahawk 芯片如何将 128 个 25G 端口聚合为 3.2T 无阻塞网络能力 subgraph SwitchASIC["Broadcom Tomahawk 交换芯片 (3.2 Tbps)"] direction TB Core["超高晶体管复杂度逻辑核"] end PortIn["128 × 25 GbE 端口输入"] --> Core Core --> PortOut["3.2 Tbps 无阻塞线速并发转发"] subgraph IndustryEcosystem["开放芯片生态体系 (摩尔定律驱动)"] Vendors["Broadcom / Cavium / Mellanox / Marvell / Barefoot / Innovium"] NextGen["6 Tbps / 13 Tbps 迭代演进"] end ``` 解耦网络垂直架构的最大红利，在于交换芯片终于能够像服务器 CPU 一样步入摩尔定律的飞速迭代轨道。当芯片与网络系统脱钩后，市场上迅速涌现出由 Broadcom、Cavium、Mellanox、Marvell、Barefoot 以及 Innovium 共同构成的活跃竞争生态；后续 6 Tbps 和 13 Tbps 的芯片将以同等的价格区间陆续推向市场。与此同时，光学领域也在借助硅光子学（Silicon Photonics）与多芯片封装技术，逐步迈入类似摩尔定律的成本与密度改善通道。 在演讲的最后，Hamilton 话题一转引出了软件定义网络（SDN）。他指出，虽然 SDN 是近年来行业热炒的概念，但 AWS 早在 EC2 成立第一天起就必须落地 SDN，否则根本无法在多租户环境下保障虚拟网络的隔离与安全。然而到了 2011 年左右，AWS 敏锐地意识到：网络数据包处理这类计算具有高度重复、全天候高频运转的特征，继续单纯消耗昂贵的通用主机 CPU 算力是极其浪费的；唯有将网络协议处理下沉并卸载到专用硬件中，才是云计算架构继续前行的必由之路。 - Broadcom Tomahawk 芯片单片吞吐达 3.2 Tbps，支持 128 个 25GbE 端口无阻塞全速运转。 - 参与开放网络交换芯片竞争的厂商包括：Broadcom、Cavium、Mellanox、Marvell、Barefoot、Innovium。 - 后续规划的 ASIC 容量已排期至 6 Tbps 及 13 Tbps，保持成本与摩尔定律同步。 - 硅光子技术（Silicon Photonics）正在解决光模块无法享受摩尔定律的技术与成本瓶颈。 - AWS 自 2011 年起启动网络协议栈向专用硬件下沉（Hardware Offloading）的技术研发路线。 ### [31:01](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1861s) 硬件卸载的优势与物理定律的限制 将网络虚拟化代码从服务器主处理器卸载并下沉到智能网卡（NIC）上，能带来诸多连锁收益。首先，服务器端释放出了更多的 CPU 核心资源供业务使用；其次，将那些难以捉摸却持续产生干扰的微小扰动（例如 TLB 刷新）彻底移出服务器。此外，系统的安全性也获得了实质性提升——即便虚拟机管理程序（Hypervisor）遭受攻破，攻击者依然无法直接控制网络，因为网卡上运行着一套独立的实时操作系统（RTOS），承载着 AWS 专属的全套自研软件体系。 ```mermaid flowchart LR accTitle: 传统架构与网卡卸载架构对比 accDescr: 对比传统网络虚拟化占用主机CPU资源与通过自研网卡硬件卸载释放资源并提升安全隔离 subgraph 传统架构["传统主机架构"] A["主机 CPU 核心"] -->|"运行 Hypervisor & 网络虚拟化"| B["业务实例受扰动 (TLB Flush)"] end subgraph 卸载架构["硬件卸载架构 (NIC)"] C["独立 RTOS & 自研软件"] -->|"完全下沉"| D["硬件卸载 NIC 芯片"] E["释放的主机 CPU"] -->|"纯净算力"| F["业务虚拟机"] end ``` James Hamilton 指出，这背后存在一条适用于所有计算层级的基本运行规律：“如果你卸载到硬件，大致来看，延迟大约只有十分之一，功耗大约只有十分之一，成本也大约只有十分之一。”（“If you offload the hardware, you run rough numbers, roughly a tenth latency, roughly a tenth the power, roughly a tenth the cost.”） 针对外界认为“必须定制网络设备才能达到 AWS 数据中心带宽”的论调，James Hamilton 予以反驳。他解释称，增加带宽在工程上其实毫不费力，无非是堆叠更多并行链路，无论用谁的商用设备都能办到，只是采买商业设备成本高昂罢了；真正具有挑战性的是延迟，因为这受限于物理学定律——光在光纤中的传播速度是不可逾越的客观极限。软件工程师度量性能习惯用毫秒，而一旦下沉至硬件，度量单位便进入纳秒与微秒级。 - 卸载到硬件的综合效益：延迟约降至 1/10、能耗约降至 1/10、成本约降至 1/10。 - 带宽问题本质上是资金问题（堆叠并行链路），而延迟问题则是受物理规律制约的工程瓶颈。 - 软件与硬件的时间度量差距：软件通常在毫秒级（ms），而定制硬件直接推进至纳秒（ns）与微秒（μs）级。 --- ### [33:34](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2014s) 跨足半导体：垂直整合的核心竞争力 AWS 已经正式涉足半导体芯片领域。AWS 部署的每一台服务器都搭载了至少一枚此类包含数十亿晶体管的自研定制芯片，部分服务器甚至搭载多枚。在底层逻辑上，硬件实现能带来极致的低延迟与高能效比，而具备硅芯片的实现能力意味着 AWS 能在组织架构内部完成全栈打通。 在同一个基础设施团队下，AWS 汇聚了自研硅芯片的数字电路设计工程师、网卡硬件架构师以及配套的底层软件开发团队。James Hamilton 强调，拥有从硬件到软件的横向与纵向端到端掌控力，是 AWS 保持创新节奏的命脉所在：“当你同时拥有横向与纵向的控制权时，我们就能保持我们习惯的速度前行，能以我们习惯的速度做出调整，能以我们习惯的速度响应客户需求。”（“When you own horizontal and vertical, we get to move at the pace we're used to, we get to make changes at the pace we're used to, we get to respond to customer requirements at the pace we're used to.”）这种架构自研能力，确保了 AWS 能够在未来服务器网络资源爆发式增长的趋势中，以极具竞争力的低成本迅速跟进升级。 - 芯片规模：单颗芯片集成数十亿个晶体管。 - 部署密度：AWS 新上架的每一台服务器内部均至少部署 1 块该定制芯片，部分机型搭载更多。 - 组织协同：基础设施团队内部同时整合了芯片数字电路设计师、硬件工程师与底层软件开发者。 --- ### [35:05](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2105s) 电力开关柜故障排查与固件重写 谈及故障排查哲学，James Hamilton 提到除了对自己系统的事故进行严格复盘外，更要借鉴业界的罕见故障案例。他援引了一家知名航空公司的灾难性断电事故：由于配电开关柜（Switchgear）故障并锁死了备用发电机，直接导致航班大面积取消，首席财务官（CFO）随后披露该故障造成了高达 1 亿美元的营收损失，相当于整整蒸发了该航司单月营收的 2%。 ```mermaid flowchart TD accTitle: 开关柜故障切断备用发电机机制 accDescr: 展现市电故障后，开关柜因怀疑短路而闭锁发电机的传统逻辑与AWS重写固件后的强切逻辑对比 A["市电电网故障 / 产生浪涌"] --> B["开关柜检测到接地异常信号"] B --> C{"传统厂商固件逻辑"} C -->|"判定存在安全风险"| D["锁定发电机回路 / 导致机房全盘断电"] B --> E{"AWS 自定义固件逻辑"} E -->|"坚持供电优先"| F["强行并网加载"] F --> G["分支电路空开跳闸隔离 / 冗余发电机兜底"] end ``` 这种故障属于极小概率事件，单家公司一生可能仅遇一次，但在 AWS 的超大规模集群中却曾发生过。James Hamilton 回忆，当时他恰好在 AWS 弗吉尼亚州的一处数据中心现场，亲历了同样的危机。数据中心的标准配电流程是：市电穿过开关柜供给不间断电源（UPS）；一旦市电中断，开关柜会先等待数秒（因为多数电网波动转瞬即逝，不值得贸然点火启动发电机）；若市电未恢复，备用柴油发电机将在 7 到 8 秒内拉升至 1800 RPM，并在 15 到 20 秒内全面接管负载。然而传统开关柜存在一个“防卫过当”的设计：一旦检测到数据中心内部可能存在接地短路，开关柜便会以“安全保护”为由直接锁定，拒绝接入发电机。 事故发生 6 小时后，开关柜厂商代表赶赴数据中心现场，并坚称这种在短路风险下断开发电机的保护逻辑是“唯一合规的安全做法”，拒不道歉。然而 AWS 数据中心负责人对此极度愤怒，因为发电机已经轰鸣运转，机房负载却白白丢失。在发现市场上所有开关柜供应商的默认保护策略都如出一辙后，AWS 决定在沿用传统商业开关柜硬件的基础上，全面重写底层控制固件，彻底颠覆了设备制造商的默认规则：即使信号提示数据中心内部疑似存在短路风险，固件依然强制切入备用发电机为机房供电。 针对这一激进决策带来的潜在隐患，James Hamilton 逐层拆解了风险评估逻辑： - **市电端偶发浪涌**：绝大多数故障实则源于数据中心外部。在 Hamilton 亲历的弗吉尼亚故障中，起因是一名酒驾司机撞倒了外部铝制电线杆，导致高压线两相短路并向数据中心输入了巨额浪涌，开关柜却误判为内部严重短路；在此类场景下，强制接入发电机完全不会受损。 - **机房分支短路**：即便机房内部真有局部短路，对应的分支断路器（Branch Circuit）也会自行跳闸将其物理隔离，次级供电架构会迅速接管，其余绝大部分服务器完全不受影响。 - **系统高层直连短路**：在最极端的高端严重短路情况下，强行接入确实可能烧毁发电机。一台工业级柴油发电机造价高达 75 万美元，但 James Hamilton 直言：“从我的角度来看，那是 75 万美元。我们非常节俭，我们绝不想蒙受 75 万美元的损失；但另一方面，我们绝对不希望丢弃负载，因此我们愿意承担这种风险。”（“And from my perspective, that's $3/4 million. We're very frugal, we do not want $3/4 million damage, but on the other hand, we certainly don't want to drop the load, and so we'll take that risk.”） - **冗余安全裕度**：AWS 所有数据中心均严格遵循“冗余且可并发维护”（Redundant and Concurrently Maintainable）的标准进行设计，即便一台 75 万美元的发电机真的在短路中烧毁，机房内配备的冗余备份系统也能在同一时刻平滑顶上，整体运行不受中断。 这类故障不仅波及航空公司，2013 年超级碗（Super Bowl）比赛期间长达 34 分钟的断电停赛也是完全相同的开关柜逻辑缺陷所致。而通过自主重写开关柜固件，AWS 彻底杜绝了此类隐患在其数据中心再次发生。随后，James Hamilton 将演讲重心引向了底层硬件设计的下一个主题：定制存储服务器。 ### [41:02](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2462s) 极致密度存储机架与 2778 磅的工程怪兽 James Hamilton 刚开始谈论硬件设计时，语气中满是工程师对纯粹物理极限的自豪。团队早前曾为单机架容纳 880 块硬盘的设计欢呼雀跃，认为这已经是不可思议的密度，然而工程团队并未止步，反而造出了一个更惊人的“庞然大物”。在标准 42U 机架内（没有通过加高机架等任何手段作弊），他们硬生生塞入了 1,110 块硬盘。 以当年的原始规格计算，单机架容量便达到了 8.8 PB；若换上当时的最新硬盘，容量将一举跃升至 11 PB。James 感叹道，一个并不算庞大的机架居然能塞下 11 PB 的数据，更令人咋舌的是这机架里光是硬盘本身的总重就达到了 2,778 磅（约 1.26 吨）。“功重比（power-to-weight ratio）”已经真真切切成为现代超大规模数据中心必须考量的物理指标。 - **单机架硬盘容量**：1,110 块硬盘安装于标准 42U 机架内。 - **存储容量**：按最初设计为 8.8 PB，若采用同期全新磁盘可达 11 PB。 - **物理重量**：仅内部硬盘组件的总重就高达 2,778 磅。 --- ### [42:00](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2520s) 反直觉的计算服务器：空旷机箱背后的能效与电费经济学 展示完极致密度的存储怪兽，James 话锋一转，带来了一台初看之下甚至可能让人觉得枯燥的计算服务器。机箱内部异常空旷，任何外行人第一眼看到都会忍不住质问：“伙计，发生什么事了？里面空得还能再塞进第二台服务器，你们到底有没有认真设计？”然而，这台服务器的所有内部留白，完全是为了极致的热学与能效架构（thermal and power efficiency）服务的。当时评估的所有高密度方案实际耗电量都更高，效率更差，而眼前这台看似空荡的设计才是真正的能效冠军。相比之下，OEM 厂商卖给客户的高密度服务器虽然外形紧凑三到五倍，却牺牲了能效，并且收取高昂的溢价。 在看不见的细节中，机箱内配备了转换效率超过 90% 的定制电源，主板上的电压调节模块（Voltage Regulators）效率同样突破了 90%。James 结合数据中心的核心指标 PUE（电源使用效率，即数据中心总输入电量与送达服务器电量的比值）算了一笔账：在 AWS 运行良好的数据中心里，PUE 通常控制在 1.12 到 1.15 之间，意味着整个数据中心在散热、配电与照明等方面的额外损耗仅占 12% 至 15%，高达 80% 多后期的电力都能实打实地输送给计算节点。由于 AWS 每年采购数亿美元的电力，电源效率只要提升微不足道的 1%，节省下来的开支就极其可观。正如 James Hamilton 所说：“1% 乘以数亿美元是个极其诱人的数字，拿来干点什么都会很有趣，所以这绝对值得大做文章。”有趣的是，某家大型云厂商前不久在技术博客上公开了一款新服务器，其架构与 AWS 许多年前的这款老设计惊人地相似，这让 James 欣然一笑。 ```mermaid flowchart LR accTitle: 数据中心电力与能效损耗分布 accDescr: 展示从电网总输入到数据中心开销及最终抵达计算节点的电力流向与PUE计算逻辑。 subgraph PowerIn["电力输入端"] A["电网总输入功率 (PUE 分母)"] end subgraph Overhead["基础设施损耗 (12% - 15%)"] B1["制冷与散热损耗"] B2["配电传输与变压损耗"] B3["照明及辅助设施"] end subgraph Servers["服务器计算节点 (PUE 分子: 85% - 88%)"] C1[">90% 转换效率电源"] C2[">90% 效率电压调节器"] C3["优化风道的空旷机箱散热"] end A --> B1 A --> B2 A --> B3 A -->|"PUE 1.12 - 1.15"| C1 C1 --> C2 --> C3 ``` - **PUE 表现**：AWS 优质数据中心 PUE 稳定在 1.12 至 1.15 区间，系统开销极低。 - **关键部件转换率**：专用电源与板载电压调节器双双达到 90% 以上的高能效转换。 - **规模效益**：在数亿美元的电力基数下，1% 的供电效率改进即可转化成数百万美元的成本节约。 --- ### [44:32](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2672s) “它已经死了，吉姆”：现场啤酒干杯大型机之死 话题随后转向了古老的大型机（Mainframe）。大屏幕上亮起旧时代巨型计算机的图片，唤起了 James 早年在 IBM 的职业记忆。他如数家珍地回忆起自己经手过的 IBM 3083、3081，以及最后的巨兽 IBM 3090 Model 600J。那台机器足有一辆中型卡车那么大，是发射极耦合逻辑（ECL）时代的绝唱——这是一种能效极低的技术，也是 IBM 转向 CMOS 工艺前的最后一代系统。为了降温，该系统直接用水冷却导热模块（TCM），模块内部通过精密弹簧死死顶在 CPU 芯片上进行热传导。 尽管这些机器是三十年前的技术奇迹，但在当今时代，它们早已属于历史遗产。James 引用《星际迷航》的名梗调侃：“它已经死了，吉姆（It's dead, Jim）。”现实中依然有许多企业在暗地里运行着这些老古董。为了帮客户走出旧时代的泥潭，James 在舞台上端起事先准备好的一杯冰啤酒，面向全场观众高呼：“让我们为大型机之死干杯！”他笑着安慰台下那些还在偷偷运行大型机的企业代表，不必为此感到羞愧，AWS 会帮助他们完成向现代云架构的迁徙。随后，他隆重邀请 Infosys 高级副总裁兼首席技术官 Navin Budhiraja 登台分享应对方案。 --- ### [46:42](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2802s) 大型机遗留系统的三大痛点：成本暴增、敏捷度丧失与人才断层 Navin Budhiraja 登台后首先调侃现场宏大的场馆规模甚至能容纳他们公司的大部分员工，接着正式宣布 Infosys 在 AWS 上推出大型机迁移实践（Mainframe Migration Practice）。尽管大型机系统稳定支撑了全球企业 30 多年，甚至当下依然表现称职，但企业为了维系这些机器所承受的沉重代价已经难以为继。 Navin 指出大型机面临的三大核心危机： 1. **费用失控**：全球业界保守估计仍有约 3,000 万 MIPS（每秒百万级指令）的大型机算力在跑，折算下来每年消耗企业超过 1,000 亿美元的维护支出；随着数据爆炸，原本为批处理设计的大型机被要求承载现代在线实时访问，账单更是水涨船高。 2. **丧失业务敏捷性**：业务部门迫切希望快速迭代推出新功能，而传统大型机架构死板脆弱，根本无法满足这种交付速度。 3. **人才断崖式短缺**：大型机开发人员年事已高，高校不再培养相关人才。Navin 分享了一件家庭趣事：“就像我问我女儿她觉得什么是 COBOL 时，她还以为是某种奇异的宝石，因为听起来很像 Cobalt（钴）。” ```mermaid flowchart TD accTitle: 大型机三大现实挑战 accDescr: 展现大型机当前面临的成本、业务敏捷性以及技能人才短缺三个维度的挑战。 MF["企业传统大型机 (Mainframe)"] subgraph Challenges["三大迁移驱动力"] C1["成本失控: 3000万 MIPS\n每年超 1000 亿美元开销"] C2["缺乏敏捷: 批处理架构\n难以应对现代实时在线需求"] C3["人才断层: COBOL 开发者老龄化\n高校停止输出相关技能"] end MF --> C1 MF --> C2 MF --> C3 ``` - **市场存量**：全球仍有约 3,000 万 MIPS 的大型机工作负载在线运行。 - **经济成本**：大型机软硬件与运维开销每年保守估计超过 1,000 亿美元。 - **代际脱节**：新生代工程师对 COBOL 毫无概念，维护团队面临严峻的自然减员风险。 --- ### [49:02](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2942s) 大型机迁移 AWS 的三阶路径与 Infosys Mana 智能洞察 面对庞大复杂的代码遗产，Navin 将大型机迁移规划为层层递进的三种路径，企业可根据自身业务节奏灵活推进： 1. **仿真迁移（Rehosting）**：将现有的 COBOL 与 JCL 应用程序直接搬移至 AWS 上的仿真运行环境中。大型机专有数据库（如 IMS）被替换为 Amazon RDS 等关系型数据库，同时适配安全认证与备份机制，仅此一步就能迅速帮企业降低 30% 到 35% 的成本。 2. **模块化重构与混合互通**：将阻碍业务敏捷发展、需要高频迭代的关键业务模块挑选出来，在 AWS 上用原生代码重构，再通过 AWS API Gateway 与留在大型机上的系统平滑互通，兼顾演进速度与稳定性。 3. **全面云原生重写（Rewrite）**：将整个遗留系统全盘替换为现代化云原生微服务，这是终极目标，虽然周期较长但能从根本上解除技术负债。 ```mermaid flowchart TD accTitle: 大型机向 AWS 演进的三种迁移路径 accDescr: 从重托管仿真、局部微服务重构到全面云原生重写的三阶段迁移架构图。 subgraph Source["原有大型机架构"] M1["COBOL / JCL 业务代码"] M2["大型机专有数据库 (如 IMS)"] end subgraph Path1["阶段一: 仿真迁移 (Rehost)"] R1["AWS 仿真运行环境"] R2["Amazon RDS 等关系型数据库"] M1 -.->|"即时节省 30%-35% 成本"| R1 M2 -.-> R2 end subgraph Path2["阶段二: 局部云原生拆分"] G["AWS API Gateway 互通"] F1["高频迭代业务模块"] F2["AWS 云原生微服务"] R1 <--> G <--> F2 F1 --> F2 end subgraph Path3["阶段三: 全面重构 (Rewrite)"] N["完整端到端云原生系统架构"] F2 -.-> N end ``` 大型机迁移之所以长期被视为畏途，根源在于系统体量庞大——历经 30 年迭代，动辄几千万行代码，且几乎“完全没有任何有效文档（no documentation）”。为此，Infosys 借助带有 AI 特性的平台“Infosys Mana”作为专家的外骨骼装甲（Iron Man suit），扫描分析历史源代码，自动梳理代码结构、业务规则以及模块间的紧密依赖关系，识别出废弃无用的僵尸逻辑，为客户输出精准的迁移评估报告与客观决策路径，解决企业对项目复杂度与周期的不确定性焦虑。 ### [52:37](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3157s) James Hamilton 引言：从阿波罗登月到 NASA JPL 的太空探索 前一位演讲者在台上结束了有关企业级最佳实践的分享，呼吁现场观众尽早加入这场云上转型的旅程。随后，AWS 副总裁兼杰出工程师 James Hamilton 重新走上台前。Hamilton 回忆起自己年轻时的过往，谈到他在第一份正式工作之前，对自己影响最深、至今记忆最清晰的并非大型机，而是 NASA 和载人航天项目。他感慨自己那种探索未知、勇于迎接艰巨挑战的好奇心与勇气，很大程度上正是来自当年的 NASA——在那时，NASA 敢于宣布“我们要把人送上月球”，哪怕在一开始他们根本完全不知道该如何实现。 Hamilton 对 NASA 取得的成就由衷赞叹，并兴奋地向全场引出当晚的重磅特邀嘉宾：来自 NASA 喷气推进实验室（Jet Propulsion Laboratory，简称 JPL）的首席技术与创新官（Chief Technology and Innovation Officer）Tom Soderstrom。在全场观众的热烈掌声中，Tom Soderstrom 登台，准备揭秘 NASA JPL 如何借助 AWS 与全新技术手段去回答关乎全人类命运的宇宙终极问题。 > “It probably many of much of my curiosity and willingness to take on tough challenges was learned from NASA taking on we're going to put a man on the moon when they have no idea how to do it and so I'm just blown away by the company I think it's just absolutely absolutely marvelous what they've done.” —— James Hamilton --- ### [54:19](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3259s) 好奇号火星着陆：“恐怖七分钟”与新一代的太空探索观 Tom Soderstrom 登台后幽默地表示，能在这么大的场馆空间里与大家畅谈“太空”，是一件极为快意的事。他指出，NASA 和 JPL 一直在探寻关乎全人类的大答案，如今随着新技术、全新人才队伍以及新型合作伙伴关系的涌现，他们正在以全新的方式寻找答案。尽管这是一个云计算大会，但他保证一定会将宇宙探索与云计算紧密交织在一起。 为了说明这种范式转变，Soderstrom 带领观众重温了四年前那个惊心动魄的历史时刻：2012 年“好奇号”（Curiosity）火星车着陆火星。那是一场极其疯狂的工程奇迹——将一辆重达 2,000 磅（约 900 公斤）的火星探测车送往 1.5 亿英里之外，在完全自主无干预的状况下，经历“恐怖七分钟”（Seven Minutes of Terror），探测器从每秒数千公里的宇宙超高速骤降至完全静止。当时 JPL 尝试了一项前所未有的创举：采用 AWS 云计算服务，使全球公众能够在科学家们从控制室看到火星照片的同一秒钟，同步看到传回的画面。大屏幕随即重现了当年控制室内控制人员屏息以待、直至听到“确认着陆，我们在火星上安全了”后全场沸腾相拥的真实录像。 ```mermaid flowchart TD accTitle: 好奇号火星着陆与数据同步分发流程 accDescr: 展示好奇号火星车经历恐怖七分钟着陆，并将图像数据通过AWS实时同步给科学家与全球公众。 subgraph Entry["火星进入与着陆 (恐怖七分钟)"] direction TB E1["高速进入火星大气层<br/>速度数千公里/秒"] --> E2["反推减速系统增压介入"] E2 --> E3["全自主悬停起重机着陆<br/>(2,000磅漫游车)"] end subgraph Ground["地空数据分发"] direction TB G1["确认安全着陆信号"] --> G2["火星表面图像传回地球"] G2 --> G3["AWS 云端实时广播平台"] G3 --> G4["JPL 控制室科学家"] G3 --> G5["全球公众同步接收观看"] end Entry --> Ground ``` Soderstrom 动情地强调，控制室里每一个人的职业生涯与心血都押注在那几分钟内。他对比了不同世代的人看待太空的方式：老一代人儿时仰望星空，看到的只是模糊的照片，后来在太空竞赛时期目不转睛地守在电视机前观看人类登月，角色仅仅是“旁观者”；而年轻一代则完全不同，他们从小手持装有 NASA 和 JPL 应用的智能手机，不再满足于仅仅“被动观察”或质疑“我们能不能到达”，而是默认“这一定会发生”。新生代渴望深度参与，他们通过增强现实（AR）、移动设备甚至亲自成为宇航员去触碰宇宙。因此，这不再是国家与国家之间的太空竞赛，而是全人类迈向太空的共同探索。 > “Touchdown confirmed, we're safe on Mars.” —— NASA 任务控制中心广播录音 - **好奇号漫游车重量**：2,000 磅（约 900 公斤）。 - **火星距离**：约 1.5 亿英里。 - **降落关键过程**：7 分钟内完成自主剧烈减速，被称为“恐怖七分钟”。 - **公众参与度**：借助 AWS 云端弹性架构，全球公众与控制室科学家在毫秒级差距内同步见证火星着陆图像。 --- ### [57:24](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3444s) 地球水资源监测：应对海量观测数据的云端革命 接着，Soderstrom 将目光拉回支撑这套太空探索体系的基础设施上。与 Hamilton 前面介绍的强大地面光纤网络不同，NASA JPL 的网络运行在深空——这就是“深空网络”（Deep Space Network, DSN）。JPL 在地球各处战略性地部署了大型巨型天线阵列，随着地球自转，确保地面随时能与分布在太阳系内外各个角落的约 30 个航天器与探测仪器保持不间断通信。公众只要掏出智能手机，就能实时查阅这些航天器的当前动态。 在保护地球家园方面，现有的两颗对地观测卫星——土壤水分主动被动探测卫星（SMAP）与轨道碳观测者（OCO）每天绕地球运行，监测极地冰盖、二氧化碳和水分，每天产生约 1 TB 的数据。然而，NASA 正在准备发射两个革命性的全新对地观测任务：SWOT（地表水和海洋地形卫星）与 NISAR（NASA-ISRO 合成孔径雷达卫星）。这两项任务将利用先进雷达从轨道向地球地表发射脉冲并反射，高精度绘制海洋、河流、冰盖乃至地下含水层与水库的水位变化。 这两颗新卫星带来的数据量是前代的 100 倍——它们将持续产生高达每天 100 TB 的数据流，传输带宽要求恒定维持在 100 Gbps 左右。这样规模的海量数据吞吐与运算，彻底超出了传统物理数据中心的承载上限，JPL 必须依靠云计算来破局。Soderstrom 坦言，这不仅仅是计算基础设施的转移，更彻底革新了 JPL 的运作方式：在 OCO 任务期间，他们发现了 AWS 的竞价实例（Spot Market），从而能够以极微小的零头成本（Pennies on the dollar）完成海量科学计算。这些云端数据将直接服务于防汛预警、旱灾预报、农田灌溉以及城市规划，为整个人类社会带来实实在在的福祉。 > “It's 100 terabytes per day, 100 gigabits per second all the time, much too big for our data centers, so what are we going to do? We're going to use cloud computing.” —— Tom Soderstrom - **现有对地观测任务**：SMAP（土壤水分探测）与 OCO（轨道碳观测），日均数据量 1 TB。 - **新一代任务（SWOT / NISAR）**：通过雷达测绘全球水体分布（海洋、湖泊、河流、冰架与水库）。 - **NISAR / SWOT 数据规模**：持续产生约 100 Gbps 数据流，日均数据量暴增至 100 TB（前代的 100 倍）。 - **运营成本革新**：引入 AWS Spot Market 竞价实例，以原本成本的一小部分实现超大规模高通量计算。 --- ### [01:00:27](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3627s) 小行星重定向任务与木卫二欧罗巴的生命探索 关于保护地球的另一个重大命题，是防范潜在毁灭性的小行星撞击。Soderstrom 透露，科学家近期在地球附近发现了多达 1,600 颗近地小行星（Near-Earth Asteroids）。如果未来真有一颗可能撞向地球，人类该如何改变其轨迹？为了验证这一设想，NASA 与 JPL 构思了“小行星重定向任务”（Asteroid Redirect Mission, ARM）。任务规划看似简洁：飞往一颗直径约 400 米的小行星，在其表面抓取一块重达数吨的巨石，将其拖拽并部署到绕月轨道上，随后派遣宇航员登石开采，并演练利用引力拖车（Tractor Beam）技术进行小行星轨道偏移偏转。 这项任务面临巨大的工程极限挑战：小行星引力极低，飞船在触碰抓取时极易直接弹飞；巨石本身可能牢固嵌在岩体中；现有化学推进剂根本无法提供足够的推力送探测器往返。为此，工程师们研发了“太阳能电力推进技术”（Solar Electric Propulsion, SEP）——利用氙气（Xenon Gas）与光子推进，仅需传统燃料的十分之一，却能产生三倍的功率与两倍的能效。如果将这种单台产生 50 千瓦电力的离子推进器进行并联组合，达到 300 千瓦总功率，就能成为未来把人类送上火星并安全带回的终极推进系统。而该项目从立项伊始就全面构建在云端，不仅能像初创公司一样敏捷迭代转向，还能与商业工业界高效协同开发。 ```mermaid flowchart LR accTitle: 太阳能电力推进与深空任务演进 accDescr: 展示氙气光子推进系统如何并联达到300千瓦动力，支撑小行星重定向与未来火星往返载人任务。 subgraph Propulsion["太阳能电力推进 (SEP)"] A["氙气离子电推<br/>(1/10燃料消耗)"] --> B["单台输出 50 kW 功率"] B --> C["多机并联系统<br/>总功率达 300 kW"] end subgraph Missions["深空探索目标应用"] C --> D["小行星重定向任务 (ARM)<br/>抓取数吨巨石入绕月轨道"] C --> E["未来火星载人往返任务"] end ``` 在演讲的尾声，Soderstrom 将目光投向了深空中最有可能存在地外生命的地点——木星的卫星“木卫二”（Europa）。孕育生命需要两大核心要素：能量与液态水，而欧罗巴这两者兼备，其内部蕴含的液态水量甚至达到了地球总水量的两倍。欧罗巴表面覆盖着一层厚达 10 英里的巨型冰盖，如同北冰洋的冰层一样在持续流动，因此地表没有任何陨石撞击坑。尽管 10 英里厚的冰层构成了屏障，但冰壳内部存在大量的液态水湖泊，被认为是太阳系中最有可能蕴藏地外生命的场所。NASA 正计划派遣一颗轨道器飞越 4 亿英里前往欧罗巴，测绘冰层缝隙并寻找着陆点，随后执行全自主的着陆任务。面对如此极端的环境与深空通信延迟，全自主着陆所需的庞大模拟仿真与海量数据处理，如果离开云计算将完全无法实现。 > “Can you imagine how much data and simulation that will take because it has to be completely automated? Without cloud computing, there's no way we could do it.” —— Tom Soderstrom - **近地威胁监控**：近期已测绘记录 1,600 颗近地小行星。 - **小行星重定向任务（ARM）**：捕获 400 米直径小行星上的多吨级巨石并拖入绕月轨道。 - **太阳能电力推进（SEP）参数**： - 推进剂：氙气（Xenon Gas）。 - 效能优势：燃料消耗仅需 1/10，功率提升 3 倍，效率提升 2 倍。 - 扩展能力：单模块 50 kW，并联至 300 kW 可作为载人往返火星的动力总成。 - **木卫二（Europa）探测**： - 水资源储量：约为地球水资源总量的 2 倍。 - 地表特征：10 英里厚流动冰壳，内部存在液态水湖泊。 - 任务距离：目标飞行距离约 4 亿英里，完全依赖云端高性能模拟以实现百分之百的自主安全降落。 ### [01:03:51](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3831s) 寻找“地球2.0”：从针孔观测到数十万台服务器的模拟 Tom Soderstrom 探讨了人类未来面临的一个宏大命题：如果未来某天人类需要向外太空迁徙，我们是否有处可去？能否找到“地球2.0”？他介绍了系外行星（Exoplanets）的研究现状。系外行星是指围绕自身恒星公转的行星，其运行规律正如地球绕着太阳转动一样。以往天文学家寻找系外行星的方法相当质朴：通过“针孔”长期注视深空中的同一位置，捕捉光能的微弱起伏——这种起伏可能源于行星掠过恒星前方导致的凌日，或是由于恒星与行星相互引力产生的轨道摆动。 在短短几年内，人类通过这种针孔观测法已经确认了 3,000 颗系外行星，另有 2,400 颗尚待证实。而在这些行星中，是否有适宜人类生存的类地行星？截至目前，科学家仅筛选出了 21 颗可能宜居的行星。但这些星球极为遥远，最近的一颗距离地球也有 4 光年。按照现有的火箭推进技术，抵达那里需要整整 1,000 年。不过 Soderstrom 指出，如果未来改用超微型纳米卫星（Nano Satellite），行程有望缩减至 20 年。这种设想在当下看似科幻，但正如几年前将重达 2,000 磅的探测器降落在火星上也曾被视作天方夜谭一样，科技创新的推进速度远超想象。未来随着詹姆斯·韦伯太空望远镜（James Webb Space Telescope）投入使用，观测口径将彻底打破“针孔”的局限。随之而来的科学计算、物理模拟和复杂数学运算，将需要数十万台服务器协同处理。正是因为云计算的存在，NASA 不再需要自行采购并维护这些庞大的硬件资产。 - **系外行星统计**：已确认系外行星 3,000 颗，待确认 2,400 颗；其中仅有 21 颗被认定为潜在宜居的类地行星。 - **星际航行距离与时间**：最近的潜在宜居行星距地球 4 光年；传统推进需耗时 1,000 年，而纳米卫星方案预计耗时约 20 年。 - **算力需求**：韦伯望远镜时代的数据解算与模拟需要数十万台云端服务器提供弹性算力支持。 --- ### [01:05:46](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3946s) 漫步火星：好奇号的发现与 Mars 2020 的五分钟决策挑战 Soderstrom 幽默地谈到火星探索，调侃道截至目前去过火星的人类“只有马特·达蒙（电影《火星救援》主角），而且幸运的是他顺利回来了”。谈及现实，好奇号（Curiosity）火星车在四年前着陆，甚至像当代年轻人一样用巨大的机械臂旋转角度拍下了火星“自拍照”以检查自身状况。火星传回的地貌照片与美国加州死亡谷（Death Valley）惊人地相似，包括沙丘地貌（如纳米布沙丘 Namib Dune），这表明约 40 亿年前火星与地球一样曾有水流冲刷，具备孕育生命的物理环境。然而，科学界至今尚未找到生命存在的确凿证据（Smoking Gun）。 为此，NASA 和 JPL 正在筹备“Mars 2020”任务以寻找生命印记，并为未来宇航员登陆铺路。在以往的任务中，地面团队每次仅有 20 分钟窗口来决定火星车次日的全部行动规划。但因为火星距离地球多达 1.5 亿英里，通信延迟与分析窗口极其紧张，一旦超出这 20 分钟，火星车当天就只能停留在原地“空转”，极大浪费了昂贵的在轨时间。通过引入云计算的海量算力结合机器学习算法，JPL 计划将这一规划周期由 20 分钟极限压缩到 5 分钟，彻底避免火星车“旷工”。云计算在空间探索中的角色已从最初展示绚丽图片的辅助工具，跃升为每日不可或缺的底层任务核心系统。 关于云计算对于火星任务的关键转变，Tom Soderstrom 总结道：“So cloud computing has gone from being a very nice to have and engaging you all with the pictures to something Mission critical every single day.” ```mermaid flowchart LR accTitle: Mars Rover Daily Operation Planning Process accDescr: Comparison of daily planning timelines between the traditional Curiosity approach and the AWS-accelerated Mars 2020 workflow. subgraph Traditional["以往好奇号流程"] A["火星数据下传"] --> B["20分钟窗口人工计算分析"] B --> C{"超出窗口?"} C -- 是 --> D["火星车停留原地停工1天"] C -- 否 --> E["下发次日行动指令"] end subgraph AWSMars2020["Mars 2020 云端流程"] F["火星数据下传"] --> G["AWS 云计算 + 机器学习"] G --> H["5分钟内完成全套规划"] H --> I["指令即时上行，零停工"] end ``` - **火星环境证据**：纳米布沙丘与水流冲刷遗迹印证了 40 亿年前火星具备与早期地球相似的生命条件。 - **Mars 2020 改进特性**： - 更强韧、更窄且具更深凹槽的新型车轮，提升攀爬陡坡能力。 - 配备专用麦克风，用于收录真实的火星环境声音。 - 具备钻探并密封保存岩芯样本的能力，留待后续载人或无人航天器取样返回。 - 搭载现场制氧技术验证装置（MOXIE 前身概念），为未来载人登火做资源准备。 - 配备对转双桨火星直升机（机智号原型雏形），在相当于地球 10 万英尺高度的超稀薄火星大气中升空侦察。 --- ### [01:09:03](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4143s) 快速原型、开源漫游车“Rovie”与全民太空探索 Soderstrom 展示了 JPL 如何借力轻量化、玩具般的原型来极速缩短航天技术的研发测试周期。现场展示了一款由桌面 3D 打印机耗费 3 美元成本制作的微型轮式机器人，它不仅在实验室中成功攀爬玻璃表面，还被带上失重飞机（Vomit Comet）并在微重力环境下成功完成玻璃攀爬测试，从荒诞想法到送入空间站验证仅用了短短几个月。 随后他正式介绍了专为科普外联设计的开源小型火星漫游车“Rovie”。Rovie 是大型火星漫游车的微缩复制版，核心软硬件均基于开源生态与商业现货：采用 Arduino 单片机、树莓派（Raspberry Pi）、开源软件以及 AWS IoT 和 AWS Lambda 无服务器架构构建。传统操控需要使用手柄遥控，而 JPL 为其接入了 Alexa 语音助手，用户可以直接通过自然语言发号施令，甚至向它询问关于火星的科学知识。JPL 宣布将全套设计图纸作为公开蓝图向全球免费开放，供学校、大学、博物馆乃至普通家庭自行复刻。Soderstrom 动情地鼓励台下所有人参与火星探索，并提到前一晚仅用一天便从零实现自动化弹性伸缩的无服务器（Serverless）黑客松，强调在场工程师构建的算力正在为人类探索宇宙铺路，未来的第一批火星漫步者可能就是台下观众的孩子。 - **3 美元火星原型**：利用桌面 3D 打印技术快速验证航天级附着机构，并在失重飞机上完成空间站模拟环境测试。 - **Rovie 漫游车构成**：集成 Arduino、Raspberry Pi、AWS IoT、AWS Lambda，并支持 Amazon Echo / Alexa 语音交互与知识问答。 - **开源共享**：漫游车机械与电气蓝图完全开源，旨在降低全球科研教育机构接触航天工程的门槛。 --- ### [01:12:03](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4323s) James Hamilton 回归：机器学习浪潮与底层系统扩展性 伴随着全场对 Soderstrom 的掌声，James Hamilton 重新登上舞台。他对即将登台的 IoT 演示充满期待，并将话题聚焦在 Tom 重点提及的“机器学习”（Machine Learning）上。Hamilton 坦言，科技行业充斥着各种言过其实的概念炒作，但他坚信这一次机器学习带来的变革是实打实的技术拐点。在他看来，机器学习必将成为未来 IT 领域占据主导地位的工作负载（dominant workload），甚至会让他此前展示的惊人云基础设施增长数据看起来“微不足道、仅仅是个起点”。 作为见证过多轮技术范式转移的资深系统架构师，Hamilton 强调解决问题的关键永远是底层系统的可扩展性。他指出，在现实工程中，“问题的规模增长速度永远比底层系统硬件的扩容速度更快”，因此构建在具备极致横向扩展能力的分布式架构之上是开展机器学习的唯一出路。随后，Hamilton 邀请 AWS 负责机器学习的副总裁 Matt Wood 登台，深入讲解 AWS 在人工智能与机器学习领域的战略布局。 James Hamilton 对此作出了极具洞察力的判断，明确说道：“It's got to be built on scalable infrastructure because it's always the case that the problem size gets bigger faster than the systems get bigger it's always the case so scalability is number one.” --- ### [01:13:26](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4406s) Matt Wood 登场：驱动现代 AI 爆发的三位一体（The Trinity） AWS 机器学习副总裁 Matt Wood 博士登台致谢并展开演讲。他回顾了近五到六年来客户对人工智能爆发出的巨大热情与业务动能。Matt Wood 指出，现代人工智能的跨越式发展并非偶发，而是由三个相互支撑的核心要素共同驱动的，即“AI 的三位一体”（The Trinity）： 第一是**算法演进（Algorithms）**。Matt Wood 提到，他早在 20 年前攻读机器学习博士学位时，基础数学与算法模型其实就已经成型；尽管核心底层原理并未发生根本动摇，但这些算法在近几年日趋复杂和精密，能够处理前所未有的非线性模式。 第二是**海量多样化数据的汇聚（Diverse Data）**。云计算的普及与 AWS 基础设施的发展，打破了传统本地数据中心的物理边界与存储容量天花板。曾经因机房壁垒被孤立的数据现在得以无上限地汇入云端对象存储与数据湖。当海量、多源且异构的数据流经不断进化的算法模型时，奇妙的化学反应便开始发生。 第三是**效用计算与 GPU 算力的按需调用（Utility Computing & GPUs）**。现代复杂深度学习算法依赖极致的并行计算能力，而云计算将昂贵的专用硬件变成了“只需一行 API 调用即可按需获取”的通用基础设施服务，使得任意规模的开发者和企业都能瞬时调度大规模 GPU 集群进行模型训练与推理。 Matt Wood 精辟概括了这一时代的转变：“And this is really being enabled by the cloud and with AWS because the data center walls that used to box in the collection of data are just melting away in the cloud where you can collect and store as much data as you need.” | 三位一体核心支柱 | 历史局限性 | 云计算时代的突破 | |:--- |:--- |:--- | | **算法演进 (Algorithms)** | 20 年前理论框架已确立，但因计算限制停留于理论与浅层模型 | 算法演化更为精密复杂，深度神经网络得以在深层架构中落地 | | **海量多样化数据 (Diverse Data)** | 受制于本地数据中心物理磁盘与存储集群上限，数据分散在孤岛中 | 云端存储使数据中心边界彻底“融化”，可无上限收集并吞吐多元数据 | | **效用计算基础设施 (Utility Computing)** | 专用超级计算机造价极高、配置周期长，普通企业无法获取 GPU 算力 | 通过简单的 API 调用即可秒级获取成千上万颗 GPU，实现算力弹性化 | ### [01:14:40](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4480s) 深度学习的崛起与三大核心挑战 数据量的大幅增长、算法精细度的提高以及超大规模计算能力，共同驱动了新一轮人工智能浪潮。其中，深度学习（Deep Learning）成为最核心的推动力。深度学习本质上是一组利用人工神经网络、通过多层分层结构进行特征检测的统计机器学习技术。它成功解决了计算机视觉、语音分析、自然语言处理、个性化推荐以及自动驾驶与机器人等领域最棘手的计算难题。 ```mermaid flowchart LR accTitle: 深度学习数据科学面临的三大核心支柱挑战 accDescr: 展示深度学习在可编程性、可移植性与性能三大维度上面临的具体挑战与要求。 subgraph P1["可编程性 (Programmability)"] A["网络层数极多（上千层）"] --> B["需要简化网络定义与开发门槛"] end subgraph P2["可移植性 (Portability)"] C["云端训练模型"] --> D["轻量部署至移动端/车载/CPU"] end subgraph P3["性能 (Performance)"] E["大规模训练性能"] --> F["低延迟推理执行"] end ``` 然而，现实场景中的深度神经网络极为庞大繁琐，某些顶尖图像识别算法包含数千层网络，结构错综复杂。讲台上的演讲者指出，这种复杂性给数据科学领域带来了三个“P”开头的根本性挑战： 1. **可编程性（Programmability）**：定义包含成千上万层的复杂网络结构门槛极高，开发过程十分痛苦； 2. **可移植性（Portability）**：模型必须具备极高的内存利用效率，以便在云端完成训练后，能够无缝移植到移动设备、车载系统或各种端侧 CPU 进行推理； 3. **性能（Performance）**：无论是训练海量参数模型，还是面向端侧用户的实时推理，系统吞吐与速度都是决定成败的关键。 - 深度学习应用覆盖：图像识别（如识别图片中是猫还是比格犬、具体犬种）、语音分析、NLP、精准推荐及自主机器人。 - 复杂网络挑战：行业顶尖图像识别模型的深度和复杂度导致人工编写和优化极其困难。 ### [01:16:26](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4586s) AWS P2 实例与深度学习 AMI 的赋能 为了应对深度学习在算力上的极端渴求，AWS 推出了新一代 GPU 计算实例 P2。单台 P2 实例最高可提供多达 40,000 个 CUDA 核心，极大地加速了大规模并行运算。此外，AWS 在 Marketplace 中发布了预打包的深度学习系统镜像（Deep Learning AMI），开箱即用集成了包括 MXNet、TensorFlow、Caffe、Theano、Torch 和 CNTK 在内的主流开源框架。 演讲者在谈到这些深度学习框架的作用时感叹：“These frameworks things like MXNet, TensorFlow, Caffe, Theano, Torch and CNTK put deep learning into the hands of mere mortals.” 开发者不再需要将精力耗费在底层的 CUDA 核心调优等繁重工作中，只需专注于算法本身。此外，AWS 还配套提供了 CloudFormation 模板，帮助开发者一键拉起具备弹性伸缩（Auto Scaling）能力的大规模 P2 集群，从而在短时间内调度极致算力。 - **P2 GPU 实例能力**：单实例最高可访问 40,000 个 CUDA 核心。 - **预置框架生态**：集成 MXNet、TensorFlow、Caffe、Theano、Torch、CNTK。 - **一键自动化编排**：通过 CloudFormation 模板支持弹性自动伸缩的集群部署。 ### [01:17:56](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4676s) 行业落地案例：医疗诊断、计算知识与自动驾驶 凭借 P2 实例与深度学习镜像的易用性，众多机构和企业在发布之初就将其应用到实际生产场景中，产生了巨大的现实价值。 在医疗健康领域，斯坦福大学（Stanford University）利用深度学习开展糖尿病视网膜病变（Diabetic Retinopathy）的早期筛查。糖尿病视网膜病变是全球致盲的主要原因之一，在美国占所有新增失明病例的 12%。该疾病会在视网膜深处产生极微小的动脉瘤，传统上只能依靠专业眼底照片进行极其细微的肉眼排查。研究表明，若能在早期发现并控制糖尿病病情，90% 的病变与失明是可以预防和治愈的。斯坦福团队训练深度学习模型自动扫描眼底图像，大幅提升了微动脉瘤的筛查效率。 在知识计算领域，Wolfram 基于这套技术构建了计算知识引擎 Wolfram Alpha。系统将网络信息结构化，并在非结构化数据上叠加抽象概念，用户可以直接提出复杂的自然语言查询（例如询问经典专辑《Pet Sounds》是谁制作的，引擎能精准解析出 The Beach Boys）。 在自动驾驶领域，初创企业图森未来（TuSimple）展示了在实际道路测试中的技术突破： - **目标与车道线检测**：在白天和夜间复杂光照条件下实现高可靠的目标与车道线识别。 - **语义分割（Semantic Segmentation）**：对路面与行驶空间进行逐像素级别的精确识别。演讲者指着演示视频幽默地补充道，视频拍摄于中国，“如果你在中国开过车，显而易见，那就是堵车”。 - **三维空间精确定位**：通过双目立体视觉相机（Stereo Vision）与深度学习算法，仅凭平面图像即可实现厘米级的车体 3D 定位。 ### [01:20:25](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4825s) MXNet 的架构优势：混合编程与极致轻量 在众多开源框架中，演讲者重点分析了 MXNet 的设计优势，阐述其如何完美平衡前面提到的三大挑战。首先在可编程性与多语言支持上，MXNet 拥有统一的前端接口与后端执行引擎。不论开发者使用的是 Python、Scala、Julia、JavaScript、MATLAB 还是 Go，前端代码都会统一编译为高效的底层计算图，确保最佳性能。 传统深度学习框架往往在命令式（Imperative）与声明式（Declarative）之间二选一： - **命令式编程**（如纯 Python/Ruby 脚本）：灵活、符合人类直觉，易于编写控制循环，但由于运行逻辑动态交织，系统难以进行全局编译优化； - **声明式编程**（如 SQL 或 TensorFlow、Theano、Caffe 的早期图模式）：优化空间极大、计算图跨语言复用度高，但写法死板，动态控制流非常繁琐。 ```mermaid flowchart TD accTitle: MXNet 混合编程与多语言编译架构 accDescr: 展示前端多语言接口通过混合命令式与符号式执行，最终统一编译到底层后端引擎。 subgraph FrontEnd["多语言前端支持"] L1["Python / Scala / Go"] L2["Julia / JavaScript / MATLAB"] end subgraph CoreParadigm["MXNet 混合模型 (Mix of Networks)"] M1["命令式模型 (Imperative)<br/>迭代循环 / 参数更新 / 特征提取"] M2["声明式符号执行器 (Symbolic/Declarative)<br/>计算图变换与全局性能优化"] end subgraph Execution["底层高效后端"] BE["统一下沉编译后端<br/>(超低内存占用 + 极高并发)"] end FrontEnd --> CoreParadigm CoreParadigm --> BE ``` “In fact that's where the name comes from mxnet is a mix of networks and this means that you can manipulate the graph of layers in the way that you might want to with declarative uh in this case symbolic executors and you're also able to mix that with some of the features of imperative programming languages such as iteration loops and parameter updates or feature extraction.” 这种混合特性让开发者既能保留脚本语言的迭代控制力，又能享受符号计算图带来的算子融合与并行调度优化。 此外，MXNet 在内存使用效率和可移植性上表现极为亮眼。一个拥有上千层的深度神经网络，在 MXNet 中运行时占用内存不到 4 GB。这种极致的轻量化特性使得模型能够轻松走出大型数据中心，无缝嵌入到计算资源受限的移动终端、物联网传感器、无人机机器人，甚至可以直接通过 JavaScript 在网页浏览器中快速运行。而在性能维度上，面对大规模训练时编写并行程序的痛点，MXNet 也在底层通过分布式引擎解决了分布式训练的横向扩展难题。 ### [01:23:46](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5026s) 深度学习框架的扩展性挑战与 MXNet 的并行优化 在训练复杂的深度神经网络时，计算复杂度会随着网络层数的增加而急剧攀升。在前向和后向传播的参数更新中，涉及跨越数百乃至数千个张量运算的复杂计算，其计算量与层数呈正比关系。网络规模越大，计算密集的特征就越明显，扩展难度也随之激增。为了在支持构建更精细模型的同时处理海量训练数据，AWS 迫切需要能够实现线性扩展的框架。 MXNet 展现出极具吸引力的架构特性。它能够自动解析数据依赖关系，将串行代码无缝转换为并行执行流，并通过协调一致的键值存储（Key-Value Store）将数据自动并行化分发到 CUDA 核心中。这种机制使得计算不仅能在单颗 GPU 内的多个核心间高效分摊，还能跨越单台宿主机上的多颗 GPU 进行扩展，极大地方便了开发者在更大规模的数据集上构建更深层的模型。 ```mermaid flowchart TD accTitle: MXNet 自动并行化机制 accDescr: 展示 MXNet 如何解析串行代码、分解数据依赖并分发至单卡与多卡 GPU subgraph Logic["代码输入与解析"] A["串行代码 (Serial Code)"] --> B["解析数据依赖 (Dependency Parsing)"] end subgraph ParallelEngine["并行执行引擎"] B --> C["协调式键值存储 (Coordinated KV Store)"] C --> D["单 GPU 核心级并行 (Multi-core)"] C --> E["单机多 GPU 扩展 (Multi-GPU)"] end subgraph ClusterScale["分布式集群"] E --> F["跨节点集群扩展 (P2 Instances)"] end ``` - **依赖解析**：MXNet 能够自动拆解原本串行的执行代码，准确识别数据依赖路径并转化为高度并行的工作流。 - **高效分发**：依托协调式键值存储系统，系统可在无需人工深度调优的情况下，将数据均匀分布在成千上万个 CUDA 核心上。 - **扩展维度**：同时支持单芯片内的多核心并行以及单节点内部跨 GPU 的协同计算。 ### [01:24:48](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5088s) MXNet 的单机多卡与跨节点扩展基准测试 为了验证 MXNet 的实际加速表现，团队在图像分析任务上运行了一组针对不同 GPU 数量的基准测试。在理想的线性扩展状态下，当 GPU 数量翻倍时，计算速度应当相应翻倍。测试数据显示，在单台机器内从单张 GPU 扩展到多张 GPU 时，MXNet 实现了高达 91% 的线性扩展效率。 然而，对于最复杂的深度学习模型和海量数据流，单台机器所能容纳的 16 颗 GPU（例如单个 P2 实例规格）仍然无法满足需求，开发者必须能够横向扩展到大规模节点集群。为此，团队在跨越 16 台 P2 实例（即总算力提升至单实例的 16 倍）的大规模集群上重复了该测试。在算力扩展 16 倍的极度严苛场景下，MXNet 的并行效率仅下降了 3%，仍然保持在 88% 的超高水准。 为了让用户能够亲自验证这些性能表现，AWS 当天发布了全新的深度学习基准测试套件。用户可以通过提供的 CloudFormation 模板直接一键拉起测试集群，登录主节点即可复现演讲中展示的全部数据图表。 - **单机效率**：在单实例多 GPU 环境下，图像分析框架测得的并行扩展效率达 91%。 - **多机效率**：集群算力扩展 16 倍（从 1 台 16 GPU 实例扩展到 16 台实例）后，扩展效率仍维持在 88%。 - **基准套件开放**：通过 CloudFormation 模板提供即开即用的基准测试代码，支持客户在自己的真实业务代码上验证性能。 ### [01:26:47](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5207s) MXNet 成为 AWS 首选深度学习框架 鉴于 MXNet 表现出的卓越性能以及客户的积极反馈，AWS 正式宣布 MXNet 成为 AWS 的首选深度学习框架。这意味着 AWS 将向 MXNet 开源社区全面贡献核心代码，重度投资提升其可编程性与开发者体验，编写更加详尽的技术文档和示例代码，并构建全套外围工具链。此外，MXNet 还将作为 AWS 即将推出的下一代人工智能服务（AI Services）的底层技术基石。 尽管 AWS 对 MXNet 倾注了大量资源，但其核心策略依然是保持平台的完全开放，确保市场上所有的深度学习工具都能在 AWS 上获得极佳的运行表现。这种策略与 AWS 在数据库服务（RDS）上的实践一脉相承——用户可以自主选择 MySQL、PostgreSQL、MariaDB、Oracle 或 SQL Server。AWS 致力于在高性能 GPU 基础设施之上包容所有的深度学习工作负载。 讲台随后交还给 James Hamilton。James 登台后对 Matt Wood 赞不绝口，并特别强调跨服务器的扩展能力才是行业里最关键的技术突破：“Did you see that multi-server scaling in MXNet? That's the golden one. Love that one.” - **战略定位**：MXNet 正式被选为 AWS 首选深度学习框架（Deep learning framework of choice）。 - **生态投入**：AWS 将在可编程性、开发者体验、示例代码、文档体系及调试工具上进行持续投入。 - **架构基础**：MXNet 将全面驱动 AWS 未来的人工智能服务体系。 - **多引擎并存策略**：全面支持 TensorFlow、Caffe 等主流框架，保障开放性与灵活性。 ### [01:28:13](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5293s) 可再生能源承诺与 50% 绿色目标 James Hamilton 将演讲重点转向可持续发展与可再生能源。早在一年多前，AWS 就公开作出了实现 100% 使用可再生能源的长期承诺。James 直言，做出这样的承诺经历了极其慎重的深思熟虑，因为 AWS 绝不空谈大话：“We're not afraid of big challenges. We will sign up for difficult tasks, but we have to see a path where we actually can reasonably get there, and we're there.” 虽然 AWS 在俄勒冈（Oregon）等区域已经实现了 100% 绿色能源供电，如果客户把所有负载部署到俄勒冈区域，确实可以立刻达成 100% 绿色能源指标。但现实情况是，客户需要将数据存放在离最终用户最近的地方，计算任务也需要就近进行。部分地区目前电网的清洁能源比例依然较低，AWS 绝不会通过剥夺客户的地域选择权来投机取巧，而是选择迎难而上，在每个区域推动本地能源结构的清洁化改造。 在过去一年中，AWS 的绿色能源比例从上一年的 25% 稳步提升。今年初团队立下的目标是达到 40%，而在业务规模飞速扩张、总用电容量几乎翻倍的前提下，完成这一目标意味着能源团队采购和接入的清洁能源必须成倍增长。James 透露，由于能源团队的高效执行，AWS 将在当年年底超额完成目标，达到 45%。与此同时，James 正式宣布了下一阶段的目标：到下一年年底，AWS 的可再生能源占比将达到 50%。 - **历史基准**：前一年可再生能源供电占比为 25%。 - **当年进展**：原定 40% 的年终目标被超额突破，年底实际达到 45%。 - **全新公布目标**：宣布到下一年（2017年）年底，AWS 全球基础设施的可再生能源比例将达到 50%。 - **业务考量**：在总算力规模翻倍的背景下逆势提升可再生能源比例，展现了庞大的净新增清洁电力接入量。 ### [01:31:10](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=5470s) 970 兆瓦清洁能源布局与闭幕致辞 为了让宏观目标落实到具体的工程实践中，James Hamilton 逐一公布了正在上线运营的一系列大型清洁电力项目。这批分布各地的风电与光伏项目单体规模极为庞大，分别包括 150 兆瓦、80 兆瓦、208 兆瓦、100 兆瓦、189 兆瓦以及 180 兆瓦。将这些项目全部相加，AWS 正在为电网直接带来高达 970 兆瓦的新生可再生能源装机容量。 为了让现场观众对 970 兆瓦的规模有直观的感受，James 分享了自己和妻子 Jennifer 驾驶游艇沿着哥伦比亚河航行、穿过邦纳维尔水坝（Bonneville Dam）的私人经历。他感叹邦纳维尔水坝是人类工程史上的惊人奇迹，而该水坝的总发电容量约为 1300 兆瓦。AWS 目前独立推动的新增可再生能源项目群，已经逼近了这座巨型国家级水坝的总容量。 针对“风不会一直吹、太阳不会一直照”的自然环境波动质疑，James 强调，在综合计算了所有场站的全年环境与气象折损系数后，这批新能源项目每年将稳定输出高达 260 万兆瓦时（2.6 million MWh）的清洁电能。正是凭借这一批实打实的重型基础设施投资，AWS 才能胸有成竹地持续推动绿色云服务的演进。 在演讲的最后，James 向全场观众表达了由衷的感谢，希望大家在接下来的 re:Invent 期间收获丰硕的技术干货，并宣布当晚活动正式圆满结束。 - **总装机容量**：新落成的风能与太阳能项目合计装机容量达 970 兆瓦。 - **单个项目明细**：涵盖 150 MW、80 MW、208 MW、100 MW、189 MW 和 180 MW 的多座大型场站。 - **工程类比**：总容量直逼装机约 1300 兆瓦的著名水利工程邦纳维尔水坝（Bonneville Dam）。 - **年发电总量**：计入各类环境因素折损后，预计每年稳定输出 260 万兆瓦时（2.6 million MWh）的可再生电力。 ## Notable quotes > [08:29](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=509s) 我的准则是，如果有一个数据包，经手它的人越多，它被成功送达的可能性就越低，事情就是这么简单。 — James Hamilton > [03:48](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=228s) 如果你对需求信号的响应非常缓慢，你能做什么？你只能过度配置，事情就是这么简单。 — James Hamilton > [22:58](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1378s) It turns out as big as the cost gain is—and oh my God, it is a big cost gain—the biggest gain is actually reliability. — James Hamilton > [31:01](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=1861s) If you offload the hardware, you run rough numbers, roughly a tenth latency, roughly a tenth the power, roughly a tenth the cost. — James Hamilton > [33:34](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=2014s) When you own horizontal and vertical, we get to move at the pace we're used to, we get to make changes at the pace we're used to, we get to respond to customer requirements at the pace we're used to. — James Hamilton > [01:12:03](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4323s) It's got to be built on scalable infrastructure because it's always the case that the problem size gets bigger faster than the systems get bigger it's always the case so scalability is number one. — James Hamilton > [01:05:46](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=3946s) So cloud computing has gone from being a very nice to have and engaging you all with the pictures to something Mission critical every single day. — Tom Soderstrom > [01:16:26](https://www.youtube.com/watch?v=AyOAjFNPAbA&t=4586s) These frameworks things like MXNet, TensorFlow, Caffe, Theano, Torch and CNTK put deep learning into the hands of mere mortals. — Matt Wood ## Mentioned James Hamilton, Tom Soderstrom, Navin Budhiraja, Matt Wood, Brad Porter, AWS, Amazon, NASA JPL, Infosys, Hawaiki Cable, Broadcom, Broadcom Tomahawk, Cavium, Mellanox, Marvell, Barefoot, Innovium, IBM, Stanford University, Wolfram Alpha, TuSimple, Curiosity, Mars 2020, SMAP, OCO, SWOT, NISAR, ARM (Asteroid Redirect Mission), Europa, James Webb Space Telescope, Rovie, Alexa, AWS IoT, AWS Lambda, Amazon EC2, Amazon RDS, AWS Direct Connect, CloudFormation, EC2 P2 Instances, Deep Learning AMI, MXNet, TensorFlow, Caffe, Theano, Torch, CNTK, Infosys Mana, Bonneville Dam ## Video description Learn more about AWS at: http://amzn.to/2g4uGgn Watch AWS VP & Distinguished Engineer James Hamilton kick off re:Invent 2016 discussing AWS innovation at scale. Also hear Tom Soderstrom talk about how NASA JPL is answering cosmic questions in new ways using AWS. [Source](https://www.youtube.com/watch?v=AyOAjFNPAbA)

[8:29] James Hamilton: 我的准则是，如果有一个数据包，经手它的人越多，它被成功送达的可能性就越低，事情就是这么简单。 [3:48] James Hamilton: 如果你对需求信号的响应非常缓慢，你能做什么？你只能过度配置，事情就是这么简单。 [22:58] James Hamilton: It turns out as big as the cost gain is—and oh my God, it is a big cost gain—the biggest gain is actually reliability. [31:01] James Hamilton: If you offload the hardware, you run rough numbers, roughly a tenth latency, roughly a tenth the power, roughly a tenth the cost. [33:34] James Hamilton: When you own horizontal and vertical, we get to move at the pace we're used to, we get to make changes at the pace we're used to, we get to respond to customer requirements at the pace we're used to. [72:03] James Hamilton: It's got to be built on scalable infrastructure because it's always the case that the problem size gets bigger faster than the systems get bigger it's always the case so scalability is number one. [65:46] Tom Soderstrom: So cloud computing has gone from being a very nice to have and engaging you all with the pictures to something Mission critical every single day. [76:26] Matt Wood: These frameworks things like MXNet, TensorFlow, Caffe, Theano, Torch and CNTK put deep learning into the hands of mere mortals.

[54:19] Tom Soderstrom: It's no longer about a space race between countries; it's a race for Humanity into space. [41:02] James Hamilton: If this power supply is 1% better, 1% times hundreds of millions is getting to be a pretty interesting number—like, you could have fun with that. [30:23] James Hamilton: Whenever you've got workload that's very repetitive and happening all the time as almost any network packet processing is you're really better off into taking some of that down into hardware [46:42] Navin Budhiraja: These mainframe programmers are not getting any younger, we're not producing more of them—it's like my daughter, when I asked her what does she think COBOL is, she thought it was some exotic stone because it sounds like cobalt.

[0:01] ladies and Gentlemen please welcome vice [0:03] president and distinguished engineer of [0:05] Amazon web services James [0:09] [Music] [0:12] [Applause] [0:13] [Music] [0:18] Hamilton wow welcome welcome welcome [0:22] it's good to get it started on what's [0:23] going to be a super action-packed week [0:26] this is this is going to be a really big [0:28] week um I remember my first reinvent [0:31] there was 6,000 people here I thought it [0:33] was phenomenal I learned more than you [0:36] can possibly imagine I It Blow blew me [0:38] away in the first reinvent seeing what [0:40] customers are doing with the product [0:42] already phenomenal just absolutely [0:44] phenomenal and so when I see what's here [0:48] today and here there's [0:50] 32,000 people [0:52] here [0:55] wow do you think this Cloud thing is [0:57] going to happen it looks pretty big [1:00] [Laughter] [1:02] amazing thanks very much for taking time [1:05] out of your super busy work lives in [1:07] order to come here to learn about AWS [1:11] thank you for taking time out of your [1:12] home life to be able to come here and [1:13] spend a week with us in AWS our job is [1:16] to make sure we deliver enough content [1:18] for you so that when you go back to work [1:20] you're able to get more done your [1:22] companies are more successful and you're [1:24] going to come back next year because we [1:25] love being able to actually get a little [1:28] closer and talk in a little bit more [1:29] detail about some of the things that [1:31] we're doing and that in fact is what I [1:33] get to do today the plan is just to give [1:36] you a little background is we're going [1:37] to we're going to dive in spend some [1:40] time on infrastructure partly because [1:42] that's where I work and partly because I [1:44] think it's super important to understand [1:46] some of the unique advantages of AWS so [1:49] that's the first thing after that we're [1:51] going to have a few people come up and [1:53] help me and give give present some other [1:55] angles and I'm going to finish off at [1:58] the end of the show talking about big [2:00] stuff that we're doing with [2:01] sustainability and renewable energy [2:03] super exciting hope you enjoy it let's [2:06] jump in I always start with this slide [2:09] I'm in love with this slide we update it [2:12] but it doesn't matter if we if we never [2:14] updated it if we left it the way it was [2:16] three years ago it'd be wonderful this [2:18] is a crazy number think what this means [2:21] just imagine forget about all the [2:23] Innovation forget about all the things [2:25] I'm going to talk about today just [2:26] imagine figuring out how to do this what [2:29] you need to do is you need to get all [2:31] the component manufacturers delivering [2:32] to server man delivering to the server [2:35] manufacturer the server manufacturer has [2:37] to build boards assemble servers package [2:40] them up test them push them into the [2:42] supply chain the logistic channel has to [2:44] deliver them to their data centers [2:46] trucks have to back up to the bright [2:47] data center deliver racks there has to [2:50] be enough data center space built there [2:52] has to be networking there has to be [2:54] power the servers got to be just put [2:56] installed by by technicians and it all [2:59] has to to be brought [3:01] up and then we do it again tomorrow this [3:04] is amazing I mean just I'm just shocked [3:07] of what the team's capable of of doing [3:09] if if if I was to think when I joined [3:11] AWS uh a few years ago that anything [3:14] even approximately that like this was [3:16] possible how would you do it if it [3:18] sounds like the moon mission I mean it [3:20] just seems impossible so this is [3:22] happening every day if you work with me [3:24] and say let's say is it likely is it [3:27] likely that Amazon uses around the same [3:30] number of server resources as a Fortune [3:32] 500 company my take is proba they [3:35] probably use a lot more but let's just [3:37] be conservative and say it's about the [3:39] same that means we're taking on a [3:40] Fortune 500 company every day this is [3:44] this is this is happening this is [3:45] happening in a very big way right now [3:48] I'm excited by this slide this is the [3:50] first time this has been shown outside [3:52] of Amazon it originated from a actually [3:54] it was an internal technical [3:56] presentation done by Brad Porter I fell [3:58] in love with it this this is absolutely [4:01] wonderful what it shows on the x axis is [4:04] of course time you can see events [4:06] happening Prime day 15 the fourth [4:09] quarter uh Prime day 16 see wow Prime [4:12] day 16 pretty big deal um on the Y AIS [4:15] it's virtual servers so we've got [4:18] virtual servers in the y-axis they won't [4:20] let me show you the y-axis it did H it [4:22] did exist internally but I will give you [4:24] a scale point I'll give you a scale [4:26] point if you walk down there and look at [4:27] Prime day 16 that is tens of thousands [4:30] of servers tens of thousands of virtual [4:34] servers brought online just before they [4:37] were [4:37] needed they delivered the goods and they [4:40] were brought offline as soon as they [4:42] weren't needed anymore that is a [4:44] phenomenal piece of news that is what [4:47] that is what's happens if you get good [4:49] at managing the cloud most workloads I [4:52] mean we talked a lot about elasticity [4:54] years and years ago and it turns out the [4:57] reason why this conference sold out a [4:59] long time ago and there's 32,000 people [5:01] here it's not because of elasticity it's [5:04] because it's Topline growth it's because [5:07] it allows companies to be agile to try [5:10] new ideas to free up Innovation to to [5:15] test things out without we waiting [5:17] months and months and months to get [5:18] resources for things that work to deploy [5:21] them quickly and when something works [5:23] live double down go big go fast and [5:27] that's why people are using the cloud [5:28] that alone is probably enough it also [5:30] happens to be a heck of a lot cheaper [5:32] I'm going to go through a bazillion [5:34] reasons tonight why it is a heck of a [5:36] lot cheaper the funny thing this slide [5:39] has nothing to do with any of those [5:40] things this is gravy this is what you [5:43] get if you're in the cloud you've taken [5:46] your workload you've you've put it in [5:48] the cloud it's it's it's costing the [5:50] company less the company's actually [5:52] producing better results it's it's it's [5:55] a better place to work because there's [5:56] more things happening and yet you can [5:58] still go as you get better at managing [6:00] the cloud and start to exploit the [6:02] unique capabilities you start to get [6:04] this type of gain when you start talking [6:06] about tens of thousands of servers let's [6:08] get a scale point on that tens of [6:10] thousands of servers is a midsize data [6:13] center that's pretty big that's a [6:15] midsize data center that came to life [6:17] lived for a few weeks and was brought [6:19] back down a midsize data center 150 [6:23] million 250 million ballpark in that [6:26] area think about what this means [6:30] conventional companies running on [6:31] premise what they have to do is they've [6:34] got a provision to that Peak and because [6:36] it takes a long time to get service [6:38] through the acquisition cycle and [6:39] actually get them in actually get them [6:41] tested you would never dare run that red [6:43] line I've shown there you'd want to [6:45] bring resources on F much earlier and [6:48] you'd want to have more resources [6:50] available because you're very slow to [6:52] respond to demand signals and if you're [6:54] slow to respond to demand signals what [6:56] do you do you have to over-provision [6:57] that's as simple as that and so you [6:59] provision so many different ways that [7:02] you've got a $200 million asset you've [7:05] committed to it for 3 years and it does [7:07] absolutely nothing this gain is not why [7:10] this room's full this is the gain that's [7:12] this is the gain that is that is [7:14] available if you start to exploit the [7:17] unique capabilities of the cloud that's [7:19] why I'm excited by this [7:21] slide here's the Amazon Regional Network [7:24] 14 14 um regions all across the glob [7:29] Globe four more announced for for next [7:32] year great news I love what we've got [7:35] we're going to have 18 we're going to [7:37] have 18 regions at that point and these [7:40] regions are real regions these are AWS [7:42] regions regions these are not hey I I [7:45] stuck two uh um stuck two racks in [7:50] opposite ends of the same Data Center [7:52] and they're relatively independent and [7:54] there's a wall between them so fires [7:55] unlikely to spread and there probably [7:57] won't be a flood so those are [7:59] availability zones no our availability [8:03] zones are real they're they're separate [8:05] buildings and they actually do survive [8:07] through all of those different faults [8:09] and I'll show you in detail because I I [8:11] want you to see exactly what the [8:13] difference is and the best way to know [8:15] the difference is to see the difference [8:17] simple as [8:20] that 68 points of present presence [8:23] spread out throughout the entire globe [8:29] this is the one that's great you haven't [8:31] seen this before I got a question two [8:33] years ago saying does Amazon have a [8:35] private Network do do you deploy uh [8:38] private networks is there any you you [8:40] spend on that a lot of companies talk a [8:42] lot about it have you considered [8:43] that yeah we thought about it that is [8:47] all 100% Amazon controlled resources [8:51] that's AWS if you're flowing between one [8:54] region and another it's flowing on that [8:56] Network it's that network is managed by [8:58] one company [8:59] it's not passed from one provider to [9:01] another Transit provider to another [9:03] interconnection site to another [9:04] interconnection site these [9:05] interconnection sites are wonderful [9:07] wonderful very committed individuals but [9:09] my rule is if you've got a packet the [9:12] more people that touch it the less [9:14] likely it is to get delivered it's as [9:15] simple as that it's it's just one [9:18] administrative domain is way better than [9:21] many administrative domains and [9:22] sometimes when the internet weird things [9:24] happen like one company is not getting [9:26] along exactly with another company and [9:28] they're trying to work on a contract and [9:30] maybe the resources get a little [9:31] squeezed during that time to kind of [9:34] Rush the contract along we're not going [9:36] to do that so if it's running on our [9:38] Network it's under our operational [9:39] control we give you better quality of [9:41] service and we always have assets we [9:44] always have assets to be able to survive [9:46] a fa there's no way we a single link [9:49] will ever will ever have any impact on [9:50] anyone in this room because we have the [9:53] capacity to survive a link failure and [9:54] we engineer it that way simple it's it's [9:56] it's we be crazy not to [10:02] this by the way is not just a little [10:05] tiny 10 gig Network this is 100 Gig [10:07] Network every one of those links that I [10:09] show you 100 Gig every one of them 100 [10:12] Gig absolutely everywhere and of course [10:14] 100 Gig is not enough for for many [10:16] places and so it's many many parallel [10:19] 100 Gig links all over the place so this [10:21] is a relatively this is a pretty [10:23] important asset when we started this [10:26] I've got to admit I was a little [10:27] concerned because it's really really [10:29] really expensive and so I'm concerned [10:32] the networking team is 100% committed [10:34] that this is the right thing to do um [10:36] from a quality of service perspective [10:38] absolutely the right thing to do and you [10:39] know something if you the team is really [10:42] good at finding great value and so these [10:45] private resources that we have available [10:48] they're short-term leases long-term [10:49] leases they're dark fiber that are lit [10:52] under irus there one several cases now [10:56] we're laying our own cable and so [10:59] everything's available we'll do anything [11:01] that whatever is most cost effective to [11:03] get the resources that we need to be [11:05] able to serve is what we do and because [11:07] we're not religious about there's one [11:08] true way and we have to it it's we get [11:11] good value let me show you an example [11:14] this is our latest project this is the [11:17] hauk transpacific cable the reason I [11:20] wanted to show it to you there was a the [11:22] groundbreaking in New Zealand was last [11:24] week it's kind of a big deal this this [11:27] this is a transpacific cable runs 14,000 [11:31] km at its steepest point it's 6,000 M [11:35] below the sea 6,000 M below the sea [11:39] that's about 3 [11:41] miles 3 miles below the [11:44] ocean interesting challenge with I can't [11:47] resist telling you this because I was so [11:49] captivated by it myself you start to get [11:51] every time you get involved with [11:53] technology you learn it's always harder [11:55] than it looks you know how hard could it [11:56] be to sing a fiber between Australia and [11:59] in the US doesn't seem that bad [12:01] shouldn't be a problem turns out signal [12:04] to noise ratios being what they are you [12:06] have to have repeaters every 60 to 80 [12:08] kilm that's unfortunate okay got it I [12:12] understand they have to be able to work [12:14] for 20 years without service oh that's [12:17] that's okay I understand I understand oh [12:20] and and they have to work three miles [12:22] under the [12:24] sea you got okay I'm with you I'm with [12:28] you and and these repeaters are [12:30] electrically powered o that's not good [12:33] see now you've got now you've got [12:36] electrical power 3 miles underneath the [12:38] sea supposed to last 20 years and you've [12:40] got to get power to it I mean those [12:42] really really really long extension [12:44] cords you see used in some in some Lawns [12:47] it doesn't feel like the right thing so [12:49] you got to find a way to get power to [12:51] these things and what they do if you [12:52] look closely you'll see the copper [12:54] sheeting that wraps the fiber the fiber [12:57] is actually wrapped in Copper so if you [12:58] look at itose L you'll see there's a [13:00] there a bundle of fibers some insulation [13:03] and then a couple layers of copper now [13:05] the problem is these are a lot of [13:08] repeaters and so you have to have a lot [13:10] of copper because it's carrying a lot of [13:13] current it takes a lot of power to run [13:14] them all and you can't do that it's just [13:16] it's not coste effective to do that so [13:18] what's the trick same trick that gets [13:20] played on on Long Hall transmission and [13:22] terrestrial terrestrial power lines and [13:25] that is if you need a lot of power you [13:27] can either deliver a lot of amperage [13:29] which means you need a lot of conductor [13:30] or a lot of voltage and of course they [13:33] go to a lot of voltage so the reason why [13:35] those those pipes those conductors are [13:37] relatively small is because they're [13:40] running very high voltage this these [13:42] these devices are running on direct [13:44] current and it's actually 10,000 volts [13:46] positive DC and 10,000 volts negative DC [13:51] one more little tidbit just because I [13:52] think it's a really interesting one if [13:54] you look closely you'll say but there's [13:55] two [13:56] conductors what if one fails what if if [13:59] someone anchors at the wrong spot what [14:01] if a fisherman trolling gets a little uh [14:03] aggressive and hopes to catch something [14:05] bigger what do you do then if one of [14:07] these one of these conductors is is open [14:10] to the Sea if it shunts to the Sea it's [14:13] it's it's you it's you're down [14:15] absolutely down there's not a third [14:17] conductor so how do you have redundancy [14:19] in this cable it's a super interest well [14:21] I think it is it's an interesting trick [14:23] so you've got 10,000 volts positive [14:25] 10,000 volts negative this 10,000 volt [14:28] negative shun to see so it goes up to [14:31] zero it floats up to zero so what you do [14:34] if you're managing the cable is you lift [14:36] this one to 20,000 volts now you still [14:38] have 20,000 volts heading to every [14:41] repeater it's the same it's the same [14:42] voltage level as before same difference [14:44] of potential you're using the seawater [14:47] as the as this as the third wire if you [14:50] will really cool trick service the go [14:53] service the cable fault and then shift [14:55] it back down to say stay the same way [14:57] again so when the few times where two [14:59] actually when you need to you've got to [15:01] you've got redundancy kind of [15:05] surprising okay what we've got here is [15:08] back to that beautiful Network I had to [15:09] show it one more time now what we're [15:11] going to do is we're going to choose one [15:13] of those regions I'm going to choose a [15:15] fully developed region because I want to [15:16] show you how big it can get and all and [15:19] the full richness that's possible inside [15:21] a region so dive into a region let's see [15:24] what we've got first thing we're going [15:25] to look at it's an actual AWS region [15:28] this isn't fictional this isn't what I [15:29] hope it'll be someday this isn't you [15:32] know uh artist rendition this is what's [15:37] there is actually what's there so every [15:39] one of our regions there's 14 in them [15:41] worldwide there's going to be 18 [15:42] worldwide every one of our regions has [15:44] at least two azs when I say an AZ I mean [15:47] a building we'll come back to that a [15:49] separate [15:50] building most of our most of our regions [15:53] have 3 A's all of the new ones we're [15:55] building we going 3 a right now it's [15:58] just it feels like the right place for [15:59] us to be this particular one is five azs [16:03] so relatively relatively big from a [16:05] scale perspective every region has two [16:09] Transit centers the the job of the [16:12] transit center is to provide [16:14] connectivity to the region to the rest [16:16] of the world our our private Network our [16:19] the Amazon Global Network connects up [16:21] into Transit centers customers that are [16:23] direct connecting to us are are hooking [16:25] up through pops or possibly up and [16:27] through Transit centers all the the C [16:29] everyone we're peering with through the [16:31] transit centers all of our Transit [16:33] providers through the transit centers [16:35] and so two Transit centers it's another [16:37] constant we'll always [16:40] have now we've got we need to wire this [16:43] up we've got five azs the first thing [16:46] we're going to do is we're going to wire [16:48] inside each a we're we're going to run [16:51] fiber to hook up each a then we're going [16:54] to run fiber to hook the azs up with [16:57] each other [16:59] then we're going to run fiber to hook [17:00] the transit centers up into all the azs [17:04] you see what's going on there is a lot [17:06] of redundant fiber here and the word [17:08] redundant is is a wonderful thing in the [17:10] networking world because it means when [17:12] things go wrong when someone decides to [17:15] dig a hole in the wrong spot things keep [17:17] running you don't you don't feel that [17:19] redundancy when you're running but when [17:21] but when you when you don't read about [17:23] it it's because of that redundancy we've [17:26] got in this particular design in this [17:28] particular [17:29] we have 126 unique spans which is a [17:32] pretty substantial number and get this [17:35] of course all of those many of those [17:37] strands are more than a fiber in fact [17:40] there's [17:42] 242,4 72 strands throughout that 126 [17:47] links there's a lot of fiber strands [17:49] here's another interesting little tidbit [17:51] at least it caught my interest is we we [17:53] use 2-in conduits the industry pretty [17:55] much runs on 2-in conduits well we're [17:58] running a lot of fiber between these [18:00] buildings do you want to dig another [18:02] hole and run another 2in conduit like [18:04] not especially and fibers are small so [18:07] how You' think it wouldn't be that hard [18:09] but you need to have strength otherwise [18:11] the fibers will break when they're [18:12] pulled so you need to have a core that's [18:15] strong enough structurally that it can [18:17] that can last second thing is the the [18:19] the whole bundle has to be armored [18:21] sufficiently well that when the [18:22] construction workers pull it it survives [18:24] and it's able to stand weather and [18:26] environments that [18:27] Underground and so company that we're [18:29] working with is really doing phenomenal [18:32] work is it's it's they start in a pretty [18:35] conventional place it's very con very [18:38] common to have ribbon cables you know [18:40] fibers that are actually in ribbons and [18:42] what they're doing is they're taking the [18:44] ribbons folding them into a V and [18:46] stacking them and so it forms kind of a [18:49] v that fills up a a quadrant and then [18:52] they're doing it again and doing it [18:53] again and doing it again and by the time [18:55] you're' done that somehow they get 3,4 [18:58] 56 fibers we're the first company to [19:01] deploy this technology we absolutely [19:03] love it it saved us a ton of money [19:06] because of we're running so much fiber [19:08] and you might ask if you're the [19:09] networking World instead of running a [19:11] lot of fiber you've always got a choice [19:13] there's other things you can do one of [19:14] them is is I showed you back there on [19:16] the on the hauk cable that is running [19:20] every fiber pair is running 100 waves of [19:25] 100 Gig you can run parallel waves on [19:27] the same fiber and and and and so [19:29] instead of running 100 fibers you can [19:31] run one fiber and 100 waves on it and so [19:33] what you saw back there was a 30 terbit [19:36] fiber with with with six fibers 30 [19:39] terbit cable with six fibers so why [19:41] don't we use the same thing here and the [19:43] reason is current technology has dwdm or [19:46] cwdm that's coarse wave division [19:49] multiplexing or dense wave multi wave [19:51] division multiplexing um cost more this [19:54] bottom line is it's it's more cost [19:56] effective for short distances to run [19:59] independent fibers for long distances it [20:01] always wins to run uh to run multiple [20:03] waves on the same fiber silicon [20:06] photonics will probably change that our [20:08] plant will probably eventually end up [20:10] with with running multi-wave I'm very [20:13] confident that'll happen but it's not [20:14] happening anytime soon it'll it'll be a [20:16] little bit it'll be a little bit of time [20:18] yet so almost all of those are single [20:21] fiber not everyone but almost all of [20:24] them okay let's jump into a fol kazz [20:28] again remember this is a specific region [20:31] these are these are actual numbers from [20:33] that region every a is one plus data [20:37] center no AZ pardon me no data center [20:41] has two azs no games like I told you [20:44] third one is all we the network links [20:46] we' got that covered final thing is this [20:49] one I found this one I should know I [20:51] should know these numbers but in fact it [20:53] blew me away we have several not one [20:56] several AES these are azs are a part of [21:00] a region 14 regions worldwide an single [21:03] AZ we have several at [21:06] 300,000 [21:08] servers wow big [21:11] numbers here's a data center this is one [21:14] where we kind of go backwards where a [21:16] lot of the numbers I've showed you I [21:17] find to be big numbers and surprising [21:20] numbers and considerably bigger than [21:21] they were last time I showed them these [21:23] numbers haven't really gone up that much [21:25] last time I showed you I think it was 25 [21:27] to 30 megawatts [21:29] right now I'm saying 25 to 32 almost all [21:32] of our new builds are 32 megawatts why [21:34] aren't we building bigger facilities we [21:37] could easily build 250 megawatt [21:39] facilities I've been in 60 megawatt [21:41] facilities nothing challenging about it [21:43] you run whatever you want here's what's [21:45] going on it's it's it's you know the [21:47] reason is the same reason we do [21:49] everything it's data we just use the [21:52] data and so when you're when you're [21:54] scaling up a data center when you're [21:56] very small and you add scale you get [21:59] really big gains in cost advantage and [22:02] as and and as you get the bigger and [22:04] bigger and bigger it's a log rithm it [22:06] just gets flatter and flatter and [22:07] flatter and it starts to get to the [22:09] point where the gains of going to a lot [22:11] bigger are relatively small the negative [22:14] gain of a big data center is linear and [22:17] that is if you have a data center that's [22:19] 32 Meg and it's 880,000 servers it's [22:22] it's bad if it goes down but not not but [22:25] we're actually have a sufficient scale [22:27] that you don't notice it we can work [22:29] through that but if you double that [22:32] 160,000 triple that quadruple that start [22:36] to get upwards of of half a million if [22:38] that goes down the amount of network [22:40] traffic that the to heal all of the [22:42] problems it's it's it's a it's not a [22:44] good place to be so our take right now [22:47] is this is about the right size facility [22:49] it cost us a little tiny bit more to [22:51] head down this path but we think it's [22:53] the right thing for [22:55] customers that's a region let me show [22:58] you a little bit on networking always [23:00] have to have my rant on Old School [23:02] networking because it held back the [23:04] industry for so long vertically [23:06] integrated networking equipment where [23:08] the where the as6 the hardware the [23:11] protocol stack is supplied by um single [23:15] companies is it's it's really it's the [23:17] way the Mainframe used to dominate [23:19] servers and it's it's actually an [23:21] interesting observation if you look at [23:23] where the networking world is it's it's [23:26] it's sort of where the server world was [23:29] 20 or 30 years ago it started off with [23:32] you you buy a Mainframe and that's it [23:35] and it comes from all one company the [23:36] networking world is the same place and [23:38] we know what happened in the server [23:39] world as soon as you as soon as you chop [23:41] up these vertical Stacks you've got [23:43] companies focused on every layer and [23:45] they're innovating together and they're [23:46] they're all competing you get great [23:48] things happening and so the same thing [23:51] starting to happen in the network world [23:52] it is a wonderful place to be and it's [23:55] what's what's happening is it it's it's [23:57] going to cause it is causing it's [24:00] already causing the ratio of networking [24:03] to server is going up in other words Ser [24:07] for a given server size the amount of [24:09] networking required to support it is [24:10] going up and partly it should have gone [24:13] up before but networking was [24:14] artificially expensive and so server [24:16] resources were getting stranded um now [24:19] when they're moving to commodity um this [24:21] no longer happening we run our own [24:24] custombuilt networks um routers so these [24:27] routers that particular one happens to [24:29] be uh topof rack form factor these [24:31] routers are built to our specs and this [24:34] is the wonderful thing we have our own [24:36] protocol development team when I rant [24:38] about how poorly served we were by [24:41] vertically integrated routers I mostly [24:42] talk about cost and it was cost that [24:45] caused us to to to to go to actually [24:47] head down our own path but it turns out [24:50] as big as the cost gain is and oh my God [24:52] it is a big cost gain um the biggest [24:55] gain is is actually reliability what [24:57] happens is networking gear is very [24:59] expensive every company has people like [25:02] me that that have big ideas and they say [25:04] oh I've got a requirement I'd like you [25:06] to add some incredibly complicated piece [25:07] of code to your system and so they say [25:10] sure and after a while the networking [25:12] gear is absolutely completely [25:14] unmaintainable and the next release [25:16] comes out they don't test all that stuff [25:18] that people like me ask for because [25:19] nobody uses it anyway and it doesn't [25:21] work it just doesn't work our networking [25:24] gear has one requirement from from from [25:27] us that's the only source of [25:28] requirements and we show judgment and [25:31] keep it simple we actually it's our [25:34] phones that ring and it's our pages that [25:36] ring if it doesn't work so it's well [25:38] worth keeping it simple as fun as it [25:41] would be to add a lot of really really [25:42] tricky features we just don't do it [25:44] because we want reliability so it's a [25:47] much more reliable system and I honestly [25:49] wouldn't have guessed that when we [25:50] headed down this path I was making [25:51] excuses saying initially it won't be as [25:54] reliable it was way more reliable from [25:56] day one way more from day one second [25:58] thing is okay you've got a problem what [26:00] do you do well if a pager goes off we [26:03] can deal with it right now it's our data [26:05] centers we can go and figure it out and [26:06] we can fix it we've got the code we've [26:08] got skilled individuals that work in [26:10] that space we can just fix it if you're [26:12] calling another [26:14] company it's going to be a long time [26:17] it's they have to duplicate something [26:19] that happened at the scale I showed you [26:21] in their Test Facilities how many Test [26:24] Facilities look like that there's not [26:27] one on the planet so it's just it's 6 [26:29] months the most committed best quality [26:33] most most serious company it takes 6 [26:37] months it's just it's a terrible place [26:39] to be so we love where we are right now [26:42] we jumped on 25 gig early 25 gig it [26:45] looks like a crazy decision if you look [26:46] at it and I was heavily involved with [26:48] this decision so I want I I'll defend it [26:51] um you know the industry standards are [26:53] 10 gig and 40 why the heck would you [26:55] build 25 it's just like you asking for [26:58] trouble and old by the way it's 25 was [27:01] really new at the time and there's a bit [27:03] of an optic shortage shortage happening [27:05] at the time so it's it's risky as well [27:08] but here's what's going on if you can't [27:10] take that if if you're not willing to [27:11] find a way to solve the Optics problem [27:13] you have to run 40 gig so that's where [27:15] most of the world went we're confident [27:18] that 25 is the right answer and I'll [27:20] show you why real fast it's super [27:22] simple 10 gig is single wave that's 10 [27:27] gig 40 gig is four waves it's still the [27:31] same thing it's it's basically it's all [27:33] it's not quite this bad but it's 40 gig [27:35] is almost four times the Optics cost of [27:38] 10 gig so it's just not a great place to [27:41] be 25 gig is one wave it's almost the [27:45] same as 10 gig again not quite true it's [27:47] a little bit more money but it's almost [27:50] the same price as as as as as 10 gig and [27:52] so what that means is on this model we [27:55] can run 50 gig which is more ban with [27:59] and we get to and we get to do it at [28:01] much less cost because we're only [28:03] running we're only running two waves and [28:04] from an Optics perspective it's [28:06] absolutely the right answer I am totally [28:08] we buy enough that it doesn't matter I [28:10] mean the vendors are extremely happy to [28:12] serve us and so it's not a problem but I [28:14] believe this is where the industry is [28:16] going to end up and the reason is it's [28:17] whenever you've got the right answer [28:19] when when when it looks that good it'll [28:22] happen and it has happened we are [28:24] deploying these by The Unbelievable uh [28:27] numbers um [28:28] which is good I mean I'm glad we [28:31] are here is the um here's the Asic that [28:35] runs in our routers today I am super [28:38] excited about this because remember I [28:40] referred earlier to saying hey servers [28:43] went down this path and and Di [28:44] verticalized you can not you can now buy [28:47] A6 individually well are very large [28:49] numbers um you can buy A6 without buying [28:52] the rest of the gear um we're we work [28:54] with broadcom this particular one is a [28:56] broadcom Tomahawk it is [28:58] at the time when it was released the [29:00] most complex by transistor count a um [29:03] application specific integrated circuit [29:05] there is on the planet these are [29:06] monsters these are absolute monsters but [29:09] the beautiful thing about this this is a [29:11] 3.2 terbit part what does that mean it's [29:15] 128 ports of 25 gig all ports can be [29:19] running all flat out with no blocking [29:22] it'll flow 3.2 terabits at the same time [29:24] through this no wonder it's a [29:26] complicated part why do I like that well [29:29] non-blocking is a wonderful place to be [29:31] but the real reason I like it is there's [29:33] a healthy ecosystem so cavium melanox [29:37] broadcom Marvel Barefoot and novium are [29:40] all building Parts there's 6 terbit and [29:43] 13 terbit PES parts coming and they will [29:46] be around the same price in just the [29:48] same way the server was and so now what [29:50] happens is if you separate off opt [29:52] basically networking gears has two costs [29:54] it has this cost and it has Optics and [29:57] that's basically all there is all the [29:58] rest is is is lost in the noise what [30:01] this means is this is on a Mo's law [30:03] Pace that is a fantastic thing well [30:08] Optics aren't but with silicon photonics [30:10] they're soon to be heading down that [30:12] same path and many of the Optics we're [30:15] we're running today are are in fact [30:17] multi-chip versions of silic silicon [30:19] photonics so good things are [30:23] happening software defined networking [30:25] big topic today super important for the [30:28] last couple years we've had it since the [30:30] beginning of ec2 because you need to [30:32] have it since the beginning of ec2 in [30:34] order to offer a secure service um as as [30:37] we do um starting around somewhere in [30:40] 2011 I believe it was we made what was [30:44] realistically a fairly obvious [30:46] observation but an important one and [30:47] that is whenever you've got workload [30:50] that's very repetitive and happening all [30:53] the time as almost any network packet [30:55] processing is you're really better off [30:57] into taking some of that down into [30:59] hardware and so what we did is we [31:01] offloaded the servers and dropped it [31:04] down and dropped that Network [31:06] virtualization code down onto the Nick [31:09] lots and lots and lots and lots of gains [31:11] follow from that first gain is more [31:14] resources are available more cores are [31:16] available on the servers good news [31:18] second gain is things that are hard to [31:21] read and hard to understand but they're [31:23] happening are little disturbances in the [31:25] server like flushing tlbs and things [31:28] like that are now moved off it's a [31:30] little bit more secure or considerably [31:31] more secure a if a if a hypervisor is [31:35] compromised you still don't have access [31:37] to the network because it's a it's a [31:39] it's a separate operating system SE [31:41] separate real-time operating system [31:43] running on the Nick running all of our [31:44] software that's all of our software [31:46] running running on that Nick and so that [31:49] offload does wonderful things another [31:52] observation [31:54] is kind of a it's a it's an obvious [31:57] observation but a super important one um [32:00] and this applies to every level of [32:01] computing and so it's kind of a nice Ru [32:03] set of rules to keep in mind and that is [32:06] if you if you offload the hardware you [32:10] run rough numbers you rough you run run [32:14] roughly a tenth latency roughly a tenth [32:17] the power roughly a tenth the cost and [32:21] so it's a big deal if you can do it [32:24] second observation is people say hey the [32:27] reason why we had to build custom [32:29] networking gear is you could never have [32:31] the bandwidth we have in our data [32:32] centers if we didn't build custom Gear [32:34] Well it's not true I mean I can give you [32:36] any bandwidth you want it's just more [32:38] parallel links and I can do it with [32:40] anyone's equipment it's it's it's not [32:42] even hard to do it's hard to pay for if [32:44] you're using some of these commercial [32:46] gear but it's absolutely not hard to do [32:48] do you know what is hard to [32:50] do latency that is physics one one is [32:54] money physics is harder physics you you [32:57] got to challenge with that it's just the [32:59] speed of light and fiber is the speed of [33:01] light and fiber it's just you know there [33:03] may come fibers that are a little bit [33:04] faster but that's basically the fastest [33:07] you're going to go so latency is key [33:09] when you move to Hardware the latency is [33:12] is fundamentally changed in fact the way [33:14] I look at it is I tell software people [33:16] your numbers the things you measure are [33:18] called milliseconds and if you put it in [33:20] Hardware the things they measure are Nan [33:23] seconds and microseconds and so you're [33:26] changing by big margins and so this is [33:31] the right place for us to go here's some [33:34] good news here's some great news do you [33:37] believe we're in the semiconductor [33:39] business isn't that great we're in the [33:42] semiconductor business that Amazon web [33:45] services is not only are we building [33:47] Hardware which I thought was pretty cool [33:49] we built this this is billions of [33:53] transistors billions and billions of [33:55] transistors every server we deploy has [33:57] at least one of these in it some will [33:59] have a lot more this is a very big deal [34:02] imagine what we can do if I'm right on [34:05] those Trends I told you on Hardware [34:07] implementation latency the cost of power [34:10] Etc if I'm right on that point and I'm [34:13] fairly confident on that one what that [34:15] means is we get to implement it in [34:17] silicone so now we've got in the same [34:20] company reporting up into our [34:21] infrastructure team we've got digital [34:24] designers working on this we've got [34:27] Hardware designers working on the Nyx [34:30] themselves and we've got software [34:32] developers and when you own horizontal [34:35] and vertical we get to move at the pace [34:37] we're used to we get to make changes at [34:39] the pace we're used to we get to respond [34:41] to customer requirements at the space at [34:43] the pace we're used to we think this is [34:45] a really big deal and if I'm right it [34:47] says that there's going to be an [34:48] acceleration of the amount of networking [34:50] resources that are available to servers [34:53] then this is a wonderful place for us to [34:54] be because we're going to be able to [34:56] step up to at a relatively low cost [34:59] because of some of the decisions I've [35:00] laid down or outlined to you so [35:03] far good news there let's look at this [35:05] one what one of the things that I do if [35:08] you if you try to manage faults by [35:10] looking at at um by looking at basically [35:14] you have a fault you go postmortem and [35:15] you say oh I shouldn't have done that [35:17] and you learn from it and then you don't [35:18] do it again it's a it's okay I mean you [35:20] should do that we we're religious about [35:21] it but it's I look at other people's [35:24] faults and try to learn from them and [35:26] this one caught my view because oh boy I [35:28] know this fault I know this Fault by [35:31] just by heart and this is a it's fun [35:33] because I know it because we run it very [35:35] large scale but it's actually a very [35:37] rare event I almost guarantee you this [35:39] company's never seen it before and [35:40] they'll probably never see it again um [35:43] but it does happen and very rare events [35:46] at very large scale happen unfortunately [35:49] is more frequently than you think let's [35:51] look at the impact to this one Chief [35:52] Financial Officer this Airline reported [35:54] they lost $100 million of Revenue went [35:56] away because of of this fault they the [35:59] the cancellations are listed there 2% of [36:01] their monthly Revenue were gone as a [36:03] consequence of this thing and the report [36:06] was switch gear failed and locked out [36:08] backup generators let's I know that one [36:12] what h i remember I happened to be in [36:14] the data center for this one it's I [36:16] don't know why but just fluke I happened [36:18] to be in one of our Virginia data [36:20] centers this exactly this event happened [36:24] um the way I should tell you the way [36:26] switch gear works and I'll tell you what [36:27] happened the way switch gear works is [36:29] the utility feeds in through the switch [36:30] gear goes down into the uninterruptible [36:32] power supply and as long as the utility [36:35] is there of course that circuit runs if [36:37] the utility fails the switch gear Waits [36:41] a few seconds just to give it this [36:44] utility usually comes back very rapidly [36:46] most faults are incredibly short it's [36:47] not worth starting a generator if it [36:49] doesn't happen the generator starts up [36:51] spins up to full RPM um wait we wait the [36:55] switch gear Waits until the voltage [36:56] stabilizes and the quality of power is [36:58] good and then swings the load over to it [37:00] the poor old generators hit 1,800 RPM in [37:03] about 7 or 8 seconds and they take load [37:05] in about 15 or 20 it is not a good job [37:08] do not apply to become a generator in in [37:10] a backup data center they get the load [37:13] hard and fast okay so that's the way [37:16] they work what goes wrong is if if [37:19] there's a fault out there that looks [37:22] like it might have been a short to [37:24] ground inside the data center the switch [37:26] gear smart and doesn't bring the [37:28] generator on into the load because it [37:31] could destroy the generator it could [37:32] damage the generator and they view it as [37:35] they view it as a safety issue which is [37:37] rubbish so what happened I'm in the [37:40] facility 6 hours later the switch gear [37:42] manufacturer came to the facility to [37:44] explain the problem to us [37:47] and the our our our data center manager [37:51] absolutely aptic just completely tic [37:54] that he's got a generator running and [37:56] and we didn't hold the load [37:58] unacceptable and it's interesting that [38:01] switch gear [38:02] manufacturers absolutely Unapologetic [38:04] it's that's the way it has to [38:06] be [38:08] fine there's there's other switch gear [38:10] manufacturers so we'll buy from someone [38:12] else they're all the same what are the [38:15] odds they're all the same and so what [38:18] we've done is the picture I'm showing [38:21] there oh actually the picture I'm [38:23] showing [38:24] here is that's that's normal commercial [38:28] switch gear and we still use that but we [38:31] changed the firmware so the firmware [38:33] that controls the switch gear does not [38:35] do what I told you and as a consequence [38:37] of not doing what I told you what [38:39] happens is if there's a fault and it [38:41] might be a short in the generator we [38:42] bring sorry in the data center we bring [38:45] the generator online we're going to do [38:46] that and the reason we do that is [38:48] because that's what you want us to do [38:50] what are the impacts maybe it's unsafe [38:51] maybe what what are the risks let's look [38:53] at it well the vast majority of the time [38:56] it's outside the data center anywhere [38:58] that's just the vast majority of the [38:59] time the one I we had experience with [39:01] the one I was at um [39:04] a it's actually it's kind of funny this [39:07] the same pole got hit twice it's it's a [39:09] longer story but anyway there there must [39:11] be a bar nearby there and and someone [39:14] drove into a utility pole an aluminum [39:16] utility pole which fell across two um [39:20] the two phases of the power lines and [39:23] the spike that hit the data center was [39:24] was was extraordinary and switch here [39:27] said very unsafe don't go um so let's [39:29] look at what can happen well in that [39:31] case it switches to the generators [39:33] nothing goes no problem at all that's [39:34] perfect let's say there was a short [39:36] somewhere in the facility then the [39:38] branch circuit kicks out everything else [39:40] runs fine back the secondary power in [39:42] those servers take over and again you're [39:44] fine okay let's say it's a fault very [39:47] high up in the system and the generator [39:48] is actually going to come into a direct [39:50] short might it might destroy the [39:52] generator might not I don't know we've [39:54] never seen it like it's honestly to my [39:56] knowledge I've never read about it's [39:57] never happened but it could happen and [39:59] so maybe it will destroy the generator [40:01] and from my perspective that's 3/4 of a [40:04] million dollars we're very frugal we do [40:06] not want 3/4 of million dollars damage [40:08] but on the other hand we certainly don't [40:10] want to drop the load and so we'll take [40:12] that risk if that happens we've got a [40:13] backup generator that to backup that [40:15] once all of our facilities are those [40:17] magic [40:18] words redundant and concurrently [40:21] maintainable which is to say you can [40:23] have a system offline and and at the [40:25] same time have a fault and and [40:27] everything still keeps running so it'll [40:29] just keep running through that no big [40:30] deal so that's what we've done on that [40:32] we're proud of it we think it's it's one [40:34] of those details that nobody would buy [40:35] AWS because we do things like that but [40:38] we had this fault twice the Super Bowl [40:41] the Super Bowl in in 2013 was down for [40:44] 34 minutes exactly the same fault it [40:47] does happen and it doesn't happen here [40:49] it hasn't happened for years because of [40:51] this let me show you a custom storage [40:54] server last year not two years ago [40:57] showed you a server that I was involved [40:59] with the the the design of it I was kind [41:02] of proud was you know [41:04] 880 discs in a single in a single rack [41:08] unit phenomenal density well of course [41:11] the team's done far better since then [41:13] and so what they've done is they've [41:15] delivered this monster that somehow in a [41:18] 42u rack not cheating it's not a tall [41:21] rack it's 42 Ur rack [41:23] 1,110 discs this by the way is an older [41:26] design I never can show you the absolute [41:29] latest but this particular design built [41:33] back when it was originally designed was [41:34] as 8.8 pedabytes just in one rack if it [41:38] was built today with with conent with [41:40] current discs that would be 11 pedabytes [41:42] 11 pedabytes in Iraq Iraq's not that big [41:47] it's it's really it's phenomenal what's [41:49] really amazing is it's [41:52] 2778 lbs of disc you know power to [41:56] weight ratio is starting to matter it's [41:59] just really [42:00] amazing let me go the other way that I [42:02] think is a piece of impressive piece of [42:04] equipment let me show you one that may [42:06] bore you slightly I love it this is a [42:08] compute server you look at it and the [42:10] first thing you look at is dude what [42:13] happened I mean you could have put a [42:14] second server in there I mean it's empty [42:17] let's you guys I thought you guys [42:20] tried turns out this is optimized for [42:23] thermal and power efficiency and every [42:26] other design we looked at at that time [42:28] actually consumes more power it's less [42:30] efficient and it's simply it's simply [42:33] this is the winning design but what the [42:35] OE what the oems are selling to [42:37] customers are probably three or four or [42:40] 5x more dense than this and they're less [42:43] efficient servers and they make up for [42:45] it by charging a lot more so I like this [42:48] design I like this design a lot things [42:50] you can see the space things you can't [42:53] see are well you can barely see you can [42:55] actually make out the name this is very [42:57] resolution you can make out the name of [42:59] the power supply company on there and so [43:01] you might be able to figure out that's a [43:02] greater than 90% power supply the [43:04] voltage Regulators you can't see those [43:06] at all also greater than 90% think about [43:08] what this means our our um our pu which [43:14] is which is a power utilization [43:16] Effectiveness this is a measure on the [43:18] numerator is the voltage that's [43:20] delivered to the servers and the [43:22] denominator is the voltage delivered to [43:24] the facility and so the difference [43:25] between those two is all the overhead [43:27] head how much did you lose in cooling [43:28] how much did you lose in power [43:30] distribution did you have lights on that [43:32] kind of thing and so a good facility and [43:34] we do run good facilities runs on the [43:36] order of about 1.12 to 1.15 in that [43:40] order and so you figure [43:42] 12% 15 12 to 15 14% are is roughly what [43:47] the overhead is which is to say 80 [43:50] something per high 80s per of the power [43:53] gets delivered to the servers and so [43:55] we're buying hundreds of millions of [43:56] dollars with a power if one if this [43:58] power supply is 1% [44:01] better 1% time hundreds of millions is [44:04] getting to be a pretty interesting [44:05] number like you could you have fun with [44:08] that so it's worth doing and that's why [44:11] this server looks the way that it that [44:13] it does and it's very different design [44:15] from what's out there in industry and [44:17] the funny thing about it is that server [44:20] one of the reasons I I showed it is it [44:21] was recent a server that looks [44:23] surprisingly similar was recently [44:25] blogged by um of the major Cloud [44:27] providers and I thought eh it's cool [44:29] that one of our old servers still [44:30] Compares nice to see it so I wanted to [44:32] show it to you okay main frames look at [44:36] that remember that most of us won't I [44:38] actually do believe it or not I started [44:40] my career at IBM never so long time ago [44:44] and I worked on 3083 3081 and the last [44:49] system was an IBM 3090 model [44:53] 600j this is a beast it's the size of of [44:56] a midsize truck it was the last of the [44:59] emitter couple logic era these are the [45:02] least efficient power prod systems in [45:05] the on the planet this is the last [45:07] system before IBM went to seos um so [45:10] these are water cooled they're directly [45:12] water cooling up to thermal conduction [45:14] modules the thermal conduction modules [45:16] are pushing with Springs to touch down [45:18] very solidly on on the CPUs in order to [45:20] conduct heat off of it these These are [45:23] beasts they're technological wonders [45:27] but it was 30 years [45:29] ago it's dead [45:33] Jim if customers are still running on [45:35] these you may not want to admit it but [45:38] there will be some in some companies [45:40] that are still running on these and we [45:42] want to help customers we want to help [45:45] customers get to a better place so let's [45:48] start off let's start off with well [45:51] let's let's start off with a toast to [45:53] the death of the Mainframe [45:58] can I have a [46:01] beer this is carefully rigged I get a I [46:04] get a chance to have a drink inste of [46:08] us cheers toast to the main frame death [46:12] of the main frame death of the main [46:14] frame and don't worry if you're still [46:17] running you don't tell anyone just we'll [46:20] handle it we'll take care of it I'd like [46:23] to invite to come up on stage to tell [46:25] you what you need I'd like to invite [46:26] invite up senior vice president and CTO [46:30] of infosis deine B [46:34] Baja deine come on [46:36] [Music] [46:42] up awesome thank [46:45] [Music] [46:50] you w this is a big room I can [46:52] definitely accommodate most of my [46:54] company in here [46:57] thanks for showing up I think it warmed [46:59] up the place it's kind of really [47:00] freezing backstage so all you guys helps [47:04] so um main frames so I'm here to [47:07] announce um as James talked about the [47:12] emphasis Mainframe migration practice on [47:16] AWS why is this um really exciting uh as [47:21] James said there a lot of companies [47:23] actually there's still a lot of [47:24] companies who are who are still uh [47:26] running this systems have been running [47:28] them for 30 plus years they done a great [47:31] job still do a great job [47:33] but why do we want to move off [47:37] this right because there are these [47:40] challenges significant challenges with [47:42] these machines [47:45] right escalating [47:47] costs so just to go and give you an idea [47:51] the way you measure these main frames is [47:52] this thing called mips right so they [47:54] about conservatively about 30 million [47:57] myips still running out there and what [48:00] that translates to typically is about [48:02] 100 plus billion dollars of cost every [48:05] year I think that's a conservative [48:06] number right and these costs are going [48:08] up more data is coming in these main [48:11] frames which are typically designed to [48:13] do batch kind of operations people now [48:15] want online access to that and so on [48:19] lack of agility right these companies [48:22] now the business wants to move at a much [48:24] more rapid Pace than what these system [48:27] were designed to do right so it kind of [48:30] prevents you from doing that and last [48:32] but not the least skill [48:34] shortage these Mainframe programmers are [48:37] not getting any younger we're not [48:39] producing more of them it's like my [48:42] daughter when I asked her what what does [48:45] she think Cobalt is she thought it was [48:47] some exotic Stone because it sounds like [48:49] Cobalt right so this is a [48:53] problem and this is really what we are [48:56] trying to uh do as part of this uh [48:59] migration practice that uh we launching [49:01] on [49:02] AWS so kind of think about it in three [49:05] uh three ways what what this can do for [49:08] you it's kind of a journey and uh [49:11] different people might do this at [49:12] different pace but broadly you can [49:15] actually really take the whole [49:17] application a cobal application in JCL [49:20] and actually move it onto AWS on a on a [49:24] kind of an emulation kind of software [49:27] you need to still make changes because [49:28] you still leave the Mainframe behind so [49:30] if you're running on a Mainframe [49:31] database like I that obviously doesn't [49:34] run on AWS you have to change that to [49:37] use a relational database like RS you [49:40] might have to change how security [49:42] security works you might have to change [49:43] how backup [49:45] works right but this thing actually [49:47] works and you can actually have a saving [49:49] up to 30 35% by just doing [49:52] this the next next kind of things we can [49:55] work on is essentially take pieces of [49:58] the main frame pieces that you feel [50:00] actually you want to evolve much faster [50:03] than others that's coming in your way to [50:05] actually evolve things at the pace that [50:07] you want to evolve them and put them in [50:10] a brand new fashion write a cloud native [50:13] code onto AWS and then make them talk to [50:16] each other through the API gateways on [50:18] AWS right that kind of addresses [50:21] somewhat the agility Challenge and [50:23] finally you can of go all the way down [50:27] you're going to rewrite the whole system [50:29] right in Cloud native format right yeah [50:31] that takes much longer but that's where [50:33] I really want to get [50:35] to so this sounds great but what is the [50:39] problem right why why haven't people [50:43] been doing [50:44] this because it this is really really [50:47] hard right these are very very large [50:50] systems right have been built over 30 [50:52] years it's not unreasonable to think a [50:55] system that is 10 million lines of code [50:57] 30 million lines of code and guess what [51:00] no [51:00] documentation right so how do you do [51:04] this and this is really where uh the [51:08] practice that we launching really kind [51:10] of shines just kind of very brief [51:13] overview of this don't have a lot of [51:14] time but the intent really is to [51:17] actually use software to amplify the [51:20] people who are the experts right like an [51:22] irn Man suit in some sense and this is [51:25] not kind of the tradition software that [51:28] it's actually has a lot of AI pieces in [51:31] it we have a technology called infosis [51:33] Mana that think of it as looks at the [51:37] piece of code and can actually discover [51:40] a lot of interesting things about it [51:42] obviously the code structures what the [51:44] code does business rules to some extent [51:47] but where where are the various pieces [51:49] of the code that are have tied [51:51] dependencies on each other what does it [51:53] make sense to actually break apart what [51:55] does it make sense to leave behind [51:57] because it's no longer being used so [51:59] long story [52:00] short take this piece of [52:03] software run it on the code that you [52:05] have and what comes out really is is [52:08] like an assessment right an assessment [52:11] that actually a machine can actually [52:12] make meaning out of and you do that to [52:15] actually then make a very very kind of [52:18] objective assessment of how you should [52:20] do the migration right and that's [52:22] fundamentally is what customers are [52:24] looking for they want predictability [52:26] they want to know that how complex this [52:29] is going to be how long it's going to [52:31] take right and what is the right way to [52:33] get there where do you start do you [52:35] rehost do you migrate do you do a [52:37] combination and so on right so that's [52:39] really what the practice is um again a [52:41] lot of details encourage you to go look [52:44] it up so what I had there actually was a [52:47] set of customers who have beg uh thank [52:50] you who have beg this journey for us so [52:53] very exciting I encourage you guys to [52:55] actually uh come take a look join us [52:58] because this is really happening and you [53:00] want to be part of this thank you and [53:02] enjoy the rest of the [53:09] show great [53:11] [Music] [53:14] job I talked a little bit earlier about [53:16] you know main frames remembering my [53:18] first job before my first job one of the [53:21] things I do remember even perhaps even [53:23] more clear is NASA and man space flight [53:27] program it just it's just absolutely [53:30] phenomenal um it it probably many of of [53:33] much of my curiosity and and and and [53:36] willingness to take on tough challenges [53:39] was learned from massau taking on we're [53:42] going to put a out on the moon when they [53:43] have no idea how to do it and so I'm [53:45] just blown away by the company I think [53:47] it's I think it's just absolutely [53:49] absolutely marvelous what they've done [53:52] um we're very lucky today I've got I've [53:54] got the chief technical off officer and [53:56] Innovation officer of of Nassau jet [54:00] propulsion Laboratories here to talk a [54:02] little bit about work that's going on [54:03] there Tom sodam come on up [54:06] please thanks [54:08] [Applause] [54:19] Tom so what a pleasure it is to be here [54:22] and talk about space with you all uh in [54:24] in such a big space [54:27] uh so what we're going to talk about is [54:29] uh we look for NASA and JPL look for the [54:32] big answers uh we're going to look for [54:34] it in new ways we now have new [54:37] technologies coming and new Workforce [54:39] new Partnerships and it's going to be an [54:41] exciting time uh these are the questions [54:44] that affect humanity and yes this is a [54:46] cloud computing conference so I will tie [54:48] in cloud computing into this uh but [54:50] first of all uh if you remember where [54:53] you were four years ago I certainly do [54:56] uh it was an exciting time where we're [54:57] going to attempt a crazy engineering [54:59] trick we're going to land a 2,000 lb [55:02] Rover 150 million miles away uh all by [55:06] itself from thousands of kilometers per [55:09] second to standstill in seven terrifying [55:11] minutes uh what we did we tried [55:13] something new we tried AWS to give you [55:17] all the pictures at the same time as we [55:19] saw them and let's see what happened and [55:21] where were you when curiosity landed on [55:23] Mars [55:29] coming up on the [55:30] [Music] [55:33] tree reports entry [55:35] interface at this time it'll begin [55:37] pressurizing the repulsion system to [55:39] increase the thrust of the system it'll [55:41] use that for all the maneuvering in the [55:42] atmosphere we're about to [55:45] do touchdown confirmed we're safe on [55:48] [Applause] [55:58] time to see where our curiosity will [55:59] take [56:00] us so these answers really matter and [56:03] you have to think about the people who [56:05] were in the room their careers hinged on [56:07] this they've been working at it for [56:08] years and years and years and it was the [56:11] generation that when we grew up we [56:14] looked into the heavens and we saw [56:17] pictures uh we didn't really know what [56:19] they were but they were [56:21] interesting then came to Space Race and [56:24] we were transfixed we were fixed in [56:26] front of the TV we were observing people [56:29] on the surface of the Moon it was [56:31] completely amazing now the Next [56:34] Generation they're not going to be happy [56:36] with just observing they're not going to [56:37] be happy with just saying can we get [56:40] there they're going to expect it to [56:41] happen they have much more knowledge [56:42] they have smartphones with NASA and JPL [56:45] apps and uh they will be expect to be [56:48] amazed by what they find not if they [56:50] find it they don't want to just [56:52] participate they don't want to just [56:54] observe they want to participate uh uh [56:56] and they want to be part of it whether [56:59] they're on the surface as my personal [57:02] Heroes when I grew up were astronauts [57:04] they will want to participate uh from [57:06] using augmented reality or uh using [57:09] their smartphones or on the surface as [57:11] astronauts so it's no longer about a [57:14] space race between countries it's a race [57:17] for Humanity into space I want to take [57:20] you on a [57:21] journey of these big questions of [57:24] discovering our universe so so first of [57:28] all how can we help protect Mother Earth [57:32] so we're going to go into detail that's [57:33] one of the big [57:35] questions then we'll go deeper into the [57:37] universe and we'll ask is there life in [57:40] space then we're going to finish on my [57:43] personal favorite Mars and see is there [57:46] or was there ever Life on Mars so those [57:49] are the big questions and we're going to [57:50] look for it in new [57:53] ways so let's talk about infrastructure [57:56] James talked about Network it's an [57:58] awesome network but it's groundbased our [58:01] network is in space and what we have is [58:04] we have the deep space Network that has [58:05] antennas that are strategical place so [58:07] as the Earth spins uh we can always hear [58:10] and and communicate with a [58:12] spacecraft we have about 30 spacecraft [58:15] uh in our solar system and beyond and [58:18] instruments uh that we track every day [58:20] for NASA and for other uh [58:24] industry so what's going on right now [58:26] well if you were to whip out your [58:27] smartphone you could find out right now [58:29] what's happening so that's a change [58:31] everybody can participate and how can we [58:33] get better results how can we improve [58:36] this infrastructure well certainly more [58:38] data and more compute power is going to [58:40] help us answer these big [58:43] questions how much data uh this is the [58:47] uh surface moisture active passive and [58:50] uh ocean carbon Observatory they're [58:52] satellites that or spacecraft that [58:53] circle the Earth and they help protect [58:55] mother Earth by looking for uh ice [58:59] carbon dioxide water and they collect a [59:02] terabyte per day that's a lot of data [59:05] now I want to go into each of those [59:07] questions a little bit deeper so we're [59:10] sending out two New [59:12] Missions these are called SWAT and nysar [59:15] they are doing the same thing they're [59:17] going to circle the Earth and they're [59:19] going to look for they're going to [59:21] bounce radar off of the ground and [59:23] they're going to bounce it and look for [59:25] the water table the water table in [59:27] oceans in rivers in the ice sheet even [59:31] down to reservoirs so they will collect [59:34] a lot of data how much [59:37] data they're 100 times bigger than the [59:40] other two how much data is that it's 100 [59:43] terabytes per day 100 gigabits per [59:45] second all the time much too big for our [59:48] data centers so what are we going to do [59:50] we're going to use cloud [59:51] computing and it's not just about the [59:54] infrastructure when we had orbiting [59:56] carbet Observatory we discovered [59:58] something called spot market and you're [60:01] all going du I know about the spot [60:03] market for us it was a revolution [60:06] because we discovered we could all of a [60:08] sudden compute at a fraction of uh the [60:11] dollar Pennies on the dollar so that is [60:13] not part of our operational way of [60:15] working so it is not just the [60:16] infrastructure but it's how we work so [60:19] that's going to help farmers it's going [60:20] to help predict us predict floods [60:23] droughts city planners it'll help [60:25] everybody body that's one of the big [60:27] questions now how else could we help [60:29] protect Mother Earth what if a big [60:31] asteroid was about to hit us uh we found [60:35] 1,600 uh Earth uh near Earth asteroids [60:38] in just uh recently and what if close on [60:43] were to come close how would we change [60:46] it or how do we move it oh we don't know [60:50] we're about to figure it out so we're [60:52] going to send out NASA and JPL is [60:54] sending out something called the as asid [60:56] redirect Mission and the idea is to very [61:00] straightforward go find an asteroid [61:02] about 400 m uh go pick up a boulder a [61:06] multi-ton boulder from that and then [61:08] bring it in orbit around the Moon where [61:10] AST astronauts can then come and mine it [61:13] and then learn how to tractor beam it [61:16] and redirect it piece of cake [61:19] right a few challenges uh an asteroid [61:23] has very low gravity so you're likely to [61:25] bounce off before you can even pick it [61:27] up the boulder the boulder might be [61:29] stuck uh and how are you going to get [61:32] there we don't have enough propulsion to [61:34] get there so that's an interesting one [61:36] the solar electric propulsion is an idea [61:39] that the NASA and JPL Engineers came up [61:40] with and what they're going to do is use [61:44] xenon gas to essentially use photons to [61:47] push the spacecraft forward it's you're [61:49] going to use 1/10th the amount of uh [61:52] fuel it's going to have three times more [61:54] power and two times s more efficient uh [61:57] you can then pair them and you can then [62:00] if you can pair enough of them each one [62:02] will generate about 50 Kow of power pair [62:04] enough of them to 300 KW and you can get [62:06] humans to Mars and back that's really [62:11] the key so what has Cloud got to do with [62:14] this well it has everything to do with [62:15] this we decided to do it differently and [62:17] actually use cloud computing to start [62:20] the project so it started in the cloud [62:22] uh now we're partnering much more easily [62:24] we're pivoting just like a startup wood [62:26] and by partnering with industry uh who [62:29] will build a lot of this uh it's going [62:30] to be an amazing [62:33] change another big question is where [62:36] could we find life and I'm sure all of [62:39] you know what this [62:40] is right maybe not but for those who [62:43] don't know it's Europa Europa is a moon [62:47] of Jupiter and it's well it's an [62:49] interesting what you need for life you [62:51] need energy and you need water Europa [62:54] has both in fact it has two times the [62:57] amount of water that Earth does and what [63:00] we can see is if you drill down on the [63:02] surface a little bit uh you notice [63:04] something interesting there are no [63:06] craters because it's a giant ice sheet [63:09] it's about 10 mil thick but it flows [63:12] like the Arctic Ocean does so we have [63:14] water and we have energy now 10 mil [63:18] thick that's a bit of a problem but [63:20] there are lakes inside of that ice sheap [63:23] that we think is one of the more likely [63:26] places to house life so we're going to [63:28] send an Orbiter to orbit Europa it's [63:31] about 400 million miles away and it's [63:34] going to find the landing spot then it's [63:37] going to land in all of these uh ice [63:40] crevices and can you imagine how much [63:42] data and simulation that will take [63:45] because it has to be completely [63:46] automated without cloud computing [63:48] there's no way we could do it and we're [63:51] using modelbased engineering and other [63:53] interesting goodies and it's all started [63:55] in the cloud [63:57] so that all of these things by the way [63:58] are happening in the next 5 to 10 [64:01] years now if we one day needed to export [64:04] Humanity would we have a place to go [64:07] could we find Earth 2.0 so there's [64:10] something called extra solar planets or [64:12] exoplanets and they are planets that [64:14] Circle their Earth just their Sun sorry [64:17] just like we Circle our Earth and how do [64:20] you find them you look through a pinhole [64:24] at the same spot in the scout for a very [64:26] long time and then you notice that there [64:29] is changes in the energy that you get [64:32] that is perhaps a a planet orbiting its [64:36] Sun or a wobble in the uh the orbit of [64:40] the planet itself from the star near it [64:44] when you do that we've found so far uh [64:47] 3,000 confirmed exoplanets in just a few [64:50] years 2400 [64:52] unconfirmed but of those are any of them [64:54] earthlike [64:55] we actually live there so far we found [64:59] 21 uh 21 by just looking through this [65:01] pinhole in just a few years problem is [65:04] they're a little far away the closest is [65:06] four light years away how far is four [65:08] light years would take a thousand years [65:10] to get there with current [65:13] propulsion so what if you were to send a [65:16] nano satellite you could actually get [65:18] there in 20 years uh science fiction so [65:21] was Landing 2,000 lb in Mars just a few [65:23] years ago uh in Innovation is moving [65:26] much faster the James web telescope is [65:28] going to go from pinhole to much larger [65:31] aperture the science the simulation the [65:34] math is going to require hundreds of [65:36] thousands of servers to do this and but [65:39] we don't have to own them anymore so [65:41] cloud computing is a really really big [65:43] deal for [65:46] us that was near of an [65:50] exoplanet now let's finish on Mars uh we [65:54] have not yet had humans on Mars there [65:57] only been one to my knowledge and Matt [66:00] Damon and he came back so that was good [66:04] uh for us to send more people we want to [66:07] prepare for it now curiosity landed four [66:10] years ago and it was an amazing thing [66:13] and like any teenager would do it took a [66:16] selfie so it took its giant robot arms [66:18] and it spun around and it took a selfie [66:20] to make sure we're healthy what have we [66:23] found so far this is a picture from so [66:27] welcome what does it look like has [66:29] anybody been to California's death [66:32] valley Death Valley looks just like this [66:35] because in fact it was formed by flowing [66:37] Waters both of them were uh and Earth uh [66:41] and Mars had the same conditions for [66:43] life about 4 billion years ago here's [66:45] another [66:46] picture it's uh the namid Dune on Mars [66:49] again it could be a sand dun on Death [66:51] Valley so we found flowing [66:54] water but we haven't found the smoking [66:56] and we found conditions for life we [66:58] haven't found the Smoking Gun we haven't [67:00] found life so JPL and NASA are going [67:02] back we're going to send uh a new [67:05] spacecraft Mars uh 2020 named for now to [67:09] see if we can uh see the science of life [67:13] and pave the way for [67:15] humans the last time we had 20 [67:18] minutes so it's amazing you you it's uh [67:22] you know 150 million miles away you only [67:24] have 20 minutes to figure out what the [67:25] the Rover is going to do the next day uh [67:27] 20 minutes means that sometimes you miss [67:29] a day the Rover stays parked that's not [67:32] a really good use of time but if we [67:34] could speed that up to 5 minutes we [67:36] wouldn't miss a day how do you do that [67:38] cloud computing so immense compute power [67:42] very fast and use some machine learning [67:45] to augment it so we've gone now and [67:47] that's what we're going to do so Mars so [67:50] cloud computing has gone from being a [67:51] very nice to have and engaging you all [67:54] with the pictures to something Mission [67:55] critical every single day now what will [67:59] this spacecraft look like or Rover [68:02] rather so if you look at it uh it's [68:04] going to have some changes it will have [68:06] new Wheels uh stronger and skinnier and [68:09] deeper grooves so it can climb uh taller [68:12] Hills it'll have a microphone so we can [68:14] hear the sounds of Mars uh it'll be able [68:17] to drill and store the rocks for future [68:21] astronauts to come and pick it up or [68:23] future missions uh even unman missions [68:27] and it's going to be the first test of [68:28] actually producing Oxygen on Mars to [68:31] prepare for next maton that comes up [68:33] there and needs some help it's also [68:34] going to have some new Landing [68:36] techniques the other thing we're looking [68:38] at is uh creating a scout a helicopter [68:41] on Mars why [68:43] not the may not be on this Mission but [68:47] how could you fly a helicopter at [68:49] 100,000 ft if we have any helicopters [68:51] here um no but we've shown that in our [68:54] lab we could do it by using encounter [68:55] rotating blades so it's possible and [68:58] that could be a scout to peek over the [68:59] hill and see what's coming [69:03] next these look remarkably like toys and [69:06] that's not a bad thing on contrary it's [69:07] speeding up how we work how we test how [69:09] we Infuse and if you saw bb-8 this is [69:13] bb-8 a toy seeing our toy Rover uh the [69:17] little guy there is a 3D printed Rover [69:19] for $3 on the desktop 3D [69:22] printer and we found that it could climb [69:25] glass [69:26] and if it could climb glass in our lab [69:28] could it do it on the space station so [69:31] we put it in the vomit Comet and it's a [69:34] zero gravity and you can see it's [69:36] climbing fine uh and it worked so this [69:39] is an idea of going from a crazy little [69:41] toy idea to actual something on the [69:43] space station in just a few months it's [69:45] unheard of [69:48] speed so this is meet Roi roie is a [69:53] Outreach Rover it's a miniature coffe of [69:56] the Mars uh exploration Rover and it's [69:58] built from arduinos Raspberry Pi's uh [70:01] open software and Amazon's iot and [70:03] Lambda tomorrow at the iot State of the [70:07] Union we're going to drive it on stage I [70:09] hope I hope it works uh and you can use [70:13] your joystick to drive it but that's so [70:16] yesterday you're going to be able to [70:17] drive it with Alexa so you can talk to [70:20] it and you'll see how that works and you [70:22] can ask it questions about Mars and [70:24] it'll answer and some other goodies that [70:26] you'll find out tomorrow if you don't [70:29] have a robot uh you can just use Alexa [70:33] So today we're announcing that tomorrow [70:36] you'll be able to use you know Amazon [70:38] dots your Amazon echos and you can ask [70:40] questions about Mars this is all about [70:44] exploring and getting crowdsourcing and [70:47] getting people to understand and care [70:48] about Mars and ask new questions uh by [70:52] the way the uh Rover is a blueprint that [70:54] everybody can have and you can build [70:56] your own and it's for schools [70:57] universities and uh museums or your [71:00] home so with this the idea is that we're [71:05] all going to be the future explorers the [71:07] things that you do the amazing compute [71:10] power the hackathon uh last night was [71:13] just amazing from zero to uh liftoff in [71:17] just one day and it all autoscaled it [71:19] was serverless and it was one the most [71:21] amazing things I've ever seen that's [71:24] what you're all going to help do and [71:26] your children are the ones that will one [71:28] day walk on Mars uh whether it is [71:31] virtually through augmented reality or [71:33] physically as astronauts but uh please [71:37] engage with us there are many NASA and [71:39] JPL people in the audience and find US [71:42] during the conference or afterwards and [71:45] let's help answer these big questions [71:48] for Humanity together and thank you very [71:50] much for listening [71:54] [Applause] [72:03] I'm going to the iot State of the Union [72:05] that sounds pretty [72:06] interesting Tom mentioned machine [72:08] learning machine learning is one of [72:11] those things where you know there's lots [72:14] of hype about lots of things this one is [72:16] extremely real this time machine [72:18] learning is going to be absolutely the [72:20] dominant workload I believe it's going [72:22] to cause the growth that I've shown you [72:24] so far to look like just getting started [72:27] small stuff machine learning is going to [72:29] be fundamentally important across the [72:31] industry it's going to open up new [72:33] Solutions it's going to allow new data [72:35] to be exploded in in very different ways [72:38] but key thing at least from my [72:41] perspective having seen a few of these [72:43] major new transitions is got to have [72:46] scalable infrastructure it's got to be [72:49] built on scalable infrastructure because [72:51] it's always the case that the problem [72:53] size gets bigger faster than the systems [72:56] get bigger it's always the case so [72:58] scalability is number one and I'm very [73:02] happy to have Matt up here [73:04] today Matt's going to be coming in to [73:07] talk about machine learning and what [73:10] we're doing with machine learning give a [73:11] little bit of a rundown Matt please come [73:13] on up [73:20] [Music] [73:26] good evening everybody thank you James [73:29] uh I'm going to talk a little bit [73:30] tonight about artificial intelligence [73:32] and we've seen tremendous interest with [73:34] our customers and tremendous momentum [73:36] behind artificial intelligence really [73:38] over the course of the last 5 to six [73:40] years and it's been driven by the [73:43] Trinity of three different areas the [73:45] first is a set of algorithms these [73:48] algorithms are actually pretty old uh I [73:50] did my PhD in machine learning 20 years [73:52] ago and a lot of the algorithms haven't [73:54] really changed [73:55] but they're starting to evolve and [73:56] become more sophisticated and that's [73:58] driving huge amount of momentum with [74:00] artificial intelligence the second area [74:03] is the ability to be able to collect and [74:05] harness increasing amounts of diverse [74:08] data and this is really being enabled by [74:11] the cloud and with AWS because the data [74:13] center walls that used to box in the [74:16] collection of data are just melting away [74:18] in the cloud where you can collect and [74:19] store as much data as you need and when [74:22] you start to drive more and Divergent [74:24] collections of data through these [74:25] algorithms wonderful things start to [74:27] happen and the third part of the Trinity [74:30] is the availability of utility computing [74:33] specifically the availability of gpus at [74:35] the end of an API call which allow you [74:37] to run the algorithms with large amounts [74:40] of data with higher degrees of [74:41] sophistication at extraordinary scale so [74:44] these three things are really fueling [74:46] the next wave of artificial intelligence [74:49] and one set of Technologies in [74:50] particular has really come uh to the [74:52] Forefront and that's something which [74:54] they call Deep learning learning deep [74:55] learning is just a set of statistical [74:58] machine learning techniques uh which [75:00] does feature detection through a series [75:01] of hierarchial layers uh trained with [75:04] artificial neural networks and it's [75:06] really found a a niche in solving some [75:09] of the hardest problems in computer [75:10] science image analysis being able to [75:12] look inside pictures like this beautiful [75:15] beagle here and be able to tell is it a [75:16] cat or is it a dog what type of dog is [75:19] it doing voice analysis natural language [75:21] processing personalizations and [75:23] recommendations and also driving a next [75:25] wave of autonomous vehicles and [75:27] autonomous robotics now the challenge is [75:30] that these hierarchies of layers in the [75:33] real world get really really complicated [75:36] uh this is a this is a real algorithm [75:38] here this is one of the image analysis [75:40] algorithms which is performs the best [75:42] and you can see just how many layers it [75:44] has and how much complexity it has and [75:46] this is a real challenge across three [75:48] different domains the first is [75:50] programmability you need to be able to [75:52] go in and Define these networks and they [75:55] can have thousands of different layers [75:56] inside them which is frankly a bit of a [75:58] pain the second is you want these [76:00] networks to be as portable as possible [76:02] and to use memory as efficiently as [76:05] possible so that you can move them from [76:06] different domains training them up in [76:09] the cloud but also running them [76:10] potentially uh on mobile devices running [76:12] them in the car or doing inference on [76:14] other CPUs so portability becomes [76:17] extremely important and the third peer [76:19] of data science is performance as James [76:21] was saying performance in training and [76:23] performance in inference of your machine [76:25] learning [76:26] models and what we've seen is a [76:29] tremendous amount of interest in running [76:31] these sorts of training of deep learning [76:33] using gpus up on AWS and a couple of [76:36] weeks ago we made available uh the next [76:38] generation of um uh uh P2 GPU instances [76:42] which came prepackaged and pre-built [76:44] with a army uh which was preconfigured [76:47] with a set of Open Source deep learning [76:49] Frameworks these Frameworks things like [76:52] mxnet tensorflow Cafe Theo torch and [76:57] cntk put deep learning into the hands of [77:00] mere mortals they take care of a lot of [77:02] the uh heavy lifting associated with [77:05] fiddling around with Cuda cor and allow [77:07] you to focus on the algorithm they allow [77:09] you to dictate and build those large [77:11] deep networks uh to run at scale and [77:14] customers have found using these across [77:17] our P2 instances incredibly easy with P2 [77:20] you can get access of up to 40,000 Cuda [77:23] cor and deep learning algorithms such as [77:25] the ones enabled through these [77:26] Frameworks are already custom built to [77:29] fly across a large number of cacor so we [77:32] package these up we put them inside an [77:34] army and from the marketplace today you [77:36] can with a single click spin up a [77:38] complete environment to run any of these [77:40] Frameworks in addition to that we made [77:42] available a cloud formation template [77:44] which allows you to prision large [77:46] elastic autoscaling clusters of these [77:48] deep learning armies so you can take any [77:50] of these Frameworks run your algorithms [77:52] at tremendous scale and customers been [77:54] doing that almost from day one [77:56] availability of P2 one really great [77:58] example is the early detection of [78:01] diabetic retinopathy at Stanford [78:03] University so diabetic retinopathy is [78:06] one of the leading causes of blindness [78:08] in the world 12% of all new cases here [78:10] in the US of blindness are caused by [78:14] diabetic retinopathy and what happens is [78:16] you get tiny aneurysms in the back of [78:18] the eye on the retina and that starts to [78:20] over time cause blindness and a paper I [78:22] was reading the other day was talking [78:24] about the uh the impact of potential [78:26] early detection the only way to detect [78:29] uh diabetic retinopathy is to take a [78:31] photo of the fundus a photo like the one [78:33] on the slide behind me and look for [78:35] these very very very small aneurysms and [78:38] if you're able to do that early enough [78:39] and treat the cause which is the uh [78:42] treatment of diabetes um then you can [78:45] cure or prevent diabetic rhinopathy and [78:47] blindness in 90% of cases so what [78:50] Stanford are doing is they're using deep [78:53] learning to go and they've trained a [78:54] system to go detect these tiny little [78:57] aneurysms so when these uh these [78:58] photographs like this are taken we can [79:00] automatically detect problems and you [79:02] can treat them early and you can prevent [79:04] one of the leading causes of blindness [79:06] uh here in the US we see other customers [79:09] like Wolfram building completely new [79:12] categories of applications and products [79:14] such as wolf from alpha wolf from alpha [79:16] is a computational knowledge engine and [79:18] it takes a web of information and starts [79:21] overlaying different concepts on top of [79:23] it so you can ask more structured [79:25] questions across really broad [79:27] unstructured data so if you need to know [79:29] who made one of the greatest albums of [79:31] all time Pet Sounds you can ask it and [79:33] it will tell you it was of course the [79:35] Beach Boys and we see tremendous [79:37] application across autonomous vehicles [79:40] there's a startup called to simple uh [79:42] these are some videos that they sent us [79:44] and they're using deep learning with [79:46] computer vision systems on cars to drive [79:49] uh really sophisticated autonomous [79:51] driving so here on the left hand side [79:53] you can see object and Lane detection uh [79:56] both during the day and at night uh you [79:58] can see semantic segmentation uh this is [80:01] pixel by pixel recognition of the road [80:03] surface uh this is obviously taken in [80:05] China and if you've ever driven in China [80:07] uh obviously that's a traffic jam um and [80:09] in addition to that they have centimeter [80:11] accurate positioning of the car in 3D [80:14] space using stereo Vision on the cameras [80:17] and that's the third heat map you can [80:18] see over there on the right hand side [80:20] which allows you to place the car [80:21] incredibly accurately using deep [80:23] learning techniques on flat [80:25] images so these deep learning Frameworks [80:28] MX net tensorflow and the rest of them [80:30] have really found a a way of materially [80:33] accelerating everything from autonomous [80:35] driving to Revolutions in in healthcare [80:38] um by making deep learning available to [80:40] more developers and I wanted to focus in [80:42] on one of these libraries today uh which [80:44] is mxnet mxnet is uh a lot of the [80:48] characteristics that developers like [80:50] when they're going off and building deep [80:52] learning so the first is programmability [80:55] mxnet supports a really broad set of [80:57] programming languages so whether you're [80:59] used to using python or Scala whether [81:01] like me you're a fan of Julia or [81:03] JavaScript or matlb or go you can use [81:06] all of the languages that you're used to [81:07] using and start running uh your deep [81:10] learning straight away because mxet has [81:13] both a front end and a back end and no [81:15] matter what you use on the front end it [81:17] gets compiled down so you get guaranteed [81:20] um uh sophistication and guaranteed [81:22] performance uh using the back end [81:25] there's some programming languages uh [81:28] which have an imperative model uh this [81:30] is great it allows you to write scripts [81:33] effectively and those scripts as anyone [81:35] that's ever written python or Ruby um [81:38] you know that it's very very flexible uh [81:40] and you can get to use a lot of the [81:41] loops and the language Native features [81:43] that make these environments uh so [81:45] productive however imperative programs [81:48] are are very hard to optimize it's very [81:50] hard to get the scale and the [81:51] performance that you need uh because [81:53] some of that flexibility is built in and [81:55] that takes away the opportunity to [81:56] optimize at the other end of the [81:58] spectrum there are declarative [82:00] programming languages SQL is a [82:02] declarative programming language you get [82:04] more chances to optimize uh what's going [82:07] on and you get to use that optimation [82:09] optimization across different languages [82:12] but it's much less flexible and this is [82:13] the approach to things like tensorflow [82:15] and Theo and Cafe approach with mxet one [82:18] of the core functional pieces of [82:19] functionality in addition to the [82:21] multiple programming languages is the [82:22] ability to be able to mix both [82:25] imperative models and declarative [82:27] languages in fact that's where the name [82:29] comes from mxet is a mix of networks and [82:32] this means that you can manipulate the [82:34] graph of layers in the way that you [82:36] might want to with declarative uh in [82:38] this case symbolic executors and you're [82:40] also able to mix that with some of the [82:42] features of imperative programming [82:43] languages such as iteration loops and [82:46] parameter updates or feature extraction [82:48] so this is an extremely powerful model [82:50] being able to mix the two together it's [82:53] also incredibly port [82:55] mxnet models fit in very very very small [82:58] amounts of memory it's incredibly memory [83:00] efficient in fact a thousand layer [83:02] network will fit in less than 4 gig of [83:05] memory and that means you can take your [83:07] models across a much more diverse set of [83:09] applications you can embed them in [83:11] Mobile you can put them inside connected [83:13] devices without many resources on board [83:15] you can use them to power robotics such [83:17] as drones you can even run it with [83:19] JavaScript directly inside your browser [83:21] so there's a huge amount of portability [83:23] through the memory memory efficiency of [83:26] mxet and finally coming back to the [83:28] third P [83:30] performance what performance enables is [83:33] larger data sets to be able to mine and [83:36] train and it enables you to build more [83:39] sophisticated models however writing [83:42] parallel programs is extremely painful [83:45] uh particularly when you're dealing with [83:46] tens of thousands of Cuda cor uh this is [83:49] even worse when you're dealing with [83:50] complex networks uh which frankly if [83:53] you're having a forward backward update [83:55] involves o to the number of layers uh [83:57] scale and that can be across hundreds or [84:00] thousands of different tensor [84:01] computations and calculations it's very [84:04] intensive and the more scale the larger [84:06] your network the worse the scaling [84:08] characteristics so we're looking for uh [84:11] Frameworks which help us achieve the [84:13] scale so we can build more sophisticated [84:15] features and drive more data through it [84:16] for training an mxet has some really [84:19] nice features it allows you to take [84:20] serial code and automatically [84:22] parallelize them and it'll pause out the [84:24] data dependencies to do that efficiently [84:26] and it'll also apply autop [84:27] parallelization of your data across [84:29] those Cuda cores using a coordinated key [84:32] value store and this allows you to be [84:34] able to scale to multiple cores on a [84:36] single GPU it also allows you to be able [84:39] to scale across gpus on a single box um [84:43] and so this allows you to take more data [84:45] and build more sophisticated models so [84:48] we ran some benchmarks or running uh an [84:51] mxet Network for image analysis cross a [84:54] set of gpus and here you can see with [84:56] the red line This is ideal scalability [84:59] this is where you want to be this means [85:00] that if you add another GPU you get a [85:03] double speed up as you go from 1 to two [85:05] and we're seeing a 91% efficiency for [85:07] image analysis Frameworks As you move [85:10] from multiple one GPU to multiple gpus [85:12] on a single box but for large networks [85:16] with sophisticated models and when [85:18] you're driving huge amounts of data [85:19] through those models you don't just want [85:21] to scale to 16 gpus which is what we [85:23] have available directly inside uh the uh [85:26] uh P2 largest P2 instances you want to [85:28] be able to create big clusters of those [85:31] P2 instances and run across as many of [85:33] them as you can so you want to be able [85:34] to scale across nodes and you want to be [85:36] able to light up as many P2 instances as [85:39] you can so we also ran the same [85:41] benchmarks across a multiple set of [85:44] actually 16 different um uh P2 instances [85:48] uh so you can see that this is the [85:49] original graph that we just looked at [85:50] that red line again is ideal scalability [85:53] and we're going to move from just a [85:54] single instance with 16 gpus to 16 [85:57] instances so a 16x increase in the [86:00] amount of computing capability on the [86:01] GPU and you can see here that you're [86:03] still maintaining close to uh ideal [86:07] efficiency 88% efficiency uh with mxnet [86:10] so just a 3% drop in efficiency when [86:13] you're scaling to 16x the capacity on [86:16] the back end um so if you want to take a [86:21] look at this yourself um you shouldn't [86:23] trust my own benchmark one of the [86:24] benefits of the cloud is you can spin [86:26] these things up uh and you Benchmark [86:29] your own code uh so today we're making [86:31] available a new deep learning Benchmark [86:33] set of codes uh you can use our cloud [86:35] formation template to spin up the [86:37] capacity that you need uh you can log [86:39] into the master node and then you can [86:40] run the exact same benchmarks that I [86:42] just used uh to draw these graphs uh [86:44] this is available today so we really [86:47] like mxnet a lot of our customers really [86:49] like mxnet and we recently announced [86:52] that mxnet is going to become the uh [86:55] deep learning framework of choice for [86:58] AWS that means that we're going to be [87:01] providing code contributions back to the [87:03] mxet open source framework we're going [87:05] to be investing in programmability and [87:08] the uh developer experience we're going [87:10] to be building out more documentation [87:11] more example codes and we're going to be [87:13] investing in a series of Tools around [87:15] mxnet and in addition to that mxnet is [87:18] going to be the foundation of our future [87:20] AI services and to hear a little bit [87:22] more about that you'll have to come back [87:24] tomorrow so as much as we like mxet uh [87:27] we remain committed to building out a [87:30] platform where all of these tools run as [87:32] well as possible this is the exact same [87:34] strategy that we have for our databases [87:36] where you can run MySQL and postgress or [87:39] Maria DB or Oracle or or SQL Server [87:42] right there on RDS we want to provide [87:44] the same flexibility so while we love [87:46] mxet we welcome all of your deep [87:48] learning workloads on this very very [87:50] high performance set of gpus and with [87:52] that I'll hand it back to James thanks a [87:53] lot [87:55] [Applause] [87:57] [Music] [87:59] thanks [88:00] B that was Dr Matt Wood general manager [88:04] of product strategy good job Matt did [88:07] you see that multi-server scaling in [88:09] mxnet that's the golden one love that [88:12] one let me talk to you about [88:13] sustainability and renewable energy we [88:15] made a commitment a little bit some [88:17] actually well more than a year back to [88:20] be 100% renewable in AWS that took a [88:23] long [88:25] long amount of thought to get to that [88:27] point because for us to make that [88:28] commitment we have to at least see a [88:30] path to get there we're not afraid of [88:32] big challenges we will sign up for [88:35] difficult tasks but we have to see a [88:36] path where we actually can can [88:38] reasonably get there and we're there I [88:40] want to show you what we've done so far [88:43] tell you announce a little bit about [88:44] what we're going to do and then show you [88:46] how we've done what what has been done [88:48] so far so let's jump into [88:52] that last year we were at [88:54] 25% and by the way we know how to hit [88:58] 100% like that so it can be done it's [89:02] just if customers put their workload in [89:06] Oregon our 100% renewable region it's a [89:10] we're 100% And for many customers they [89:12] do exactly that and we appreciate it [89:14] it's a good thing it's a great thing [89:16] Oregon's growing at a phenomenal pace [89:18] for this reason probably many others as [89:20] well however many customers want to have [89:22] their data close to their users many [89:24] customers want to have their their [89:26] computation close to their data there [89:28] are many many reasons to put data [89:30] centers in different places and it's [89:32] just a fact that some of the places [89:34] where customers want to put data today [89:37] aren't the cleanest power s locations in [89:40] the entire world so we're signing up for [89:42] a challenge because we got to make it [89:43] right everywhere and we're not going to [89:46] do that by by removing Choice from [89:48] customers we're going to do both because [89:50] that's the way it is at Amazon and so 25 [89:54] % last year we signed up for 40% this [89:58] year 40% is an interesting number [90:00] because let's just say we doubled [90:02] capacity in that period it's a little [90:04] it's bit a little bit less than doubled [90:05] but let's just say we doubled capacity [90:07] in that period the team the the the the [90:10] AWS Power Team the energy team has [90:13] signed up to to deliver 40% if they [90:17] double the amount of renewable energy [90:18] during that period will we double the [90:20] capacity they'd be at 25% that's a kind [90:23] of a tough one and so at the same time [90:26] we're growing in the way that I showed [90:27] you the team has to catch that and beat [90:30] that and hit 40% and we love them Nat's [90:34] team is doing an awesome job every [90:36] discussion I have with them I find I [90:38] learn something they're always active [90:40] debates we are going to hit 45% at the [90:42] end of this year not [90:46] bad we're really [90:48] happy today we're announcing we're [90:50] feeling good we we really think we've [90:53] got good systems in place and we know [90:55] how to do this we've announced we're [90:57] announcing today we're going to hit 50% [90:59] at the end of next year so if customers [91:02] want to be 50% green just go to the [91:04] cloud simple as that and we're tracking [91:07] to towards our goal and we're happy [91:09] about the progress how do you do it like [91:11] let make it real for [91:12] me hard work the team's delivered a lot [91:15] of basically the teams delivered a lot [91:18] of green power right there you know [91:21] instantly what we're looking at we've [91:22] got 150 megaw [91:24] 80 [91:26] megawatt another another 208 [91:31] megaw 100 [91:35] megaw 189 megawatts almost 200 megaw and [91:39] then a full another 180 megaw that's a [91:43] lot of [91:45] power just give so sum that Al together [91:48] that says that AWS is bring projects are [91:51] bringing online 9 [91:54] 7 megaw of new renewable generation 970 [92:01] megaw Jennifer and I took our boat [92:03] through the Columbia River up the [92:05] Columbia River a couple years ago one of [92:08] the one of the interesting parts of that [92:10] trip as we went through the Bonville Dam [92:12] Bonville dam is a unbelievable piece of [92:15] engineering it is astounding how big a a [92:20] piece of engineering it is and its [92:22] capacity I believe is 1,000 300 megawatt [92:25] so we're in that ballpark it's it's very [92:28] close to the bonable dam [92:31] phenomenal okay if you're like me the [92:34] first question you're going to ask is [92:35] wait a second wait a sec power is not [92:37] always blowing wind you know it's not [92:40] it's wind's not always blowing sun's not [92:42] always shining what are we going to get [92:44] if you take into account environmental [92:46] factors for all of those sites that I [92:48] told you we're going to deliver 2.6 [92:51] million megawatt hours an ually of [92:54] renewable energy we're super happy about [92:56] what's been done so far and the reason [92:58] we're willing to sign up for the goal [93:00] that I told you for 2015 is because the [93:02] team's doing good work listen we're I am [93:07] super happy to have had a chance to take [93:08] you through a little bit more detail on [93:10] what we're doing appreciate your time [93:12] and I hope you have a really good rest [93:14] of your week at at reinvent hope you [93:16] learn a lot hope hope it's worthwhile [93:19] thank you very [93:21] much thanks [93:25] w