Waymo Deploys AI Backbone to Scale Autonomous Rides to 10 Cities

Waymo部署AI底座将无人驾驶出行服务扩展至10座城市 驾驶辅助系统绝不会自然演化为完全自动驾驶出租车，因为应对纯无人驾驶挑战需要解决截然不同的质性与安全难题。 Waymo联合首席执行官Dmitri Dolgov详细阐述了规模化落地完全自动驾驶车辆的历程，解释了“Waymo Driver”如何利用结合摄像头、激光雷达与雷达的实时推理架构。他深入解析了由Driver、Simulator与Critic组成的Waymo基础模型生态，并说明闭环训练如何解决复杂的极端边缘场景。Dolgov区分了完全自动驾驶与渐进式驾驶辅助系统，并概述了自动驾驶车队的运营基础设施以及对未来城市的影响。最后，他将实现超越人类安全水平所需的技术突破，归功于Alphabet的长期信念与工程定力。 Dmitri Dolgov在苏联长大，曾移居日本和美国，并在1994年社会乐观情绪高涨时回到俄罗斯，之后再次赴美攻读研究生。 (0:01) Waymo已从科研与核心技术开发阶段转向全球规模化扩张的加速期，目前每周提供超过50万次完全自动驾驶出行服务。 (0:23) Waymo Driver在所有实时驾驶任务中均依赖车载本地算力处理，通过摄像头、激光雷达、雷达和麦克风阵列获取不间断的360度视野。 (2:51) Waymo的车外基础模型细分为三个高能力教师模型：Waymo Driver、用于云端合成训练的Simulator，以及用于评估行为表现的Critic。 (6:24) Waymo的EMMA论文表明，微调现成的视觉语言模型可以生成驾驶轨迹，但其表现类似“会说话的马”，缺乏完全自动驾驶所需的安全水平。 (9:59)

Waymo Co-CEO Dmitri Dolgov asserts that autonomous driving has transitioned from scientific research into global commercial deployment, powered by unified foundation models, multi-sensor physical grounding, and specialized operational infrastructure. Dolgov argues that full autonomy cannot be developed incrementally from driver-assist systems, requiring instead a dedicated full-stack architecture capable of satisfying extreme safety margins. AI Architecture, Distillation, and Sensor Grounding (2:51) Waymo's software architecture centers on a large off-board foundation model trained on physical world dynamics and social interactions. This model is specialized into three off-board "teachers": the Waymo Driver, a generative Simulator for closed-loop evaluation, and an evaluative Critic. The driving model is distilled into a compact, low-latency version that executes all real-time driving inference locally on the vehicle using sensor encoders and action decoders. To keep simulation efficient and verifiable, Waymo augments learned representations with structured intermediate concepts such as roads, signs, and physical objects. The physical vehicle relies on a complementary sensor suite of cameras, LiDAR, and radar rather than vision alone. In Waymo's experimental EMMA paper, an off-the-shelf vision-language model (VLM) demonstrated baseline driving trajectories, but Dolgov likens raw end-to-end models to a "talking horse"—impressive as a concept, but orders of magnitude away from the safety margins required for driverless passenger operations. Sensor fusion provides critical physical grounding; for instance, peripheral LiDAR reflections bouncing beneath a stopped bus have enabled the system to detect an obscured pedestrian's feet and anticipate their path before they emerge into view. Full Autonomy Versus Incremental Driver Assist (47:50) Dolgov rejects the perspective that consumer Level 2 and Level 3 driver-assist features (ADAS) will incrementally converge into fully autonomous Level 4 and Level 5 platforms, arguing they represent fundamentally distinct problems. Full autonomy eliminates the human driver as a fallback, meaning every additional "nine" of safety requires roughly ten times more engineering effort. Passive imitation learning—simply training models on human driving demonstrations—proves inadequate for rare, high-stakes edge cases, prompting Waymo's use of closed-loop reinforcement learning fine-tuning (RLFT). Operating domains also introduce physical and kinematic constraints that general AI decoders cannot resolve digitally. Freeway driving carries severe consequences because kinetic impact scales quadratically with vehicle speed, demanding instantaneous reactions to high-speed hazards like debris falling from trucks. In winter conditions, software world models must be paired with physical interventions, including heated sensor housings, specialized cleaning systems, and dynamic traction control on ice. Hardware Evolution and Manufacturing Cost Reductions (34:39) Waymo's deployment trajectory evolved through distinct platform generations, beginning with commercial service in Chandler, Arizona, in 2020 using fourth-generation Chrysler Pacifica minivans. While Gen 4 depended on segmented, isolated machine learning models, the fifth-generation platform deployed on Jaguar I-PACEs established a unified AI backbone that allowed the system to expand into dense environments like San Francisco and Phoenix. De-risking software on retrofitted luxury vehicles was a deliberate choice to avoid premature capital expenditures on custom automotive manufacturing. The upcoming sixth-generation platform transitions to custom-designed passenger vehicles, including the "Ojai" platform and Hyundai Ioniqs, featuring flat floors and sliding doors. Gen 6 reduces hardware costs to a fraction of Gen 5—approaching the cost of consumer ADAS suites—driven by supply chain maturity and price declines in automotive radar and solid-state LiDAR rather than mass production scale alone. Operational Scaling and Urban Second-Order Effects (46:37) Waymo has scaled commercial operations to approximately 3,000 vehicles delivering over 500,000 rides and logging more than 4 million autonomous miles weekly across 11 US cities, with public rider access in 10 and testing underway in Nashville. Fleet maintenance is increasingly automated: vehicles independently route to depots when low on power or flagged for cabin messes, park in designated stalls, and pull out automatically once unplugged. Dolgov expects autonomous vehicles to eventually serve all postal addresses across the United States, though low-density rural markets may rely on personally owned autonomous vehicles rather than ride-hailing fleets due to vehicle utilization constraints. At scale, predictable, automated driving is expected to reduce phantom traffic jams caused by human braking waves. Dolgov also forecasts that autonomous fleets will drastically reduce the need for idle parking, enabling municipalities to eliminate parking minimums and reclaim urban land. While Dolgov asserts that Waymo's core autonomous driving challenge is technologically solved, the transition to global scale remains constrained by physical edge cases like extreme winter weather, supply chain shifts, and depot automation. Waymo has declined to set a public timeline for selling personally owned autonomous vehicles to consumers, keeping its current capital focus on commercial fleet expansion.

Waymo联席首席执行官Dmitri Dolgov指出，公司已从早期底层科研探索全面跨入全球规模化部署阶段，目前每周在美提供超50万次全自动驾驶出行；他强调，全无人驾驶与辅助驾驶存在质的区别，其安全落地依赖于云端基础模型分化出的系统架构、结构化中间表征及多传感器物理融合，而非单一的端到端黑盒模型。 架构演进：基础模型与中间表征闭环 (6:24) Dolgov介绍，Waymo的实时驾驶系统完全在车端本地运行推理，非实时任务则由云端处理。其系统背后是一个庞大的云端世界基础模型，该模型被专门分化为三大离线“导师”系统：蒸馏到车端执行实时驾驶的Waymo Driver、负责生成式闭环模拟训练的Simulator，以及用于评估驾驶表现与构建奖励函数的Critic。三者共享对物理世界物体关联与轨迹预测的基础理解能力，使车辆能在行驶中实时评估多种假设并选择最安全路径。 针对行业内关于纯“端到端”方案的争论，Dolgov以Waymo关于EMMA模型的研究论文为例说明，直接微调开源视觉语言模型（VLM）生成驾驶轨迹在日常工况下表现尚可，但如同“会说话的马”，其可靠性与无人驾驶所需的安全性相差数个数量级。单纯依靠观察人类驾驶的模仿学习无法解决长尾极端场景，Waymo通过结合道路、标牌、物理几何等结构化“中间表征”，应用强化学习微调（RLFT）在闭环中进行高效模拟与实时安全校验，兼顾了机器学习的泛化能力与物理现实的可解释性验证。 传感器融合与硬件代际迭代的降本演进 (34:39) 在感知层面，Waymo坚持融合摄像头、激光雷达（LiDAR）与毫米波雷达（Radar），辅以定向麦克风阵列。Dolgov指出，LiDAR提供高精度三维点云采样且不受光照剧变影响，而Radar能穿透旧金山的浓雾或雨雪，检测视距受阻车辆。系统通过联合编码器将各传感器数据输入统一网络，而非各自独立推断后再做比对。在旧金山的一次实测中，一辆Waymo在避让停靠公交车时减速绕行，事后日志显示，车身侧边LiDAR接收到穿透车底地面的微弱反射信号，模型成功捕获了盲区行人的脚步移动并准确预测其轨迹。 在硬件发展路径上，Waymo从2020年以克莱斯勒Pacifica小巴在钱德勒市推出无人商业服务的第四代分散模型架构，跃迁至采用统一AI主干网络的第五代系统，从而攻克了旧金山与菲尼克斯的高难度路况。针对即将推出的第六代Ojai平台，车队转向以乘客为中心的定制化座舱设计。得益于雷达与激光雷达供应链的成熟，第六代传感器套件在简化硬件结构的同时，将硬件成本大幅压缩至第五代的一小部分，达到高端消费级辅助驾驶系统的价格区间，且相同算法能够直接复用到现代Ioniq等不同车型上。 辅助驾驶与全无人驾驶的本质鸿沟 (47:50) 针对辅助驾驶系统（L2/L3）是否能渐进升级为全无人驾驶（L4/L5）的观点，Dolgov明确表示两者是“根本不同的两个问题”，将其视为单一复杂度光谱上的渐进延伸是误导性的。辅助驾驶系统容许人工随时接管，对系统可靠性“9的数量级”要求显著较低；而在全无人驾驶工程中，每增加一个“9”的安全性，都需要付出十倍以上的研发代价。 不同驾驶场景的工程难点差异巨大。高速公路虽然规则清晰、常态下可预测性高，但由于碰撞后果随车速呈二次方激增，货车掉落物件等高速长尾事件极度危险；而在城区道路，驾驶难点则高度集中在上下客交互的社会生态博弈中，系统必须精确权衡临时双排停车、避让出入口及乘客需求的微小决策。此外，极寒结冰等恶劣气候涉及传感器物理加热、液体清洗及滑移路面动力学控制，属于深层机械与物理工程问题，无法直接通过数字化VLM的通用世界知识解决。 车队自主运营与城市交通的二次重塑 (46:37) Waymo目前在全美11座城市运营约3,000辆全自动驾驶车辆，向其中10座城市的公众提供服务（最新一座为纳什维尔），周行驶里程超400万英里。公司此前曾创下单日向四座新城市同时开放外部乘客运营的纪录。与早期高度依赖人工相比，如今车队场站已实现高度自动化协同：车辆在低电量或车厢脏乱时会自动返站，进出充电车位亦由车辆自主驾驶完成，仅充电插头插拔与深度清洁仍保留人工环节。 Dolgov预期，Waymo技术最终将覆盖美国邮政署所服务的各类地址，但由于低密度偏远区域车队调度效率和利用率低下，未来更适合以搭载Waymo系统的个人私家车形态落地。他预测，随着自动驾驶车辆在车流中占据主导，其平顺、可预测的行驶行为将消除人类驾驶急刹引发的交通“驻波”堵塞现象，大幅提升路网通行效率；同时，因车辆无需长期闲置停放，城市可逐步废除严格的停车位配额法规，释放出大量地面空间用于公共生活设施。 Dolgov总结认为，Waymo之所以能跨越近二十年的技术浪潮并进入多国拓展阶段（如伦敦与东京布局），在于母公司管理层长期坚持的技术定力与深厚研发投入。虽然核心驾驶技术已不存在底层原理盲区，但面对极端长尾场景、寒冷气候的物理控制以及个人私家车量产化等目标，真正的全无人系统仍需要漫长而严谨的验证周期。

[0:01] peak Russian optimism [2:51] Waymo Driver [9:59] Dmitri Dolgov: talking horse [9:59] Dmitri Dolgov: number of nines [20:31] social ecosystem of our roadways [22:29] long tail of really complicated stuff [22:29] everything is quadratic in speed [23:16] clearly moved past the stage of scientific research and deep core technology development to this new phase of accelerated global scaling and deployment [26:12] big bet on AI [29:38] feels like you're in a living room [34:39] a fraction of the cost [40:53] Dmitri Dolgov: It just blew my mind [46:37] ghost city [47:50] Dmitri Dolgov: fundamentally two different problems [47:50] Dmitri Dolgov: I think it's deceptive to think of them as incremental on one spectrum of complexity [50:49] fully, automatically orchestrated dance of autonomous vehicles [54:50] Slow is smooth and smooth is fast [57:56] Dmitri Dolgov: stamina and conviction to go the distance

[0:01] Dmitri Dolgov is Co-CEO of Waymo. [0:03] He joined Google’s self-driving car project in 2009 as one of its first [0:06] engineers, and was repeatedly promoted until he took it over in 2021. [0:10] Waymo is Google’s most successful moonshot [0:12] and now provides over 500,000 fully autonomous rides each week. [0:17] Cheers, by the way. Yeah, cheers. [0:21] You grew up in Russia, right? I grew up in Russia. [0:23] It was actually the Soviet Union. Right, exactly. [0:27] My dad is a physicist. [0:31] The Soviet Union started falling apart, and then he [0:34] had a visiting position in University, in Kyoto University for a year. [0:41] We moved there as a family. [0:42] Then he went to Berkeley, and I tagged along. [0:45] Then I graduated from high school. [0:47] I was thinking about the next thing I wanted to do. [0:49] I really like that technical school in Russia. [0:54] The Russians are serious about their physics. [0:56] They are. [0:56] I went back to Russia, and I got my bachelor's and master's there. [1:00] What year was this that you went back to Russia? [1:02] 1994. Okay. [1:04] That was almost peak Russian optimism in the sense where it was opening up. [1:09] It was. [1:10] I actually remember talking to my mom about it. [1:14] Of course, my parents grew up in the Soviet Union. [1:16] They've seen it. [1:19] They were born right before the war, and then they saw… [1:23] They lived through some really tough times. [1:26] I remember talking to my mom… [1:28] In fact, I got my green card here in the US before I went back, [1:32] and she insisted that I do it. [1:34] Actually, at that time, I wasn't thinking of coming back. [1:38] But then I was pretty excited about where Russia is and the trajectory it's on. [1:42] Being young and naive, there's no turning back. [1:48] Why did you decide to come back? [1:50] There's more of a—Yeah, I know. [1:53] It was pretty clear to me. [1:56] I wanted to continue studying math and computer science. [2:01] While the undergrad and master's that I got in physics and applied math, [2:05] I think was still an incredibly strong foundational [2:10] school of Russian math and science. [2:14] Graduate school, it was very clear to me that the best way to do it [2:18] was in the US, so I came back. [2:20] I'm struck by the founders of the two most [2:23] valuable UK companies are Russian math [2:28] nerds who both went to the same school, [2:31] Nikolay at Revolut and Alex Gerko at XTX. [2:39] It's a strong diaspora. [2:40] There's a company, not far from here, where one of the founders [2:45] also has a similar pedigree. [2:48] A company that we're closely related to. Exactly. [2:51] You know the classic engineering interview question of "What happens [2:56] when I type google. [2:57] com and hit enter?" As in, talk me through whatever you like, HTTP, DNS, and BG. [3:04] You can go down to whatever level of the stack you want. [3:06] Do you want to maybe just describe, when I take a ride in a Waymo today, [3:11] what's happening at a technical level? [3:13] What is the architecture? [3:14] Let me answer your question. [3:16] It was happening in real-time, but this is going to be only a part [3:19] of the story because we're going to be talking about the inference, [3:22] the real-time inference part of it. [3:24] If we want to have a deeper, richer, technical conversation, [3:30] I think it would be interesting also to zoom out and talk about the entire [3:35] ecosystem of what goes into building, evaluating, and deploying [3:39] the Waymo Driver. [3:40] But when you're driving around or being driven around, we think about [3:44] what we're building as a driver. Obviously, it's not a car. [3:48] It has a number of sensors that are positioned around the vehicle. [3:53] We use three different sensing modalities. [3:55] There are cameras, there's LiDARs or lasers, and there are radars. [4:00] Those are the primary ones. [4:02] There are also microphones, directional microphone arrays, [4:06] but those are the primary three for sensing the world. [4:10] They all have very nicely complementary physical properties. [4:15] They all have 360-degree coverage around the vehicle, so the Waymo Driver [4:19] sees 360 all the time. [4:23] All of the data goes into a computer, you would expect. [4:28] The software that process, now it's all AI, specialized AI in the physical world. [4:34] It processes the sensor data. [4:37] Nowadays, we talk about it using AI terminology as [4:41] encoders that take this data in. [4:44] Then there's the decoder, the action—the generative part, if you will—in the car. [4:50] The generative task there is to figure out how to drive. [4:54] That is, of course, connected through a specialized interface to the car [4:59] where we can actuate the vehicle. [5:01] That's why you see the steering wheel turn, and it drives you around. [5:06] Okay, so I get into my car, there are three main families of sensors, [5:09] LiDAR, radar, and cameras. [5:14] It is using that to first build a model of what's going on in the world, [5:17] where are all the other cars and things like that, then you say make decisions, [5:21] and then actuate that with the car. [5:23] That is the system that you're living in. [5:25] Is all that inference done locally or presumably, yes, nothing's in the cloud? [5:29] Nothing real-time? Nothing real-time in the cloud. [5:32] There are some things that can happen in the cloud, but they're not required. [5:36] Got it. [5:37] What's an example of a nice-to-have that happens in the cloud? [5:40] You can imagine a situation where we do… [5:43] Some of it is not directly related to the task of driving, [5:47] but let's say after you leave the car, we want to check that the car is not dirty. [5:53] You didn't leave anything there. [5:55] If you did leave an item, well, if you left it a mess, [6:00] then we want to send the car to one of our depots, get it cleaned up. [6:04] If you left an item there, maybe your phone, we want to detect that, [6:08] and then send it to our listening phone and let you know. [6:12] That we do by asking a model that actually lives [6:19] off-board as opposed to having to put it on the car because it's not a real-time [6:22] task related to the driving. [6:24] That's one example of something that— [6:26] There are all these debates that go on on Twitter around self-driving. [6:32] I can think of end-to the end versus the more modular approach. [6:38] There's cameras only versus array of sensors. [6:42] I can't tell, are these debates [6:46] actually interesting to an expert in the field, or do you think these are [6:49] just settled matters, and they're just grist for the algorithm? [6:53] I understand where the questions are coming from. [6:55] I do find that often the way they're posed and the way [7:01] the debate happens is losing a lot of the nuance and a lot of [7:09] detail that really matters. [7:11] To me, the most interesting technical questions are in that level. [7:15] Because the way we think about building the Waymo Driver, [7:22] it starts with a large off-board foundation model. [7:26] I can imagine building a big model that understands how [7:31] the physical world works and understands [7:35] the important properties of what it means [7:39] to drive, the social aspects of driving, and what it means to be a good [7:43] driver as opposed to a bad one. That's the foundation. [7:46] Then we specialize it into, let me [7:50] call it, three main off-board teachers. [7:53] There are still large, high-capacity off-board models. [7:56] There's the Waymo Driver, there is the simulator, [8:00] and then there's the critic. [8:03] Those then get distilled into smaller models that you can run [8:07] inference on faster. [8:09] The Waymo Driver becomes the backbone, the backbone of what's in the car. [8:14] The simulator, of course, is what powers our synthetic generative [8:18] environment that can run on the cloud for training and for evaluation [8:22] enclosed the system. [8:23] The critic is the value. [8:24] Does the simulator ever run locally? [8:27] No, it doesn't. [8:29] However, what I think is interesting, in a way, the way the decoder works, [8:33] the way the model works. [8:35] If you think about the generative task in the simulator of creating those [8:39] realistic worlds and how other people behave, how cars, pedestrians, cyclists, [8:46] and the task that you have to solve on the car in real-time, [8:51] there is this fundamental shared [8:55] capability of understanding how these [8:58] objects relate to each other and predicting what they might do [9:03] in the future if you are running on car and then generating, some sampling, those [9:07] probabilistic behaviors in the simulator. [9:09] It's different models, but this is why the shared foundation [9:12] model is able to power both. [9:16] Similarly, if you think about the critic, the job of the critic is to find [9:20] interesting events and then be opinionated about what's good [9:24] behavior and what's bad behavior. [9:27] Similar fundamental understanding. [9:29] If you're running inference on the car, you still have to figure out [9:32] which of the multiple hypotheses of these future worlds you want to take [9:37] action to steer towards. [9:39] Okay, and these are all downstream of the same foundation model? [9:43] That's right. [9:44] You start with the foundation model, then you specialize and fine-tune, [9:50] still off-board model. [9:51] Those are the teachers, and then you distill. [9:53] Each one of the teachers trains its own student, the driver, [9:58] the simulator, the critic. [9:59] You started working on self-driving 20 years ago. [10:02] As you think about the tech evolution, [10:06] is this just a scaling laws story where we [10:09] had to be able to throw enough compute at it? [10:11] Were there architectural approaches we needed to wait to have be invented? [10:16] Was it just a story of we needed 20 years of going down the wrong cul-de-sacs before [10:22] we eventually arrived at the right approach? [10:24] Knowing what you know now, could you have a successful Waymo [10:28] in market in 2015, or was there some enabling technology? [10:35] No. [10:36] Technology breakthroughs that happened over the years were critically important, [10:41] primarily in AI, but also in other areas. Compute. [10:46] They have a compute... [10:48] Now, I wouldn't characterize it as going a thousand different dead ends and then [10:53] having to retract and then finding the one right path, I would characterize it as [10:59] iterative learning and evolution, and then transformers came around, [11:02] but Transformers, for example, are very general architecture, [11:06] powers of LLMs, powers of our models. [11:09] But how you apply them to that space, I think this is where— [11:12] It didn't just fall out of Transformers. Exactly. [11:15] Then, of course, people like to talk about architectures, but architecture [11:20] is important, but really a lot of it comes down primarily to your metrics, [11:25] to your evaluation mechanisms, to all of the training recipes, and of course, data. [11:30] LLMs are good at text or tokens, specifically, and obviously perform [11:36] best at domains that have some single corpus of text they can work on, [11:41] like coding, where it's very helpful that everything was just textual already. [11:46] Part of the success has been creating textual representations for domains such [11:50] that we can then put a lens against them. [11:54] Can you describe how you encode the world that you're seeing? [12:02] Are you just building a 3D bit map, essentially? [12:07] This is where I think we get a bit into [12:12] this question of what is the interface [12:19] between the encoder and the decoder parts. [12:21] I think that touches also on the thing you flagged earlier where people like [12:25] to debate end-to-end or not end-to-end. [12:32] Let's talk a little bit about end-to-end and then get back to what is [12:36] the interface between those two. [12:38] When we say end-to-end, what do we mean? [12:41] We mean that it is some large ML model. [12:45] Typically, you don't build them monolithically. [12:48] You have different parts and different subgroups. [12:50] But what's important is that you can propagate/back prop the gradient and [12:55] the loss function all through the different layers. [12:58] Every layer, you can learn the weights and the representations [13:03] that matter for the final task. [13:04] You don't force it through some narrow funnel between, let's say, [13:08] the encoder and the decoder. [13:10] I think of a simple view of end-to-end being pixels go in, [13:13] and car actions come out, which may be a bit of an oversimplification. [13:17] That's exactly right. [13:20] This is the basic vanilla version of it. [13:24] If you think about what [13:29] will it take to build the driver that's capable of fully autonomous operations. [13:33] You think about this entire ecosystem of the driver, the simulator, the critic. [13:39] If that's all you do—pixels in, [13:42] trajectories out—it becomes very difficult [13:46] to do all of those three and achieve the high level of safety [13:51] and performance that we require, and it becomes very difficult to do it at scale. [13:56] However, it's a very easy way to get started. [13:59] You collect some data… Kind of like the LLM world. [14:05] The easiest thing you can do is pick a model. [14:11] The easiest way to get started nowadays would be just take a VLM. [14:16] It already has a language-aligned camera encoder, [14:20] and then it has a decoder that can predict, generate text, and [14:26] you can fine-tune it and say, "Instead of text, generate trajectories." Very doable. [14:31] In fact, a while ago, we published a paper called EMMA, that did exactly that. [14:36] It will actually, in the nominal case, drive pretty darn [14:43] well, which is mind-blowingly impressive. That is very funny. [14:47] There's something to it— [14:49] You're saying you can take an off-the-shelf model, [14:50] which has nothing to do with driving to start with, and you'll [14:54] get these good results. That's right. [14:55] In the normal case. [14:56] I just want to be clear. [14:57] It's orders of magnitude away from what you need to— [15:00] Yeah, you should not try it on the streets, but it works. [15:03] But for example, if you— It's like a talking horse. [15:04] It's impressive that it's talking. Exactly. [15:06] You can actually… [15:08] The product that you wanted to build was maybe a driver system, [15:13] not a fully autonomous system, then maybe that's all you need to do. [15:17] Then for that, you don't need all this other machinery of the simulator [15:20] and the critic because the number of nines is drastically lower. [15:23] This is interesting because there is some intuition behind why that works. [15:30] If you think about the hard parts of driving, it's not unlike [15:37] having a conversation, except if in the LLM world, [15:42] you're modeling a language, or may be modeling a dialogue [15:47] in the space of sentences and words. [15:50] What makes driving hard is also this multi-agent social interactive part of it. [15:57] If I do something that's going to affect you, it's going to affect somebody else, [16:01] and the history matters—it's not local and just geometric—context [16:05] matters, semantics matters. [16:07] But it's in a different… [16:08] It's not in the language of words, it's in the language of [16:11] body language, if you will. [16:15] We see that empirically validated if you do this approach. [16:19] Let's say we build this thing, just cameras, camera encoder, [16:22] pixels go in, trajectory go out. [16:24] The quality is sufficient to drive. [16:29] In the normal case, it's not sufficient to deal with the long [16:32] tail of the edge cases and hit the high bar of superhuman safety that we require. [16:38] Then you start asking the question, what else do you need? [16:43] If all you did was observing how other people [16:49] drive when you trained this system, maybe observing [16:53] just passively how people drive and how they interact, maybe also driving the car [16:58] yourself and then using imitative learning to train it. [17:02] You find that that's not enough. [17:03] You have to do something in closed loop. [17:06] You have to do things like RLFT, which is also parallel [17:11] to what we see also. RLFT? [17:14] RLFT. Reinforcement Learning-based Fine-Tuning. [17:17] Okay. [17:18] Similar to the Reinforcement Learning with Human Feedback in the LLM world, right? [17:24] You want to do maybe proper closed loop driving where you [17:31] explore all kinds of different situations, and then you give it a reward [17:34] signal to keep it in distribution. [17:37] For that, then you need a realistic simulator. [17:42] If you want to have a good RL system, you need to have an opinion for the reward [17:47] function, this is where the critic comes in. [17:48] If you have a purely end-to-end system, let's look at the simulator. [17:52] What do you do? [17:53] You're then constrained to just go from pixels to trajectory. [17:58] That's all you can run the system on, right? [18:03] It's a very high dimensional space. [18:05] It's a hard problem to generate everything. [18:08] But even if you solve that, it just becomes incredibly inefficient [18:15] to run it in the full way of pixels to trajectories and simulation [18:19] for training or for evaluation. [18:21] This is when intermediate representations come in. [18:24] There are some intermediate representations in the world in this task, [18:27] in the physical world, we know are correct. [18:30] They are not sufficient, but they're not generality limiting. [18:34] There's an object here, there's a concept of a road, [18:37] there're signs, there's speed limits. [18:38] This is where augmenting that learned representation, those learned embeddings [18:44] from the encoder decoder with that more structured representation is what we do. [18:51] We find that this gives us additional knobs to simulate in that space, [18:57] just pixels to trajectories. [19:01] It allows us to have additional safety validation layers in real-time. [19:05] It also gives us additional mechanisms to specify [19:09] the reward function for evaluation of the critic or for training. [19:14] This is, again, we've gone full circle of it. [19:17] Is it end-to-end? Yes, it is. [19:19] Then, if you want to do it at scale for full autonomy, it's augmented [19:23] with all of this other stuff. [19:25] That's very interesting on the simulating point. [19:27] It's just very hard to simulate for an end-to-end model because it's [19:30] easier to deal in intermediate representations rather [19:35] than coming up with a pixel-perfect view of the world. [19:37] You need both. [19:38] Having end-to-end architecture that's augmented with that structure allows [19:43] you to play in both of those worlds. Yeah, yeah, yeah. [19:47] What are you looking to do as a self-driving car? [19:51] It sounds funny, but I think people maybe don't realize that there are many [19:54] different things that you're looking to solve for, where you're looking to get [19:58] the person to their destination, you're looking to get them there [20:01] reasonably promptly, but also drive quite smoothly, [20:05] and also have many lines of safety, and also not annoy other drivers [20:09] and get honked at, and... [20:11] What are some of the reward functions or things you're optimizing for that maybe [20:16] are not obvious to people? [20:18] Safety is the primary focus. [20:22] But of course, we also want to be [20:25] a smooth driver for both people [20:29] in the car and other actors. [20:31] We also want it to be a predictable, well-behaved one so that it can nicely fit [20:37] into the whole social ecosystem of our roadways. [20:41] It seems like one of the issues that [20:47] has quickly emerged with self-driving [20:50] is the fact that people can't have nice things or not everyone [20:55] is nice to the robots. [20:57] Whether you're driving through a dodgy area or getting blocked, or [21:02] maybe I'm not going to drop you off here. [21:04] Maybe I'm going to go around the block and drop you somewhere better, but all [21:07] of these, as you say, other human issues. [21:10] How do you go about solving those? [21:12] A lot of the ones that you mentioned are just things that we [21:17] need to work on and understanding. [21:19] Honestly, as I said, that if we're not dropping you off exactly [21:23] where you wanted to be dropped off, or we don't give you a good [21:27] interface to tell us, that's on us. [21:30] We got to make it better. [21:32] It feels like the drop-off is actually a pretty nuanced part of the [21:36] self-driving journey. [21:37] The highway stuff and the 35-mile-an-hour roads, that is all nailed, [21:42] but there's just a lot of nuance in the drop-off experience. [21:46] I'd say they're all hard. [21:47] You picked freeways, and you picked drop-off for different reasons. [21:50] For drop-offs, you're absolutely right. [21:55] There are a few things that are maybe not obvious. [21:58] You just think about this problem, but it's understanding where you want [22:02] to go and making it as convenient as possible for you. [22:05] Pickups from drop, it's not exactly symmetric. [22:08] But then also understanding the context of the situation, where do you stop? [22:14] You don't want to block a driveway. [22:15] You don't want to double park, although in some cases where if [22:18] it's a quick one, maybe it's okay. [22:20] There's a lot of nuance that goes into doing that well so that it's smooth, [22:24] frictionless experience for the rider as well as other folks. [22:29] Freeways, for most of the time, not much happens. [22:36] They're very well-structured because we designed them that way. [22:41] But there is still that long tail of really complicated [22:49] stuff that happens where the consequences [22:55] of a bad event are much more severe. [22:58] The speed is much higher. [23:00] Everything is quadratic in speed. [23:02] But we see a lot of stuff. [23:03] Imagine grills falling off of freeways. [23:05] Imagine people getting into accidents and spinning out of control. [23:10] You see one of those flatbed trucks with just a bunch of stuff piled in it, [23:14] and you're driving behind it? I don't know. [23:15] I always find it very nerve-wracking. [23:18] I know. We've seen them leave a trail. [23:25] It's a different set of problems. [23:26] But I feel like the general sentiment with the Waymo is that the [23:33] driving has mostly now been solved by you guys, and it's a question of scaling up [23:38] and maybe some super long-tail stuff, really snowy condition. [23:41] Is that your sense internally, or is there actually much more nuance within that? [23:45] I would say it's not like we're done with engineering. [23:48] I would say that we've clearly moved past [23:53] the stage of scientific research and deep [23:57] core technology development to this new [24:00] phase of accelerated global [24:04] scaling and deployment. [24:06] We still have work to do, but I don't see today any limitations [24:12] or any gaps in the core technology. [24:14] The driving is good enough now. [24:16] Well, the core technology, I think, is good enough that I can't [24:21] think of any aspect of driving that is not supported by the fundamental technology. [24:28] Now, that said, there is a lot of work to do in specialization and in validation [24:35] before we can deploy responsibly. [24:38] We're not driving everywhere in the world. [24:40] We are planning to start operating in London and in Tokyo this year. [24:46] Do we have a driver that you're using today in San Francisco that we can [24:51] just plop down in London and go? No. [24:54] But what we're seeing is incredibly encouraging from the perspective [24:59] of is the core technology there? [25:01] Now it's a matter of collecting the data, doing some specialization and validation. [25:05] Science are different in both of those places. [25:08] People drive on the other side of the road, but that's actually [25:11] not that hard for computers. [25:12] The core technology generalizes really well, but there's still [25:16] work that you have to do. [25:18] What generalizes least well? [25:20] Increasingly, we're finding, especially now that we're able to hook the Waymo AI [25:26] to the AI in the digital world, the VLMs, and inherit the general world knowledge [25:32] from VLMs, we're seeing really strong results from zero-shot or few-shot [25:37] learning because of that general knowledge that we bring in. [25:40] But there are a few things like, say, cold weather, cold winter weather, [25:47] where it affects the entire stack. [25:49] It's not just the AI, we actually have to— The hardware. [25:52] You need the hardware. [25:54] You need to have the proper cleaning solution, heating elements in it, [25:57] and then you think about things that are completely solvable [26:02] by computers, like motion control and slippery surfaces. [26:05] That takes a bunch of work. [26:07] You don't get that for free from just pulling it, some VLM decoder. [26:12] Was it the case… [26:13] My impression, not knowing anything is [26:17] that in the early days, there was maybe a lot [26:19] of San Francisco-specific work or Phoenix-specific work in the early [26:24] markets, whether it be mapping or something else and that you guys seem [26:29] to either have solved that in generalizing it or just [26:35] scaled up your ability to do the city-specific work. [26:39] What enabled the rapid-city expansion? [26:43] We usually think about it, the capability of the Waymo Driver as well [26:47] as deployment, not primarily and directly [26:51] in that space of cities or zip codes. [26:54] I think about the operating domain. [26:57] Then the freeways, cold weather, [27:01] snow, rain, fog, density, et cetera. [27:04] That's what we are building. [27:07] That's where we're evaluating, and then that maps to a particular city, [27:11] be it within the operating domain or outside of it. [27:18] If we rewind history a little bit, our initial deployment in where we started [27:22] offering a fully autonomous commercial service for the first time was [27:27] in 2020 in Chandler, Arizona. [27:31] That was on what we called the fourth generation of the Waymo Driver. [27:34] This was, if you remember, the Pacifica minivans with different [27:38] hardware, different software. [27:39] There, we were super focused on doing the whole thing [27:46] end-to-end: learn how to build the driver, [27:51] evaluate it, deploy regularly, [27:54] operate it end-to-end 24/7 with customers, learn from the customers. [27:57] Then we were very focused on that operating domain of mostly [28:00] Chandler, which is a medium, low-complexity one. [28:03] Then, when we made the jump to the fifth generation of our system, [28:06] this is what's on the highways today, we really wanted to take a huge [28:11] bite out of that operating domain. [28:13] We collected data all over the United States, all [28:18] different states, different cities. [28:20] When we chose to deploy in the hardest parts of San Francisco, [28:24] hardest parts of Phoenix, we made a big jump on the hardware side, [28:28] and most importantly, on the software, the AI side. [28:31] I would say that was the big discontinuous jump. [28:36] That's what you're seeing now after we've scaled up and iterated all of the aspects [28:42] of building and deploying the driver. [28:44] This is now why you're seeing us go in parallel and scaling [28:50] in the US and globally. [28:51] So driver v5 was just a much more generalizable stack than v4? [28:57] What was it about it? [28:58] Was it just that it had been trained on a much wider data set? [29:04] It was when we made this big bet on AI. [29:07] There was a lot more little AI models and ML models in the fourth generation. [29:12] We made a much bigger bet and jump to AI is the backbone for the fifth generation. [29:19] AI is the backbone as the core engine, [29:24] as in you're saying that Gen 4 had lots of small little AI subsystems? [29:28] Yes. [29:31] We made that jump, and we've been iterating [29:34] and improving the model since then. [29:38] As we're seeing with Waymo rolling out widespread autonomy, it has second-order [29:42] changes on the entire system. [29:44] In this case, traffic patterns or other drivers' behavior, or eventually, [29:47] how cities are laid out. [29:49] Autonomous systems are coming in many domains. [29:52] In commerce, soon agents are going to be transacting without human intervention. [29:57] We're basically getting driverless commerce. [30:00] Stripe is building the economic infrastructure for AI. [30:03] As part of that, we're letting payments be initiated by humans or by agents. [30:08] If you want to sell to agents or if you want to let your agent spend money all [30:12] around the web, check out Stripe's Agentic Commerce Suite. [30:17] Can we talk about hardware a second? [30:19] Lots of hardware questions, but one is maybe, [30:24] everyone in this space [30:26] has a very charismatic demo of a vehicle [30:32] that is custom-made for self-driving. [30:36] It's often the van with no steering wheel, seats facing in both directions. [30:42] You guys have one. [30:44] Tesla has the steering wheel-less Cybercab. [30:48] Cruise had the Cruise Origin. [30:51] Yet, we're still driving in Jaguars that have [30:57] a steering wheel in the front and are pretty similar to consumer cars. [31:01] It's interesting to me because if we were talking about this 10 years ago, [31:06] we might say, "Well, yes, developing a custom car, [31:09] that's relatively straightforward. [31:10] We know how to put a bunch of sensors on a new car." But the software [31:14] will take a long time. [31:15] What's interesting is we've made huge progress in the software, [31:18] but interestingly, the cars are still derivatives [31:21] of cars that people are driving. [31:23] I'm curious why you just think the custom hardware has not happened as of 2026. [31:29] It's obviously a small improvement compared to Waymo is the big improvement, [31:33] but it's just interesting that it still hasn't happened. [31:36] Well, let's say our sixth generation of the vehicle and the driver [31:41] is our version of that. Oh, no, I know it is. [31:44] It is the Ojai platform. [31:45] That still has the… [31:47] We can talk about whether you want to have the seats pointed backwards or not. [31:51] I actually think it looks nice in a demo, but practically speaking, [31:55] it's maybe not the way to go. [31:56] But it is a custom-designed vehicle, and we put a lot of thought into [32:03] moving away from a car that's designed [32:07] around the driver to a car that's [32:10] designed around the passenger. [32:12] It's much more spacious. [32:14] It's happening. [32:15] It's not open to the public yet, but I took a ride in it the other day, [32:22] fully autonomously, and that's coming this year. [32:24] Yes. [32:24] How much better is it as a passenger experience? [32:26] You'll tell me once you give it a try. I love it. [32:29] It's all about the space and [32:35] the convenience of ingress and egress [32:39] and the screens and the interface of the passenger. [32:42] We put a lot of thought into every aspect of it. [32:44] It has sliding doors. It's very easy to get in. [32:47] It has a flat floor. [32:48] If you sit in the back, you can fully stretch out, [32:52] and there's so much space there. [32:54] From the outside, it looks fairly big. [32:58] But the actual footprint of that is barely larger than the I-PACE. [33:05] It's amazing that you walk in, and it feels like you're in a living room. [33:09] Yes. I guess my question is just, Waymo does [33:13] 25 million rides a year, run rate-ish, [33:18] with the Jaguar I-PACE. [33:20] It's interesting that so much scaling has happened with self-driving [33:24] so far on the old retrofit. [33:28] Maybe that's to be expected. [33:30] Well, it matches the high… I don't think it's a given. [33:37] You're right. [33:39] But if you think about the value proposition, of course, [33:43] there is the safety of it. You don't have to worry about it. [33:45] There's also the privacy, being in the car by yourself, [33:52] maybe with other folks, but not having to share that space with another human. [33:55] No, Waymo is a great product. [33:56] But I guess this is why we're seeing such consistency in car. [34:00] It drives well, very predictable. [34:05] You can go beyond that. [34:07] You can specialize even more to make the experience even more [34:10] magical around the rider. [34:12] But I guess it would have been disappointing without the specialized car, [34:17] and I think I would have been surprised if we leveled off at some other much lower [34:21] level of customer adoption because a car seems like more of an optimization [34:26] improvement, but the core of the value proposition comes [34:29] from those other factors. Yes. [34:30] I guess it just take risk on one thing at a time. [34:33] We'll start by doing the software layer, and then we'll build a specialized [34:37] car or something like that. That's right. [34:39] It's also, as you said, it's a big investment. [34:42] You have to de-risk the fundamentals. [34:44] Throughout our history, we were very [34:48] focused on setting the biggest goal [34:52] for the company to de-risk the most important questions. [34:56] We talked about the third generation where we wanted to deploy [35:00] something and go end-to-end. [35:01] We talked about what was the goal with the fourth generation, sorry, [35:03] the fifth generation, and then there's the sixth generation. [35:07] That was the sixth generation where it made sense to spend all this [35:10] effort into the custom vehicle. [35:13] The sixth generation is both the custom vehicle. [35:16] Is it also a new generation of the driving stack? [35:20] It is the new hardware. [35:24] The sensors, the hardware, the self-reliant hardware they're putting [35:28] on the Ojai vehicle is the sixth generation. [35:32] It is very different from the fifth generation. [35:34] It is simpler. [35:35] It is more capable. [35:37] It is much lower cost. [35:38] Just thinking of a fraction of the cost. [35:39] It's comparable to what you would get like a fancy ADAS system nowadays, [35:44] the driver-assist system. The software is pretty much the same. [35:49] When we talk about generalizability of the Waymo Driver, we talk about weather [35:53] conditions, we talk about cities, but it also generalizes well to different [35:58] vehicle platforms and different sensor configurations. [36:01] Okay, so Gen 6 is a new vehicle and a new sensor stack, but it's almost [36:06] a TikTok cycle happening here. It's a similar software. [36:09] That's right. [36:09] Then we're going to put the sixth generation Waymo Driver [36:14] on other vehicle platforms like the Hyundai Ionic that's coming [36:19] later in the year. [36:21] What is different about the sixth generation hardware stack, [36:24] and how did you make it cheaper? [36:26] It still has the same three sensing modalities, [36:29] but we've made significant optimizations in all three. [36:36] Unification, simplification, and there's just the… [36:41] Just writing the— [36:42] Yes, is it a classic case of manufacturing scale where we're not even— [36:46] Well, scale hasn't fully come in place. [36:47] But all of those, if you think about the supply chains, the industries, [36:53] cameras is pretty mature. [36:56] Radars, way many years ago, used to be bulky, complex, very expensive when [37:03] we were putting them on planes. [37:05] But then we started putting them on cars, now you can get a decent automotive [37:09] radar for tens of dollars. [37:12] There is a variant of the automotive radar. [37:17] It's called the imaging radar. [37:18] It gives you a richer… [37:20] That also has come down in cost drastically, but it's a little bit [37:24] behind your standard automotive radars. [37:26] LiDARs are following the same very predictable, very well-known trend. [37:31] We're writing that, and we're also learning from the previous generation [37:35] to just make improvements and simplifications and optimizations. [37:38] It's a very silly question. [37:39] What are LiDARs versus radars better at in a self-driving context? [37:43] LiDARs—Are they complementary? [37:46] They're very complementary. [37:49] It's all blasting. [37:55] Echolocation. [37:56] Effectively, like blasting photos out there, and then [38:01] they bounce off of something, they come back, you measure what comes back. [38:05] The frequencies are very different. [38:09] The laser gives you… [38:10] Its very high resolution. [38:12] You can think of it as like a laser beam that goes out, spins around. [38:16] It shoots out millions of these laser pulses per second, and then each one comes [38:22] back, and you're sampling the 3D structure of the world with very high resolution. [38:27] LiDAR for very fine-grained mapping. That's right. [38:29] Radar has much lower resolution, but because of the physics of it, it [38:36] degrades much better in adverse weather conditions. [38:41] So fog, snow, heavy rain. [38:44] It could be included by particles between it and the target. [38:49] Imagine driving in super dense fog. Yes. [38:53] We're close to San Francisco, so we probably don't have the think that hard. [38:58] It can be really hard to see. [39:00] So cameras degrade. [39:02] Laser, depending on the size of the particulates, can degrade [39:06] better or worse than camera. [39:07] Radar is not well-affected. [39:09] So you can imagine driving on a freeway, then radar will give you really good [39:13] returns for cars that are absolutely invisible in the camera space. [39:18] That's interesting. [39:19] Does that mean there are some environments where you'll be relying [39:23] significantly more on radar? [39:25] Where the performance is good enough? [39:27] Well, it's a combination of the sensors. [39:29] We rely on… [39:32] Each one is noisy. [39:36] How the noise characteristic show up in different environments is different, [39:42] but it's not like we switch from one to another. [39:44] It's not like we estimate what's happening with the world through cameras and through [39:48] radars and through LiDAR, and then we compare. [39:50] No, they're like… [39:50] There's an encoder for camera, there's an encoder for LiDAR. [39:52] There's an encoder… [39:53] They all go into the system that gives you, jointly, the best view [39:59] of what's happening in the world around. [40:02] If it's a nice, bright, sunny day, cameras are very valuable. [40:07] If it's pitch dark, or you have sun in your face, [40:09] or you're blinded by the headlights from an oncoming car, [40:12] then the camera will degrade. [40:14] There's still some noisy signal, but it will degrade, and LiDAR [40:18] is completely unaffected. [40:21] Are there technical problems that are your white whale, or you're still chasing, [40:26] or you are particularly interested in solving? [40:28] Even if they're niche for the… [40:29] We really want to have driving when it's actually snowing [40:35] nailed or steep hills in San Francisco. [40:38] Are there problems you've been very interested in historically or still are? [40:43] I'm super excited right now about the accelerating global expansion, [40:49] more cities in the United States and going internationally. [40:53] I understand I'm not answering your question [40:57] about the technology, I'll come back to that. [40:59] But really, that's the thing that I'm today most excited about. [41:02] Just getting to a place where any major [41:09] metropolitan area, [41:10] you can fly into the airport, and then take a Waymo and go [41:13] anywhere you want to go. That is insanely exciting to me right now. [41:18] Then technically, [41:21] what I'm most excited about is all of the [41:27] rapid progress in AI, and the world [41:33] models, the foundational model work. [41:36] It is just such a massive boost to how much we can simplify [41:42] the system, how much we can bring down the cost, and how we can scale globally. [41:46] There's just some magic that happens that I don't think I would have [41:51] anticipated a few years ago. [41:53] That I find from the technical perspective, just insanely thrilling. [41:58] When you talk about the progress in AI, what are the most fun parts [42:02] of it for you these days? I think it's seeing [42:07] the capability and the scaling laws [42:12] from this approach of starting with that cornerstone [42:16] of the foundational model, and then specializing to teachers [42:20] and then distilling. [42:21] You get such big wins in performance across the board. [42:28] You invest something [42:32] into the architecture or get a better data or training recipe, and then [42:37] you invested that early stage, and then it just has massive [42:40] amplification and ripple effects. [42:42] That, in some ways, is kind of magical. [42:46] Then you see it on the car. [42:49] I've had some moments where a car [42:54] does something, and you look at a log, [42:57] and I've been surprised. [43:00] It does things that I didn't think it was capable of doing. [43:05] It's that… [43:08] When you see emergent behavior, that's a proud moment? [43:12] One example, yeah. [43:13] When you build a system, and then you think you understand [43:17] how it works, and you understand fully the limits of its capability [43:22] and performance, and then it does something [43:25] almost magical, it's exhilarating. [43:28] One example I can give you, I think I've shared some videos of that [43:33] publicly in some talks, was this example where the situation [43:38] that happened in San Francisco. [43:41] A fairly [43:43] benign situation where at an intersection, our light is red, [43:47] there's near cross traffic, a bus goes by, and it stops partially blocking the lane. [43:54] Our light turns green, so we start to go. [43:58] We're nudging around the bus, and then you see a pedestrian being detected [44:03] on the other side of the bus. [44:05] Then your car responds appropriately, it slows down, goes a little bit wider, [44:09] and then a pedestrian actually emerges from the bus, and we go on our own. [44:15] The first time I looked at that log, I was like, "What's going on here?" I know [44:20] we have pretty darn good sensors, and the software is very capable, [44:25] but we don't see through stuff. [44:28] That's not how cameras or LiDARs and radars work. [44:31] It saw the pedestrian through the bus? [44:33] It saw the pedestrian on the other side of the bus. [44:35] It's not like you look at the windows, you're like, "Okay, radars shouldn't… [44:38] This massive metal box." [44:41] Look at the sensor data, and radar [44:47] shouldn't be able to go through it. [44:50] You can't see in the camera because there're reflections [44:53] and there's people on the bus. [44:54] It's not like you can see through the windows. [44:56] What is going on? [44:57] Maybe it's noise or some coincidence. [44:59] The first time I saw it, I couldn't actually believe it. [45:02] I was like, "No, there's something. [45:03] It doesn't smell right." What actually turned out was happening is that our [45:08] peripheral LiDARs bounced under the bus, and there was just a little bit of very [45:12] noisy reflection of the movement of the person's feet that was [45:16] enough for the AI models. [45:18] "Hey, likely there's a pedestrian there, and I'm going to… [45:25] I detect it as such, and moreover, there's enough data there to predict what [45:29] they're going to do." It just blew my mind. [45:33] Is this the perfect example to explain what we were talking about earlier, [45:38] the value of one, fusion across a sensor suite, but then secondly, building, [45:44] I mean, relatedly, building an intermediate representation of what's [45:49] going on, where if you're just dealing with pixels, I mean, [45:55] the person behind the bus does not exist in pixel space, and so you need to have [45:59] some representation of the world that exists to be able to reason [46:02] about the person behind the bus. [46:04] I think it's an example where using [46:11] that intermediate representation to boost [46:13] the level of performance of all parts of the model is what's happening here. [46:20] Just imagine solving this problem with a black box, [46:25] purely open loop imitative system. It could be— [46:29] Hard to impossible. [46:30] Is it impossible, no, but in practice, what would it take [46:34] to achieve that level of performance? [46:35] It's very, very difficult. [46:37] What metrics can you share on just where the business is at today in terms of [46:42] rides, revenues, cars on the roads? [46:46] We have about 3,000 cars on the roads. [46:51] We're doing about half a million rides per week. [46:56] That translates to about over 4 million fully autonomous miles per week. [47:03] We are operating in a fully autonomous [47:06] mode in 11 cities in the US, and 10 [47:11] of those, we have riders, public riders. [47:15] What's the ghost city? The ghost city? [47:17] The ghost city is Nashville. We just started there. [47:19] We just opened it up to riders [47:25] in four new cities in one day. [47:28] That was one of those little but super exciting moments where I thought [47:34] back to the history, like how long did it take us [47:37] from the first time we started fully autonomous, [47:40] rider-only operation to the first time we had external riders in four cities. [47:45] It was about eight years. [47:46] Then the other week, we just launched four in one day. [47:50] Yes. It seems now clear that in 15 years, most [47:55] miles that are driven will be autonomous. [48:00] There will be some burn in period, and it's lots of old cars in the road. [48:03] I think it'll actually take a little while. [48:06] Some of that will be by level four, level five systems expanding in new [48:12] cities and that expansion continuing. [48:15] Some of it will be, you referenced the existing driver assist [48:18] systems and getting up to level two and level three, and existing [48:23] systems across current car brands getting more and more capable. [48:28] What do you think that working your way up from the lower levels versus working your [48:33] way expanding from existing products like Waymo? [48:37] What will that convergence look like? [48:39] Because we're going to eat it from both sides. [48:43] I don't believe we will. [48:44] I actually think this— That's a great answer. [48:50] Cars will get smarter. [48:51] There's going to be advances in driver-assistance systems, and if [48:56] there is at the same time from level four autonomy, there is simplification, and [49:04] the sensors of today are not going to be the sensors of tomorrow, [49:07] so they'll be much more integrated, they'll be simpler, [49:09] there'll be much lower cost. [49:10] From that perspective, there is a path of convergence. [49:14] There's also a path of convergence from the product lines. [49:18] There's ride hailing, and you can take a ride through the Waymo app today. [49:24] Eventually, that'll be on your personal car, so that I see. [49:27] You can talk about the technology, and I see it just as fundamentally [49:31] two different problems. [49:33] There's driver-assist systems, and then there is full autonomy. [49:38] I think it's deceptive to think of them as [49:40] incremental on one spectrum of complexity. [49:46] Okay, but you think one cannot work one's way up [49:50] from driver-assist systems to full self-driving? [49:53] You think you have to start building a full self-driving system? [49:56] I think you have to tackle… [49:58] If I think about the hardest parts [50:03] of building a fully autonomous, [50:06] rider-only system, they are very different from what you do for a [50:12] driver-assist system. [50:13] Of course, some work in this space helps you. [50:16] I don't want to say you can't make the jump, but it is a qualitative jump. [50:21] When can I buy a Waymo so that I don't need to wait for it when I want to go? [50:25] When I'm ready, I can walk out the door and it's there. [50:28] I'm not going to give it a date today, but you're not the first person [50:31] to bring this up as a product request. Duly noted. [50:37] I'll add it to the list. [50:38] Just that waiting for the car, it should be nice just in the garage there [50:42] and keep your stuff in it and everything. [50:43] It's not the first time you've heard that request. [50:49] It seems to me operationally very intensive and very hard. [50:55] A self-driving car is actually not self-driving, it takes a village. [50:59] You have all of the human operator ready to step in. [51:04] There was that thundering herd incident that you guys talked about in [51:09] San Francisco that highlighted that for people. [51:12] Then there's just keeping the cars clean and keeping everything [51:16] running in that regard. [51:17] Can you describe just what the operational infrastructure that sits [51:21] behind Waymo looks like? [51:22] Sure. [51:24] I will say that we are overall in all of those areas on a path of [51:34] increasing efficiency and automation. [51:38] The number of manual steps that one had to do [51:43] five years ago to [51:46] launch a Waymo, versus where we are today, [51:51] is drastically different. [51:53] But nowadays, if you look at one of our depots [51:58] as a fully, automatically orchestrated dance of autonomous vehicles. [52:04] The way it looks like today is [52:11] cars will automatically go to pick [52:17] up their riders, serve their trips. [52:19] If for some reason they need to come back, maybe they're low on energy, [52:27] maybe somebody left a mess in the car, they will automatically come to the depot. [52:31] If it is, so cleaning today is a manual process. [52:35] It'll get flagged, and the car. [52:36] We have fleet management systems say, "Hey, car number [52:42] 378 needs cleaning." Actually, on the sensor dome, [52:46] we're able to display icons. [52:48] We'll show you like a little emoji. [52:51] They'll put their hand up, yeah. [52:52] There's people whose job it is to clean the car. [52:54] They'll come and clean it up. [52:56] If that cleaning is not required, and it's just charging, [52:59] we'll also automatically pull into a charging stall, [53:03] and we'll say, "Hey, I need charging." We don't yet have automated charging. [53:06] In the future, you can imagine that being fully automated, but a person will come [53:10] in and plug in a cable and the car will charge, and they'll say, "Hey, [53:13] now I'm ready to go." It will get unplugged and the car will [53:16] pull out of its parking stall and then go on its merry way. [53:21] One of the new Porsches, I think it is, has inductive charging, [53:25] just like your iPhone, where you just drive over the charging mat. [53:28] I was amazed that works at car scale, but yeah, possibly in the future, [53:31] they'll just be able to drive onto the charging mat, or do you think [53:34] just robotic plug-in will be easier? [53:36] We'll see. I don't know. [53:38] I think there're some questions about efficiency and how that plays [53:42] into the overall cost and which one will be most cost-beneficial. [53:46] It remains to be seen, I think. [53:48] How well-behaved are the Waymo riding population in terms of not [53:51] leaving a mess in the car? [53:56] We have wonderful riders. [53:57] We have the most amazing customers in the world. [54:02] Generally, I would say they are very good. [54:07] I think there is something about… [54:11] I talked about not having a person in the car, it's not somebody else's car. [54:15] In some ways, you want to preserve the… [54:18] I think generally people want to preserve the nice aspects of it. [54:20] It's a broken window thing where it's so clean to begin with. [54:23] I know. [54:24] I think that's the general trend that we see. [54:28] Because it's not somebody else's space, you're in it, it feels like it's your own. [54:33] You don't want to mess up your own space. [54:36] I don't want to speculate too much on the psychology thing. [54:41] However, I will say that it varies. [54:45] You can imagine a college town on a Saturday night, [54:48] and that's a different distribution. Yes. [54:50] Will I be able to get a Waymo at any address [54:56] that has USPS service in the US, or will there be some head/tail dynamic [55:01] where Ketchikan, Alaska is just never worth it? [55:06] Eventually, it will, absolutely. [55:07] There's no doubt in my mind. [55:09] I think it's just a matter of when and what modality would make [55:15] the most commercial sense. [55:17] This is for your ride-share versus privately owned. [55:19] For a ride, it's not a technical problem. [55:21] I mean, technology is solved. [55:22] But then if you're in the middle of nowhere and there's just not enough [55:26] density of the trips, does it make sense for the ride-hailing [55:28] service that Waymo is running to have cars on standby? [55:32] Probably not. [55:33] They can be deployed somewhere else, and you probably don't want [55:36] a horribly bad ETA. [55:38] This is where a personally-owned vehicle that is equipped with the Waymo Driver is [55:45] maybe how you will see it materialized. [55:47] Relatedly, what will the second-order effects of, say, majority [55:50] autonomous traffic be? [55:52] It feels like a lot of things will work better where, as you say, [55:54] when someone merges into a lane very poorly and everyone all the way back has [55:59] to slam on the brakes, that's an antisocial behavior. [56:02] It feels like higher quality and more prosocial driving will just [56:07] basically reduce traffic a little bit, even for the same number [56:10] of cars on the road. [56:11] But presumably, there'll be other second-order effects. [56:13] We'll want higher throughput traffic lights and, yeah. [56:17] How else will things change? [56:20] The first thing I think that you mentioned [56:23] is that's a huge deal. [56:27] I just need to think about traffic jams. [56:31] What's that saying with the Navy SEALs? [56:34] "Slow is smooth and smooth is fast." That's what you're like, [56:37] traffic jams are like, you accelerate abruptly, [56:41] then you come to a stop, and sometimes you have a traffic jam, like what happened? [56:44] Well, an old lady crossed the road three hours [56:47] ago, and we still have the standing wave there. [56:50] If everybody was a smooth, predictable driver and a consistent [56:55] driver, and you would still have those traffic jams at the time of, [57:00] but then the time constant should clean it out, I think, would be very different. [57:05] But longer term, things like parking lots. [57:10] Right now, if you look at what is our most [57:12] interesting pieces of land allocated to, [57:18] it's parking lots, it's garages. Why is that? [57:21] Well, because, again, your car is just sitting there 90% of the time. [57:26] If more cars become fully autonomous, then there's no need of that. [57:31] Then imagine, just imagine what you can do with your favorite city in the world if [57:37] you don't have to spend that money, that huge fraction of it on just keeping [57:42] these chunks of metal sitting around. [57:44] I don't think people often realize how big a deal parking minimums are [57:48] for the layout of the urban landscape. [57:50] The coffee shop near where I am would like to have outdoor seating but can't [57:53] because it would reclaim parking spots. [57:55] Yeah, wouldn't it be wonderful? [57:56] I have a few more questions, but I'm curious to talk about [58:00] Google's relationship with… [58:04] Self-driving where, again, it feels like right now, [58:11] Waymo is, aside from everything else AI-related, the most exciting thing [58:16] happening at Google, but it was a very long journey to get here. [58:22] I feel like you could say that Google almost started working on it [58:27] too early because you were saying there's been a bunch of recent enabling [58:30] technologies, and so did it require Google starting when it did so early, [58:34] or could one have spun up this project in 2015, 2020? [58:40] Then, how did Google keep the faith [58:45] when it almost felt like it [58:47] was perennially two years away? [58:49] Yeah, on the latter part, I just have to give credit, [58:54] huge kudos and gratitude to Larry and Sergey and Alphabet's [58:59] leadership in our company. [59:03] It is part of the culture and the DNA of the company is [59:08] to have that vision and have the stamina [59:14] and conviction to go the distance. [59:20] To the other part of the question, [59:27] was it too early? [59:29] I don't know. [59:31] I think what we've been seeing, clearly, all of the breakthroughs [59:36] that we've seen over the years have changed how we're building the system. [59:42] But the complexity of the problem is such that you need to go through [59:45] these iterative cycles. [59:48] We've seen many waves of technology. [59:51] There were breakthroughs in 2013, ImageNet came around. [59:56] There was this narrative, "Okay, that is the right time to start or be [59:59] a self-driving company." Then transformers came around and VLMs. [60:04] All of those are super powerful. [60:07] You have applications in other spaces. [60:11] In the digital world, they certainly have an impact on our AI in the physical world. [60:17] But there are no silver bullets. [60:18] They drastically reshape that early part of the curve. [60:21] It's always been the nature of this problem. [60:24] It's very easy to get started. [60:26] It's deceptively easy to get started. [60:28] But it is super hard to go the full distance. [60:32] Edge case... [60:34] It's the number of nines. [60:36] There's the standard engineering rule of [60:40] thumb that every next nine takes 10x more. [60:46] Yeah, maybe there is a more optimal path, but I don't see that there's some magical [60:51] moment where the true complexity of the problem goes away, [60:54] and then you can just take some off-the-shelf components [60:56] under your business. [60:57] If that were the case, then I think the industry would look very different today. [61:01] Last question I have. [61:04] You've been promoted a lot at Google. [61:06] It feels like Google really recognized your talents. [61:08] Just what do you think Google does? [61:10] Google is famously one of the very best in the world at technical talent and say, [61:15] the current AI wave more broadly happening, is either stuff happening [61:21] at Google or generally Google alumni. [61:25] But just what have you observed firsthand from how Google does this so well? [61:32] I would say [61:35] that culture of Google of not accepting [61:41] the status quo, having a big vision, and [61:50] investing in technical talent—the people who can go the distance and realize [61:54] the vision, that is part of the culture. [61:56] I think this is what you're seeing with the breakthroughs in AI [62:02] in the digital world, and like all of the early investments [62:04] in transformers and other fundamental [62:08] technologies, quantum computing. [62:12] I guess we are not unlike those efforts as well. [62:15] Dmitri, thank you. Yeah, thank you.