Nvidia, Uber Partner to Deploy Level 4 Robotaxis in 28 Cities by 2028

NVIDIA GTC Automotive Special Address — NVIDIA

NVIDIA

Nvidia argues that deploying commercial Level 4 autonomous fleets requires pairing end-to-end reasoning models with deterministic classical safety guardrails on standardized vehicle hardware.

AI-assisted summary of NVIDIA GTC Automotive Special Address — original source. Check the original for full context.

Source published:

Key ideas

  • NVIDIA announced a major commercial partnership with Uber to deploy Level 4 autonomous vehicle services across 28 global cities by 2028, supported by an Uber data collection fleet utilizing Hyperion 10 sensor suites. (39:07)
  • Four of the world's top ten automotive manufacturers—BYD, Geely, Nissan, and Hyundai—have officially adopted the NVIDIA DRIVE Hyperion compute and sensor reference architecture. (39:07)
  • NVIDIA unveiled Albatross 1.5, a 10-billion-parameter reasoning model featuring native routing, multimodal natural language prompt understanding, and flexible camera layouts, which ranks first on the public LingoQA benchmark. (18:55)
  • To accelerate industry-wide validation, NVIDIA is open-sourcing 7,000 hours of multi-country driving fleet data alongside NeuRecon neural reconstruction tooling, which powers 2 million daily closed-loop simulation runs internally. (22:40)
  • DRIVE Hyperion 10 integrates dual DRIVE AGX Thor SoCs built on Blackwell silicon, delivering an estimated 20x performance improvement over DRIVE Orin when executing in FP4 precision. (12:31)

Detailed notes

## Background Published on YouTube by NVIDIA on March 25, 2026, this GTC Automotive Special Address explores how autonomous vehicles are becoming the first mass-deployed physical AI agents. According to the page description, the session details the path to Level 4 autonomy through NVIDIA's DRIVE Hyperion architecture, its cloud-to-car development and simulation platform, agentic AI innovations, and partnerships across OEMs and robotaxi operators.

## TL;DR Framing 2025 as the pivotal inflection point for autonomous driving driven by reasoning and foundation models, the address presents NVIDIA's three-computer, five-layer automotive stack spanning training, simulation, and real-time vehicle compute. The presentation highlights DRIVE Hyperion 10 powered by dual DRIVE AGX Thor SoCs, the ASIL D-certified HALOS operating system and safety guardrail layer, and open-source foundation models including Albatross 1.5 and the NeuRecon simulation pipeline. It also charts the global deployment of DriveAV's hybrid end-to-end and classical architecture with Mercedes-Benz, an L4 robotaxi partnership with Uber across 28 cities by 2028, and expanding ecosystem adoption across BYD, Geely, Nissan, and Hyundai.

## Atmosphere & style The presentation carries the polished, forward-looking momentum characteristic of an executive NVIDIA GTC address, balancing high-stakes industry ambition with rigorous engineering specifics. The speaker delivers the material with an accessible, grounded confidence, interspersing macro-economic transportation metrics and technical software diagrams with lighthearted candidness about developer adoption. Watching the session feels like attending a comprehensive technical briefing that methodically connects frontier AI research to road-tested, safety-critical automotive hardware.

## Bulletin - [39:07](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2347s) NVIDIA announced a major commercial partnership with Uber to deploy Level 4 autonomous vehicle services across 28 global cities by 2028, supported by an Uber data collection fleet utilizing Hyperion 10 sensor suites. - [39:07](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2347s) Four of the world's top ten automotive manufacturers—BYD, Geely, Nissan, and Hyundai—have officially adopted the NVIDIA DRIVE Hyperion compute and sensor reference architecture. - [18:55](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1135s) NVIDIA unveiled Albatross 1.5, a 10-billion-parameter reasoning model featuring native routing, multimodal natural language prompt understanding, and flexible camera layouts, which ranks first on the public LingoQA benchmark. - [22:40](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1360s) To accelerate industry-wide validation, NVIDIA is open-sourcing 7,000 hours of multi-country driving fleet data alongside NeuRecon neural reconstruction tooling, which powers 2 million daily closed-loop simulation runs internally. - [12:31](https://www.youtube.com/watch?v=7N38fD4ksnI&t=751s) DRIVE Hyperion 10 integrates dual DRIVE AGX Thor SoCs built on Blackwell silicon, delivering an estimated 20x performance improvement over DRIVE Orin when executing in FP4 precision.

## Key takeaways - [07:28](https://www.youtube.com/watch?v=7N38fD4ksnI&t=448s) Autonomous vehicles represent the primary vanguard of Physical AI, serving as the first mass-deployed embodied robotic systems tackling the 13-trillion-mile global transportation market where autonomy currently accounts for only 0.006% of travel. - [09:57](https://www.youtube.com/watch?v=7N38fD4ksnI&t=597s) NVIDIA's automotive development model is built around a 'three-computer' paradigm spanning cloud foundation model training, cloud simulation for validation, and in-vehicle real-time inference. - [09:57](https://www.youtube.com/watch?v=7N38fD4ksnI&t=597s) The platform is organized across a five-layer stack encompassing DRIVE Hyperion hardware, the HALOS safety operating system, foundation reasoning models, the DriveAV application stack, and cloud development infrastructure. - [12:31](https://www.youtube.com/watch?v=7N38fD4ksnI&t=751s) DRIVE Hyperion 10 offers standard Base and High sensor reference designs, with the High package providing full hardware fail-operational redundancy via 14 exterior cameras, 9 imaging radars, and 1 forward LiDAR. - [16:11](https://www.youtube.com/watch?v=7N38fD4ksnI&t=971s) The HALOS operating system delivers ISO 26262 ASIL D safety certification while absorbing hardware and sensor abstraction layers directly into NVIDIA's stack, freeing OEM engineering teams from writing custom middleware. - [17:49](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1069s) HALOS incorporates certified active safety guardrails directly into the operating system runtime, pairing cutting-edge end-to-end models with classical deterministic software stacks compliant with 5-star Euro NCAP standards. - [18:55](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1135s) Albatross 1.5 was fine-tuned on 80,000 hours of curated autonomous driving data built upon the web-scale Cosmos foundation model, which was pre-trained on 20 million hours of real-world video. - [22:40](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1360s) NVIDIA is open-sourcing generative AI simulation tools NeuRecon, Harvester, Fixer, and Cosmos Transfer to extract real-world road obstacles, alter vehicle perspectives, and restyle environmental weather and lighting via text prompts. - [29:15](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1755s) Following a five-year joint engineering initiative, NVIDIA's DriveAV software stack has deployed worldwide across all 2025 Mercedes-Benz vehicles outside China, targeting full address-to-address L2++ availability throughout 2026. - [29:15](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1755s) NVIDIA's production DriveAV architecture utilizes a hybrid approach where end-to-end deep learning networks run concurrently with classical deterministic stacks on a single DRIVE Orin SoC. - [33:19](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1999s) DriveAV maintains high development velocity by logging approximately 10 model iterations per day (over 3,500 in one year) and compressing field data turnaround to closed-loop testing down to under six hours. - [35:57](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2157s) A real-time safety arbitrator continuously monitors end-to-end neural trajectories and instantaneously hands control over to classical path planning whenever safety margins are breached. - [37:19](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2239s) The software architecture creates a unified bridge from L2++ to L4 autonomy, using a shared codebase split between a primary Main ECU and a safety-redundant Satellite ECU executing automated Minimum Risk Maneuvers. - [37:19](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2239s) NVIDIA's commercial deployment roadmap schedules global L2++ scaling within the year, commercial Level 4 pilot fleets next year, and full production L4 passenger vehicles by 2028.

## Detailed walkthrough ### [00:09](https://www.youtube.com/watch?v=7N38fD4ksnI&t=9s) The ChatGPT Moment for Autonomous Driving

The session opens with an energetic welcome to GTC, reflecting on what a transformative period it has been for autonomous vehicle developers. The speaker frames 2025 as a definitive turning point for the industry, driven primarily by the lightning-fast emergence of cutting-edge AI architectures—specifically Vision-Language-Action (VLA) models, world foundation models, and reasoning models. These breakthroughs have fundamentally disrupted legacy autonomous vehicle development pipelines. Rather than causing friction or setbacks, these changes have dramatically accelerated the industry’s trajectory toward full autonomy.

Looking back across NVIDIA's journey over the past decade, the speaker recalls that ten years ago, NVIDIA introduced its first ADAS chip, Parker, which helped power the very first generation of Autopilot. Now, at another pivotal juncture, the company and its engineering teams are partnering across the broader mobility ecosystem to establish a safe, scalable route to full Level 4 autonomy. Reflecting on this turning point, the speaker emphasizes: “And at this moment, I would say that the path to full L4 level autonomy has never been this clear.”

- Technological drivers of this inflection point: Vision-Language-Action (VLA) models, world foundation models, and reasoning models. - Historical milestone: NVIDIA Parker SoC launched a decade prior to enable initial consumer autopilot systems. - Core industry objective: Unlocking scalable, safety-certified Level 4 autonomous driving across passenger vehicles, robotaxis, and delivery platforms.

---

[03:16](https://www.youtube.com/watch?v=7N38fD4ksnI&t=196s) In-Car Reasoning Demonstration with Alpamayo and Mercedes-Benz

To showcase reasoning capabilities in real-world scenarios, the presentation features a drive through San Francisco and across the North Bay in a prototyping vehicle built in partnership with Mercedes-Benz. The vehicle runs NVIDIA's Alpamayo reasoning model, which continuously perceives and evaluates the environment in real time while engaging in natural spoken dialogue with the passengers. When a passenger asks why the Transamerica Pyramid features its distinctive shape, the in-car assistant responds promptly with architectural context, explaining that William Pereira chose the tapered profile so that sunlight could reach street level rather than casting an overpowering shadow across the city blocks.

Beyond conversational infotainment, the test vehicle is prompted to narrate its internal driving thoughts aloud, demonstrating how the model evaluates traffic dynamics before executing maneuvers. As it travels, the vehicle articulates its reasoning in plain English: identifying an oncoming traffic stream when waiting for a left turn gap, planning a path around a double-parked delivery truck, and yielding smoothly to pedestrians at crosswalks or crossing vehicles. When prompted to make casual route adjustments—such as speeding up, changing lanes, taking an exit toward Sausalito for coffee, or pulling over—the system plans safe, compliant maneuvers. Highlighting how this architecture handles unpredictable environments, the narrator notes: “What sets Alpamayo apart is how it reasons through complexity, breaking down edge cases before they become problems.”

- Vehicle platform: Prototyping vehicle developed in close partnership with Mercedes-Benz. - Reasoning model: NVIDIA Alpamayo, operating continuously in real time to handle route planning and conversational queries. - Situational behaviors handled: Lane changes, unprotected left turns, double-parked obstacle navigation, pedestrian crosswalk yielding, and safe drop-off curb detection. - Commercial roadmap: In-vehicle consumer trials and customer experiences targeting late 2025.

---

[07:28](https://www.youtube.com/watch?v=7N38fD4ksnI&t=448s) Physical AI and the 13-Trillion-Mile Opportunity

Tracing the broader progression of artificial intelligence, the speaker references Jensen Huang’s keynote themes outlining the industry's progression from perception AI (sparked by AlexNet in 2012) to generative AI (unlocked by the Transformer in 2017), onward to modern reasoning and agentic models. The next frontier is grounding these agents within physical systems: physical AI. Autonomous vehicles represent the vanguard of this movement as rolling, road-going robots that will serve as the first mass-produced and mass-deployed manifestation of embodied intelligence. Framing the stakes of this shift, the speaker states: “And we believe that building and safely deploying physical AI is going to be the defining challenge of the coming decade and since automotives are just basically driving robots on the road, I'm pretty sure AV will be the first mass-produced and mass-deployed physical AI technology and applications.”

The commercial imperative behind this engineering effort is underscored by staggering global transportation figures. Every year, approximately 13 trillion miles are traveled globally across passenger cars, commercial transport, and delivery vehicles. At present, however, a mere 0.006% of those miles are driven autonomously. Bridging that vast disparity presents an unprecedented economic and technological frontier for mobility providers, OEMs, and NVIDIA alike, supporting the overarching vision that eventually every moving machine will operate autonomously.

```mermaid flowchart LR accTitle: Evolution of AI into Physical Embodiment accDescr: Progression from AlexNet perception AI in 2012 to Transformers in 2017, reasoning agents, and Physical AI in autonomous vehicles.

subgraph Perception["2012: Perception AI"] A["AlexNet<br/>Computer Vision"] end

subgraph Generative["2017: Generative AI"] B["Transformer Architecture<br/>Foundation Models"] end

subgraph Agentic["Present: Agentic AI"] C["Reasoning Models<br/>VLA & World Models"] end

subgraph Physical["Coming Decade: Physical AI"] D["Embodied Robotics<br/>Autonomous Vehicles"] end

A --> B B --> C C --> D ```

- Annual global vehicle travel: Approximately 13 trillion miles logged across personal cars, delivery vans, and motorbikes. - Current autonomous penetration: Only 0.006% of global miles driven autonomously today. - Industry vision: Transitioning 100% of mobility and logistics transport to full autonomous capability over time.

---

[09:57](https://www.youtube.com/watch?v=7N38fD4ksnI&t=597s) The Three-Computer Architecture and Five-Layer Stack

To address the immense complexity of physical AI, NVIDIA has structured its automotive strategy around Jensen Huang’s three-computer paradigm. Solving autonomy requires an ecosystem composed of a cloud training supercomputer, a cloud simulation supercomputer for verification, and an in-vehicle inference supercomputer. On top of these three computational pillars, NVIDIA is building a comprehensive five-layer platform designed to serve developers, Tier 1 suppliers, and automotive OEMs.

At the base rests the DRIVE Hyperion hardware layer, which provides a unified reference architecture spanning computing hardware and sensor suites. Directly above it sits HALOS (Hardware Abstraction and Safety Operating System), delivering safety guardrails, middleware, and operating system foundations for end-to-end models. The middle tier contains foundation and driving models such as the open-sourced Alpamayo architecture. Above that, the application tier encompasses DRIVE AV, supporting in-house full-stack software ranging from assisted driving to complete L4 driverless operations. Finally, an overarching cloud infrastructure layer provides toolchains, synthetic data generation, and foundation models to streamline model development and validation.

```mermaid flowchart TD accTitle: NVIDIA DRIVE Five-Layer Physical AI Architecture accDescr: The five architectural software and hardware layers spanning the three-computer paradigm of training, simulation, and vehicle inference.

subgraph CloudInfra["Cloud Infrastructure Layer"] L5["Foundation Models, Toolchains & Cloud Sim"] end

subgraph AppLayer["Application Layer"] L4["DRIVE AV Full-Stack Software (L2+ to L4)"] end

subgraph ModelLayer["Model Layer"] L3["Alpamayo Reasoning & Foundation Driving Models"] end

subgraph SystemLayer["Operating System & Safety Middleware"] L2["HALOS (ASIL D Safety OS, Guardrails, Middleware)"] end

subgraph HWLayer["Unified Vehicle Hardware Layer"] L1["DRIVE Hyperion (DRIVE AGX Thor & Sensors)"] end

L5 -.-> L4 L4 --> L3 L3 --> L2 L2 --> L1 ```

- The Three Computers: - Cloud Training Computer (foundation model training). - Cloud Simulation Computer (closed-loop testing and synthetic validation). - In-Vehicle Real-Time Inference Computer (in-car execution). - Five Service Layers: - Hardware: DRIVE Hyperion reference compute and sensor configurations. - Platform/OS: HALOS operating system and safety runtime middleware. - Models: Alpamayo open and proprietary reasoning driving models. - Applications: DRIVE AV software stack scaling from partial assistance to full driverless autonomy. - Infrastructure: Cloud development toolsets, foundation models, and evaluation pipelines.

---

[12:31](https://www.youtube.com/watch?v=7N38fD4ksnI&t=751s) DRIVE Hyperion 10, AGX Thor, and Sensor Reference Designs

The compute core of DRIVE Hyperion is powered by dual DRIVE AGX Thor SoCs—NVIDIA’s flagship automotive superchip architecture built on Blackwell technology. Because in-vehicle platforms are perpetually constrained by thermal envelopes and memory bandwidth, Thor incorporates native 4-bit floating point (FP4) precision support alongside FP16 and FP32. Leveraging FP4 allows the system to achieve an approximate 20x performance uplift over the previous-generation DRIVE Orin SoC, providing the compute density necessary to run complex reasoning and multimodal models locally.

Hyperion also addresses data fragmentation across carmakers by establishing a standardized, cross-compatible sensor reference architecture. NVIDIA is partnering with Uber to field a dedicated fleet equipped with the Hyperion 10 sensor specification, kickstarting widespread data collection for the ecosystem. The platform offers two primary sensor packages: a High configuration and a Base configuration. The High model provides full fail-operational redundancy designed for urban and highway L3/L4 deployments, comprising 14 high-definition exterior cameras, up to 4 interior monitoring cameras, 9 imaging radars, and 1 forward LiDAR. The cost-optimized Base package uses 10 cameras and 3 radars to deliver address-to-address, hands-free highway and urban driving, including park-to-park maneuvers.

- Central Compute: Dual DRIVE AGX Thor SoCs built on the NVIDIA Blackwell architecture. - Compute performance: Up to a 20x throughput improvement over DRIVE Orin when utilizing FP4 precision. - Hyperion 10 High Configuration (L3/L4): 14 exterior HD cameras, up to 4 interior cameras, 9 radars, and 1 LiDAR with full hardware fail-operational redundancy. - Hyperion 10 Base Configuration: 10 exterior cameras and 3 radars delivering hands-free address-to-address and park-to-park capability. - Fleet Partnership: Collaborative data collection deployment utilizing Hyperion 10 sensor suites across an Uber testing fleet.

---

[16:11](https://www.youtube.com/watch?v=7N38fD4ksnI&t=971s) HALOS: A Unified L4 Software Safety Foundation

Sitting directly on top of the Hyperion compute fabric is HALOS, designed to provide a unified, safety-certified software foundation for Level 4 autonomous operations. At its base layer, HALOS incorporates an operating system certified to automotive safety's most stringent integrity classification—ASIL D. Evolving from the legacy DriveOS runtime, this OS incorporates foundational developer toolkits and libraries including CUDA, TensorRT, and an LLM SDK to ensure large vision-language and reasoning models execute with maximum hardware efficiency. HALOS maintains broad automotive OS compatibility, supporting both safety-critical QNX and standard Linux environments.

Above the OS kernel, HALOS introduces a standardized middleware layer that redefines historical boundaries between NVIDIA and automotive OEMs. During the preceding DRIVE Orin generation, carmakers had to dedicate hundreds of embedded software engineers just to specify, write, and maintain custom hardware abstraction layers for diverse vehicle sensors and actuators. Under the HALOS architecture, NVIDIA is stepping in to absorb those abstraction responsibilities directly within the platform, freeing OEM engineering teams to concentrate their resources on differentiated user experiences, intelligent vehicle behaviors, and top-level autonomy policies.

- Safety certification: Engineered and certified to ISO 26262 ASIL D, the highest automotive functional safety rating. - Included software runtimes: DRIVE OS foundation, CUDA, TensorRT acceleration, and specialized LLM SDKs for edge inference. - Supported host operating systems: Concurrent support for enterprise Linux and safety-certified QNX environments. - Architectural evolution: Native hardware and sensor abstraction layers in HALOS eliminate the need for OEMs to deploy hundreds of engineers to bridge the gap between silicon and autonomous software.

[17:49](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1069s) HALOS Safety Guardrails and the DRIVE Hyperion Platform

The software architecture powering DRIVE Hyperion is designed to be highly generalizable across different automotive OEMs, drastically cutting the engineering overhead and integration burden required for any car manufacturer to get an autonomous driving stack up and running. Looking toward future deployments with the NVIDIA DRIVE Thor platform, NVIDIA plans to introduce comprehensive, safety-certified guardrails directly into the operating system layer. This architecture pairs cutting-edge end-to-end AI models with five-star Euro NCAP and NCAP active safety software, using a proven classical stack to serve as an indispensable safety backstop.

By building this safety foundation across the various layers of the HALOS operating system, autonomous vehicle developers can run their custom models on top of a rock-solid, standards-compliant baseline. The overarching goal is to let developers innovate rapidly without having to reinvent safety mechanisms from scratch, as the speaker outlines:

> “So in the future, hopefully for every AV developer, they can, you know, put their own end-to-end model on HALOS, you know, and benefit from the safety foundation we have built across different layers within HALOS.”

- **Standardization:** Generalizable cross-OEM software minimizes the engineering burden for deploying autonomy on DRIVE Hyperion. - **Safety integration:** Integrates NCAP and Euro NCAP 5-star active safety software directly alongside classical algorithmic fallbacks. - **Next-gen roadmap:** Full safety guardrail integration is scheduled for rollout with the upcoming DRIVE Thor generation.

---

[18:55](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1135s) Announcing Albatross 1.5: Expanding Reasoning, Routing, and Multimodal Prompts

Building on the foundation model layer, NVIDIA announced the release of Albatross 1.5, arriving barely two months after the launch of Albatross 1.0. Albatross 1.0 was a pioneering 10-billion-parameter model tailored specifically for AV reasoning and emerged as a massive hit in the developer community. Over its first two months, it became the second most downloaded model in the entire robotics category on Hugging Face, amassing over 160,000 downloads—a metric that humorously surprised the team, with the speaker remarking, “I didn't even know there are so many people working on AV.”

Albatross 1.5 retains the 10-billion-parameter footprint of its predecessor but adds critical production-ready capabilities. Foremost among these is native routing support: the model can ingest navigation guidance, waypoint sequences, or vectorized map routes and execute trajectory planning accordingly. In addition, Albatross 1.5 gains natural language prompt understanding, allowing drivers or passengers to ask the vehicle what maneuver it is executing or why a specific decision is being made. It also brings flexible sensor configurations, supporting anywhere from one to four cameras with varying fields of view and placements. These advancements helped Albatross 1.5 secure the number-one spot on the public LingoQA benchmark.

```mermaid flowchart TD accTitle: Albatross Model Training Hierarchy accDescr: Pyramid showing how internet-scale Cosmos models feed fine-tuned autonomous driving models. subgraph Foundation ["Web-Scale Base Layer"] C1["Cosmos Foundation Model<br/>(20M hours web-scale data)"] end subgraph Intermediate ["Transfer & Reasoning Layers"] C2["Cosmos Predict Transfer"] C3["Cosmos Reason"] end subgraph AVLayer ["Fine-Tuning Layer"] A1["Albatross 1.5 Model<br/>(80,000 hours curated AV data)"] end

C1 --> C2 C2 --> C3 C3 --> A1 ```

Behind the scenes, Albatross was trained on 80,000 hours of curated autonomous driving data. While 80,000 hours might sound modest for a model of this caliber, it was not trained from scratch. Instead, its backbone derives directly from the Cosmos Reason model within the Cosmos open-source family. Cosmos was pre-trained on internet-scale data comprising 20 million hours of real-world video. From there, it was adapted via Cosmos Predict transfer learning before Cosmos Reason was fine-tuned into Albatross, providing the architecture with exceptional generalization capabilities.

- **Model size:** 10 billion parameters, optimized for spatial and temporal autonomous reasoning. - **Hugging Face traction:** Albatross 1.0 achieved 160,000+ downloads, ranking #2 across the robotics category. - **Routing inputs:** Ingests waypoints, vectorized maps, or high-level navigation instructions to plan paths. - **In-cabin interaction:** Natural language prompt comprehension allows real-time passenger Q&A regarding driving maneuvers. - **Camera flexibility:** Adaptable to 1, 2, or 4 camera sensor layouts across varying fields of view and mounting locations. - **Benchmark:** Ranks #1 on the public LingoQA autonomous driving benchmark.

---

[22:40](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1360s) Production Pipeline: Open Data, Neural Reconstruction, and World Models

Deploying open-source models like Albatross into safety-critical, production vehicles requires an exhaustive fine-tuning and validation pipeline. The workflow relies on massive data ingestion—both physical and synthetic—along with fast data search, curation, model training, and continuous failure-mode discovery fed back from road testing into retraining. To help the wider ecosystem tackle this challenge, NVIDIA is open-sourcing 7,000 hours of high-quality autonomous driving fleet data gathered across 25 different countries during DriveAV development. Although not sufficient on its own to train a model end-to-end, its geographic diversity makes it the largest and most comprehensive open-source validation dataset available.

Tooling is equally critical for closed-loop validation, leading to the public release of NeuRecon (neural reconstruction). NeuRecon allows developers to take real-world sensor logs and photorealistically reconstruct camera pixels from altered vehicle poses and novel camera perspectives, which is vital for evaluating closed-loop behavior without going onto public roads. The speaker emphasized the maturity of NeuRecon by pointing to NVIDIA's internal reliance on it, noting:

> “We are running two million simulation tests based on NeuRecon on a daily basis to develop our stack. So there's a, I guess, famous phrase, we are eating our own dog food.”

Beyond simple scene replay, NVIDIA announced the open-sourcing of two generative AI models that plug directly into NeuRecon: Fixer and Harvester. Harvester extracts physical objects—pedestrians, vehicles, or road obstacles—spotted in real-world driving footage and catalogs them for reuse. Fixer then realistically inserts these harvested objects into entirely different video sequences and alters their trajectory, expanding training diversity by an order of magnitude. Combined with Cosmos Transfer, which can restyle any recorded sequence into novel weather conditions, low-light night scenes, different terrains, or foreign cities via text prompts, developers can systematically target corner-case scenarios in simulation.

```mermaid flowchart LR accTitle: DriveAV Data and Generative Simulation Pipeline accDescr: End-to-end cycle from real-world drive data through simulation modifications to in-car deployment. subgraph Ingestion ["Data Ingestion & Curation"] D1["Fleet Driving Data<br/>(7K hrs open sourced)"] D2["Harvester Model<br/>(Extracts field objects)"] end subgraph GenAI ["NeuRecon & Generative AI"] S1["NeuRecon Closed Loop<br/>(2M daily sims)"] S2["Fixer & Cosmos Transfer<br/>(Weather, lighting, actors)"] end subgraph Production ["Deploy & Feedback"] P1["Model Retraining"] P2["Vehicle Deployment &<br/>Failure Mode Feedback"] end

D1 --> S1 D2 --> S2 S2 --> S1 S1 --> P1 P1 --> P2 P2 -->|Edge cases| D1 ```

- **Open dataset:** 7,000 hours of multi-country driving footage covering 25 nations for testing and benchmarking. - **NeuRecon scale:** Powers 2,000,000 closed-loop autonomous simulation runs every single day within NVIDIA. - **Harvester & Fixer:** Generative models designed to rip real-world entities from footage and dynamically insert them into new simulated runs. - **Cosmos Transfer:** Text-guided environmental transfer that converts clear daylight runs into storms, snow, night, or foreign cityscapes.

---

[29:15](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1755s) DriveAV Evolution: From Parker to Thor and the Mercedes-Benz Global Rollout

NVIDIA's role in autonomous vehicles spans a decade of hardware and software co-development. The journey began 10 years ago when the NVIDIA DRIVE Parker SoC powered the first commercial iterations of Autopilot, followed by the Xavier and Orin architectures, which catalyzed the mass rollout of intelligent driving assistance in China and worldwide. In modern EV markets, particularly China, advanced AV capabilities have become an absolute commercial necessity. Looking ahead, DRIVE Thor is carrying forward this legacy as the centralized supercomputing brain within the DRIVE Hyperion architecture, already moving into start of production (SOP) with leading global OEMs.

On the software side, NVIDIA’s DriveAV team has been building autonomy stacks in parallel for a decade, pioneering deep neural network path planning as early as 2017 with PilotNet. That foundational research turned into major commercial production partnerships: Mercedes-Benz in 2020, followed by Jaguar Land Rover (JLR) and Lucid Motors. In 2025, after five years of deep joint engineering, NVIDIA’s software stack deployed globally across all Mercedes-Benz vehicles outside of China, setting the stage for full point-to-point, address-to-address Level 2++ autonomous capabilities to roll out across the Mercedes fleet throughout 2026.

Technologically, DriveAV uses a hybrid architecture that pairs an end-to-end neural network policy with a classical, deterministic software stack. While end-to-end foundation models provide a smooth, human-like driving feel, traditional automotive validation mandates the functional safety (FuSa) and strict ISO 26262 compliance offered by classical stacks. Combining both allows NVIDIA to deliver human-like driving dynamics without compromising system interpretability or safety verification. In NVIDIA's production build, this entire hybrid stack—including the production version of Albatross—executes efficiently on a single DRIVE Orin system-on-chip:

> “And this kind of interpretable safety is very important to the industry. So having a hybrid stack, you can achieve both. Number one, you can still have human-like driving experience, and meanwhile, without compromising, basically, the safety interoperability.”

- **Compute milestones:** Evolution spanning Parker (10 years ago), Xavier, DRIVE Orin, to the mass adoption and SOP of DRIVE Thor. - **Production OEM programs:** Strategic Tier-1 software and hardware partnerships with Mercedes-Benz, Jaguar Land Rover, and Lucid Motors. - **Mercedes-Benz rollout:** Technology deployed worldwide across the 2025 Mercedes lineup (ex-China), with full address-to-address L2++ functionality rolling out across 2026. - **Hybrid stack architecture:** Couples end-to-end AI (Albatross) for smooth planning with an ISO 26262/FuSa-compliant classical stack for deterministic verification. - **Efficiency:** The complete production hybrid stack runs entirely on a single NVIDIA DRIVE Orin SoC.

[33:19](https://www.youtube.com/watch?v=7N38fD4ksnI&t=1999s) Hybrid Architecture and High-Velocity DriveAV Model Iteration

NVIDIA's internal DriveAV development couples an end-to-end, LiDAR-less, mapless deep learning stack on one side with a proven classical stack on the other. Sitting directly above both pipelines is a dedicated safety arbitrator designed to evaluate planned paths in real time and guarantee vehicle safety throughout execution. This dual-architecture design serves as the engine across NVIDIA’s consumer-facing feature set, providing automated parking, active safety mechanisms, hands-free Highway L2+, and address-to-address or park-to-park urban navigation under L2++.

To bring these end-to-end models into production, the DriveAV team has prioritized rapid engineering velocity backed by a massive dedicated GPU cluster. Having initiated development on this end-to-end model roughly one year prior, the team has already logged over 3,500 distinct model iterations—maintaining a pace of approximately ten new versions deployed every single day.

Simulation infrastructure plays an indispensable role in maintaining this pipeline cadence. NVIDIA’s closed-loop toolchain has compressed the entire cycle from field collection to ready-to-test model down to under six hours, with plans to trim that turnaround to just a few hours. The pipeline incorporates an extended version of the open-sourced AlpaSim simulator, specifically adapted to support closed-loop testing of hybrid stacks where classical algorithms and end-to-end neural network models run concurrently against identical scenarios.

- **Stack Composition:** Mapless and LiDAR-free end-to-end model running in parallel with a classical software stack, coordinated by a real-time safety arbitrator. - **Feature Coverage:** Automated parking, active safety, hands-free Highway L2+, and urban address-to-address L2++. - **Iteration Cadence:** 3,500 total iterations completed within the first year of development, averaging roughly 10 model releases per day. - **Turnaround Window:** Less than 6 hours from initial field data capture to closed-loop validation and replay testing, targeting a reduction to several hours.

---

[35:57](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2157s) Safety Arbitrator Guardrails and Global Scaling of L2++

To illustrate how safety guardrails operate between the co-running stacks, the presentation showcases a real-world driving scenario involving a cyclist riding along the road shoulder. As the shoulder narrows and forces the cyclist inward toward vehicle traffic, the end-to-end model (designated as AlphaMario, highlighted in green) plans an initial path. Simultaneously, the safety arbitrator monitors the projected path against the classical planning modules and detects that the neural network's margin of error is insufficient for safe clearance.

The speaker explains the arbitrator's immediate intervention logic: “Essentially, it will say, okay, this current trajectory is not safe enough.” Without perceptible lag or hesitation, the system executes an instantaneous handoff to the classical planner's trajectory, steering clear of the cyclist while preserving vehicle stability and passenger safety.

With the hybrid software architecture validated in mixed driving environments, NVIDIA has launched a worldwide scaling campaign for its L2++ platform. Development vehicles are operating across multiple continents, with the goal of expanding coverage across the entirety of the United States and selected metropolitan hubs throughout Europe before the end of the year, followed by broader international availability the next year.

- **Arbitrator Transition:** Zero-delay trajectory switching from the end-to-end AlphaMario model to classical trajectories whenever safety boundaries are breached. - **Edge-Case Handling:** Dynamic handoffs during real-time road obstructions, such as disappearing shoulders that force cyclists into active traffic lanes. - **Rollout Milestones:** Complete United States coverage and initial European city deployments scheduled by the end of the year, expanding to full global availability the following year.

---

[37:19](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2239s) Scalable L4 Architecture and the Autonomous Deployment Roadmap

A central design strength of NVIDIA's automotive software stack is its direct architectural bridge between driver-assist capabilities and full Level 4 autonomy. Instead of requiring a ground-up rewrite for driverless operations, the production L2++ foundation extends cleanly into an L4-certified dual-computing layout split between a Primary Electronic Control Unit (Main ECU) and a Secondary Safety Redundant ECU (Satellite ECU).

```mermaid flowchart TD accTitle: L4 Dual ECU Architecture and Fail-Safe Flow accDescr: Diagram illustrating the Main ECU running L2++ with supplementary sensors alongside a Satellite ECU executing classical safety backup and Minimum Risk Maneuvers.

subgraph MainECU["Main ECU (Primary Compute)"] A["L2++ DriveAV Stack (End-to-End & Classical)"] B["Supplementary Sensor Inputs"] B --> A end

subgraph SatECU["Satellite ECU (Safety Redundant Compute)"] C["Classical Redundant Stack"] D["Minimum Risk Maneuver (MRM)"] C --> D end

A <-->|"Arbitration & Status Heartbeat"| C A -->|"Single Point of Failure"| D ```

Under this configuration, the Main ECU runs the core L2++ stack while ingesting supplementary sensor streams routed from the Satellite ECU to elevate perception range and confidence. The Satellite ECU hosts an independent, classical-based safety redundant stack programmed to initiate an automated Minimum Risk Maneuver (MRM) if any single point of failure disables the primary compute block. This common code repository allows commercial partners to develop and deploy both L2++ and L4 vehicles concurrently without bifurcating engineering efforts.

NVIDIA's overarching deployment roadmap sets concrete milestones for bringing these autonomous systems to public roads: - **Current Year:** Global scaling of urban address-to-address L2++ capabilities. - **Next Year:** Initial operational rollouts of dedicated Level 4 commercial pilot fleets. - **2028:** Full commercial deployment of Level 4 autonomous systems in production passenger vehicles alongside OEM partners.

---

[39:07](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2347s) Hyperion Ecosystem Expansion and the Uber Level 4 Partnership

Industry adoption of the NVIDIA DRIVE Hyperion reference platform continues to accelerate, with major global carmakers standardizing on its compute and sensor architecture. The speaker highlights that new original equipment manufacturers are entering the Hyperion fold almost every month, with four leading automakers recently signing on: BYD, Geely, Nissan, and Hyundai. Representing four of the world’s top ten automotive manufacturers, their adoption reflects the operational and economic advantages of relying on a shared, safety-certified compute and sensor baseline rather than custom proprietary hardware.

In parallel with OEM partnerships, NVIDIA announced an alliance with Uber aimed directly at commercializing Level 4 robotaxis. Under the terms of the agreement, the two companies plan to roll out autonomous L4 vehicle services across 28 global cities by 2028. Beyond in-cabin autonomy and embedded chips, NVIDIA emphasizes its role as a full-stack enterprise AI provider, utilizing NVIDIA Omniverse digital twins across automotive factory design, production planning, vehicle aerodynamics, and enterprise software.

Reflecting on the scale of the automotive industry's transition, the speaker stresses the necessity of collective development: “Autonomy is a daunting task. It's a grand mission, but it really needs everybody to work together.” NVIDIA aims to unify OEMs, Tier 1 suppliers, and independent AV software developers behind open platforms to close the performance gap between early prototypes and production-grade safety reliability.

- **New Hyperion Partners:** BYD, Geely, Nissan, and Hyundai join the ecosystem, uniting four of the world's top ten automakers around common compute and sensor architectures. - **Uber Global L4 Alliance:** Formal partnership established to deploy autonomous Level 4 technologies across 28 metropolitan markets globally by 2028. - **Enterprise Digital Twins:** Industrial use of NVIDIA Omniverse for end-to-end automotive design, virtual factory floor simulation, and assembly optimization. - **Industry Safety Target:** Collective effort required to advance autonomy from early single-digit statistical benchmarks toward upper-90s reliability.

---

[41:14](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2474s) Key Takeaways: The Next Phase of NVIDIA DRIVE

Concluding the special address, the speaker reviews the central pillars defining NVIDIA DRIVE’s next stage of growth, structured across three specialized computing domains and five integrated layers of platform services:

First, on the physical vehicle hardware front, DRIVE Hyperion is expanding across top-tier OEMs to standardize sensor layouts and onboard compute, while the strategic data partnership with Uber creates an extensive, diverse training repository accessible to the broader developer ecosystem. Second, NVIDIA is significantly upgrading DriveOS (HALOS), moving it beyond a standard automotive operating system into an active real-time safety guardrail layer capable of supervising end-to-end AI stacks on Hyperion hardware.

Third, on the application layer, NVIDIA announced AlphaMario 1.5, the next generation of its end-to-end driving policy network. DriveAV engineering remains focused on scaling L2++ capabilities globally this year while extending the core code architecture into Level 4 redundancy. Finally, across cloud and simulation infrastructure, NVIDIA is open-sourcing its foundational automotive models, neural reconstruction tools, and Cosmos world foundation and transfer models, equipping developers across the mobility sector to enrich their simulation datasets and compress validation cycles.

## Notable quotes > [07:28](https://www.youtube.com/watch?v=7N38fD4ksnI&t=448s) building and safely deploying physical AI is going to be the defining challenge of the coming decade

> [00:09](https://www.youtube.com/watch?v=7N38fD4ksnI&t=9s) the path to full L4 level autonomy has never been this clear.

> [39:07](https://www.youtube.com/watch?v=7N38fD4ksnI&t=2347s) Autonomy is a daunting task. It's a grand mission, but it really needs everybody to work together.

> [03:16](https://www.youtube.com/watch?v=7N38fD4ksnI&t=196s) What sets Alpamayo apart is how it reasons through complexity, breaking down edge cases before they become problems.

## Mentioned AlexNet, AlpaSim, Alpamayo, AlphaMario, AlphaMario 1.5, BYD, Cosmos, Cosmos Predict, Cosmos Reason, Cosmos Transfer, CUDA, DRIVE AGX Thor, DRIVE AV, DRIVE Hyperion, DRIVE Hyperion 10, DRIVE Orin, DRIVE OS, DRIVE Parker, DRIVE Thor, DRIVE Xavier, Fixer, Geely, HALOS, Harvester, Hugging Face, Hyundai, Jaguar Land Rover, Jensen Huang, LingoQA, Lucid Motors, Mercedes-Benz, NeuRecon, Nissan, NVIDIA, NVIDIA Omniverse, PilotNet, QNX, TensorRT, Uber, William Pereira

## Video description Autonomous vehicles (AVs) are emerging as the first mass-deployed physical AI agents, reshaping mobility and driving new demands in AI, safety, and scalable development. This special address explores the industry’s path to Level 4 autonomy and how NVIDIA is helping accelerate it. With the DRIVE Hyperion architecture, manufacturers (OEMs) and AV developers can build scalable, safety-certified autonomy solutions. NVIDIA will present its cloud-to-car platform—spanning training, simulation, and in-vehicle compute—to support developing and validating next-generation drive-policy models. The session will also touch on how agentic AI is poised to transform AVs end to end, from automating data-driven development and validation to enabling more intelligent, adaptive in-car experiences. Gain insights into L4 software architectures, the DRIVE Hyperion platform, and ecosystem collaboration across OEMs, AV developers, mobility providers, and robotaxi companies.

[Source](https://www.youtube.com/watch?v=7N38fD4ksnI)

Reading summary

NVIDIA frames autonomous vehicles as the leading edge of physical AI, asserting that emerging foundation, reasoning, and vision-language-action models have clarified the pathway to Level 4 autonomy across global mobility. Physical AI and the Three-Computer Architecture (7:28) NVIDIA traces the progression of artificial intelligence from perception in 2012 to generative transformers in 2017, arguing that the coming decade will focus on physical AI deployed through embodied systems like vehicles. With roughly 13 trillion miles traveled globally each year and only 0.006% currently driven autonomously, the company positions road transportation as the first mass-market application for physical intelligence. To address the scale of this problem, NVIDIA outlines a three-computer architecture spanning cloud training supercomputers, cloud simulation supercomputers, and in-vehicle inference hardware. This operational model is realized through a five-layer stack combining DRIVE Hyperion hardware, the HALOS safety operating system, foundation reasoning models such as Alpamayo, DRIVE AV software, and cloud-based simulation and data infrastructure. DRIVE Hyperion 10, Thor, and HALOS Safety (12:31) At the hardware tier, DRIVE Hyperion 10 centers on dual DRIVE AGX Thor SoCs built on Blackwell architecture, introducing native FP4 precision to achieve up to a 20x performance uplift over the prior DRIVE Orin SoC within vehicle thermal constraints. Hyperion provides standardized sensor reference designs, including a fail-operational High package for L3/L4 with 14 exterior cameras, up to 4 interior cameras, 9 radars, and 1 forward LiDAR, alongside a cost-optimized Base package for hands-free driving. Directly above the hardware, the HALOS operating system and middleware layer integrates ISO 26262 ASIL D safety certification, CUDA, TensorRT, and edge LLM runtimes across Linux and QNX environments. By absorbing hardware and sensor abstraction into HALOS, NVIDIA aims to eliminate the need for automakers to write custom low-level device software, allowing OEMs to deploy end-to-end models over standardized active safety guardrails. Albatross Reasoning Models and Generative Simulation (18:55) Within the model layer, NVIDIA introduced Albatross 1.5, a 10-billion-parameter reasoning model derived from the open-source Cosmos foundation model family. Trained on 80,000 hours of curated autonomous driving data following Cosmos pre-training on 20 million hours of web video, Albatross 1.5 supports waypoint and vectorized map routing, natural language passenger dialogue, and flexible configurations of one to four cameras, ranking first on the public LingoQA benchmark. To validate these systems without relying solely on road testing, NVIDIA open-sourced 7,000 hours of fleet driving data spanning 25 countries alongside NeuRecon, a neural reconstruction engine that powers two million daily closed-loop internal simulations. This setup integrates the generative tools Harvester and Fixer to extract real-world objects and insert them into novel trajectories, while Cosmos Transfer restyles recorded footage across weather, lighting, and urban environments. Hybrid DriveAV Stack and Deployment Milestones (29:15) NVIDIA's production software stack, DriveAV, employs a hybrid architecture pairing a mapless, end-to-end neural network with an ISO 26262-compliant classical deterministic stack. A real-time safety arbitrator continuously monitors both paths and executes zero-delay handoffs to classical trajectories whenever safe margins are compromised, as demonstrated during edge cases like cyclists entering traffic lanes. NVIDIA reports that this complete hybrid stack executes on a single DRIVE Orin chip and achieves rapid iteration, logging over 3,500 model revisions in its first year. The architecture scales from consumer driver-assistance to full driverless operation by bridging L2++ software into an L4 dual-ECU setup, pairing a primary compute unit with a redundant satellite unit running automated Minimum Risk Maneuvers. NVIDIA's production roadmap began with software deployments in non-China Mercedes-Benz vehicles in 2025 targeting address-to-address L2++ in 2026, alongside OEM adoption from BYD, Geely, Nissan, and Hyundai, commercial Level 4 fleet pilots next year, and an alliance with Uber to deploy L4 services across 28 cities by 2028. While NVIDIA presents a unified hardware and software path toward consumer Level 2++ and commercial Level 4 autonomy by 2028, realizing these targets requires extensive industry-wide coordination, rigorous validation of hybrid fail-safes, and closing the statistical reliability gap between developmental prototypes and production-grade safety standards.

Source quotes

[7:28] building and safely deploying physical AI is going to be the defining challenge of the coming decade [0:09] the path to full L4 level autonomy has never been this clear. [39:07] Autonomy is a daunting task. It's a grand mission, but it really needs everybody to work together. [3:16] What sets Alpamayo apart is how it reasons through complexity, breaking down edge cases before they become problems.

Original transcript

[0:09] Good afternoon, everybody. [0:11] Welcome to GTC. [0:13] Hopefully, you are enjoying it. [0:15] And first of all, I want to say, what a year, especially for us AV developers. [0:23] In 2025, I would say that it's also the chat GPT moment for autonomous driving technologies. [0:31] Mostly because of the rapid progress of AI technologies. [0:37] We have VLA, vision language action models, and we have word model foundation models, and we also have reasoning models. [0:47] With these technologies, I would say over the last few years, we have been seeing the way we develop AV have been fundamentally changed. [0:59] Sorry. [1:03] Good timing. [1:10] On the other hand, these changes didn't really slow us down, though. [1:17] And on the contrary, we are seeing, you know, the whole industry is moving much faster towards full autonomy. [1:26] And at this moment, I would say that the path to full L4 level autonomy has never been this clear. [1:37] And also the mission to make everything moves autonomous have never been this real. [1:45] And NVIDIA actually have been working on this problem. [1:48] I would say the autonomy challenge for a long time, for more than a decade actually. [1:54] I remember 10 years ago, we produced our first, I would say, ADAS device or ADAS chip which is called, sorry. [2:08] which is called the Parker, and that device actually enabled the first generation of autopilot. [2:17] And 10 years from that, today, when we're looking at a pivotal moment of AV industry, obviously NVIDIA is working super hard, my team is working super hard with the rest of the industry and the ecosystem to find a path towards scalable and a safe level for autonomy. [2:45] And in this presentation, I will just walk you through what we have been doing. [2:53] To kick off the presentation, actually I would like to first show a video which is what my team did recently, a drive from San Francisco around the North Bay based on a reasoning model in a prototyping vehicle from our partner, Mercedes. [3:16] All right, let's get the video rolling. [3:20] Today we're going to experience the reasoning capabilities of the Alpameo model that powers the intelligence in our cars. [3:27] Throughout the drive, you'll see the car is constantly evaluating its surroundings in real time. [3:35] You can also ask the car for information about your surroundings. [3:39] Hey Mercedes, why is the Transamerica Pyramid shaped like that? [3:44] Architect William Pereira chose the tapered design so more sunlight could reach the streets below instead of casting a huge shadow. [3:52] We've also asked the car to narrate its thinking out loud today, so you'll hear it call out what it's observing and why. [3:57] I'm changing lanes to the right to follow my route. [4:14] Alpamayo is continuously reasoning in the background, so that when something unexpected happens, the car is already ready to act. [4:20] I want to turn left, but there is oncoming traffic, so I'm waiting for a gap. [4:30] There's a double parked vehicle in my lane. [4:33] I'm going around it. [4:37] There's a pedestrian in the crosswalk ahead. [4:39] I'm going to yield. [4:48] There's a vehicle crossing my path ahead. [4:50] I'm going to yield. [4:56] Thank you for watching. [5:03] What sets Alpamayo apart is how it reasons through complexity, breaking down edge cases before they become problems. [5:17] Hey Mercedes, can we speed up? [5:20] Sure, I'll speed up. [5:25] Hey Mercedes, make a lane change to the right. [5:30] Sure, initiating lane change. [5:34] You know, I think maybe we should go to Sausalito to grab a coffee. [5:37] Hey Mercedes, can you take the next exit please? [5:41] Sure, turning right ahead. [5:56] Hey Mercedes, pull over here. [6:00] Sure, I'll find a safe place to drop you off. [6:11] Thank you for watching. [6:18] From dense city streets to highways and bridges, in both real and simulated environments, Alpameo reasons through it all, continuously building a safer, smarter driving experience. [6:37] There's a pedestrian in the crosswalk ahead. [6:39] I'm going to yield. [6:44] Hey Mercedes, can you pull over here? [6:49] Sure, I'll find a safe place to drop you off. [7:04] All righty, hopefully this video, thank you. [7:09] I think my team does deserve a round of applause because they have been working super hard. [7:14] And hopefully, towards the end of this year, we can really have the customers and other folks to be able to experience this in a real car. [7:28] So, hopefully this video, again, gives us a glimpse of the future, how a reasoning model can be used to tackle the autonomy task. [7:40] And this slide, obviously, I believe Jensen has been using the slides in a few of his keynotes, and, you know, this shows the journey we have been through in the AI technology development. [7:56] With LXNet in 2012, I think, and then Transformer in 2017, we have been seeing perception AI evolve into generative AI, and with the reasoning model, which is happening right now, [8:11] We are introducing agentic AI, and my team and obviously the rest of the industry are working on also putting this AI technology into a physical embodiment, which is the physical AI. [8:25] And we believe that building and safely deploying physical AI is going to be the defining challenge of the coming decade and [8:35] Since automotives are just basically driving robots on the road, I'm pretty sure AV will be the first mass-produced and mass-deployed physical AI technology and applications. [8:52] And why is autonomous driving so important? [8:56] I think there's a few numbers here. [8:59] So the most important number here is actually the middle one. [9:04] Every year, across all different segments, roughly speaking, there's 13 trillion miles being driven by a car. [9:16] Or a delivery motorcycle, a delivery car. [9:24] And obviously there's other use case as well. [9:27] And as of today, only 0.006% of the mileage are driven autonomously. [9:35] And we do believe that in the near future, every mile, everything that moves will become autonomous. [9:46] And this gap is the huge opportunity in front of all of us, even for the size of NVIDIA. [9:57] And to tackle this challenge, as I said in the beginning, at this pivotal moment, we are also pivoting our main, basically, product, which is NVIDIA Drive, to a slightly different form. [10:17] As Jensen mentioned, physical AI problem is a three computer problem. [10:23] So in Drive AV, in NVIDIA Drive, we have really three computers, the training computer and the simulation computer in the cloud, and the inference computer in the car. [10:36] And we are going to build five layers of services on these three computers. [10:41] And on the bottom layer, obviously, it's our hardware. [10:45] And here, it's the Drive Hyperion. [10:48] And it provides a unified reference design for computer and sensors. [10:56] And on top of that, it's the operating system and the platform software. [11:01] This is what we call the Halos. [11:05] Halos provides the middleware that interacts the model and our hardware, and also in the future we want Halos to be able to provide safety guardrails for anybody who wants to deploy an end-to-end model on our Hyperion hardware. [11:30] And on top of that, we have the model layer, where we use, I guess you guys have heard about AppMio, and we have open-sourced a version of AppMio, and we will keep developing on that. [11:44] And on top of that, it's applications. [11:46] This is where DriveAV come into play. [11:50] And we are working our partners to deliver from partial to full autonomy in our basically in-house full stack. [12:01] And, last but not least, infrastructure is very important in the age of AI. [12:08] We are building a bunch of foundation models and tools in the cloud to help developers to develop efficient models to be able to run our hardware and the full stack. [12:28] Let's get into a little bit into the details. [12:31] And as I said, Hyperion is a unified vehicle reference architecture towards L4. [12:37] And as a call of it, obviously, it's a L4 ready, you know, computer architecture, which is based on a dual, basically, AGX SOAR, which is our flagship device of current generation for in-car SOC. [12:56] And on top of that, we are also providing a reference design for the sensor suite in Hyperion as well. [13:05] And this is very important because only when you have a unified sensor architecture, the data will become shareable across different OEMs, different platforms. [13:17] And on top of that, we are also working with our partner, Uber, to build a fleet of basically Hyperion, the current generation called Hyperion 10, Hyperion 10 sensor set and to do data collection and give a sort of jumpstart to the whole industry. [13:35] I think this is very important of our strategy as well. [13:42] A little bit more regarding our current flagship device, Ajax SOAR. [13:48] This is really a in-vehicle AR supercomputer. [13:53] It's based on the Blackwell architecture, which is the same architecture as most of the company are using to train the foundational models. [14:02] And it supports basically, you know, obviously different positions, starting from 32 to 16. [14:09] It also supports, you know, floating point 4, 4-bit floating point, which is very important because for in-car devices, you are always limited by memory. [14:21] memory bandwidth, and this can effectively increase your in-car computer capability. [14:28] And as compared to the previous generation, if we can, you know, use, you know, FP4, that's really basically a 20x boost in terms of performance as compared to Orion. [14:42] And the census suite. [14:44] Again, to make a census suite common is very important, as I said, from a data-shareable perspective. [14:53] And we have two models. [14:56] On the right side is the high model, which we believe will be able to sufficient for a high-level economy, including highway and urban L3 and L4. [15:06] And right now, this sensor suite has 14 high-definition cameras and up to four internal cameras, and it has nine radars and one LiDAR. [15:19] This sensor is designed to have full sensor redundancy, so that whenever you have a single point of failure, the system will still be able to operate safely. [15:32] On the left side is our base model, and this is obviously more focused on the cost, you know, saving in terms of at a bomb level, but with 10 cameras and 3 radars, this system will be able to deliver a full, hand-free... [15:56] Highway and urban, address-to-address, even park-to-park capability, which is, you know, what we are building on with our customers in DriveAV. [16:11] And on top of Hyperion, we build Halos. [16:17] Halos, as I said, is a unified software safety foundation for L4. [16:23] It actually have multiple layer of software, you know, defined within Halos. [16:28] At the bottom layer, it's our operating system, which is certified to the highest ASO level, which is ASO Delta. [16:38] And this is based on the legacy DriveOS, basically software, and DriveOS will become part of Halos moving forward. [16:49] And it provides a library, including CUDA, TensorRT, and LMSDK, so that we can enable developers to deploy large language models and AI models for AV in the most effective way. [17:07] And it also supports both Linux and the QNX, basically, operating system. [17:14] And on top of that, it's middleware. [17:17] So middleware is very important. [17:19] In our current generation, which is Orem-based generation, we used to have a separate responsibility between NVIDIA and OEM. [17:29] And OEMs actually have to spend hundreds of engineers to define this sensor and vehicle, let's say, abstract layer. [17:38] And moving forward, in Halos, as part of Halos operating system, Halos OS layer, we will take more responsibility. [17:49] And because much of this software is generalizable across different OEMs. [17:54] We can significantly reduce the engineering burden on the OEM side to be able to deploy their stack on Hyperion. [18:04] And on top of that, in the future, in the SOAR generation, we also want to introduce, as I said, the safety guardrails for the system. [18:16] This will include our five-star EN-CAP and N-CAP active safety software, and it will have our classical stack to serve as a guard rail. [18:30] So in the future, hopefully for every AV developer, [18:34] They can, you know, put their own end-to-end model on halos, you know, and the benefit from the safety foundation we have built across different layers within halos. [18:55] Now let's go to the model layer, and today I'm very proud to announce that we are going to release our Mario 1.5. [19:06] And our Mario 1.0 was released only, I think, two months, slightly more than two months ago. [19:13] And it was a 10 billion parameter model, which is also the first. [19:21] Recently model for AV and over the last two months it was actually number two in the most downloaded application or model in the whole robotic space on Hugging Face. [19:38] It was downloaded more than 160,000 times. [19:43] I didn't even know there are so many people working on AV. [19:47] But, you know, we are very happy about, you know, what we, you know, the result. [19:53] And in our Pomayo 1.5, it's still a 10 billion parameter model, but we are going to make it more powerful. [20:01] So what are the changes? [20:03] Number one, we are going to introduce routing capability into the model in different formats. [20:09] It can be waypoints or it can be a nav guidance from a, you know, let's say, a navigation software or in other some, let's say, vectorized format, then the model will be able to follow the route. [20:28] And the number two, we are also introducing text prompts capabilities as you what do you have seen in the in the in the video in the opening video The driver or the passenger can ask what the car is trying to do [20:43] And we are also introducing multiple configurations in the camera support. [20:50] It can support one or two or four cameras with different FOVs and placements and so on and so forth. [20:57] We believe that with this model, it can hopefully add one more capability. [21:07] One more capable model to our ecosystem and help the industry, you know, build more advanced models. [21:16] And as a matter of fact, you know, our Mario 1.5, you know, ranked number one in the open dataset lingo QA. [21:32] This is going a little bit behind the scenes regarding how we train our model, the open source model. [21:38] And we trained our model with 80,000 AV data. [21:44] Maybe it doesn't sound too much, but we are not training our Mario model from scratch. [21:50] Our Mario model was actually the backbone was taken from the Cosmos Reason model, which was obviously part of the Cosmos Open model family. [22:02] And Cosmos, the foundation model, was trained with internet-scale data, with 20 million hours of real-world data. [22:11] And then on top of that, there's basically a Cosmos predict transfer and then the Cosmos reason model was trained on top of that. [22:20] So this pyramid shows us basically how much data the actual model of Mario inherit from the foundation model. [22:30] And hopefully it will also show great generalization capability in the field as well. [22:40] However, taking the Mario open source model to production will still take quite a bit of work. [22:48] Hopefully, this is not very surprising to anybody here. [22:52] So, to build a production-ready, basically, model, you have to make sure the model is fine-tuned to work well in all different scenarios. [23:03] And this is a typical pipeline, which is what we use for internal effort as well, which I'm going to show you in a later slide, regarding how to fine-tune the out-model. [23:13] and get it ready for production. [23:15] So you would need lots of data set, obviously, either through data collection or through synthetic data, which Cosmos can actually help over there as well, which is also open source to public. [23:27] And... [23:30] In the middle, obviously, you have to prepare the data set and train the model and then basically when you release to the car and then basically you have to get the, let's say, the failure mode from the road and basically feedback the data to the model training. [23:52] So, the tools are very important here. [23:55] One is basically, obviously, when we prepare the dataset, you need a good search and curation capability. [24:02] And also, on the right side, you know, simulation and the validation is super important in the gen AI, you know, kind of way of developing AV. [24:14] And so, [24:18] To help, you know, again, the ecosystem to be able to train the model, so we are not only open sourcing the model itself, we are also open sourcing data, which is right now, at this point, it's 7,000 hours of high-quality data, which we collected in the driver AV development phase. [24:41] To the best of my knowledge, this is by far still the largest data set, open source data set out there. [24:47] And this is probably not enough for training, but this data has good, basically, diversity. [24:52] It's collected into, you know, in 25 countries. [25:00] And it will be very well suited for evaluation and testing. [25:07] And we are also, you know, as I said, tooling is very important. [25:11] You need good tooling for search and curation, and you need good tooling for validation and assimilation. [25:19] And towards this end, we are also basically make, you know, Neurac, which is neural reconstruction, as a tool to be available to public soon. [25:31] Not yet, but in the next few weeks. [25:34] And the Neurac is a super powerful, basically, tool, which is absolutely needed for developing AV stack, especially the end-to-end model. [25:43] So this video. [25:46] Hopefully, I haven't clicked it yet. [25:48] Okay, so this video shows the, I guess, the cameras. [25:54] And you can see, you know, we can really reconstruct the pixel when the vehicle is changing a different pose. [26:01] This is very critical for any, basically, closed-loop evaluation of the stack of all of the end-to-end model. [26:10] So this... [26:12] This technology is pretty mature. [26:15] It's getting more and more mature as we speak as well, partly because it's massively utilized in our internal effort to drive AV development. [26:25] We are running two million simulation tests based on NeurIQ on a daily basis to develop our stack. [26:33] So there's a, I guess, famous phrase, we are eating our own dog food. [26:40] And I'm very confident about the quality of this tool, and hopefully this can help the community. [26:52] Well, Nurex is not only about basically doing just replay, you know, different trajectory and reconstruct the pixel. [27:00] You can do more about that. [27:02] And as part of the, you know, Nurex open-source effort, we also open-sourced [27:09] Fixer. [27:10] These are two Gen AI models. [27:11] One is Fixer. [27:13] The other one is basically Harvester. [27:16] The Harvester can actually harvest any object you see from the field and save it for other use. [27:23] For example, you can plant it in other videos. [27:27] So this slide shows how we can leverage this capability, not only use NeurIQ for replay, but also use it to insert and modify behaviors of the object. [27:37] By doing this, we can increase the data diversity or variety by auto-magnitude, you know, from the data we collect from the field. [27:48] So, there's a few examples. [27:50] I will just quickly click through them. [27:54] Essentially, you can, you know, obviously you can just do the reconstruction. [27:58] You can also basically insert a scooter here. [28:01] I'm not sure if you guys can see. [28:03] I cannot see very well. [28:04] You know, the screen in front of me is extremely small. [28:08] And then, basically, you can also modify the trajectory when this kind of object shows up and then feed it back, train a different model, and deploy it to the car. [28:22] And the other one which is extremely important and extremely powerful is Cosmos Transfer. [28:29] Again, Cosmos Transfer, as I showed you guys briefly in the data pyramid, is a part of the basically Cosmos family. [28:37] And what it does is basically you can render any sequence into a different environment. [28:43] And is there some example here? [28:46] Can we go back to the previous slide, please? [28:49] Yeah, so there's a multiple example here. [28:52] Obviously, you can just add a prompt of different weather, different lighting, and even in different cities, in different essential terrain, so this is, again, another way to bring a lot more variety and diversity into a data set and train the model. [29:15] All right, last but not least, let me spend time talking about the driver AV. [29:22] And we have a pretty sizable team working on driver AV right now. [29:27] And obviously, NVIDIA is pretty well known to work with the industry to provide AV SOCs. [29:34] As I mentioned, 10 years ago, Parker enabled the first version of Autopilot. [29:40] And the next generation, basically Xavier and Orin, they really... [29:48] started the massive transform or bring up of you know AV technology in China and the rest of the world and right now basically at least in the China market you cannot really sell a car new car without a good AV capability in it. [30:08] And we are obviously we announced the SOAR as well. [30:12] It has been in the market for a few years and you know actually there's already OEMs, you know taking SOAR to to SOP And we are also seeing very good trend in the rest of the world market as well of SOAR adaptation adaptation and [30:31] SAW is, as I said, part of the Hyperion, basically, ecosystem, Hyperion platform and, you know, for everybody who's adopting Hyperion, we'll use SAW as an, you know, in-car AI supercomputer. [30:46] It's probably less well known that NVIDIA worked on AV stack also for a long time. [30:53] DriveAV was kicked off 10 years ago as well. [30:57] And in 2017, actually, we had our internal, which was a little bit ahead of its time, I would say, end-to-end model-based approach, basically, for AV, which is called a pilot net. [31:12] And then, with many years of progress, we secured a partnership with Mercedes in 2020, which was five years ago. [31:26] And then, as the JLR happened and Lucid happened, and I cannot say much yet, but we'll have more OEMs getting into the DriveAV ecosystem. [31:41] And this year, no, last year, 2025 was the first year, after five years of partnership, we deployed our technology to all Mercedes vehicles globally, except China. [31:55] And we are expecting to roll out our address-to-address L2++ capability throughout this year in Mercedes vehicles, which is gonna be super exciting. [32:08] And we are also working on, as we speak, JLR and Lucid customer fleet as well. [32:19] Our driver AV stack is actually a hybrid end-to-end classical stack. [32:25] This is very important because we get the human-like drive experience from the Android model. [32:31] This is very well known. [32:32] But the whole industry has been developing AV and ADAS based on basically FUSA and ISO 26262 standards. [32:43] And this kind of interpretable safety is very important to the industry. [32:47] So having a hybrid stack, you can achieve both. [32:51] Number one, you can still have human-like driving experience. [32:56] And meanwhile, without compromising, basically, the safety interoperability. [33:04] So this... [33:06] Slightly busy diagram shows our stack. [33:09] What's important here is, you know, our Mario, which is the production version of Mario, is our end-to-end model running the stack. [33:15] By the way, we are running everything here in one ORN. [33:19] And basically, this is a LIDAR list and a mapless stack. [33:23] And on the right side, it's our classical stack. [33:26] And on top of that, basically, we have a safety arbitrator, which can guarantee safety, you know, when the stack is running. [33:38] And with this stack, obviously, we can, we are delivering parking, we are delivering active safety, and we are also delivering advanced, you know, AV features, including Highway L2+, with handle-free capability, and L2++, which is the address-to-address or park-to-park in urban scenarios. [34:05] And this slide shows a little bit how we take our model to production in the internal DriveAV effort. [34:14] It's very similar to the slide I showed before, and we're iterating super fast. [34:21] We have iterated, basically, we started developing this end-to-end model about a year ago, and now we have iterated 3,500 times. [34:28] And roughly speaking, we are iterating 10 versions every day. [34:39] And basically, we are also using, obviously, a sizable GPU cluster to be able to iterate this fast. [34:49] And also, as I mentioned, simulation is very important for this end-to-end model development. [34:59] And right now, basically, our technology has been enabled from [35:06] Test in the field until a model is ready. [35:09] Again, every new rack is a model. [35:12] You have to build this model and use it in replay and the closed-loop evaluation. [35:18] Right now, we can contain the whole hours within turnaround time within six hours. [35:23] And we believe that we can take it down to a few hours in the near future. [35:29] So, again, this is a massive effort, and the middle block is, you know, is similar to the AlpaSIM we open sourced to public, but this one is slightly more because it have to be able to do a closed-loop evaluation of a hybrid stack. [35:47] So you can run a classical stack and the Android model and use this to get a good evaluation result. [35:57] Well, this one explains roughly how guardrails work with a classical stack running, again, side by side by the end-to-end model. [36:08] We have a cyclist right on the side, and because his shoulder is ending, to end up basically coming to the road. [36:18] So, when it's green, it shows how Mario was running it. [36:20] When it's classical, the arbitrator changed the model to classical. [36:25] Essentially, it will say, okay, this current trajectory is not safe enough. [36:31] The classical, you know, trajectory is better. [36:34] Then it will seamlessly, almost like with zero delay, it can transfer, it can transit, I would say, to a classical trajectory and to maintain safety. [36:49] And as we speak, we just started basically, you know, worldwide scaling as well for our L2++ stack. [36:56] And this is showing we are basically driving pretty much everywhere in the world. [37:01] And we expect that by the end of this year, we should be able to scale the technology to all of the U.S. [37:08] and a few cities in Europe. [37:11] And the next year, we want to basically make it, you know, available globally. [37:19] And another important point to hit is basically our current software structure is actually very much scalable or extendable to L4. [37:29] So this slide shows what the architecture will look like when we enable this kind of capability, our stack, in the L4 setting. [37:38] You will have a main ECU. [37:41] And you also have a satellite ECU. [37:43] In the main ECU, it's pretty much running the L2++ stack we are running right now. [37:50] But on top of that, it will take additional sensors from the satellite ECU to make the main ECU capability even better. [38:00] And on the satellite ECU, this is our safety redundant stack. [38:05] It's mostly going to be classical based. [38:08] But it will also have the MRM or minimum risk maneuver when basically, you know, a single point of failure happens. [38:19] So in this way, basically we can use the same code repository and delivering L2++ and L4 together to our partners. [38:32] All right, this is our high-level roadmap. [38:37] As I said, basically, this year, we are trying to scale L2++ from address to address. [38:44] And starting from next year, we want to start the initial deployment of our L4 fleet. [38:51] And by the end in 2008, [38:55] You know, we will basically deploy L4 in passenger vehicles with our partners. [39:07] And basically, I want to, this is a, you know, Hyperion is, you know, catching a lot of attention, I would say, and, you know, we are having new OEMs joining, basically, Hyperion ecosystem almost, I would say, every month. [39:24] And, you know, we have four new OEMs joining the Hyperion ecosystem, BYD, Geely, Nissan, and Hyundai. [39:34] All of them are, you know, fall of the top 10, basically, OEMs globally, and again, we see great benefit of, you know, having a common sensor suite and a compute platform across different OEMs. [39:53] And we also tried, I'm very happy to announce our partnership with Uber towards L4. [40:01] And by 2028, we agreed to deploy L4 technologies in 28 cities globally. [40:12] We are also basically, obviously NVIDIA is not only an AV company, you know, of course we are not only a chip company, we are a full stack AI company, and we are using our AI capabilities to work with OEM globally on many different areas as well, including basically using Omniverse Digital Twin for design and manufacturing and enterprise. [40:43] And this is the ecosystem we are working with. [40:47] Autonomy is a daunting task. [40:49] It's a grand mission, but it really needs everybody to work together. [40:54] And we are very happy to work with everybody in the automotive, basically, industry, including OEMs, tier ones, or AV software developers, to work together to get from 0.006% to upper 90s. [41:07] This is the mission. [41:14] All right, I think I'm running a little bit out of time, but this is the key takeaways of what I talked about today. [41:22] Again, Drive AV is working, sorry, NVIDIA Drive is getting into a new phase, three computers. [41:31] Five layers of service, and we are trying to basically, on the Hyperion side, we're trying to get as many as possible EMs to share the same sensor in a computer platform, and we are also trying to work with Uber to create a large data set for the ecosystem developer AV. [41:58] And we are bringing halos, we are enhancing halos significantly from just the operating system to basically a safety guard rail so that everybody can deploy end-to-end or AV stack on Hyperion, our hardware platform. [42:20] And we announced Alpamario 1.5, which is the next generation of Alpamario and also basically on the application side, NVIDIA is working super hard to take Drive AV to the next step, which is really basically this year it's about scaling of L2++ and also get the technology extended to L4. [42:46] And on the infrastructure side, because simulation and the validation with AI technology is super important, we are also open sourcing, you know, our foundation models and also new rack and the Cosmos transfer models so the industry can help, can use to enhance their data set and accelerate their AV development. [43:11] So this is what I want to talk about today. [43:14] Thank you for your time, and hopefully you can enjoy the rest of the afternoon.