Nvidia Prepares 'World Action Models' to Skip Robot Teleoperation

英伟达筹备“世界动作模型”以跳过机器人遥操作 英伟达预测，通过将视频和动作视为一等公民来复刻大语言模型的指数级扩展定律，人类将在2040年前达到机器人技术树的绝对终点。 在本次演讲中，Jim Fan概述了英伟达通过模仿大语言模型的演进阶段来攻克机器人技术的战略路线图。他介绍了从视觉-语言-动作模型向“Dreamer”等世界动作模型的转变，后者通过视频预测扩展直接学习物理规律。为了突破数据瓶颈，英伟达利用Dex-UMI等人类可穿戴设备以及“Dream Dojo”等神经模拟器，来满足灵巧操作的对数线性扩展定律。最终，Fan描绘了一副文明级别的技术树，预测到2040年将实现物理图灵测试、软件配置的机器人机队以及自动自我改进的硬件。 英伟达 Jim Fan 提出机器人学的“大平行”（The Great Parallel）路线，主张直接复刻 LLM 从预训练、对齐到强化学习与自主研究的发展路径，以世界模型替代语言模型加速具身智能演进。 (0:02) 针对传统 VLA 模型过度偏向语言语义而缺乏物理理解的短板，英伟达推出世界动作模型（WAM）Dreamer，通过视频生成预训练模拟物理世界，并结合动作微调实现端到端的物理任务执行。 (0:31) 为突破人类遥操作的数据采集瓶颈，英伟达采用 Dex-UMI 外骨骼及 EgoScale 框架，基于 2.1 万小时第一人称人类视频预训练，仅需不足 0.1% 的机器人本体数据即可掌握高难度灵巧操作。 (7:43) Fan 预计机器人将在 2 至 3 年内通过物理图灵测试，并有 95% 的把握在 2040 年前彻底点满包含物理 API 与自主迭代研发在内的机器人科技树。 (16:59)

Jim Fan, leader of Nvidia's embodied autonomous research group, argues that robotics can match the rapid ascent of large language models by adapting their developmental playbook—replacing text tokens with world-state predictions, scaling human demonstration data via wearables, and generating reinforcement learning environments through neural simulation. Shift from Language Models to World Action Models (0:31) Fan contends that prevailing Vision-Language-Action (VLA) models such as RT-2 and PaLM-E dedicate disproportionate parameter capacity to language processing, making them proficient at semantic nouns but fundamentally weak at physical verbs and dynamics. In contrast, large-scale video generative models like Sora and VEO3 implicitly learn foundational physical rules—such as gravity, buoyancy, reflection, and lighting—simply by predicting subsequent pixels across massive visual datasets. To bridge generative video and physical control, Nvidia developed "Dreamer," categorized as a World Action Model (WAM). Dreamer jointly decodes prospective visual world states alongside high-dimensional, continuous motor actions. By visually projecting its actions several seconds into the future, the model executes zero-shot manipulation tasks, succeeding when its internal video simulation accurately predicts the physical interaction. Scaling Demonstration Data via Wearable Human Interfaces (7:43) Traditional robotic teleoperation faces a severe throughput ceiling, bounded by hardware downtime and human labor limits to roughly three usable recording hours per robot each day. To scale data capture without physical robots, researchers use data wearables such as the Universal Manipulation Interface (UMI) and Nvidia's Dex-UMI exoskeleton, which maps human finger motion directly to five-finger dexterous robot hands to gather manipulation data at lower cost and higher speed. Nvidia deployed this wearable data strategy in its "Ego Scale" framework, which was pre-trained on 21,000 hours of egocentric human video with zero robotic demonstrations. Fine-tuning required only 50 hours of motion-capture glove data and 4 hours of teleoperation—less than 0.1 percent of the total dataset—to execute delicate tasks like sorting cards, handling syringes, and folding shirts. Fan notes that this setup demonstrates a clean log-linear scaling law between egocentric video pre-training hours and validation loss. Generating Parallel Environments with Neural Simulation (13:59) While physical reinforcement learning can push specialized tasks—such as automated GPU assembly—to a 100 percent success rate, scaling hardware across millions of diverse training environments is impractical. Nvidia addresses this through a "real-to-sim-to-real" pipeline, converting smartphone 3D room scans into physics engine assets and procedurally generating synthetic variations termed "digital cousins." To bypass the limits of standard physics engines entirely, Nvidia introduced "Dream Dojo," a data-driven neural simulator. Rather than executing mathematical physics equations, Dream Dojo accepts continuous action signals and outputs real-time RGB video and sensor states directly from data. This architecture allows reinforcement learning to run concurrently across real hardware stations and massive inference compute, operationalizing the principle that compute directly substitutes for physical training environments. The Robotics Technology Tree and Projected Milestones (16:59) Fan outlines the trajectory of robotics as a three-stage civilizational technology tree. The first milestone is a "physical Turing test," where a robot's dexterity and energy efficiency are indistinguishable from a human's, which Fan estimates is two to three years away. The second milestone is a "physical API" allowing software commands to orchestrate robot fleets in fully autonomous, lights-out manufacturing plants and automated wet laboratories. The final milestone is "physical auto research," wherein robots autonomously design and assemble future generations of hardware. Comparing the field's trajectory from AlexNet in 2012 to agentic systems in 2026, Fan highlights the compounding effect of exponential growth over linear projections. Based on this historical curve, Fan forecasts with a stated 95 percent certainty that the robotics technology tree will be fully achieved by 2040. Fan's framework hinges on the premise that physical dynamics can be largely solved through video pre-training and neural simulation. However, his aggressive timeline—including achieving a physical Turing test within two to three years and reaching the end of the robotics technology tree by 2040—represents an optimistic forecast that assumes hardware scaling and neural simulators can reliably overcome real-world physical edge cases without severe physical bottlenecks.

英伟达具身智能研究负责人Jim Fan主张，机器人学应当复刻大语言模型（LLM）从状态模拟、动作对齐到强化学习的演进路径，通过世界动作模型与穿戴式数据采集技术攻克具身智能的物理终局。 从语言动作模型转向世界动作模型 (0:31) 过去三年具身模型主要由视觉-语言-动作（VLA）架构主导，如PaLM-E与RT-2。Fan指出这类模型本质上属于语言优先的架构，其参数空间高度偏向语言模块，长于处理名词语义关联（例如将可乐罐移动到泰勒·斯威夫特照片旁），但在物理常识与动词交互推理上存在明显短板。与此同时，Sora和VEO3等互联网规模视频生成模型的进展表明，大规模预测下一状态像素能够内生涌现重力、浮力、反射和折射等物理法则，无需人工编写物理规律。 为将物理模拟转化为行动控制，英伟达提出了世界动作模型（WAM）“Dreamer”。Dreamer通过动作微调联合解码下一世界状态与高维连续电机动作，使机器人在脑海中“预演”未来数秒的物理演变并执行零样本物理任务。在该机制下，机器人动作的成败直接取决于视频预测的准确性：预测精准则动作成功，若视频发生幻觉则执行失败。 外骨骼与人眼视频打破遥操作瓶颈 (7:43) 数据采集长期受制于物理瓶颈。Fan指出，传统机器人遥操作受限于单机每日24小时的物理上限，且由于机械故障频发，单机单日实际有效采集时间通常仅约3小时，人力成本极为高昂。为此，通用操作界面（UMI）以及英伟达研发的Dex-UMI五指外骨骼穿戴设备，允许操作员在脱离机器人本体的情况下直接通过双手采集灵巧操作数据，显著提升了采集速度与动作成功率，且训练过程无需传统遥操作数据在环。 在预训练层面，英伟达推出Ego Scale框架，在包含2.1万小时自然人类第一视角视频（Ego-Exo）且完全没有机器人数据的基础上完成预训练。该方案仅额外加入50小时动捕手套数据与4小时传统遥操作数据（占比不足总训练混合数据的0.1%），即在分拣卡片、操作注射器和折叠衬衫等精细任务中实现了自主控制策略。研究人员同时发现，人类第一视角视频预训练时长与验证损失之间遵循严格的对数线性神经缩放定律。 虚实迁移流程与神经仿真器环境扩展 (13:59) 针对后训练强化学习所需的海量交互环境，英伟达推行“真-仿-真”（Real-to-Sim-to-Real）工作流。尽管在真实机台上直接运行强化学习可使特定硬件装配任务（如自主组装GPU）达到100%成功率，但物理硬件无法线性扩展至数百万个并行环境。该工作流支持通过手机拍摄房间照片，经由3D世界扫描管线提取可交互物体，在传统物理仿真器中自动合成场景，并无限生成物理属性各异的“数字表亲”对象以增加训练泛化度。 为进一步摆脱传统物理与图形引擎的计算限制，英伟达开发了纯数据驱动的神经仿真器“Dream Dojo”。Dream Dojo接收连续动作信号作为输入，无需物理方程或图形着色管线即可实时输出RGB视频帧和传感器状态。该系统使强化学习能够在真实机器人工作站、图形计算核心及高负载推理算力之间大规模并行展开，贯彻了“算力即环境即数据”的后训练理念。 技术树三大里程碑与2040年终局预测 (16:59) Fan将机器人技术突破比作策略游戏《文明》中的科技树，并划定了通往终局的三大关键里程碑：首先是达到单位能量输入对应同等人力输出、旁观者无法区分人机动作的“物理图灵测试”，Fan预估该目标将在2至3年内实现；其次是提供支持软件调度机群的“物理API”，从而驱动将设计图纸直接转化为实物的“暗灯工厂”与无人化化学/生物湿实验室；最终目标则是实现机器人全自主设计、改进和制造下一代机体的“物理自主研究”。 对比2012年AlexNet初步识别猫狗到2026年行业普遍研讨自主智能体研究的发展轨迹，Fan认为技术进步呈现指数级而非线性加速。以2026年为中点向前推演14年，Fan声称有95%的把握认为人类将在2040年彻底点亮机器人技术树的终点，在当代研究者的生命周期内解决通用具身智能问题。 该报告系统阐述了英伟达将LLM与视频生成范式向物理实体迁移的模型与数据战略。然而，将终极技术树攻克时间定为2040年且给出95%置信度属于演讲者的主观预测；世界动作模型在应对复杂长程物理交互时的视频幻觉抑制，以及极低比例遥操作数据在开放物理环境中的安全边界，仍是待实际工程检验的核心变量。

[0:02] Jensen Huang: To Elon and the OpenAI team, to the future of computing and humanity... [0:02] Ilya Sutskever: If you believe in deep learning, deep learning will believe in you. [0:02] Jim Fan: speed running AGI [0:02] Jim Fan: the great parallel [0:31] Jim Fan: AI video slop [0:31] Jim Fan: head-heavy in the wrong places [7:43] Jim Fan: Given his salary, I think this is by far the most expensive teleop trajectory ever collected. [7:43] Jim Fan: throwing tantrums [13:59] Jensen Huang: The more you buy, the more you save. [13:59] Jim Fan: Compute = Environments = Data. [16:59] Jim Fan: civilizational technology tree [16:59] Jim Fan: lights-out factories [16:59] Jim Fan: printers of atoms that turn markdown files into physical products [18:51] Jim Fan: born just in time to solve robotics [18:51] Jim Fan: If you believe in robotics, robotics will believe in you.

[0:02] And up first, I'm delighted to introduce [0:04] my friend Jim Fan. [0:06] Uh Jim leads the embodied autonomous [0:08] research uh group at Nvidia, otherwise [0:10] known as Nvidia robotics. [0:12] Um [0:13] I think that robot robots are just one [0:15] of the most thrilling things that's [0:16] going to happen. Uh a car basically is a [0:18] big robot, but I'm excited for robots [0:19] can go beep boop and lift things for us. [0:22] And so, Jim was Jim was a standout at [0:24] last year's AI Ascent, and we're [0:26] delighted to have you back. [0:28] Thanks, [applause] everyone. Thanks. [0:30] So, it was a summer day in 2016. [0:32] Actually, right in this office that [0:35] we're sitting in. [0:36] >> [snorts] [0:37] >> There's a guy in shiny leather jacket, [0:39] you know, big biceps, hurling this large [0:42] metal tray. [0:43] And on this [0:45] large piece of metal, he wrote, "To Elon [0:48] and the OpenAI team, to the future of [0:50] computing and humanity, I present you [0:52] the world's first DGX-1." [0:55] So, that was the first time I met [0:56] Jensen. [0:57] And as any good intern would do, I [1:00] rushed to getting line to sign my name [1:02] on it. [1:03] So, can you spot it, my name? [1:05] It's here. And can you spot another? [1:08] That's Andre, right there. [1:09] So, Andre, we're going to the Computer [1:11] History Museum. [1:13] I feel like a dinosaur. [1:16] You know, back then, I had no clue what [1:19] I was signing up for. [1:21] And then, no one can describe what [1:23] happened next better than Ilya himself. [1:26] If you believe in deep learning, deep [1:28] learning [1:29] will believe in you. [1:31] And oh boy, did deep learning believe in [1:33] all of us big time. [1:36] Three step functions, six years. [1:39] That's how all it took to bring us here [1:41] today. [1:42] The first tick, GPT-3, [1:44] pre-training. [1:46] Next token prediction is really about [1:48] learning the rules of grammar, the shape [1:50] of language. It's about simulating how [1:53] thoughts and code and strings in general [1:56] should unfold. [1:57] 2022 Instruct GPT [2:00] supervised fine-tuning aligned the [2:02] simulation to do useful work. [2:05] 01 reasoning [2:07] using reinforcement learning to surpass [2:09] imitation learning and finally auto [2:11] research accelerating the whole loop [2:14] beyond what's humanly possible. [2:17] So, as Andre said [2:18] all the labs are getting to the final [2:20] boss fight. [2:22] So, for LLMs [2:24] they're in the thick of the end game. [2:27] And honestly, I'm very jealous. [2:29] Look at how happy Andre was, big smile [2:31] on his on his face. [2:33] The LLM folks [2:35] are having the party of their lifetime. [2:37] They're speed running AGI on mystical [2:40] creatures literally called mythos. [2:43] So, why can't robotics get a piece of [2:45] fun? [2:47] So, as any self-respecting scientist [2:49] would do [2:50] I copy homework and I give it a new [2:52] name. [2:53] I call it the great parallel. [2:56] So, instead of simulating strings [2:58] can we simulate next physical world [3:00] state? [3:02] And then we can align through action [3:03] fine-tuning [3:04] onto a thin slice of that simulation [3:06] that matters for real robots. [3:09] And we let reinforcement learning carry [3:12] the last mile. [3:14] And that's it. [3:16] The great parallel copying the LLM [3:17] success. [3:18] If you can't beat them, join them. [3:20] So, please join me in a new episode [3:23] robotics the end game. [3:26] I'm sorry, I just couldn't resist. Nano [3:27] bananas too good. Thanks, Demis. [3:31] So, how do we play the end game? It [3:33] boils down to two things, model strategy [3:35] and data strategy. [3:37] Let's look at the model first. [3:39] The last 3 years [3:41] were dominated by VLAs or visual [3:44] language action models and models like [3:46] PaLM and Goot fall in this category. [3:49] So, we assume that the pre-training is [3:51] done by a VLA [3:52] and we simply graph an action head on [3:54] top of it. [3:56] But really, if you think about these [3:57] models, they're LVAs because the most [4:00] amount of parameters are dedicated to [4:02] language. So, language is first-class [4:05] citizen followed by vision and action. [4:08] And by design, VOAs are great at [4:10] encoding knowledge and nouns, but not so [4:13] much at physics and verbs. It's kind of [4:16] head-heavy in the wrong places. [4:18] This is my favorite example from the [4:20] original VOA paper. [4:22] Move the Coke can to a picture of Taylor [4:24] Swift. Yes, it has not seen Taylor Swift [4:26] before. Yes, it's able to generalize. [4:29] But this is not quite the pre-training [4:30] ability that we're looking for. [4:32] So, what's the second pre-training [4:34] paradigm? [4:35] And I always thought that it would be [4:37] something glorious. [4:39] Unfortunate, it turns out [4:42] that this is AI video slop that we call. [4:45] You know, I can watch these cats playing [4:47] banjo on security cam all day. It's peak [4:49] internet. [4:51] But really, [4:53] look at this. No one can take this [4:54] seriously [4:56] >> [laughter] [4:56] >> until we realize that these video models [4:59] are learning to simulate next world [5:01] state internally. [5:03] So, these are some rollouts from VEO3. [5:06] You can see that the models, they pick [5:07] up gravity, buoyancy, lighting, [5:10] reflection, refraction all by [5:12] themselves. None of this is coded in. [5:15] Physics emerge by predicting the next [5:18] blob of pixels at scale. [5:21] And even visual planning emerges. [5:23] Look at how VEO solves these mazes. [5:27] It solves them by running simulation [5:29] forward in pixel space. [5:32] And draw attention to the lower right [5:34] corner here. This is my favorite [5:35] example. Let's watch. And you blink if [5:37] you miss how VEO3 solves this one. [5:41] >> [laughter] [5:43] >> It's super smart. You know, VEO3 figures [5:45] out that if you're not looking, geometry [5:47] is optional. [5:49] I call this physics law. [5:53] So, how do we make these world models [5:55] useful? [5:56] Well, [5:57] we do action fine-tuning. [5:59] We align the superposition of all [6:02] possible future states and [snorts] [6:04] collapse that onto a thin slice that [6:07] matters for real robots. [6:09] Introducing Dreamer. [6:12] It's a new type of policy model [6:14] that dreams a couple seconds into the [6:16] future and acts accordingly. [6:19] And you know that motor actions, they're [6:21] high-dimensional continuous signals. [6:24] So, that looks just like pixels. [6:26] We can render it at the same time as we [6:28] render the videos. So, Dreamer jointly [6:32] decodes the next world states and next [6:34] actions. [6:35] And as a result, it's able to zero-shot [6:38] solve tasks [6:39] and verbs that it has never seen in [6:41] training. [6:44] And as a robot executes, we can [6:45] visualize what it's dreaming about. [6:47] And the correlation is very tight. [6:49] If the video prediction works, the [6:51] action works. [6:52] If the video hallucinates, the action [6:54] fails. [6:55] So, once again, vision and action are [6:57] now first-class citizens. [7:00] And we have a lot of fun with Dreamer. [7:02] So, we just roll the robot around in our [7:04] lab and then type random things into the [7:06] prompt box. [7:08] And of course, Dreamer is not going to [7:10] get all of these tasks 100% robust, but [7:13] it's kind of like GPT-2. [7:15] It's trying to get the shape of the [7:16] motion correct in every case. [7:19] So, Dreamer is our first step towards [7:22] open-ended, open-vocabulary prompting [7:25] for robotics. [7:27] And we [snorts] call this new type of [7:28] model world action models or WAM. [7:32] So, let's all take a moment of silence [7:35] for our dear friend VOS. [7:37] They served us well. Rest in peace. [7:40] Long live world action models. [7:43] >> [clears throat] [7:43] >> And next, data strategy. [7:46] This is Nvidia's chief scientist, Bill [7:49] Dally, operating teleoperation inside [7:51] our lab. [7:53] And given his salary, I think this is by [7:55] far the most expensive teleop trajectory [7:57] ever collected in our data set. [8:01] The past 3 years have been dominated by [8:04] teleoperation. It's the golden era. [8:07] All right, VR headsets, [8:09] extremely optimized [8:11] latency for streaming, and these complex [8:14] rigs that look like medieval torture [8:15] devices. [8:17] You know, so much investment in [8:18] industry, so much pain and suffering. [8:21] And yet, for teleop, it's upper bounded [8:24] by 24 hours per robot per day, the [8:27] fundamental physical limit. [8:28] And actually, who am I kidding? It's [8:30] more like 3 hours per robot per day, and [8:32] only when the robot god is merciful, [8:34] because they throw tantrums all the [8:36] time. [8:37] So, how can we do better? [8:39] Well, how about this? [8:41] You just wear the robot hand on your own [8:44] hand. [8:45] So, this is called UMI, or universal [8:47] manipulation interface, and it's a [8:49] deceptively simple idea. [8:51] You wear the robot actuator on your hand [8:54] and directly collect the data as humans, [8:56] while getting the rest of the robot body [8:59] out of the loop. [9:01] Yet, I will say UMI is perhaps one of [9:03] the greatest papers ever written in [9:04] robotics data, [9:06] and it spawned two unicorn startups. [9:09] On the left hand side is Joles improving [9:11] this design, so you can wear the gripper [9:12] here, and on the right hand side, Sunday [9:15] made these three-finger data gloves. [9:18] So, last year, we took a one step [9:20] further. [9:21] We designed this exoskeleton that has a [9:23] one-to-one mapping with five-finger [9:25] dexterous robot hands, and we call it [9:27] Dex-UMI. [9:28] Let's look at it in action. [9:31] On the left, [9:32] the human directly collecting the data [9:34] always the fastest. [9:35] On the right, look at how difficult [9:37] teleop is. [9:38] Right? The human operator, here one of [9:40] our most skilled PhDs, he has to align [9:43] very carefully, right? And then it's [9:45] super slow. Also, the success rate is [9:48] very low as well. [9:49] And in the middle, you just wear this [9:51] exoskeleton and you collect data [9:53] directly. [9:55] And we train a robot policy on this [9:57] data. [9:58] So, here what you see is a fully [9:59] autonomous [10:01] rollout of a policy that's trained on [10:03] zero teleoperation data. [10:06] So, we're able to break the curse of 24 [10:09] hours per robot per day and see how [10:11] happy these robots are because they no [10:13] longer need to be in the loop for data [10:15] collection. [10:16] So, is this the answer? Have we solved [10:18] scaling for robotics? [10:21] Anyone driving Tesla or Waymo here? [10:23] Anyone? [10:24] Right? [10:26] You know, when you're driving, [10:27] you're actually contributing to the [10:30] biggest physical data flywheel. [10:33] And the beauty is you don't even feel it [10:35] during FSD because the data upload is an [10:38] ambient process. [10:40] Yet, wearing these old me or data [10:41] wearables, it's still cumbersome, right? [10:43] It's intrusive. It's not as seamless as [10:46] just driving to work. [10:48] So, we need [10:50] an FSD equivalent. [10:53] The data collection needs to get out of [10:55] the way, fade into the background so we [10:57] can capture the full glory of human [10:59] dexterity across all walks of lives, [11:02] across all labors of economic value. [11:06] So, we're going all in on human [11:08] egocentric videos that come with these [11:10] detailed annotations like hand position [11:13] tracking and dense language annotations. [11:17] Introducing Ego Scale. [11:20] Where 99.9% [11:21] of the training that goes into this is [11:23] based on human egocentric videos. [11:26] And the result is an end-to-end policy [11:29] that maps directly from the camera [11:30] pixels here to 22 degrees of freedom [11:33] high dexterity robot hands. [11:35] What you see here is fully autonomous. [11:39] We pre-train Ego-Exo on 21K hours of [11:43] in-the-wild egocentric human data [11:46] with zero robot data whatsoever. [11:49] And during pre-training, we predict [11:50] these hand joints and wrist poses. [11:54] Then action fine-tuning, we collect only [11:56] 50 hours of high precision mocap data [11:59] gloves and 4 hours of teleop. [12:02] That's 4 hours of teleop. [12:04] Less than 0.1% [12:06] of our training mix. [12:09] And with this, Ego-Exo is able to [12:11] generalize to these very dexterous tasks [12:13] like sorting card [12:15] or manipulating syringe, [12:17] right? Over transferring the liquid, you [12:20] know, someday we might have robot nurses [12:22] at home. Might as well try this. [12:25] And for these tasks, it takes only one [12:27] shot demonstration at test time to learn [12:29] different shirt folding strategies. [12:33] And perhaps the most fascinating finding [12:35] from the paper [12:36] is that we discovered this neural [12:39] scaling law [12:40] for dexterity. [12:42] It's a very clean relationship between [12:44] the amount of hours we put into [12:46] pre-training and the optimal validation [12:48] loss. In fact, it's a clean log-linear [12:52] mathematical equation. [12:53] 6 years after the original neural [12:55] scaling law for language models. [12:59] So, if we put all of these data [13:00] strategies on this chart, X axis is [13:03] alignment to the robot hardware, Y axis [13:05] is scalability, this is what it looks [13:07] like. Teleop, the least scalable. Data [13:10] wearables, you can go up to hundreds of [13:12] thousands of hours. [13:14] And egocentric video, if we're able to [13:16] spin the FSD flywheel, easily 10 million [13:19] hours in the next year or so. [13:22] And if we draw a line here, everything [13:24] to the left of this line is the [13:26] paradigm, sensorized human data. [13:29] So, let me make a few predictions. [13:31] In the next year or two, we'll see [13:33] Teleop dropping and dropping to almost [13:35] negligible amount. [13:37] And then, there will be an ensemble of [13:40] data wearables custom-designed for [13:42] different hardware and use cases. [13:44] And finally, the main diet for robotics [13:46] will be egocentric videos. [13:49] So, a moment of silence for our dear [13:52] friend Teleop. You have served us well. [13:54] Rest in peace. Long live sensorized [13:56] human data. [13:59] Are we done with the data strategy yet? [14:01] Did you notice I put two rings on data [14:02] strategy? [14:04] What's the outer ring here? [14:06] All the old and frontier labs [14:09] have spent significant budget now on [14:11] acquiring millions of coding [14:13] environments to do reinforcement [14:14] learning. [14:15] So, robotics is the same. We're in [14:17] urgent need to scale up environments. [14:20] And of course, you can always [14:23] do reinforcement learning directly on [14:24] the real robot. [14:25] So, in our lab, we use RL to push [14:27] certain tasks to almost 100% success [14:30] rate, so you can do these continuous [14:32] execution for hours on end. [14:35] You know, it's kind of therapeutic to [14:36] see these robots assembling GPUs just by [14:38] themselves. Or as a wise man would say, [14:40] "Good boy, this task has been approved [14:42] by my boss." [14:45] Yet, we can't get [snorts] to 1 million [14:48] environments because that would require [14:50] 1 million robots if you do it the [14:51] previous way. [14:53] So, we need a better way. [14:55] Here, let's say you take an iPhone [14:57] picture [14:58] and you can pass it through this 3D [15:01] world scan pipeline to extract all the [15:03] objects and then automatically [15:06] synthesize them again inside a classical [15:08] physics simulator. So, all these objects [15:11] are actually interactive after the scan. [15:14] And then, you can augment this [15:15] infinitely in simulation with variations [15:18] that we call digital cousins. [15:21] So now, iPhone basically become [15:24] a pocket world scanner. [15:26] In this process that we call real to sim [15:28] to real, and in this way, we have a [15:31] scalable way to port the physical world [15:33] into the digital world. [15:35] But still this method relies on a [15:37] classical [15:38] graphics engine. Can we do better? [15:41] Introducing [15:42] Dream Dojo. [15:44] So, it's our spin on video world model [15:46] and turning them into full-fledged [15:49] neural simulators. [15:51] Dream Dojo takes us input these [15:53] continuous action signals and outputs [15:56] the next RGB frames as well as sensor [15:58] states in real time. [16:00] Not a single pixel you see here is real. [16:03] And Dream Dojo is able to capture and [16:05] learn the mechanics of different robots [16:07] through a purely data-driven approach. [16:10] There's no physics equation, no graphics [16:12] engine involved in this process. [16:16] So, the new post-training paradigm for [16:18] robotics is a massively parallel RL [16:21] system [16:22] that runs on a few real robot stations, [16:25] a bunch of graphics cores running world [16:27] scans, and heavy inference compute [16:29] running world models. [16:32] Or, as this equation goes, compute now [16:35] equals environment now equals data. Or, [16:39] as a wise man would say, [16:40] the more you buy, the more you save. [16:43] And this message has been approved by my [16:44] boss. [16:47] So, that's it. Putting it together, [16:49] the great parallel that robotics will [16:51] follow. And it's happening as we speak. [16:54] And we're looking at the beginning of [16:56] the end game. [16:59] You guys play the video game [17:01] Civilization? [17:03] Still my favorite. [17:05] I like to think of my research as [17:07] unlocking game achievements on the [17:09] civilizational technology tree. [17:12] >> [clears throat] [17:13] >> And there are three more achievements to [17:15] unlock for robotics, and then we're [17:16] done. [17:17] I can retire, and I can't wait for that. [17:20] The first [17:22] is passing the physical Turing test. [17:25] Across a wide range of activities, you [17:28] cannot tell the difference between a [17:31] human doing a task or a robot doing it. [17:33] Maybe not drunk humans, but [17:36] you know. [17:37] Physical Turing test is about unit [17:39] energy in and unit labor out. [17:43] And just by judging at the sexy pose of [17:45] this robot, I think the work is cut out [17:46] for us. [17:47] So, maybe it's 2 to 3 years away. [17:51] And next, physical API. [17:53] You have a whole fleet of robots, [17:55] and they can be configured just like any [17:57] other software using APIs and command [17:59] lines, orchestrated someday by Opus 9.0. [18:04] And if we have this physical API, we'll [18:06] be able to realize lights-out factories. [18:09] Those are essentially printers of atoms. [18:11] They take us input design in markdown [18:14] files, and then output fully assembled [18:16] products, completely autonomous. [18:19] Or these wet labs [18:21] that automate scientific discoveries in [18:24] chemistry, [18:25] biology, and medicine. [18:28] And the final stop, physical auto [18:30] research. When the robots start to [18:33] design, improve, and build the next [18:35] iteration of themselves, far beyond [18:38] what's humanly possible. [18:40] So, you might ask, is this too science [18:42] fiction? Like, are we going to see this [18:44] in our lifetime? [18:46] Well, [18:48] it took the AI community [18:50] 14 years [18:51] to go from the first forward pass of [18:53] AlexNet in 2012, [18:55] a model that barely recognized cat [18:57] versus dog, [18:58] to AI ascent today, 2026, where we talk [19:01] about agentic auto research. [19:04] And let's just add another 14 years. How [19:06] about that? [19:08] 2026 is right in the middle of 2012 and [19:12] 2040. [19:14] And technology does not advance [19:15] linearly. It advances exponentially. [19:19] So, [snorts] I can say with 95% [19:22] certainty that we'll get to the end of [19:24] the end game, [19:25] the end of the technology tree, [19:28] by 2040. [19:30] And we'll still be all We'll still be [19:32] young. [19:34] If you believe in robotics, robotics [19:36] will believe in you. [19:38] And to all of us here, [19:40] sitting here, [19:41] I think our generation [19:43] was born too late to explore the Earth [19:46] and too early to explore the stars, but [19:48] we are born just in time to solve [19:51] robotics. [19:53] Thank you. [19:54] >> [applause] [19:58] [applause]