Runway, Niantic, Odyssey Scale World Models for Physical AI
Ai4
AI leaders argue that scaling video models and real-world ground-truth data will bypass costly physical trial-and-error to unlock automation across the physical economy.
## Background
This panel discussion, titled "AI That Understands The Real World," was published by the channel Ai4 on YouTube on August 21, 2026. The video features industry leaders discussing how artificial intelligence is transitioning from generating digital content to understanding space, motion, physics, and three-dimensional environments through world models and spatial intelligence.
## TL;DR
Moderated by Jared Perlo, the panel brings together Anastasis Germanidis (Runway), Inhi Cho Suh (Niantic Spatial), and Oliver Cameron (Odyssey) to explore the evolution of world models beyond language-centric AI. The speakers discuss concrete applications, such as urban 3D digital twins, synthetic video datasets to overcome data bottlenecks in robotics, and infinite simulation environments. They compare data strategies for physical AI—ranging from observing human tasks to utilizing multi-layer enterprise sensory data—while debating model architectures and scaling paradigms. Ultimately, the discussion highlights how bridging the gap between digital simulation and physical reality will drive real-world automation, enterprise efficiency, and spatial reasoning.
## Atmosphere & style
The panel carries an intellectually sharp, forward-looking, and pragmatic tone, blending high-level research concepts with practical industrial applications. The dynamic between moderator Jared Perlo and the three founders feels collaborative and deeply informed, as each panelist draws on distinct expertise—from autonomous driving and spatial mapping to creative video models. It feels like an engaging, high-level industry briefing where complex technical debates around AI scaling laws and physical simulation are grounded in real-world economic realities.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=b47WtyhqpPk&t=4s) AI is undergoing a fundamental shift from language comprehension to understanding the physical three-dimensional world, enabling applications in robotics, autonomous systems, and spatial intelligence.
- [01:41](https://www.youtube.com/watch?v=b47WtyhqpPk&t=101s) World models are defined differently across industries, ranging from predictive foundation models for autonomous systems to high-fidelity 3D digital twins and scalable video generation models.
- [06:44](https://www.youtube.com/watch?v=b47WtyhqpPk&t=404s) Video world models overcome traditional physics simulators by learning complex physical dynamics and non-rigid behaviors directly from large-scale pixel data.
- [06:44](https://www.youtube.com/watch?v=b47WtyhqpPk&t=404s) Generating synthetic video variations directly solves the primary data availability bottleneck facing robotics training without relying on expensive physical hardware.
- [11:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=689s) Human observation footage allows video models to build intuitive physics knowledge that can be transferred to physical robots using minimal target-specific data.
- [12:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=749s) Physical AI relies heavily on secure, private multi-layer enterprise data, particularly for GPS-denied complex environments like industrial refineries.
- [13:40](https://www.youtube.com/watch?v=b47WtyhqpPk&t=820s) General world models eliminate the need for thousands of hours of specialized domain training by leveraging broad prior visual and physical world knowledge.
- [18:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1109s) Simple, data-scalable model architectures trained on next-token prediction consistently outperform complex hand-designed frameworks as compute and data scale up.
- [22:00](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1320s) Grounding digital simulations in real-world visual and sensory data is essential for effective sim-to-real workflows and localized operational compliance.
- [24:20](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1460s) The single greatest bottleneck in physical AI deployment is generalization, as specialized robotic policies fail under subtle real-world layout changes and long-tail conditions.
- [29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) World simulators provide the missing compilation and execution sandbox for the physical world, unlocking the majority of the global economy anchored in physical operations.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=b47WtyhqpPk&t=4s) Introduction and the Shift to World Models
The panel opens with an introduction framing a fundamental shift in artificial intelligence: after decades of excelling at language understanding, AI is now learning to comprehend the physical world. World models grant AI the ability to understand space, movement, and three-dimensional environments, unlocking applications across robotics, autonomous systems, and creative tools.
The session features three industry leaders who are shaping this domain:
* **Anastasis Germanidis**: Co-founder and Co-CEO of Runway, an applied research lab merging art and science. Runway built the first publicly available AI video model and operates the largest world model deployment in production.
* **Inhi Cho Suh**: CEO of Niantic Spatial, who is developing real-world foundational models for physical AI so humans and machines can work together safely and efficiently.
* **Oliver Cameron**: Co-founder and CEO of Odyssey, who previously helped launch the first fully driverless service in San Francisco and is now building general-purpose world models to predict and interact with the physical world.
---
[01:41](https://www.youtube.com/watch?v=b47WtyhqpPk&t=101s) Defining World Models Across Different Domains
Moderator Jared Perlo sets the stage by noting that "world models" can mean many different things depending on the application, prompting each founder to share their background and operational definition.
* **Oliver Cameron (Odyssey)**: Recalls entering the driverless car space in 2015, when systems were too primitive because they could not accurately predict the world. Over seven years, scaling AI models and datasets made it possible to predict the trajectories of pedestrians and vehicles with superhuman accuracy. Odyssey was founded on the premise of taking that predictive capability and expanding it into general-purpose foundation models that simulate both virtual and physical environments for robotics, autonomous vehicles, drones, gaming, defense, and healthcare.
* **Inhi Cho Suh (Niantic Spatial)**: Shared that after decades in enterprise software, a trip to Mount Everest inspired her transition into physical AI. Experiencing the Himalayas firsthand made her realize that screen-based digital work and static photographs fail to capture real-world context and presence. Niantic Spatial focuses on building real, high-fidelity 3D models rather than pure synthetic simulations.
* **Anastasis Germanidis (Runway)**: Founded Runway eight years ago on the core belief that useful intelligence must learn directly by observing the world, as human language alone cannot capture everything humans care about. Runway began with video and visual data, discovering that as video generation models are scaled with more data and compute to predict pixels, they naturally learn physics, object behavior, and complex human tasks.
> "Language does not capture everything that we as humans care about... we learn as we grow up from observation by trying different things in the world and seeing what happens." — Anastasis Germanidis
---
[06:44](https://www.youtube.com/watch?v=b47WtyhqpPk&t=404s) Concrete Applications: Digital Twins, Robotics, and Infinite Simulators
Moving past abstract terminology, the panel highlights high-value real-world implementations currently in production.
* **3D Digital Twins for Urban Management**: Niantic Spatial partnered with the city of Rancho Cordova (near Sacramento, California), HMC Architects, and NVIDIA to build the first multi-square-mile 3D digital twin of a city. The goal is to simulate complex real-world scenarios including traffic management, urban revitalization, last-mile drone delivery, and fleet routing over vast areas.
* **Overcoming Robotics Bottlenecks**: Germanidis explains that the primary bottleneck in robotics is data availability rather than compute. Collecting physical robot data requires expensive hardware scaling. Video world models solve this by generating synthetic video variations of tasks, lighting, objects, and environments. Furthermore, when fed direct camera feeds from a robot, video models can predict subsequent frames and determine appropriate actions. Unlike traditional physics simulators that struggle with non-rigid materials (such as cloth or fluid dynamics), video models trained on massive datasets simulate complex physics naturally.
* **Infinite Learning Environments**: Cameron references DeepMind's *AlphaStar*, where AI agents reached superhuman performance by playing inside *StarCraft*. However, their intelligence was capped by the constraints of that specific environment. World models provide an "infinite simulator" where AI agents learn without environmental ceilings.
- **Key Data Point**: Runway's platform serves over 50 million creators, filmmakers, artists, and marketers, alongside robotics and life sciences research labs.
---
[11:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=689s) Data Strategies for Physical AI
Addressing the data shortage that plagues large language models (LLMs), the panelists compare their distinct approaches to sourcing, training, and leveraging spatial data.
* **Human Observation for Robotics Transfer (Runway)**: Germanidis shares a surprising finding: video models trained on vast amounts of footage depicting humans doing household tasks, playing sports, or interacting with objects build a rich intuition for spatial motion and physics. Robotics teams can transfer this broad human knowledge to physical hardware using only a few hours of actual robot interaction data (whether captured via egocentric head-mounted cameras or third-person views).
* **Private Enterprise Data & Multi-Perspective Sensing (Niantic Spatial)**: Cho Suh highlights that physical AI relies heavily on non-public, highly secure enterprise data—such as construction sites, shipyards, airports, and industrial facilities. For instance, Niantic Spatial works with major oil and gas companies to map massive oil refineries (some spanning 5x5 square miles). These GPS-denied environments are packed with dense metal scaffolding, valves, and compressors, requiring 3D high-fidelity mesh modeling to track changes and guide maintenance teams. Additionally, physical AI requires multi-layer data captured from the ground, air, drones, and satellites.
* **Rethinking Autonomous Driving Data (Odyssey)**: Cameron argues that current autonomous driving development is fundamentally flawed because it forces models to watch thousands of hours of driving footage. Humans learn to drive around age 17 after only ~30 hours behind the wheel because they bring 17 years of prior world knowledge—understanding basic rules, physics, and visual context. General world models allow AI systems to draw on massive general world knowledge and adapt to driving with only a fraction of specialized driving data.
> "A human does not need thousands and thousands of hours of driving... we spend about 30 hours on average learning to drive... because we understand the basic rules of the road, we understand physics." — Oliver Cameron
---
[18:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1109s) Debating Model Architectures and Data Paradigms
The discussion turns to industry debates—such as Yann LeCun's approach focused on world navigation versus relying purely on video datasets harvested from platforms like YouTube.
Germanidis observes that a major lesson from the recent history of large language models is that the simplest architectures and training paradigms tend to scale the best. He notes that in the mid-2010s, researchers proposed highly complex architectures, but simpler, data-scalable frameworks ultimately prevailed.
<FollowUp label="Want to continue with part 2 of this transcript?" query="Write a detailed, faithful walkthrough of part 2 of the AI That Understands The Real World transcript using the same structure and formatting."/>
[19:31](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1171s) The Case for Scaling Video Models and World Simulators
The discussion centers on the foundational architecture for building intelligent systems capable of reasoning and operating over long horizons. Reflecting on the evolution of large language models, the primary driver of capability was not just novel techniques, but massive scale in compute and data. While small-scale language models struggled to form coherent sentences, scaling next-token prediction enabled them to solve complex software engineering tasks over multi-day periods. Applying this exact same thesis to video models suggests that scaling moving pictures naturally develops a sophisticated internal understanding of physics and real-world simulation. While alternative approaches like JEPA (Joint-Embedding Predictive Architecture) or explicit 3D-based methods show promise, the simplest approach—scaling video models—tends to scale the best.
Beyond simple visual generation, training models on hyper-detailed video data acts as an intensely attentive learner. By observing every micro-interaction and detail across vast datasets, these models will develop a comprehension of physics and reality that surpasses human intuition, ultimately unlocking major scientific and physical breakthroughs.
- At small scales, video and language models produce simple or barely coherent outputs, whereas massive scale unlocks long-horizon reasoning and complex problem-solving.
- Scaling next-token prediction in video serves as a direct analog to the scaling laws that powered language models.
- Alternative world-modeling frameworks currently being explored include Meta's JEPA and dedicated 3D spatial representations.
[22:00](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1320s) Ground Truth, Physical Diversity, and Sim-to-Real Workflows
Transitioning from digital generation to real-world deployment requires grounded physical data. Across oil refineries, warehouses, semiconductor labs, and autonomous farms, every physical space operates under different structural and environmental realities. Because there is only one real world, the most effective strategy begins with capturing ground-truth sensory and visual data from where people work and machines reside, spanning ground sensors, aerial capture, and private enterprise spaces.
Simulations built on top of this real-world data enable key operational optimizations, such as predictive maintenance or region-specific emergency evacuations. For instance, safety evacuations in Southeast Asia differ drastically from Western Europe due to distinct infrastructure and cultural patterns of movement. Furthermore, moving from "sim-to-real" out of the lab requires embedding complex local policies into agentic workflows, considering that drone and robotics regulations vary widely across cities, states, and provinces.
- Industrial environments like semiconductor fabs, agricultural sites, and warehouses represent distinct physical domains requiring tailored ground-truth data capture.
- Emergency evacuation models must factor in localized infrastructure and human behavioral movement patterns.
- Regulatory frameworks for ground robotics and aerial drones remain highly fragmented across different jurisdictions.
[24:20](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1460s) Key Bottlenecks: Exploration, Deployment, and Generalization
When evaluating persistent bottlenecks in world model development, three core challenges emerge: application exploration, scaled real-world deployment, and generalization. Unlike language models, which saw rapid developer adoption early on—such as Jasper utilizing early GPT models for marketing copy—world models have not yet experienced their wide developer exploration phase. Current efforts remain heavily concentrated in obvious domains like autonomous vehicles and classical robotics.
Additionally, scaling deployment into physical spaces presents severe operational friction. Environments with mixed human-machine interactions require rigorous pre-deployment testing rather than deterministic controls. The single greatest barrier to autonomous real-world operation remains generalization. Standard robotic demonstrations often rely on policies overfitted to hyper-specific environments; moving a table by a few inches or altering fine-grained conditions frequently causes these specialized systems to fail. True real-world usefulness demands pre-trained models that can handle the long-tail uncertainty, unpredictable layout changes, and dynamic edge cases inherent to daily life.
- Early language model commercialization was catalyzed by targeted application layers like Jasper for marketing copy generation.
- Overfitted robotic policies perform impressive tasks (like backflips or precision manipulation) in controlled environments but break when minor spatial shifts occur.
- Robust physical intelligence requires large-scale pre-training to resolve unpredictable long-tail events in dynamic human environments.
[29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) Closing Visions: Physical AI, Abundance, and World Simulators
In closing reflections, the panel emphasizes that physical AI and world models represent a major frontier, given that roughly 80% of the global economy exists within the physical world. Universal world simulators capable of learning from multi-modal inputs will act as transformational technology, reducing the reliance on slow, expensive physical trial-and-error.
A major reason language models excel at coding and mathematics is the existence of perfect, instant digital compilers and simulators that allow millions of parallel execution tests. In contrast, real-world tasks lack a native simulator, leading to absurd failures when purely text-based LLMs attempt real-world management tasks (such as Anthropic's experiment where Claude failed to manage a physical vending machine business). Building scalable world simulators will bridge this gap, accelerating progress across robotics, biology, enterprise operations, and physical automation to save lives and create unprecedented abundance.
- Approximately 80% of the global economy is anchored in physical operations and real-world assets.
- LLMs succeeded rapidly in code and math due to perfect automated execution sandboxes, a resource that has historically been absent for physical reality.
- Reference made to Anthropic's experiment attempting to have Claude autonomously manage a real-world vending machine business to illustrate current LLM real-world planning limitations.
## Notable quotes
> Language does not capture everything that we as humans care about... we learn as we grow up from observation by trying different things in the world and seeing what happens. — Anastasis Germanidis
> A human does not need thousands and thousands of hours of driving... we spend about 30 hours on average learning to drive... because we understand the basic rules of the road, we understand physics. — Oliver Cameron
## Mentioned
Jared Perlo, Anastasis Germanidis, Inhi Cho Suh, Oliver Cameron, Runway, Niantic Spatial, Odyssey, NBC News, Mount Everest, Himalayas, Rancho Cordova, Sacramento, HMC Architects, NVIDIA, DeepMind, AlphaStar, StarCraft, Yann LeCun, YouTube, JEPA, Meta, Jasper, GPT, Anthropic, Claude
## Video description
Jared Perlo - AI Reporter, NBC News
Anastasis Germanidis - Co-Founder & Co-CEO, Runway
Inhi Cho Suh - CEO, Niantic Spatial
Oliver Cameron - CEO, Odyssey
Today’s AI can generate text, images, and video, but the next frontier is systems that understand space, motion, physics, geography, and the structure of the physical world. This panel brings together leaders building world models and spatial intelligence systems to discuss how AI will move from creating digital content to simulating, mapping, and eventually acting in real-world environments.
Runway、Niantic与Odyssey拓展物理AI世界模型
AI行业领袖认为，扩大视频模型与现实世界真值数据规模将绕开代价高昂的物理试错，从而释放实体经济的自动化潜能。
该圆桌讨论由 Jared Perlo 主持，汇集了 Anastasis Germanidis（Runway）、Inhi Cho Suh（Niantic Spatial）和 Oliver Cameron（Odyssey），共同探讨超越以语言为中心的世界模型演进。嘉宾们探讨了具体的应用场景，例如城市 3D 数字孪生、用于克服机器人技术数据瓶颈的合成视频数据集，以及无限仿真环境。他们对比了物理 AI 的各种数据策略——从观察人类任务到利用多层企业传感器数据——并探讨了模型架构与扩展范式。讨论最终强调了弥合数字仿真与物理现实之间的差距将如何推动现实世界自动化、企业效率与空间推理能力的提升。
AI 正从语言理解全面迈向物理世界智能，Runway、Niantic Spatial 与 Odyssey 等团队正致力于构建通用世界模型，赋能空间计算、机器人与自动驾驶。 (0:04)
视频世界模型通过生成高质量合成视频变体，有效突破了机器人训练对高昂实体硬件数据的依赖，并能自然模拟传统引擎难以处理的非刚体物理交互。 (6:44)
物理 AI 的落地依赖工业设施等多视角企业专有数据；同时模型通过学习人类日常视频构建物理常识直觉，仅需少量目标数据即可迁移至硬件执行。 (11:29)
实践表明，采用基于下一个 Token 预测（next-token prediction）的简单视频架构在算力与数据扩展下表现优异，复现了语言模型的 Scaling Laws 并展现出深度物理推理潜能。 (19:31)
鉴于全球约 80% 的经济依赖于物理实体，通用世界模拟器将为现实世界提供类似代码编译器的沙盒环境，解决实体 AI 在复杂长尾场景下的泛化瓶颈。 (29:33)
Reading summary
Industry leaders Anastasis Germanidis (Runway), Inhi Cho Suh (Niantic Spatial), and Oliver Cameron (Odyssey) argue that artificial intelligence must transition from text models to foundational world models that comprehend physical space, motion, and 3D reality to automate real-world systems.
Scaling Next-Token Prediction to Video and Physics (19:31)
Panelists argue that language alone cannot capture the totality of human experience or the physical environment. Germanidis and Cameron maintain that the scaling laws that empowered large language models apply directly to visual media: scaling next-token prediction over massive video datasets enables systems to implicitly learn intuitive physics, spatial movement, and object behavior. While alternative paradigms such as Meta's Joint-Embedding Predictive Architecture (JEPA) or explicit 3D modeling are under exploration, Germanidis asserts that simpler architectures optimized for large compute and data scale have historically proven most effective.
Bridging Data Bottlenecks Through General Simulation (6:44)
Collecting real-world physical and robotic interaction data is capital-intensive and slow, presenting a primary bottleneck for embodied AI. World models serve as infinite simulators that generate synthetic video variations of tasks, environments, non-rigid materials, and lighting conditions. Cameron contrasts this with autonomous driving methods that ingest thousands of specialized driving hours, noting that humans learn to drive in roughly 30 hours because they already possess general physical intuition. Similarly, Germanidis notes that models pre-trained on broad video footage of humans performing everyday tasks require only a few hours of physical hardware interaction data to transfer those skills to robots.
High-Fidelity Enterprise Digital Twins and Local Deployment (22:00)
Transitioning models from lab simulations to real-world environments requires ground-truth data from actual facilities. Cho Suh highlights Niantic Spatial's work capturing secure, GPS-denied enterprise environments—such as multi-square-mile oil refineries with dense metal scaffolding—using multi-layer sensors across ground, aerial, and satellite perspectives. In an urban deployment with NVIDIA and HMC Architects in Rancho Cordova, California, Niantic Spatial built a multi-square-mile 3D digital twin to model traffic flow, drone logistics, and urban planning. Deploying these sim-to-real models requires navigating fragmented regional policies and accounting for localized human movement during emergency evacuations.
Generalization Barriers in Dynamic Physical Environments (24:20)
Unlike language models, which saw rapid early software commercialization through tools like Jasper, world models remain constrained to specialized robotics and vehicle applications. The primary technical hurdle is generalization: conventional robotic control policies often overfit to controlled settings, causing systems to fail when minor spatial shifts, such as moving a table by a few inches, occur. Pre-trained world models are intended to handle these dynamic edge cases and long-tail uncertainties in unstructured human spaces.
The panel underscores that because roughly 80% of the global economy operates in the physical realm, world models address a critical structural gap: while language models accelerated rapidly due to digital compilers and automated execution sandboxes, physical reality lacks native simulators. Panelists project that universal world simulators will bridge this divide, though the sector must still overcome fragmented municipal regulations, data capture constraints in non-public facilities, and the challenge of general developer adoption.
## Background
该内容由 YouTube 频道 Ai4 于 2026 年 8 月 21 日发布。视频记录了一场由 NBC News 科技记者 Jared Perlo 主持的圆桌论坛，邀请了 Runway 联合创始人 Anastasis Germanidis、Niantic Spatial 首席执行官 Inhi Cho Suh 以及 Odyssey 首席执行官 Oliver Cameron，共同探讨人工智能如何跨越数字内容生成，迈向理解空间、运动、物理规律并能在现实世界中模拟与行动的世界模型与空间智能前沿。
## TL;DR
论坛深入探讨了世界模型从理论概念走向产业落地的核心路径与现实挑战。三位行业领袖分别阐述了基于海量视频像素预测、企业级真实 3D 空间孪生以及通用环境模拟的技术探索，提出利用人类视频中内化的物理直觉来破解实体机器人数据匮乏的困境。针对自动驾驶的机械灌流数据反思以及机器人杂技演示在长尾场景中的易碎性，嘉宾们指出了过度拟合与泛化能力的本质瓶颈。大家一致认为，类似代码环境的通用世界模拟器将成为物理 AI 进化的关键底座，深刻重塑占全球经济 80% 的实体产业。
## Atmosphere & style
对谈整体洋溢着冷静务实与前瞻远见并存的专业探讨氛围。主持人节奏明快、直击行业痛点，三位企业掌舵人既各自捍卫所押注的技术路线——无论是视频扩展法则的暴力美学还是深扎工业现场的地面真值——又在揭露当前机器人与自动驾驶研发误区时展现出难得的清醒与坦诚。整个交流摒弃了空洞的概念炒作，穿插着喜马拉雅山感悟、庞大炼油厂无 GPS 困境与自动售货机模拟失败等鲜活案例，传递出物理智能在破晓前夜直面真实世界残酷复杂度的厚重质感。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=b47WtyhqpPk&t=4s) 人工智能正从理解人类语言迈向物理世界模型，空间智能将赋予模型理解三维空间、物体运动与物理法则的能力，成为驱动具身智能与现实自主系统的基石。
- [02:06](https://www.youtube.com/watch?v=b47WtyhqpPk&t=126s) 世界模型的三大主流演进路径分别以不同维度切入：Odyssey 聚焦通用预测与全行业模拟，Niantic Spatial 侧重宏观真实空间的 3D 数字孪生，Runway 则通过海量视频像素预测让模型自发涌现物理规律。
- [06:44](https://www.youtube.com/watch?v=b47WtyhqpPk&t=404s) 机器人发展的核心瓶颈在于物理数据匮乏而非算力；视频生成模型不仅能合成非刚体与复杂动力学训练数据，还能让机器人基于摄像头输入直接端到端预测下一步物理动作。
- [09:10](https://www.youtube.com/watch?v=b47WtyhqpPk&t=550s) 世界模型的终极杀手级应用是充当其他 AI 的“无限学习模拟器”，打破单一封闭仿真环境的上限，为通用智能提供无边界的进化空间。
- [11:51](https://www.youtube.com/watch?v=b47WtyhqpPk&t=711s) 通过在大规模人类行为视频中预先吸收物理直觉与空间常识，世界模型能打破硬件数据瓶颈，使机器人仅需数小时的真实硬件数据即可快速泛化到具体任务。
- [14:05](https://www.youtube.com/watch?v=b47WtyhqpPk&t=845s) 物理 AI 的关键数据深锁于企业私有工业场景（如炼油厂、仓库），这些无 GPS 且金属密集的极端环境需要高精度 3D Mesh 空间建模与更稳健的 Sim-to-Real 微观校准能力。
- [16:56](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1016s) 传统自动驾驶依靠海量行车视频灌注的做法违背了人类认知常理；人类只需 30 小时学车是因为具备生活常识与物理先验，世界模型能以同等逻辑显著降低自动驾驶的训练门槛。
- [18:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1109s) 技术架构往往遵循极简扩展法则，正如大语言模型最终由“下一个 Token 预测”统领，大规模视频生成预测同样能通过数据与算力缩放涌现出超越人类极限的物理机制理解。
- [22:00](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1320s) 真实世界的“地面真值”不可替代，不同地域在基础设施和行为规范上存在本质差异（如紧急疏散流程），空间智能必须扎根现场的多模态真实感知数据。
- [23:37](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1417s) 从虚拟模拟走向实体落地（Sim-to-Real）面临着各地分散的政策法规制约，同时世界模型当前缺少像早期 LLM 那样由繁荣开发者生态驱动的应用探索。
- [27:11](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1631s) 现有高难度机器人演示大多基于过度拟合的策略模型，极微小的环境扰动就会导致系统崩溃；唯有依赖大规模预训练的世界模型，才能解决现实长尾场景的泛化难题。
- [29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) 全球经济中 80% 属于实体经济，缺乏试错环境此前制约了现实智能的发展；通用世界模拟器将提供零成本试错底座，全面释放物理经济与科学探索的巨大潜能。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=b47WtyhqpPk&t=4s) 圆桌论坛开场：从语言智能迈向物理世界模型
过去数十年间，人工智能在理解人类语言方面取得了突破性进展，但如今技术前沿正迎来新的跃迁——AI 正在学习理解真实的物理世界。主持人 Jared Perlo 在开场中指出，世界模型（World Models）正在赋予 AI 理解空间、运动和三维环境的能力，这为更强大的机器人、自主系统以及各类创意工具开启了全新的大门。本场论坛汇聚了三位在世界模型与空间智能领域的领军人物：Runway 联合创始人兼联席 CEO Anastasis Germanidis（Runway 推出了首个公开可用的 AI 视频模型，并拥有目前业界最大规模投产的世界模型部署）、Niantic Spatial 首席执行官 Inhi Cho Suh（专注于为物理 AI 构建真实世界基础模型，使人机协作更高效安全），以及 Odyssey 联合创始人兼 CEO Oliver Cameron（曾助力在旧金山落地全球首个全无人驾驶出租车服务，现致力于打造通用世界模型）。
世界模型的概念内涵极为丰富，三家企业从各自独特的路径探索这一前沿领域，这也正是本次对谈探讨空间智能未来的出发点。
---
### [02:06](https://www.youtube.com/watch?v=b47WtyhqpPk&t=126s) 嘉宾背景与世界模型的定义：预测、空间现实与视觉观察
三位嘉宾分别阐述了各自进入该领域的契机以及对“世界模型”这一核心概念的理解。Oliver Cameron 回忆称，自己在 2015 年投身无人驾驶领域时，自动驾驶技术因无法准确预测物理世界而远未成熟；但经过五至七年的发展，团队将大规模 AI 模型与海量数据集相结合，使得系统展现出预测行人与车辆轨迹的“超人级精准度”。这种如魔法般的突破促使他在三年前创立了 Odyssey，旨在打造能够理解并模拟虚拟与物理世界的通用基础模型，赋能机器人、无人车、无人机、游戏、国防和医疗等跨行业场景。
```mermaid
flowchart TD
accTitle: 三种不同的世界模型技术演进路径
subgraph Odyssey["Odyssey: 通用预测模拟"]
A1["自动驾驶轨迹预测"] --> A2["物理/虚拟世界通用基础模型"]
A2 --> A3["全行业应用与无限模拟"]
end
subgraph NianticSpatial["Niantic Spatial: 真实物理建模"]
B1["喜马拉雅山真实体验启发"] --> B2["真实大尺度3D数字空间"]
B2 --> B3["实体人机协同与现实孪生"]
end
subgraph Runway["Runway: 视觉观察与像素预测"]
C1["海量视频数据预训练"] --> C2["学习真实物理法则/非刚体动态"]
C2 --> C3["创意生成 + 机器人动作控制"]
end
```
Inhi Cho Suh 分享了她的心路历程：在企业级软件与数字化 AI 领域深耕数十年后，一次喜马拉雅山珠峰之行让她深刻意识到，人类长期受限于屏幕输入与数字化工作，但世界本质上是物理的，“喜马拉雅山的照片根本无法还原其真实的壮阔”。Niantic Spatial 的核心愿景正是构建非单纯人工模拟、而是忠实还原真实世界的 3D 模型，为人类、机器与 AI 智能体提供扎根物理现实的交互底座。
Anastasis Germanidis 则强调了 Runway 创立八年以来的核心信念：人类如果仅依赖语言描述来学习，是无法构建出真正实用的通用智能的，因为语言并不能完全涵盖人类所感知的全部世界；智能源于对世界的直接观察与试错。Runway 从视频与视觉数据切入，通过不断扩展模型预训练与后训练的数据和算力规模，发现像素预测模型能够自发学会物理定律与人类关注的各种复杂任务。目前 Runway 不仅服务着超过 5000 万创作者、电影制作人与艺术家，其技术更已深入生命科学以及机器人世界模拟等关键领域。
---
### [06:44](https://www.youtube.com/watch?v=b47WtyhqpPk&t=404s) 现实世界模型的杀手级应用：数字孪生、机器人与无限模拟器
在落地应用层面，世界模型已经从抽象概念转化为解决实际复杂问题的工具。Inhi Cho Suh 介绍了 Niantic Spatial 联合英伟达（NVIDIA）及 MCI 为加州萨克拉门托周边的兰乔科尔多瓦市（Rancho Cordova）构建的首个城市级 3D 数字孪生项目。该项目覆盖数平方英里的宏大空间尺度，市政管理团队和市长议会能够基于该高精模型进行交通管理、城市空间活化规划、无人机航线设计以及末端物流配送等多种前瞻性场景的宏观模拟仿真。
```mermaid
flowchart LR
accTitle: 视频世界模型驱动机器人系统闭环
subgraph DataGen["合成与仿真"]
D1["视频生成模型"] --> D2["无限环境/光照/任务变体"]
D1 --> D3["非刚体/流体动力学模拟"]
end
subgraph ActionLoop["机器人执行与闭环"]
D2 --> E1["观察机载摄像头画面"]
D3 --> E1
E1 --> E2["预测下一步状态与动作"]
E2 --> E3["端到端控制机器人执行任务"]
end
```
Anastasis Germanidis 深入剖析了世界模型对机器人领域的革命性推动。他指出，当前机器人学的核心瓶颈并非算力，而是**数据匮乏**——现存数据远远不足以消耗现有的充沛算力，且实体硬件采集数据的周期漫长、成本极高。Runway 的视频模型在此展现出双重核心价值：
- **突破传统仿真局限的数据生成**：能够随着算力扩展，自动生成针对不同机器人形态、任务变体、物体材质、复杂光照与多变环境的大规模训练数据。传统仿真在处理非刚体（如布料）和复杂流体力学时往往表现欠佳，而海量视频数据训练出的模型能够高度逼真地模拟这些物理现象。
- **从观察直接预测动作**：机器人输入机载摄像头画面后，视频模型能够预测接下来会发生什么，并进而直接推断并输出完成特定任务所需的物理动作，实现高难度场景的端到端动作决策。
Oliver Cameron 则指出了世界模型的“终极杀手级应用”——**充当其他 AI 的终极学习环境（无限模拟器）**。他以 DeepMind 的 AlphaStar 为例：过去让智能体在《星际争霸》（StarCraft）中博弈，虽然能诞生出超越人类水平的游戏玩家，但智能体的发展上限始终被单一游戏环境所封顶。而通用的世界模型能够充当无边无际的模拟环境，彻底打破特定封闭场景对智能进化的桎梏，从而孕育出人类前所未见的通用智能。
随后，讨论进一步延伸至所有具身与物理智能必须直面的核心基石——数据获取方式的差异与挑战。
### [11:51](https://www.youtube.com/watch?v=b47WtyhqpPk&t=711s) 从人类视频中学习物理直觉：破解机器人数据匮乏的飞轮
在探讨数据收集与处理方式时，Runway 联合创始人兼联合首席执行官 Anastasis Germanidis 指出，机器人领域的数据采集之所以极其艰难，是因为它受制于现有的硬件规模。开发者能够扩展的数据量完全取决于拥有的机器人数量；而为了将机器人部署到真实世界中，它们必须足够好，但要做到足够好，又必须先用海量数据进行训练——这就陷入了一个难以启动的飞轮困境。然而，Runway 在物理 AI（Physical AI）研究中最令人惊喜且极具前景的发现是：模型可以直接从人类执行各种任务的视频数据中学习，而最终只需要极少量的机器人硬件数据即可完成适配。
由于 Runway 的模型已经吸收了海量关于人类进行体育运动、做家务、物体交互以及各种尝试与失败的视频案例，它们已经内化了关于人类在空间中如何运动以及基础物理规律的直觉。要将这种直觉转移到机器人身上，通常只需要几个小时的真实机器人数据。无论是通过佩戴在头部的摄像头采集的第一人称视角（Egocentric data），还是第三人称视角数据，模型都能有效吸收。对于世界模型而言，数据的规模与多样性远比数据的特定形式更为关键。
```mermaid
flowchart TD
accTitle: 物理 AI 的世界模型知识迁移机制
subgraph Pretraining["通用世界先验预训练（海量规模）"]
A["人类行为视频<br/>（体育、家务、交互、失败案例）"] --> C["世界模型 / 物理直觉<br/>（空间运动、重力、碰撞规律）"]
B["多视角视觉数据<br/>（第一人称视角 + 第三人称视角）"] --> C
end
subgraph FineTuning["下游任务快速迁移（极小规模）"]
C --> D["注入极少量硬件数据<br/>（仅需数小时的机器人特定数据）"]
D --> E["实体机器人 / 自动驾驶精准执行"]
end
```
- **硬件约束困境**：机器人数据的扩展速度直接受限于实体硬件数量，形成了“没有数据就无法部署，没有部署就无法采集数据”的循环。
- **直觉迁移**：模型从人类运动和物体交互中建立物理直觉，将海量人类常识转移给机器人仅需数小时的真实硬件数据支持。
---
### [14:05](https://www.youtube.com/watch?v=b47WtyhqpPk&t=845s) 企业私有空间与极端工业场景的 3D 空间智能
Niantic Spatial 首席执行官 Inhi Cho Suh 从空间尺度和企业私有数据的角度剖析了物理 AI 的独特挑战。与可以通过互联网公开抓取文本的大语言模型不同，物理 AI 所需的关键数据大部分并非公开可用，而是深锁在企业内部。这些环境包括极其复杂的室内外设施，例如仓库、炼油厂、机场、造船厂和建筑工地。这些私有数据不仅包含高度机密的信息、专利、安全合规策略，而且物理环境极其严苛。
以 Niantic 正在合作的顶级石油与天然气公司为例，美国大型炼油厂的占地面积可达 5x5 平方英里（约 65 平方公里），比 4 英里长的拉斯维加斯大道在体量上庞大得多。在这类布满阀门、压缩机、脚手架和密集金属构件的极端环境中，GPS 完全失效，如何进行空间导航、变化检测以及引导工人进行精准检修是巨大难题。Niantic 的技术能够以高保真度重建这些三维空间，并在网格（Mesh）级别进行电子化操作。此外，在与 Flexion 合作进行从仿真到真实世界（Sim-to-Real）的迁移时，环境微变也极具挑战——机器人在平整干净的舞台地面上训练良好，但一旦面对地毯或地面洒水，就必须具备重新校准平衡、电机与传感器的能力。
- **空间数据来源的多样性**：涵盖地面多传感器数据、无人机航拍数据及高空卫星数据，覆盖从微观网格到宏观地理的不同尺度。
- **无 GPS 工业场景的高保真重建**：在金属密集、管线错综复杂的 5x5 平方英里超大型炼油厂内，利用 3D Mesh 空间模型替代传统定位技术。
- **仿真到真实的微观泛化**：应对现实世界中未见过的边缘工况（如地毯阻力、地面水渍），使机器人在受控的小数据集下学会重新校准机动与平衡。
---
### [16:56](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1016s) 自动驾驶的反思：利用世界模型实现 30 小时快速泛化
Odyssey 首席执行官 Oliver Cameron 结合自己在自动驾驶领域的经历，对当前的自动驾驶研发路径提出了根本性的反思。他认为行业在构建自动驾驶汽车的底层逻辑上走入了误区：目前的做法是强行给模型灌注成千上万个小时的人类驾驶视频，但这与人类学习驾驶的方式截然相反。人类在学车时，绝不需要坐在电视机前观看几千个小时旧金山街头的行驶画面。
相反，一个 17 岁的青少年第一次坐上驾驶座时，平均只需要大约 30 个小时的实际练习就能掌握驾驶。这是因为在坐进驾驶室之前，人类就已经通过 17 年的生活经验理解了基础物理法则、碰撞的后果、道路的基本规则，并且多年来一直在潜移默化地观察父母如何开车。世界模型带来的颠覆性机遇正在于此：它预先具备了对人类、物体、运动、医疗、国防以及各类物理交互的广博先验知识；在此基础上，它只需要极小量（如几十小时）的特定驾驶数据，就能迅速适应并完成复杂的自动驾驶任务。
> "一个 17 岁的人第一次坐到方向盘后面，平均只花大约 30 个小时学车……世界模型让我们有机会构建一个拥有无比庞大常识的模型，进而只需极少量的数据就能适应驾驶任务。" —— Oliver Cameron
---
### [18:29](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1109s) 架构之争与极简扩展法则：为何押注视频生成模型
针对主持人 Jared Perlo 提出的关于技术路线分歧的提问——包括 Yann LeCun 正在推进的 JEPA 架构（联合嵌入预测架构）、基于 3D 几何的方法，以及纯粹依靠海量视频预训练的世界模型路线——Anastasis Germanidis 给出了坚定的解答。他回顾了大语言模型的发展历史：在 2010 年代中期，学术界和工业界曾提出过大量极其复杂的推理与长程规划架构，但最终脱颖而出的却是最简单的范式——将“下一个 Token 预测（Next Token Prediction）”推向极致规模。在小规模时，语言模型连完整句子都组织不好；而在超大规模下，它们能够连续数天解决复杂的软件工程难题。
Anastasis 强调，Runway 在视频领域做出了与语言模型完全相同的类比押注。小规模的视频生成模型连连续画面都难以生成，但在巨大的算力和数据扩展下，它们能够自发涌现出对物理规律极其精密的理解，并能模拟现实世界的复杂演变。尽管 JEPA 和纯 3D 方案各具前景，但历史经验表明，最简单且易于规模化扩展的架构往往具有最强的生命力。Oliver Cameron 对此深表认同，并补充道，与一个物理理解深度超越人类认知极限的模型进行交互极其令人兴奋——如果一个模型能够细致入微地观察世界上存在过的每一段视频中的每一个物理细节，它所揭示出的物理和科学规律将带来人类无法想象的突破与发现。
- **大语言模型历史映射**：复杂的专门推理架构最终输给了简单可扩展的下一个 Token 预测；视频生成模型的物理理解同样依靠海量数据与算力的扩展法则（Scaling Law）。
- **超越人类感知的物理洞察**：全量视频训练催生出对现实世界细粒度物理机制的深度建模，未来有望反哺基础科学与工程发现。
---
### [22:00](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1320s) 真实世界的“地面真值”：跨地域与复杂工业物理实体的多模态感知
Niantic Spatial 的 Inhi Cho Suh 提出了截然不同的务实视角。她以自己走访 50 多个国家的个人与业务经历切入，强调真实世界的复杂性和不可替代性：“只有一个世界，那就是真实的世界（There's only one world. It's the real one）。”当一个人真正走进半导体实验室、高度自动化的农场、大型物流仓库或化工厂时，会发现每个物理工作场所都有其独特的物理与文化规则。
Niantic 的核心逻辑是必须从物理实体的“地面真值（Ground Truth）”出发：首先在人类工作和机器运转的真实场所采集多模态数据，进而构建高精度仿真系统以优化预测、设备维护、运营调度及安全疏散。例如，东南亚地区的应急疏散模式与西欧国家截然不同，因为两地的基础设施布局、人流移动习惯以及安全文化存在本质差异。因此，构建真正能落地的空间智能，第一步必须扎根于地面传感器、航空视觉以及企业私有物理空间的深度多模态数据采集。
- **物理与文化的地域差异**：基础设施的差异导致作业与安全规范具有极强的地域特征（如东南亚与西欧的紧急疏散流程差异），无法仅靠单一虚拟数据统一概括。
- **地面真值驱动的仿真流**：先采集真实企业环境的多源传感器与视觉数据，再以此为基础生成上层的高保真仿真，用于预测性维护与自主决策。
### [23:37](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1417s) 从实验室到现实：政策监管与“虚拟到现实”（Sim-to-Real）的跨越
在讨论世界模型与代理系统落地时，Inhi Cho Suh（Niantic Spatial 首席执行官）指出了一个关键制约因素：政策、工作流与智能体集成点之间的协调。在物理世界部署自主设备无法一刀切，法律规范在不同地域之间差异极大——无人机并不能在所有空域随意飞行，每一个城市、州和省份对于空中及地面机器人、物流配送等技术都有截然不同的监管条例和安全规范。因此，不仅要加速现实世界到模拟环境（Real-to-Sim）的建模，更关键的是与像 Runway 这样的团队深度合作，共同攻克将虚拟模拟成果反哺到物理实体世界（Sim-to-Real）的难题，让技术真正走出实验室。
随后，主持人 Jared Perlo（NBC News）承接这一话题展开探讨。他指出，“世界模型”（World Model）这一概念其实在计算机科学和机器人学领域已经被探讨和钻研了数十年。如今坐在如此气派的宴会厅中，看着所有人都在热烈讨论“世界模型”，整个领域展现出前所未有的热度。他向在座嘉宾抛出核心问题：当前阻碍世界模型发展的核心瓶颈究竟是什么？如果只能集中所有精力攻克一个痛点，那会是什么？
### [24:54](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1494s) 开发者生态与规模化部署难题
Oliver Cameron（Odyssey 首席执行官）认为，当前世界模型最核心的瓶颈在于“探索的速度”（Speed of exploration）。相比语言模型（LLM），世界模型尚未形成繁荣的开发者生态。回顾大语言模型的发展路径，开发者社区构建了海量各具创意的应用，将底层技术落到具体业务中；在早期 GPT-2、GPT-3 阶段，最标志性的“杀手级应用”（Killer App）是营销文案生成（例如 Jasper 公司），随后才迎来整个行业的爆发式增长。而在世界模型领域，这类开发者驱动的生态繁荣还未发生。目前大部分应用仍然局限于最显而易见的赛道，比如自动驾驶、机器人等，但世界模型的探索空间远不止于此。
```mermaid
flowchart TD
accTitle: LLM与世界模型的探索模式对比
subgraph LLM ["语言模型（LLM）路径"]
L1["基础模型 (GPT-2/3)"] --> L2["首个杀手级应用 (Jasper 营销文案)"]
L2 --> L3["开发者生态爆发与长尾应用繁荣"]
end
subgraph WM ["世界模型（World Models）现状"]
W1["底层物理与空间建模"] --> W2["显性场景 (自动驾驶 / 工业机器人)"]
W2 -.-> W3["待解锁：广泛的开发者生态与未知探索"]
end
```
紧接着，Inhi Cho Suh 补充了另外两大瓶颈：数据获取的多样性与规模化部署（Scale Deployment）。
- **数据复杂性**：不同物理任务和作业场景所需的数据集完全不同，且极难获取。
- **物理环境异质性**：现实中的每个物理结构和操作参数各不相同，规模化部署前必须经过极其严苛的系统性测试。
- **人机混合环境的非确定性**：在纯机器人运作的确定性封闭空间部署相对容易，但多数现实场景是人类、自动化设备与无人作业系统交织并存的复杂环境，这给系统规模化落地带来了巨大挑战。
### [27:11](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1631s) 机器人的泛化瓶颈与长尾场景困境
Anastasis Germanidis（Runway 联合创始人兼联席首席执行官）指出，构建能够自主在现实世界运行的系统，其最大障碍在于“泛化能力”（Generalization）。公众每天都能看到令人惊艳的机器人演示视频，例如机器人完成高难度的后空翻、复杂的杂技动作或精密操作。然而，这些演示中的机器人策略模型（Robot Policy）绝大多数都是针对特定环境、特定任务与特定场景进行过度拟合（overfitted）训练出来的。
如果场景发生一丝微小的偏离——哪怕仅仅是将桌面任务中的桌子移动几英寸——那些看似完美的演示就会瞬间崩溃，机器人将彻底无法完成既定动作。传统机器人研究往往专注于特定任务的高精求解，在可完全预测的环境中表现良好；但一旦遭遇长尾事件（Long-tail events）或突发情况，脆弱性便暴露无遗。如果想要机器人走进家庭日常，适应千差万别的房间布局，甚至陪同用户去商店购物并协助日常琐事，系统就必须具备应对不确定性与意外场景的泛化能力。这种能力无法仅靠特定任务的微调获得，而必须依赖于大规模预训练（Large-scale Pre-training）所赋予的通用智能。
### [29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) 总结与展望：世界模拟器与实体经济的未来
在对谈尾声，主持人邀请三位嘉宾分享对世界模型当下进展及未来 1 至 5 年趋势的寄语与展望：
- **Oliver Cameron (Odyssey)**：世界模型将推动基础物理学层面的重大突破，帮助人类重新认识和发现现实世界的本质，进而带来颠覆性的创新。最终，世界模型赋能的自主系统将创造巨大的物质财富、拯救生命并改善全人类的生活质感。
- **Inhi Cho Suh (Niantic Spatial)**：全球经济中约 80% 根植于物理世界。世界模型与物理 AI（Physical AI）正在释放这 80% 物理经济的潜能，其带来的可能性将远远超出当下的想象。只要保持好奇心，各行各业从今天起就能探索这场变革。
- **Anastasis Germanidis (Runway)**：能够通过视频以及多模态感知学习的“通用世界模拟器”（Universal World Simulator），将成为这个时代最具变革性的技术。
Anastasis 特别强调了模拟器对智能演进的决定性作用：大语言模型之所以在编程（Coding）和数学领域取得惊人进展，是因为软件代码本身拥有“完美模拟器”——可以瞬间并发运行数百万个程序，快速验证代码是否正确执行，并让模型从错误中反复试错迭代。然而，现实世界此前根本不存在这样的模拟器。正因如此，当 Anthropic 让 Claude 尝试在真实商业环境中模拟运营一台自动售货机时，模型表现出了极其滑稽且惨烈的大溃败（"failed pretty spectacularly"）。现实物理世界试错成本极高，一旦拥有了强大的世界模拟器，机器人学、生物学乃至组织化运转中的海量现实难题都将被彻底解锁。
## Notable quotes
> [02:06](https://www.youtube.com/watch?v=b47WtyhqpPk&t=126s) 喜马拉雅山的照片根本无法还原其真实的壮阔 — Inhi Cho Suh
> [16:56](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1016s) 一个 17 岁的人第一次坐到方向盘后面，平均只花大约 30 个小时学车……世界模型让我们有机会构建一个拥有无比庞大常识的模型，进而只需极少量的数据就能适应驾驶任务。 — Oliver Cameron
> [22:00](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1320s) 只有一个世界，那就是真实的世界（There's only one world. It's the real one） — Inhi Cho Suh
> [24:54](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1494s) 世界模型最核心的瓶颈在于“探索的速度”（Speed of exploration） — Oliver Cameron
> [27:11](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1631s) 泛化能力（Generalization） — Anastasis Germanidis
> [29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) 全球经济中约 80% 根植于物理世界。世界模型与物理 AI（Physical AI）正在释放这 80% 物理经济的潜能，其带来的可能性将远远超出当下的想象。 — Inhi Cho Suh
> [29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) 通用世界模拟器（Universal World Simulator），将成为这个时代最具变革性的技术 — Anastasis Germanidis
> [29:33](https://www.youtube.com/watch?v=b47WtyhqpPk&t=1773s) failed pretty spectacularly — Anastasis Germanidis
## Mentioned
Jared Perlo, NBC News, Anastasis Germanidis, Runway, Inhi Cho Suh, Niantic Spatial, Oliver Cameron, Odyssey, NVIDIA, MCI, Rancho Cordova, AlphaStar, DeepMind, Flexion, Yann LeCun, JEPA, Jasper, Anthropic, Claude
## Video description
Jared Perlo - AI Reporter, NBC News
Anastasis Germanidis - Co-Founder & Co-CEO, Runway
Inhi Cho Suh - CEO, Niantic Spatial
Oliver Cameron - CEO, Odyssey
Today’s AI can generate text, images, and video, but the next frontier is systems that understand space, motion, physics, geography, and the structure of the physical world. This panel brings together leaders building world models and spatial intelligence systems to discuss how AI will move from creating digital content to simulating, mapping, and eventually acting in real-world environments.
[Source](https://www.youtube.com/watch?v=b47WtyhqpPk)
Source quotes
[2:06] Inhi Cho Suh: 喜马拉雅山的照片根本无法还原其真实的壮阔
[16:56] Oliver Cameron: 一个 17 岁的人第一次坐到方向盘后面，平均只花大约 30 个小时学车……世界模型让我们有机会构建一个拥有无比庞大常识的模型，进而只需极少量的数据就能适应驾驶任务。
[22:00] Inhi Cho Suh: 只有一个世界，那就是真实的世界（There's only one world. It's the real one）
[24:54] Oliver Cameron: 世界模型最核心的瓶颈在于“探索的速度”（Speed of exploration）
[27:11] Anastasis Germanidis: 泛化能力（Generalization）
[29:33] Inhi Cho Suh: 全球经济中约 80% 根植于物理世界。世界模型与物理 AI（Physical AI）正在释放这 80% 物理经济的潜能，其带来的可能性将远远超出当下的想象。
[29:33] Anastasis Germanidis: 通用世界模拟器（Universal World Simulator），将成为这个时代最具变革性的技术
[29:33] Anastasis Germanidis: failed pretty spectacularly
Source quotes
Anastasis Germanidis: Language does not capture everything that we as humans care about... we learn as we grow up from observation by trying different things in the world and seeing what happens.
Oliver Cameron: A human does not need thousands and thousands of hours of driving... we spend about 30 hours on average learning to drive... because we understand the basic rules of the road, we understand physics.
Original transcript
[0:04] Good morning everybody. Thank you so much for
being here on this Thursday morning. For decades,
[0:10] AI has excelled at understanding language. Now,
it's learning to understand the physical world.
[0:19] World models are giving AI the ability to
understand space movement and three-dimensional
[0:25] environments, opening the door to more
capable robots, autonomous systems,
[0:32] and all sorts of creative tools. To talk about the
latest 3D innovations are three leaders helping
[0:39] define this exciting new frontier. Anastasis
Germanus is co-founder and CEO of Runway,
[0:46] the AI lab building AI that simulates the world by
merging art and science. Runway created the first
[0:54] publicly available AI video model and now has the
largest world model deployment in production. In
[1:02] Chosa is the CEO of Niantic Spatial where she
is building real world foundational models
[1:08] for physical AI. So people and machines can
work efficiently and safely together. Oliver
[1:16] Cameron is the co-founder and CEO of Odyssey.
After helping launch the world's first fully
[1:23] driverless service in San Francisco, he is now
building generalpurpose world models that enable
[1:28] AI to predict and interact with the real world. So
without further ado, please join me in welcoming
[1:35] Anastasis Germanus, Ini Chosa, and Oliver Cameron.
Thank you so much. I think there is a world,
[1:41] if not a galaxy, a universe of exciting topics to
discuss here. So, world models can mean a zillion
[1:47] different things and I really excited to have you
on the panel today because I think each of you
[1:52] and your companies is approaching this exciting
new frontier in a very different way. So, maybe
[1:57] to kick things off, I'm curious to hear from each
of you just a little bit more about where you're
[2:01] coming from, what your company's working on, and
how you kind of approach and define world models.
[2:06] Awesome. Excellent to be here. My name is Oliver
Cameron, co-founder and CEO of Odyssey. And before
[2:12] Odyssey, my background was in driverless cars. And
it turned out that when I jumped into the field in
[2:19] 2015, uh driverless cars were not ready. And one
of the reasons they were not ready is because they
[2:26] could not predict the world. These technologies
at the time were too primitive. And so over the
[2:32] course of a five, six, sevenyear period, we
threw artificial intelligence at the problem.
[2:37] And it turned out that if you scale these models,
you scale the data sets that all of a sudden these
[2:43] models could predict the world. They could predict
where that pedestrian was going to be next. They
[2:48] could predict where that car was going to be next
and do that with superhuman accuracy. And so that
[2:53] felt magical to me, like literal breakthrough.
uh and so started a company called Odyssey
[3:00] really premised on the idea that we could take
a technology like that that could understand
[3:04] and predict the world and build general world
models that would be applicable to many different
[3:10] industries. And so we've been working on this
problem now for 3 years. To answer your question
[3:15] of what a world model I believe is, it's a
foundation model that understands and can simulate
[3:21] the world, both the virtual world and the physical
world for robots, driverless cars, drones, games,
[3:27] defense, healthcare, many other industries. Carson
in hi, my name is Inh. Great meeting all of you.
[3:36] Um I started at Niantic Spatial actually only a
few months ago and Niantic Spatial is building
[3:43] real world models for physical AI and prior
to Niantic Spatial I spent a couple decades
[3:50] in enterprise software and digital AI. I took a
little trip to Everest and I realized, you know,
[4:01] we're often stuck in the screen typing
and doing digital work, but the world is
[4:07] actually physical and the way we think about
relevance of things and people and context,
[4:13] you actually have to go there and visit it. So
net is the pictures for the Himalayas don't do
[4:19] it justice and having a real world 3D model that's
real not simulated is part of our desire to create
[4:29] real experiences that machines humans um AI agents
can build upon Hi everyone I'm anastasis um I'm
[4:40] co-founder and coco at Runway um so Runway
is an applied research company we started
[4:46] 8 years ago at this point and for kind of for
all this time our main goal has been and our main
[4:52] belief has been that in order to actually uh build
uh like actually useful uh intelligence in the
[4:59] world uh you need to learn directly by observing
the world. Um just learning that from how humans
[5:06] describe the world is not enough. Uh language
does not capture everything that we as humans care
[5:12] about. uh we learn as we grow up from observation
by trying and and trying different things in the
[5:18] world and seeing what happens. Um and so our our
mission has always been um starting from video and
[5:26] visual data and building models that can represent
that at higher and higher scales. Um so we uh we
[5:34] started building our video generation models um
back about half a decade ago and ever since we've
[5:40] been scaling uh and improving the pre-training
and post-training of those models. Um and we've
[5:47] seen that as we increase the amount of data
and amount of compute that we put into video
[5:52] uh models, so models that learn to predict pixels,
they actually learn uh a lot about the world that
[5:58] we care about. uh they learn about physics, they
learn about a variety of tasks that humans care
[6:04] about. Uh and they can be useful for a variety of
things. They can be useful for um kind of creative
[6:10] expression. So we have a a platform of more
than 50 million kind of creators and filmmakers,
[6:16] artists, marketers that are using our platform,
but they can also be very useful for solving
[6:21] real problems in the real world. Um and that whe
whether that's in robotics or life sciences or
[6:26] other uh other domains uh a big focus for us has
been especially in how our models can be uh useful
[6:34] in robotics for simulating scenarios that are
difficult to simulate otherwise uh and building
[6:39] models to learn how to act in the world and take
actions. Awesome. Thank you. I think that's a
[6:44] great overview. So I think you gave me a great
segue uh there Anastasis because I think world
[6:50] models can sound very abstract perhaps you know
a bit ironically it can mean so so many different
[6:54] things. I'm curious so far what some of the most
interesting applications are that you've seen uh
[7:01] in your companies customers coming to you saying
wow this isn't just some kind of you know nebulous
[7:06] concept but there's actual real meat on the
bones here. So uh maybe kicking off with India,
[7:10] I'm curious from Niant's point of view, what are
you know some really exciting applications you're
[7:14] seeing? Well, one of the exciting applications
we actually announced this week was Tuesday.
[7:19] We're working with the city of Cordova, Rancho
Cordova, which is in just outside of Sacramento,
[7:25] California. And we are creating the very first
3D digital twin of the city in partnership with H
[7:33] uh MCI as well as uh Nvidia. And what the team
and mayor council and the partner ecosystem
[7:42] wants to do is create a model that allows them
to then simulate a number of scenarios. So you
[7:49] can imagine scenarios for traffic management,
scenarios for planning, urban revitalization,
[7:57] planning for maybe drones and fleet management or
last mile delivery. So there's a number of things
[8:03] that you could start to do when you can see things
at massive scale. I mean we're talking miles,
[8:09] square miles, many many square miles of this.
See, I'm curious if you could share a little
[8:14] bit more about the robotic applications you just
mentioned there on this. Yeah. So one of the one
[8:20] of the biggest bottlenecks in robotics is data. Um
it's actually not compute. uh if there is uh you
[8:26] there there is not enough data to utilize all the
compute available for robotics applications right
[8:31] now. Um it's very time consuming to collect data.
You need to build uh kind of massively scale up
[8:37] the hardware to parallelize uh the amount of data
collection that you can do. Um so video models are
[8:43] actually especially useful in both generating
data. So generating uh different examples of
[8:49] a particular uh robot embodiment performing
different tasks. Um and you can scale that
[8:55] basically you scale that together with compute.
You can generate as much as uh variations of the
[9:01] tasks, the objects, the lighting, the environments
um as as you can with a with a video model. Uh and
[9:08] once you have a great video model, we actually
see that you can actually make it also predict
[9:12] actions. So if you if you give it uh observations
directly that come from the robot. So a camera
[9:18] that might be attached on the robot or next to it
uh it can predict what happens next and by doing
[9:24] so it can predict what actions to take in order
to solve a specific task. Um and so simulation has
[9:30] always been a very big part of robotics. Uh but
traditional simulation does not work very well for
[9:37] um for non-rigid objects for things you know like
cloth or um uh or dealing you know simulating kind
[9:44] of fluid dynamics correctly. U whereas a video
model because it has learned directly from massive
[9:50] scales of data is able to actually simulate those
those things quite well. Uh and so we've seen kind
[9:56] of robotics robotics labs use our models in order
to simulate what happens as a result of particular
[10:02] actions that the robot uh the robot can take
and then uh deploy our models directly uh into
[10:08] into the robots to decide what actions to take
and and solve tasks that uh are have been very
[10:14] difficult to solve with traditional methods. Maybe
coming back to you Oliver. Yes. I think one of the
[10:20] most important or the killer applications of wolf
models will be as a learning environment for other
[10:25] AIs. And so if you wind back to some of the uh
deep mind work uh called Alphaar where effectively
[10:35] what they did was put agents inside Starcraft
allow those agents to play allow those agents to
[10:42] learn all inside Starcraft. What you saw is you do
that long enough, you get a superhuman Starcraft
[10:48] player. Now, the challenge there is the ceiling
of performances uh imposed by the environment.
[10:56] Starcraft specifically, the agents only know
Starcraft. They can only work within it. Now,
[11:01] what I think world models really offer is this
ability to be this infinite simulator. And so,
[11:08] instead of agents just learning inside a single
environment, they can learn inside infinite
[11:12] environments. and the ceiling of intelligence that
would uh that is there with Starcraft suddenly is
[11:19] removed and I I I don't know what intelligence
emerges from that but I I think it would be
[11:23] very cool. Excited to check back in. Um I think
[clears throat] you we've all mentioned or kind of
[11:29] hit on at least tangentially this key point about
data. So I think a lot of people in the room are
[11:34] probably familiar with the fact that today's large
language models just simply can't get enough data.
[11:38] just hoovering up all the data that exists on the
internet, spending billions of dollars to create
[11:42] customized environments to harvest more data,
getting more people to label data. And I think
[11:48] each of your companies has a slightly different
approach to gathering data, the types of data
[11:51] you're really interested in and exciting about.
So maybe working, you know, back to front here
[11:56] with Anastasis, how are is runway thinking about
gathering and approaching data? So as I mentioned,
[12:03] data and robotics is very difficult to collect
because you're bound by the hardware that you
[12:06] have available. Uh, and so you can only scale data
by the number of robots you have. And in order to
[12:12] deploy those robots in the real world, they need
to be good enough. But in order for them to be
[12:16] good enough, they need to be trained on enough
data. So there is this interesting kind of a
[12:20] um flywheel that needs to be built. Um but
what's the most kind of surprising and and and
[12:27] um and promising aspect of that we've seen about
how we our models going to be applied in physical
[12:34] AI is that you can actually learn directly from
data of humans performing uh different kinds of
[12:40] tasks and then you only need a very small amount
of data directly uh that's kind of from robot uh
[12:46] from robot hardware. Uh so because our model
has learned from so much uh so many examples
[12:53] of humans playing sports, doing household tasks um
and examples of physics um of you know different
[13:00] interactions with objects, people trying to
perform something and failing um they they've
[13:06] gained this intuition about you know how humans
move in space. They've gained this intuition about
[13:11] uh kind of this intuition about physics. Uh and
it's actually very than much easier to transfer
[13:18] that to um teaching a robot how to do something
and you only need a few hours of robotics data
[13:25] uh in order to transfer all the knowledge that has
been built from examples of humans doing tasks and
[13:31] sometimes that's egocentric data which means
you know uh attaching a camera kind of on on
[13:36] on someone's head and then having them perform a
specific task. Sometimes it's third person view.
[13:41] our model can learn from both kinds of data
and actually the scale and diversity matters
[13:46] more than the kind of very specific type. Um but
that's that's the most uh amazing fact that you
[13:53] can essentially learn directly from just examples
of humans doing different tasks and transfer that
[13:59] into uh into teaching robots how to do the
same tasks with very little additional amount
[14:05] of robotic data which is expensive to collect. I
would touch on data on a couple levels. Um first
[14:14] is you actually need a variety of data and for us
it's capturing data on the ground in the air think
[14:22] aerial data, drones, satellite data and what's
the size and scale of what you want to operate and
[14:29] look at and run models on. So that's one piece.
The second piece on physical AI, what makes it
[14:35] really challenging is that most of physical
AI data is actually not publicly available,
[14:40] which is the opposite of large language models
where you're, you know, it's it's easier to uh
[14:45] get from the internet. It's private data. It's in
enterprises. We're talking complex indooroutdoor
[14:50] facilities, warehouses, oil refineries, um
airports, uh you know, shipyards, construction
[14:59] sites. this important confidential information
that may also contain, you know, patents. It
[15:06] may contain security things. It may contain uh
policies. And for us, we've been working actually
[15:12] with some of the largest oil and gas companies to
prove out our technology to be able to recreate
[15:19] spaces in oil refineries. Oil refineries are
massive. One of the largest ones here in the
[15:24] US is 5x5 square miles. I mean, think about the
biggest strip that's only four miles long. Now,
[15:31] just, you know, volumewise, it's massive. And
in a refinery, you have valves, compressors,
[15:37] scaffolding, you have a lot of metal. It you
don't have GPS. It's a it's a highly intensive
[15:43] um environment. How do you actually navigate the
space? How do you think about change detection
[15:48] in that space? How do you understand um how to get
the right people to go do fixed maintenance repair
[15:54] in these large complicated spaces? And the ability
to see that environment in 3D in high fidelity and
[16:03] then operate it electronically at a mesh level.
It is part of the unique capabilities that we
[16:11] get to unlock. So I would say private data in the
enterprise is one of the key ingredients. There's
[16:16] not enough of it. You need it to train it. The
second one um is around the work we're doing with
[16:22] flexion because it's about smaller data sets that
can be uh contained in order to go from simulation
[16:28] to real. So um when a robot sees the floor, it
might have been trained on the stage floor which
[16:33] is like clean, but what happens if there's carpet,
right? What happens if water spilled on the floor?
[16:39] Like as a human, we sort of modify for a robot
in that real scenario. if it becomes very messy,
[16:44] you have to be able to train that in a way
that it understands how to recalibrate the
[16:49] balance or the motors or sensors. So, we think
about data both small and at large scale. Uh,
[16:56] one of my reflections from driverless cars
was that really we're building driverless
[17:00] cars wrong. And one of the reasons I think we're
building driverless cars wrong is because today
[17:07] to make a car drive by itself, you need to show
it thousands and thousands of hours of driving.
[17:13] And a human does not need that, right? We don't
sit in front of a television when we're learning
[17:19] to drive and watch thousands of hours of someone
zipping through the streets of San Francisco.
[17:24] Instead, what we do is when we are 17 years old,
we get behind the wheel for the first time and we
[17:30] spend about 30 hours on average learning to
drive. And even before we get to the wheel,
[17:37] we understand the basic rules of the road. We
understand physics. We understand not to bump into
[17:43] things. We understand that I've watched my parents
drive on this side of the road all my life up
[17:48] until the age of 17. And then we can adapt to the
task of driving in just 30 hours. And so to me,
[17:55] world models really present this new opportunity
to say a world model has this incredibly vast
[18:02] knowledge of the world. It has knowledge of
people and robots and driving and healthcare
[18:10] and defense and general things and sports and
all these different concepts. And it's then
[18:16] able to adapt with a very tiny amount of actual
driving data like ours to the task of driving.
[18:23] And that just feels like a totally new way to
think about autonomous systems. Gotcha. Well,
[18:29] I thank my parents for being relatively good
drivers. But I'm curious if we can hang out on
[18:33] this data subject for a little bit. I think people
are taking many different bets in the world model
[18:37] space. Uh Yan Lun, a pioneer in the AI space,
is off in France developing his own world model
[18:43] startup, really focusing on navigating through
the physical world. Other people think that you
[18:47] can learn pretty much everything you need just
from video, watching YouTube videos, gathering
[18:52] uh, you know, other types of egocentric videos. So
I'm curious for any panelists here kind of why are
[18:58] you making the bets you're making and you know is
there a video versus real world trade-off? Is you
[19:02] know the room under the tent for everybody? How do
we make those choices? So I think one big lesson
[19:09] from the language model uh the the recent history
of language models is that the simplest approach
[19:16] tend to scales the best. Um there were um if if
you look back into the you know mid2010s uh there
[19:24] were a lot of very complex architectures that
were being floated around about how um how you
[19:31] build systems that can reason and they can kind of
think for long horizons of time. And what ended up
[19:37] happening is that the approach that worked the
best uh is next token prediction and scaling
[19:42] that as to to an a massive uh a massive amount of
scale and at small scales those models you know
[19:50] language models could barely form sentences.
Uh at large scales they can they can you know
[19:56] solve very complex software tasks for uh for you
know and spend a day or days doing so. Um and the
[20:04] only you know there is a lot of techniques that
had to be uh invented along the way but really
[20:10] the the most important thing was scale was scale
of compute and data. Um and the same in our mind
[20:17] uh applies to video. Um it's the same models that
at small scale can barely generate you know moving
[20:24] pictures. uh at large scale can build a really
uh really sophisticated understanding of physics
[20:30] and they can simulate a wide range of things in
the real world. Um and so we're really taking
[20:36] the anal analogous bet of the thing that the what
we saw work really well in language uh to to the
[20:43] video domain. uh there is approaches out there
that um you know take there is more uh kinds of
[20:51] new kinds of architectures that are being floated
like JPA there is more uh 3D based approaches and
[20:57] I think those are all promising paths uh but in
our mind uh the the approach that tends to scale
[21:03] the best is the simplest approach and and that's
why we've we've been so bullish on this idea that
[21:08] scaling video models is actually the best path we
have available to uh to building really powerful
[21:14] systems that understand the world and complete
both. I agree with that. I I think um one of the
[21:21] reasons we have taken our approach is that there's
just something really interesting to be learned by
[21:29] interacting with a model that understands physics
to a level of depth that we as humans I don't
[21:36] think will be able to comprehend. And I think if
you just imagine this incredibly attentive learner
[21:42] that is paying attention to every single detail,
every facet in every video of everything ever,
[21:48] that thing is going to know things about our
reality, physics, science that we just can't
[21:54] imagine, and it's going to lead to breakthroughs
and discoveries. So that sounds cool to me.
[22:00] you know, I I love to travel and I want to
use that as the entry point. And for someone
[22:08] that has that background just personally and as a
business uh experience, I've been to 50 different
[22:15] countries, having seen real physical workplaces.
And when you step in an oil refinery, you step
[22:25] in a warehouse, you step in a semiconductor lab,
you step into a farm where you have autonomous,
[22:33] you know, vehicles, you have you step into
different worlds in different parts of this
[22:37] planet. This is the first point. There's only
one world. It's the real one. So our thesis is
[22:44] really simple. It's sort of let's start with
the ground truth as well as capture the datas
[22:48] in the places people work and where machines
are going to reside and then on top of that
[22:53] let's generate the simulations that optimize
whether it's intelligent you know predictions
[22:59] or maintenance or operations or safety evacuation
like the way you would do emergency evacuation in
[23:07] Southeast Asia is very different than other parts
of Western Europe because the infrastructure is
[23:14] different and there's a level of reality to the
culture of the way people move and the operations
[23:20] and the safety. And so our sort of thesis is
simple which is first you're going to need a
[23:25] lot of different kinds of data. You're going to
need sensors and visual data. Um you're going to
[23:30] need it at the ground in the air in the private
spaces that enterprises and people reside. The
[23:37] second piece is um the policies and the workflows
that need to be integrated into those agentic
[23:43] uh integration points are going to be varied
depending on the rules. I mean you can't fly a
[23:48] drone everywhere. It it every city and state and
province has very different rules about robotics
[23:54] and deliveries in the air and on the ground. So
those have to be taken into consideration. And
[23:59] then you know there different safety regulations.
So the point is I think we can accelerate not
[24:06] only the simulation from the real world to sim but
also really partner well with runway and others to
[24:13] unlock that sim to real out of the lab and that's
where we want to focus. Awesome. Well I think
[24:20] that's a a great jumping off point to uh maybe
getting into some of the barriers you're facing
[24:25] some of the key bottlenecks. So I think zooming
out at at a high level, the term world model has
[24:29] actually existed for years, decades. You know,
some computer scientists have been thinking about
[24:33] it. Obviously, roboticists to varying degrees have
been hammering away at this problem for decades.
[24:38] So now, you know, we're in this gigantic beautiful
ballroom and we're talking about world models and
[24:42] it's awesome to see all the interest in world
models as a term now. But I'm curious to hear
[24:46] what's kind of one key persistent bottleneck uh
that you're you know is just really slowing down
[24:52] progress or you know if you could just crack
this one problem but something where you're
[24:54] really putting all your energy towards solving.
Um I I feel one of the bottlenecks is the speed
[25:01] of exploration of world models. So with language
models, the developer community has done wonderful
[25:08] things to place language models into applications
ABCD. And in world models that has yet really to
[25:15] take place where you've had a world model that's
been adopted by the developer community and put
[25:20] into all of these applications, some of which you
couldn't comprehend. And if we wind back to GPT2,
[25:26] early GPT3, the killer application was marketing
generation, marketing copy generation, company
[25:32] called Jasper, if you guys remember. Um, and so
there was obviously an explosion that happened
[25:38] after that that has yet to happen with World
Models. Um, and that's why I I think the most
[25:42] obvious things are happening right now, driverless
cars, robots, uh, all these other applications,
[25:48] but there's so much more I think to explore. I if
anyone has any other thoughts. Well, I was think
[25:54] you know I I was thinking about a couple things
mainly because the data piece is complicated as
[26:00] you know was shared by everybody. There's just a
variety of data um that's not necessarily easily
[26:06] accessible and each set is unique depending on the
nature of like the work and tasks to be done. Uh
[26:12] but the real challenge I think is scale deployment
because every environment that you want to operate
[26:20] in the real world in the physical world is so
different and the parameters are so different and
[26:26] the structures are different and so uh figuring
out how to do rigorous testing before you scale
[26:33] so that you ensure you really are able to do that
whether it's you know workforce transformation
[26:42] of people, whether it's deployments at of large
scale devices, whether it's semi. You know,
[26:49] it's one thing if it's a deterministic space,
meaning, hey, in most of those spaces, you're
[26:56] going to have only robots, but there are a lot of
spaces where you're going to have these devices
[27:01] and human and unmanned um operation systems.
And and so this is sort of the intersection of
[27:07] that skilled deployment that I think is really
challenged. That's why I was pausing a bit. I
[27:11] was thinking about where the where the gap is. I
would say the the biggest challenge I believe in
[27:18] building systems that can operate autonomously in
the real world is this idea of generalization. Um
[27:23] so you know you see very impressive kind of robot
demos every day of you know robots doing back
[27:30] flips or doing very complicated like acrobatics
or diff like very difficult fine grain tasks.
[27:35] And in the majority of those demos, the uh the the
robot policy, which is the model that basically
[27:43] takes actions for the robot, uh has been trained
and kind of overfeitted specifically to that exact
[27:50] uh environment, to that exact task, to that exact
uh to that kind of exact scenario. Um and if
[27:57] uh if the scenario kind of deviates even a bit
from the the setting that the model was trained on
[28:04] uh then the the the the robot doesn't generalize
to it. It doesn't perform as well. Uh so if you
[28:10] have a tabletop task and maybe you move
the table by a few inches very often the
[28:15] that impressive demo would break down and the
robot would not be able to perform the task.
[28:19] Um and that's because uh traditionally um
you know the work in robotics has been very
[28:26] specialized towards solving particular tasks very
well. Uh and if there is no nothing um no longtail
[28:33] uh events or uh unexpected scenarios that happen
then that's totally fine. Uh if the environment
[28:38] is fully predictable that's totally fine. But if
you want a robot that can, you know, u come that
[28:44] can be in your kind of household and perform kind
of the variety of different tasks in in in that
[28:50] uh you perform in your home uh that can operate
at different kind of room layouts and different
[28:55] scenarios. They can kind of go uh outside and
kind of follow you to the store and like help you
[29:01] uh kind of along kind of your all the daily tasks
that you want to uh that you want help with. Then
[29:06] you need models that generalize. You need models
that can deal with uncertainty. you need models
[29:10] that can um you know handle the unexpected
scenarios that happen all the time in the real
[29:15] world. Um and so that's that's why models that
have this more general intelligence that comes
[29:20] from uh kind of large large scale pre-training
uh is is I believe so important and so key to
[29:27] uh to solving robotics. Awesome. Thank you. Well,
there's so much to talk about here. I feel like
[29:33] we could talk for hours if not years, but we
are running up against time. So I'm wondering,
[29:37] you know, three amazing companies here working on
world models, some really exciting vantage points.
[29:42] Maybe just some very quick closing thoughts from
each of you about something that the audience
[29:46] should walk away with both about where where world
models are today and then where they should expect
[29:50] them to go in the next year, 5 years. I think
world models will make physics breakthroughs.
[29:57] They will again discover things about this
thing reality and that will lead to incredible
[30:05] uh innovation. It will also lead to autonomous
systems that create abundance, save lives,
[30:11] improve lives, and we have a lot of work
to get there, but it's going to be great.
[30:17] you know, 80% of the world's economy is in the
physical world. And what we're doing in terms
[30:23] of the work around world models and physical
AI is going to unlock that opportunity in very
[30:28] different ways that are even just beyond our
imaginations. And that's what I'm excited about
[30:34] and um excited to see kind of the possibilities
what we can actually start doing now today in
[30:40] almost every space around the world if people
are curious. Yeah, I think building um I agree
[30:48] with all that and building um world simulators and
universal world simulators that can um learn from
[30:55] uh video and they can increasingly learn from
other modalities and other observations of the
[30:59] world will be the most transformational technology
of our time. uh there is you know so many problems
[31:05] that involve interacting with the real world and
are very involve very expensive data collection
[31:11] of just actually needing to um do you know a lot
of trial and error in the real world. Uh and being
[31:17] able to simulate those scenarios will be the key
to actually uh making progress in those tasks. Uh
[31:23] if you look at language models you know they've
made those incredible breakthroughs in coding
[31:27] and math. Uh but uh because the because we have
perfect simulators for those domains, we can we
[31:34] can spin up millions of programs and test out
have have models kind of test out different
[31:40] uh different programs, see how what the you
know if they execute correctly and learn from
[31:44] their mistakes. But we have no such simulator
for the real world. And so you will routinely
[31:50] see language models fail at very basic things
like running a vending machine business. There
[31:55] was a very interesting art kind of experiment
that Anthropic did that had Claude try to run a
[32:01] vending machine business and kind of fail pretty
spectacularly. Uh and that's because we don't
[32:06] have great simulators of the real world. And the
moment we have those simulators, we will unlock
[32:12] so much so much progress in solving like actual
real world tasks, in solving tasks in robotics,
[32:17] in biology, in kind of uh in uh in models that can
kind of operate and kind of organizations. Um and
[32:25] so it's a it's a very very exciting future ahead
of us with with world models. Awesome. Well, thank
[32:30] you so much to our panel and thank you so much
to all of you in the audience for joining us and
[32:34] diving into the world of world models. Uh we have
some exciting sessions coming up. So thank you.
