World Labs Acquires SceniX to Solve Robotics Data Bottlenecks

World Labs收购SceniX以破解机器人数据瓶颈 李飞飞（Fei-Fei Li）创立的World Labs正将其Marble生成模型与SceniX的仿真技术栈相结合，构建real-to-sim-to-real管线，用合成环境取代缓慢的物理数据采集。 World Labs收购了SceniX，旨在通过将其3D生成模型Marble与SceniX的仿真及物理技术栈相结合，解决机器人领域严重的数据稀缺瓶颈。讨论详细说明了纯视频模型为何难以保持物理一致性，以及real-to-sim-to-real管线如何实现可扩展的策略训练、反事实推理和快速评估。World Labs并不制造硬件或独立的机器人大脑，而是提供面向仓库和商业空间等半结构化环境、与具体具身形式无关的数字基础设施。在接下来的两年里，这支横跨两岸的整合团队力求在高价值物理自动化任务中打造标杆级企业客户部署。 World Labs收购SceniX，将生成式3D世界模型与仿真技术栈相结合，以攻克空间智能与物理机器人学习难题。 (0:00) 机器人技术面临严重的数据瓶颈，因为与LLM不同，它缺乏网络规模的物理token，这使得合成数据生成变得至关重要。 (6:04) World Labs的Marble模型能根据稀疏输入生成几何一致的3D环境，从而简化物理场景重建。 (6:04) 机器人基座模型必须作为一种多模态“全能模型”运行，将动作与文本、RGB帧和深度图融合在一起。 (10:55) 生成式视频模型缺乏空间和物理上的时间一致性，因此可靠的策略训练离不开扎实的3D仿真。 (12:35)

World Labs and recently acquired startup SceniX are developing generative 3D simulation infrastructure to bypass the severe data scarcity bottleneck in robotics, aiming to make robotic training and evaluation scalable without requiring massive physical data collection. Resolving the Physical Robotics Data Bottleneck (6:04) Unlike large language models that train on trillions of internet text tokens, robotics faces an acute shortage of physical training and evaluation data. World Labs utilizes its base generative world model, Marble, to synthesize geometrically consistent 3D environments from text prompts, single images, or sparse multi-view inputs. This foundation integrates with SceniX's real-to-sim-to-real pipeline, turning real environments into digital replicas so systems can avoid slow, expensive, and fragile physical data collection. Simulation, Counterfactual Reasoning, and Video Limitations (12:35) Pure video generation models often fail basic physical consistency by dropping objects or distorting geometry when camera perspectives shift. Yunzhu Li and Fei-Fei Li argue that simulation provides the structural physics and spatial persistence necessary for robust policy transfer. Rather than requiring molecular-level accuracy, simulations use domain randomization across lighting, mass, and friction to bridge the gap to reality. Simulation Mechanics for Training and Evaluation (19:39) Addressing arguments from researchers like Sergey Levine who favor physical data collection over drift-prone simulations, Fei-Fei Li points to autonomous vehicle development like Waymo, which relies on billions of simulation hours. Simulation enables counterfactual reasoning—testing scenarios that are too dangerous, rare, or costly to collect physically—while allowing systematic temporal acceleration beyond real-world constraints. Furthermore, digital environments resolve the wall-clock evaluation bottleneck, enabling rapid benchmarking between model checkpoints without dangerous hardware runs. Embodiment-Agnostic Tools for Semi-Structured Workspaces (27:17) World Labs is building embodiment-agnostic software infrastructure rather than physical robot hardware or standalone brains. The platform supports diverse physical form factors, from single industrial arms to mobile manipulators, allowing external developers to train or fine-tune models from scratch. The immediate commercial strategy targets semi-structured commercial environments—such as warehouses, commercial kitchens, hotels, and electronics assembly—as a practical stepping stone before attempting highly unpredictable home settings. Pragmatic Milestones and Measured Optimism (33:35) Yunzhu Li and Fei-Fei Li stress maintaining measured optimism, contrasting low-power biological intelligence—noting the human brain runs on approximately 20 to 30 watts—with the stricter reliability requirements of physical robotics compared to language models. Operating as a bicoastal entity across San Francisco and New York with modular tech stacks, World Labs' two-year roadmap focuses on proving quantitative automation value in select high-frequency verticals to establish lighthouse enterprise customer deployments. World Labs and SceniX have oriented their pipeline around semi-structured industrial tasks, but closing the energy efficiency and adaptability gap with biological systems remains an open, long-term challenge as systems transition toward fully unstructured physical environments.

李飞飞创立的 World Labs 通过收购具身智能初创企业 SceniX，试图将其三维生成基座模型 Marble 与“真实-模拟-真实”（Real-to-Sim-to-Real）仿真流水线相融合，以破解通用机器人领域的数据匮乏、反事实推理及物理评测瓶颈。该团队主张打造包含动作维度的多模态世界模型，并优先为半结构化工业场景提供与硬件本体解耦的数字仿真与训练基础设施。 破解数据瓶颈：从生成视频到三维模型 (6:04) 通用机器人研发与依赖抓取海量互联网文本的大语言模型（LLM）截然不同，面临严峻的数据匮乏难题。依赖外骨骼或人工遥操作采集真实物理数据不仅进度缓慢、设备损耗大，而且成本高昂。World Labs 开发的基础生成模型 Marble 能够依据文本、单张图片或稀疏多视角输入，直接生成几何一致的三维资产与场景（采用高斯泼溅或 3D 网格表征），大幅降低了对真实环境进行张量重建的时间与算力开销。 针对机器人学界关于纯视频生成模型与可控三维仿真的分歧，团队指出纯视频模型仅预测未来像素，容易出现物理规律幻觉、物体在镜头移动时突兀消失或几何畸变等问题。World Labs 与 SceniX 采取混合方案，利用三维世界模型在光照变化、遮挡和视角切换下维持严格的空间与物体一致性，从而构建出兼顾空间几何逻辑与可扩展性的合成数据飞轮。 动作条件全模态模型与正向模拟 (10:55) World Labs 将机器人的基础模型界定为包含动作维度的多模态“全能模型”（Omni-Models），使其摆脱仅能被动输出文本或像素画面的局限。在技术架构上，该模型必须将动作（如机械臂关节控制指令）与文本、RGB 图像帧、深度图及边界框等信号同步纳入输入与输出空间，实现双重功能融合。 当全能模型输入当前环境帧与特定动作时，它充当“正向模拟器”（Forward Simulator），能够准确预测物理世界在执行该动作后的状态转变；而当模型输入当前状态与既定目标时，它则化身为“策略模型”（Policy Model），输出完成该操作所需的一系列物理操控动作。这种正向模拟与策略生成的双向能力，为企业客户基于自身具体工业任务进行微调提供了通用底层支撑。 模拟环境的反事实推理与规模化评测 (19:39) 针对部分学术观点认为模拟环境必然与现实存在“仿真鸿沟”因而必须依赖物理实测，李飞飞与李昀烛（Yunzhu Li）明确指出模拟与真实数据采集是互补的演进关系。李飞飞强调，物理现实数据永远无法覆盖全部组合情况，模拟最不可替代的认知价值在于“反事实推理”（Counterfactual Reasoning）——即推演那些尚未发生、无法安全发生或缺乏现实数据的极端边缘场景，正如自动驾驶公司 Waymo 重度依赖数百亿小时模拟一样。 在开发流程中，评测（evals）是机器人模型迭代的关键反馈信号。李昀烛指出，在物理世界中验证一个模型检查点究竟达到 90% 还是 99.9% 的可靠性，受限于真实物理时钟，极其危险、缓慢且昂贵。SceniX 的仿真环境通过结构物理学与领域随机化（Domain Randomization）调节光照、摩擦力和质量分布，不仅能在数字端打破人类遥操作的时序速度上限，还能以极低成本对策略可靠性进行高精度的规模化评测。 实体无关定位与半结构化商业路径 (27:17) 在商业定位与硬件路线方面，李飞飞澄清 World Labs 不生产实体机器人本体，也不构建封闭的单一机器人“大脑”，而是提供本体无关（embodiment-agnostic）且算法无关的基础软件设施。外部机器人企业可将各类专属控制器、单臂、移动底盘或定制夹爪接入其生成的虚拟世界，开展从头训练或策略微调。 关于商业化落地场景，团队将应用环境划分为结构化（传统工厂）、半结构化（如电商物流仓储、商业厨房、酒店）及完全无结构化（充满儿童与宠物的家庭环境）三个梯级。李昀烛援引的一项涉及千余项日常任务的调研显示，三分之一的自动化诉求集中于基础清洁与整理等脏乱重复工序。团队认为现阶段模拟无须追求分子级别的绝对数学精度，只要具备防止真实迁移崩溃的物理保真度即可。因此，World Labs 将电子制造装配与仓储物流等半结构化场景确立为优先推进的商业化跳板。 李飞飞与李昀烛提醒，相较仅消耗 20 至 30 瓦能量的人脑以及仍需人类复核把关的语言模型，物理机器人面临着更为严苛的即开即用可靠性标准，维持“有节制的乐观”是应对具身智能鸿沟的核心准则。合并后的 World Labs 确立了两年内聚焦高价值垂直赛道打造灯塔标杆客户的目标，并开放了涵盖纯环境数字化（Real-to-Sim）与端到端软硬件部署（Real-to-Sim-to-Real）的合作管线。

## Background 该视频由投资机构 a16z 于 2026 年 8 月 3 日发布在 YouTube 频道上。内容聚焦于 World Labs 宣布收购机器人初创公司 SceniX 后，a16z 合伙人 Martin Casado 与李飞飞及李昀烛展开的深度对谈，围绕两家团队如何联手利用物理仿真与大世界模型攻克具身智能的数据瓶颈展开。 ## TL;DR World Labs 联合创始人兼 CEO 李飞飞与 SceniX 联合创始人李昀烛在对谈中深入剖析了两家团队整合的战略动因与技术架构。针对当前机器人研发面临的真实数据匮乏与高成本评测难题，团队阐述了依托 Marble 基础模型与 Real-to-Sim-to-Real 管线构建物理与几何一致性世界模型的路径。李飞飞强调仿真在支持“反事实推理”与极端边缘场景学习中的不可替代性，并指出具身基础模型必须同时兼具前向仿真器与策略模型的双向输出能力。面对当前人形机器人赛道的过热预期，团队倡导审慎克制的工程实用主义，主张以形态无关和模型无关的基础设施优先攻克半结构化工业场景，打磨高可靠性与高能效的商业化标杆用例。 ## Atmosphere & style 整场对话在严谨前沿的硬核技术研讨与轻松幽默的师生老友互动之间自如切换，充满科技领袖的高维视野与一线极客务实作风的碰撞。李飞飞以深邃敏锐的认知哲学把控宏大叙事，而李昀烛则用“让机器人干实事”的工程师直觉拆解微观落地阻碍，配合 Martin Casado 节奏明快、直击痛点的商业追问，使谈话张弛有度。现场频繁爆发关于五岁孩子与狗弄乱客厅、机器人视频必须十倍速快进的会心笑声，在理性剥离人形机器人浮躁泡泡的同时，传递出对物理世界 AI 底层基建演进坚定而克制的乐观感染力。 ## Bulletin - [01:08](https://www.youtube.com/watch?v=-tabaM5l3s0&t=68s) World Labs 宣布收购具身智能初创团队 SceniX，将其 3D 世界生成基座 Marble 与 Real-to-Sim-to-Real 技术栈紧密整合，攻坚机器人在物理空间中的数据与落地瓶颈。 - [12:35](https://www.youtube.com/watch?v=-tabaM5l3s0&t=755s) 针对纯视频生成模型在接触交互中频繁发生物体闪烁或凭空蒸发的物理幻觉，World Labs 主张融合几何物理先验与端到端学习，构建具备严格 3D 一致性的混合世界模型。 - [23:01](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1381s) 机器人落地的最大隐形瓶颈在于系统评测（Evaluation），高保真数字虚拟世界通过严密的真实对齐验证，能够以数倍于物理原子的迭代效率安全区分模型表现。 - [29:17](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1757s) 团队呼吁对通用人形机器人保持“审慎克制的乐观”，商业化应优先攻克仓储、物流等半结构化场景，并提供形态与模型均无关的底层空间智能基础设施。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=-tabaM5l3s0&t=0s) 空间智能（Spatial Intelligence）旨在赋予机器生成、理解、推理及与空间交互的能力，大世界模型是维持多视角与物理几何一致性的基石。 - [05:14](https://www.youtube.com/watch?v=-tabaM5l3s0&t=314s) 机器人学面临与大语言模型根本不同的数据荒漠，无法单纯依靠互联网数据，唯有借助高保真仿真与合成数据飞轮才能解锁 Scaling Law。 - [10:56](https://www.youtube.com/watch?v=-tabaM5l3s0&t=656s) 具身基础模型具备双向机制：输入视频帧与动作时作为前向仿真器预测物理演变，给定任务目标时则作为策略模型输出动作决策。 - [12:35](https://www.youtube.com/watch?v=-tabaM5l3s0&t=755s) 纯视频生成模型缺乏底层 3D 空间结构，交互时常违背因果逻辑；未来的世界模型必须结合几何物理结构与数据驱动学习的混合架构。 - [16:22](https://www.youtube.com/watch?v=-tabaM5l3s0&t=982s) 机器人环境建模无须做到百分之百微观精确，只要抓住刚度与摩擦力等核心力学先验并配合域随机化，即可有效达成真实物理迁移。 - [18:13](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1093s) 仿真的本质认知功能是“反事实推理”，使智能体在完全零现实风险的前提下穷举推演极端罕见工况，学会应对复杂边缘场景。 - [20:31](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1231s) 真实世界遥操作采集效率低下，仿真系统通过系统化域随机化覆盖全状态空间，并能在物理可信前提下突破人类作业的极限速度。 - [23:01](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1381s) 实体评测受限于物理法则导致“挂钟时间”极其昂贵缓慢，数字世界通过真实对齐（Real-to-sim alignment）极大压缩了算法验证周期。 - [27:15](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1635s) World Labs 平台坚持形态无关（Embodiment Agnostic）与模型无关（Model Agnostic），定位为供各类机器人大脑训练与测试的基础设施底座。 - [30:04](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1804s) 商业自动化遵循从结构化走向半结构化的客观规律，当前阶段盲目追求通用人形机器人进入复杂家庭环境并非最具实用性的工程路径。 - [32:31](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1951s) 面对 AI 狂潮需秉持“审慎克制的乐观”，人脑仅需 30 瓦能耗即可完成敏捷空间交互，机器人在软硬件能效与可靠性上与人类仍有巨大鸿沟。 - [39:03](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2343s) World Labs 确立旧金山与纽约双总部架构，两年内的核心目标是以模块化 Real-to-Sim-to-Real 管线在重点垂直工业场景打造标杆灯塔案例。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=-tabaM5l3s0&t=0s) 引言与空间智能的定义 人类智能的核心之一在于能够在脑海中进行大量的推演与模拟。与纯粹基于真实世界采集的数据不同，模拟承担着一个极其关键的角色——进行“反事实推理”（counterfactual reasoning）。World Labs 正在攻克 AI 的下一个前沿领域，即“空间智能”（Spatial Intelligence）。所谓空间智能，是指赋予机器生成、理解、推理以及与空间交互的能力，无论这种空间是物理世界还是虚拟世界。而通往空间智能这一终极目标的必经之路，正是构建大世界模型（Large World Models）。 这种大世界模型旨在构建一个具有全局一致性的世界——在空间、时间、多视角以及不同交互行为下均保持连贯一致。虽然物理世界的互动极具吸引力，但现实本身完全可以是一个“多元宇宙”（multiverse）。World Labs 的技术愿景不仅是服务于物理实体，更是为广大的创作者和开发者提供能够在各种虚拟空间（如视觉特效 VFX、游戏、设计等）中自由创造与行动的基础工具。 - 空间智能的核心要素：生成（Generate）、理解（Understand）、推理（Reason with）与交互（Interact with）。 - 大世界模型（Large World Model）是实现跨时空、多视角和复杂交互下“物理与几何一致性”的技术基石。 --- ### [01:08](https://www.youtube.com/watch?v=-tabaM5l3s0&t=68s) 机器人与物理交互：World Labs 收购 SceniX 的契机 主持人 Martin Casado 抛出疑问：以往大家普遍认为让模型在物理世界中“行动（Act）”是一个遥远的未来目标，为何 World Labs 此时会选择收购一家机器人初创团队 SceniX？Fei-Fei Li（李飞飞）解释道，机器人技术始终是空间智能最重要、最深刻的应用场景之一。尽管虚拟创作空间价值巨大，但真正让机器能够在物理空间中实施动作，是未来 AI 体系中最激动人心的能力，也是 World Labs 长期使命中既定的一部分。 SceniX 的联合创始人 Yunzhu Li（李昀烛）现任哥伦比亚大学助理教授，博士毕业于 MIT，曾在斯坦福大学跟随 Fei-Fei 开展博士后研究。他强调自己是一个“非常讲求实用的人”，其科研目标始终只有一个：“我希望机器人能真正奏效（My north star is I want the robot to work）”。通用机器人当前面临的核心瓶颈主要集中在训练（Training）与评估（Evaluation）两大环节，而 SceniX 创立的初衷正是搭建一套完整的 Real-to-Sim-to-Real（真实-模拟-真实）技术栈，借此攻克通用机器人的落地难题。 ```mermaid flowchart LR accTitle: Real-to-Sim-to-Real 数据闭环 accDescr: 展示从物理世界捕捉、数字模拟环境训练与反事实评估，再部署回真实机器人的闭环流程。 subgraph S1["真实物理世界"] RealEnv["真实环境感知与捕捉"] Deploy["策略部署与实体机器人运行"] end subgraph S2["数字模拟世界"] SimEnv["高精度物理与渲染模拟环境"] TrainEval["可扩展合成数据训练与反事实评估"] end RealEnv -->|"高一致性映射 (Alignment)"| SimEnv SimEnv -->|"海量生成/零成本试错"| TrainEval TrainEval -->|"闭环策略迁移"| Deploy ``` --- ### [05:14](https://www.youtube.com/watch?v=-tabaM5l3s0&t=314s) 偶遇客户：Marble 模型与机器人数据瓶颈 关于两家团队的结合，Fei-Fei 分享了一段颇具戏剧性的幕后故事：许多人猜测由于 Yunzhu 曾是她的博士后，两家公司肯定早就私下串通好要合并，但事实完全出乎意料——SceniX 最初其实是 World Labs 的“付费客户”。World Labs 在去年初冬（11月至12月左右）低调发布了代号为 Marble 的基础模型第一代版本，SceniX 团队看到后立刻注册签约成为客户。Fei-Fei 偶然在后台客户名单里发现了熟悉的名字，当场打电话给 Yunzhu 惊呼：“哇，原来这家公司是你的！” 这一戏剧性巧合揭示了两家公司底层需求的惊人互补： - **Marble 基础模型**：能够接收单张或多张图像、文本等提示词作为输入，直接生成具备几何一致性的三维世界，并支持以 3D Gaussian Splatting 或网格（Mesh）等多种 3D 几何形式输出。 - **机器人的根本痛点**：机器人学与大语言模型（LLM）有着本质差异。LLM 可以在互联网上获取近乎无限的海量文本数据，而机器人领域无论在训练还是系统评估上都极度匮乏数据。 - **解锁 Scaling Law 的钥匙**：机器人系统如果要实现涌现与泛化，必须找到一条能够释放规模法则（Scaling Law）潜力的路径，而模拟与合成数据正是撬动这一法则的核心杠杆。 --- ### [07:13](https://www.youtube.com/watch?v=-tabaM5l3s0&t=433s) 团队互补性与合并决策背后的技术协同 两家团队的技术背景具备高度的结构互补性。SceniX 的三位联合创始人带来了极为互补的硬核基因： - **Yunzhu Li（李昀烛）**：全栈机器人学技术领袖，从底层硬件交互到高层策略建模均有深入研究，同时兼具极佳的学术与工程影响力； - **Changxi Zheng（郑昌玺）**：哥伦比亚大学教授，在物理模拟（Simulation）与计算机图形学领域享誉全球，拥有维塔数码（Weta Digital）特效制作、腾讯研发及深厚创业经历； - **Sunonni（工程负责人）**：资深工程架构师，此前创立的公司被亚马逊收购，长年在亚马逊负责跨技术栈的大规模计算机视觉工程系统。 在具体技术分工上，SceniX 擅长在物理模拟、可微分渲染与机器人闭环控制等下层技术，其过往的 Real-to-Sim-to-Real 流程在物理环境的外观、精细几何以及动态交互（即施加动作后环境如何演变）的重构上偏向于“密集重构（Dense Reconstruction）”，计算与数据成本相对较重。而 World Labs 则在稀疏重构（Sparse Reconstruction）、3D 视觉几何以及前沿生成模型（如 Marble）上积累了雄厚的突破性成果。 这种端到端能力的打通，使 SceniX 得以借助 World Labs 的轻量化稀疏生成模型，将现实世界以极低成本高保真地映射进模拟引擎，从而彻底补齐了从“世界模型生成”到“物理仿真与机器人落地下游”之间的关键断环。 ### [10:56](https://www.youtube.com/watch?v=-tabaM5l3s0&t=656s) 构建机器人具身基础模型：前向仿真器与策略模型的结合 Martin Casado 抛出了一个核心问题：World Labs 是否正在为机器人研发基础模型？李飞飞（Fei-Fei Li）给出了明确的回应，证实 World Labs 正在全力打造底层基座模型。她指出，当前最令人振奋的技术前沿是全能全模态模型（Omni-models），它们不仅能够接收文本、图像等多模态输入，而且能输出多模态表征。对于机器人领域的基础模型而言，输出的定义至关重要——它绝不仅限于生成对世界状态的描述，更必须包含物理动作（Actions）的输出，这正是具身智能不可或缺的组成部分。 李云珠（Yunzhu Li）从技术机制上对这种多模态基础模型进行了深层剖析。他解释道，机器人基础模型必须综合处理单帧画面、文本、图像以及深度（Depth）等各种模态，而“动作”在其中扮演着双向转换的核心角色。当模型将视频帧与动作作为输入时，它本质上是一个“前向仿真器”（Forward Simulator），能够精确预测当智能体施加某一动作后，周围环境将会发生怎样的物理演变；而当模型将动作作为输出时，它则转变为一个“策略模型”（Policy Model），旨在根据给定的目标任务，计算出机器人在现实环境中究竟应该执行什么动作才能逐步逼近目标。这种双向贯通的模型架构不仅能赋能学界与工业界深刻理解环境并进行交互，更为下游具体的机器人垂直应用提供了可微调的骨干网络（Backbone），从而满足客户对稳定度与效率的高严苛要求。 ```mermaid flowchart TD accTitle: 机器人基础模型的双向运作架构 accDescr: 展示了多模态基础模型作为前向仿真器与策略模型的双向机制 subgraph InputModality ["多模态输入"] direction TB A["感知数据 (文本 / 图像 / 深度 / 视频帧)"] end subgraph FoundationModel ["多模态全能模型 (World Labs Backbone)"] direction TB B{"核心模型机制"} B -->|帧 + 动作作为输入| C["前向仿真器 (Forward Simulator)<br/>预测环境状态演变"] B -->|目标状态作为引导| D["策略模型 (Policy Model)<br/>输出逼近目标的动作"] end subgraph Downstream ["下游落地"] direction TB E["机器人垂直应用微调<br/>(满足工业级可靠性与效率)"] end A --> FoundationModel C --> Downstream D --> Downstream ``` --- ### [12:35](https://www.youtube.com/watch?v=-tabaM5l3s0&t=755s) 纯视频生成模型与真实空间一致性的鸿沟 谈及当前投资圈与机器人赛道的现状，Martin 提出了一个尖锐的对比：市场上绝大多数机器人初创公司都在追逐“纯视频生成模型”（Video-only models）这一热门路径，而 World Labs 与 SceniX 却选择深耕 3D 表征与物理仿真。针对这种分歧，李云珠直言不讳地指出了视频生成模型的硬伤所在——“一致性”（Consistency）的缺失。要让机器人学会在物理世界中自处，虚拟环境必须在空间几何、时间演进、视点切换以及物体交互等全维度维持严密的真实一致性。 李云珠举了一个非常直观的物理交互漏洞：在现有的许多视频预测模型中，当机器人推开一个物体时，由于缺乏深层空间几何支撑，物体经常会“不可思议地凭空消失”。这种严重的物理幻觉根本无法为机器人提供有效的动作学习信号。World Labs 开发的 Marble 世界生成技术恰恰填补了这块基石，它生成的世界天然具备 3D 几何与物理一致性。李云珠指出，他们构建的物理仿真基础设施可以提供极强的初始动量，随后通过数据飞轮与真实环境中的机器人策略执行相融合；未来的世界模型绝非单纯的“硬编码物理仿真”或“纯数据端到端学习”，而是一种融合了两者优势的混合架构——既能捕捉物理问题的本质结构，又具备依赖海量数据与算力持续扩展（Scaling）的潜力。 - **一致性的四大维度**：机器人学习必须依赖在空间、时间流逝、多视角变换以及物理接触交互中始终如一的环境。 - **纯视频模型的典型缺陷**：在处理外力接触与遮挡时容易发生物体闪烁、形变或凭空蒸发，破坏了因果逻辑。 - **混合路径架构**：以几何与物理结构为初始骨干，结合实际执行采集的数据飞轮，既保证底层逻辑正确，又保留数据驱动的学习扩展性。 --- ### [14:30](https://www.youtube.com/watch?v=-tabaM5l3s0&t=870s) 极客务实主义：“让机器人干实事”与脏活累活的需求 在谈话中，Martin 聊到了李飞飞多年来以“3D与空间智能”为指路明灯的宏大远景，随即好奇李云珠是否也有着类似的哲学追求，还是纯粹的技术实用派。李云珠笑着给出了极其干脆的回答："My north star is to make robots work."（我的北极星指标就是让机器人在真实环境中干成实事。）他回忆起自己在李飞飞实验室做博士后时的一项工作：为了构建机器人基准测试集，团队曾向公众发起过广泛调研，统计大众最希望机器人为自己做什么。 调研收集到了上千个具体任务，令人惊讶的是，其中整整三分之一的需求都高度集中在“清洁”（Cleaning）上。人们最渴望机器人代劳的，恰恰是那些枯燥、繁复且脏乱的日常杂务（"dull and dirty tasks"）。李飞飞随后高度赞扬了 SceniX 团队的务实特质。虽然李云珠和甘创（Chuang Gan）都来自学术界前沿，但团队从成立第一天起，本能反应就是扎进真实的产业一线，紧密对接工业实验室、智能仓储物流以及电子装配产线的实际设计伙伴与客户需求。在李飞飞看来，这种不玩学术花架子、直奔产业最痛点的极客务实作风，正是促成双方强强联合的关键动因。 --- ### [16:22](https://www.youtube.com/watch?v=-tabaM5l3s0&t=982s) 逼真度辩证法：机器人建模并不需要百分之百的绝对精确 Martin 提出了一个关于技术跨界整合的深刻疑问：World Labs 过去涉足了大量影视与艺术等创意场景，在这些领域，“失真”或非物理现象有时甚至是一种风格化的艺术表达；然而机器人交互却必须极为精确，两者在底层逻辑上如何调和？李云珠对此给出了出人意料的专业解答：从长远来看这两者完全可以统一，且机器人技术中的环境建模“其实根本不需要做到百分之百绝对完美”。 为了阐明这一点，李云珠以当下发展成熟的足式机器人与飞行器为例进行了对比分析： | 机器人形态 / 应用 | 真实环境中的复杂挑战 | 仿真环境的处理方式 | 迁移成功的原因 | |:--- |:--- |:--- |:--- | | **四足机器人 / 双足人形** | 在积雪、泥地或灌木丛等复杂非结构化地形中稳定行走 | 无需高保真建模每一片雪花或每一根树枝的微观物理特性 | 抓住地形刚度与摩擦力的核心物理结构，配合数字空间内的**域随机化**（Domain Randomization） | | **无人机 / 扫地机器人 (Roomba)** | 空气动力扰动、室内多样化家具摆放与地面材质变化 | 建立宏观动力学与边界轮廓模型，忽略无关的视觉/微观几何冗余 | 核心力学结构保真，模型控制策略对微小偏差具备内在鲁棒性 | 李云珠强调，SceniX 与 World Labs 正在联合攻克的关键科学课题，正是去厘清“环境建模究竟需要达到怎样的保真度阈值”，才能让智能体在虚拟数字世界中训练出的能力，平滑、无损地迁移回物理现实之中。 --- ### [18:13](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1093s) 数据飞轮演进与李飞飞的“反事实推理”认知观 面对“仿真环境终将偏离物理现实，唯有海量真机实采数据才可信”的学术观点（例如 Sergey Levine 所代表的流派），李云珠指出这并非二元对立的矛盾命题。仿真系统本质上是一个预测环境状态演化规律的模型，它不等于僵化的纯经典力学方程，而是物理先验与机器学习的有机结合体。在整个数据演进飞轮中，早期因真实数据匮乏，系统更侧重依赖物理和几何先验以锁定结构一致性；而随着机器人部署铺开，来自真实场景的数据持续回流，模型将自然地过渡为以学习为主导的环境模拟器。 李飞飞随后从认知科学的角度进行了深层次的升华。她指出，将仿真与真实世界对立起来是一种思维误区；人类智能本身就是在头脑中进行无休止的“内部仿真”。物理数据固然重要，但仿真扮演着物理数据永远无法替代的独特认知功能——**反事实推理**（Counterfactual Reasoning）。 > "There's a very important role simulation plays that real world data doesn't play, which is counterfactual reasoning — is that you play out events that hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it." > —— 李飞飞 人类每天都在脑海中推演那些从未发生过、在现实中过于危险而不能发生、或者由于样本极度稀缺而无法收集数据的极端边缘场景（Corner cases），并在这种脑内推演中学会了如何采取行动。机器人要想真正走向通用与安全，同样必须借助世界模型所赋予的反事实推演能力，这也是纯粹依赖现实世界动作采样的传统方案所无法企及的维度。 ### [20:31](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1231s) 仿真在机器人中的核心价值：反事实推理、可靠性与超越人类的速度 李飞飞借由西班牙队夺冠的比赛做比，指出任何高水平竞技在开赛前都会进行白板推演或数字模拟，仿真的本质是“反事实推理”（Counterfactual Reasoning）。这种能力对机器人而言至关重要，因为真实物理世界根本不可能提供足够量级且包含所有边界情况的数据。她以自动驾驶行业为例：Waymo 官方早已公布其系统经过了数十亿小时的仿真训练，Waymo 对仿真的依赖甚至远远超过了纯实景路测数据。在李飞飞看来，自动驾驶汽车在广义上其实只是“最简单形态的轮式机器人”，仿真在更复杂的机器人学习中必然占据绝对核心的位置。 李允珠进一步从工程与商业落地角度，将仿真带来的核心价值拆解为“可靠性”（Reliability）与“效率”（Efficiency）两个层级： 1. **可靠性**：机器人要在真实物理环境中稳定作业，模型必须系统性地覆盖所有可能的状态空间。仿真支持系统化的域随机化（Systematic Randomizations），工程师可以随意控制光照、摩擦力、物体几何形状、材质类别乃至各种底层物理参数，确保系统拥有足够的泛化冗余度。 2. **效率**：目前机器人领域主流依赖遥操作（Teleoperation）采集数据，但操控外骨骼或遥操作设备的采集速率往往比人类真实干活还要慢。然而客户并不满足于人类速度，他们要求机器人超越人类作业极限。在真实世界中，你不能靠简单粗暴地给电机“加速”来提升作业效率，因为重力与动力学规律是固定的；但在仿真中，算法可以系统性地推演加速后的运动状态，在充分考虑动态环境变化的前提下，训练出兼具极高作业速度与稳定性的策略。 ```mermaid flowchart TD accTitle: 仿真平台赋能机器人学习的双重核心价值 accDescr: 展示仿真系统如何通过提升可靠性与突破效率瓶颈，为机器人训练与部署提供核心支持。 subgraph Reliability ["可靠性（Reliability）"] A["系统化域随机化"] --> B["覆盖极端物理参数<br/>光照 / 摩擦力 / 几何结构"] B --> C["全状态空间覆盖与泛化"] end subgraph Efficiency ["效率（Efficiency）"] D["突破遥操作采集上限"] --> E["仿真动态系统加速"] E --> F["超越人类极限作业速度"] end C --> G["高可靠、超高速真实世界部署"] F --> G ``` ### [23:01](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1381s) 两大落地场景之一：为何“评测”（Evaluation）是机器人最大的隐形瓶颈 当马丁·卡萨多（Martin Casado）追问客户具体的业务使用场景时，李允珠明确指出，核心诉求集中在“评测”（Evaluation）与“训练”（Training）两端，而其中最容易被行业忽略的恰恰是评测。在传统 AI 领域，“Eval” 是开发者赖以生存的日常工具，但在物理机器人领域，评测的难度完全不可同日而语。机器人评测要求团队必须精确量化一个模型权重 Checkpoint 的表现是 95% 还是 99.9% 的成功率。在工业落地中，最致命的衡量指标是“挂钟时间”（Wall-clock time）——团队需要耗费多长时间才能在统计学上区分出 90% 成功率的模型与 92% 成功率的模型。 如果这一评测过程全部放在真实世界中进行，迭代周期将比大语言模型（LLM）慢上数个数量级。马丁和李飞飞调侃道：“毕竟真实世界里原子（Atoms）必须穿过空间，必须遵从物理定律，你去看网上的机器人演示视频，全都是被 8 倍速、10 倍速快进过的！”李允珠强调，真实物理测试不仅慢，而且危险且极其昂贵。SceniX 与 World Labs 打造的数字化虚拟环境经过了严格的“真实世界对齐验证”（Real-to-sim alignment），如果在仿真中 Checkpoint A 的表现优于 Checkpoint B，那么在真实场景中它大概率同样表现更优。这种确定性让客户敢于将海量、高危、复杂的评测流程全面转移到仿真中，极大压缩了验证时间。 ### [25:36](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1536s) 两大落地场景之二：全场景可控训练与旺盛的市场需求 在训练层面，仿真所提供的“可控性”是纯物理采集无法比拟的。真实世界遥操作不仅速度极其缓慢，还会受到硬件本体数量、操作台搭建数量以及复杂的数据清洗与运营成本的严重制约。而在高保真的数字化世界中，工程师可以精确界定自己究竟覆盖了哪种概率分布，并在该分布内做出机器人能够百分百可靠运行的确定性断言。 李飞飞透露了一个内幕细节：早在 World Labs 与 SceniX 正式洽谈收购之前，World Labs 发布的 Marble 就已经遭遇了客户需求的“排队轰炸”。大量早期的前沿机器人研发团队、具身基础模型团队，乃至下游极具务实工业落地场景的客户，纷纷打来电话寻求高保真数字环境的支持。World Labs 当时甚至没有足够的精力去一一承接这些需求，这充分证明了全行业对于物理可信仿真训练环境的饥渴程度。 - **遥操作瓶颈**：实体遥操作受到物理设备台数与人工人时的严格线性束缚，单位时间边际采集成本极高。 - **分布可验证性**：仿真训练让开发者能清晰定义与追踪训练集覆盖的数据分布范围，建立模型确定性边界。 - **市场井喷信号**：在技术完全打通之前，从模型层到底层应用层的主流机器人企业已自发形成强大的商业买方诉求。 ### [27:15](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1635s) World Labs 的定位：做形态无关与模型无关的“世界级”基础设施 针对外界普遍的猜疑——“李飞飞做机器人是不是意味着 World Labs 要买 3D 打印机造硬件、自己写机器人大脑？”马丁在此作出了明确澄清。李允珠指出，World Labs 正在构建的是一套底层的软件基础设施，它的本质是创造一个能够让外部智能体（Agent）和机器人“大脑”在其中感知、学习与评测的虚拟世界。这一平台天然具备两大核心特性： 1. **形态无关（Embodiment Agnostic）**：现阶段各家机器人硬件架构千差万别，有的客户采用单机械臂，有的采用双臂系统；有的使用固定基座机械臂，有的使用移动复合机器人（Mobile Manipulators）；末端执行器更涵盖了简单的二指夹爪（Grippers）到复杂的仿生灵巧手。World Labs 的系统能够极度顺畅地挂载并集成各类物理本体，将其直接置入数字世界进行任务闭环。 2. **模型无关（Model Agnostic）**：平台不绑定特定的算法模型结构，既可以利用生成的数据从零训练专属模型，也能用于对现有的多模态大模型进行后训练（Post-training），例如视觉-语言-动作模型（VLA）或世界-动作模型（WAM）。 World Labs 的边界非常清晰：专注于打造最高标准的物理可信世界基础设施，让各种形态的机器人在其间打磨出能够在真实环境中稳定工作的泛化能力。 ### [29:17](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1757s) 人形机器人过热反思：从全结构化走向半结构化现实 马丁顺着话题切入关于人形机器人（Humanoids）的现实讨论，提到李允珠此前曾表示目前外界对人形机器人的落地预期“过于激进”，未来更可能是优先在仓储等受限场景中分步落地。 李允珠剖析了物理世界机器人自动化的客观历史演进规律，强调机器人技术的落地从来不是一蹴而就的，它必须经历三层物理环境复杂度的递进： - **全结构化环境（Fully Structured Environments）**：人类对环境中的所有物理位姿、几何构型和流程具备绝对先验知识和 100% 的精准控制。最典型的代表就是传统的汽车制造流水线和封闭式工厂车间，这类场景早在几十年前就已经完成了高度自动化。 - **半结构化环境（Semi-Structured Environments）**：具有一定的既定布局和业务规则，但存在大量动态变量（例如货品位姿不同、货箱规格各异、有人机协作盲区等），现代智能仓储与物流分拣中心便是典型的半结构化场景。 - **非结构化环境（Unstructured Environments）**：完全开放且极度不可预测的空间，比如杂乱的家庭居所、城市人行道及野外大自然。 现实中机器人落地的真正挑战，在于如何稳健地从全结构化车间跨越到充满变量的半结构化场景，随后才能谈论彻底征服完全非结构化的人类日常世界。这也是为何必须依托强大的仿真系统与世界模型，先在可控的半结构化工业场景中建立足够的确定性。 ### [30:04](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1804s) 半结构化环境与人形机器人的通用身体悖论 李云珠深入剖析了机器人面临的环境分级。在工业机器人统治了数十年的高度结构化工厂之外，存在着诸如亚马逊仓库、餐厅、酒店等“半结构化环境”（semi-structured environments）。在这些场景中，运营方对空间拥有一定的控制力以降低自动化难度，但仍必须面对衣物、软性杂物等形态不可控的物体。而真正的“终极挑战”（grand challenge）是像普通人家中那样的完全非结构化环境。马丁·卡萨多随即打趣道，尤其是他自己的家——屋里养了三条狗，还有一个五岁的孩子，现场顿时爆发出一阵笑声。李云珠指出，算法的鲁棒性取决于对潜在突发场景的充足覆盖度；因此在商业落地上，先专注于半结构化环境，再逐步迈向完全非结构化环境，是一条更加务实且可持续的工程路径。 李飞飞从演化生物学与工程哲学的交叉视角进行了深刻补充。人类身体是自然演化在非结构化生存环境中千锤百炼的结果——如果人类作为一个物种唯一的生存目标只是爬树，双手和身体形态绝不会长成现在的模样。人类躯体演化成了极其通用但并不在特定单项任务上绝对最优的形态，一切都是为了在未知的野外生存。然而，从商业实用主义与技术落地的角度来看，用通用的人形身体去解决非结构化环境的所有问题，不仅是当前最艰难的命题，甚至根本不是正确的切入方式。更聪明的做法是针对特定细分问题采用特化形态的机器人。正因如此，SceniX 平台的核心要求是做到“本体无关”（body-agnostic），使其空间仿真与训练基础设施能够适配各种不同的机械本体与半结构化场景。 ```mermaid flowchart TD accTitle: 机器人运行环境的复杂度演进阶梯 accDescr: 展示从高度结构化工业环境到完全非结构化家庭环境的难度递进与当前技术落地重点 subgraph Structured ["结构化环境（工业制造）"] A["高精度固定工位<br>确定性刚性物体"] end subgraph SemiStructured ["半结构化环境（商业与仓储）★当前重点"] B["仓储物流 / 餐厅 / 酒店<br>空间受控 + 局部不可控物体"] end subgraph Unstructured ["非结构化环境（家庭终极挑战）"] C["家庭起居空间<br>宠物、儿童、高动态复杂干扰"] end Structured --> SemiStructured --> Unstructured ``` --- ### [32:31](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1951s) 机器人能效悖论与审慎的乐观主义 马丁·卡萨多从经济学视角提出了一个尖锐的对照：以生成式语言模型为例，AI 生成散文或编写代码的速度比人类快一万倍，成本极低，因为人类大脑在符号处理上并非最极致高效，因而在算力与经济账上极具吸引力。然而在物理世界中，人类大脑与身体在三维空间导航、物品抓取等动作上的能效高得不可思议。马丁由此发问：在可预见的未来（比如五年内），我们是否可能造出在基础体力劳动（如最低时薪杂务）上达到人类能效水平的机器人？还是说这永远无法实现？ 李云珠认为这将经历极其漫长的时间。真实的机器人是一个极其复杂的硬核工程系统，从顶层的大脑决策算法、底层控制软件、硬件执行机构，细化到机械手指表面的摩擦系数，系统各层必须严丝合缝地配合迭代。尽管技术演进速度早已超越了他读博初期的想象，但要真正达到人类级别的能耗效率与通用作业能力，行业必须保持冷静与校准。李飞飞更是给出了精辟的总结：“马丁，在当今的 AI 领域，最难的一件事就是保持恰到好处、审慎克制的乐观（measured optimism）。”她指出，人脑仅需约 30 瓦的功率就能运转，即使是大语言模型在单位功耗比上与人脑相比都相去甚远；虽然在特定软件任务上 AI 展现了超高价值，但在物理机器人领域，人类目前距离这种能效奇迹还差得很远。 - **人脑能效对比**：人类大脑维持全身运转和高敏捷三维交互的能耗仅约为 30 瓦（30 watts）。 - **机器人系统复杂性**：必须跨越硬件动力学、环境物理交互（如指尖摩擦系数）、软件与模型端的多重集成壁垒。 --- ### [35:05](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2105s) 实体世界对可靠性的苛刻要求与数字世界的解法 当被问及这种高难度现实是否改变了团队的战略雄心时，李云珠表示，与 World Labs 的结合极大地拓宽了前瞻边界。他强调了机器人与大语言模型在容错率上的根本鸿沟：当前顶尖的大语言模型虽然能力惊人，但普通用户绝不敢盲目放权让它直接代订机票或酒店房间，通常需要人在回路（human-in-the-loop）检查确认；然而对机器人模型而言，现实世界的诉求是“开箱即用”（out of the box）并且具备绝对的物理可靠性。 实体环境中的机器人不允许出现灾难性试错，而且物理世界中收集失败案例和极端边缘情况数据的成本极高且极具危险性。因此，依托 World Labs 打造的可扩展数字世界（scalable digital worlds）与高保真仿真环境，让机器人能够安全、高效地进行无风险的试错学习与严谨评估，成为用合成数据替代现实高危高成本采集、突破机器人具身智能瓶颈的核心解法。 --- ### [36:34](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2194s) World Labs 与 SceniX 的整合节奏与双城布局 马丁观察到早期技术并购往往面临“立即合流”还是“保持独立长期演进”的选择难题，并向李飞飞询问两家团队的实际整合节奏。李飞飞阐明了非常审慎克制的组织逻辑：管理团队经过深入探讨，决定绝不急于将两家公司的代码库和工程人员盲目揉碎混为一谈。她形象地比喻道，团队不会“像拌一碗综合沙拉一样急急忙忙地混在一起”。SceniX 已经拥有了相对完整独立的技术栈、明确的产品路线和现有企业客户群。当前的策略是聚焦于仿真基础设施与动作条件基座模型层面的深度协同，SceniX 团队已经作为第一方核心内测用户直接调用 World Labs 的 Marble 世界模型底盘。 在地理布局上，李飞飞宣布 World Labs 正式升级为一家横跨东西海岸的双总部协同公司。公司总部设在旧金山，李云珠也将迁居旧金山湾区；马丁随即笑着插话欢迎他来到这场技术“文艺复兴的佛罗伦萨”。同时，World Labs 将在纽约设立长期办公室，以此广纳美东顶尖学术与工程人才。为了支持跨地域研发，两个办公室都将部署实体机械臂与测试设备，以此打磨和验证远程操控与自动化工程架构——这正是未来为全球企业客户提供远程仿真与部署服务所必需的核心基石。 --- ### [39:03](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2343s) 两年愿景与 Real-to-Sim-to-Real 的商业落地 在被要求勾勒未来两年的理想成功形态时，李飞飞给出了非常清晰且聚焦的商业蓝图：在两年的时间窗口内，World Labs 与 SceniX 联合打造的系统基础设施，将在少数极其关键的垂直自动化领域完成真实验证。通过切切实实帮助这批客户解决严苛的物理自动化痛点，树立一批无可争议的“灯塔客户”（lighthouse examples），从而拉动整个通用空间智能平台的规模化商业扩张。 当马丁进一步追问其他机器人初创公司应当在何种研发阶段接入 World Labs 平台时，李云珠详细阐释了其 Real-to-Sim-to-Real 闭环的模块化服务机制。这套工业级基础设施并非铁板一块，客户可以根据自身的痛点灵活接入：有的客户硬件已经就位，仅仅需要采购前端的“Real-to-Sim”数字化能力，将其关心的现实物理场景与作业任务快速转译至高精度的仿真数字孪生之中，以便在其间开展海量的测试基准评测。 ### [40:27](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2427s) 平台灵活性与紧贴落地部署的产业实用场景 李云珠（Yunzhu Li）进一步阐述了他们所构建平台的灵活性与服务模式。不同客户的技术需求差异很大：一部分客户需要完整的 Real-to-Sim-to-Real 闭环管线，以确保在仿真中验证的策略能够直接部署并运行在自家的实体硬件上；另一部分客户可能只需要针对现有系统进行严格的评估测试。因此，整套平台的底层架构设计得非常弹性，能够根据合作伙伴的具体痛点进行定制化适配。 李云珠特别强调，目前与他们接洽合作的客户普遍已经处于非常接近“实际部署（deployment）”的成熟阶段，正在解决极其务实、具体的现实任务。在这些应用场景中，自动化方案一旦跑通上线，就能立即兑现商业价值。许多客户手头甚至积攒了成百上千个等待自动化改造的高价值场景。通过 SceniX 与 World Labs 的合力，团队能够针对这些场景开发出高度可靠的工程解决方案（部分典型场景已在官方博客中做了具体展示），并持续深挖和攻关真实环境落地中所面对的各种核心边界条件与硬性物理约束。 - **管线灵活性**：平台支持端到端的 Real-to-Sim-to-Real 闭环，亦可单独拆解为仿真测试或策略评估模块，以适配不同客户自研硬件与算法的集成需求。 - **高确定性商业价值**：合作客户大多聚焦于临近投产阶段的具体工作流，只要机器人方案落地，即可迅速创造直接的生产力收益。 ### [41:28](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2488s) 商业合作全面开启：何时联系 World Labs 都不算早 随着访谈进入尾声，Martin Casado 抛出了一个极为务实且针对行业受众的问题：“对于一家机器人公司而言，现在主动联系 World Labs 到底是太早还是太晚？” 李飞飞（Fei-Fei Li）毫不犹豫地给出了干脆利落的回答：“一点都不早，也绝不算晚（No）。”她真诚而热情地表示：“我们希望所有人都来联系我们，我们渴望深入了解你们具体的实际用例（We want everybody to call us. We want to learn about your use case）。”现场在欢声笑语中达成共识——对于探索物理世界 AI 解决方案的企业来说，任何时候开启沟通都正当其时。李飞飞与团队明确表示 World Labs 目前已经“全面开门营业（open for business）”。Martin 随后向所有正在涉足机器人项目或运营机器人初创企业的听众发出呼吁，建议大家紧密跟踪 World Labs 的最新进展并积极寻求合作，整场对话在轻松而充满信心的氛围中圆满落幕。 ## Notable quotes > [01:08](https://www.youtube.com/watch?v=-tabaM5l3s0&t=68s) My north star is I want the robot to work — 李云珠 (Yunzhu Li) > [05:14](https://www.youtube.com/watch?v=-tabaM5l3s0&t=314s) 哇，原来这家公司是你的！ — 李飞飞 (Fei-Fei Li) > [14:30](https://www.youtube.com/watch?v=-tabaM5l3s0&t=870s) My north star is to make robots work. — 李云珠 (Yunzhu Li) > [18:13](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1093s) There's a very important role simulation plays that real world data doesn't play, which is counterfactual reasoning — is that you play out events that hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. — 李飞飞 (Fei-Fei Li) > [23:01](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1381s) 毕竟真实世界里原子（Atoms）必须穿过空间，必须遵从物理定律，你去看网上的机器人演示视频，全都是被 8 倍速、10 倍速快进过的！ — 马丁·卡萨多 (Martin Casado) 与 李飞飞 (Fei-Fei Li) > [32:31](https://www.youtube.com/watch?v=-tabaM5l3s0&t=1951s) 马丁，在当今的 AI 领域，最难的一件事就是保持恰到好处、审慎克制的乐观（measured optimism）。 — 李飞飞 (Fei-Fei Li) > [36:34](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2194s) 像拌一碗综合沙拉一样急急忙忙地混在一起 — 李飞飞 (Fei-Fei Li) > [41:28](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2488s) 一点都不早，也绝不算晚 — 李飞飞 (Fei-Fei Li) > [41:28](https://www.youtube.com/watch?v=-tabaM5l3s0&t=2488s) We want everybody to call us. We want to learn about your use case — 李飞飞 (Fei-Fei Li) ## Mentioned World Labs, SceniX, a16z, Martin Casado, Fei-Fei Li (李飞飞), Yunzhu Li (李昀烛 / 李云珠), Changxi Zheng (郑昌玺), Sunonni, Chuang Gan (甘创), Sergey Levine, Marble, ImageNet, Waymo, Roomba, 麻省理工学院 (MIT), 斯坦福大学 (Stanford University), 哥伦比亚大学 (Columbia University), 亚马逊 (Amazon), 维塔数码 (Weta Digital), 腾讯 (Tencent), Real-to-Sim-to-Real, 3D Gaussian Splatting ## Video description Last week, World Labs announced its acquisition of SceniX, bringing together two teams working on one of AI's biggest unsolved problems: how to give machines a true understanding of the physical world. Martin Casado sits down with Fei-Fei Li, co-founder and CEO of World Labs, creator of ImageNet, and pioneer of spatial intelligence, alongside Yunzhu Li, co-founder of SceniX and assistant professor at Columbia University. They discuss why World Labs acquired SceniX, how simulation can unlock the next generation of robotics, and why training robots may require a fundamentally different approach than training language models. The conversation explores real-to-sim-to-real pipelines, world models, robotics foundation models, evaluation, synthetic data, and why the future of AI depends not just on understanding language—but on understanding and interacting with the physical world. Timestamps: 00:00 - Intro 01:08 - World Labs & SceniX 06:04 - Marble & the Data Bottleneck in Robotics 07:13 - How the Two Teams Come Together 10:55 - Building a Foundation Model for Robotics 12:35 - Video Models vs Real-to-Sim-to-Real 19:38 - Why Simulation is Essential for Robot Learning 23:01 - Training, Evaluation & Real Customer Use Cases 29:17 - Humanoids, Semi-Structured Environments & the Grand Challenge 36:56 - Integration Plans & What Success Looks Like in Two Years Resources: Follow Fei-Fei Li on X: https://x.com/drfeifei Follow Yunzhu Li on X: https://x.com/YunzhuLiYZ Follow Martin Casado on X: https://x.com/martin_casado Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures. [Source](https://www.youtube.com/watch?v=-tabaM5l3s0)

[1:08] 李云珠 (Yunzhu Li): My north star is I want the robot to work [5:14] 李飞飞 (Fei-Fei Li): 哇，原来这家公司是你的！ [14:30] 李云珠 (Yunzhu Li): My north star is to make robots work. [18:13] 李飞飞 (Fei-Fei Li): There's a very important role simulation plays that real world data doesn't play, which is counterfactual reasoning — is that you play out events that hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. [23:01] 马丁·卡萨多 (Martin Casado) 与 李飞飞 (Fei-Fei Li): 毕竟真实世界里原子（Atoms）必须穿过空间，必须遵从物理定律，你去看网上的机器人演示视频，全都是被 8 倍速、10 倍速快进过的！ [32:31] 李飞飞 (Fei-Fei Li): 马丁，在当今的 AI 领域，最难的一件事就是保持恰到好处、审慎克制的乐观（measured optimism）。 [36:34] 李飞飞 (Fei-Fei Li): 像拌一碗综合沙拉一样急急忙忙地混在一起 [41:28] 李飞飞 (Fei-Fei Li): 一点都不早，也绝不算晚 [41:28] 李飞飞 (Fei-Fei Li): We want everybody to call us. We want to learn about your use case

[0:00] Fei-Fei Li: My north star is I want the robot to work. [19:39] Fei-Fei Li: There a very important role simulation plays that real world data doesn't play, which is counterfactual reasoning... you play out events that haven't happened or cannot happen or you don't have enough data to make it happen in real world.

[0:00] We are building the next frontier [0:01] [music] of AI which is what we call [0:04] spatial intelligence. [0:05] >> As Cynics, we are developing what we [0:07] call a real to see to real pipeline. We [0:10] can replace all the data, all the [0:13] evaluation we need in the real [0:14] environment by using the data that can [0:16] generate at a scalable way in our [0:19] [music] digital world. [0:19] >> Think about human intelligence. We do a [0:22] lot of simulation in our head. Why? [0:24] [music] [0:25] There's a very important role simulation [0:28] plays that real world data doesn't play [0:30] which is counterfactual. Reason [0:32] >> what we are building is a consistent [0:34] world consistent both over space over [0:37] time over different [music] viewpoints [0:39] and over different type of interactions. [0:41] My north star is I want the robot to [0:43] work. [0:43] >> The world we live in can be multiverse [0:45] that we create technology to allow [0:48] people builders developers to act within [0:51] different [music] spaces. Do you believe [0:53] we'll ever be able to build robots that [0:56] have the power efficiency of a human [0:59] being? How far away are we from this? Is [1:01] this like 5 years or this is like never? [1:03] >> The TLDDR is [1:08] >> All right. Well, it's great to have you [1:10] both here. So, Feay, for the listeners [1:13] that may not have the background, maybe [1:14] you can give an overview of what World [1:17] Labs does. [1:18] >> Yeah. Well, World Lab has is a [1:20] two-year-old startup. I I I think we [1:23] should just recognize it's a frontier [1:26] model lab. It's uh we we are building [1:29] the next frontier of AI which is what we [1:32] call spatial intelligence. And uh [1:35] spatial intelligence is about [1:39] um creating AI that has the ability to [1:43] generate [1:45] uh understand [1:47] reason with and interact with spaces [1:50] whether it's physical or virtual. And of [1:53] course uh a means to an end towards [1:55] spatial intelligence is building large [1:58] world models. And that's what uh World [2:00] Labs is mostly focused on. [2:02] >> Yeah. So you've been saying this since [2:04] the very beginning which is um you know [2:07] the machine's ability to perceive and [2:09] reason about spaces and act on spaces [2:12] but I always had the assumption that the [2:14] acting on spaces was some like longist [2:17] future thing but now you're acquiring a [2:20] robotics company and so maybe talk a [2:22] little bit about the timeliness of this [2:23] and the intentions. [2:24] >> Yeah. So first of all, it doesn't just [2:27] take robotics to act within spaces or to [2:31] interact, right? I mean, look at the [2:34] creative field, whether it's VFX or [2:37] gaming and or design. Many use cases, [2:40] you can create and act within virtual [2:43] spaces. World Warlab's thesis has always [2:46] been that um the the the the world we [2:50] live in can be multiverse that we create [2:53] technology to allow people builders [2:56] developers to act within different [2:59] spaces. [3:00] >> Having said that the ability to act [3:03] within the physical space is one of the [3:05] most exciting and most profoundly [3:08] important capability of the future AI [3:12] world. So robotics is very much that. So [3:17] worldlap has always believed that [3:19] robotics is a important [3:23] application as well as use case of uh [3:26] spatial intelligence and world modeling. [3:29] So by joining force with uh inviting [3:32] scen [3:34] team to world labs is part of our [3:36] long-term vision and uh mission. We [3:40] we've always committed to that. Amazing. [3:43] So, Yunu, you're the co-founder of [3:44] Scenix. So, maybe provide everyone with [3:47] a quick um overview of your background [3:49] and what Scenix does. [3:50] >> Yeah. So, I'm Yunu. So, I'm currently [3:53] co-founder of Cynics and also assistant [3:55] professor at Columbia University. Wow. [3:58] >> So, my research started from my PhD at [4:01] MIT and then postto with Fay really at [4:04] the Stanford [laughter] University. [4:05] >> That's great. [4:06] >> The world is small. [4:07] >> The world is small. [4:07] >> It is. [4:08] >> Throughout my career, my goal has been [4:10] very simple. trying to help the robots [4:12] better perceive and interact with the [4:14] physical world. So I'm a very practical [4:16] person. I want my robot to work in the [4:18] real physical environments. [4:19] >> So for cynics the unique opportunity we [4:23] see is that there has been a lot of like [4:24] a bottlenecks. Right now we see faced by [4:27] the developments of general purpose [4:29] robots especially around training and [4:32] also around evaluations. So as we are [4:35] developing what we call a real to sim to [4:38] real pipeline. Okay. We want to map the [4:40] real environments into the digital world [4:42] that has the best alignments with the [4:44] real environments. By alignments we mean [4:46] that whatever happens in the digital [4:48] world is also going to happen in the [4:50] real environments [4:51] >> such that we can replace all the data [4:54] all the evaluation we need in the real [4:56] environment by using the data that can [4:58] generate at a scalable way in our [5:00] digital world. So that is how everything [5:03] started in Synex. We put together a very [5:06] very strong and best teams around [5:08] robotics, robot learning and also [5:09] simulation and rendering trying to build [5:11] this realtom real stack to solve some of [5:13] the key bottlenecks. [5:14] >> It's amazing that you two work together. [5:16] >> Yeah. And uh there is a funny story here [5:19] because you would think because we [5:22] worked together he was my amazing postto [5:24] we've been talking about this and world [5:27] lab um um integration for a long time. [5:30] It's actually not true. They came into [5:33] World Wars as a customer. [5:35] >> Really? [5:35] >> When we when we released the first [5:37] version of our generative model called [5:40] Marble last winter around uh November, [5:43] December, [5:44] >> Cynics just signed up. [5:47] >> No kidding. AS A CUSTOMER. [5:48] >> YES. [laughter] [5:49] And I didn't even know what it was. And [5:52] then I realized this is Vindrew's [5:54] company. I called Vindro. I'm like, [5:57] "Wow, this is your company." and you [6:00] guys and and then we realized there's so [6:03] much synergy. [6:04] >> Maybe Faith could just quickly describe [6:06] what Marble is. [6:07] >> Yeah, Marble is the code name for the [6:09] base model that World War has been [6:11] training and iterating on that the [6:13] fundamental capability right now of [6:15] Marble that is publicly released is to [6:18] take a a prompt. It can be an image, it [6:21] can be a few images and uh or a text and [6:26] turn that into a geometrically [6:30] consistent world that can be represented [6:32] in 3D geometry whether it's gausian [6:35] splat or mesh. Really what Synix team is [6:38] doing is trying to solve this extremely [6:42] difficult problem in robotics which is [6:44] the lack of data. M [6:46] >> the lack of data in training, the lack [6:49] of data in uh evaluation. This is very [6:52] very different from language models [6:54] where data is abundant on the internet. [6:57] >> And we know that um in order for [7:00] robotics to work, we have to somehow [7:03] unlock the power of scaling law. But [7:06] where does that come from? This is [7:07] something that that is a profound [7:09] problem that everybody's battling with [7:12] in in robotics. It'd actually be great [7:13] to talk about this energy like you have [7:16] put together a very very talented team. [7:17] You have put together a very talented [7:19] team and so like to what extent is there [7:20] overlap to what extent is this an [7:22] extension? Maybe talk a little bit about [7:23] that. Yeah, that's actually like how [7:24] complimentary it is. [7:25] >> It's it's actually the the TLDDR is is [7:28] very complimentary and with the shared [7:31] mission. So is one of the three uh [7:33] technical co-founders. The other two are [7:36] Changi Jan, another Colombia professor [7:39] who has been a world-class technologist [7:43] in simulation. Wow. And Changi has his [7:45] background in also um VFX. He worked at [7:48] Weta, he worked at Tencent, he's being [7:52] an entrepreneur. Uh then there's Sunonni [7:54] who is a phenomenal engineering leader [7:58] who was also in a uh startup uh that was [8:02] acquired by Amazon many years ago. So he [8:04] worked in many different tech stacks in [8:07] the computer vision field in uh in [8:10] Amazon. So when when we started talking [8:13] more seriously I recognized that uh a [8:17] couple of things that Phoenix has from a [8:19] talent point of view is extremely [8:21] complimentary to to uh worldaps. one is [8:25] obviously [8:26] incredible um uh thought leadership and [8:29] and just technical prowess in robotics [8:33] right so uh from really from hardware [8:37] full stack robotics and even when he was [8:40] my posttock at Stanford at that time you [8:43] already had your faculty offer so you [8:45] were there only for one year I wanted [8:47] you for more than one year but he had to [8:49] go become a have the real job so uh he [8:52] was a full stack researcher in in [8:54] robotics from modeling to to hardware. [8:57] Uh and of course uh VR and his student [9:00] uh students at Synenix was that pool of [9:04] talent world hasn't had yet. Then on the [9:07] Changi side is just incredible [9:09] simulation um um capability, right? he's [9:14] such a senior um researcher and [9:17] technologist in simulation and and uh [9:20] what world labs is doing is very much um [9:24] interfacing the world of simulation. So, [9:27] so I think what they don't have um [9:30] obviously is on the generative model [9:32] side as well as the computer vision 3D [9:36] reconstruction side we're also very [9:38] strong at world labs. So that's a [9:40] technology that scenics needs. So [9:43] together these uh these two sides come [9:45] together and make it much more complete. [9:48] Fe's motivation in this as like this is [9:50] an extension and a compliment to get [9:52] into robotics you know having been in [9:54] your situation which is deciding when to [9:56] sell a company it would be great to hear [9:57] from you on like how you think about [9:59] joining world labs and kind of the fit [10:00] there and like why you made the decision [10:02] to do it. Yeah. So at the very beginning [10:05] we were deciding okay we want to just [10:08] keep going but after chatting with F [10:10] after seeing all the synergies that [10:11] happening in the middle it just makes [10:13] perfect sense for the forces to join [10:15] each other. So in any sense as cynics [10:18] what we have been doing is real to seem [10:19] to real is to do this reconstruction of [10:22] the environment. So we capture the [10:24] appearance of the environment geometry [10:26] of the environments and also the [10:27] dynamics of the environment meaning how [10:29] the environment is going to change when [10:30] you apply actions. So this tensor [10:32] reconstruction right now is still a [10:35] little bit on the heavier side and what [10:37] world labs right now has been doing [10:39] involves a lot of profound capabilities [10:42] around sparse reconstruction and [10:44] generations. So we see a lot of [10:46] opportunities of leveraging like a [10:48] marble and other like capabilities as [10:51] world labs in order to do very efficient [10:53] reconstructions and modeling of the [10:55] environments. [10:56] >> So can we expect a foundation model for [10:58] robotics from world labs? [11:01] World Lab is building a foundation [11:03] model. As you know, Martin, we're [11:04] building a base model and uh as the [11:08] technology has been evolving, [11:11] some of the most exciting base models [11:13] are omniodels, right? They take they [11:16] take multimodal input, they have [11:18] multimodal outputs. And uh what is a [11:22] foundation model for robotics? Uh it's [11:25] very likely going to involve actions. M [11:28] >> it's very likely going to involve the [11:31] output of actions in addition to the [11:34] state of the world and we're definitely [11:36] not ruling this out. [11:38] >> Yeah. Great. [11:38] >> So for example for the foundation models [11:40] it's essentially needs to be a [11:41] multimodal model. So it has to take into [11:44] account frame text image deps and [11:47] different kind of modalities and action [11:48] is a very very important parts of that [11:51] modalities. So if you think about frame [11:53] actions as inputs that is essentially a [11:56] forward simulator that is going to [11:57] predict how the environment is going to [11:59] change when you apply a specific action. [12:01] When the action is output this is [12:03] essentially a policy model that is [12:05] trying to predict given a specific goal [12:07] like what should be the action you take [12:09] in the real environment to get you [12:10] closer to that goal. So this kind of [12:12] omni models actually can benefit a lot [12:15] and actually provide huge amount of [12:17] values for the robotics communities in [12:19] trying to understand how to model the [12:20] environments and at the same time how to [12:22] act in the environments and this can [12:24] also acts as a backbone for you to [12:27] fine-tune into specific robotic [12:29] applications to making sure it's really [12:31] leave up to the reliability and [12:33] efficiency that expected by the clients. [12:35] you know if uh Yun if you don't uh if [12:37] you don't mind a kind of a lay investor [12:39] question I see a lot of robotics [12:41] companies and a very popular approach [12:44] right now for the robotics companies [12:45] that come in is like we'll use a video [12:47] model you know and like you know that's [12:50] the the predominant method where this is [12:53] you know 3D and simulation it's a very [12:55] different approach and so maybe you [12:57] could contrast [12:59] you know this popular approach of just [13:00] using video only versus kind of what the [13:02] ambition here is [13:03] >> yeah so in order to create words where [13:06] the robot can learn the words as I [13:07] mentioned is to capture the essential [13:09] structure of the problem and one of the [13:11] very important necessary like [13:13] requirements for those words will be [13:15] consistency so that is where I actually [13:18] see there's very very strong synergies [13:20] with marble because what we are building [13:22] is a consistent world consistence both [13:25] over space over time over different [13:27] viewpoints and over different type of [13:29] interactions and marble the generated [13:31] words from marble is also provides an [13:33] infrastructure a component of that [13:35] entire words that we believe is [13:37] necessary for the robot to learn. [13:39] Imagine if a robot push an object [13:40] forwards. The object just magically [13:42] disappear which has been a problem of [13:44] many of the existing like video [13:46] prediction models. This won't provides [13:48] good enough signal for the robot to know [13:50] like what is the right thing to do. But [13:52] obviously right now there has been a lot [13:53] of investigation on building better and [13:55] better and stronger and stronger like a [13:57] video models. So we actually see a way [13:59] where some of the infrastructure we [14:01] build can provide as a initial momentums [14:04] and to going through this data flywheel [14:07] of going from this like a more [14:09] simulationdriven models into like a [14:11] robot policy models which going to do [14:13] the execution in the real environment [14:15] collecting new data the data will come [14:17] back in where the model doesn't [14:18] necessarily have to be physics only or [14:20] learning only but somewhere in the [14:22] middle which be able to capture the [14:24] essential structure of the problem but [14:26] at the same times be able to scale and [14:28] become better and better as you [14:29] accumulate more data. [14:30] >> You know, um I've worked now fay very [14:33] closely for a while and and and [14:37] you've always had this north star which [14:39] has driven this and you know you you've [14:40] articulated variously as kind of 3D and [14:42] and in a number of other ways. And I'm [14:44] just wondering for you is there also a [14:46] similar philosophical northstar or [14:48] you're more the pragmatic [14:50] like I am like build the system like do [14:53] the thing. [14:54] >> My northstar is to make robots work. [14:56] >> Amazing. Yeah. in the real environment. [14:57] I'm a very practical person. I want the [14:59] robot to work. One interesting thing [15:01] that's actually coming from my [15:03] collaborations with FIP during my [15:04] postto, we are building this kind of [15:06] benchmark. We actually send out surveys [15:08] asking the general public what they want [15:11] their robots to do for them. [15:12] >> Yeah. [15:12] >> Among the thousand tasks we collected, [15:15] onethird of the tasks are about clean. [15:18] >> People just don't like to do those like [15:19] a d and dirty tasks. And those are the [15:22] scenarios we really want to making sure [15:24] we have robotic solutions to deal with. [15:26] >> One thing I really like about Synenix uh [15:29] Martin especially uh uh continuing your [15:32] question there's a lot of robotics [15:34] companies building models and all that. [15:37] One thing I truly like about Synenix is [15:40] is Renu and his co-founders have such an [15:43] incredibly pragmatic approach to [15:46] robotics. They especially they come from [15:49] academia right Sunny doesn't but Vindra [15:52] and Chani come from academia but their [15:55] first instinct is work with design [15:57] partners and customers in real industry [16:01] whether it's is uh labs at um industry [16:04] labs or or warehouses or um um [16:08] >> electronics [16:09] >> electronics [16:11] you know uh assembly that is such a [16:14] refreshing actually a refreshing way of [16:16] approaching [16:17] robotics and that that really tr made me [16:20] very excited to work with them. [16:22] >> Maybe this is for you and but I'll just [16:24] be this is this is personal curiosity [16:25] which is it seems to me that for [16:27] robotics you have to be pretty exact. I [16:29] mean not perfect but pretty close. But [16:31] for the creative use cases which [16:32] worldaps has done a lot of you kind of [16:34] don't need to because you know I mean [16:36] you know even sometimes like being wrong [16:39] is stylistic or intentional or whatever. [16:41] And so from a technical perspective what [16:45] is the challenge here for reconciling [16:47] these two things or do they never get [16:48] reconciled like there always be two [16:50] points in the design space? So they will [16:53] be like reconciled in the in the long [16:55] terms of course and um modeling of the [16:58] environments um doesn't have to be [17:01] perfect. The model doesn't have to be [17:03] perfect in robotics. [17:04] >> And by the way is there again this is [17:05] pure curiosity but is there like um a [17:08] bit more formal way to say that like [17:10] what does that mean not to be perfect? [17:11] It has to be pretty close. [17:13] >> So so let me putting this way for [17:15] example models over the developments of [17:17] all different kind of robotic [17:18] applications has been a very important [17:20] cornerstones. Yeah. If you look at all [17:22] the existing robotic applications like [17:24] plane, drones, Roomba or even for [17:27] quadripad robots, bipedal robots, model [17:29] has been the way [17:30] >> for them to actually work and be able to [17:32] transfer from simulation to the real [17:33] environment. [17:34] >> I see. [17:34] >> But if you look at those locomotion [17:36] robots like quadriped robots, bipedal [17:38] robots, they can walking on snows, they [17:40] can walking on bushes. But you don't [17:41] need to have a simulator. They can [17:43] simulate all the bushes and snows like [17:45] very precisely. you needs to have a [17:47] simulation that capture the essential [17:49] structure of the problem and do a whole [17:51] different kind of randomizations inside [17:53] the digital environment. So that is what [17:55] we're aiming for. So basically with [17:57] Synenix and together with word labaps [17:59] we're trying to investigate like what is [18:01] the level of fidelity we needs to model [18:04] the mass massive worlds besides the [18:05] robots I said we'll be able to transfer [18:07] the robotic systems training the [18:09] simulated environment digital worlds [18:11] back into the real scenarios. As an [18:13] investor, I've heard other researchers [18:15] say like Sergey Lavine say simulation [18:19] will always eventually deviate from the [18:22] physical world and real world data [18:24] collection is absolutely critical. And [18:26] so maybe talk a little bit about like [18:28] the viability of this approach where [18:30] simulation is a cornerstone as opposed [18:32] to some other approach. [18:35] >> So they don't contradict with each [18:37] other. So if you think about the [18:39] simulation, simulation is essentially [18:41] trying to predict how the environment is [18:43] going going to change when you apply the [18:45] actions and this is essentially a model [18:48] of the worlds. It doesn't necessarily [18:49] have to be pure physics. It can be a [18:52] combination between both physics and [18:53] also learning. We are collecting real [18:55] world data. We will be using those real [18:57] world data. It just at different stages [19:00] of this like a data fly. Maybe at the [19:02] very beginning we have stronger [19:03] emphasize on we have more physics to [19:05] making sure we have the right [19:06] consistency and right structure for us [19:08] to learn the uh the the worlds for us to [19:11] train the robot policies. But as we [19:13] accumulate more and more data both [19:15] through data collection and also through [19:16] the collaboration with our clients we'll [19:19] have the data that will be moving [19:21] towards more towards more learning based [19:23] like modeling of the environments. So [19:25] this kind of transition and also this [19:27] kind of data flow is really an enabling [19:29] factors of both getting the best of both [19:32] physics and the and and geometry and [19:34] consistency as well as all the power and [19:36] magics from the data and compute. [19:39] >> I want to add to this and be slightly [19:42] philosophical here is there isn't a a [19:47] binary choice between simulation or no [19:50] simulation. All this come um in together [19:53] um to to make robotics work. Think about [19:57] human intelligence. We do a lot of [19:59] simulation in our head. You know why? [20:02] There's a very important role simulation [20:05] plays that real world data doesn't play [20:08] which is counterfactual reasoning is [20:10] that you play out events that c hasn't [20:14] happened or cannot happen or you don't [20:16] have enough data to make it happen in [20:18] real world. And while you play it out, [20:22] you learn how to act in it. Humans do [20:24] this all the time. We probably don't, [20:27] you know, we just I know you were at [20:28] World Cups. [20:30] >> I was at the World. [20:31] >> Congratulations to Spain winning. I'm [20:34] sure in the planning of every game there [20:37] is simulation, whether it's digital or [20:39] on the on the whiteboard or whatever, [20:42] that simulation, the role simulation [20:44] plays is counterfactual reasoning. And [20:46] that's really important in robotics [20:49] because we just do not have cannot [20:51] possibly have enough real world data for [20:54] that. Here's a real life uh example the [20:57] the industry of self-driving cars. Whimo [21:01] has officially said they use billions of [21:04] hours of simulation [21:06] >> and and actually Whimo is more [21:09] simulationheavy than just real world [21:12] data heavy. So these are real examples [21:15] and and as you know Martin Andrew too [21:17] cars are the simplest kind of robots. [21:20] >> Yeah. [21:20] >> Yeah. So so clearly simulation plays a [21:24] huge role in robotic learning. [21:26] >> I also want to add to that. So like [21:29] there are if you put things more [21:31] specific simulation can provide two [21:33] levels of benefits. The first one is [21:36] reliability and the second one is [21:38] efficiency. So for reliability, if [21:41] you're thinking about a robotic system [21:42] working reliable in the real [21:43] environments, you need data to provide [21:46] systematic coverage of all the state [21:48] space and the variations that robots [21:50] might encounter. That's how you can [21:52] learn of how that is robust. So with [21:54] simulation, you can do systematic [21:56] randomizations and control and the [21:58] variations of lighting, frictions, [22:00] geometries, object types and also all [22:03] different kind of physical parameters to [22:04] making sure you have sufficient coverage [22:06] of the state space. So this is what can [22:08] give the robotic systems reliability and [22:11] second is about efficiency. So right now [22:14] many people are doing tele operation and [22:16] if you look at many of the television [22:19] device imagining all the actual [22:21] skeletons you are using you're actually [22:23] collecting the data at a speed that is [22:25] actually slower than human actually [22:26] doing the task. [22:27] >> But for many of our clients human speed [22:29] to them is not good enough. They want [22:31] faster than human speeds. Yeah. [22:33] >> So for the robot to like move faster, [22:35] it's not as simple as just drive the [22:37] robot faster because the gravity doesn't [22:39] change. [22:40] >> But in simulation, you can do systematic [22:42] speed up of the robots behaviors to [22:44] train the robots such that it considers [22:46] all the dynamics changes of the [22:48] environments. So this is what's can give [22:51] like our clients for them efficiency. So [22:54] both for the reliability and efficiency [22:56] though there are some kind of like a [22:58] very unique like values where simulation [23:00] can provide. You've talked about the [23:01] technology and the platform, what it [23:03] does. Maybe talk about the specific use [23:04] cases people use it for. [23:07] >> There are essential like two specific [23:09] use cases especially around both [23:10] training and also around evaluations. [23:13] >> Okay. [23:13] >> Um starting from the evaluations. [23:15] >> So evaluation is something like people [23:18] often overlooked in the robotics [23:21] >> but if you are tuning like robotic [23:23] models you have to know how well it [23:25] works and that is the only source of [23:27] information for you to iterate. [23:28] >> Yeah. By the way, a lot of a lot every [23:31] every AI person really understands what [23:33] eval are and uses it all the time. NonAI [23:36] people, it often means something a [23:37] little different. So maybe it's even [23:38] worth just describing specifically what [23:40] you mean by evaluation. [23:41] >> Okay. So what I mean by evaluation is [23:43] you'll be able to understand for this [23:45] specific checkpoints how well does it [23:48] perform? Does it perform for example 95% [23:50] of the time or 99.9% [23:53] of the time? And the key criteria people [23:56] use in industry is how long does it [23:58] take? How long in walk work walk time [24:01] does it take for you to distinguish [24:03] between a checkpoints that is 90% from a [24:06] checkpoint that is 90 92 points. And if [24:08] you only do that in the real environment [24:10] that's just takes so long for you to do [24:13] the distinguishment. And if you really [24:15] think about also the robotic evaluations [24:17] right now people are doing in the real [24:18] environments the iteration speeds is [24:21] multiple orders of magnitude slower than [24:24] iterations of those language models. [24:26] >> Yeah. [24:26] >> So not only is like the robotic tasks [24:29] very varied very diverse. [24:31] >> Oh yeah because like you actually have [24:32] to do the thing right [24:36] atoms have to move through space. [24:38] [laughter] [24:38] >> Exactly. But only there are [24:39] >> the laws of physics have to do. [24:41] >> Have you watched those robotics videos? [24:43] Every video has like 10 x 8x because it [24:47] moves so slowly. [24:48] >> Exactly. So not only it's slow, it's [24:51] dangerous, it's costly, but at the same [24:53] time the speeds is also like multiple [24:56] orders of magnitudes like slower. So [24:58] some of our clients actually needs this [25:00] digital environment that's can be used [25:03] to evaluate their robotic like systems [25:05] and because our digital environment has [25:07] proven alignments with the real world. [25:09] So meaning whatever happens in the sim [25:11] is also likely to happen in the real [25:13] environment. If a checkpoint is working [25:15] better in the simulation is also highly [25:17] likely to also work better in the real [25:19] environments as we have also been [25:20] discussed in the blog post. So that [25:22] actually give our clients very strong [25:25] confidence in actually using the data [25:28] using the signal from the digital [25:29] environment to do scalable safe and much [25:32] faster evaluations of their robotic [25:34] systems. Great. So that is on the [25:36] evaluation then on the training. So on [25:38] the training sides, so basically like I [25:40] also mentioned it's about [25:42] controllability. So you want to control [25:44] all the different possible variations of [25:46] states, parameters, lighting, frictions, [25:48] physical parameters like even object [25:50] geometry, object types. So you want to [25:52] making sure you have sufficient coverage [25:54] of all different kind of scenarios such [25:56] that you'll be able to generate like a [25:58] informative data for your robots to be [26:01] robust. And this is just going to be so [26:03] hard to do just in the real environments [26:05] like we discussed. If you do tele [26:06] operation, the speed at which you're [26:08] collecting data is slow. You're also [26:10] limited by like how many robots you [26:12] have, how many tele operation device you [26:14] have. There's like a whole different [26:16] kind of like challenges around all the [26:18] data operations around it. But in [26:20] simulation everything can be [26:22] controllable, everything can be [26:24] systematic and everything can be [26:26] understand at a level where you know [26:28] exactly and making claims about exactly [26:31] what distribution you have covered to [26:33] develop confidence about within that [26:35] distribution. we know the robot will [26:36] work. So those kind of confidence and [26:38] efficiency and scalability is something [26:40] that our clients also value to use our [26:43] like a digital words for the training of [26:45] robotic systems. [26:46] >> Here's the crazy thing even before uh [26:49] Synix and we are talking our inbound [26:52] customers for Marble were already seeing [26:54] this kind of demands. we just cannot [26:57] serve these customers. But we are [26:59] already getting a lot of phone calls [27:00] from from robotics early stage robotics [27:04] companies who are developing their [27:06] models all the way to downstream very [27:09] pragmatic use cases and uh we're seeing [27:12] these uh these needs. When people hear [27:15] you're going into robotics, what they're [27:17] going to envision is you're pulling out [27:19] a 3D printer and you're going to be [27:21] making hardware and then you're going to [27:23] be programming the brain of a robot and [27:26] sticking it in the robot and then you've [27:28] got a robot. And I don't think that's [27:30] what you guys are talking about here. So [27:31] maybe talk about what you know where [27:34] this fits in the life cycle of creating [27:36] a robot and like where you will end and [27:39] where the rest of the ecosystem will [27:42] will begin. So what we have been [27:44] building you can imagine is a [27:46] infrastructure like with the softwares [27:49] around this infrastructures for people [27:51] to for them build words such that robot [27:53] can learn and evaluate and this [27:55] infrastructures is naturally model [27:59] agnostic and embodiment agnostic. So I [28:02] just want to be very clear just because [28:03] this is actually a very subtle for you [28:04] it's obvious but it's a very subtle [28:06] point which is um [28:08] from what you said that's not building a [28:10] robot it's building an environment which [28:13] another company can place their robot [28:15] brain [28:16] >> exactly [28:16] >> to navigate and to learn. [28:18] >> Yeah. So for our customers right now [28:20] they have all different kind of robots. [28:22] Some are using for them single robot arm [28:24] some are using bio some are using a [28:26] fixed arm. Some are using like a mobile [28:28] manipulators. Some using grippers, some [28:30] are using some more elaborate versions [28:31] of the end factors. So our platform [28:33] right now is just naturally embodiment [28:35] agnostic. We can very easily integrate [28:38] different kind of robotic embodiment be [28:40] able to put them into the works we [28:42] generated with digitalized such as we [28:44] will be able to give those individual [28:46] robots capabilities of doing the right [28:48] tasks and at the right levels of [28:50] reliability and efficiency in the real [28:52] environments. [28:53] >> Yeah. [28:53] >> And we are also for example model [28:55] agnostic. So we can just using the data [28:58] generated by our words to train [29:00] different models either from scratch or [29:02] doing post training of existing [29:04] foundation models like vision language [29:06] action models or word action models. So [29:09] to us it doesn't matter we just want to [29:11] making sure we have the infrastructure [29:13] we have all the words such as the robot [29:15] can work reliably in the real [29:16] environment. you know, you you you have [29:18] uh told me um that you think uh a lot of [29:23] the predictions around humanoids were a [29:25] little bit aggressive and we're likely [29:26] to see more constrained rollouts like [29:28] warehouses or whatever. Can you talk a [29:30] little bit about that and like how that [29:31] impacts what you're going to be tackling [29:33] here at uh the like world labs? [29:37] >> So that's a very good question. So if [29:39] you look at for example all the uh [29:41] progressions of robotic applications in [29:43] the real environments it has always [29:45] followed the trend from going from fully [29:48] structured environments into [29:49] semiructured environments and then into [29:51] unstructured environments. [29:53] >> For fully structured environments what [29:54] do we mean that you have knowledge and [29:57] control over all the configurations [29:59] within the environments [30:00] >> like factories [30:00] >> like factories or for car manufacturing [30:03] mice those has been automated for [30:04] decades. [30:05] >> Yeah. Yeah. Yeah. And then you have for [30:06] example semiructured environments which [30:09] you have certain controls over the [30:10] environments for example like the Amazon [30:13] warehouses or for example like [30:16] restaurants hotels where you have [30:18] certain control over the environment to [30:19] just make the task easier for your [30:21] robots but there are obviously many [30:23] other like objects or for example [30:25] clothes those are the object you don't [30:26] have control and then for the [30:28] unstructured environments it's like your [30:29] home and my host [30:31] >> those is I would say the the grand [30:34] challenge Especially Especially my [30:36] house, trust me. Three dogs, [laughter] [30:38] >> five-year-old. [30:39] >> Yes, dogs. [30:41] >> Exactly. [30:42] >> If you're thinking about where does the [30:43] robustness coming from, robustness [30:45] coming from a sufficient coverage of the [30:47] scenarios that robots might encounter. [30:49] So, it's so much easier and more [30:51] approachable at least like right now to [30:53] focus more on the semiructured [30:55] environments before we move on to fully [30:57] unstructured environments. So, we will [30:59] move into that direction. It's just we [31:00] want to uh take a more sustainable and [31:03] more realistic approach towards it. [31:05] >> I think your point here is that [31:08] humanoids mimics human body and [31:11] evolution has optimized human body for [31:14] unstructured environment. [31:17] And so our fingers, our legs are not the [31:22] best apparatus to do one thing. For [31:25] example, if if if our only goal as a [31:29] species is to climb trees, we will not [31:31] have this body necessarily, right? So, [31:34] we'll have different kind of fingers. [31:36] But what humans end up having are [31:38] involve evolved into is this this body [31:42] shape that can be very general but not [31:45] necessarily best at everything. And that [31:48] is for the survival of unstructured [31:50] environment. But but but from a business [31:53] point of view, from a pragmatic [31:56] um technology point of view that this [32:00] unstructured environment and a [32:02] generalized body is actually the hardest [32:05] problem to solve. It's not necessarily [32:08] even the right way to solve the problem. [32:10] It's we specialize. So we take more more [32:14] specialized body to solve a narrower [32:18] problem. But the challenge for scenics [32:20] is is that to be more body agnostic so [32:24] that their infrastructure can serve [32:26] different bodies and different [32:28] semistructured environments. [32:31] >> You know a common lens to look at [32:33] exactly this question is an economic [32:34] lens right which is um [32:36] >> well you compare it to like generative [32:38] LLMs where they can create uh pros or [32:42] code 10,000 times faster than a human [32:45] being a bunch cheaper than a human [32:46] being. So the economic case makes sense [32:48] because our brains aren't very efficient [32:50] at that. However, our brains and our [32:53] bodies are very efficient at 3D [32:55] navigation, right? You know, like moving [32:57] through the world or picking things up. [32:58] And so this is just a prediction [33:00] question, but do you believe we'll ever [33:02] be able to build robots, at least in the [33:05] foreseeable future, that have the power [33:08] efficiency of a human being when it [33:10] comes to to menial tasks? So let's say [33:12] just basically you know minimum wage or [33:15] something like that like how far away [33:17] are we from this? Is this like 5 years [33:19] or this is like never I think it's going [33:22] to take a very long time. So if you [33:24] really think about like a robots in the [33:26] real environment in the end it it will [33:28] always be a system. So every working [33:30] robot in the real environment is a [33:31] system work. It's you need to be very [33:33] mindful and thoughtful about how the [33:35] system are coming together. the [33:36] hardware, the software, the brain, even [33:39] like to the details of for example [33:41] what's the friction coefficients of your [33:43] of your fingers. So there's a lot of [33:44] things you have to consider to make [33:46] these things uh a reality and it will [33:48] take iterations. But what I am excited [33:51] about is that I have always been at the [33:54] state of the arts of robot learning and [33:56] also trying to push the state of the art [33:57] forward. Yeah. [33:58] >> But the state-ofthe-arts always moving [34:00] faster than I expected. So what's I'm [34:03] focusing on and trying to investigate [34:05] right now is very different from for [34:07] them when I started my PhD. So this is a [34:10] speak to how fast the whole ecosystem [34:13] has been evolving and all the moving [34:15] pieces started coming together or [34:16] building this robotic systems but we [34:18] also have to be like calibrated about [34:21] our predictions. So we will see a lot of [34:24] like a progress but to achieve for [34:25] example human level efficiency and [34:27] capabilities it will take uh longer. [34:30] Martin, the hardest thing in today's AI [34:33] is to have the right measured optimism, [34:36] [laughter] [34:36] >> right? [34:37] >> That's right. It's totally true. Yeah. [34:39] >> I mean, even LLM does not have human [34:42] brain efficiency. Human brain operates [34:44] on 30 watts. [34:46] >> Yeah, that's true. [34:46] >> That's so we are far from that. So, [34:49] >> but that but I mean [34:52] performance [34:53] to power it may be close, right? in [34:57] narrow task like software [34:59] >> like generating an image or or software [35:01] engineering than it is right. Yeah, I [35:02] think so. I don't think we're anywhere [35:04] close when it comes to robotics. Does [35:05] this change how you think about your um [35:08] like strategically the level of ambition [35:10] that your team can go after? I mean does [35:12] it is it changed that or is it still [35:14] very much in line with what you expected [35:16] to do when you started? [35:17] >> It definitely changed the trajectories [35:19] in a very profound manners. So we see a [35:22] lot of unlock in be able to do this [35:24] whole process do the modeling of the [35:26] environments yeah much more efficient [35:28] and much more scalable manners [35:29] especially in partner together with word [35:32] labaps and I also want to add to f if [35:34] you think about for example the current [35:36] states of the language models [35:38] >> so those are models that's with [35:40] incredible capabilities [35:41] >> but still you don't just blind trust it [35:44] to book your flight tickets or make your [35:46] hotel reservations you still hopefully [35:48] there's still a person who reading the [35:50] output from those language models. [35:52] >> Yeah. [35:52] >> But that is very different from how [35:54] people and we'll be using for them [35:55] robotic models because for robotic [35:57] models out of the box the robots has to [36:00] work reliably in the real environment. [36:02] >> So and we don't even have the data we [36:05] don't even have all the necessary [36:06] infrastructures around those for the [36:08] robots to just out of box work reliable [36:10] in the real environments. So for that [36:12] reasons be able to create this digital [36:14] worlds this scalable digital worlds [36:16] where the robot can learn in and [36:18] evaluates within that is going to unlock [36:20] so much more potentials for be able to [36:23] replace all the like a costly and unsafe [36:26] data in the real environments with the [36:28] data generated from the the the the [36:30] words for the robots to be able to do [36:32] scalable learning and evaluations. [36:34] >> You know I've I've seen uh I've seen [36:35] many of kind of these kind of [36:37] integrations. they actually work very [36:38] well at this stage when they have this [36:40] much alignment which is great. Um but [36:42] there's always like this question of do [36:44] you integrate now into what's happening [36:46] now or do you keep things quite separate [36:48] and provide kind of like a long-term [36:51] trajectory that will you know be [36:54] realized you know in in in the year time [36:56] frame. How are you thinking about this [36:58] fee? Is this something that integrates [36:59] right away or is this kind of a separate [37:01] longer term? [37:03] >> This is a great question. I think at [37:04] this point you know Changi Sunonny [37:08] Justin Ben and have been talking about [37:10] this at this point we are going to take [37:14] it thoughtfully we're not rushing to [37:18] integrate [37:19] everything from codebase to teams [37:22] because I think Synex does have a very [37:25] uh um wellthought and I wouldn't call it [37:29] standalone completely but fairly uh [37:32] contained [37:34] um tech stack as well as their customers [37:37] as well as the the the kind of products [37:40] they're building. We're going to take [37:42] time. We definitely will we already on [37:45] the simulation side as well as the the [37:48] potential base model um action condition [37:51] model side. We already are starting to [37:53] talk and also they are using Marvel as a [37:56] internal customer. So we we will be [37:59] integrating but uh we're not rushing to [38:02] blend the team as like a full salad [38:06] bowl. [38:07] >> Yeah. [38:07] >> How are you thinking about geographies [38:09] with this with scenics move? Is it going [38:10] to stay in the same place? [38:12] >> We're going to Vundra is going to move. [38:14] >> Oh well, welcome here. [laughter] Moving [38:16] to San Francisco. Yeah, Florence and the [38:18] Renaissance. Perfect. [38:19] >> I think we our labs is officially [38:22] becoming a by coastal company where the [38:25] headquarter is in San Francisco. I've, [38:28] you know, I live in Palo Alto. I feel [38:29] like I'm in a [laughter] different [38:31] state. We [gasps] we but uh we but um [38:36] I'm actually excited that we're going to [38:39] have an office in New York that can help [38:41] us to attract talent on East Coast and [38:44] also we have been talking about making [38:46] sure that in both offices we set up the [38:49] robots so that we get to [38:53] basically test out and and and uh mature [38:56] our engineering stack so that we can [38:59] work with robots remotely because we [39:01] have to do that for for our customers. [39:03] Anyway, [39:03] >> so may maybe just to be very concrete, [39:05] Fay, maybe let's just pencil out like [39:07] what is the the perfect success case in [39:09] two years? Like what product do you [39:11] have? Who's engaging with it? How do [39:13] they use it? Just just the crisp like [39:16] what? We're very happy that uh Synix [39:19] team and World Apps team will have [39:26] validated [39:28] um [39:30] customers in in a small number of [39:34] important vertical use cases where our [39:38] system, our infrastructure has proven to [39:40] be truly beneficial to their automation [39:44] needs And these customers became our [39:50] lighthouse examples to scale our [39:53] business. [39:54] >> And how how early let's say someone [39:57] listening to this [39:59] uh is running a robotics company? How at [40:02] what stage do they engage with world [40:05] labs? Is it really early on? Is it [40:07] somewhere in the middle? [40:10] So right now um for our customers [40:12] because we are building this kind of [40:14] real to sim to real pipelines like where [40:16] the simulation is essentially the words [40:18] we're going to provide the training [40:19] evaluation grounds. [40:21] >> Some customers they needs only the real [40:23] to sim part [40:24] >> they want to digitalize the task they [40:26] care about and be able to do the [40:27] evaluations of their robotic systems. [40:30] >> Some customers they need this real to [40:31] sim to this entire pipeline such as they [40:34] will be able to like have policies like [40:36] running on their hardwares. So our [40:38] platform is also designed in a way that [40:40] is flexible depending on what our [40:42] clients needs and at the same times uh [40:44] the clients we are working with are [40:46] actually pretty close to the deployments [40:49] like a stage. So basically they are [40:51] working on very very practical tasks. [40:53] those tasks when replaced when we have a [40:55] robotic solutions that are there can [40:57] just create value immediately and they [41:00] have like at least like uh tons or [41:02] hundreds of like this kind of situations [41:05] they are thinking about to do the [41:06] automations for. So as like together [41:09] with word labaps we'll be able to [41:11] develop reliable solutions for those [41:13] scenarios as we have already showed we [41:15] have a number of scenarios already [41:16] instantiated in our blog post and we'll [41:19] be able to further our investigation to [41:21] see how they can actually solve the key [41:23] like like requirements and and and also [41:26] constraints faced by the real world [41:28] deployments. [41:28] >> Great. I want to be very specific about [41:29] this. Is it ever too late or too early [41:32] to call World Labs if you're a robotics [41:34] company? [41:34] >> No. [41:35] We we want everybody to call us. We want [41:38] to learn about your use case. [41:39] >> Wonderful. If you listen, if you're [41:41] listening to this and you're anywhere [41:42] close to a robotics project or robotics [41:44] company, uh please track World Labs. [41:46] >> Yes. Thank you. [laughter] [41:49] >> Definitely open for business. [41:51] >> We are open for business. All right. [41:52] >> Not too early. [41:53] >> All right. If you're doing robotics, [41:55] call World Labs. Thank you both very [41:56] much for coming. [41:57] >> Thank you. [42:00] [music]