World Labs Launches Atlas to Replace Video Drift With 3D Control

Theo Jaffee, Sofia Puccini, Justin Johnson

Direct spatial grounding and camera trajectory inputs eliminate the coherence failures of generative video models, creating a horizontal foundation layer that can simulate environments and adapt robotics policies within minutes.

AI-assisted summary of World Models, Robotics, and the Future of 3D AI — original source. Check the original for full context.

Source published: 2026-09-13

Key ideas

World Labs announced Atlas, a foundational multimodal world model designed around three core capabilities: generating fictional 3D spaces, sparse 3D reconstruction from 1 to 100+ images, and physics simulation for agents and robotics. (0:00)
Unlike its predecessor Marble, which routed all outputs through a 3D Gaussian splat bottleneck, Atlas features a redesigned architecture that bifurcates 2D direct pixel streaming from explicit 3D mesh and splat reconstruction. (5:00)
By grounding reference images at coordinates along 3D camera paths, Atlas mitigates the temporal coherence drift common in generative video, enabling stable generations lasting up to a minute. (3:40)
Atlas demonstrates real-to-sim adaptation for robotics, using casual smartphone imagery to reconstruct environments and stage RL fine-tuning simulations for embodied foundation models within minutes. (10:00)

Detailed notes

## Background Published on September 13, 2026, on an audio podcast feed hosted by Theo Jaffee and Sofia Puccini, this episode features World Labs co-founder Justin Johnson. According to the show description, the discussion centers on Atlas—World Labs' multimodal world model—and outlines how AI systems capable of generating, reconstructing, and simulating 3D environments could become a horizontal foundation for fields ranging from entertainment to robotics.

## TL;DR World Labs co-founder Justin Johnson breaks down Atlas, a foundational multimodal world model capable of 3D scene generation, sparse physical reconstruction, and physics simulation. He details how Atlas moves past the architectural bottlenecks of their earlier model, Marble, by enabling direct 2D video generation alongside explicit 3D Gaussian splat exports, solving video drift with native camera trajectory inputs. The discussion explores transformative applications, including 'real-to-sim-to-real' pipelines that adapt robots to specific environments in minutes and video game development guided by fine-grained directorial control rather than random generation. Johnson also demonstrates the system's spatial fidelity using a three-phone frozen-time splash capture, while looking ahead to virtual reality interfaces and recursive chip design.

## Atmosphere & style The dialogue has an energetic, forward-looking builder ethos, balancing nuanced technical architecture with accessible, imaginative metaphors. Hosts Theo Jaffee and Sofia Puccini steer the conversation with provocative questions about the future of human software development and incumbent AI labs, giving the talk an intimate insider feel. Johnson responds with grounded candor, rejecting slot-machine generative hype in favor of deliberate creative control and illustrating complex spatial breakthroughs with memorable, tactile anecdotes like capturing oat milk splashes from consumer iPhones.

## Bulletin - `00:00` World Labs announced Atlas, a foundational multimodal world model designed around three core capabilities: generating fictional 3D spaces, sparse 3D reconstruction from 1 to 100+ images, and physics simulation for agents and robotics. - `05:00` Unlike its predecessor Marble, which routed all outputs through a 3D Gaussian splat bottleneck, Atlas features a redesigned architecture that bifurcates 2D direct pixel streaming from explicit 3D mesh and splat reconstruction. - `03:40` By grounding reference images at coordinates along 3D camera paths, Atlas mitigates the temporal coherence drift common in generative video, enabling stable generations lasting up to a minute. - `10:00` Atlas demonstrates real-to-sim adaptation for robotics, using casual smartphone imagery to reconstruct environments and stage RL fine-tuning simulations for embodied foundation models within minutes.

## Key takeaways - `00:00` World models represent a general horizontal layer for visual and physical understanding, positioned as an analog to LLMs in textual reasoning. - `00:00` Atlas operates across three primary tasks: novel world generation, sparse environment reconstruction, and physical simulation. - `03:40` Atlas uses native 3D camera trajectories and spatial reference points rather than pure text prompts to eliminate video hallucinations and temporal drift. - `05:00` The Atlas architecture decouples direct 2D video pixel generation from explicit 3D Gaussian splats, removing the architectural bottleneck present in World Labs' earlier Marble model. - `06:50` Explicit 3D representations like Gaussian splats and polygonal meshes remain essential for local client-side rendering on VR headsets and integration into existing VFX and gaming pipelines. - `10:00` Real-to-sim pipelines enable evaluation and reinforcement learning fine-tuning of robotic foundation models in precise digital twins of targeted physical spaces. - `12:35` Incumbent AI labs focused heavily on LLMs may face an Innovator's Dilemma, creating an opening for dedicated horizontal world model platforms. - `14:40` Future game development may shift toward coding agents querying world model APIs to build and stage 3D environments on the fly. - `14:40` World Labs designs generative 3D tools for deterministic, director-level control rather than non-deterministic, slot-machine-style random generation. - `18:25` Atlas achieved dynamic 'bullet time' spatial freeze-frame reconstructions, such as a fluid droplet splash, using input from only three synchronized consumer smartphones. - `20:00` Camera trajectories are treated as native multimodal inputs within Atlas, replacing vague text prompt commands with precise spatial choreographies. - `20:00` Standard Newtonian physics inaccuracies in world models are typically addressable via supervised fine-tuning, whereas quantum or relativistic modeling would require architectural redesigns. - `21:28` Long-term explorations for 3D world models include spatial chip design and native virtual reality interfaces for exploring generated environments.

## Detailed walkthrough ### `00:00` Introducing Atlas and the Thesis Behind World Models

The conversation opens with World Labs co-founder Justin Johnson explaining the foundational concept behind their research, contrasting world models directly with language models. While LLMs emerged as horizontal, general-purpose engines built to process discrete text tokens, Johnson explains that an equivalent, fundamental horizontal layer is needed for physical space:

> “And our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds.”

Hosts Theo Jaffee and Sofia Puccini introduce Johnson right after World Labs announced Atlas—billed as the world's first multimodal world model capable of generating image and video frames with precise camera trajectory control and reconstructing spaces in 3D. When asked to define Atlas and describe its potential use cases, Johnson clarifies that this announcement is a foundational model release rather than an immediate consumer product release. Products will follow, but Atlas serves as the base model built to power future World Labs offerings.

Johnson breaks Atlas down into three primary capabilities: 1. **Generation:** Creating entirely new, fictional environments from text or image prompts where a user can direct and fly a camera through space that never existed before, unlocking applications in VFX and gaming. 2. **Reconstruction:** Modeling existing physical environments using sparse reconstruction. A user can input as few as one photo or up to 100 or more; as more photos are provided, Atlas reconstructs the real-world scene with increasing fidelity. 3. **Simulation:** Turning a reconstructed 3D space into an interactive arena to simulate physical actions, such as modeling how a robot might behave or navigate within that specific environment.

When Jaffee asks whether World Labs' ultimate ambition leans toward training environments for robotics or building an immersive metaverse for human interaction, Johnson responds that it is “all of the above.” The mission is not restricted to one vertical; it aims to become the foundational visual and spatial intelligence layer across entertainment, virtual reality, architecture, construction, and robotics.

```mermaid flowchart TD accTitle: Atlas Core Capabilities accDescr: Atlas processes text, image, and spatial prompts into generation, reconstruction, and simulation.

subgraph Inputs["Input Signals"] A["Prompts & Images"] B["Camera Trajectories"] end

subgraph Engine["Atlas World Model Engine"] C["Unified Multimodal Intelligence"] end

subgraph Outputs["Core Capabilities"] D["Generation: Novel 3D Worlds"] E["Reconstruction: Real Spaces to 3D"] F["Simulation: Physics & Robotics"] end

A --> C B --> C C --> D C --> E C --> F ```

---

`03:40` Spatial Control vs. Video Generation Hallucinations

Sofia Puccini brings up a classic limitation with generative video and procedural world synthesis: after roughly 10 seconds, traditional video generation models drift, lose coherence, and begin hallucinating uncontrolled changes. Johnson explains that Atlas was designed from inception around direct spatial control and explicit spatial contexts to eliminate this drifting behavior.

Instead of treating reference images as flat, isolated 2D inputs, Atlas grounds them directly in 3D space. Creators can stage reference images at specific 3D coordinates, establishing concrete reference points across an environment:

> “So the idea here is that even as you go to really long generations, it's not a slot machine.”

- Users guide the video output by steering the camera path with precise pixel-level control rather than relying solely on open-ended text descriptions. - Reference images can be dropped like "breadcrumbs" along a 3D camera trajectory, enabling continuous, coherent video generations extending up to a minute without visual degradation or drift. - This methodology gives artists and engineers true directorial control, replacing random output generation with reproducible choreography.

---

`05:00` Rebuilding the Stack: Moving Beyond Marble and Gaussian Splats

Theo Jaffee brings up a viral open-source project created by their friend Tyce from OpenAI, which built a *Grand Theft Auto*-style recreation of San Francisco using a massive Gaussian splat where users could drive cars and walk around. Jaffee asks whether Atlas will enable similar experiences. Johnson confirms it will, explaining that there are two technological paths to interactive worlds: - **Explicit 3D pipelines:** Generating 3D Gaussian splat scenes or meshes that developers import into traditional engines like Unreal or Unity. - **Neural frame generation:** Bypassing traditional game engines entirely to stream generated frames directly without requiring an intermediate explicit 3D mesh.

Johnson explains how Atlas differs from World Labs' earlier release, Marble. While both models pursue multimodal inputs and outputs, Marble routed all generations through an architectural bottleneck:

> “Marble could output videos, Marble could output 3D scenes, Marble could output meshes. But everything was kind of bottlenecking through a 3D Gaussian splat representation.”

In Marble, any video generation required first generating a 3D Gaussian splat scene and rendering it back down into 2D frames. Atlas was completely redesigned from the ground up with a unified, scalable architecture that splits 2D and 3D pathways much earlier. If a user only needs high-fidelity video, Atlas produces direct pixels without passing through an explicit 3D splat intermediate.

```mermaid flowchart LR accTitle: Architectural Shift from Marble to Atlas accDescr: Marble bottlenecked outputs through Gaussian splats, whereas Atlas separates 2D pixel generation from explicit 3D reconstruction.

subgraph Marble["Marble Architecture"] M1["Inputs"] --> M2["3D Gaussian Splat"] M2 --> M3["Rendered 2D Video"] M2 --> M4["3D Assets"] end

subgraph Atlas["Atlas Architecture"] A1["Inputs"] --> A2["Unified Model Layer"] A2 -->|"Direct Pixel Path"| A3["Direct 2D Video"] A2 -->|"Spatial Path"| A4["3D Gaussian Splats & Meshes"] end ```

---

`06:50` The Continuing Role of Explicit 3D in Production Workflows

When asked if the industry is outgrowing Gaussian splats, Johnson pushes back against a purely neural view. While future systems might eventually stream real-time pixels directly from centralized server farms, that approach remains far too computationally expensive today. Explicit 3D assets such as Gaussians and polygonal meshes remain essential for client-side rendering on VR headsets and mobile devices, as well as for fitting into existing industry pipelines.

Johnson outlines the internal interface World Labs developed for working with Atlas: - Creators drag reference images onto an interactive 3D spatial canvas. - The interface assigns proxy geometries to those images in 3D space. - The user navigates the scene with first-person shooter (FPS) camera controls to establish exact viewpoints and flight paths.

Meeting existing creative professionals where they work is critical for real-world adoption:

> “So people in gaming and VFX and design and architecture, they have their existing workflows that work on 3D representations. And, you know, instead of trying to switch them over all at once to a fully AI-native thing, if you can get with AI into their workflows and meet them where they are, then that's another great use of splats and other explicit 3D representations.”

As the segment concludes, Jaffee references *Microsoft Flight Simulator*, highlighting how it reconstructs a functional 3D digital twin of the Earth using satellite imagery, and prompts Johnson on what technical hurdles remain before Atlas can generate interactive worlds of that magnitude dynamically.

`10:00` Explicit 3D, Client Streaming, and Real-to-Sim Robotics

Theo Jaffee questions whether world models will soon be capable of generating content quickly enough to reproduce something like Microsoft Flight Simulator or an interactive Google Maps where users can freely traverse any space on Earth. He wonders whether current architectures represent the right technological trajectory or if it is simply "overkill." Justin Johnson counters that this future will arrive "sooner than you expect" and explains that World Labs designed Atlas specifically to accommodate two parallel technological paths: 1. **Explicit 3D representation:** Generating scenes as explicit 3D Gaussian splats or 3D point clouds, which the client device can then freely interpolate and render locally. 2. **Video frame streaming:** Generating and streaming novel view video frames dynamically to the client.

Sophia Puccini turns the discussion toward robotics, specifically asking how "real-to-sim" pipelines—taking real video footage and turning it into an interactive simulation—will evolve the field. Johnson explains that real-to-sim serves two critical functions: evaluation and targeted adaptation. In high-stakes settings like factories, evaluating whether an embodied agent functions properly requires testing it directly inside an accurate digital replica of its target operating environment. While developers could theoretically spend months gathering physical demonstrations or teleoperating robots on site, simulations provide an infinitely repeatable alternative.

```mermaid flowchart TD accTitle: Real-to-Sim-to-Real Pipeline Architecture accDescr: Diagram showing how phone captures feed into Atlas to generate simulations, allowing RL fine-tuning for robotics before physical deployment.

subgraph Capture["1. Physical Space Capture"] A["Casual iPhone Photos / Video"] B["Natural Language Task Spec"] end

subgraph AtlasEngine["2. World Labs Atlas Engine"] C["3D Space Reconstruction"] D["Interactive Simulation Staging"] end

subgraph PolicyTuning["3. Policy Adaptation"] E["Pre-trained Robotics Foundation Model"] F["RL Fine-Tuning in Generated Sim"] end

subgraph Deployment["4. Real-World Execution"] G["Adapted Robot Deployed to Exact Space"] end

A --> C B --> D C --> D E --> F D --> F F --> G ```

Puccini asks whether general-purpose robots will always require an environment-specific simulation to scale reliably. Johnson outlines the two prevailing approaches in the robotics community: - **In-context imitation learning:** Supplying a general foundation model with a single visual demonstration of a task inside its context window, expecting the robot to infer what to do on the fly. - **Real-to-sim-to-real adaptation:** Capturing a physical space (such as a podcast studio) using casual smartphone imagery, having Atlas reconstruct the 3D geometry and stage the interactive simulation, running an automated software agent to synthesize tasks described in natural language, and using reinforcement learning (RL) to fine-tune the robotics foundation model within minutes.

As Johnson summarizes: > “And now you've got a robot maybe in the span of a couple minutes even that could come in and is perfectly adapted to this space.”

---

`12:35` OpenAI's Sora, Innovator's Dilemma, and World Models as a Horizontal Platform

The conversation shifts to the history of video generation. Theo notes that in late 2023 and early 2024, OpenAI originally pitched Sora in technical reports under titles like *Video generation models as world simulators*, mirroring the core philosophy of World Labs. However, Theo observes that OpenAI seemingly abandoned world modeling, pivoting toward consumer social media experiences with Sora 2 before pausing it altogether.

When asked why OpenAI shifted away from world models, Johnson refrains from speculating on internal company politics but emphasizes World Labs' distinct strategic thesis: > “I don't want to comment on OpenAI in particular. I don't really know much about their internal strategy or discussions. But our thesis is that world models are this general horizontal platform that can be used to go after a lot of different applications. And there's going to be a ton of value unlocked there.”

He suggests that incumbent AI labs might be facing a classic Innovator's Dilemma: > “I think there is maybe a sense of a bit of innovators dilemma, right? Like you're sitting on LLMs and you've got this as a really powerful thing and you're leading the pack there. It doesn't necessarily make sense to try to invest in that next thing. But, you know, for us, we think that there is another category of models out here. There's a ton of opportunities out here and we want to be the ones to go and grab them.”

---

`14:40` "Vibe Coding" Video Games, Human Creativity, and Director-Level Control

Sophia and Theo explore consumer applications, predicting that world models will first hit virtual reality, augmented reality, and video game development. Theo draws an analogy to software engineering, suggesting that while the industry is currently "vibe coding" websites, people will soon vibe code full interactive 3D video games using world models.

Johnson agrees that rapid deployment and open accessibility are central to their mission. Atlas is designed to expose general capabilities across simulation, 3D reconstruction, and generation via APIs. In this paradigm, coding agents can query world models and stitch these spatial capabilities together into playable games without requiring creators to manually assemble game engines or hand-place millions of environment assets.

Theo wonders aloud if upcoming landmark titles like *Grand Theft Auto VI* (GTA 6) might represent "the last great piece of human written software before all future video games are... kind of, generated on the fly with stuff like Atlas." Sophia pushes back, arguing that video games are an art form that will always retain a human in the loop.

Johnson agrees with Sophia, contextualizing the shift within the broader history of animation and digital art: - **Early Disney animation:** Hand-painting physical cel animations was a groundbreaking technology in its time, followed later by synchronization of sound and picture. - **Pixar's computer graphics revolution:** Transitioning from practical sets to computer-generated rendering expanded creative storytelling rather than eliminating human directors. - **The generative AI transition:** Generative 3D tools serve as the next evolution of digital cameras, brushes, and rendering engines.

Johnson adamantly rejects building "generative slot machines where you just like pull the thing and hope you get out a good generation" without having personal ownership over the output. Instead, World Labs focuses on providing precise, deterministic director-level control: > “Like we want to build these tools that have really deep control, right? Like whether it's 3D control, whether it's text control, that you feel like you're a director in the thing guiding this model and deciding exactly what it outputs rather than just like the model spits something out and maybe I can stitch it together after the fact.”

---

`18:25` Physics Simulation and Three-Phone "Bullet Time" Flyarounds

Theo asks about the limits of Atlas’s physics understanding, probing edge cases like the extreme gravitational forces and spaghettification around a black hole or difficult everyday phenomena like dynamic hair. Johnson admits with a laugh that he hasn't tried simulating a black hole yet—joking that he might test it that afternoon and tweet the result—though he suspects Atlas would fall back on standard collision models rather than simulating general relativity.

However, Johnson reveals an unexpected breakthrough in the model's spatial rendering: synchronized multi-view "bullet time" shots. Using just three consumer smartphones mounted on standard tripods, the team captured dynamic real-world events simultaneously. Atlas was able to process these three synchronized video streams and render seamless, frozen-moment 3D camera flyarounds: - **The strawberry in oat milk demo:** World Labs tested the system by dropping a strawberry into a bowl of oat milk. At the exact millisecond of impact, as the milk splashes upward, Atlas reconstructs the dynamic splash in full 3D, allowing users to freeze time and pilot an interactive virtual camera all around the fluid droplet structure. - **Input efficiency:** Reconstructing full 3D spatial freeze-frames typically requires expensive specialized multi-camera rigs; Atlas generated the entire scene from only three standard iPhone angles.

Johnson highlights this as proof of the immense power of 3D spatial understanding: even when producing standard 2D outputs for VFX or cinema, grounding the model in explicit 3D geometry on the input side gives creators unprecedented control over camera angles, framing, and dynamic motion.

`20:00` Native Camera Control and Physical Simulation Limits in Atlas

```mermaid flowchart LR accTitle: Physics Regimes in World Models accDescr: Diagram contrasting Newtonian physics handled by Atlas with non-Newtonian physics requiring architectural changes.

subgraph Solvable["Standard Regimes"] direction TB A["Macro / Newtonian Physics"] --> B["Covered by Pre-training Data"] B --> C["Edge Cases Solved via SFT"] end

subgraph Complex["Non-Newtonian Regimes"] direction TB D["Extreme / Quantum Physics"] --> E["Black Holes & Relativistic Mechanics"] D --> F["Nanoscale & Microscopic Mechanics"] E & F --> G["Requires New Data & Architectures"] end ```

Justin Johnson explains the necessity of providing human creators with direct spatial control surfaces when interacting with generative models. Relying purely on natural language descriptions falls short when orchestrating a complex scene; typing commands like "pan in," "pan out," or "truck left" fails to capture the nuanced motion a director wants. To solve this, Atlas is built with camera trajectories as a native, multimodal input format, allowing users to physically manipulate and choreograph viewpoints with high precision. When asked about the system's ability to model extreme physics like black holes or atomic-scale interactions, Johnson distinguishes between standard macroscopic physics and non-Newtonian regimes. Because standard Newtonian mechanics are already well represented in pre-training datasets, any persistent flaws at that scale can usually be corrected through supervised fine-tuning (SFT). However, depicting relativistic phenomena or quantum mechanics would demand significant revisions to both model architectures and training pipelines.

- Justin Johnson emphasizes direct control over text prompting: “Because I don't just want to write in text like pan in, pan out, truck left.” - On native camera input handling: “Atlas has camera control as a native input type. It's multimodal and one of those modalities is cameras.” - Addressing everyday mechanics versus exotic physics: “So anything that's like roughly Newtonian that doesn't work at is probably just a quick SFT away.” - On the boundaries of current world modeling: “Once you're down in the quantum mechanics realm, I think then you'd have to rethink a bit the architecture and the data side.”

`21:28` Chip Design Dreams, VR Navigation, and Closing Thoughts

The conversation turns to the future possibilities of spatial modeling at microscopic scales. Johnson shares an early ambition from the founding of World Labs: utilizing 3D world models to engineer next-generation silicon chips that could subsequently be used to train even larger foundation models. The hosts consider how humans will ultimately interface with these simulated environments, noting that conventional keyboard-and-mouse navigation (like WASD controls) is bound to give way to virtual reality as the primary interface for exploring and prompting 3D worlds. Johnson notes that World Labs has already integrated virtual reality with their system internally for nearly a year, describing the immersion of standing inside a completely generated space as a profound shift. The episode concludes with expressions of anticipation for what creators will build once Atlas is fully released to the public.

- Justin Johnson on recursive hardware design: “And how cool would it be to use a 3D world model to design a chip that we then use to train the next generation world model?” - On stepping into generated environments: “We've had Marvel in VR for like almost a year at this point. That is a really transformative experience if you guys haven't tried it, right?” - Johnson on the subjective experience of the technology: “headset and I'm in this complete AI generated world that I created. It's quite amazing.”

## Notable quotes > `00:00` our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds. — Justin Johnson

## Mentioned Theo Jaffee, Sofia Puccini, Justin Johnson, World Labs, Atlas, Marble, Tyce, OpenAI, Sora, Sora 2, Unreal Engine, Unity, Microsoft Flight Simulator, Google Maps, Grand Theft Auto VI, Disney, Pixar, iPhone

## Video description World Labs co-founder Justin Johnson joins MTS hosts Theo Jaffee and Sophia Puccini to discuss Atlas, World Labs’ latest world model, and the broader case for AI systems that understand and interact with the physical world. Justin explains how Atlas approaches three core tasks: generating new worlds, reconstructing real environments from images, and simulating how objects or robots might behave within them. Underlying it is a bigger thesis: just as language models became general-purpose engines for working with text, world models could become a horizontal layer for visual and physical intelligence across industries from entertainment and gaming to construction and robotics. They also explore how world models could change video games and creative tools, why precise spatial control matters, and the potential for “real-to-sim-to-real” robotics, where a few photos of a physical environment could eventually be enough to build a simulation and adapt a robot to that specific space.

[Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/e578e083-6b40-4815-a479-28213e86988a/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=e578e083-6b40-4815-a479-28213e86988a&feed=JGE3yC0V)

中文简报

World Labs推出Atlas，以3D控制取代视频漂移直接的空间锚定与摄像机轨迹输入消除了生成式视频模型的一致性缺陷，构建了一个能够在数分钟内模拟环境并调整机器人策略的横向基础层。 World Labs联合创始人Justin Johnson详细介绍了Atlas，这是一款具备3D场景生成、稀疏物理重建和物理模拟能力的基础多模态世界模型。他阐述了Atlas如何通过支持直接的2D视频生成以及显式3D高斯泼溅（Gaussian splat）导出，突破其早期模型Marble的架构瓶颈，并利用原生摄像机轨迹输入解决了视频漂移问题。讨论探讨了颠覆性的应用场景，包括在数分钟内使机器人适应特定环境的“现实到模拟再到现实”（real-to-sim-to-real）管线，以及通过细粒度导演级控制而非随机生成来驱动的视频游戏开发。Johnson还通过三部手机拍摄的定格水花捕捉展示了该系统的空间保真度，并展望了虚拟现实接口和递归芯片设计的前景。 World Labs发布了Atlas，这是一款围绕三大核心能力构建的基础多模态世界模型：生成虚构3D空间、基于1到100多张图像的稀疏3D重建，以及面向智能体和机器人技术的物理模拟。 (0:00) 与此前将所有输出都经由3D高斯泼溅瓶颈处理的前身Marble不同，Atlas采用了重新设计的架构，将2D直接像素流与显式3D网格及高斯泼溅重建分流处理。 (5:00) 通过将参考图像锚定在3D摄像机路径的坐标上，Atlas缓解了生成式视频中常见的时间一致性漂移，实现了长达一分钟的稳定生成。 (3:40) Atlas展示了面向机器人技术的“现实到模拟”适配能力，能够利用随手拍摄的智能手机图像重建环境，并在数分钟内为具身基础模型搭建强化学习微调模拟。 (10:00)

Reading summary

World Labs co-founder Justin Johnson argues that physical space requires a fundamental horizontal intelligence layer comparable to language models, introducing the Atlas foundation model to unify 3D generation, real-world reconstruction, and interactive simulation. The Core Capabilities of World Models (0:00) Johnson contrasts world models with large language models, explaining that while LLMs handle discrete text tokens, physical intelligence requires visual and spatial comprehension. Atlas is introduced not as a consumer application, but as a foundational base model designed around three core functions: generating fictional 3D spaces from text or image prompts, reconstructing real-world physical scenes from anywhere between one and over one hundred photos, and staging interactive simulations for physics and robotics. Johnson contends this foundation will serve as an infrastructure layer spanning gaming, VFX, architecture, and embodied AI rather than targeting a single vertical. Deterministic Spatial Control Versus Video Drift (3:40) Traditional generative video models typically drift and hallucinate uncontrolled changes after roughly 10 seconds. Johnson states that Atlas prevents this degradation by replacing open-ended text prompting with explicit 3D spatial anchoring. Creators can place reference images at exact 3D coordinates like breadcrumbs along a planned path, enabling continuous, coherent video generation for up to a minute without visual decay. Atlas treats camera trajectories as a native multimodal input modality, allowing users to direct camera paths with pixel-level precision rather than relying on ambiguous text commands like panning or trucking. Architectural Decoupling and Industry Pipelines (5:00) Atlas modifies the pipeline of World Labs' earlier model, Marble. Marble routed all video generation through an architectural bottleneck by first creating a 3D Gaussian splat and rendering it down into 2D frames. Atlas redesigns this stack into a unified architecture with separate pathways, producing direct video pixels when requested while maintaining a distinct spatial pathway for 3D meshes and Gaussian splats. Johnson defends the continued necessity of explicit 3D representations, noting that streaming generated neural frames directly from servers remains computationally prohibitive. Formats like polygonal meshes and splats enable client-side rendering on VR headsets and mobile devices and integrate seamlessly into existing production tools used by gaming and VFX studios. Simulation and Real-to-Sim Adaptation for Robotics (10:00) In robotics, Atlas is positioned to accelerate real-to-sim-to-real pipelines as an alternative to teleoperation or gathering physical demonstrations on site. A user can capture a physical setting using casual smartphone imagery, have Atlas reconstruct the geometry, and run an automated agent to stage interactive environments. Pre-trained robotics foundation models can then undergo reinforcement learning fine-tuning within the simulated space in minutes before physical deployment. To demonstrate spatial reconstruction capabilities from minimal consumer hardware, Johnson highlights a test where three synchronized iPhones captured a strawberry dropping into oat milk; Atlas reconstructed the dynamic fluid impact in full 3D, allowing users to navigate a frozen-moment camera flyaround around the splash. Physical Boundaries and Long-Term Horizons (20:00) Regarding physical accuracy, Johnson distinguishes standard Newtonian mechanics from extreme non-Newtonian environments. Because everyday physics are heavily represented in pre-training data, visual errors at normal scales can generally be remedied through supervised fine-tuning. Conversely, simulating microscopic scales, quantum mechanics, or relativistic phenomena like black holes would require entirely new datasets and architectural adjustments. Looking ahead, Johnson notes internal experiments interfacing with generated environments using VR headsets, and cites an early founding ambition to use 3D spatial modeling to aid the physical design of next-generation silicon chips. While Atlas demonstrates multi-camera reconstruction and direct camera trajectory steering, Johnson acknowledges that direct real-time neural streaming remains constrained by computational costs, leaving explicit 3D representations essential for near-term production, while modeling non-Newtonian physics remains outside current architectures.

精炼阅读摘要

World Labs 联合创始人 Justin Johnson 指出，物理空间需要一个类似于大语言模型的底层通用智能层，该公司推出的 Atlas 基础世界模型旨在通过直接空间控制与物理理解，统一场景生成、稀疏三维重建与物理仿真。世界模型作为空间基础层 (0:00) Justin Johnson 认为大语言模型解决了离散文本 token 的通用处理，但在物理空间领域仍缺乏一个基础的横向底座。Atlas 被定位为多模态世界模型的基础层，而非单一的消费级产品，其核心能力涵盖三维空间生成、物理环境稀疏重建以及交互式物理仿真。用户只需提供一张至数百张照片，系统就能逐步提高保真度重建真实场景。World Labs 的长远目标不是单一垂直领域，而是覆盖影视特效、游戏开发、建筑施工、虚拟现实与具身机器人等全流程的空间智能基础设施。空间锚定消除视频漂移 (3:40) 传统视频生成模型在生成时长超过约 10 秒后往往会出现漂移与不可控幻觉，Justin Johnson 解释称 Atlas 从设计之初就引入了直接空间控制与显式空间上下文。创作者可以将参考图像直接放置在三维坐标中作为空间锚点，并通过精确的像素级相机轨迹来引导画面演进，而非仅仅依赖模糊的自然语言提示。在三维轨迹上像面包屑一样布设参考图，可实现长达一分钟且不发生质量衰退或漂移的连续连贯视频生成，从而将随机抽卡式的生成转变为高确定性的导演级运镜编排。解耦显式三维与像素生成架构 (5:00) World Labs 此前发布的 Marble 模型存在结构瓶颈：所有视频、三维场景与网格的输出都必须经过三维高斯溅射（3D Gaussian Splat）的中间表征，导致生成视频需要先生成高斯场景再渲染为二维帧。Atlas 对此进行了重构，在架构早期将直接二维像素通道与三维几何通道分流。对于仅需高质量视频的场景，Atlas 无需构建显式三维实体即可直接输出像素；而对于需要集成到现有游戏引擎或在 VR 设备上进行本地渲染的工业工作流，系统仍保留了显式的高斯点云与多边形网格输出通道。 Real-to-Sim 加速具身机器人适配 (10:00) 在机器人领域，Atlas 推动了“真实到仿真再到真实”（Real-to-Sim-to-Real）工作流的建立。Justin Johnson 解释，与其在现实工厂中耗费数月进行物理示教或远程遥操作，不如使用普通智能手机拍摄环境视频，由 Atlas 在几分钟内重建三维几何并搭建交互式仿真环境。软件智能体随后可根据自然语言任务要求，在仿真环境中通过强化学习快速微调预训练的机器人基础模型，使具身智能体在极短时间内完成对全新物理场景的精准适配与评估。原生相机控制与物理模拟边界 (20:00) Atlas 将相机运动轨迹作为原生多模态输入，允许创作者通过第一人称视点拖拽或设备位姿直接操控画面，克服了文本输入在精细运镜上的表达缺陷。例如，团队仅用三部普通 iPhone 同步拍摄草莓落入燕麦奶的瞬间，Atlas 即可完成流体飞溅的三维瞬时重建并支持全自由度视角环绕。面对物理规律模拟的边界，Justin Johnson 表示宏观牛顿力学已被大规模预训练数据覆盖，细微偏差多可通过监督微调纠正；但黑洞引力、相对论或纳米尺度的量子力学现象，仍需在模型架构与训练数据层面上进行重构。 World Labs 尝试通过原生相机轨迹与显式空间坐标赋予生成模型高度确定性，并寄望该技术未来能用于芯片设计等底层空间架构；然而，极端非牛顿物理规律的建模仍受限于现有架构，全神经实时像素流生成在算力成本上也依然面临落地挑战。

Detailed notes — zh

## Background 本期播客于 2026 年 9 月 13 日由 Theo Jaffee、Sofia Puccini 和 Justin Johnson 在 RSS / web 平台上联合发布。节目单集页面介绍显示，World Labs 联合创始人 Justin Johnson 作客 MTS 播客，探讨其最新世界模型 Atlas，以及能够理解并与物理现实交互的 AI 系统如何在娱乐、游戏、建筑和机器人等行业中充当通用的横向底层基座。 ## TL;DR Justin Johnson 详细阐释了 World Labs 新发布的基座世界模型 Atlas 的三大核心能力：生成全新世界、稀疏视图 3D 重建以及高保真物理仿真。与传统视作“生成式老虎机”的纯扩散视频模型不同，Atlas 原生支持摄像机运动轨迹与 3D 空间上下文控制，并兼顾显式 3D（如 3D 高斯泼溅）与直接视频流生成的双轨技术路线。节目还探讨了极速适配机器人的“Real-to-Sim-to-Real”闭环工作流、为何 OpenAI 在 Sora 后未能坚守世界模型基座的创新者窘境，以及仅凭三部 iPhone 捕捉草莓落入燕麦奶即实现震撼“子弹时间”的突发物理能力与 VR 沉浸交互的未来前景。 ## Atmosphere & style 对话充满前沿极客特有的兴奋感与深度技术审视，节奏紧凑明快，在硬核架构讨论与未来工业想象之间自如切换。Justin Johnson 展现出兼具学者严谨与创业者敏锐的从容气度，与主持人们关于 GTA 6 是否是“最后一款人类代码软件”、用世界模型设计芯片训练下一代 AI 等思想实验撞出诸多火花。整场交流既打破了盲盒抽卡式生成的幻梦，又让人深切感受到戴上 VR 头显真正步入 AI 生成的三维现实时那份令人屏息的变革力量。 ## Bulletin - `00:00` World Labs 联合创始人 Justin Johnson 发布全新基座世界模型 Atlas，提供生成、稀疏 3D 重建与仿真三大核心能力，旨在成为视觉与物理智能的横向通用层。 - `06:48` Atlas 抛弃了将 3D 高斯泼溅作为强制中间瓶颈的架构，在底层提早解耦 2D 像素直出与 3D 几何资产生成，兼顾轻量渲染与高保真输出。 - `10:50` 基于 Atlas 的“Real-to-Sim-to-Real”流程仅需手机拍摄数张照片即可重建环境，配合 Coding Agent 与强化学习，可在数分钟内让机器人完美适配全新实体工作空间。 - `20:00` Atlas 将 3D 摄像机轨迹设计为原生输入模态，彻底摆脱传统文本生成视频的“老虎机抽卡”模式，赋予创作者导演级的高精度空间控制力。 ## Key takeaways - `00:00` 世界模型被定位为平行于大语言模型的全新通用水平层，专注解决通用视觉理解、空间几何构建与物理规律仿真。 - `00:00` Atlas 具备强大的稀疏重建能力，仅凭 1 到 100 余张 2D 照片即可精准复原真实物理空间的 3D 结构。 - `03:08` Atlas 通过将参考图像锚定在 3D 空间作为“面包屑”轨迹并引入像素级相机控制，解决了传统视频生成模型在长时程下画面崩溃与漂移的问题。 - `05:00` Atlas 同时兼顾显式 3D（生成高斯场景与网格）和神经渲染（端到端流式生成视频帧）两条技术演进路线，适应不同算力与硬件场景。 - `06:48` 相比上一代产品 Marble，Atlas 打破了必须经由 3D 高斯作为中间转换的瓶颈，使高精度 2D 像素与 3D 几何结构能够按需独立输出。 - `06:48` 传统影视特效、游戏与移动端硬件依旧高度依赖显式 3D 资产，AI 工具融入既有管线比完全依赖云端像素串流更具现实可行性。 - `10:50` “Real-to-Sim-to-Real”范式替代了成本高昂的人工遥操作采集，利用世界模型重建虚拟空间并进行强化学习微调，实现低成本、极速部署机器人。 - `13:16` OpenAI 未在 Sora 之后持续深耕世界模型，或源于大语言模型赛道一马当先所带来的“创新者窘境”，从而给专注于通用物理底座的新团队留下了历史性机遇。 - `14:40` Atlas 为编程智能体提供了标准 API 接口，使 Coding Agent 能将环境重建与代码逻辑编织起来，实现“Vibe Coding”直接构建互动 3D 游戏。 - `16:04` World Labs 明确反对将 AI 创意工具做成盲盒式的“生成式老虎机”，坚持让创作者坐在驾驶座上，通过 3D 空间交互获得深度的导演级掌控力。 - `18:18` 实验证明 Atlas 仅需三部普通手机的多视角拍摄，即可完美重构水滴飞溅的高动态瞬间，实现高保真的 360 度“子弹时间”运镜穿行。 - `20:00` 文本提示词在描述 3D 运动时存在天然局限，Atlas 将摄像机控制提升为原生输入模态，支持创作者直接抓取机位进行精准变焦和轨迹运镜。 - `20:41` 宏观牛顿力学物理规律已存在于预训练数据中并可通过 SFT 快速修复，但黑洞或纳米量子等极端非牛顿物理场景则需要重构模型架构与训练数据。 - `22:04` 键盘鼠标不足以发挥世界模型的潜力，戴上 VR 头显直接沉浸于 AI 原生生成的 3D 全景空间，才是世界模型最本质的交互与提示形态。 ## Detailed walkthrough ### `00:00` 世界模型与 Atlas 的三大核心能力 Justin Johnson 作为 World Labs 的联合创始人，介绍了公司刚发布的基座模型 Atlas。他澄清这并非最终的产品发布，而是一个底层基座模型，未来 World Labs 的各类产品都会基于它来构建。Justin Johnson 解释道，如果说语言模型是处理离散文本序列的通用水平层，那么世界模型则是对应视觉与物理理解的全新通用水平模型。Atlas 的核心能力主要由三个维度构成： ```mermaid flowchart TD accTitle: Atlas 三大核心能力架构 accDescr: 展示 Atlas 作为基础世界模型，具备生成、重建与仿真三项核心功能及对应下游应用。 subgraph AtlasModel["Atlas 世界模型基座"] direction TB G["1. 生成 (Generation) 文本/图像 Prompt -> 全新虚构世界"] R["2. 重建 (Reconstruction) 稀疏图像输入 (1至100+张) -> 3D空间建模"] S["3. 仿真 (Simulation) 物理/视觉环境 -> 机器人与物体行为模拟"] end subgraph Applications["下游行业应用"] App1["创意设计与游戏 / VFX"] App2["VR / 元宇宙 / 建筑"] App3["具身智能与机器人训练"] end G --> App1 R --> App2 S --> App3 ``` - **生成（Generation）**：用户可以输入文本 Prompt 或图像 Prompt，在 3D 空间中自主引导相机运镜（比如运镜穿行），让模型从无到有生成以前从未存在过的全新世界，服务于游戏和视觉特效（VFX）等创意行业。 - **重建（Reconstruction）**：针对现实中真实存在的空间进行稀疏重建（Sparse Reconstruction）。用户只需提供少至 1 张、多至 100 张以上的照片，输入的图片越多，模型对真实物理空间的 3D 还原精度就越高。 - **仿真（Simulation）**：将真实空间重建成 3D 之后，进一步模拟物体或机器人在该环境中的物理行为，主要面向机器人训练与视觉特效。当被主持人问及 World Labs 的终极愿景究竟是为机器人提供训练环境，还是构建供人类沉浸其中的元宇宙时，Justin Johnson 强调这是兼而有之的全局布局：“Our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds.” --- ### `03:08` 摆脱“抽卡”模式：空间上下文与像素级精确运镜主持人提出目前视频生成模型普遍存在“时间跨度一长就崩溃变形”的问题——例如生成到 10 秒左右画面就开始逻辑错乱、脱离参考图并产生各种扭曲幻觉。Justin Johnson 说明了 Atlas 在底层设计上的根本不同：它从一开始就是围绕“空间控制”（Spatial Control）和“空间上下文”（Spatial Context）构建的。 - **3D 空间定位与锚定**：在 Atlas 中，输入的参考图像不仅仅是一组扁平的 2D 像素，而是被直接定位和固定在三维几何空间中。 - **空间面包屑轨迹（Breadcrumbs）**：用户可以在三维场景的不同位置摆放多张参考图，就像在轨迹上沿途留下“面包屑”。 - **像素级运镜控制**：生成视频时，用户不仅依赖文本描述，还能直接操作相机轨迹（Pixel Camera Control），精确指定相机在每一帧的移动路径。 Justin Johnson 强调，通过这种类似导演视角（Directorial Control）的干预机制，即便生成长达一分钟的连续视频，输出结果也不会像传统的扩散模型那样失控：“So the idea here is that even as you go to really long generations, it's not a slot machine. You're generating and directing and really controlling what the model outputs.” --- ### `05:00` 双轨并行路线与内部 3D 画布工具主持人提到 OpenAI 员工 Tyce 曾做过一个类似 GTA 的旧金山演示项目，利用整座城市的巨大 3D Gaussian Splatting（高斯泼溅）资产让用户在其中漫游和驾驶。Justin Johnson 指出，实现这类交互式虚拟世界其实存在两条完全不同的技术路线，而 Atlas 原生兼具这两种能力： | 技术路线 | 实现机制 | 优势与适用场景 | Atlas 的支持方式 | |:--- |:--- |:--- |:--- | | **显式 3D 路线 (Explicit 3D)** | 生成 3D Gaussian Splats 或网格（Mesh），导入传统游戏引擎进行光栅化渲染 | 便于编辑、可复用资产、适合客户端与轻量设备实时漫游 | Atlas 原生支持输出 3D 高斯场景并导出到引擎 | | **纯神经渲染路线 (Frame Generation)** | 跳过一切游戏引擎与显式几何体，模型端到端直接预测并串流生成 2D 视频帧 | 视觉拟真度极高、光影与材质自然，省去建模环节 | Atlas 具备直接渲染连续高质量 2D 像素帧的能力 | 关于创作者的具体使用流程，Justin Johnson 透露团队目前有一套非常惊艳的内部工具：用户可以像使用第一人称射击游戏（FPS）的控制器一样，把参考照片直接拖拽到一个三维立体画布（Spatial 3D Canvas）中，图片会自动吸附生成对应的空间代理几何体（Proxy Geometry），创作者再用类似游戏摇杆的操作去规划相机穿行的轨迹。 --- ### `06:48` 架构跃迁：从 Marble 到 Atlas 的解耦演进 Justin Johnson 详细对比了 World Labs 之前的第一代产品 Marble 与现在的基座模型 Atlas 之间的底层差异。虽然两者都追求多模态的输入与输出，但底层流水线已经经过了彻底重构： ```mermaid flowchart LR accTitle: Marble 与 Atlas 渲染流水线演进对比 accDescr: 对比 Marble 以 3DGS 为中介瓶颈的旧架构与 Atlas 提早解耦 2D/3D 的全新原生架构。 subgraph MarblePipeline["早期产品 Marble（强耦合瓶颈架构）"] direction LR M_In["输入多模态数据"] --> M_GS["核心瓶颈： 必须先构建 3DGS"] M_GS --> M_Out1["渲染输出 2D 视频"] M_GS --> M_Out2["导出 3D Mesh / 场景"] end subgraph AtlasPipeline["新一代模型 Atlas（提早解耦双通道架构）"] direction LR A_In["输入多模态数据"] --> A_Core["统一世界模型基座 (提前解耦 2D/3D)"] A_Core -->|"直出原生像素"| A_2D["高质量 2D 图像与视频"] A_Core -->|"按需生成几何"| A_3D["3DGS / 显式 3D 资产"] end ``` - **打破 3DGS 瓶颈**：Marble 的所有输出（无论是视频还是网格）都必须先经过 3D Gaussian Splatting 这一层中间表示作为瓶颈，再由高斯场景重采样或渲染成像素。而 Atlas 在模型底层更早的阶段就将 2D 渲染与 3D 几何处理彻底解耦。 - **像素直出与几何按需输出**：在 Atlas 中，如果最终交付目标只是 2D 视频（例如 VFX 镜头），模型会直接端到端输出精致的 2D 像素，无需忍受高斯转译带来的失真和瓶颈；只有在确实需要几何结构的场景下，模型才会进一步生成 3D 资产。 - **传统工作流与显式 3D 的现实价值**：面对“高斯技术是否正在过时”的疑问，Justin Johnson 明确反驳，指出高斯与网格依然是行业必需的工具。即便未来可能出现强大的云端服务器集群直接实时串流生成像素，目前的高昂算力成本也难以承受。移动端设备、VR 头显的客户端渲染，以及好莱坞 VFX、游戏工业与建筑设计现有的管线，都高度依赖显式 3D 资产。与其强迫整个行业瞬间颠覆流程，用 AI 生成高斯资产并融入现存管线才是务实的选择。对话最后，主持人引申出《微软模拟飞行》（Microsoft Flight Simulator）结合卫星数据重建地球的高保真案例，探讨 Atlas 未来动态生成超大规模世界的技术边界。 ### `10:00` 实时场景流式传输与显式 3D 技术路径主持人们探讨了世界模型是否能以足够快的速度生成内容，从而实现类似《微软模拟飞行》（Microsoft Flight Simulator）或全可遍历版 Google Maps 的效果，并质疑这是否属于“杀鸡用牛刀”。Justin Johnson 明确回应称这会比预想中来得更快，而 Atlas 在设计之初就同时兼顾了两条不同的技术路线：一条是显式 3D（Explicit 3D）路线，即通过生成 3D 高斯场景（3D Gaussian Splatting）或 3D 点云场景，随后由客户端完成画面插值；另一条路线则是直接流式传输视频帧。Justin Johnson 指出：“Atlas 就是为了兼顾这两种技术演进路线而构建的（Atlas was built to handle both of those tech trees）”，并且此类能力很快就能在实际系统中落地。 ### `10:50` 机器人的 Real-to-Sim-to-Real 与数分钟极速适配谈及“现实到仿真”（Real-to-Sim）在机器人领域的价值时，Justin Johnson 强调了仿真在最终部署环境中的评估与微调作用。传统做法需要采集大量人类遥操作（teleoperation）或实体机器人在特定场景的演示数据，而通过世界模型重建特定真实工厂或工作空间，能让机器人大模型在极度匹配的环境中直接检验与迭代。针对如何加速通用机器人的普及，他对比了两种演进路径：一种是纯上下文学习（in-context learning），即机器人仅需观察一次任务演示便能领悟并执行；另一种则是极具潜力的“Real-to-Sim-to-Real”闭环。 ```mermaid flowchart TD accTitle: Real-to-Sim-to-Real 工作流 accDescr: 展示从手机拍摄到 Atlas 重建、自然语言指定任务、强化学习微调以及最终机器人部署的闭环流程。 subgraph Capture["现实采集"] A["手机拍摄若干照片或随手录像"] end subgraph Simulation["Atlas 仿真构建"] B["Atlas 重建显式三维场景"] C["自然语言描述任务需求"] D["Coding Agent 自动搭建仿真场景"] end subgraph Adaptation["模型微调与部署"] E["强化学习 (RL) 微调基础大模型"] F["数分钟内完成机器人现场部署"] end A --> B B --> D C --> D D --> E E --> F ``` 用户只需拿着手机在录音室或工位周围随手拍几张照片或简短视频，Atlas 便能重建三维物理空间。随后，用户用自然语言描述希望机器人完成的任务（如挪动桌上的麦克风），由 AI 智能体自动组装出专属仿真环境，并利用强化学习（RL）对通用的机器人基础大模型进行微调。Justin Johnson 总结道：“现在你可能仅花上几分钟的时间，就能让一台机器人进入现场并完美适配这个空间（And now you've got a robot maybe in the span of a couple minutes even that could come in and is perfectly adapted to this space）。”这种类似“极速入职培训”（fast onboarding）的工作流，避开了极其昂贵且缓慢的人工微调。 ### `13:16` 创新者窘境：OpenAI 为何未坚持世界模型底座对话转向了 OpenAI 早期的战略转变。Theo Jaffee 提到，在 2023 至 2024 年间 Sora 最初发布时，其官方技术报告标题就是《视频生成模型作为世界模拟器》（Video generation models as world simulators），这与 World Labs 的愿景高度吻合；然而后来的 Sora 2 却走向了偏向消费级娱乐的 TikTok 仿制品，似乎放弃了将视频模型作为底层物理世界模拟器的研究方向。面对主持人关于 OpenAI 为何放弃该方向的提问，Justin Johnson 表示自己不便评判 OpenAI 的内部策略，但他给出了独到的商业与技术洞察：“我认为这里面可能存在一点创新者窘境的意味，对吧？你手里握着大语言模型，这是一个极其强大的杀手锏，而且你在这条赛道上一马当先，此时再去全力押注下一个全新的物种并不一定符合其商业逻辑（I think there is maybe a sense of a bit of innovators dilemma, right? Like you're sitting on LLMs and you've got this as a really powerful thing and you're leading the pack there. It doesn't necessarily make sense to try to invest in that next thing）。”他进一步指出，World Labs 的底层信念在于，世界模型是一层全新的横向通用技术底座，具有巨大的未开发价值，而他们正全力去抢占这个定义未来模型的机会。 ### `14:40` 消费级前沿、Vibe Coding 游戏与智能体互操作随着“Vibe Coding”（氛围式编码）从构建网页扩展到游戏创作，主持人们畅想普通用户未来是否能轻松生成自己的可交互 3D 世界，并好奇世界模型将如何与现有 LLM 栈结合。Justin Johnson 确认了这一愿景，他认为世界模型必须同时对人类创作者和 AI 智能体开放。 - **横向通用平台定位**：Atlas 同时整合了仿真、三维重建、显式 3D（点云/网格）生成以及 2D 画面生成等多维能力。 - **与 Agent 深度协作**：通过提供标准 API，编程智能体（Coding Agents）可以直接调用世界模型的能力，自动将重建环境与逻辑机制编织在一起，让用户能够极其敏捷地构建全新的交互体验。 - **统一工具生态**：如同在软件栈中增加 API 订阅一样，世界模型能够自然地与 LLM 和其他工具无缝互操作（interop），充当智能体观察与改变物理三维世界的“物理引擎”。 ### `16:04` 创意控制权与反对“生成式老虎机” 主持人 Theo Jaffee 大胆猜测即将到来的《侠盗猎车手 6》（GTA 6）是否会成为“最后一部由人类手工编写的伟大软件”，未来的游戏是否将全面转向由 Atlas 这类模型实时动态生成。Sophia Puccini 与 Justin Johnson 均明确表示反对这种彻底取代人类创意的论调。 Justin Johnson 回顾了创意媒介的发展史：从迪士尼早期手绘每一张赛璐璐胶片（cell），到首次攻克音画同步技术，再到皮克斯（Pixar）引领的电脑三维动画（CG），每一次技术飞跃都提供了新的表达形式，而创意人员始终坐在驾驶座上。他格外强调了 World Labs 在产品哲学上对精确控制的坚持： > “我们绝不想打造那些生成式老虎机——你只是拉一下拉杆，然后祈祷能掉出来一个好的生成结果，或者觉得那个作品根本不属于你、你无法控制它，也预料不到会吐出什么东西来（We never wanted to be building these generative slot machines where you just like pull the thing and hope you get out a good generation. Or you don't feel like that generation was yours or that you controlled it or that you knew what you were getting out）。” 他阐述道，无论是通过三维空间控制还是文本约束，核心都在于让使用者拥有像电影导演一样的深度掌控力（director-level control），精确引导模型输出意图，而不是只能在模型盲盒输出后做被动的修补。 ### `18:18` 物理极端情况与令人惊艳的三镜头“子弹时间” 在讨论模型面对特殊物理现象的表现时，主持人询问如果让 Atlas 模拟“黑洞”（black hole）以及黑洞引力下的“面条化”（spaghettification）等非传统碰撞现象，模型是否能合理应对。Justin 打趣表示自己还没试过黑洞，或许当天下午就应该尝试一下并发布在 Twitter (X) 上，但他推测模型目前在黑洞这类超常规极端物理上的表现可能不会特别出彩；不过对于头发（hair）这类复杂物理资产，模型的还原度已经相当不错。随后，Justin 分享了一个让研发团队自身都深感震撼的突发优势（emergent capability）：利用 Atlas 实现极其逼真的“子弹时间”（bullet-time shots）。 - **极简硬件需求**：仅使用三部普通 iPhone 架在三脚架上，对同一动态事件进行多角度同步拍摄。 - **草莓滴落实验**：测试案例是一颗草莓掉入一碗燕麦奶的瞬间。当草莓触碰牛奶、飞溅起奶花的刹那，时间被瞬间定格（freeze frame）。 - **空间漫游穿梭**：Atlas 仅凭借这三个视角的输入，就完美重构了水滴飞溅的高动态瞬间，允许虚拟摄像机自由环绕奶花做 360 度运镜飞越。 Justin 强调，这种从极稀疏视图（三视角）恢复出高保真可导航冻结时间的能力完全超出了团队最初的预期。对于电影视觉特效（VFX）而言，即便最终交付的成片依然是 2D 画面，但在输入侧引入显式 3D 结构以及自由调整摄像机机位的能力，彻底重塑了影视特效画面的创作空间与自由度。 ### `20:00` 3D控制界面与原生摄像机控制 Justin Johnson 深入阐述了模型提供 3D 控制界面（control surfaces）对于人类导演的必要性。如果创作者只能依靠文字提示词输入“向内平移（pan in）”、“向外平移（pan out）”或“向左移动（truck left）”，是根本无法实现精准视线调度的。文本在表达三维空间中的细微动态轨迹时存在天然局限，创作者真正需要的是能够直接“抓住摄像机”，让它完全按照心意变焦和移动。因此，World Labs 的 Atlas 模型将摄像机控制设计为原生输入模态，使其成为一个真正支持三维摄像机运动的多模态模型。 ```mermaid flowchart LR accTitle: Atlas模型的原生摄像机控制机制 accDescr: 展示了文本提示与原生3D摄像机抓取控制两种方式的差异，以及Atlas如何将摄像机轨迹作为多模态输入 subgraph Traditional["传统文本提示方式"] A["文本指令 ('pan in', 'truck left')"] -->|"表达模糊、缺乏空间精度"| B["不可控的生成结果"] end subgraph AtlasApproach["Atlas 原生多模态方案"] C["创作者直接抓取摄像机"] -->|"连续、精确的3D运动轨迹"| D["Atlas 原生摄像机控制模态"] D --> E["高保真、完全可控的3D场景输出"] end ``` - Justin Johnson 指出：“Atlas has camera control as a native input type. It's multimodal and one of those modalities is cameras.” - 相比于二维画面后处理或抽象的文字提示，直接在三维视口中操纵摄像机让导演对生成镜头的掌控力达到了专业制作级标准。 ### `20:41` 物理边界与微观/宏观边缘案例：从黑洞到量子芯片针对模型在面对黑洞或纳米级微观物理等极端边缘场景时的表现，Justin Johnson 分析了物理模拟的不同范畴。他认为在宏观层面，大致属于牛顿力学（Newtonian physics）范畴的物理规律已经存在于预训练数据中，只要在此范围内表现不够理想，通常“只需要做一次快速的 SFT（监督微调）就能解决”。但对于非牛顿力学物理（如黑洞物理）或者微观粒子所处的量子力学领域，仅仅依靠现有管线是不够的，必须在底层架构和训练数据层面重新构思。谈及这些前沿探索，Justin 分享了团队创立初期的一个愿景与内部玩笑：利用三维世界模型直接辅助设计半导体芯片，进而用这些更强的芯片去训练下一代世界模型。 - Justin Johnson 解释物理机制分类时强调：“So anything that's like roughly Newtonian that doesn't work at is probably just a quick SFT away. If you want something that's sort of non-Newtonian, right? Like black hole physics or like, you know, microscopic stuff. Once you're down in the quantum mechanics realm, I think then you'd have to rethink a bit the architecture and the data side.” - 他谈到创办 World Labs 时的终极构想：“But one of our dreams we joked when starting the company is you want to build these 3D world models. And how cool would it be to use a 3D world model to design a chip that we then use to train the next generation world model?” ### `22:04` VR 交互界面与世界模型的未来对话转向了交互形式的终极演进。主持人和嘉宾探讨了键盘 WASD 与鼠标操作在漫游三维世界时的局限性，一致认为世界模型最自然的交互模态与 Prompt 提示手段最终必然是虚拟现实（VR）。Justin 透露，团队并不是把 VR 视作遥远的远景，World Labs 早在将近一年前就已经实现了在 VR 头显中运行模型（Marble 阶段）。当用户戴上头显，整个人直接置身于完全由 AI 实时生成的全景三维虚拟世界中时，那种沉浸感是“极具变革性（transformative）”的体验。节目最后，主持人们对 Atlas 正式向全社会开放后的创作生态表达了极高期待，并感谢 Justin Johnson 的深度技术分享。 - Justin Johnson 分享自己在头显中的亲身体验：“That is a really transformative experience if you guys haven't tried it, right?... headset and I'm in this complete AI generated world that I created. It's quite amazing.” - 空间交互的升级路径从传统的键盘鼠标（WASD + 鼠标转动）彻底跨越至原生头显沉浸式漫游与实时交互。 ## Notable quotes > `00:00` Our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds. — Justin Johnson > `16:04` We never wanted to be building these generative slot machines where you just like pull the thing and hope you get out a good generation. Or you don't feel like that generation was yours or that you controlled it or that you knew what you were getting out — Justin Johnson > `20:00` Atlas has camera control as a native input type. It's multimodal and one of those modalities is cameras. — Justin Johnson > `20:41` But one of our dreams we joked when starting the company is you want to build these 3D world models. And how cool would it be to use a 3D world model to design a chip that we then use to train the next generation world model? — Justin Johnson ## Mentioned Justin Johnson, Theo Jaffee, Sofia Puccini, World Labs, Atlas, Marble, OpenAI, Tyce, Sora, Sora 2, 3D Gaussian Splatting, Microsoft Flight Simulator, Google Maps, Grand Theft Auto (GTA), GTA 6, Disney, Pixar, Twitter (X), iPhone ## Video description World Labs co-founder Justin Johnson joins MTS hosts Theo Jaffee and Sophia Puccini to discuss Atlas, World Labs’ latest world model, and the broader case for AI systems that understand and interact with the physical world. Justin explains how Atlas approaches three core tasks: generating new worlds, reconstructing real environments from images, and simulating how objects or robots might behave within them. Underlying it is a bigger thesis: just as language models became general-purpose engines for working with text, world models could become a horizontal layer for visual and physical intelligence across industries from entertainment and gaming to construction and robotics. They also explore how world models could change video games and creative tools, why precise spatial control matters, and the potential for “real-to-sim-to-real” robotics, where a few photos of a physical environment could eventually be enough to build a simulation and adapt a robot to that specific space. [Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/e578e083-6b40-4815-a479-28213e86988a/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=e578e083-6b40-4815-a479-28213e86988a&feed=JGE3yC0V)

Source quotes

[0:00] Justin Johnson: Our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds. [16:04] Justin Johnson: We never wanted to be building these generative slot machines where you just like pull the thing and hope you get out a good generation. Or you don't feel like that generation was yours or that you controlled it or that you knew what you were getting out [20:00] Justin Johnson: Atlas has camera control as a native input type. It's multimodal and one of those modalities is cameras. [20:41] Justin Johnson: But one of our dreams we joked when starting the company is you want to build these 3D world models. And how cool would it be to use a 3D world model to design a chip that we then use to train the next generation world model?

Source quotes

[0:00] Justin Johnson: our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds.

Original transcript

[0:00] Language models are these general horizontal engines for processing streams of discrete text, discrete tokens. Those have had tons of applications from, you know, everything that we know and love today. And our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding, that can be used to generate, simulate, reconstruct worlds. And if we can build these with the right generality, they should be applicable to tons of different industries, right? From entertainment to VR to construction to robotics. We live our lives in this physical built space all around us and we need models to help us with these things as well. Language models gave AI a way to work with words. What happens when models can understand and simulate the physical world? World Labs co-founder Justin Johnson joins Theo Jaffe and Sofia Puccini on MTS to discuss Atlas and the broader idea behind world models. They explore how models could generate and reconstruct environments, what separates them from video generation, and applications across gaming, VFX, and robotics, including turning a few photos of a real space into a simulation for training a robot. We're live with Justin Johnson, who is a co-founder of World Labs, which is a startup that builds spatial intelligence products And just today they announced Atlas the world first multimodal world model that generates image and video frames with pixel camera control and reconstructs them in 3D Model the world, move the camera, and simulate space and time. So congrats on the launch. The launch video looks really cool. Welcome to MTS. Yeah, thanks for having me. Absolutely. So tell us a little bit more about what this product is specifically, what you can do with it, what people will use it for. Yeah, so this is not a product launch. This is a model announcement. So the products are coming later. This is the base model. It's going to be used to power our future products from World Labs. But at its heart, it's a world model. It does basically three different kinds of things. Generation, reconstruction, and simulation. Generation is you want to generate new worlds that don't exist. I can start from a text prompt. I can start from an image prompt. Then I can direct the camera and sort of fly through that world and whatever. Have the model generate a world that never existed before. So that obviously has tons of applications across many things in creativity. right, from VFX to gaming to all the kinds of use cases that people are using there. The second major capability of Atlas is reconstruction, right? There sometimes I don't want to generate a new world. Sometimes I have an existing space in the real world that I'd like to model and bring into virtual reality or the virtual world in some way So there what Atlas can do is sparse reconstruction I can take just a few photos of a space as few as one up to 100 or more And the more photos you give it the more you can reconstruct that existing real world space accurately. And the third thing we can do is simulation, right? And this is going after more robotics. Yeah. Well, both VFX and robotics. But there, we want to be able to take an existing space, recreate that space in 3D, then simulate how a robot might behave in that space. And we show some examples of all these different use cases on the blog. Yeah. Is the ultimate vision for WorldLab something more like environment for teaching robotics how to work? Or is it more like a metaverse that people will like spend time in? All of the above, probably. Or is it just kind of all of the above? Yeah, I think it's all of the above. The central thesis here is that there's a new, there's a different kind of horizontal model that we can build called a world model. Right? Like language models are these general horizontal engines for processing streams of discrete tax discrete tokens. Those have had tons of applications from, you know, everything that we know and love today. And our thesis is that there exists another category of model called world models that should be based in visual understanding, should be based in physical understanding that can be used to generate, simulate, reconstruct worlds. And if we can build these with the right generality, they should be applicable to tons of different industries, right? From entertainment to VR to construction to robotics You know we live our lives in this physical built space all around us and we need models to help us with these things as well Right So the model is the input that would go to all those industries like the input intelligence I curious like how this has evolved and why Atlas is a big jump here. So I would imagine, like, one of the difficulties with procedurally generating worlds is that, like, after a certain point, like, I feel like, you know, with video models, like 10 seconds in, I feel like the output could start to get jumbled, especially if you have like a reference image. And then it starts to take its own direction. It starts to like hallucinate in its own way. So how do you fix this thing of like at a certain horizon, things would start to get jumped or distorted? Yeah, that's a great question. So here we built Atlas specifically around spatial control and the notion of a spatial context. So when you give Atlas reference images, they aren't just images. They're actually grounded in 3D space. So you can stage those images in 3D and say this reference image should be here. This reference image should be here. This reference image should be here. And when you generate a video from Atlas, you're not just giving it a text prompt. You also go in and steer the camera and say, I want the camera to fly this way, this way, this way, this way with exact precise pixel control. So then we show some examples where even up to a minute of video, we give it more reference images and sort of leave breadcrumbs of those images along the trail in 3D. And then you get that direct tutorial control. [5:00] over the video that you're generating. So the idea here is that even as you go to really long generations, it's not a slot machine. You're generating and directing and really controlling what the model outputs. Yeah. So... Yeah, we were just talking right before we went on about... Our friend Tyce from OpenAI. did this project that went kind of viral that was like gta but it's san francisco so it like i think it was like a giant gaussian splat of the whole city or something and you can like walk around in it you can steal cars and drive around you can fly around it's not very high resolution definitely not as much as atlas you think Ultimately, this is something you'll be able to do with Atlas? Yeah, I think so. And I think there's actually two different futures here of like the tech path that gets us there. One is sort of the explicit 3D route that you mentioned, right? Like the idea of generating a Gaussian splat reconstruction, then importing that explicit asset into a game engine or something else. This is something Atlas can do, right? Like it can be used to generate, you know, 3D Gaussian splat scenes. And then you could import them and generate and build games around them. Another direction where this could go is, you know, you skip the engine entirely and go directly for frame generation, right? outputting the frames without any explicit 3D in the mix anywhere. And Atlas is built to do this as well. So it's one model that both handles, you know, 2D when you need it, 3D when you need it, and can handle both modalities in a native way. If you're somebody using this world model and you have all these inputs, how would you... How would you steer Atlas to do what you intended? Like if you're like a film producer or video game developer, how would you work with it? Yeah, we have some internal tooling for this, but it's going to take a little bit more iteration on the product side to figure out the right way to expose that. But the internal tooling is actually pretty awesome. You can basically take images, drag them into this interface, have this sort of spatial 3D canvas for all your input images. Then they end up with this proxy geometry in 3D space, and you can steer your camera kind of with an FPS type control to decide exactly where you want this thing to go. And that's a pretty complicated user experience or user interface. to expose these capabilities in nice ways. How does it differ from Marble, which is WorldLab's first product? Yeah, so it's a totally rebuilt stack of models, right? I think the central thesis is the same. We want to build multimodal models that can input many different modalities, output many modalities. This is something that Marble did as well. But one way in which they differ is that Marble really put Gaussian splats really at the center of many things that it did. So Marble could output videos, Marble could output 3D scenes, Marble could output meshes. But everything was kind of bottlenecking through a 3D Gaussian splat representation. you were always generating a Gaussian scene, then rendering those Gaussians back to images or videos. And this is really nice, but we wanted to break that dependency. So Atlas was built fully from the ground up to both rethink the architecture, to make it more unified, more easy to scale, and to have that branch between 2D and 3D happen earlier. So that when we're generating 2D outputs or video outputs from Atlas, you're not bottlenecking through a Gaussian splat scene anymore. The model is directly giving you those beautiful 2D pixels. up to 3D only for the applications that you actually know that you need 3D. So I guess at this stage, are we outgrowing Gaussians as a technology? Or what is their role within all of this? I think Gaussians are a tool just like anything else. And you got to use the right tool for the job, right? If you're working on something like VFX, then the ultimate output is a 2D image or a 2D image or 2D video. And maybe that's all you need. But for other applications, and other explicit 3D representations still have their place. I think there might be a world in the future where we're all streaming pixels directly off these models in real time from powerful server farms, but we're not quite there yet. And even if we are, it's going to be pretty expensive. So something like a Gaussian or a mesh is really useful for client-side rendering, especially on mobile or VR devices, and also really useful for integrating with existing workflows and pipelines. I think we all believe in the future of technology, and we're excited about building these solutions that give us whole new experiences. than these, it takes a lot longer for these things to permeate than we expect, right? So people in gaming and VFX and design and architecture, they have their existing workflows that work on 3D representations. And, you know, instead of trying to switch them over all at once to a fully AI-native thing, if you can get with AI into their workflows and meet them where they are, then that's another great use of splats and other explicit 3D representations. Got it. Yeah, one of my favorite video games ever is Microsoft Flight Simulator, When you fly around the world and it uses like a lot of not quite real time, but it uses satellite data. in order to reconstruct a 3D model of the world, basically, in reasonably high fidelity, not incredibly high, reasonably high, do you think Alice will eventually be able to What will it take to get Atlas from where it is now to able to dynamically generate [10:00] content quickly enough to... do what Flight Simulator does? And like, is this even like the right tech tree for that? Imagine like Google Maps in this world, like you could just traverse like any space, basically. Is it kind of overkill though? I mean, I think it'll happen sooner than you expect, right? And that'll happen in one of two ways. Like there's one sort of, and actually, you know, Atlas was built to handle both of those tech trees. One is the explicit 3D route, right? Like maybe we're generating a Gaussian scene or a 3D point cloud scene and then interpolating that on the client. That's something Atlas can do streaming the frames. Right. And that's something that I think we'll be able to do pretty soon. Okay. I'm also really curious about reel-to-sim for robotics. So, you know, taking like a reel video and then building a simulation. So how is this useful and how do you see it? this possibly being like helping evolve the field. Yeah, so I think there's a couple different ways that this is useful. One is for evaluation, right? Like you definitely want to evaluate robotics in the final situation, in the final environment where they're actually going to end up, right? If you're going to have a robot working in a particular factory environment with a particular type of machine in a particular environment, like you want to make sure the robot works exactly in that intended environment. matches the final use case as accurately as possible. Then even if you've got your big pre-trained robotics foundation model that knows about a lot of robotics actions in general, then you could either fine tune that thing directly in that final simulation that matches the application or even do evaluations to make sure that it's going to work. So that gives you a different avenue than you could just collect a ton of demonstrations of a robot either working in that space or a person teleoperating the robot in that space. But simulation gives you a different angle to simulate those environments. Yeah, interesting. So is there a world where you have your general pre-trained robot, but then you always need a realtism of a specific robot? environment? And could that get us basically to like a working like general robot faster by scaling this? Maybe. I think there's a couple different tech trees that people are working on. One is this notion of fully in context learning, right? Like maybe I've got a robotics foundation model, then I can kind of demonstrate the robot once how a task should be performed. And that's enough for the robot to figure out, like stick that demonstration in the context. And now the robot can figure out that task. Another version is more this, you know, what we're calling real to sim to real. There, maybe I've got my pre-trained robotics foundation model, but I want to adapt it to this particular environment, right? Like this studio, like this table, like moving this microphone around. Yes. Then there's a version of that where you could come in here and take just a couple of casual videos with your phone or a couple of images with your phone, use Atlas to reconstruct the space, and now stage all kinds of robotics interactions in this studio space in particular. list, have a general simulation, maybe describe in natural language what you think, what kind of task you want the robot to do, have an agent go build, you know, build that simulation for you, fine tune your thing, like RL, fine tune your general purpose robotics foundation model. And now you've got a robot maybe in the span of a couple minutes even that could come in and is perfectly adapted to this space. Yeah, like a very fast onboarding. Exactly. Yeah, that'd be great. I'm kind of curious about all the way back in 2023, 2024, when the original version of Sora came out, opening eyes paper on it was called like video generation models as world simulators, which is seems like very similar to the vision of world labs. And then, you know, they did Sora as that. They did Sora 2 basically as like a TikTok clone that didn't seem that interested in the fundamental research direction of video models as world simulators. It seemed like a consumer product instead. And then they dropped Sora. And now they're not working on world models at all. So why do you think they decided to drop it? Well, I don't want to comment on OpenAI in particular. I don't really know much about their internal strategy or discussions. But our thesis is that world models are this general horizontal platform that can be used to go after a lot of different applications. And there's going to be a ton of value unlocked there. I think there is maybe a sense of a bit of innovators dilemma, right? Like you're sitting on LLMs and you've got this as a really powerful thing and you're leading the pack there. It doesn't necessarily make sense to try to invest in that next thing. But, you know, for us, we think that there is another category of models out here. There's a ton of opportunities out here and we want to be the ones to go and grab them. Yeah. Do you think that there's like I'm curious about the mass consumer applications of world models are the first ones. I feel like VR or AR feels natural and then video games feels very natural. I feel like we could start shifting in a direction where everyone's vibe coding video games and world models could be a part of that. Whereas right now we're just vibe coding websites. We're already kind of doing that, right? Yeah, I think we're moving towards that. Do you see world models being that big? [15:00] rapidly deployed. so that everyone can kind of play around with them. Yeah, definitely. And that's what we want to do. So we want to get this... That's kind of like fits into our thesis about these models as horizontal general things, right? So if we want to expose these models to a lot of people, let them be consumable by humans and agents alike, and let them interoperate with other tools. Yeah. So if we can provide a model that has all these general capabilities in simulation, in reconstruction, in generation, in explicit 3D, in 2D, and then we can have a coding agent go and stitch all these capabilities together, then we can let people really quickly iterate on really cool experiences. So I'm really excited to see once we get this into people's hands, what kinds of cool experiences people will build with it. Right. Yeah, I'm also curious. how you think they'll naturally work with LLMs. So are we all going to basically be like adding another subscription to our AI stack? And do you think agents are very easily able to like, work with world models in theory? Yeah, definitely. I don't see why not, right? Like, if you want, once you offer something on an API, like, you know, you get certain things in certain They interop with other pieces of software. I think we've always wanted to build these things for both humans and agents to consume. I do wonder if GTA 6 is just the last great piece of human written software. before all future video games are... kind of, uh, generated on the fly with stuff like Alice. I don't know. I disagree with that. I think there will always be, like, human in the loop with... Because video games are an art. I'm not saying there won't be humans in the loop, but I'm saying... But, like, fully hand-stitched, like, video game? Yeah, like, you know, you won't need to, like... you know, physically place like millions of different assets in a game engine or anything like yeah but this is just the evolution of technology in general right as we see like you know from the early, like some of the early Disney animation films where they go and they hand paint every cell, like that was already a new type of technology that enabled new, a new way to make movies and creative experiences at that time. Right. Or some of the early Disney films figured out how to sync audio and video for the first time. And that was a really transformative experience. But, you know, new technologies come out and creative people always find really interesting and innovative ways to enable new experiences. So moving from, you know, live, like live action, generated films like the Pixar era of CG into AI. I view these as just new creative tools, and they'll always be people like creative people behind the wheel that will be interested in creating new experiences. Yeah, I can imagine like a Pixar, but in 3D, like in like a 3D navigable space sort of thing emerging. Like I think this is going to be really good for creative specifically. Yeah, I think so. Right. And what we've wanted to build on all these things is this aspect of creative control. Right. We never wanted to be to be building these generative slot machines where you just like pull the thing and hope you get out a good generation. Or you don't feel like that generation was yours or that you controlled it or that you knew what you were getting out. Like we want to build these tools that have really deep control. right? Like whether it's 3D control, whether it's text control, that you feel like you're a director in the thing guiding this model and deciding exactly what it outputs rather than just like the model spits something out and maybe I can stitch it together after the fact. Yeah, totally. How does Atlas perform with difficult physics stuff? For example, if I wanted to do a simulation of a black hole or something, would it be able to say, "Oh, this is a scenario where normal object collisions don't really apply. You'll need to do spaghettification or whatever." Yeah, black hole is a good one. I haven't tried that. Maybe I'll try it this afternoon and tweet it. But I should do that. You should. You should. I'm actually really curious to hear. Yeah, my guess is it probably won't do something super exciting for black holes. We'll see. We'll see. Or what about just like hair or something? Hair is pretty good. I think one of the most amazing applications we got out of this thing that was actually a bit of a surprise to us how well it worked were these bullet time shots. So we can do this thing where you can take a couple of regular iPhones and stick them on tripods, just like three iPhones. Take synchronized capture of some event happening. And then you can use Atlas to do these bullet time reframing shots, like freeze frame fly around. And this worked better than any of us were expecting. It's completely insane. our headline or examples there is a strawberry dropping into a bowl of oat milk. And then we can like, as soon as the strawberry hits the milk, the milk splashes up, we freeze time and like go in there and fly all around it. Like it's just incredible. This works from just three input views. Yeah, I'm also curious about that added capability of being able to adjust where the camera is. What part does that help with? I can imagine that helps if you're trying to make a movie or something. But yeah, how is that a useful capability? value of 3D on the input side as well. Right? Like even in a situation like a VFX shot or something like that, where the ultimate output is 2D. [20:00] for the model to offer these 3D control surfaces to let the human director figure out what kind of scene the model should output. In this case, we can do this There you can control precisely where does the camera move. Because I don't just want to write in text like pan in, pan out, truck left. I want to be able to grab the camera and make it zoom exactly as I like. And there's really no way to describe that explicitly in text. You really want to be able to just grab the camera and make it do exactly what you wanted. So Atlas has camera control as a native input type. It's multimodal and one of those modalities is cameras. Yeah. So if you think it wouldn't do a black hole correctly now, like what more would be required in order to get it to handle all kinds of edge cases in physics? Oh, I mean, so I think there's a couple of different regimes of physics. So like anything that's like roughly Newtonian, like we, that was in the pre-training data. So anything that's like roughly Newtonian that doesn't work at is probably just a quick SFT away. If you want something that's sort of non-Newtonian, right? Like black hole physics or like, you know, microscopic stuff. Once you're down in the quantum mechanics realm, I think then you'd have to rethink a bit the architecture and the data side. But the black hole example is really interesting. I'll have to give this a try. Yeah, yeah. I would definitely be very curious to see. Or like at a nanoscale. Yeah, that's probably not going to work. Yeah. Not yet, not yet. But one of our dreams we joked when starting the company is you want to build these 3D world models. And how cool would it be to use a 3D world model to design a chip that we then use to train the next generation world model? thinking that. Yeah, yeah, yeah. Because we're talking about quantum stuff. You have your like semiconductor. Also, to me, what feels very natural is that like the natural modality for navigating these and even prompting the world models is just going to be VR, basically. Yeah. Because you can like, I guess, do WASD and then like move your mouse around to traverse a world. But I think eventually you will have to basically like... Well, not just eventually. We've had Marvel in VR for like almost a year at this point. That is a really transformative experience if you guys haven't tried it, right? I haven't tried it. headset and I'm in this complete AI generated world that I created. It's quite amazing. Yes. Yeah. Yeah, absolutely. Yeah, I can't wait to see all of the amazing stuff that people are going to do with Atlas once it's fully out into the world. Yeah. Yeah. Justin, thanks so much for coming on DS. Yeah, thanks for having me. Absolutely. This was awesome. This was really awesome, yeah. and Spotify. Follow us on X at A16Z and subscribe to our sub stack at A16Z.substack.com. Thanks again for listening and I'll see you in the next episode. This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product. This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with A16Z. Such advertisements, Management LLC, A16Z, or any of its affiliates. Information is from sources deemed reliable on the date of publication, but A16Z does not guarantee its accuracy. you