Reactor CEO Says Existing AI Infrastructure Fails Real-Time Robotics

@Scale

Stateless generative AI infrastructure cannot support physical automation because latency lapses cause collisions, requiring stateful video models grounded in robotic force telemetry.

AI-assisted summary of World Models for Robotics from Reactor | Alberto Taiuti from Reactor — original source. Check the original for full context.

Source published: 2026-10-30

Key ideas

World models mark a paradigm shift from passive, asynchronous AI (such as LLMs and offline diffusion models) to real-time, interactive, action-conditioned state machines running at 60 FPS. (3:33)
While internet-scale video provides rich visual common sense and kinematics, embodiment telemetry from physical robots—such as force, torque, and tactile feedback—is essential to ground simulations in physical reality. (6:22)
Deploying real-time world models requires entirely re-architecting AI infrastructure, shifting priorities from raw throughput to ultra-low latency, model distillation, and high-performance network transport layers. (10:21)
Interactive neural generation threatens traditional multi-million-dollar 3D asset pipelines, enabling applications ranging from instant game environments to generative, pixel-by-pixel user interfaces. (15:13)

Detailed notes

## Background This talk was published on YouTube by the @Scale channel on October 30, 2026. Titled 'World Models for Robotics from Reactor | Alberto Taiuti from Reactor', the video did not include any accompanying show notes or description.

## TL;DR Alberto Taiuti, CEO and co-founder of Reactor, explores how real-time interactive world models represent the next transformative frontier beyond passive, asynchronous generative AI. He defines world models as action-conditioned state machines that require strict causality, low latency, persistence, and physical accuracy, combining internet video common sense with embodied robotic telemetry. Taiuti details the full-stack infrastructure overhaul necessary to run distilled, stateful neural simulations across optimized transport pipelines at 60 frames per second. Through live interactive demos, he illustrates how neural rendering will replace multi-million-dollar traditional 3D graphics pipelines and pave the way for generative, pixel-by-pixel software applications.

## Atmosphere & style The presentation carries an energized, visionary, and technically grounded tone, moving seamlessly from Carl Sagan philosophical reflections to hardcore distributed systems engineering. Taiuti speaks with casual authority and infectious enthusiasm, leveraging his pedigree across Apple Vision Pro, Skydio, and Luma AI to demystify bleeding-edge concepts. The live interactive demos—such as maneuvering a character or a dog through photorealistic neural scenes in real time—provide compelling visual proof, giving the session the palpable thrill of watching the future of spatial computing unfold live on screen.

## Bulletin - [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) World models mark a paradigm shift from passive, asynchronous AI (such as LLMs and offline diffusion models) to real-time, interactive, action-conditioned state machines running at 60 FPS. - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) While internet-scale video provides rich visual common sense and kinematics, embodiment telemetry from physical robots—such as force, torque, and tactile feedback—is essential to ground simulations in physical reality. - [10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) Deploying real-time world models requires entirely re-architecting AI infrastructure, shifting priorities from raw throughput to ultra-low latency, model distillation, and high-performance network transport layers. - [15:13](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=913s) Interactive neural generation threatens traditional multi-million-dollar 3D asset pipelines, enabling applications ranging from instant game environments to generative, pixel-by-pixel user interfaces.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=0s) World models represent the future of computing and inference, transitioning artificial intelligence into reactive, real-time systems that respond immediately to user and agent actions. - [00:00](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=0s) Interactive simulations can be rendered on the fly at 60 FPS from a single static seed image without relying on pre-baked 3D geometric assets or standard game engines. - [02:28](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=148s) Reactor was founded by Alberto Taiuti (former co-founder/CTO of Luma AI and Vision Pro contributor) to democratize and scale infrastructure specifically for interactive world models. - [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) Current mainstream AI architectures remain inherently passive, operating asynchronously over high-latency request-response cycles rather than supporting tight interactive feedback loops. - [05:16](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=316s) At a fundamental engineering level, a world model is an action-conditioned state machine that updates an internal environment state and continuously streams resulting observations. - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) Internet video provides an observational foundation of visual causality and physics, but lacks the tactile and kinetic telemetry required for mechanical authenticity. - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) A mutually reinforcing flywheel exists between robotics and world models: robot fleets capture mechanical ground truth data that turns visually plausible simulations into physically accurate engines. - [08:38](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=518s) A true world model must fulfill five fundamental properties: causality, interactivity, real-time latency, environmental persistence, and physical accuracy. - [08:38](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=518s) Existing commercial applications already leverage world models, including Waymo running edge-case autonomous driving evaluations and embodied AI training via world-action models. - [10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) Legacy AI serving stacks fail world models because language models and image generators prioritize batch throughput over the ultra-low latency required for real-time closed-loop robotic control. - [12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) Achieving real-time stateful inference requires distilling large video backbones for causal, autoregressive execution and maximizing utilization of specialized silicon. - [12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) Low-latency network transport engineering and optimized compression codecs are just as critical as machine learning weights for delivering interactive streaming world models. - [15:13](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=913s) Prompt-driven neural simulation collapses multi-month, multi-million-dollar 3D development cycles into near-instantaneous environment generation. - [17:03](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=1023s) Upcoming iterations of world models will feature deep persistence, enabling users to serialize, store, restore, and collaboratively share virtual dynamic worlds. - [17:03](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=1023s) World models will redefine software engineering through generative interfaces where entire graphical user interfaces are rendered pixel by pixel dynamically from code scripts rather than pre-compiled UI widgets.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=0s) Introduction to World Models and Live Demo

Alberto Taiuti opens his talk expressing immense excitement for a rapidly developing piece of AI technology: world models. In his view, world models represent the future of computing, inference, and artificial intelligence as a whole. Before diving into the technical mechanics, Taiuti grounds his talk with an inspiring perspective from his personal hero, astronomer Carl Sagan, using the famous aphorism: “Carl Sagan, who said that if you wish to make an apple pie from scratch, you must first invent the universe.” He reflects that while it is simple and conventional to get bogged down in the minute, day-to-day details of engineering, building transformative technology requires stepping back to understand the grand scheme of why the work matters on a global scale.

To demonstrate what a world model actually does, Taiuti launches straight into an interactive, real-time live demo. Stepping inside a fully rendered virtual environment initialized from a single still image, he controls an animated character named Jumplo. Every frame appears dynamically on screen without being stored as pre-baked geometric assets, textures, or standard game engine data. Instead, the visual scene is computed on the fly by an AI video model generating 60 frames per second on Reactor's backend servers and streaming the pixel feeds directly to the display. Taiuti demonstrates spatial consistency by pivoting the camera fully around, illustrating how objects maintain their relative positions and do not vanish or drift. He triggers discrete interactive events inside the environment, such as shifting the ambient lighting across the base, all updating in real time. He emphasizes that producing continuous, high-framerate interactive simulations at scale historically demanded months of intensive human engineering and astronomical capital expenses.

- The demo environment is generated frame-by-frame on remote servers and streamed live at 60 FPS without pre-existing 3D assets. - Initialized from a static seed image, the model maintains spatial consistency as the user pans and navigates the camera. - The environment handles interactive state shifts, such as altering dynamic lighting effects across the virtual station.

---

[02:28](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=148s) Alberto Taiuti’s Background and the Founding of Reactor

Taiuti introduces himself as the CEO and co-founder of Reactor, an infrastructure platform engineered to democratize access to world models and run them efficiently at web scale. His passion for simulated 3D spaces has driven his trajectory across the frontier of spatial computing and computer vision. Prior to founding Reactor, Taiuti served as the CTO and co-founder of Luma AI, where he spearheaded deep learning infrastructure for scalable 3D reconstruction and generative video.

His background also includes engineering work at Apple, where he contributed to the Vision Pro headset during its early phases as a stealth project, an experience that deepened his fascination with simulated virtual realities. Additionally, he worked as a real-time AI systems engineer at autonomous drone manufacturer Skydio. Building real-time flight stacks for drones brought home the realization that physical-world robotics cannot scale without rich, responsive, and physically grounded simulations.

- **Current Role**: CEO and co-founder of Reactor, building foundational infrastructure to run interactive world models at scale. - **Luma AI**: Former CTO and co-founder, focusing on generative video models and 3D reconstruction pipelines. - **Apple**: Core contributor to the Apple Vision Pro spatial computing platform while under confidential development. - **Skydio**: Real-time engineer on embedded autonomy and drone AI stacks.

---

[03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) From Passive AI to Real-Time Interactive Systems

Surveying the broader trajectory of artificial intelligence, Taiuti argues that existing mainstream architectures are fundamentally limited by their passive nature. Today, AI is dominated by Large Language Models (LLMs) alongside diffusion-based image and offline video generators. While these models represent impressive milestones—enabling code generation, text dialogues, and creative asset pipelines—they operate asynchronously. A user submits a prompt, waits seconds or minutes for inference to complete, and receives a static block of text, an image, or an exported clip.

Taiuti asserts that the next era of computing will be defined by reactive systems where both logic and the rendering of pixels occur interactively at the edge or via high-speed streaming. He explains: “Whereas in the future, more and more systems and AI systems will be reactive. They will be real time. They will be systems that you can poke at them and they poke back at you.” Tracking the historical trend lines of AI modalities reveals a clear trajectory moving from low-bandwidth to high-bandwidth interfaces: starting from text-to-speech and audio, advancing through static images, progressing to offline video synthesis, and ultimately arriving at real-time, interactive video engines that react instant-by-instant to human or agent input.

```mermaid flowchart LR accTitle: Evolution of AI Modalities accDescr: Progression from passive text generation to interactive real-time video simulation.

subgraph Passive["Passive & Asynchronous"] A["Text & Audio"] --> B["Static Image"] B --> C["Offline Video"] end

subgraph Active["Reactive & Real-Time"] C --> D["Interactive Video"] D --> E["World Models"] end ```

---

[05:16](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=316s) Defining the World Model: The Core State Machine

Taiuti strips away semantic hype to define precisely what constitutes a world model from an engineering standpoint. At its structural core, a world model is an interactive system that maintains an internal representation of the state of its environment. When it receives an external intervention or control signal—an action—it updates that internal state, produces the appropriate observation, and yields the subsequent state, repeating this transition continuously inside a tight feedback loop. Taiuti puts it plainly: “And so at the mo at the most basic fundamental aspect of things, a world model is a state machine.”

Because real-time interactive systems must account for everything that has previously unfolded, an internal state is mandatory. It cannot evaluate queries in a vacuum like an isolated stateless transformer. Although Taiuti notes that high-performance world models encompass far more complexity than a traditional deterministic automaton, viewing the model fundamentally as an action-conditioned state transition machine clarifies how passive generative models transform into responsive simulators.

```mermaid flowchart TD accTitle: World Model Loop accDescr: The state machine feedback loop showing actions updating internal state and emitting observations.

subgraph StateLoop["Continuous Simulation Loop"] S1["Current State (S_t)"] --> Trans["World Model Transition"] Act["Input Action (A_t)"] --> Trans Trans --> S2["Next State (S_t+1)"] Trans --> Obs["Streamed Observation / Pixels (O_t+1)"] S2 -.->|"Feedback"| S1 end ```

---

[06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) Data Sourcing: Internet Video and the Robotics Flywheel

Building rich world models requires massive datasets to teach models how reality functions. Taiuti points out that raw internet video offers a tremendous repository of visual common sense. By watching vast numbers of video clips, neural backbones naturally absorb causal relationships, basic kinematics, fluid motion, lighting cues, and everyday physical events—such as people walking or rain falling to the ground. Leveraging internet-scale video datasets allows researchers to build visual simulators purely through observational learning.

However, internet video has a fatal blind spot: it records appearance rather than mechanical reality. Pure pixel feeds lack measurements of contact dynamics, friction coefficients, joint torques, and physical forces. This is where the convergence between world models and modern robotics becomes critical. As fleets of physical robots are deployed into real-world environments, their onboard sensor suites capture the precise physical ground truths that video platforms lack: force-torque measurements, tactile contact feedback, and inertia.

Taiuti describes a mutually reinforcing flywheel where robotics and generative video backbones lift one another: “And this can then be incorporated in world models to bridge the gap between a simulation that looks correct to one that actually is.” Web video provides the visual and semantic backbone, while embodiment data converts visual approximations into physically accurate engines.

- **Video Modality Strengths**: Internet video contains billions of natural demonstrations of causality, geometry, motion, and optical properties. - **Video Modality Weaknesses**: Fails to capture non-visual physical interactions, including surface friction, inertia, impact forces, and internal dynamics. - **The Robotics Flywheel**: Deployed physical robots gather continuous sensory and force data; integrating this telemetry into video-conditioned world models turns visual hallucinations into physically authentic simulations.

---

[08:38](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=518s) The Five Core Properties and Commercial Applications

Taiuti outlines the five essential properties that distinguish a true world model from an ordinary generative pipeline:

1. **Causal**: It respects temporal ordering and conditions future frames on the chronological history of prior events. 2. **Interactive**: It takes incoming control signals and responds immediately without human-perceptible latency. 3. **Real-Time**: It computes states rapidly enough to support closed-loop control tasks, such as preventing a walking robot from losing balance and falling. 4. **Persistent**: It stores persistent representations of the environment, ensuring objects and spaces remain stable when revisited or panned away from. 5. **Physically Accurate**: It adheres to the underlying mechanics of mass, gravity, and contact, bridging the gap between visual fidelity and real-world dynamics.

Taiuti emphasizes that these concepts have already advanced beyond academic theoretical research into critical industrial infrastructure. In robotics and embodied AI, world-action models (WAMs) enable autonomous agents to hallucinate consequences and evaluate candidate policies prior to real-world execution. In gaming and interactive media, real-time pixel generation is spawning dynamic entertainment formats and intelligent avatars serving as responsive digital tutors. Finally, in autonomous mobility, companies such as Waymo routinely run autonomy stacks inside scalable world simulations, evaluating edge-case driving policies safely and cost-effectively before running vehicles on public roads.

[10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) Why Current AI Infrastructure Fails World Models

World models are no longer purely theoretical concepts; they have already moved into production environments today. Alberto Taiuti highlights that one of the areas he is most energized by is generative user interfaces driven by world models, where every single pixel on screen can be synthesized dynamically in real time according to user intent. He points out that prominent tech figures share this excitement, noting that Elon Musk observed that the future of AI will largely center on video understanding and generation. Because video captures such a rich, dense substrate of information about the physical universe, it acts as an ideal medium through which models can learn foundational world dynamics, unlocking entirely new interactive realms and computational systems.

However, Taiuti warns that the infrastructure built for earlier waves of generative AI is fundamentally inadequate for powering world models. The systems developed to date are inherently slow, passive, and non-interactive. Conventional language models generate discrete tokens without continuous streaming, and image generation models are completely stateless—they generate an image without needing to track or incorporate historical context. By contrast, a world model cannot function under such constraints because it must be strictly causal, continuously streaming, and state-aware.

Latency tolerances highlight this structural mismatch. When an LLM takes a full second to return an answer, that delay might be acceptable for a text chat interface, but as Taiuti emphasizes:

> “So an LLM that gives you an answer in a second is not good enough if you're if you're using it to pilot a robot, for example. it needs to respond immediately otherwise the robot may hit a wall.”

To prepare for an AI paradigm where world models occupy center stage, engineering efforts must decisively shift away from legacy, high-latency, stateless assumptions and prioritize continuous, immediate generation.

[12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) Full-Stack Engineering for Real-Time, Stateful Inference

Operating in true real time imposes an unforgiving computational budget: every single frame or actuation step must be resolved almost instantaneously, whether the target is rendering directly to a client monitor or actuating a physical robotic joint. Achieving this performance requires compressing massive video foundation backbones through model distillation so they run autoregressively and causally across substantially fewer inference steps. Taiuti emphasizes that engineers must deeply understand the underlying hardware architectures; failing to squeeze optimal performance out of specialized silicon prevents these technologies from ever becoming scalable or democratized.

Because a world model is explicitly stateful, every link in the delivery pipeline—from server-side frame generation in the cloud down to client-side reception—must be completely re-architected. Infrastructure teams must systematically prioritize minimal latency over raw throughput, and value rock-solid reliability and safety over mass, batched asset generation. Everything matters: the physical geographic proximity of server clusters, the choice of video resolution, video compression codecs, and the network transport protocols. Taiuti points out that network engineering is just as critical as machine learning research:

> “And also the transport layer that you choose is crucial to get right because otherwise you can have the fastest best word model in the world and it's still going to be useless if you cannot move the bites that it generates across the internet and across clients quickly enough.”

```mermaid flowchart LR accTitle: Real-Time World Model End-to-End Pipeline accDescr: Pipeline showing inputs feeding an autoregressive backbone down to low-latency transport and client execution.

subgraph Server["Cloud / Inference Engine"] A["Context & Inputs"] --> B["Distilled Backbone (Causal/Autoregressive)"] B --> C["Real-Time Frame / Action Generation"] end

subgraph Delivery["Low-Latency Network Stack"] C --> D["Video Compression & Codecs"] D --> E["Optimized Transport Layer"] end

subgraph Client["Edge & Application Layer"] E --> F["Client Screen (60 FPS Stream)"] E --> G["Robot Actuation / Policy"] end ```

Beyond raw transport, democratizing world models requires creating standardized, developer-friendly interfaces. Taiuti envisions an ecosystem where anyone equipped with an LLM and an automated coding agent can instantiate a world model to spin up custom simulations, robotics applications, or dynamic video games. Achieving this vision demands built-in primitives for persistent sessions, state tracking, hyper-optimized inference engines, and comprehensive observability and security guardrails to monitor high-frequency autonomous systems safely.

[15:13](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=913s) The Reactor Vision and the Real-Time Dog Demo

Reactor is being built specifically to serve as the dedicated platform for real-time world models, meeting the architectural demands of continuous, stateful execution. Urging the audience to explore existing models firsthand, Taiuti remarks that the capabilities unlocked across the open ecosystem over the preceding twelve months were previously unthinkable, setting up an even faster pace of progress over the coming year.

Drawing on his professional background in 3D computer graphics, Taiuti expresses awe at what neural generation can accomplish from minimal prompts. To illustrate the shift away from manual 3D production pipelines, he presents a live interactive demo where he controls a dog navigating a photorealistic park:

> “And generating a world like this before would have taken months and millions of dollars, whereas now you can just prompt it from an image and you're ready to go.”

In the demo, Taiuti steers the dog to walk forward, explore the environment, and leap on command. The entire scene streams interactively at 60 frames per second straight from Reactor’s cloud servers to the display. He stresses that scenes do not need to feature humans or fantastical aesthetics to demonstrate technical power; rendering an everyday, naturalistic scenario interactively with dynamic physics and camera responsiveness illustrates how neural rendering can supplant conventional game engines and asset suites.

- **Streaming performance:** Generated live and streamed at a steady 60 frames per second directly from remote servers. - **Cost and labor disruption:** Tasks that formerly demanded production budgets running into millions of dollars and multi-month studio timelines can now be instantiated instantly from a single image and a text prompt.

[17:03](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=1023s) Persistent Worlds and Pixel-by-Pixel Generative Software

Looking at upcoming developments, Taiuti predicts that interactive world models will soon gain deep persistence. Instead of ephemeral streams that vanish once a session ends, future worlds will maintain states that users can serialize, store, restore, and share collaboratively. Creators will be able to invite other human participants into their generated spaces or deploy autonomous agents into them to run spatial exploration and rigorously evaluate policy models for robotics and gaming.

In his closing demonstration, Taiuti shows an experimental application created entirely by a third-party developer who used a world model to synthesize a software user interface pixel by pixel from scratch. Rather than assembling traditional UI layouts from pre-compiled UI widgets, buttons, and frameworks, the application renders dynamic visual interactions entirely on the fly via code scripting:

> “And so I think this in many ways is the future of application and software development because you can just put anything on screen at any moment just from scripting it and it's going to change the way that we interface with computers really forever.”

Taiuti concludes with a call to action for software developers, roboticists, and researchers to immerse themselves in world models today, framing real-time neural generation as an inevitable paradigm shift in human-computer interaction and embodied AI.

## Notable quotes > [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) Whereas in the future, more and more systems and AI systems will be reactive. They will be real time. They will be systems that you can poke at them and they poke back at you. — Alberto Taiuti

## Mentioned Alberto Taiuti, Reactor, Carl Sagan, Jumplo, Luma AI, Apple, Apple Vision Pro, Skydio, Elon Musk, Waymo, Scale

[Source](https://www.youtube.com/watch?v=RMspGHdVg-Q)

中文简报

Reactor首席执行官称现有AI基础设施无法满足实时机器人需求无状态生成式AI基础设施无法支持物理自动化，因为延迟失误会导致碰撞，这需要基于机器人力量遥测数据的有状态视频模型。 Reactor首席执行官兼联合创始人Alberto Taiuti探讨了实时交互式世界模型如何成为超越被动、异步生成式AI的下一个变革性前沿。他将世界模型定义为以动作为条件的状态机，需要严格的因果关系、低延迟、持久性以及物理准确性，并将互联网视频常识与具身机器人遥测技术相结合。Taiuti详细阐述了在优化的传输流水线上以每秒60帧运行精简的有状态神经模拟所需的全栈基础设施革新。通过现场交互式演示，他展示了神经渲染将如何取代价值数百万美元的传统3D图形管线，并为逐像素生成的软件应用铺平道路。世界模型标志着从被动、异步AI（如大语言模型和离线扩散模型）向以60 FPS运行的实时、交互式、以动作为条件的状态机的范式转变。 (3:33) 虽然互联网规模的视频提供了丰富的视觉常识和运动学信息，但来自物理机器人的具身遥测数据（如力、扭矩和触觉反馈）对于让模拟根植于物理现实至关重要。 (6:22) 部署实时世界模型需要彻底重构AI基础设施，将重点从纯吞吐量转向超低延迟、模型蒸馏和高性能网络传输层。 (10:21) 交互式神经生成对价值数百万美元的传统3D资产管线构成了挑战，能够实现从即时游戏环境到逐像素生成用户界面的各类应用。 (15:13)

Reading summary

Alberto Taiuti, CEO and co-founder of Reactor, argues that artificial intelligence is shifting from passive, asynchronous generators toward real-time, interactive world models. In his view, scaling embodied robotics, dynamic simulations, and next-generation software interfaces requires stateful, causal systems supported by entirely re-engineered, low-latency computing infrastructure. From Passive Modalities to Causal State Machines (3:33) Taiuti contrasts mainstream generative architectures—such as Large Language Models and offline image or video diffusers—with reactive systems. Current systems operate asynchronously, requiring users to wait seconds or minutes for a static completion. In contrast, Taiuti asserts that the next computational paradigm belongs to real-time systems that dynamically update visual outputs in response to user or agent interventions. Grounding Internet Video with Robotic Telemetry (6:22) Building robust world models requires training on web-scale video datasets, which supply foundational visual common sense regarding motion, lighting, and everyday physical causality. However, Taiuti notes that internet video possesses an inherent limitation: it records surface appearance rather than underlying mechanical reality, omitting contact dynamics, friction coefficients, and applied forces. To bridge this divide, Taiuti points to a flywheel between simulation and embodied robotics. As autonomous physical robots deploy, their onboard sensors collect force-torque, tactile, and inertial ground truths. Integrating this physical telemetry into video foundation models transforms visual approximations into physically accurate simulation engines capable of evaluating robot control policies. Infrastructure Constraints of Real-Time Stateful Inference (10:21) Existing AI serving infrastructure is ill-suited for world models because it prioritizes batched throughput over minimal latency. While a one-second response time is tolerable in conversational chat, Taiuti emphasizes that such delays in robotics could cause a machine to crash into a wall. Closed-loop control tasks demand instantaneous, causal, and continuous state updates. Executing world models in real time requires full-stack engineering overhauls. Massive foundation models must be compressed via distillation into autoregressive backbones that run in fewer inference steps on specialized silicon. Additionally, Taiuti argues that low-latency network transport layers, geographic proximity of inference clusters, and optimized video compression codecs are just as essential as machine learning research to prevent transmission bottlenecks. Simulated Production Environments and Generative Interfaces (15:13) In live demonstrations, Taiuti highlights neural environments generated from a single static image and streamed at 60 frames per second directly from remote cloud servers, bypassing traditional 3D graphics assets and game engines. While commercial autonomous driving platforms like Waymo already employ world simulations to test safety-critical edge cases, Taiuti demonstrates consumer applications ranging from interactive characters to photorealistic natural scenes. Looking forward, Taiuti forecasts that world models will incorporate session persistence, allowing users and autonomous agents to collaboratively explore and modify saved environments. Furthermore, he highlights early experiments where dynamic software user interfaces are synthesized pixel by pixel via code scripting rather than assembled from traditional, pre-compiled widget frameworks. While Taiuti presents real-time world models as an inevitable paradigm shift for computing and robotics, the transition depends on solving steep engineering challenges, including millisecond-level inference latencies, persistent session state tracking, and collecting non-visual physical telemetry to prevent simulations from diverging from physical reality.

精炼阅读摘要

Reactor 联合创始人兼首席执行官 Alberto Taiuti 指出，人工智能正从被动的离线生成转向低延迟、可交互的世界模型；这类模型本质上是维持内部状态的状态机，正在成为机器人控制、仿真与新型软件界面的底层计算范式。从被动生成到交互式状态机 (3:33) Taiuti 认为，当前以大语言模型和扩散模型为主流的 AI 系统具有异步和被动的局限性：用户提交提示词后需要等待推理，最终仅获得静态文本、图像或离线视频切片。相比之下，计算界面的演进趋势是从低带宽的文本与音频逐步过渡到高带宽的实时交互视频。未来的核心系统需要具备双向交互能力，能够实时响应外部操作并即时反馈。世界模型的定义与五大核心特征 (5:16) 从工程架构来看，世界模型本质上是一个动作条件驱动的状态机。它在内部维护环境的状态表征，当接收到外部动作指令后更新内部状态，并连续输出下一阶段的观测结果与像素画面。Taiuti 提出，真正的世界模型必须满足五大核心特征：具备因果性（以时序历史为依据生成后续帧）、具备交互性（无明显感知延迟地响应控制信号）、具备实时性（支持机器人闭环控制等高频场景）、具备持久性（视角移开或返回时空间物体保持稳定），以及具备物理准确性（遵循质量、重力和碰撞等机械力学规律）。互联网视频与机器人数据的协同闭环 (6:22) 训练世界模型需要庞大的物理世界常识，而网络海量视频虽然蕴含丰富的运动轨迹、光照规律和因果演示，但仅记录了像素外观，缺乏摩擦系数、关节扭矩、接触力学和惯性等底层物理参数。Taiuti 提出，这可以通过世界模型与实体机器人的双向飞轮来弥补：实际部署的机器人通过机载传感器采集力矩和触觉等物理真值数据，再将这些数据回流并入视频模型，从而使视觉仿真演进为具备真实力学精度的模拟环境。目前，世界动作模型（WAM）已用于具身智能策略的预演评估，自动驾驶公司 Waymo 亦采用类似模拟验证边缘驾驶策略。现有基础设施的短板与全栈工程重构 (10:21) Taiuti 强调，此前为离线、无状态生成设计的 AI 基础设施无法直接支持世界模型。例如在机器人控制场景中，大语言模型常见的单秒级延迟会导致机器撞墙，因此系统必须实现极低延迟的即时响应。要实现毫秒级生成，工程团队必须对视频基础模型进行深度蒸馏，使其以更少推理步数实现自回归与因果计算，同时必须针对硬件特性进行优化。全栈流媒体管道与实时演示 (12:23) 除了模型压缩，交付管道的端到端重构同样关键。系统必须优化服务器地理分布、视频编解码器与底层网络传输协议，以确保云端生成的像素流能够即时跨网分发给客户端。Taiuti 在现场演示了单张静态图片即时初始化的虚拟场景，以及操纵小狗在写实公园内奔跑跳跃的场景；这些画面完全由 Reactor 云端模型以 60 FPS 动态推理并实时推流生成，无需预先制作传统的 3D 几何网格或贴图资产，大幅颠覆了以往耗资巨大、耗时数月的 3D 开发管线。持久化共享环境与逐像素动态软件 (17:03) 面向未来发展，Taiuti 预测世界模型将从瞬时会话走向持久化世界，用户与开发者将能够保存、恢复并多端共享生成的空间，甚至允许自主智能体长期在其中探索并优化控制策略。此外，现场还展示了由第三方开发者利用世界模型完全逐像素动态生成的软件界面，该方案不再依赖预先编译的按钮和控件布局，而是纯粹依据代码逻辑在屏幕上即时渲染视觉交互，展示了生成式底层改变计算交互方式的潜力。 Taiuti 呼吁开发者、机器人专家与研究人员尽早投入世界模型的研发与应用。然而，要将无状态的被动式视频生成彻底转变为高保真、低延迟且具备物理精确性的生产级基础设施，仍高度依赖网络传输、硬件推理极致优化以及物理感知数据的深度整合。

Detailed notes — zh

## Background 该视频由 YouTube 频道 @Scale 于 2026 年 10 月 30 日发布。视频内容为 Reactor 联合创始人兼首席执行官 Alberto Taiuti 的主题演讲，页面未提供额外的文字简介。 ## TL;DR Reactor 联合创始人兼 CEO Alberto Taiuti 深入探讨了“世界模型”（World Models）如何引领下一代 AI 范式转移。他指出，AI 正在从被动、无状态的文本和图像生成，走向低延迟、有状态且具备交互性的实时视频与物理模拟。通过结合互联网规模的视频数据与实体机器人的动力学传感反馈，世界模型能够从纯粹的视觉逼真跨越至物理真实。针对现有基础设施高延迟和无状态的瓶颈，Taiuti 演示了 Reactor 实现的 60 FPS 实时交互式推演，并展示了世界模型在机器人仿真训练以及颠覆性“生成式 UI”软件界面上的广阔应用前景。 ## Atmosphere & style 演讲洋溢着极客式的激情与对前沿技术的坚定信念，Taiuti 以 Carl Sagan 的经典名言开场，奠定了宏大而深远的叙事基调。他的语调从容自信、逻辑严密，穿插着数个令人惊艳的 60 FPS 实时交互演示，将复杂的数学状态机抽象概念与直观震撼的动态视觉无缝衔接。整场分享节奏紧凑且富有感染力，不仅展现出深厚扎实的全栈工程沉淀，更向观众传递出一种亲历人机交互与软件开发历史性剧变的兴奋感。 ## Bulletin - [00:00](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=0s) Reactor 首席执行官 Alberto Taiuti 展示了基于单张静态图像初始化的实时世界模型，该系统能够以 60 FPS 的超高帧率从云端逐帧流式推演并呈现可交互的虚拟环境，无需依赖本地三维多边形资产。 - [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) 世界模型在计算本质上是一个有状态的状态机，通过融合互联网级视频的因果运动常识与真实机器人的触觉受力、摩擦力及动力学数据，实现从“视觉逼真”到“物理法则精确”的跨越。 - [10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) 传统 AI 架构的无状态与离散非流式特性无法满足毫秒级机器人控制与交互需求，世界模型需要围绕自回归因果蒸馏、极致低延迟网络传输与状态保持全面重构底层基础设施。 - [15:55](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=955s) 生成式 UI 与逐像素原生动态界面的出现，标志着人机交互与软件开发正在从传统静态预设组件转向完全由世界模型脚本驱动的实时像素流演变。 ## Key takeaways - [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) 世界模型代表了 AI 从被动生成走向主动实时交互的核心范式转移，其系统目标是实现毫秒级“动作-反馈”的即时闭环。 - [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) 世界模型的底层计算抽象是一个状态机（State Machine），需要持续维护内部状态并在外部动作指令干预下推导出下一个状态。 - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) 互联网海量视频数据为模拟器提供了物体运动、因果关系和时空连续性的先验常识，是构建世界模型骨干的天然富矿。 - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) 纯二维视频数据缺乏真实世界的接触力、关节力矩及地面摩擦等深层物理信息，必须引入真实机器人部署采集的传感数据进行对齐与补充。 - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) 成熟的世界模型必须同时满足五大核心特征：因果性（Causal）、可交互性（Interactive）、实时性（Real-time）、持久性（Persistent）以及物理精确性（Physically accurate）。 - [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) 自动驾驶和具身智能机器人等物理系统严重依赖高保真世界模型进行低成本仿真测试，免去实体物理试错所带来的高昂代价。 - [10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) 现有的 LLM 与文生图基础设施以无状态计算和秒级延迟为主，无法支持需要连续因果流式推演和亚秒级响应的世界模型。 - [12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) 要满足实时生成的严苛时间预算，算法层面必须对巨量视频基础骨干网络进行极致蒸馏，以极少的推理步数实现因果自回归生成。 - [12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) 系统传输层是端到端体验的瓶颈，网络架构必须从追求高吞吐量转向极致低延迟，合理优化服务器部署位置、视频编码与传输协议。 - [14:25](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=865s) Reactor 致力于构建易用的标准化平台，通过会话管理（Sessions）、状态持久化、推理加速与内建安全机制降低世界模型的应用门槛。 - [15:55](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=955s) 通过第一人称视角实时演示证实，世界模型已支持动态探索、环境持久化存储以及多 Agent 共同接入进行强化学习策略评测。 - [15:55](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=955s) 生成式 UI 打破了传统软件工程中预设控件与渲染引擎的限制，允许开发者直接用脚本实时控制屏幕上每一个像素的生成。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=0s) 世界模型与从零创造宇宙：实时演示与背景介绍 Reactor 联合创始人兼首席执行官 Alberto Taiuti 开场便直奔主题，表达了对“世界模型”（World Models）这一技术的极高热情。他强调，世界模型是一类正在快速演进的新型 AI，在诸多层面上代表了未来计算、推理乃至整个 AI 领域的发展方向。在正式展开技术探讨前，Taiuti 引用了自己偶像 Carl Sagan 的经典名言来阐明构建事物的格局观：“Carl Sagan 曾说，‘如果你想从零开始做一块苹果派，你必须先发明宇宙。’（Carl Sagan, who said that if you wish to make an apple pie from scratch, you must first invent the universe.）”他认为，开发者关注眼前的细节固然重要，但更关键的是要站在宏观视角理解长远目标，而世界模型正是这种“自下而上构建整个模拟宇宙”的具象化体现。紧接着，Taiuti 展示了一段引人注目的实时技术演示（Live Demo）。他步入了一个仅凭一张单张静态图像初始化生成的虚拟世界中。画面中的一切场景，全是由运行在 Reactor 基础设施上的视频生成模型以 60 FPS（每秒 60 帧）的超高帧率、逐帧实时演算并从云端服务器流式传输到屏幕上的，本地甚至不存在任何预设的三维多边形资产。在演示中，Taiuti 操控角色“jumplo”移动并 360 度转动摄像机镜头，展示了令人惊叹的环境一致性——镜头移开后再转回来，原先位置的物体依然稳定存在；他甚至触发了场景中的动态事件，例如实时改变基地的灯光照明。以往要以大规模实现这种效果不仅极其昂贵，而且往往需要工程团队耗费数月研发，而现在世界模型却能实时推演完成。 Alberto Taiuti 随后介绍了自己的技术背景与 Reactor 的创立初衷： - **Reactor**：由 Taiuti 联合创立并担任 CEO，致力于构建世界模型的基础设施平台，将世界模型“重新民主化”并支持大规模部署落地。 - **Luma AI**：Taiuti 曾任联合创始人兼 CTO，主导过大规模 3D 重建基础设施与视频生成技术的研发。 - **Apple**：曾在 Apple 参与 Vision Pro 研发，当时该头显仍处于高度机密的内部项目阶段，这段经历进一步点燃了他对 3D 虚拟模拟世界的执念。 - **Skydio**：曾在这家无人机机器人公司担任 AI 技术栈的实时系统工程师，深刻认识到构建高逼真度现实世界模拟器对于赋能机器人技术的不可替代性。 --- ### [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) 从被动生成到主动交互：AI 范式转移与世界模型的本质回顾当代人工智能的发展历程，Taiuti 梳理了技术模态的演进路径：业界从最初的 LLM（大语言模型），逐步拓展至图像生成、视频生成模型。这些系统固然推动了人机交互与即时应用生成的巨大进步，但它们共享一个致命的局限性——它们本质上都是“被动（Passive）、缓慢且非实时的”。例如，用户向大语言模型提问或生成单张图片，往往需要等待数秒甚至数分钟才能拿到输出结果。然而，未来的 AI 系统必将转向实时响应与交互机制，形成一种“你戳它一下，它立刻戳回你一下”的即时反馈闭环；甚至屏幕上呈现的每一个像素，都将由 AI 在毫秒级时间内实时运算推导生成。AI 发展正从低维的文本、音频，迈入高维的图像、视频，并最终走向可随人的意图自由塑造的实时交互视频。 ```mermaid flowchart LR accTitle: AI Modality Evolution towards World Models accDescr: Diagram tracing the historical evolution of AI modalities from passive low-bandwidth inputs to real-time interactive world models. subgraph Passive["被动与非实时系统"] A["文本 / 音频"] --> B["图像生成"] B --> C["离线视频生成"] end subgraph Interactive["交互与实时闭环"] D["世界模型 (World Models)"] E["实时像素生成 (60 FPS)"] D <--> E end C -->|"范式转移: 状态化与交互"| D ``` 针对“究竟什么是世界模型”这一核心问题，Taiuti 揭示了其底层数学与计算本质：世界模型在最基础的抽象层面上，就是一个带状态的转移系统。被动模型每次都是无状态或静态推断，而交互系统必须将当前环境的瞬时状态纳入考量，才能准确生成下一时刻的演进。“在最基本、最核心的维度上，世界模型本质上就是一个状态机（At the most basic fundamental aspect of things, a world model is a state machine）。”其运转逻辑极为精炼：模型维护一个内部状态（Internal State），接收来自外部的一个动作指令（Action Input），预测计算并转移至下一个状态（Next State），并在循环回路中不断自我迭代。 --- ### [06:22](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=382s) 视频数据与机器人力学的联姻：构建物理真实的模拟器为了能够模拟真实的物理世界乃至完整的应用程序，海量训练数据的来源成为了关键挑战。Taiuti 指出，人类极其幸运地拥有“视频”这一天然模态。互联网规模的视频数据本身就蕴含着海量关于客观世界的暗度信息——它天然记录了物体的运动规律、因果关系、时间流动以及基础视觉光影与碰撞。当 AI 模型吞吐了互联网量级的视频后，它便学会了观察并推演动态规律，从而构成了高保真模拟器的坚实骨架。然而纯视觉数据存在天然的阿喀琉斯之踵：二维互联网视频无法直接测得摩擦力（Friction）、真实动力学（Dynamics）以及力（Forces）。巧合的是，伴随世界模型的成熟，机器人具身智能领域也在经历爆炸式发展。当实体机器人越来越多地部署于真实物理环境中，它们采集回来的正好是视频所欠缺的触觉接触力、关节力矩及地面摩擦等深层物理数据。将这些物理传感数据回灌至世界模型中，便补齐了“看起来逼真（Looks correct）”与“物理法则上真实无误（Actually is correct）”之间的鸿沟，这也正是当下“世界动作模型”（World Action Models, WAM）与视频骨干网络备受瞩目的核心原因。 ```mermaid flowchart TD accTitle: Marriage of Video Data and Robotics for Physical Reality accDescr: Shows how internet-scale video datasets and robotic sensor feedback combine to form physically accurate world models. subgraph DataSources["数据输入源"] V["海量互联网视频<br>(因果、运动、重力、视觉常识)"] R["真实机器人部署<br>(受力、摩擦力、动力学参数)"] end subgraph CoreModel["世界动作模型核心"] WM["世界模型 (状态机推演)"] end subgraph OutputState["目标与成效"] Out["从'视觉逼真'跨越到'物理精确'"] end V -->|"提供常识先验"| WM R -->|"补齐物理真值"| WM WM --> Out ``` Taiuti 总结了真正成熟的世界模型必须同时具备的五大核心属性： - **因果性（Causal）**：模型生成后续状态时，严格将前序发生的事实作为前置约束，具备时间单向性。 - **可交互性（Interactive）**：能够即时响应外部干预，接受动作输入并即刻给出环境反馈。 - **实时性（Real-time）**：推演输出的延迟极低、吞吐极高，足以满足机器人控制回路，防止实体机器人在动态平衡中摔倒。 - **持久性（Persistent）**：具备状态记忆能力，哪怕视线移开，曾生成的历史场景与物体在重访时依然恒定存在。 - **物理精确性（Physically accurate）**：深度结合动力学真实法则，而非仅停留在像素层面的天马行空。在应用落地层面，世界模型早已跨出象牙塔，在具身智能与机器人策略训练、新形态生成式游戏与交互影视、动态数字人与 AI 导师等领域全面铺开。尤其是对于现实复杂场景的孪生仿真，它免去了动辄数百万美元的实体试错成本；例如像 Waymo 这类领先的自动驾驶巨头，已经在积极依赖高保真仿真环境来规模化验证其自动驾驶系统。 ### [10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) 世界模型的产业现状与传统 AI 架构的技术瓶颈 Alberto Taiuti 指出，世界模型其实今天已经进入了生产环境应用。在众多方向中，最令他兴奋的领域之一是通过世界模型实现的“生成式 UI”（Generative UI）——系统可以随心所欲地在屏幕上实时生成任何像素。不仅是行业从业者，许多领军人物也对此极度看好，正如 Alberto 所引述的：“Elon Musk who said that you know in many ways the future of AI is going to be video understanding and generation and this is because video contains so much information that it is very useful as a medium to learn about the world.” 视频蕴含的信息密度极大，是理解物理世界的最佳媒介，这将赋予人们构建前所未有的系统与虚拟世界的能力。然而，现有的基础设施完全无法支撑世界模型的运行。过去几年行业构建的 AI 系统本质上是缓慢、被动且缺乏交互性的。例如，现存的大语言模型（LLM）系统虽然能够吐出 Token，但并不具备连续流式（continuously streaming）生成的能力，而世界模型必须具备因果性（causal）并持续不断地生成；在延迟方面，普通模型几秒钟给出回答尚可接受，但面对世界模型则完全失效。Alberto 强调：“So an LLM that gives you an answer in a second is not good enough if you're if you're using it to pilot a robot, for example. it needs to respond immediately otherwise the robot may hit a wall.” 此外，现有系统基本是无状态的（stateless），图像生成模型画图时无需在乎前序状态，但世界模型必须依赖历史状态。因此，整个基础设施必须推倒重来，为世界模型占据 AI 核心地位的未来重新设计。 - **现有 AI 架构缺陷**：高延迟、离散非连续流式、被动响应、无状态（Stateless）。 - **世界模型核心需求**：连续流式输出、因果自回归（Causal Autoregressive）、亚秒级极低延迟、强状态保持（Stateful）。 ```mermaid flowchart TD accTitle: Traditional AI vs World Model Architecture accDescr: 对比传统AI架构与世界模型在延迟、状态管理和输出模式上的核心差异。 subgraph Traditional["传统 AI 架构（如 LLM / 文生图）"] T1["高延迟响应（以秒计算）"] T2["无状态计算（Stateless）"] T3["离散 Token / 单次资产生成"] end subgraph WorldModel["世界模型架构要求"] W1["极低延迟交互（实时预算）"] W2["有状态追踪（Stateful）"] W3["连续流式生成（Streaming 60 FPS）"] end Traditional -.->|"无法适配物理交互"| Clash["机器人碰撞 / 交互中断"] WorldModel -->|"端到端全链路优化"| RealTime["实时机器人控制 & 生成式 UI"] ``` --- ### [12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) 实时全栈工程挑战：从模型蒸馏到网络传输要让系统真正做到“实时运行”，意味着每次生成事件都受限于极其微小的“时间预算”（time budget），无论是将画面呈现在屏幕上还是直接下发指令控制机器人。为达成这一目标，算法层面必须将庞大的视频基础骨干网络（video backbones）进行深度蒸馏，使其以极少的推理步数完成生成，同时保持因果性与自回归特性。此外，开发者必须对底层硬件有极深理解，否则根本无法压榨出硬件的极致性能，也就无法实现这项技术的普惠化。在系统与网络工程层面，世界模型的“有状态”特性引发了更严苛的挑战。从云端世界模型生成帧或动作，到最终客户端接收，每一个环节都必须重构。系统架构的权衡取向必须从“优先吞吐量”彻底转向“优先低延迟”，并将安全与可靠性置于批量资产生产之上。Alberto 提到：“The transport layer that you choose is crucial to get right because otherwise you can have the fastest best word model in the world and it's still going to be useless if you cannot move the bites that it generates across the internet and across clients quickly enough.” 服务器的物理选址部署、视频流的分辨率设定与压缩编码算法，以及底层的传输协议，都会直接决定世界模型能否正常工作。 ```mermaid flowchart LR accTitle: End to End Low Latency Pipeline accDescr: 展示世界模型从云端蒸馏推理到网络传输再到客户端呈现的全链路延迟优化流程。 subgraph Cloud["云端计算节点 (Edge / Optimized Server)"] A["视频骨干网络蒸馏"] --> B["超少步数因果生成"] B --> C["高压缩比低延迟编码"] end subgraph Network["传输网络 (Transport Layer)"] C --> D["极低延迟网络传输协议"] end subgraph Client["终端交互 (Client)"] D --> E["60 FPS 实时流式呈现 / 机器人控制"] end ``` --- ### [14:25](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=865s) Reactor 的平台定位与生态标准化为了彻底释放世界模型的潜力，必须降低其使用门槛，使其对任何人来说都易用且易懂。Alberto 呼吁对世界模型的接口进行尽可能深入的标准化，让普通开发者配合 LLM 和编码智能体（Coding Agent）就能直接调用世界模型，开发出全新的仿真系统、机器人落地场景或新型视频游戏。这套体系需要围绕会话管理（Sessions）、状态持久化、极致推理加速以及系统可观测性（Observability）展开，同时必须在超高速运行的动态环境中内建强安全机制。正是基于这些痛点，Alberto 和他的团队创办了 Reactor，致力于打造专属于这一新范式的平台底座。他展示了当前行业中涌现的大量世界模型，并强烈建议现场观众亲自上手体验：“This is not about reactor. It's more about you know these models are available to try and you should really spend time with them to appreciate how fast things are evolving and how amazing already the things that you can do with them are.” 他感慨道，当前展示的许多模型在仅仅一年前还是天方夜谭，因此必须紧跟演进节奏，为一年后的飞跃做好准备。 - **核心基础设施要素**：统一标准接口、会话机制（Sessions）、状态保持与恢复、内建安全机制、系统可观测性。 - **生态目标**：赋能编码 Agent 与开发者快速构建机器人策略、仿真引擎及交互式游戏。 --- ### [15:55](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=955s) 实机演示：60 FPS 实时小狗视角漫游与软件交互革命拥有 3D 图形学背景的 Alberto 直言，能够仅用一张图片和几行文字提示就实时生成一个完整可交互的世界，这在以前简直不可思议。过去要制作类似水准的虚拟世界，往往需要耗费数月时间、庞大团队和数百万美元的成本。他在现场进行了第二次演示：以一只小狗的第一人称视角在公园里奔跑漫游。他在屏幕前操控小狗跳跃、奔跑，画面根据交互指令实时动态演化。 Alberto 兴奋地阐述道：“And again, this is all being generated in real time streaming in 60 frames per second from our servers live to the screen.” 该画面完全由云端服务器实时流式生成并传输，帧率稳定在 60 FPS。更重要的是，世界的演变即将迎来更多突破：这些世界将具备持久性（Persistent），状态可以被完整存储、恢复并跨用户分享，人们可以邀请其他玩家或 AI Agent 共同进入该世界。智能体可以在其中自由探索并进行机器人运动策略（Robotics Policies）或游戏机制的端到端强化学习测试。在演讲尾声，Alberto 展示了一个社区开发者构建的激进 Demo——利用世界模型从零开始“逐像素”（pixel by pixel）动态生成一款前所未见的新型应用界面。他总结道：“And so I think this in many ways is the future of application and software development because you can just put anything on screen at any moment just from scripting it and it's going to change the way that we interface with computers really forever.” 这种摆脱传统静态 UI 控件、由脚本驱动任意像素实时生成的模式，将彻底颠覆人机交互与软件工程的未来形态。 - **实时演示性能**：基于单张图像提示启动，云端实时串流，稳定输出 60 FPS 动态可交互画面。 - **未来演进特性**：世界状态持久化（State Save/Restore）、多人协同接入、自主 Agent 策略仿真与评测。 - **人机交互范式转变**：从静态渲染 UI 转向由世界模型实时生成的动态像素级原生界面。 ## Notable quotes > [10:21](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=621s) So an LLM that gives you an answer in a second is not good enough if you're if you're using it to pilot a robot, for example. it needs to respond immediately otherwise the robot may hit a wall. — Alberto Taiuti > [12:23](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=743s) The transport layer that you choose is crucial to get right because otherwise you can have the fastest best word model in the world and it's still going to be useless if you cannot move the bites that it generates across the internet and across clients quickly enough. — Alberto Taiuti > [03:33](https://www.youtube.com/watch?v=RMspGHdVg-Q&t=213s) At the most basic fundamental aspect of things, a world model is a state machine — Alberto Taiuti ## Mentioned Alberto Taiuti, Reactor, Carl Sagan, Luma AI, Apple, Vision Pro, Skydio, Waymo, Elon Musk [Source](https://www.youtube.com/watch?v=RMspGHdVg-Q)

Source quotes

[10:21] Alberto Taiuti: So an LLM that gives you an answer in a second is not good enough if you're if you're using it to pilot a robot, for example. it needs to respond immediately otherwise the robot may hit a wall. [12:23] Alberto Taiuti: The transport layer that you choose is crucial to get right because otherwise you can have the fastest best word model in the world and it's still going to be useless if you cannot move the bites that it generates across the internet and across clients quickly enough. [3:33] Alberto Taiuti: At the most basic fundamental aspect of things, a world model is a state machine

Source quotes

[3:33] Alberto Taiuti: Whereas in the future, more and more systems and AI systems will be reactive. They will be real time. They will be systems that you can poke at them and they poke back at you.

Original transcript

[0:00] Awesome. [0:06] Really great to be here. I'm excited [0:08] today to tell you about a piece of [0:09] technology called world models and why [0:12] you should be as excited about them as I [0:15] am. World models are a new type of AI [0:18] that are evolving very rapidly and that [0:21] they are in many ways the future of [0:23] computing and inference and AI. And so [0:26] today I want to introduce you to them, [0:28] tell you all about them, why you should [0:30] be excited about them, like I said, and [0:32] in general just take you through a [0:34] journey that I hope by the end of the [0:35] talk will make you feel excited to [0:37] explore more about them. And before we [0:41] even start, I always like to start my [0:43] talks by mentioning my hero Carl Sean, [0:45] who said that if you wish to make an [0:47] apple pie from scratch, you must first [0:49] invent the universe. I think that this [0:52] quote really symbolizes in many ways the [0:54] way that I like to think about how you [0:57] build things and what you choose to work [0:58] on because it's all fair and well to [1:02] focus on the nitty-gritty details of [1:03] things and what you have in front of you [1:05] that you're working on, but it's also [1:07] very important to think about the grand [1:08] scheme of things and and why you're [1:10] doing something on a larger scale. And [1:12] so this is a very inspiring quote that I [1:14] think also relates to world models and [1:16] and why they're awesome. And so without [1:18] further ado, I'd love to first introduce [1:20] you to world models and give you a live [1:23] demo of them. So as you can see here, [1:27] I'm going to step into this virtual [1:29] world [1:29] >> started from an image [1:30] >> and all of this is being generated in [1:32] real time as we speak frame by frame [1:36] streamed from our servers live to the [1:38] screen. None of this exists in here. All [1:41] of this is being made by a video model [1:43] running at 60 frames per second and [1:45] streaming the content to the screen [1:47] using the character jumplo. [1:50] >> And look at how the consistency of the [1:52] model allows me to turn the camera [1:54] around [1:55] >> and for things to remain there. And [1:57] remember that all of this is being [1:58] generated through effectively what is a [2:00] video model running at 60 frames per [2:03] second at scale through reactor. And I [2:07] can even, you know, have certain events [2:10] happen in the scene and I can have other [2:12] things happen like the the lights in the [2:15] base changing. [2:17] And all of this is happening in real [2:18] time as we speak. And doing this at [2:20] scale before would have been absolutely [2:23] incredibly costly and require many, many [2:26] months of people working on this. [2:28] Now that you've seen what a world model [2:30] is, I'd love to introduce myself. So I'm [2:33] Alberto and I am the CEO and co-founder [2:35] of a company called Reactor. And at [2:37] Reactor, we're building a infrastructure [2:39] platform for world models to [2:41] redemocratize them and make them [2:43] available at scale. And previously I was [2:47] also CTO and co-founder of a company [2:49] called Luma AI where I was working on [2:52] infrastructure for 3D construction and [2:54] also video generation at scale. I also [2:57] worked at Apple on the Vision Pro [2:59] because my passion is simulated 3D [3:01] worlds. And so I worked on the Vision [3:03] Pro back when it was still a secret [3:05] project and it was a very amazing [3:07] experience. And that even more [3:08] heightened my passion and interest for [3:10] 3D world models and and in general [3:13] simulations and virtual worlds. And I [3:15] also worked on a robotics company called [3:17] Skyio where I was working as a real-time [3:20] engineer on the AI stack. And that also [3:24] even more made me understand how [3:26] important it is to have amazing [3:28] simulations of the world that can help [3:30] robotics and other fields. [3:33] And so to introduce the idea of world [3:35] models, I thought it was interesting to [3:37] talk a little bit about the history of [3:39] how AI has been evolving and where it's [3:42] going. And so for example, when you [3:45] think about AI today, you think about [3:47] LLMs and sometimes you think about image [3:49] and video generation models. And all of [3:52] these technologies are extremely [3:53] powerful and they've been extremely [3:55] useful in advancing humanity and getting [3:58] us to systems where you can talk to [4:01] computers. You can generate entire [4:03] applications on the go and on the fly. [4:06] However, one common aspect of all these [4:10] models is that they are all passive. [4:13] They are all slow and they're not real [4:15] time. meaning that when I ask an LLM for [4:18] a question or I ask for an image to be [4:20] generated, I'm often left waiting for [4:22] minutes before I see an output. Whereas [4:26] in the future, more and more systems and [4:29] AI systems will be reactive. They will [4:31] be real time. They will be systems that [4:33] you can poke at them and they poke back [4:34] at you. And even more so, even pixels, [4:38] even the things that you see on a screen [4:40] will be generated only real time through [4:42] the power of AI. And this in many ways [4:45] is the next natural step in this history [4:48] of how AI has been evolving and how [4:51] things have evolved in the field where [4:53] we've been going from modalyses that are [4:56] low resolution you know text to audio [4:58] then from audio to image from image to [5:00] video and now finally real time and [5:02] interactive video that can be shaped as [5:05] you see it as you use it as you think [5:07] about it. And so this is really the next [5:10] step in AI and why world models are are [5:13] going to play a role there. And so [5:16] before we even continue, I think it's [5:18] important to talk about what a world [5:21] model is. And a world model is like I [5:24] said this evolution in AI where we are [5:27] going from systems that are passive to [5:28] systems that are interactive but also [5:31] systems that are taking into [5:33] consideration the state of things when [5:36] producing their next output. Because if [5:39] a system is real time and interactive, [5:41] it has to keep in mind what's been [5:43] happening in the world at that moment [5:45] before it gives you an output. And so in [5:48] its most basic form, a world model is [5:50] actually a type of AI that has a state, [5:54] an internal state, receives an input as [5:57] an action, and then produces the next [5:58] state and then repeats that in a loop. [6:02] And so at the mo at the most basic [6:04] fundamental [6:06] aspect of things, a world model is a [6:09] state machine. However, of course, it's [6:11] much more than that, but that's how you [6:13] should picture it when thinking of [6:14] systems that are in theory passive and [6:17] slow, but actually become interactive in [6:19] real time by becoming a world model. [6:22] And now that we've thought we talked [6:25] about this and we talked about the [6:27] ability to go from systems that are [6:28] passive, slow, and not real time to [6:31] systems that are, we also need to think [6:33] about how do we get there? And we want [6:36] to start simulating worlds. We want to [6:38] start simulating entire applications and [6:40] and and and systems. And so where do we [6:43] get all of that data? Well, fortunately [6:45] for us, we have a modality video that [6:49] already contains enormous amounts of [6:52] information about the world. If you [6:54] think about videos, they already tell [6:55] you about physics, causality, motion, [6:58] bite, and and other such properties. [7:00] Because when you see a person walking or [7:02] you see the rain falling, inherently you [7:04] can observe what happens in the video [7:06] and learn about it. And that's what we [7:08] do with AI. We can show a lot of samples [7:12] of something and have the AI learn about [7:14] the system and being able to replicate [7:16] that. And since we have internet scale [7:19] data of video, we can produce very [7:22] powerful simulators that learn about the [7:25] world by observing it. And so through [7:29] this system, we can already build very [7:31] powerful simulators aka world models [7:33] that run in real time. But however, [7:35] internet video data does not contain [7:38] important aspects of the world such as [7:40] friction, dynamics, forces. And the [7:44] beautiful thing here is that in in [7:45] tandem, in parallel with the evolution [7:47] of world models, the robotics field is [7:49] also going through a transformation and [7:52] becoming very important and evolving [7:54] quickly. And so as more and more robots [7:57] are deployed in the field, we will see [7:59] more and more data generated around the [8:01] very things that I just said we lack, [8:03] which are forces, frictions, dynamics, [8:06] etc., etc. And this can then be [8:09] incorporated in world models to bridge [8:11] the gap between a simulation that looks [8:14] correct to one that actually is. And [8:18] this is why there is this heightened [8:20] interest right now on things like world [8:22] action models and video backbones [8:25] because in many ways they are enabling [8:28] robotics and robotics in return is [8:30] enabling them to be correct and useful. [8:32] And all of this the underlying all of [8:34] these concepts are world models. [8:38] And in order to really understand why a [8:40] world model is interesting is that it's [8:42] got five amazing properties. The main [8:45] one is that it's causal. meaning that a [8:47] world model thinks about what came [8:49] before when producing the next outputs [8:51] and it takes that into consideration but [8:54] also that they are interactive meaning [8:55] that if you poke at them they they reply [8:58] back at you immediately. They are real [9:00] time so that for robotic systems they [9:02] actually produce outputs quickly enough [9:04] that the robot doesn't fall or something [9:06] like that. They are persistent meaning [9:09] that they remember what they saw before [9:10] and what they generated before and also [9:13] they are physically accurate and like I [9:15] said that comes from marrying robotics [9:17] with world models and and AI and machine [9:19] learning and this is happening a lot [9:21] these days and so to really think of [9:24] like what a world model should be it [9:25] should have these five properties and [9:28] this is not something that's just in [9:29] academia like this is already very [9:32] impactful in today's industries many of [9:35] them but they're all underlying that is [9:37] the concept of world models to all of [9:38] these applications of them. So in [9:40] robotics and embodied AI again uh there [9:43] is an explosion right now of [9:45] applications for world action models and [9:47] what they can do in gaming and and [9:50] interactive media. New types of games [9:51] are being created, new types of movies, [9:53] new types of experiences that weren't [9:55] possible before and world models are [9:57] enabling that. We also have advances in [9:59] avasars and digital people where you you [10:02] can learn from a digital tutor much more [10:04] easily today. And of course simulating [10:07] real worlds where you can try your [10:10] robotics policies and make sure that [10:11] they behave correctly without having to [10:13] spend a lot of money. And to give you an [10:15] idea here, companies like Whimo, they [10:18] actually actively test their AV systems [10:21] in world models. So these systems, world [10:24] models are already in production today. [10:26] One of the areas that I'm most excited [10:28] about are generous is generative UI [10:30] through world models whereby we can put [10:33] any pixel on screen that is [10:37] generated in real time as as we want and [10:41] a lot of people are already excited [10:42] about this including Elon Musk who said [10:45] that you know in many ways the future of [10:47] AI is going to be video understanding [10:49] and generation and this is because video [10:51] contains so much information that it is [10:54] very useful as a medium to learn about [10:56] the world. And so this is going to [10:59] generate the ability for us to really [11:04] create systems and worlds that weren't [11:06] possible before. [11:08] The challenge that we have is that world [11:10] models cannot really run on what we've [11:12] been building up to today because up to [11:14] today we've been building systems for AI [11:17] that are slow, passive, and not [11:19] interactive like I said. So for example, [11:22] we have systems that produce tokens but [11:25] that are not continuously streaming. And [11:27] like we said, world models actually need [11:29] to be causal and generate continuously. [11:32] So that doesn't work. We have systems [11:34] that are very very high in latency in [11:36] the sense of like even if the output [11:39] happens after just a few seconds, that [11:41] is not good enough for a world model. So [11:43] an LLM that gives you an answer in a [11:45] second is not good enough if you're if [11:47] you're using it to pilot a robot, for [11:49] example. it needs to respond immediately [11:51] otherwise the robot may hit a wall. [11:54] They're stateless. Whatever input you [11:56] gave before to these systems, they don't [11:58] need to know. When you're making an [11:59] image, the the image generation model [12:02] doesn't need to know what came before, [12:04] but a world model does. And so for this [12:07] reason, many many systems that we have [12:08] today haven't been built for that [12:10] future. And we need to start building [12:11] for a future where world models are [12:14] front and center of of of AI and they [12:18] require you know these types of of [12:20] properties. [12:23] And so to really get a little bit more [12:26] in the details of how we should run [12:28] these systems, consider that when we say [12:30] that something should run in real time, [12:32] it means that every time that we make a [12:35] generation, we have a very small budget [12:37] of time to make something happen. either [12:40] appear on the screen or control a robot. [12:42] And in order to get there, we need to [12:44] make systems that are very very [12:45] efficient and also causal and also [12:48] regressive. And so it's very important [12:50] that we take large video backbones and [12:53] we distill them into systems that are [12:54] much faster that do fewer steps to [12:57] generate the output and also that take [13:00] into consideration what happened before [13:02] so that they can be causal. Like we [13:04] said, it's also super important that we [13:07] understand the hardware that we're [13:08] working with deeply. Otherwise, we [13:11] cannot squeeze the performance out of [13:12] them. We and and we cannot achieve the [13:15] democratization of this technology that [13:16] we want. [13:19] And like I said, since a world model is [13:20] stateful, there are many more [13:22] considerations that we need to do to [13:24] make them happen and make them useful. [13:26] We need to think about, you know, from [13:27] when the world model is running and [13:29] frames or actions are generated in the [13:32] cloud to the client receiving them. [13:34] Every single step of the way, you have [13:36] to build systems that are optimized for [13:39] this use case where we prefer latency [13:42] over throughput and where safety and [13:45] reliability is much more important than [13:47] than large mass production of of assets. [13:50] And so it requires many more [13:52] considerations than normal. For example, [13:54] latency is extremely important, i.e. [13:57] where you pick your servers. The [13:59] resolution at which you stream video and [14:02] and at which you compress video becomes [14:04] crucial to understand and to really [14:07] optimize. And also the transport layer [14:09] that you choose is crucial to get right [14:12] because otherwise you can have the [14:14] fastest best word model in the world and [14:17] it's still going to be useless if you [14:18] cannot move the bites that it generates [14:21] across the internet and across clients [14:23] quickly enough. [14:25] And finally, in order to really unlock [14:28] the power of world models, we should [14:30] strive to make them useful and easy to [14:33] understand and use for anybody. Meaning [14:36] that we should think about the [14:38] interfaces to use them and standardize [14:41] them as much as possible so that anybody [14:43] with an LLM and some coding agent can [14:46] get a world model and build a new type [14:48] of simulation, a new type of robotics [14:50] use case or a new type of video game. [14:53] And we should have things around [14:55] sessions, statefulness [14:57] and faster and faster inference. Of [15:00] course, observability is going to be key [15:02] here because we are moving from systems [15:04] that are passive that we can observe to [15:06] systems that are very, very fast. And so [15:08] having security built in becomes very [15:11] important. [15:13] And so that's why we're building Reactor [15:15] today because we believe that this is [15:17] the future of AI and that we should [15:19] build a platform that is dedicated to [15:20] this amazing technology that's that's [15:22] coming and is here actually in many [15:25] ways. And here I wanted to show you just [15:27] how many models already exist that you [15:29] should go and try for yourself. This is [15:31] not about reactor. It's more about you [15:33] know these models are available to try [15:35] and you should really spend time with [15:36] them to appreciate how fast things are [15:39] evolving and how amazing already the [15:41] things that you can do with them are. [15:43] And many of these models and what they [15:45] enable wasn't possible just a year ago. [15:48] So imagine a year from now. And so it's [15:50] very important that you familiarize [15:52] yourself with what's coming down the [15:53] line. [15:55] And so with that in mind, I'd love to [15:58] show you another demo that I think is [16:00] mind-blowing. And coming from the 3D [16:01] world, it's an understatement to say [16:04] that I am amazed that I that I am [16:06] capable of doing this with just an image [16:07] and some text and to do something that [16:09] would have taken millions of dollars and [16:11] probably many many people and many [16:13] months to to make. And so I'm going to [16:15] give you another amazing demo here real [16:17] quick. [16:19] So, for the next demo, I'm going to [16:21] become a dog and actually walk through [16:23] the park and show you that once again, [16:25] it doesn't even have to be humans. It [16:26] doesn't even have to be a, you know, [16:29] scene that doesn't exist and that is [16:31] nonrealistic. This is literally just a [16:33] dog walking in the park. And I can make [16:35] the dog jump and we can walk around and [16:38] be very happy. And again, this is all [16:41] being generated in real time streaming [16:44] in 60 frames per second from our servers [16:46] live to the screen. And generating a [16:49] world like this before would have taken [16:51] months and millions of dollars, whereas [16:53] now you can just prompt it from an image [16:55] and you're ready to go. And I think this [16:58] is absolutely mind-blowing and the [16:59] future of computing in many ways. [17:03] So yeah, and I think this really should [17:06] drive home that what is possible today [17:08] is already mind-blowing, but that many [17:10] many improvements are also just around [17:12] the corner. For example, these worlds [17:14] will become persistent. They will have [17:16] states that can be stored, restored, and [17:19] shared with other people. You can invite [17:21] people into this world. You can use them [17:23] for video generation. You can use them [17:24] for simulation. You can use them for a [17:27] lot of things. And also, you can invite [17:30] agents into this world. worlds and have [17:33] them explore them but also test policies [17:36] both robotics and even just for games [17:39] and beyond that and none of these would [17:42] have been possible even a year ago. So I [17:44] am very excited about what it means here [17:46] given that we've just gotten started and [17:48] so many improvements are coming down the [17:50] pipeline and I think this makes me very [17:52] excited about the future of world models [17:54] and AI in general. [17:56] And just to leave you with something [17:58] that's really cool, I wanted to show you [18:00] a demo that someone built where they [18:02] took a world model and generated a new [18:05] type of application from scratch pixel [18:07] by pixel that looks something very very [18:10] different from what you would expect a [18:12] normal application to look like. And so [18:14] I think this in many ways is the future [18:16] of application and software development [18:19] because you can just put anything on [18:21] screen at any moment just from scripting [18:23] it and it's going to change the way that [18:25] we interface with computers really [18:27] forever. So I hope that made you excited [18:30] about the the the topic of world models [18:33] and I hope that you go and explore them [18:35] and and find out more about them and [18:37] yeah thank you very much for having me.