Google Launches Gemini Robotics ER 2 for Video Understanding, Task Orchestration, and Multi-Robot Collaboration

Google Launches Gemini Robotics ER 2 for Video Understanding, Task Orchestration, and Multi-Robot Collaboration Google has launched Gemini Robotics ER 2, an embodied reasoning model designed as a high-level brain for robots. It enables continuous video understanding, multi-step planning, tool orchestration, and multi-robot collaboration, handing off motor execution to lower-level vision-language-action (VLA) models. Serves as a high-level reasoning model that plans tasks and natively calls tools or VLA models without jarring execution pauses via bidirectional streaming in the Gemini Live API. Achieves 57.4% accuracy in continuous progress classification and 91.3% accuracy with 0.96s mean absolute distance in precision moment-finding. Enables multi-robot collaboration across heterogeneous systems, demonstrated with Apptronik's Apollo 2 and Franka F3 Duo. Advances spatial intelligence and safety metrics, successfully halting robots when humans are nearby and resuming only when clear. Agentic Orchestration and Low Latency Gemini Robotics ER 2 orchestrates physical tasks by declaring low-level control interfaces—such as VLA models or navigation APIs—as tools while streaming video, audio, or text directly. Integrated with the Gemini Live API, the bidirectional streaming endpoint minimizes latency, allowing the robot to reason about next steps simultaneously with execution. In a demonstration, the model orchestrated Boston Dynamics' Spot via navigation and manipulator APIs to fetch items upon natural language commands. Temporal Intelligence and Progress Tracking To address the challenge of determining task completion, the model introduces continuous progress classification and precision moment-finding over continuous video feeds. It reached 57.4% accuracy in categorizing video frames across five progress intervals, enabling mid-task adjustments. For precision moment-finding—identifying the exact frame to transition tasks, such as stopping a pour—it reached 91.3% accuracy and a 0.96-second mean absolute distance at four times the execution speed of larger models. Multi-Robot Collaboration and Spatial Reasoning The model allows heterogeneous robots with different physical capabilities to communicate via shared semantic understanding and complete shared workflows. In spatial intelligence, ER 2 improves on success and failure detection from raw video feeds to catch mid-execution errors, enhances spatial visual question answering, and expands instrument reading across ten instrument formats including digital displays, linear scales, rulers, and thermometers. Safety Benchmarks and Deployment Evaluated on Safety Instruction Following and Human Proximity benchmarks, ER 2 enforces physical constraints and detects human presence, pausing humanoid robots when a person enters proximity and resuming once clear. A new benchmark was also introduced to assess foundation models acting as safe VLA orchestrators. The model is accessible to developers through Google AI Studio, the Gemini API, and in private preview on the Gemini Enterprise Agent Platform.

谷歌推出具身推理模型 Gemini Robotics ER 2，强化视频理解与多机器人协作 谷歌 DeepMind 正式发布具身推理模型 Gemini Robotics ER 2。该模型作为机器人的高层大脑，通过持续视频理解实现任务编排、实时自我修正与多机器人协作，显著提升了物理世界中的交互与执行效率。 Gemini Robotics ER 2 可调用低层 VLA 模型和谷歌搜索等外部工具，支持边行动边思考。 模型具备时间智能，在进度分类任务中达到 57.4% 准确率，在关键瞬间定位任务中实现 91.3% 准确率与 0.96 秒平均绝对距离。 新增多机器人协作支持，并提升了原始视频故障检测、10 种仪表读取及安全避障能力。 模型已通过 Gemini API、Google AI Studio 公开发布，并在 Gemini Enterprise Agent Platform 提供内部预览。 任务编排与低延迟流式交互 Gemini Robotics ER 2 充当高层具身推理代理，负责协调多步骤物理任务，并将运动执行下放给底层视觉-语言-动作（VLA）模型或导航 API。在真实 VLA、仿真 VLA 及人类遥操作三种模式下，其工具编排表现均优于前代 ER 1.6。通过整合 Gemini Live API 的双向流式端点，模型消除了停顿思考的延迟，使机器人能够在执行当前动作的同时规划下一步。 时间感知与关键节点判定 针对机器人判断任务完成时机的难题，该模型引入了连续进度分类与精准瞬间定位两项核心能力。模型将连续视频划分为五个进度区间，以 57.4% 的准确率实现动态调整与失败重试；在判定停止倒水等关键时刻的任务中，模型达到 91.3% 的准确率，并在提供亚秒级执行延迟的同时将计算开销显著降低。 多机协同、空间智能与安全基准 该模型允许轮式车与人形机器人等不同形态设备基于共享语义进行任务交接与协作。空间智能方面，模型可直接在原始视频流中检测溢出或滑落等中途故障，并支持数字屏、标尺等 10 种仪表的读数识别。此外，模型在安全指令遵循与人类接近基准上取得提升，当人员靠近时可自动停止人形机器人作业，并在环境安全后自主恢复工作。

Jul 30, 2026 | Gemini Robotics ER 2 represents a step change in powering robots with video understanding, task orchestration, and multi-robot collaboration — making it possible for robots to be more helpful in the physical world. Steven Hansen Senior Staff Software Engineer Peng Xu Staff Software Engineer For robots to assist humans in everyday environments, accurate spatial reasoning is not enough. Robots must also think fast, timing their decisions and reasoning with the real-time speed of the physical world. That’s why today we’re launching Gemini Robotics ER 2 , our most capable “embodied reasoning” model for robotics. Think of Gemini Robotics ER 2 as a high-level brain for robots. It allows robots to chat with humans, understand the physical world, and plan multi-step tasks. It then hands off motor execution to any given lower level vision-language-action (VLA) model. Gemini Robotics ER 2 can also natively call tools like Google Search to find information, or any other user-defined function. The design of Gemini Robotics ER 2 allows the robot to “think” about what comes next while simultaneously performing its actions. Gemini Robotics ER 2 represents a significant upgrade over Gemini Robotics ER 1.6 . By watching continuous video feeds, robots can now track their own progress, adapt if something goes wrong, and know exactly when to move on to the next step. We are also introducing multi-robot collaboration, enabling robots to work together in shared spaces and complete complex workflows a single robot could not do alone. Gemini Robotics ER 2 is now publicly available to developers via the Gemini API , Google AI Studio , and in private preview on Gemini Enterprise Agent Platform . To help you get started, we’re sharing examples of how to configure the model and prompt it to power more useful physical AI tasks. Advancing physical agentic capabilities Most tasks in the physical world are complex and require multiple steps to complete. Gemini Robotics ER 2 is a physical agent, orchestrating steps for the robot and enabling it to self-correct, and generalize to more novel situations. To build an agentic setup, developers can declare low-level control interfaces — like Vision-Language-Action (VLA) models or navigation APIs — as tools, and stream multimodal video, audio, or text directly into the model. Gemini Robotics ER 2 improves this tool orchestration workflow. We can evaluate its performance with robots in simulation, using real-world robot control, and even pair it with a human controlling the robot remotely. Gemini Robotics ER 2 consistently outperforms ER 1.6 for tool orchestration across three control modes: real VLA, sim VLA, and human tele-op. In robotics, high-level reasoning depends on execution speed. Gemini Robotics ER 2 integrates into the Gemini Live API , using a bidirectional streaming endpoint optimized for latency-sensitive tasks. The result is fluid orchestration: Gemini Robotics ER 2 commands action models and robotics APIs to complete multi-step tasks without the jarring “stop-and-think” pauses. To illustrate this, we’ve built a demo with Spot from our partners at Boston Dynamics . We use Gemini Robotics ER 2 to orchestrate Spot APIs , such as navigation and manipulator movement, creating an interactive robot that fetches objects for you. Gemini Robotics ER 2 powered Boston Dynamic Spot fetches a popcorn snack up on a natural language command. The code is available on Github with other examples. Unlocking temporal intelligence for robust task completion One of robotics’ hardest challenges is knowing when a task is done. Gemini Robotics ER 2 brings a step-change in video understanding and progress tracking to verify that complex tasks — such as tightening a light bulb or tying a trash bag — are complete to specification before switching to the next task. In this update, we’ve made progress on two foundational capabilities for task progress understanding: progress classification and moment finding. Continuous progress classification Progress classification refers to a robot’s ability to track progress towards task completion. In our evaluations, we assign each frame in a video feed into five levels of progress (0-20%, 20-40%, 40-60%, 60-80%, 80-100%). By quantifying task progress, Gemini Robotics ER 2 provides robots with real-time situational awareness, and allows them to adjust actions on the fly or retry failed steps without restarting an entire workflow. Gemini Robotics ER 2 achieves 57.4% accuracy on progress classification tasks, outperforming previous generation models and competing frontier models. Precision moment-finding Moment-finding measures a model's ability to identify the exact video frame where a critical event takes place (i.e. when to stop pouring coffee into a cup). Gemini Robotics ER 2 achieves significant gains in performance on moment finding, enabling robots to precisely switch between tasks, verify success and suggest corrections. For moment-finding tasks, Gemini Robotics ER 2 achieves 91.3% accuracy and a 0.96s mean absolute distance. It competes closely with much larger model categories, but delivers this precision at a fraction of the compute cost and 4x the execution speed—the sub-second latency actually required to safely operate physical robotics in the real world. Multi-robot collaboration No single robot fits every task — a wheeled rover excels indoors, while a humanoid robot may excel at uneven terrain. Gemini Robotics 2 enables multi-robot collaboration, allowing diverse machines to communicate via a shared semantic understanding to handoff and complete complex tasks. See how Gemini Robotics ER 2 enables Apptronik ’s Apollo 2 and Franka F3 Duo to collaborate here . Improving general spatial intelligence Gemini Robotics ER 2 advances our core spatial reasoning capability, as measured by three benchmarks: Success/failure detection: Now operates on raw video feeds rather than static snapshots to catch mid-execution failures like spills, slips, or misalignments. General instrument reading: Extends beyond circular dials and sight glasses to include digital displays, linear scales, rulers, and liquid thermometers. We tested it across 10 different types of instruments. Enhanced spatial VQA: Improves Visual Question Answering throughGemini’s advancements in multi-modal understanding. Gemini Robotics ER 2 consistently achieves the highest accuracy across all core capabilities, with highlights including success detection (image/video), Question Answering (ERQA), and generalized instrument reading. Advancing safety for embodied intelligence Gemini Robotics ER 2 is our safest model, achieving significant gains on Safety Instruction Following and Human Proximity benchmarks, which evaluate how a model adheres to physical constraints during reasoning tasks and spatial awareness for detecting humans. We found that Gemini Robotics ER 2 successfully halts a humanoid robot when a person is nearby and autonomously resumes work only once the area is clear. To advance safety for physical agents, we’re introducing a benchmark that evaluates a foundation model's ability to act as a safe VLA orchestrator by testing its capacity to enforce safety constraints, monitor the environment, assess physical feasibility, and seek human clarification. For details, see our safety technical report . Gemini Robotics ER 2 outperforms ER 1.6 and other frontier models on Safety Instruction Following and Human Proximity benchmarks. Looking ahead, our plans are to push these models towards even more complex tasks to accelerate the development of helpful robots and support the robotics community. Get the latest news from Google in your inbox Sign up for our newsletters with product updates, event information, special offers, and more. Your information will be used in accordance with Google's privacy policy. You may opt out at any time.