Google DeepMind Launches Agentic Video Understanding for Gemini Models

Google DeepMind Launches Agentic Video Understanding for Gemini Models Google DeepMind has introduced agentic video understanding across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite, replacing static fixed-frame processing with dynamic video exploration to cut costs and token use while boosting accuracy. Reduces token usage by up to 88% and analysis costs by up to 66%, while increasing accuracy by up to 7% on benchmark tests. Replaces fixed-rate ingestion with an agentic loop that dynamically fetches specific video frames, audio, or transcripts as needed. Supports capabilities such as sub-second moment retrieval, needle-in-a-haystack search in long-form media, anomaly detection, and object counting. Available via the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform, with rollouts planned for the Gemini app and YouTube's 'Ask YouTube' tool. Mechanism and Benchmark Performance Traditional video processing ingests media at a fixed frame rate, typically defaulting to 1 FPS, which often forces developers to choose between missing fine details or absorbing large token costs on long-form video. Agentic video understanding integrates Gemini's core reasoning with native video tools in an agentic loop, allowing the model to decide what visual segments, audio signals, or transcripts to retrieve and at what speed. Across standard benchmarks, this method cuts token consumption by up to 88% and costs by up to 66% while delivering an accuracy increase of up to 7%, positioning Gemini 3.7 Flash at the accuracy-to-cost pareto frontier. Core Capabilities and Use Cases The dynamic inspection process enables capabilities previously difficult under static sampling. These include sub-second moment retrieval for editing tight cuts and state changes, answering complex queries across multi-hour videos without processing millions of tokens, and resampling targeted time windows at higher frame rates for anomaly detection. It also enables accurate tracking and counting of repeated physical actions and objects over time. Availability and Product Rollout The capability is currently accessible for uploaded files and YouTube links through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform by designating processing as 'agentic'. It adheres to standard Gemini API token pricing without added feature fees. DeepMind plans to roll out the capability to general users in the Gemini app across Flash and Flash-Lite models, as well as integrate it into YouTube's 'Ask YouTube' feature on video watch pages.

Google推出Gemini Agentic视频理解功能：Token消耗最高降低88%，成本降幅达66% Google DeepMind宣布在Gemini 3.7 Flash、3.6 Flash和3.5 Flash-Lite模型中推出Agentic视频理解功能。该技术通过动态检索与检查视频片段，显著降低Token消耗与处理成本，并提升长视频分析准确率。 在基准测试中，该功能最多可减少88%的Token消耗和66%的分析成本，同时准确率最高提升7%。 区别于固定帧率的静态处理，该技术通过Agent循环自主决定查看的目标片段、采样速度以及模态（画面、音频或转录文本）。 解锁了亚秒级瞬间检索、长视频“大海捞针”搜索、高帧率重采样异常检测及精准动作与物体计数等应用。 已在Google AI Studio与Gemini Enterprise Agent Platform的Gemini API上线，按标准Token计费，后续还将接入Gemini应用与YouTube相关功能。 性能表现与效率提升 传统的视频处理以固定帧率（默认1 FPS）静态摄取媒体流，容易在长视频中引发高昂Token成本或遗漏关键细节。Gemini通过将核心推理与原生视频工具结合，动态搜索和检查目标片段。测试显示，该技术最高可降低88%的Token用量和66%的分析成本，并提升最高7%的准确率；其中Gemini 3.7 Flash在准确度与成本的帕累托前沿表现最优。 工作机制与核心应用场景 该功能赋予模型目标导向的自主选择权，通过调用内部工具动态决定观看视频的内容、速度和模态（视觉帧、音频或转录文本），免去了开发者手动构建复杂管道的负担。在应用层面，它支持亚秒级的状态变化检索以辅助精准剪辑、在数小时视频中执行复杂查询、针对异常区域提高采样帧率复查，以及准确追踪重复动作和不同物体。 上线平台与后续推广 目前开发者可通过Google AI Studio及Gemini Enterprise Agent Platform的Gemini API使用该功能，涵盖视频上传与YouTube链接，只需在配置中将处理模式设为agentic，且不加收额外功能费。此外，该技术即将向Gemini应用中的所有用户开放，并在未来数月内整合进YouTube视频播放界面的Ask YouTube功能。

Sep 01, 2026 | Our new agentic feature for video analysis cuts token consumption by up to 88%, reduces costs by up to 66%, and boosts quality by up to 7%. Rohan Doshi Senior Product Manager, Google DeepMind Mario Lučić Research Director, Google DeepMind Your browser does not support the audio element. Listen to article [[duration]] minutes This content is generated by Google AI. Generative AI is experimental Today, we’re launching agentic video understanding across our latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis. Similar to agentic vision , which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools to improve performance and unlock new capabilities for video processing like sub-second moment retrieval, more accurate anomaly detection, precise counting and more. The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. Benchmarks Unlike current ‘static’ processing, where the model ingests the video at a fixed frames-per-second rate (default 1 FPS, adjustable via API), agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect target video segments across visual frames, audio, and transcripts. Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%. These efficiency gains are especially pronounced on long-form video (from 10-minute how-to guides to 90-minute lectures and multi-hour recordings), where static processing forces developers to choose between high token costs or techniques that drop critical details. Activating agentic video understanding drops token consumption by up to 88% and boosts accuracy by up to 7% with Gemini 3.7 Flash. While these gains span all three supported models, Gemini 3.7 Flash with agentic understanding offers the best possible quality overall and the best combination of quality and cost efficiency, putting it at the accuracy-to-cost pareto frontier among tested models for video understanding. Using agentic video understanding places Gemini 3.7 Flash at the accuracy-to-cost pareto frontier for video analysis. How it works Instead of static processing where the model ingests media streams at a fixed frame rate, agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the moments and signals needed. While developers could previously do this manually, with agentic video understanding, Gemini can accomplish it through an agentic loop, invoking an internal tool to load the relevant part of the video file, significantly reducing development overheads. Capabilities and use cases Agentic video understanding transforms how developers can process long-form video content across a variety of demanding applications. Sub-second moment retrieval : Pinpoint split-second state changes and tight cut boundaries that are easily missed at 1 FPS, making precise automated video editing possible. Long-form needle-in-a-haystack search : Answer complex queries across multi-hour videos without consuming millions of tokens. Anomaly detection : Resample interesting time windows at higher FPS to inspect rapid motion and subtle visual artifacts. Counting action & object : Accurately track repeated physical movements and distinct objects over time. Real-world results Many of our early access partners saw strong performance while testing with agentic video understanding. Here’s what they have to say: Getting started Agentic video understanding is available via the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform , launching across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. It uses standard Gemini API token pricing with no additional feature fee. To enable it, simply set processing to "agentic" in the API configuration. Read our developer guide to get more insights into the feature and how to get started. from google import genai client = genai.Client() interaction = client.interactions.create( model="gemini-3.7-flash", input=[ { "type": "video", "uri": "https://youtu.be/7Z5Vy9JBANs", "processing": "agentic" }, { "type": "text", "text": "What are the 3 most important announcements in this keynote?", }, ], ) print(interaction.output_text) We are also bringing the efficiency and quality improvements of agentic video understanding to billions of users across Google products. The feature will roll out to all users in the Gemini app across Flash and Flash-Lite models soon. And in the coming months, agentic video understanding will also power YouTube's ‘ Ask YouTube’ feature on the video watch page, leveraging Gemini to deliver higher-quality answers grounded in the visuals. Acknowledgement for their contribution to this work: Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team. Get the latest news from Google in your inbox Sign up for our newsletters with product updates, event information, special offers, and more. Your information will be used in accordance with Google's privacy policy. You may opt out at any time.