Meta Uses Autonomous Agents to Double Media Model Training Speed

@Scale

Inverting traditional human-in-the-loop workflows allows autonomous agents to monitor live inference signals, prune failing GPU runs early, and enforce strict production latency limits.

AI-assisted summary of Flywheel of Media Models & Product Velocity | Rushaan Mahajan from Meta — original source. Check the original for full context.

Source published: 2026-10-30

Key ideas

Meta is inverting legacy human-in-the-loop workflows into an 'agents-in-the-loop' framework where autonomous agents observe telemetry, prune failing runs, and iterate on media models across training and deployment. (0:05)
Autonomous sidekick agents double model training velocity by autonomously tuning hyperparameters, generating code diffs, and enforcing production latency envelopes like 100 ms limits. (14:03)
Production video frame interpolation models across Meta's Family of Apps double frame rates on low-end 15–20 fps captures, treating initial deployment as a starting line for continuous agentic retraining. (16:42)
Sandboxing agents prevents GPU compute budget overruns and suppresses noisy, microscopic updates in favor of perceptible step-change improvements in visual fidelity. (19:24)

Detailed notes

## Background This talk was published on YouTube by @Scale on October 30, 2026, featuring Rushaan Mahajan from Meta's Media Foundation team. No additional show notes or description were provided by the channel.

## TL;DR Rushaan Mahajan details how Meta accelerates media AI development by inverting traditional human-in-the-loop workflows into an autonomous 'agents-in-the-loop' continuous flywheel. By deploying secondary Vision-Language Models as automated 'visual judges' across training and inference, autonomous sidekick agents catch visual defects, tune hyperparameters, and optimize models under strict compute and latency budgets. The architecture enables massive-scale applications—such as capacity-aware golden hour photo editing and deep learning video frame interpolation—turning production checkpoints into continuous learning launchpads rather than static one-off releases.

## Atmosphere & style The presentation is crisp, highly pragmatic, and delivered with the brisk, authoritative tempo of a scaled systems architect. Mahajan avoids speculative AI hype, grounding his narrative entirely in concrete production realities like GPU cluster economics, millisecond latency gates, and real-time artifact detection. Watching the talk feels like an insider briefing on frontier infrastructure orchestration, blending industrial-scale engineering rigor with an energetic vision for automated model iteration.

## Bulletin - [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) Meta is inverting legacy human-in-the-loop workflows into an 'agents-in-the-loop' framework where autonomous agents observe telemetry, prune failing runs, and iterate on media models across training and deployment. - [14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) Autonomous sidekick agents double model training velocity by autonomously tuning hyperparameters, generating code diffs, and enforcing production latency envelopes like 100 ms limits. - [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) Production video frame interpolation models across Meta's Family of Apps double frame rates on low-end 15–20 fps captures, treating initial deployment as a starting line for continuous agentic retraining. - [19:24](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1164s) Sandboxing agents prevents GPU compute budget overruns and suppresses noisy, microscopic updates in favor of perceptible step-change improvements in visual fidelity.

## Key takeaways - [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) The traditional human-in-the-loop paradigm is being replaced by an agents-in-the-loop architecture that continuously co-designs training runs and production inference. - [01:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=102s) Serving viral features like 'golden hour' image editing requires capacity-aware models that run inference across hybrid CPU and GPU pipelines to control infrastructure costs. - [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) Visual judges leverage quantitative metrics and qualitative Vision-Language Models (VLMs) to inspect pixels directly for temporal flicker, brand logo clarity, and structural coherence. - [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) Automated visual evaluation prevents GPU cluster idle time by catching artifacts early and adjusting training parameters while human researchers are away. - [08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) Autonomous sidekick agents sample live production traffic around the clock to detect geographic edge cases and adapt models to regional bandwidth constraints. - [08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) Teams translate high-level product goals into structured intent frameworks that define strict compute budgets, visual eval rubrics, and queries per second (QPS) thresholds. - [11:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=663s) A daily grounded report synchronizes researchers and overnight agent experiments without generative hallucinations, outlining latency drift and visual judge scores. - [14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) The media harness integrates visual judges directly across both training and serving, enabling autonomous agents to double model training velocity. - [14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) Agents prevent operational drift by pruning underperforming checkpoints early and enforcing strict latency caps before models reach production. - [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) Neural video frame interpolation models replace legacy heuristic algorithms across Meta's Family of Apps to smoothly double low capture frame rates (15–20 fps). - [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) Shipping a media model to production is treated as the starting point rather than the end of a project, feeding edge cases back into continuous agentic retraining. - [19:24](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1164s) Strict operational sandboxes prevent autonomous agents from running wild, exceeding allocated GPU cluster quotas, or deploying noisy, negligible metric updates.

## Detailed walkthrough ### [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) Inverting Human-in-the-Loop: Agents in the Training and Inference Flywheel

The media AI flywheel is accelerating rapidly, driven by the tight co-design of underlying models and consumer-facing products to maximize feature velocity. For product teams, the primary target is delivering high model quality at ultra-low latency to unlock fluid user experiences. Historically, production workflows relied on human-in-the-loop paradigms where human operators gated and approved model operations. In modern media infrastructure, this architecture is inverted into an "agents-in-the-loop" framework, where autonomous agents continuously observe, act, and iterate across both training and deployment lifecycles.

```mermaid flowchart TD accTitle: Co-designed Agentic Media Model Flywheel accDescr: Feedback loop connecting agent-monitored model training, production inference deployment, and visual evaluation.

subgraph TrainingLoop ["Training Lifecycle"] TR["Model Training (GPU Fleet)"] RA["Researcher + Agent Sidekick"] EV["Offline Eval (Reference Metrics)"] TR --> EV EV --> RA RA -->|"Adjust architecture & parameters"| TR end

subgraph ProdLoop ["Production Deployment"] INF["Inference Engine (GPU/CPU)"] VJ["Visual Judge (VLM & Reference-Free)"] PD["Production Traffic & User Feedback"] INF --> PD PD --> VJ end

TR -->|"Push new checkpoint"| INF VJ -->|"Online feedback loop"| RA ```

In this co-designed architecture, offline model training and online production deployment feed directly into one another. Autonomous agents partner with research teams to tweak training parameters, evaluate checkpoints, and manage production rollouts. When new checkpoints reach production, user engagement data and operational telemetry are routed straight back into the training loop, creating an ongoing engine of continuous improvement. As Rushaan Mahajan emphasizes: “We know about human in the loop where a person gates every action of the model. I like to think about it as where is agents in the loop inverting this.”

- **Core product goals:** Maximize model fidelity while driving down latency to encourage frequent user interactions. - **Continuous refinement:** Agents actively adjust training parameters and architectures alongside researchers rather than relying on manual intervention. - **Production feedback loops:** Real-world traffic serves as direct input to retrain and update models iteratively.

---

[01:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=102s) The Golden Hour Example: Capacity-Aware Image Editing at Scale

A concrete manifestation of this model-product co-design is the "golden hour" editing feature. Mobile photographers frequently capture sunrises or sunsets seeking a viral aesthetic, but natural obstacles—such as an unwanted cloud obscuring the light or a stray tree in the frame—often detract from the shot. Standing up a generative media model that can cleanly erase or modify these elements on consumer devices requires balancing high fidelity with strict operational economics.

To serve editing requests at Meta's scale, models must be explicitly "capacity aware." Simply scaling up raw compute is unsustainable; models are engineered to refine user prompts and run inference heterogeneously across both GPU and CPU hardware. This hybrid compute approach drastically lowers per-inference serving costs while delivering near-instant, real-time responses to the end user. Mahajan explains his background and focus: “Last year I gave a talk at Scale when we were a startup building Sema Labs where we were improving the quality of video as we think about image, video, and all the deployments that come at scale. This year I'm going to be expanding my talk here at Metascale where we are actually finding the winning model recipe.”

- **Representative viral use case:** Natural camera capture edits, specifically removing obstructions like clouds or trees during golden hour shots. - **Capacity-aware serving:** Hybrid execution pipelines utilizing both CPUs and GPUs to minimize serving expense while hitting interactive latency thresholds. - **Speaker background:** Rushaan Mahajan leads AI media efforts on the Media Foundation team across Meta’s family of apps, previously co-founding Sema Labs.

---

[04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) Visual Judges: Combining Quantitative Baselines and Qualitative VLMs

To maintain media fidelity across both training and serving, teams deploy "visual judges." A visual judge is a secondary vision model acting as an automated critic, inspecting visual generations to score and refine the primary model's outputs. Visual evaluation spans both quantitative and qualitative signals:

These automated judges solve a major operational challenge: human researchers cannot monitor massive GPU training clusters around the clock. When physical media artifacts occur—such as temporal flicker across sequential video frames or structural smearing during rapid motion—an autonomous agent detects the metric deviation immediately. Instead of allowing expensive GPU clusters to sit idle or churn through failed runs while an engineer sleeps, agents intervene, apply adjustments, and resume training. Furthermore, Vision-Language Models (VLMs) fundamentally change qualitative assessment. As Mahajan succinctly highlights: “An LLM can read the logs and a VLM reads the pixels.” By looking directly at image outputs, VLMs spot issues like garbled text or unreadable brand logos, feeding corrections into the online loop to ensure subsequent checkpoints resolve the defects.

- **Offline evaluation:** Rigorous reference-based comparisons against baseline ground truth during training runs. - **Online visual judges:** Scoring real production traffic via reference-free quantitative signals and VLM-based qualitative critique. - **Artifact detection:** Real-time flagging of temporal flicker, motion blur, and structural smearing across video pipelines. - **Dashboard consolidation:** AI agents ingest raw monitoring dashboards, freeing researchers from having to manually review dozens of tracking interfaces.

---

[08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) Researchers and Sidekick Agents: Personalized Models and Continual Learning

Engineering teams are routinely bottlenecked by finite compute capacity and the manual labor required to tune media models. Treating agents as research sidekicks alleviates these resource constraints across the entire model lifecycle. Sidekick agents continuously sample live production traffic, diagnose geographic edge cases, and propose targeted fine-tuning strategies. For instance, network constraints, connection bandwidth, and visual aesthetic preferences vary widely by country; an agent can identify these regional divergences and orchestrate the creation of personalized models tailored to local conditions.

This dynamic underpins a broader framework of continual learning. In standard development workflows, product teams define high-level ambitions—such as delivering smoother video motion or crafting dedicated golden-hour filters—which previously required researchers to spend weeks scoping training recipes manually. With continual learning, teams translate these requirements into a formal "intent" that encapsulates explicit goals, evaluation criteria, inference queries per second (QPS), and hardware constraints. As Mahajan states: “That's the golden experience, right: an agent that is sampling prod traffic, it's flagging issues on researchers' behalf, and it's finding the winning model recipe.” The agent sidekick actively runs iterations within the designated training GPU budget, ensuring that the resulting architecture fits the exact inference capacity available in production.

- **Agent as sidekick:** 24/7 sampling of live production traffic to uncover edge cases and flag failures without human prompting. - **Geographic personalization:** Autonomous adaptation of media architectures to match regional network bandwidth and localized user preferences. - **Structured intent framework:** Formalizing high-level product goals into precise compute budgets, eval rubrics, QPS limits, and deployment milestones.

[11:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=663s) Researcher-Agent Collaboration and the Media Model Context

In modern machine learning co-design, the traditional operational boundaries between training and inference are dissolving. Autonomous agents continuously monitor inference signals from production systems and feed those real-world learnings directly back into model retraining. Rushaan Mahajan emphasizes that this shift resolves longstanding bottlenecks where researchers were routinely blocked by slow feedback loops and complex product trade-offs. Rather than leaving the human researcher overwhelmed by balancing model quality, virality, and resource constraints, the agent acts as an autonomous collaborator that drives continuous optimization.

A foundational element of this partnership is establishing shared context through a single, structured morning report. Mahajan explains that after an agent runs experiments overnight, the researcher needs immediate clarity on what progress was made, spanning both quantitative metrics and qualitative evaluations:

> “Continual learning is where agents are continuously understanding inference signals and driving that into model training.” — Rushaan Mahajan

> “This report is grounded without hallucinations and it makes sure that the media model context is shared.” — Rushaan Mahajan

To maintain scientific rigor and prevent human frustration, every claim generated by an agent must be fully reproducible and backed by verifiable data rather than generative hallucinations. The morning report explicitly highlights gating criteria: whether latency has drifted higher, how the visual judge scored perceptual improvements, and whether users will noticeably prefer the new outputs. Armed with this grounded context, researchers begin their workdays unblocked, issuing precise intents and launching focused training batches alongside their autonomous agent partners.

```mermaid flowchart LR accTitle: Researcher and Agent Continual Learning Loop accDescr: Diagram illustrating how inference signals, autonomous overnight agents, structured reports, and researcher direction form a closed loop. subgraph Overnight["Overnight Agent Loop"] A["Inference Signals"] --> B["Autonomous Agent Optimization"] B --> C["Visual Judge & Metric Gates"] end subgraph Morning["Morning Researcher Handoff"] C --> D["Grounded Daily Report"] D --> E["Researcher Direction & Intents"] E --> B end ```

[14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) Finding the Winning Recipe and the Media Harness

The primary objective of this researcher-agent dynamic is discovering the optimal "winning recipe" for media models under tight production constraints. Autonomous agents evaluate available cluster capacity, parse live production signals, and make real-time decisions to prune doomed training checkpoints early while starting promising new hyperparameter sweeps. Mahajan notes that by handling these adjustments autonomously, agents double training velocity and prevent common engineering bottlenecks—such as underutilized, idling GPUs or models drifting to 200 milliseconds when downstream product requirements demand a strict 100-millisecond latency envelope.

This continuous feedback process operates within an end-to-end framework Mahajan calls the "media harness." The harness embeds visual judges directly into both the training loop and deployment inference pipeline, replacing traditional, narrow software loss metrics with rich qualitative assessments:

> “As a model trains and gets ready for production and delivering that final checkpoint, we need to make sure a visual judge is always in that training and deployment inference loop at the same time.” — Rushaan Mahajan

By allowing agents to write code diffs, submit automated bug fixes, and interpret subjective media quality at scale, engineering teams can support the long tail of user feature requests without running out of compute capacity.

- Autonomous agents double media model training velocity (2x faster) by pruning underperforming runs early and tuning hyperparameters. - Latency gating prevents models from exceeding strict production limits (e.g., maintaining a required 100 ms cap when unoptimized runs hit 200 ms). - Automated code diff generation enables agents to patch training pipelines directly without human engineering intervention.

[16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) Video Frame Interpolation at Meta Scale

To ground these systems in production reality, Mahajan highlights Meta's deployment of deep learning models for video frame interpolation. Globally, hundreds of millions of user videos suffer from low capture frame rates on budget hardware, producing stuttering, degraded motion playback. While legacy architectures relied on heuristic software algorithms to interpolate motion, Meta replaced these pipelines with neural networks trained jointly by researchers and agents:

> “Between every frame, we are able to generate and insert a frame that keeps the temporal context.” — Rushaan Mahajan

> “I like to say this is the worst it's ever going to be. It's going to constantly improve.” — Rushaan Mahajan

Deploying across Meta's Family of Apps provides a massive stream of real-world production data that feeds back into model training. The system reliably ingests lower-frame-rate content—such as 15 fps or 20 fps video—and doubles the frame rate while preserving temporal consistency across diverse scenes. Mahajan notes a cultural paradigm shift within Meta’s engineering teams: shipping a model checkpoint to production is no longer treated as the conclusion of a project. Instead, the initial deployment serves as the starting line, while autonomous agents continually ingest production edge cases to train and deploy superior successor checkpoints in an ongoing cycle.

- Replaced legacy algorithmic motion smoothing with deep learning video frame interpolation models that insert synthesized frames while maintaining temporal coherence. - Optimized across diverse mobile inputs, successfully doubling frame rates on constrained 15 fps and 20 fps mobile video captures. - Converted product delivery from static one-and-done releases into continuous, rolling checkpoint deployments driven by production telemetry across Meta's Family of Apps.

[19:24](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1164s) Guardrailing Agents for Step-Change Product Impacts

While agent autonomy dramatically accelerates development, unconstrained agents introduce serious operational risks if left without rigorous guardrails. Mahajan warns against letting agents deploy continuous, microscopic modifications—such as pushing daily updates that yield negligible 0.0001% metric improvements—which introduce systemic instability without tangible consumer value. Instead, agents are sandboxed to accumulate learnings, retrain models, and deliver noticeable step changes in visual fidelity so users immediately feel the upgrade in product capability.

Strict sandboxing also enforces hard operational constraints across infrastructure budgets:

> “One thing we don't like to do is make sure the agents can run wild and actually create a problem in terms of exceeding the GPU budget.” — Rushaan Mahajan

Agents operate within explicit sandbox boundaries, preventing them from exhausting allocated GPU compute quotas or missing downstream latency targets. By reading every training run, evaluation score, and runtime metric against strict gates, the system maintains a predictable deployment cadence, allowing teams to deliver high-quality, delightful media experiences while keeping computational costs under control.

- Strict infrastructure sandboxes prevent autonomous agents from overrunning dedicated GPU cluster budgets during hyperparameter sweeps. - Agents are tuned to target noticeable, step-change improvements in consumer experience rather than landing negligible, fractional gains. - The media harness monitors all evaluations and performance metrics simultaneously to preserve system safety and efficiency.

## Notable quotes > [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) An LLM can read the logs and a VLM reads the pixels. — Rushaan Mahajan

> [11:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=663s) Continual learning is where agents are continuously understanding inference signals and driving that into model training. — Rushaan Mahajan

> [14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) As a model trains and gets ready for production and delivering that final checkpoint, we need to make sure a visual judge is always in that training and deployment inference loop at the same time. — Rushaan Mahajan

## Mentioned Rushaan Mahajan, Meta, Scale, Sema Labs, Media Foundation team

[Source](https://www.youtube.com/watch?v=WHwOVk-kXfM)

中文简报

Meta 利用自主智能体将多媒体模型训练速度提升一倍颠覆传统“人在回路”工作流，使自主智能体能够监控实时推理信号、及早终止失败的 GPU 任务，并执行严格的生产延迟限制。 Rushaan Mahajan 详细阐述了 Meta 如何通过将传统的“人在回路”（human-in-the-loop）工作流转变为自主的“智能体在回路”（agents-in-the-loop）持续飞轮，从而加速多媒体 AI 的开发。通过在训练和推理过程中部署辅助视觉语言模型充当自动化“视觉裁判”，自主辅助智能体能够在严格的算力和延迟预算下捕捉视觉缺陷、调优超参数并优化模型。该架构支撑了诸如感知容量的黄金时刻照片编辑以及深度学习视频插帧等大规模应用，使生产检查点成为持续学习的启动平台，而非静态的一次性发布。 Meta 正在将传统的“人在回路”工作流转变为“智能体在回路”框架，由自主智能体在整个训练和部署过程中监测遥测数据、及早裁剪失败的运行任务并迭代多媒体模型。 (0:05) 自主辅助智能体通过自主调优超参数、生成代码 diff 以及执行诸如 100 毫秒上限等生产延迟限制，使模型训练速度翻倍。 (14:03) Meta 旗下各应用生产环境中的视频插帧模型使 15–20 fps 低端采集视频的帧率翻倍，并将初次部署视为由智能体驱动的持续再训练起点。 (16:42) 对智能体进行沙箱化管理可防止 GPU 算力预算超支，并抑制杂乱的微小更新，从而推动视觉保真度实现肉眼可见的跨越式提升。 (19:24)

Reading summary

Rushaan Mahajan from Meta argues that scaling consumer media AI requires inverting conventional human-in-the-loop oversight into an 'agents-in-the-loop' flywheel, where autonomous systems co-design training sweeps, monitor live production inference, and enforce strict hardware budgets. Inverting Oversight With Agentic Feedback Loops (0:05) Rather than relying on human operators to manually approve model iterations, modern media architectures deploy autonomous agent sidekicks directly into both offline training clusters and production serving pipelines. Offline model training on GPU fleets and real-world deployment continuously feed into one another: agents monitor telemetry, evaluate checkpoints, and route live engagement data straight back into the retraining cycle to accelerate feature velocity without waiting for manual intervention. Multimodal Visual Judges for Continuous Evaluation (4:14) To monitor large-scale training and inference without continuous human supervision, teams use automated visual judges spanning quantitative metrics and Vision-Language Models (VLMs). While offline training uses reference-based comparisons against sharp baselines, production inference relies on reference-free scoring alongside VLMs that inspect outputs directly for semantic alignment, brand logo legibility, and text clarity. When video artifacts like temporal flicker or structural smearing emerge, autonomous agents detect the regression, adjust parameters, and resume cluster execution immediately, preventing GPUs from idling overnight. Capacity-Aware Inference and Intent Frameworks (1:42) Deploying generative features across Meta's Family of Apps—such as 'golden hour' camera capture edits that erase clouds or trees—requires explicit capacity awareness. Because unconstrained scaling of compute is unsustainable, inference workloads run heterogeneously across hybrid CPU and GPU pipelines to drive down per-request costs while maintaining real-time responsiveness. High-level feature requests are translated into formal 'intents' that define target queries per second (QPS), latency limits, and hardware budgets, which agents then satisfy by tailoring personalized models to regional network constraints. Autonomous Optimization and Frame Interpolation Deployments (14:03) Through what Mahajan calls a 'media harness,' agents parse production telemetry, prune failing training sweeps early, generate automated code diffs, and enforce latency caps (such as holding models to a 100-millisecond threshold when unoptimized runs reach 200 milliseconds). Mahajan reports this doubles media training velocity. In production, this setup supports neural video frame interpolation across Meta's Family of Apps, doubling frame rates on low-end 15 fps and 20 fps captures while preserving temporal context. Under this model, initial checkpoint deployment acts as the start of continuous edge-case ingestion rather than the end of development. Guardrailing Agents Against Micro-Gains and Overruns (19:24) Unconstrained agent autonomy introduces operational hazards, such as exceeding dedicated GPU quotas during hyperparameter sweeps or destabilizing systems with negligible 0.0001% metric adjustments. Engineering guardrails sandbox autonomous sidekicks so that every retraining batch targets discernible step-change improvements in consumer experience. To maintain scientific reproducibility, agents hand off structured daily morning reports grounded in verifiable metrics and judge scores, keeping researchers and autonomous agents aligned without hallucinated claims. The framework shifts media product delivery from discrete, manual releases into an autonomous, continuous improvement cycle. However, its effectiveness depends on enforcing hard sandboxes to ensure agents pursue perceptible user upgrades rather than destabilizing pipelines or exhausting finite GPU allocations.

精炼阅读摘要

Meta媒体基础团队负责人Rushaan Mahajan指出，现代多媒体AI基础设施正在反转传统的“人机协同”（Human-in-the-Loop）架构，转向由自主智能体全程主导训练调优、生产推理与视觉评估的“智能体协同”（Agents-in-the-Loop）飞轮模式。智能体反转生产流水线 (0:05) 传统多媒体模型研发中，人工操作员通常负责审核并门控模型的每一步操作，导致反馈周期冗长且难以兼顾算力与产品迭代速度。Meta将离线训练与线上部署闭环打通，引入自主智能体作为研究员的“副驾驶”。智能体能够24小时不间断分析线上流量指标、检测物理介质伪影，并自主调整训练超参数与模型架构。这种架构将用户真实的交互流量与运维遥测数据直接作为训练集的重训输入，从而打破训练与推理之间的工程壁垒。算力感知与视觉评估体系 (1:42) 在大规模应用场景中，单纯堆叠算力不可持续。以手机摄影的“黄金时刻”（Golden Hour）擦除杂物功能为例，模型必须具备“算力感知”能力，通过CPU与GPU异构计算流水线，在大幅压缩单位推理成本的同时满足实时低延迟交互需求。为了在离线训练和在线服务中量化多媒体质量，Meta构建了“视觉裁判”（Visual Judges）系统。该系统在离线阶段采用有参考基准的量化指标对比真实像素，在线上阶段依托无参考指标与视觉语言模型（VLM）直接读取像素，实时诊断时间闪烁、结构模糊以及品牌标识破损等语义级缺陷。研发协同与媒体治理框架 (11:03) 为了让研究员从繁琐的手动监控中解脱，智能体在夜间自主运行实验、提早剪枝低效检查点并生成无幻觉的早间量化报告。Mahajan指出，自主智能体使媒体模型的训练速度翻倍（2倍），且能直接生成代码补丁修复训练管线，严格守住推理延迟门槛（例如将未优化的200毫秒延迟压至产品规定的100毫秒上限）。在被称作“媒体治理框架”（Media Harness）的闭环中，产品团队的高阶诉求被结构化为包含明确计算预算、QPS上限与硬件约束的“意图”，由智能体在既定GPU配额内寻找最佳模型配方，甚至能针对不同地区的网络带宽与审美偏好实现本地化模型个性化定制。规模化插帧落地与安全沙箱 (16:42) 在Meta旗下应用家族中，数百万人拍摄的低帧率视频（如15 fps或20 fps）通过深度学习视频插帧模型实现了帧率翻倍，并在保持时间上下文一致性的同时替代了传统启发式平滑算法。该团队将产品发布视角从一次性交付转变为滚动检查点持续迭代。与此同时，Meta对智能体施加了严格的基础设施沙箱限制。Mahajan强调，必须防止智能体无节制消耗GPU集群预算，且系统严禁为了0.0001%的微小指标增益频繁发布琐碎更新，必须确保每次模型迭代都能为终端用户带来感知明确的阶跃式画质提升。该方案展示了智能体驱动下多媒体模型研发的生产力跃升，但系统仍高度依赖严格的沙箱门控来防止GPU资源超支；如何在追求阶跃式产品提升与保持自动化迭代稳定性之间维持长期平衡，仍是团队必须持续应对的工程课题。

Detailed notes — zh

## Background 本内容由 YouTube 频道 @Scale 于 2026 年 10 月 30 日发布。视频源页面未提供详细简介，核心记录了 Meta 媒体基础模型团队的 Rushaan Mahajan 所作的技术演讲。 ## TL;DR Meta 的 Rushaan Mahajan 深入探讨了如何通过“智能体在回路”（Agents-in-the-loop）架构打破传统研发瓶颈，实现媒体基础模型与终端产品的深度协同设计。通过引入视觉裁判（Visual Judges）、意图定义语言（Intent DSL）以及多媒体测试基架（Media Harness），Agent 能够 24/7 全天候监控指标并优化训练，将迭代速度提升 2 倍。演讲结合 Meta 视频插帧模型的落地案例，展示了从低帧率优化到生产数据反哺训练的持续学习闭环，强调通过严密的安全护栏与算力约束，为用户打造代际跃迁级的产品体验。 ## Atmosphere & style 整场演讲呈现出极具前瞻性且务实笃定的工程极客风格，节奏紧凑而逻辑严密。Rushaan 展现出资深一线架构师的敏锐与自信，将深夜算力浪费、抗拒 Agent 幻觉等研发痛点娓娓道来，既有对自动化未来的兴奋与野心，又保持着对算力预算与代际质量跃升的冷静克制。聆听过程中能直观感受到超大规模工业级 AI 研发的工业化美感，充满着令人振奋的创新张力。 ## Bulletin - [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) Meta 提出“智能体在回路”（Agents-in-the-loop）新范式，将模型训练生命周期与线上推理部署深度打通，利用 Agent 24/7 监控与参数调优，使训练迭代速度提升 2 倍。 - [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) 引入视觉裁判（Visual Judges）作为二级视觉模型，结合 LLM 解析系统日志与 VLM 审视像素级瑕疵，构建兼顾定量与定性的自动化闭环评估体系。 - [08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) 采用“意图定义语言”（Intent DSL）将宏观业务诉求结构化为评估目标、交付时间线及 GPU 算力预算，前置约束高 QPS 下的推理算力上限。 - [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) 以 Meta 实拍视频插帧模型为例，模型不仅能对 15/20 fps 低帧率视频实现倍帧，更确立了模型上线即在后台持续迭代新 Checkpoint 的“持续学习”研发文化。 ## Key takeaways - [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) AI 媒体模型需与终端产品进行深度协同设计（co-design），以平衡极低时延、生成质量与交付速率等多重业务诉求。 - [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) 从传统的“人在回路”（Human-in-the-loop）逆向转向“智能体在回路”（Agents-in-the-loop），由 Agent 担任全天候观察者与持续迭代者。 - [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) 打造“容量感知”（capacity-aware）架构，支持跨 GPU 与 CPU 混合运行推理，在削减计算成本的同时保障毫秒级实时交互。 - [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) 视觉裁判（Visual Judges）作为二级视觉模型，填补了生产环境中缺乏基准图时的“无参考”定性评估空白。 - [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) LLM 负责读取调度日志，VLM 负责深入像素层审视时序闪烁、运动模糊及 Logo 锐利度等细节，分工明确。 - [08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) 引入结构化意图定义语言（Intent DSL），将业务需求规范为核心评估指标、时间线、GPU 训练预算与线上推理约束四大维度。 - [08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) Agent 能够对全球不同区域的生产流量进行自适应抽样，识别网络带宽与审美偏好差异，主动规划轻量化或个性化模型。 - [11:49](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=709s) Agent 在夜间自主迭代后，需向研究员生成标准化、抗幻觉的晨间报告，确保顶层指标与定性评估具有坚实的可复现性。 - [14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) 多媒体测试基架（Media Harness）赋能 Agent 实时研判并在训练发散时果断止损，自动生成 diff 修复补丁，实现 2 倍的迭代加速。 - [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) Meta 视频插帧模型针对全球低端设备拍摄的抽搐卡顿问题，在相邻两帧间精准生成插入帧，使低帧率视频流畅度翻倍。 - [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) 研发文化发生蜕变：模型推向生产环境不再意味着项目结项，而是依靠生产数据回传由 Agent 持续产出更优质的 Checkpoint。 - [19:24](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1164s) 为 Agent 构建严格的沙箱护栏机制，防范其超出算力预算，追求带来显著质感飞跃的代际跃迁（step change）而非微小代码改动。 ## Detailed walkthrough ### [00:05](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=5s) AI 媒体模型的飞轮与模型/产品协同设计在 AI 与多媒体的交汇处，飞轮效应正在以前所未有的速度不断加速运转。整个体系中最引人入胜的环节，莫过于将基础模型与终端产品进行协同设计（co-design），以此来大幅提升功能迭代与交付的速率（feature velocity）。业务产品团队在研发过程中往往面临着恒定的多重诉求：追求极低的时延（lower latency）、要求更高的模型质量（model quality），同时还要创造出引人入胜的极致用户体验。为了支撑起这种标杆级的产品体验，全方位加速 AI 媒体模型的生命周期已成为重中之重。在传统的机器学习范式中，行业普遍依赖“人在回路”（Human-in-the-loop）模式，由人工对模型的每一步关键动作或决策进行审查和把关。然而，Rushaan Mahajan 提出了一个逆向的思维框架： > “We know about human in the loop where a person gates every action of the model. I like to think about it as where is agents in the loop inverting this.” 在“智能体在回路”（Agents-in-the-loop）的新架构下，Agent 扮演了不知疲倦的观察者、行动者与持续迭代者，尤其深度嵌入在模型的训练生命周期中。整个系统将模型训练生命周期与线上推理及部署生命周期深度绑定。通过打通线上产品部署到线下训练的端到端反馈闭环，评估（eval）与训练实现了紧密协同。Agent 与算法研究员并肩协作，持续调优训练超参数与模型架构；一旦新版本推向生产环境，真实的线上流量与用户交互数据又会反哺并放大整个飞轮的势能。当模型训练得当且推理时延被大幅压降时，用户与产品的互动意愿便会迎来爆发式增长。以图像编辑中的“黄金时刻”（golden hour）摄影风格为例，许多用户渴望获得具有病毒式传播潜力的视觉效果——无论是在日出还是日落时分拍摄的手机原生照片，用户既希望通过提示词直接生成理想的光影氛围，也希望对自然实拍图进行精细修饰。这种图像编辑功能是一大高频刚需场景：用户捕捉了一张构图极佳的照片，但画面中可能恰好飘来一朵遮挡主体的乌云，或者背景里突兀地出现了一棵多余的树木，他们渴望一键移除这些瑕疵。生产环境的真实数据能够精确揭示用户的交互痛点和修改意图。然而，要让此类前沿模型真正落地，核心在于打造具备“容量感知”（capacity-aware）特性的架构，使其能够兼顾极低成本与超大规模部署。在具体工程实现中，系统首先接收并改写精炼用户的输入提示词以提升生成质量，同时模型被设计为能够无缝跨 GPU 与 CPU 混合运行。这种算力分摊机制显著削减了单次推理的计算成本，使用户在输入修改指令后能够瞬间看到画面更新，真正达成了毫秒级响应的实时体验，而这一切流畅体验的坚实后盾，正是底层严密的模型训练生命周期。 ```mermaid flowchart LR accTitle: AI Media Model Flywheel accDescr: Training, inference, and production data feedback loop with agents in the loop. subgraph Offline["线下训练生命周期"] A["研究员与 Agent 协作"] --> B["训练超参数/架构优化"] B --> C["离线评估 (Offline Eval)"] end subgraph Online["线上生产与部署"] D["生产环境模型部署"] --> E["容量感知推理 (GPU/CPU)"] E --> F["用户实时编辑体验"] end C -->|"模型交付上线"| D F -->|"生产交互数据反馈"| A ``` - **演讲者背景**：Rushaan Mahajan，目前在 Meta 负责媒体基础模型团队（Media Foundation Team）的 AI 媒体前沿研发工作，业务范围横跨 Meta 旗下的整个应用家族（Family of Apps）。在此之前，他曾在初创公司 Cyma Labs 致力于提升规模化场景下的视频生成质量，并在去年的 Scale 大会上发表演讲；本次演讲则重点探讨在 Meta 这种超大规模（Metascale）场景下如何寻找制胜模型秘方（winning model recipe）。 - **演讲核心议程**：重点覆盖生产环境中的视觉裁判（Visual Judges）、多媒体内容的持续优化与演进，以及 AI Agent 如何从单纯的辅助工具升格为算法研究员的默契队友。 --- ### [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) 视觉裁判（Visual Judges）与多模态评估体系视觉裁判（Visual Judges）是一种全新引入的模型形态。Rushaan Mahajan 将其定义为一种二级视觉模型（secondary vision model），它专门充当反馈回路的核心探针，其职责在于仔细审视并评估主模型生成的视觉产出，进而指导并精炼主模型的输出质量。在量化评估基线方面，多媒体指标通常可以划分为“基于参考”（reference-based）与“无参考”（reference-free）两类。定量指标具备计算开销低、反馈速度极快的天然优势，构成了模型质检最基本也是最关键的信号源。在基于参考的评估中，系统以原始像素作为客观基准线（例如对比原始高清晰度画面与最终输出画面），以此来校验生成结果是否忠实反映了用户预期，并衡量画面质量相对基线的偏移程度。但在真实的生产环境中，绝大多数场景并不存在天然的原始参考基准，这便引出了无参考指标的必要性——在缺乏对照图的情况下，系统必须建立一套标准（例如 1 到 5 分，或 1 到 100 分的评分量表），自主评定一次图像生成或局部擦除编辑是否达到高水准。视觉裁判正是填补定性评估空白的关键角色，它不仅能自动执行高精度字幕标注（captioning），还能在定性维度上直接纠正并精细打磨主模型的输出缺陷。面对复杂的生产与训练环境，Agent 必须以 24/7 全天候运转的方式，不间断地同时监控定量与定性指标。在大规模 GPU 集群上训练基础媒体模型时，保持全量算力的高效运转是极为昂贵且严肃的任务。过去经常发生这样的情况：深夜由于某些边界异常，模型训练在特定阶段发生静默崩溃或发散，而负责的研究员正在熟睡，导致庞大的 GPU 集群长时间处于空转闲置状态，白白浪费数额巨大的算力成本。而在 Agent 实时巡检机制下，Agent 会密切跟踪各项媒体质量指标的变化趋势，一旦发现训练劣化趋势便立即触发干预或参数调整。 ```mermaid flowchart TD accTitle: Dual Visual Evaluation Architecture accDescr: Offline eval and online visual judge monitored 24/7 by agents and VLMs. subgraph Metrics["多维度评估信号"] Qn["定量指标 (Quantitative) - 便宜、快速、低开销 - 基于参考 / 无参考"] Ql["定性指标 (Qualitative) - 视觉裁判 (二级视觉模型) - 图像描述与结构打磨"] end subgraph Lifecycle["监控与反馈闭环"] M1["训练阶段：离线评估 (Offline Eval) 监控时序闪烁、运动模糊等"] M2["生产阶段：在线视觉裁判 (Online Judge) 监控线上真实流量与品牌呈现"] end Qn & Ql --> Lifecycle Lifecycle --> Ag["24/7 巡检 Agent"] Ag -->|"LLM 读日志 + VLM 读像素"| FB["自动调优 / 交付新 Checkpoint"] ``` 在多媒体模型的训练过程中，存在诸如“时序闪烁”（temporal flicker）或快速运动画面下的“结构性涂抹”（structural smearing）等典型且棘手的肉眼可见缺陷。为了彻底解决这类问题，Meta 在交付流水线的每一个阶段都部署了双重质检机制： 1. **线下阶段（Offline Eval）**：通过内置的定量与定性指标，持续对训练过程中的每个 Checkpoint 进行自动化打分； 2. **线上阶段（Online Visual Judge）**：在生产流量入口部署视觉裁判模型，结合定量特征提取，对海量用户请求的输出进行实时判定。在监控形态上，过去算法研究员需要肉眼紧盯几十个复杂的监控仪表盘，而如今这类仪表盘的数量正在急剧精简——因为能够“阅读仪表盘”的 Agent 已经接管了底层数据流，并直接提炼出最核心的决策信号汇报给研究员。更具突破性的是，大语言模型（LLM）与视觉语言模型（VLM）在监控中展现出了截然不同的专长分工： > “An LLM can read the logs and a VLM reads the pixels. This is the difference.” 大型语言模型擅长从系统报错、节点调度日志中解析故障根因，而视觉语言模型则直接深入像素层进行语义审视。例如，VLM 能够直接判定画面中渲染出的文本字符是否自洽连贯，或者画面的主体边缘是否足够锐利、商业品牌 Logo 的细节是否高清无损。如果在生产端发现特定品牌的 Logo 生成质量欠佳，这种定性的像素级负反馈会被立即打包回传给训练端，并在产线进行专项增强与微调。随着优化后的新 Checkpoint 被重新热更新交付至生产环境，所有线上输出的 Logo 模糊问题便迎刃而解。这种由生产环境流量中的在线裁判所驱动的“在线反馈回路”（online feedback loop），彻底打破了以往模型上线即固化的僵局，使模型即使身处生产阶段也能与业务共同持续演进。 --- ### [08:29](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=509s) Agent 作为研究员副手与“意图定义”（Intent DSL）在大规模工程体系中，算法工程师常常受制于算力瓶颈与繁杂的模型优化琐事，甚至在最初的概念构思与方案探索阶段就陷入效率泥潭。将 Agent 深度引入日常研发、扮演研究员的贴身副手（sidekick），正在彻底重塑多媒体模型的全生命周期。一个充当副手的 Agent 能够自主、持续地对生产流量进行自适应抽样分析。这种线上探针的能力极为广泛且具针对性：Agent 可以对不同国家或地区的实际请求样本进行聚类解析。例如，在某些特定国家，网络带宽和连接质量较为受限，用户在审美上对图像色彩和画质又有独特的偏好诉求，Agent 便能敏锐识别出这些地域特征，进而建议乃至自动构建定制化的轻量化个性化模型。它不仅能代表研究员 24 小时全天候巡视生产流量、快速定位长尾不良案例（bad cases），还能在海量实验组合中探寻出胜率最高的模型超参方案。这套闭环从根本上将创作、生成与终端消费有机融为一体。这一切能力的底层基石在于“持续学习”（continual learning）机制。在实际的产品协作流程中，业务产品团队提出的初始诉求通常是高度宏观的——例如“我们想要一个能够适配日落黄金时刻的模型”，或者“我们希望视频生成的动态画面更加丝滑流畅”。然而，这种高度抽象的商业诉求无法直接指导底层的训练算力分配。为了填平这一鸿沟，演讲者引入了一个全新的结构化“意图定义语言”框架（Intent / Intent DSL）。一份严谨的意图定义需要明确规范核心评估目标（eval goal）、划定训练所允许消耗的 GPU 算力预算（compute budget）、预设可接受的模型迭代轮次，并与产品团队预期的上线时间线（timeline）深度对齐。在过去，仅将这样一个产品意图转化为具体的实验方案，算法研究员就需要耗费数周时间进行人工推演和算力核算。如今，研究团队与专属 Agent 紧密协同，共同快速解析并迭代这份意图定义：在设计阶段，Agent 会实时校验各项训练策略，确保绝不超出既定的 GPU 算力预算红线；而在面向生产端时，面对动辄每秒成千上万的高并发查询请求（QPS）和庞大的线上推理洪峰，系统能够前置计算出推理侧所能承载的算力极限，并将这些严格的生产环境硬约束直接反哺并锚定在早期的模型结构设计中。 ```mermaid flowchart TD accTitle: Intent DSL Workflow accDescr: Turning high-level product goals into constrained training budgets via Intent DSL. P["业务团队宏观需求 - 黄金时刻画面滤镜 - 视频运动平滑度提升"] --> I["意图定义规范 (Intent DSL) - 核心评估指标 (Eval Goal) - 交付时间线 (Timeline) - 离线训练 GPU 预算上限"] subgraph AgentCollab["研究员 + Agent 协同闭环"] I --> D["算力与超参自动化探索"] D --> E["前置约束模拟 考虑线上高 QPS / 推理算力上限"] E --> F["训练架构与部署参数收敛"] end F --> S["生产级模型交付与持续监控"] ``` - **生产流量抽样与个性化落地**：Agent 能够在生产环境中执行自动化流量分流与抽样，针对不同区域的带宽限制与画质偏好差异，主动规划针对性的轻量化或个性化模型版本。 - **意图规范（Intent DSL）的四大核心维度**： - 核心评估指标（Evaluation Goal） - 研发交付时间表（Timeline） - 离线训练算力预算（GPU Budget） - 生产端推理算力约束（Inference Compute / QPS Constraints） ### [11:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=663s) 训练与推理的协同设计与持续学习在模型训练的联合设计（co-design）中，传统的界限正被彻底打破，训练与推理不再是割裂的两个阶段，而是紧密结合协同运行。持续学习（continual learning）的核心在于让 Agent 持续捕捉和理解推理端的线上信号，并将这些真实反馈反哺回模型的训练闭环中。Rushaan Mahajan 强调，必须将这种“意图驱动”视作颠覆整个行业的关键一步。以往研究人员在推进多媒体模型时常受到多重瓶颈的掣肘。随着媒体产品对传播力和病毒式扩散（viral）的要求越来越高，整个体验中涉及的权衡（trade-offs）也呈指数级上升。当研究员与 Agent 共同面对并权衡这些复杂因素时，不仅能迅速扫除阻碍，更能打造出颠覆性的产品体验。 ```mermaid flowchart LR accTitle: 训练与推理协同闭环 accDescr: 展示推理信号经由Agent反哺模型训练，并由视觉评估器监督的闭环结构。 subgraph Production ["生产端 / 推理"] A["生产推理请求"] --> B["推理信号与用户反馈"] end subgraph AgentLoop ["Agent 持续学习系统"] B --> C["Agent 意图分析"] C --> D["超参数微调 & 训练作业分发"] end subgraph Training ["训练端"] D --> E["模型训练与检查点生成"] E --> F["视觉评估器 (Visual Judge)"] F --> A end ``` - 训练与推理的协同打破了传统 AI 研发壁垒，推理侧信号成为训练侧的核心输入。 - 研发人员不再需要孤军奋战，通过将系统权衡与设计意图传递给 Agent，大幅加速工程解阻。 --- ### [11:49](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=709s) 研究员与 Agent 的协作：晨间标准化报告与抗幻觉机制研究员与 Agent 之间最引人入胜的协作细节在于“多媒体模型上下文”（media model context），也就是所谓的“一份报告的清晰度”（clarity in one report）。Agent 在夜间自主执行任务，当研究人员第二天清晨醒来时，面临的最关键问题就是：如何迅速掌握 Agent 在昨夜完成的所有改进？Rushaan Mahajan 坦言自己经历过无数次这样的清晨，而这种日复一日的积累在数周内会产生巨大的复利效应。为了让 Agent 能向研究人员清晰传达惊人的进展，建立标准化的报告风格至关重要。虽然每位研究员偏好的展现形式各异，但核心顶层指标（topline features）必须统一呈现：既要包含定量评估（quantitative eval）与定性评估（qualitative eval），也要明确系统硬性指标（gates）的变化——新模型的延迟是否上升？视觉质量是否获得实质提升？来自视觉裁判（visual judge）的细分反馈是否证实新输出更受用户欢迎？ ```mermaid flowchart TD accTitle: Agent 标准化早报生成与交互流程 accDescr: 展示夜间Agent自主迭代后，生成抗幻觉报告并协助研究员启动新一轮训练的流程。 A["夜间自主运行: 实验与评估"] --> B["生成标准化报告"] subgraph ReportContent ["报告核心模块"] B --> C["定量指标 (Latency / Gates)"] B --> D["定性评估 (Visual Judge 评分)"] B --> E["可复现性验证 (严防幻觉)"] end C & D & E --> F["晨间交付研究员"] F --> G["研究员下达新指令: 启动新训练任务"] ``` 最核心的原则是杜绝虚假宣称：必须确保每一项结论都具备可复现性，并拥有扎实的定量数据支撑。正如 Rushaan Mahajan 所指出的： > “因为研究人员最深恶痛绝的一件事，就是 Agent 产生幻觉。”（“cuz the one thing that researchers don't like is when an agent has made hallucinations.”）这份基于事实、毫无幻觉的报告让媒体模型的上下文得以完整对齐。研究员每天清晨可以胸有成竹地下达指令，规划启动多批新的训练作业。Agent 与研究人员并肩作战，将时间花在加速创新上，而不是浪费在排查幻觉错误中，这正是当下 AI 媒体生成领域顶级模型配方（model recipes）不断涌现的原因。 --- ### [14:03](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=843s) 自动化配方搜寻与多媒体测试基架（Media Harness）寻找制胜的多媒体模型配方，被视作人机协作的“黄金体验”。Agent 会全天候监控底层集群算力（capacity）、新接入的产品功能需求以及真实生产环境传回的数据。当检测到某些训练检查点（checkpoints）注定无法达到上线标准时，Agent 会果断中止这些训练任务，并根据反馈调整超参数，主动开启更有潜力的新训练任务。通过将定量和定性反馈深度结合，Agent 成功让训练迭代速度提升了 2 倍（2x faster），促成了更多产品的快速发布。如果缺乏 Agent 的协同，研发团队就会陷入常见的工程泥潭：GPU 算力白白空转，或者模型延迟飙到了 200 毫秒，而产品团队给出的硬性指标却是 100 毫秒以内。随着大规模部署的推进，产品需求愈发苛刻，Agent 成为消化这些复杂约束的必备工具。 Rushaan Mahajan 将这套闭环系统概括为“多媒体测试基架”（media harness）： | 维度 | 传统训练流程 | 多媒体测试基架（Media Harness + Agent） | |:--- |:--- |:--- | | **评估方式** | 单一的定量工程指标 / 基础软件算法 | 定量指标 + 视觉评估器（Visual Judge）的定性感知反馈 | | **检查点筛选** | 人工定时检查，算力耗尽后才被动评估 | Agent 实时研判，提前终止劣质检查点，释放 GPU 资源 | | **缺陷修复** | 研发人员手动编写补丁排查 | Agent 自动编写 diff 与修复方案，并同步上下文 | | **长尾需求交付** | 难以顾及海量碎片化长尾功能 | 在算力约束下，通过人机协同快速覆盖各类产品功能与长尾请求 | 在这套机制下，视觉评估器被无缝嵌入到训练和部署推理的统一闭环中，Agent 自动产出代码补丁（diffs），确保训练流与生产评估时刻维持在最佳状态，使得长尾场景下的海量用户需求得以高效落地。 --- ### [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) 核心实战案例：Meta 视频插帧模型的落地与迭代为了将理论落到实处，Rushaan Mahajan 深入剖析了 Meta 内部的一个经典应用场景：视频抽搐与低帧率优化。在世界范围内，由于很多用户使用低端手机拍摄，视频原始采集质量不高，画面运动时常伴有明显的卡顿（stuttery）。过去这类问题主要依赖传统软件算法解决，但到了 2026 年，团队决定联合研究员与 Agent，为产品团队超高速训练一套定制的视频插帧（video frame interpolation）深度学习模型。该模型的核心任务是在视频相邻两帧之间，准确预测并生成插入帧，同时严密维持画面的时间连贯性（temporal context）。其工程复杂度极高，不仅要设计精妙的损失函数，还要适应五花八门的输入规格——例如针对 15 fps 和 20 fps 的输入视频，模型必须采取不同的处理策略。借助 Meta 规模化平台（Metascale）的优势，团队能够直接利用真实生产数据来推动模型再训练：研发出的模型在处理低帧率视频时成功实现帧率翻倍。用户端数据反馈表明，低帧率场景下的效果立竿见影；而在应对相对较高帧率的输入时，团队则在此基础上引入更多特征增强，从而在实现帧率翻倍的同时带来惊艳的视觉观感。 ```mermaid flowchart LR accTitle: 视频插帧模型生产闭环 accDescr: 展示Meta视频插帧模型在低帧率视频输入下的倍帧处理及上线后的持续升级路径。 A["用户低帧率视频 (如 15/20 fps)"] --> B["插帧模型: 生成中间帧并维持时序连续"] B --> C["帧率翻倍输出 (丝滑流畅)"] C --> D["Meta 系列应用 (AI 创作场景) 线上灰度"] D --> E["生产数据回传"] E --> F["Agent 后台持续微调并产出新 Checkpoint"] F -.->|"无缝热替换"| B ``` Rushaan Mahajan 强调，Meta 旗下应用家族（Family of Apps）丰富的 AI 创作场景，为生产环境验证和数据反哺提供了绝佳土壤。最令他感慨的一点是研发文化的蜕变： > “过去我们把模型发布到生产环境就意味着项目结项，而如今检查点刚在生产环境发布，我们其实已经在持续不断地生成更好的检查点。”（“So nowadays, instead of having a model go to production and say that's the checkpoint, this project is done, we're actually saying the checkpoint is released in production, but we constantly have improved checkpoints.”）就像大语言模型领域常说的“智能便宜到无需计量”（intelligence is too cheap to meter），视觉媒体领域也进入了持续进化的快车道。从 AI 生成图片到 AI 编辑，再到针对手机镜头实拍素材的画质修复模型，现在的系统呈现是它“有史以来最差的状态”，因为未来每一天它都会变得更好。 - 传统视频插帧依赖低效的软件滤波算法，新方案通过 Agent 协同研发专用视觉模型替代。 - 视频插帧模型有效解决了全球海量低端设备拍摄造成的画面顿挫与卡顿。 - 生产环境发布不再是研发流程的终点，模型上线的同时，Agent 已在后台基于回流数据持续产出性能更优的检查点。 --- ### [19:24](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1164s) Agent 护栏机制与代际跃迁式产品体验为了保证 Agent 在自动化运转时规范可控，必须为其建立严密的安全护栏（guardrails）与操作框架。Agent 必须细致入微地阅读每一次实验运行记录、评估结果和基准指标。Rushaan Mahajan 明确指出，团队追求的不是每天向生产环境提交带来 0.001% 微小改动的高频代码，而是真正具有质感跃升（step change）的代际体验： > “当我们在生产环境中发布一个新模型时，我们希望上一代模型与下一代模型之间的差距，在质感改进上能让人感受到一次代际跃迁。”（“When we ship a model to production, we want the gap between the previous model and the next model to feel like a step change in quality improvement.”）只有当 Agent 沉下心来执行持续学习、重跑关键训练任务并实现实质突破后，将改动部署到线上才能让用户惊呼：“哇，我昨天还做不到这个效果，但今天能明显看到质的飞跃。” 此外，在视觉媒体任务中，对 Agent 进行严格约束防范其“脱缰乱跑”至关重要。最忌讳的情况就是放任 Agent 自由调度，导致其超出算力预算、耗尽 GPU 配额。必须为 Agent 构建完善的沙箱隔离与测试运行环境，让它在严格遵守算力上限与指标红线的前提下自主探索，最终为用户交付真正惊艳的产品体验。Rushaan Mahajan 在演讲最后号召大家携手创新，共同加速拓宽多媒体 AI 的前沿边界。 ## Notable quotes > [19:24](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1164s) When we ship a model to production, we want the gap between the previous model and the next model to feel like a step change in quality improvement. — Rushaan Mahajan > [04:14](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=254s) An LLM can read the logs and a VLM reads the pixels. This is the difference. — Rushaan Mahajan > [16:42](https://www.youtube.com/watch?v=WHwOVk-kXfM&t=1002s) So nowadays, instead of having a model go to production and say that's the checkpoint, this project is done, we're actually saying the checkpoint is released in production, but we constantly have improved checkpoints. — Rushaan Mahajan ## Mentioned Rushaan Mahajan, Meta, Cyma Labs, Scale [Source](https://www.youtube.com/watch?v=WHwOVk-kXfM)

Source quotes

[19:24] Rushaan Mahajan: When we ship a model to production, we want the gap between the previous model and the next model to feel like a step change in quality improvement. [4:14] Rushaan Mahajan: An LLM can read the logs and a VLM reads the pixels. This is the difference. [16:42] Rushaan Mahajan: So nowadays, instead of having a model go to production and say that's the checkpoint, this project is done, we're actually saying the checkpoint is released in production, but we constantly have improved checkpoints.

Source quotes

[4:14] Rushaan Mahajan: An LLM can read the logs and a VLM reads the pixels. [11:03] Rushaan Mahajan: Continual learning is where agents are continuously understanding inference signals and driving that into model training. [14:03] Rushaan Mahajan: As a model trains and gets ready for production and delivering that final checkpoint, we need to make sure a visual judge is always in that training and deployment inference loop at the same time.

Original transcript

[0:05] The flywheel of AI and media is [0:07] constantly accelerating. [0:09] One of the fascinating parts is where [0:11] models and products are being [0:13] co-designed to improve feature velocity. [0:17] I want to discuss where the golden [0:19] experience and the winning recipe lies. [0:22] Part teams constantly are looking at [0:24] lower latency, the model quality being [0:27] higher, and making an amazing user [0:29] experience. [0:31] To make this an amazing product [0:33] experience, accelerating the AI media [0:36] model life cycle is critical. We know [0:39] about human in the loop where a person [0:42] gates every action of the model. I like [0:44] to think about it as where is agents in [0:47] the loop inverting this. agents are [0:50] actually watching, acting and iterating [0:52] continuously, [0:54] especially in the training life cycle. I [0:57] like to describe it as we have a [0:59] training vertical in the training life [1:00] cycle co-designed with our inference and [1:03] model deployment life cycle. Part of [1:06] this is we want to get feedback from the [1:09] product deployment into the training [1:11] life cycle. And so eval training are [1:15] co-designed. We have a agent that works [1:18] with the researcher to constantly [1:20] improve the training parameters and [1:21] architecture and then finally making [1:24] that push to production and getting that [1:27] production data that enhances this whole [1:29] flywheel. That is where model quality [1:32] can actually benefit the user [1:34] experience. When a model is trained [1:36] really well with lower latency, you see [1:39] users interacting more. [1:42] I like to give this fat great example of [1:45] golden hour. Golden hour is where many [1:48] users want a viral look. Whether it's [1:51] our capture on our phones with sunrise [1:53] or sunset, [1:55] there is a prompt that can go into it. [1:58] So you want to generate what this looks [1:59] like or you have a natural camera [2:02] capture and you want to edit it. This [2:05] editing use case with image editing is a [2:08] huge viral case where we wanted to stand [2:10] up an AI model that could actually help [2:13] users where they took an amazing picture [2:16] but a cloud was in the way of what they [2:18] wanted or a tree was in the landscape [2:21] and they wanted that taken out. [2:23] Production data tells us where we can [2:26] actually see the users interacting and [2:28] actually wanting that change. What's [2:30] important is actually making a model [2:32] that is able to be deployed at scale. [2:35] And this is where we like to think about [2:37] a model that's capacity aware. Right? [2:39] When we develop this model, we think a [2:41] lot of it as taking a user prompt and [2:45] refining it to be able to improve the [2:48] final output. So as an example, when a [2:51] user is saying, I want that cloud [2:53] removed, we actually are able to have a [2:55] model that runs on both GPUs and CPUs [2:59] and is able to make the inference time [3:03] less expensive. So what does this do? [3:05] Now the user is quickly making the [3:07] prompt and the changes happening in real [3:09] time. That's the real-time experience. [3:12] But backing this up comes at the [3:14] training model life cycle. [3:17] I want to go into our agenda today. [3:19] Starting with visual judges and where [3:21] they stand in production, the continuous [3:24] media improvement and how researchers [3:27] and agents are actually teammates. A [3:30] little bit about myself. My name is [3:32] Rashan and I do a lot of leading AI [3:35] media work here at Meta across the media [3:38] foundation team where we span the family [3:40] of apps. Last year I gave a talk at [3:43] scale when we were a startup building [3:45] Sema Labs where we were improving the [3:47] quality of video as we think about image [3:51] video and all the deployments that come [3:54] at scale. This year I'm going to be [3:56] expanding my talk here at Metascale [3:59] where we are actually finding the [4:01] winning model recipe. What does it look [4:03] like to make that golden experience and [4:05] how training and inference are [4:08] co-designed to make a lovely agentic [4:11] experience in product? [4:14] Let me jump into where visual judges are [4:17] playing a role. This is a new type of [4:19] idea. I think about visual judges as a [4:21] secondary vision model that acts as a [4:24] feedback model inspecting visual outputs [4:27] and refining the primary model [4:29] generations. [4:31] So an example of where reference-based [4:33] and reference free metrics play a role [4:35] is in quantitative baselines. So [4:39] quantitative metrics are really critical [4:40] because we think about them as cheap and [4:42] fast information that is a critical [4:45] signal. Reference metrics use the [4:47] original pixels as a baseline. So we [4:50] have a sharp image and then we have an [4:52] output image and we're trying to say is [4:54] this actually what we wanted and is the [4:56] quality of this output actually good [4:58] compared to the baseline. Reference free [5:01] is where there is no original baseline [5:03] and this is a great example in [5:04] production. Right? In training we might [5:06] have the baseline and we might have the [5:08] edited but in production we might only [5:11] have the edited. And so how do we have a [5:13] reference metric across maybe a scale as [5:15] an example a one to five or 1 to 100 and [5:18] actually being able to say this is a [5:20] good kind of generation or edit or [5:23] output. Visual judges play that [5:26] qualitative role. They're able to help [5:28] us with captioning able to help us [5:31] refine the primary model output. [5:35] Agents really need to monitor all these [5:38] metrics, quantitative, qualitative, and [5:41] continuously. I like to think of agents [5:44] as being able to offer a researcher an [5:46] assistance at 24/7 monitoring. So in the [5:50] training vertical, you constantly have a [5:52] model training and often times we have [5:55] this on a GPU fleet where we want to [5:57] make sure that all GPUs are actually [6:00] being used. When an agent is able to [6:02] monitor the metrics, it sometimes see, [6:04] hey, your training run is actually [6:06] failing in this part. But instead of the [6:09] researcher being asleep and basically [6:11] having the GPUs go idle, this is where [6:14] the agents are monitoring those media [6:16] quality metrics and saying, "Let's make [6:18] an adjustment." Now, a good example I [6:21] have here is anything from temporal [6:23] flicker to structural smearing in fast [6:27] motion. These are some really tangible [6:29] things that happen in media models, [6:32] right? And so we want to make sure that [6:34] at every stage of the delivery pipeline, [6:36] we score the outputs in training and [6:40] production. In training, we call this [6:42] the offline ebel process with the [6:45] metrics. And in production, we think [6:47] about it as the online visual judge with [6:50] the quantitative signals. Both of these [6:52] play a critical role. And in both cases, [6:55] agents can continuously monitor 247. [6:58] One of the fascinating things is [6:59] previously researchers had many [7:01] dashboards to view this. Now those [7:04] dashboards are actually going down [7:06] because agents are able to view those [7:09] those metrics and give us researchers [7:12] the signal we need. This is really [7:15] important because vision models can [7:17] observe and self-evaluate your training [7:20] and production. An LLM can read the logs [7:23] and a VLM reads the pixels. This is the [7:26] difference. A VLM can actually evaluate [7:28] is the text in the image coherent. Is [7:31] the frame actually sharp where the brand [7:34] logo is very clear? And this is an [7:36] interesting feedback mechanism. In the [7:38] past, we had as I said these [7:40] quantitative eval metrics. Now we're [7:42] getting feedback where the model is like [7:45] hey you're actually not as good at the [7:47] specific brand logo where we need to [7:50] enhance that and that's qualitative [7:52] signal that comes to the model and in [7:54] production we make that adjustment and [7:56] suddenly all the outputs and logos look [7:58] really good. This is important a judge [8:01] on production traffic is what we like to [8:03] call an online feedback loop. This is an [8:06] online feedback loop that drives the [8:09] training and is actually able to have [8:11] models that when a feature is in [8:12] production, the model is still improving [8:14] because we can deliver a new checkpoint [8:16] and offer a better kind of product [8:19] experience. [8:21] So visual judges at metascale are really [8:23] important to get right because we want [8:25] both those signals quantitative and the [8:27] qualitative. [8:29] I like to think of agent observations as [8:32] a critical part and I talk to my [8:34] teammates all the time of where this [8:36] plays a role as agents and researchers [8:39] become closer teammates. As I said, [8:41] there's a ton of use cases where an [8:44] engineer is bottlenecked. It's [8:46] bottlenecked on capacity, bottlenecked [8:48] on improving the media model. But how [8:51] can actually the ideation phase and the [8:55] whole model life cycle be improved when [8:57] a researcher has a sidekick agent? [9:00] Right? The sidekick is able to [9:02] constantly evaluate prod traffic. It [9:04] samples pro traffic traffic and says in [9:06] these countries actually we can improve [9:08] the model by enhancing the quality in [9:10] this way and in these countries we can [9:13] develop personalized models because they [9:15] have specific requirements based on [9:17] their bandwidth their connection but [9:19] also what do they find visually [9:20] interesting and this is really [9:22] interesting when we get this [9:24] personalized model goal we're able to [9:27] have agents that actually deliver on [9:29] that mission that's the golden [9:31] experience right an agent that is [9:33] sampling ling prod traffic. It's [9:35] flagging issues on researchers behalf [9:38] and it's finding the winning model [9:40] recipe. This is a feedback loop and the [9:42] feedback loop actually can happen 24/7 [9:45] and can consistently enhance the [9:47] industry experience as we think about [9:49] models that are capacity constrainted [9:51] but also models where you want to [9:54] include generation, creation and [9:56] consumption. [9:59] I like to think of all this at the core [10:01] of continual learning. Continual [10:03] learning establishes this idea where a [10:06] product team can come to us and say we [10:08] have a goal. We have a goal to make a [10:10] model for golden hour to make a model to [10:13] make videos smoother motion. These are [10:16] all highle goals but we need specific [10:19] details and that's where I'd like to [10:20] establish this framework of an intent [10:23] right something that has a goal in eval [10:28] gives us the compute budget and it also [10:31] says here is the goal and the iteration [10:33] that we can have this and the timeline [10:35] which product teams want it. Often times [10:38] researchers took this and spent weeks [10:40] doing it. Now a researcher team is [10:43] working with agents to actually iterate [10:46] on this intentl right they're able to [10:48] design this making sure that we're not [10:50] exceeding a GPU budget in training and [10:53] then we're able to say hey at inference [10:55] time with a ton of QPS or a ton of [10:58] inference traffic what is the compute we [11:01] get at inference time we can factor that [11:03] in in the code design of training this [11:06] is where the boundaries are being broken [11:08] down and training and inference works [11:10] together. Continual learning is where [11:12] agents are continuously understanding [11:15] inference signals and driving that into [11:18] model training. I like to really focus [11:21] on this intent as an industry changing [11:24] step. Researchers typically were [11:26] bottlenecked by many things. This [11:28] unblocking is happening as we make [11:31] improvements and make media products [11:34] more viral. There's a lot more [11:35] trade-offs that go into the whole [11:37] experience and making sure we [11:39] acknowledge that with the researcher and [11:41] agent creates this amazing experience [11:44] that delivers amazing product [11:46] experiences. [11:49] A fascinating part of the researcher and [11:52] agent collaboration is the media model [11:55] context. This is where I like to say [11:58] clarity in one report. So when an agent [12:01] works overnight with a researcher, how [12:03] does the researcher come back the next [12:05] morning? And I say this myself having [12:06] gone through this many times, wake up in [12:09] the morning and actually have an [12:11] understanding of what are all the [12:12] improvements the agent made overnight. [12:14] And this happens days compounds into [12:17] weeks and what happens where the agent [12:19] has made amazing progress and how does [12:22] the agent share that with the [12:24] researcher. And so part of this is [12:26] actually having a standard report style. [12:29] And every researcher has their own [12:31] report style they enjoy. But what's [12:33] important is some of the topline [12:35] features. Whether that means [12:37] quantitative eval, qualitative eval, but [12:40] also what is some details around the [12:43] gates. Is the latency higher of this new [12:46] model? Is the model maybe producing much [12:50] better quality outputs? and a specific [12:52] part of the the visual judge giving [12:55] feedback is showing, hey, we're actually [12:57] producing outputs that the user might [12:59] like a lot more. How does the agent [13:02] share that with the researcher? And [13:05] that's where I like to say every claim [13:07] is reproducible. It has a quantitative [13:10] backing cuz the one thing that [13:12] researchers don't like is when an agent [13:13] has made hallucinations. So this report [13:16] is grounded without hallucinations and [13:19] it makes sure that the media model [13:21] context is shared. So a researcher can [13:23] go into the day and say I want to launch [13:26] these many training jobs. And so they [13:28] start the morning giving that agent [13:30] those instructions whether that's [13:31] through the intent or also through the [13:34] the media model context. And now the [13:37] agent and the researcher are working [13:39] side by side and making accelerating [13:42] progress instead of having bottlenecks [13:45] like hallucinations where the agent is [13:47] making mistakes. This is really critical [13:49] because the last thing we want is an [13:51] agent making hallucination and the [13:52] researcher not happy about that. This is [13:55] why we're seeing amazing model recipes [13:58] coming out and actually the whole [14:00] industry in AI media advancing. [14:03] So, as I said, the golden experience is [14:06] really finding the winning media model [14:09] recipe with the agent. You have an agent [14:11] that's continuously improving and seeing [14:15] where capacity is, what are some of the [14:17] new product features that have come in [14:19] and actually what is production data [14:21] telling us. It's finding checkpoints [14:23] that are actually not going to make it [14:24] to production and stopping them and [14:26] starting new training runs to say, hey, [14:29] maybe these small hyperparameter [14:31] improvements and hyperparameter tuning [14:32] can get us an improved result. And [14:35] that's where it takes the feedback in [14:36] the quantitative and the qualitative and [14:39] it makes training velocity 2x faster and [14:42] it's seeing more products shipping. [14:45] Without the agent, you have all these [14:46] engineering bottlenecks. Whether that's [14:48] GPUs idling or the model getting to 200 [14:51] milliseconds, but the product team said [14:52] we want it at 100 milliseconds, right? [14:55] Or we're seeing capacity exhausting. A [14:58] lot of this comes when you see [15:00] deployments at scale, right? You're [15:01] starting to understand how an agent is [15:03] actually able to fit a lot of the [15:05] requirements. Requirements are getting [15:08] more challenging and agents are here to [15:09] help us. [15:12] I like to think of this all in the media [15:15] harness. As a model trains and gets [15:18] ready for production and delivering that [15:20] final checkpoint, we need to make sure a [15:22] visual judge is always in that training [15:25] and deployment inference loop at the [15:28] same time. And agents are constantly [15:30] writing fixes to make sure the training [15:32] loop is as best as possible and we [15:35] constantly have the best eval in [15:36] production. At the end of the day, eval [15:39] play a role both in training and in [15:41] inference. This all comes together at [15:44] the long tail. There's a long tale of [15:46] features and now pod teams are able to [15:48] constantly offer and ask for more things [15:50] and research teams are able to drive [15:53] more velocity. This whole hardness comes [15:56] to be when you think about the evolution [15:59] of model training from software [16:01] algorithms to also visual judges where [16:05] we previously had quantitative metrics [16:07] to qualitative. And we're starting to [16:09] see why agents writing the diffs, [16:12] writing the fixes, and giving that media [16:15] context to the researcher plays an [16:18] important role. I like to think of this [16:21] as when you're capacity constrained, [16:24] what are all the different features that [16:27] you want to actually deploy and how do [16:29] you have researchers and agents working [16:31] together side by side to make the [16:34] longtail actually be deployed and how do [16:37] we actually take care all the features [16:39] and all the user requests. [16:42] I want to ground us in a core meta use [16:45] case. A lot of videos across the world [16:48] come from low frame rates because maybe [16:50] phones are capturing at lower quality, [16:54] but also the motion feels stuttery. [16:57] Taking care of this was previously done [16:59] with a software algorithm. But here in [17:02] 2026, we said we can train a model and [17:05] do it fast for the product team. And [17:07] this is the difference. Alongside a [17:10] researcher and an agent, we were able to [17:12] train a model that does video frame [17:14] interpolation. Between every frame, we [17:16] are able to generate and insert a frame [17:18] that keeps the temporal context. Now, [17:21] there are so many parts to this. There's [17:23] the loss functions. There's making sure [17:26] that across different frame rates, what [17:29] if the video was 15 fps input versus [17:32] whether it was 20 fps input? but also [17:35] how do we make sure that we get the best [17:37] version of the model into production and [17:40] get data. And this was a fascinating use [17:42] case. At Metascale, we love shipping and [17:45] offering that production data to drive [17:48] the model training. And so we were able [17:50] to develop a model that was taking low [17:54] frame rate videos, doubling the frame [17:56] rate, and then passing some of that [17:58] context. We saw that users loved at low [18:01] frame rates the model was doing very [18:04] well but at higher frame rates we had to [18:06] add more features in and actually how do [18:08] we still double the frame rate but [18:10] actually provide an amazing user [18:11] experience. What's amazing is the AI [18:14] creation surfaces that we have here at [18:16] the family of apps is really important [18:19] because it gives us the ability to to [18:20] test in production get core data and go [18:24] back and constantly have the model [18:25] improve. What I want to say about this [18:27] deployment that fascinated me is that [18:29] every time we shipped a model to [18:30] production, many people celebrated that [18:33] while their agents actually were [18:34] continuously improving the model. So [18:37] nowadays, instead of having a model go [18:39] to production and say that's the [18:40] checkpoint, this project is done, we're [18:43] actually saying the checkpoint is [18:44] released in production, but we [18:46] constantly have improved checkpoints. So [18:48] what does that mean? The user experience [18:50] is continuously getting better. Now, we [18:53] like to say intelligence is too cheap to [18:55] meter, which is important in the LLM [18:57] world. Well, now we're starting to say [18:59] visual media, whether that's generation [19:02] or creation, is constantly improving. [19:05] And I like to say this is the worst it's [19:07] ever going to be. It's going to [19:08] constantly improve. AI generated images [19:10] to AI editing to also improving ML [19:14] models that are able to work on camera [19:16] captured inputs. The whole life cycle is [19:19] improving and parked experiences are [19:20] getting so much better. [19:24] To make sure that we have agents that [19:26] are streamlined and guardrail, I like to [19:28] think about a couple frameworks that are [19:30] really important. Whether it's reading [19:32] every run, every eval, and every metric, [19:35] and making sure that an agent is [19:38] actually able to help us make that [19:40] product improvement. When we ship a [19:42] model to production, we want the gap [19:44] between the previous model and the next [19:45] model to feel like a step change in [19:47] quality improvement. And that's where [19:50] the agent needs to make sure it's not [19:51] just landing changes every day, right? [19:53] It's not like we want to land a change [19:55] and this is a 0001% improvement. [19:58] Instead, it's actually doing continuous [20:00] learning and saying let's relaunch a [20:03] couple training jobs. Let's improve [20:04] this. And when the improvement gets [20:06] landed in production, the user [20:08] experience feels like a true step up. [20:10] This is where the user is like, whoa, [20:12] yesterday I was not able to do this, but [20:14] today I'm actually able to see a step [20:15] change in improvement. And making sure [20:17] that agents are guardrail to actually [20:21] produce the best outputs, especially in [20:23] a visual medium, is critical. One thing [20:26] we don't like to do is make sure the [20:28] agents can run wild and actually create [20:30] a problem in terms of exceeding the GPU [20:32] budget. We want to make sure that an [20:34] agent is always given the right sandbox [20:36] and and environment to be able to work [20:39] with our capacity constraints, work with [20:42] our metrics, and also ship a user [20:45] experience that actually feels [20:46] delightful. [20:48] Thank you. And I'm excited to accelerate [20:50] the media frontier. There's so many [20:52] golden opportunities to make amazing [20:55] part experiences. Let's innovate [20:57] together.