Google Cloud Warns Multi-Agent Cascades Drain Enterprise AI Budgets

The Enterprise AI Imperative - Scaling Agents for Transformational Value With Google Cloud — Ai4

Ai4

Unmonitored multi-agent architectures multiply downstream model calls by up to sixteenfold, exhausting annual IT budgets within weeks unless organizations enforce strict governance and model right-sizing.

AI-assisted summary of The Enterprise AI Imperative - Scaling Agents for Transformational Value With Google Cloud — original source. Check the original for full context.

Source published: 2026-08-14

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 14, 2026, this keynote features Ben Yevin, an AI Solutions Specialist at Google Cloud. According to the video description, the presentation introduces a practical framework for connecting enterprise AI capabilities to the bottom line, addressing common implementation roadblocks while managing the full agent lifecycle from prototype to global production.

## TL;DR Ben Yevin distinguishes standalone foundation models from complete agent orchestration systems, deconstructing what enterprise scale means across user concurrency, agent volume, and infrastructure management. He demonstrates the Gemini Enterprise Agent Platform, illustrating how modular multi-agent architectures—such as an automated CFO earnings call preparation workflow—serve both pro-code developers and low-code business users. The talk underscores the necessity of lifecycle governance, brand safety filters, and automated evaluation agents to maintain production trust. Finally, Yevin addresses tokenomics discipline, cautioning teams against downstream multi-call amplification and emphasizing model right-sizing between Flash and Pro variants to ensure tangible business ROI.

## Atmosphere & style The presentation maintains a pragmatic, brisk, and approachable tone, blending technical clarity with conversational humor, self-deprecating stage moments, and upbeat AI-generated songs. Yevin acts as a candid practitioner-guide, puncturing industry ambiguity and hype with relatable war stories about runaway cloud budgets, miscalculated query sizing, and CFO scrutiny. Watching the talk feels engaging, grounded, and direct, translating complex enterprise architecture into structured, actionable insights.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) There remains persistent confusion in the market regarding the fundamental definitions and differences between foundation LLMs, apps, and autonomous agents. - [02:16](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=136s) An LLM acts merely as the reasoning 'brain,' whereas an agent is the operational system that coordinates planning, memory, tools, and execution to run complex workflows without context loss. - [03:23](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=203s) Enterprise scale must be deconstructed across four dimensions: horizontal agent quantity, concurrent user volume, unified delivery surfaces, and managed operational runtimes. - [04:26](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=266s) The Gemini Enterprise Agent Platform unifies the agent lifecycle across four core pillars: build, scale, govern, and optimize. - [05:54](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=354s) Scaling enterprise adoption requires supporting both pro-code software developers via Agent Development Kits (ADKs) and non-technical business users via visual low-code/no-code studios. - [05:54](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=354s) Gemini Enterprise functions as a single pane of glass and agent hub, allowing employees to access and interact with agents connected to internal SaaS tools and data lakes. - [07:14](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=434s) Modular multi-agent architectures break down monolithic prompt limits by assigning dedicated sub-agents to specific tasks like market scraping, historical analysis, and deck creation. - [09:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=544s) Robust governance and observability tooling—including session inspection, audit logs, and query blocking—are essential to prevent uncontrolled LLM sprawl and token waste. - [11:09](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=669s) Ensuring brand safety and trust at high throughput requires automated verification, such as LLM-as-a-judge evaluation loops, sandboxed code execution, and strict access controls. - [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) Multi-agent workflows cause dramatic query amplification, where a single user prompt can trigger dozens of downstream calls, drastically multiplying infrastructure and throughput requirements. - [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) Optimizing tokenomics requires right-sizing models—routing low-complexity or latency-sensitive tasks to lighter models like Flash or Flash-Lite and saving Pro models for deep reasoning. - [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) Sustained enterprise AI investment demands strict ROI discipline to prove business value and prevent teams from burning annual compute budgets prematurely. - [15:34](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=934s) Successful scaling hinges on an architectural checklist encompassing clear problem definition, persona alignment, capacity planning, safety controls, and cost discipline.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) Level-Setting the Agent Confusion

Ben Yevin, an AI Solution Specialist at Google Cloud who joined the team around its inception two and a half years ago and has spent over six years at Google, opens by shifting the focus from how to construct individual agents to what scaling them actually means in an enterprise context. Before diving into the mechanics, he pauses when his speaker notes accidentally project on screen—laughing that the audience doesn't need to see that—before insisting on a necessary baseline. While he usually avoids basic definitions, a recent customer pulled him aside in a sidebar conversation asking fundamental questions: "What is an agent? What's the difference between an agent and a Gemini app? What's the difference between an agent and an LLM?"

Even for someone who sits in the seat every single day talking to organizations about building, testing, and deploying to production, the interaction was eye-opening. It took him a minute to process how to explain concepts that practitioners often take for granted. To capture the persistent ambiguity in the market—epitomized by the sentiment that "no one really knows what an AI agent is"—he cues up an upbeat track generated using Google's Gen Media tools, singing about a fast-approaching digital world with "AI agents in our veins."

[02:16](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=136s) LLMs vs. Agents: Brains vs. Orchestration Systems

Moving past the music, the distinction between a standalone foundation model and an agent comes down to the difference between a raw reasoning engine and a complete operational system. A large language model serves as the "brain" giving a response, but attempting to execute complex multi-step workflows with an LLM alone quickly runs into walls. Even with an exceptionally robust prompt, single-model workflows inevitably lose context, begin to stray, and risk hallucinations—even though providers have worked aggressively to mitigate and redact them.

An agent, by contrast, functions as the overall worker and orchestration framework connecting multiple moving parts. It pairs the underlying model with planning, reasoning, memory, tools, and runtime execution. Rather than merely supporting an interactive chat conversation, the agent coordinates external actions and multi-step executions to accomplish a defined mission.

```mermaid flowchart TD accTitle: AI Agent System Architecture accDescription: Architectural diagram showing how an AI agent coordinates models, memory, planning, tools, and runtime execution.

subgraph CoreSystem ["AI Agent System"] LLM["LLM (The Brain / Reasoning Engine)"] Plan["Planning & Reasoning"] Mem["Memory Bank & Sessions"] Tools["Tool & API Integrations"] end

subgraph Environment ["Execution & Delivery"] Runtime["Managed Agent Runtime"] Tasks["Task Execution & Actions"] end

LLM <--> Plan Plan <--> Mem LLM <--> Tools CoreSystem --> Runtime Runtime --> Tasks ```

[03:23](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=203s) Deconstructing Enterprise Scale

As agents proliferate across organizations, enterprise leaders are forced to confront what "scale" actually means in practice. The definition is rarely uniform across teams or business units: - Scaling horizontally across agent quantity: moving from a single bespoke agent to hundreds of specialized agents. - Scaling across user concurrency: expanding access from a single pilot user to hundreds or thousands of employees simultaneously. - Surface and delivery scaling: deploying unified interfaces so both highly technical engineers and everyday non-technical business users can access organizational agents. - Operational scaling: transitioning from self-managed underlying servers and custom hosting to fully managed runtimes and shared execution engines.

[04:26](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=266s) The Gemini Enterprise Agent Platform

Addressing Google's sprawling naming ecosystem—from the public Gemini app and Gemini Enterprise app to Gemini in BigQuery and Gemini in Google Workspace—the central framework for enterprise application and agent orchestration is the Gemini Enterprise Agent Platform (the evolution of what was formerly branded as the Vertex AI agent platform). The architecture centers on four core pillars: build, scale, govern, and optimize.

The interface is dense with "New" and "GA" (General Availability) indicators, reflecting rapid product development intended to give enterprises full control over their deployments. Scaling cannot succeed without governance and observability: teams must monitor active sessions, extract logs, inspect user queries, verify that agents are sticking to their intended mandates, and ensure the business is not merely "burning 14 trillion tokens a day asking for the weather report."

[05:54](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=354s) Building for Both Pro-Code and Low-Code Personas

Building agents requires accommodating two distinct organizational personas: technical developers using pro-code workflows, and business users needing no-code or low-code environments. For engineers, an Agent Development Kit (ADK) paired with dedicated developer tools and scalable runtimes allows for programmatic construction. For non-technical staff, visual natural-language interfaces like Agent Studio provide intuitive graphical configuration.

To bridge these groups, Google introduced Gemini Enterprise the application. It acts as an enterprise-wide agent hub and single pane of glass, coupling the core Gemini interface with connectors into internal SaaS applications and siloed enterprise data lakes. Within this unified portal, employees can interact with agents built by their colleagues—whether those agents were configured via natural-language prompt builders by business staff or authored in code by engineering teams.

[07:14](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=434s) Case Study: The Earnings Prep Call Agent

Demonstrating the power of no-code generation inside Gemini Enterprise, an agent built for Google's Chief Financial Officer illustrates how complex financial workflows are orchestrated. Rather than manually writing intricate code, the author described the desired behavior in unstructured natural language—essentially "word vomited all over the page"—and instructed the system to generate an agent configuration prompt.

Pasting that prompt into Gemini Enterprise automatically spun up a modular multi-agent structure: - A central root agent coordinating high-level objectives. - A market-crawling sub-agent tasked with scanning real-time financial market movements. - A research sub-agent responsible for pulling and synthesizing historical company earnings reports. - An executive briefing agent that compiles findings into structured briefing memos. - A presentation agent that formats the output directly into executive-ready prep documents and slide presentations.

Attempting this task with a single monolithic prompt would likely fail as the model drops context and loses focus across historical records and market datasets. Breaking the workload across specialized sub-agents—each grounded in dedicated enterprise data and running on tailored models—enables non-developers to build, deploy, and share sophisticated production workflows without touching code.

```mermaid flowchart LR accTitle: Multi-Agent Earnings Call Preparation Flow accDescription: Workflow diagram showing a root orchestrator agent delegating tasks across market research, historical analysis, briefing, and presentation sub-agents.

User["CFO / Finance Team"] --> Root["Root Orchestrator Agent"]

subgraph SubAgents ["Specialized Sub-Agents"] Crawl["Market Crawling Agent"] History["Historical Earnings Agent"] Brief["Executive Briefing Agent"] Deck["Presentation Builder Agent"] end

Root --> Crawl Root --> History Crawl --> Brief History --> Brief Brief --> Deck Deck --> Output["Final Prep Document & Deck"] ```

[09:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=544s) Managing the Lifecycle: Scale, Governance, and Optimization

Once technical teams build agents, the primary hurdle is deployment without drowning in infrastructure operations. Within the Gemini Enterprise application layer, runtime, hosting, and compute management are abstracted away entirely. For engineering groups operating through technical code paths, the platform's scaling layer supplies managed runtimes, persistent session handling, and shared memory banks—enabling an organization to grow from one agent to hundreds without manually configuring virtual machines or clusters.

Equally critical are governance and cost optimization. Governance tooling provides deep visibility into user sessions, extracts audit logs, monitors prompt inputs, and enforces policy controls such as dynamically blocking problematic queries or restricting user access. These guardrails prevent "LLM sprawl" and uncontrolled "tokenomics" from cannibalizing corporate IT budgets before value can be verified.

[11:09](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=669s) Production Trust, Brand Safety, and Verification

Scale is fundamentally tied to trust: as interaction volumes swell, organizations need automated ways to guarantee output fidelity. Deploying human-in-the-loop validation remains an essential strategy, but high-throughput systems increasingly rely on automated verification architectures: - Specialized evaluation agents such as LLM-as-a-judge and self-reflection loops that inspect outputs against strict predefined criteria before delivery. - Dedicated code execution sandboxes that run and validate generated code snippets before passing them back to human developers. - Brand protection filters that screen for abusive inputs, mitigate negative public relations risks, and prevent proprietary IP from leaking across external network perimeters. - Strict identity and role-based access controls to guarantee that agents and users only retrieve enterprise data they are explicitly permitted to view.

[12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) Tokenomics, Quota Sizing, and ROI Discipline

Uncontrolled costs frequently derail enterprise AI initiatives, often because architectures are misaligned with their underlying workloads. Not every interaction requires a massive, computationally expensive frontier model; routine, latency-sensitive tasks can run efficiently on lightweight models like Flash or Flash-Lite, reserving high-powered Pro or specialized coding models strictly for complex reasoning.

The math behind multi-agent architectures often surprises teams. During a sizing exercise for a client running an external customer platform, the customer estimated system load at just one query per second (QPS) and requested Provisioned Throughput to guarantee capacity and prevent HTTP 429 rate-limit errors. However, a single user query triggered an automated cascade of 16 downstream LLM and agent calls. The true workload was 16 QPS rather than 1 QPS, causing their infrastructure estimate to skyrocket from 105 Generative Scale Units (GSUs) to roughly 1,500 GSUs.

By auditing the architecture, substituting heavy models for Flash and Flash-Lite variants, and eliminating redundant intermediate steps, the team brought the workload under control. Such discipline is urgent: stories circulate across LinkedIn of enterprises exhausting their entire annual AI budget in a single month. When one customer approached their CFO pleading for emergency funds, the executive agreed on one strict condition: the team had to return with proven ROI showing that resources were powering measurable business value rather than employees "building day trading systems on the weekend."

[15:34](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=934s) Architectural Checklist and Closing

Ben wraps up the session by summarizing the critical strategic decisions every organization must address when scaling agents: - Problem definition: pinpoint the exact business challenge before writing prompts or provisioning infrastructure. - Developer persona alignment: establish clear entry points for non-technical business teams versus specialized software engineers. - Capacity and scaling scope: determine whether scale targets agent counts, concurrent users, or infrastructure self-management. - Brand fidelity and safety: implement multi-layered filters and evaluation loops to avoid viral PR disasters and data leaks. - Cost architecture: right-size models across sub-agents, monitor multi-call amplification, and continually optimize token utilization.

To close, he shares one final AI-generated track created by feeding a screenshot of his summary slide into Google's Gen Media tools with a simple prompt asking for a song about the lightbulb moment when everything clicks into place. Leaving the audience with the upbeat track, he invites attendees to connect on LinkedIn and discuss questions off-stage.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) What is an agent? What's the difference between an agent and a Gemini app? What's the difference between an agent and an LLM? — Ben Yevin

> [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) no one really knows what an AI agent is — Ben Yevin

> [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) AI agents in our veins — Ben Yevin

> [04:26](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=266s) burning 14 trillion tokens a day asking for the weather report. — Ben Yevin

> [07:14](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=434s) word vomited all over the page — Ben Yevin

> [09:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=544s) LLM sprawl — Ben Yevin

> [09:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=544s) tokenomics — Ben Yevin

> [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) building day trading systems on the weekend. — Ben Yevin

## Mentioned Ben Yevin, Google Cloud, Google, Ai4, Gemini, Gemini Enterprise, Gemini in BigQuery, Gemini in Google Workspace, Vertex AI, Agent Development Kit (ADK), Agent Studio, Flash, Flash-Lite, Pro, LinkedIn

## Video description Ben Yevin - AI Solutions Specialist, Google Cloud

While AI technology is advancing rapidly, many organizations still struggle to translate innovation into measurable business results. This keynote bridges the gap between technical execution and strategic value creation, exploring the common roadblocks that stall enterprise AI initiatives. We will introduce a practical framework for connecting AI capabilities directly to your bottom line, while teaching you the "how" of scaling. Learn how to accelerate the journey from prototype to global production by building AI agents capable of handling millions of interactions and taking real-time action. Join us to discover comprehensive strategies for managing the entire agent lifecycle and ensuring your data and systems grow seamlessly to drive true transformational impact.

[Source](https://www.youtube.com/watch?v=OOMEVQCUX5Q)

中文简报

Google Cloud警告多智能体级联耗尽企业AI预算缺乏监控的多智能体架构会使下游模型调用量激增高达16倍，除非企业落实严格治理并合理匹配模型规模，否则年度IT预算将在数周内耗尽。 Ben Yevin将独立基础模型与完整的智能体编排系统进行了区分，从用户并发量、智能体规模及基础设施管理等维度剖析了企业级规模的内涵。他展示了Gemini Enterprise Agent Platform，阐释模块化多智能体架构（如CFO财报电话会议准备自动化工作流）如何同时服务专业代码开发者与低代码业务用户。演讲强调了生命周期治理、品牌安全过滤和自动化评估智能体对于维持生产级信任的必要性。最后，Yevin探讨了Token经济学规范，告诫团队防范下游多次调用的放大效应，并强调在Flash与Pro变体之间合理匹配模型规格，以确保切实的商业ROI。大语言模型（LLM）仅充当推理“大脑”，而智能体（Agent）则是结合了规划、记忆、工具调用与运行时执行的完整操作体系，能够避免长流程任务中的上下文丢失与幻觉。 (2:16) Gemini Enterprise 智能体平台围绕构建、扩展、治理与优化四大支柱，既通过 ADK 支持专业开发者，又借助无代码 Studio 赋能业务人员，作为统一入口连接企业内部 SaaS 工具与数据湖。 (4:26) 以财报筹备智能体为例，模块化多智能体架构将单一庞大 Prompt 拆分为市场抓取、历史财报分析与简报生成等专用子智能体，显著提升复杂工作流的执行效果。 (7:14) 企业扩展智能体必须警惕 Token 成本失控与算力级联放大，单次用户查询可能触发 16 次下游调用，需合理搭配 Flash 等轻量模型并建立严格治理与评估机制以证明实际 ROI。 (12:37)

Reading summary

According to Google Cloud AI Solution Specialist Ben Yevin, scaling artificial intelligence in the enterprise requires moving beyond standalone language models to orchestrated multi-agent systems supported by managed runtimes, automated governance, and rigorous token cost discipline. Moving From Raw Models to Orchestrated Multi-Agent Systems (2:16) A standalone foundation model functions primarily as a reasoning brain, but attempting to execute complex workflows with a single prompt causes systems to lose context, drift from objectives, and produce hallucinations. Yevin contrasts raw models with AI agents, which act as operational workers that integrate underlying models with planning, reasoning, memory banks, tool interfaces, and managed runtime execution. To demonstrate this distinction, Yevin highlights an earnings call preparation workflow built for Google's Chief Financial Officer. Instead of relying on a fragile monolithic prompt, the system generated a modular multi-agent architecture from natural-language instructions. A root orchestrator coordinated specialized sub-agents responsible for market crawling, historical earnings analysis, executive briefing generation, and presentation slide creation. Distributing tasks across specialized sub-agents grounded in proprietary data maintains context and output fidelity where a single model fails. Unifying Technical and Non-Technical Development Workflows (5:54) Enterprise agent adoption requires addressing two separate user personas: software engineers building through code and non-technical business professionals relying on low-code or no-code tools. Technical developers require Agent Development Kits (ADKs), programmatic code paths, and scalable infrastructure, whereas business users need visual configuration environments like Agent Studio that operate via natural-language prompts. To prevent architectural siloing between these groups, Google uses the Gemini Enterprise application as an organization-wide agent hub. The application integrates directly with enterprise data lakes and SaaS software, providing a unified single pane of glass where business users and engineers can share, discover, and deploy agents built across both code and visual authoring pipelines. Operational Scaling and Automated Trust Guardrails (9:04) Enterprise scale involves more than increasing agent counts; it requires expanding user concurrency, unifying delivery interfaces, and shifting from self-managed virtual machines to fully managed runtimes that abstract hosting, compute, and session management. However, Yevin notes that deploying at scale quickly collapses without observability and administrative guardrails to verify active sessions, inspect user prompts, and prevent model sprawl. Because manual oversight cannot handle high throughput, enterprise systems must rely on automated verification architectures. These include LLM-as-a-judge evaluators, self-reflection loops, and isolated code execution sandboxes that test generated code before delivery. Complementary protections—such as role-based access controls, audit logging, and brand safety filtering—prevent internal data leaks across network boundaries and mitigate reputational risks. Controlling Tokenomics and Multi-Call Workload Amplification (12:37) Unmanaged compute costs frequently stall enterprise deployments, with Yevin noting reports of organizations burning through annual AI allocations within a month. A primary culprit is multi-call amplification within agent architectures. In one customer sizing exercise, a client estimated demand at one query per second (QPS) for provisioned throughput. However, a single incoming query triggered 16 downstream model and agent calls, ballooning actual capacity requirements from 105 Generative Scale Units (GSUs) to approximately 1,500 GSUs. Controlling this expenditure requires architectural auditing and model right-sizing. Rather than routing every step to expensive frontier models, organizations substitute lightweight models like Flash or Flash-Lite for routine, latency-sensitive steps, reserving compute-heavy Pro models strictly for complex reasoning. Yevin stresses that financial executives increasingly require verified return on investment before releasing emergency budgets, demanding proof that agent networks deliver measurable business value. Yevin concludes that successful agent scaling depends on addressing five prerequisites: clear problem definitions, persona-aligned development environments, realistic capacity sizing, robust brand and evaluation guardrails, and disciplined cost optimization against cascading agent calls.

精炼阅读摘要

Google Cloud 的 AI 解决方案专家 Ben Yevin 指出，企业规模化部署 AI Agent 的核心挑战并非构建单体模型，而是建立涵盖治理、编排架构及严密成本管控的完整运行体系。区分基础模型与 Agent 编排系统 (2:16) 在企业实际应用中，单一大语言模型与 Agent 存在本质区别。LLM 本质上是提供推理与回答的“大脑”，但若直接依赖单一模型处理复杂的多步骤工作流，即便编写了详尽的 Prompt，模型依然容易丢失上下文、偏离任务目标并产生幻觉。相比之下，Agent 属于整体工作与编排框架，它将底层模型与规划、长期记忆、工具调用以及托管运行环境结合起来，从而协调外部操作以完成具体业务目标。面对企业在多维度上的扩展需求，规模化不仅意味着将单一 Agent 扩展至数百个专用 Agent，还涵盖支持成百上千员工的高并发访问、面向技术与非技术人员的统一交互界面，以及从自建服务器迁移至全托管运行时与共享执行引擎的运维转型。双轨开发与多 Agent 协同实践 (5:54) 为消除技术人员与业务部门之间的壁垒，Google 将原 Vertex AI Agent 平台演进为 Gemini Enterprise Agent Platform，划分为构建、扩展、治理和优化四大支柱。平台同时支持两类人群：专业开发者可利用 Agent Development Kit（ADK）与代码工具进行系统级搭建；业务人员则能通过可视化自然语言界面（如 Agent Studio）进行低代码配置。集成的 Gemini Enterprise 应用程序充当企业 Agent 中心，让员工可以统一调用跨部门构建的各类工具，并连接内部 SaaS 与孤岛数据。这一架构的可行性在一个针对 Google 首席财务官财报电话会议准备的案例中得到了验证。业务人员无需编写代码，只需用非结构化自然语言提出需求，系统便自动生成包含多子 Agent 的协同流程：主 Agent 统筹全局，市场抓取子 Agent 扫描实时行情，研究子 Agent 提取历史财报，简报子 Agent 汇总提炼，演示子 Agent 则生成汇报文档。这种将庞大任务解构为专业子任务的模式，避免了单模型在海量异构数据中丢失上下文的问题。生产级信任、品牌安全与自动化评估 (11:09) 当 Agent 调用量激增时，企业必须建立自动化机制以确保输出的准确性并防范风险。尽管人工在环（Human-in-the-loop）仍然是重要的验证策略，但在高吞吐量系统中，依赖自动化审核架构已成为必然趋势。这包括引入专门的模型作为评估裁判（LLM-as-a-judge）与自我反思循环，在输出交付给用户之前依据预设规则进行自动检验。此外，生产环境的防护体系还包括运行代码沙箱以先行验证生成的代码片段、部署品牌保护过滤机制以阻断违规输入并防止内部知识产权泄露，以及实施严格的身份认证与基于角色的访问控制（RBAC），确保 Agent 和终端用户只能检索被授权访问的企业专有数据。调用放大效应与 Token 成本控制 (12:37) 多 Agent 架构的隐性成本往往超出企业预期。Ben Yevin 举例说明，某客户在构建外部平台时，原以为系统负载仅为每秒 1 次查询（1 QPS），并申请了预留吞吐量（Provisioned Throughput）以避免 HTTP 429 限流错误。然而架构评估显示，单个前端用户查询会在底层触发级联效应，产生 16 次下游 LLM 和 Agent 调用，导致真实负载达到 16 QPS，其基础设施需求估算也随之从 105 个生成扩展单元（GSU）飙升至约 1500 个 GSU。为避免企业年度 AI 预算在极短时间内耗尽，企业必须建立严谨的成本架构。并非所有环节都需要高开销的前沿大模型，轻量级模型（如 Flash 或 Flash-Lite）足以应对日常及对延迟敏感的任务，高成本的 Pro 系列或专用代码模型应严格保留给复杂推理任务。同时，企业必须通过审计消除冗余调用，并向管理层出具可量化的投资回报率（ROI）证明，以确保计算资源真正服务于核心业务业务增长。规模化推行企业 Agent 要求决策者在动工前明确具体业务痛点、划分开发人员权限、定义扩容范围，并在架构设计中同步落实品牌防护与模型分层策略。未建立调用放大监控与投资回报证明的系统，将面临严重的预算失控风险。

Detailed notes — zh

## Background 本内容由 Ai4 于 2026 年 8 月 14 日发布在 YouTube 平台上。视频记录了 Google Cloud 的 AI 解决方案专家 Ben Yevin 发表的主题演讲，页面描述指出该演讲旨在弥合技术落地与战略业务价值之间的鸿沟，介绍如何构建、治理并规模化扩展能够处理海量交互的企业级 AI Agent，从而推动真正的业务转型。 ## TL;DR 演讲首先厘清了大语言模型（作为思考大脑）与 AI Agent（作为具备记忆、规划和工具调用能力的完整执行系统）的本质区别。随后介绍了基于 Gemini Enterprise Agent Platform 的全生命周期落地框架，涵盖无代码 Agent Studio、专业代码开发、全托管运行时以及全套治理监控能力。针对企业落地痛点，演讲深入探讨了通过人机协同（Human-in-the-Loop）与 LLM-as-a-Judge 构建可信安全屏障，并结合多智能体级联放大的实际案例，提出了分层选型和精准容量规划以防止成本与 Token 蔓延的核心策略。 ## Atmosphere & style 整场演讲呈现出既专业务实又幽默轻松的科技大会现场氛围。讲者善于用自嘲与即兴生成的 AI 歌曲化解行业认知模糊的严肃感，使现场互动生动有趣。随着话题切入真实客户踩坑案例、架构隐患与 CFO 预算压力，叙事节奏转入严谨硬核的工程与商业视角，给观众带来一种从行业迷雾中抽丝剥茧、豁然开朗的清晰感与落地掌控力。 ## Key takeaways - [01:43](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=103s) 大语言模型（LLM）是推理大脑，而 AI Agent 是整合了任务规划、长期记忆、工具调用与运行时的完整执行系统。 - [01:43](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=103s) 单纯依赖复杂繁琐的 Prompt 无法替代系统工程，长链路任务必须依靠 Agent 架构编排来防止上下文丢失与幻觉产生。 - [03:23](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=203s) 企业级规模化（Scale）涵盖三个维度：Agent 数量增长、全员并发覆盖以及面向技术与非技术用户的统一分发触达。 - [03:23](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=203s) Gemini Enterprise Agent Platform 围绕构建（Build）、规模化（Scale）、治理（Govern）与优化（Optimize）四大支柱管理智能体全生命周期。 - [05:54](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=354s) Gemini Enterprise 应用扮演企业“统一门面”，原生打通数据孤岛与 SaaS 软件，支持无代码与代码构建的 Agent 统一发布。 - [05:54](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=354s) 采用根智能体（Root Agent）与子智能体（Sub-agents）的多层专业化分工，非技术人员也可在数分钟内快速组装复杂业务流程。 - [08:58](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=538s) 托管式运行时、Session 会话管理与 Memory Bank 帮助企业摆脱底层服务器运维负担，有效防止 Agent 蔓延与资源浪费。 - [11:01](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=661s) 规模化部署需依托人机协同（Human-in-the-Loop）与质检代理（LLM-as-a-Judge）机制，确保输出合规并杜绝敏感数据泄漏与越权风险。 - [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) Multi-Agent 系统存在调用放大效应（如 1 次用户请求级联触发 16 次调用），容量与吞吐量规划必须基于真实内部链路计算。 - [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) 控制 LLM 成本的关键在于分层选型与精准降级，常规任务采用 Flash 等轻量模型以大幅压低开销并证明业务 ROI。 - [15:34](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=934s) 智能体成功落地需以真实业务问题为导向，并将成本管控、品牌安全与风控治理深度融入架构迭代的全生命周期。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) 重新对齐概念：什么是 AI Agent 与 LLM 的本质区别 Google Cloud 的 AI 解决方案专家 Ben Yevin 登台开场。他在该团队成立之初（约两年半前）便已加入，在 Google 供职已超过六年。在切入当天的核心议题“如何规模化扩展智能体（Scaling Agents）”之前，他幽默地注意到投影屏幕上意外切出了自己的演讲备忘录，笑着调侃大家不需要看那个。随后他决定先做一件平时很少做的事——重新校准并定义“什么是 AI Agent（智能体）”。之所以要从最基础的概念谈起，是因为最近有一位客户在私下交流时非常认真地问他：“到底什么是 Agent？Agent 和 Gemini 应用有什么区别？Agent 和底层大语言模型（LLM）又有什么区别？”作为一名整天与企业客户探讨如何构建、测试、上线和扩展 Agent 的一线专家，Ben 坦言自己当时愣了一会儿，甚至花了一分钟才组织好如何通俗准确地回答。行业内其实存在一种认知模糊——正如他展示的一句名言所言，“到现在其实并没有人真正完全搞懂 AI Agent 到底是什么”。为了活跃气氛，Ben 播放了一段使用 Google 的 Gen Media 媒体生成工具制作的 AI 歌曲，欢快地唱出“未来已来，AI Agent 流淌在我们血管中，但没人真正知道它是什么”。 ```mermaid flowchart TD accTitle: Agent Architecture Comparison subgraph LLM["大语言模型 (LLM)"] A["Reasoning Engine (思考与推理大脑)"] end subgraph AgentSys["AI Agent 完整智能体系统"] B["规划与分解 (Planning)"] C["上下文与记忆 (Memory Bank)"] D["外部工具与动作 (Tools & Actions)"] E["协同编排与运行时 (Orchestration & Runtime)"] end A --> AgentSys ``` ### [01:43](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=103s) 从大脑到完整系统：AI Agent 的多维度编排架构 Ben 深入剖析了 LLM 与 Agent 的核心差异： - **LLM 是大脑与推理引擎**：它负责理解输入并给出单次的语言生成与推理结果。 - **Agent 是完整的系统与执行者（Worker）**：它负责多任务之间的协同编排（Orchestration），将大脑的推理能力与记忆、规划、外部工具和运行时环境连接起来。如果仅试图通过一个极其庞大繁复的 Prompt（提示词）让单一 LLM 去完成复杂的多步骤任务，模型往往会在长链路中“逐渐丢失上下文、偏离目标，甚至产生幻觉”。虽然研发团队一直在极力减少和纠正幻觉，但单靠模型本身无法替代系统工程。Agent 的本质是通过系统架构来编排多项动作，而非仅仅进行一问一答的对话。它涵盖了任务规划与拆解、长期记忆保持、工具调用以及底层执行运行时（Runtime），从而可靠地把预定目标端到端地交付完成。 ### [03:23](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=203s) 企业级规模化落地：定义规模与 Gemini Enterprise 平台演进随着 Agent 开始大量涌入企业内部，企业管理者和技术团队普遍面临关于“规模化（Scale）”的困惑。Ben 指出，在谈论落地前，首先必须明确企业口中的“规模化”具体指什么： - 是指 Agent 的数量规模（从运行 1 个 Agent 增长到 100 个 Agent）？ - 是指用户并发与覆盖规模（从单个员工试用推广到全公司成百上千名员工日常使用）？ - 还是指分发与触达的规模（如何将技术团队或业务团队构建的所有 Agent 统一呈现在所有技术与非技术用户面前）？为了支撑这种全方位的规模化需求，Google 推出了 **Gemini Enterprise Agent Platform**（即此前 Vertex AI 平台的全面升级与重塑）。该平台覆盖了 Agent 的全生命周期，围绕四大核心支柱构建： 1. **构建（Build）**：提供面向非技术人员的无代码 Agent Studio 与面向专业开发者的代码开发套件。 2. **规模化（Scale）**：提供托管式 Agent 运行时、会话管理与 Memory Bank（记忆库）。 3. **治理（Govern）**：提供全面的日志追踪、会话洞察、安全准入与阻断机制。 4. **优化（Optimize）**：监控分析 Token 消耗与成本效益，避免陷入“每天烧掉 14 万亿 Token 却只用来问天气预报”的低效境地。 ### [05:54](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=354s) 无代码 Agent 构建：Gemini Enterprise 应用与实战案例针对 Agent 的构建路径，Ben 区分了两种主要形态：基于 GUI 自然语言交互的无代码体验（Agent Studio），以及面向专业开发者的 Pro-code 模式。在此处，他重点介绍了企业级入口级产品——**Gemini Enterprise 应用程序**。该应用不仅是企业的统一智能对话入口，更扮演着企业专属的“Agent 统一门面（Central Pane of Glass）”。它原生连接企业的各类数据孤岛与 SaaS 软件，支持集中搜索与直接触发业务动作；同时，无论是由业务人员通过无代码方式创建的 Agent，还是由开发团队通过代码套件构建的复杂 Agent，都可以统一发布在该门户中供全员使用。为了直观展示其易用性，Ben 展示了他亲自为 Google 首席财务官（CFO）准备财报电话会议而搭建的“财报预备智能体（Earnings Prep Call Agent）”： - **生成过程**：他直接将大脑中构想的 Agent 需求用自然语言“一股脑倾倒出来”，让 AI 自动生成一份无代码提示词。随后将提示词粘贴到 Gemini Enterprise 的创建界面中，系统瞬间自动生成了一个包含根智能体（Root Agent）与多个子智能体（Sub-agents）的多层架构。 - **功能分工**：不同子智能体各自负责特定专业任务——有的负责全网抓取并分析宏观市场动态，有的负责调取该公司的历史财报与业务背景，有的负责撰写专项摘要简报，最终由协调者汇总生成完整的会议准备备忘录及演示文稿。 - **业务价值**：即使是完全不会写代码的业务人员，也可以在几分钟内可视化调整底层模型、修改指令描述、挂载企业专属数据源，快速完成企业级复杂智能体的构建、部署与全员共享。 ### [08:58](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=538s) 开发者视角：代码优先构建、运行时托管与全面治理对于技术开发人员而言，可以通过 Google 生态或外部开源开发工具（如各类 ADK）进行高代码（High-code/Pro-code）开发。但构建完成之后，企业面临的最大挑战仍然是如何将这些 Agent 交付给业务一线。在传统自建模式下，企业需要自行搭建并维护复杂的底层服务器集群与执行环境。而在 Gemini Enterprise 体系中： - **全托管基础设施与运行时**：平台原生封装了底层基础设施，企业无需管理复杂的服务器配置即可直接获得托管运行时、Session 会话管理和持久化记忆库（Memory Bank），轻松实现从 1 个 Agent 到 100 个 Agent 的无缝扩展。 - **企业级可观测性与治理管控**：提供细粒度的审计日志与会话透视能力，安全团队能够清晰洞察各部门员工输入了哪些查询、Agent 的实际执行轨迹是否符合预期，并具备在违规或异常时一键阻断访问的管理权限。 - **Token 经济学（Tokenomics）与治理防蔓延**：帮助企业全面治理“Agent 泛滥（Agent Sprawl）”，精确评估计算与 Token 开销，确保每一笔算力投入都能产出明确的业务价值。 ### [11:01](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=661s) 规模化代理中的信任与安全治理机制随着企业内部大语言模型（LLM）的快速扩散与无序扩张（LLM Sprawl），企业的预算正在被迅速蚕食。在规模化部署智能体（Agent）的过程中，“信任”（Trust）是不可或缺的核心要素。当企业拥有成百上千个智能体或数千名员工在使用智能体时，管理者必须面对一个根本问题：如何确保这些智能体的输出是合规且合法的？针对这一挑战，企业可以采用 Google Cloud 极其推崇的“人机协同”（Human-in-the-Loop）机制，由人工对智能体执行的动作和输出进行最终审核；而对于更复杂的生产场景，企业则可以构建专门的端到端质检智能体（QA Agent），例如“自我反思智能体”（Self-reflection Agent）或“以大模型为评判者”（LLM-as-a-Judge）架构。在这类架构中，核心业务代理完成任务后，质检代理会依据预先设定的格式和规则对结果进行终审校验，并输出验证报告。 ```mermaid flowchart TD accTitle: 智能体验证与安全防护流水线 subgraph Execution["核心任务执行与安全"] A["用户/系统请求"] --> B["业务执行 Agent"] B --> C["代码执行 Agent<br/>(安全沙箱审查)"] end subgraph Evaluation["信任与输出质检"] C --> D{"质量与合规校验"} D -->|自动化| E["自我反思 Agent / LLM-as-a-Judge"] D -->|人工介入| F["Human-in-the-Loop 审核"] E --> G["合规格式化输出"] F --> G end subgraph Protection["企业边界防护"] G -.-> H["品牌声誉保护"] G -.-> I["防数据外泄与权限管控"] end ``` 在安全性方面，安全不仅仅局限于代码层面的校验（如在循环末端配置代码执行代理来复核开发人员编写的代码），更关乎企业级防护的核心诉求： - **防范滥用与品牌保护**：确保输出内容符合企业品牌基调，防止模型生成违规内容并引发公关危机。 - **防止数据泄漏**：杜绝企业敏感数据或专有知识无意中泄露至外部互联网。 - **细粒度访问控制**：建立严密的权限体系，确保用户仅能访问其被授权的数据资产，严格阻断越权访问。 --- ### [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) 控制大模型成本蔓延与容量规划实战成本失控是当前企业在大模型落地过程中最普遍的痛点。解决成本问题的关键在于明晰智能体的具体职责、合理设计系统架构，并根据工作负载精准选择最合适的模型等级。高 Token 消耗的繁重任务并不一定需要昂贵的高级编程模型；如果只是常规的通用任务，完全可以采用轻量、低延迟的 Flash 系列模型来压低成本。在实际客户支持中，曾有企业在评估预置吞吐量（Provisioned Throughput，用于保障 LLM 吞吐容量并避免 429 请求超限错误）时陷入误区。该客户构建了一个复杂的 Multi-Agent 生态系统，并声称其系统负载仅为 1 QPS（每秒查询数）。然而经过底层链路深挖发现，外部终端用户发起的这 1 次查询，在后台级联触发了 15 到 16 次不同的 Agent 及大模型调用。这意味着真实的系统负载不是 1 QPS，而是 16 QPS。这直接导致其算力容量评估从最初预估的 105 个 GSU（Generative Scale Units，生成式扩展单元）暴增至 1,500 个 GSU。针对这种调用放大效应，架构优化的关键路径在于分层选型与精准降级： - **避免过度配置**：无需在所有链路上统一使用顶配的 Pro 系列模型，常规步骤可全面降级至 Flash 或更小尺寸的轻量模型。 - **杜绝预算“闪崩”**：社交媒体上屡见不鲜的案例显示，不少企业的全年 AI 预算在上线第 1 个月就消耗殆尽。当业务团队向 CFO 申请追加预算时，CFO 的普遍反馈是：“我可以追加预算，但你们同样会很快花光。请先优化架构并证明其实际的业务 ROI（投资回报率），证明员工不是在利用公司算力搞周末日间交易系统的副业，而是在交付切实的业务价值。” --- ### [15:34](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=934s) 智能体规模化落地核心原则与总结智能体的规模化（Scale）与定义因企业而异，但成功落地必须围绕几个核心问题建立清晰的方法论： - **业务定位与问题定义**：厘清开发人员或业务人员究竟要解决什么具体业务问题，而非盲目跟风。 - **架构与扩展维度**：明确系统架构与规模化的具体含义——是指从 1 个扩展到 100 乃至 1,000 个智能体，支持 100 到 1,000 名用户，还是指底层算力基础设施的自动化纳管。 - **受众与专业化分工**：确定系统服务于业务人员还是技术人员，合理编排专职代理以保持业务专注。 - **品牌安全与风控**：确保面向最终用户的体验始终忠于企业品牌形象，防止错误信息传播或在社交媒体上引发负面舆情。 - **全生命周期成本管控**：将成本、安全与可信度融入架构的每一次迭代之中。演讲最后，Ben 播放了一段由生成式多媒体（Gen Media）工具根据其 PPT 总结页截图一键生成的歌曲，歌词生动唱出“模糊的画面开始消散，灵光闪现，一切瞬间清晰到位”，呼应了智能体从混乱探索走向成熟落地的过程。Ben 鼓励参会者在 LinkedIn 上与其保持联系，共同探讨企业级 AI 的落地实践。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) 到底什么是 Agent？Agent 和 Gemini 应用有什么区别？Agent 和底层大语言模型（LLM）又有什么区别？ — Ben Yevin > [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) 到现在其实并没有人真正完全搞懂 AI Agent 到底是什么 — Ben Yevin > [00:04](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=4s) 未来已来，AI Agent 流淌在我们血管中，但没人真正知道它是什么 — Ben Yevin > [01:43](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=103s) LLM 是大脑与推理引擎，Agent 是完整的系统与执行者（Worker） — Ben Yevin > [01:43](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=103s) 逐渐丢失上下文、偏离目标，甚至产生幻觉 — Ben Yevin > [03:23](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=203s) 每天烧掉 14 万亿 Token 却只用来问天气预报 — Ben Yevin > [12:37](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=757s) 我可以追加预算，但你们同样会很快花光。请先优化架构并证明其实际的业务 ROI（投资回报率），证明员工不是在利用公司算力搞周末日间交易系统的副业，而是在交付切实的业务价值。 — Ben Yevin > [15:34](https://www.youtube.com/watch?v=OOMEVQCUX5Q&t=934s) 模糊的画面开始消散，灵光闪现，一切瞬间清晰到位 — Ben Yevin ## Mentioned Ben Yevin, Google Cloud, Gemini Enterprise, Gemini Enterprise Agent Platform, Vertex AI, Agent Studio, Memory Bank, LLM-as-a-Judge, GSU (Generative Scale Units), Flash 系列模型, Pro 系列模型, Gen Media, Ai4, LinkedIn ## Video description Ben Yevin - AI Solutions Specialist, Google Cloud While AI technology is advancing rapidly, many organizations still struggle to translate innovation into measurable business results. This keynote bridges the gap between technical execution and strategic value creation, exploring the common roadblocks that stall enterprise AI initiatives. We will introduce a practical framework for connecting AI capabilities directly to your bottom line, while teaching you the "how" of scaling. Learn how to accelerate the journey from prototype to global production by building AI agents capable of handling millions of interactions and taking real-time action. Join us to discover comprehensive strategies for managing the entire agent lifecycle and ensuring your data and systems grow seamlessly to drive true transformational impact. [Source](https://www.youtube.com/watch?v=OOMEVQCUX5Q)

Source quotes

[0:04] Ben Yevin: 到底什么是 Agent？Agent 和 Gemini 应用有什么区别？Agent 和底层大语言模型（LLM）又有什么区别？ [0:04] Ben Yevin: 到现在其实并没有人真正完全搞懂 AI Agent 到底是什么 [0:04] Ben Yevin: 未来已来，AI Agent 流淌在我们血管中，但没人真正知道它是什么 [1:43] Ben Yevin: LLM 是大脑与推理引擎，Agent 是完整的系统与执行者（Worker） [1:43] Ben Yevin: 逐渐丢失上下文、偏离目标，甚至产生幻觉 [3:23] Ben Yevin: 每天烧掉 14 万亿 Token 却只用来问天气预报 [12:37] Ben Yevin: 我可以追加预算，但你们同样会很快花光。请先优化架构并证明其实际的业务 ROI（投资回报率），证明员工不是在利用公司算力搞周末日间交易系统的副业，而是在交付切实的业务价值。 [15:34] Ben Yevin: 模糊的画面开始消散，灵光闪现，一切瞬间清晰到位

Source quotes

[0:04] Ben Yevin: What is an agent? What's the difference between an agent and a Gemini app? What's the difference between an agent and an LLM? [0:04] Ben Yevin: no one really knows what an AI agent is [0:04] Ben Yevin: AI agents in our veins [4:26] Ben Yevin: burning 14 trillion tokens a day asking for the weather report. [7:14] Ben Yevin: word vomited all over the page [9:04] Ben Yevin: LLM sprawl [9:04] Ben Yevin: tokenomics [12:37] Ben Yevin: building day trading systems on the weekend.

Original transcript

[0:04] Welcome to the talk. If you were in the session earlier, we you heard him talking about [0:09] how to build agents and the different options we have. The goal for me today is really to [0:14] talk about scaling agents and what that means. So, you know, I'm Ben. I'm an AI [0:21] solution specialist here at Google Cloud. I've been on the team since it was founded 2 and 1/2 [0:25] years ago or so at this point and I've been at Google for for just over 6 years. So, for today, [0:32] really what we're looking at Oh, my speaker notes are up there. You guys don't need to see that. [0:39] Uh first thing, level set. I usually don't do this, but we're going to talk about what is an [0:45] AI agent and I say that because recently I was in a session with a customer who side barred me on [0:50] the side and was like, "What is an agent? What's the difference between an agent and a Gemini app? [0:55] What's the difference between an agent and an LLM?" So, we're going to level set there. [0:59] We'll talk about scaling agents and then we'll talk about security, governance, and cost. [1:06] So, I love this quote because it it's it's a little dated at this point, [1:10] but no one really knows what an AI agent is. So, what I found to be interesting about this [1:16] is in that customer interaction, he genuinely did not know what an agent was. And from somebody [1:22] who sits in the seat all day every day and is constantly talking to customers and you know, [1:28] enterprises about how to build agents and scale them and put them into production and test, [1:33] I was like a little bit of miffed and honestly, it took me like a minute to figure out how to [1:37] answer the question. Um so, we're going to see if this works. I used AI to create a song. [1:43] The future's here. It's coming fast. A digital world that's built to last. We've got the power, [1:52] got the brains, got AI agents in our veins. No one really knows what an AI agent is. [music] [2:04] All right, that's a little plug. Used Google's Gen Media tool set there where we uh to to create [2:10] the soundtrack and the song. Uh I've got another one towards the end. Uh so hope that was a little [2:16] uplifting for the day. Uh but coming back to this, LLMs, agents, right? LLM is the brain. It's It's [2:22] what's giving you the response. The agent is the orchestration between all the different pieces [2:28] of what you're trying to achieve, right? Like it says the worker up here. But think about having [2:33] to do multiple things. If you did that with just an LLM, you could have a very robust prompt. But [2:38] as we heard earlier, it starts to lose context. It starts to stray. Sometimes it could hallucinate, [2:43] though we worked very hard to reduce and redact that hallucination. And at the end of the day, [2:51] the the the clear definition is orchestrating multiple things versus just doing having a chat [2:57] conversation. All right, for the visual learners, same thing here, right? Your LLM is the brain at [3:04] the top. It's that reasoning engine. It's what gives you that output. But then there's all these [3:08] other pieces that are part of the orchestration of what you're trying to achieve. The agent would [3:12] be the whole system here that you see. And then you have like the brain, the LLM, the memory, [3:17] the planning, the reasoning. So that's what, you know, the AI agents do. So in particular, [3:23] uh agents are now flooding organizations, right? And that brings a lot of questions as to scale. [3:31] What does scale mean? Does scale mean I'm going from one agent to 100 agents? Does scale mean I'm [3:37] having one agent being used by one person to 100 people? Right? Does scale mean how do I put all [3:45] of the agents that the organization organization has built in front of the users, right? There's [3:50] technical users, there's business users. So, the first step is how do you actually define scale? [3:56] Right? We'll go through this real quick. This is a more the last slide about agents and what they do, [4:01] right? They plan and execute tasks. They have the different pieces, their models, [4:06] the tools, the orchestration, and the runtime. And that's the difference between an LLM and an agent, [4:13] right? Your agent has all of these, it's a it's a it's a system that works to do the task it was [4:19] set out to do. Whereas an LLM will give you a response based off of what you're asking. [4:26] Okay, so I have this slide. We're going to talk about it twice. Uh but the the main things that [4:31] I want to point out here, this is the Google is really good at this, like Gemini Enterprise, [4:38] if if anybody's ever heard of that. Uh there's Gemini the app, there's Gemini Enterprise the app, [4:45] there's Gemini Enterprise Agent Platform, there's Gemini in BigQuery, there's Gemini in Workspace. [4:52] Gemini's everywhere. But from Google Cloud perspective for application development in AI [4:59] and for agent development and for orchestration, we have the Gemini Enterprise Agent Platform. [5:04] For those true Google fans, this used to be called Vertex. This is the the rebranded Vertex platform. [5:11] Same concepts, but what's important here, you have got build, scale, govern, and optimize. [5:17] And what you can see, it's probably hard for everybody to see, the little blue boxes [5:22] next to each of the words, they either say new or GA. And the fact that we have all of this is just [5:28] to show how we're investing in the platform to give people the ability to do those things like [5:34] scaling and governing, security, optimizing. How do we How do we track what the enterprise is doing [5:40] with their agents? How do we get the logs out? How do we know what each person is actually doing? [5:44] Is the agent actually doing what it was set out to, or are we just, you know, [5:49] burning 14 trillion tokens a day asking Fable for the weather report? [5:54] Okay. Uh, in the last session we heard a little about a little bit about this. I want to build an [5:58] agent. Uh, we spoke about Agent Studio, which I would refer to as like a GUI, a natural language [6:05] experience, no code. Uh, and then you have your pro code, your your technical users, uh, [6:12] who are going to develop an ADK using dev tools, who are going to need an a runtime to deploy the [6:18] agents and get them at scale. Um, so from here I'm going to talk about one that we didn't mention in [6:23] the last session, which is Gemini Enterprise the application. So this is Google's agentic platform [6:30] that we are putting out for the enterprise. And ultimately what it is is it's the Gemini app [6:36] coupled with the connections to all of your data silos, right? And your SaaS applications, so you [6:42] can bring them all in to a central location and make them searchable and make them usable and have [6:47] actions in there, plus an agentic system, which means that this is the place where all of your [6:52] users inside your company can go to interact with all of the agents that the enterprise has built. [6:58] That could be built in this platform by business users who are not using code. [7:02] It could be built outside of this platform, uh, in like an ADK type fashion and brought in as that [7:08] like central pane of glass. How do we give people access to all of the agents that we've built? [7:14] So this is Gemini Enterprise the app. This is an agent that I built, um, and you could see in the [7:19] upper left, uh, natural language. You don't see the whole query there, it's a screenshot. [7:24] I wasn't sure that I was allowed to do demos. If I would have, I'd be in the platform right now. [7:29] Um, but you can see up in the left, I used AI to create my AI agent. Uh, I just word [7:36] vomited all over the page everything that was in my mind about the agent I wanted it [7:41] and I said create a no-code prompt for me. So, on the upper left, it created the no-code prompt. [7:46] In Gemini Enterprise, I went to create an agent. I pasted the prompt, and the prompt [7:51] built this agent, right? You have a root agent with multiple sub-agents. You can click into each [7:56] one of them. You could change the model. You could change the description of what you want it to do. [8:01] You can ground it in certain information. This particular agent is an earnings prep call agent. [8:06] So, it was built for the CFO of Google to prepare for upcoming earnings calls. And you could see the [8:14] different pieces that you have in an agent. Like, yeah, could you do this with one huge prompt? [8:18] Probably. Would the results come way you want it? Probably not, because somewhere along the lines [8:23] you'll lose that context and that memory. But you could see like it's looking at the [8:28] it's going to crawl the market. It's going to look for it's going to create a brief. [8:32] It's going to look at all of the historical context around earnings for that company. [8:36] And then it's going to put it all together into a prep document and presentation. So, the the [8:41] reason why I show this is for non-developers, there's a way for you to build agents. There's [8:46] a way for you to deploy them. There's a way for you to scale them. There's a way for you [8:50] to share them across your organization. All of that can be done in the Gemini Enterprise app. [8:58] Okay. So, now talk a minute here about uh high-code agents, pro-code agents. [9:04] Right? We heard it earlier. There's the Google ecosystem. There's the external ecosystem. [9:10] Uh but ultimately, you need to determine what do you want your agent to do and how do you want to [9:15] do it? And then you'll come in here and you'll use these tools to build it. From there, you have to [9:21] figure out how do you get this into the hands of the people that are actually using the agents? [9:25] And if they're all technical and you have your own platform, which a lot of enterprises do, they'll [9:29] go there. But if you're looking to extend this out to business users, like once the work is done, you [9:34] need something else like the the Gemini Enterprise app, right? So, there's different ways of [9:38] surfacing this, which then goes to scaling, right? In the Gemini Enterprise app, you don't need to [9:44] worry about runtime, you don't need to worry about infrastructure. Like, you you bought it, [9:49] you have it. Your employees that use it are able to do whatever they need to do uh without having [9:55] to manage the underlying infrastructure. It's all included, right? And you could see uh up here [10:01] uh in build. That's one of the things you could do. Uh if you choose to do or follow the technical [10:07] path, right? Then you have the things under scale, right? Like your agent runtime, your sessions, [10:13] your memory bank. These are the things that allow you to go from uh one agent to 100 agents [10:19] without having to worry about the infrastructure, because the infrastructure was managed. So, [10:24] you deploy it and you you assign it, but you don't actually manage the underlying infrastructure. [10:28] You just get it going. And then on top of that, you have the ability to to govern the platform, [10:33] right? And what does that mean? Like, it's your insights into the sessions that people are using [10:38] with your agents. It's pulling out the logs. It's seeing what what queries people are putting. It's [10:42] being having the ability to stop somebody from using an agent or from blocking it from something [10:47] of that nature. And then, of course, the ability to optimize. Um you know, I'm sure everybody in [10:52] this room has heard the term tokenomics. Uh and, you know, how IT or agent sprawl, [11:01] LLM sprawl is eating enterprises budgets. Um and how do you optimize against something like that? [11:09] Okay. So, from scale, included in scale is trust. As you have more agents or more people [11:16] using agents, how do you trust the output of those agents? How do you know it's legitimate? [11:22] Are you using something that we are very favorable of, human in the loop, right? Does there a human [11:26] that comes in and checks what the agent has done and what they're trying to do? Or like, [11:31] are you really sophisticated and you're creating an end agent, a QA agent like here you see there's [11:37] a self-reflection agent, an LLM as a judge, but after your agent has done everything that you want [11:42] it to do, there's one more step to validate all of that stuff and produce a report against or produce [11:48] an output against a set of you know format that you've defined ahead of time. So, how do you trust [11:56] it? We give you those tools to make that happen. How do we think about safety? In this particular [12:02] example, they're talking about safety within code. So, there's a code execution agent that was added [12:07] at the end of the loop to come back and check the developer's code, but more often than that or more [12:12] importantly than that, like safety is about abuse, it's about protecting your enterprise's brand, [12:18] it's about ensuring safety that you know, you're not leaking information outside into the internet, [12:25] it's about ensuring that people have access to what they're supposed to have access to [12:30] and they don't have access to what they're not supposed to have access to. And then costs. So, [12:37] I've had a handful of conversations in the last month or so about costs being out of control [12:42] and I think it comes down to you know, number one going back to defining what is your agent doing, [12:48] how are you architecting it and then when you choose your model that you're choosing the right [12:52] model for the workload, right? Like you can have a very heavy workload from a token perspective, [13:02] but maybe it's not coding, right? And you don't need one of those high-powered coding models. [13:06] Maybe you have a a workhorse workload that you can use a flashlight model for because you just [13:11] need the results fast, you need it cheap. I had a conversation with one of our customers about [13:16] a week ago and we were sizing their environment and they built like a whole multi-agent ecosystem [13:22] and we were looking at something called provision throughput which gives the LLMs [13:25] guaranteed capacity and throughput for you to get a response and not have the 429 errors. [13:31] But, when they gave me the metrics to size it, something didn't make sense. I was like, [13:35] wait a minute. Like, you have you have 15, 16 different agent and agents and LLM calls, [13:42] but you're telling me that there's only like one query per second. And then when we dove deeper, [13:46] like, yeah, there's one query per second from the first query of of whoever puts that [13:52] query into the system. This happened to be an external customer on their platform. Uh but, [13:58] that query trigger triggered 16 LLM calls. So, it's not really one query per second. [14:04] It's 16 queries per second. Right? Okay, well, that drastically changed the size of [14:09] the environment that we were talking about. Right? We went from, you know, 105 GSUs to 1,500 GSUs, [14:16] uh generative scale units for for those of you that don't know that term. That's what we use [14:21] to measure provision throughput. Uh and what we saw was there were a lot of places to optimize, [14:27] right? Like, we're not going to use Gemini 3.5 uh flash for this. We're going to use 3.1 flashlight [14:33] for that. Or we don't need to use a pro model for this. We're going to use a flash model for that. [14:38] So, all of those things are are what customers are starting uh to to look at when they are thinking [14:45] about these agentic ecosystems. Um you know, there's a lot of very public customers uh or or [14:50] companies, I should say, out there. You see it on LinkedIn. You see it on social media. Uh their AI [14:56] budget for the year uh was consumed in 1 month. Right? Okay, well, now you don't have money for [15:03] the rest of the year. What are you going to do? You're going to go to the CFO and get more budget? [15:08] Which one of my customers did. And what did the CFO say? Okay, I'll give you more budget, but [15:12] you're going to blow through that, too. There's no there's nothing else there. Why don't you refine [15:16] what you're doing and come back and show us how you deliver the on the ROI. Right? Show us how [15:21] what you're actually doing is not your employees building day trading systems on the weekend. Uh [15:28] but they're actually building enterprise stuff that is is producing and returning value to us. Um [15:34] So, I know I'm coming up here on time. So, takeaways from this, right? Like, where do [15:42] you want your developers or business users to start? What is the problem you're trying to solve? [15:47] How are you architecting the agent or the ecosystem for what you're looking for? [15:53] How do you scale it? Does scale mean one to 100 to 1,000 agents? Does scale mean one to 100 to 1,000 [15:59] people? Does scale mean managing infrastructure? Does scale mean having managed infrastructure? [16:05] Are you looking for business users? Are you looking for technical users? Right? How do [16:10] you add specialized agents? How do I ensure that the agent stays on brand? How do I ensure that [16:15] whatever I'm doing, if it's a user experience, our brand is kept true. We're not leaking things. [16:22] We're not putting out bad press. We're not doing things that are going to go viral on the internet [16:27] uh on social media because something happened that shouldn't have happened. How do we manage it? [16:34] How do we look at it from a safety perspective? And uh how do we manage the costs, right? All [16:40] of these play into scale and everybody has a different definition of scale, uh the same way [16:45] most people have a different definition of agents. Um so, let me see if my last slide is here. I [16:53] I think uh it might be gone. I had another another song. Maybe I can find it real quick. [17:01] [music] The fuzzy picture [singing] starts to fade. [music] A perfect insight has been made. [17:10] The lightbulb moment [singing and music] shining bright, [17:14] everything just clicks into place. [singing] Oh, everything [music] just clicks into place. [17:28] [music] [17:29] Uh so, last plug on the Gen Media piece. I put a very simple prompt in, like, "Hey, [17:34] create a song for me based off of a off of the picture that I've attached." The picture [17:39] was a screenshot of my takeaway slide. And the first song was very similar. Give me an upbeat uh [17:46] music song about uh what is an AI agent and how nobody actually knows how to define it. Anyway, [17:53] my name is Ben. Feel free to connect with me on LinkedIn. Uh I'll be here for a little bit [17:56] if anybody has any questions, but that is it for today. So, enjoy the rest of the show.