Google Cloud Tech
Scaling enterprise AI agents requires replacing manual prompt engineering with automated trajectory evaluations, ephemeral machine identities, and standardized deployment templates.
AI-assisted summary of From prototype to production: 45 minutes to a reliable Gemini Enterprise Agent Platform agent — original source. Check the original for full context.
Source published:
Key ideas
- Booking.com evolved its generative AI stack from basic 2023 wrappers to an enterprise gateway and production agent platform handling multi-vertical travel workflows at scale. (2:30)
- Agent evaluation must assess both final responses and internal execution trajectories, using exact, in-order, or any-order matching against golden test datasets. (13:35)
- Application Design Center (ADC) and Gemini Cloud Assist translate local agent codebases into Terraform templates and cloud architectures in roughly 30 minutes. (23:15)
- Model Armor acts as an inline threat filter stationed between end users and LLMs to prevent PII leakage, prompt injections, and system prompt extraction without fragile prompt engineering. (30:06)
- OpenTelemetry has been extended with GenAI-specific signals to capture model metadata, token consumption, and multi-turn tool traces alongside conversational BigQuery analytics. (35:57)
Detailed notes
## Background Published on YouTube by Google Cloud Tech on June 25, 2026, this Google Cloud Next session presents a production-first guide to deploying hardened agents on the Gemini Enterprise Agent Platform. The session description highlights practical architectures for connecting agents via Model Context Protocol (MCP) servers, securing them with enterprise guardrails, monitoring them with real-time observability, and ensuring accuracy through automated evaluations.
## TL;DR Engineers from Booking.com and Google outline the architectural transition required to move autonomous GenAI agents from local prototypes into hardened, enterprise-grade cloud deployments. Drawing on Booking.com's high-throughput travel platform and a multi-agent London concierge sample application, the speakers detail four foundational pillars: multi-turn response and trajectory evaluation, accelerated cloud deployment using Application Design Center (ADC) and Gemini Cloud Assist, runtime security enforced by ephemeral agent identities and Model Armor proxies, and day-zero observability powered by OpenTelemetry and BigQuery analytics. The session demonstrates how platform-level governance, agent gateways, and golden architectural templates allow organizations to safely scale collaborative AI agents without compounding manual engineering overhead.
## Atmosphere & style The session carries the crisp, rapid-fire momentum of an applied engineering masterclass, balancing large-scale corporate credibility with hands-on console demonstrations. Alibek Datbayev and Maria-Irène Călinoiu set a grounded foundation with real production metrics from Booking.com, before Manasa Kandula and Naz Bayrak engage in an energetic tag-team delivery filled with natural onstage rapport and self-deprecating banter about audience clapping and minor AV screen hiccups. The mood remains pragmatic and transparent throughout, openly addressing production realities like inevitable system failures and high token costs while equipping viewers with concrete, actionable architecture patterns.
## Bulletin - [02:30](https://www.youtube.com/watch?v=fkCTifAqVGg&t=150s) Booking.com evolved its generative AI stack from basic 2023 wrappers to an enterprise gateway and production agent platform handling multi-vertical travel workflows at scale. - [13:35](https://www.youtube.com/watch?v=fkCTifAqVGg&t=815s) Agent evaluation must assess both final responses and internal execution trajectories, using exact, in-order, or any-order matching against golden test datasets. - [23:15](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1395s) Application Design Center (ADC) and Gemini Cloud Assist translate local agent codebases into Terraform templates and cloud architectures in roughly 30 minutes. - [30:06](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1806s) Model Armor acts as an inline threat filter stationed between end users and LLMs to prevent PII leakage, prompt injections, and system prompt extraction without fragile prompt engineering. - [35:57](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2157s) OpenTelemetry has been extended with GenAI-specific signals to capture model metadata, token consumption, and multi-turn tool traces alongside conversational BigQuery analytics.
## Key takeaways - [02:30](https://www.youtube.com/watch?v=fkCTifAqVGg&t=150s) Booking.com runs hundreds of production ML models serving 500,000 requests per second with a p999 latency target around 50 milliseconds. - [08:45](https://www.youtube.com/watch?v=fkCTifAqVGg&t=525s) Production conversational agents require tailored serving tiers like provisioned throughput rather than default on-demand endpoints to guarantee latency and tokens per minute. - [08:45](https://www.youtube.com/watch?v=fkCTifAqVGg&t=525s) Building enterprise AI platforms requires investing equally in platform infrastructure and internal developer enablement, documentation, and hackathons. - [13:35](https://www.youtube.com/watch?v=fkCTifAqVGg&t=815s) Evaluating non-deterministic agents requires trajectory evaluation to inspect tool calls, parameters, and sequence fidelity rather than black-box response checks alone. - [16:11](https://www.youtube.com/watch?v=fkCTifAqVGg&t=971s) Trajectory evaluation matching rules range from strict exact matches for high-stakes operational workflows to flexible any-order matches for open-ended recommendations. - [18:32](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1112s) Automated simulation runs benchmark multi-turn conversations against golden JSON datasets and stream granular traces directly into BigQuery. - [20:05](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1205s) Gemini Cloud Assist and the Application Design Center bridge the prototype-to-cloud gap by generating Terraform configurations and visual architecture canvases from source code. - [25:52](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1552s) An enterprise Agent Registry acts as a centralized catalog to prevent duplicate tooling, while an Agent Gateway enforces platform-level communication boundaries. - [27:58](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1678s) Agent Identity dynamically provisions ephemeral, least-privilege IAM credentials at runtime, producing granular audit trails tied to exact agent versions. - [30:06](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1806s) Model Armor secures agents against PII exfiltration, prompt injection, and jailbreak attempts at the network layer rather than relying on endless prompt engineering. - [33:38](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2018s) Embedding native observability into SDKs shifts telemetry from an expensive day-two afterthought to a day-zero operational standard. - [35:57](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2157s) OpenTelemetry extensions capture GenAI-specific metrics, including model checkpoint versions, inference latency, temperature settings, and tool invocations. - [38:00](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2280s) The Agent Development Kit BigQuery plugin streams conversational turns and metadata into SQL tables for business sentiment and token cost analysis. - [42:02](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2522s) ADC golden templates allow platform teams to encode security guardrails, IAM standards, and observability pipelines into pre-approved architectures that scale across development teams.
## Detailed walkthrough ### [00:13](https://www.youtube.com/watch?v=fkCTifAqVGg&t=13s) Introduction and the Booking.com & Google Partnership
Alibek Datbayev, an engineering manager at Booking.com leading the AI Platform team, opens the session on taking reliable agents from prototype to production. He introduces his colleague Maria-Irène Călinoiu alongside Googlers Manasa Kandula and Naz Bayrak, setting the stage for a practical masterclass on hardened enterprise AI. Alibek highlights that Booking.com has collaborated with Google for years beyond traditional performance marketing: “It’s a battle tested partnership that is built on deep trust and technology that actually delivers.” Booking.com actively leverages Gemini and Vertex AI to unlock automated, smarter travel experiences centered around its foundational mission—making it easier for everyone to experience the world.
To emphasize that mission, Alibek asks the audience for a show of hands on who traveled internationally to attend the conference, observing that almost the entire room raised their hands. He follows up by asking how many used the Booking.com app to plan their trip, expressing excitement at the strong response. He underscores that travel fosters personal growth and cultural understanding precisely because people rediscover what they love and explore unfamiliar territory, directly embodying the core purpose behind the products his team supports.
[02:30](https://www.youtube.com/watch?v=fkCTifAqVGg&t=150s) Scale of the Booking.com Platform and Evolution of Gen AI
Alibek provides concrete numbers demonstrating the sheer magnitude of Booking.com's operations. Rather than functioning simply as a hotel reservation website, it operates as a fully integrated multi-vertical travel ecosystem covering accommodations, flights, rental cars, attractions, and ground transport. The platform's scale is backed by massive production metrics:
- **6.8+ billion** guest arrivals facilitated since 2010. - **175,000** global destinations covered across 70 countries. - **140** international offices, with roughly 7,000 employees based at the global headquarters in Amsterdam, Netherlands. - **10+ years** of running predictive machine learning models in production. - **Hundreds of production ML models** handling approximately 500,000 requests per second with a p999 latency target around 50 milliseconds.
```mermaid flowchart LR accTitle: Booking.com Generative AI Evolution accDescr: Timeline of Booking.com AI maturity from Phase 0 experimentation to an enterprise gateway and production agent platform. subgraph S1["Phase 0: 2023"] A["ChatGPT Moment<br/>Simple wrapper apps"] end subgraph S2["Phase 1: Early 2024"] B["Stack revamp<br/>In-house models<br/>Multi-provider setup"] end subgraph S3["Phase 2: Gateway"] C["Single API gateway<br/>Built-in governance<br/>Usage monitoring"] end subgraph S4["Phase 3: Production"] D["Enterprise Ecosystem<br/>Evals & safety guardrails<br/>Agentic capabilities"] end A --> B B --> C C --> D ```
Turning to generative AI, Alibek traces the architectural progression since the "ChatGPT moment" in 2023. Phase 0 began with basic API wrappers to get a foot in the door. By early 2024, Booking.com revamped its core stack to incorporate in-house models across a multi-provider landscape. The pivotal milestone occurred when the platform unified all access under a single internal gateway featuring built-in governance, cost attribution, and usage monitoring. Today, Booking.com manages the complete generative lifecycle, spanning advanced evaluations, enterprise safety guardrails, and production-ready agentic capabilities to scale multiple travel use cases rapidly.
[05:47](https://www.youtube.com/watch?v=fkCTifAqVGg&t=347s) Gemini Use Cases at Booking.com
Maria-Irène Călinoiu takes over to detail four distinct applications powered by Gemini models across customer-facing, internal engineering, and partner-facing workflows. The flagship customer-facing project, built as a proof of concept alongside Google engineers, is titled **Know Before You Go**. It tackles the stressful gap between booking an itinerary and arriving at the destination. Travelers can dynamically generate customized itineraries based on group dynamics or time constraints and figure out airport-to-hotel transit. Under the hood, the system relies on Gemini grounded with Google Maps and Google Search to merge real-time external knowledge with complex reasoning.
```mermaid flowchart TD accTitle: Gemini Use Cases at Booking.com accDescr: Four production and POC generative AI use cases across customer, internal, partner, and multimodal touchpoints. subgraph External["Customer & Partner Touchpoints"] A["Know Before You Go<br/>(Grounding via Maps & Search)"] C["Partner Analytics<br/>(Review signal extraction)"] D["Reels to Reality<br/>(Video URLs to itineraries)"] end subgraph Internal["Developer Productivity"] B["BigBot Registry Chatbot<br/>(Data discovery & workflows)"] end ```
Beyond customer planning, Maria-Irène outlines three other deployments: - **BigBot (Internal Data Registry):** A developer-facing chatbot that helps engineers locate data assets across a vast corporate repository, ask architectural questions, and build natural language automated workflows. - **Partner Analytics (Host Facing):** Instead of using large language models solely for text generation, this application performs structured signal extraction on raw customer comments and reviews. Property partners can immediately spot recurring operational pain points, such as complaints regarding pool maintenance or breakfast variety, and prioritize repairs. - **Reels to Reality (Multimodal Travel Planning):** Recognizing that travelers draw heavy inspiration from social video, this project accepts direct YouTube URLs into Gemini to parse video contexts and automatically assemble concrete, bookable itineraries within the Booking.com app.
[08:45](https://www.youtube.com/watch?v=fkCTifAqVGg&t=525s) Production Learnings: Serving Tiers and Platform Enablement
Maria-Irène emphasizes that finding a suitable model checkpoint is minor compared to solving how models are served and consumed internally. Booking.com experimented extensively across Vertex AI serving tiers, learning early on that standard on-demand endpoints were neither fast nor predictable enough for conversational applications like Know Before You Go. The team migrated to provisioned throughput to guarantee dedicated tokens per minute, and adopted priority processing to handle demanding multimodal workloads like Reels to Reality. As Maria-Irène observes, “The real takeaway has been that we are trying to match the serving strategy with the production need, rather than just assume that a default is going to do the trick.”
The second operational challenge centers on knowledge enablement. Because the AI platform team provides self-service infrastructure rather than writing application code for every product vertical, building the platform constitutes only half the battle. The other half involves running internal hackathons, compiling up-to-date documentation on model providers, and guiding product teams on which architecture fits their latency and cost profile. She encapsulates the company's trajectory: “Booking.com has been moving from isolated experiments to a governed multi-provider production ready AI ecosystem.”
[11:13](https://www.youtube.com/watch?v=fkCTifAqVGg&t=673s) The Four Pillars of Production Agent Architecture
Manasa Kandula steps up to transition the presentation from Booking.com's user journey into the core engineering curriculum for deploying Gemini Enterprise agents. She identifies the central anxieties development teams encounter once they push past initial toy prototypes: guaranteeing agent quality against strict corporate metrics, maintaining control as the sheer count of tools and downstream agents scales up, and keeping production systems locked down against novel security vulnerabilities.
To resolve these enterprise bottlenecks, Manasa outlines the four core pillars that she and Naz Bayrak will unpack across the remainder of the session: 1. **Pre-Production Agent Evaluation:** Implementing automated evaluation techniques, benchmarks, and multi-turn scoring rubrics before code ever touches real users. 2. **Accelerated Code-to-Cloud Deployment:** Best practices for packaging code and moving from local development into managed cloud infrastructure cleanly and quickly. 3. **Runtime Agent Identity and Security:** Provisioning fine-grained agent identities, implementing model guardrails, and enforcing least-privilege tool security during live execution. 4. **End-to-End Observability and Auditability:** Establishing complete tracing, inspection, and debugging capabilities to isolate failures and monitor agent reasoning paths in real time.
[12:25](https://www.youtube.com/watch?v=fkCTifAqVGg&t=745s) Introducing the Speakers and the London Concierge Sample Demo
Naz Bayrak introduces her colleague Manasa Kandula, an application modernization specialist, and introduces herself as an AI specialist, noting that both are based out of Google’s Amsterdam office. To ground the discussion in practical enterprise architecture, they walk through a multi-agent travel application inspired by their colleagues at Booking.com. At the core of the system is the **London travel concierge**, the primary front-facing agent designed to interact directly with users and provide personalized itinerary recommendations.
The London concierge orchestrates tasks by communicating directly with two downstream agents: a ticket agent and an activities agent. To handle dynamic conditions, the system connects to external environments through Model Context Protocol (MCP) servers. Because London weather is notoriously unpredictable, the concierge relies on a weather MCP server for live updates. In parallel, it queries a custom London data MCP server that bridges the agent to proprietary enterprise information stored in Cloud SQL, ensuring that itinerary and venue suggestions reflect authoritative business data rather than generic web knowledge. The entire infrastructure is backed by Vertex AI Agent Engine and hardened using Model Armor to maintain enterprise guardrails.
```mermaid flowchart TD accTitle: London Concierge Multi-Agent Architecture accDescr: Architecture showing the London travel concierge routing requests to sub-agents and external MCP servers backed by Cloud SQL, Agent Engine, and Model Armor.
User(["End User"]) --> MainAgent["London Travel Concierge\n(Main Orchestrator)"]
subgraph SubAgents ["Collaborating Sub-Agents"] TicketAgent["Ticket Agent"] ActivitiesAgent["Activities Agent"] end
subgraph DataIntegrations ["MCP Server Integrations"] WeatherMCP["Weather MCP Server\n(Live Weather)"] DataMCP["London Data MCP Server"] CloudSQL[("Cloud SQL\nPostgreSQL + Vector")] DataMCP --> CloudSQL end
MainAgent --> TicketAgent MainAgent --> ActivitiesAgent MainAgent --> WeatherMCP MainAgent --> DataMCP
subgraph EnterpriseInfra ["Platform Foundation"] AgentEngine["Vertex AI Agent Engine"] ModelArmor["Model Armor Security"] end ```
[13:35](https://www.youtube.com/watch?v=fkCTifAqVGg&t=815s) Evaluation Paradigms: Response vs. Trajectory
Evaluating agents is fundamentally more complex than evaluating traditional machine learning models because agents operate non-deterministically. Naz highlights this inherent unpredictability: “Agents are non-deterministic and this brings some challenges to our evaluation systems.” She divides agent evaluation into two major components: - **Response Evaluation:** Treating the agent as a black box, this method disregards everything occurring internally during execution and focuses strictly on the final output. This mirrors standard LLM evaluation methodologies. - **Trajectory Evaluation:** Unique to autonomous multi-agent systems, this method peers into the internal reasoning path. It inspects which specific tools were invoked, which parameters were supplied, which sub-agents were engaged, and the precise sequence of collaboration. Evaluators match this execution path against a predefined "golden goal" dataset to verify operational fidelity.
[14:30](https://www.youtube.com/watch?v=fkCTifAqVGg&t=870s) Evolution from Human-in-the-Loop to LLM-as-a-Judge and Automated Metrics
Four years ago, evaluation relied almost entirely on human review: an evaluator inspected an output and manually determined whether it was good enough based on business criteria. However, manual human evaluation is inherently subjective, expensive, and difficult to scale across thousands of production requests. The industry adapted by moving toward **LLM-as-a-Judge**, where a secondary language model is tasked with grading the output of the primary model. Naz explains: “Now the advantage of using LLM as Judge is very scalable. It's efficient and consistent.” Despite these benefits, LLM-as-a-Judge has clear constraints: it is fundamentally bounded by the evaluator model's intelligence and struggles with complex deterministic reasoning, such as verifying code modernization.
To build rigorous evaluation pipelines, teams incorporate automated metrics. While conventional AI practitioners rely on classical statistical metrics like RMSE or mean absolute error, LLM-based agent evaluation uses automated metrics such as hallucination, groundedness, and coherence scores (often computed via an evaluator LLM). For agent systems specifically, SDKs combine these response scores with automated trajectory metrics to measure tool-calling accuracy alongside text quality.
[16:11](https://www.youtube.com/watch?v=fkCTifAqVGg&t=971s) Trajectory Metrics: Exact Match, In-Order Match, and Any-Order Match
When configuring trajectory evaluation within an agent SDK, architects must define how strictly an agent's execution path must match the golden benchmark dataset. Naz breaks these down into three primary matching strategies:
- **Exact Match:** The agent must replicate the benchmark path identically. Every tool called, every input argument provided, and the absolute sequence of executions must match the golden test set flawlessly. The final prose output is secondary to whether the operational mechanics were executed with absolute precision. This is essential for operations like the ticket agent, where applying the wrong promotional code or charging the wrong fee has real financial repercussions. - **In-Order Match:** The agent must hit the required milestones in the correct sequence, but deviations or side detours are permitted. If an agent spontaneously triggers a Google Search tool that was not in the golden test set, latency may increase slightly, but the run still passes as long as the essential tools execute in the required order. Naz notes: “In order match so you can detour, but as long as you get to the right destination, meaning you call the right tools, that's OK.” - **Any-Order Match:** The agent must call all required tools, but the execution order is irrelevant. This is common in exploratory or recommendation tasks; for example, whether the London concierge gathers recommendations for Soho or Westminster first does not matter, as long as both relevant data sets are retrieved for the user.
While automated trajectory metrics offer objectivity and massive scale, they have limitations: they struggle to quantify creativity, and systems can appear to "cheat" or pass artificially if an engineer selects an overly lenient metric for a strict operational workflow.
[18:32](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1112s) Simulation and Granular Metric Logging Demo
In a live walkthrough of their sample configuration, Naz demonstrates how metrics and test cases are structured using the agent SDK. The setup loads a configuration containing both response metrics (hallucination, safety) and trajectory metrics (tool invocation fidelity). Test datasets are created directly within the SDK user interface by engaging in multi-turn dialogues with the agent and saving high-quality runs as a golden test set in JSON format.
During automated simulation runs: - The system plays back the input prompts against the agent. - The agent’s outputs and tool invocations are executed and compared turn-by-turn against the golden baseline. - Individual scores are recorded for every conversation turn (prompt, expected response, actual output, and metric results). - The granular execution trace and aggregated conversation-level metrics are streamed and persisted into BigQuery, providing an audit trail to decide if an agent is ready for production.
[20:05](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1205s) Accelerating Developer Velocity: Gemini Cloud Assist and Application Design Center
Naz yields the floor to Manasa Kandula, prompting lighthearted applause from the audience. Manasa quips: “We're going to be switching back and forth often, so don't need to clap every time, but we appreciate it.” Manasa focuses on developer velocity, noting that while modern code-assist tools make hacking together a local prototype fast and easy, the transition to production often stalls when moving code into a secure, hardened enterprise cloud architecture. The shift to AI agents introduces unfamiliar runtimes, evolving security perimeters, and new architectural paradigms that significantly increase cognitive load on developers.
To bridge this gap, teams can use **Gemini Cloud Assist** alongside the **Application Design Center (ADC)**: - **Gemini Cloud Assist:** Functions as an AI cloud architect. It parses codebases, interprets developer intent, and maps required agent primitives to idiomatic Google Cloud infrastructure. In the London concierge app, which depends on a PostgreSQL database with the `pgvector` extension, Gemini Cloud Assist automatically recommends provisioning a Cloud SQL for PostgreSQL instance configured as a private instance to satisfy enterprise isolation requirements. - **Application Design Center (ADC):** Acts as a centralized repository to store, manage, and deploy approved architectural templates. It ensures governance and delivers out-of-the-box observability metrics from day one. - **Developer Workflow via MCP:** Developers can interact with their codebase through Gemini CLI in their IDE, backed by Gemini Cloud Assist and ADC MCP servers. These MCP tools analyze local code, generate appropriate deployment architectures, deploy resources straight to Google Cloud, and troubleshoot deployment discrepancies inline.
The demonstration shows an IDE housing the concierge agent codebase, verifying first that the MCP configuration has correctly mounted the Application Design Center and Gemini Cloud Assist MCP servers to manage the deployment lifecycle.
[23:15](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1395s) From Codebase to Cloud: Automated Architecture and Deployment with ADC
Manasa demonstrates how an agent can take a raw local application and deploy it onto Google Cloud using automated infrastructure tooling. Starting from a single natural-language instruction—“design a Cloud architecture from our code base”—the agent scans the entire local repository, analyzes its internal components, and deduces how services map and connect to one another. Once it builds that baseline mental model, it invokes the Gemini Cloud Assist MCP tool to translate those application requirements into a complete architecture. It generates Terraform templates, writes them directly back into the local codebase, and registers the deployment specification into the Application Design Center (ADC).
Within ADC, developers are presented with a visual design canvas where they can inspect the architecture, review configurations, and trace component dependencies before anything goes live. Once approved, the agent initiates the deployment pipeline via Cloud Build. In the demonstration, a deployment process that typically takes 10 to 15 minutes is fast-forwarded to reveal automated status checks, deployment logs, and the provisioning of a public Load Balancer IP address. Manasa pulls this IP address directly from the output and loads the live travel concierge application in the browser, noting that bridging the gap between a local prototype and an enterprise-grade cloud deployment usually burns days of engineering time spent matching cloud primitives and applying security baselines. As Manasa explains: “And we managed to do this in a matter of about half an hour using these two tools.”
```mermaid flowchart TD accTitle: Codebase to Cloud Deployment Flow accDescr: Automated transition from local code to cloud architecture using MCP and ADC subgraph Design ["1. Architecture & Design"] A["Local Codebase"] --> B["Agent Code Analysis"] B --> C["Gemini Cloud Assist MCP"] C --> D["Terraform Templates"] D --> E["ADC Visual Canvas"] end
subgraph Deploy ["2. Approval & Deployment"] E --> F["Developer Approval"] F --> G["Cloud Build Pipeline"] G --> H["Google Cloud Infrastructure"] H --> I["Load Balancer IP"] end ```
- The agent required only a single natural-language instruction to parse the entire codebase without manual configuration mapping. - Infrastructure-as-code artifacts were produced as native Terraform templates written directly to the project directory. - ADC provided a visual representation of all cloud resources, relations, and configurations prior to execution. - Automated Cloud Build routines verified operational health and deployment logs, concluding with a functional external Load Balancer IP.
---
[25:52](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1552s) Scaling the Ecosystem: Agent Registry and Agent Gateway
As organizations build multiple autonomous agents and Model Context Protocol (MCP) servers, they immediately hit scaling bottlenecks around visibility, duplicated engineering effort, and perimeter control. Manasa introduces two core architectural answers: the Agent Registry and the Agent Gateway. The Agent Registry serves as a centralized internal catalog and unified source of truth for all deployed agents and MCP tools across an organization. Instead of separate development groups reinventing duplicate capabilities—such as multiple variations of ticket-booking integrations—teams can discover existing services, view their specifications, and consume them directly.
However, discovery without governance leads to architectural chaos and severe security risks. With dozens of agents operating across an enterprise, organizations cannot allow an unconstrained web of agent-to-agent communication. For instance, while a customer-facing travel concierge must communicate with an external ticketing agent to purchase flights, it must be explicitly barred from ever reaching an internal human resources agent. Rather than burdening individual application developers with hand-coding custom authorization and routing rules between models, the Agent Gateway shifts policy enforcement to the infrastructure level. Manasa emphasizes the platform-first role of this mechanism: “You can think of it as the governance layer for your agents and MCP tools.”
- **Agent Registry:** Functions as an enterprise catalog and single source of truth to promote asset discovery and prevent redundant engineering. - **Agent Gateway:** Acts as the centralized governance proxy, enforcing boundary policies regarding which agents and MCP tools are permitted to call one another. - **Platform-Level Policy:** Eliminates developer burden by applying communication guardrails at the infrastructure layer rather than relying on custom application code.
---
[27:58](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1678s) Runtime Security: Ephemeral Agent Identity and Audit Trails
Naz Bayrak steps up to break down runtime production safety into three critical pillars: identity, threat defense, and observability/auditability. The first line of defense is Agent Identity, a native capability designed to give autonomous models their own enterprise credentials. Traditionally, developers assign agents broad, long-lived service account roles whose entire lifecycle, rotation, and privilege scope must be manually administered. If an agent executes an action under a generic service account, incident responders cannot distinguish which specific agent or model version initiated the downstream API call.
Under the new Agent Identity architecture, an agent is dynamically assigned a unique, least-privilege IAM identity at deployment and runtime that disappears when the workload terminates. When the concierge agent accesses a ticketing backend, it signs and authenticates using its own distinct, scoped identity. If anomalous behavior occurs, security operations teams are not left staring at ambiguous service-level calls; they receive a granular log specifying the exact agent instance, version number, and execution context. As Naz states: “And you'll be able to have an irrefutable audit trail to trace back exactly what happened.”
- Traditional architectures require teams to maintain permanent service roles and manual credential lifecycles for agents. - Agent Identity binds ephemeral, runtime IAM credentials to specific agent instances without developer management overhead. - Identifiers and permissions persist only while deployment workloads are actively running and vanish upon shutdown. - Operational logs provide immutable attribution containing the exact agent name, build version, and identity context for every action.
---
[30:06](https://www.youtube.com/watch?v=fkCTifAqVGg&t=1806s) Defending the Frontier: Live Threat Mitigation with Model Armor
Securing live generative AI applications involves protecting sensitive data and company reputation against bad actors without relying on fragile, endless system prompt tweaks. Naz outlines the primary threats confronting customer-facing agents: leaks of personally identifiable information (PII), unauthorized commentary on business competitors, toxic or biased outputs, and deviations from brand voice. Rather than forcing engineers to spend weeks crafting and evaluating defensive prompts that can still be bypassed, Google Cloud introduces Model Armor—an inline security proxy stationed directly between incoming user prompts and the underlying foundation model. Naz summarizes its design objective: “Model Armor resides between the user and your LLM to prevent all of these from happening.”
```mermaid flowchart LR accTitle: Model Armor Security Proxy accDescr: Incoming user requests are filtered by Model Armor before reaching the LLM A["End User / Attacker"] --> B["Model Armor Inspection Layer"] subgraph Inspection ["Threat Filtering Engine"] B -.-> C["PII / Exfiltration Detection"] B -.-> D["Prompt Injection & Jailbreak"] B -.-> E["System Prompt Leakage"] B -.-> F["Malicious URL Interception"] end B -->|"Sanitized Traffic"| G["Foundation LLM / Agent"] B -->|"Blocked Attack"| H["Safe Fallback Response"] ```
To demonstrate its capabilities, Naz runs a live attack suite against the travel application UI. An attacker first attempts data exfiltration by coaxing the model into dumping customer credit card records; Model Armor detects the intent and suppresses the response. Successive attacks escalate through direct prompt injections, creative jailbreaks, and system instruction extraction (“prompt leak”) attacks where the user tries to compel the agent to reveal its internal guidelines. The interface shows the attacker becoming increasingly frustrated as the filter neutralizes safety violations and flags malicious URLs. To validate that the system does not simply drop all user inputs, Naz executes a standard travel query, which passes through cleanly, followed by a final test against sensitive data detection that is immediately blocked.
- Intercepts threats natively in the network path, replacing fragile, long-winded system prompt instructions. - Shields systems against automated data exfiltration attempts targeting PII and financial records. - Mitigates direct and indirect prompt injections, jailbreaks, and attempts to extract system instructions. - Inspects and filters malicious URLs and policy-violating language while allowing legitimate business interactions to flow freely.
---
[33:38](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2018s) Observability: Moving Production Telemetry from Day Two to Day Zero
Manasa returns to address the reality of operating generative AI workloads in production environments, opening with a direct question to the audience: “How many of you have had production applications that have had no issues.” When not a single person raises their hand, she highlights the universal reality of production failures and the critical need for deep observability. The fundamental challenge of modern software operations is diagnosing unexpected behaviors quickly, yet observability is routinely deferred. As Manasa observes: “And one of the things that frustrates me is that observability is often referred to as a day two operation.”
Observability is traditionally treated as an afterthought because implementing it is tedious and expensive. Application developers must manually instrument source code with tracing spans, while platform engineers must extract signals from cloud infrastructure and stitch together complex custom dashboards that correlate application context with system telemetry. Because of this high initial effort, teams rarely implement proper observability at launch ("day zero"), leaving production agents flying blind. Manasa explains that Google Cloud resolved this dilemma by integrating deep telemetry natively into the platform SDKs, capturing essential execution signals automatically from the moment of deployment.
- The audience poll demonstrated that virtually every production system experiences operational faults. - Traditional observability efforts require heavy manual code instrumentation and complex custom cloud dashboard engineering. - High upfront implementation friction causes development teams to defer monitoring strategies until post-launch ("day two"). - Google Cloud embeds baseline telemetry into the agent platform SDK to provide native visibility on "day zero" without manual instrumentation overhead.
[34:54](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2094s) Unified Observability with Application Design Center (ADC)
Application observability is enabled out of the box with the click of a single button. By utilizing Application Design Center (ADC) to deploy the entire application, developers instantly gain platform-level dashboards. What sets ADC apart from standard cloud monitoring is that instead of treating infrastructure as disconnected silos—leaving teams to piece together separate signals from a Cloud SQL Postgres database, individual Cloud Run instances, and other isolated services—it binds every resource together under a single, unified application context. This allows engineering teams to monitor and operate the application as an integrated whole rather than juggling fragmented metrics across a Google Cloud project.
Alongside the core infrastructure metrics, the deployment provides a dedicated Agent App dashboard. This dashboard surfaces specialized GenAI operational telemetry critical for running production AI workloads, including token count consumption and latency metrics segmented by specific models.
- **Unified Context:** ADC eliminates operational silos by correlating signals from Cloud Run, Cloud SQL Postgres, and other resources into a single application-level topology. - **Out-of-the-Box Metrics:** Automated dashboards track real-time GenAI performance indicators, such as request latency per model and cumulative token usage.
---
[35:57](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2157s) Diagnosing GenAI Agents with OpenTelemetry Standards
Once monitoring dashboards detect an anomaly or performance regression, engineers require deep diagnostic instrumentation to isolate the root cause. Google Cloud actively champions OpenTelemetry (OTel), the open, vendor-neutral standard used across the industry to construct resilient telemetry pipelines without vendor lock-in. To address modern AI application stacks, Google Cloud collaborated with the open-source software (OSS) community to extend the core OpenTelemetry protocol, adding native telemetry signals specifically tailored for GenAI architectures.
Before these extensions, telemetry conventions only accommodated traditional application interactions. Because GenAI systems introduce non-deterministic model behaviors, execution trajectories, and external tool calls, extended OpenTelemetry semantics now capture granular model metadata alongside conventional execution traces.
As the speaker attempts to switch over to a live demonstration to display one of these GenAI execution traces, the screen switch encounters a brief technical hitch. The speaker reassures the audience with a smile: “It's worth it, I promise. It just takes a couple of seconds to come up.” After a brief pause waiting for the AV feed, the speaker decides to defer the live trace view until the end of the presentation so the AV team can assist, ensuring only a single screen switch is needed.
---
[38:00](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2280s) End-to-End Traces and Business Analytics with BigQuery
While technical metrics confirm that an application is running cleanly without throwing errors, teams still face an existential operational question: “The thing is, the question here is, how do you know if your agents are actually delivering value to your end customer?” To evaluate real-world impact, the agent uses the BigQuery analytics plugin provided by the Agent Development Kit (ADK). This plugin captures every conversation turn, extracts all conversation metadata and message payloads, and streams them into a structured BigQuery table for high-powered SQL analytics.
```mermaid flowchart LR accTitle: GenAI Trace and Analytics Pipeline accDescr: End-to-end trace execution and conversational data export to BigQuery.
subgraph AgentExecution ["Agent Execution Turn"] U["End User"] --> A["Gemini 2.5 Flash Agent"] A -->|"tool call"| T["London Activities Search Tool"] T -->|"tool result"| A A -->|"final response"| U end
subgraph ObservabilityLayer ["OTel & ADK Analytics"] A -.->|"OTel GenAI spans"| TR["Trace Explorer"] A -.->|"ADK BigQuery Plugin"| BQ[("BigQuery Analytics")] BQ --> S["SQL Sentiment & Token Analysis"] end ```
Switching successfully to the live console, the speaker inspects an end-to-end trace of a single conversational turn. The waterfall trace details every internal hop and latency breakdown: the initial user request, the call to the large language model, the agent invoking an external tool to search for London activities data, the tool returning results to the model, and the model rendering the final output. The trace spans are explicitly tagged with GenAI metadata, highlighting that the agent used Gemini 2.5 Flash with a temperature setting of 0.01 and consumed 3,658 tokens for that single turn—prompting the speaker to joke, “We need to optimize our agent. But anyway, so you can see all of this in one space.”
Turning to the BigQuery console, the speaker reviews interaction records for a single day, capturing timestamps, conversation events, attributes, and user prompts. Running a SQL sentiment analysis query sorts the conversations by customer satisfaction score on a 1-to-5 scale: - **Five-star interaction:** A satisfied user reviewing an itinerary states, *“Yes, day 3 is absolutely perfect. Those are exactly the kind of things I want to do,”* receiving a top rating of 5/5. - **One-star interaction:** Exporting lower-scoring conversations into a Google Sheet reveals a frustrated user complaining, *“This is really not working out, so you can't find anything,”* resulting in a bottom rating of 1/5. - **Token consumption analytics:** A custom dashboard graphs average daily token consumption by interaction type, enabling engineers to identify heavy prompt chains and optimize cost profiles.
---
[42:02](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2522s) Scaling Enterprise Governance with ADC Golden Templates
Deploying, hardening, and instrumenting a single agent requires thoughtful configuration, but scaling that manual process across an enterprise with 10 or 100 agents quickly becomes impossible. As the speaker emphasizes, “Are you going to be able to scale the steps that we talked about one by one for all your agency and organization, scaling manual oversight is really difficult.”
Application Design Center solves this scaling challenge through reusable architectural governance. While ADC can scan an individual developer's repository to generate a bespoke deployment template, an enterprise platform engineering team can instead establish an organization-wide “golden template” for all agentic applications. This golden template bakes in mandatory enterprise policies, MCP tool connection standards, IAM permissions, security guardrails, evaluation gates, and telemetry pipelines by default. Individual developers simply take the pre-approved platform template, configure their specific agent logic and tools, and deploy into production with guaranteed compliance.
---
[43:26](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2606s) Session Recap: The Four Pillars of Production Agents
With two minutes remaining, the presentation transitions back to Naz Bayrak to summarize the core pillars required to move GenAI agents from prototypes to hardened production systems:
1. **Evaluation:** Assessing agents through both response evaluation and trajectory evaluation, using automated metrics to validate multi-step tool execution paths. 2. **Efficient Cloud Deployment & Registries:** Packaging agents efficiently for Google Cloud and establishing centralized registries for discovering and sharing agent tools and capabilities. 3. **Identity, Access, and Threat Modeling:** Enforcing least-privilege IAM specifically mapped to agents at deployment time, and actively mitigating common agent security vulnerabilities. 4. **Observability, Auditability, and Platform Governance:** Implementing comprehensive OTel tracing and BigQuery conversational analytics, while using standardized ADC golden templates to scale compliance and governance across teams without manual overhead.
Naz wraps up by inviting attendees to continue the discussion outside: “Given that I see that we are almost out of time, we will be in the corridor in case any of you would like to ask any questions so that we get to understand your use cases and see how we can help you better.”
## Notable quotes > [08:45](https://www.youtube.com/watch?v=fkCTifAqVGg&t=525s) The real takeaway has been that we are trying to match the serving strategy with the production need, rather than just assume that a default is going to do the trick. — Maria-Irène Călinoiu
> [33:38](https://www.youtube.com/watch?v=fkCTifAqVGg&t=2018s) And one of the things that frustrates me is that observability is often referred to as a day two operation. — Manasa Kandula
> [13:35](https://www.youtube.com/watch?v=fkCTifAqVGg&t=815s) Agents are non-deterministic and this brings some challenges to our evaluation systems. — Naz Bayrak
## Mentioned Alibek Datbayev, Application Design Center, BigBot, BigQuery, Booking.com, Cloud Build, Cloud Run, Cloud SQL, Gemini, Gemini 2.5 Flash, Gemini Cloud Assist, Google Cloud, Google Maps, Google Search, Know Before You Go, Manasa Kandula, Maria-Irène Călinoiu, Model Armor, Model Context Protocol, Naz Bayrak, OpenTelemetry, PostgreSQL, Reels to Reality, Terraform, Vertex AI, Vertex AI Agent Engine
## Video description Simple agents are easy. Reliable agents are hard. Join us for a high-velocity master class on the production-first approach to Gemini Enterprise Agent Platform. We’re breaking down the complex transition from a basic prompt to a hardened enterprise tool. First, connect your agent by using Model Context Protocol (MCP) servers for secure data context. Next, protect it by implementing enterprise-grade security guardrails. Then, inspect it by setting up observability to monitor agent behavior in real time. And finally, perfect it by running automated evaluations to ensure accuracy at scale. In this session, expect no fluff – just the architecture you need to ship with confidence.
Watch more Google Cloud Next 2026 Sessions → https://goo.gle/gcloud-next-2026 Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
Speakers: Alibek Datbayev, Manasa Kandula, Naz Bayrak, Maria-Irène Călinoiu
BRK2-014 #GoogleCloudNext
[Source](https://www.youtube.com/watch?v=fkCTifAqVGg)