Databricks Unveils Omnigent, Buys Panther to Curb Enterprise AI Costs

Data + AI Summit Keynote 2026 | Day 2 — Databricks

Databricks

Chief Executive Officer Ali Ghodsi warns runaway token costs will become prohibitive within twelve months unless enterprises abandon frontier model worship to ground autonomous agents in proprietary lakehouse context.

AI-assisted summary of Data + AI Summit Keynote 2026 | Day 2 — original source. Check the original for full context.

Source published:

Key ideas

  • Databricks introduced Omnigent, an open-source meta-harness abstraction layer that standardizes cross-harness coordination, session handoffs, contextual policy enforcement, and financial guardrails across proprietary frameworks like Claude Code and OpenAI Codex. (18:56)
  • The company launched Agent Bricks alongside Unity AI Gateway to centralize enterprise governance over models, agents, and MCP tools, using an 'advisor pattern' and dynamic data tainting to optimize inference costs and prevent privileged data leaks. (40:00)
  • Databricks announced Genie ZeroOps for Machine Learning to act as an autonomous primary responder for production model operations, reversing traditional on-call workflows to automatically diagnose, retrain, and queue drifted models for deployment. (102:43)
  • Addressing domain-specific agentic workflows, Databricks unveiled CustomerLake for 1-to-1 Infinity Campaigns and Lakewatch, an open agentic security lakehouse accelerated by the acquisition of Panther Labs. (120:00)

Detailed notes

## Background Published on YouTube by Databricks on June 18, 2026, this keynote captures Day 2 of the Data + AI Summit before an audience of over 30,000 data professionals. According to the description, the session covers new open-source innovations, major updates across the Databricks Platform, and autonomous agentic solutions tailored for enterprise marketing and cybersecurity.

## TL;DR Databricks leadership argues that raw model intelligence is no longer the bottleneck in enterprise AI, framing proprietary context, cost control, and avoiding vendor lock-in as the true production hurdles. To bridge fragmented agent frameworks, Databricks unveils Omnigent, an open-source meta-harness that standardizes multi-agent orchestration, contextual security policies, and live collaboration. The presentation introduces Agent Bricks with Unity AI Gateway for intelligent model routing, Databricks Apps to govern enterprise 'vibe coding,' and Genie Code and Genie ZeroOps for autonomous machine learning lifecycle management. The keynote concludes with major domain-specific platforms: CustomerLake, an agentic Customer Data Platform for hyper-personalized marketing, and Lakewatch, an open agentic security lakehouse built in tandem with the acquisition of Panther Labs.

## Atmosphere & style The presentation delivers the polished, high-energy cadence typical of a flagship tech keynote, alternating between visionary executive manifestos and dense, live terminal and UI demonstrations. Speakers maintain an urgent, pragmatic tone, steering clear of passive model hype to focus on the unglamorous mechanics of production deployment—such as token budgets, sandboxing, data tainting, and on-call fatigue. The narrative carries a strong collaborative rhythm as platform leads and engineering founders hand off seamless demonstrations, cultivating a sense of technical momentum and enterprise inevitability around agentic systems.

## Bulletin - [18:56](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1136s) Databricks introduced Omnigent, an open-source meta-harness abstraction layer that standardizes cross-harness coordination, session handoffs, contextual policy enforcement, and financial guardrails across proprietary frameworks like Claude Code and OpenAI Codex. - [40:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2400s) The company launched Agent Bricks alongside Unity AI Gateway to centralize enterprise governance over models, agents, and MCP tools, using an 'advisor pattern' and dynamic data tainting to optimize inference costs and prevent privileged data leaks. - [01:42:43](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6163s) Databricks announced Genie ZeroOps for Machine Learning to act as an autonomous primary responder for production model operations, reversing traditional on-call workflows to automatically diagnose, retrain, and queue drifted models for deployment. - [02:00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7200s) Addressing domain-specific agentic workflows, Databricks unveiled CustomerLake for 1-to-1 Infinity Campaigns and Lakewatch, an open agentic security lakehouse accelerated by the acquisition of Panther Labs.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=0s) Enterprise AI bottlenecks have shifted from raw model intelligence to the challenge of infusing proprietary enterprise context into agent reasoning loops. - [00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=0s) Inference and token consumption are projected to become cost-prohibitive for enterprises within 6 to 12 months without architectural interventions like semantic ontologies and advisor routing patterns. - [11:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=671s) The proliferation of custom, fragmented agent software has created an 'agent harness problem,' where incompatible execution environments cannot share state, collaborate, or enforce unified enterprise controls. - [20:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1200s) Omnigent provides an open-source meta-harness architecture featuring an execution runner and a centralized server to unify CLI, desktop, and web agent interactions across diverse models and tools. - [21:20](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1280s) Omnigent replaces static tool-call permissions with dynamic contextual policies that restrict sensitive external actions based on an agent's prior data access history. - [36:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2171s) Agent Bricks abstracts the 99% of infrastructure work required beyond the core agent loop by providing multi-model choice, Genie Ontology grounding, Agent Memory Services, and ephemeral VM Sandboxes. - [46:51](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2811s) Unity AI Gateway shifts organizations from 'tokenmaxxing' to 'valuemaxxing' through intelligent model routing, centralized MCP skill registries, and runtime data tainting to restrict data egress. - [01:09:16](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4156s) Databricks Apps solves the prototype-to-production gap of enterprise vibe coding by bundling managed authentication, AppSpaces administrative boundaries, and serverless micro-VM runtimes. - [01:37:07](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5827s) AI Runtime delivers serverless, on-demand GPU capacity without upfront commitments, enabling distributed deep learning, fine-tuning, and LLM pre-training natively in the lakehouse. - [01:42:43](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6163s) Genie Code for Machine Learning integrates with team-specific MLflow experiments, feature stores, and validation suites to act as an automated senior ML engineer. - [01:42:43](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6163s) Genie ZeroOps for ML addresses the reality that teams spend 60% to 80% of bandwidth maintaining models, placing an autonomous agent on primary on-call triage while human engineers serve as approvers. - [01:52:48](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6768s) Buyer behavior is transitioning to delegate shopping agents acting over APIs in seconds, rendering traditional waterfall marketing campaigns obsolete. - [02:00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7200s) CustomerLake establishes an agentic CDP natively inside the lakehouse, leveraging Profile Agents for LLM-driven identity resolution and Campaign Agents to execute continuous 1-to-1 Infinity Campaigns. - [02:20:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8400s) Legacy SIEM platforms cannot withstand automated AI cyberattacks due to 30-day retention cutoffs, high licensing fees, and an inability to ingest rich contextual telemetry like code commits and Slack chats. - [02:22:29](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8549s) Lakewatch and the acquisition of Panther Labs merge detection engineering with the open lakehouse, deploying defensive agents that automate OCSF log normalization, detection authoring, and alert triage. - [02:43:40](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9820s) Databricks' overarching strategy anchors the agentic era in open standards across data formats, models, and harnesses, unified under Unity Catalog governance without cloud lock-in.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=0s) Infusing Enterprise Context into AI

Databricks co-founder and CEO Ali Ghodsi takes the stage to kick off Day 2 of the Data + AI Summit, joking with the crowd about the early start: “All right, super excited. Wow, everybody made it 8:00 AM. It was hard for me, actually.” Recapping the core message from Day 1, Ghodsi argues that the industry's obsession with chasing raw model capabilities misses the real enterprise bottleneck. As Ali Ghodsi states: “We don't need more intelligence for the AI, we need more context.” The real objective for Databricks and its customers is infusing the proprietary enterprise context trapped across disparate organizational systems directly into models to unlock autonomous, production-ready AI agents.

To ground this approach, Ghodsi outlines four non-negotiable pillars required to solve the enterprise context equation: 1. **Choice and Openness:** Organizations must retain the freedom to run any model, leverage any cloud or infrastructure, and store data in open file formats. Ghodsi warns that architectural lock-in has been the primary factor handicapping enterprise velocity over the last two to three decades as teams layered proprietary software into bloated stacks. As Ali Ghodsi puts it: “Avoiding lock-in, I think, is one of the most important things that Databricks can help with.” 2. **Control and Governance:** Enterprises require granular security policies, strict access controls, and transparent auditability spanning agents, data, and underlying compute infrastructure. 3. **Cost Controls:** Unchecked model consumption poses an existential risk to widespread deployment. While companies are already feeling the pinch of inference and token usage, Ghodsi cautions that the industry is only scratching the surface: “I think in the next 6 to 12 months, most organizations are going to see that the costs are going up so much that it's actually prohibitive to their companies.” 4. **Context via Genie Ontology:** By continuously computing in the background across an organization’s entire data footprint—reaching beyond the Lakehouse—the Genie ontology builds an ambient institutional index. Ghodsi likens this background synthesis to “a PageRank that Google did, but for the enterprise,” which drastically accelerates agent execution, slashes unnecessary token burn, and elevates response quality.

[03:21](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=201s) Keynote Roadmap: Agent Bricks, Genie Code, Apps, Lakewatch, and CustomerLake

Ghodsi pivots to previewing the major architectural launches and technical deep dives scheduled across the keynote, starting with Agent Bricks presented by Casey and Ankit. Designed for developers building custom agents, Agent Bricks provides isolated virtual machine sandboxes for secure code execution, persistent memory services, and native integration with Omnigent, the open-source meta-harness. A standout architectural pattern demonstrated during this segment pairs Agent Bricks and Omnigent with the Unity AI Gateway to tame token costs using an advisor pattern. Under this setup, a fast, cost-effective open-source model handles primary task execution and invokes a frontier, high-cost model strictly as a fallback tool when it encounters an obstacle.

```mermaid flowchart TD accTitle: Advisor Pattern for Cost-Optimized Agent Execution accDescr: Small open-source model executes tasks directly and calls an expensive frontier model as a tool advisor only when stuck.

subgraph Execution["Agent Runtime"] Task["Task or Code Request"] --> SmallLLM["Cost-Effective Open Model (Primary Agent)"] SmallLLM --> Check{"Agent Stuck?"} end

subgraph Escalation["Advisory Tool"] Check -- "Yes (Escalate)" --> Gateway["Unity AI Gateway"] Gateway --> SmartLLM["Frontier Large Model (Advisor Tool)"] SmartLLM --> Guidance["Structured Hint / Resolution"] Guidance --> SmallLLM end

Check -- "No (Proceed)" --> Output["Task Completed"] ```

The keynote agenda also expands into specialized agentic workloads across data science, application delivery, and domain platforms: - **Genie Code for Machine Learning and ZeroOps:** Building upon its data engineering automation capabilities, Genie Code is expanded into an autonomous data science and machine learning agent. It writes end-to-end ML pipelines, automatically troubleshoots runtime failures, and establishes a self-healing loop that democratizes advanced model development. - **Databricks Apps and the Agent System of Record:** As organizations build custom vibe-coded apps for hundreds of thousands of employees, Databricks Apps introduces governed authentication, data authorization, and enterprise runtime guarantees to bridge the prototype-to-production gap. - **Lakewatch and Panther Integration:** A walkthrough of Lakewatch—the agentic Security Information and Event Management (SIEM) system launched two months prior—covering its expansion and integration with Panther following Databricks’ agreement to acquire the security operations platform. - **CustomerLake:** An educational dive into the Customer Data Platform (CDP) reimagined as an agentic system of record, designed to autonomously orchestrate customer touchpoints at massive scale. - **Omnigent Open Source:** Chief Technologist Matei Zaharia will introduce Omnigent, an open-source meta-harness open-sourced over the weekend that serves as the architectural foundation for Databricks’ own products, including Genie. Teasing Zaharia’s entrance, Ghodsi jokes with the event production team: “And I want a request here for the camera crew: please zoom in on his shoes, okay? And make sure you capture that, because that's going to be a really important part of his talk.”

[06:33](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=393s) Mukesh Ambani on Democratizing AI for 1.5 Billion Indians

Before welcoming Matei Zaharia to unveil Omnigent, Ghodsi introduces a video address from Mukesh Ambani, Chairman and Managing Director of Reliance Industries, underscoring their joint mission to deliver digital sovereignty and affordable AI across the developing world. Ambani explains that high inference costs often price emerging economies out of advanced artificial intelligence, necessitating an architecture centered on high-efficiency open models and scalable data infrastructure.

Addressing the summit delegates, Ambani frames the Reliance-Databricks collaboration as a national modernization initiative: “Today, with Reliance Intelligence, our mission is to democratize intelligence for 1.5 billion Indians. Our motto is simple: AI everywhere for everyone.” Ambani outlines a vision where accessible AI elevates farmers, local neighborhood shopkeepers, healthcare outcomes, and public education across India. Furthermore, he emphasizes that technical architectures engineered to operate under India's demanding cost, scale, and language constraints provide an exportable blueprint for the entire Global South, where four-fifths of humanity lives.

Ambani highlights the technical magnitude of the migration, noting that Reliance has consolidated petabytes of operational data across Jio telecommunications, retail networks, green energy, materials, and media onto a unified Databricks intelligence platform to serve over 800 million customer relationships. Ambani describes this transformation as a fundamental operational shift: “For many years, enterprises used data mainly to look back. The next generation will use data to make better decisions, act faster, and continuously improve. This is the shift from dashboards to decisions.”

- **Reliance Intelligence Scale:** Migrated multi-petabyte datasets across diverse verticals (Jio, retail, energy, materials, and media). - **Consumer Footprint:** Powers real-time decision-making across more than 800 million active customer relationships. - **Global Reach:** Targets population-scale accessibility for 1.5 billion Indian citizens while generating open architectures applicable to the Global South.

[10:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=600s) Reliance and Databricks: Powering Scale in India

Speaking about the deployment of Databricks across Reliance, the presenter highlights that decision-makers are already leveraging Databricks Genie to query enterprise data in natural language and receive real-time, actionable intelligence. This capability is now scaling to thousands of business leaders across the entire organization. India is described as one of the most demanding and rigorous testing environments for artificial intelligence globally, driven by massive scale, diverse languages, demanding affordability constraints, and rapid adoption speed.

As the Reliance speaker reflects on the broader potential of this enterprise deployment: “If an AI solution works for India, it can serve the world.” He expresses gratitude to Databricks CEO and Co-founder Ali Ghodsi for his partnership, friendship, and shared vision, noting that the two companies are establishing a global blueprint for harnessing data and AI responsibly, openly, and at enterprise scale.

[11:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=671s) The Explosion of Coding Agents and the "Harness" Problem

Databricks Co-founder and Chief Technologist Matei Zaharia takes the stage to open the second day of keynotes. He announces a brand-new open-source project released by the Databricks engineering team over the preceding weekend. The initiative addresses recurring architectural roadblocks encountered across the industry as developers build, run, and scale AI agents. While coding agents have achieved widespread developer adoption and agents like Genie prove that specialized agentic workflows succeed, their very ease of creation has sparked a new problem: when custom software is this easy to create, developers naturally attempt to "vibe code" their own solutions rather than relying on standard infrastructure.

Zaharia explains that creating custom software around models leads directly to what he terms the "agent harness" problem. A language model in isolation simply predicts and generates text tokens. The software wrapped around that model—providing the interface to file systems, execution environments, toolsets, user interfaces, and permission models—is the harness. Products such as Claude Code, Codex, and countless custom agent SDKs each introduce their own proprietary harnesses. Because developers often co-design the harness alongside the underlying language model, the ecosystem has exploded with dozens of incompatible harnesses. Zaharia compares shopping for agent frameworks to browsing a pet store: you can pick an agent harness in any color, size, or shape, but none of them talk to one another.

```mermaid flowchart TD accTitle: The AI Agent Harness Architecture accDescr: Diagram contrasting raw language models, proprietary agent harnesses, and external system interfaces.

subgraph Models["Underlying Intelligence"] LLM["Foundation Language Model<br/>(Predicts / Generates Tokens)"] end

subgraph Harness["Agent Harness Layer"] H["Harness Software<br/>(Claude Code, Codex, Custom SDKs)"] UI["User Interface"] SEC["Security & Permissions"] TOOL["Tool & File Connectors"] end

subgraph World["External Environment"] FS["File Systems & Codebases"] DB["Databases & APIs"] USR["Human Developers"] end

LLM <--> H H --- UI H --- SEC H --- TOOL UI <--> USR TOOL <--> FS TOOL <--> DB ```

[13:48](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=828s) Three Challenges of Interoperability: Composition, Collaboration, and Control

Because fast coding cannot resolve systemic architectural divergence, Zaharia outlines three core interoperability challenges that siloed harnesses cannot fix on their own:

1. **Agent Composition:** Engineers rarely stick to a single agent. When troubleshooting complex bugs or distributing tasks, developers frequently query multiple coding agents in parallel, feeding them different prompts and comparing outputs. While running loops or multi-agent workflows inside a single harness is straightforward, coordinating across different harnesses (for instance, bridging Claude Code and Codex) is brittle and fragmented. Zaharia notes emerging architectural patterns such as multi-agent loops and "advisor" frameworks, where lightweight, cost-effective models handle routine volume while calling larger frontier models when blocked. Research from Harvey AI, Anthropic, Databricks Genie, and Zaharia’s own lab at UC Berkeley confirms that mixing multi-size models improves both operating costs and answer quality. 2. **Human-Agent and Agent-Agent Collaboration:** A typical developer desktop has devolved into a chaotic mosaic of terminal windows, Claude Code sessions, Slack channels, and Google Docs, leaving users manually copy-pasting code snippets across isolated harness interfaces. Zaharia questions why developers cannot seamlessly transfer an active coding session—complete with full state and context—directly to another team member when stepping away from their desk. 3. **Enterprise Control and Security:** Agents are vulnerable to prompt injection, cost overruns, and catastrophic operational errors such as accidental database deletions. While transactional lakes like Lakebase can revert accidental table drops, security controls remain fundamentally weak. Zaharia points out the absurdity of current human-in-the-loop authorization dialogs, where models interrupt developers with unreadable prompts asking: “Do you want to run this 100-line Python script? Yes or no.” Expecting human users to scrutinize every underlying agent script provides neither real security nor acceptable developer velocity.

Summarizing the coordination hurdle facing engineering organizations, Matei Zaharia observes: “If everyone is just vibe coding their own thing, they will still be incoherent.”

[18:56](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1136s) Introducing Omnigent: The Open Meta-Harness

To resolve the fragmentation across agent implementations, Databricks recognized the need for an open abstraction layer positioned directly above individual agent harnesses: a "meta-harness." Instead of replacing existing frameworks, the meta-harness creates a unified interface that standardizes agent composition, team collaboration, and policy-driven control.

Zaharia announces the open-source release of **Omnigent**. Designed to accommodate any underlying harness, Omnigent natively integrates with leading coding agents (including recent additions such as Cursor and Antigravity, added right after launch in response to developer demand) as well as custom multi-agent architectures built on tools like the OpenAI Agents SDK. By sitting across varied frameworks, Omnigent enables centralized auditing, seamless session handoffs, cross-agent loops, and robust policy enforcement without locking organizations into a single vendor's agent runtime.

[20:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1200s) Omnigent Architecture: The Runner, the Server, and Uniform Interfaces

Omnigent is designed to solve the fragmentation of agent harnesses by providing an open meta-harness architecture structured around a clean, minimal interface: sending messages in, receiving event streams out, and invoking tools. Under the hood, Omnigent consists of two primary architectural components: a local runner and an optional centralized server. The runner acts as a lightweight wrapper around any underlying agent harness. It sandboxes execution to enforce system-level security constraints, continuously monitors agent processes, and ensures runtime stability without requiring proprietary infrastructure.

```mermaid flowchart TD accTitle: Omnigent Architecture Overview accDescr: Diagram illustrating how agents run inside an Omnigent Runner and connect to an optional Omnigent Server with multi-modal interfaces.

subgraph Execution ["Agent Execution Layer"] A["Agent (Claude / Codex / Custom)"] R["Omnigent Runner\n(Sandboxing & Monitoring)"] A <--> R end

subgraph Server ["Omnigent Server (Postgres, Docker, OpenTelemetry)"] H["Central Session History"] C["Agent Catalog & Sharing"] M["Central MCPs & Skills"] P["Security & Cost Policies"] end

subgraph Interfaces ["Uniform Access Interfaces"] T["Terminal CLI"] W["Web UI"] N["Native App / Mobile"] API["REST API"] end

R <-->|"State, Events & Policies"| Server R --> T Server --> W Server --> N Server --> API ```

Connecting the runner to an Omnigent server unlocks enterprise-grade centralization and governance. The server maintains a unified session history across all agent interactions, provides a catalog for publishing and sharing agents across teams, standardizes Model Context Protocol (MCP) integrations and agent skills, and distributes centralized artifacts and security policies. Crucially, regardless of which underlying harness or runtime is used, Omnigent abstracts the presentation tier into a uniform interface: developers retain their native terminal CLI experience, while simultaneously gaining access through a web browser, a native desktop application, mobile clients, and a programmable REST API. The entire system is fully open source, requires no dependencies on Databricks, and is built on standard commodity infrastructure including Docker, PostgreSQL, and OpenTelemetry to support deployments ranging from individual developer laptops to large-scale enterprise clusters.

- The core Omnigent interface standardizes message inputs, event stream outputs, and tool-calling primitives across disparate agent runtimes. - The server stack relies on open-source, non-proprietary standards (Docker containers, PostgreSQL database, OpenTelemetry tracing). - Developers do not sacrifice their terminal workflows; Omnigent layers Web, native desktop, mobile, and REST interfaces on top of existing command-line interfaces.

---

[21:20](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1280s) Composition, Real-Time Collaboration, and Contextual Governance

Standardizing agent runtimes into this uniform meta-harness layer unlocks three fundamental capabilities: composition, collaboration, and control. For composition, engineers can combine heterogeneous agents across different harnesses and reinforcement-learning-tuned models (RLMs) using simple YAML declarative definitions. Developers can seamlessly swap the underlying harness of an active, running agent session mid-flight without losing execution state or history. For collaboration, Omnigent introduces live multi-user sessions, enabling teammates to jump directly into an ongoing agent workspace, converse with the agent, inspect diffs, and comment on modified files simultaneously in real time.

Addressing the third pillar—control—the speaker highlights how conventional agent tooling relies on crude, static tool-level permissions that merely approve, deny, or prompt the user per invocation. Omnigent replaces this with state-aware, contextual policy enforcement where runtime permissions dynamically adjust based on the agent's prior actions and data access patterns:

> “Today in most agent tools, the control is just for each type of tool call. Do I allow it or deny it or ask the user? They're very static. We support contextual policies where the permissions vary based on what the agent did before.”

Under this model, if an agent inspects confidential internal files, the policy engine automatically revokes its ability to call external communication tools, such as sending external emails; conversely, if the agent has accessed only public marketing collateral, outbound communications remain authorized. Omnigent also introduces strict financial guardrails—a capability missing from existing agent frameworks—allowing operators to set hard expenditure limits (such as capping a session's API consumption at $5.00), paired with an OS-level sandbox to prevent malicious or accidental host system damage.

---

[23:20](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1400s) Demo: Wrapping Claude with Omnigent and Bridging Terminal with Web UI

Moving to a live demonstration, the presenter showcases Omnigent running between a remote development machine inside Databricks and a local laptop. Wrapping an existing agent requires nothing more than prepending the CLI command with `omni`, demonstrated via `omni claude`. The runner initializes, loading Databricks' internal configuration of Claude Code, pre-wired with internal Databricks MCP servers and custom prompt behaviors—including insider Databricks terminology like “archiving.” When prompted in the terminal to explain the current workspace repository (`What is this project?`), Claude parses the repository and responds directly within the terminal interface.

```mermaid flowchart LR accTitle: Bi-directional UI Synchronization in Omnigent accDescr: Diagram showing bi-directional synchronization of inputs and outputs between the terminal CLI and the Omnigent Web UI.

U1["Developer (Terminal CLI)"] <-->|"omni runner"| Session["Omnigent Active Session State"] U2["Developer (Web Browser / Native App)"] <-->|"WebSockets / API"| Session Session <-->|"Tool Calls / File Edits"| Project["Project Workspace (README.md)"] ```

While running an agent in the terminal is familiar, Omnigent simultaneously exposes that exact session inside a web UI. The browser view renders a rich chat interface, an embedded live mirror of the terminal output, and a visual file explorer showing modified assets and diffs. Demonstrating cross-surface synchronization, the presenter submits a prompt from the web interface instructing the agent to add a joke to the top of the project's `README.md`. The input instantly pipes into the running terminal process, which executes the edit.

Inspecting the changes through the web UI's markdown visualizer, the presenter highlights how developers can review diffs, leave line-by-line comments (e.g., requesting the removal of previous jokes and keeping just one), and instruct the agent to address all visual review feedback in a single batch. To circumvent the friction of losing agent sessions inside cluttered browser tabs, the presenter also demonstrates Omnigent's dedicated native desktop application, which provides persistent access, desktop notifications, and focused window management.

---

[26:35](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1595s) Switching Harnesses Mid-Session and Orchestrating Multi-Agent Workflows with Poly

A central capability of Omnigent is the ability to fork an ongoing session and switch harnesses without sacrificing context. Demonstrating this, the presenter forks the active Claude session directly from the interface and switches the runner over to OpenAI Codex. Codex spins up in the terminal, instantly inheriting the complete conversational history and file context established by Claude. The presenter prompts Codex to add a second joke to the `README.md`, showing how a developer who gets stuck on a complex coding problem with one model or harness can switch to another without restarting from scratch.

```mermaid flowchart TD accTitle: Poly Multi-Agent Supervisor Workflow accDescr: Flowchart illustrating the Poly multi-agent supervisor orchestrating parallel worker agents to evaluate project tasks.

Supervisor["Poly Supervisor Agent"] Task["Evaluate & Rate README Jokes"] Supervisor --> Task

subgraph ParallelWorkers ["Parallel Sub-Agent Execution"] Worker1["Claude Code Worker\n(Runs evaluation)"] Worker2["Codex Worker\n(Runs evaluation)"] end

Task --> Worker1 Task --> Worker2

Worker1 -->|"Rating & Critiques"| Aggregator["Supervisor Aggregation"] Worker2 -->|"Rating & Critiques"| Aggregator Aggregator --> Output["Consolidated Verdict:\nJoke 1 vs. Joke 2"] ```

Beyond point-to-point harness switching, Omnigent natively facilitates multi-agent hierarchical composition. The presenter launches Poly, an included open-source multi-agent coding supervisor that orchestrates multiple autonomous agents across distinct harnesses. Executing Poly in the same project directory, the presenter tasks it with evaluating the jokes added to the documentation:

> “So we actually include an example here, Poly, which is a multi-agent coding supervisor that can use both Claude and Codex to do subtests.”

Poly automatically inspects the environment to verify whether Claude and Codex are installed, spins up two dedicated worker processes in parallel—one running Claude Code and the other running Codex—and assigns each worker the task of reviewing and grading the jokes. The developer can inspect each worker's discrete terminal stream in real time as they perform their analysis. Poly's supervisor layer synthesizes their independent evaluations, reporting back a comparative judgment (rating the first joke as solid, and the second as inferior). The presenter notes that this pattern extends far beyond simple reviews—such as parallelizing the triage of open GitHub issues and verifying in codebase history whether bugs have been patched—before pivoting to demonstrate real-time engineer collaboration.

- Poly automatically detects available local CLI agents (such as Claude Code and Codex) and spawns isolated sub-agents as background workers. - Developers can monitor both high-level supervisory summaries and raw, per-worker terminal execution trees simultaneously. - Composition enables practical parallelization of developer tasks, including bug investigation, PR reviews, and multi-model consensus evaluations.

[30:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=1800s) Live Collaboration, Stateful Policies, and Open-Sourcing Omnigent

Continuing his live demonstration of Omnigent, Matei Zaharia simulates a collaborative debugging session by inviting Ankit, an engineer on the Databricks AI team, into his workspace. Zaharia rings Ankit on a live call, catching him off guard. Ankit answers with mock exasperation: “Oh man, Matei’s calling me on vacation?” Matei asks him to review an update to a repository's `README` file: “I know the agent said that joke one is better, but I kind of like joke two.” With session-sharing enabled, Ankit immediately views the active session, leaves comments, and issues direct prompts to the running agent, demonstrating how teams can seamlessly hand off complex agent workflows. Zaharia emphasizes that because every session and full interaction history is retained on the user's server, teams can retroactively inspect and analyze all agent activity.

Turning to governance, Zaharia highlights Omnigent’s built-in stateful policy controls, tracking per-agent token spending in real time. Because these policies are contextual and evaluate current runtime state, developers and centralized administrators can establish programmatic constraints, such as restricting an agent from spending more than a specified budget threshold without human approval. To demonstrate, Zaharia inputs a mock condition prompting for confirmation if spending exceeds $0.20 after an agent has already consumed $0.25. When he commands it to execute a task requiring extensive file reading, the agent pauses mid-execution and asks for user permission. Zaharia approves it, showing that the system will execute until reaching an absolute $5 ceiling, where it shuts off completely:

> “So this is a really cool feature, again, on top of any agent, Claude, Codex, whatever it is, and it's just scratching the surface.”

```mermaid flowchart TD accTitle: Omnigent Stateful Policy and Collaboration Flow accDescr: Flow diagram showing user collaboration, agent execution with stateful budget checkpoints, and centralized policy enforcement. subgraph Collaboration ["Live Multi-User Workspace"] U1["Primary Developer (Matei)"] <-->|"Shares session"| U2["Reviewer (Ankit)"] U1 & U2 -->|"Prompt & instruct"| AG["Running Agent"] end subgraph RuntimeGovernance ["Stateful Policy Engine"] AG -->|"Execute tool calls"| POL{"Cost > Budget Policy?"} POL -->|"Exceeds threshold"| HIL["Pause & Request Approval"] HIL -->|"Approved"| EXEC["Resume Execution"] POL -->|"Hard limit reached"| STOP["Halt Agent Run"] end ```

Matei Zaharia explains that Databricks chose to open source Omnigent because agent development must move above the silo of individual software development kits (SDKs) and harnesses:

> “We think this meta harness layer is going to be needed, and it will benefit from being open the same way that data formats and sharing protocols and stuff like that benefit from being open, because it's an ecosystem thing where people sort of have to agree on a way to do it, to make it work well.”

The project has already drawn significant community contributions since its launch. Beyond programmatic workflows, Omnigent provides a complete open-source collaborative user interface—including mobile responsiveness—eliminating the need for teams to build bespoke frontends for internal agent tooling. Zaharia hints at upcoming ecosystem extensions and automated agent optimization capabilities, wrapping up by welcoming attendees to adopt Omnigent.

- **Collaborative features**: Full session-state sharing, inline commentary, and live handoffs to team members. - **Stateful policy engine**: Enforces dynamic spending limits (e.g., pause at $0.20 for human-in-the-loop signoff; hard kill at $5.00). - **Backend persistence**: Complete session and trace histories stored on-premises or on private enterprise servers. - **Cross-model and cross-harness support**: Works agnostically across Claude, OpenAI Codex, and other backends.

---

[34:44](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2084s) Databricks Free Edition: Bringing Enterprise Tooling to Learners

A recorded transition video outlines the adoption and educational scope of the Databricks Free Edition. The platform serves hundreds of thousands of users seeking hands-on mastery of production data and AI skills, ranging from agent design to biomedical literature synthesis and fragmented environmental data unification.

Educators and industry professionals spotlight how providing university students access to tools like Lakeflow Designer on the Free Edition bridges the traditional divide between academic instruction and enterprise deployment. When incoming hires possess hands-on familiarity with the platform, they bypass the onboarding learning curve and address core business challenges from their first week on the job. The program offers learners free, permanent access to an authentic production environment.

- **Target audience**: Students, researchers, and transitioning professionals developing enterprise data and AI skills. - **Core capabilities**: Hands-on access to production Databricks tooling, including Lakeflow Designer. - **Pricing tier**: Free indefinitely for registered learners.

---

[36:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2171s) Agent Bricks: Overcoming the Infrastructure Hurdle in Enterprise AI

Casey Uhlenhut, Director of Product Management at Databricks, takes the stage to shift the keynote's focus toward enterprise builders. While the summit's opening day explored consuming AI through interfaces like Databricks Genie, day two is aimed directly at developers constructing mission-critical autonomous agents.

Uhlenhut reviews the growth of Agent Bricks, Databricks' agent developer platform launched one year prior. Since debut, developers have built over 100,000 custom agents on the platform, MLflow GenAI experimentation and tracing has exceeded one billion downloads, and the Databricks AI Gateway has processed more than one quadrillion tokens. Enterprise adoption includes: - **7-Eleven**: Accelerated maintenance and repair workflows. - **MasterCard**: High-throughput fraud detection across billions of financial transactions. - **AstraZeneca**: Accelerated extraction of commercial insights for drug discovery teams.

Despite these milestones, Uhlenhut cautions that standard industry demonstrations misrepresent the difficulty of production agent engineering:

> “When you see agent demos and agent pilots, you're usually just seeing the agent loop, but the agent loop is only 1% of the work that you have to do. It's the other 99% that's gonna get you.”

```mermaid flowchart LR accTitle: Production Agent Development Breakdown accDescr: Diagram contrasting the 1 percent agent loop against the 99 percent infrastructure iceberg. subgraph Pilot ["1% Visible Work"] LP["Agent Loop / Prompting"] end subgraph Production ["99% Enterprise Infrastructure"] direction TB AUTH["Authentication & Identity"] MCP["Model Context Protocol (MCP) Integrations"] EXEC["Long-Running Reliability & Timeout Handling"] GOV["Context Retrieval & Budget Guardrails"] end Pilot -.->|"Deceptively simple"| Production ```

Developers tasked with building hundreds or thousands of enterprise agents inevitably become bogged down building foundational infrastructure rather than optimizing applications. Uhlenhut identifies the three primary bottlenecks facing these engineering teams: 1. **The Rapidly Shifting Frontier**: Models and agent harnesses evolve weekly, rendering six-month-old architectures obsolete and forcing teams to replatform their entire software stack to stay competitive. 2. **Context Fragmentation**: Competitive differentiation relies on corporate data, yet that context is scattered across disparate formats and silos—from unstructured PDF documents to external REST APIs—making accurate context selection difficult. 3. **Privileged Access Risks and Runaway Costs**: Because enterprise agents can execute mutations and access private records, they represent the most privileged actors in an organization. Developers face risks of accidental data leakage or catastrophic production database deletions, while unchecked execution loops risk exhausting cloud budgets before an application ever reaches deployment.

[40:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2400s) Agent Bricks: Unlocking Developer Choice Across Frontier and Custom Models

To keep engineers from getting trapped in the weeds of spinning up and managing bespoke infrastructure, Databricks is introducing Agent Bricks—a dedicated agent developer platform anchored on three core pillars: choice, context, and control. As the speaker explains, “Agent Bricks is our developer platform where we provide answers to all of these so that developers can focus on building agents, rather than on building infrastructure.” The first pillar, choice, ensures that developers are never locked into a single ecosystem. Agent Bricks provides out-of-the-box, native access to every leading frontier model, including Claude Opus, OpenAI's GPT series, Google Gemini, Alibaba's Qwen, Moonshot AI's Kimi, and, announced in partnership with SpaceX and xAI, the Grok model family.

Frontier models alone, however, cannot solve every enterprise challenge. Certain specialized or domain-critical tasks require tailored reinforcement learning (RL). Agent Bricks includes a complete, built-in model platform enabling organizations to train, serve, and deploy custom RL models optimized for their unique workflows right alongside frontier endpoints. Highlighting this capability, the speaker points out that Databricks’ internal Genie research team used this exact RL infrastructure to build custom models that actually outperformed Opus on their specific benchmarks.

Beyond model endpoints, choice extends directly to agent authoring frameworks and meta-harnesses. Developers can build and deploy agents using Databricks' newly open-sourced Omnigent harness, or stick with popular external frameworks like LangGraph and the OpenAI Agent SDK. To prevent painful engineering rewrites every time a new benchmark leader emerges, the platform exposes a single, unified interface across all models and agents. In the words of the speaker: “So we believe choice is about giving you the flexibility to stay at the cutting edge without having to re-platform your solution every time, and we're able to do this by giving you a unified interface across all of your models so that you can easily switch amongst them.”

- Native model access spanning Claude Opus, GPT, Gemini, Qwen, Kimi, and the newly announced Grok family. - Built-in reinforcement learning platform for training task-specific models that can run head-to-head with or beat top commercial models. - Native support for Omnigent, LangGraph, OpenAI Agent SDK, and custom agent harnesses. - A unified multi-model API eliminating the need to re-platform when switching underlying LLMs.

---

[42:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2531s) Solving the Context Problem: Unity Catalog, Document Intelligence, and Agent Memory Services

The second pillar of Agent Bricks is context—the information that grounds an agent in an organization's business, assets, and users. The speaker frames this plainly: “The best model in the world, though, is only as good as the information you give it, which takes us to context.” In Agent Bricks, all enterprise context is centrally cataloged and governed within Unity Catalog. Everything from tabular datasets and dashboards to external APIs becomes discoverable and callable as tools by agents. To solve the discovery and relevance challenge, the Genie Ontology acts as a rich semantic metadata layer sitting across all agentic assets, allowing agents to accurately retrieve and select exactly the right data without hallucinating irrelevant context.

```mermaid flowchart TD accTitle: Agent Bricks Context Layer accDescr: Diagram showing how enterprise context flows through Unity Catalog and specialized services to ground custom agents.

subgraph DataSources["Enterprise Context & Tools"] UC["Unity Catalog\n(Tables, Dashboards & APIs)"] DOCS["Unstructured Documents\n(PDF Contracts, Invoices)"] EXT["External APIs\n(Managed MCPs)"] end

subgraph Processing["Context Optimization Services"] ONT["Genie Ontology\n(Semantic Metadata Layer)"] DI["Document Intelligence\n(Parsing & Extraction)"] MEM["Agent Memory Services\n(Preferences, Sessions, History)"] end

subgraph AgentExec["Agent Runtime"] AGENT["Enterprise Agent\n(Omnigent, LangGraph, SDKs)"] end

UC --> ONT --> AGENT DOCS --> DI --> AGENT EXT --> AGENT MEM <--> AGENT ```

Handling unstructured enterprise data requires specialized tooling. Organizations frequently possess millions of complex invoices, contracts, and scanned PDFs that standard parsers mangle. Databricks' Document Intelligence provides purpose-built models for classification, parsing, and extraction engineered specifically for optimal placement on the quality-cost curve so teams do not go broke ingesting massive backlogs of files. Furthermore, for data residing outside Databricks, managed Model Context Protocol (MCP) integrations provide out-of-the-box authentication and secure connectivity to external APIs.

Finally, to support personalized user experiences without requiring developers to spawn independent, hard-coded agent instances for every single employee or customer, Databricks announced Agent Memory Services. These managed APIs store user preferences, conversational state, and long-term session histories. Addressing the sheer engineering lift required to build reliable memory, retrieval, and search, the speaker emphasizes: “Context is probably the most difficult problem that exists in the agent space today, which is why we have an entire research team dedicated to just solving the hardest problems here.” Innovations produced by this research team are continuously piped into both Agent Bricks and Genie.

- Context assets (structured tables, dashboards, external APIs) are registered, governed, and callable as tools through Unity Catalog. - Genie Ontology provides a semantic metadata layer so agents selectively retrieve the right context rather than drowning in irrelevant data. - Document Intelligence features custom parsing, extraction, and classification models tailored for complex PDF contracts and invoices. - Managed external MCP connectors handle authentication and data integration for systems outside Databricks. - Agent Memory Services provide managed APIs for storing long-term user preferences, multi-turn conversation histories, and persistent sessions.

---

[45:18](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2718s) Control via Databricks Sandbox: Safe, High-Speed Code Execution

The third pillar of Agent Bricks is control. As agents evolve from simple chat interfaces into autonomous systems that generate and run code, the risk of an unconstrained model running amok in enterprise systems becomes a major blocker. Developers often find themselves caught between wanting agents to write code to solve novel problems and fearing that an agent might accidentally wipe out an active database. Acknowledging this tension, the speaker jokes, “They can do like amazingly creative things if you just kind of let them cook, but you don't want them to take down production.”

To provide a secure, bounded execution environment, Databricks announced Databricks Sandbox. Databricks Sandbox provides ephemeral virtual machines where agents can safely execute generated code without any danger of contaminating or corrupting production environments. Unlike generic cloud virtual machines or container environments, Databricks Sandbox is natively wired into the data layer. It provides instant, downscoped access to enterprise datasets using OnBehalfOf permissions, ensuring that the running code operates strictly within the security clearance of the initiating user or identity. Crucially for autonomous workflows where agents iterate through trial-and-error loops, these sandboxes launch in mere milliseconds, allowing agents to spin environments up, execute analytical scripts, and tear them down on the fly without latency penalties.

- Instant-launch virtual machines designed for secure, isolated agentic code execution. - Native integration with enterprise data using downscoped OnBehalfOf permissions. - Millisecond startup and teardown times matched to rapid, multi-step agent reasoning loops. - Complete isolation from production infrastructure, eliminating risks of unintended system destruction or data corruption.

---

[46:51](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2811s) Unity AI Gateway: Unified Governance, Agent Registry, and Contextual Security Policies

Building toward what the speaker calls the “pièce de résistance” of the platform, Databricks introduced major capabilities in Unity AI Gateway. As the speaker explains: “Unity AI Gateway is our centralized governance, security, and cost control solution that allows you to unify governance across models, agents, MCPs, and skills.” It provides an operational control plane designed to tame sprawling, shadow AI development across large organizations through five key capabilities.

```mermaid flowchart LR accTitle: Contextual Security Policy Dynamic Tainting accDescr: Flowchart illustrating how accessing sensitive PII data dynamically taints an agent's session state and limits permitted actions.

subgraph State1["Public Data Access"] A1["Agent queries public web / docs"] --> B1["State: Untainted"] B1 --> C1["Action: Can post to public web & send email"] end

subgraph State2["Sensitive Data Access"] A2["Agent queries customer PII table"] --> B2["State: Tainted"] B2 --> C2["Action: Block public publishing"] B2 --> D2["Action: Allow internal email only"] end ```

The first capability addresses enterprise sprawl: the Agent Registry. Recognizing that hundreds or thousands of agents are being built across business units using mismatched tools, the Agent Registry acts as a single centralized system of record. Teams can register internal Databricks agents, external agents, MCP tools, and discrete agentic skills into one searchable inventory. The second capability introduces dynamic, contextual security policies based on data tainting. The speaker illustrates this with a concrete scenario: if an agent interacts solely with public data, it is allowed to publish its findings to the public web. However, the moment that same agent accesses sensitive customer data containing personally identifiable information (PII), its state is dynamically tagged as "tainted." From that point forward, the gateway alters its permitted actions—it may still be allowed to draft an internal email to a colleague, but any attempt to publish data to an external or public destination is strictly blocked.

The speaker highlights why this dynamic policy engine is distinct to the platform: “And this is only possible on Agent Bricks, because our agent platform is built and integrated into our data platform.” In addition to contextual tainting, Databricks has partnered with leading security vendors to embed turnkey security guardrails directly into the gateway. Finally, the speaker turns to the pervasive issue of runaway inference budgets—humorously labeled “tokenmaxxing”—where unconstrained iterative loops drain project budgets before applications ever reach production, setting up the gateway's advanced cost-control and rate-limiting features.

- Centralized Agent Registry functioning as the single system of record for all enterprise agents, external agents, MCP endpoints, and skills. - Stateful, contextual security policies that dynamically alter agent permissions (e.g., blocking public egress after touching sensitive PII). - Deep integration between data catalog governance and model runtime security. - Security partnerships offering pre-configured behavioral and content guardrails. - Centralized cost controls, monitoring, and budget boundaries to curb runaway token consumption.

[50:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3000s) Unity AI Gateway: Budgets, Intelligent Routing, and Agent Tracing

Casey expands on the governance and economic challenges enterprises face when managing autonomous agents. With Unity AI Gateway, organizations can move beyond simple post-hoc spend tracking to proactively enforce spend limits for individual users and entire teams through dedicated budgets. However, simply knowing how much has been spent is not enough; enterprise operators need automated, intelligent cost control so engineers and administrators do not have to micromanage every token.

```mermaid flowchart TD accTitle: Unity AI Gateway Intelligent Routing Architecture accDescr: Flow showing incoming prompts routed by complexity across model tiers, multi-cloud fallbacks, and logging into Unity Catalog.

subgraph Client["Developer / Agent Layer"] A["Developer or Agent Prompt"] end

subgraph Gateway["Unity AI Gateway"] B{"Intelligent Routing Engine"} C["Cheap / Fast Models\n(Trivial Tasks)"] D["Expensive / Powerful Models\n(Complex Tasks)"] E["Multi-Cloud & Multi-Provider\nFallback Protection"] end

subgraph Storage["Governance Layer"] F["MLflow Tracing Engine"] G[("Unity Catalog Trace Table")] H["Genie Analysis & Debugging"] I["Lakewatch SIEM / Alerting"] end

A --> B B -->|"Low Complexity"| C B -->|"High Complexity"| D B -.->|"Provider Outage"| E C --> F D --> F F --> G G --> H G --> I ```

To solve this, Unity AI Gateway introduces intelligent routing. The gateway evaluates the contextual complexity of incoming requests, automatically dispatching lightweight, straightforward questions to inexpensive, high-speed models while reserving costly, highly capable frontier models for deeply complex tasks. Because the architecture operates as a multi-cloud and multi-AI layer, smart routing also grants resilience: organizations can establish dynamic fallback chains spanning both distinct model providers and underlying cloud infrastructures to ensure high availability.

Finally, the gateway integrates full agent tracing powered by MLflow tracing. The system captures the full execution graph—every input, every output, and intermediate agent reasoning steps—and centrally records them into a customer-owned table managed within Unity Catalog. Because trace logs contain proprietary context and enterprise secrets, storing them in Unity Catalog allows enterprises to maintain complete data ownership. Once stored, teams can query these traces using Databricks Genie to diagnose failures or connect them directly to LakeWatch (Databricks' SIEM offering) to catch data leakage and trigger incident alerts in real time. Casey emphasizes:

> “So this is what AI governance should look like in the enterprise, where all of your assets can be centrally managed, you can have contextual policies to enforce security, you have an open ecosystem where you can connect to partners, and you can have the cost controls that you need.”

- **Centralized Asset Management**: Unifies tools, models, and agents under cohesive governance rules. - **Enterprise Observability**: Automatically persists granular MLflow traces into governed Unity Catalog Delta tables. - **Security & Incident Response**: Integrates directly with LakeWatch for automated real-time threat detection and leakage alerting.

---

[52:37](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3157s) Demo: Developer Coding Agents, Smart Budgets, and Token Economics

Ankit Mathur, a software engineer at Databricks, takes the stage after returning from vacation to demonstrate how Unity AI Gateway manages real-world developer workloads. Addressing the audience as an active practitioner, Mathur points out that engineering organizations are experiencing explosive cost growth from autonomous and interactive AI assistants. The top 1% of AI spenders now incur costs reaching $90,000 per user per year. Cost escalations are no longer confined to human software engineers drafting isolated scripts; employees across an entire company now trigger multiple agents running in parallel, which execute persistently as long as compute and API access remain unchecked.

To demonstrate how the gateway reclaims control, Mathur sets up his terminal environment using standard tools like Codex, simply pointing the tool’s endpoint directly at the Unity AI Gateway. As the session launches, inference requests and Model Context Protocol (MCP) tool calls are immediately funneled through a centralized governance choke point. MCP integrations authenticate once through a central registry rather than requiring fragmented credentials per machine.

```mermaid flowchart LR accTitle: Unified AI Gateway Developer Budgeting accDescr: Flow from local developer terminal to Unity AI Gateway showing real-time budget tracking and gradual model downgrade policies.

subgraph Terminal["Developer Environment"] A["Terminal / Codex CLI"] end

subgraph Gateway["Unity AI Gateway Layer"] B["Unified Central Choke Point\n(Single-Sign-On MCPs)"] C{"Real-Time Spend Tracker\n($50/day Budget Limit)"} D["Standard / Frontier Models\n(< Threshold)"] E["Token-Efficient Guidance\n(e.g., Databricks Kimi)"] F["Strict Policy Enforcement\n(At 100% Cap)"] end

A --> B B --> C C -->|"Normal Spend"| D C -->|"Approaching Limit"| E C -->|"Exhausted Limit"| F ```

Mathur walks through an initial developer query asking for a coffee shop near Moscone Center. The request executes transparently, but behind the scenes, the gateway aggregates inference into a cross-tool budget. In Mathur’s case, his profile is bound to a $50-per-day allowance. Instead of employing blunt, productivity-killing hard cutoffs, the gateway enforces nuanced, gradual policies. As a developer nears their ceiling, the gateway begins issuing contextual warnings and guiding them toward token-efficient alternatives, such as the newly released Databricks Kimi model, which operates at a fraction of standard frontier pricing. Active sessions are allowed to finish gracefully to prevent breaking developer workflows, but new sessions automatically default to high-efficiency alternatives based on real-time burn rates.

Mathur captures the scale of the developer spend dilemma when setting up the demonstration:

> “The top 1% of spenders are spending $90,000 per user per year, and that exponential isn’t showing signs of slowing down.”

- **Cross-Tool Budgeting**: Consolidates developer consumption across distinct command-line interfaces and editors under a unified policy. - **Graceful Soft-Throttling**: Prevents sudden workflow interruptions by advising model transitions during active work and applying restrictions to subsequent sessions. - **Centralized MCP Authentication**: Single-point authorization eliminates distributed API token leaks across local machines.

---

[55:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3300s) Omnigent Meta-Harness Routing and Unity Catalog Tracing Analytics

Mathur advances the demonstration by coupling the Unity AI Gateway with Omnigent, the open-source agent meta-harness. When an agentic system acts as a meta-harness, it operates at a layer above isolated individual tasks, allowing it to deconstruct complex user intents and make multi-tiered model routing decisions across sub-agents.

When Mathur issues a simple local recommendation request, Omnigent recognizes the query's low complexity, directing it through the gateway to an inexpensive model that returns an answer almost instantaneously with minimal latency. However, when he prompts it with a heavy software engineering task—instructing it to locate the top four bugs in the open-source Omnigent repository—the meta-harness partitions the objective and routes each sub-task to the most cost-effective tier: 1. **Distribution & Orchestration**: A balanced, medium-quality model splits the work. 2. **Repository Surface Scan**: A massive token-consumption task routed to Databricks Kimi to maintain low latency and rock-bottom token cost. 3. **Edge Case Review**: Dispatched to Gemini 3.5 Flash for high reasoning density at low price points. 4. **Core Logic & Deep Auditing**: Assigned to premier, expensive frontier models where code correctness is mission-critical.

Each delegated task runs in parallel inside isolated Databricks Sandboxes, preserving enterprise security boundaries while multiple sub-agents execute simultaneously.

Mathur points out that optimization requires measurement, which is enabled by routing all agent traces directly into a centralized Unity Catalog trace table. Because these logs can expose sensitive corporate data, access is protected using standard Databricks governance mechanisms, including Attribute-Based Access Control (ABAC) and governed tags. With all trace data centralized, engineering teams can use Genie to interrogate past sessions. Mathur reveals that an audit of his own team's developer traces before adopting the gateway showed that 90% of sessions had defaulted to top-tier models like Claude Opus for trivial prompts like "hi" or standard boilerplate tasks. Centralized tracing provides the exact data needed to pinpoint which developer workflows should be routed to cheaper models and which paved paths should be formalized into reusable, catalog-governed agent skills. Mathur observes:

> “Because without data, we can’t be data driven. And today, coding agent sessions aren’t centralized to a single place.”

- **Task-Specific Decomposition**: Dispatches repository-wide scans to high-throughput budget models while routing critical logic audits to frontier reasoning engines. - **Databricks Sandbox Execution**: Parallel sub-agents execute code in secure, isolated VMs to eliminate environmental security risks. - **Enterprise Trace Governance**: Protects sensitive agent interactions using Unity Catalog ABAC policies and governed classification tags. - **Data-Driven Cost Optimization**: Historical trace analysis through Genie revealed that 90% of internal developer sessions previously overpaid for frontier models on trivial interactions.

[01:00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3600s) Unity AI Gateway Wrap-Up and Agent Bricks Platform Overview

Ankit wraps up his demonstration of the Unity AI Gateway by emphasizing developer token efficiency and full operational observability. With features like configurable budget policies, the intelligent model router, and end-to-end tracing, organizations can finally alter their AI spending trajectory while continuing to deploy autonomous systems aggressively. Pointing out that raw intelligence remains one of the scarcest and most expensive resources in modern computing, Ankit highlights the core philosophical shift enabled by the gateway: transitioning enterprise development away from wasteful brute-force consumption toward deliberate value creation. As Ankit summarizes, “And so hopefully we can take the world from the old world of token maxing to something a little bit more like value maxing with the Unity AI gateway.”

Casey returns to the stage to contextualize where Unity AI Gateway sits within the broader Agent Bricks platform. She positions Agent Bricks as an end-to-end foundation built directly on top of Databricks’ core data engine, designed around three primary tenets: - **Choice**: Ensuring organizations retain architectural flexibility across underlying foundation models, distinct agents, and runtime harnesses without being locked into a rigid platform or forced to replatform. - **Context**: Supplying agents with rich metadata and lineage so they can retrieve the exact enterprise data required for complex workflows. - **Controls**: Enabling autonomous agent operation through the centralized governance of Unity AI Gateway and the isolation of Databricks Sandbox, which runs execution payloads safely against downscoped data.

```mermaid flowchart LR accTitle: Agent Bricks Architecture accDescr: Three core pillars of the Agent Bricks platform providing choice, context, and control. subgraph AgentBricks["Agent Bricks Platform"] direction TB Choice["<b>Choice</b><br/>Flexible Models, Agents & Harnesses"] Context["<b>Context</b><br/>Metadata & Enterprise Data Access"] Control["<b>Controls</b><br/>Unity AI Gateway & Databricks Sandbox"] end Choice --> AgentBricks Context --> AgentBricks Control --> AgentBricks ```

---

[01:02:44](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3764s) Satya Nadella on Systems of Context, Token Capital, and Enterprise IP

Databricks CEO Ali Ghodsi introduces Microsoft Chairman and CEO Satya Nadella to reflect on their nearly decade-long strategic partnership. Ali notes how quickly the landscape has transformed over the past year, prompting Satya to explain how customer mindsets have fundamentally evolved. Satya explains that enterprise leaders have moved past fixating solely on base foundation models and their standalone benchmark scores. Instead, organizations are focused on establishing their own proprietary frontier capabilities powered by their private business logic and intellectual property. Satya characterizes this industry-wide pivot: “So what I mean by that is, instead of just talking about here is the latest model and its capabilities, we are now going from frontier model worship to actually building a frontier ecosystem, building one company at a time.”

Ali connects this evolution to themes from Microsoft’s recent Build conference regarding the "future of the firm," where proprietary data and private IP form an organization's primary competitive moat. Satya expands on this by contrasting legacy enterprise software architectures with modern AI systems. While earlier enterprise eras centered on static systems of record (databases and ERPs) and systems of engagement (interfaces and communications), modern agentic AI demands dedicated *systems of context*. Without rich, curated contextual grounding, models engage in repetitive, costly token consumption.

Furthermore, enterprise data is not a static repository; it is an active, continuously evolving stream that includes execution histories and agent run traces. Satya shares a compelling example demonstrated at Microsoft's CEO Summit: a lightweight 5-billion-parameter model from Microsoft AI (MAI) successfully outperformed massive, frontier-class models on enterprise evaluations. This occurred not because of proprietary model magic, but because the smaller model was continuously tuned on high-quality enterprise data, reinforcement learning environments (RLE), and real execution traces. Satya warns that enterprises currently wasting tokens by running frontier models over raw data without retaining or learning from their traces are inadvertently leaking their most valuable asset—a dynamic he predicts will reverse quickly.

---

[01:05:40](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3940s) Deep Azure Integrations and Compounding Token Capital

Ali reinforces Satya's point, describing how poorly grounded agents enter inefficient loops—performing what amounts to a stochastic random walk over raw data, repeatedly burning tokens and driving up latency to extract insights that should have been captured structurally. Ali outlines the expanding native integrations between Databricks and the Microsoft ecosystem, including: - Embedding Databricks Genie directly within Microsoft 365 Copilot and Microsoft Teams. - Native integration across Microsoft Entra and Agent 365. - Exposing the Databricks Genie Model Context Protocol (MCP) server inside Microsoft Copilot Studio, allowing developers to build custom agents powered by Databricks context. - Bidirectional read/write interoperability across Delta Lake and Microsoft Fabric's OneLake.

Satya highlights how crucial these touchpoints are for enterprise economics, emphasizing that surfacing context right where users collaborate enables teams to accumulate compounding advantages in both workforce skills and computational assets. When Ali asks what the next decade of partnership and enterprise technology holds, Satya offers a foundational perspective on corporate longevity:

> “I mean, I look, I think the next 10 years for us will be a lot of what we talked about. That if a firm is about the tacit knowledge and learning, in fact, I always say this, which is you can always buy a tool, you can even sometimes outsource a job or a task, but you can never as a firm outsource your learning. So in this new world, what does it mean to build that tacit knowledge that is compounding both human capital and token capital?”

Satya concludes that while today’s enterprises rely on human capital navigating digital systems to build operational knowledge, the future belongs to organizations that compound human capital alongside "token capital"—autonomous systems trained on an organization’s own data and traces to generate sustained business value.

---

[01:09:16](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4156s) Databricks Apps: Breaking the Enterprise Data Bottleneck

Following Satya Nadella's appearance, Justin de Brabant, Director of Product Management at Databricks, takes the stage to address application development. Justin begins by reflecting on Databricks Apps, which launched the previous year with the goal of democratizing data and AI development across technical and non-technical teams alike.

He emphasizes how dramatically software creation economics have inverted over the past twelve months. With the explosion of autonomous coding agents and assisted development workflows, the historical friction and financial expense associated with writing application code has virtually collapsed. However, this shift has exposed a much harsher constraint for real-world enterprise deployments. Code generation may be trivial, but building production-grade enterprise software remains bottlenecked by governed, secure, and permissioned access to high-stakes internal data.

[01:10:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4200s) Databricks Apps: Governed Data Access and Massive Customer Momentum

Enterprise software applications are fundamentally different from standalone code repositories because their utility depends entirely on access to underlying enterprise data. As the speaker explains, the core hurdle teams face when attempting to move from an initial prototype into production is establishing secure, governed data access. To resolve that friction, Databricks Apps was engineered as an enterprise application platform with data security and governance built directly into its foundation. Over the past year, adoption has surged dramatically: more than 5,000 customers are actively deploying applications on the platform, and over 150,000 apps have been created—representing a sixfold increase compared to the previous year. Crucially, these systems are not sitting idle; hundreds of thousands of end users interact with them weekly, including many employees who previously had no direct touchpoints with enterprise data and AI tools in their daily workflows.

The speaker highlights that speaking with customers reveals the sheer variety of problems being solved across industries. Two aerospace leaders demonstrate this diversity: - **Thales:** The multinational aerospace and avionics company overhauled its in-flight intelligence platform for commercial airlines. The application aggregates system health telemetry, content engagement statistics, and passenger satisfaction signals across entire global airline fleets. This operational view allows the Thales operations team to anticipate cabin hardware and service issues before passengers ever notice them. The solution stitches together multiple Databricks components—real-time ingestion via Lakebase, lakehouse analytics, Genie conversational interfaces, and machine learning models forecasting operational needs—into a single interface. - **Joby Aviation:** The manufacturer of all-electric vertical takeoff and landing aircraft operates with extreme supply chain and material planning complexity behind every build. Joby created a unified application that maps demand, parts inventories, and procurement orders across all manufacturing programs in its fleet, allowing planning teams to detect parts shortages and bottlenecks well before they cascade into manufacturing delays.

As the speaker emphasizes when comparing these varied implementations, “These are all very, very different applications. Different teams, different applications. And this is what you see across all of our customers. Everybody is really innovating and doing different things across all their different industries.”

[01:12:44](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4364s) The Platform Foundation: Govern, Host, and Build

Databricks delivers these capabilities through three foundational pillars: Govern, Host, and Build. Each addresses a specific engineering hurdle that teams encounter when operationalizing internal web applications.

```mermaid flowchart TD accTitle: Databricks Apps Architecture Pillars accDescr: The three functional pillars of Databricks Apps comprising Govern, Host, and Build.

subgraph Govern["1. Govern: Identity & Data Controls"] direction TB G1["Databricks SSO Managed Auth"] G2["Unity Catalog Permissions (App or User Level)"] G3["Real-time Observability via ZeroBus into Managed Tables"] end

subgraph Host["2. Host: Managed Cloud Infrastructure"] direction TB H1["Configurable Instances & Auto-Scaling"] H2["Load Balancing & Sticky Sessions"] H3["Versioned Builds & Zero-Downtime Deployments"] end

subgraph Build["3. Build: Developer Experience & Ecosystem"] direction TB B1["Framework-Agnostic (Python / JavaScript)"] B2["AppKit TypeScript SDK (Lakebase, Genie, Jobs)"] B3["Coding Agent Plugins, Skills & MCPs"] end

Govern --> Host --> Build ```

- **Govern:** Identity is managed out of the box through Databricks SSO, eliminating the need to configure a separate identity provider or handle unique credentials for every application. Fine-grained authorization is enforced through Unity Catalog, giving administrators the flexibility to govern data access at either the overall application identity level or the individual end-user session level. Platform teams also get native, real-time observability: every user action and backend service invocation is streamed into ZeroBus in real time and persisted into Unity Catalog managed tables for audits and operational analysis. - **Host:** Databricks provides managed, cloud-agnostic container infrastructure. While the platform launched a year prior with only a single always-on instance type, it now supports customizable instance sizing and horizontal auto-scaling. The routing layer includes native load balancing and sticky session handling for multi-instance applications, complemented by versioned builds and zero-downtime rolling deployments to safeguard production releases. - **Build:** The runtime is framework-agnostic, supporting standard Python and JavaScript toolkits. Developers can accelerate delivery using AppKit, a native TypeScript SDK that includes pre-built integration plugins for Lakebase, Genie, autonomous agents, Databricks Jobs, and core Lakehouse tables. Furthermore, the platform integrates with AI coding agents through custom Model Context Protocol (MCP) servers, plugins, and skills created alongside developer ecosystem partners.

Reflecting on the platform's architectural philosophy, the speaker notes: “At the core of Databricks governance for apps is our managed auth offering. We have out-of-the-box identity through Databricks SSO. You don't have to have a separate IDP. You don't have to have a separate login for each individual app. It's all managed through Databricks.”

[01:16:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4571s) The Prototype-to-Production Gap in the Vibe Coding Era

While central data teams initially dominated internal app creation, the nature of enterprise software creation is experiencing a massive demographic shift. Industry forecasts predict that 1 billion new custom enterprise applications will be built by 2028, with the vast majority created by citizen developers rather than traditional software engineers. To illustrate the operational friction this creates for IT and platform teams, the speaker recounts an internal experience involving a Sales Operations manager at Databricks:

The sales ops manager sought to streamline her department's Quarterly Business Review (QBR) process. Her goal was to combine customer usage and consumption metrics, recent Salesforce meeting logs, and pipeline revenue forecast models into a single interactive dashboard. Using Replit, she vibe-coded a working prototype, demonstrated it to sales leadership to widespread praise, and believed the application was ready for immediate rollout.

However, when she presented the prototype to the internal platform engineering team to take it live, the project stalled. Platform engineers immediately pressed her on core operational realities: what specific production data tables were being queried, what sharing policies applied, what dependencies were bundled, who paid for compute costs, and how security was maintained. Because the sales ops manager did not possess administrative rights to production environments containing sensitive customer records, moving the prototype to production deteriorated into weeks of back-and-forth reviews.

The speaker highlights that this dynamic is no longer an isolated incident, but an organizational reality: “And this isn't about just happening once. This is happening thousands of times across your organization. And I guarantee some of you on the platform team have experienced this. Vibe coding is taking over. And unless we have a governed solution to it, it's going to overrun us.”

[01:18:24](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4704s) Scaling Citizen Development: Introducing AppSpaces and Serverless Micro Apps

To reconcile the speed of citizen vibe coding with enterprise security and fiscal discipline, Databricks interviewed enterprise customers and identified two fundamental product requirements: multi-application governance and cost-efficient execution for lightweight tools. In response, Databricks announced two major capabilities:

- **AppSpaces:** Standard governance in Databricks Apps historically operated on a per-app basis, which becomes unmanageable when hundreds of non-technical creators deploy thousands of distinct utilities. AppSpaces introduces a multi-application administrative boundary. Platform administrators can define governance, network, and security policies once at the space level, automatically inheriting those rules across all apps deployed inside it. Admins can restrict access boundaries—such as mandating that all applications deployed within a Sales Operations space can only be viewed by authenticated members of the sales organization. AppSpaces also provides centralized operational management, tracking aggregate compute spend, resource consumption, and user traffic across all child applications. - **Serverless Micro Apps:** While always-on, auto-scaling instances are essential for mission-critical, high-traffic tools, they are financially inefficient for lightweight departmental dashboards accessed only occasionally, such as an internal ops report referenced twice a month. Serverless Micro Apps provides a dedicated runtime featuring fractional CPU and memory allocations paired with dynamic scale-to-zero capabilities when idle. When an incoming request arrives, the application cold-starts in seconds. Despite utilizing shared underlying serverless infrastructure, each application operates inside a fully isolated execution sandbox to preserve multi-tenant security.

As the speaker summarizes the platform scaling challenge: “How do we unlock the next wave of builders without sacrificing security, governance, or budgets? And this is what you told us. First off, we need governance and ops that scale to thousands of apps.”

[01:20:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4800s) Announcing Genie App Builder: Context-Aware Vibe Coding

The presenter addresses the final major obstacle data teams encounter when trying to move vibe-coded applications into production: tools need to understand not just raw enterprise data, but the complete business context in which apps are deployed. This includes specific compute resources, operating environments, and sharing policies. In response, Databricks announced the release of Genie App Builder, designed specifically to bring business-level awareness into the vibe-coding experience. As the speaker explains: “Genie App Builder is vibe coding that understands your business.”

Rather than relying purely on generic code generation, Genie App Builder is context-aware and leverages the Genie Ontology to vibe code with deep semantic comprehension of an organization’s business logic and underlying data assets. Because it is 100% managed within Databricks, the platform delivers native enterprise security and governance guarantees. It also enforces production quality by default by integrating the AppKit SDK alongside custom skills, ensuring that enterprise development best practices are baked into every application generated out of the box.

```mermaid flowchart TD accTitle: Genie App Builder Architecture accDescr: Flow diagram showing how prototype code is brought into an App Space, governed, connected to enterprise systems, and deployed via Serverless Micro VMs.

subgraph Ingestion ["1. Prototype Ingestion"] A["GitHub Repo / Prototype"] --> B["Genie App Builder UI"] C["Docs & Wireframes"] -.-> B end

subgraph Governance ["2. Governed App Space"] B --> D["Supply Chain Ops App Space"] D --- E["On-Behalf-Of (OBO) Credentials"] D --- F["Approved Sharing Policies"] D --- G["Allowed Connectors"] end

subgraph Resources ["3. Databricks Resources"] D --> H["SQL Warehouse"] D --> I["Model Serving Endpoint"] D --> J["Lakebase Real-time Instance"] D --> K["Embedded Genie Agent"] end

subgraph Deployment ["4. Execution & Preview"] H & I & J & K --> L["Serverless Micro VMs (Side-by-Side Preview)"] L --> M["One-Click Production Deployment"] end ```

---

[01:21:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=4871s) Demo: Building and Deploying an App with Genie App Builder

Stepping into the role of a Supply Ops Manager for the “Brickster Cup,” the presenter showcases an inventory command center app designed to track site-level inventory, monitor demand forecasts as tournament teams advance, and execute direct operational actions like triggering production runs. The workflow begins not from scratch, but from an existing prototype vibe-coded using one of Databricks’ vibe-coding partners and stored in a GitHub repository.

Navigating to the Genie App Builder homepage, the presenter walks through configured app spaces—designated project workspaces curated by system administrators that bundle allocated computes, data assets, on-behalf-of (OBO) credential scopes, sharing policies, and external connectors:

* **Resource Linking:** Upon importing the GitHub prototype into the `Supply Chain Ops` app space, the Genie App Builder immediately inspects the available space inventory and auto-wires necessary backend components: a SQL warehouse, a model-serving endpoint for inventory forecasts, a Lakebase instance for real-time inventory tracking, and default questions for an embedded Genie conversational agent. Real data promptly begins streaming into the interactive preview. * **Governance and Permission Boundaries:** When the app attempts to load production manufacturing data, the builder flags that the current app space lacks permission. Rather than bypassing corporate rules, the user is offered an admin access request flow; for the V1 release, the presenter simply asks the agent to strip out the manufacturing data and its corresponding chart. * **Side-by-Side Micro VM Previews:** Changes can be evaluated visually through a proposed-versus-current preview window. Under the hood, serverless micro VMs spin up multiple isolated instances of the application concurrently, enabling developers to review code and styling changes (such as turning a chart green) before approving them. * **Boundary Enforcement:** To verify governance guardrails, the presenter attempts to configure a custom agent to send daily summaries through a Slack connector, which fails gracefully because Slack has not been whitelisted for this space. Similarly, attempting to update sharing permissions to “everyone at my company” is blocked because the space strictly restricts distribution to the designated supply operations group. * **Deployment and Developer Hub:** With resources confirmed, hitting deploy transitions the app into production within seconds. The presenter highlights that all showcased capabilities are available immediately, and directs users to the new developer portal at `developers.databricks.com` for prompts, architectural blueprints, sample apps, and practical implementation guides.

---

[01:28:01](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5281s) Customer Story: Virtue Foundation Powers Clinical Care with Genie

A video presentation highlights the humanitarian work of the Virtue Foundation, a non-governmental organization focused on delivering surgical and clinical healthcare to underserved populations across low- and middle-income countries. Using Genie, the organization developed a custom agent trained on decades of proprietary healthcare operational data, transforming historic travel logs and clinical field records into an authoritative, actionable source of truth.

A representative from the Virtue Foundation describes the dramatic efficiency leap unlocked by their custom agent, noting: “Using Genie has allowed us to take what would have taken a research assistant months. We’re now able to get answers to questions in seconds.” By asking plain-language questions—such as identifying total hospital counts in Ghana and filtering specifically for those equipped for cardiology—the team surfaces mission-critical medical intelligence in real time. Ultimately, the speaker emphasizes the profound human impact of agentic data systems: “The fact that we’re solving real problems for real people’s lives is just a humbling experience.”

---

[01:29:22](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5362s) Introducing Agentic ML Engineering with Mike Del Balso

Mike Del Balso, Director of Product Management at Databricks, steps onto the stage, opening with a lighthearted acknowledgment of the physical distance from the wings: “Justin was right, this is a deceptively long walk.” Del Balso establishes his roots in foundational enterprise machine learning, recounting his early career on the Google Ads machine learning infrastructure teams that helped catalyze the modern MLOps movement. He then details his subsequent work helping start the machine learning team at Uber, where he led the creation and expansion of Michelangelo, Uber’s pioneering internal ML platform, setting the stage to explore the evolution from traditional MLOps to agent-driven machine learning systems.

[01:30:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5400s) The Technical Debt of Production Machine Learning and Why GenAI Accelerates Demand

Reflecting on their career, the speaker shares how they previously ran a production machine learning startup that was recently acquired by Databricks, where their team continues to tackle the industry’s defining challenge. Across the entire machine learning field, the north star has always been “to make machine learning as easy and automated as possible, so we can bring intelligent decisions to all of the important places, all the important decisions across our business.” Despite countless attempts across the tech landscape, most efforts have struggled because production ML remains fundamentally difficult.

To illustrate the point, the speaker presents a well-known diagram from a foundational paper published by their Google team roughly twelve years ago. In the visual, the actual machine learning code occupies just a tiny red box in the center, surrounded by massive blocks representing all the supporting systems required to build, deploy, monitor, and maintain production models. Under the hood, this operational friction is fundamentally a data problem. While some industry observers predicted that generative AI would dampen interest in traditional machine learning, the speaker explains that GenAI has actually accelerated it: every new piece of software and AI-driven user experience creates more products to recommend, prices to optimize, fraud vectors to catch, and loan applications to evaluate.

Because production machine learning is notoriously hard to engineer and maintain, enterprises end up rationing it to only a tiny fraction of potential use cases. Most operational decisions across an enterprise continue to rely on static logic, rigid heuristics, and rules of thumb, leaving enormous value on the table.

- In the past year alone, organizations have generated more operational software than ever, exponentially increasing the volume of automated decisions required. - Despite having thousands of business decisions ripe for intelligent modeling, most enterprises only run a handful of production models due to operational complexity. - Replacing intelligent ML with statically coded rules and heuristics results in companies “leaving money on the table” by making systematically dumber business decisions.

---

[01:33:04](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5584s) Why Agents Are the Only Way to Automate Machine Learning

Automating this machine learning loop requires more than raw pipeline code; it requires capturing the human intuition present at every stage. Currently, highly trained data scientists and PhDs are constantly in the loop, deciding which features to engineer, diagnosing model convergence, aligning training metrics with business objectives, and deciding when a candidate model is safe to promote into production. Accelerating this lifecycle demands an automation layer capable of evaluating tradeoffs and making decisions in the exact same manner as expert practitioners.

The speaker emphasizes that agents are uniquely positioned to solve this problem, remarking: “And there's actually only one technology that can do this. And that's agents.” While agentic workflows have already reshaped software engineering, they have failed to transform machine learning for two structural reasons: 1. **Lack of Integrated Infrastructure:** Only a tiny handful of elite tech companies have built the unified operational infrastructure required to run the full machine learning loop automatically. 2. **Missing Operational Context:** Agents cannot make high-quality modeling decisions because they lack access to the rich institutional knowledge, historical performance metrics, and contextual data that human engineers rely on.

Databricks aims to resolve both bottlenecks by pairing an agent-ready infrastructure stack with domain-specialized agents capable of driving the loop end-to-end.

---

[01:34:52](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5692s) Agent-Ready ML Infrastructure: Serverless, Single Data Model, and Unified Governance

Databricks has spent over a decade establishing its position as an industry leader in data science and machine learning, sitting at the top right of analyst evaluation matrices with more than 500 million models trained on the platform over the past year alone. However, that entire existing stack was architected specifically for human practitioners. Over the past 18 months, Databricks redesigned and optimized the platform to make it fully “agent-ready” across three foundational pillars.

```mermaid flowchart TD accTitle: Agent-Ready Machine Learning Infrastructure accDescr: Three pillars of Databricks agent-ready infrastructure feeding into a single source of truth for agents.

subgraph Pillars ["Agent-Ready Stack Pillars"] A["Serverless Compute<br/>(Auto spin-up/down, zero cluster state)"] B["Unified Data Model<br/>(Training, features, serving & monitoring)"] C["Governed Actions<br/>(Unity Catalog permissions for agent tools)"] end

subgraph Lifecycle ["ML Lifecycle Source of Truth"] D["Governed Delta Tables"] E["Autonomous Agents"] end

A --> D B --> D C --> D D <--> E ```

First, the entire environment is now serverless out of the box so that autonomous agents do not waste cycles configuring clusters, tuning driver nodes, or managing infrastructure state. Second, the entire machine learning lifecycle is instrumented with a unified data model: feature stores, training datasets, model serving endpoints, inference logs, and monitoring metrics all stream into the same governed tables. This shared foundation gives agents an immediate view of cause and effect whenever they alter features or retrain models. Third, Databricks expanded Unity Catalog beyond traditional data and asset governance (models, experiments, endpoints) to govern the specific actions and tools agents are permitted to invoke. As the speaker explains: “We now have governance on the actions that your agents can take so that you can trust any changes that your agents are making to automate this machine learning loop.”

---

[01:36:44](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5804s) Announcing AI Runtime: Serverless GPU Compute for Deep Learning and LLMs

To eliminate the operational hurdles of scaling advanced neural architectures and large language models, Databricks is introducing AI Runtime. This new compute tier brings serverless, on-demand GPU capacity to the platform with zero upfront commitments, allowing teams to instantly spin up deep learning and LLM training workloads at arbitrary scale.

Having run AI Runtime in private preview over the previous two months, hundreds of organizations have already migrated their deep learning workloads onto it. Highlighting real-world enterprise adoption, the speaker references Merck's deployment of the platform to accelerate their latest drug discovery models. Building on that momentum, Databricks is officially expanding AI Runtime to support multi-node distributed training, unlocking massive datasets and large-scale model architectures within the governed enterprise lakehouse environment.

- **On-Demand Elasticity:** Provides serverless GPU instances with no upfront reservation or capacity lock-in. - **Deep Learning & LLMs:** Built specifically to support neural network training, fine-tuning, and large language model pre-training. - **Multi-Node Expansion:** Now supports multi-node distributed GPU training to train on massive datasets at any scale. - **Enterprise Proven:** Hundreds of organizations, including pharmaceutical giant Merck, have already deployed production deep learning workloads on the runtime during preview.

---

[01:38:13](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=5893s) Announcing Genie Code for Machine Learning

With the underlying infrastructure optimized for automation, the speaker pivots to the agent layer designed to execute production ML tasks: Genie Code for Machine Learning. Available immediately, this tool upgrades Databricks’ core coding agent with specialized intelligence and best practices derived from seeing more real-world machine learning deployments than any other vendor in the ecosystem.

Genie Code for Machine Learning is natively plugged into the Databricks platform, giving it direct programmatic awareness of experiment tracking, feature stores, and model deployment pipelines. Crucially, its context engine has been overhauled to understand how real-world data science teams collaborate and evaluate tradeoffs. Instead of treating machine learning code in isolation, the agent draws upon the broader institutional context—the unspoken assumptions, baseline metrics, and validation goals that human practitioners use—so it can co-pilot feature selection, run automated evaluations, and work seamlessly alongside data scientists and ML engineers.

[01:40:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6000s) Genie Code for ML: Ontology-Powered Machine Learning Engineering

Machine learning decisions hinge on deep institutional context that is typically scattered across an organization's entire platform: historical experiments, specific business metric definitions, past failures and successes, and established evaluation scripts. Without direct access to this context, an AI agent runs the risk of referencing incorrect datasets, choosing improper evaluations, and generating outputs that appear superficially plausible while remaining fundamentally flawed. To address this gap, Databricks integrated Genie Code with Genie Ontology. By grounding the agent in the platform's ontology, Genie Code learns exactly how an internal machine learning team operates, how models are structured, and how performance is validated.

```mermaid flowchart TD accTitle: Genie Code Grounded in Genie Ontology accDescr: Diagram illustrating how Genie Ontology feeds institutional context into Genie Code to handle feature engineering, training pipelines, and production evaluations.

subgraph ContextSource["Genie Ontology"] A["Team Workflows & Data Lineage"] B["Historical MLflow Experiments"] C["Existing Evaluation Scripts"] end

subgraph AgentCore["Genie Code Agent"] D["Contextual Feature Engineering"] E["Pipeline & Script Reuse"] F["Standardized Model Evaluation"] end

subgraph Outcome["Production Delivery"] G["Senior MLE-Level Outputs"] H["Accelerated Deployment"] end

A --> D B --> E C --> F D --> G E --> G F --> G G --> H ```

This integration enables Genie Code to engineer features adhering strictly to the patterns already used by the engineering team, reuse verified training scripts, and seamlessly plug into existing pipelines. It also respects and runs established evaluation suites so engineers can verify that a newly trained model meets the organization's existing production bar. Highlighting this paradigm shift, the speaker notes:

> “And that allows Genie code to really act like a senior machine learning engineer on your team.”

The operational impact of this architecture was demonstrated with Danfoss, a multinational engineering firm, which went from having no machine learning in place to deploying a complete production machine learning system in just 90 minutes—a milestone that typically requires weeks, months, or quarters across enterprise environments.

- **Feature Engineering Alignment:** Builds new model features using the exact coding patterns and workflows already maintained by the user's team. - **Evaluation Reuse:** Automatically discovers and applies existing evaluation scripts rather than inventing synthetic metrics, ensuring benchmark integrity. - **Danfoss Case Study:** Transitioned from zero machine learning to a fully functioning production ML deployment in 90 minutes.

---

[01:42:43](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6163s) Announcing Genie ZeroOps for ML

While speeding up model creation represents a massive breakthrough, building models accounts for only a fraction of day-to-day work; real-world machine learning teams spend 60% to 80% of their operational time maintaining production applications. Accelerating model creation by 100x risks creating 100x the operational burden, which no human engineering organization can scale to meet. Models in production face endless vectors of failure, including data drift, inference server latency spikes, broken upstream pipelines, and a long tail of unpredictable edge cases.

To eliminate this bottleneck, Databricks announced Genie ZeroOps for ML, an autonomous operational agent described by the speaker as “the biggest launch in machine learning in the past 10 years.” ZeroOps for ML fundamentally rethinks traditional observability. Standard monitoring setups observe metrics, identify an anomaly, and fire an alert demanding human intervention. In contrast, Genie ZeroOps for ML detects the issue, explores hypotheses, diagnoses the root cause, and synthesizes a verified fix before contacting the engineer.

> “And the flip here is that for the whole history of machine learning, we've always had a team member on call for our models. And zero ops for ML inverts that and allows an agent to be on call and your team is just in a position to approve the recommendations from that agent.”

- **The Maintenance Bottleneck:** ML engineering teams routinely spend 60% to 80% of their bandwidth managing, debugging, and maintaining active production deployments. - **Problem-to-Solution Shift:** Replaces basic metric threshold alerts with autonomous root-cause analysis and executable candidate fixes. - **Inverted On-Call Paradigm:** An autonomous agent serves as the primary tier on-call responder, while human engineers remain in the loop solely to evaluate and approve production deployments.

---

[01:45:22](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6322s) Demo: Genie ZeroOps for ML in Action

Amber takes the stage to demonstrate Genie ZeroOps for ML on a live production workload at the fictitious organization Bricksports Global. The team maintains a critical production model forecasting ticket sales for every match in the Data and AI World Cup. Because ticket sales forecasts directly guide tens of millions of dollars in event staffing and venue operations spend, forecasting errors risk heavy financial losses or empty seats due to ticket overpricing.

Amber begins by prompting Genie Code to set up an alert on the production endpoint. The agent provisions and registers the rule automatically, configured to trigger whenever observed ticket sales deviate outside a ±5% margin of forecasted demand. Moving the timeline forward to a scenario at 5:30 p.m. on a Friday, an email notification shows the alert fired 30 minutes earlier due to a sudden drop in forecast accuracy.

```mermaid flowchart LR accTitle: Genie ZeroOps Incident Remediation Workflow accDescr: Flowchart tracing an incident from alert trigger through automated hypothesis testing, feature creation, retraining, and one-click deployment.

subgraph Detection["Detection & Triage"] A["±5% Forecast Alert Fires"] B["ZeroOps Inbox Thread Created"] end

subgraph Investigation["Autonomous Diagnosis"] C["Query Unity Catalog & Run SQL"] D["Inspect Feature Values & Lineage"] E["Root Cause: Missing Knockout Stage"] end

subgraph Remediation["Autonomous Fix"] F["Engineer Tournament Stage Features"] G["Retrain & MLflow Model Registry"] H["Human Approval & 1-Click Rollout"] end

A --> B B --> C C --> D D --> E E --> F F --> G G --> H ```

Rather than requiring manual troubleshooting during off-hours, the alert email includes a button linking directly to the investigation in the Genie ZeroOps inbox. Over the 30 minutes between the alert and the engineer checking the notification, the system worked through dozens of hypotheses completely in the background:

> “Work that would have taken me hours to do manually, Genie Code root caused it before I even realized there was an alert.”

Genie Code queried Unity Catalog tables, executed diagnostic SQL queries, checked feature distributions, and evaluated MLflow experiment runs. It isolated the exact failure: the model lacked a feature capturing the tournament stage, leaving it unable to anticipate the surge in fan interest and demand accompanying the knockout rounds.

After identifying the root cause, Genie Code located unutilized matchup and tournament stage metadata, constructed the missing features, retrained the model with hyperparameter tuning, and logged the run to the MLflow Model Registry using Databricks best practices. Leveraging end-to-end data lineage, it validated the candidate model against the original evaluation benchmark, verified that forecast accuracy returned to target thresholds, and presented the engineer with a single-click deployment interface to route a canary slice of live traffic to the patched model.

- **Alert Criteria:** Configured to fire if actual ticket sales diverge beyond ±5% from forecasted values. - **Autonomous Root-Cause Analysis:** Inspected training notebooks, queried Unity Catalog tables, monitored drift across feature distributions, and executed MLflow runs without human direction. - **Tournament Stage Drift:** Identified that the model failed to incorporate tournament phase features, causing predictions to falter as soon as the tournament reached knockout matches. - **Automated Remediation:** Feature engineered the tournament stage and matchup attributes, executed hyperparameter sweeps, logged artifacts into MLflow, verified accuracy against historical eval benchmarks, and queued the model for one-click deployment.

[01:50:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6600s) Genie ZeroOps Resolves Stale Models and Agentic ML Engineering Arrives

Wrapping up the live demonstration, the presenter points out the newly deployed Version 2 of the model. Under normal circumstances, diagnosing and fixing a stale production model is a major, cross-functional incident that drags on for hours or even days, racking up potential business losses in the tens of millions of dollars. Genie ZeroOps for ML triaged and resolved the failure swiftly with minimal disruption to the team. As machine learning enters what Mike dubbed the "100x era," organizations face an explosion in model volume—and with a hundred times more models comes a hundred times the operational complexity. The core mission is to shield engineering teams from that mounting complexity, automating operational toil so practitioners can return to building breakthrough AI.

```mermaid flowchart LR accTitle: Genie ZeroOps Incident Remediation accDescr: The automated cycle of model drift detection, remediation via Genie ZeroOps, and zero-downtime deployment. subgraph S1["Detection"] A["Stale Production Model"] --> B["Alert & Root-Cause Triage"] end subgraph S2["Remediation"] B --> C["Genie ZeroOps for ML"] C --> D["Candidate Model v2"] end subgraph S3["Production"] D --> E["Zero-Downtime Rollout"] end ```

The speaker recaps the three foundational pillars showcased during the ML portion of the keynote:

- **AI Runtime:** Serverless GPU compute for training and fine-tuning deep learning models and large language models (now in public preview). - **Genie Code for Machine Learning:** An embedded agentic assistant integrated directly into local developer environments (now generally available). - **Genie ZeroOps for Machine Learning:** An autonomous operations agent designed to monitor, maintain, and self-heal production ML systems (slated for release in Summer 2026).

Reflecting on the historical vision of the discipline, the presenter emphasizes that the industry has spent decades trying to automate machine learning. With serverless GPU infrastructure paired with agent loops, ML engineering can finally execute a hundred times faster across a hundred times more models, driving granular personalization into every product interaction and optimizing business decisions at scale. The speaker leaves the audience with a closing charge:

> “Let your agents do the work now, so your team can focus on what's next.”

---

[01:52:48](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6768s) The Agentic Shift in Buyer Behavior and the Collapse of Waterfall Marketing

The keynote shifts from internal machine learning infrastructure to customer-facing applications. The host introduces Tassos Argyros, General Manager for CustomerLake. Argyros shares a personal retrospective: having spent ten years building a database company and another decade building a marketing technology platform to bridge raw data with intuitive business interfaces, leading CustomerLake represents a natural synthesis of those two decades. He grounds his talk in a central reality of modern business: roughly 80% of companies report that they compete primarily on the basis of customer experience.

```mermaid flowchart TD accTitle: Waterfall Campaigns versus Agentic Buying Reality accDescr: Contrast between slow traditional marketing workflows and high-speed multi-channel agentic shopping. subgraph Traditional["Traditional Waterfall Campaign"] M1["Marketing Strategy"] --> D1["Data Eng (Customer 360)"] D1 --> O1["Marketing Ops (Audiences)"] O1 --> C1["Creative Content"] C1 --> A1["Activation & Review (Weeks)"] end subgraph Agentic["Agentic Era Reality"] B1["Consumer Delegate Agent"] -->|"APIs / Seconds"| CH["Multiple Headless Channels"] CH -->|"Instant Decisions"| BUY["Automated Checkout"] end ```

However, the mechanics of customer experience are facing structural disruption because AI is fundamentally altering how consumers buy:

- **Autonomous Agent Delegation:** Consumers no longer evaluate purchases in isolation. When Argyros asks the room how many people consult an LLM before buying goods, roughly 80% raise their hands. Chatbot advice is rapidly transforming into autonomous delegation, where personal agents conduct research, price discovery, shipping comparisons, and checkout. - **Compressed Decision Cycles:** Traditional buyer journeys spanning days or weeks are collapsing into transactions completed in seconds or minutes. - **Headless, Multi-Channel Commerce:** Because shopping has become API-driven, shopping agents execute concurrent inquiries across dozens of channels simultaneously.

Argyros contrasts this high-velocity reality with traditional marketing practices. Historically, marketing departments ran waterfall campaigns: marketing set strategy, data engineering aggregated static Customer 360 datasets, marketing ops carved out broad segments of millions of people, creative teams generated static copy, and marketers activated campaigns over weeks-long rollouts. While adequate for slow-moving human consumers, waterfall workflows are unsuited to agentic buyers operating across APIs in real time. As Argyros explains:

> “When agents buy on multiple channels at the same time, in seconds, you can't spend weeks designing a single campaign.”

---

[01:57:12](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7032s) Introducing Infinity Campaigns and the Architecture of the Agentic CDP

To match the speed of autonomous consumer agents, businesses must invert their operational model: instead of blasting campaigns to audiences of millions, organizations need an agent for every customer to serve audiences of one. Argyros introduces **Infinity Campaigns**, an architecture designed to deliver real-time, continuous personalization.

```mermaid flowchart LR accTitle: Infinity Campaign Closed Loop accDescr: Continuous signal processing loop between an individual customer and their dedicated campaign agent. subgraph Loop["Continuous Engagement Engine"] A["Individual Customer"] -->|"Behavioral Signals"| B["Dedicated Customer Agent"] B -->|"Determines Next Best Action"| C["Dynamic Content Generation"] C -->|"Personalized Interaction"| A end ```

The name "Infinity Campaigns" reflects three core design principles rather than a mere visual resemblance to the infinity symbol:

1. **Evergreen and Self-Adapting:** These campaigns eliminate fixed start and end dates. Because autonomous agents power the loop, the campaign continuously adapts to shifting customer signals and evolving business objectives without requiring manual teardowns and redesigns. 2. **Infinite Content Permutations:** Instead of choosing from a handful of pre-rendered creative assets, agents draw from an essentially limitless pool of tailored copy, recommendations, and pricing variations tailored to an individual customer's precise context. 3. **Sub-Second Response Latency:** Unbound by human processing bottlenecks, agents respond instantaneously across any channel the customer (or the customer's buying agent) queries.

Throughout this transformation, human oversight remains intact. The marketer functions as the “chief strategist, the chief orchestrator, and the chief approver,” defining guardrails, priorities, and rules while delegating execution to an army of agents.

To ground the concept in enterprise architecture, Argyros turns to Customer Data Platforms (CDPs). Historically, a CDP sits as a business application between raw enterprise storage and execution layers (such as email tools, push notification engines, and ad platforms). Its traditional mandate is two-fold: ingesting raw transactional logs to build organized Customer 360 tables, and providing business users with a graphical user interface to slice audiences and configure journey steps.

Argyros highlights the central tension that Infinity Campaigns aim to resolve:

> “The marketer is the chief strategist, the chief orchestrator, and the chief approver of what the agents are doing. But now they have an army of agents that can do all the one-to-one personalization work that humans were just too slow to do before.”

[02:00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7200s) The Rise of the Agentic CDP and CustomerLake

Traditional Customer Data Platforms (CDPs) were designed to send audience and customer records downstream into operational marketing tools such as email platforms and advertising networks—a process known as reverse ETL. However, legacy CDPs operated as middleware completely detached from the underlying data warehouse or data lake. In the agentic era, building effective autonomous workflows requires more than raw data pipelines; agents demand deep business context situated directly where the data and compute reside. To unlock true "infinity campaigns," organizations require an entirely new class of platform: the agentic CDP.

An agentic CDP must fulfill three fundamental architectural criteria: 1. **Embedded in the Lakehouse:** It must live directly inside the data foundation where data, enterprise context, and autonomous agents are co-located. 2. **LLM-Core Architecture:** Large language models cannot be treated as superficial copilot sidecars; agents must form the core engine driving data transformation, reasoning, and operational execution. 3. **Power Infinity Campaigns:** It must support evergreen, real-time, continuously adapting campaigns capable of deploying individual agents to deliver one-to-one personalization for every single customer.

To lead this new category, Databricks announced CustomerLake, an agentic CDP built natively on the Databricks Data Intelligence Platform. Because CustomerLake is embedded directly in the lakehouse, customer data never leaves the security boundary, and all data and agent governance established in Unity Catalog carries over automatically. Designed with an autonomous engine yet tailored for business users through a democratized, business-first user interface, CustomerLake is built to power personalized customer experiences "a billion times a day." The platform is split into two primary operational modules: Profile Agents, which automate the raw-data-to-Customer-360 pipeline, and Campaign Agents, which empower marketing teams to execute Infinity Campaigns.

As the speaker highlighted regarding this shift: > “Now, in order to make agents work, you need data, but you also need context, and you need data and context to be where the agents live.”

```mermaid flowchart TD accTitle: CustomerLake Architecture accDescr: Architecture of CustomerLake showing Profile Agents transforming raw data into unified profiles, which Campaign Agents use to power Infinity Campaigns and reverse ETL.

subgraph DataPrep ["Data Ingestion & Context"] A["LakeFlow Connect & Lakehouse Data"] B["Databricks Marketplace Hydration"] C["Databricks Clean Rooms (3rd-Party)"] end

subgraph ProfileEngine ["Profile Agents"] D["Semantic Tagging & Cleaning"] E["Agentic Identity Resolution (Rules + LLM)"] F["Schema Mapping & Survivorship"] end

subgraph CoreLakehouse ["Unity Catalog & Lakehouse Storage"] G[("Governed Customer 360")] end

subgraph CampaignEngine ["Campaign Agents"] H["Genie-Powered Audiencing"] I["Infinity Campaigns (1:1 Agent Personalization)"] J["Native Managed Reverse ETL Connectors"] end

A --> D B --> D C --> E D --> E E --> F F --> G G --> H H --> I I --> J ```

---

[02:03:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7380s) Profile Agents and Agentic Identity Resolution

Profile Agents automate the intricate engineering journey from disparate raw datasets to a business-ready Customer 360. Triggered by natural language prompts outlining specific business use cases, these agents orchestrate an autonomous ingestion, transformation, enrichment, identity resolution, and evaluation loop. Data preparation begins using LakeFlow Connect to bring in third-party application sources, augment existing lakehouse tables, or hydrate profiles with third-party datasets from the Databricks Marketplace. Profile Agents automatically tag data semantically and resolve data hygiene flaws, including standardizing malformed email addresses, physical locations, and phone numbers.

At the center of Profile Agents is agentic identity resolution for first-party data—matching records belonging to the same individual across disparate datasets that lack common primary keys. Where traditional methods relied strictly on brittle static SQL rules or opaque, slow-to-train machine learning models, agentic identity resolution establishes a self-improving loop: - Agents first generate and test deterministic matching rules. - When ambiguous edge cases cause rule failure or inconclusive matches, reasoning LLMs step in to make probabilistic determinations. - If uncertainty remains, the system routes the record to human reviewers for feedback. - Every manual adjudication and confirmed match feeds back into the loop, allowing the agents to continually refine their underlying heuristics and LLM prompts.

For third-party data enrichment, CustomerLake introduces privacy-first external identity resolution powered by Databricks Clean Rooms, enabling cross-partner customer matching without exposing raw data outside the lakehouse. Finally, Profile Agents handle survivorship logic, schema mapping, and continuous data evaluation, verifying that final Customer 360 data products meet quality thresholds before downstream business consumption.

The speaker summarized the superiority of this hybrid resolution system: > “Historically, this has happened through static SQL rules or machine learning. Both approaches have significant limitations, and we think the agentic identity resolution wins on all fronts over these traditional approaches.”

---

[02:06:15](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7575s) Campaign Agents, Infinity Campaigns, and Managed Reverse ETL

Campaign Agents serve as the marketer-facing interface of CustomerLake, automating end-to-end campaign operations. Starting with a high-level natural language prompt, Campaign Agents review business goals and context to draft a complete campaign brief. Once human marketers inspect and approve the strategic brief, autonomous agents are deployed to handle hyper-personalized content generation and execution.

Campaign Agents also reinvent foundational CDP tasks like audiencing. Marketers can explore, segment, and refine audiences conversationally using conversational prompts powered by Genie, or work through a visual UI tailored for marketing operations teams. Crucially, any audience segment can be converted into an operational campaign with a single click.

At the center of this workflow are Infinity Campaigns. Because Infinity Campaigns run on LLMs rather than conventional predictive scoring models, teams do not have to wait months for data scientists to train, validate, and deploy complex machine learning pipelines or decision engines. Personalization starts out of the box on day one, complete with simulation environments, guardrails, and integrated measurement. To close the operational loop, Databricks announced built-in, native managed reverse ETL connectors within CustomerLake: - The platform launches with approximately 20 out-of-the-box managed connectors spanning leading advertising networks, marketing automation suites, and customer experience platforms. - Over a dozen additional native connectors are scheduled for release by the end of the year, with ongoing expansion planned thereafter.

As the speaker emphasized regarding the immediate time-to-value of this architecture: > “The beauty of Infinity Campaigns, because it's built around LLMs, is that it can start personalizing immediately.”

---

[02:08:53](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7733s) Demo: CustomerLake Campaign Agents in Action

The presentation transitioned to a live demonstration led by Databricks presenter Justin, who joked as he walked back onto the stage, “Encore demo. I'm flattered.” Justin outlined the scenario: assuming Profile Agents have already unified, cleansed, and resolved identities into a cohesive Customer 360 lakehouse table, his demonstration focused on showing how a non-technical marketing lead can use Campaign Agents and Genie to build segments and launch 1-to-1 Infinity Campaigns.

Assuming the persona of a marketing manager for an athletic apparel retailer named "Bricksport," Justin brought up the unified CustomerLake campaigns dashboard. The overview screen provided a high-level command center displaying active marketing campaigns, key monitored audience segments, campaign metrics, and the status of active reverse ETL sync pipelines pushing data downstream. Justin navigated into the Audiences tab, displaying existing customer segments alongside their current population counts and percentage growth over time, before initiating the creation workflow for a brand-new customer audience.

[02:10:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=7800s) Demo: Generating Audiences with Genie and Running Infinity Campaigns

Justin steps through a live demonstration of CustomerLake's campaign features, replacing rigid, manual filtering workflows with natural-language prompt execution inside Genie. He inputs the following target criteria:

> “Include our most valuable Team USA fans, who have spent more than 500 on merch this season. and exclude anyone who already bought the USA Fan Kit.”

Genie scans the existing catalog of attributes and audience segments, returning a qualified audience of 800,000 customers. The criteria remain entirely transparent and editable, allowing marketers to tweak thresholds directly. Justin inspects the underlying query generating the segment:

> “If you've ever supported marketing by pulling lists and writing custom campaigns, you can get really complex sequel to power some of these campaigns. And we got this from one prompt with Genie.”

```mermaid flowchart LR accTitle: Genie Audience to Infinity Campaign Workflow accDescr: Natural language prompt transforms into SQL, builds an audience segment, and powers 1-to-1 Infinity Campaign agent decisions.

subgraph Audience["Audience Generation"] A["Genie Natural Language Prompt"] --> B["Automated Complex SQL"] B --> C["800,000 Customer Audience"] C --> D["Dynamic Insights Dashboard"] end

subgraph Campaign["Infinity Campaign Execution"] D --> E["Infinity Campaign Engine"] E --> F{"Agent Reasoning per User"} F -->|"Marcus"| G["Defer: Frequency cap / timing"] F -->|"High-intent fan"| H["Matchday Offer: 15% off seat upgrade"] F -->|"General attendee"| I["Matchday Guide: Kickoff & parking"] end ```

After accepting the audience, CustomerLake produces an automated, dynamic insights dashboard dedicated to that specific cohort. Moving from audience building to continuous activation, Justin showcases an active **Infinity Campaign** set up for an "in-venue fan experience." The configuration interface sets the operational bounds for the autonomous system: - **Campaign Guardrails & Objectives**: Explicit conversion goals, metric trackers, frequency capping rules, and messaging catalogs (e.g., a matchday guide vs. a matchday promo offer). - **One-to-One Agent Reasoning**: In the Agents tab, the platform reveals the agent's individualized decisions profile by profile. For a fan named Marcus, the agent decides to "defer" sending anything. Drilling into the decision trail exposes the complete chain of reasoning: timing, real-time incoming signals, and frequency cap guardrails that prevented spamming. - **Dynamic Content Personalization**: For another fan, Genie triggers a personalized offer featuring 15% off a seat upgrade. For an attendee needing logistical support, it dispatches a matchday guide containing kickoff times and parking information instead. - **Aggregate Reporting**: Justin inspects total opens, clicks, and conversion rates directly within the campaign console before handing the stage back.

---

[02:14:14](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8054s) The Agentic CDP Era and CustomerLake Launch

Returning to the stage, the presenter highlights that true, continuous 1-to-1 personalization at scale is no longer theoretical:

> “The area of true one-to-one personalization It's here. It's very, very, very exciting.”

CustomerLake was built alongside forward-leaning enterprise design partners and an ecosystem spanning data providers, content engines, ad-tech platforms, activation pipelines, and systems integrators. The presenter formally outlines the rollout details: - **Availability**: CustomerLake is officially launched in **Private Preview**, accessible by reaching out directly to Databricks account teams. - **Summit Sessions**: A dedicated **CDP breakout track** and a special **Marketing Forum at 1:00 PM** are opened across the event schedule, featuring presentations by the Databricks engineering and product leads behind CustomerLake, alongside initial enterprise adopters. - **Strategic Vision**: Summarizing the milestone, the speaker notes how special it is to unite business context, data architecture, and autonomous agents under one roof “to build something that provides the perfect customer experience a billion times a day.”

---

[02:16:25](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8185s) AI-Driven Cyber Threats and the Launch of Lakewatch

Andrew Tu, General Manager of Lakewatch, takes the stage to introduce Databricks' security lakehouse strategy. Originally previewed two months prior at the RSA Conference, the initiative was significantly accelerated by Databricks' acquisition of Panther Labs, announced the previous day by CEO Ali Ghodsi.

Tu warns that modern LLMs have fundamentally shifted the economics and capabilities of offensive cyber operations: - **Instant Vulnerability Discovery**: Capabilities previously restricted to nation-state threat actors—requiring dedicated teams of reverse engineers hunting zero-days over months or years—can now be generated immediately by anyone utilizing advanced LLMs across open-source and proprietary software alike. - **Automated End-to-End Exploitation**: As detailed in recent research from Anthropic regarding attacks directed at the Mexican government, malicious actors used Claude to run automated kill-chains. The attackers prompted an agent to discover open server vulnerabilities, instruct the agent to exploit them, use vulnerabilities inside a crontab configuration to escalate to root access, and exfiltrate sensitive files—ultimately stealing 200 million personnel records and gigabytes of classified data.

Tu argues that the only viable defensive posture is deep architectural alignment between security operations and enterprise data engineering:

> “And now just about anyone with access to an LLM can put on an attack at a scale and speed that was previously unimaginable. That's the world we live in. And the only way that we're going to be successful in defending against this is if security and data teams work together.”

Tu breaks down the traditional Security Operations Center (SOC) model into three distinct operational pillars: 1. **Telemetry Ingestion**: Aggregating massive streams of machine data, endpoint metrics, server traces, and identity/access management logs from laptops and cloud infrastructure into central servers. 2. **Detection Engineering**: Crafting detection rules and SQL queries to identify suspicious behavior—such as impossible travel anomalies where an account logs in from San Francisco and then minutes later from the opposite side of the world. 3. **Incident Response (IR)**: Pushing flagged anomalies downstream to analysts, who must triage alerts and determine whether an event represents routine developer debugging or a catastrophic compromise.

[02:20:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8400s) The Failure of Legacy SIEMs Against Agentic Threats

The speaker explains how traditional security operations center (SOC) teams work: analysts investigate incoming alerts one by one, determining whether an anomaly is a genuine breach or merely a false positive. When a threat is verified, responders take targeted mitigation actions such as blocking an offending IP address or triggering a mandatory password reset. However, the legacy tooling underpinning this workflow—specifically traditional Security Information and Event Management (SIEM) systems—is fundamentally broken for the modern threat landscape. As the speaker emphasizes:

> “The challenge is the tools they're using are outdated. They typically use a system called a SIM, a security information event management tool, and these systems don't -- they're not designed for the scale of threat that we're facing today.”

Traditional SIEMs suffer from severe ingestion constraints driven by an unforgiving combination of technical bottlenecks and cost structures. To cope with steep licensing fees and storage limits, security teams are forced into operational compromises: - **Aggressive Data Pruning:** Organizations discard event data older than 30 days because retaining historical telemetry for months or years is cost-prohibitive. - **Upstream Data Filtering:** Pipelines drop high-volume telemetry deemed lower priority, risking critical blind spots in post-incident forensics. - **Exclusion of Contextual Data:** Modern rich telemetry—such as large language model (LLM) execution logs, source code changes, Jira tickets, and Slack messages—is omitted entirely, even though it provides crucial context (for example, proving whether an engineer was legitimately running a VPN to troubleshoot a service or experiencing an active machine takeover).

Compounding the data challenge is an unsustainable manual burden. A typical large enterprise managed roughly 7,000 alerts per week in 2020; today, that figure exceeds 30,000 alerts weekly. With a single investigation taking an analyst an average of 30 minutes—tracing pre-incident behavior, historical login origins, and downstream actions—an organization would require an impractical team of over 400 analysts just to keep pace. As automated, agent-driven attacks accelerate in volume and velocity, human-only defense models fail completely.

> “So the reality is that with the tools we have today, we don't stand a chance in meeting these agentic threats. We're dealing with incomplete data because we can't afford and we can't scale to bring it in. We're dealing with humans fighting agent attackers. And we're dealing with proprietary data silos where you can't analyze all the data because it's locked up in different locations.”

---

[02:22:29](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8549s) Lakewatch: The Agentic Security Lakehouse Architecture

To solve these architectural bottlenecks, Databricks built **Lakewatch**, an agentic security lakehouse designed natively on the Databricks platform. Built around open table formats, Lakewatch eliminates proprietary data lock-in and dramatically reduces storage costs, allowing enterprises to ingest and retain all structured, semi-structured, and unstructured telemetry without arbitrary retention cutoffs. Unity Catalog provides enterprise-wide data governance and auditability, while Agent Bricks and Genie inject autonomous AI agents across the entire security operational lifecycle:

- **Ingestion & Normalization:** Autonomous agents apply Extract, Transform, Load (ETL) pipelines and normalize diverse schemas into standard formats. If an upstream log format changes, the agents automatically adapt the pipeline to maintain uninterrupted data flow. - **Rule Authoring & Detection:** Agents assist security engineers in researching emerging attack vectors and authoring complex SQL-based detection logic. - **Investigation & Remediation:** Incoming alerts trigger agents that construct complete forensic case dossiers, allowing human analysts to simply review findings and approve proposed remediation actions.

```mermaid flowchart TD accTitle: Lakewatch Agentic Security Lakehouse Architecture accDescr: Telemetry flows through ingestion into Unity Catalog and Open Formats, where Genie and Agent Bricks orchestrate automated ETL, detection, and investigation.

subgraph Ingest["1. Multimodal Ingest"] A["Security Telemetry<br/>(Cloud, Endpoint, Network)"] B["Contextual Data<br/>(LLM Logs, Code, Jira, Slack)"] end

subgraph Platform["2. Databricks Platform Core"] C["Open Data Formats (Delta Lake)"] D["Unity Catalog<br/>(Central Governance & Access)"] C --- D end

subgraph Agents["3. Agentic Layer (Genie & Agent Bricks)"] E["Auto-ETL & OCSF Normalization"] F["Automated Detection Engineering"] G["Background Case Investigation"] end

subgraph Operations["4. Security Analyst Workflow"] H["Dedicated Security UI & Pre-built Incident Cases"] end

A --> C B --> C C --> E E --> F F --> G G --> H ```

---

[02:24:43](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=8683s) Demo: Automated OCSF Normalization and Genie Rule Authoring

Michael Anderson, Tech Lead for Lakewatch, takes the stage to demonstrate the live product, which integrates directly into the Databricks unified app switcher:

> “And Lakewatch is our agentic security lake house, and it's designed for the entire Sim journey. And that begins with data.”

Lakewatch ships with dozens of security-specific connectors supporting multimodal and heterogeneous data streams. Following its initial preview at the RSA Conference, Lakewatch introduced an automated normalization pipeline powered by Genie that maps incoming logs into the Open Cybersecurity Schema Framework (OCSF). By landing normalized data directly into curated gold tables, SOC teams can write a detection once and run it across disparate logs from multiple cloud providers and third-party vendors.

Michael shows how Lakewatch scales detection management across tens of thousands of simultaneous rules with native MITRE ATT&CK tagging and coverage heatmaps. Rather than spending days or weeks researching emerging threat actor tactics, security engineers can prompt Genie to generate production-grade detection rules in minutes.

Entering the prompt *“Please research and write me a detection for Storm-2460”* (noting with a laugh as the prompt processes, “This is how you know it's live”), Michael highlights the coordinated multi-agent workflow: - **Background Threat Intelligence Research:** An autonomous research agent crawls web databases, tracking CVE disclosures, threat blogs, and attack path reports for the complex multistage malware Storm-2460. - **Pipeline Introspection & Sampling:** Concurrently, Genie inspects the environment's telemetry pipelines, discovering connected CrowdStrike sensor feeds transformed into the OCSF `process_activity` gold table. Genie samples active records to confirm prerequisite fields exist (identifying certutil/search utility command patterns without active malicious indicators). - **SQL Synthesis and False-Positive Validation:** Combining web research with internal telemetry schema, Genie drafts a detection SQL query covering all seven stages of the malware kill chain. It validates the query against the past 24 hours of data and runs historical backtests over the previous 30 days to ensure zero false positives. - **Turnkey Rule Generation:** Genie packages the validated SQL into a production rule pre-tagged with relevant MITRE ATT&CK tactic/technique metadata, compressing a week-long engineering cycle into roughly two minutes. - **Agentic Case Summarization:** Turning to the Lakewatch incident cases console, Michael demonstrates that cases marked with a star icon have been automatically investigated in the background by Genie (such as analyzing an anomalous rate-limiting surge alert generated by an enterprise Web Application Firewall), delivering pre-compiled forensic summaries to triage analysts.

[02:30:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9000s) Demo: Autonomous Threat Investigation with Genie and Lakewatch

Continuing his live demonstration of Lakewatch, Michael reviews an auto-generated incident report. Genie does not simply summarize an event; it actively hunts down related cases, matching attack patterns against the web application firewall (WAF) and tracking activities originating from the same IP address. It cross-references external threat intelligence feeds to check for known malicious reputations and inspects internal perimeter logs to determine whether the suspicious traffic breached defenses beyond the perimeter. Furthermore, Genie continuously learns from how human analysts handled prior investigations, automatically running updated analysis routines so that human staff can bypass repetitive runbooks and focus exclusively on novel aspects and threat mitigation.

To illustrate how this works on signals that usually get ignored, Michael navigates to an uninvestigated informational alert labeled "suspicious API request." In a typical security operations center (SOC) flooded with tens of thousands of alerts weekly, low-fidelity statistical deviations from baseline traffic are routinely discarded due to lack of bandwidth. Michael triggers Genie to investigate the case live. Genie immediately inspects the raw event details and identifies an incoming POST request where a parameter named `readContext` deviates sharply from established traffic baselines.

```mermaid flowchart TD accTitle: Genie Autonomous Threat Investigation Workflow accDescr: Sequence of analytical steps taken by Genie to triage a low-fidelity alert into a critical security finding. subgraph Triage["Initial Triage"] A["Informational Alert: POST request"] --> B["Detect anomaly in readContext parameter"] B --> C["Query external threat intelligence feed"] end subgraph DeepDive["Code & Context Analysis"] B --> D["Inspect endpoint source code"] D --> E["Identify hardcoded backdoor logic"] E --> F["Check commit history & author details"] end subgraph Resolution["Report & Escalation"] C --> G["IP enriched: Google Cloud origin"] F --> H["Scan Jira tickets: zero authorization"] G --> I["Escalate to high severity true positive"] H --> I end ```

Working concurrently, Genie queries threat feeds to check whether the origin IP is part of a known botnet (finding no external hits), identifies the microservice hosting the endpoint, and scans the application source code. Within seconds, Genie spots a backdoor: when the `readContext` parameter matches a specific string, the code completely bypasses all access control checks and directly returns the decryption key. Genie then broadens its probe autonomously, pinpointing the exact commit date (which matches the date of the anomalous request), locating a duplicate instance of the backdoor elsewhere in the repository, checking the committer's manager, and querying Jira for associated tickets. Finding no Jira tickets or legitimate work authorization for the change, Genie assembles a comprehensive incident brief noting that the request originated from Google Cloud infrastructure and recommending an immediate escalation to high severity. Reflecting on the speed and thoroughness of the automated triage, Michael emphasizes the paradigm shift:

> “This is the first time in history that my capacity to investigate cases and my capacity to be thorough in investigating every single thing in my infrastructure is no longer bounded or limited by the size of my SOC team.”

- Genie analyzes low-fidelity signals that human SOC teams lack the bandwidth to investigate manually. - The automated code inspection identified a hardcoded backdoor that bypasses access control and exposes decryption keys. - Investigation history, commit records, and project management tickets (Jira) were correlated without manual analyst intervention.

---

[02:33:15](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9195s) Open Security Lakehouse: Breaking Silos and Fighting Agents with Agents

Taking the stage after the demo, Andrew underscores that Lakewatch allows organizations to analyze all their enterprise telemetry without being constrained by traditional ingestion limits or runaway licensing costs. Because modern cyberattacks increasingly deploy autonomous agents to probe attack surfaces, defensive postures must counter with equally sophisticated defensive agents. Lakewatch achieves this by breaking out of proprietary vendor silos, storing security data in open formats within an open ecosystem that accepts feeds from any source.

Although Lakewatch launched only a few months prior, enterprise adoption has ramped quickly, as showcased at the previous day's Cyber Forum featuring organizations such as Atlassian, National Australia Bank (NAB), and Adobe. Andrew highlights Adobe's deployment, where the security team faced petabytes of telemetry that overwhelmed their legacy SIEM infrastructure. By ingesting those petabytes into an open security lakehouse on Databricks, Adobe applied machine learning and agentic workflows to uncover sophisticated threats that were previously impossible to detect. Andrew urges technical leaders across disciplines to align their efforts:

> “To sum it up, the only way that we can meet today's security threats is by having security and data teams work together.”

- Lakewatch eliminates data silos by keeping enterprise telemetry in open lakehouse storage formats. - Enterprises deploy AI defense agents to counter automated, agent-driven attacks. - Adobe migrated petabytes of telemetry from legacy systems to Databricks to run advanced AI/ML threat detection.

---

[02:35:45](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9345s) Databricks x Panther: Rebuilding the AI SOC from First Principles

Reynold Xin takes the stage to discuss Databricks' agreement to acquire cybersecurity company Panther, welcoming Panther's founder and CEO, Jack Naglieri. Recognizing that many in the audience come from data engineering and analytics backgrounds rather than cybersecurity, Reynold invites Jack to outline his history. Jack explains that he spent roughly eight years in cyber operations, eventually leading detection and response engineering at Airbnb. At Airbnb, standard commercial SIEM products proved too slow, expensive, and operationally burdensome to handle modern cloud-scale workloads. His team built and open-sourced an internal detection platform, which ultimately served as the founding inspiration for Panther. Today, technology companies including Coinbase, Docker, and Block rely on Panther to outpace attackers.

Reynold highlights an unusual decision in Jack's founder journey: stepping down from day-to-day operational CEO responsibilities a few years earlier to spend 100% of his time writing code and hacking on early LLMs, before later returning to the CEO seat. Jack explains that as large language models advanced, he realized AI could not simply be bolted onto manual legacy triage workflows; the SOC needed to be redesigned from first principles around autonomous agents. Starting with LLM-generated detection rules, his research quickly advanced to full end-to-end alert triage, removing the human bandwidth bottleneck that has constrained security teams for decades. Addressing the broader conference theme of whether security teams will eventually operate like data teams, Jack offers a nuanced perspective: security personnel are incident response specialists who must understand attack mechanics, not data infrastructure plumbing. The goal of combining Panther's detection application layer with the Databricks data lakehouse is to deliver cloud-scale data processing and AI agents to security operators without burdening them with pipeline maintenance.

> “And during that time, we found that the typical off-the-shelf SIEMs just didn't really work for our use cases.”

> “So agents really allowed us to break that fundamental scaling problem in the SOC.”

- Jack Naglieri founded Panther after developing custom, open-source security data tooling to replace legacy SIEMs at Airbnb. - Key enterprise customers running on Panther include Coinbase, Docker, and Block. - Jack temporarily stepped away from CEO duties to prototype first-principles agentic workflows for end-to-end alert triage. - The Databricks and Panther combination pairs enterprise data infrastructure with specialized security detection, response, and compliance applications.

[02:40:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9600s) Transition and Recap: Laying the Agentic Data Foundation

Wrapping up the discussion on security and Panther, Jack Naglieri highlights the immense value of giving security agents access to broader organizational datasets so they can enrich context, triage alerts, and empower security teams with greater response accuracy. With warm applause, Databricks CEO Ali Ghodsi retakes the stage to close out the two-day keynote and weave together all the announcements. He reiterates the foundational thesis introduced on day one: while modern AI models are extraordinarily capable, they are ineffective without enterprise context. As Ali Ghodsi summarizes: “We started the keynote by saying that we have all this AI that's amazing. But we have to give it all this context that we have at the bottom.” Over the two days, Databricks demonstrated how its integrated platform feeds clean, controlled, governed context directly into AI agents.

Ali recaps the base data layer required to make enterprise AI reliable and real-time: * **Lakeflow:** Serves as the unified ingestion layer, expanded with hundreds of native connectors to pull data from diverse enterprise sources into the lakehouse in any open format. * **Lakehouse RT:** Powered by the Raiden engine—which Ali jokingly describes as “Reynold's dream engine”—bringing blazing-fast execution speeds that allow executive dashboards to render instantaneously and AI agents to query underlying data with minimal latency. * **Lakebase:** Serverless Postgres built directly into Databricks to significantly drive down total cost of ownership (TCO) for data-backed applications. * **LakeTap:** Unlocks unified hybrid querying by seamlessly combining the Lakehouse and Lakebase, serving as the prime operational bridge for transactional and analytical workloads.

---

[02:41:20](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9680s) Governance, Genie Ontology, and the Fleet of Specialized Agents

Moving up the stack from raw data to governance and context, Ali highlights how Unity Catalog and Unity AI Gateway ensure safe and cost-effective execution. He reflects fondly on the live demo presented by Ankit Mathur, which showcased the advisor pattern: using a lightweight, open-source model to handle the majority of tasks and calling an intelligent advisor model only when expensive or complex reasoning is required. On top of governance sits the Genie Ontology, an automatically constructed enterprise knowledge graph that updates continuously in the background to supply business context to agents.

To make this accessible across the entire enterprise, Databricks introduced Genie 1 as a single pane of glass that eliminates the complexity of navigating individual workspaces or multiple data warehouses. Business users can query enterprise data directly without being overwhelmed by technical data tooling. Ali urges data teams to deploy this broadly: “Please help your organization get Genie 1 in everyone's hands.” He explains that administrators can achieve this simply by enabling Automatic Identity Management (AIM) in Databricks and granting non-technical colleagues a lightweight consumer role.

Beyond Genie 1, specialized Genies and agentic tools handle dedicated lifecycle workflows: * **Genie Agents:** Intelligent digital coworkers that handle recurring operational tasks and scheduled data automations. * **Genie Code for ML:** Highlighted in Mike Del Balso’s demo, enabling data scientists and engineers to build sophisticated machine learning prediction models, troubleshoot code paths, and maintain automated real-time performance monitoring. * **Genie ZeroOps for ML and Data Engineering:** Ali’s personal favorite agent, designed to autonomously maintain, diagnose, and repair data engineering pipelines whenever upstream or downstream jobs fail. * **Genie App Builder:** Demonstrated by Justin as part of Databricks Apps, enabling users to vibe-code custom internal applications and micro-apps directly on governed lakehouse infrastructure.

---

[02:43:40](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9820s) Agent Bricks, Lakewatch, CustomerLake, and the Open Vision

Ali recaps the developer and operational layers unveiled during the keynote. For developers building custom logic, Matei Zaharia introduced Omnigent, an open-source meta-harness powering Agent Bricks alongside managed developer services such as Agent Memory Services, secure VM Sandboxes, and dedicated storage. Through Databricks Apps, organizations can deploy agile, serverless micro-apps that scale to zero to maintain minimal run costs. For domain-specific workflows, the keynote showcased Lakewatch—an open agentic security lakehouse integrated with Panther to automate triage, alert resolution, and continuous threat hunting—and CustomerLake, an agentic Customer Data Platform (CDP) designed for real-time Customer 360 insights and automated Infinity Campaigns.

```mermaid flowchart TD accTitle: The Databricks Agentic Enterprise Architecture accDescr: An architecture diagram showing data foundation, governance, context, and specialized agent applications.

subgraph AppsAgents["Agentic Apps & Experiences"] G1["Genie 1 & Consumer Role"] GTools["Genie Code & Genie ZeroOps"] Apps["Databricks Apps & Genie App Builder"] SecMkt["Lakewatch (Security) & CustomerLake (CDP)"] end

subgraph DevContext["Context & Agent Infrastructure"] Ontology["Genie Ontology (Enterprise Knowledge Graph)"] AgentBricks["Agent Bricks & Omnigent Meta-Harness"] Memory["Agent Memory Services & Sandbox Execution"] end

subgraph GovLayer["Unified Governance & Cost Control"] Unity["Unity Catalog & Unity AI Gateway"] Advisor["Advisor Pattern (Cost & Model Routing)"] end

subgraph DataLayer["Agentic Data Foundation"] Lakeflow["Lakeflow (Ingestion & Connectors)"] RT["Lakehouse RT (Raiden Engine)"] Lakebase["Lakebase Serverless Postgres & LakeTap"] end

DataLayer --> GovLayer GovLayer --> DevContext DevContext --> AppsAgents ```

In closing, Ali abstracts the entire architectural vision into Databricks' core commitments: open-source freedom, rigorous cost controls, unified governance, and continuous enterprise context. Summarizing the ultimate platform goal, Ali emphasizes: “In short, we want you to have any data, any model, and any harness, no cloud lock-in, for your whole infrastructure.” He encourages attendees to spend the rest of the conference exploring technical breakout sessions, engaging with ecosystem partners in the expo hall, and joining the evening celebration at Oracle Park featuring The Chainsmokers.

- **Universal Interoperability:** Freedom of choice across models, data formats, and agent harnesses with zero cloud lock-in. - **Unified Governance:** A central control plane in Unity Catalog and Unity AI Gateway to track permissions, enforce data security, and audit agent spend. - **Enterprise Context:** Automated, background graph generation via Genie Ontology that grounds every agent action in enterprise reality.

## Notable quotes > [00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=0s) We don't need more intelligence for the AI, we need more context. — Ali Ghodsi

> [36:11](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=2171s) When you see agent demos and agent pilots, you're usually just seeing the agent loop, but the agent loop is only 1% of the work that you have to do. — Casey Uhlenhut

> [52:37](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=3157s) Instead of just talking about here is the latest model and its capabilities, we are now going from frontier model worship to actually building a frontier ecosystem, building one company at a time. — Satya Nadella

> [01:52:48](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=6768s) When agents buy on multiple channels at the same time, in seconds, you can't spend weeks designing a single campaign. — Tassos Argyros

> [00:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=0s) I think in the next 6 to 12 months, most organizations are going to see that the costs are going up so much that it's actually prohibitive to their companies. — Ali Ghodsi

> [02:30:00](https://www.youtube.com/watch?v=sn9My5Pj0mE&t=9000s) This is the first time in history that my capacity to investigate cases and my capacity to be thorough in investigating every single thing in my infrastructure is no longer bounded or limited by the size of my SOC team. — Michael Anderson

## Mentioned 7-Eleven, Adobe, Agent 365, Agent Bricks, Agent Memory Services, AI Runtime, Airbnb, Ali Ghodsi, Alibaba Qwen, Amber, Andrew Tu, Ankit Mathur, Anthropic, Antigravity, AppKit, AppSpaces, AstraZeneca, Atlassian, Block, Bricksport, Bricksports Global, Casey Uhlenhut, Claude, Claude Code, Claude Opus, Coinbase, CrowdStrike, Cursor, CustomerLake, Danfoss, Data and AI World Cup, Databricks, Databricks Apps, Databricks Clean Rooms, Databricks Free Edition, Databricks Jobs, Databricks Marketplace, Databricks Sandbox, Databricks SSO, Delta Lake, Docker, Genie, Genie 1, Genie App Builder, Genie Code, Genie Code for Machine Learning, Genie Ontology, Genie ZeroOps, Genie ZeroOps for Machine Learning, GitHub, Google, Google Cloud, Google Gemini, Harvey AI, Jack Naglieri, Jio, Jira, Joby Aviation, Justin de Brabant, Lakebase, Lakeflow, LakeFlow Connect, Lakeflow Designer, Lakehouse RT, LakeTap, Lakewatch, LangGraph, MasterCard, Matei Zaharia, Merck, Michael Anderson, Michelangelo, Microsoft, Microsoft 365 Copilot, Microsoft Copilot Studio, Microsoft Entra, Microsoft Fabric, Microsoft Teams, Mike Del Balso, MITRE ATT&CK, Model Context Protocol (MCP), Moonshot AI Kimi, Mukesh Ambani, National Australia Bank, Omnigent, OneLake, Open Cybersecurity Schema Framework (OCSF), OpenAI, OpenAI Agents SDK, OpenAI Codex, OpenAI GPT, OpenTelemetry, Oracle Park, Panther Labs, Poly, PostgreSQL, Raiden, Reliance Industries, Reliance Intelligence, Replit, Reynold Xin, RSA Conference, Satya Nadella, Serverless Micro Apps, Slack, SpaceX, Tassos Argyros, Thales, The Chainsmokers, UC Berkeley, Uber, Unity AI Gateway, Unity Catalog, Virtue Foundation, xAI Grok, ZeroBus

## Video description Watch the full keynote from day 2 of Data + AI Summit, which brought together 30,000+ data professionals across data engineering, scientists, analysts, architects, and business leaders.

The speakers covered new open source innovations, major updates to the Databricks Platform, and marketing and security solutions built for the agentic era.

0:00 Databricks CEO and Co-founder Ali Ghodsi on how context + AI unlocks enterprise agents that work 3:21 Product updates and innovations you’ll explore in this keynote 11:11 The AI harness problem: anyone can build their own agent harness, making interoperability hard 18:56 Why we built Omnigent, an open meta-harness, and how it works 23:20 Demo: Omnigent OSS 33:05 Why we open sourced Omnigent 34:44 What’s new in Databricks Free Edition 36:11 Engineers get stuck building infrastructure, and Agent Bricks solves this problem 40:28 Agent Bricks provides choice across models, agents, and harnesses 42:11 Agent Bricks solves the context problem 44:23 Announcing Agent Memory Services, powered by Lakebase 45:18 Announcing Databricks Sandbox: VMs for secure code execution 46:51 New capabilities in Unity AI Gateway: centralized governance, security, and cost controls over agents, models, and tools 52:37 Demo: from toxenmaxxing to valuemaxxing with Unity AI Gateway 1:09:16 Governed access data is the biggest roadblock to production for AI apps 1:13:04 How Databricks Apps brings app vibe coding in the enterprise 1:16:35 The prototype to product gap with vibe coded apps 1:17:53 What’s new in Databricks Apps 1:21:11 Demo: Building an app with Genie App Builder 1:28:01 Virtue Foundation powers healthcare with Genie 1:29:22 The only way to automate ML is with agents 1:37:07 Announcing AI Runtime: Serverless GPU compute for enterprise deep learning 1:38:13 Announcing Genie Code for ML: an agent for production ML engineering 1:42:43 Announcing Genie ZeroOps for ML: an agent that runs your ML systems 1:45:22 Demo: Genie ZeroOps for ML 1:51:01 Agentic ML engineering is here 1:52:48 AI is changing how people buy 1:57:12 Infinity Campaigns, a new approach to marketing campaigns, powered by the agentic CDP 2:01:33 Announcing CustomerLake: an agentic CDP to power the perfect customer experience – a billion times a day 2:08:53 Demo: CustomerLake Campaign Agents for always-on engagement 2:16:25 Agents are fundamentally changing security 2:22:29 Lakewatch: fight every security threat with agents 2:24:43 Demo: Lakewatch, an open agentic security lakehouse 2:35:45 Databricks x Panther 2:40:36 The Agentic Data Foundation

[Source](https://www.youtube.com/watch?v=sn9My5Pj0mE)

Reading summary

Databricks argues that enterprise AI adoption is no longer limited by raw model intelligence, but by an absence of deep enterprise context, open interoperability, and state-aware governance across organizational data. Across Day 2 of the summit, company leaders presented an integrated architecture designed to connect autonomous agents directly to governed lakehouse data while abstracting infrastructure management. Open Agent Interoperability via Omnigent (18:56) Matei Zaharia identified three major interoperability challenges hampering current coding workflows: brittle composition across siloed agent harnesses, fragmented collaboration across disjointed developer tools, and static, tool-level permissions. In response, Databricks announced Omnigent, an open-source meta-harness abstraction layer positioned directly above individual agent runtimes. Rather than replacing frameworks, Omnigent provides a standardized operational interface across tools such as Claude Code, Cursor, Antigravity, and custom OpenAI Agents SDK deployments. Defined through declarative YAML configurations, Omnigent enables teams to swap underlying agent harnesses mid-flight without losing state or session history. It supports live multi-user collaboration, real-time bi-directional synchronization between terminal CLIs and web browser interfaces, and centralized auditing. Omnigent replaces binary tool approvals with contextual, stateful governance policies, such as automatically adjusting an agent's permissions or prompting for budget confirmation once cumulative token spending crosses defined monetary thresholds. Grounding Agents with Agent Bricks and Gateway Controls (40:00) Databricks introduced Agent Bricks to decouple agent development from underlying infrastructure management, focusing on developer choice, enterprise context, and centralized control. The platform provides out-of-the-box endpoints for major frontier models—including Claude, GPT, Gemini, Qwen, Kimi, and Grok—alongside built-in reinforcement learning infrastructure. Databricks noted that its internal Genie research team used this RL framework to build specialized models that surpassed Claude Opus on domain-specific benchmarks. Enterprise context is organized through Unity Catalog, supported by the Genie Ontology semantic metadata layer, Document Intelligence, and Agent Memory Services to prevent hallucinations. To govern deployments, Databricks unveiled capabilities within the Unity AI Gateway, including an Agent Registry to catalog assets and automated dynamic data tainting, which restricts public-facing actions once an agent touches sensitive internal tables. The gateway also features intelligent routing, directing low-complexity tasks to cheaper models while reserving complex logic for frontier systems. Transitioning Citizen Prototypes into Governed Applications (80:00) Justin de Brabant highlighted an economic shift: while generative AI and coding agents have made writing application code trivial, shipping production-grade enterprise software remains constrained by secure access to internal data. Addressing forecasts that citizen developers will create one billion enterprise applications by 2028, Databricks introduced Genie App Builder to bridge the prototype-to-production divide. Genie App Builder provides context-aware "vibe coding" integrated with the Genie Ontology to align generated software with enterprise business logic. When users import prototypes from external platforms or GitHub repositories, the system binds them into curated "App Spaces." These spaces automatically provision and govern necessary backend dependencies—including SQL warehouses, Lakebase instances, and model serving endpoints—while enforcing organizational data permissions and running on serverless infrastructure. Industrializing ML Operations with Genie ZeroOps (102:43) Mike Del Balso argued that while generative AI accelerates enterprise demand for predictive machine learning, operational maintenance remains a severe bottleneck, with teams spending 60% to 80% of their time maintaining live models. To support high-velocity model deployment, Databricks unveiled AI Runtime, providing serverless, on-demand GPU capacity with multi-node distributed training, citing production drug discovery workloads at Merck. To mitigate downstream maintenance burdens, Databricks introduced Genie ZeroOps for ML alongside Genie Code. Inverting traditional on-call structures where alerts demand immediate human intervention, ZeroOps autonomously monitors production models for drift and degradation, investigates root causes, tests candidate fixes, and packages verified remediations for engineer sign-off prior to zero-downtime deployment. Specialized Engines for Agentic Security and Marketing Databricks extended its lakehouse framework into domain operations with CustomerLake and Lakewatch. CustomerLake serves as an agentic Customer Data Platform embedded directly in the lakehouse. It utilizes Profile Agents to automate data preparation and execute hybrid identity resolution—combining deterministic rules, probabilistic LLM reasoning, and human escalation—while Campaign Agents deploy continuous, real-time "Infinity Campaigns" generated through natural language prompts. In cybersecurity, Databricks announced Lakewatch, accelerated by its acquisition of Panther Labs, to counter automated LLM-driven offensive cyber campaigns. Contending that legacy SIEM platforms force dangerous telemetry pruning due to licensing costs, Lakewatch establishes an agentic security lakehouse that normalizes streaming logs to the Open Cybersecurity Schema Framework (OCSF). Powered by Genie, Lakewatch autonomously investigates low-fidelity alerts, maps MITRE ATT&CK coverage, and authors detection rules from natural language threat intelligence prompts. Databricks presents context unification and open meta-harnesses as the definitive solution to enterprise AI deployment, contrasting its architecture with fragmented proprietary stacks. However, the operational viability of this model relies on whether organizations can successfully organize their internal data catalogs, manage token consumption across expanding agent networks, and enforce stateful governance at scale.

Source quotes

[0:00] Ali Ghodsi: We don't need more intelligence for the AI, we need more context. [36:11] Casey Uhlenhut: When you see agent demos and agent pilots, you're usually just seeing the agent loop, but the agent loop is only 1% of the work that you have to do. [52:37] Satya Nadella: Instead of just talking about here is the latest model and its capabilities, we are now going from frontier model worship to actually building a frontier ecosystem, building one company at a time. [112:48] Tassos Argyros: When agents buy on multiple channels at the same time, in seconds, you can't spend weeks designing a single campaign. [0:00] Ali Ghodsi: I think in the next 6 to 12 months, most organizations are going to see that the costs are going up so much that it's actually prohibitive to their companies. [150:00] Michael Anderson: This is the first time in history that my capacity to investigate cases and my capacity to be thorough in investigating every single thing in my infrastructure is no longer bounded or limited by the size of my SOC team.

Original transcript

[0:00] Yeah. Thank you. - Welcome to the stage, Databricks co-founder and CEO, Ali Godsey. All right, super excited. Wow, everybody made it 8:00 AM. It was hard for me, actually. All right, so today we have a super exciting program. But I want to recap a little bit what we talked about yesterday, and then I'm going to tell you sort of the talks that are coming up, and then we'll get into it. OK. So as I mentioned yesterday, We think that... the AI is plenty smart already. We don't need more intelligence for the AI, we need more context. If we can just infuse all the context that we have inside all of these organizations that you all represent. to the AI. then we can actually do magic. And that's our job, essentially. So, how do we do that? And we said that there's really context itself is four things. that we sort of have to nail. Number one, we need choice. So openness is extremely important for us. The fact that we can choose any model, any infrastructure, any cloud, any file format. Avoiding lock-in, I think, is one of the most important things that Databricks can help with. That's a mission we've been on since we started the company, since we, you know, our roots in this conference, the open source community, Because... over the last 20-30 years, the main thing that's slowing down organizations is the fact that they're getting locked in to various types of software that they keep adding to their complicated stacks. So that's number one. Number two, we have to be able to control the AI, make sure that we have security rules, policies and auditability over the AI, the data, and the infrastructure. And then cost controls. This is going to get extremely expensive. So I think you all already know that AI can get very expensive quickly, but I think we're just scratching the surface. I think in the next 6 to 12 months, most organizations are going to see that the costs are going up so much that it's actually prohibitive to their companies. And then finally context. We talked about it yesterday. Genie ontology is how you can actually compute in the background continuously and learn from all the data that you have in the organization, not just in Databricks and Lakehouse, And that really helps speed up the agents and it actually lowers the cost and actually increases the quality. So it's kind of like index, if you will, kind of like a page rank that Google did, but for the enterprise. Okay, so this is what the architecture we presented yesterday looked like. I'm not going to walk through all of it here, but I'm just going to highlight the talks that are coming up, OK? So one of the things I'm really excited about is that Casey and Ankit are going to get up here and talk about agent bricks. So this is the developer side. of agents, so if you want to build your own agents, Agent Bricks gives you all the tools for that. It gives you a sandbox, it gives you memory, and it also comes with open source Omnigens that we're going to talk about today. But when you combine Agent Bricks, Omnigent, and Unity AI Gateway, you can actually start doing really interesting things with cost control. So we'll see a demo of that and I'm very excited about that. I hope that most of you actually look into how to do this. So the idea is You use this advisor pattern. And the advisor pattern, the way it works, is that you use a much cheaper, smaller model. It could be an open source model. But you tell it, hey, here's a tool you can use. This tool is a much smarter, bigger model that's expensive. but only use it when you get stuck. And if you do that, you can now use that with your coding agents. You can use that with any AI work that you're doing in the organization. And we've seen that really curb the costs. while at the same time allowing you to keep the intelligence that you have. So that's a really exciting demo and an exciting talk that we're going to hear about this. We're gonna go back a little bit to GenieCode. I mentioned yesterday that GenieCode is really good at writing data engineering pipelines. We saw with zero ops that it could automate the headaches of pipelines going down and troubleshooting them and fixing them. But Genie code is also really amazing data scientists and machine [5:00] Learning agent. So we'll hear a whole talk and demo by how actually you can write whole pipelines and also troubleshoot the pipelines and fix them and sort of get into the automated loop of machine learning and democratize that inside the organization. So that's a really interesting talk. We're going to look at apps. This is very exciting. As I said, we think there's new-- agentic system of record or agent system of record that's appearing. where you can actually start democratizing apps inside of your organizations to tens of thousands of employees, hundreds of thousands of employees, And the key thing there is, how do you actually authenticate and authorize and give access to the data? But we're going to go far beyond that, so a very exciting talk on that. And then we're going to see Lakewatch, which is the security information event management system that we launched two months ago. We're going to see how we've extended that. We'll also hear a little bit from Panther. the company that we announced that Databricks has agreed to acquire. And then finally, customer lake. This is the CDP. On Lakewatch and CDP, We also made it a little bit more tutorial-like so that we can bring everybody along. If you don't have a background in how sims work or how customer data platforms work. But next up we're actually gonna have Matei come on stage and he's gonna talk about the new exciting open source project called OmniGent. that he actually open sourced over the weekend, and it's central to everything we do at Databricks. It's actually the underpinnings of Genie. built that on that pattern ourselves. So that's super exciting. And I want a request here for the camera crew. Please zoom in on his shoes. Okay? And make sure you capture that, because that's going to be a really important part of his talk. But before we hand it over to Matej, I wanted to show you a quick video from a very important deep partner of ours. which is Mukesh Ambani of Reliance Industries in India. So Mukesh and Reliance Industries, they're on this amazing mission that we've been partnering with them on. which is how do you democratize AI? How do you democratize data to everyone in the world There's a lot of folks that cannot afford very expensive models. So how do we reduce our lines on these very expensive models and have maybe open source models that are more available and where the costs are much more manageable, and then actually get it out to everybody and actually help with sovereignty in the many different nations that need AI as well. So very excited about that. So with that, I want to play you this video. Thank you. Dear Ali and all the delegates at the Databricks AI Summit. Namaste. Thank you. Greetings from India. Thank you for inviting me. to address this prestigious gathering of the global data and AI community. Today, with Reliance Intelligence, our mission is to democratize intelligence for 1.5 billion Indians. Our motto is simple. AI Everywhere. for everyone. We want AI to create. a more equal India, bring prosperity to every farmer, entrepreneur and neighborhood shopkeeper, transforming education healthcare and public services for every Indian. Solutions that work for India can serve other countries in the global south, where four-fifths of humanity resides. Partnering with the best AI companies in the world allows us to make progress in this mission. That is why we chose Databricks. What impressed us most was not only the technology, It was the alignment of vision. Ali and his team understood that our ambition was not simply to modernize infrastructure. It was to build the foundation for innovation, productivity, customer value and societal value. Together, we are creating a unified intelligence platform across Reliance. We have migrated petabytes of data across geo, retail, energy, materials, media, and beyond. to serve more than 800 million customer relationships. This is one of the most ambitious enterprise data transformations undertaken anywhere in the world. for many years. Enterprises use data mainly to look back. the next generation will use data to make better decisions Act faster. and continuously improve. This is the shift from dashboards to decisions. Our leaders [10:00] are already using Databricks Gini to ask questions in natural language and receive useful intelligence. in real time. We are now scaling this to thousands of decision makers across Reliance. Friends, India is perhaps the most demanding test bed for AI because of our scale of aspirations, diversity, language needs, affordability constraints, and speed of adoption. If an AI solution works for India, it can serve the world. Together with Databricks, we hope to create a blueprint for how enterprises can harness data and AI responsibly openly and at scale. Ali Thank you for your partnership, friendship and vision. The best of what Reliance and Databricks can build together is yet to come. My best wishes to all the participants of your summit. Thank you and have a good day. Welcome to the stage, Databricks co-founder Matej Zaharia. I. Thank you. all right hi everyone welcome uh to the second day of keynotes um so i'm excited to kick off the morning today uh and talk about a new open source project we launched um over the weekend we just released it saturday this is a new layer from our engineering team for working and building with agents and it's based on patterns and challenges that we saw coming up everywhere where people are using agents, and we realized that we're going to need this new layer on top of the current Agentex stack to make it really affected. So I'll talk about what that means, but we call it a meta harness, harness of harnesses, if you know what an agent harness is. Okay. So as everyone probably knows, agents are really starting to work. Coding agents are the first type that were really successful. Pretty much every developer has probably used them at this point. But the same is true in other fields. And you saw with Genie and with other agents that you can make very effective agents. And especially because coding is a lot easier now, you might wonder, do we even need agent infrastructure at all? If I don't like the current agents, can I just Vibecode my own? So it seems like, hey, maybe we're kind of done with at least simple software and we're set. And this is what the world should look like. Agents are everywhere. Things are working. But actually when you use agents, it can be quite annoying if you're a manager. It can be very scary because of security issues and stuff like that. So, you know, the world isn't really perfect yet. So there is a problem. And actually I didn't realize until I was making the talk and looked very carefully at this image that we even see part of the problem in the image. It's the harness, the thing around our little agent there. Okay. So what's an agent harness? So agent harness is basically the software around the language model that is the agent's interface to the world. So this is something like Cloud Code as a coding agent. Does Cloud the model-- that's-- a thing that just predicts that outputs tokens. And then there's cloud code, the harness, which connects it to files and a user interface and security permissions and stuff like that. And then there's codecs, and then there's any custom agent you make, you have some kind of software around it called the harness. And because it's so easy to build harnesses, and because it actually helps to co-design the harness with the language model, everyone is building them, and indeed everyone has. so many agent harnesses here. These ones here are just some of the popular ones for coding. And then if you make custom agents, there are lots of agent SDKs and stuff like that. There's just a ton of these out there. Actually, this weekend I was at the pet store, and this reminded me a lot of it. Pick the harness for your agent, any shape, any color, you can put it in there. So what's the problem with these? Well, the issue is that agents sometimes have to interoperate, and there are also a lot of common challenges that the harnesses have where it would help to have a common layer that lets us interface across them. And that requires people to sort of build the layer and agree. If everyone is just Vibe coding their own thing, they will still be incoherent. So, you know, there's no amount of like faster coding will solve this problem. So what are some of the challenges with interoperability of these? So the first is composing agents. When I'm doing coding and when a lot of the engineers at Databricks do coding, they use many coding agents at the same time. If there's a bug and one of the agents is stuck, they ask another one. [15:00] They give them different inputs. They ask them to work on different things in parallel. So how do you compose agents with different harnesses? It's easy to do within one of them, like cloud code or codex, but what if you want to use both? And there's a lot of other techniques for using multiple models to get better cost, better quality, and so on. The second one that's interesting is collaborating across humans and agents. You know, if I'm using one coding agent and someone else is using another one, how do we collaborate? Because ultimately we have to get stuff done together. So it's actually a huge problem that that we think we can improve on. And then the final one is I've got all these agents, all these different pieces of software, the harnesses, how do I control all of them for cost and security? So we start these problems repeatedly in Databricks, actually from two angles. First, the engineering side where we have our own info around coding agents. And, you know, all the thousands of engineers are using them. And also on the customer facing side where we build a GenTech products like Genie and we have to keep changing how they work and we build custom agents for our customers. So let me dive into HFD a little bit more. So composition, there's so many patterns emerging now for how to combine agents to get a better result. So if you've looked at coding, maybe you've heard of loops. This is how you have some agents maybe supervise each other or run in a loop to try to finish a task. And there's all these patterns for loops. Another pattern that's emerging that's really interesting is called advisors. So you can have a small, fast, a cheap model that's doing a bunch of the work, and then an expensive model. And maybe the expensive model is like driving the small one, getting it to do the busy work or maybe the small one is calling to the expensive one and it's stuck. So Harvey AI had this blog post on custom train model with a frontier model as an advisor. Anthropic has a post about it. Cloud Code actually uses multiple sizes of models for different things. Actually, my research lab at UC Berkeley, we wrote a paper on advisor models too. And then there's many other patterns of composing them. We have a post about how in Genie, if you use multiple LLMs, you can get better quality and better cost. So that's the composition that does kind of continue to be an issue as agents and models evolve. Second one is collaboration. Whose desktop looks like this? This is what my life is like all the time. I've got all these Agent Windows open, plus things like Slack and Google Docs, And all I'm doing is copy-pasting text across them. This is pretty ridiculous. There should be an easier way to collaborate, but each agent has its own harness, its own app, and it's stuck there. And again, your company managing this, it's also a huge pain to manage everything. So just as a simple thing, when I am working with one of these coding agents, and I have to go away or whatever, why can't I just hand my session to another engineer and have them continue with the full context. That's a simple thing that any harness would benefit from. And then the final one is control. Agents can be hacked through prompt injection, so you shouldn't really trust them. Agents will also make mistakes even without being hacked, like deleting databases and so on. With Lakebase you could just to revert the database, but in general, it'll be an issue. And they can also cost a lot. And people might say, "Okay, well, agents are very dangerous. Let's ask users for permission." Here's the average permission prompt when Claude asked me something. It's like, do you want to run this 100-line Python script? Yes or no. It's a huge beta as a user, just waiting, just hoping that users will review every action is not enough either. So this control and security layer is another common problem across harnesses that we want to solve. So we think that we need a new layer above the harness level to manage and build with agents. And we also think the layer should be open so anyone can plug in their own harness, their own workflows, their own tools. And we call this layer the meta harness. So that's what we decided across Teams and Databricks, and then we decided we have to make it open source, because that's going to be much better if anyone can contribute and it works with the latest things. And that's what we just released. with Omnigent. So how does Omnigent work? So at the bottom you've got existing agents. It works with coding agents. Actually, just since Saturday we added two more, Antigravity and Courser, that it supports because people were asking for them. It also works if you're building custom agents with something like OpenAI Agents SDK. You've got a lot of the same features on top. Now, all these harnesses are different pieces of software, sometimes tailored to that LLM. But what we do is, on top of them, we give them all a common interface so we can do the collaboration and composition [20:00] and sharing that Omnigens try to do. And the interface is basically, you know, you can send messages in, you can get a stream of events out, they can call tools, that's what it is. So Omnigens then has two pieces. First, there's a runner, which wraps around your agent and can enforce security and other types of controls by sandboxing it and just monitors it and makes sure it's running. And then you can also optionally connect Omnigens to a server where you get collaboration and central policy features. So with your server, if you connect your agents to that, you get a central history for everything, you can get a catalog to share agents with people, you can set up MCPs and skills centrally, you can also set up security policies and share artifacts out. And then on top of this, no matter what kind of agent you have, you also get a uniform interface. So you still get the agent's terminal UI if it has that, but any agent's also accessible on the web, native app, mobile, and also REST API. And Omnigens is open source. It does not require Databricks in any way. We developed it so you can just run it anywhere. And we can use it both at small scale and large scale. So it just uses standard technologies like Docker and Postgres and OpenTelemetry to run the server. Okay, so what does this give you now that you have this layer? So there's three different things. So first for composition, you can now easily combine agents that use multiple harnesses and RLMs. You can actually just write a little YAML file to define an agent that's using multiple of these together. It's also super easy to switch agent harnesses while the agent is running and continue to run. So you can also connect me with a different one, which I'll show in a bit. Second, you get collaboration. So we support live collaboration. You can even invite other people to write to your session and talk to the agents and comment on files and things in real time. So this makes it way easier to work with these. Once this is available, like all my work I try to do through this. And then finally, for control, we looked a lot at how can we make security around agents better. And we actually came up with what I think is a much nicer, more powerful model than what you have today. Today in most agent tools, the control is just for each type of tool call. Do I allow it or deny it or ask the user? They're very static. We support contextual policies where the permissions vary based on what the agent did before. So for example, if the agent read a confidential document, maybe we don't let it send an email outside the company anymore. But if it only read public marketing docs, then it's okay to send an email. So tracking state makes that much more powerful. And we also have policies on cost, which none of the agent frameworks do. So you can tell her, don't spend more than $5 on this, for example. We also built a very flexible OS sandbox to protect the agent from harming your machine. So that's three things that it does. To show you a little bit of this in action, I'd like to show a demo. All right, cool. So I'm here with Omnigine setup. So this here is actually my dev box inside Databricks. It's like a machine that I develop on, and then there's my laptop. So the first thing I'll show you with it is how it just wraps existing agents. So I'm just going to start Claude, but with this Omni in front of it. And that's just going to start it. It's starting this runner. And then, as you'll see, we'll basically just have the standard Claude interface. So this is actually the instance of Claude in Databricks that's been configured with all our settings. It's got all these MCPs and different things in there, specifically for Databricks. And I can ask it to do stuff. So currently, I'm in the Omnigine project. So what is this project? I can ask it. And you'll see it's doing stuff. And we even customize these things that Claude says, archiving. If you don't know what archiving is, don't worry. So there it is. It means something in Databricks. Okay. So now I'm just starting Claude. So that doesn't seem that exciting. But what do I get to Omnigine even as an individual developer with an existing agent? One of the things I get is a web UI for my agent, too. So the same session I was doing, I have it on the web. I can see all the files and things in here if I want. And I can see it here in a nice chat form. I can even see the same terminal in here if I want. So it's the same thing. [25:00] Let me ask the agent to do something like Actually, since I was doing this before, I'm going to check out the readme first. Oops, sorry. because I was trying this before, so let me do add a joke to So I'll just tell it to add a joke to the top of the readme. Okay? So I sent a message from the web. Now, in the terminal, you still see it here. So like we do this mapping of user input and output across the to interfaces. And, okay, it's saying, "Hey, I've added a joke in there. If I look here, it says that it added it." Now, the other nice thing about the Web UI, is, of course, it's much more visual. So if I look here, I can see there's a changed file to read me. I can see there's the joke at the top. But this is just like a markdown file, so I can just view the whole thing I'm going to close this to make it a little bit bigger here for you. Let's move this, okay. All right, so this is it. I can view the diff. And I can even put in comments about it. So for example, I guess there were a few because I was doing this before, but maybe I can tell it like I'll just tell it to delete the jokes. Okay. And then, I don't know, I could try to change other things, but I can leave some comments in here and tell it to address all of the comments and this is just sending it so i get this nice visual interface and it's it's doing its thing um the other thing we built for it that's like very very helpful. I'll just tell it. I'll tell it to just add one. The other thing we built is actually a native app. So in my browser, sorry, Yeah. If I go here, in my browser, I've got all these tabs. It's hidden all the time. I closed all my tabs this morning. But in the native app, it's very easy to find it, and you get notifications and stuff from the system. So let's look at this again. Okay, we got a joke in there. Okay, so what else can I do? The other thing I can easily do is I can switch agents. So I can take the same session and say, hey, let's actually continue from here with codex. and just fork the session. and it's going to start Codex here, and you'll see that it has the same history. Okay, so now in the terminal, this is actually OpenAI Codex. And I can just tell it add a second. joke too. And you'll see it adding that in a second. This is a little bit slower, but it's going to be in there. So this is pretty cool. If I'm stuck with one agent, I can switch to another one, and then there are two of them. But then the super interesting thing that really is enabled by composition is I can also use a multi-agent So we actually include an example here, Poly, which is a multi-agent coding supervisor that can use both Cloud and Codex to do subtests. And I'll start it in the same folder. And Talit asks, Thank you. Thank you. Okay, so I'll tell it to ask these guys to both rate the jokes at the top of the readme. And you'll see when it starts that it's going to kick off a couple of them. It's checking to see if I have Cloud and Codex installed. It's launching some workers. So in here, you'll see some workers will pop up. um, there's a cloud worker and then it's going to start a codex worker too. And it's telling them to rate these things. So these are again, these are just cloud code and codex. They do the same things you could do Otherwise, I can click on these guys and even see you know, what they're doing in here, there's the terminal, they're trying to rate things, etc. If I go back at the top, you'll see that the supervisor hears And eventually it's going to say, OK, so first one, pretty good. Second one, not quite as good. So that's what Foley thinks. Okay. So obviously I could do more things. I use these to do all kinds of other stuff, like for example, thing I was doing last night is I asked to find all the open GitHub issues about something and then investigate them in parallel and see if they've been fixed in the code. So there's a lot of cool stuff you can do with this that, you know, I don't have Time to show. But the final thing is, so that was a composition of agents. The other thing you can do that's really cool is collaboration. So let's say I think I have a great change to the project here, and I want to get an engineer to review and commit this. [30:00] - Call one of our engineers. Uh... Oops, sorry. And this is Ankit. He's on the AI team, and I think he's a good reviewer for Just read me. So I can just invite him to it. And let me just give him a call. Here, let's see. Oh man, Matei's calling me on vacation? Thank you. Thank you. Hey, Matej. Hey man, you busy? Not really. OK. I got this important change to the readme. Can you take a look and tell me which version you prefer? Thanks, yeah, I shared my session with you. All right, let me take a look. Thank you. I know the agent said that joke one is better, but I kind of like joke two. Thank you. So as you can see, thanks Ankit, that was very easy. So I invited him, he can see the same session, he can leave comments and things, and he can send a message in there and tell my agent to do things. So I can hand off work to someone else. And obviously all the history, all the sessions, any agent, whatever, are also stored on your server, so you can analyze them later and do things. Okay, so final thing I'll show is policies. Actually, I guess I can show it in here too. So the stateful policies. So notice at the top here, we actually track the cost of each agent, how much it's spent. And we also allow you to set policies. And these are contextual. They can depend on the current state. So one of the policies we allow is set a budget on it as it's spending in this session. Don't let it spend more than, say, a dollar without asking the user. So I'll just make something very silly here. It already spent 25 cents or whatever, so I'll say, ask me if you're trying to spend more than 20 cents. And then I'll tell it how, you know, I'll just tell it to do something that's going to involve, you know, reading a lot of files. And you'll see, as it's thinking, eventually when it begins to do stuff, it's going to ask me for permission because it's out of budget. So this is like pretty nice. You can also tell it like launch a sub-agent but only let that agent spend a dollar. There's all kinds of things. And if you're managing this for a team, you can set policy centrally. And if I approve it here, it's going to continue. And eventually, when it gets to $5, it's just going to stop. So this is a really cool feature, again, on top of any agent, cloud, codex, whatever it is, and it's just scratching the surface. You'll hear more about these soon. Cool, so that's the demo. So let me just wrap up with a couple more things in the slides. Thank you. quote. So one thing you may be wondering is there's a lot of functionality in this. Why are we open sourcing it? So we really think working with agents has to go above the level of a single harness and SDK and so on. We think this meta harness layer is going to be needed, and it will benefit from being open the same way that data formats and sharing protocols and stuff like that benefit from being open, because it's an ecosystem thing where people sort of have to agree on a way to do it, to make it work well. So that's why we're making it open source. We use it a lot internally. We hope you use it as well. And let's figure out together how to build things with this. We already got a lot of really great contributions since Saturday. The other cool thing about open sourcing, this is everyone we saw building an agent wants an app like what I showed. Maybe not for coding, but with collaboration, sharing, security, all that stuff. So now you've got an open source app you can use to build any agent with a great interface. I didn't show it, but you can even get a mobile interface to your agent through this. So we have a lot more coming up, more ecosystem support, also auto-optimization of the agents, which you'll hear about hopefully in a while. future events. So that's Omnigent. We really think that AI needs a layer above the harness. It makes it way easier to work. Certainly, our team has been working with this all the time, developing not just the engineers, but product managers and designers. and so on. And we invite you to check it out. It's early on, but we think it'll be an important layer. Thank you. Thank you. Hundreds of thousands have come to Databricks to learn the skills that actually matter in data and AI through Databricks Free Edition. From building agents, to analyzing biomedical literature, to unifying fragmented environmental data sets, there are endless ways to apply data. [35:00] - Apply Free Edition. - Getting students access to the Free Edition before they enter the workforce, helps bridge that gap between classroom learning and industry practice. It gives them access to a modern data and AI platform that is extremely relevant for employers. One of the biggest advantages of working with students with prior Databricks experience is that they can get started much more quickly. Rather than learning the platform, they can spend their first few weeks working to solve real business problems. and Lakeflow Designer on Free Edition, where you can learn on a production platform. This means you get virtually all of Databricks for free forever. Hundreds of thousands of people have already built data and AI projects and learned on Databricks. Sign up and start building today. Welcome to the stage, Databricks Director of Product Management, Casey Uhlenhut. Thank you. Hi, everyone. So yesterday was about consuming AI. You saw how Genie allows you to get AI-powered answers to all of your knowledge workers. but today is about developers. Developers are the teams who are building agents that run your business and provide amazing customer experiences in your products. and Agent Bricks was built for them. Agent Bricks is our agent developer platform. We launched it a year ago and we've seen amazing success so far. Over 100,000 custom agents have been built on AgentRank since we launched. We've had over a quadrillion tokens go through our AI gateway, and our MLflow solution, which provides Gen AI experimentation and tracing, has had over a billion downloads. And our customers are seeing a lot of value from this solution as well. Where 7-Eleven is able to repair maintenance fixes faster, MasterCard is able to detect fraud across millions and billions of transactions. And AstraZeneca is able to help their drug discovery team get commercial insights faster. This is all amazing, but it's only one piece of the picture, and we're only just beginning. When you see agent demos and agent pilots, you're usually just seeing the agent loop. but the agent loop is only 1% of the work that you have to do. It's the other 99% that's gonna get you. where you have to figure out authentication, you have to figure out how to connect to MCPs, how you're going to manage identity, how you're going to have this run for a long time without breaking or hitting timeouts. and You have to do all this work? for just one agent. but you're trying to build thousands of agents in your company. So it's no wonder that we find that developer teams end up stuck building infrastructure rather than building agents. And as we've talked to developer teams, there's really three problems that start to emerge for them. The first is that the frontier seems to change every week. A new model comes out, a new agent harness comes out, like you just saw with Omnigent. And now the solution that you built six months ago is at least one generation behind, if not more. And if you want to be able to change to be on the cutting edge, you might have to actually change stacks. which then means you have to replatform your entire solution. The next problem that you have is that your competitive advantage is in your data. But your data can be fragmented across many different sources, and your data can come in many different formats, all the way from PDFs to external APIs. And it's really hard to help your agent select the right context from all this data. And lastly, agents are the most privileged actor in your enterprise. where they have access to sensitive customer data, and they can take real actions. And that's why you're seeing the headlines emerge with tons of these different scary headlines where agents are deleting production databases, they're leaking sensitive data, And to make it even worse, these agents can spend a lot of money and not a lot of time. So as a developer, you could end up blowing through your budget before you even got to production. [40:00] These are the challenges that we keep hearing over and over again, which is why we believe an agent developer platform must provide choice, context, and control. Agent Bricks is our developer platform where we provide answers to all of these so that developers can focus on building agents, rather than on building infrastructure. So now we're going to go through how Agent Bricks provides each of these values. So we'll start with choice. Agent Bricks provides native model access in integration with all of the frontier models. So you get Opus, you get GPT, you get Gemini, you get Quen, and even recently you get Kimi. And today we're excited to announce in partnership with SpaceX that you now get access to the Grok model family as well. Thank you. But sometimes the frontier models aren't enough. You might have a very specific task that you need to accomplish, and it turns out you're going to need to build a custom reinforcement learning model. So we also provide an entire model platform for you to use reinforcement learning and build custom models that are trained for your specific task. In fact, our own Genie research team recently used RL and was able to actually outperform Opus. on their tasks in there. And you're able to serve and deploy these models alongside all the rest on the Agent Bricks platform. But choice isn't just about models. When you're building custom agents, you want to use your own framework or build your own. And so today we're excited to announce the Omnigent that you just saw with Matei. You can now build and deploy all of those agents inside of Databricks. And of course, you can continue to work with your agent framework of choice, whether it's the OpenAI, Agent SDK, LANG graph, or many more. So we believe choice is about giving you the flexibility to stay at the cutting edge without having to re-platform your solution every time, and we're able to do this by giving you a unified interface across all of your models so that you can easily switch amongst them. Thank you. The best model in the world, though, is only as good as the information you give it, which takes us to context. Context is how your agent understands about your users and your business. And as we mentioned before, context is just data. So what we do in Agent Bricks is we allow all of your context to be centrally managed inside Unity Catalog. So you can connect your external data there and all of your other data is accessible in there. And all of this is callable as tools by your agent. So your agent can discover all of them, including dashboards. but Discovering context isn't enough. You also need to make sure their agent pulls the right context. And that's why the genie ontology announcement we made yesterday is such a big deal. Geneontology provides the metadata layer over all of your agentic assets so that your agents can effectively and efficiently retrieve and select context. So now that we have this foundation set up, the next hard part is sometimes your context comes in complex formats. You might have millions of PDF contracts and invoices that you wish you could feed to your agent. but you can't. And that's why we built document intelligence. Our document intelligence solution provides custom built models. for parsing, extraction, and classification. And our solution is the best on the quality cost curve out there because we know you have millions such of these documents and we don't want you to break the bank just so that your data or your agents can get access to that valuable data. And of course, we know that not all of your data is inside Databricks, so we allow you to securely connect to all of your external APIs with our managed external MCPs. And we'll provide some of them out of the box for you so you don't even have to worry about all those authentication issues. The last part about context is sometimes when you're building these custom agents, you want to personalize them per user. But you don't want to build an agent per user. That's way too much. So instead you want something that's like agent memory. So today we're excited to announce our agent memory services, where you can connect your custom agents to our managed APIs, and we will store user preferences, conversation history, and sessions for you. so that you can personalize your content. to your users. Thank you. Context is probably the most difficult problem that exists in the agent space today, which is why we have an entire research team dedicated to just [45:00] Solving the hardest problems here. So our research team is working to solve agent memory, agent retrieval, and agent search. And all of the solutions and techniques that they come up with are then baked into all of our agent brick services that we provide for developers, as well as they're integrated into Genie. So now that you have connected your agents to all of your knowledge, now you want to act on it. but safely. And so that takes us to control. There are two main products that we want to discuss in this control section. So as we mentioned earlier, agents can run wild out there and go rogue in your enterprise and even delete a production database. But you kind of want these agents to run code. They can do like amazingly creative things. if you just kind of let them cook. but you don't want them to take down production. So you want to give them some kind of walled environment that they can operate in. And that's why today I'm excited to announce Databricks Sandbox. Databricks Sandbox provides VMs that you can have your agents securely run and execute code in. But this isn't just your generic VM solution. We are the only VM out here that provides you out of the box access to your data. So we will allow you to have downscoped access using OnBehalf of permissions to your data so that your agent can get creative with your data inside that VM that won't affect the rest of your system. Thank you. Also, these sandboxes are incredibly fast. They take just milliseconds to launch, so you can create and destroy these sandboxes just as fast as your agent can work. which makes this extremely exciting for being able to work and play with your context securely in your agents. And now, for the piece de resistance. Unity AI Gateway. Unity AI Gateway is our centralized governance, security, and cost control solution that allows you to unify governance across models, agents, MCPs, and skills. And this is probably the most exciting product that we have that's really going to be a game changer, that allows you to run your agents autonomously in your enterprise. There are five new capabilities that we've added to AI Gateway that we need to go through. So I need you all to buckle up because we're going to dive right in. So the first, remember those thousands of agents? that your team is building? Well, they're all going to probably build them with different tools and platforms in there, and that's okay. But wouldn't it be great if you just had like one inventory of all the agents that were built in your enterprise? Well, now that's possible with our agent registry that's part of Unity AI Gateway. You can register all of your external agents, all your agents built on Databricks, all your MCPs, all your skills, So you have one system of record of all of the agentic assets in your organization. The next capability that we have is related to security. We provide contextual security policies. This one's best explained through an example. Imagine you have an agent that touches sensitive customer data with PII. You want to change the state. of that agent to now say, "Hey, it's been tainted, because now it has access to sensitive data." So when it takes its next action, you probably want to change what it can actually do. So you may allow it to still send an email to a coworker. but now you absolutely do not want to allow it to publicly post things. that data. However, If in a different agent session with that same agent, it only touched public information, then it's totally okay for it to then access and publish things to the public web. And this is the best illustration of what a dynamic and stateful policy can do that's contextual to your data. And this is only possible on Agent Bricks, because our agent platform is built and integrated into our data platform. And so this is the kind of security policy that then is going to allow you to bound your agent to safely run autonomously. Because at every action and every step your agent takes, we can change how the policies are run. On the same hand with security, we have now partnered with many security organizations to allow you to have security guardrails that you can use to further control your agent behavior. Now that we've covered security, let's go into the thing that we're all thinking about, token maxing. Cost controls are a huge issue for many developers today and their organizations, where you can end up deleting your budget before you really even got that far ahead in your project. [50:00] So now with AI Gateway, we support budgets, where you can set spend limits for different users and user groups in there. But what you really probably want is not just to know how much you've spent. You would actually rather there be some intelligent system out there that kind of helped you control your spend automatically so you don't have to really think about it. And that's where intelligent routing comes in. With intelligent routing, we can figure out what the best model is for the task based on the complexity. So we can route trivial questions to cheap, fast models, and we can route complex questions to expensive and powerful models. The other thing that we provide with Smart Routing is because we are multi-cloud and multi-AI, you can have the most highly available agent out there where you can set up fallbacks across your model providers, as well as across your clouds. And the last major capability that we've added to Unity AI Gateway is agent tracing. Using MLflow tracing, we can log every input, every output, and everything that happened in the middle of your agents, and centrally store them all in a Unity catalog table. and you own that table and all of that data, because it is super sensitive to your enterprise. Once all of those traces are in Unity Catalog, you can then use Genie to analyze them to debug what's going wrong with your agent. Or you can connect it to our SIM system with LakeWatch and get real-time incident alerts when data leakage happens. So this is what AI governance should look like in the enterprise, where all of your assets can be centrally managed, you can have contextual policies to enforce security, you have an open ecosystem where you can connect to partners, and you can have the cost controls that you need. All of this will allow you to run your agents autonomously. It also works really well for admins in your enterprise, where now you can actually safely and responsibly democratize AI across your organization. So on that note, I'm very excited to welcome to the stage my colleague, Ankit Mathur, who's going to show a demo of how you can use Unity AI Gateway to democratize access to coding agents across your developer org. With that, please help me welcome to the stage Ankit. Thank you. Thank you. Hi everyone, my name is Ankit and I'm a software engineer at Databricks. And yes, I am back from vacation. Today I'm going to show you how our Unity AI Gateway will help you roll out AI at your company while keeping costs in control. Now, as a software engineer, I myself use AI every single day. And what we're seeing is that leaning on AI is starting to become incredibly expensive as costs are growing exponentially. The top 1% of spenders are spending $90,000 per user per year, and that exponential isn't showing signs of slowing down. What started happening is that we don't just have engineers writing code anymore. Every single person at your company is using agents They're using them in parallel, and the models are good enough now that they're going to keep working uninterrupted for as long as we let them. So it's no surprise-- that costs are growing out of control. But are we using AI responsibly? And what's going on inside of that bucket of spend? That's what the Unity AI Gateway can help you answer. And so in today's demo, I'm going to demo three of the six things that Casey talked about. The first will be budgets, the second will be smart routing, and the third will be agent tracing. So let's jump right in and we'll start our day exactly where every other developer starts their day, which is inside of the terminal. And so with the Unity AI Gateway, you can easily point the coding tools that you already use towards the gateway. So all I need to do is something like this. Thank you. Thank you. And boom, codex starts just as it normally would with a couple differences. It's now pointed at the Unity A gateway. inference and MCPs are tracked through a central choke point. and I can continue to see exactly the same codex UI that I was using before. So I can ask a very simple question like, Find me a coffee shop. near Moscone Center. And it will work just as normal. Except this time, your MCPs are getting routed through a central registry, so you only need to authenticate once. and your inference is being tracked. Now on the right here, you'll see something interesting. which is that because all of our coding tools can point at the same gateway, we're able to specify and define a budget that spans all of the tools that we're using. [55:00] As an engineer, I have a budget of $50 a day. And the policies are a lot more subtle. So here you can see that Codex actually was able to warn me that I'm nearing a limit on my smart budget policy. So what does that look like? These are not as simple as hard caps. Instead, they're far more subtle. Since your developers are being extremely productive with AI, you probably don't want to stop them. And so when they start their day, you can use any coding tool and any model that's available to you, because all of these are available through the gateway. But as you get closer to the top of your limit, we might start guiding you towards a more token efficient model like Databricks Kimmy. which comes out this week and is significantly cheaper than all the other models. So that's great. But then I can also go when I get to 100 percent, I might actually start enforcing that that happens. Now, my session here continues just as normal, because I don't want to get in the way of active But if I was to, for example, start something new, I can type the same thing, gateway codex, And this time what would happen is that it's aware of our real-time budgets, because our budgets are burning down in real time. And so we can take a look at this and it's automatically recommending that we switch the more token efficient model. So this is awesome and it's a lot better than what we've already got, but can we do any better? And the answer is yes, because we have Omnigint. And when Omnigint can connect to the Unity AI gateway, we can do a lot more sophisticated stuff. Specifically, when you have a meta harness, you can make decisions at the higher level than an individual task, which allows you to make intelligent routing decisions. So here we see the intelligent router when Omnigent connects through the gateway. And for that same query that I had before, which was relatively simple, What happens is that when I start running it, the intelligent model router makes a decision. And it's like, hey, this is actually a pretty simple local recommendation. I can use a cheap model for this. And it has this great advantage that it has really great latency when it answers my question. But if I was to do something slightly more complex, like in the Omnigen open source repo, Find the top four bugs in the repo. Now because Omnigen is a meta-harness, it's able to take a look at that work and distribute it to other coding agents. And so for that distribution task, it actually picked a simpler medium quality model. But for each of the individual things it's doing, it's able to make a more sophisticated decision. So for example, for a top level scan of the repository, which is very token expensive, It picked a cheaper model like Databricks Kimmy For an edge case review, it picked Gemini 3.5 Flash, which is extremely intelligent but cheap. And for core logic reviews and deep audits, it picked medium and expensive models because their quality is extremely important. Now all of these things get kicked off in the Databricks sandbox so they can work in parallel while still maintaining enterprise-grade security. So I can click into any individual one of these agents, like for example, the surface scan, And look at how fast Kimmy is running. Latency is another great advantage of being able to pick a cheaper model for the task. And so this kind of work can get done much faster, which frees you up to continue your work and work on something else. All right, so that's cool, and I'm glad that we were able to do this. It's important to have a system like this in place so that you're having those intelligent routing decisions being made. But how do you know if that system is working? Well, this is where agent tracing is incredibly impactful. Because without data, we can't be data driven. And today, coding agent sessions aren't centralized to a single place. And so that's where, again, the Unity AI Gateway, when you connect a coding tool to it, can automatically route those traces to Unity Catalog. Now, this is some of the most sensitive data in your organization. because it is extremely sensitive. Your CEO might have access to context that you don't have access to, so you don't want just anyone to be looking at it. That's why when it's in Unity Catalog, you can use the same tools to govern that data as you already do for your most sensitive data in Databricks. Things like ABAC policies and governed tags mean that only the right folks have access to it. All of this goes into a central trace table that's available in the Unity AI gateway. But another advantage of data being in a central place is that you can use Genie 1 to analyze it. So before we rolled out the router, for example, within our own engineering team, I was able to take a look at a sample of our coding traces. and assess that actually because our default model was the most expensive one, like Opus, 90% of our sessions were overkill. That's already way more data than I would have had otherwise. Otherwise, I would have just been guessing. And the nice thing about that data being in one place is I can also then look at what kinds of things are going on in there. Why was it overkill? Well, you know, for example, was it just people saying hi or hello? Was it paved path tasks that are actually not so complex? This can help you guide what skills you need to build as well, Those skills can also be put in Unity Catalog's AI gateway [60:00] and that will allow you to be more token efficient with the things that your developers and engineers are doing on a day-to-day basis. And it also gives you that full level of observability. So where does this leave us? We hope that with the Unity AI gateways, budget policies, intelligent model router, and tracing, we can start to bend that exponential curve. and still be aggressive about deploying AI within our organizations, but be more mindful about exactly which models we're using. Because intelligence is one of the most precious resources that we have in the ecosystem. And so hopefully we can take the world from the old world of token maxing to something a little bit more like value maxing. with the Unity AI gateway. That's all I have. Thank you very much. And back to Casey to wrap up Agent Bricks. Thank you. Thank you Ankit for that amazing demo. So as you can see, Agent Bricks is our complete end-to-end agent developer platform. We believe it's important to provide choice so you have flexibility to always remain at the cutting edge without having to replatform your organization. solutions. We provide you context so that your agents have access to the right metadata to grab the right data for all of your agent tasks. and And as you saw with Unity AI Gateway, we provide controls so that you can run these agents autonomously in your enterprise, and also that Databricks Sandbox so that you can have it securely execute on downscoped data. While I did show you a lot of features today, we didn't go through everything that we've launched in the last year, and so here's a list of all the many other features that we've implemented on Agent Bricks today. Thank you. Again, Agent Bricks is our agent developer platform. It is uniquely integrated into our data platform, which allows us to provide capabilities that no one else can. You saw some of that with our intelligent routing with Unity AI Gateway. You'll see it with Databricks Sandbox when you get to play with that, and you'll see it with our contextual security policies as well. Thank you for all the things that you have built on Agent Brick so far and all the feedback that you've given us. It's really helped us build an amazing solution here and we can't wait for what you build next. Now, I'm excited to introduce one of our longest running partners, where we're going to hear from Satya Nadella from Microsoft. Thank you. - Sakya, welcome back. I love this, we do this every year. Our partnership's stronger than ever. You know, things are moving so fast. Since just last year we talked, you know, such a big opportunity ahead. I'm curious, what have you seen in the market? Where are things headed? First of all, Ali, it's fantastic to be back with you. And obviously, thank you for the partnership. The two companies have worked together now close to 10 years. And here we are both innovating and really helping our mutual customers. The thing that I would say, even since we last talked to now, uh, A lot has changed, but I would say the core structure of how customers are evaluating what AI can do for them is changing. So what I mean by that is, instead of just talking about here is the latest model and its capabilities, we are now going from frontier model worship to actually building a frontier ecosystem, building one company at a time. where even us partnering and enabling every enterprise to be able to fully participate at that frontier with their own IP, I think is going to be key. Yeah, that makes a lot of sense. I saw your build keynote and in a sense, it's like the future of the firm. and what will it look like, what kind of IP people you have. And it seems that data is more important than ever. really protecting that data that each firm has. and building IP that you yourself own and protecting that, that's how you're going to compete in the future. Is that sort of how you're thinking about it? That's exactly right. In fact, there are two parts about data, right? One is the context, which both of us deeply care about in terms of building the systems, which are the context systems. In the past, we said, hey, here is the system of record or system of engagement. But now we are creating those systems of context, which, by the way, will make by definition, even your use of a frontier model token efficient, right? The one way to waste a lot of tokens is not have the best context. we are doing is all about helping companies at that layer. Then the next piece is of course you have this massive data asset and it's not a static asset it's an ever-changing continuous asset. [65:00] Including the agent traces. So, if you can take that data And tune, in fact, like, in fact, one of the things that I love, you were at our CEO Summit and you saw this, is a good 5B model from MAI is able to beat frontier class evals. Because why? Not because of some magic of ours. One, it's a good lineage model that was, you know, good learning model. But the magic is the enterprise data. When you put it into an RLE at the enterprise level and feed it its traces, it beats, you know, a much more, you know, big, large frontier model because the value is coming from your data. So right now a lot of people are token maxing, using frontier models with their context and in fact leaking the traces. I think that this is going to reverse in a hurry. Yeah, makes a lot of sense. It's like this for loop. It just does this random walk going over your data again and again and again, processing the same thing, spending tokens, taking a lot of time. when that context could have been gathered. So we're super excited. So we're really integrated now with the whole Azure platform. So we have Databricks Genie. We integrate it with M365 co-pilot. IQ, I think, is really amazing, what Microsoft is doing with that. Entra, Agent 365. teams. I'm wondering, these deep integrations that we're putting together, how does it change things for customers? No, it's fantastic to see all this, right? Genie inside a Teams, inside a co-pilot. That means I can get to all of that context that you have been able to bring together and right where I'm using it along with, you know, these surfaces. The other thing that I love about it is you also expose the MCP server of Genie in Copilot Studio. So it's available for somebody to build their agent more efficiently with your MCP. And also, of course, the work that we are doing and even the read-write through OneLake and what you are having, that's fantastic. And so this ability to bring our innovation together to essentially power this agentic future where the enterprise feels that they can get the best economics of tokens, they can build their own token capital and this context here, I think is just fantastic to see. So yeah, zooming out, you know, it's been almost a decade together. you know, what will the future look like between our firms? And what's companies going to do out there? with all this AI, what do you foresee in the future, next few years, maybe 10 years is hard to look out? It's fascinating, right? I mean, obviously, we got together in what was, at least at that time, you know, the opportunity for you and for us to sort of rebuild what is cloud-native data tier look like and reimagine that and bring, in some sense, great innovation and great value to customers. I mean, I look, I think the next 10 years for us will be a lot of what we talked about. that if a firm is about the tacit knowledge and learning, in fact, I always say this, which is you can always buy a tool, you can even sometimes outsource a job or a task, but you can never as a firm outsource your learning. So in this new world, what does it mean to build that tacit knowledge that is compounding both human capital and token capital? I think what the two companies are super aligned on in terms of sense of purpose and mission, as well as the technology innovation, is to help every firm accrue that value for themselves. And today they have unbelievable human capital that uses digital systems to build that tacit knowledge. Tomorrow they'll have human capital and token capital that is building that tacit knowledge that really drives enterprise value for them. And that's what I hope to achieve partnering deeply with you. Yeah, that's so well put. Very excited about that future, helping these firms actually figure out how they can the core DNA, what will it look like in the AI era and how can they keep that and actually compete with that in the market. So very excited about this. Thank you so much for the partnership, Satya. Looking forward to another decade together. Absolutely. Thank you so much, Ali. Welcome to the stage, Databricks Director of Product Management, Justin de Brabant. It's a longer walk than it looks. Everybody, I'm Justin. Let's talk about apps. So... We launched apps last year with a mission to democratize data and AI. And as you've seen over the course of the talks yesterday and today, and certainly in your own experiences, we're no longer constrained by the cost of writing code. um Coding agents have proliferated, code is cheap, But the bottleneck is enterprise data. [70:00] Thank you. Enterprise apps are not just code. They need access to the data. And the real challenge in going from prototype to production is governed access to that data. And that's why we built the first enterprise app platform with data security and governance at the foundation. And that's what Databricks apps is. Thank you. And over the last year, you all have been busy building. It's incredible. We have over 5,000 customers using apps. 150,000 apps have been built. That's 6x growth since last year. And it's not just the apps are being built, they're being used by hundreds of thousands of users on a weekly basis, many of them. that weren't using data and AI in their day-to-day before. So... The favorite part about my job is that I get to talk to you all. And every PM will say they love talking to customers, and that's usually true. Obviously there's some conversations that are harder than others, feature requests, support issues, etc. But with apps it's a little bit different. Because when I go into these customer conversations, I get to hear firsthand what you're building. I get to see the excitement of, hey, I built a thing that I didn't know was possible, and I want to show it to you. And it's really cool. So I figured I'd start this talk today by showing you a few examples of some of the awesome things that our customers are building. So first off, Thales. Thales is a multinational aerospace company building software and avionics, for global airlines. Dallas reinvented their in-flight intelligence platform. bringing together system health, content engagement, passenger satisfaction across the entire global fleet. And with this app, the Thalys Ops team is able to anticipate cabin issues before passengers notice. ultimately improving the passenger experience. And even though This is delivered through an app. It's powered by the best of the Databricks platform. We have Lakebase for real-time data, we have analytics in the lake house, we have Genie, we have machine learning models powering the forecast. It's everything brought together in one cohesive app. Or finally, Jovi. Joby is a manufacturer for all electric air vehicles. And behind every single air vehicle that they build, is an enormous material planning complexity challenge. They built an app that provides a single unified view for demand, inventory, parts, and orders across every manufacturing program in their fleet. So planners can spot shortages. before they become manufacturing problems and eventually delays. There's a theme of airlines, aircraft, air industry. I want to call out that what I think is cool here Is... These are all very, very different applications. Different teams, different applications. And this is what you see across all of our customers. Everybody is really innovating and doing different things. across all their different industries. So how does the platform deliver that? We have three unique buckets of capabilities. First, we help you govern. We provide governed access to enterprise data, Host, we provide infrastructure for every application across the enterprise. And finally, build. We provide you tools and integrations to do AI-first development, and build production-ready apps. Let's double click into each. Thank you. So first, govern. At the core of Databricks governance for apps is our managed auth offering. We have out-of-the-box identity through Databricks SSO. You don't have to have a separate IDP. You don't have to have a separate login for each individual app. It's all managed through Databricks. Thank you. Governance comes out of the box with Unity Catalog. You can use Unity Catalog to manage resources and data that the app has access to, either at the app level, or the individual app user level. So there's a lot of power and configurability. And finally, we now support real-time observability out of the box. So as a platform team, you get real-time insights not only into who's using the app, but what they're doing, what services they're accessing. And this is all ingested into ZeroBus in real time and made available to Unity Managed Tables. Thank you. Host. Thank you. Databricks provides managed containers and infra, secure cloud-agnostic deployments across our stack. When we launched last year, we only had a single always-on instance type for apps. There's one instance. Now, we have configurable instances and horizontal auto-scaling. We also do load balancing and sticky sessions across these multi-instance apps. So no app is too big. And finally, we help you iterate safely. We now have version builds, [75:00] and zero downtime deployments to ensure risk-free releases. And finally, build. Apps is framework agnostic. You can use Python or JavaScript across a range of the most popular app frameworks. You can build production apps faster with AppKit. This is our TypeScript apps SDK. It helps with pre-built plugins for building apps using LakeBase, Genie, Agents, Jobs, Lakehouse, etc. It really makes the hard parts of app building super easy. And finally, it works with coding agents. We've invested a ton with our partners to build custom skills, MCPs, and plugins to make it easier for coding agents to build and deploy Databricks apps. And of course, we're not doing this alone. We're powered by a very strong partner ecosystem, the best AI coding partners out there. So this is exciting, but this is still only the beginning. One billion. That's how many new custom enterprise apps are forecasted by 2028. One billion. Process that. It's huge. And the really interesting aspect about this is that most of them are projected to be built by citizen developers So many of the apps that I just described and the challenges that we addressed were for central teams building these applications teams with access to Databricks resources and the right access to data. It's a completely different set of challenges as we try to democratize innovation of who can build these apps to these citizen developers. So to highlight that, I want to share a quick story. We'll call this the prototype to production gap. This is a real story of a sales ops manager here at Databricks. This person wanted to build an app to streamline their team's QBR process. So as part of that app, they wanted to pull in customer consumption, recent conversations from Salesforce, forecast in our model, and bring it all into one cohesive app. And she built a prototype. She used Replit to Vibecode up a prototype, was super excited about it, showed it a lot with her leadership, thought all the hard work was done. It was not. She went to then to our platform engineering team, and immediately our platform engineering team said, okay, what data do you need? What's the sharing policies, dependencies, code, cost? And it ended up being a lot of back and forth to get the app actually deployed to production Because the sales ops manager obviously didn have access to our production environments with all of our customer information And this isn't about just happening once. This is happening thousands of times across your organization. And I guarantee some of you on the platform team have experienced this. Vibe coding is taking over. And unless we have a governed solution to it, it's going to overrun us. So we spent the last couple quarters going out and talking to you all and asking this question, how do we unlock the next wave of builders without sacrificing security, governance, or budgets? And this is what you told us. First off, we need governance and ops that scale to thousands of apps. The governance model of apps today is largely a single-app basis. How do we scale to many thousands of apps built by hundreds of developers? And today we're excited to announce a new governance construct called AppSpaces. AppSpaces helps you govern and operate thousands of apps through multi-app policies. You can apply governance and security policies at all apps that are deployed in that space. As an admin you can restrict who the apps in that space can be shared with So in my sales ops example maybe those apps can only be shared with the sales team And finally it provides scalable operations How do you track app usage, cost, and other operational metrics for all apps that are deployed in that space? The next challenge that we heard is we need more cost-effective infra for lightweight apps. This idea of apps infrastructure being always on and now auto-scaling is great for big mission-critical apps, but it's not great when you have these lightweight or low-use apps. You know, maybe it's an Ops dashboard that is used a couple times a month. So we need more cost-effective solutions there. And we're excited to announce a completely new apps architecture called Serverless Micro Apps. Yeah, cost. I love it. So, serverless microapps helps you run lightweight apps effectively. It does it through fractional CPU and memory with dynamic scale to zero, which many of you have been asking for. It has fast start-ups, so it starts back up when the app is used on the order of seconds. And even though this is shared infra, it's still with full isolation, so we're providing [80:00] using micro VMs. Amen. And finally, The last thing that we heard, the last major challenge... is that we need Vibe Coding tools that understand your business. Not just your data, but the whole context of where you're deploying these apps. The resources, the environment, the sharing policies, etc. And that's why we are super excited to announce the release of Genie App Builder. Yeah. Genie App Builder is vibe coding that understands your business. It's context-aware. It leverages Gini Ontology to Vibecode with a deep semantic understanding of your business and your data. It's embedded, it's 100% Databricks managed. So it provides the strongest security and governance Guarantees? And And finally, it's production quality by default. It uses the AppKit SDK and custom skills to ensure that high quality apps are out of the box. So all the best practices are baked into all the apps that Genie App Builder creates. Okay, let's see all this in a quick demo. Thank you. So for this demo, I'm going to be a Supply Ops Manager for the Brickster Cup. And what I've done is I've created an app that's going to be my inventory command center. So it's going to help me track inventory levels across various sites, demand, forecast, how that's changing as certain teams advance or not. And then it's going to let me actually take action directly from the app to adjust my inventory levels, trigger new production runs, et cetera. And what I'm going to do is I'm actually going to start from a prototype that I Vibecoded using one of our Vibecoding partners, and I'm going to use Genie App Builder to integrate this into my production environment, connect it to live data, and ultimately deploy it. Okay? So let's get going. Okay, so here we are in the Genie App Builder homepage. You can see the kind of prompt-based text up here. Here we have a set of templates. So these are per app space. You can configure custom app templates. These are configured by your admins. If we go up here and look into the app space, we can see the list of app spaces that I have available to me. the amount of apps that are deployed in each, I can go into my supply chain ops app space and see the resources that are allocated to this space. So what data, what computes, what OBO credentials and scopes I can use, how this apps can be shared, any sort of external connectors. So it's all the governance settings for this app space. I'm going to go back out, and we're going to build an app. and we're going to build it in this supply chain ops space. So now as an app builder, I can select the space I'm trying to build in, and I have all the resources available to me in that space. Now, like I said before, we're actually going to start from an existing prototype that I've Vibecoded. This is available in this GitHub repo here, so I'm just going to import this. It will attach this as context into Genie App Builder. I could also add additional context. So down here, if I had documents or screenshots of prototypes, I could also attach those. For simplicity, I'm just going to use this GitHub prototype. and say, let's build an app based on this. Okay? So now we're going to kick off the Builder UI. Thank you. Okay, so as this populates, you can see the prototype app that I have over here on the right. There's no live data attached to it. The layout, it has my inventory, recommended actions, demand across my different sites. And I'm going to go ahead and say, let's connect this to my Databricks environment. So now what Genie App Builder is doing is it's going through all the resources that I have available to me in the app space, And it's attaching those to the various components of the app. So I found the SQL warehouse, the serving endpoint that I need for my inventory forecast, lake-based instance for the real-time inventory data. It's also looking at the data that I need. So I now have my warehouse inventory, And you can see it's actually, over here on the right, it's starting to propagate this data in real time. So now the app preview is being propagated with the real data. Over here I have a Genie agent and now it's gone into a set of default questions for this Genie agent. And over here you can see I have a slight issue. So this is my production manufacturing data. It looks like I don't have access to this in this space, so I can go over here and actually request access to the admin. This isn't going to happen immediately. The admin needs to approve this. And you can see that I need this data for this chart here. So what I'm going to do is I'm going to remove this manufacturing data. And what it should do, there we go, what it should do is now it's removed [85:00] The graph and the manufacturing data from this. And I can go up here and we also have this preview capability so I can actually see the current. and the proposed preview. And what's actually happening under the scenes is we're actually using these serverless micro VMs to spin up multiple instances of the app and visually preview them side by side. I'm going to go ahead. This looks fine. I'm going to approve the changes. I don't need this data for the V1 of the app. I can also, it's not just about the integrating the data, I can also go up here and makes simple visual changes. So here, Let's just say-- Let's turn this graph green. There we go. Should update. All right. So now the graph is green. And then what I can do is let's test the boundaries of the app space. So what I'll do is I'm going to ask Genie App Builder to send me a Slack summary based on the recommendations from this agent. So let's see. Custom agent, and I essentially want a daily summary here. Now, in order to do this, I need a Slack connector configured at the space. And unfortunately, it's not available. So this is Genie App Builder respecting the governance boundaries of the app space that I've set up. Same thing, let's check the sharing policies. So let's say we can share this app with a Everybody. at my company Probably not the best thing to do. And yes, this app space only allows sharing apps with the supply ops group. So Genia app builder is enforcing the governance boundaries of the space that it's being deployed in. I can also go up here and see the inventory that is attached to this app, so what the app is actually using, and also what else is available in this app space to me that the app is not using, and you can swap out these. as necessary. And then I can quickly go up here and press deploy. And now you can see this app is deploying. And I now have this app deployed in production. And it spun it fast, on the order of seconds. So that's it. That's how we go from... Thank you. That's how we go from prototype to production using Genie App Builder. Um... All of what I just showed is available today, so talk to your account teams for how to get access. So, All right. The other thing that if you want to get started to check out is we have a new developer hub. Developers.databricks.com. Check it out. There's prototypes. prompts to get going, we have a bunch of sample apps, how-to guides, It's really going to help you kind of get going and get the wheels turning with ops. Thank you. So hopefully I inspired you, showed you some customers, building some really cool things. On the note of customers building cool things, let's take a look at a video from one of our customers at the Virtue Foundation and what they're building with Genie. Virtue Foundation is an NGO. The mission is to deliver clinical care to those in need in low and middle income countries. GNI has allowed us to build a custom agent that's specific to us. The agent has the skills and the tools that utilizes your own proprietary data and fundamentally you begin to see insights that you couldn't see before. Built off my traveled and worked for years. And this becomes data we can all action upon because it becomes a source of truth. I am most excited about the maturation of this agentic capability. Ask a question, give me an answer. How many hospitals are in Ghana? There's my answer. How many hospitals do cardiology in Ghana? There's my answer. Using Gini has allowed us to take what would have taken a research assistant months. We're now able to get answers to questions in seconds. the most important part of this is that this is connected back to real human lives. The fact that We're solving real problems for real people's lives. is just a humbling experience. Welcome to the stage, Databricks Director of Product Management, Mike Del Balso. All right. Justin was right, this is a deceptively long walk. Uh... Hey, everyone. So my whole career, I've actually been working in production machine learning. I started out at Google working on the teams, the ads machine learning teams, that actually kind of kicked off the whole MLOps space. Then I helped start the machine learning team at Uber and helped build and lead the ML platform there called Michelangelo. [90:00] And for the past few years, I've been running my own production machine learning startup which recently joined Databricks, And we've been working on those same problems here at Databricks. And the whole time, my whole career... the whole machine learning industry has had one goal, in a single dream, which is to make machine learning as easy and automated as possible, so we can bring intelligent decisions to all of the important places, all the important decisions across our business. And many companies, many have tried and many have failed. And I'm really excited to talk to you today because... That's finally changing. So this diagram here, many of you may recognize this, this is from a paper that my team at Google published something like 12 years ago. And the point of this paper is to really explain how hard it is to do production machine learning. You know, this diagram shows, if you look at the red, that's the machine learning code. And everything around it is the point of the diagram. All these other things around it are all these other systems you practically need to build... and run and maintain to run true ML systems in production. It's a really hard problem. And as a hint, this is all a data problem under the hood. And why are we talking about this right now? Well, we're building more software than ever. And you may have thought that Gen AI would reduce the demand in machine learning. but it's actually only accelerated it. with all of the new software and all the new product experiences we're building... We have more products to recommend. We have more prices to offer. We have more fraud to detect. more loans applications to decide on. So we have a lot more decisions to make in the company. And the problem is that it's actually really hard to do this. And so we only use, we ration machine learning. We only use it in a handful of places, right? Think in your company, how many models you have in production, compared to All of the other decisions that you make in the company that could be automated. And because we're not using... intelligent machine learning there, We're leaving money on the table. We're basically making dumber decisions. We're using heuristics and rules of thumb and statically coded logic. And what we really want to do... is automate... Machine learning so we can... make it really easy to scale to all the important decisions, we have in the company, and keep up with all the scale of the new software, all the new decisions we're going to add as we deploy all this new software. And we just think back to that diagram I showed you. It's a really hard problem technically... But also, you know, to automate this, You know, there's people in the loop in every single stage. There's a person making... There's an expert, there's a data scientist, someone with a PhD making a decision about which feature to build, which model looks good, what are we really optimizing for? Which model should we promote to production? So if we're gonna automate machine learning, We need to do a lot to accelerate these folks. and enable the automation to make decisions in the same way they do. And there's actually only one technology that can do this. And that's agents. Now agents have done an immense amount to completely revolutionize how we build. Software. All of software engineering. But they haven't had that same impact with machine learning yet. We're still kind of building and operating machine learning in the same way. uh Now why is that? Well, there's two reasons. One, is only a handful of companies even have the right infrastructure. It was a really hard problem. You saw the diagram of all the stuff you got to build. to run this complete machine learning in an automated way. The second problem, is that if we want agents or software to automate and make the same decisions that our machine learning engineers are making, They need to have all the same type of information so they can make the same quality of decisions. We can really automate that. And that context is just not available today to our agents. And I'm really excited because Databricks is the first company that can solve this. And we're doing this in two ways. The first is with agent-ready machine learning infrastructure. So getting that infrastructure into a place that your agents can actually drive throughout that whole loop. Thank you. Second, of course, We need the agents that can drive the loop. with your context, to do it in the same way that your teams would do. So let's look at both of these, right? Agent-ready machine learning infrastructure, If you're here, you're probably building... your ML models on Databricks. Databricks has been at this for more than a decade now. More than 500 million models have been trained on Databricks. in just the past year alone. And so Databricks already has a complete... leading platform for machine learning. And that's why Databricks is recognized as the leader in data science and ML. Databricks is the dot [95:00] And the... The highest up, most top, and most to the right. And so that's why you're probably building your ML models on Databricks today. So what's the problem? Well, the problem is that we built all of this and we solved all of these problems for humans today. to build ML. But now we're going to have agents do it for us. And that's what we've spent the past 18 months optimizing for at Databricks. We made the stack agent ready. So what do I mean by that? First thing, we made everything serverless. We don't want agents to have to configure clusters and maintain cluster state and stuff like that. Everything is serverless out of the box, easy for an agent to spin up and spin down. and scale to whatever scale they need. Secondly... This whole data loop, this whole machine learning life cycle is now instrumented with a single... data model. So your training data. your features, your serving data, your monitoring data, your inference, They all land in the same governed tables. That allows your agents to immediately see the impact of any changes they make. and have a single source of truth for the entire machine learning lifecycle. making it much easier for them to optimize and drive this loop And thirdly, everything is governed by default. So you probably already know that we govern all the machine learning assets. So your models, your experiments, your serving endpoints, all of that's already governed in Unity Catalog today. Yeah. But we now have governance on the actions... that your agents can take so that you can trust any changes that your agents are making... to automate this machine learning loop. And I wanna, I'm really excited because we're extending this further today. We are launching a new compute type in Databricks that makes it really easy to build deep learning models and LLMs And that is called AI Runtime. Thank you. This is serverless GPUs available on demand at whatever scale you need for no upfront commit. You can use this to build deep learning models and LLMs And you know, we've actually had this in preview for the past two months, and we've seen huge adoption from it. There's hundreds of companies... that have already moved their deep learning workloads to this platform and are finding huge success. Yesterday, Ali mentioned... Merck's latest drug discovery models, and we've been really fortunate to partner with them and help them build on this platform. So I wanna announce one more thing, which is we're expanding it today to support multi-node training. So now you can build these models at any scale you need on any size of data. We're really excited about this because this brings a whole new domain of machine learning within the Databricks. platform and there's no reason that you shouldn't be bringing this to your deep learning teams today. Yeah. Thank you. Okay, so that was the infrastructure... - Yes. Been talking about infrastructure for a long time, right? Let's talk about the agents to actually drive this loop, right? Getting to a more interesting thing here. Agents that drive the loop, it's important for them to do it with your context in the same way that your team would actually build machine learning, right? Let's talk about the first part here. It's building. . So this is building the models, evaluating them, etc. Extremely excited also to announce Genie Code for Machine Learning. This is an enhanced set of capabilities and intelligence for real world machine learning tasks built into GenieCode, and it's available today. Nobody has seen more machine learning in the real world than Databricks. So we've taken all of those lessons. what works, what doesn't work, what good looks like in production, and we've built that into GenieCode. Genie code is also natively integrated with the complete machine learning stack. So it knows how to use... All of those components, all that infrastructure I just showed better than any other coding agent. And what I'm most excited about is we've upgraded the context in GenieCode. So it now knows... how your machine learning teams and how your data science teams actually work. so it can make recommendations and do work alongside your teams. And I'll show you exactly what I mean by that. So you remember this diagram? This diagram's a little bit deceptive because, as I mentioned, this is from like a decade ago, and that was the problem we were thinking about at that time. That was solving the human problem. But when people are making decisions and driving that whole loop, They're actually... Drawing on many other sources of information. [100:00] to make these decisions, right? What experiments did we run already? What does this business metric mean? What worked, what didn't work? Where do we keep that evaluation script? And all this context is... It is scattered across our whole platform. And if you don't have access to this data, if your agent doesn't have access to this data, It's going to have... It's going to use the wrong data. It's going to use the wrong evaluation. And it's going to tell you something that kind of looks right. But it won't be right. and you won't be able to trust it. So what we've done... Yesterday, Ken announced Genie Ontology. We've built all of this into, integrated it with Gini Ontology and empowered Gini code with Ontology. So all of that context is built into GenieCode, and this allows GenieCode to know how your machine learning teams work and how they Build models, evaluate models. Now let me show you. specifically what I mean. Right? This allows Genie code-- To do things like building features... in the same way that your team builds features today using the exact same patterns It allows Genie code to reuse the training scripts and understand What are our training workflows? How do we actually build models? What's our existing pipeline? And reuse that so it actually fits in with the workflows that your team already has. But it's not just about building. It's also about bringing these models to production and figuring out what are the better models. And so Genie code also respects the evals you have. It finds the eval scripts that you have. And it reuses those. So in Genie code... says some a model is good you can actually trust it because you know it's holding a model to the same bar that your team already trusts. And that allows Genie code to really act like a senior machine learning engineer on your team. We found that GenieCode performs way better in real world data science and machine learning tasks, than other leading coding agents now. And this allows companies like Danfoss, a multinational engineering company, To go from no machine learning, to A complete... production machine learning system in just 90 minutes. And this is a type of project that would take weeks, months. And I know for a lot of companies, a lot of customers in this room, sometimes it takes quarters, right? It's really hard to do this. This is a gigantic speed-up. And so we're really excited about that, and that's just the build phase here. But the bad news is that the build phase, if you talk to a real world ML team, building is not the majority of their job. They're actually spending 60, 70, 80% of their time on maintaining... their machine learning applications. It's just part of the problem. And if we're building models 100 times faster, We're going to be building 100 times more models. And so we can't... take on 100 times the maintenance burden. No team can scale to that. And so that's why we need to think about the operating side of this loop as well, right? After a model's in production, How do we keep it healthy? Thank you. And you know, anything can go wrong with machine learning models. Data can drift. The model server, the inference server, can have latency spikes. There can be any number of reasons for errors. Upstream data pipelines can break. There's a long tail of a thousand issues. And to solve this and automate this... Something I'm really excited about, and I think it's the biggest launch in machine learning in the past 10 years. I'm very excited to introduce... Gini's your ops for machine learning. This is an agent that runs your machine learning systems for you. So there's a really big paradigm shift with this. Instead of typical monitoring systems that, you know, are looking at your metrics and when they find an issue, they bring you a problem and say, hey, this thing looks wrong. Zero ops for ML doesn't stop there. It finds the problems... And then it takes it upon itself to discover the root cause Come up with a fix autonomously and bring you the solution. So instead of getting problems given to you all the time, you're getting solutions brought to you. And so it monitors every model continuously, it investigates these issues and resolves them autonomously, but it still leaves your team in the loop to approve before anything deploys. And the flip here is that for the whole history of machine learning, we've always had a team member on call for our models. And zero ops for ML inverts that and allows an agent to be on call and your team is just in a position to approve the recommendations from that agent. [105:00] That allows... Jeanne is your option for ML to solve all of these kinds of problems we just talked about. while still leaving your team in the loop just for the final approval. And so rather than me showing you a bunch of more slides about this, I'd like to invite Amber on stage to actually show us how it works live. Amber. Thank you. Thank you. Thank you. Thanks, Mike, for setting the stage, and hello, everyone. My name is Amber, and I'm here to show you how Genie Zero Ops not only finds issues with your ML systems, but fixes them automatically. At Bricksports Global, my team and I have a model in production that forecasts ticket sales for every game of the Data and AI World Cup. And this model is mission critical because that forecast affects tens of millions of dollars in event staffing and operations spend. And we don't want to end up with a lot of empty seats because we overpriced tickets. With this model, I need to be alerted as soon as any issues happen. So I'm going to quickly use Genie code to help create some alerts. I'm going to create an alert. Okay. And I'm going to ask Genie Code to create an alert that will trigger if observed ticket sales are not within plus or minus 5% of the forecasted sales for any game. Here we can see how Genie code builds and registers the agent for me automatically. No extra setup needed. All right, I'll allow this alert creation here. And the alerts created. Now let's fast forward to the end of the week. It's Friday at 5:30 p.m. I'm coming out of back-to-back meetings. I open up my email and crap. That alert was fired 30 minutes ago and the ticket forecast has suddenly become less accurate. This is a critical issue where every minute could cost my company millions of dollars. So I need to just drop everything and debug what's going on. But wait, there's this new button. View investigation in Genie Zero Ops? That sounds promising. Let's see what it does. When I tap it, it opens up the Genie Zero Ops inbox, where we can see an open thread with the highest severity. So I'm just gonna go and click into that in order to see the investigation that GenieCode started automatically about 30 minutes ago. So instead of waiting for me to investigate this issue manually, GenieCode got started immediately and has already explored and tested dozens of hypotheses. That's amazing for me because these issues are costly and every second counts. see what it did. First, GenieCode tries to understand what triggered the alert, and this is where GenieCode's context and ontology shines. Most agents would provide a surface-level generic suggestion that still requires me to do a majority of the investigation myself. But Genie code goes deeper. It's powered by the ontology of my full ML project, so it analyzes notebooks that built the model. It looks at unity catalog tables. It runs SQL queries. It's checking for changes in the feature values. It's running MLflow experiments and more. Overall, it really approached the problem like a real ML expert, continuously iterating until it found the root cause. And my favorite thing is all this is done in the background. Work that would have taken me hours to do manually, GD Code root caused it before I even realized there was an alert. And it looks like it figured out the underlying issue. our model is missing a feature that conveys the stage of the tournament. This means that a model was unable to account for the increased demand and fan detention that the knockout rounds receive. So as soon as the tournament entered the knockout stage, our forecast became less accurate, as you can see here. And then it didn't stop there. After figuring out this issue, Genie code went even further. It started building and testing for the solution it figured out that we had existing data for both the game's tournament stage and matchup, and then it created features based on that underutilized data. Genie code then decided to retrain a model with these features, did hyper parameter tuning registered the model and MLflow all while following best practices for ML on Databricks. So That's awesome, and it essentially solved the problem for me automatically. And since Genie code has full lineage across our ML workflow, it reused the same evaluation criteria from the previous model to confirm the new model's accuracy. And it gets our forecast back to where it needs to be. Now it's time to roll out this new model to a small amount of traffic, and I'm feeling really good because I can trust what Genico did because I saw the evaluations it used. Now I can deploy this model in one click. Let's do it. [110:00] All right. See version two. And I'm done. Now, normally a stale model like this is a major team-wide incident that massively affects our business for hours, if not days. potentially costing tens of millions of dollars. But Genicode Zero Ops resolved it quickly and with minimal disruption to my team. And as Mike mentioned earlier, we're entering the 100x era of ML. But with a hundred times more models comes a hundred times the complexity. And that's really what we have to protect teams from. And that's really the opportunity here. Give ML teams the tools that they need to stop spending all this time fighting operational toil and start building breakthrough AI. And that's Genie ZeroOps for ML. Thank you. Okay, pretty cool, right? going to have to jog here. Yeah. Okay, so we saw three things today. The first is AI runtime and public preview serverless GPUs, you can build your deep learning models and your LLMs with it. Go try it now. Second, G&E code for machine learning is generally available It's built into the Genie code you have right now on your laptop, so you can start using it today. And third, Genie Zero Ops for machine learning. It's coming this summer, so we're really excited to work with you with that. at that time. So think back to the dream I started this talk with, the machine learning, Industry has always been about automating machine learning, and we're finally at the point where we have the infrastructure and we have the agents to drive the loop. And that allows us to build machine learning 100 times faster 100 times more models. And we're entering this era now where we can scale ML to all of the decisions across our business so we can have truly personalized customer experiences at every part of our product and we can really optimize every business decision. So we're really excited to see what you build. Let, Your agents do the work now. So your team can focus on what's next. So thank you very much. Okay. Thank you. So I'd like to introduce, speaking of what's next, the next speaker, He's the GM for Customer Lake, and he's here to tell us about the future of customer data and marketing, - Tassos Argyros. Good morning. How's everyone doing? Woo! Okay. Um... It's a real pleasure to be here with you today. This is actually very, very special for me. I'll tell you why. um... I spent the last 20 years building two companies. I spent 10 years building a database company, But then I really wanted to do something that would bring the power of data to the business. So I spent another decade building a marketing technology company. One that would combine data but also combine a UI for the business. That was my passion. And it's really special to be here because in some ways I'm back at my roots. but what I'm about to show you is something that truly brings the power of data and agents Do that business. But before we talk about this, I want to talk to you about customer experience. Why do we talk about it? The best companies differentiate on customer experience, 80%. say that they compete primarily on the basis of the experiences they create for their customers. And... We live in a very interesting time. Because AI is dramatically changing how people buy and as a result how those experiences need to be designed. Why is that happening? First of all, humans do not buy on their own anymore. How many of you do you consult with an LLM before you buy anything, be honest. Okay, I think this is like 80%. Very soon, it's not just going to be advice from a chatbot, it's going to be delegating agents to do all the research. all the price discovery, all the buying, figure out shipping, Everything will happen through agents. This means that buying decisions are now happening in seconds or minutes. Not days or weeks, which was a buyer journey until now. And finally, because [115:00] - Commerce is going headless. and agents are really good with APIs, Agents can be on multiple channels buying, researching, all at the same time. So this creates new challenges, right, to how things were done before. Because the way things were done before is very simple. Someone in a company, usually marketing, would start with the universe of customers, create segments, we call them audiences, that would have millions of people and then activate those audiences... through campaigns. And the way these campaigns would be designed to begin with was through this big waterfall model, right? So marketing would do the strategy, Data engineering would put the Customer 360 data together. Marketing ops will create the audiences. Creative would do content, marketers would activate measure, right, and monitor the campaign as they were running. This waterfall worked, but it had some issues, right? First of all, it takes a really long time and it's very rigid. It's hard to change dynamically. So, for all these limitations though, these waterfall campaigns worked fine up to now. But... Do they work for the era of agentic buying? And the answer is no, right? Of course not. When agents buy on multiple channels at the same time, in seconds, You can't spend weeks designing a single campaign. But what if... What if we could personalize every interaction for every customer in real time all the time? for their customers and their agents. That would be a game changer. But to do that, we need to think differently. We need to invent a new type of campaign, one that doesn't rely on this waterfall model, and one that can personalize audiences of one instead of audiences of millions. What if we could do this with agents, right? So agents create the new challenge on the buyer side. What if agents are also the solutions? What if we could have an agent for every customer? And then what its agent would do is to process the latest signals, from every single customer, determine what's the next best action and create a highly personalized content just for that individual customer. And from there on, that would go back to the customer. The customer would process, transact, and then create more signals that would then feed back to the agent. in this infinity-like loop. We think this is the future and this is such a new approach. to how marketing can do campaigns. that he deserved a new name. We call it infinity campaigns. And we call it infinity, not because the symbol looks like an infinity symbol, But importantly, these are campaigns that are evergreen. that don't need to have a start and an end. Because we have agents, the agents continuously adapt, so the same campaign can adapt to new customer behaviors, new business goals, and it can keep running. Also, infinity campaigns can provide content that's highly personalized, right? You can pick from and multitude infinitely large number of variations of content for what each individual customer needs that particular moment. And finally, if you employ agents, they can respond very, very fast. You don't have any limits in how fast these agents can respond. Now, it's important to note that human remains in control. The marketer is the chief strategist the chief orchestrator, and the chief approver. of what the agents are doing. But now they have an army of agents that can do all the one-to-one personalization work that humans were just too slow to do before. Okay, so hopefully I'm convincing you that this new concept of Infinity Campaign is It's very cool. But the question is, can we make it work? How do we make this happen? And the way... this would have happened in the past, was through CDPs. And let me talk a little bit about what CDPs are. For those of you that don't know, CDPs stands for Customer Data Platform. And it's essentially a business... application. that sits between the data and all the marketing tools, the customer tools that actually send the communications to the customers. So what do CDPs do? First, they can turn raw data into Customer 360, organized data ready for business use. Then they have a business UI where marketers can come in and build audiences and journeys. [120:00] And finally, they send data to marketing tools that serve content to the end customer, like email tools or advertising platforms. And that process is called reverse ETL, because the data goes from the CDP into the content or the advertising platform. The challenge here is that CDPs were middleware, so they were detached from the data. And it's not just about data anymore. Now, in order to make agents work, you need data, but you also need context, and you need data and context to be where the agents live. So we think you need a new generation of CDPs to make infinity campaigns work, and we call this new generation the agentic CDP. The agentic CDP is not just some general concept. There's three very specific things that a CDP needs to do in order to be agentic. First, it has to be embedded in the lake house, because that's where the data, the context, and the agents increasingly live. Second, it has to be built with LLMs at the core of its architecture. This is not just about having some co-pilot to help you do some operations better. As we discussed before, the agent is the core of how this new type of CDP works. And finally, it has to power infinity campaigns. It needs to have the capability to have evergreen, real-time, continuously adapting campaigns that personalize every individual customer with individual agents. So, agentic CDP is a new category. We think there's going to be several new players there. However, we are one of the very first to launch an agentic CDP, and we call this Customer Lake. Thank you. Thank you. Customer Lake is an agentic CDP built specifically for the Databricks platform. It's embedded, meaning data never has to leave the lake house, and all the governance for data and agents that you've done with Unity Catalog will be transferred immediately into Customer Lake. It's democratized. It's built for the business users, so it has a business-first UI, and it is autonomous. It can empower one-to-one personalization with agents to provide the perfect customer experience a billion times a day. Customer Lake comes with two modules. The first is Profile Agents, which automates the process of going from raw data to Customer 360. The second is Campaign Agents. That's the business-first product that allows marketers to power Infinity Campaigns with agents. Let's double-click and talk about it a little bit more. There's a lot of capabilities that we're introducing here, and in the remaining of this talk, we're going to talk a little bit more about these two products and also show you how it works. Okay, let's start with Profile Agents. Profile Agents are agents that automate the process of creating Customer 360 data that is business-ready. So with the prompt that defines a use case, Profile Agents get to work. They ingest, transform, enrich data, resolve customer identities, and evaluate profiles to generate this agentic loop that keeps improving quality. The first step is to prepare customer data for use. With LakeFlow Connect, we can ingest third-party data from applications or obviously use LakeHouse data that's already there. Our agents will semantically tag data and also improve quality such as fix emails, addresses, or phone numbers. And we can also bring third-party data in, which we call data hydration, through the Databricks marketplace. The second step is one of the key core features of Profile Agents, which is identity resolution. What is identity resolution? Identity resolution for first-party data, meaning data you already own. This is data you have collected from your own customers. It's the process of identifying the same customer across different data sets that do not share the same customer key. So you have information about the customers. You may have names, emails, addresses that don't quite match, but probabilistically, you can tell it's the same customer. This is very important because if you don't do this, you may end up thinking the same customer is different people, and that leads to massive problems for marketing and customer experience. So we introduce here a new approach to identity resolution that we call the agentic identity resolution. We have agents that come up with rules on how to resolve this identity. [125:00] If these rules are inconclusive, LLMs take over and try to make a determination. And if needed, it falls back to human feedback. And as more and more identities are being identified and resolved, the agents learn. and they enable this agent loop that make the rules better the LLM learning better and the whole process continuously improves. Historically, this has happened through static SQL rules or machine learning. Both approaches have significant limitations. and we think the agentic identity resolution wins on all fronts over these traditional approaches. And when it comes to third-party data, meaning data that comes from third-party partners, We are introducing today a way to resolve identities through those partners without your data ever leaving the lake house. So this is a privacy-first approach. to third party identity resolution, that we enable through the Databricks cleanrooms. And the last step in profile agents, after having ingested, transformed quality check the data. and identify the identities. is to output the right data products for the right use cases. So that involves schema mapping, Survivor shape. and also data evaluation so that the agents can self-improve the quality of the final output I've knitted. So that's profile agents, helping you go from raw data to business-ready customer 360, faster than ever before. So let's talk about our second product here, campaign agents. Campaign agents are agents that automate campaign work. So given a prompt, the agents can create the right campaign brief from business context and business goals The marketers review it and approve it, and then we deploy agents to do personalization. What's cool also is that campaign agents also do traditional CDP functionality like audiencing. But of course, we do it in a way that's Gini-powered, So everything can happen through prompts, but we also have a business UI that marketing and marketing operations can very quickly review the work of the agents I prove it. and with one click you can take audiences and turn them into campaigns. Infinity campaigns is the heart of what campaign agents do. The beauty of Infinity Campaigns, because it's built around LLMs, is that it can start personalizing immediately. You don't have to spend months building ML models. You also don't need to use ML based decisioning. that takes a really long time to start to work. It provides exceptional campaign performance out of the box, And of course, batteries are included. We have guardrails, you can do simulation runs, and measurement. Everything is built in. And for the first time... We're introducing a feature that so many of you have been asking. Reverse ETL, we can now have native managed connectors to multiple products in the advertising, marketing, and customer experience ecosystem, that with one click you can make it work. We're launching with approximately 20 connectors. to a rich ecosystem of partners. We have a dozen more connectors coming by the end of the year. and many more beyond that. Okay, so it's time to see Customer Lake in action, so I want to invite Justin. to give a demo of Customer Lake. Welcome, Justin. Thanks, Tasso. Encore demo. I'm flattered. Yeah. Um... So in this demo, we're going to give a quick overview of campaign agents. The assumption here is that you've already used profile agents to ingest, model, and resolve your customer data. And what I'm going to walk through is using campaign agents, to build audiences using Genie, and then Infinity campaigns to decision on a one-to-one basis. Okay, so let's jump into it. So Now I'm a marketing manager for Bricksport, and here I can see my campaigns dashboard. This is my overview of my active campaigns. key audiences that I'm tracking, my most active reverse ETL syncs, Um... kind of my overview dashboard. Let's jump over here, though, into the audiences. Here's a list of audiences that I've created. I can see the size of these audiences, the percent change over time. What we're going to do is we're just going to create a new audience. And as I do this, I could create this audience using [130:00] Rules and filters. But we're going to use Genie. And I'm going to copy and paste a prompt in here. "Include our most valuable Team USA fans, who have spent more than 500 on merch this season. and exclude anyone who already bought the USA Fan Kit. Okay? So let's see what Genie comes up with. Jeannie's gonna go and actually generate this audience. So it's gonna look at the attributes and the audiences that I already have created. And there we go, Jeannie generated this audience. So now I have an audience of 800,000 customers. I can see the qualification criteria here. If I wanted to, I could go in here and tweak this. Right? editable I'm going to trust Jeannie and go ahead and accept this. I come back out here, and now I have some quick insights that Jeannie generated. We'll come back to that in a second. I actually want to show this. You can view the sequel that is backing this audience. So this is starting to look probably more familiar to many of you in the audience. If you've ever supported marketing by pulling lists and writing custom campaigns, you can get really complex sequel to power some of these campaigns. And we got this from one prompt with Genie. So Let's go ahead and save this, and I'll get brought out to the main Insights dashboard here. And I can see kind of insights on this audience. These are automatically generated by Genie. So dynamically, every audience is getting its own unique dashboard. And then I could immediately use this in a new campaign or even just a new sync. So the point is all these audiences are actionable. This is not just insights. You can immediately run this in a campaign. Now, for the sake of time, I'm going to go over here, and I'm going to find a live Infinity campaign. In this example, it's my in-venue fan experience. So let's quickly look into this running campaign here and see how it's configured. Alright, so I can see the audiences to include, the goals, conversion metrics, any sort of guardrails that I wanted to apply. So this is kind of the constraints that you're putting on the agent. I can see the messages that I made available as part of this Infinity campaign. So here I have a matchday guide and a matchday offer. We'll come back to those in a second. um And the really interesting part is looking at this agents tab. So this is where the agent is actually looking on a one-to-one basis. at each individual customer. So you can see a list of sample profiles here. Here we have Marcus. This agent decided to defer, so not send anything to Marcus right now. Let's take a look at why. We can actually see The reasoning behind this decision from the agent What was evaluated? What was the timing that was considered? What sort of data signals were considered as part of this decision? Any sort of guardrails that were applied, like I said, frequency caps, etc. If we Go into somebody who got the offer here. We see this person got the matchday offer. And same thing. So, Jeannie decided to send this personalized offer, and this is a preview of what they actually got. So this is a 15%. offer off a seed upgrade. Let's find somebody who actually got just the matchday guide. So here we go. If we preview this, we can see this customer. got just a summary of the matchday kickoff, parking information, etc. So, Campaign Agents is personalizing every single one of these messages for each individual customer in this campaign continuously. I can go out and see an overview of the insights on this campaign. I can look at performance by individual messages, opens, clicks, conversions, etc. to get kind of an aggregate view of how this campaign is doing. And with that, I'm back at the homepage. That's a quick run through of Customer Lake campaign agents. The world's first agentic CDP. Thank you. Back to Utah. Thank you, Justin. So the... The area of true one-to-one personalization It's here. It's very, very, very exciting. So in developing Customer Lake, we were very fortunate and honored to work with some of the top innovators globally. some of which were kind enough to let us put their logos up on the screen. These are world-class data and marketing teams. that are pushing the boundaries of customer experience with data, agents and marketing strategy. Thank you to all our early Customer Lake partners. And similarly, We have been building Customer Lake surrounded by an elite ecosystem of partners. that jointly serve our customers in terms of data, content, advertising, activation, and services. Thank you to all of our partners that are joining us today for this launch. [135:00] So with that, I'm excited to say that Customer Lake is in private preview today. If you're interested, please talk to your account team. Thank you. Please talk to your account team. if you want to learn more. We also have a brand new CDP track In the breakouts... And we also have a marketing forum today at 1 p.m. You can find both Search for Customer Lake and in your Data and AI Summit app. And you will see both the marketing forum and the breakout session. That's the place to learn more. Some of the breakouts is from the teams, the engineers, the product folks that actually built customer lake and we also have some of our early customers talking about their first experiences with Customer Lake as well. Make sure to check it out. So with that, thank you for your time. It's a very special moment for me to be able to bring together the worlds of business, customer experience, and data. all in one place, to build something that provides the perfect customer experience. a billion times a day. Thank you. Thank you. And now, Andrew Tu, GM of Lake Watts, to talk about Lake Watts. Thank you. Bye. Thank you. Hello everyone, I'm Andrew. I'm excited to talk to you today about Lakewatch, which is our new security Lakehouse product. We announced this just about two months ago at the RSA conference. And this is an area that's really strategic for us. You heard Ali share just yesterday the acquisition of Panther Labs. which is joining us to bolster the LakeWatch effort. And you'll hear right after me conversation with Panthers CEO Jack, together with our co-founder, So you'll learn a lot more about that soon. But I want to start with why. We've probably all read about the latest LLM models being able to find previously unknown vulnerabilities in just about any piece of software out there today. whether it's open source or closed source, what used to take nation-state level capabilities You had to have teams of reverse engineers and hackers working for months or years to find these vulnerabilities Now, anyone with access to the right LLM. can find these things instantly. And what's more is AI is actually automating the entire attack process too. We're seeing this happening in reality today. Anthropic reported recently that hackers were using Claude to attack the Mexican government. And you can actually read the entire chat log. It's really fascinating. You can hear-- you can see the hackers ask the agent to go and search for vulnerabilities in a range of servers It finds some. They ask it to attack one of the servers. It's able to break in. Then they ask the agent to escalate privileges to root access. and the agent finds a vulnerability in a cron tab entry and gains root access. They ask it to go and search for sensitive data. And in the end, the hackers walk away with, 200 million personnel records, and gigabytes of sensitive data. So now just about anyone with access to an LLM can put on an attack at a scale and speed that was previously unimaginable. That's the world we live in. And the only way that we're going to be successful in defending against this is if security and data teams work together. Let me explain why. To start, most companies today have a security operations team. that's tasked with defending them. And this team does three things. first. They... ingest a bunch of data from a wide range of data sources. Think employee laptops, servers, identity and access systems. All of that telemetry data, log data comes into a central server. Then they write detection rules on top of that data. So these are queries for potentially malicious behavior. So for example, if I were to log in from San Francisco today, and then a couple minutes later log in halfway around the world, That might be a sign that my machine was compromised. Or maybe it's just a sign I was debugging. So finally, all of those alerts go to an incident response team, which is a team of analysts [140:00] that are looking through every one of those and investigating whether this is a truly a threat or just a false positive. If it's a real threat, they can take some remediating action, like blocking my IP address or forcing me to reset my passwords. So that's what these teams are doing. The challenge is the tools they're using are outdated. They typically use a system called a SIM, a security information event management tool, and these systems don't -- they're not designed for the scale of threat that we're facing today. The first problem is that you can't even get all your data in. This is a mix of technical limitation and pricing problem. So what security teams are doing is they do a number of backflips to reduce the amount of data coming into these SIM tools. For example, they'll throw away data that's older than 30 days. because they can't afford to store years worth of data. Second, they'll run data filtering tools. This. filters down the incoming data, taking away things that are maybe less valuable. Hopefully, you're not missing anything by doing that. And then there's an entire class of data that today's solutions can't even ingest and can't consider. These are things like LLM logs, code, JIRA, Slack messages. This is really valuable context that would help an analyst determine whether Andrew was... using a VPN and maybe debugging a service, or whether indeed my machine was compromised. The second issue is that the process today is incredibly manual. If you look at these incident responders, the analysts who have to determine if a particular case ticket is malicious or not, They were dealing with over 7,000 alerts a week in 2020. That number has gone up to over 30,000 alerts a week for a typical large organization. And it might take them 30 minutes to investigate one of those, to see, you know, what was Andrew doing before this happened? Has he ever logged in from this location before? What did he do after? And so you multiply that out and already you would need a team of over 400 people to meet the number of alerts you're getting today. And this is only going to get harder and harder as we see more alerts agentic attacks happening at larger scale and at faster speeds. So the reality is that with the tools we have today, we don't stand a chance in meeting these agentic threats. We're dealing with incomplete data because we can't afford and we can't scale to bring it in. We're dealing with humans fighting agent attackers. And we're dealing with proprietary data silos where you can't analyze all the data because it's locked up in different locations. The solution here is a big data platform. And that's why we built Lakewatch, the agentic security lake house, This is built on top of the Databricks platform. So all of your data is stored in open formats. We make it very cheap and easy to bring in data in lots of different ways. On top of that, you have Unity Catalog, so you can govern access to the data. ensure compliance, And then we build in Agent Bricks and Genie on top, so you can automate all of these manual processes and workflows And finally, we give you this in a nice UI that security teams are familiar with. and can use easily. And so the result is that you can leverage all of your data, Whether it's structured, unstructured, semi-structured data, anything you need can come into your security lake house. And then we can apply agents at every step of the process. on the ingest path, we can have the agents automatically apply ETL on that data, do data normalization to make it easier to work with. If a schema changes, the agents can automatically correct the ingestion pipeline to ensure you don't have any interruption. in your data that's coming in. On the detection path, the agents can help author can help author the queries and detection rules for you. And on the investigation path, the agents can automatically investigate these incidents so that the analysts are presented with a full case, and they can just verify that the agent did the right task and recommended the right action. And the result of all of this is that we can meet the scale and speed of threats that are coming in today. I'm going to hand it to our tech lead for Lakewatch, Michael Anderson, to give you a demo of the product. Good morning, folks. [145:00] Thank you. So I'm here inside Lakewatch. When you have it enabled, speak to your account team, you'll see it show up here in the app switcher. And Lakewatch is our agentic security lake house, and it's designed for the entire Sim journey. And that begins with data. So, Over here on the data sources side, Databricks is already great at bringing data into your Lakehouse. But Lakewatch adds dozens of security-specific connectors that can bring all of your security data, including multimodal data, into the Lakehouse. for analysis. Once that data lands inside the lake house, We also have tools to normalize that data. into the Open Cybersecurity Schema Framework, or OCSF. In fact, this is a new feature that we've added since our announcement at RSA. There's now an entirely automated Gini-powered normalization pipeline that will analyze your data and construct this entire normalization pipeline for you. And the benefit of having normalized data landing on your gold tables is that you can write your detections once and do your investigation in a single uniform way, even if you have products from multiple clouds and multiple different vendors that have data that is very heterogeneous. Of course, once that data has landed in your leghouse, Lakewatch is also designed for scalable detection. We support... tens of thousands of detection rules running against that data with features for managing those rules at scale such as mitre tagging and mitre coverage analysis So you can really work out exactly where you need to improve your detection coverage. Of course, because I have brought a bunch of data into my lake house, including information about my business and my code and so on, I can actually use Genie to automate the process of creating a detection rule too. This page here, when you come here to create a detection rule, this is something that would typically have taken a stock analyst days, sometimes weeks to do. You have to go out and research a threat, you have to craft a detection, you have to validate it. It's an incredibly time-consuming process. But now, I can actually go in here and I can ask Jeannie, Please research And write me-- ooh. This is how you know it's life. Write me a detection... for Storm2460. Storm2460 is a malware, it's a fairly complex malware that has multiple steps in its attack chain. Genie is going to go out and it's going to kick off a background research agent That's going to go and search the web, look for articles and CVEs and analysis about how this particular tag path works. And in parallel, it's going to go and look at my data. Now it understands my entire data pipeline. It knows I have CrowdStrike connected. It knows that's flowing through my normalization pipeline into my process activity OCSF table. It's going to read that table, look at the schema, look at the data that I have in there, so it knows what it has available to construct this detection. And then it's going to go and combine that. Now it's busy sampling it to make sure that All of the data really actually is there. And the web research has come back, so it's going to combine that with the web research and start to form some hypotheses for how this can be detected. So building out a detection SQL. It's busy checking that I do indeed have the indicators in my data. It looks like I have lots of search utility activity. Fortunately, it's all legitimate search utility, haven't been breached yet. But that's the data that I need to detect this malware. And now it's going to write the SQL. It's tested it over the past 24 hours, no indication of attack. Now it's testing it over the last 30 days to make sure that it isn't getting any false positives. Because often you may get false positives and it would need to iterate on this, refine the detection SQL to make sure that I have a good, trustworthy and valid detection. So now it's come back, the detection is validated, zero false positives, all of the data that this relies upon is present in my environment. And it's now going to take that detection SQL and turn that into a rule which means I've got the metadata that I need to manage this alongside thousands of other rules, and it's going to give me a little bit of a report here. This is how I'm detecting it. It's targeting all seven phases. of this malware. You can see it's pre-populated my detection SQL for me here on the left. And then it's also giving me an overview of the MITRE metadata that it has tagged this rule with. so that I can... sort of manage this in my corpus of detections. So this is a massive acceleration for your SOC team. We've taken something that would have been a week, and it's now executed in a couple of minutes. But the security lake house in Genie doesn't just change how I author my detections, it also changes how I investigate and respond to what those detections produce. If I go over here into my security cases, you will see that this The star here shows that many of these have actually been investigated by Genie in the background. So if I look here, this is an alert from my WAF. There's been a rate-limiting surge, but Genie has filled [150:00] - Not an entire report here, it's gone and found related cases, both a similar pattern attacking the WAF, but also similar sort of activities done by the same IP address. It's gone and looked for these entities in the threat intelligence feed to see if this is a known malicious IP address. It's actually gone and looked inside my perimeter as well to see, is there any indication that this IP has done something past the WAF? Like, did it breach in any way? And it's given me this whole report so that by the time an analyst opens a case, they don't have to repeat the same runbooks and the same repetitive tasks. All of that has been done for them. And in fact, Genie will learn from how analysts have investigated prior cases, so that the next time a similar case comes along, it does the updated analysis on it. All an analyst needs to do is focus on the novel aspects of this case and focus on mitigating it. I can actually show you this in process. If I go back to my cases and I look at informational, you see there's one here that Jeannie has not investigated yet. This is a suspicious API request. This is a very low fidelity signal. The reality is when a SOC team is getting tens of thousands of alerts a week, they don't have time to investigate something. like this, where it's just a statistical differentiation from an established baseline. But with Genie, we now can. I can go in here, I can ask Jeannie to investigate this case, It's going to come in and pull up the event here and look through the event details and start to understand what's going on. Looks like it's a post request. The reason it's suspicious is this read context parameter deviates from the established baseline. And it's now actually going to find the service that provides this endpoint and look through my source code and try and understand what is the significance of this new pattern of request. In parallel, it's also going to check my thread feed to see this IP address. Is this known to be malicious or part of a kind of a known botnet or something like that? It doesn't find anything on that side. So it's going to keep looking through the code and understand what the significance of this is. Thank you. Looks like it's found something here in the code. This readContext parameter has a very specific pattern, and when that pattern is present, there's actually a hard-coded backdoor in the code. It skips all of the access control checks and directly returns the decryption key. So this is... This is an amazing finding. I found this very rapidly. It's actually going to extend its investigation now and try and work out who introduced this into the code. When was it introduced? Does it exist in more places or has this employee introduced other things into the code? And it finds it was committed to, a particular day, the same day that the suspicious request was made, it exists in two places in the code, it's now trying to work out Um... who this person's manager is, and if there's any Jira tickets that explains maybe why they did this, what do they normally work on, And it's looking at the JIRA tickets, and there's no JIRA tickets related to this change. So there's no good explanation for this. So it's busy drafting up a report now. It tells me actually from the enrichment data, I can see this IP address comes from Google Cloud. Here's the exploit, here's the context. I recommend escalating this to high severity and treating this as a true positive. So this is... really a pivotal moment in security. This is the first time in history that my capacity to investigate cases and my capacity to be thorough in investigating every single thing in my infrastructure is no longer bounded or limited by the size of my SOC team. With the power of Genie and the data that Lakewatch lets you bring into your security lake house, We can investigate anything at arbitrary scale without having to have everything being driven by humans sitting in front of their machine. Thank you very much. I'll hand it back to Andrew. Thank you. So what Michael just showed you is how with LakeWatch, you can now analyze all of your data. You're not limited by scale or cost concerns there. you can fight Agents. on the attacking side with agents on the defense. And you can break out of proprietary data silos because all of your data is stored in open formats, in an open ecosystem, and you can load data from anywhere you like. We launched Lakewatch just a couple months ago, and we already have a number of marquee customers using it. Just yesterday we had the Cyber Forum and we heard from Atlassian, Adobe, and NAB. But to give you one example, Adobe wanted to analyze petabytes of data, which was too much for their existing legacy solution, They came to Databricks, loaded all that data, and they're now able to apply AI and ML techniques to [155:00] find previously impossible to find threats. To sum it up, the only way that we can meet today's security threats is by having security and data teams work together. So if you are a data and AI expert, I want to encourage you to reach out to your security colleagues and see how you can help. And if you're on the security side, You should move to using an open security lake house so that you can analyze all of the data and you can apply agents to find the latest threats. Thank you, and I'm going to hand it to Reynolds to tell you more about the Panther acquisition. All right, yesterday, Ali talked about the Databricks' agreed to acquire Panther, a cybersecurity company. providing a leading AI stock platform. And today we have a jack on stage. founder and CEO of Panther with us. So Jack, I think many of the audience here are coming more from a data background, so they probably haven't heard that much about you and Panther. Can you tell us a little bit about your background and what Panther does? Yeah, of course. So I'm Jack, I'm the founder and CEO of Panther. We're a cybersecurity company with the mission of making security teams smarter and faster than attackers. And my background has been security. I've been working in cyber operations for about eight years. And prior to Panther, I was leading detection and response engineering at Airbnb. And during that time, we found that the typical off-the-shelf Sins just didn't really work for our use cases. They were really slow, really expensive, very operationally burdensome. We did what I would not recommend doing, which is building your own solution. But we ended up open sourcing that and that became the inspiration for Panther. I see. And who uses Panther? Oh, well, Panther is used by some of the best technology companies in the world. Coinbase, Docker, Block, many others. And as we're getting to know each other, I realized you did something fairly unconventional. from a founder's perspective. So you were the CEO, at some point a few years ago you told the board, "Hey, I want to step back from my operational role and focus on actually building. and spent 100% of my day hacking. and then later on step back as a CEO. Can you tell us a little bit about that story and why did you go back to the hands-on typing code? Yeah, so when AI was really coming up with LLMs, as LLMs were becoming much more advanced, we wanted to ask ourselves what does this look like to change the paradigm where agents really operate in the center of the stock and we couldn't just think about agents as attaching to the old manual workflows we really wanted to fundamentally rethink things. Where that started is simple code gen, like can I use an LLM to create a detection rule for me, like the ones that we just saw in the demo. And then that progressed into, can AI actually triage this alert fully end to end? And that was just a massive breakthrough. just because the limiting factors for so many years have just been around can we actually process the amount of alerts that we are generating every single day? So agents really allowed us to break that fundamental scaling problem in the SOC. and I spent all of my time just trying to figure out from first principles, how do we really rebuild this with agents in mind? - I heard the resulting product was pretty amazing. It's the core reason that people buy Panther. The other theme of this conference is actually for the data audience here, we've been talking about, hey, we want -- we think security teams in the future operate a little bit more like data teams. you start to get to understand more about how to actually operate data and use data to defense. What's your perspective on that? Yeah, so my perspective here is security teams should always be focused on security. That's their core specialty. They are operations people. They have a really great understanding of how attacks work. And in the event of an incident, they really just have to focus on that end to end. So what we've always thought of is like, how can we give them the benefits of a really great data platform, huge amounts of security data without the burden? So that's really how we've focused on it. And I think that they've gotten a lot of the benefits, especially as we've gone more into AI and agents. So we want them to actually benefit from the data infrastructure without having to get into the nitty-gritty details of how it works. And maybe looking forward with the acquisition, what are you most excited about? Yeah, so I think that Databricks and Panther are quite complementary. I think that where we focus is really on the application side of things. how can we use all of this data that's collected across the enterprise to adequately detect and respond to threats that could cause harm to the business? and also do compliance. Right. Like these are the things that drive the business forward. [160:00] keep it safe. and just really support the growth of the underlying company. So I think that there's a lot of really interesting use cases where the security agents can have access to the broader data sets, and then they can better enrich and triage and allow security teams to just have more accuracy in their response. Okay. Well, I don't want to keep you here for too long. Thank you very much, Jack. And we're now going to pass back to Ali to just Give us some closing thoughts. Thank you. Thank you. Thank you. Yeah, super excited about Panther. It's going to be awesome. Okay, so I just want to recap last two days. So this is the end of our keynote. We started the keynote by saying that we have all this AI that's amazing. But we have to give it all this context that we have at the bottom. And I hope that we've convinced you in the last two days that Databricks helps you actually get that context that they'd control into the AI. So what did we go through? So we started with talking about Lakeflow, and Lakeflow is really now the one place where you can get all your data from the different sources, and we've expanded that with hundreds of connectors. into your lake house. into any open format that you like. So that's awesome. Then we talked about Lakehouse RT, That's the Raiden engine, Reynolds' dream engine. And it's blazing fast. So you got to try it out. You can now get the dashboards to render extremely fast. The agents can move much faster on your data. And then we talked about lake base. So Lakebase is serverless Postgres, lowers your TCO significantly on any apps you want to build. But the really cool thing about leg base was the fact that you can now actually combine Lakehouse and Lakebase in what we called LakeTap. So this is one of the main reasons you should use Lake Base. OK, so that was our Gentec data. Now we have our data ready. for AI, it's fast accessible. We then talked about governance. So there's Unity Catalog and Unity AI Gateway. My favorite demo was Ankit after his vacation showed that you can actually use Unity AI Gateway to use this advisor pattern. The advisor pattern will use a small open source model for most of your work. and then it uses a tool and uses an advisor that's much smarter for expensive tasks. That way you can get your costs down significantly. So that's unified governance. Then on top of that, the most important thing we demoed yesterday was Gini ontology. Okay, so it's a graph that we construct of all the enterprise knowledge that you have, and we do that continuously in the background and feed that to the agents. The primary agent we introduced, one, Gini1, which is just a single pane of glass for all of Databricks. No more workspaces. no more different data warehouses or anything like that. You just have one place, any business user in your organization can go in and ask questions from that data. So-- "Please help your organization." get Genie 1 in everyone's hands. And you do that by turning on what's called automatic identity management, AIM, in Databricks. Then everyone can log in to Databricks. and then you give them a consumer role, so that they can actually access Genie 1 without having to see all of the rest of Databricks, which can be it can be daunting for someone who's not from a data and AI background. Genie Agents is your intelligent coworker that actually automates things, can do things on schedule. Gini code. We saw today Mike Delmaso. show that you can actually build really advanced AI models, machine learning models that do predictions, and you can automatically troubleshoot them and continuously monitor them. We also saw Zero Ops, that's my favorite genie because it just maintains data engineering pipelines. If they go down, it can fix them automatically for you. So this, I think, is a huge-- big advantage for any of us that manage data pipelines. Finally, we saw Justin demo the app builder. So now you can Vibe code your own applications on top of Databricks, and it's part of Databricks apps. Okay, and then this morning we started with Matei on the developer side. So developers who want to write their own code or they want to... decoding, how they can use this meta harness, as we called it, Omnigent as part of Agent Bricks. And we showed you all the different parts of Agent Bricks that we've launched like memory or storage and so on. Okay, so that's agentic development. Apps. That's a really great way to democratize access to everything in the organization. So it's not just Genie App Builder you can use with apps, we also showed how you can now get these micro apps that are low cost, and they can scale up and down quickly. Lake Watch, this is the sim. We just had-- on stage here, Panther, and we talked about how you, with agents, can actually automate a lot of the toil that the security teams are doing. triaging detections or alerts that come up. or doing threat hunting automatically. And then finally... customer lake. This is a customer data [165:00] that lets you do Customer 360 and Infinity campaigns. OK, so let's zoom out. What does this look like? Well, it basically is the four things that we mentioned yesterday that we think are essential for getting the context. from the data and the processes into your AI. Choice. Open source remains extremely important to us. It's cost controls and governance controls, and then that context layer with Genie ontology on top. So in short, we want you to have any data any model. and any harness, no cloud lock-in, for your whole infrastructure. We want to have one place to govern all your data in AI, and one place where you can do all of your cost controls for all the agents and the AIs. And on top of that, we have enterprise context. for all of your AI and all of your infrastructure. So that concludes our talk today. Thank you for coming last two days. We have great sessions, rest of the day, so please go to the sessions, go to the expo hall, work with all of our partners, look at the sponsors, and then finally we have an awesome party tonight at Oracle Hall with Chainsmokers, so see you all there. Thank you. Thank you. Baby, how you feel?