Databricks Launches AI Agents to Automate Machine Learning Operations

Agentic machine learning with Genie Code (includes demo) — Databricks

Databricks

Machine learning cannot scale across corporate decision-making until autonomous agents take over ongoing model maintenance, which currently consumes up to 80% of engineering time.

AI-assisted summary of Agentic machine learning with Genie Code (includes demo) — original source. Check the original for full context.

Source published:

Key ideas

  • Databricks introduced AI Runtime with serverless, on-demand GPUs and multi-node distributed training support for deep learning and LLMs, entering public preview with zero upfront commitments. (7:46)
  • Genie Code for ML is now generally available, embedding domain-specific engineering heuristics and native integrations across Databricks feature engineering, training, serving, and monitoring. (8:57)
  • Genie ZeroOps for ML launches later in the summer to invert production operations, autonomously investigating alerts, engineering missing features, retraining models, and presenting production-ready fixes for human approval. (13:20)
  • Danfoss leveraged Genie Code's ontology-driven context to build and deploy a complete production ML system in 90 minutes, bypassing the weeks or months of typical engineering lead time. (9:57)

Detailed notes

## Background Published by Databricks on YouTube on June 24, 2026, this keynote presentation highlights Genie Code's upgraded intelligence for machine learning engineering. According to the description, the release delivers native integrations across every Databricks ML Platform layer, including feature engineering, model training, serving, and monitoring.

## TL;DR Databricks leaders Mike Del Balso and Amber Roberts outline how autonomous AI agents can overcome the decades-old technical debt and maintenance burden of production machine learning. The keynote introduces AI Runtime for serverless GPU-powered deep learning training, Genie Code for ML to construct context-aware pipelines using organizational tribal knowledge, and Genie ZeroOps for ML to automate operational triage and patching. A live demonstration illustrates Genie ZeroOps autonomously diagnosing an accuracy regression in a ticket forecasting model, engineering missing tournament features, retraining the pipeline, and serving a verified candidate model for human approval.

## Atmosphere & style The presentation carries the confident, polished cadence of a major developer keynote, balancing seasoned industry reflection with forward-looking technical ambition. Mike Del Balso establishes an authoritative yet accessible narrative, connecting his foundational MLOps background at Google and Uber to modern generative AI pressures. Amber Roberts's live demo introduces relatable operational tension by framing the scenario around a high-stakes Friday afternoon alert, delivering a satisfying and pragmatic payoff as autonomous agents replace weekend firefighting with a single click of human governance.

## Bulletin - [07:46](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=466s) Databricks introduced AI Runtime with serverless, on-demand GPUs and multi-node distributed training support for deep learning and LLMs, entering public preview with zero upfront commitments. - [08:57](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=537s) Genie Code for ML is now generally available, embedding domain-specific engineering heuristics and native integrations across Databricks feature engineering, training, serving, and monitoring. - [13:20](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=800s) Genie ZeroOps for ML launches later in the summer to invert production operations, autonomously investigating alerts, engineering missing features, retraining models, and presenting production-ready fixes for human approval. - [09:57](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=597s) Danfoss leveraged Genie Code's ontology-driven context to build and deploy a complete production ML system in 90 minutes, bypassing the weeks or months of typical engineering lead time.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=0s) The core machine learning code represents only a tiny fraction of a production system, while the surrounding data ingestion, serving, and monitoring infrastructure creates massive technical debt. - [01:12](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=72s) Generative AI is accelerating the demand for traditional ML by generating massive volumes of new software that require automated decision points like pricing, fraud detection, and recommendations. - [03:43](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=223s) Automating ML has historically lagged behind software engineering automation due to a lack of cohesive, closed-loop infrastructure and the absence of rich tribal context for autonomous agents. - [04:44](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=284s) Databricks redesigned its platform over 18 months to make it agent-ready using 100% serverless compute, a unified instrumented data model, and action-level governance in Unity Catalog. - [07:46](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=466s) AI Runtime provides serverless GPU infrastructure that scales on demand without upfront commitments, supporting both single-node and multi-node distributed training for frontier models. - [09:57](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=597s) Genie Code incorporates organizational ontology and tribal knowledge to reuse existing pipelines, adhere to internal evaluation scripts, and generate production-grade code that matches senior engineer output. - [13:20](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=800s) Because teams spend 60% to 80% of their time maintaining existing deployments, multiplying development speed by 100 will create an unsustainable operational crisis without automated maintenance. - [14:26](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=866s) Genie ZeroOps inverts traditional alerting by placing an autonomous agent on call 24/7 to triage alerts, test hypotheses, and retrain models while human engineers act as approvers. - [16:07](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=967s) In a live demonstration, Genie ZeroOps detected a ticket forecasting failure, diagnosed an omitted knockout-stage feature, engineered the new feature, retrained the model, and prepared a fix within 30 minutes. - [20:19](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=1219s) Operators can review the agent's evaluation benchmarks and roll out candidate models to live traffic with a single click, eliminating high-severity operational downtime. - [21:40](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=1300s) Automating the end-to-end development and operations lifecycle enables enterprises to manage 100 times more models and replace static business heuristics with intelligent ML decisions.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=0s) The Hidden Technical Debt of Production Machine Learning

Mike Del Balso, Director of Product Management at Databricks, steps onto the stage, quipping that the walk out was deceptively long. He reflects on his career dedicated to production machine learning, tracing his roots back to the Google Ads ML teams that helped spark the MLOps discipline, co-founding the ML team at Uber where he built and led the Michelangelo platform, and running his own production ML startup that recently joined Databricks. Across that entire journey, he notes, the machine learning industry has chased a single, persistent dream: “And the whole time, my whole career, the whole machine learning industry has had one goal, and a single dream, which is to make machine learning as easy and automated as possible, so we can bring intelligent decisions to all of the important places, all the important decisions across our business.”

Displaying the iconic technical debt diagram from a paper his Google team published roughly twelve years earlier, Del Balso emphasizes that actual machine learning code occupies just a tiny red box in the center. Everything surrounding it consists of the massive, complex peripheral systems required to build, run, and maintain true production deployments—systems that are fundamentally data problems under the hood. While many expected generative AI to dampen the demand for traditional machine learning, Del Balso argues it has achieved the exact opposite. Companies are deploying vast amounts of new software, creating an explosion of critical decision points: more product recommendations, dynamic pricing calculations, fraud detection evaluations, and automated loan decisions.

Because production ML remains painfully difficult, organizations are forced to ration it, reserving models for only a handful of critical workflows. When compared against the thousands of other business decisions that could be automated, companies continue to rely on static logic, rigid heuristics, and simple rules of thumb. As Del Balso puts it: “And because we're not using intelligent machine learning there, we're leaving money on the table.” Automating this workflow is exceptionally hard because human experts—such as data scientists and PhDs—are embedded at every stage of the lifecycle deciding which features to craft, which models look viable, what objective metrics to optimize, and which candidates deserve promotion to production.

- Google Ads ML engineering teams originally laid the conceptual foundation for modern MLOps. - Mike Del Balso co-created and led Michelangelo, Uber's pioneering internal machine learning platform. - The classic Google paper illustrating the "hidden technical debt" of ML systems was published approximately 12 years prior.

```mermaid flowchart TD accTitle: The Reality of Production Machine Learning Systems accDescr: A diagram contrasting the small ML code core with the vast peripheral infrastructure required in production.

subgraph Periphery["Vast Data & Operational Infrastructure"] DataIngest["Data Collection & Verification"] FeatEng["Feature Extraction & Pipelines"] ResourceMgmt["Resource Management & Serving"] Monitor["Analysis & Monitoring Tools"] end

subgraph Core["The Small Center"] MLCode["ML Code"] end

DataIngest --> MLCode FeatEng --> MLCode MLCode --> ResourceMgmt MLCode --> Monitor ```

---

[03:43](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=223s) Why AI Agents Are the Only Path to Full ML Automation

Del Balso asserts that software and traditional code alone cannot solve this bottleneck; artificial intelligence agents represent the only technology capable of automating machine learning end-to-end. While AI coding agents have already completely overhauled general software engineering, machine learning workflows remain stuck in their traditional manual patterns. Del Balso explains that this lag is driven by two specific blockers across the industry:

1. **Infrastructure maturity:** Only a small handful of companies possess the sophisticated, cohesive infrastructure required to support a fully closed, automated machine learning loop. 2. **Context deficit:** Agents cannot replicate the choices of expert machine learning engineers unless they have access to the exact same depth of domain, pipeline, and operational context.

Databricks aims to bridge this gap by delivering both agent-ready machine learning infrastructure and specialized agents capable of navigating that infrastructure with deep organizational context. Del Balso stresses that agents must do more than execute blind scripts: “Second, of course, we need the agents that can drive the loop with your context to do it in the same way that your teams would do.”

---

[04:44](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=284s) Building Agent-Ready ML Infrastructure on Databricks

Turning to infrastructure, Del Balso notes that Databricks has spent over a decade constructing its data science and machine learning platform, leading to its industry recognition as the top-right leader in data science and ML platforms. However, that entire surface was designed for human practitioners. Over the past 18 months, Databricks focused on redesigning and optimizing the entire ML stack to make it completely agent-ready through three core architectural pillars:

- **100% Serverless Operations:** Agents should never spend time configuring compute clusters, diagnosing environment dependencies, or managing machine state. Compute is completely serverless out of the box, allowing agents to instantly spin up, scale, and tear down resources on demand. - **Unified Instrumented Data Model:** The end-to-end lifecycle—spanning training datasets, feature tables, inference logs, serving payloads, and live monitoring metrics—is unified inside the same governed data model. This single source of truth allows an agent to immediately observe the downstream impact of an upstream pipeline tweak. - **Default Governance for Agent Actions:** While machine learning assets like models, experiment tracking runs, and serving endpoints already live under Unity Catalog, governance has now been extended to the precise actions and mutations agents can perform, guaranteeing auditability and operational trust.

As Del Balso summarizes: “The problem is that we solved we built all of this and we solved all of these problems for humans to build an ML. But now we're going to have agents do it for us.”

- Over 500 million machine learning models were trained on the Databricks platform in the past year alone. - Unity Catalog governs both static assets (models, features, endpoints) and the runtime actions permitted to autonomous agents.

---

[07:46](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=466s) Introducing AI Runtime and Genie Code for ML

Del Balso announces the launch of **AI Runtime**, a brand-new Databricks compute architecture built specifically for training deep learning models and large language models (LLMs). AI Runtime offers on-demand, serverless GPUs that require zero upfront financial commitments, enabling teams to scale elastic GPU compute instantly. During its initial two-month preview, hundreds of enterprises migrated deep learning workloads onto the runtime—including pharmaceutical giant Merck, who partnered with Databricks to train advanced drug discovery models. Expanding the capability further on launch day, Del Balso announces that AI Runtime now includes full support for multi-node distributed training, opening up large-scale foundation model development on arbitrary dataset sizes.

With the underlying infrastructure transformed into an agent-ready foundation, Del Balso transitions to the agents built to drive the lifecycle: **Genie Code for machine learning**. Available immediately, Genie Code for ML introduces upgraded engineering intelligence specifically tuned for real-world machine learning tasks, rather than generic web development or basic script generation. It incorporates operational heuristics drawn from Databricks’ vast production exposure and features native, out-of-the-box integrations with every layer of the Databricks ML ecosystem, including feature stores, model training runs, deployment infrastructure, and live system monitoring.

Del Balso captures the motivation behind tailoring the model: “Nobody has seen more machine learning in the real world than Databricks. So, we've taken all of those lessons, what works, what doesn't work, what good looks like in production, and we've built that into Genie code.”

- AI Runtime provides serverless, on-demand GPU clusters with no upfront resource commitments. - The platform was validated in private preview across hundreds of enterprises, notably powering Merck's drug discovery models. - Multi-node distributed training support is added to AI Runtime to accommodate frontier deep learning datasets and architectures. - Genie Code for ML is directly integrated across the Databricks feature engineering, model training, model serving, and pipeline monitoring APIs.

[09:57](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=597s) Context and Ontology in Genie Code

Mike Del Balso highlights that what excites him most about the latest release is the upgraded context built into Genie Code, which enables the agent to understand how data science and machine learning teams actually operate. Reflecting on an older ML platform diagram from a decade prior, he notes that earlier tools only solved the human interface problem. In reality, when human engineers drive an ML lifecycle, they make decisions based on institutional memory and tribal knowledge scattered across an entire platform: remembering past experiments, interpreting business metrics, recalling past failures, and locating company-approved evaluation scripts. Without direct access to this distributed context, an autonomous agent risks using the wrong datasets, running improper evaluations, and producing output that looks plausible on the surface but cannot be trusted.

```mermaid flowchart TD accTitle: Genie Code Context and Ontology Architecture accDescr: Flowchart showing how Genie Ontology feeds tribal knowledge and institutional context into Genie Code to automate model building. subgraph Context["Genie Ontology (Tribal Knowledge)"] A["Feature Patterns & Workflows"] B["Historical Experiments & Metrics"] C["Existing Training Pipelines"] D["Standardized Evaluation Scripts"] end

subgraph Agent["Genie Code Agent Loop"] E["Feature Engineering"] F["Pipeline Construction"] G["Evaluation & Benchmarking"] end

subgraph Outcome["Production Delivery"] H["Senior ML Engineer Level Output"] I["Rapid Production ML Deployment"] end

Context --> Agent A --> E B & C --> F D --> G Agent --> Outcome E & F & G --> H --> I ```

To resolve this limitation, Databricks integrated Genie Code with Genie ontology, which was unveiled the previous day by Ken. By infusing ontology into Genie Code, the agent understands how a company's specific ML teams build, test, and validate models. It constructs features using the exact conventions already adopted by the organization, reuses existing training pipelines, and honors proprietary evaluation scripts. Because it evaluates candidate models against the team's existing baseline metrics, data scientists can rely on its recommendations. Del Balso explains that this context allows Genie Code to act like a senior machine learning engineer embedded directly on the team, outperforming other leading coding agents on real-world data science tasks.

- Danfoss, a multinational engineering corporation, leveraged this capability to move from zero machine learning footprint to a complete production ML system in just 90 minutes. - Traditional deployment workflows for comparable systems typically require weeks, months, or entire quarters of engineering effort. - “We've found that Genie code performs way better on real-world data science and machine learning tasks than other leading coding agents now,” Del Balso states.

---

[13:20](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=800s) The Maintenance Crisis and Introducing Genie ZeroOps for ML

While speeding up model development is an enormous leap forward, Del Balso cautions that the initial build phase accounts for only a fraction of an ML engineer's daily reality. Real-world teams spend between 60% and 80% of their working hours maintaining existing applications rather than creating new ones. Accelerating model development by a factor of 100 will inevitably lead to organizations creating 100 times more models, creating a proportional operational burden that no engineering staff could realistically support. Once a model is deployed, teams face a long tail of operational failures: feature values drift over time, inference servers encounter latency spikes, upstream data pipelines break, and edge-case errors degrade performance.

```mermaid flowchart LR accTitle: Traditional Monitoring versus Genie ZeroOps Paradigm accDescr: Comparison showing the shift from traditional alerting to autonomous investigation, remediation, and human approval. subgraph Traditional["Traditional Monitoring"] T1["Metric Degradation"] --> T2["Alert Fired"] --> T3["Page On-Call Engineer"] --> T4["Manual Root Cause & Patch"] end

subgraph ZeroOps["Genie ZeroOps for ML"] Z1["Continuous Monitoring"] --> Z2["Autonomous Root Cause Investigation"] --> Z3["Generate Fix & Retrain"] --> Z4["Human Review & One-Click Approval"] end ```

To automate the operational side of the lifecycle, Del Balso introduces Genie ZeroOps for machine learning, describing it as the single biggest launch in machine learning over the past decade. Traditional monitoring tools merely track metrics and page human operators when a threshold breaks, leaving engineers to do the heavy lifting of triage and repair. In contrast, Genie ZeroOps for ML detects an issue, autonomously investigates the root cause, tests hypotheses, and delivers a completed, production-ready fix directly to the team. This completely inverts the industry's historical operational model: an agent remains on call around the clock, while human engineers step into the supervisory role of reviewing and approving validated solutions.

- Production ML systems regularly succumb to a long tail of thousands of operational failure modes, including data drift, inference latency spikes, and broken data dependencies. - Genie ZeroOps monitors every deployed model continuously while enforcing human-in-the-loop governance prior to any production deployment. - “And zero ops for ML inverts that and allows an agent to be on call and your team is just in a position to approve the recommendations from that agent,” explains Del Balso before welcoming Amber Roberts to the stage for a live demonstration.

---

[16:07](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=967s) Live Demo: Autonomously Resolving Issues with Genie ZeroOps

Amber Roberts, Staff Tech Marketing Engineer at Databricks, demonstrates how Genie ZeroOps pinpoints and resolves machine learning defects in real time. She frames the demonstration around a real-world scenario at Berkshire Sports Global, where her team maintains a mission-critical production model that forecasts ticket sales for every match of the Data and AI World Cup. Because ticket sales forecasts directly guide tens of millions of dollars in venue staffing and operational overhead—and mispricing risks leaving stadiums half-empty—predictive precision is essential. To safeguard the pipeline, Roberts prompts Genie Code to establish an automated alert that fires whenever actual ticket sales deviate by more than plus or minus 5% from forecasted values. Genie Code generates, configures, and registers the alerting agent without requiring manual infrastructure setup.

Roberts then fast-forwards the scenario to Friday at 5:30 p.m., just as an engineer emerges from back-to-back meetings to find that the accuracy alert triggered 30 minutes earlier. Instead of an engineer spending the weekend manually tracing error logs, the notification features a "View investigation in Genie ZeroOps" link that navigates directly into the ZeroOps inbox. While the engineer was away, Genie Code had already opened an incident thread marked with the highest severity and spent the preceding half hour testing dozens of hypotheses. Leveraging the ontology of the workspace, the agent inspected model notebooks, queried Unity Catalog tables, ran SQL diagnostics, monitored feature distributions, and evaluated recent MLflow experiment runs to identify the failure point.

The agent determined that the model suffered from an omitted feature representing the specific tournament stage. Without this variable, the model failed to account for the surging consumer demand and fan attention associated with knockout matches, leading to a sudden loss in predictive accuracy as group play ended. Rather than stopping at diagnosis, Genie Code discovered existing, unutilized match metadata, synthesized new features for tournament rounds and team matchups, retrained the candidate model, performed hyperparameter optimization, and registered the updated artifact in MLflow. Because it preserved complete lineage across the project, Genie Code applied the team's original validation criteria to benchmark the new model against historical standards before presenting the finished fix for approval.

- “And this model is mission critical because that forecast affects tens of millions of dollars in event staffing and operation spend,” Roberts emphasizes. - The monitoring alert was defined to trigger whenever observed ticket sales deviated beyond a ±5% margin relative to predicted game attendance. - Genie ZeroOps autonomously explored and eliminated dozens of potential failure modes across Unity Catalog, SQL queries, and MLflow runs within 30 minutes of the alert firing. - The identified root cause was a missing knockout-stage demand feature, which Genie Code resolved by engineering features from existing match data and retraining the pipeline following Databricks best practices.

[20:19](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=1219s) Deploying the Retrained Model and Eliminating Operational Toil

Amber Roberts wraps up the live demonstration by reviewing the results of the automated fix generated by Genie Code. The new candidate model restores model accuracy and brings the forecasting pipeline back to expected operational levels. Because she was able to inspect the exact evaluation metrics and validation benchmarks used by the agent, Amber feels fully confident rolling out the new candidate to a small percentage of production traffic. With a single click, she promotes and deploys version two of the model, bringing the remediation process to an end without manual coding.

```mermaid flowchart LR accTitle: Genie ZeroOps Incident Remediation Flow accDescr: Progression from alert and agent evaluation to single-click production rollout.

subgraph Evaluation ["Evaluation & Verification"] A["Alert: Degraded Accuracy"] --> B["Genie Code Diagnoses & Retrains"] B --> C["Review Evaluation Benchmarks"] end

subgraph Deployment ["Rollout"] C --> D["Deploy Candidate (v2)"] D --> E["Route Low % of Traffic"] E --> F["Incident Fully Resolved"] end ```

Amber contrasts this seamless experience with the traditional engineering reality of diagnosing and patching stale models in production. She notes that in standard environments, model degradation triggers high-severity team incidents that consume days of engineering time and risk tens of millions of dollars in business losses. By automating root-cause diagnosis, feature retraining, and verification, Genie ZeroOps eliminates this operational drag, allowing data teams to manage massive model fleets without linear increases in headcount. As Amber emphasizes: “Give ML teams the tools that they need to stop spending all this time fighting operational toil and start building breakthrough AI.”

- **One-click deployment:** Engineers can promote agent-generated model versions (such as version two demonstrated) directly into serving endpoints after verifying evaluation results. - **Incident severity mitigation:** The agent resolves stale-model incidents that typically disrupt entire teams for hours or days, directly averting potential eight-figure business impacts. - **Scaling ML operations:** Addressing operational toil is presented as the primary requirement for surviving the transition into the "100X era of ML," where organizations must support a hundredfold increase in model volume and complexity.

[21:40](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=1300s) Recap, Availability, and the Automated ML Era

Mike Del Balso returns to the stage to summarize the keynote's three major platform announcements and outline their release statuses. First, Databricks AI Runtime is available in public preview, providing serverless GPU infrastructure optimized for deep learning models and LLMs. Second, Genie Code for ML is generally available directly within existing Genie Code environments on user laptops. Third, Genie ZeroOps for ML will arrive in the summer, delivering full-loop autonomous incident management.

Reflecting on the overarching history of the field, Mike reminds the audience that the long-standing vision of the machine learning discipline has always been full automation. By combining agent-ready, managed infrastructure with intelligent agents capable of driving the development and maintenance loops, teams can now develop systems 100 times faster and manage 100 times more models. Mike states the strategic outcome clearly: “The machine learning industry has always been about automating machine learning, and we're finally at the point where we have the infrastructure and we have the agents to drive the loop.” This capability enables businesses to scale ML into every granular operational decision and deliver personalization across all touchpoints while engineers direct their energy toward future breakthroughs.

- **AI Runtime:** In public preview; delivers serverless GPU support designed for building deep learning architectures and large language models. - **Genie Code for ML:** Generally available (GA); integrated directly into the local Genie Code tooling on developer machines. - **Genie ZeroOps for ML:** Slated for release later in the summer to handle automated monitoring, retraining, and operational remediation. - **Vision for 100X ML:** Autonomous agents close the loop on routine ML workflows, allowing companies to apply optimized ML decision-making across all core products.

## Notable quotes > [21:40](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=1300s) The machine learning industry has always been about automating machine learning, and we're finally at the point where we have the infrastructure and we have the agents to drive the loop. — Mike Del Balso

> [07:46](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=466s) Nobody has seen more machine learning in the real world than Databricks. So, we've taken all of those lessons, what works, what doesn't work, what good looks like in production, and we've built that into Genie code. — Mike Del Balso

> [20:19](https://www.youtube.com/watch?v=C1rEj2VQtwU&t=1219s) Give ML teams the tools that they need to stop spending all this time fighting operational toil and start building breakthrough AI. — Amber Roberts

## Mentioned Databricks, Genie Code, Genie ZeroOps, AI Runtime, Unity Catalog, MLflow, Mike Del Balso, Amber Roberts, Google, Google Ads, Uber, Michelangelo, Merck, Danfoss, Berkshire Sports Global

## Video description Genie Code now comes with upgraded intelligence for ML engineering and native integrations across every Databricks ML Platform component: feature engineering, model training, serving, and monitoring.

Learn more: https://www.databricks.com/blog/whats-new-ai-platform-agents-ml-engineering-our-deep-learning-platform-and-new-capabilities

00:00 — Mike Del Balso, Director, Product Management, Databricks 01:12 — The hidden tech debt of ML systems 03:43 — AI agents are the the only way to automate ML 04:44 — Agent-ready infrastructure with Databricks 07:46 — Introducing AI Runtime 08:55 — Driving the agent loop with context 09:19 — Introducing Genie Code for ML 13:29 — The looming AI maintenance burden 14:26 — Introducing Genie ZeroOps for ML 15:49 — Justin introduces Amber Roberts, Staff Tech Marketing Engineer, Databricks 16:08 — Amber demos Genie ZeroOps for ML 21:47 — Mike wraps up

Presenters: Mike Del Balso, Director, Product Management, Databricks Amber Roberts, Staff Tech Marketing Engineer, Databricks

[Source](https://www.youtube.com/watch?v=C1rEj2VQtwU)

Reading summary

Databricks' Mike Del Balso argues that the machine learning industry has reached a turning point where autonomous AI agents—supported by serverless, context-aware infrastructure—can finally achieve the goal of full-lifecycle automation for production models. The Persistent Gap in Production ML (0:00) Despite the long-standing industry ambition to automate decision-making, production machine learning remains restricted to a narrow set of critical workflows. Del Balso notes that while models are essential for tasks like fraud detection and dynamic pricing, organizations often default to static logic and heuristics due to the intense manual labor required to manage the 'hidden' peripheral infrastructure surrounding actual ML code. This resource rationing leaves significant business value unrealized. Infrastructure Requirements for Agent-Led ML (4:44) To transition beyond manual processes, Databricks argues that machine learning systems must evolve into 'agent-ready' environments. This necessitates three foundational pillars: fully serverless compute that eliminates manual resource management, a unified data model that tracks the lifecycle from training to monitoring, and granular governance. By extending Unity Catalog to govern both static assets and the specific runtime actions autonomous agents can perform, organizations can maintain auditability while allowing agents to autonomously navigate complex technical environments. Context-Driven Intelligence via Genie Ontology (9:57) Generic coding agents often fail in machine learning because they lack the tribal knowledge required to build and validate models successfully. By integrating Genie Code with 'Genie Ontology'—which captures institutional history, past experiments, and standardized evaluation scripts—Databricks aims to mimic the decision-making process of a senior ML engineer. This context-aware approach has allowed early adopters, such as Danfoss, to deploy production ML systems in approximately 90 minutes, compared to traditional timelines spanning weeks or months. Scaling Operations with Genie ZeroOps A critical challenge in the '100X era' of ML is the operational burden of maintenance, which consumes up to 80% of an engineer's time. Genie ZeroOps for ML seeks to invert this model by enabling agents to autonomously detect, investigate, and remediate operational failures—such as data drift or latency spikes—before human engineers provide final approval. In a live demonstration, an agent correctly identified missing feature data as the cause for model degradation, retrained the model, and prepared a candidate for deployment within 30 minutes, drastically reducing the traditional manual triage process. Databricks concludes that the combination of AI Runtime, Genie Code, and the upcoming Genie ZeroOps provides the necessary framework to scale model development and maintenance by a factor of 100. The ultimate success of this approach depends on the agent's ability to maintain high precision while operating within the specific institutional context of its host organization.

Source quotes

[21:40] Mike Del Balso: The machine learning industry has always been about automating machine learning, and we're finally at the point where we have the infrastructure and we have the agents to drive the loop. [7:46] Mike Del Balso: Nobody has seen more machine learning in the real world than Databricks. So, we've taken all of those lessons, what works, what doesn't work, what good looks like in production, and we've built that into Genie code. [20:19] Amber Roberts: Give ML teams the tools that they need to stop spending all this time fighting operational toil and start building breakthrough AI.

Original transcript

[0:00] Welcome to the stage, Databricks [0:02] Director of Product Management, Mike Del [0:05] Balso. [0:12] >> All right. [0:13] Justin was right. This is a deceptively [0:15] long walk. [0:18] Hey everyone. So, my whole career I've [0:20] actually been working in production [0:21] machine learning. I started out at [0:23] Google working on the teams, the ads [0:25] machine learning teams, that actually [0:28] kind of kicked off the whole ML ops [0:30] space. [0:31] Then I helped start the machine learning [0:32] team at Uber and helped build and lead [0:34] the ML platform there called [0:35] Michelangelo. [0:37] And for the past few years, I've been [0:38] running my own production machine [0:40] learning startup, [0:41] which recently joined Databricks, and [0:43] we've been working on those same [0:44] problems here at Databricks. [0:46] And the whole time, my whole career, the [0:48] whole machine learning industry has had [0:51] one goal, and a single dream, [0:53] which is to make machine learning as [0:55] easy and automated as possible, so we [0:58] can bring intelligent decisions to all [1:00] of the important places, all the [1:02] important decisions across our business. [1:06] And many companies, many have tried and [1:08] many have failed, and I'm really excited [1:10] to talk to you today because [1:12] that's finally changing. [1:14] So, this diagram here, many of you may [1:16] recognize this. This is from a paper [1:19] that my team at Google published [1:21] something like 12 years ago. And the [1:23] point of this paper is to really explain [1:25] how hard it is to do production machine [1:27] learning. You know, this diagram shows, [1:28] if you look at the red, that's the [1:30] machine learning code, [1:32] and everything around it is the point of [1:33] the diagram. All these other things [1:35] around it are all these other systems [1:36] you practically need to build and run [1:39] and maintain to run true ML systems in [1:41] production. It's a really hard problem, [1:44] and as a hint, this is all a data [1:45] problem under the hood. [1:49] And why are we talking about this right [1:50] now? [1:52] Well, we're building more software than [1:53] ever, [1:54] and you You have thought that GenAI [1:56] would reduce the demand [1:58] in machine learning, but it's actually [2:00] only accelerated it. With all of the new [2:02] software and all the new product [2:04] experiences we're building, [2:06] we have more products to recommend, we [2:08] have more prices to offer, [2:10] we have more fraud to detect, [2:13] uh more loans applications to decide on, [2:15] so we have a lot more decisions to make [2:17] in the company. [2:18] >> [snorts] [2:19] >> And the problem is that it's actually [2:21] really hard to do this, and so we only [2:23] use we ration machine learning and only [2:25] use it in a handful of places, right? [2:27] Think in your company, how many models [2:29] you have in production compared to all [2:31] of the other decisions that you make in [2:33] the company that could be automated. And [2:35] because we're not using [2:37] intelligent machine learning there, [2:39] we're leaving money on the table. [2:41] Right? We're using we're basically [2:42] making dumber decisions. We're using [2:44] heuristics and rules of thumb and [2:45] statically coded logic. [2:47] And what we really want to do [2:49] is automate machine learning so we can [2:53] make it really easy to scale to all the [2:55] important decisions [2:57] we have in the company [2:59] and keep up with all the scale of the [3:01] new software, all the new decisions [3:02] we're going to add as we deploy all this [3:03] new software. [3:05] And we just just think back to the [3:06] diagram I showed you, it's a [3:08] it's a really hard problem technically, [3:10] but also, you know, to automate this, [3:13] you know, there's people in the loop in [3:15] every single stage. There's a person [3:16] making there's expert, there's a data [3:19] scientist, someone with a PhD making [3:21] decision about which feature to build, [3:23] which model looks good, what are we [3:25] really optimizing for? [3:26] Which model should we promote to [3:27] production? [3:29] So, if we're going to automate machine [3:30] learning, [3:31] we need to do a lot to accelerate these [3:33] folks [3:34] and enable the automation to make [3:36] decisions in the same way they do. [3:38] And there's actually only one technology [3:39] that can do this. [3:42] And [clears throat] that's agents. [3:44] Now, agents have done [3:45] an immense amount to completely [3:47] revolutionize how we build [3:49] software, all of software engineering. [3:52] But they haven't had that same impact [3:54] with machine learning yet. [3:56] We're still kind of building and [3:57] operating machine learning in the same [3:58] way. [3:59] >> [snorts] [3:59] >> Now, why is that? [4:02] Well, there's two reasons. One [4:04] is only a handful of companies even have [4:07] like the right infrastructure. It's a [4:08] really hard problem. You saw the diagram [4:10] of all the stuff you got to build [4:12] to run this complete machine learning [4:15] loop in an automated way. [4:18] The second problem [4:20] >> [snorts] [4:20] >> is that if we want agents or software to [4:23] automate and make the same decisions [4:26] that our machine learning engineers are [4:27] making, [4:29] they need to have all the same type of [4:30] information so they can make the same [4:32] quality of decisions. We can really [4:33] automate that. And that context is just [4:36] not available today to our agents. [4:38] And I'm really excited because [4:40] Databricks is the first company that can [4:41] solve this. [4:42] And we're doing this in two ways. The [4:44] first is with agent-ready machine [4:46] learning infrastructure. So, getting [4:48] that infrastructure into a place that [4:50] your agents can actually drive [4:52] throughout that whole loop. [4:54] >> [snorts] [4:55] >> Second, of course, [4:57] we need the agents that can drive the [4:59] loop with your context to do it in the [5:01] same way that your teams would do. [5:04] So, let's look at both of these, right? [5:06] Agent-ready machine learning [5:07] infrastructure. [5:08] If you're here, you're probably building [5:11] your ML models on Databricks. [5:13] Databricks has been at this for more [5:15] than a decade now. More than 500 million [5:18] models have been trained on Databricks [5:20] in just the past year alone. [5:23] And so, Databricks already has a [5:24] complete leading platform [5:27] for machine learning. [5:30] And that's why Databricks is recognized [5:32] as the leader in data science and ML. [5:34] Databricks [5:36] is the dot in the most the highest up, [5:39] most top, and most to the right. [5:41] And so, that's why you're probably [5:42] building your ML models on Databricks [5:43] today. [5:46] >> [snorts] [5:46] >> So, what's the problem? [5:49] Well, the problem is that we solved we [5:51] built all of this and we solved all of [5:53] these problems for humans to build an [5:55] ML. [5:57] But now we're going to have agents do it [5:59] for us. [6:01] And that's what we've that's what we've [6:02] spent the past 18 months optimizing for [6:05] at Databricks. [6:07] We made the stack agent ready. [6:09] So, what [snorts] do I mean by that? [6:11] First thing we made everything [6:12] serverless. We don't want agents to have [6:15] to configure clusters and maintain [6:16] cluster state and stuff like that. [6:18] Everything is serverless out of the box, [6:20] easy for an agent to spin up and spin [6:22] down, [6:23] and scale to whatever uh scale they [6:25] need. [6:27] Secondly, [6:29] this whole data loop, this whole machine [6:31] learning life cycle is now instrumented [6:33] with a single [6:35] uh data model. So, your training data, [6:38] your features, your serving data, your [6:40] monitoring data, your inference, [6:42] they all land in the same governed [6:44] tables. [6:45] That allows your agents to immediately [6:49] see the impact of any changes they make [6:52] and have a single source of truth for [6:54] the entire machine learning life cycle, [6:57] making it much easier for them to [6:58] optimize and drive this loop. [7:01] And thirdly, [7:02] everything is governed by default. [7:04] >> [clears throat] [7:05] >> So, you probably already know that we [7:06] govern all of our machine learning [7:08] assets. [7:09] So, your models, your experiments, your [7:12] serving endpoints, all of that's already [7:14] governed in Unity Catalog today. [7:17] But we now have governance on the [7:19] actions that your agents can take, [7:23] so that you can trust any changes that [7:25] your agents are making to automate this [7:28] machine learning loop. [7:31] And I want to I'm really excited because [7:33] we're extending this further today. [7:35] We are launching a new compute type in [7:38] Databricks that makes it really easy to [7:40] build um [7:42] uh deep learning models and LLMs [7:45] and that is called AI run time. [7:48] >> [snorts] [7:49] >> This is serverless GPUs available on [7:52] demand at whatever scale you need for no [7:55] upfront commit. [7:56] You can use this to build deep learning [7:59] models and machine and LLMs and you know [8:02] we've actually had this in preview for [8:04] the past 2 months and we've seen huge [8:05] adoption from it. There's hundreds of [8:08] companies that have already moved their [8:10] deep learning workloads to this platform [8:12] and are finding huge success. [8:14] Yesterday Ali mentioned [8:17] Merck's latest drug discovery models and [8:19] we've been really fortunate to partner [8:20] with them and help them build on this [8:22] platform. [8:24] So I want to announce one more thing [8:26] which is we're expanding it today to [8:27] support multi-node training. So now you [8:30] can build [8:31] these models at any scale you need on [8:33] any size of data. [8:35] We're really excited about this because [8:37] this brings a whole new domain of [8:38] machine learning within the Databricks [8:40] platform and there's no reason that you [8:43] shouldn't be bringing this to your deep [8:44] learning teams today. [8:46] >> [snorts] [8:48] >> Thank [applause] you. [8:51] Okay, so that was the infrastructure. [8:54] Been talking about infrastructure for a [8:55] long time, right? Let's talk about the [8:57] agents to actually drive this loop, [8:59] right? This is [9:00] getting to a a more interesting thing [9:01] here. Agents that drive the loop, it's [9:03] important for them to do it with your [9:04] context in the same way that your team [9:06] would actually build machine learning, [9:08] right? So let's talk about the first [9:10] part here, it's building. [9:12] >> [snorts] [9:13] >> So this is building the models, [9:15] evaluating them, etc. Extremely excited [9:18] also to announce Genie code for machine [9:20] learning. This is an This is an enhanced [9:22] set of capabilities and intelligence for [9:25] real-world machine learning tasks [9:27] built into Genie code and it's available [9:29] today. [9:32] >> [applause] [9:34] >> Nobody has seen more machine learning in [9:37] the real world than Databricks. So, [9:38] we've taken all of those lessons, [9:41] what works, what doesn't work, what good [9:43] looks like in production, and we've [9:44] built that into Genie code. [9:47] Genie code is also natively [9:49] integrated with the complete machine [9:51] learning stack. [9:52] So, it knows how to use all of those [9:55] components, all that infrastructure I [9:57] just showed, better than any other uh [9:59] coding agent. [10:00] And what I'm most excited about is we've [10:03] upgraded the context in Genie code. [10:05] So, it now knows how your machine [10:08] learning teams and how your data science [10:10] teams actually work, so it can make [10:13] recommendations and do work alongside [10:14] your teams. And I'll show you exactly [10:16] what I mean by that. [10:18] So, you remember this diagram? [10:20] This diagram's a little bit deceptive [10:22] because, as I mentioned, this is from [10:24] like a decade ago, and that was the [10:25] problem we were thinking about at that [10:26] time. That was solving the human [10:27] problem. [10:29] But when people are making decisions and [10:31] driving that whole loop, they're [10:32] actually [10:34] drawing on many other sources of [10:35] information [10:37] to make these decisions, right? What [10:39] What experiments do we run already? What [10:42] uh What does this uh business metric [10:44] mean? What What worked, what didn't [10:46] work? Where do we keep that evaluation [10:47] script? So, that and all this context is [10:51] is scattered across our whole platform. [10:53] And if you don't have access to this [10:55] data, if your agent doesn't have access [10:57] to this data, [10:59] it's going to have [11:00] the wrong It's going to use the wrong [11:01] data. It's going to use the wrong [11:02] evaluation. [11:04] And it's going to tell you something [11:05] that kind of looks right, [11:07] but it won't be right. [11:09] And you won't be able to trust it. [11:12] So, what we've done, [11:14] yesterday Ken announced Genie ontology, [11:17] we've built all of this into integrated [11:20] it with Genie ontology and empowered [11:22] Genie code with ontology. So, all of [11:24] that context is built into Genie code, [11:27] and allow This allows Genie code to know [11:29] how your machine learning teams work and [11:31] how they build models, evaluate models. [11:34] And now let me show you specifically [11:36] what I mean, right? This allows Genie [11:39] code [11:40] to do things like building features in [11:43] the same way that your team builds [11:45] features today, using the exact same [11:46] patterns. [11:48] It allows Genie code to reuse the [11:51] training scripts and understand what are [11:53] our training workflows. How do we [11:54] actually build models? What's our [11:55] existing pipeline? [11:57] And and reuse that. So it actually fits [12:00] in with the workflows that your team [12:02] already has. [12:04] But it's not just about building. [12:06] It's also about, you know, bringing [12:08] these models to production and figuring [12:09] out what are the better models. And so [12:11] Genie code also respects the evals you [12:14] have. It finds the eval scripts that you [12:16] have, [12:17] and it reuses those. So when Genie code [12:19] says some a model is good, you can [12:21] actually trust it because you know it's [12:23] holding a model to the same bar that uh [12:26] your team already trusts. [12:30] And that allows Genie code [12:32] to really act like a senior machine [12:34] learning engineer on your team. [12:36] We've found that Genie code performs [12:39] way better on real-world data science [12:41] and machine learning tasks [12:43] than other leading coding agents now. [12:47] And this allows companies like Danfoss, [12:49] a multi-national engineering company, [12:53] to go from no machine learning [12:55] to [12:56] a complete production machine learning [12:59] system in just 90 minutes. [13:01] And this is a type of project that would [13:02] take weeks, months, and I know for a lot [13:06] of companies, a lot of customers in this [13:08] room, sometimes it takes quarters, [13:10] right? It's really hard to do this. [13:12] Uh this is a gigantic speed up. [13:15] And so we're really excited about that, [13:17] and that's just the build phase here. [13:19] >> [clears throat] [13:20] >> But the bad news is that the build [13:23] phase, if you talk to a real-world ML [13:26] team, [13:28] building is not the majority of their [13:30] job. They're actually spending 60, 70, [13:32] 80% of their time [13:34] on maintaining their machine learning [13:37] applications. It's just part of the [13:38] problem. And if we're building models [13:40] 100 times faster, [13:43] we're going to be building 100 times [13:44] more models. [13:45] And so we can't take on 100 times the [13:49] maintenance burden. No team can scale to [13:51] that. [13:52] And so that's why we need to think about [13:54] the operating side of this loop, as [13:56] well, right? After a model's in [13:57] production, [13:59] how do we keep it healthy? [14:01] >> [snorts] [14:02] >> And you know, anything can go wrong with [14:04] machine learning models. [14:06] Data can drift. [14:07] The model server, the inference server, [14:10] can have latency spikes. There can be [14:11] any number of reasons for errors. [14:14] Up- upstream data pipelines can break. [14:17] There's a long tail of a thousand [14:18] issues. [14:20] And to solve this and automate this, [14:23] something I'm really excited about, and [14:24] I think it's the biggest launch in [14:27] machine learning in the past 10 years. [14:29] I'm very excited to introduce Genie Zero [14:32] Ops for machine learning. This is an [14:33] agent that runs your machine learning [14:35] systems for you. [14:37] So there's a really big paradigm shift [14:38] with this. Instead of typical monitoring [14:41] systems that, you know, are are looking [14:44] at your metrics, and when they find an [14:46] issue, they bring you a problem and say, [14:48] "Hey, this this thing looks wrong." [14:51] Zero Ops for ML doesn't stop there. It [14:54] finds the problems, [14:56] and then it takes it upon itself to [14:58] discover the root cause, [14:59] come up with a fix autonomously, and [15:01] bring you the solution. [15:03] So instead of go getting problems given [15:05] to you all the time, you're getting [15:05] solutions brought to you. [15:08] And so it it monitors every model [15:10] continuously. It investigates these [15:12] issues and resolves them autonomously, [15:14] but it still leaves your team in the [15:16] loop to approve before anything deploys. [15:19] And the flip here is that for the whole [15:22] history of machine learning we've always [15:23] had a team member on call for our [15:25] models. [15:27] And zero ops for ML inverts that and [15:29] allows an agent to be on call and your [15:32] team is just in a position to approve [15:34] the recommendations from that agent. And [15:37] that allows [15:38] Genie zero ops for ML to solve all of [15:40] these kinds of problems we just talked [15:41] about [15:43] while still leaving your team in the [15:44] loop just for the final approval. [15:47] And so rather than me showing you a [15:49] bunch of more slides about this, I'd [15:51] like to invite Amber on stage to [15:52] actually show us how it works live. [15:55] Amber. [15:56] Thank you. [16:07] >> Thanks Mike for setting the stage and [16:10] hello everyone. My name's Amber and I'm [16:12] here to show you how Genie zero ops not [16:14] only finds issues with your ML systems [16:17] but fixes them automatically. At [16:19] Berkshire Sports Global, my team and I [16:21] have a model in production that [16:23] forecasts ticket sales for every game of [16:25] the Data and AI World Cup. And this [16:27] model is mission critical because that [16:30] forecast affects tens of millions of [16:32] dollars in event staffing and operation [16:34] spend. And we don't want to end up with [16:36] a lot of empty seats because we over [16:38] priced tickets. [16:40] With this model, I need to be alerted as [16:42] soon as any issues happen. So I'm going [16:45] to quickly use Genie code to help create [16:47] some alerts. [16:48] Going to create an alert. [16:52] And I'm going to ask Genie code to [16:54] create an alert that will trigger if [16:56] observed ticket sales are not within [16:58] plus or minus 5% of the forecasted sales [17:01] for any game. [17:02] Here we can see how Genie code builds [17:04] and registers the agent for me [17:05] automatically. No extra setup needed. [17:10] All right, I'll allow this um [17:13] yeah, alert creation here. [17:18] And the alert's created. Now, let's fast [17:21] forward to the end of the week. It's [17:23] Friday at 5:30 p.m. I'm coming out of [17:26] back-to-back meetings. I open up my [17:28] email and crap. That alert just fired 30 [17:31] minutes ago and the ticket forecast has [17:33] suddenly become less accurate. This is a [17:36] critical issue where every minute could [17:38] cost my company millions of dollars. So, [17:40] I need to just drop everything and debug [17:42] what's going on. But wait, there's this [17:45] new button. [17:46] View investigation in Genie Zero Ops? [17:49] That sounds promising. Let's see what it [17:51] does. [17:53] When I tap it, it opens up the Genie [17:54] Zero Ops inbox where we can see an open [17:56] thread with the highest severity. [18:00] So, I'm just going to go and click into [18:01] that in order to see the investigation [18:03] that Genie code started automatically [18:05] about 30 minutes ago. [18:07] So, instead of waiting for me to [18:09] investigate this issue manually, Genie [18:11] code got started immediately and has [18:13] already explored and tested dozens of [18:15] hypotheses. [18:17] That's amazing for me because these [18:18] issues are costly and every second [18:20] counts. Let's see what it did. [18:24] First, Genie code tries to understand [18:25] what triggered the alert and this is [18:27] where Genie code's context and ontology [18:29] shines. Most agents would provide a [18:32] surface-level generic suggestion that [18:34] still requires me to do a majority of [18:36] the investigation myself. [18:39] But Genie code goes deeper. It's powered [18:41] by the ontology of my full ML project, [18:43] so it analyzes uh notebooks that built [18:46] the model. It looks at Unity catalog [18:48] tables. It runs SQL queries. It's [18:51] checking for changes in the feature [18:54] values. It's running ML flow experiments [18:56] and more. [18:58] Overall, it really approached the [18:59] problem like a real ML expert, [19:01] continuously iterating until it found [19:03] the root cause. And my favorite thing is [19:05] all this is done in the background. Work [19:08] that would have taken me hours to do [19:09] manually, Genie code root caused it [19:11] before I even realized there was an [19:13] alert. [19:14] And it looks like it figured out the [19:16] underlying issue. Our model was missing [19:18] a feature that conveys the stage of the [19:20] tournament. This means that the model [19:22] was unable to account for the increased [19:23] demand and fan attention that the [19:25] knockout rounds receive. [19:27] So, as soon as the tournament entered [19:28] the knockout stage, our forecasts became [19:31] less ac- accurate, as you can see here. [19:36] And then, it didn't stop there. After [19:38] figuring out this issue, Genie code code [19:40] went even further. It started building [19:43] and testing for the solution. It figured [19:45] out that we had it existing data for [19:47] both the games, tournament stage, and [19:49] match-up. And then, it created features [19:52] based on that underutilized data. [19:54] Genie code then decided to retrain a [19:57] model with these features, did [19:59] hyperparameter tuning, registered the [20:01] model in MLflow, all while following [20:03] best practices for ML on Databricks. [20:07] That's awesome, and it essentially [20:08] solved the problem for me automatically. [20:11] And since Genie code has full lineage [20:13] across our ML workflow, it reused the [20:15] same evaluation criteria from the [20:17] previous model to confirm the new [20:19] model's accuracy. And it gets our [20:21] forecast back to where it needs to be. [20:23] Now, it's time to roll out this new [20:25] model to a small amount of traffic, and [20:27] I'm feeling really good because I can [20:29] trust what Genie code did cuz I saw the [20:31] evaluations it used. Now, I can deploy [20:33] this model in one click. Let's do it. [20:41] All right. [20:43] See version two. [20:45] And I'm done. [20:51] >> [applause] [20:53] >> Now, normally a stale model like this is [20:55] a major team-wide incident that [20:57] massively affects our business for [20:58] hours, if not days, potentially costing [21:01] tens of millions of dollars. But, Genie [21:03] code zero ops resolved it quickly and [21:05] with minimal disruption to my team. [21:08] And as Mike mentioned earlier, we're [21:09] entering the 100X era of ML. But with [21:12] 100 times more models comes 100 times [21:15] the complexity. And that's really what [21:16] we have to protect teams from. [21:19] And that's really the opportunity here. [21:21] Give ML tool Give ML teams the tools [21:24] that they need to stop spending all this [21:27] time fighting operational toil and start [21:29] building breakthrough AI. And that's [21:31] Genie Zero Ops for ML. Thank you. [21:38] >> Okay. [21:40] Pretty cool, right? [21:41] Going to have to jog here. [21:45] >> [snorts] [21:46] >> Okay, so we saw three things today. [21:48] The first is AI runtime in public [21:51] preview, serverless GPUs. You can build [21:53] your deep learning models and your LLMs [21:55] with it. [21:56] Go try it now. [21:58] Second, Genie code for machine learning [22:01] is generally available. [22:03] It's built into the Genie code you have [22:04] right now on your laptop, so you can [22:06] start using it today. [22:08] >> [snorts] [22:08] >> And third, Genie Zero Ops for machine [22:10] learning is coming this summer, so we're [22:13] really excited to work with you with [22:14] that at that time. [22:17] So think back to the dream I started [22:19] this talk with. The machine learning [22:22] industry has always been about [22:23] automating machine learning, and we're [22:25] finally at the point where we have the [22:28] the infrastructure and we have the [22:29] agents to drive the loop. [22:32] And that allows us to build a hunt [22:33] machine learning 100 times faster, [22:36] 100 times more models, [22:38] >> [snorts] [22:38] >> and we're entering this era now where we [22:40] can scale ML to all of the decisions [22:43] across our business, so we can have [22:45] truly personalized customer experiences [22:47] at every part of our product, and we can [22:50] really optimize every business decision. [22:53] So we're really excited uh to see what [22:55] you build. Um [22:57] let your agents do the work now, so your [23:00] team can focus on what's next. So, thank [23:02] you very much. [23:04] >> [snorts] [23:05] [music] [23:10] [music]