Uber Automates Routine Coding to Scale Engineering Output

Uber: Leading engineering through an agentic shift - The Pragmatic Summit — The Pragmatic Engineer

The Pragmatic Engineer

By moving developers from synchronous pair programming to autonomous background agents, Uber successfully offloads high-volume toil and shortens the product delivery lifecycle.

AI-assisted summary of Uber: Leading engineering through an agentic shift - The Pragmatic Summit — original source. Check the original for full context.

Source published:

Key ideas

  • Uber transitioned from synchronous IDE 'pair programming' to asynchronous 'peer programming,' with engineers acting as supervisors for long-running, multi-step autonomous agents. (3:36)
  • The developer platform team now prioritizes toil automation as their highest strategic pillar, as 70% of early agentic requests focused on maintenance tasks like dependency updates and bug fixes. (6:41)
  • Uber's internal AI developer tool spend has exploded by over 6x since 2024, necessitating a tiered model routing strategy to balance high-reasoning tasks with cost-efficient execution. (36:01)
  • Success metrics have shifted from raw code volume to business outcomes, specifically measuring the compression of end-to-end cycle time from design to production feature delivery. (35:24)

Detailed notes

## Background Published on YouTube by The Pragmatic Engineer on March 10, 2026, this presentation from The Pragmatic Summit features Anshu Chada and Ty Smith from Uber's Developer Platform team. According to the video description, the session explores how Uber is leading its engineering organization through an agentic shift, supported by updated internal metrics showing widespread adoption of autonomous background coding and command-line agents.

## TL;DR Chada and Smith detail Uber's strategic evolution from synchronous tab-completion tools to asynchronous, multi-agent peer-programming workflows focused on eliminating maintenance toil. To enable autonomous agents securely within their private monorepos and microservice architecture, Uber engineered an internal platform suite including Minion for background execution, Code Inbox for triaging the resulting code explosion, U Review for noise-filtered code feedback, and Shepherd for large-scale migrations. Despite surging developer satisfaction and massive task automation, the team confronts real-world friction such as developer cultural inertia, legacy system debt, and a more than sixfold increase in AI compute expenses. To maintain enterprise viability, Uber employs tiered model routing to curb token burn and measures business success through end-to-end product delivery cycle times rather than raw code volume.

## Atmosphere & style The presentation carries a grounded, candid, and highly pragmatic tone, pairing Chada's executive-level organizational framing with Smith's detailed architectural walkthroughs. Rather than indulging in uncritical AI boosterism, the speakers speak transparently about friction points, from reviewers feeling overwhelmed by synthetic pull requests to facing intense CFO scrutiny over spiraling model costs. Watching the session feels like attending an insider retrospective on the messy reality of enterprise transformation, marked by practical problem-solving and an organizational willingness to let internal tools become obsolete as frontier models evolve.

## Bulletin - [03:36](https://www.youtube.com/watch?v=i1tZN41VKcE&t=216s) Uber transitioned from synchronous IDE 'pair programming' to asynchronous 'peer programming,' with engineers acting as supervisors for long-running, multi-step autonomous agents. - [06:41](https://www.youtube.com/watch?v=i1tZN41VKcE&t=401s) The developer platform team now prioritizes toil automation as their highest strategic pillar, as 70% of early agentic requests focused on maintenance tasks like dependency updates and bug fixes. - [36:01](https://www.youtube.com/watch?v=i1tZN41VKcE&t=2161s) Uber's internal AI developer tool spend has exploded by over 6x since 2024, necessitating a tiered model routing strategy to balance high-reasoning tasks with cost-efficient execution. - [35:24](https://www.youtube.com/watch?v=i1tZN41VKcE&t=2124s) Success metrics have shifted from raw code volume to business outcomes, specifically measuring the compression of end-to-end cycle time from design to production feature delivery.

## Key takeaways - [01:15](https://www.youtube.com/watch?v=i1tZN41VKcE&t=75s) AI has been promoted to one of Uber’s six core corporate strategic pillars, focused on 'superhuman' augmentation rather than developer headcount reduction. - [14:25](https://www.youtube.com/watch?v=i1tZN41VKcE&t=865s) The transition to agentic workflows relies on the 'Minion' background agent platform, which runs inside Uber's existing CI infrastructure to ensure secure, low-latency access to internal monorepos. - [09:36](https://www.youtube.com/watch?v=i1tZN41VKcE&t=576s) A cross-functional 'tiger team' was established to build a centralized, production-grade Model Context Protocol (MCP) gateway to govern enterprise-wide agent connectivity. - [10:15](https://www.youtube.com/watch?v=i1tZN41VKcE&t=615s) The AIFX CLI serves as the standard entry point for developers, ensuring consistent configuration management, prompt defaults, and secure MCP server discovery. - [19:48](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1188s) Code Inbox was created to mitigate the 'code explosion' risk, serving as a smart triage dashboard that filters noise and enforces review Service Level Objectives (SLOs). - [21:24](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1284s) U Review, an internal platform, uses a 'Review Grader' layer to filter out low-confidence AI comments and ensure only high-value feedback reaches developers. - [24:16](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1456s) AutoCover, using Uber's internal LangFX framework, generates roughly 5,000 unit tests per month with significantly higher quality than generic LLM-based test generators. - [25:46](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1546s) Shepherd is a dedicated campaign management tool that allows for fleet-wide code migrations by orchestrating multi-phase, scoped pull request generation. - [30:53](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1853s) The team prioritizes architectural abstraction over specific tools, allowing them to remain 'unmarried' to technologies as the rapid pace of AI evolution renders existing solutions obsolete. - [32:43](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1963s) Cultural adoption is driven by peer-to-peer evangelism; grassroots promotion by influential engineers is significantly more effective than top-down executive mandates. - [36:01](https://www.youtube.com/watch?v=i1tZN41VKcE&t=2161s) Tiered model routing is the primary defense against spiraling AI costs, assigning high-cost frontier models only to complex architectural tasks. - [24:48](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1488s) The shift to AI-generated code necessitates an increased focus on rigorous unit test validation via an adversarial 'Critic Engine' to prevent brittle test mocks.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=i1tZN41VKcE&t=4s) Introduction and the ROI-Driven Agenda

Anshu Chada opens the session by welcoming the audience and introducing the presentation team from Uber's Developer Platform organization. Chada leads Developer Platform at Uber, while co-speaker Ty Smith is a principal engineer who has served as one of the primary engineering voices steering both Uber's recent agentic shift and its broader, multi-year artificial intelligence strategy. Eager to leave ample time for audience questions and debate at the end, Chada outlines a packed, three-part agenda designed to showcase both technical breakthroughs and bottom-line impact:

1. The core motivations driving Uber’s aggressive pivot toward agentic AI, alongside the tangible return on investment (ROI) that the engineering organization has already captured. 2. A technical deep dive led by Ty Smith into the specific platforms, systems, and integrations that unlocked these productivity gains. 3. A concluding exploration of the non-technical frictions encountered along the way, ranging from culture, org design, and developer mindsets to measurement frameworks and escalating cloud costs.

Chada emphasizes that this initiative is not speculative experimentation, remarking: “We're going to walk through what has motivated our push into agentic AI and some of the key ROI that we we've we've manifested.” He underscores the business reality behind their effort by adding, “And there is key ROI.”

---

[01:15](https://www.youtube.com/watch?v=i1tZN41VKcE&t=75s) Uber's Strategic AI Shift: Superhumans, Not Replacements

Machine learning and artificial intelligence are foundational to Uber's core business; proprietary AI models have powered the company’s marketplace matching and dynamic fares platforms for years, providing its defining competitive edge. What is fundamentally new over the past few years, however, is embedding AI directly into the internal software development lifecycle and developer productivity tooling. This internal adoption has elevated AI from a localized tool into one of six overarching corporate strategic pillars established by Uber CEO Dara Khosrowshahi. While the company initially termed this transition a migration from a human-and-early-AI company to a "generative AI-powered" enterprise, Chada quips that the label has already become passé in favor of becoming an "agentic-powered company."

Crucially, the leadership team explicitly rejects the narrative that AI exists to replace the human engineering workforce. Framing the executive vision, Chada references Khosrowshahi's stance: “Dara made this quote, which is, you know, AI AI is enabling people to become superhumans um in terms of their productivity and the impact that we can realize for our end users.” The goal is radical augmentation: using generative and agentic tools to eliminate repetitive engineering drudgery—such as library upgrades, technical migrations, and routine bug fixing—so engineers can dedicate their cognitive energy to creative problem-solving and rapid product feature delivery.

- AI has driven Uber’s external algorithmic core (fares and driver-rider matching) for nearly a decade. - CEO Dara Khosrowshahi designated AI adoption as one of Uber’s six company-wide strategic shifts. - Internal AI efforts focus squarely on offloading toil rather than reducing developer headcount.

---

[03:35](https://www.youtube.com/watch?v=i1tZN41VKcE&t=215s) From Pair Programming to Peer Programming

Reflecting on the evolution of developer tooling, Chada contrasts the early days of generative coding in 2022 and 2023 with the present paradigm. The introduction of tools like GitHub Copilot established the era of "pair programming," defined by synchronous inline tab-completions and in-IDE chat sidebars. While Uber’s internal tracking showed a respectable 10% to 15% bump in overall developer velocity from synchronous pair programming, it was ultimately an incremental optimization of traditional software workflows.

The true inflection point arrived over the past year with the transition to asynchronous "peer programming." Backed by advanced frontier models capable of sustained multi-step reasoning over hours rather than sub-second completions, engineers can now delegate end-to-end tasks to autonomous agents and step back into supervisory roles. Chada highlights this structural evolution in how software engineers operate: “And that's that's all culminated in this model where we imagine developers acting as their own tech leads.” Developers formulate intent, hand off long-running maintenance workloads—such as dead code elimination, documentation generation, and complex dependency updates—and provide high-level course corrections only when an agent requests review. Once dismissed as a social media punchline, "vibe coding" has matured into a viable enterprise workflow adopted across high-performing tech companies like Ramp and Anthropic.

```mermaid flowchart LR accTitle: Evolution of AI-Assisted Engineering accDescr: Diagram illustrating the shift from synchronous pair programming to asynchronous agentic peer programming.

subgraph Pair["Pair Programming (2022-2023)"] direction TB P1["Synchronous Tab-Completion"] P2["IDE Chat Sidebars"] P3["10-15% Velocity Bump"] P1 --> P2 --> P3 end

subgraph Peer["Peer Programming (Present)"] direction TB A1["Asynchronous Agent Hand-off"] A2["Developer as Tech Lead"] A3["Autonomous Execution (Hours)"] A1 --> A2 --> A3 end

Pair -->|"Paradigm Shift"| Peer ```

---

[06:41](https://www.youtube.com/watch?v=i1tZN41VKcE&t=401s) Toil Automation and the Virtuous Adoption Cycle

When Uber's Developer Platform team first rolled out autonomous agentic capabilities to the engineering organization, developers organically gravitated toward maintenance rather than net-new feature creation. Chada reveals that 70% of the initial workloads submitted to the platform were categorized as engineering toil. This behavioral pattern was driven by structural task characteristics: maintenance tasks like dependency upgrades, deprecations, and library migrations feature unambiguous start and end states, clear compiler feedback, and deterministic validation criteria, whereas greenfield features demand business context, UI experimentation, and complex product trade-offs.

Because these bounded toil tasks yielded substantially higher task accuracy and completion rates, developer trust surged rapidly. Chada describes how this dynamic unlocked self-reinforcing adoption across the company: “As soon as we made our agentic workflows available to developers, what we saw is 70% of the workflows that developers are pushing into the system were toil tasks.” High accuracy encouraged engineers to feed increasingly complex maintenance pipelines into the agentic system, sparking a virtuous cycle that led Chada’s Developer Platform leadership team to establish toil automation as their single highest strategic priority for internal AI tooling.

- 70% of all early agentic requests submitted by Uber engineers were toil-oriented workloads. - Bounded maintenance tasks (e.g., framework updates, dead code removal) exhibited significantly higher completion accuracy than open-ended product work. - Proven success in toil reduction drove organizational buy-in and made autonomous workflows a top platform priority.

---

[07:41](https://www.youtube.com/watch?v=i1tZN41VKcE&t=461s) Uber's Agentic Architecture: Standing on the Shoulders of Michelangelo

Handing over the presentation to Ty Smith, the discussion transitions from strategic motivation to system architecture. Smith emphasizes that Uber’s modern agentic ecosystem was not invented in a vacuum or assembled overnight. Instead, the platform team explicitly designed its agentic infrastructure on top of nearly a decade of mature machine learning investments: “And so we really see ourselves as building on the shoulders of giants.”

```mermaid flowchart TD accTitle: Uber Agentic Platform Architecture accDescr: Layered architectural diagram showing how Uber builds agentic workflows on top of Michelangelo, organizational context, and specialized agent runtimes.

subgraph Enablement["Governance & Enablement"] E1["Cost Control & Measurements"] E2["Developer Education & Guardrails"] end

subgraph Agents["Agent Execution Layer"] A1["Minion Background Agent Platform"] A2["Automated Test Generation"] A3["Industry Clients (IDE / CLI)"] end

subgraph Context["Organizational Memory & Context"] C1["Source Code & Repositories"] C2["Internal Docs, Jira & Slack"] C3["Central MCP Gateway"] end

subgraph Foundation["Michelangelo ML Foundation"] M1["Model Gateway (OpenAI, Anthropic Proxies)"] M2["Internal Model Hosting & Inference"] end

Foundation --> Context Context --> Agents Agents --> Enablement ```

At the foundation sits Michelangelo, Uber’s historic machine learning platform. Historically utilized for model training and feature stores, Michelangelo was modernized to include a unified Model Gateway. This gateway securely proxies requests to leading external frontier model providers (including OpenAI and Anthropic) while managing access to proprietary internal models.

Directly above the model gateway lies the contextual infrastructure that provides agents with institutional memory. To operate with human-level efficacy, agents must ingest internal source code, architecture documentation, Jira tickets, and active Slack discussions. Above this contextual tier, the team supports both external industry clients (such as IDE-based tooling and CLI agents) and purpose-built internal platforms, such as Minion (Uber’s asynchronous background agent runtime) and specialized test generation pipelines. The entire stack is capped by an engineering enablement layer focused on cost attribution, observability, and education.

---

[09:36](https://www.youtube.com/watch?v=i1tZN41VKcE&t=576s) The Cross-Functional Tiger Team and Central MCP Gateway

As the Model Context Protocol (MCP) gained widespread industry traction, Uber recognized it as a powerful mechanism to connect autonomous agents with company-wide services and data repositories. Rather than permitting fragmented, unvetted MCP integrations across individual engineering groups, platform leadership moved decisively to centralize infrastructure, security, and access control.

Smith explains that the company formed an elite, cross-functional "tiger team" drawn from infrastructure, platform, and security organizations across Uber. Their mission was to architect a unified enterprise MCP strategy and build a production-grade Central MCP Gateway. Smith explains their rapid mobilization: “This became a very popular piece of technology in the industry last year, and we moved very quickly to make sure that this was deployed and secure for our engineers so that we could make them as productive as possible.” The resulting gateway acts as an authenticated, auditable proxy between agent clients and underlying service infrastructure, standardizing authorization and governance across all internal and external MCP servers.

[10:15](https://www.youtube.com/watch?v=i1tZN41VKcE&t=615s) Platformizing Agent Tooling: Michelangelo, MCPs, and the AIFX CLI

To make agentic development scalable across thousands of engineers, Uber recognized early on that ad-hoc local configurations would quickly devolve into chaos. The developer platform team set up an infrastructure ecosystem anchored by a central Model Context Protocol (MCP) registry and sandbox. This sandbox gives engineers a safe playground where they can discover new MCP servers, test them against internal systems, and verify their behavior with comprehensive telemetry and logging before using them in workflows. Concurrently, Uber leveraged Michelangelo—its proprietary internal machine learning platform—to provide both code SDKs and no-code agent builders. Equipped with visualization, telemetry, and distributed tracing, these tooling foundations enable any team across the company to build custom agents that tap securely into internal services and proprietary datasets, while publishing them back to a shared registry for cross-company discovery and reuse.

Crucially, these agents needed to run uniformly across multiple environments, including Uber's DevPod remote development infrastructure, local developer laptops, background execution runners, and production services. Because engineers rely on a diverse and shifting set of clients—ranging from Claude Code and Codex to Cursor—Uber created a unified command-line entry point called the AIFX CLI. As Ty Smith explained: “One thing that we recognized we needed to platformize pretty quickly was a central ability to provision and configure and update the clients, the agent clients themselves, the ability to install and discover MCPs from the registry, configure those inside of the agent clients, deploy standard configuration management so that people who are just new to the space are having more effective prompts and configurations right away.”

```mermaid flowchart TD accTitle: Uber Agent Platform Architecture accDescr: Diagram showing how AIFX CLI bridges registries to developer clients and runtime environments.

subgraph Registries ["Central Registries & Builders"] R1["MCP Registry & Sandbox"] R2["Michelangelo Agent Builder<br/>(SDKs & No-Code)"] end

subgraph PlatformLayer ["Management Layer"] CLI["AIFX CLI<br/>(Client config, updates, prompt defaults)"] end

subgraph Clients ["Developer Clients"] C1["Claude Code"] C2["Cursor"] C3["Codex"] end

subgraph Runtimes ["Execution Environments"] E1["DevPod (Remote Dev)"] E2["Local Laptops"] E3["Minion Background Agents"] E4["Production Services"] end

Registries --> CLI CLI --> Clients Clients --> Runtimes ```

- **Michelangelo Agent Platform:** Supports both SDK-driven and no-code creation of internal agents with built-in tracing, telemetry, and access to internal microservices and datasets. - **AIFX CLI:** Serves as the developer-facing gateway to ensure baseline system prompts, approved client versions, and standardized MCP server connections across the engineering organization. - **Universal Environments:** Ensures consistent execution behavior whether an agent is executed inside an engineer's remote DevPod instance or deployed into background infrastructure.

---

[12:25](https://www.youtube.com/watch?v=i1tZN41VKcE&t=745s) The Multi-Agent Shift and Why Third-Party Infra Didn't Fit

The arrival of agentic systems has fundamentally disrupted the historic developer loop. In traditional software engineering, developers spent a modest fraction of their schedule planning, committed the bulk of their time to hands-on code authorship, and dedicated a smaller slice to peer review, cycling through a tight edit-build-run-verify cycle inside standard IDEs. The initial wave of AI integration altered this slightly by introducing interactive in-IDE chat or terminal agents (like Cursor or early Claude Code sessions) where the developer continually watched outputs, approved terminal execution steps, and steered the model command by command.

However, developer behavior at Uber quickly advanced to an asynchronous, multi-agent paradigm where engineers fire off long-running tasks autonomously in the background. Ty Smith highlighted the psychological shift driving this pattern: “This gets into this place where as an engineer you're giving a prompt, you're waiting for some time while it's running. You're thinking, 'Oh, what am I going to do? Am I going to go have a coffee or browse Reddit? Might as well kick off another background agent.' And so they get into this mode of the new flow looks like running several agents at once.”

While external commercial vendors were beginning to offer their own background agents, they almost universally hosted compute on their own third-party cloud infrastructure. Handing over enterprise background execution to external environments presented immediate security and operational hurdles for Uber, as their workloads required direct, low-latency access to massive internal monorepos, private networks, and internal microservices. Ty Smith noted that while third-party hosting might evolve over time, “having the ability to bootstrap on our own infrastructure was really important to us and allowed us to move really quickly.”

- **Workflow Divergence:** The traditional edit-build-test loop has been superseded by an asynchronous model where developers orchestrate and supervise multiple parallel background jobs. - **Infrastructure Sovereignty:** Running background agents on vendor-hosted runners introduces isolation from enterprise monorepos and internal network perimeters, prompting Uber to build in-house execution runners.

---

[14:25](https://www.youtube.com/watch?v=i1tZN41VKcE&t=865s) Minion in Action: Uber’s Autonomous Background Agent Platform

To run background agents entirely within their security perimeter, Uber built Minion, an autonomous background agent execution platform. Minion integrates directly with state-of-the-art agent CLIs and SDKs while running on top of Uber's existing continuous integration (CI) platform. Because it operates natively inside Uber's CI cluster, Minion tasks spin up with the target monorepos already checked out, full network access to internal microservices pre-authenticated, and connections to internal MCP servers configured via AIFX. Minion surfaces across multiple interfaces: a dedicated web UI, Slack notifications and commands, GitHub pull request comments, the AIFX CLI, and programmatic APIs that other internal services can invoke.

Walking through an actual production use case, Ty Smith demonstrated how an engineer resolved an unexpected crash. A user had reported an error message encountered while running a command on macOS. Rather than switching contexts, checking out branches, and reproducing the bug locally, Smith pasted the raw command, operating system details, and error output directly into the Minion web dashboard. The Minion interface provides curated prompt templates with parameter placeholders, lets the author target specific monorepos or pull requests, and allows selecting the underlying driver agent (such as Claude Code). Because Uber has been in a multi-year migration from Phabricator to GitHub, Minion seamlessly supports outputting code changes as either a Phabricator Differential diff or a GitHub pull request.

The system also includes a built-in prompt improver. When Smith entered his raw report, the interface flagged a red indicator signaling that the prompt quality was sub-optimal and carried a lower probability of success; clicking the improver analyzed the text and suggested automated additions that Smith accepted. Once launched, Minion pinged the author via Slack with a task tracking link. Exactly seven minutes later, a follow-up Slack message confirmed the job was complete. The dashboard provided full inspection logs and retry options, but in this case, Minion had already opened a clean GitHub pull request co-authored by the Minion bot and Smith, populated with a linked Jira ticket and an explicit test verification plan.

```mermaid flowchart LR accTitle: Minion Task Execution Flow accDescr: End-to-end flow from developer prompt entry in Minion web UI to an opened GitHub pull request.

subgraph Input ["Task Creation"] A["Developer Prompt<br/>(Error + Environment)"] --> B["Prompt Improver<br/>(Quality Analysis)"] B --> C["Select Target Repo &<br/>Driver (Claude Code)"] end

subgraph Execution ["Minion Platform (CI)"] C --> D["Uber CI Runner<br/>(Monorepo + MCPs)"] D --> E["Autonomous Fix &<br/>Verification Run"] end

subgraph Output ["Completion & Notification"] E --> F["Slack Ping<br/>(7 min elapsed)"] E --> G["Generated Pull Request<br/>(Co-authored + Jira + Tests)"] end ```

- **Monorepo Readiness:** Minion avoids slow environment provisioning by running directly on CI compute nodes where monorepo caches and network routes already exist. - **Prompt Quality Guardrails:** An automated prompt evaluator flags under-specified instructions with a visual warning and offers an inline prompt rewriting engine to maximize task completion rates. - **Seven-Minute Turnaround:** In the demonstrated real-world bug fix, Minion understood the Mac-specific crash, implemented the fix, verified tests, and produced a review-ready PR with Jira metadata in seven minutes.

---

[18:41](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1121s) Code Explosion and Tackling Context Switching with Code Inbox

The proliferation of autonomous agents has flipped the traditional engineering bottleneck on its head. Because agents can generate code orders of magnitude faster than humans can write it, developers are spending an unprecedented percentage of their working hours planning prompts and performing code reviews. Ty Smith highlighted the cultural friction this causes across teams: “They're spending more and more time in planning and code review because there's so much more code being generated that they're being forced to do it. This probably isn't the favorite type of work the developers love doing, code review.”

This code explosion carries severe operational risks. When developers face dozens of pull requests waiting for review, cognitive fatigue sets in. Review turnaround times slow down, and worse, reviewers risk rubber-stamping diffs and letting critical bugs slip through into production simply because the sheer volume of code overwhelms human attention. Furthermore, managing dozens of asynchronous tasks across Minion, GitHub, and local environments introduces crippling context-switching overhead whenever an agent requires intervention or finishes a run.

To tame this distraction stream, Uber's developer platform team developed Code Inbox, a specialized application acting as a unified triage dashboard for pending code reviews. Rather than inundating an engineer with raw chronological GitHub notifications and Slack pings, Code Inbox filters out passive noise and surfaces strictly actionable items. A pull request or agent execution only appears in the engineer’s queue when it actively demands their input or decision, keeping engineers focused on unblocking their teammates and agents without drowning in automated notifications.

- **Review Burden Shift:** The primary time sink for developers has inverted from writing code to reviewing massive volumes of synthetic, AI-generated pull requests. - **Code Inbox Triage:** Centralizes incoming pull requests across systems into a filtered feed, hiding PRs that are stalled or waiting on others and elevating only those requiring immediate human intervention.

[19:48](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1188s) Code Inbox: Smart Reviewer Assignments, Noise Control, and Blast Radius Analysis

To streamline the human side of code review, Uber invested heavily in "Code Inbox," an intelligent routing and notification engine designed to pair pull requests with the right reviewers without spamming engineers. Rather than relying on static code owners files, the system analyzes compliance boundaries, working history, and real-time developer context such as time zones and calendar availability. Once assigned, Code Inbox monitors progress against strict Service Level Objectives (SLOs), providing visibility into pending reviews and automatically triggering reassignments or leadership escalations when reviews stall.

Notification delivery is deeply tuned to developer psychology and daily schedules. The platform batches notifications to prevent notification fatigue and coordinates with Slack to respect calendar focus blocks, holidays, and out-of-office statuses. For organizations that already rely on team-level Slack review channels or triage queues, Code Inbox injects incoming reviews directly into those workflows. In parallel, the platform estimates the risk profile of each incoming patch by evaluating its blast radius, total surface area, and whether it touches a lightweight test or a core production service. These risk insights are surfaced upfront so developers know when a change warrants heightened scrutiny or an extra reviewer.

- Analyzes code ownership, compliance rules, historical contributions, time zones, and calendar availability to assign reviewers. - Enforces strict review SLOs with automated escalation and reassignment for stalled PRs. - Batches Slack notifications and halts alerts during developer focus time and scheduled time off. - Plugs directly into existing team-level Slack code review queues. - Computes blast-radius estimates and service criticality to flag high-risk changes directly to reviewers.

---

[21:24](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1284s) U Review: An In-House Graded AI Code Review Pipeline

Addressing the actual content of code reviews led Uber to build U Review, an internal platform tailored to provide intelligent, contextual feedback on incoming changes. While Ty Smith acknowledged the proliferation of commercial review tools like CodeRabbit and Graphite—noting that Uber frequently tests and integrates external offerings—he emphasized that Uber's internal complexity and its ongoing migration from Phabricator to GitHub necessitated a unified, proprietary platform. U Review acts as a controlled gateway that runs a preprocessor over code diffs, orchestrating a suite of plugins that include internal defect bots, best practice checkers, Model Context Protocol (MCP) servers, and third-party tools via an open API.

```mermaid flowchart LR accTitle: U Review AI Code Review Pipeline accDescr: Pipeline showing code changes flowing through preprocessing, grading, deduplication, and dual-UI rendering.

subgraph Ingestion["Input & Analysis"] Diff["Code Change"] --> Prep["Preprocessor & Plugins"] Prep --- Bots["Defect Bots & MCPs"] Prep --- Ext["External Review Bots"] end

subgraph Quality["Filtering & Evaluation"] Prep --> Grader["Review Grader"] Grader --> Dedupe["Deduplication & Categorization"] end

subgraph Delivery["Surfacing to Developer"] Dedupe --> UI["Dual UI Layer"] UI --- Phab["Phabricator UI"] UI --- GH["GitHub PRs"] end ```

The core breakthrough in U Review is its dedicated "Review Grader" layer. Ty Smith pointed out that off-the-shelf bots often generate low-value noise and pedantic nits just to surface feedback. U Review runs specialized evaluation models at each layer of the pipeline to measure model performance against specific tasks, ensuring that only high-confidence, actionable comments reach the author. Over the past year of iteration, the platform saw an increase in the volume of best-practice suggestions while simultaneously sustaining a high resolution rate from engineers, validating that the comments are genuinely helpful rather than annoying. Because Uber operates across two code review ecosystems during its transition, U Review powers custom interfaces in both GitHub and Phabricator with built-in feedback loops. As Ty Smith explained when discussing why aggressive filtering is essential:

> “And we really only want to put the high confidence changes that the developer really needs to focus on, not little nits.”

- Combines internal defect bots, architectural best practices, MCP integrations, and external vendor bots through a single API. - Implements a Review Grader stage to eliminate low-confidence comments, style nits, and duplicates. - Uses distinct, benchmarked LLM models optimized for each stage of evaluation and categorization. - Tracks comment resolution rates as a primary success metric to ensure developers fix flagged issues rather than dismissing them as spam. - Features dual-UI compatibility supporting both legacy Phabricator diffs and modern GitHub pull requests.

---

[24:12](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1452s) AutoCover and the Critic Engine: Scalable, High-Quality Unit Test Generation

As AI dramatically accelerates the volume of incoming code, maintaining rigorous CI verification and automated testing becomes critical to prevent subtle defects from slipping through review. To solve this, Uber created AutoCover, a specialized unit test generation system built by an engineer whom Ty Smith spotted in the summit audience. While generic coding assistants and command-line tools can write test boilerplate, Uber discovered that building a domain-specific agent atop LangFX—Uber's internal LLM framework built on LangChain—produced test suites of substantially higher engineering quality.

AutoCover generates and merges roughly 5,000 unit tests every month across Uber's repositories, achieving nearly triple the quality rate of generic LLM agents. To protect against the hazard of brittle "change detector" tests and false confidence from inflated line coverage, the team incorporated an adversarial "Critic Engine." This module evaluates whether generated tests genuinely verify business logic or merely mirror implementation details. Uber subsequently decoupled the Critic Engine into a standalone "Test Validator" tool, enabling developers to run automated quality grading against both human-written and AI-generated tests across the company.

- Generates and successfully merges approximately 5,000 automated unit tests per month across Uber. - Outperforms off-the-shelf LLM coding tools by nearly 3x in test quality by leveraging Uber's internal LangFX framework. - Incorporates a Critic Engine to catch change-detector tests, superficial assertions, and brittle test mocks. - Offers the Critic Engine as an independent Test Validator for grading human-written and AI-authored tests alike.

---

[25:46](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1546s) Auto Migrate and Shepherd: Managing Large-Scale Code Migrations at Scale

Transitioning to code maintenance and toil reduction, Ty Smith reflected on the industry narrative surrounding AI-generated code. Hearing tech executives announce to news outlets and shareholders that massive percentages of their codebases were now AI-authored prompted Uber's platform team to investigate how engineering organizations like Google and Meta achieved those milestones. They realized those companies had previously built foundational infrastructure for Large-Scale Changes (LSCs), creating a structural prerequisite that generative AI could amplify. Lacking that foundational automation, Uber launched a major cross-functional program called "Auto Migrate" to industrialize codebase-wide modifications across four stages: problem identification, code transformation, automated validation, and campaign management.

```mermaid flowchart TD accTitle: Auto Migrate Four-Stage Architecture accDescr: The four key phases of Uber's Auto Migrate platform powered by Shepherd.

P1["1. Problem Identification"] -->|"Slice PRs & assess risk"| P2["2. Code Transformation"] P2 -->|"Run LLM agents or OpenRewrite"| P3["3. Validation"] P3 -->|"Verify via CI, tests & canary signals"| P4["4. Campaign Management"]

subgraph P4["4. Campaign Management (Shepherd Platform)"] YAML["Declarative YAML Config"] Engine["PR Generation & Rebase Engine"] Route["Queue Routing & Code Inbox Integration"] YAML --> Engine --> Route end ```

To manage the logistical complexity of distributing hundreds of automated pull requests to code owners, Uber engineered Shepherd, a campaign management platform. Migration authors define their campaigns using a declarative YAML file, specifying either a prompt for an AI agent or a deterministic script using tools like OpenRewrite. Shepherd orchestrates the PR lifecycle from end to end: generating targeted diffs strictly scoped to individual ownership boundaries, rebasing and refreshing PRs on defined schedules, routing them into appropriate team review queues, and synchronizing with Code Inbox. Demonstrating Shepherd in action, Ty Smith showcased a campaign upgrading internal Java services to Java 21 using deterministic OpenRewrite recipes, where the platform automatically generated clean, correctly scoped pull requests directly to the respective service owners. Describing the strategic realization that prompted the project, Ty Smith noted:

> “And we'd had a lot of discussions and seen that they had fundamentals that Uber hadn't invested before. Which was the ability to kind of scale out large-scale changes so that the AI can then build on top of that.”

- Organizes large-scale refactoring into four distinct phases: problem identification, transformation, validation, and campaign management. - Leverages both non-deterministic LLM agents (such as Claude Code) and deterministic engines (such as OpenRewrite). - Validates automated diffs using CI pipelines, unit tests, and staging or production canary signals before requesting human approval. - Exposes a dedicated Shepherd web UI driven by YAML configurations to coordinate, rebase, and track hundreds of migration pull requests. - Integrates directly with Code Inbox to deliver scoped PRs into team queues without manual intervention. - Successfully orchestrated fleet-wide upgrades of internal Java services to Java 21.

[29:02](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1742s) Automated Performance Fixes and Minion Integration

Ty wraps up the technical deep dive by demonstrating another practical application integrated into the Minion platform as an agent. Within Uber’s programming systems group, separate analysis tools continuously run to detect performance issues and generate high-quality diagnostic data. While Ty initially mixes up the name with an internal project called Dr. Fix, he clarifies that this particular agent ingests the performance data and generates code changes (PRs or diffs) to resolve the bottlenecks.

These automated changes are then piped directly through Shepherd, Uber's code review system. Each generated change includes standardized documentation explaining precisely how it was tested, how it was verified, and what the human developer needs to look for in order to review and land the code safely. With the deep dive concluded, Ty passes the presentation over to Anshu Chada to address the non-technical hurdles of leading an agentic shift.

```mermaid flowchart LR accTitle: Automated Performance Remediation Architecture accDescr: Diagram showing performance analysis tools generating data, feeding into the Minion platform agent, and creating PRs in Shepherd with verification details.

subgraph Analysis["Detection"] A["Programming Systems Tools"] -->|"generate diagnostic data"| B["Performance Issues"] end

subgraph AgenticCore["Minion Platform"] B --> C["Automated Fix Agent"] end

subgraph Review["Verification & Review"] C -->|"generate PRs / diffs"| D["Shepherd Platform"] D --> E["Standardized Review Brief<br/>(Tests, Verification, Safety)"] end ```

[29:53](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1793s) Business Uncertainty, Tool Churn, and Build vs. Buy

Anshu steps up to examine the human and business realities that complicate large-scale AI adoption. Using the Olympics as a timely visual analogy, he illustrates how the podium of leading AI technologies keeps rotating: the most powerful models for a given task change constantly, and the line between building in-house versus buying third-party SaaS shifts under their feet. In an engineering organization of Uber's scale, initiating projects like AutoCover (automated test generation) or AutoMigrate represents a massive capital and human commitment involving dozens of engineers over many months; leadership cannot afford to pivot on a whim every quarter.

To navigate this volatility without paralyzing execution, Uber leans on two core principles: architectural abstraction and organizational detachment from internal tools. By maintaining strict abstraction layers across platforms like Minion, the team can swap underlying foundation models or components seamlessly whenever superior technology emerges. More fundamentally, the engineering organization has had to cultivate a mindset that embraces obsolescence:

> “And so it's really important for us to not be married to the tech that we're building and being okay if something comes along like if the the co-founder cursor talked about the the auto the test coverage system that that might be coming a couple weeks. I'm really excited about that. It might make our auto auto cover infrastructure obsolete and that's okay because at the end of the day we need to deliver impact for Uber.”

[32:04](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1924s) People Challenges: Legacy Debt and Cultural Inertia

Beyond model selection, cultural and technical inertia present stubborn obstacles. Uber's monorepo and microservice landscape contain fifteen years of legacy code, spanning both world-class infrastructure and obscure, archaic systems that few remaining employees comprehend. Exposing this heterogeneous codebase to agents via Model Context Protocol (MCP) endpoints has proven to be an ongoing struggle.

Simultaneously, developer habits die hard. To show executive leadership what was possible, Anshu ran a hands-on session where four VPs landed production code in just 24 minutes—the first time any of them had shipped code in years. Yet despite such eye-opening demos, broader grassroots adoption across the company lagged behind expectations. Engineers are hardwired to write code from scratch in an IDE; handing autonomy over to background agents feels like an unnerving leap of faith. Top-down mandates to hit specific adoption metrics produced shallow improvements, but real cultural momentum only occurred through peer-to-peer evangelism:

> “Um so, that's been the the tactic that we're we're pushing on now is key promoters pushing techniques to their peers, because those promoters are typically engineers, and engineers trust other engineers as opposed to directors like me.”

- Top-down executive decrees artificially inflate tracking metrics without necessarily creating sustainable workflow shifts. - Showcasing tangible, peer-led wins in developer channels triggered organic spikes in tooling adoption. - Retrofitting older systems with modern MCP interfaces remains one of the heaviest engineering lifts for agent integration.

[34:13](https://www.youtube.com/watch?v=i1tZN41VKcE&t=2053s) Measuring Business Value Beyond Activity Metrics

Objective internal indicators demonstrate that generative AI is transforming developer productivity at Uber: developer net promoter scores (NPS) and self-reported engineering satisfaction sit at all-time company highs, while AI-attributed code volume continues to surge. The adoption curve hit an aggressive inflection point once high-reasoning models like Claude 3.5 Sonnet and Opus arrived and were paired with Uber's Minion platform. A wide gap has emerged between casual users and power users who leverage agentic workflows at least 20 days a month.

However, Anshu stresses that these figures are strictly activity metrics rather than commercial outcomes. As AI infrastructure expenditures balloon, executive leadership expects a clear financial return:

> “You know, our CFO has has asked me uh what is the impact of this? Right? I You know, I can't put him to diffs. I need to show him what's the impact on on revenue.”

To bridge the gap between engineering activity and enterprise value, the team is instrumenting the end-to-end product delivery pipeline. Instead of counting diffs or landed lines of code, the platform team is benchmarking the cycle time from an initial product design artifact to the moment a feature experiment goes live in production, proving whether agentic tools actually compress time-to-market.

[36:01](https://www.youtube.com/watch?v=i1tZN41VKcE&t=2161s) Confronting Spiraling AI Costs with Model Routing

Anshu bluntly summarizes the operational reality facing platform leaders: “the cost of AI is too damn high.” Since 2024, Uber’s spending on AI development tooling has exploded by at least 6x, evolving from a line item an engineering director could absorb within a team discretionary budget into an enterprise expenditure requiring explicit sign-off from the CFO. With GPU and memory hardware prices keeping external pricing elevated at vendors like Anthropic, Uber has had to adopt aggressive internal cost controls.

Rather than asking individual software engineers to manually balance cost versus capability, Uber's platform infrastructure automates the routing. For compound workflows orchestrated in Minion, the platform routes upstream reasoning and high-level architectural planning to premier, high-cost frontier models, while delegating low-level, repetitive code execution and linting to cheaper, high-throughput models. This keeps developer friction to a minimum while reining in token burn, a discipline that requires continuous tuning as Uber expands its enterprise toolbelt across Cursor, JetBrains AI, and Warp terminal environments.

- AI developer tooling expenses increased over 6x between 2024 and 2026. - Strategic tiered routing uses frontier models for project planning and cost-efficient models for localized task execution. - Newly onboarded tools like JetBrains AI and the Warp command line agent introduce distinct commercial billing structures that require continuous optimization.

## Notable quotes > [03:36](https://www.youtube.com/watch?v=i1tZN41VKcE&t=216s) And that's that's all culminated in this model where we imagine developers acting as their own tech leads. — Anshu Chada

> [18:41](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1121s) They're spending more and more time in planning and code review because there's so much more code being generated that they're being forced to do it. — Ty Smith

> [22:12](https://www.youtube.com/watch?v=i1tZN41VKcE&t=1332s) We really only want to put the high confidence changes that the developer really needs to focus on, not little nits. — Ty Smith

> [33:44](https://www.youtube.com/watch?v=i1tZN41VKcE&t=2024s) Um so, that's been the the tactic that we're we're pushing on now is key promoters pushing techniques to their peers, because those promoters are typically engineers, and engineers trust other engineers as opposed to directors like me. — Anshu Chada

## Mentioned Uber, Anshu Chada, Ty Smith, Dara Khosrowshahi, Michelangelo, Minion, AIFX CLI, Code Inbox, U Review, AutoCover, Shepherd, Claude Code, Cursor, GitHub, Phabricator, OpenRewrite, Model Context Protocol (MCP), Anthropic, OpenAI, LangFX

## Video description With Ty Smith and Anshu Chada, Uber Dev Platform. At The Pragmatic Summit: www.pragmaticsummit.com

Update on 11 March: the Uber team shared updated numbers as of March 2026:

- 84% of devs at Uber are agentic coding users (either using CLI-based agents or making more agentic requests than tab-completion in their IDE)

- 65-72% of code is AI-generated inside IDE-based tools. This number is, naturally, 100% for AI command line tools like Claude Code.

- Claude Code usage nearly doubled in 3 months — from 32% in Dec to 63% in Feb, while IDE-based tools (Cursor, IntelliJ) have plateaued.

See more context in this article: https://newsletter.pragmaticengineer.com/p/how-uber-uses-ai-for-development

Watch the session with Q&A also included: https://newsletter.pragmaticengineer.com/p/the-pragmatic-summit-recordings

0:00 Intro 6:11 Toil automation 8:02 Uber’s agentic platform architecture overview 11:16 The cross-functional tiger team 17:46 Async multi-agent workflow model 20:32 Minion background agent platform 26:55 Agent logs, observability, and debugging workflows 31:20 AI code review pipeline and quality grading 36:50 Testing 40:38 Migrations 43:55 Non-technical challenges

[Source](https://www.youtube.com/watch?v=i1tZN41VKcE)

Reading summary

Uber transitioned its developer platform from synchronous coding assistants to autonomous, multi-agent workflows aimed at offloading repetitive maintenance toil rather than reducing developer headcount. To make this shift practical at scale, the engineering organization built internal execution and review systems while managing escalating cloud token costs and persistent cultural inertia. From Inline Pair Programming to Asynchronous Peer Programming (3:35) Uber's early experiments with generative tooling—such as GitHub Copilot for inline tab-completions and in-IDE chat—yielded a modest 10% to 15% velocity bump. The primary operational inflection occurred when the company transitioned to asynchronous "peer programming," where engineers delegate multi-step, multi-hour maintenance tasks to autonomous agents and act effectively as their own tech leads. When agentic capabilities were opened to developers, roughly 70% of early workloads organically targeted engineering toil such as framework upgrades, deprecations, and library migrations. In-House Infrastructure for Background Multi-Agent Execution (14:25) Because external vendor runners could not access Uber's private microservices or large internal monorepos with low latency, Uber platformized its own agent runtime named Minion. Operating natively on Uber's continuous integration cluster, Minion executes tasks via state-of-the-art agent CLIs with pre-checked-out repositories and network permissions. The platform provides a web UI, Slack integration, prompt evaluation guardrails, and unified configuration via the AIFX CLI. In one live scenario, Minion resolved a Mac-specific crash, generated test verification, and opened a co-authored GitHub pull request with Jira links in seven minutes. Industrializing Review, Testing, and Monorepo Refactoring (18:41) Synthetic code generation shifted the engineering bottleneck toward code review, creating cognitive fatigue and the risk of uncritical rubber-stamping. Uber addressed this review deluge with Code Inbox, an intelligent triage system that filters non-actionable alerts, respects developer focus schedules, and assesses change blast radius. It also deployed U Review, an internal review pipeline equipped with a "Review Grader" model that filters out pedantic nits, retaining only high-confidence defect and architectural guidance across both Phabricator and GitHub interfaces. To ensure codebase integrity during this volume surge, Uber deployed AutoCover, an agent built on its internal LangFX framework that generates roughly 5,000 merged unit tests per month. The system includes an adversarial Critic Engine—also available as a standalone Test Validator—to eliminate brittle "change detector" tests. For broad structural refactoring, the Auto Migrate program pairs deterministic engines like OpenRewrite with agentic models, coordinating PR generation, rebasing, and queue distribution through a campaign management engine called Shepherd. Overcoming Cultural Friction and Spiraling Token Expenses (32:04) Despite rapid demonstrations—such as four vice presidents landing production code in 24 minutes—grassroots developer adoption faced headwinds from deeply ingrained IDE habits and fifteen years of legacy system debt. Top-down usage mandates inflated tracking numbers artificially, leading platform leadership to pivot toward peer-to-peer evangelism by engineering champions to build genuine trust. With AI tooling costs expanding over 6x between 2024 and 2026, Uber instituted dynamic model routing in Minion to restrict frontier models to complex planning while assigning routine execution to cheaper models. As platform spending faces CFO scrutiny, leadership is working to move beyond vanity activity metrics—like pull request counts—to prove measurable ROI by tracking cycle time reductions from product design to live production experiments.

Source quotes

[3:36] Anshu Chada: And that's that's all culminated in this model where we imagine developers acting as their own tech leads. [18:41] Ty Smith: They're spending more and more time in planning and code review because there's so much more code being generated that they're being forced to do it. [22:12] Ty Smith: We really only want to put the high confidence changes that the developer really needs to focus on, not little nits. [33:44] Anshu Chada: Um so, that's been the the tactic that we're we're pushing on now is key promoters pushing techniques to their peers, because those promoters are typically engineers, and engineers trust other engineers as opposed to directors like me.

Original transcript

[0:04] Awesome. Good afternoon, folks. Thank [0:06] you so much [music] for joining. I [0:07] really appreciate it. [0:09] Yes, my name is Anshu. I lead the [0:10] developer platform organization at Uber. [0:13] And Thai is a principal engineer. He's [0:14] been one of the the leading engineering [0:17] voices that's led our agentic shift, but [0:19] also our overall AI strategy over the [0:21] past couple years. [0:25] All right, let me try to get out of the [0:26] way. [0:27] We have a pretty [0:29] packed agenda, but I want to try to get [0:30] to the end as fast as possible because [0:31] I'm curious about your folks' [0:33] perspective on what I'm going to talk [0:35] about. [0:36] We're going to walk through what has [0:38] motivated our push into agentic AI [0:42] and some of the key ROI that we we've [0:45] we've manifested. And there is key ROI. [0:48] I'm really excited about the impact that [0:50] that we realized for Uber. [0:52] And then I'm going to hand off to Thai [0:53] and he's going to get into the [0:53] specifics, the actual technologies that [0:55] we built or integrated that has resulted [0:58] in that AI [1:00] that impact. [1:01] And then I'm going to end the talk [1:03] talking about some of the non-technical [1:05] challenges that we've been dealing with. [1:07] Organizational and cultural basically [1:09] people challenges, measurement, and of [1:11] course cost. [1:15] Okay, so AI is not new to Uber. [1:19] Our fares platform, our matching [1:21] platform has been using AI methodology [1:24] for years and years. It's one of the [1:25] things that sets us apart from the [1:27] competitors. [1:28] But over the past few years using AI as [1:32] part of the the engineering productivity [1:34] and engineering life cycle, that's [1:36] fairly new. [1:37] And [1:39] it's AI's integration into not just [1:42] engineering, but all aspects of what [1:44] Uber employees do [1:46] has become [1:47] a bigger and bigger part of what we want [1:49] to focus on. So much so that Dara has [1:52] stated that AI is one of our six [1:55] strategic shifts. We move from a [1:58] early from a human {slash} early [2:00] AI-powered company into a generative [2:03] AI-powered company. Now, I will say that [2:05] that term is really passe these days. [2:08] Nowadays, it's more fashionable to be an [2:10] agentic-powered company, but the concept [2:12] still holds where from the metrics, from [2:14] the data that that we've gathered, [2:17] um Dara made this quote, which is, you [2:19] know, AI AI is enabling people to become [2:21] superhumans um in terms of their [2:24] productivity and the impact that we can [2:26] realize for our end users. [2:28] So, from that standpoint, we want to [2:31] enable all tasks that people do at Uber [2:34] um to be supported by generative AI to [2:38] augment human productivity. [2:40] And that last part is really important [2:42] because um what we're not pushing for is [2:47] AI automate all humans in in the [2:50] company, [2:51] right? And especially on the on the [2:53] engineering side, what we found is we [2:56] want to focus on enabling our engineers [2:58] to focus on creative work rather than [3:01] toil. [3:02] I'm going to get into some of the [3:02] metrics um and the impact that we [3:05] realized from that, but the as we've [3:07] unlocked AI, what we found is [3:10] when we push some of the boring stuff to [3:13] it, [3:14] upgrades, migrations, bug fixes, [3:18] um not only does it result in much [3:20] higher satisfaction for our engineers, [3:21] they're able to push our product and [3:23] create features for end users in ways [3:26] that we didn't even thought was possible [3:28] and at velocities that were just have [3:29] just been incredible. [3:31] So, this has been the place that we've [3:32] really been uh [3:34] been doubling down on. [3:35] Now, one of the reasons we've been able [3:37] to push on this is the capabilities of [3:39] uh of the technology and the industry. [3:42] And [3:43] part of it first part of it is we going [3:46] from um pair programming to peer [3:49] programming. So, [3:51] if you think about back in the good old [3:53] days of 2022 and 2023, [3:56] uh when GitHub Copilot first came out, [3:58] it was pretty uh novel way of augmenting [4:01] development. You had a system where you [4:03] could do synchronous tab completion and [4:06] IDE chat window that would help [4:08] developers move faster. [4:10] We saw it [4:11] ourselves in our in our metrics. We saw [4:13] about a [4:13] maybe a 10 to 15% bump in overall dev [4:16] velocity. [4:18] This is pretty phenomenal, but this by [4:19] itself didn't push us in the direction [4:22] that that we've seen over the past, [4:24] let's say, year, [4:25] where the paradigm has shifted to [4:28] peer programming, [4:29] where you could hand off workloads that [4:31] are running asynchronously, [4:34] uh [4:35] and the models that that we use are so [4:38] good, they're so accurate, that all you [4:41] need to do with the AI agent is to [4:43] redirect in certain ways, maybe give it [4:45] some some course correction. [4:49] Um and that's that's all culminated in [4:52] this model where we imagine developers [4:55] acting as their own tech leads. [4:58] Right? Developers are are uh directing [5:00] AI agents using a variety of the [5:02] different models and capabilities that [5:03] are available to be able to execute uh [5:06] asynchronously and come back for for [5:08] direction. Now, [5:10] this doesn't work for every single task, [5:12] but again, when we think about some of [5:14] the toil work that developers need to [5:16] do, dead code cleanup, writing docs, [5:18] library migrations, these all seem basic [5:22] um these are basic operations, but [5:25] they're absolutely essential for [5:27] maintaining a healthy codebase. [5:29] But, they don't by themselves help to [5:30] grow the business. So, pushing these [5:32] workloads to AI agents in effect helps [5:35] to grow the business. [5:38] Another thing that's really helped is [5:40] the growth of the capabilities of the [5:42] system. So, on the right we see a [5:44] logarithmic diagram that shows [5:47] uh how long some of the models have been [5:48] able to execute over the period of time. [5:51] We go from, you know, less than 1 second [5:53] to [5:54] agents that can operate for hours and [5:55] hours. [5:57] And that's helped with this paradigm [5:59] shift that's happened where, again, when [6:01] Copilot was first introduced, it was [6:03] still augmenting traditional [6:05] development. [6:06] Um but as the capabilities have gotten [6:08] better, we've seen that this concept of [6:10] vibe coding, which was just a joke a [6:12] couple years ago, becoming much more [6:15] prominent and much more of a serious [6:17] concept. [6:18] Um and it's resulted in companies like I [6:20] have an example for Ramp. I know they [6:22] have a talk today. Um but there other [6:24] companies that have had similar [6:25] examples. In fact, even Anthropic talked [6:28] about how they were able to release code [6:30] work very quickly using a variety of [6:32] agents. [6:34] Uh this example is not unique, but it is [6:35] representative of where the industry has [6:37] been going so far. [6:40] Okay. So, talking about toil, um Ty is [6:43] going to talk about the um the agentic [6:45] system that we've deployed out. As soon [6:47] as we made our agentic workflows [6:49] available to developers, what we saw is [6:52] 70% of the workflows that developers are [6:54] pushing into the system were toil tasks. [6:57] There's a couple reasons for that. One [6:58] is [6:59] the accuracy of these tasks [7:02] was much higher compared to the more [7:04] ambiguous workloads. [7:05] And it makes sense, right? Like the [7:07] start and end state of some of these [7:09] tasks, if you think about a library [7:10] upgrade or a migration, is much more [7:13] straightforward versus say building a [7:15] brand new feature or an experience out [7:17] that requires an experiment. [7:19] Because the accuracy was higher, [7:21] developers were more likely to push more [7:23] workloads into the agentic system that [7:24] were toil oriented. And it became a [7:27] virtuous cycle that that we saw. [7:29] Uh and based on that, based on the [7:31] success, we we uh pushed for making this [7:35] as one of uh my org's developer [7:36] platform's top priorities in terms of AI [7:39] augmentation. [7:41] Okay, I'm going to hand off the time now [7:43] to get into specifics. I'll start by [7:45] saying that we are not building this in [7:47] isolation at Uber. Uber's had a long [7:49] history of building AI solutions and [7:52] having a lot of engineers across the [7:54] organization building infrastructure. [7:55] And so we really see ourselves as [7:57] building on the shoulders of giants. [7:59] Where you know we have our our historic [8:02] Michelangelo platform which has had some [8:04] public content in the past that provides [8:06] things like a model gateway so that we [8:08] can proxy and talk to the main frontier [8:10] models or host internal models, [8:12] traditional inference training platforms [8:15] and all the other things you would [8:16] expect in an ML platform that within the [8:18] last couple years has really started to [8:20] lean more into like the agentic side and [8:22] the APIs that we're using to talk to [8:25] open AI and Anthropic and those folks. [8:29] On top of that, we have a lot of the [8:30] traditional infrastructure and context [8:33] at Uber that we would want to take [8:34] advantage of. Things like having access [8:36] to our source code, our engineering [8:38] documentation, [8:40] Jira tickets, Slack information. Like [8:41] these are all things that to have an [8:43] effective agent have organizational [8:45] memory it needs to start to get access [8:47] to. One of the key ones that I'll dig a [8:50] little more into in the later slides is [8:52] our deployment of MCPs throughout Uber. [8:55] On top of that, we see a lot of industry [8:57] agents. We really take a perspective of [9:00] trying to enable the latest and greatest [9:02] for our engineers allowing them to [9:03] experiment, allowing them to have a a [9:06] learning culture and use the best of [9:08] class. So that means that there's a lot [9:10] of clients that are coming in that folks [9:11] are using and we use a lot of those to [9:13] build specialized agents. This could be [9:16] our background agent platform that we're [9:18] going to be talking about, our test [9:20] generation platform, or many other kinds [9:22] of internal ones. And then [9:24] at the at the top of that we have you [9:26] know the engine engineering enablement [9:28] phrase that's been going around the [9:30] industry. It's you know measurements and [9:31] cost control and education and [9:33] everything else that you would expect. [9:36] So let's [9:38] dig in a little bit to how we think [9:39] about uh MCPs. This became a very [9:41] popular uh piece of technology in the [9:44] industry last year, and we moved very [9:46] quickly to make sure that this was uh [9:48] deployed and secure for our engineers so [9:50] that we could um make them as productive [9:53] as possible. So, we ended up putting a a [9:56] tiger team together from across the [9:57] company. They came together and designed [10:00] a strategy and built a uh central MCP [10:03] gateway. [10:04] Um this allows us to both to [10:07] proxy external and internal MCPs from [10:09] our service infrastructure and expose [10:11] those in a consistent way to engineers [10:13] that handle things like authorization, [10:15] um [10:16] telemetry, logging, everything else that [10:18] you might expect. We also provide a [10:20] registry and a uh a sandbox so that [10:22] developers can come in, they can play [10:24] with these MCPs, they can make sure that [10:27] uh it's going to do what they're [10:28] expecting, and that they can discover [10:30] new ones. [10:34] Continuing on with that, we also have [10:37] uh Eduber uh through our ML our [10:39] Michelangelo platform built [10:42] agent uh the ability to build agents [10:44] with both SDKs and with no-code [10:46] solutions. Building agent builders, the [10:48] ability to visualize, have telemetry, do [10:51] tracing. Um that way as folks around the [10:54] company are building some of these [10:55] solutions that have access to uh the [10:58] internal services, the data sets, [11:00] we can reuse these in other systems. [11:03] This can be discoverable, and that we [11:05] can provide a registry that's then can [11:07] be found by other engineers or uh [11:09] non-engineering alike [11:11] to to deploy these. And they're deployed [11:13] consistently in a lot of our [11:15] environment. This can be uh from our uh [11:18] dev pod infrastructure, which is our [11:19] remote um dev environment, local [11:22] laptops, uh through our background [11:24] agent, which is called minions. We're [11:25] going to introduce that here in a [11:26] minute. Or deploy these in production. [11:31] So, we we talked about the agents, the [11:33] kind of registry there, the MCPs, the [11:35] registry there, all the different agent [11:37] clients that our engineers might be [11:39] using, be it Cloud Code or Code X or [11:42] Cursor. [11:43] Uh one thing that we recognized we [11:44] needed to platformize pretty quickly was [11:46] a central ability to provision and [11:50] configure and update uh the clients, the [11:53] agent clients themselves, the ability to [11:54] install and discover MCPs from the [11:56] registry, configure those inside of the [11:58] agent clients, [12:00] uh deploy standard configuration [12:01] management so that people who are just [12:03] new to the space are having more [12:05] effective uh prompts and uh [12:07] configurations right away. [12:09] And management in connection into our [12:12] background task infrastructure. [12:14] Uh so we built this tool called AIFX CLI [12:17] uh and it is the kind of the forefront [12:19] of what developers are using to access [12:22] our agentic infrastructure. [12:25] So, let's let's take a minute before I [12:27] jump into our our specific product and [12:29] think about the traditional developer [12:31] workflow. [12:33] If you looked at how people were [12:34] spending time a little bit in planning, [12:36] probably a lot in code authorship [12:37] historically, and then a small amount in [12:39] review. [12:42] And then typically they'd be in this [12:44] edit, run, build, run loop of editing [12:46] their code, building it, doing the [12:47] verification, using some standard IDEs. [12:51] Now of course this has been changing [12:52] significantly with the agentic world. [12:55] And so if we look at what the the first [12:58] agent workflow looked like, it might [13:00] look something like this. You have a [13:01] developer who's in the middle of using [13:03] Cursor or Cloud Code, they're giving a [13:05] prompt, it's asking for the ability to [13:08] proceed and approve commands, and [13:10] they're very interactive in the loop [13:11] trying to drive it to an outcome that [13:13] they want. Um [13:16] but what we're seeing emerge now in the [13:18] industry and at Uber is both background [13:21] agents that are running fully [13:23] autonomously, as well as the ability for [13:26] uh multiple of these to be run at once. [13:28] Right? This gets into this place where [13:30] as an engineer you're giving a prompt, [13:32] you're waiting for something [13:34] to you're waiting for some time while [13:36] it's running. You're thinking, "Oh, what [13:37] am I going to do? Am I going to go have [13:38] a coffee or browse Reddit? Might as well [13:41] kick off another background agent." And [13:43] so you they get into this mode of the [13:44] the new flow looks like running several [13:46] agents at once, right? [13:49] This this sounds great. We're I think us [13:52] and a lot of the industry is trying to [13:53] to push towards this. But a lot of [13:55] challenges start to emerge with this [13:57] different way of working. One of them [13:59] for us was we wanted these background [14:02] agents to be running autonomously [14:04] and looking at the external vendors that [14:06] were offering, you know, tools like like [14:09] cursor and cloud code and codex. All of [14:12] them are running their background agents [14:13] in other people's infrastructure. And [14:16] while we can get there, while that may [14:17] make sense long term, having the ability [14:19] to bootstrap on our own infrastructure [14:21] was really important to us and allowed [14:23] us to move really quickly. And so we [14:25] built a product called Minion. Minion is [14:28] our formal background agent platform. [14:32] It's built on top of state-of-the-art [14:34] agents, CLIs, and SDKs. [14:37] This leverages all of Uber's existing [14:40] infrastructure. It runs on our CI [14:41] platform. This has our mono repos [14:43] checked out, ready to work in quickly, [14:45] handles all of the network access into [14:47] the rest of the infra, [14:49] allows the connection to all of those [14:51] MCP servers that we talked about earlier [14:53] through AIFX. [14:55] It's integrated for the developer in a [14:57] bunch of different work workflows and [14:59] panes of glass. There's the web [15:01] interface we're looking at here, which [15:02] is one of the main [15:05] interaction paradigms, but it's also [15:07] available through Slack, through GitHub [15:09] PRs in the code review process, through [15:11] the CLI that we saw earlier, and we have [15:14] APIs exposed so that it can start to be [15:17] connected to by other workflows and [15:19] other services [15:20] throughout the rest of Uber. [15:22] And one other powerful thing is this [15:24] offer is good default. So when people [15:26] are coming here and kicking off these [15:27] background jobs, you know, they're [15:29] giving a prompt, they're expecting a PR [15:31] out of this. [15:32] They may not be giving the the ideal [15:34] prompt or it have the ideal setup for [15:36] it. And we can provide great defaults [15:38] for each of our mono repos, make sure [15:40] that this is more likely to have a [15:42] successful task that the engineers [15:43] authoring than if they were, you know, [15:46] if they just did this locally and they [15:48] didn't have a lot of the code empty [15:50] setup or the other context that may want [15:53] to provide. So let's walk through a demo [15:55] real quick of what using Minions is like [15:58] with an example. [15:59] So we have this web interface and in [16:01] this this is one of that I actually ran. [16:04] We had a user report an error. [16:06] They said, "Hey, this is crashing on my [16:07] machine when I run this command. Here's [16:09] what the error is." And I threw that [16:11] into into Minion. I said, "Hey, you [16:13] know, we're having this issue. The [16:15] user's on a Mac. Here's the error [16:16] they're seeing. [16:18] Here's Here's the command that was run." [16:20] And so you can see a few things here [16:21] that are cool. One, we have these [16:23] existing templates that users can choose [16:25] from that are well-written prompts that [16:26] have placeholders they can fill in. We [16:30] have the ability to choose and run in [16:31] our different mono repos. I can run in [16:34] both on a branch or we can switch it to [16:36] a follow-up task of existing PRs or [16:38] diffs. [16:39] We have all the task history here. Then [16:41] we have some cool things here. We have [16:44] I can select the agent. So in this case [16:45] I'm going to run it in cloud code. Put [16:47] it out as a GitHub PR. We've been in a [16:48] long multi-year migration from [16:50] Fabricator to GitHub. So having this [16:53] dual mode is important for our internal [16:54] engineers. [16:56] And one interesting thing you'll see is [16:58] this red icon here. What this indicates [17:00] is that this wasn't a great quality [17:02] prompt and it would have less chance of [17:03] successful success. So one tool that we [17:06] built into this was a prompt improver. [17:08] The ability to analyze the prompt and [17:10] make suggestions [17:12] that the user can accept on how to have [17:14] a more [17:15] a higher chance of success. [17:18] Now, once that kicks off, this is [17:20] running, you know, background agents can [17:22] take a little bit of time. We ping the [17:23] users on Slack, give them links so that [17:26] they can go ahead and track this. And a [17:28] few minutes later, in this case, it was [17:30] 7 minutes later, the Slack notification [17:32] pings them again. It says, "Hey, the [17:34] Minion task is done. You can go look at [17:37] the PR here. You can go look at the [17:38] artifacts." [17:40] So, let's say let's not go to the PR [17:42] quite yet. Let's jump back into the task [17:44] completion. I have a view here now where [17:46] I can see what ran. I can investigate [17:49] the agent logs if I need to. If this [17:51] failed, I can retry or have follow-up [17:53] tasks. Let's say it failed, then I can [17:57] search through the logs here and start [17:58] to try to understand what the agent was [18:00] doing and maybe give a follow-up. But, [18:02] in this case, it it was successful. We [18:04] got a PR out of that immediately. [18:06] It was a very straightforward one. [18:08] Our Minion bot co-authors this with the [18:10] person that kicked it off. Here we have [18:12] a link Jira. We have the test plan of [18:14] how it verified. You can see it was [18:17] authored here by which agent Claude was [18:20] Minion was running. [18:21] And it was a very straightforward fix [18:22] that we got. So, [18:25] this was a very simple workflow, [18:27] but it was very [18:29] much easier for the developer to just [18:30] say, "Dump in a prompt. Hey, here's a [18:32] problem the user is having." and get a [18:34] PR of that as opposed to all of the [18:36] context switching that they would need [18:37] to traditionally do. [18:41] Right now, [18:42] the work the workflow for the developers [18:44] has changed and is changing further. [18:46] They're spending more and more time in [18:49] planning and code review because there's [18:51] so much more code being generated that [18:53] they're being forced to do it. [18:55] This probably isn't the favorite type of [18:58] work the developers love doing, code [19:00] review. [19:01] And there's a lot of challenges with [19:03] that. If people are doing code review [19:04] and it's taking more time, they're maybe [19:06] slowing down. They maybe let more bugs [19:08] in because they're missing it in review [19:10] because there's much more. [19:11] So, let's jump into a few of the [19:13] investments we made to try to improve [19:15] that. [19:17] So, one of the big problems is context [19:19] switching amongst all the background [19:20] agents. This could be on PRs that are [19:22] coming out or the agent itself needing [19:24] attention. So, we built a tool called [19:26] code inbox, which was designed to try to [19:28] help with this situation. It's a unified [19:30] inbox for PRs that a developer needs to [19:33] review. And what's interesting about [19:34] this is it's designed to try to remove [19:37] noise. So, only bringing out the [19:39] actionable ones that are directly [19:40] relevant for a user then when it needs [19:42] attention, not when it's, you know, [19:44] sitting there waiting for someone else. [19:48] And we put a lot of work into the to [19:50] smart assignments with code inbox so [19:52] that we try to find the most relevant [19:54] person to review the code both from a [19:56] ownership and compliance perspective, [19:58] but also the history of how that person [20:00] was working, uh their time zone [20:03] availability, their calendar [20:04] availability. Uh and we we try to find [20:07] the right person and assign and then [20:09] have uh strict SLOs that we track so [20:11] they can see how long it's been [20:12] assigned, help reassign, do automatic [20:15] reassignment or escalation if necessary. [20:17] And then this does a a smart job of the [20:20] Slack notifications to devs. So, it's [20:22] doing thing like batching notifications [20:24] so they don't see uh a bunch of noise or [20:26] accounting for their focus time so it's [20:28] not bothering them in the middle of it [20:29] or, you know, their um holiday time if [20:32] they're out. Uh it'll also [20:35] uh [20:36] handle integrating this into teams' [20:38] existing processes. So, if teams have [20:41] existing Slack usage with code review [20:43] queues, uh we can plug directly into [20:45] that and inject the reviews at at that [20:47] level as well. [20:49] >> [snorts] [20:49] >> Some of the other cool stuff that we [20:50] built into this one was we tried to [20:52] understand the risk of the change and [20:54] we're going to continue to invest in [20:55] that. Uh there's a much different risk [20:58] profile to a small change in test versus [21:00] a change that's in one of our key [21:02] services. And so, we we try to highlight [21:04] that here by analyzing the surface area, [21:07] the the blast, how much that's going to [21:09] affect, what type of service that's [21:11] hitting, um and then make those [21:13] estimates so that we can raise that to [21:15] the developer, so they might put more [21:16] scrutiny on the review or or bring in [21:18] another person or, you know, whatever [21:19] decision they might want to make for a [21:21] riskier change. [21:24] So, in the code review space, I want to [21:26] move on to a second product. Uh this one [21:29] we talked about the notifications and [21:31] bringing context awareness, [21:32] uh but this is our product uh it's [21:34] called U Review, and this one is [21:36] designed more at the review help itself. [21:39] There's a bunch of external products [21:40] right now. We've all seen them in the [21:41] market, everything from Code Rabbit to [21:44] to Graphite. [21:46] All of those are are trying to solve [21:48] this problem, and we've played with a [21:49] bunch and we'll continue to use external [21:51] ones as well, but what we found is we [21:53] had a lot of internal context. We had a [21:55] lot of complexity like the migration [21:57] between Fabricator and GitHub that made [22:00] it make sense for us to have a platform [22:03] that we were controlling the surface [22:04] area for the comments coming through. Uh [22:07] and so what this work How this works is [22:08] we have a preprocessor for the code, and [22:10] at that point we have a set of plugins [22:11] that are going to run. There can be [22:13] general defect bots that are analyzing [22:15] it. Um it can be pulling from best [22:17] practices or MCPs or other types of [22:19] information around the organization. [22:22] Uh and we also have an API so that we [22:24] can plug in external bots. So, if we [22:26] were using, you know, one of those [22:28] external code review tools, we can just [22:29] plug it into the API here and have it [22:32] surfaced with the rest of the comments [22:33] that are coming in to the developer to [22:35] help minimize duplicate comments or or [22:38] extra noise. [22:40] That then runs through a review grader. [22:42] Um this has been one of the common [22:43] problems that we've seen a lot of low [22:45] value comments surface from those [22:47] because they'd rather uh give something [22:49] to the developer to do even if it isn't [22:52] maybe necessary. Uh and we really only [22:54] want to put the high confidence changes [22:56] that the developer really needs to focus [22:58] on, not little nits. And so, this [23:01] continues through the flow. It looks for [23:02] duplicates from these different systems [23:04] uh and finally categorizes these. Now, [23:07] each one of these layers [23:09] we've done evaluations and have [23:10] different models running based on the [23:13] the performance that each model has on [23:16] the type of behavior. [23:19] This has been something we've been [23:20] working on for most of the last year. [23:22] So, we saw some growth and some progress [23:24] in this system as as it matured. [23:27] One we saw that we were able to get [23:29] higher quality comments at a higher [23:31] rate. [23:33] As as we invested and integrated [23:35] additional best practices and other [23:36] rules, [23:38] we also saw the rate of the comments [23:41] and the best practices increase while [23:43] maintaining a high rate of comments [23:46] being addressed. This is the the [23:49] specific piece of feedback that we're [23:51] looking at to make sure that this isn't [23:53] noise, that developers are actually [23:54] fixing these and it's not just annoying [23:56] them. And then here's a screenshot in [23:58] Fabricator, not in GitHub, where I [24:00] mentioned we have to have the dual kind [24:02] of UI because of the two systems at the [24:05] moment. And so this was a custom one we [24:07] built to try to have a feedback loop for [24:09] the developer. [24:12] In the code review space, it would it [24:15] would [24:15] wouldn't be complete if I didn't talk [24:17] about the verification, the validation, [24:19] CI, test. I think that's the other big [24:22] part that we are really concerned about [24:24] to make sure that the those mistakes [24:27] aren't slipping through code reviews [24:28] more code is coming in. So, we built a [24:30] system called Auto Cover that we've [24:31] talked a little bit about in the past. [24:33] Actually, the author for it is saw him [24:36] around here somewhere. [24:38] Um [24:39] This this was a system that we designed [24:41] to generate unit tests. [24:43] Now, you might say, "Well, you can just [24:45] do that with cloud code or or many other [24:47] products." And you can, but what we [24:48] found is by really focusing on this [24:50] project, building a custom agent on top [24:53] of our internal LangFX [24:56] SDK built on LangChain, we were able to [24:58] get a much higher quality type of unit [25:00] test output. [25:02] Um, and so at this point we're seeing [25:03] about 5,000 tests generated and merged [25:06] per month around the company from this. [25:08] And um, almost a 3x rate of quality [25:10] versus something that would be generated [25:12] from your typical generic agent. Now, as [25:15] we were doing this, we were quite [25:16] concerned with, you know, bad quality [25:19] tests, change detector tests, things [25:21] like that coming in. And uh, so we we [25:23] built into this a critic engine. So it [25:25] has both the generation and the critic [25:27] engine. And we separated that out into [25:29] an independent test validator that now [25:31] developers can use independently, [25:33] whether it's a human generated test or [25:34] an AI generated test. Uh, which is great [25:37] to help up level uh, the test quality in [25:39] general and ignore any um, false [25:41] confidence that we might get from having [25:43] the higher coverage. [25:46] So I'm going to talk about one more [25:47] category before I hand it back to Anshu. [25:49] Uh, we've talked about um, you know, [25:51] authoring code initially in the code [25:52] review process, but code maintenance is [25:55] a big area. And you know, at the [25:56] beginning Anshu was talking about the [25:58] toil work. This is where a lot of folks [26:00] kind of consider toil the heaviest. [26:03] So as we were looking at how to how to [26:06] build this out, we we looked at the [26:08] space, we looked at the messages coming [26:10] out from other companies. You know, [26:11] where their CEOs are going up in front [26:13] of the news or to their their boards or [26:16] their investors and saying X percent of [26:18] our code is generated by AI now. And we [26:21] were looking at that and saying, well, [26:22] how is how is some of this done? And [26:23] some of the companies were very mature [26:26] companies, like like Google or Meta. And [26:28] we'd had a lot of discussions and seen [26:30] that they had fundamentals that Uber [26:33] hadn't invested before. Which was the [26:35] ability to kind of scale out large-scale [26:38] changes so that the AI can then build on [26:40] top of that. And and so we got together [26:42] last year and we decided we needed to [26:43] run a big program to create a scalable [26:46] version of how we handle large-scale [26:48] change. We call this auto migrate. Um, [26:50] we broke this program up into kind of [26:52] four key areas. Uh, we have the problem [26:54] identification area where someone would [26:56] looking at a migration or an upgrade and [26:58] deciding what the risk of the migration [27:00] is or like what the surface area of the [27:02] the changes or how to cut up the PRs so [27:05] that they they make sense and reduce [27:06] risk. You have the code transformer [27:08] piece which could be an agent, you know, [27:10] we could be using uh, you know, cloud [27:11] code or any others, but it could be [27:13] something deterministic like open [27:14] rewrite which we've made a lot of [27:15] investments with. [27:17] Then it gets into the validation phase [27:19] where we need to understand how we get [27:21] confidence that that automated change is [27:23] going to be successful uh, and that it's [27:26] not just relying on human review. So, [27:27] this might be CI or unit test or [27:29] sometimes even uh, you know, staging or [27:31] production signal, um, but this is a key [27:33] area as we think about it. And then [27:35] finally, [27:36] um, the the area of campaign management [27:38] was something specifically that we [27:40] needed to build from scratch. You know, [27:42] the ability if you have 100 PRs that [27:44] need to go out to developers for a [27:45] migration, how do you get those all into [27:47] the right spot? How do you track those? [27:49] How do you make sure those folks are [27:50] notified? How do you refresh this? [27:52] And so, this was this became the key of [27:55] the platform that we called Shepherd. [27:57] Um, here's introducing the experience [28:00] with Shepherd. At the surface, it's a [28:03] web UI where developers can go, um, [28:05] migration authors specifically, and they [28:07] can track all of the PRs that are [28:09] associated with a migration. [28:11] It will uh, it allows them to define [28:14] those simply through a YAML file where [28:15] they can either give a prompt if it's an [28:17] agent or they can point it to the script [28:19] that's going to be handling it. And then [28:20] Shepherd is going to take care of [28:22] generating those PRs, refreshing them on [28:25] whatever cadence you defined, keeping [28:27] those fresh for the developers, [28:28] notifying the the people that need to [28:30] review it, getting it in the right [28:31] queues, integrating with code inbox, the [28:33] last product I showed. [28:36] So, let's walk through two quick um, [28:39] demos of this. One, here's a a PR using [28:42] one of the deterministic transformers. [28:44] This used open rewrite. Um, we had [28:46] Shepherd uh, generate all of the PRs to [28:48] move our Java services to Java 21. Here [28:51] we had this PR generated that [28:54] um, [28:54] correctly found the owners, created a [28:57] limited PR in just the space of the code [28:59] owners that upgrades it to Java 21, and [29:02] we can see that generated here in a [29:04] small change needed for that upgrade. [29:07] Here's one more where uh, it's similar, [29:09] but in this case it's using the Minions [29:11] platform. It's integrated with it as an [29:12] agent. So, we have separate tools in our [29:15] programming systems group to do [29:16] analysis, find performance issues. A lot [29:19] of these are generating really good [29:20] data. Um, one of these uh, we called um, [29:23] Dr. Fix. [29:25] Actually, it's a different one, sorry. [29:26] It's not about Dr. Fix. Um, but it it [29:28] identified these performance issues, and [29:30] it was able to generate a lot of PRs to [29:33] and or diffs in this case to account for [29:35] those, run those through Shepherd, [29:37] have a a standard thing that accounts [29:40] for how how it was tested, how it was [29:42] verified, what the developer needs to [29:43] know to review it safely. [29:46] And with that, we've walked through kind [29:48] of the major deep dive. I want to hand [29:49] it back to Anshu to talk about the [29:52] couple last topics. [29:53] >> All right. Um, so Ty talked through a [29:56] lot of the engineering investments that [29:57] we made to make um, you know, the the [30:00] Sagantec shift. Uh, I'm going to talk [30:02] through a few um, non-technical [30:05] challenges that we're still dealing [30:06] with. [30:07] First up is uh, on the people side [30:10] um, and the business side. So, on the [30:12] business side um, I have a diagram here [30:14] that uh, it's very topical since the [30:16] Olympics are on right now. Um, the the [30:19] leaders when it comes to AI tech is are [30:22] changing pretty frequently. Um, you [30:24] know, the the models that are the most [30:26] powerful for certain tasks, whether you [30:29] should build something in-house versus [30:31] use a SaaS provider, these [30:34] uh, these decisions need to be revisited [30:37] on a pretty regular basis. [30:39] Unfortunately, in a large organization [30:41] like ours, some of the investments that [30:42] Ty talked about, whether it's auto cover [30:44] or auto migrate, these are not trivial [30:46] decisions to make. We need to commit [30:49] dozens of people on projects that might [30:52] be running for months. So, we can't just [30:55] change our mind after a quarter. [30:57] >> [snorts] [30:57] >> Uh there's there's two things that that [30:59] we've done to mitigate this. One is [31:01] seemingly pretty basic, making sure that [31:03] we have the right abstraction layers in [31:04] place. [31:06] We talked about the Ty showed them the [31:08] Minions infrastructure. Under the [31:09] covers, if we need to swap out the model [31:11] or we need to swap out the technology [31:13] that we're using, we're we're now able [31:15] to do so if if a better technology comes [31:18] around that can solve some of the [31:20] underlying [31:21] pieces more effectively, we can do so. [31:24] Um but the the second part is just [31:27] having this [31:29] this this belief that the tech we're [31:32] building um [31:34] will likely be replaced with something [31:36] better in the industry. And so it's [31:39] really important for us to not be [31:40] married to the tech that we're building [31:42] and being okay if something comes along [31:44] like if the the co-founder cursor talked [31:47] about the [31:49] the auto the test coverage system that [31:52] that might be coming a couple weeks. I'm [31:53] really excited about that. It might make [31:55] our auto auto cover infrastructure [31:58] obsolete and that's okay because at the [32:00] end of the day we need to deliver impact [32:02] for Uber. [32:04] The second part is another is a people [32:07] problem. [32:08] So, a lot of the challenges that Ty [32:10] alluded to deals with like, you know, [32:13] historic infrastructure that's been [32:15] built out over the last 10 to 15 years [32:17] at Uber. [32:18] We have some really sophisticated code [32:21] that we built out and then we have some [32:22] really um I would say archaic code that [32:27] you know, very few people know about. [32:29] Getting that technology integrated into [32:33] to places where AI can reach it is [32:35] challenging. [32:36] E- just getting MCP endpoints set up to [32:39] reach to different parts of our [32:40] ecosystem has been a challenge. [32:43] Similarly, [32:44] the tech that Ty talked about, like I've [32:46] seen in action, it's magic. I I ran a [32:48] demo session with some of my VPs, and in [32:50] 24 minutes I had four VPs land code for [32:53] the first time in years. Um it was it [32:55] was a pretty amazing experience. They [32:56] were pretty satisfied by it, too. [32:58] Um but our adoption for this technology [33:01] has been relatively slow. It's been [33:03] slower than I've expected. And it's Part [33:05] of it is because [33:06] we're trying to have developers uh do [33:09] something that they're so not used to. [33:10] They're used to looking at code and [33:12] generating from scratch, operating in [33:14] their IDE, and we're telling them to [33:15] take a risk by operating in a very [33:17] different way. [33:19] In both of these cases, um you know, [33:22] we've tried different tactics to get [33:24] around this people issue. We've tried a [33:26] top-down approach, you know, [33:28] directives from leaders to say, "You [33:30] must do X, Y, and Z. You must adopt." [33:33] It's had some impact, as you track, as [33:35] you folks know. You track a metric, it's [33:36] going to go up, or it's going to [33:38] improve. [33:39] The more successful um [33:42] technique that that we've applied is [33:43] actually just sharing wins. [33:46] So, as we share examples between [33:48] different engineers, [33:49] uh cool things that they've tried that [33:51] have resulted in in wins, adoption of [33:53] that technology has uh has erupted. [33:57] Um so, that's been the the tactic that [33:59] we're we're pushing on now is key [34:01] promoters pushing techniques to their [34:04] peers, because those promoters are [34:06] typically engineers, and engineers trust [34:08] other engineers as opposed to directors [34:10] like me. [34:13] Okay. Uh I'm going to touch on [34:15] measurements now. So, um [34:17] we have tons and tons and tons of [34:19] metrics. Uh I can say with confidence [34:21] that objectively AI is having positive [34:24] impact. [34:25] Um our net promoter score, our overall [34:28] developer experience at Uber, has never [34:29] been higher. [34:31] Um [34:32] the the self-reported net satisfaction [34:35] developers have and their productivity [34:36] has never been higher. [34:38] The amount of code that we're landing [34:40] through AI is is amazing. The overall [34:43] engineering velocity is fantastic. And [34:45] you can see the graph over here. We see [34:47] the inflection point where when we were [34:49] introduced agentic um the Minions [34:52] agentic system along with when the [34:54] models became really, really good like [34:56] Sonnet and Opus being introduced, uh the [34:59] the delta between developers that are [35:01] using it very casually versus the ones [35:03] that are the the the power users that [35:05] are using at least 20 days a week, it's [35:07] only exploded. Uh it's only the [35:10] uh deviation has only gone up. [35:12] So [35:13] uh I'm really pleased about this. Now, [35:15] the the issue is that these are activity [35:17] metrics, [35:18] right? These are not necessarily [35:20] business outcomes. And when we start [35:22] talking about the costs of this [35:24] technology, [35:26] you know, our CFO has has asked me uh [35:28] what is the impact of this? Right? I You [35:30] know, I can't put him to diffs. I need [35:32] to show him what's the impact on on [35:34] revenue. [35:35] Um I'm sure you folks are dealing with [35:37] the same problem. We This is not [35:39] necessarily a solved problem for us. One [35:41] of the tactics that we're taking this [35:43] year is to instrument our uh overall uh [35:46] feature infrastructure so that we can [35:48] uh time from when a, you know, a design [35:51] is first created to when an experiment [35:53] is is launched in production. [35:55] And then seeing how we're able to speed [35:58] that uh pipeline up. [36:01] And then speaking of costs, the cost of [36:04] AI is too damn high. [36:06] Um you know, since 2024, our our costs [36:10] have gone up at least 6x. Now, we'll say [36:12] that this technology is amazing. It like [36:14] there's again, no question that it's had [36:16] positive impact. But it's gone from [36:18] something that I can self-fund using my [36:21] own budget to something that I need to [36:22] ask permission from, you know, the CFO. [36:25] What that's necessitated, especially [36:27] where we went from a model where, you [36:28] know, it's it's not necessarily cursor's [36:30] or Entropic's fault that it's going up, [36:32] the GPU costs are high and memory costs [36:34] are really high. So, we've had to be [36:36] more responsible about how we use [36:39] tokens, how we think about what's the [36:41] right model for the job, and then [36:43] helping developers select those models. [36:46] So again, going back to the the example [36:48] with Minions, we helped developers think [36:50] about [36:51] um the right model to form the plan for [36:54] the uh for the project. And then, lower [36:56] cost but still pretty effective models [36:59] to do the execution. [37:01] Uh we don't necessarily want developers [37:02] to think about it, but we want to be [37:03] able to help them uh help have the [37:06] infrastructure decide for them so that [37:09] we reduce the friction for them, but [37:10] then we also optimize our costs. [37:12] Um but this is this is something that we [37:15] continuously have to keep on evaluating [37:18] and adjusting. Um especially as new [37:21] technologies introduced. So, like this [37:23] year we introduced uh JetBrains AI and [37:25] um and Warp, which we hadn't introduced [37:27] in the past, all have their own costing [37:29] model and all have their own [37:30] complexities with regards to how [37:32] developers are using them.