Ramp Automates 75% of Code as AI Pushes Development Bottlenecks to PMs

Lenny's Podcast

Automating code generation merely moves development bottlenecks downstream to problem definition and team coordination, requiring companies to re-engineer their internal software factory rather than manage individual features.

AI-assisted summary of What product looks like when coding is solved | Geoff Charles (Ramp CPO) — original source. Check the original for full context.

Source published: 2026-09-25

Key ideas

Ramp's in-house Slack coding agent Inspect powers 75% of all pull requests and enabled non-engineers to submit over 1,000 PRs in a single month, while its Review Buddy agent autonomously processes 93% of code reviews. (7:58)
Testo, an autonomous browser QA agent navigating app flows via plain-English prompts, caught 425 bugs across 30 days prior to production release. (10:30)
By treating organizational coordination as an API connected to Notion, Linear, and Slack, Ramp's internal agent Gadget resolves 85% of product questions autonomously. (11:31)
Automating micro-loops from bug ingestion to testing and release resolves 60% of reported UX issues within 24 hours, keeping human PMs in the loop only for lightweight Slack approvals. (13:16)

Detailed notes

## Background Published on YouTube by Lenny's Podcast on September 25, 2026, this session features Ramp CPO Geoff Charles speaking live at the Lenny and Friends Summit in San Francisco. The talk explores how building faster with AI requires systematically identifying and eliminating bottlenecks across the entire product development lifecycle, while demonstrating how Ramp deploys autonomous agents across problem discovery, scoping, testing, and coordination to fundamentally reshape the role of product managers.

## TL;DR Drawing parallels to Formula 1 engineering and endurance motorsport, Geoff Charles argues that software velocity depends on system-wide flow rather than isolated coding speed, because automating one phase merely pushes the bottleneck downstream. He details Ramp's internal agentic toolchain—including Glass for defining specs, Inspect for rapid code generation, Review Buddy for auditing PRs, Testo for autonomous browser QA, and Gadget for organizational coordination—which resolves 85% of internal inquiries and fixes 60% of reported UX issues within 24 hours. Charles contends that automating these micro-loops liberates product managers from reactive busywork, dividing the future PM discipline into three distinct archetypes: Technical PMs who build the internal factory, Tastemakers who set subjective quality standards, and General Managers who orchestrate end-to-end business outcomes. Product organizations must ultimately stop obsessing over individual features and focus on re-engineering the software factory that produces them.

## Atmosphere & style The presentation is fast-paced, pragmatic, and high-energy, opening with self-deprecating humor about Charles totaling a $500 car in the 24 Hours of Lemons before shifting into a rigorous operational breakdown. Blending motorsport case studies from Ferrari and Audi with real-world architecture diagrams and live company metrics, the talk carries a tone of technical precision and executive urgency. It feels like an authentic, insider dispatch from the frontier of agentic engineering, balancing warnings about organizational coordination fatigue with an inspiring rally cry for teams to raise their ambitions.

## Bulletin - [07:58](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=478s) Ramp's in-house Slack coding agent Inspect powers 75% of all pull requests and enabled non-engineers to submit over 1,000 PRs in a single month, while its Review Buddy agent autonomously processes 93% of code reviews. - [10:30](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=630s) Testo, an autonomous browser QA agent navigating app flows via plain-English prompts, caught 425 bugs across 30 days prior to production release. - [11:31](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=691s) By treating organizational coordination as an API connected to Notion, Linear, and Slack, Ramp's internal agent Gadget resolves 85% of product questions autonomously. - [13:16](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=796s) Automating micro-loops from bug ingestion to testing and release resolves 60% of reported UX issues within 24 hours, keeping human PMs in the loop only for lightweight Slack approvals.

## Key takeaways - [00:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=6s) In professional auto racing, the driver dictates only ~15% of performance; real speed comes from the car and pit crew eliminating bottlenecks across the broader system. - [00:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=6s) Formula 1 pit stops shrank from 67 seconds in the 1950s to 1.8 seconds today through process redesign and tooling rather than demanding mechanics work harder. - [03:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=186s) AI does not permanently remove product bottlenecks; it merely shifts them downstream from code generation directly onto product managers. - [03:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=186s) To synthesize fragmented customer feedback across Gong, Zendesk, and telemetry, Ramp built an ETL and vector-search insight pipeline alongside an automated daily AI 'hate podcast'. - [04:01](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=241s) Generic AI chat interfaces fail without context; Ramp's Project Glass agent acts as an AI tech lead by integrating directly with Snowflake, PRDs, and the codebase to generate validated requirements and design system prototypes. - [07:58](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=478s) Inspect deploys functional preview environments in under five seconds via Slack, enabling PMs to iterate quickly and generating three-quarters of Ramp's code. - [07:58](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=478s) Review Buddy mitigates PR review fatigue by evaluating security, code standards, and the author's prompt history, autonomously approving 93% of PRs so engineers focus only on the riskiest 7%. - [10:30](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=630s) Testo automates end-to-end QA by running headless browser tests across ~100 state permutations from plain-language instructions, catching 425 bugs in 30 days. - [10:30](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=630s) When code generation and QA are solved, human attention and organizational coordination emerge as the primary constraint on velocity. - [11:31](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=691s) Under the thesis that 'every question is an API,' Ramp's Gadget agent answers roadmaps, pricing, and launch inquiries directly from internal systems of record. - [13:16](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=796s) Automating small, low-level feedback loops prevents PMs from chasing easy dopamine hits and resolves 60% of UX issues in under 24 hours, freeing teams to pursue ambitious bets. - [14:32](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=872s) Product velocity cannot be captured by a single metric; leaders should hire intuitive 'drivers' with an uncompromising bar for speed and empower them to challenge leadership. - [14:32](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=872s) Resource-constrained teams can outperform incumbents by picking a specific dimension of excellence and engineering around it, mirroring Audi's Le Mans wins through diesel efficiency. - [16:25](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=985s) The PM role is diverging into three future archetypes: Technical PMs who build internal factories, Tastemakers who guide aesthetic and intuitive design, and General Managers who drive end-to-end business outcomes. - [17:55](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=1075s) Product leaders must shift their focus from inspecting individual features to continually architecting and upgrading the software factory that builds them.

## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=6s) The 24 Hours of Lemons and System-Level Velocity

Geoff Charles opens by praising the earlier speakers and joking that he has rewritten his presentation three times after being inspired to incorporate storytelling. To illustrate the concept of velocity, he shares a personal anecdote about his love of speed. He once signed up to race—not at the prestigious 24 Hours of Le Mans, but at the 24 Hours of Lemons, an endurance race where participants enter beat-up $500 cars and try to survive a full day on the track. His race ended abruptly when he suffered a severe crash. While he emerged uninjured, the car was totaled.

Reflecting on the wreck, Charles notes that while driver error caused his crash, in professional motorsport the driver accounts for only about 15% of the overall race outcome. The real performance advantage lies in the interplay between the driver and the car, and between the car and the pit crew. "Winning isn’t just about driving faster," Charles explains; even world-class drivers are constrained by the capabilities of the broader system. Winning requires systematically eliminating the bottlenecks surrounding the driving itself.

To show how dramatic systemic improvements can be, Charles highlights Formula 1 pit stops: in the 1950s, changing tires took 67 seconds; today, top teams execute pit stops in 1.8 seconds. This 37-fold improvement did not happen by ordering mechanics to work 37 times harder, but by dissecting the process, introducing specialized roles, upgrading equipment, and practicing relentlessly. Furthermore, F1 innovation is continuous—90% of a car's 16,000 components are redesigned every year, with only 10% carrying over to the next season. Charles challenges the audience of product leaders to consider what their organizations would look like if 90% of their codebase turned over annually. In software, the race clock starts the moment a customer experiences pain and only stops once a working solution is in their hands. AI does not permanently eliminate bottlenecks; it merely shifts them downstream. The winning product teams will be those that identify the newly formed bottleneck fastest and re-engineer their factory around it.

- In professional auto racing, the driver dictates only ~15% of the race outcome; system integration accounts for the remaining 85%. - F1 pit stops dropped from 67 seconds in the 1950s to 1.8 seconds today through process and tooling redesign. - Only 10% of the 16,000 components on an F1 car carry over from one season to the next.

---

[03:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=186s) Shifting the Bottleneck to PMs and Step 1: Identifying Customer Pain

In the standard product development lifecycle—identify, define, build, and improve—software engineers have aggressively automated their own work with AI tools, making code generation dramatically easier. Charles jokes that product managers should "be as lazy as engineers" and automate their own overhead, because the primary bottleneck in product organizations has now shifted directly onto PMs. Product managers suddenly face an overwhelming demand to define more problems, facilitate collaboration, coordinate launches, and test more deliverables.

```mermaid flowchart TD accTitle: Ramp Customer Insight Pipeline accDescr: Customer pain data flows from siloed sources into an ETL pipeline and vector search, feeding an insight agent that surfaces findings via Slack, dashboards, and audio digests.

subgraph Inputs ["Fragmented Feedback Sources"] G["Gong Calls"] Z["Zendesk Tickets"] L["LogRocket Sessions"] E["Exec Escalations"] end

subgraph Processing ["Customer Insight Engine"] ETL["ETL & Data Pipelines"] VS["Vector Search & Clustering"] PR["Product & Feature Mapping"] end

subgraph Interfaces ["Distribution Layer"] SA["Slack Query Agent"] DB["HTML Insights Dashboard"] PC["AI Audio Hate Podcast"] end

Inputs --> ETL ETL --> VS VS --> PR PR --> Interfaces ```

The first stage of this loop is identification. Customer pain signals are scattered across disparate silos: Gong call recordings, Zendesk tickets, LogRocket telemetry, surveys, and angry escalations sent to executives. Sifting through this volume manually is impossible. Even modern LLMs with 1-million-token context windows can ingest less than 0.5% of Ramp's Gong transcripts alone. Ramp initially set up a lightweight internal Slack channel dubbed the "hate channel" that automatically broadcast raw customer gripes. While entertaining, it quickly became an unmanageable flood of noise.

To solve this systematically, Ramp built a dedicated customer insight agent powered by traditional ETL pipelines, vector search, and clustering algorithms. The agent is grounded in Ramp's org chart, product hierarchy, and active feature sets, allowing it to aggregate related feedback and make it traceable back to specific accounts. To distribute these insights across the company, the team built a queryable Slack interface, an HTML dashboard, and even an automated "hate podcast" where team members can listen to an AI-generated morning audio digest of customer complaints. Charles emphasizes that synthesizing data this way does not replace talking to customers; rather, it directs PMs to the exact users they need to interview.

---

[04:01](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=241s) Step 2: Defining Products with AI (Project Glass)

Once a problem is identified, the next hurdle is defining the solution—moving from a raw concept to an actionable, fully scoped specification. Charles points out that standard AI chat interfaces are frustratingly unhelpful when they greet the user with a blank prompt: "What do you want to build?" Asking a product manager what they want without prior context renders an intelligent model far less useful. To create actionable output, AI must be wired directly into enterprise systems.

Ramp tackled this by creating an internal agent named Glass. Glass integrates with Snowflake data warehouses and user research repositories to articulate precise "jobs to be done" supported by both qualitative customer verbatims and quantitative behavioral metrics. Because Glass is indexed on Ramp’s codebase, product strategy documents, and PRD guidelines, it effectively functions as an AI tech lead. Instead of product managers interrupting senior engineers to ask whether a technical path is feasible or might break an existing dependency, they validate requirements directly with Glass.

Finally, Glass bridges the gap between spec and execution by generating functional prototypes mapped to Ramp's actual design system and frontend components. Charles observes that engineers rarely want to read bloated, abstract specifications, nor do they want unconstrained mockups that cannot be implemented. What engineers need is the complete package: solid qualitative and quantitative evidence proving the problem exists, crisp technical requirements ready to feed into automated coding agents, and an interactive prototype demonstrating the desired experience.

---

[07:58](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=478s) Step 3: Building at Scale with Inspect, Review Buddy, and Testo

With definitions automated, the execution bottleneck moves into the build phase. Charles outlines how Ramp systematically addressed each sequential constraint across coding, code review, and QA testing:

```mermaid flowchart LR accTitle: Ramp Automated Engineering Pipeline accDescr: Software delivery flow showing Inspect generating code, Review Buddy reviewing PRs, and Testo running headless browser validation.

subgraph CodeGen ["Code Generation"] PM["PM / Non-Eng Input"] --> INS["Inspect (Slack Agent)"] INS --> PRV["Deployed Preview (<5s)"] end

subgraph Review ["Verification"] PRV --> RB["Review Buddy (93% Auto-Approved)"] RB -->|"7% High Risk"| ENG["Senior Human Review"] end

subgraph Testing ["QA Validation"] RB --> TST["Testo (Browser QA Agent)"] ENG --> TST TST --> REL["Production Release"] end ```

The first obstacle was raw implementation speed, which Ramp solved by developing Inspect, an in-house coding agent. Rather than using generic coding tools, Ramp built Inspect to deeply understand their specific repository architecture and coding standards. Running directly through Slack, Inspect can generate functional code and deploy an interactive preview environment in under five seconds, allowing product managers to test working software almost instantly. Inspect has facilitated over one million sessions, writes 75% of Ramp's pull requests, and enabled non-engineers to submit over 1,000 PRs in a single month.

The massive influx of code created an immediate secondary bottleneck: PR review fatigue. Engineers found themselves buried under an avalanche of pull requests. In response, Ramp introduced Review Buddy, an agent that audits incoming PRs for security standards, code quality, and style conventions. Review Buddy can even inspect the prompt chains used to create the code, evaluating the author's intent and prompt hygiene. The agent automatically reviews and processes 93% of all Ramp pull requests, freeing senior engineers to focus exclusively on the remaining 7% of high-risk, architectural PRs.

The final bottleneck shifted to quality assurance. Traditionally, product managers spend hours configuring staging environments, seeding test fixtures, and clicking through user journeys to verify implementation details. Ramp automated this with Testo, an autonomous browser-based QA agent. Testo accesses production-like test fixtures, spins up the application across 100 different parameter combinations, and navigates the UI like a real user. PMs give Testo plain-language instructions—such as "pay an invoice, but amortize it"—and the agent executes the flow, identifying functional blockers and interface issues before release.

- **Inspect**: Ramp's custom Slack-based coding agent; powers 75% of all PRs, has run over 1,000,000 sessions, and enabled non-engineers to merge 1,000+ PRs in a single month. - **Review Buddy**: Automated code review agent that audits prompt history, security, and quality, autonomously handling 93% of Ramp pull requests while routing only the 7% riskiest PRs to human staff. - **Testo**: Autonomous browser agent that runs headless QA sessions across ~100 realistic state permutations using plain-English prompts.

[10:30](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=630s) Catching Bugs with Testo and the Shift to Coordination

Geoff Charles details how Ramp’s automated testing agent, Testo, catches issues, blockers, and confusing UI elements that violate their design system or design language before they ever hit production. Rather than letting customers discover these defects, Testo feeds thoughtful quality and design critiques right back into the product development loop. In just the last 30 days alone, Testo caught 425 bugs internally.

```mermaid flowchart TD accTitle: Ramp Automated QA and Feedback Loop accDescr: Testo scans code for bugs and design issues, routes feedback back to builders, and prepares releases for coordination. subgraph Testing["Testo Agent"] A["Code & UI Artifacts"] --> B["Testo Automated Review"] B --> C{"Quality Checks"} C -->|"425 bugs caught (30 days)"| D["Feed back to builders to fix"] end subgraph Bottleneck["Next Bottleneck"] C -->|"Passed checks"| E["Shipping rapidly"] E --> F["Human Attention Bottleneck"] end ```

Once code generation and testing are solved, you begin shipping massive amounts of product at high velocity. That immediately exposes the next systemic bottleneck: human attention and coordination. Product managers become completely inundated with pings, status inquiries, and cross-functional questions. Striking the balance between too much process (which slows builders down) and too little process (which leaves everyone confused) requires treating coordination fundamentally differently.

- Testo caught 425 bugs in the last 30 days before customer release. - When development speed surges, human attention and organizational coordination replace coding as the primary constraint.

---

[11:31](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=691s) Treating Questions as APIs: Gadget and Autonomous Coordination

Ramp approaches human coordination with a core technical thesis: "Every question is an API." To execute on this, they built an internal tool named Gadget. Gadget understands the intent behind inbound inquiries and connects them to the company's formal systems of record: roadmap documents and specs in Notion, customer call notes, Linear tickets, and Slack conversations. For AI agents to function effectively, the entire company's operational context must be legible to machines.

```mermaid flowchart LR accTitle: Gadget Organizational Knowledge Architecture accDescr: How Gadget connects team questions to internal systems of record and handles status and launch artifacts. subgraph Input["Queries & Roles"] Q1["Status & Roadmaps"] Q2["Sales & GTM Inquiries"] Q3["Launches"] end subgraph GadgetAgent["Gadget Engine"] G["Gadget API Parser"] end subgraph Sources["Legible Context"] N["Notion Specs & Roadmap"] L["Linear Tickets"] S["Slack Discussions"] end subgraph Outputs["Autonomous Actions"] O1["Auto-updates Notion & alerts owners"] O2["Answers pricing / geos (85% solved)"] O3["Writes Help Docs, Emails, Blog Posts"] end

Q1 --> G Q2 --> G Q3 --> G G <--> N G <--> L G <--> S G --> O1 G --> O2 G --> O3 ```

When someone asks if a project is on track or who owns the next milestone, Gadget provides an evidence-backed answer, pings owners who are overdue on deliverables, and directly updates the roadmap in Notion. For the sales team asking about feature availability in Brazil, pricing tiers, or specific use cases, Gadget resolves the request or routes it to the exact owner. Furthermore, Gadget handles launch operations by analyzing the finished feature to draft help center documentation, marketing blog posts, and customer announcement emails.

- **85% of questions** previously directed to product managers are now resolved completely autonomously by AI. - Unanswered queries are routed to the proper human, and the resulting answers are ingested to train the system for future inquiries.

---

[13:16](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=796s) Automating Micro-Loops to Reclaim Ambition

Product managers naturally succumb to quick dopamine hits by picking off small, visible, reactive tasks with high certainty. Citing Claire Vo's summit talk on ambition, Charles warns that spending energy on tiny items prevents teams from being truly ambitious. The antidote is automating oneself out of small feedback loops entirely.

At Ramp, an autonomous AI pipeline manages these low-level cycles: it ingests issues, deduplicates and ranks them against the Linear backlog, generates a plan, writes the code, pushes it through CI/CD and testing, and updates the knowledge base upon launch. The human PM or engineer is kept in the loop merely via lightweight Slack approvals (e.g., verifying "ready to code"). By having thousands of small loops running unattended in the background, 60% of UX issues identified by customers, sales reps, or internal staff are resolved within 24 hours, freeing PMs to tackle larger structural bets.

---

[14:32](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=872s) Racing Lessons: Velocity, Resource Constraints, and Drivers

Charles addresses the three most frequent questions teams face when operating this way:

1. **How do you measure product velocity and maintain quality?** Velocity cannot be reduced to a clean single metric like a lap time. Charles references Niki Lauda joining Ferrari in 1974, testing the car, bluntly telling Enzo Ferrari it handled and braked terribly, and immediately sitting down with the engineers to rebuild it. To know if you are moving fast enough, hire "drivers" who possess an intuitive, uncompromising understanding of true speed, grant them authority, and let them challenge leadership.

2. **How do you compete with constrained resources or token limits against massive incumbents?** In 2006, Audi entered endurance racing with an inherently slower car. Instead of attempting to outrun rivals on pure top speed, they focused on diesel fuel efficiency, minimizing pit stops and maximizing track time to capture three consecutive 24 Hours of Le Mans victories. Organizations must embrace their constraints, pick a distinct dimension to achieve world-class excellence, and systematically attack their single biggest bottleneck.

---

[16:25](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=985s) The Three Future Paths for Product Managers

Addressing the fear that autonomous agent loops will automate away the PM role, Charles counters that offloading execution work to engineering-led agentic systems is liberating rather than threatening. He outlines three emerging archetypes for product managers:

| PM Archetype | Primary Focus & Domain | Core Responsibility | |:--- |:--- |:--- | | **The Technical PM (The Factory Builder)** | Internal tooling, development pipelines, and organizational architecture | Removing drag, diagnosing bottlenecks, and shipping the internal software that enables AI agents and humans to build customer products faster. | | **The Tastemaker (The Driver)** | Product aesthetics, customer intuition, and design language | Acting as the driver holding the steering wheel, setting an exacting quality standard, and delivering subjective judgment that AI cannot emulate. | | **The General Manager (The GM)** | Cross-functional business outcomes across GTM, Ops, and Product | Expanding past classic product boundaries to direct product, marketing, sales, and operations toward unified business metrics. |

---

[17:55](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=1075s) Final Takeaways: Invest in the Software Factory

Charles summarizes three guiding principles for the modern product organization: 1. **Speed is about flow, not just typing code:** True speed comes from locating and systematically eliminating your core development bottleneck—every team has one. 2. **Bottlenecks constantly shift:** Solving one constraint immediately exposes the next; great teams continually cycle forward to address the new impediment. 3. **Focus on the factory:** Product leaders must spend less time scrutinizing isolated features and significantly more time architecting the engine that manufactures software.

He concludes by acknowledging that everything Ramp built is already iterative and bound to be surpassed, invoking Enzo Ferrari’s classic maxim: "The best Ferrari ever built is the next one." In the same way, the best product you will ever launch is your next one, which relies entirely on the quality of the software factory you assemble today.

## Notable quotes > [00:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=6s) Winning isn’t just about driving faster — Geoff Charles

> [03:06](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=186s) be as lazy as engineers — Geoff Charles

> [04:01](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=241s) What do you want to build? — Geoff Charles

> [11:31](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=691s) Every question is an API. — Geoff Charles

> [16:25](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=985s) The Technical PM (The Factory Builder) — Geoff Charles

> [16:25](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=985s) The Tastemaker (The Driver) — Geoff Charles

> [16:25](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=985s) The General Manager (The GM) — Geoff Charles

> [17:55](https://www.youtube.com/watch?v=ZG8Mf3P9xzI&t=1075s) The best Ferrari ever built is the next one. — Enzo Ferrari

## Mentioned Geoff Charles, Ramp, Lenny's Podcast, Lenny and Friends Summit, 24 Hours of Lemons, 24 Hours of Le Mans, Formula 1, Gong, Zendesk, LogRocket, Slack, Project Glass, Snowflake, Inspect, Review Buddy, Testo, Gadget, Notion, Linear, Claire Vo, Niki Lauda, Ferrari, Enzo Ferrari, Audi

## Video description At Lenny and Friends Summit, Ramp's CPO Geoff Charles explains why building faster with AI means finding bottlenecks across the entire product development process. He shows how Ramp uses AI agents to uncover customer problems, shape ideas, review and test code, coordinate launches, and fix UX issues. As each bottleneck shifts, he argues that product leaders need to keep improving the system they use to build and rethink the PM's role within it.

Recorded live at Lenny and Friends Summit on September 10, 2026, in San Francisco.

[Source](https://www.youtube.com/watch?v=ZG8Mf3P9xzI)

中文简报

Ramp实现75%代码自动化，AI将研发瓶颈转移至产品经理代码生成自动化只是将研发瓶颈向下游转移至问题定义和团队协同，这要求企业重构内部的软件工厂，而非局限于管理单一功能。 Geoff Charles将软件工程比作F1赛车与耐力赛，指出软件研发速度取决于全系统的流通效率，而非孤立的代码编写速度，因为单一环节的自动化只会将瓶颈推向研发下游。他详细阐述了Ramp的内部Agent工具链——包括用于定义规范的Glass、快速生成代码的Inspect、审查PR的Review Buddy、用于浏览器自主QA的Testo，以及用于组织协同的Gadget；该工具链能自主解决85%的内部询问，并在24小时内修复60%报告的UX问题。Charles认为，将这些微循环自动化能将产品经理从被动的繁杂事务中解放出来，并将未来的PM专业分化为三种典型原型：负责构建内部工厂的技术型PM、确立主观质量标准的品味把控者（Tastemaker），以及统筹端到端业务成果的通用管理者（General Manager）。产品团队最终必须停止执着于单一功能，转而专注于重构生产这些功能的软件工厂本身。 Ramp的内部Slack编码Agent Inspect驱动了全部Pull Request（PR）的75%，并使非工程人员在单月内提交了超过1,000个PR，同时其Review Buddy Agent可自主处理93%的代码审查。 (7:58) 能够通过自然英语Prompt自主操作应用流程的浏览器QA Agent Testo，在生产环境发布前的30天内捕获了425个Bug。 (10:30) 通过将组织协同视为连接Notion、Linear和Slack的API，Ramp的内部Agent Gadget自主解决了85%的产品问题。 (11:31) 将从Bug录入到测试和发布的微循环自动化，可在24小时内解决60%已报告的UX问题，人类产品经理仅需在Slack中参与轻量级审批。 (13:16)

Reading summary

Geoff Charles, Chief Product Officer at Ramp, argues that as artificial intelligence automates software coding, the primary bottleneck in product organizations shifts downstream to customer problem definition, testing, and operational coordination. Sustaining product velocity requires organizations to continuously re-engineer their internal software factory rather than relying on individual output. System Velocity and Downstream Bottlenecks (0:06) Drawing an analogy to professional motorsport—where Charles notes the driver dictates only roughly 15% of the outcome while system integration accounts for the remaining 85%—he argues that product velocity depends on eliminating systemic bottlenecks. Formula 1 pit stops fell from 67 seconds in the 1950s to 1.8 seconds today through tooling, specialized roles, and process redesign rather than sheer effort. Similarly, F1 teams redesign 90% of their car components every year, prompting Charles to ask product leaders how their organizations would operate if 90% of their codebase turned over annually. In software development, the delivery clock begins when a user experiences a problem and ends only when a functioning solution reaches them. Because software engineers have automated large parts of code generation with AI, Charles states that the primary constraint has transferred to product managers, who face growing demands to surface problems, align stakeholders, and validate deliverables. Grounded Discovery and Scoping Agents (3:06) To identify customer pain across fragmented silos—such as Gong call recordings, Zendesk tickets, LogRocket telemetry, and executive escalations—Ramp developed an internal customer insight agent using ETL pipelines and vector search. Because modern LLMs with 1-million-token context windows can process less than 0.5% of Ramp's Gong transcripts alone, the pipeline maps clustered feedback directly to company org charts and feature sets, distributing updates through Slack and an automated audio digest. Charles stresses that this synthesis does not replace direct customer contact, but instead directs managers to the exact users they need to interview. For solution definition, Ramp implemented an agent named Glass that connects to Snowflake data warehouses, user research repositories, and Ramp's existing codebase. Glass acts as an automated technical lead, establishing validated user stories and generating design-system prototypes. This allows product managers to verify feasibility and architectural dependencies without interrupting senior engineering staff. Autonomous Build, Review, and QA (7:58) To prevent execution from stalling during the build phase, Ramp deployed dedicated agents across coding, code review, and QA. Its custom Slack coding agent, Inspect, writes 75% of Ramp's pull requests and generates live preview environments in under five seconds, enabling non-engineers to submit more than 1,000 PRs in a single month. To resolve the resulting review backlog, Ramp introduced Review Buddy, an agent that audits code quality and prompt hygiene; it autonomously reviews 93% of pull requests, escalating only the 7% riskiest submissions to human engineers. Testing bottlenecks were addressed through Testo, an autonomous browser agent that simulates user interactions across approximately 100 state permutations from plain-English instructions. Charles reports that Testo caught 425 bugs and design-language inconsistencies internally over a 30-day window, resolving issues before code reached end customers. Automated Coordination and Future Product Roles (11:31) When development speed surges, human attention and organizational coordination replace engineering as the critical constraint. Treating internal communication as an API, Ramp created Gadget, an agent that indexes roadmaps in Notion, Linear tickets, and Slack discussions to autonomously resolve 85% of inbound questions directed to product managers. Additionally, autonomous pipelines handle low-level maintenance loops—ranking, coding, testing, and deploying minor fixes—resolving 60% of reported UX issues within 24 hours under lightweight human Slack approvals. Addressing concerns that AI will eliminate product management, Charles asserts that automating operational overhead frees PMs to pursue ambitious bets. He outlines three emerging archetypes: the Technical PM who designs internal build pipelines and tooling; the Tastemaker who acts as an intuitive driver enforcing subjective design quality; and the General Manager who steers cross-functional outcomes across marketing, operations, and product. Charles concludes that product leaders must prioritize architecting the software development factory over scrutinizing isolated product features. He cautions that any operational setup is inherently temporary, requiring organizations to continually identify and dismantle whichever bottleneck emerges next as automation advances.

Source quotes

[0:06] Geoff Charles: Winning isn’t just about driving faster [3:06] Geoff Charles: be as lazy as engineers [4:01] Geoff Charles: What do you want to build? [11:31] Geoff Charles: Every question is an API. [16:25] Geoff Charles: The Technical PM (The Factory Builder) [16:25] Geoff Charles: The Tastemaker (The Driver) [16:25] Geoff Charles: The General Manager (The GM) [17:55] Enzo Ferrari: The best Ferrari ever built is the next one.

Original transcript

[0:06] Hello everyone. [0:08] First off, it's been absolutely amazing [0:10] to see the speakers before me. I want to [0:12] give one more round of applause for the [0:14] incredible amount of content and [0:15] preparation. [0:16] I'm inspired. I've rewritten my speech [0:19] three times. I learned how to do [0:20] storytelling. And so here's a story for [0:23] you. I learned from Ron about velocity. [0:25] We love speed. [0:27] I love speed. [0:29] I love speed so much that I once signed [0:31] up to race cars [0:34] at Le Mans. I'm just kidding. [0:36] Lemons. 24 hours of lemons. [0:39] It's a race where you buy a $500 car and [0:42] you race around for 24 hours [0:45] hoping to win. [0:47] Unfortunately for me, [0:49] my car did not make it. [0:54] I crashed. [0:56] I crashed hard. [0:58] I was okay. The car unfortunately was [1:01] not. Now, this crash was entirely my [1:03] fault as the driver. [1:05] But in real professional racing, [1:08] the driver is rarely the problem. [1:11] In fact, the driver is only 15% [1:14] of the impact on the race. The real [1:17] impact [1:18] is the interaction between the driver [1:20] and the car, between the car and the [1:22] team. In other words, [1:25] winning isn't just about driving faster. [1:28] The best drivers in the world can only [1:30] perform up to the level of the system. [1:34] Winning the race is about removing the [1:36] bottlenecks around the driving. [1:40] That's the key point I want to drive [1:41] today. Take for example the pit stop. In [1:43] the 1950s, it took 67 seconds [1:46] to change tires. [1:48] Today, [1:49] 1.8 seconds. [1:52] How did 67 seconds turn into 1.8? [1:56] Certainly not by asking the mechanic to [1:58] work 37 times harder. All you leaders, [2:00] you know what I'm talking about. [2:02] They found the bottleneck [2:04] and they iterated to remove them. [2:06] Specialized functions, better [2:08] technology, more practice. [2:10] And F1 is relentless. 90% [2:14] of the parts of a car change every year. [2:18] Only 10% of the 16,000 parts of an F1 [2:21] vehicle carry over to the next year. [2:23] Now, I want you to also think about what [2:25] if 90% of your code changed every year? [2:30] Because that is the bar. Why am I [2:32] telling you all this? [2:33] We're in a race. [2:35] Speed is everything. Execution is [2:37] everything. [2:38] And we're in a race to build with AI. [2:41] For the race car, [2:43] it starts when the car comes in for new [2:45] tires and the clock stops when it [2:47] leaves. For us, the clock starts when [2:50] there's pain and it leaves when the [2:52] customer has a product that solves it. [2:54] AI simply removes the bottleneck, [2:58] but moves it. [3:00] And the best team, the winning team, is [3:01] the team that can find the bottleneck [3:02] faster, remove it, and move on to the [3:04] next one. [3:06] Now, we all know the product development [3:07] life cycle. We identify, [3:10] we define, we spend a lot of time [3:12] building, [3:13] and then we improve. [3:14] Now, the engineers, [3:16] fantastic engineers, [3:18] they've been able to automate a lot of [3:20] their work and coding is now much [3:22] easier. [3:24] So, here's a tip. Be more like [3:25] engineers. Be as lazy as engineers. [3:29] Because the reality is that the [3:30] bottleneck has now shifted to us. [3:33] We need more things to define. We need [3:35] more things to collaborate. We need more [3:36] things to coordinate, more things to [3:38] ship, more things to test, more things [3:39] to release. The bottleneck has shifted [3:41] to us and so we need to invest [3:43] in our own factory. The speed of this [3:45] loop [3:46] determines the outcome of the race. [3:48] And the question becomes, how quickly [3:50] can you identify the bottleneck and [3:51] redesign the factory around it? [3:54] What I'm going to show you today are [3:55] five of these steps and how Ramp [3:58] has gone far in iterating and automating [4:01] through it. The first step is identify. [4:04] Identify the problem. Identify the pain, [4:05] right? The issue [4:07] is that that pain lives somewhere and [4:09] there's a lot of pain, a lot of data. [4:11] Gong, Zendesk, LogRocket, a survey, [4:15] very angry email to an executive. The [4:18] problem is that context lives all over [4:19] the place. Too many silos, too many [4:21] opinions. And so the first bottleneck to [4:23] identify is how to sift through the [4:25] noise. [4:27] And LLMs are great, [4:29] but a 1 million token window, [4:31] that's less than 0.5% of Gong [4:33] transcripts at Ramp. So we start small. [4:37] We created a hate channel and every day [4:40] it posts all the lovely things that our [4:42] customers say. And yes, these are real [4:44] quotes. [4:45] >> [snorts] [4:47] >> That got out of hand really quickly. [4:51] >> [gasps] [4:51] >> Yes, I know. I am not proud of it. No [4:54] one's perfect. [4:56] And so we had to redesign from scratch. [4:59] We built an actual customer insight [5:02] agent. [5:03] That agent pulls from all the data [5:04] sources at a company. [5:07] It uses traditional [5:08] ETL and pipelines. [5:10] It uses actual vector search. [5:13] It has clustering around the same [5:15] context, understands our product, and [5:17] understands our teams, [5:18] and understands the features that we [5:20] have, [5:21] and it's accessible to the rest of the [5:22] organization. Once we had this working, [5:25] we asked ourselves, "How do we get this [5:28] to as many hands as possible?" [5:30] And so we again experimented. [5:32] A Slack agent where you can ask [5:34] questions, a nice HTML dashboard that [5:37] people can log in and read. My favorite [5:39] one was the podcast. [5:41] The hate podcast. Great way to start [5:44] your day. You listen in, [5:46] 100 customers yelling at you about all [5:48] the ways your product is broken. The [5:50] concept is simple, right? Get people to [5:52] access to this data as fast as possible. [5:54] That doesn't mean don't talk to [5:55] customers, it means you actually [5:56] identify exactly which customers to talk [5:58] to. [5:59] Because the data is traceable. Okay, [6:01] once you've found [6:03] what to [6:04] focus on, the next step is to define [6:06] your product. How do you go from you [6:08] know, concept to a fully scoped idea? [6:11] And so, we turn to AI, right? [6:15] And the AIs all kind of look the same. [6:17] They ask you, "What do you want to [6:18] build?" [6:20] And you stare. [6:22] You have this super intelligent being [6:24] asking you what you want to build. Is [6:26] that really the best question to ask? Is [6:27] that really the best starting point? [6:29] Right? The goal with AI is to connect it [6:31] to your systems. [6:33] That is how you're going to make AI [6:34] extremely extremely actionable. [6:37] And so, we did that. We built our own AI [6:38] agent called Glass, and we gave it all [6:40] the context it needs [6:42] to do the work. [6:44] It connects to all our systems so that [6:46] it understands both the data with [6:48] Snowflake, it understands our user [6:49] research, [6:51] and it can nail exactly the job to be [6:53] done that our customers are asking for. [6:55] That gives us specificity with both [6:57] qualitative and quantitative data. Glass [6:59] also understands our product strategy. [7:02] It understands how we define product [7:04] specs. [7:05] It understands our code base. And so, [7:07] instead of a PM bothering an engineer [7:09] asking them, "Is this possible? Would [7:11] this break something? What am I [7:12] missing?" you ask AI. [7:15] AI can be your best thought partner. [7:17] So, now AI is your tech lead. [7:19] And finally, you can build a prototype. [7:23] Because it understands our product [7:24] principles, it understands our design [7:26] system, it understands our code base, [7:27] you can actually define and build a [7:28] prototype that actually works within [7:31] your product. Your engineering team [7:33] don't want any of these individual [7:35] things. They don't want your prototype. [7:37] It's useless. [7:38] They don't want your long spec. That's [7:41] useless. They want the combination [7:43] of qualitative and quantitative data to [7:45] convince them that this is a problem, [7:47] the actual requirements that they can [7:49] use for their coding agents, and a [7:51] prototype that they can actually get [7:52] inspired by. [7:54] That is the next contract. So, now you [7:57] have [7:58] a strong foundation for which to go. The [7:59] next step is to build. I mentioned that [8:01] building is no longer the bottleneck. [8:03] Obviously, it's a bit more nuanced than [8:04] that. When we focus on one bottleneck, [8:06] the next one appears. [8:07] The first bottleneck was coding. [8:09] Everyone should have a coding agent at [8:11] this point. We built Inspect. [8:13] Inspect is our own coding agent, and the [8:14] reason why we built it ourselves was [8:16] because we wanted to have a strong [8:18] harness, and we wanted to make sure that [8:20] our coding agent really understood our [8:22] codebase. [8:23] It works inside of Slack. [8:25] It's fully provisioned. [8:27] It can run under 5 seconds, and it can [8:29] return an actual product in a deployed [8:32] preview, [8:33] so that the actual product manager can [8:35] actually use and interact with the code [8:36] that's being built. Anyone at Ramp can [8:39] now code. [8:40] Inspect has a million sessions. 75% of [8:43] our PRs is actually built by Inspect, [8:46] and a thousand of those PRs over the [8:48] last month was submitted by a [8:50] non-engineer. [8:52] One tip here, though, is that coding [8:53] agents are really good when you have a [8:55] strong architecture and a strong [8:57] codebase. [8:59] This expands how much of your company [9:01] can actually use AI. You now are [9:03] shipping a lot of code. The next step is [9:05] code reviews. Becomes the next [9:06] bottleneck. Engineers are just [9:08] constantly combing through a ton of [9:10] code. And so, we focused on that. We [9:12] built Review Buddy. Review Buddy fully [9:14] understands our codebase, fully [9:16] understands our actual quality checks, [9:18] our security concerns. It finds the [9:20] right human reviewer to loop in, [9:23] and it also understands the context, all [9:25] the prompts that led you to build that [9:26] code, so it can actually audit how you [9:28] used AI in the first place. So, the [9:30] review and the collaboration is now [9:31] grounded in visible context. 93% of our [9:35] PRs [9:36] are now automatically handled by Review [9:38] Buddy, so that the highest quality [9:40] engineers are spending time on the 7% of [9:43] PRs that truly matter. [9:44] That have a lot of questions around are [9:46] we building this correctly? [9:48] Are there any risks? [9:50] The next bottleneck is testing. [9:53] Previously PMs, I'm sure you're familiar [9:56] with this, we spin up a QA environments, [9:59] we try to tweak different fixtures, we [10:01] click through the screens, we make sure [10:02] that the engineer didn't skip out on [10:03] that one feature we really liked. We [10:06] automated that. We built Testo. Testo is [10:08] a browser-based QA agent. It spins up [10:11] the product [10:12] in 100 different combinations based on [10:14] the actual production data that we have [10:16] access to, and it runs in the product [10:18] like a user. [10:20] We give it instructions. We say, "Hey, [10:22] pay an invoice, [10:24] but amortize it." [10:26] And it clicks around and actually gives [10:28] us feedback, both blocking feedback, [10:30] bugs, issues, blockers, and honestly, [10:33] extremely thoughtful design and quality [10:35] feedback around things that were [10:36] confusing, things that break our design [10:38] system or break our design language. And [10:40] that goes back [10:41] into the product development life cycle. [10:43] Testo in the last 30 days has caught 425 [10:45] bugs. [10:48] I think these are bugs better caught by [10:49] us than our customers. [10:51] And they send it back through the loop [10:53] to fix. So, now you're shipping a lot of [10:55] product. [10:57] The next step is to coordinate. [11:00] How do you make sure that everyone in [11:01] your company [11:02] is in the loop? [11:04] Things are moving so fast, you're [11:05] shipping so many different things. [11:07] The next bottleneck becomes human [11:09] attention. We've all been there. PMs [11:11] being inundated with notifications. [11:14] And so, our job becomes coordination. [11:16] I see a few people coordinating right [11:18] now. [11:19] Too much process, [11:22] right? And slows down the builders. [11:25] Too little process and everyone is [11:26] confused. [11:27] And so, human attention becomes that [11:29] bottleneck. How do you solve this? Well, [11:31] every question [11:32] is an API. Every question is an API. And [11:35] you architect a system. We build [11:37] gadgets. [11:38] A gadget essentially understands the [11:41] intent of these questions and connects [11:43] it to the formal record, the road map in [11:46] Notion, the specs in Notion, the [11:49] customer calls in Notion, Slack, Linear, [11:52] our tickets. The principle here is [11:54] simple. For you to empower an agent, the [11:56] agent needs to be able to understand and [12:00] read the organization. [12:01] And the organization needs to be legible [12:03] to your agents. [12:05] I'll give you an example. A question is [12:06] like, "What's the status of this [12:07] project? What's the status of this [12:09] launch? Are we on track?" An agent [12:10] handles that. [12:12] It understands the full picture. [12:14] It understands who owns the next steps [12:16] and answers with evidence. But not only [12:17] that, the agent updates our road map. [12:20] The The agent gives status updates. The [12:22] agent pings people who are late on their [12:25] deliverables. There's a lot of fun ways [12:27] you can actually scale. [12:28] Or another question, sales, asking [12:30] questions about, you know, "What is this [12:32] feature? How do I sell it? Is it [12:34] available in [12:35] Brazil? [12:36] What's the use case? [12:38] What's the price?" [12:40] All of these things are fully understood [12:41] in gadget. If it does not understand, [12:43] routes it to the right person. But we [12:44] can take a step further. A launch. [12:47] Gadget understands the product and so it [12:50] can write the help center article. [12:51] It can help write the blog post. It can [12:54] help write the customer email. [12:56] So on and so forth. [12:57] Again, start one place and you'll see [13:00] how far it expands. [13:01] 85% [13:03] of questions that are being asked to PMs [13:05] now are fully answered with AI. And [13:07] those that are not are answered and fed [13:09] back into the system. So you're shipping [13:11] a lot of products now. Everyone's fully [13:12] aware and coordinated. Everything is [13:14] perfect. [13:16] But that's just the start, right? You [13:18] are constantly improving on the product [13:20] and the loop starts again. Now, PMs, we [13:22] we naturally gravitate towards the small [13:25] reactive things, the things that we have [13:27] high confidence in, the things that are [13:28] easy, [13:29] that are visible, the dopamine hit, oh [13:31] we've solved something. I think that is [13:33] a huge mistake. Claire, I think had a [13:35] great talk around how to be ambitious. [13:37] You don't be You're not ambitious [13:38] because you're focusing on small things. [13:40] You need to automate your way out of [13:42] these small loops. And so we did just [13:44] that. [13:46] For most small things, [13:48] an AI fully runs the loop. It routes it [13:51] to the right team. It matches it with [13:52] your backlog in linear. It dedupes. It [13:54] counts. It ranks. It runs the plan. [13:58] It has a human in loop through Slack [13:59] around, "Hey, is this ready to go? I'm [14:01] ready to code." It writes the code. It [14:03] runs through test and CICD just like we [14:05] talked about. And when it launches, it [14:08] runs through the knowledge base. These [14:09] are small loops. [14:11] And humans are in the loop only just [14:13] barely. Thousands of little loops with [14:16] thousands of little things, so you can [14:18] focus on the big ones. Thanks to these [14:19] autonomous loops, 60% 60% of UX issues [14:23] that are identified [14:25] by either a customer or a salesperson or [14:27] sales person or ourselves are fixed [14:30] within 24 hours. So at this page, you're [14:32] probably asking yourselves three [14:33] questions. [14:35] The first question is, okay, how do we [14:38] know we're moving faster? [14:40] Right? How do we move moving faster with [14:41] quality? After all, you know, we talk [14:42] about product velocity, how do you [14:44] measure it? It's certainly harder than, [14:46] you know, a lap time at a racetrack. [14:48] Well, some things are not easily [14:51] quantifiable. [14:53] Niki Lauda in 1974 [14:56] took a lap with a Ferrari car and told [14:58] Enzo, "This car is a piece of It [15:00] drives poorly. It handles poorly. It [15:02] brakes poorly." Got out of the car, went [15:04] to the engineer, [15:06] fixed it. [15:07] That is not dissimilar to the culture at [15:09] Ramp. I'll start there. [15:11] But the point I'm trying to make here [15:13] is that you you will know if you're [15:15] moving fast enough [15:16] if you actually hire a driver that knows [15:19] what speed looks like. And you put them [15:21] in charge [15:22] and you actually allow them to challenge [15:24] you. [15:25] That might be hard for some leaders, but [15:27] that's what we need to do to be [15:29] challenged by people who know what great [15:31] speed looks like. [15:33] The second question [15:35] might be, what about our budget? We have [15:37] limited resources. We have limited [15:39] resources in terms of engineers or [15:42] simply with tokens. How do we compete [15:43] with companies with no constraints? In [15:46] 2006, Audi asked [15:48] Audi had a bad car, a very slow car. And [15:51] so they asked themselves, how can we win [15:53] if we can't go faster [15:55] than anyone else? [15:58] Their answer? [15:59] Fuel efficiency. [16:01] Fewer stops, [16:04] more time on the track. [16:05] And they won them all three years in a [16:07] row. [16:08] So constraints force you to choose a [16:10] dimension in which you can be [16:12] world-class. [16:14] So embrace your constraints, [16:16] but not your bottlenecks. Find the one [16:18] thing that you're really, really good [16:19] at. Find that one bottleneck that you [16:21] think can 10x your company and just [16:24] start there. [16:25] And the last question, something that [16:27] we've talked a lot about today, is the [16:28] future role of product. Are we [16:30] automating our jobs? [16:32] What's going to happen if these loops [16:33] are running without PMs? [16:36] Well, [16:37] the first thing I'll say is [16:39] I think it's amazing for for product [16:41] managers to be out of the loop for a lot [16:43] of things that engineers can drive. This [16:45] helps us get a ton of leverage. [16:47] And I don't think we should be scared [16:49] about that. [16:50] Our jobs, though, I think will evolve in [16:52] three ways. There are three tracks that [16:54] I see for product managers. The first [16:56] track [16:57] is the technical track and something [16:59] that I don't think we are investing [17:00] nearly enough. It's the PM that's [17:02] building the actual factory, the [17:04] technical product manager that is [17:06] identifying bottlenecks and removing the [17:08] drag in your organization. They're not [17:09] shipping products for the customer, [17:12] they're shipping the product that helps [17:13] you build the product for the customer [17:15] or helps AI do so. The second role, [17:18] arguably one of the most important [17:19] roles, certainly the one that gets the [17:21] most limelight, [17:23] is a tastemaker. It's the driver. [17:24] Ultimately, you need someone who's [17:27] holding the steering wheel and doing [17:29] things that AI cannot do and holding the [17:30] bar for what great product, great taste [17:32] looks like. And finally, the GM. [17:36] I think PMs will absolutely evolve and [17:38] expand their role to not just the [17:40] product organization, but the marketing [17:42] organization, the sales organization, [17:43] the growth organizations, the [17:45] operational organizations. PMs will [17:47] become GMs and they will own the actual [17:49] business outcome and lead the entire [17:51] function. So, we walked through a lot. [17:55] I'll recap three main takeaways. The [17:57] first is that speed is not about just [17:59] coding faster, it's identifying and [18:02] removing the bottleneck in your build [18:04] process. And everyone has one. [18:06] Scientifically, there is always one. [18:10] Two, as you as you actually solve that [18:13] first bottleneck, the next one appears. [18:16] And your job is to move through them as [18:17] fast as possible. And lastly, us as [18:20] product leaders, we need to obsess a [18:22] little bit less about the product that [18:23] we're delivering and a little bit more [18:25] about the factory that helps us build [18:28] products faster. Now, I just walked [18:30] through a lot here. [18:33] A lot of great things that we built. [18:36] And I did so because the reality is that [18:39] everything you've seen here is outdated. [18:41] Just like F1, teams will copy each [18:43] other. I want you to copy us. I want you [18:45] to actually hopefully surpass many of [18:47] the things you see today. And I'm sure [18:49] people here have gone much further. [18:52] The reality is that it's all about the [18:55] next season. [18:56] And so, [18:58] I'll leave with this. [19:00] Enzo Ferrari said, "The best Ferrari [19:02] ever built [19:03] is the next one." [19:05] The best product you'll ever build is [19:07] the next product that you'll launch. [19:09] That begins in the factory. That begins [19:11] in the software factory that you build [19:12] today. [19:14] I hope you copy us. [19:15] I hope we get to copy you. Let's share [19:18] more together as a community around how [19:19] we are building these products [19:21] so we can deliver more for our [19:22] customers. And I hope you all stay stay [19:25] safe out there. Thank you. [19:27] >> [applause]