Google's Robby Stein Says AI Erases Execution Value, Demands Taste

Lenny's Podcast

Because AI tools can build software in minutes, product managers lose the scarcity value of operational execution and must rely entirely on decision-making judgment and deep human empathy to survive.

AI-assisted summary of What it takes to be a top PM today | Robby Stein (Google Search) — original source. Check the original for full context.

Source published: 2026-09-28

Key ideas

As AI tools commoditize conventional engineering execution and roadmap coordination, the core differentiator for product managers shifts toward editorial taste, judgment, and high-conviction decision-making. (0:06)
Diagnosing why users fail to engage requires inverting the evaluation question to 'Why aren't you doing what we built this for?'—a method that revealed social anxiety as the hidden blocker behind Instagram Stories sharing and led to Close Friends. (8:39)
Google Search is scaling product quality assurance by deploying autonomous AI agents that run real-world queries, capture rendered UI screenshots, audit fidelity like LaTeX math rendering, and trigger automated bug fixes. (13:55)
Product craft demands equal focus on removing friction and sparking delight; subtle micro-interactions like a color-cycling blinking cursor and responsive search-box motion communicate humanity and intentionality. (16:48)

Detailed notes

## Background Published on YouTube by Lenny's Podcast on September 28, 2026, this talk features Robby Stein (Google Search) speaking live at the Lenny and Friends Summit in San Francisco. Drawing on his product leadership across Instagram Stories, Reels, and AI-era Google Search, Stein explores what makes products genuinely beloved when AI can generate code and features in minutes, presenting a three-part framework centered on deep human needs, rigorous root-cause diagnosis, and meticulous craft.

## TL;DR Robby Stein argues that as automated AI tools commoditize raw execution, the core value of a product manager shifts almost entirely to high-judgment decision making and taste. He outlines an unwritten three-part playbook drawn from his work at Instagram and Google: uncovering latent user motivations using the Jobs to Be Done framework, diagnosing why initial product iterations fail by treating user feedback like machine learning optimization epochs, and elevating craft by both ruthlessly eliminating friction with automated evaluation agents and infusing soul through responsive motion, color, and haptics. Rather than chasing novel technology or growth hacks, he urges builders to ground their roadmaps in the fundamental emotions and needs of the humans using their software.

## Atmosphere & style The presentation carries the warm, reflective tone of a seasoned builder delivering a masterclass packed with candor, humility, and hard-earned war stories. Stein balances self-deprecating humor about awkward mattress showroom tests and failed feature launches with precise, analytical frameworks on agentic evaluation and social friction. It feels both inspiring and practical, seamlessly bridging high-level organizational shifts down to the microscopic delight of a blinking colored cursor.

## Bulletin - [00:06](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=6s) As AI tools commoditize conventional engineering execution and roadmap coordination, the core differentiator for product managers shifts toward editorial taste, judgment, and high-conviction decision-making. - [08:39](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=519s) Diagnosing why users fail to engage requires inverting the evaluation question to 'Why aren't you doing what we built this for?'—a method that revealed social anxiety as the hidden blocker behind Instagram Stories sharing and led to Close Friends. - [13:55](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=835s) Google Search is scaling product quality assurance by deploying autonomous AI agents that run real-world queries, capture rendered UI screenshots, audit fidelity like LaTeX math rendering, and trigger automated bug fixes. - [16:48](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1008s) Product craft demands equal focus on removing friction and sparking delight; subtle micro-interactions like a color-cycling blinking cursor and responsive search-box motion communicate humanity and intentionality.

## Key takeaways - [00:06](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=6s) Traditional PM execution has lost its scarcity value because modern AI agents and developer tooling allow individual builders to create what once took large engineering teams years to ship. - [00:06](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=6s) Rather than acting as generalists who handle varied tasks moderately well, top product managers must become specialized students of taste, judgment, and decision-making. - [03:41](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=221s) Clayton Christensen's Jobs to Be Done framework demonstrates that roadmaps often default to obvious features while missing the true, underlying emotional and functional drivers of human choice. - [06:11](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=371s) Text-only conversational interfaces collapse during visual discovery journeys, compelling Google to invest heavily in multimodal search that lets users refine aesthetic attributes across multi-turn chats. - [06:11](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=371s) Conversational AI recommendations for local discovery only resonated with users once grounded in canonical data such as interactive maps, real-time operating hours, and verified star ratings. - [08:39](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=519s) Initial product builds are almost always flawed; achieving product-market fit requires treating iteration like machine learning training epochs by continuously ranking shortcomings, diagnosing them, and testing fixes. - [08:39](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=519s) Instagram Stories struggled with casual sharing until the team inverted their research to identify audience conflict with family and acquaintances, ultimately cutting complex features down to Close Friends. - [08:39](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=519s) Instagram Reels initially stumbled in Brazil by assuming short-form videos should be ephemeral, misjudging creators' primary motivation to build audiences, gain virality, and establish businesses. - [12:31](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=751s) AI product diagnosis at Google revealed that search systems often master technical product specifications but fail without conversational mechanisms that proactively elicit human context, like a child's height when buying a backpack. - [13:55](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=835s) Product craft is divided into two distinct responsibilities: ruthlessly eliminating user friction and intentionally designing details that elicit positive emotional responses. - [13:55](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=835s) Google deploys autonomous agent fleets to systematically search, review UI screenshots, verify specific rendering rubrics like visual media trays, and file or fix bugs without manual QA overhead. - [16:48](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1008s) Micro-details in the redesigned Google search box—such as an energetic expansion animation, tactile haptics, and a cursor cycling through Google colors—give software an emotional soul. - [17:43](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1063s) The durable PM playbook across technological waves rests on three foundational pillars: deeply understanding people's needs, rigorously diagnosing product blockers, and crafting simple, joyful user experiences.

## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=6s) The PM’s Superpower in the Age of AI: Decision Making

Robby Stein opens by expressing his genuine love for product management and building, noting that after nearly two decades in the craft—something he jokingly admits is "very upsetting to be able to say out loud"—it has never been a more fascinating time to build products. Over those twenty years, he has spanned the gamut from early-stage startups to products operating at massive planetary scale, such as Instagram and Google Search, where billions of people interact with experiences his teams designed. At the same time, he candidly points out that he has also launched things that "zero people use today... absolutely zero." Across both the triumphs and the failures, a repeatable pattern emerged about what separates beloved products from forgotten ones.

At Instagram, Stein’s teams were responsible for shipping the original Instagram Stories, building Reels, reshaping feed ranking, and developing Direct Messaging. Today at Google Search, he is helping lead the effort to reimagine search for the AI era—designing an experience where users can ask anything across text, audio, and video while tapping into the entirety of the web's knowledge. Yet rather than focusing on high-level industry transformations, Stein zeroes in on the day-to-day craft of product work: how to shape an experience people genuinely love, especially when AI tools and automated agents allow a single person to build in minutes what once required years of engineering and massive teams.

Historically, the product manager's primary value centered on execution—organizing roadmaps, driving momentum, and functioning as a hybrid product-and-project manager. Stein argues that dynamic has fundamentally shifted. When almost anything imaginable can be built effortlessly with AI, conventional execution loses its scarcity value. While conventional wisdom says a PM's craft is simply doing many different things moderately well, Stein contends that PMs must be the undisputed best at one specific capability: decision making. Because many product decisions inevitably prove wrong, top PMs must become dedicated students of taste, judgment, and the decision-making process itself.

- **Career Track Record**: Nearly 20 years in consumer tech, spanning zero-user failures, startups, and billion-user features like Instagram Stories, Reels, Feed Ranking, Direct Messaging, and Google Search in the AI era. - **The Core Evolution**: The role of the PM has moved from operational coordination ("getting stuff done") to editorial judgment, taste, and high-conviction decision making.

```mermaid flowchart TD accTitle: The Changing PM Value Proposition accDescr: Diagram illustrating the shift from traditional PM skills to AI-era PM judgment. subgraph Traditional["Traditional Era"] A["Coordination & Process"] B["Project Management"] C["Generalist Execution"] end subgraph AIEra["AI Era"] D["Taste & Curation"] E["Deep Human Empathy"] F["High-Judgment Decision Making"] end Traditional -->|"Shift driven by AI agents"| AIEra ```

---

[03:41](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=221s) Chapter 1: Deep Human Understanding and the Job to Be Done

To unpack how high-judgment decisions are made, Stein introduces an unwritten three-chapter playbook he has refined over his career. Chapter one begins with the foundation: understanding people deeply. Building a great product is impossible without isolating the fundamental needs driving human behavior. Stein highlights Clayton Christensen’s seminal book *Competing Against Luck* and the Jobs to Be Done (JTBD) framework, which posits that customers do not merely purchase or open products; they "hire" them to accomplish specific jobs in their lives.

To illustrate how to uncover these hidden motivations, Stein recounts a hands-on offsite the Instagram team ran with the creators of the JTBD framework. Participants were instructed to dissect their most recent major purchase by mentally reconstructing every tiny sensory and emotional detail of that day: where they were, who they were with, what they were reading, wearing, and feeling. For Stein, that purchase was a mattress. Wandering through a showroom with his wife, they narrowed the selection down to three or four options that felt equally comfortable.

Suddenly, his wife made a peculiar request: "Can you do me a favor and jump on this? Do a barrel roll." Stein obliged with an awkward flop onto the bed. His wife immediately declared, "We’re getting this bed right now. Thank you. We’re done." When he asked why, she explained that her non-negotiable priority was having a mattress that would not wake her up whenever he tossed and turned at night. Stein remarks that while this realization was "a little offensive," it revealed the true underlying job: motion transference. A standard mattress PM would likely default to obvious roadmap additions like cooling gels, eco-friendly materials, or competitive price cuts. Yet without digging into the raw human context, that PM would completely miss the real decision-making driver.

- **Recommended Framework**: Clayton Christensen’s Jobs to Be Done from *Competing Against Luck*. - **The Mattress Lesson**: The conventional roadmap items (cooling, eco-friendly fabrics, discounting) were irrelevant compared to the actual hiring criteria (motion transference to avoid sleep disruption).

---

[06:11](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=371s) Unearthing Hidden Jobs in Google's AI Search

Stein translates this need for deep human empathy directly into digital consumer software, drawing on recent experiences developing AI-native search at Google. Two years ago, when the industry’s initial conversational chatbots were strictly text-based interfaces, the team noticed users turning to search for visual, inspirational discovery journeys—such as hunting for furniture. Describing aesthetic taste purely in prose breaks down quickly; nobody can adequately evaluate a couch through text alone. To solve this, Google became one of the earliest teams to invest heavily in multimodal understanding, retrieval, and knowledge infrastructure. This allowed users to converse back and forth visually: prompting for a couch, refining by abstract textures like "fuzzy," shifting color palettes to "emerald green," and transitioning smoothly into a curated grid of matching throw pillows.

A second breakthrough occurred during the early days of Google's AI mode exploration. Users were asking conversational queries about neighborhoods and restaurant recommendations, but the early AI answers felt hollow. The team realized the experience was missing the foundational authority people hire Google for: seeing locations pinned on interactive maps, checking real-time operating hours, verifying price tiers, and evaluating verified star ratings. Bringing Google's trusted local entity data directly into the conversational interface instantly converted it into one of the highest-rated, most popular features on the platform based on user thumbs-up feedback.

Looking ahead, Stein sees enormous potential in using AI itself to scale this discovery methodology. Rather than manually parsing user interviews, PMs can run transcripts through specialized models to systematically identify and categorize underlying jobs to be done. Furthermore, Stein is exploring autonomous AI interview agents programmed with Clayton Christensen's deep-interviewing methodology. Such agents could conduct open-ended, empathetic conversations at massive scale, interrogating users about their circumstances, daily routines, and motivations to pinpoint the exact problems they are hiring software to solve.

- **Multimodal Search Insight**: Text-only chatbots fail for aesthetic inspiration. Google built multimodal retrieval so users could refine visual concepts ("emerald green", "fuzzy", accent pillows) across multi-turn chats. - **Local AI Mode Entity Grounding**: Conversational recommendations only resonated once paired with canonical Google local data—maps, hours, star ratings, and pricing. - **Future AI Applications in JTBD**: Using AI agents to analyze interview transcripts for latent jobs, and deploying autonomous AI research agents to conduct deep contextual user interviews at scale.

[08:39](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=519s) Diagnosing Root Causes: Iterating Like a Machine Learning Epoch

Once you have established an understanding of what people want, the next essential step is diagnosing root causes. Robby Stein points out that after you grasp the initial user problem and formulate a vision, you build a first version—and that first version is almost always "not great." In fact, virtually every successful product he has been part of started out rough. The secret to bridging the gap between a flawed initial build and true product-market fit lies in treating product iteration like an optimization loop. Product managers must take a rigorous analytical view of the exact shortcomings in their product space, rank those issues by priority, diagnose them deeply, fix them, and then recursively ask the same diagnostic questions again. Stein compares this ongoing cadence directly to an epoch in machine learning model training: each batch of updates and changelists pushes the product incrementally closer to genuine market fit.

```mermaid flowchart TD accTitle: Iterative Product Debugging Loop accDescr: A recursive feedback loop showing how product teams diagnose root causes, test fixes, and iterate toward product-market fit much like model training epochs. subgraph Diagnosis ["1. Diagnose Root Causes"] A["Identify User Failure Point ('Why aren't you using it?')"] --> B["Cluster Leaf Node Feedback Into Themes"] B --> C["Quantify Weights via Large Surveys"] end subgraph Execution ["2. Target & Iterate"] C --> D["Isolate True Blocker (e.g., Audience Burden)"] D --> E["Ship Focused Prototype (Prune Complex Variations)"] end subgraph Epoch ["3. Evaluate Next Epoch"] E --> F{"Does It Meet Fit?"} F -- "No" --> A F -- "Yes" --> G["Scale Product Experience"] end ```

To illustrate this dynamic, Stein shares two key stories from his time at Instagram. When Instagram Stories initially launched, it took off quickly, but getting to that point required clearing immense internal hurdles, including earlier experiments that awkwardly forced stories directly into the feed. Even after launch, a critical issue lingered: a large portion of the user base was reluctant to share. Instead of asking why people *did* use the product, the team inverted the question: "Why aren't you doing the thing that was our goal?" User responses formed a massive cloud of feedback, but once clustered by theme and quantified across a survey of thousands of users, the primary blocker became glaringly obvious: social anxiety and audience conflicts. People resisted posting casual moments because their exes, teachers, aunts, or siblings followed them. Stein emphasizes that no creative camera tool or playful feature could bypass that social friction. The team spent two years iterating through messy permutations—including a confusing concept called "Favorites" with hidden profile backdoors and feed-posting toggles—before ruthlessly stripping away extraneous mechanics to launch a single, focused feature: Close Friends inside Stories.

The second Instagram example centered on the early launch of Reels in Brazil. The product team initially reasoned that since users were recording goofy dances and lip-syncing, they would naturally want that content to disappear rather than remain permanently pinned to their profiles. That assumption resulted in a major launch failure. In hindsight, the miss was obvious: users who spent hours choreographing and perfecting an intricate dance did not want their work to vanish after 24 hours. More fundamentally, the team realized that creators—even young students—aspired to be entertainers, build businesses, and reach massive audiences. Rather than fearing that their entire school would see their video, creators actively wanted their whole school and the wider internet to discover it. Instagram completely pivoted Reels from an ephemeral experiment into a permanent, viral video surface, ultimately driving massive global engagement.

- **The Diagnostic Inversion:** When diagnosing engagement drop-offs, shift from asking *"Why did you use this?"* to rigorously asking *"Why aren't you doing the thing we built this for?"* - **Instagram Stories & Close Friends:** Qualitative feedback revealed users were terrified of casual sharing because their ex, aunt, or teacher was watching. It took two years of cycling through failed variations (such as "Favorites" and private profile backdoors) before stripping the feature down to the dedicated Close Friends list inside Stories. - **Instagram Reels in Brazil:** The team incorrectly assumed short-form video should be ephemeral to hide embarrassing moments, misjudging creators' underlying ambitions to achieve virality and build content businesses.

---

[12:31](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=751s) AI-Assisted Diagnosis and Collaborative Google Search

Diagnosing product shortcomings has evolved dramatically with the rise of modern AI tooling. At Google, Stein’s team uses internal systems, including the Google Anti-Gravity environment, to tap into streams of opted-in users who actively share feedback while engaging with products. Instead of relying solely on disjointed bug logs, these systems capture rich qualitative comments about where an experience fell flat, which an AI model can then synthesize, categorize, and stack-rank automatically.

Stein shares a concrete example from shopping on Google: a user researching backpacks. While Google excelled at canonical product facts—identifying exact dimensions, materials, and which bag had higher capacity—a parent pointed out a crucial blind spot in the feedback: "It's awesome that you knew about each backpack... but you should have asked me about my son and the height. Like, that's a pretty key thing or it's going to be too heavy for my kid." The system had all the technical specs but lacked human context. Aggregating feedback revealed that this was a pervasive, high-ranking user need across multiple queries. The insight propelled Google Search toward becoming more conversational and collaborative, proactively guiding searchers through the discovery process rather than assuming users already know every technical specification required to reach their goal.

---

[13:55](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=835s) Craft: Scaling Pain Reduction with AI Agents and Reigniting Delight

Once a product survives the iterative loops and achieves clear product-market fit, it reaches the stage where it is ready for prime time—the company-wide rollout and mass customer launch. At this juncture, Stein argues that the defining differentiator is craft. Craft is often dismissed or overlooked, but Stein defines it simply: users possess an intuitive radar and "can detect whether the creators of a product should care about the product that they created." He splits craft into two primary vectors: 1. **Eliminating user pain:** Ensuring the product works flawlessly and without friction. 2. **Sparking joy and delight:** Determining how the product makes the user feel on an emotional level.

On the pain-reduction side, the historical norm required product managers and QA teams to act as obsessive power users, manually testing every nook and cranny of the software. To scale that process, Stein personally built an internal agentic tool powered by Google’s Anti-Gravity system. The tool acts as an autonomous user: it visits Google, enters real-world queries across diverse topics, asks follow-up questions, captures screenshots of the rendered output, and audits each step against an exhaustive evaluation rubric.

```mermaid flowchart LR accTitle: Autonomous Quality Assurance Agent Loop accDescr: An automated evaluation pipeline where an AI agent interacts with Google Search, captures screenshots, and assesses output fidelity against specific rubrics. subgraph Interaction ["Agent Execution"] Q["Prompt Library (Math, Visuals, Tasks)"] --> Ag["AI Agent Searcher"] Ag --> S["Capture Full Response & UI Screenshot"] end subgraph Verification ["Multimodal Evaluation"] S --> R{"Rubric Check"} R -->|"Math Query"| M["Verify LaTeX Rendering (No garbled syntax)"] R -->|"Visual Query"| V["Verify Visual Trays (e.g., bioluminescence)"] end subgraph Correction ["Triage"] M --> F["Log Breakages & Trigger Auto-Fixes"] V --> F end ```

The system evaluates both textual accuracy and UI fidelity. For example, when testing a school homework query, the agent checks whether complex math expressions are rendered cleanly in LaTeX rather than outputting garbled characters. For visual topics like "bioluminescence," the agent verifies that the interface injects a rich visual media tray rather than serving a wall of dry text. Google has expanded this approach into fleets of specialized agents that identify UI breakages, detect off-spec responses, and increasingly submit automatic code fixes—effectively transforming manual QA into an automated, highly scalable loop.

Finally, Stein turns to the second element of craft: sparking genuine joy. He touches on the dramatic redesign of Google’s search box, which was reimagined after more than twenty years and unveiled at Google I/O. The challenge was substantial: how do you reinvent one of the most recognizable interfaces on the internet to support multimodal AI inputs—expanding fluidly, accepting context from documents, photos, and videos—without alienating hundreds of millions of users? Stein emphasizes that the core objective was ensuring the experience still felt unmistakably "Google" while delivering a unmistakable sense of upgraded intelligence and wonder: making users feel that "AI is here."

[16:48](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1008s) Giving a Product Soul Through Color, Motion, and Haptics

To convey the sense of an intelligent system inside the product actively helping the user, the design team introduced a gradient of color representing the underlying AI power and wove together several subtle, tactile details. For instance, as the cursor blinks, it quietly cycles through the different Google colors. Robby shares that he cannot count how many times someone has noticed that tiny detail and felt compelled to post about it on X, admitting, "It just blows me away every time." Similarly, when a user taps into the search box, the interface reacts as if it is "excited"—it jumps out slightly and loads the glowing ring, projecting an eager energy that says, "Ooh, like we're getting ready to load. Like we're here, you know?"

That visual responsiveness is paired with haptics, giving the interaction a genuine physical sensation alongside its emotional warmth. When all of those elements come together into a single cohesive experience, they transcend mere utility. As Robby puts it, "through motion, through color, through feel, through touch, you know, and the intentionality of the experience, you communicate the humanity behind the product and you give it a soul." He notes that this layer of craft is a vital part of the product creative process that far too many people overlook.

- **Blinking cursor**: Cycles continuously through the classic Google colors, frequently sparking organic user posts on X. - **Interactive search box**: Jumps out when tapped and animates a loading ring to feel responsive and "excited." - **Haptic feedback**: Pairs physical touch with visual motion and color to make the AI experience feel tangible and human.

[17:43](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1063s) The PM Playbook in the AI Frontier

Bringing his talk to a close, Robby synthesizes these lessons into an overarching playbook for product managers operating in the new AI frontier. While AI makes building possible in a totally new light, the fundamental responsibilities of product building have not changed. First, you have to understand people deeply, uncovering the core needs and problems they are actually feeling. Second, you must be rigorous and analytical in diagnosing the product, constantly asking, "What are the things I need to fix in order to get the product to win and to work for people against that objective?" Finally, you have to build and refine the product so that it is simple to use, works seamlessly, and makes people feel awesome when they use it.

```mermaid flowchart LR accTitle: Robby Stein's Three-Part PM Playbook in the AI Era accDescr: A sequential flow showing the three core responsibilities of a product manager: deeply understanding human needs, rigorously diagnosing what to fix, and crafting a simple, delightful experience.

A["1. Understand People Deeply (Core needs & problems)"] --> B["2. Diagnose Rigorously (Analytical fixes to win)"] B --> C["3. Craft with Soul (Simple, works great, feels awesome)"] ```

Because of this, Robby believes the craft of product management is more critical now than ever before. "I think it's never been more important time to be a PM and to do product and to be a builder," he emphasizes, adding that "the skill of making great decisions that can lead to great products has also never been, you know, more important." With humility, he acknowledges that his framework is "just one way" to build products, joking that the audience has probably heard ten other approaches at the summit that day that are just as valid, and encourages everyone to adopt whatever makes sense for them. He leaves the audience with a final challenge for the next time they sit down to work on their roadmaps: spend a little less time thinking about "cool features kind of out of the blue," throwing new technology at a problem, or devising new growth tactics to get a product in front of users, and instead "spend more of your time thinking about the actual people and the humans on the other side of the screen that you're building for."

## Notable quotes > [00:06](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=6s) zero people use today... absolutely zero. — Robby Stein

> [03:41](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=221s) Can you do me a favor and jump on this? Do a barrel roll. — Robby Stein's wife

> [03:41](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=221s) We’re getting this bed right now. Thank you. We’re done. — Robby Stein's wife

> [08:39](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=519s) Why aren't you doing the thing that was our goal? — Robby Stein

> [12:31](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=751s) It's awesome that you knew about each backpack... but you should have asked me about my son and the height. Like, that's a pretty key thing or it's going to be too heavy for my kid. — Google Search user

> [13:55](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=835s) can detect whether the creators of a product should care about the product that they created. — Robby Stein

> [13:55](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=835s) AI is here. — Robby Stein

> [16:48](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1008s) Ooh, like we're getting ready to load. Like we're here, you know? — Robby Stein

> [16:48](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1008s) through motion, through color, through feel, through touch, you know, and the intentionality of the experience, you communicate the humanity behind the product and you give it a soul. — Robby Stein

> [17:43](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1063s) I think it's never been more important time to be a PM and to do product and to be a builder — Robby Stein

> [17:43](https://www.youtube.com/watch?v=sTgM_sbLMNg&t=1063s) spend more of your time thinking about the actual people and the humans on the other side of the screen that you're building for. — Robby Stein

## Mentioned Robby Stein, Clayton Christensen, Lenny Rachitsky, Lenny's Podcast, Lenny and Friends Summit, Google, Google Search, Google Anti-Gravity, Google I/O, Instagram, Instagram Stories, Instagram Reels, Close Friends, Competing Against Luck, Jobs to Be Done (JTBD), X

## Video description When AI can help build almost anything, what makes a product people love? At the Lenny and Friends Summit, Google's Robby Stein draws on his work on Instagram Stories, Reels, and AI search to argue that a PM’s most important skill is making great decisions. He shares a three-part approach: understand people’s needs, diagnose why a product isn’t working, and refine the details that make it useful and delightful.

Recorded live at Lenny and Friends Summit on September 10, 2026, in San Francisco.

[Source](https://www.youtube.com/watch?v=sTgM_sbLMNg)

中文简报

Google的Robby Stein表示：AI消解执行价值，更需依赖品味随着AI工具能在数分钟内构建软件，产品经理失去了日常执行层面的稀缺价值，必须完全依赖决策判断力与深刻的人性共情才能立足。 Robby Stein认为，随着自动化AI工具让纯粹的执行能力同质化，产品经理的核心价值几乎完全转向了高判断力的决策制定与品味。他结合自己在Instagram和Google的工作经验，总结了一套未曾成文的三部分工作法：利用“待办任务”（Jobs to Be Done）框架挖掘潜在的用户动机；将用户反馈视同机器学习优化轮次，以此诊断早期产品迭代失败的原因；以及通过借助自动化评估Agent坚决消除阻碍，并融入灵动的动效、色彩与触觉反馈来赋予产品灵魂，从而提升工艺水准。他呼吁开发者不要盲目追逐新奇技术或增长技巧，而应将产品路线图扎根于使用软件的人类的基本情感与需求之中。随着AI工具使常规的工程执行和路线图协调工作变得同质化，产品经理的核心差异化优势正转向编辑品味、判断力以及高确定性的决策能力。 (0:06) 诊断用户为何不买账需要将评估问题反转为“你为什么不做我们为此设计的功能？”——这种方法曾揭示出社交焦虑是Instagram Stories分享背后的隐性障碍，并催生了“密友”（Close Friends）功能。 (8:39) Google Search正通过部署自主AI Agent来扩大产品质量保障规模，这些Agent可运行真实查询、捕获渲染后的UI截图、审查类似LaTeX数学渲染的保真度，并触发自动化缺陷修复。 (13:55) 产品工艺需要兼顾消除摩擦与激发愉悦；诸如色彩循环变换的闪烁光标和响应灵敏的搜索框动效等细微微交互，能够传递出人情味与用心设计。 (16:48)

Reading summary

Robby Stein argues that as artificial intelligence commoditizes routine software execution, a product manager's primary value shifts from operational coordination to high-judgment decision-making rooted in human empathy, rigorous diagnosis, and intentional craft. Decision Making as the Core PM Competency (0:06) Historically, product managers derived value from operational execution—managing roadmaps, coordinating teams, and driving delivery momentum. Stein contends that because modern AI tools and agents allow individual builders to produce complex software in minutes, basic execution has lost its scarcity value. Instead of being generalists who perform varied tasks moderately well, PMs must specialize in high-conviction decision-making, taste, and product judgment to separate enduring products from those that fail. Uncovering Latent Human Needs (3:41) Grounded in Clayton Christensen’s Jobs to Be Done framework, Stein asserts that effective product strategy requires isolating the hidden emotional and contextual drivers behind user behavior rather than relying on surface-level requests. In physical retail, for instance, a mattress buyer's decisive hiring criteria may be motion isolation to prevent waking a partner, rather than standard roadmap features like cooling gel or sustainable fabrics. Stein applies this principle to digital products, noting that Google's conversational AI search initially underperformed in visual queries and local discovery. Users shopping for furniture required multimodal image retrieval to articulate abstract concepts like "emerald green" or "fuzzy," while local recommendations succeeded only after incorporating established local data such as interactive maps, operating hours, and verified ratings. Stein projects that autonomous AI agents will soon conduct deep, Christensen-style discovery interviews at scale to systematically identify these latent jobs. Iterative Root-Cause Diagnosis (8:39) Stein compares the journey from a flawed initial release to product-market fit to an optimization loop in machine learning, where each batch of updates functions like a training epoch. When target behaviors fail to materialize, teams should invert their inquiry from asking why active users engage to rigorously asking why intended users do not. At Instagram, early Stories engagement stalled because users experienced social anxiety over diverse follower audiences (including teachers, exes, and relatives). Resolving this barrier required two years of prototyping before the team stripped away extraneous mechanics to launch Close Friends. Similarly, the initial release of Reels in Brazil incorrectly assumed short-form video should disappear after 24 hours to mask embarrassing attempts; the team discovered that creators actually sought permanent distribution to build audiences, prompting a pivot to an evergreen, viral format. At Google, aggregated user feedback on shopping showed that while the engine provided precise backpack specifications, it failed parents by neglecting human factors like a child's height and weight limits. Balancing Friction Elimination with Product Soul (13:55) Once a product demonstrates market fit, Stein identifies craft as the decisive differentiator, divided into reducing friction and generating delight. To automate pain reduction, Stein built autonomous internal agents via Google's Anti-Gravity environment. These agents run queries, capture UI screenshots, audit outputs against rubrics (such as checking LaTeX rendering on math problems and verifying visual trays on descriptive queries), and log breakages or trigger automatic code fixes. On the emotional side, Stein emphasizes micro-interactions that communicate the humanity behind the software. During the multimodal redesign of Google’s search box, the team implemented subtle cues: a blinking cursor that cycles through brand colors, an interface that elevates dynamically when tapped, and tactile haptic feedback. According to Stein, these tactile and visual details give software a perceptible "soul" and signal responsiveness to the user. Stein frames this three-part discipline—deep human empathy, rigorous analytical diagnosis, and soulful craft—as a roadmap for builders navigating the AI transition. While acknowledging that alternative frameworks exist, he cautions product managers against defaulting to novel technology or growth tactics at the expense of understanding the people on the other side of the screen.

Source quotes

[0:06] Robby Stein: zero people use today... absolutely zero. [3:41] Robby Stein's wife: Can you do me a favor and jump on this? Do a barrel roll. [3:41] Robby Stein's wife: We’re getting this bed right now. Thank you. We’re done. [8:39] Robby Stein: Why aren't you doing the thing that was our goal? [12:31] Google Search user: It's awesome that you knew about each backpack... but you should have asked me about my son and the height. Like, that's a pretty key thing or it's going to be too heavy for my kid. [13:55] Robby Stein: can detect whether the creators of a product should care about the product that they created. [13:55] Robby Stein: AI is here. [16:48] Robby Stein: Ooh, like we're getting ready to load. Like we're here, you know? [16:48] Robby Stein: through motion, through color, through feel, through touch, you know, and the intentionality of the experience, you communicate the humanity behind the product and you give it a soul. [17:43] Robby Stein: I think it's never been more important time to be a PM and to do product and to be a builder [17:43] Robby Stein: spend more of your time thinking about the actual people and the humans on the other side of the screen that you're building for.

Original transcript

[0:06] Everyone, it's so great to be here. And [0:09] I love being here to get to talk about [0:10] one of my favorite things ever. [0:13] Being a PM, [0:14] product manager, [0:16] and builder. [0:17] And what's pretty incredible is that [0:19] it's never been so interesting to be [0:22] building products. And after 20 years [0:24] almost of building, particularly in [0:25] consumer journeys, it's very upsetting [0:27] to be able to say out loud. Um but after [0:29] almost 20 years of building, been [0:31] fortunate to build all kinds of [0:32] different products over my career. Built [0:35] startups, and I've also helped lead [0:37] teams and products through really big [0:38] changes like Instagram and Google [0:40] Search, where now billions of users are [0:43] using products that our teams have [0:45] created over the years. Also, I've built [0:47] some things that zero people use today, [0:50] by the way. Absolutely zero. And what [0:52] I've learned throughout this process is [0:54] that there is a little bit of a pattern [0:56] of the kinds of things that we've done [0:58] to make the products really great. [1:00] Sometimes it works out, sometimes it [1:01] doesn't, but there has been a repeatable [1:04] pattern that I've seen emerge from the [1:05] things that have done really well and [1:07] the things that have not. And that's [1:09] really what I'm here to share today [1:11] uh and impart to all of you. There's [1:12] lots of things to talk about, but I [1:14] wanted to give you just a little bit of [1:16] extra background on me. So, I spent some [1:17] years at Instagram, where my team uh has [1:19] built out Instagram Stories and shipped [1:22] it originally, and built all of the [1:23] experiences around it. We have Reels, uh [1:26] feed ranking, and direct messaging. So, [1:28] a whole bunch of products that came out [1:29] of that era. Uh and then now at Google [1:32] Search, where I am working through a [1:35] brand new uh experience with the team to [1:38] help think through what is Google Search [1:40] in the AI era, so that you can truly ask [1:42] anything on your mind and get the most [1:44] helpful information imaginable across [1:47] everything, whether it's videos or [1:49] audio, text. And you can tap into all of [1:52] the knowledge of the of the web and of [1:54] Google and connect the information [1:56] that's out there. But, today I'm not [1:57] talking about the transformation of [2:00] product, although that is something I [2:01] really deeply enjoy. I'm talking about [2:04] the craft of product. What does it take [2:06] to build? What does it take to have a [2:07] product that people love and to shape [2:10] that and make that? And how do you do [2:11] that in the age of AI that ultimately, [2:14] and from my opinion, any one of you can [2:16] do now with just a handful of agents or [2:19] kind of AI assistants which took us [2:21] years and whole teams of people to do um [2:24] in process here. Let's uh just mention [2:27] one thing I wanted to say around the the [2:29] role of PM before we get started. And [2:31] that is really the [2:33] the fundamental shift that we're all [2:34] feeling is that we used to be, I think [2:36] as PMs, a big part of our value was [2:39] around getting stuff done. Was around [2:40] organizing, creating energy and [2:42] momentum, and being almost like a [2:44] project manager and a product manager. [2:46] And I think in some cases that's still [2:47] true, but really if in the world of [2:50] today, almost anything you can think of [2:52] can get built, [2:54] it's more clear to me than ever that the [2:56] actual true value of PM is actually [2:59] around judging. It's around taste. It's [3:01] around doing something extremely well. [3:03] And I think what's something I've [3:05] thought about a lot is that people say [3:06] the PM's main craft is that you do lots [3:10] of things pretty well. [3:12] And I think that it is somewhat true, [3:14] but I think above all else actually the [3:16] craft of PM is one simple thing that you [3:19] should be the absolute best at. And that [3:21] is decision making. [3:23] It's about making decisions and you [3:24] should be students of what it takes to [3:26] make great decisions. And it's a [3:28] humbling thing cuz most decisions, many [3:30] decisions are wrong, and these are just [3:32] the things that I've learned over the [3:33] years that have helped me and the teams [3:35] make decisions that have led to some of [3:37] the great experiences that are now out [3:39] there today. There's kind of a book [3:41] that's been unwritten uh in my mind that [3:43] I've been kind of thinking about um [3:45] three chapters around the process that, [3:47] you know, I've used and teams have used [3:48] to build and I'm here to share that [3:50] today really for the first time in this [3:51] format. So, chapter one is understanding [3:55] people deeply. It all starts with [3:57] people, obviously. And you can figure [3:59] out lots of things that you can build, [4:00] but it comes back to the very root of [4:03] what are the fundamental needs of a [4:05] person. And I love the book um Clayton [4:07] Christensen wrote Competing Against [4:09] Luck. I'm sure many of you have read it [4:10] and are probably read the Jobs to Be [4:12] Done framework, which I really do love. [4:14] The thought is that we don't use [4:16] products, we hire them to do things for [4:18] us, and we have to know what those [4:19] things are. And it's best articulated [4:21] how to extract those kinds of insights [4:23] through a story. And so, actually um a [4:26] team that we had uh at Instagram, we did [4:28] a offsite with the people who created [4:30] the Jobs to Be Done framework, and we [4:31] actually went through this process. So, [4:32] you can all think about this with me. [4:34] What was the last major purchase that [4:37] you made? So, quickly think about it. [4:40] And for me, it was a bed. So, here are a [4:42] bunch of beds. And then what you're [4:44] supposed to do, and the book actually [4:46] talks about how to do this, we've really [4:47] internalized this with our process, is [4:49] to put yourself back into that moment. [4:50] And if you're talking to a person, ask [4:52] them details about their life. What was [4:54] it like that day? Where were you? What [4:55] were you reading? Who were you with? [4:57] What were you wearing? Were you with [4:58] your kids? Were you by yourself? Were [4:59] you in a store? Were you online? And you [5:01] really deeply put yourself in the [5:03] position of that user. And for me, I'm [5:05] looking at this sea of beds with my [5:06] wife. And basically, we were kind of [5:09] trying different things out, and at the [5:11] end of the day, three or four felt [5:13] great, but my wife weirdly turned to me [5:15] and said, "Can you do me a favor and [5:17] jump on this?" Basically. Uh she's like, [5:19] "Do a barrel roll." I was like, "What?" [5:21] Uh all right. And so, I did like a huge [5:23] flop, I guess, and she was like, "We're [5:25] getting this bed right now. Thank you. [5:26] We're done." And I was like, "What? This [5:27] is so weird." She's like, "No, the main [5:29] thing that I need is a bed that will not [5:31] wake me up if you like move." And if you [5:33] think about it, that's like a little [5:35] offensive, but also I was like, "Okay, [5:37] it's real." And if I was a PM for beds, [5:39] then I might be like classically [5:40] thinking, "Quick, what's the road map [5:42] for beds?" Right? You might be thinking, [5:43] "Okay, uh how about cooling? Everyone's [5:45] doing cooling." Like, yeah, cooling, [5:47] sure. Or like eco uh or [5:50] just pure price. Like, those things do [5:52] matter, but for us, the thing that made [5:53] the decision on that day is a concept of [5:55] transference. And if you didn't talk to [5:57] me and you didn't learn that, the PM, [5:59] the team, they would never know that. [6:01] And so, that's what it means to discover [6:03] the job to be done for someone and how [6:05] products, things, customers hire what [6:08] your services do, your products do to [6:09] help them. And very quickly, just [6:11] talking two quick examples in the [6:12] digital world, one from Google. Early in [6:15] our AI experiences, we realized that [6:16] people actually came for inspirational [6:19] journeys. But 2 years ago, chatbots were [6:21] only text. So, we were one of the first [6:23] teams to look at this need and say, "How [6:25] do you get inspired by what to buy or or [6:28] what kind of [6:29] uh couch to get through text? How do you [6:32] describe a couch to someone?" You can't. [6:35] And so, we were one of the earliest [6:36] teams to really invest in [6:37] state-of-the-art multimodal [6:38] understanding, retrieval, and knowledge [6:41] systems, so you could actually ask about [6:43] a couch, get an incredible visual [6:45] response, and do multi-turn conversation [6:47] with it to say, "Oh, I actually want [6:49] emerald green or I want fuzzy." Like, [6:51] what does fuzzy mean? Or like that throw [6:52] pillow, how does it create attention on [6:53] the throw pillow? And do a turn and then [6:55] now do a grid of throw pillows. Like, [6:57] these were really hard things many years [6:58] ago, and obviously you see them now in [7:00] many places, but that was a core need [7:02] and a core insight that we saw. [7:04] And then last example was very early in [7:06] our AI mode exploration, which is a way [7:08] that you can use Google Search now to [7:10] ask anything and have a conversation [7:11] with Google, was people came and it had [7:13] all this incredible information, like, [7:15] here this is a neighborhood and like a [7:17] food question, but it was missing [7:19] something. It was missing something that [7:21] people were coming to Google for and [7:22] were expecting, which was the knowledge [7:25] and the accuracy and the experience of [7:28] actually seeing the places you go, [7:30] seeing them on the map, getting the [7:31] Google level authority of the star [7:34] rating, the closing time, what kind of [7:35] food it is, how expensive it is. It's [7:37] completely missing from the product. And [7:39] when we brought that together, so that [7:40] now you can talk to Google, ask about [7:43] food, and then get this rich experience [7:45] that brings all the goodness of seeing [7:47] the food and clicking on an entity now [7:49] and actually seeing the description that [7:50] you can trust, that was the magical [7:53] product. And it's still one of the most [7:54] popular things on the product today [7:56] based on thumbs up ratings and other [7:57] things that we follow. But again, back [7:59] to a human inside of the why behind they [8:02] were using the experience and what [8:04] caused them to use it in the first [8:05] place. [8:07] And so AI can actually change a lot of [8:08] this and a lot of the talk today is how [8:10] AI can apply to these. I think on this [8:12] one, I'm still thinking about some of [8:13] these uh myself, but one of the things [8:16] I'm excited to try more is how you can [8:18] actually take the transcript data of [8:19] these user interviews and actually have [8:21] it specifically sourced jobs to be done [8:23] and scale that much more. Another idea [8:25] I've been thinking about um is actually [8:27] an agent that can interview people with [8:28] the exact methodology that I just [8:30] described so that it can really ask a [8:33] user more at scale to talk about why [8:35] they're using the product they're using [8:36] and what those underlying problems were. [8:39] So, that is understanding people. Next [8:41] up, you have a seed. You have something [8:44] you're working towards. The next step, [8:46] diagnosing root causes. So, the way I [8:49] think about it is you have this initial [8:50] vision of this problem you're solving [8:51] for people, this initial understanding [8:53] of what people want. And then you make a [8:55] thing. And that thing is usually not [8:57] great. [8:58] And most of the things that I've ever [9:00] been a part of building started not [9:02] great. Everything did, really. Um and [9:05] what's incredible about this process is [9:08] by having a clear analytical view of the [9:11] exact problems that are in your product [9:13] space, ranking them in priority and [9:15] understanding them deeply, and then [9:17] actually fixing them and then asking it [9:19] recursively again and really looping [9:20] through, you basically are doing your [9:22] own training process, kind of like an [9:24] epoch of model training a little bit. [9:26] And each update you do with this change [9:28] list gets you closer to the market fit [9:30] you want until you have that fit, and [9:32] then it works. There's two stories I [9:33] wanted to kind of articulate here, both [9:35] uh from Instagram. Uh one was when we [9:38] originally launched stories it actually [9:40] did take off. It was one of the earliest [9:41] things that worked pretty well out of [9:43] the box, but we had many internal [9:45] versions where stories were fit into the [9:46] feed that didn't work great. So there [9:49] was a ton of [9:50] headache to get to the the final [9:51] version. But one thing that was [9:53] particularly clear was that not everyone [9:55] was adopting it or felt comfortable [9:57] sharing. And really the whole premise [9:59] was hey this is an opportunity to share [10:00] your life throughout the day. Why aren't [10:02] you doing this? And the process that we [10:03] followed was pretty simple. We asked a [10:05] very clear root cause of why you were [10:08] not able to do the thing that was our [10:09] goal. So it's kind of the inverse. The [10:11] first one's like why did you do the [10:12] thing? Now you have a thing and it's [10:13] like well why aren't you doing the [10:14] thing? And it turns out that if you ask [10:16] that question people answered think of [10:18] all the leaf nodes like a cloud behind [10:19] me of all of the answers people had. And [10:22] then if you were to group them by theme [10:24] it turned out that the top one when you [10:26] sorted them [10:27] was something around audience problems [10:29] more or less. Like my ex is on it. My [10:31] teacher is on it. My aunt is on it. My [10:34] sister is on it. I'm not going to like [10:35] just put a goofy thing on like people [10:37] are on it. And so what we realized in [10:39] seeing that and then what we did was we [10:41] then did a very large quantity survey [10:42] thousands of people taking that as an [10:44] input and then we were able to quantify [10:46] the actual weights across them. And what [10:48] we figured out was that if we couldn't [10:49] overcome that burden there's no creative [10:51] tool there's no like interesting product [10:53] idea that would work. And so the team [10:55] actually spent two years cycling through [10:58] different ideas in the space until [11:00] we came across close friends and then it [11:02] ended up working. And there were all [11:03] kinds of weird permutations in this like [11:05] multiverse where like it's called [11:07] favorites and it also has on your [11:09] profile a back door that you can like [11:12] see a like a private version of your [11:14] profile and you can also post to your [11:15] feed under that kind of close friends [11:17] label. Super confusing no one understood [11:19] it. Like why would I put this like very [11:21] raw photo in the middle of like a glitzy [11:22] feed of all this stuff that's happening. [11:24] The only thing that worked was within [11:26] stories. So we cut out the entire [11:27] product relaunched it as a stories [11:29] experience and it worked. [11:31] Reels was the second example. We had a [11:33] We had a a launch for Reels in Brazil [11:36] and we thought at the time that if [11:39] you're doing goofy things and dances and [11:41] posting them, why would you want them to [11:42] live on your profile for everyone to see [11:44] forever, which made sense and the team [11:45] felt that way. So, we launched it, huge [11:48] failure. So, it turns out that if you [11:50] invest a bunch of time doing an awesome [11:51] dance, you don't want it to go away in a [11:54] day, which was an ephemeral format. It's [11:55] very obvious in hindsight. But, the the [11:58] bigger insight was the people who were [11:59] posting them, even if they were young, [12:01] you know, people, they wanted to be [12:03] entertainers. They wanted to make [12:05] businesses out of their content and they [12:06] wanted it to live on and go viral. And [12:08] so, we did a massive update. We talked [12:10] about this change list, where now it's [12:12] its own format. Instead of being [12:13] ephemeral, it lives like forever and it [12:16] can be something your whole school sees [12:17] instead of the opposite, like, oh, I [12:19] don't want my whole school to see it. [12:20] It's like, no, that's actually what you [12:21] want. You want everyone to get the joy [12:24] from this product. And then, obviously, [12:25] that take has taken off and done quite [12:27] well over the years. So, two really [12:29] clear examples um of how that's [12:31] happened. And with AI, this is actually [12:33] possible to do a lot easier now using [12:35] internal tools and I think things that [12:37] you can all do every day. And we use, [12:38] you know, internally Google [12:39] anti-gravity, but we have a system that [12:41] effectively lets users that have opted [12:44] into this, they want to talk to us more [12:46] and they've decided that they're they're [12:48] comfortable sharing as they're using [12:49] Google products with us. And so, what [12:52] will happen on the side is we will then [12:53] ask them, "Hey, what did you think of [12:55] this? Did it Where did it miss the [12:56] mark?" And they will put comments in. [12:58] And so, we imagine this huge library of [12:59] feedback from people now that's [13:01] available. Well, now, the model can take [13:03] that and can both take qualitative [13:05] feedback like above. This is not This is [13:06] a way a shopping example for backpacks. [13:08] It's like, it's awesome that you knew [13:09] about each backpack and it's awesome [13:11] that you knew which was larger and the [13:13] sizing cuz Google has all this canonical [13:15] information, but you should have asked [13:16] me about my son and the height. Like, [13:19] that's a pretty key thing or it's going [13:20] to be too heavy for my kid. And it was [13:22] so obvious that when you stack rank [13:25] them, this was a very common kind of [13:26] feedback. so this concept of being [13:28] collaborative and and creating a [13:30] conversation so you can actually help [13:32] the the person make progress with to the [13:34] ultimate objective. They don't actually [13:36] know all the things that they need to [13:37] know to answer it was absolutely [13:39] critical to getting the user to vote and [13:40] say this was awesome. And so we invested [13:43] a ton of energy in this and it's been [13:44] one of the biggest things we've done in [13:46] terms of engagement, in terms of usage, [13:48] in terms of value and helpfulness, you [13:49] know, that we've seen out of the [13:50] product. So that's kind of chapter two, [13:53] uh, which is around understanding the [13:55] root causes. Now, if you're lucky, you [13:57] have a great seed idea, you've done this [13:59] process of finding the market fit by [14:01] going looping through, seeing your [14:03] problems, analyzing them rigorously, and [14:05] fixing them aggressively. Maybe you get [14:07] to the stage now where the product is [14:09] ready for primetime. You're going all [14:11] out you're going to all your customers, [14:12] you're sending the blast email. And [14:14] really this is about the fit and finish [14:16] and final elements of the product, which [14:17] is around craft. And I think this is [14:19] often overlooked, but basically the way [14:21] I think about craft is the feeling that [14:23] I think everyone has a general sense and [14:25] I believe people can detect whether the [14:27] creators of a product should care about [14:29] the product that they created, [14:30] basically. [14:32] And there's two things that I think [14:33] drive this. One is does the product work [14:36] perfectly? And is there any user pain, [14:38] basically, in that experience? And the [14:40] second is how does it make me feel? Do I [14:42] feel something, joy, excitement using [14:44] it? And so I'm going to talk about each [14:45] real quick. Um, on the pain side is [14:48] really not just about obsessing over it [14:50] and you all being power users of your [14:52] own product, but actually how to scale [14:54] that now with AI, which is really [14:56] incredible. And so I actually built a [14:57] tool that I'm sharing here. It's a real [15:00] tool that I built, um, internally using [15:02] the anti-gravity system to basically [15:05] look at all of the potential questions [15:07] we might ask. So here's a sample here on [15:09] the side. [15:10] Use the product as an agent, so like [15:12] literally like go to Google, type in a [15:14] question, go to the AI and ask a [15:16] follow-up, and then record that [15:18] experience, screenshot it, and then [15:19] evaluate what you see based on a really [15:22] comprehensive rubric. [15:23] And what we found out was that the model [15:25] actually was incredibly capable to look [15:27] at things like, "Hey, there's a math [15:28] question." or this is like a school [15:30] homework thing. Is it doing a great job [15:32] rendering math in LaTeX? [15:34] Or is it, you know, [15:36] emitting weird uh characters? Or if you [15:38] ask about bioluminescence, is it adding [15:40] a visual tray, which is what the [15:42] response should have if you're asking [15:43] about something visual? Or is it just [15:44] text? And you can go through and you not [15:47] only can you get to the best response, [15:48] but you can see breakages really easily [15:50] and the team easily and the team has now [15:52] invested in a bunch of agents that can [15:53] go figure out when stuff is just broken [15:56] or off spec, and then increasingly fix [15:59] it itself. And you have this feedback [16:00] loop and where [16:02] used to have maybe a QA team, you you [16:03] may need a PM, or you might need people [16:05] on your team who are obsessively using [16:06] every single part of the product. I [16:08] think many of those days are now over, [16:10] where we can scale this process much [16:12] more elegantly for everyone. And then [16:14] the last thing I'm going to say [16:15] is around, you know, delight and really [16:18] sparking joy. And this is a little ex- [16:20] perience. This is what the Google search [16:21] box looks like now. We we reinvented [16:23] this the search box after 20 years, [16:25] announced it at I/O, what an AI-based [16:27] search box should be. It should grow [16:28] with you. It should be able to upload [16:30] things, and have context around files, [16:32] around videos, around photos. What What [16:34] should that be like? And And how [16:36] daunting of an experience is that? And [16:38] what we did was we came up with a a [16:40] system that said, "Okay, we wanted [16:41] Google to still feel Google." And but we [16:44] wanted it to be this upgrade that felt [16:45] like AI is here. It's in There's AI [16:48] inside that's helping you. And so [16:49] there's this concept of this kind of [16:51] gradient of color that really represents [16:53] the AI kind of power behind the product. [16:56] And you can actually see a couple nice [16:58] details here that the design team [16:59] brought together. One was actually in [17:01] the cursor, if you watch it blinking, [17:03] it's cycling through the different [17:04] Google colors. [17:06] And I I can't tell you how many times [17:07] I've seen this pop up on X that like [17:09] someone noticed this and and and and and [17:10] just and felt like posting it. It just [17:12] blows me away every time. Or if you tap [17:14] in the search box, it's kind of excited, [17:16] it like jumps out kind of and then loads [17:18] the ring. It's like, "Ooh, like we're [17:20] getting ready to load. Like we're we're [17:21] here, you know?" It has this kind of [17:22] feeling. And it actually has real [17:24] feeling with haptics. So, you actually [17:25] see Imagine all that coming together as [17:27] an experience. I mean, this is one small [17:29] example of just by using the product and [17:31] through motion, through color, through [17:32] feel, through touch, you know, and the [17:34] intentionality of the experience, you [17:36] communicate the humanity behind the [17:37] product and you give it a soul. And I [17:39] think many people overlook this, you [17:41] know, part of the the product creative [17:42] process. [17:43] And then really, when you bring this [17:45] kind of all together, this is kind of [17:47] the the wrap here. I think that this [17:48] really means that you can have a a [17:50] playbook for what it means to be a PM in [17:52] this new, you know, AI frontier. And [17:55] it's you have to still do all the same [17:57] things. Like, a lot of this is not new, [17:59] but it's it's possible in a totally new [18:01] light with AI. So, you have to [18:03] understand people deeply, get at their [18:05] core needs and problems that they're [18:06] feeling. You have to be rigorous and [18:08] analytical at sorting out, "What are the [18:10] things I need to fix in order to get the [18:11] product to win and to work for people [18:13] against that objective?" And I need to [18:15] build it so that it's awesome and it's [18:16] simple to use and it works great and it [18:18] makes people feel awesome using it. And [18:20] that's really the playbook. And I think [18:22] that now it's exciting that the craft of [18:24] PM is so critical. I think it's never [18:26] been more important time to be a PM and [18:28] to do product and to be a builder. And I [18:30] think the skill of making great [18:32] decisions that can lead to great [18:33] products has also never been, you know, [18:35] more important. So, hopefully all of you [18:37] can take a little bit of something. This [18:38] is just one way. There's so many ways to [18:40] build products. You've probably heard 10 [18:41] other ones today that were just as valid [18:43] as this one. You should decide on what [18:44] makes sense for you, but hopefully you [18:46] take one thing away um that you can take [18:48] away with you and to build as well. And [18:50] hopefully the next time you're thinking [18:51] about your products, you're spending a [18:53] little bit less time thinking about, [18:55] "Hey, here's cool features kind of out [18:56] of the blue or here's some tech we're [18:58] going to throw at it or here's a new way [18:59] we're going to get it in front of [19:00] people." And you spend more of your time [19:02] thinking about the actual people and the [19:04] humans on the other side of the screen [19:07] that you're building for. [19:08] Thank you so much. [19:10] >> [applause]