AI Coding Surge Shifts Bottlenecks to Product Strategy, Panelists Say

与会专家表示：AI编程热潮正将瓶颈转向产品战略 随着自动化智能体消除工程约束，产品管理的价值正决定性地从撰写需求工件转向重大的战略研判与直接的客户探索。 本次研讨探讨了人工智能如何使产品管理的核心瓶颈从战术性工件撰写转向高杠杆的战略判断与优先级排序。尽管各机构正在利用生成式 AI 自动编写用户故事、精简 Jira 工作流并快速搭建交互式实时原型，但行业领袖告诫切勿以算法输出取代深度的客户同理心与面对面的需求发掘。讨论还梳理了团队架构的演进，包括精炼的三人实验小组，以及产品经理向亲力亲为的技术构建者与商业总经理两极分化的趋势。 AI 正在重塑产品管理，组织的主要瓶颈已从工程交付吞吐量转移到产品战略方向与假设验证的速度上。 (0:04) 产品团队应专注于优化驱动决策的上游商业判断，而非单纯加速撰写 PRD 和发布说明等下游文档产物。 (1:39) 企业级智能搜索与 AI 系统的相关性本质上是一种由业务上下文和权衡策略界定的规则，而非基础大语言模型天然具备的属性。 (20:16) 生成式 AI 将产研生命周期转变为“信任并验证”模式，使产品经理的时间分配从 70% 撰写文档颠倒为 70% 用于战略评估与决策判断。 (14:55) 坚持人工手写战略文档能确保思考的深度与清晰的权衡界限，AI 负责搜索探索决策空间，而人类牢牢掌控最终的综合决策。 (18:44)

As generative AI automates routine software development and documentation, product leaders argue that the primary operational bottleneck has shifted from technical execution to upstream product judgment, discovery, and strategic prioritization. Shifting Priorities from Artifacts to Upstream Decisions (1:39) Rather than merely accelerating document drafting or meeting summaries, enterprise leaders argue that artificial intelligence delivers the most value by enhancing strategic evaluation. Chika Anajemba emphasizes optimizing the judgment behind artifacts rather than the documents themselves, using AI to test roadmaps against business constraints and surface hidden risks. Saurav Gupta notes that Eli Lilly has shifted its product management posture from slow drafting to a "trust and verify" model, reallocating the 70% of time previously spent assembling artifacts into high-leverage decision-making, while advancing projects across a "5 Ps" framework: Proof-of-Concept, Pilot, Product, Platform, and Performance. Simultaneously, engineering acceleration has exposed product prioritization as the main organizational choke point. Rohan Suri notes that because AI coding tools allow engineers to deliver substantially more output with unchanged headcount, determining what to build and validating bets with customers has become the primary constraint. Consequently, Nooks has increased its hiring focus on product management. Operational Workflows and the Principle of Contextual Relevance (9:19) Panelists illustrate internal tooling successes while stressing the limitations of automated outputs without strict business rules. Natalie Barnes explains that Zumba replaced an initiation process taking up to a week with multi-model AI agents and Claude skills; once a feature is scored, the system provisions Jira boards, drafts Confluence hubs, generates business rules, and outputs user stories. In early stages, product managers use AI design skills to build functional UI prototypes, letting designers focus on styling systems and engineers test feasibility before sprint planning. However, technical deployment often exposes contextual gaps. Anajemba recounts that Citi's enterprise search pilot confidently returned irrelevant outputs for financial queries like "interest rates" because the engine split the terms across disparate documents. She emphasizes that "relevance is a policy and not a property," meaning search and prioritization require clear human weighting schemes—such as recency versus authority—rather than purely algorithmic scale. Preserving Customer Empathy and Human Synthesis (17:49) Despite aggressive AI adoption, leaders warn against allowing automation to displace direct human intuition and discovery. Suri highlights that Nooks deliberately requires all PRDs and strategy briefs to be handwritten by product managers. While AI slices quantitative data and pulls verbatim interview quotes, humans must author the final two-page document because writing forces clear trade-offs and prevents the spread of uncurated AI-generated material. Anajemba warns that the greatest threat facing product managers is behavioral displacement—using automated ticket analysis as an excuse to avoid sitting with customers and navigating ambiguity. Suri and Barnes add that machine models cannot pick up on unspoken social cues, vocal defensiveness, or environmental tracking anomalies, such as Zumba's cookie banner update that caused an artificial 15% drop in analytics traffic which AI alone could not explain. Evolving PM Archetypes and Organizational Topologies (28:06) As routine execution is commoditized, product management is polarizing into specialized profiles. Suri observes a squeeze on middle PMs who handle formulaic requirements, while demand increases for technical "builder PMs" and commercial "general manager PMs" focused on business metrics and go-to-market execution. At Eli Lilly, Gupta cites frameworks indicating future PMs will orchestrate hybrid groups of human colleagues and autonomous agents, balancing time-to-market against quality rework and compliance. AI also enables leaner pod structures tailored to risk. At Nooks, high-risk 0-to-1 initiatives operate with one PM and two engineers tasked with validating or terminating an experiment within two to three months, whereas scaled platform teams pair one PM with seven to ten engineers. To manage the resulting surge of concurrent projects without administrative bloat, leadership relies on AI to monitor and synthesize project communications across internal channels. While AI tooling dramatically accelerates prototyping, administrative overhead, and code delivery, panelists agree that the durable core of product management remains anchored in commercial judgment, qualitative customer relationships, and defining success criteria.

来自金融、制药、健身与企业级软件领域的资深产品领导者指出，生成式AI通过消除战术摩擦与工程编码瓶颈，正将产品管理的关键制约由下游文档撰写彻底转向前端的战略判断、需求洞察与业务取舍。企业不应仅利用AI优化交付物本身的生成速度，而应利用其提升产生交付物背后的决策质量与实际商业价值。 瓶颈转移：从文档起草走向战略决策 (1:39) 花旗集团（Citi）的Chika Anajemba指出，AI在下游战术执行（如撰写PRD、总结会议、润色发布说明）上表现卓越，但真正的蓝海在于赋能上游决策。产品经理应利用AI转化客户信号、压力测试路线图并揭示潜在业务风险，其核心准则是“不要只优化产出物，而要优化产生该产出物的判断力”。Nooks首席产品官Rohan Suri同样观察到，过去六至九个月内，AI代码生成能力使工程团队产出显著激增，这反倒消除了研发产能瓶颈，使“构建什么”以及“产品团队的战略优先级与决策速度”成为企业新的核心卡点。 这种重心重塑深刻改变了产品经理的时间分配。礼来公司（Eli Lilly）AI转型高级总监Saurav Gupta说明，生成式AI使团队从缓慢的手工起草转向“信任并验证”模式。传统模式下PM需耗费70%的时间收集材料和起草文档、仅留30%进行战略决策；而在AI赋能后，70%的时间被释放给战略评估与判断，仅需30%用于审查与验证模型产出。面对医药健康的严监管环境，Gupta提出从概念验证（POC）到试点、产品、平台并最终聚焦性能的“5P”成熟度框架，以系统性遏制无实质产出的原型蔓延。 工作流重构：端到端自动化与即时原型 (9:19) 内部AI工具链正逐步接管产品运营中的机械化环节。Zumba副总裁Natalie Barnes分享，团队通过构建端到端的多模型智能体系统与Claude定制技能（涵盖技术产品申请TPR、范围界定及设计原型），实现了项目发起流程的自动化。这套系统可根据公司KPI自动生成Jira看板、Confluence项目中心、业务规则与结构化用户故事，将以往需耗费数天甚至一周的行政事务大幅压缩，有力支撑了服务全球逾10万名认证教练和1500万每周学员的庞大业务体量。 即时交互原型的普及彻底重构了敏捷开发与对齐机制。礼来公司的数据显示，AI使产品管理效率提升了30%；PM可以在干系人会议中即刻生成可交互UI原型，将抽象争论转化为对具体切片（Gupta比作“最小可行披萨”）的评估，冲刺规划的重心随之从“开发什么”转向“排除什么”以及深挖异常路径与故障模式。在Nooks，仅需10分钟即可搭建云端原型并向客户验证概念，甚至使产品经理能直接向工程团队提交初步代码拉取请求。 AI的局限：手写PRD与业务相关性政策 (17:49) 多位专家提醒，生成式AI虽然擅长海量信息处理，却缺乏真实共情与情境直觉。Nooks虽然研发自主GTM智能体，但坚持要求所有PRD与战略文档必须由产品经理纯手写完成并保持在数页内。Rohan Suri强调，“写作即思考”，将战略起草外包给模型会剥离PM深度权衡、推敲假设和做出抉择的能力；Nooks明确界定AI仅用于发掘输入和检索决策空间，最终的综合与定夺必须由人类掌控，以此防范团队生成不经推敲的“AI垃圾（slop）”。 在搜索与用户洞察层面，人类判断同样不可替代。Chika Anajemba以花旗银行智能搜索上线为例指出，系统曾将“利率（interest rates）”拆解匹配为通用的“兴趣”与“比率”，这表明“相关性是一项政策，而非内在属性”，其排序权重必须由深谙业务情境的人类定义。Chika进一步警告，AI的最大威胁并非模型幻觉，而是“行为替代”，即促使产品经理停止坐下来与客户沟通并直面模糊性。AI能够快速分析上万张工单，但无法感知访谈中的微妙情绪或洞察未被言明的潜意识需求。 团队拓扑重塑、角色分化与价值评估 (28:06) 基础文档撰写的商品化推动了产品经理角色的极化与挤压。Rohan Suri指出，仅依循程式化执行的“中间层PM”正面临生存危机，行业加速向两类画像分流：一类是技术背景深厚、能编写原型并与少数工程师探索前沿垂直领域的“构建型PM”（Builder PM）；另一类是商业与销售敏感度极高、擅长资源调配与GTM定位的“业务/总经理型PM”（GM PM）。在组织拓扑上，Nooks针对高风险的“0到1”创新设立仅由1名PM与2名工程师构成的精炼小组，在两至三个月内高频验证并决定去留；成熟平台线则维持1名PM配比7至10名工程师，甚至由工程经理直接负责产品。 面对激增三至四倍的并发项目，企业对ROI的评估也从单纯关注速度转向多维价值验证。Saurav Gupta指出，礼来公司从上市速度、返工与缺陷预防、等效“数字劳工”产出及监管合规四大支柱评估成效。Natalie Barnes亦强调，Zumba将AI概念严格置于A/B测试中验证用户价值，而非孤立追踪Token消耗；同时，管理层通过部署Claude对内部Slack频道进行动态归纳，以自上而下的AI智能体统筹监控大量并发项目，避免微观管理。 生成式AI并没有削弱产品经理的战略价值，而是加速了工程交付并接管了低附加值的文档起草，从而将行业竞争推向更高维度的商业直觉、需求真实性洞察与战略取舍；然而，如何在极简小组的高速并发试错与金融医药等强监管环境的安全合规之间保持动态平衡，仍是企业规模化落地中需要持续探索的议题。

## Background 该内容由科技行业大会频道 Ai4 于 YouTube 平台发布，记录了一场题为“AI 赋能产品管理（AI Powered Product Management）”的专家圆桌论坛。研讨会汇集了来自 Zumba、Meta、Citi、Eli Lilly 以及 Nooks 等企业的资深产品与工程领袖，深入探讨产品经理如何将生成式 AI 作为副驾驶，加速需求起草、用户研究与原型设计，同时厘清 AI 提效与人类不可替代判断力之间的边界。 ## TL;DR 圆桌嘉宾围绕 AI 在金融、医药、消费健身与企业服务等领域的落地实践展开探讨，指出 AI 的核心价值在于解放以往占据产品经理 70% 精力的机械性文档与战术事务，使瓶颈转移至更高价值的商业决策与判断力上。研讨深入分析了“相关性是业务策略而非算法属性”、AI 原型重塑产研评审与敏捷交付的模式，并警示团队切勿因“AI 疲劳”而将提示词答案误当成真实用户洞察。最后，嘉宾展望了未来产品团队向“人 + 智能体”微型 Pod 演进的趋势，以及 PM 角色向深耕技术的 Builder PM 和聚焦商业落地的 GM PM 两极分化的必然方向。 ## Atmosphere & style 整场研讨氛围专业、务实且充满一线实战的锐利思辨，完全没有浮于表面的概念炒作。多位跨行业领袖在对 AI 原型与自动化提效表达兴奋的同时，始终保持着对技术局限性的理性克制，尤其在讨论手写 PRD 的必要性与算法业务定义时迸发出极具深度的思想火花。对话节奏紧凑而直击痛点，既展示了技术爆发带来的组织焦虑，又传递出坚守人类商业直觉、现场同理心与高阶判断力的坚定底气。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=4s) AI 在产品管理中的核心定位并非取代战略愿景，而是通过接管下游机械工作来放大并增强产品决策与判断力。 - [01:39](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=99s) 产品经理应当将 AI 应用重点从加速撰写 PRD 等下游产出物，转移到上游的业务判断、风险分析与路线图假设压力测试。 - [03:48](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=228s) 受监管行业应推行“5P 模型”（PoC -> Pilot -> Product -> Platform -> Performance），避免 PoC 泛滥并迈向平台化与业务结果导向。 - [07:40](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=460s) 随着 AI 编码智能体大幅提升工程开发效率，产研瓶颈已从工程实现端转移至产品端——“决定做什么”与“客户反馈闭环”成为新挑战。 - [09:20](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=560s) 企业级场景中的“相关性”是一项业务策略而非纯算法属性，必须结合业务上下文与人类定义，而非单纯依赖更复杂的模型。 - [11:29](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=689s) 通过定制化多模型 AI Agent 与专属技能，团队可实现项目看板、业务规则与基础故事的自动化生成，大幅缩短运营层流转周期。 - [17:49](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1069s) Nooks 坚持核心 PRD 必须由产品经理纯人工手写，利用 AI 检索与切片数据，但通过亲笔撰写来强迫思维清晰化并直面业务权衡。 - [22:07](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1327s) AI 辅助高保真原型使产品评审重心从抽象争论“做什么”转向攻坚边缘用例与非理想路径，有效提升产研协同与敏捷评审效率。 - [26:06](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1566s) 警惕过度依赖自动化输出导致的“AI 疲劳”，现成的 Prompt 答案不能替代面对面客户访谈与深度情境洞察。 - [32:58](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1978s) 未来产品经理将面临两极分化：要么是贴近代码与底层方案的 Builder PM，要么是主导战略、组织与商业结果的 GM PM，传统通用型 PM 空间被挤压。 - [37:14](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=2234s) 衡量生成式 AI 的 ROI 应聚焦上线时间、交付质量与返工率、数字员工复用度及合规性，而非单纯计较 Token 消耗成本。 - [39:02](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=2342s) 得益于 AI 带来的单兵效能飞跃，0 到 1 创新产品已能够以“1 名 PM + 2 名工程师”的极简微型 Pod 敏捷探索与试错。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=4s) 圆桌开场与主持人背景介绍 会议由 Meta 机器学习软件工程经理 Amir Emadzadeh 主持。Amir 简要介绍了自己的专业背景：他拥有加州大学洛杉矶分校（UCLA）电子工程博士学位，长期深耕于 AI 技术的跨行业落地应用，曾在自动驾驶企业负责车辆定位技术研发，在生物医药领域构建用于优化临床试验的 AI 产品，目前在 Meta 带领团队负责广告排序模型的 AI 产品开发。 Amir 指出，AI 正在迅速成为产品经理（PM）工具箱中的核心组件，其根本目的并非取代战略愿景，而是为了“放大与增强”战略愿景。在接下来的研讨中，来自软件服务、消费级健身、全球金融以及生物医药等多个行业的领军人物将分享他们如何将 AI 融入日常工作流程、加速市场与用户研究，并重新定义产品团队的领导方式。随后，Amir 邀请各位嘉宾分别进行自我介绍，并分享各自今年在产品管理中应用 AI 的“第一优先级（Number One Priority）”。 ```mermaid flowchart TD accTitle: AI 在产品管理中的定位 subgraph CoreGoal["AI 赋能产品管理的核心定位"] A["战略愿景 Strategic Vision"] -->|放大与增强 Amplify| B["产品决策与执行 Execution"] end subgraph MultiIndustries["横跨多元行业的实践场景"] C["全球银行业 (Citi / Goldman)"] D["生物医药 (Eli Lilly)"] E["消费健身 (Zumba)"] F["企业级 GTM (Nooks)"] end B -.-> MultiIndustries ``` --- ### [01:39](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=99s) 优化判断力而非仅产出文档：金融领域的 AI 产品观 曾在高盛（Goldman Sachs）和花旗集团（Citi）拥有超过 16 年金融服务行业经验的 Chika Anajemba 首先发言。Chika 的职业生涯始终专注于将宏观战略落地为具备企业级影响力的产品、平台和数字化能力。 谈及今年的首要任务，Chika 直言，目前 AI 在产品经理的“下游工作（Downstream work）”中表现亮眼，例如快速撰写产品需求文档（PRD）、提炼会议纪要、清洗发布说明（Release notes）等。然而，AI 赋能产品管理的最大机遇，在于重构上游工作流以帮助产品经理“做出决策，而不仅仅是加速产出文档”。 Chika 强调，团队应当利用 AI 将零散的用户信号转化为具备优先级的深度洞察，而不是停留在个案反馈；同时，AI 应该用于对照战略目标与业务约束对路线图决策进行压力测试，并在实际研发执行前系统化分析风险、底层假设与依赖项。 > “问题不应当仅仅是‘我怎样用 AI 更快写出 PRD’，而应该是‘我怎样用 AI 做出更好的产品决策’——不要只去优化产出物本身，要去优化孕育产出物的判断力。” - **核心观点**：从“加速撰写下游交付物（PRD / 会议纪要）”转向“赋能上游业务判断与决策”。 - **关键应用场景**：客户信号结构化优先级排序、路线图假设压力测试、执行前依赖与风险分析。 --- ### [03:48](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=228s) 强监管环境下的 AI 落地与 5P 转型思维 礼来制药（Eli Lilly and Company）AI 转型与产品管理高级总监 Saurav Gupta 拥有 21 年产品管理经验，长期致力于为患者、医疗提供方及护理人员打造数字产品。Saurav 指出，在受到严格监管的医疗健康领域，AI 的负责任落地尤为关键，因为一旦出错，面临的绝不仅仅是“糟糕的用户体验”，而是直接关乎患者健康结局（Patient outcomes）、患者生命安全以及严苛的合规风险。 Saurav 提出了今年 AI 赋能产品团队的三大核心任务： 1. **结果导向转型**：借助 AI 工具让产品经理从繁琐事务中解脱出来，转向真正以结果（Outcome-driven）为核心。 2. **重塑工作交付方式**：彻底改变团队的工作模式，以适应市场、患者及医疗服务提供方日益加快的需求变化节奏。 3. **摆脱 PoC 蔓延，推行 5P 模型**：打破业内普遍存在的概念验证与试点泛滥（PoC and pilot sprawl），转向可复用的产品与平台化思维。 ```mermaid flowchart LR accTitle: Saurav Gupta 的 5P 演进模型 subgraph Exploration["探索阶段"] P1["1. PoC (概念验证)"] --> P2["2. Pilot (试点)"] end subgraph Scale["规模化与复用"] P2 --> P3["3. Product (产品)"] P3 --> P4["4. Platform (平台)"] end subgraph Evaluation["持续度量"] P4 --> P5["5. Performance (效果与闭环)"] end ``` - **5P 演化路径**：从 PoC（概念验证）与 Pilot（试点），跨越到可复用的 Product（产品）与 Platform（平台），并最终以 Performance（实际业务效果与敏捷迭代）为衡量准绳。 --- ### [05:37](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=337s) 全球化健身社区的运营自动化与产品战略聚焦 Zumba 健身产品与设计副总裁 Natalie Barnes 介绍了 Zumba 的全球业务版图。Zumba 远不止是一家拉丁舞健身品牌，其核心业务是为全球超过 10 万名认证教练构建数字化教学与社群体验，这些教练每周为全球约 1,500 万名 Zumba 学员授课。 作为产品、设计和 QA 团队的负责人，Natalie 面对的是跨越多语言、多文化习俗的复杂全球市场。她指出，团队今年的核心目标是利用 AI 实现运营层（Operational level）事务的自动化与极致优化。 通过将日常反复消耗精力的运营性流程交由 AI 处理，团队能够将省下的宝贵时间重新投资于产品判断与战略思考中，集中精力深入评估“应该推出什么产品、不应该推出什么产品”，从而为全球教练和学员创造最大的实际价值。 --- ### [07:40](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=460s) 消除产品瓶颈以匹配 AI 时代的工程交付速度 B2B 营收智能平台 Nooks 的首席产品官 Rohan Suri 介绍了其公司的业务：Nooks 专注于构建底层基础设施，帮助企业部署“营收智能体（Revenue Agents）”来创建、推进并转化销售管线（Pipeline）。 Rohan 分享道，在过去 6 到 9 个月中，得益于 AI 编码与工程智能体的飞跃，Nooks 在人员编制和资源保持不变的情况下，极大提升了产品路线图的雄心壮志（Ambition），开始大规模启动之前积压在 Backlog 中的高价值赌注。 然而，研发速度的大幅跃升导致团队瓶颈发生了转移：编码智能体在代码实现上极其高效，使得研发端的瓶颈被打破，而产品团队本身反而成为了新的瓶颈——“究竟该构建什么产品？”、“如何建立足够敏捷的客户反馈闭环？”以及“如何快速推向市场（GTM）？”成为了新的核心挑战。因此，产品组织的首要任务是加速产品决策与验证流程，防止产品规划成为工程交付的阻碍。 --- ### [09:20](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=560s) 企业级智能搜索重构：“相关性是一种策略，而非单纯属性” Amir 随后向 Chika 和 Natalie 提问，希望她们分享团队近期成功落地或正在积极研发的内部 AI 工具及工作流案例。 Chika 分享了他们为旗下专有业务平台主导的“智能搜索平台重构（Intelligent Search Replatforming）”项目。该项目将 AI 能力直接嵌入一线搜索功能中，横跨 5 个不同的市场业务部门，服务超过 10 万名月活跃用户（MAU）。 在落地过程中团队发现，在企业级复杂场景中将 AI 嵌入搜索并满足每个细分业务群体的需求极具挑战。例如，某个业务板块的用户在搜索特定术语时，AI 模型给出的结果可能混杂了适用于其他业务板块的内容。这要求产品团队必须深入业务一线，重新定义不同板块下“相关性（Relevance）”的具体标准。 Chika 总结道：“相关性是一项业务策略，而不是一个单纯的技术属性（Relevance is a policy and not a property）。”它绝不是构建好 AI 模型后再随意添加的补充插件，而必须从项目立项第一天起，就作为核心业务上下文深度融入到整个流程之中。 ### [11:29](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=689s) Natalie Barnes：AI Agent 重塑 Zumba 的端到端产品运营工作流 Natalie Barnes 分享了 Zumba 在过去一年中如何高度聚焦于将 AI 深度融入产品运营层（Operational Layer）。在引入 AI 之前，团队的标准工作流程冗长繁复：产品经理或业务方首先提出项目需求，进入范围界定（Scoping），随后使用 RICE 评估模型进行评分与优先级排序；确定立项后，产品经理必须在 Jira 中手动配置项目、自定义字段和任务看板以符合规范，接着在 Confluence 建立项目中心枢纽页面，撰写详细业务规则（Business Rules）与测试计划。整套流程走下来，往往需要数天甚至整整一周的时间。 为了打破这一效率瓶颈，Natalie 与现场参会的同事 Yelva 及内部 AI 团队共同研发了专用的多模型 AI Agent。当一个项目完成优先级排序后，AI Agent 会全自动生成 Confluence 项目主页、业务规则页面以及基础 User Stories。虽然产品经理依然需要审查细节并补充上下文，但这一机制极大地缩减了机械化文档工作，让团队能以极快速度进入研发交付环节。 ```mermaid flowchart TD accTitle: Zumba 基于 AI 的产品流程自动化架构 subgraph Input["需求输入与评估"] A["业务方/PM 提出项目需求"] --> B["Scoping 与 RICE 评分"] B --> C["完成优先级排序"] end subgraph AIAgent["AI Agent 自动化生成"] C --> D["AI Agent 触发"] D --> E["生成 Jira 项目与看板配置"] D --> F["生成 Confluence 中心枢纽"] D --> G["生成业务规则与 User Stories"] D --> H["关联测试计划"] end subgraph Skills["Claude 专属技能扩展"] I["TPR 撰写 Skill<br/>(内置业务规则与 KPI)"] J["Scoping 范围界定 Skill"] K["Claude Design 原型设计 Skill"] end subgraph HumanReview["人工审核与交付"] E & F & G & H --> L["PM 审查校验与细节补充"] K --> M["交付研发团队快速构建"] end ``` 此外，Zumba 还在内部利用 Claude 构建了多种定制化“技能（Skills）”： - **TPR（技术/产品需求）编写技能**：深度注入了 Zumba 的内部业务规则与 KPI 知识库，使 AI 生成的需求文档高度契合业务目标，直接用于指导后续评估。 - **范围界定技能（Scoping Skill）**：辅助团队精准拆解 TPR 中的技术与业务边界。 - **Claude 原型设计技能（Design Skill）**：允许产品经理直接生成高保真交互原型并快速同步给工程团队。 Natalie 强调，工程师团队同样在全面应用 AI 进行辅助开发。团队的终极目标不是让 AI 接管一切，而是“利用 AI 承揽运营层面的机械工作，将省下来的时间全力投入到高价值的人类商业判断中”。 --- ### [14:55](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=895s) Saurav Gupta：瓶颈转移——从文档起草转向高价值商业决策 针对生成式 AI 如何改变产品日常战术工作的问题，Saurav Gupta 指出，GenAI 带来的最本质变化在于**“工作流瓶颈已经彻底从内容起草（Drafting）转移到了决策制定（Decision Making）”**。在礼来（Eli Lilly）等企业的实践中，GenAI 已经贯穿软件开发生命周期（SDLC）与产品全生命周期的每个阶段，涵盖需求拆解、User Story 编写、自动化测试生成以及上线后的指标监控。团队逐步确立了“先信任后验证（Trust and Verify）”的敏捷心态，摆脱了过去大量耗时低效的人工手动操作。 在用户研究领域，GenAI 带来了“广度与速度”的巨大飞跃，能快速为产品方向提供全景指引；而在医疗与医药等严肃行业，产品需要直接面对真实的患者与医护人员，因此团队通过“AI 调研（速度与广度）+ 人类定性研究（深度与同理心）”的混合模式，实现了兼具效率与深度的用户洞察。在日常会议中，PM 甚至可以直接利用 AI 实时生成产品原型和交互界面进行讨论。尽管这会带来“原型看起来过于成熟，导致业务方误以为马上就能上线”的预期管理挑战，但它极大加快了产品验证的节奏。 Saurav 总结道，过去产品经理往往要将 **70% 的时间耗费在信息拼装与文档草拟**上；而在 GenAI 赋能后，这 70% 的精力被完全释放出来，转而投向**商业判断、方案权衡与关键决策**——这正是产品管理者最不可替代的核心价值所在。 --- ### [17:49](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1069s) Rohan Suri 与 Chika Anajemba：人类直觉、手写文档与“相关性”的本质定义 当主持人将话题引向“AI 缺乏同理心与真实生活体验，人类直觉在何时必须推翻 AI 结论”时，Rohan Suri 提出了一个略带争议却极其坚定的观点：**在 Nooks，每一份 PRD（产品需求文档）都坚持纯人工手写，绝不交由 AI 直接生成**。 Rohan 解释，撰写一份 1 到 2 页的 PRD 或战略文档，其核心输入是数据分析和客户访谈记录。Nooks 充分利用 AI 去检索海量对话、提取精准的用户原话引用、进行多维度数据切分；但最终将这些输入凝练为 PRD 的过程，必须由产品经理亲自落笔。因为“书写过程本身会强迫你建立清晰的思考”，迫使 PM 直面最底层的业务权衡。他提到，团队早期曾发现市场开拓（GTM）团队滥用 AI 生成了大量充斥冗余信息的“垃圾内容（Slop）”，公司随即明确立下规矩：**“我们决不允许 AI 代替人类做决策；AI 的定位是提供输入并帮助探索决策空间，但敲定战略与撰写文档的责任永远属于人类。”** ```mermaid flowchart LR accTitle: Nooks 与 Citi 对 AI 与人类边界的实践认知 subgraph Nooks["Nooks: 决策与文档原则"] N1["海量客户对话与数据"] --> N2["AI: 检索切片与提取引述"] N2 --> N3["人类 PM: 手写核心 PRD<br/>(强制思维清晰化与战略定夺)"] end subgraph Citi["Citi: 算法与业务定义"] C1["智能搜索返回不准<br/>(利率查询混淆)"] --> C2["归因分析: 权重与意图错位"] C2 --> C3["人类介入: 重构业务成功标准<br/>('相关性是一种策略，而非属性')"] end ``` 随后，Chika Anajemba 分享了花旗银行（Citi）在落地智能搜索平台 POC 时遭遇的真实案例。当时搜索模型返回的文本结构工整、语气极其自信，但实际检索效果却令人大失所望。例如当用户搜索“利率（Interest Rates）”时，系统返回了大量单独包含“Interest（兴趣/利息）”或单独包含“Rates（比率）”的文章，真正讨论“Interest Rates”的核心内容反而被淹没。 Chika 由此提出了一个深刻洞见：**“相关性（Relevance）是一种业务策略（Policy），而非算法的固有属性（Property）。”** - **AI 并没有出故障**：模型只是在忠实执行某人显式或隐式设定的权重机制（如关键词重合度、时效性、权威度或广度）。 - **破局之道在业务定义而非算法本身**：团队没有盲目追求“构建更复杂的模型”，而是深入业务一线重新向真实用户调研，弄清楚在他们的工作场景中“相关性”究竟意味着什么，进而重新调优排序权重。 - **人机分工的关键**：AI 负责按既定规则高效运算，而定义“什么是成功、什么是真正的业务相关”，必须由具备行业上下文的人类来主导。 ### [22:07](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1327s) 从静态 PRD 到高保真原型：重塑产研协作与设计流程 Natalie Barnes 分享了团队在产品初期阶段让产品经理（PM）亲自动手制作原型的探索与实践。以往 PM 如果只是拿着一份 PRD（产品需求文档）、Confluence 页面或 PPT 来阐述构想，工程师在评审时的反应往往充斥着各种理论性质的质疑：“这到底怎么运转？点击这里后数据流向哪里？”这是因为大家都在抽象层面空谈。一旦开始通过工具把原型直观地搭建出来，系统各模块如何串联运作便一目了然。 对于复杂项目，团队鼓励 PM 必须制作原型，以便工程师清晰理解交付目标；而对于小型项目，PM 甚至可以直接生成带代码的原型交付给研发。这种模式不仅让后期的技术质询大幅减少，还激发了工程师在遇到问题时主动提出更优的替代解法。对于产品设计师而言，AI 赋能的原型制作也能直接代劳约 75% 的前置结构工作，设计师随后只需专注于视觉规范、品牌指南（brand guidelines）与细节打磨，整个产研流水线的流转速度显著提升。 ```mermaid flowchart TD accTitle: AI 原型驱动的产研协同流程 subgraph Ideation["需求与原型阶段"] A["PM 构思业务需求"] --> B["AI 快速生成高保真原型 / 代码"] end subgraph Collaboration["多方协同深化"] B --> C["产品设计: 补充品牌与样式规范 (完成剩余 25%)"] B --> D["研发团队: 明确非理想路径与异常边界"] end subgraph Delivery["高效落地"] C --> E["加速高质量交付并优化架构方案"] D --> E end ``` --- ### [24:02](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1442s) 敏捷评审的范式转移：“最小可行披萨”与售前极速验证 Saurav Gupta 从度量指标切入，指出团队实测 AI 对产品管理生产力的提效幅度已达 30%，但这远非终点。AI 原型彻底改变了 Sprint Planning（冲刺规划会）的讨论重心：团队不再把时间浪费在争论“要做什么”，而是能够迅速对“不做什么”达成共识。过去的敏捷设计普遍擅长描绘“理想路径”（happy path），却容易忽视异常情况；现在借助原型，团队可以在评审中集中火力攻坚边缘用例（edge cases）、非理想路径（non-happy paths）以及对应的测试方案，极大完善了“准备就绪定义”（Definition of Ready, DoR）与“完成定义”（Definition of Done, DoD）。 Saurav 还提出了一个生动的类比——“MVP（Minimum Viable Pizza，最小可行披萨）”：现在的协作就像直接端出一块热腾腾的切片披萨去验证市场，而不是干巴巴地兜售底层面饼，使讨论变得极具象且可落地。紧接着，Rohan Suri 从初创公司角度印证了这一趋势：在 Nooks，团队在售前阶段（pre-sales）广泛使用原型做概念验证。如今借助云端工具，制作并部署一个交互原型的成本“几乎缩短到了 10 分钟”，PM 甚至可以直接提交 PR（Pull Request）来呈现功能原型，大幅加快了从概念到实际执行的闭环。 - **30% 效率提升**：AI 工具在产品需求定义与原型阶段带来的可量化生产力飞跃。 - **10 分钟极速原型**：初创团队在售前与早期验证中搭建云端交互概念的时间成本降至极低。 - **聚焦非理想路径**：评审焦点从功能罗列转向异常流、边界测试用例与容错设计。 --- ### [26:06](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1566s) 警惕 AI 疲劳：别把提示词答案当成客户真实洞察 针对当前 SaaS 工具全盘 AI 化带来的“AI 疲劳”与过度依赖自动化输出的现象，Chika Anajemba 发出了深刻警示。她指出，AI 在产品管理中最危险的隐患并非“给出错误答案”，而是“诱使我们放弃那些真正关键的人工行动”——即停止了与客户坐在一起面对面深聊、停止了在复杂模糊（ambiguity）中探索核心问题的耐心。 Chika 强调，AI 确实可以在“你的咖啡变凉之前，飞速处理完 10,000 张客服工单”，这具备切实的提效价值，但它永远无法替代“你尚未与客户进行的那场对话”。当任何问题都可以通过一段 Prompt（提示词）快速获得解答时，PM 极易落入将“现成答案”混同于“深度洞察”（confuse answers for insights）的陷阱。如果团队只顾沉迷于花哨的数据看板和生成报告，而没有挖掘出能指导商业决策的真实洞察，产品体验会在无形中悄然恶化，团队却还在为虚假的“高生产力”自嗨。为此，她总结了一条金句原则： > "AI reads the room, but people still have to be in it."（AI 可以洞察局势与全貌，但人必须身在现场。） --- ### [28:06](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1686s) 未来 PM 的能力重塑：商业判断力、情境感知与人机协作 面对用户故事编写、需求撰写和早期原型逐渐被 AI 自动化的现状，Saurav Gupta 直言，社区对“PM 是否会被取代”存在普遍焦虑，但他认为未来企业不仅不会削减 PM，反而需要更多具备新型素质的产品经理。过去占据 PM 70% 精力的执行琐事将被全面商品化（commoditized），未来的 PM 必须进化为结果导向、具备极强商业判断力与全局视野的人才。他引用了 Claude 团队的“AI 采纳阶梯”（AI Adoption Ladder）理念与 MIT 教授 Paul Cheek 的著作观点，指出未来组织将向全面 AI 原生（AI-native）演进，团队将变成一半人类、一半自主代理（autonomous agents）的混合体，PM 的核心竞争力在于判断何时调度 AI Agent、何时与人协同。 Natalie Barnes 补充指出，人类的**判断力**与**现实业务情境（Context）**是 AI 永远无法自主补全的壁垒。她举了一个切身案例：团队在网站上线了隐私政策勾选框，由于部分用户未同意授权，Mixpanel 无法追踪这部分行为，导致分析看板中同比流量出现了约 15% 的异常跌幅；连接了数据看板的 Claude 只能基于现有指标进行归因，却完全缺失“合规弹窗导致统计口径变化”这一真实业务背景，最终仍需 PM 手动调取 Google Search Console 进行交叉验证。 | 评估维度 | 传统产品经理 (PM 1.0) | AI 时代战略型产品经理 (PM 2.0) | |:--- |:--- |:--- | | **日常精力分配** | 70% 耗费在撰写 PRD、拆解 User Story、画线框图 | 聚焦商业结果、边界探索与高阶价值判断 | | **原型验证周期** | 依赖设计与前端排期，耗时数天或数周 | 借助 AI 工具在 10 分钟到数小时内自主生成代码原型 | | **团队协同形态** | 纯人工产研团队跨职能沟通 | 管理“人 + 自主 AI Agent”的高度混合型团队 | | **数据洞察来源** | 被动依赖仪表盘与提示词生成的表面结论 | 结合现实环境 Context（如合规/业务突发变量）进行深度研判 | 在谈及未来的人才招聘标准时，Natalie 强调她最看重两点：**对新技术的持续好奇心（Curiosity）**以及**将 AI 视为运营杠杆的熟练度**。AI 能够协助解决 70% 到 75% 的常规链路，但临门一脚的商业定夺与环境纠偏始终离不开人类的介入。优秀的 PM 应当拥抱工具，从而将省下的精力全力倾注于最具杠杆效应的高阶判断之中。 ### [32:58](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1978s) 初创企业中产品经理与工程师配比的变化及两极分化趋势 在初创企业环境下，控制成本与提高资金效率是永恒的主题。Nooks 的首席产品官 Rohan Suri 分享道，大约六个月前，团队决定将产品路线图的野心与目标翻倍，但这并没有导致盲目扩张工程师团队，而是优先招募更多产品经理（PM）。在如今 AI 大幅赋能工程开发的背景下，工程师的单兵产出与效能显著提升，导致“产品经理与工程师的配比（PM-to-Eng Ratio）”反而朝着“每位工程师配备更多 PM”的方向发展。产品端（即明确要做什么、怎么做）正在成为新的产能瓶颈。 针对 PM 职能的发展，Rohan Suri 指出行业正在出现两极分化的趋势，夹在中间的传统通用型 PM 空间正被挤压： 1. **Builder PM（技术构建型 PM）**：这部分 PM 极其贴近代码、技术架构与底层方案，往往拥有深厚的工程背景以及出色的产品直觉。他们通常与 1-2 名工程师组成精简敏捷的小组，直接探索全新的垂直领域与市场，以极高的技术理解力主导路线图方向。 2. **Business PM / GM PM（商业/业务总经理型 PM）**：这部分 PM 则将技术实现完全放权给工程师，聚焦于驱动整体商业结果。他们负责组织架构（Staffing）、人员调配（确保对的人做对的事）、探索正确战略押注、推动 GTM（Go-to-Market）以及赋能销售团队。他们具备极强的人际交往能力与商业嗅觉。 ```mermaid flowchart TD accTitle: AI时代产品经理的角色分化 subgraph Squeezed["被AI挤压的中间层"] Mid["传统通用型 PM<br/>（仅负责撰写简单 PRD / 基础需求）"] end subgraph Divergence["两极分化的新型 PM 角色"] direction LR Builder["Builder PM（构建型）<br/>• 极度贴近技术与底层代码<br/>• 强工程背景 + 敏捷探索新业务"] GM["Business / GM PM（商业型）<br/>• 聚焦商业结果与战略押注<br/>• 负责 GTM、人员调配与销售赋能"] end Mid -.->|"AI 自动化基础需求撰写<br/>迫使能力向两端演进"| Divergence ``` - 撰写简单的需求文档不再是 PM 工作的核心壁垒，AI 能够轻松完成此类基础工作；真正的难点在于“选择押注哪个方向”以及“提出并解答关键业务问题”。 - 在客户发现（Customer Discovery）环节中，捕捉客户的微妙肢体语言、社交线索或防备情绪，属于高度依赖人性的能力，AI 在短期内依然无法替代真实的人际连接。 --- ### [37:14](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=2234s) 衡量生成式 AI 投资回报率（ROI）与成效的核心指标 现场观众就“企业如何衡量使用 GenAI 的成效与 ROI 指标”进行了提问。多位嘉宾结合各自行业的落地实践给出了具体的考量维度： Saurav Gupta 从企业 AI 转型的角度归纳了四大核心考量维度： 1. **上线时间 / 成果产出时间（Time to Market / Time to Outcomes）**：评估产品迭代与交付周期的整体压缩程度。 2. **返工率与交付质量（Quality & Rework Rate）**：单纯追求速度毫无意义，必须确保产出质量稳定，避免因幻觉或低级错误导致大量返工。 3. **数字员工转化与复用（Digital Workers Conversion）**：将可复用的智能体（Reusable Agents）转化为具备稳定生产力的“数字员工”，量化其带来的实际效能收益。 4. **合规性（Compliance）**：在受监管或企业级场景中，确保生成式 AI 应用始终符合合规与安全要求。 Natalie Barnes 则从产品设计与管理视角补充指出，他们重点关注产品上市时间，并通过大规模 A/B 测试来验证 AI 驱动的新特性是否真正为终端客户创造了核心价值。她特别提到，团队目前并不过度纠结于“Token 消耗成本”，而是将重心完全放在“如何跑得更快、如何更快地向客户交付切实价值”上。 --- ### [39:02](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=2342s) AI 对团队架构（Pods）的影响与多管线管理实践 在讨论 AI 对团队组织形态的影响时，Rohan Suri 介绍了 Nooks（约 300 人规模）基于不同产品性质所采用的“差异化 Pod（小敏捷组）结构”： - **0 到 1 探索型产品**：组建极度精简的 Pod（通常为 1 名 PM 配备 2 名工程师）。该小组全权负责寻找设计合作伙伴（Design Partners）、快速运行实验并在数月内决定验证成果。如果假设不成立，团队必须果断“杀死（Kill）”该项目。得益于 AI 提升单兵能力，这种微型 Pod 在过去是不可想象的。 - **成熟平台型产品（Platform Products）**：维持相对传统的比例（如 1 名 PM 与 1 名设计师对接 7 至 10 名工程师）；在技术属性极强且工程主管能力突出的领域，甚至直接由工程经理（EM）兼任产品负责人，不再额外配备 PM。 在管理多条并行推进的产品管线时，项目通常由产品负责人从客户对接、设计伙伴招募一路跟进到 GA（全面开放可用）阶段的评审门禁。尽管并行项目数量可能激增 3 至 4 倍，但管理者可以借助 AI 工具（例如利用 Claude 自动检索并总结各个 Slack 频道的进展与讨论）轻松掌握全貌，有效降低了多线协作的管理负荷。 ## Notable quotes > [01:39](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=99s) 问题不应当仅仅是‘我怎样用 AI 更快写出 PRD’，而应该是‘我怎样用 AI 做出更好的产品决策’——不要只去优化产出物本身，要去优化孕育产出物的判断力。 — Chika Anajemba > [09:20](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=560s) 相关性是一项业务策略，而不是一个单纯的技术属性（Relevance is a policy and not a property）。 — Chika Anajemba > [11:29](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=689s) 利用 AI 承揽运营层面的机械工作，将省下来的时间全力投入到高价值的人类商业判断中。 — Natalie Barnes > [14:55](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=895s) 工作流瓶颈已经彻底从内容起草（Drafting）转移到了决策制定（Decision Making）。 — Saurav Gupta > [17:49](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1069s) 我们决不允许 AI 代替人类做决策；AI 的定位是提供输入并帮助探索决策空间，但敲定战略与撰写文档的责任永远属于人类。 — Rohan Suri > [26:06](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1566s) AI reads the room, but people still have to be in it.（AI 可以洞察局势与全貌，但人必须身在现场。） — Chika Anajemba > [32:58](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=1978s) 撰写简单的需求文档不再是 PM 工作的核心壁垒，AI 能够轻松完成此类基础工作；真正的难点在于‘选择押注哪个方向’以及‘提出并解答关键业务问题’。 — Rohan Suri > [37:14](https://www.youtube.com/watch?v=qHmNh6D5S_o&t=2234s) 团队目前并不过度纠结于‘Token 消耗成本’，而是将重心完全放在‘如何跑得更快、如何更快地向客户交付切实价值’上。 — Natalie Barnes ## Mentioned Amir Emadzadeh, Chika Anajemba, Saurav Gupta, Natalie Barnes, Rohan Suri, Meta, Goldman Sachs, Citi, Eli Lilly and Company, Zumba, Nooks, Claude, Jira, Confluence, Mixpanel, Google Search Console, Slack, Paul Cheek, MIT ## Video description Natalie Barnes - VP of Product Management & Product Design, Zumba Amir Emadzadeh - Software Engineering Manager ML, Meta Chika Anajemba - Former Senior Vice President, IT Project Group Manager, Citi Saurav Gupta - Senior Director, AI Transformation, Eli Lilly & Company Rohan Suri - Chief Product Officer, Nooks AI is quickly becoming an indispensable tool in the product manager’s toolkit—not for replacing strategic vision, but for amplifying it. This panel will explore how PMs can use AI as end users to accelerate market research, generate customer insights, draft requirements, and even prototype ideas before handing them to design and engineering. Panelists will share real-world examples of leveraging generative AI for user story creation, clustering customer feedback, and producing creative variations of product concepts. The conversation will highlight where AI meaningfully speeds up product work, and where human intuition and leadership remain irreplaceable. Attendees will leave with a practical playbook for how to make AI a true co-pilot in the product management process. [Source](https://www.youtube.com/watch?v=qHmNh6D5S_o)

[1:39] Chika Anajemba: 问题不应当仅仅是‘我怎样用 AI 更快写出 PRD’，而应该是‘我怎样用 AI 做出更好的产品决策’——不要只去优化产出物本身，要去优化孕育产出物的判断力。 [9:20] Chika Anajemba: 相关性是一项业务策略，而不是一个单纯的技术属性（Relevance is a policy and not a property）。 [11:29] Natalie Barnes: 利用 AI 承揽运营层面的机械工作，将省下来的时间全力投入到高价值的人类商业判断中。 [14:55] Saurav Gupta: 工作流瓶颈已经彻底从内容起草（Drafting）转移到了决策制定（Decision Making）。 [17:49] Rohan Suri: 我们决不允许 AI 代替人类做决策；AI 的定位是提供输入并帮助探索决策空间，但敲定战略与撰写文档的责任永远属于人类。 [26:06] Chika Anajemba: AI reads the room, but people still have to be in it.（AI 可以洞察局势与全貌，但人必须身在现场。） [32:58] Rohan Suri: 撰写简单的需求文档不再是 PM 工作的核心壁垒，AI 能够轻松完成此类基础工作；真正的难点在于‘选择押注哪个方向’以及‘提出并解答关键业务问题’。 [37:14] Natalie Barnes: 团队目前并不过度纠结于‘Token 消耗成本’，而是将重心完全放在‘如何跑得更快、如何更快地向客户交付切实价值’上。

[1:39] Chika Anajemba: Don't just optimize the artifact, but optimize the judgment that produces the artifact. [9:19] Chika Anajemba: Relevance is a policy and not a property. [18:44] Rohan Suri: We don't let AI make the decisions. We let AI provide inputs and help us search the decision space, but ultimately we're going to be the ones writing it. [26:06] Chika Anajemba: what it makes us stop doing, which is sitting with customers and sitting with ambiguity. [26:06] Chika Anajemba: AI reads the room, but people still have to be in it.

[0:04] Hi everyone, welcome to our AI powered [0:07] product management session. I'm Amir [0:09] Ahmad Zada and I'll be moderating [0:11] today's uh conversation. Just a quick [0:14] background about me. Um I am with Meta. [0:17] I uh my team builds AI products um for [0:21] ranking models uh in ads. Uh my [0:24] background is um in AI uh applying AI in [0:28] different industries. I have worked for [0:30] self-driving car companies doing [0:32] localization and also biotech uh [0:35] building um AI products for improving [0:37] clinical trials. I hold a PhD in [0:40] electrical engineering from UCLA and um [0:43] as you know AI is fast becoming a core [0:46] part of the product manager toolkit not [0:49] to replace replace strategic vision but [0:51] to amplify it. Over the next 40 minutes [0:54] or so, we will hear from this incredible [0:58] panel of leaders who are putting AI to [1:00] work in uh their day-to-day jobs um from [1:04] across several different industries from [1:06] software to um to consumer fitness to [1:10] global banking and pharma um as they [1:13] share how they are using AI to speed up [1:15] research and uh rethink what it means to [1:18] lead product teams. So uh I'd like [1:21] everyone um to start by giving a quick [1:24] intro about themselves and also tell me [1:27] uh about one number one priority uh this [1:32] year they have uh when it comes to [1:34] applying AI to their product management [1:37] maybe we can start from you. [1:39] >> Sure. Thanks Amir. Hi everyone. I'm [1:42] Chica An Amba and I help organizations [1:46] solve complex business problems by [1:49] turning strategy into products, [1:51] platforms and capabilities that create [1:54] enterprise impact. Whether that's [1:56] through data, digital transformation or [1:59] AI that has been a focus of my career [2:02] which is helping organizations move from [2:04] ambitious ideas and challenges to [2:07] execution at at scale. I've done that uh [2:11] for 16 plus years across financial [2:13] services most recently at city and [2:15] previously at Goldman and it's the [2:18] perspective that I'm excited to bring to [2:20] today's discussion. So um in answering [2:23] your question so I think one of the [2:25] priorities for product managers this [2:27] year you know uh so actually let me [2:30] start with first off I think AI has been [2:33] very good in the downstream work for [2:36] product managers whether that's in [2:37] writing PRDS um summarizing meeting [2:41] notes or cleaning up release notes but I [2:44] think there's a big big opportunity for [2:46] AI and product management to build [2:49] better product management workflows that [2:51] enable product managers to make [2:52] decisions and not just make documents or [2:56] you you know write documents faster. Um [2:58] and so you know whether that's in [3:00] turning uh customer signals into [3:04] prioritized insights instead of just re [3:06] relying on anecdotal feedback stress [3:09] stress testing roadmap decisions against [3:12] strategic objectives and business [3:15] constraints or analyzing risks [3:18] assumptions and dependencies before [3:21] execution begins. There's a huge [3:23] opportunity there and I think that's [3:25] something that we should be looking at. [3:27] So in other words, you know, the [3:28] question shouldn't just be how can I use [3:30] AI to write faster PRDs, but or or to [3:34] write PRDs quicker, but how can I use AI [3:37] to make better product decisions? So [3:40] don't just optimize the artifact, but [3:43] optimize the judgment that produces the [3:45] artifact. [3:46] >> Sounds great. How about you, Rob? What's [3:48] your perspective? [3:49] >> Hey guys, my name is Raf Gupta. I'm with [3:51] Eli and Lily and I'm responsible for [3:53] product management and AI transformation [3:55] over there. uh I've been in uh product [3:59] management for last 21 years and trying [4:02] to create a lot of digital products for [4:04] like real people and hard problems our [4:06] patients providers and caregivers and my [4:09] focus for this year is just making sure [4:12] how can we adopt or enable AI [4:15] responsibly in a regulated environment [4:17] right because anything wrong going in [4:19] that environment you're not talking [4:21] about bad user experience but you're [4:23] talking about patient outcomes patient [4:25] safety as well as uh compliance risk. Uh [4:29] my top priority for this year uh is [4:32] three-fold. Uh one, how can we make our [4:36] product managers uh to get away and [4:39] become more outcomedriven by using AI? [4:41] Second, how the work gets done, right? [4:44] We need to change how the work gets done [4:46] with all these tools enabled so that we [4:48] can go faster because market needs are [4:51] changing faster, our patient needs are [4:53] changing faster. uh what our uh provider [4:56] needs is changing faster. And third uh [4:59] very important thing for us actually for [5:01] our organization is how can we actually [5:05] uh move away from this uh p and pilot [5:08] spraw right uh what I call is 5s um [5:12] which is moving from p uh to pilot and [5:16] move actually towards product and [5:18] platform mindset. think more about [5:21] reusable products, reusable platforms [5:24] and eventually look at uh fifth P which [5:27] is performance which is how is actually [5:30] anything that you're putting out using [5:32] AI is performing and how can you can [5:34] learn and adopt quickly. [5:36] >> Awesome Natalie. [5:38] >> Good morning everyone. My name is [5:40] Natalie Barnes. I am the VP of product [5:42] at Zuma Fitness. [5:44] Um, I'm sure most of you probably are [5:46] thinking, you know, isn't that just a [5:48] Latin dance fitness um, company? Yes, it [5:52] is. But there's actually a business [5:53] behind it. And that business is ensuring [5:56] that our core customers, which are well [5:59] over 100,000 instructors worldwide who [6:02] take a training and then they become a [6:04] part of our community. It's building [6:06] those digital experiences for them to be [6:08] able to teach classes to about 15 [6:12] million uh students, Zuma students on a [6:15] weekly basis. And my team and I actually [6:18] focus on making sure that those digital [6:21] experiences actually add value. And then [6:25] the fact that we are a global company, [6:26] we have to make sure that we do it in [6:29] many different languages. We have to [6:31] understand many different cultural [6:33] norms. So as the leader of the product [6:37] team, the product design team and the QA [6:40] team, our focus is consistently trying [6:42] to figure out how can we add the most [6:44] value for them. And with AI, I think one [6:50] of our goals for this year is making [6:51] sure that we optimize or automate the [6:56] operational level of what we do with AI. [7:00] That way we can reinvest that time in [7:03] judgment calls and making sure that you [7:05] know we make better decisions for the [7:08] products that we put out there and for [7:09] the products that we don't put out [7:10] there. So for us it's more about how do [7:12] we leverage AI to the point where those [7:15] operational things that we do on a [7:17] regular basis become you know just part [7:21] of the process that we don't have to [7:24] worry about and spend time on anymore. [7:27] Instead, we focus on how do we add the [7:30] most value for our customers? How do we, [7:32] you know, spend more time on strategy [7:34] and yeah, I'm going to share hopefully a [7:36] lot of that with you guys today. [7:38] >> Sounds great. How about you, Rohan? [7:41] >> Hi, good morning everyone. Um, I am I'm [7:44] Rohan. I'm one of the I'm chief product [7:47] officer at a company called Nooks. Uh at [7:50] Nooks we effectively help companies we [7:53] we build infrastructure to help [7:54] companies build revenue agents to create [7:58] close and expand pipeline. So everything [8:01] go to market related um since we sell to [8:04] other businesses. So a lot of like [8:05] companies like uh that you guys might be [8:07] working at um I think um you know uh [8:12] because of AI in the last probably like [8:15] 6 to9 months we've been able even given [8:18] the same headcount and resources been [8:21] able to dramatically increase the [8:23] ambition of our product roadmap. Um, and [8:26] uh, I think probably every everyone here [8:28] is probably in that position where [8:30] they're saying, "Hey, can we can we do [8:31] more? Can we take the bets that were in [8:33] the backlog now and now actually, um, [8:36] potentially execute on those?" And so, [8:39] um, that's something that we started [8:40] about 6 months ago. Um, and just trying [8:43] to be much much more ambitious in terms [8:44] of the road map we have. And uh one of [8:47] our big priorities is you know making [8:50] sure now that uh product isn't the [8:52] bottleneck because effectively what you [8:54] have is agents are very very good at [8:56] engineering and coding. Uh but now the [8:58] bottleneck is like okay like what do you [8:59] build? How do you get fast enough [9:01] customer feedback loops? How do you take [9:02] that to market? And so a very very big [9:05] priority for me and the whole product [9:07] org is to um kind of remove that [9:10] bottleneck so we can you know really [9:11] execute on a more ambitious roadmap and [9:14] taking a lot of our new products and [9:15] bets to market. So yeah excited to share [9:18] more about that. [9:19] >> Sounds good. Um this question uh is for [9:23] um Chica and Natalie. Uh, I want you [9:26] guys if possible to share a story about [9:29] an internal AI tooling initiative or [9:32] workflow that your teams have either [9:35] successfully rolled out or is actively [9:37] developing. Maybe Chica, you go first. [9:41] >> Yeah, sure. Um, I can share on on that. [9:44] Um, so we recently uh led a intelligent [9:48] search replatforming for one of our [9:50] proprietary platforms. Um and that you [9:53] know involved in embedding AI into the [9:56] frontline capabilities particularly for [9:59] search. Um so that was very interesting. [10:01] You know we had to cover five different [10:03] market divisions. Um you know it's a [10:06] platform that ser serves over 100,000 [10:09] monthly active users. Um and so one of [10:12] the things that we found, you know, when [10:14] we implemented that was [10:16] that it's not very easy to, you know, [10:20] when you're embedding AI to [10:24] to cater to every C customer or client's [10:26] needs. Um, and one of the ways that that [10:29] surfaced was, you know, just to give an [10:30] example, when a user, one particular [10:33] business segment would search on a [10:35] particular term and it would throw up [10:38] results that were not necessarily just [10:41] for that uh uh user, but could also [10:44] apply for others. And so that really [10:47] meant that we had to, you know, uh, [10:48] engage with the business to understand, [10:51] okay, what does relevance actually mean [10:53] for you? Because, you know, we talk [10:55] about building better models. Um but [10:57] it's not just always about improving [10:58] better models. It's very important that [11:01] you we understand the context for the [11:03] business in order to be able to surface [11:05] the right results. Um so one thing that [11:08] I say is relevance is a policy and not a [11:10] property. You know it's not an [11:12] afterthought that you embed when you [11:14] that you um sort of put in as an add-on [11:17] when you are uh enabling AI. It very [11:20] much has to be part and parcel of the [11:22] process from the get-go in order to get [11:25] surface the right results for your [11:27] clients. [11:29] >> Awesome. How about you Natalie? [11:31] >> Yeah, we were hyperfocused this year on [11:34] making sure that we can leverage AI as [11:38] much as possible as I mentioned before [11:40] in the operational layer. So our typical [11:42] process before was you you create uh or [11:45] either the product manager or the [11:47] business owner creates a project [11:49] request. That project request then goes [11:51] to scoping. Then from scoping once it [11:55] gets prioritized once it gets scored we [11:57] use the rice method to score it. Once it [11:59] gets scored it gets prioritized then we [12:01] start we create a jer project. And that [12:05] jer project needs its custom fields. It [12:07] needs everything and and the boards and [12:10] everything else that we do to make sure [12:11] that the project follows our standards. [12:14] After that, the product manager will [12:16] create a confluence page which is kind [12:19] of like the central hub for their [12:20] project where all the links are in [12:22] there. Every detail about that project's [12:24] in there. So the rest of the business [12:25] have an idea as to you know what exactly [12:28] this project is about. In addition to [12:31] that, we create the business rules which [12:32] links off of that confluence page. [12:34] recreate the the uh test plan which is a [12:38] part of that confluence page as well. So [12:40] there's a whole process that the product [12:42] manager went through and that process [12:43] could take anywhere from a few days to a [12:46] week. We decided I sat down with one of [12:49] my product manager who's here in the [12:51] audience Yelva and we decided we need to [12:54] find a way to optimize that process. So [12:57] we built our own we have our own AI [13:00] team. We built a um an agent using [13:04] different models that actually takes [13:06] once a project gets prioritized once you [13:09] know it's scoped and it's ready and it [13:11] gets prioritized [13:12] that uh agent creates the confluence [13:16] page it creates that business rules page [13:18] it creates the user stories it doesn't [13:21] end there of course the product manager [13:23] has to go through and make sure that we [13:25] have all the details that we need and [13:26] you know everything is in line and we [13:29] always end up having to add things to [13:31] that to that uh confluence page. But [13:35] that already took up a lot of the the [13:38] you know time and it it gets us to that [13:41] product build a lot faster. We also went [13:44] as far as to use claude. We use claude [13:46] internally to create a skill that [13:49] creates a TPR and that skill actually [13:52] has all the context of our business. It [13:54] understands you know what our business [13:56] rules are. It understands our KPIs. So [13:59] then it can create a product or a [14:02] project request that actually has [14:05] everything um we need in order to scope [14:08] that that um project. And then we also [14:10] created a skill for scoping and uh for [14:15] scoping the TPR. [14:17] And finally, we al also have a design [14:20] skill where a product manager can now go [14:23] into uh claw design and create a [14:26] prototype of that project and then we [14:28] can send it off to the engineers. Our [14:31] engineers are also using um AI to build [14:35] and we've been you know experimenting [14:38] throughout the process. We're not done [14:40] with that process. The ideal is for us [14:43] to get from we have an idea and then we [14:46] use AI to help us you know with the [14:49] operational parts of it but we actually [14:51] make those judgment calls with the time [14:53] that we've saved. [14:55] >> Awesome. Um this next question is for [14:59] Rob. Uh we hear a lot about AI [15:02] accelerating market research clustering [15:04] feedback overdrafting requirements. [15:08] Where are you seeing Gen AI make the [15:10] most meaningful impact in speeding up [15:12] the day-to-day tactical work of your [15:14] POS? [15:16] >> Yeah, so I would say with Gen AI um the [15:20] bottleneck has shifted from drafting to [15:23] actually decision making, right? And [15:24] that's where I think the magic happens [15:26] and you can really get like a [15:28] productivity out of it. Um we've been [15:31] using Gen AI uh in different u phases [15:34] and and different areas of product [15:36] management. one exactly what Natalie [15:38] said, right? We have JN AI end to end [15:40] for a product uh product development [15:42] life cycle right from from the time you [15:44] are actually understanding the [15:46] requirements all the way till the time [15:48] you like you uh deploy the product and [15:51] then monitor it uh every part of the [15:54] SDLC or product development life cycles [15:56] we have AI enabling it whether it's uh [15:58] requirements creation user stories test [16:01] test automation um where or whether it's [16:04] for even monitoring [16:07] uh what metrics we need to go after and [16:09] all. So more and more with genai we're [16:11] trying to like move towards trust and [16:13] verify mindset rather than trying to [16:16] spend a lot of time like manually doing [16:19] the work. Uh we are actually also uh [16:23] using uh generative AI uh to augment our [16:27] user research. uh so when you look into [16:29] like user research you can have uh like [16:32] AI based research which can help you [16:33] with the speed and breadth and you can [16:35] directionally actually know how uh the [16:39] research like how your product uh [16:41] product should come around but you can [16:43] also then augment that with uh uh with [16:46] the human user research which is [16:47] important in our space because you are [16:49] tackling with uh with real um patients [16:52] and providers. So we augment our user [16:54] research so that we can uh with that we [16:57] can actually also get more depth and and [17:00] work towards that. Uh other thing where [17:02] we are using gen AI is very much in like [17:05] prototyping like we are in the meetings [17:07] and actually we can learn things we can [17:09] quickly prototype and actually we can [17:10] show that what prototype is this what [17:12] like should look like and even define [17:14] the the experiences that really helps us [17:17] to like uh get in uh get hands-on in [17:20] like developing product very quickly. Uh [17:23] again that comes with a risk and [17:25] prototype that looks like a finish and [17:27] there's an expectation that they would [17:28] be out very much quickly in the market [17:30] but that's like level setting. Uh so [17:33] just to like round up I think where most [17:35] our product managers were spending uh [17:37] 70% of their time in assembling and [17:40] drafting and now that 70% of time they [17:43] can now focus more on like decision [17:45] making and judgment and that's where we [17:46] need them to be focusing on. [17:49] >> Makes sense. Um I have actually a [17:51] related question now for um Rohan and [17:53] Chica. Um as you mentioned AI is [17:56] brilliant at synthesizing syn [17:58] synthesizing data but it lacks empathy [18:00] and lived experience. [18:03] Can you share an example where AI [18:05] generated insights or prototypes looked [18:08] great on paper but human intuition and [18:11] leadership had to step in and say wait [18:14] we need to go in a different direction. [18:16] Maybe Rohan you can start. Yeah, we um [18:20] so maybe this is controversial but like [18:23] um at Nooks every every purity is [18:25] handwritten. Um we don't uh we don't [18:28] generate that with AI um we use we use [18:32] AI to provide the synthesis or sorry to [18:35] gather the inputs to make the decisions [18:38] that go into that PRD. So if you think [18:40] about every you know every PRD or [18:42] strategy doc that you want to write [18:44] there's some level of inputs typically [18:46] for a lot of products that that inputs [18:48] is data and then it's also um like [18:50] customer conversations and so what we [18:52] use AI to do is to query that to [18:55] understand it say hey we spoke to [18:57] someone what was that pull out the [18:59] quotes that are relevant um to uh find [19:03] the you know um cut it in different ways [19:05] do that data analysis but ultimately we [19:08] actually write all of our docs and [19:10] they're not long. I mean, they're [19:11] usually like maybe a couple pages. Um, [19:14] but we tried to write that by hand. Um, [19:16] what we saw when is when you do that [19:19] that that writing process itself forces [19:22] clarity of thought. It forces you to [19:24] think how are you actually approaching [19:26] the problem? What are the key decisions [19:27] you want to be making? Um, and it [19:30] doesn't again it doesn't take that long. [19:31] And these docs are like you know really [19:33] important like they they are deciding [19:35] your strategy and how you're going to [19:36] approach and whether this you know makes [19:38] sense whether this bet works or not. And [19:41] so um you know that's something that I [19:43] think early on we even found like I [19:45] think our product team was pretty good [19:46] at this but like our go to market team [19:49] was like really bad where like they were [19:50] just like creating this slop all over [19:52] the place and we kind of had to be like [19:53] guys like you know this is this is how [19:55] we do it. We don't make decision. We [19:56] don't let AI make the decisions. We let [19:58] AI provide inputs and help us search the [20:01] decision space, but ultimately we're [20:03] going to be the ones writing it. Um, and [20:05] so that's kind of how we've learned to [20:06] operate. And I think it's it's made our [20:08] pods better because we can just have [20:10] more inputs and have more data, but but [20:12] it's still still handwritten. [20:14] >> How about you, FA? [20:16] >> Yeah, sure. Um, so as I mentioned [20:18] earlier, uh, we, you know, implemented [20:21] an intelligent search uh, platform. Now, [20:24] when we ran the PC, you know, the [20:26] results looked clean um until someone [20:29] searched for the term interest rates and [20:32] the results were well written, [20:34] confidently returned, but mostly [20:36] irrelevant. So, it threw up a lot of [20:38] articles that referenced interest. Some [20:41] of them referenced rates, but very few [20:43] actually referenced interest rates, [20:45] which is what the user really wanted. Um [20:47] and you know for us what that uncovered [20:51] you know as I mentioned is that [20:53] relevance is not a pol is a policy and [20:56] is not a property. So we saw this [20:59] happening in search right but this [21:01] pattern I think is something that shows [21:03] up anywhere where AI is either ranking [21:06] prioritizing or synthesizing whether [21:09] that's in recency versus relevancy [21:12] breath versus authority or keyword [21:14] overlap versus user intent. In this case [21:17] the AI wasn't broken. It was actually [21:20] doing what it was optimized to do. [21:22] However, right, the model is executing [21:25] on a waiting scheme that someone [21:27] explicitly or implicitly made a decision [21:30] decision on. And so for us, what we had [21:34] to do is, you know, we went back to the [21:36] users to understand what the definition [21:38] of relevance meant for them in the [21:40] context of their work. And we focused on [21:43] tuning the ranking strategy accordingly. [21:46] So it wasn't about building a better [21:48] model, but it was about building [21:50] building a better definition of success. [21:54] >> Awesome. Um, my next question is, have [21:58] your teams used AI to prototype concepts [22:02] before handing them off to design and [22:04] engineering? How was this change the [22:07] conversation during your planning? Maybe [22:10] Natalie. [22:13] Yeah, we have definitely experimented [22:15] with prototyping or product managers [22:18] prototyping especially in that initial [22:19] phase. We've always found that just [22:22] showing a PRD if a product manager comes [22:24] up with an an idea and shares the [22:28] confluence page or just their ideas or a [22:31] presentation about what they want to [22:33] achieve. [22:35] The conversation almost always from the [22:38] engineers or you know well how is this [22:41] going to work? How is that going to [22:42] work? Where does that go to? And it's [22:44] questions that they haven't really [22:45] thought about yet because they're [22:47] thinking about the product theoretically [22:50] when you start building it and you start [22:53] understanding how everything works [22:55] together. I think it makes a huge [22:56] difference. So we have the more complex [22:59] projects that we have. We do encourage [23:02] our team to build a prototype so the [23:05] engineers can understand exactly what [23:07] they're um what they're working on. And [23:10] the questions later on are are much less [23:13] it's more about you know okay maybe we [23:16] should use this instead of that or maybe [23:18] they they the engineers actually come up [23:20] with better solutions for us if there's [23:23] a problem that they encounter with a [23:24] prototype. Also for the product [23:26] designers, it actually gets them, you [23:29] know, it saves them time as well because [23:32] they get to about maybe 75% of the way [23:34] there and then the product designers can [23:36] then add their styling and brand [23:39] guidelines and so on and and it makes [23:42] the process a lot quicker for us. So we [23:44] do experiment a lot with prototype and [23:46] when it's a more complex project [23:48] sometimes when it's a smaller project [23:49] too we do um go ahead and create those [23:53] prototype and send it straight to [23:54] engineer with the code and that has [23:56] worked for us as well but it's [23:58] definitely something we're working on. [23:59] >> Awesome. How about you R? [24:02] >> Very much what Natalie said right so for [24:04] us um actually we measure these stuff [24:07] too. So our AI productivity and product [24:09] management actually last u we measured [24:10] was has improved by 30% um by no margins [24:14] that's enough right but we we continue [24:16] on to like push hard um what we are able [24:19] to do is actually change the [24:21] conversation that's happening our sprint [24:23] planning right when you can prototype it [24:26] you can show uh it's not about now what [24:28] to build it's all about what not to [24:30] build uh there's more clarity uh you can [24:33] now start like having more conversation [24:35] about non-happy paths Right? Like we all [24:37] are very good designing for happy path [24:40] but not in non-happy path. So you can [24:41] start thinking about in sprint planning [24:43] actually [24:45] which are the age cases where this can [24:47] fail which are the tests that we need to [24:49] do all around it. Uh so that's where [24:51] like uh it has helped us more we could [24:54] have better definition of done better [24:56] definition of ready and the conversation [24:58] looks u obviously uh I call what it is [25:01] like MVP minimal viable pizza. So you [25:04] can actually sell a slice of pizza [25:06] rather than trying to like sell the [25:08] base. So I think that becomes more um [25:11] more something which is uh tangible [25:14] actually to have a conversation about [25:16] >> makes sense. Um startup environment [25:19] what's your perspective there Rohan? [25:22] >> Yeah we um so I think like uh our team [25:25] definitely uses prototypes. We do it a [25:27] lot like pre-sales if you want to test [25:28] concepts very early, you know, it's like [25:32] um I mean the cost of creating a [25:34] prototype now and like if you put it in [25:35] a cloud is like quite literally you know [25:38] 10 minutes of work. Um and so uh you [25:42] know I think it allows you to test [25:43] things, allows you to communicate [25:44] internally a lot more effectively. Um [25:47] and then also like when you do have PMs [25:49] just going straight to the product like [25:50] you can just put up a PR for for your [25:52] concept that you want to do. Um and that [25:54] worked that also works pretty well. So [25:56] then also it's then what's nice is it's [25:58] faster than they go execute and actually [26:00] build that. So um we see a lot of that [26:01] as well. [26:02] >> Yeah. [26:03] >> Awesome. All right. This next question [26:06] is specifically for you Chica. With AI [26:10] integrated into nearly every SAS tool a [26:12] PM uses today, are you seeing any AI [26:17] fatigue or over reliance on automated [26:19] outputs at the expense of talking to [26:22] actual customers? [26:25] >> Yeah. So what I would say about that is [26:27] that I think the biggest risk for AI [26:30] well actually it might be an overstretch [26:32] to say the biggest risk but one of the [26:34] biggest risks that I see for AI and [26:37] product uh management is not so much [26:39] that it gives wrong answers but it's [26:42] what it makes us stop doing um you know [26:44] which is sitting with customers um you [26:47] know talking with them and sitting with [26:49] ambiguity right so AI can um process [26:53] 10,000 support tickets, you know, before [26:55] your coffee is cold. And that's real [26:58] value, but it cannot replace the [27:00] conversations you haven't yet had. And [27:03] so, you know, being able to sit with the [27:05] customer to understand their needs, to [27:08] understand, you know, what the real [27:09] problems are is critical, right? And I [27:12] think that, you know, there's a risk [27:14] that, you know, when every question can [27:16] get answered with a prompt, we start to [27:19] confuse answers for insights. And so, [27:22] you know, we really need to state take a [27:23] step back and really think about what [27:26] are the customer insights that we want [27:28] to surface that will actually help us [27:30] make better product decisions that will [27:33] ultimately lead to better measurable [27:36] business value for our customers. So for [27:39] me I think you know like that's one area [27:41] where you know if PMs are relying too [27:43] much on fancy dashboards and answers and [27:46] you know not really generating insights [27:49] um the customer experience quietly [27:51] worsens and we tell ourselves that we're [27:53] getting more productive. So my simple [27:55] rule is AI reads the room but people [27:58] still have to be in it. [28:00] >> Yeah, makes sense. [28:06] >> Cool. Okay, my next question is if AI [28:09] continues to automate user story [28:11] creation, requirement drafting and early [28:14] prototyping, how does the fundamental [28:16] role of the product manager change over [28:19] the next few years? What skills will you [28:22] be looking for in your next strategic [28:24] hire? Maybe we start from the rough. [28:27] >> Yeah. So, I know um for everybody in [28:30] this room and our community, right, [28:32] there's a lot of anxiety with like [28:33] product managers will be needed a lot. [28:35] uh I I I would say we would need more [28:38] product managers actually than not. uh [28:40] what is going to change is the makeup of [28:42] the product managers right like what we [28:44] used to do um maybe 70% of time that [28:48] going to be commodatized right what we [28:51] really need to focus is our product [28:53] managers are actually becoming very [28:56] outcomedriven understanding the judgment [28:58] the business focused right um there's [29:01] two ideas I really look after if anybody [29:04] follows um Boris journey from Claude [29:06] right he has uh like AI adoption ladder [29:10] right and it talks about eventually the [29:12] whole organization is going to become AI [29:15] native right and eventually more and [29:17] more agents are the ones that we'll be [29:19] supervising uh initially it's going to [29:21] be more assisted gated then it's going [29:23] to be supervised anatomy and then uh uh [29:26] supervised autonomy and then eventually [29:29] you would be AI native and then um [29:31] there's this uh interesting book that I [29:34] recently read from Paul Cheeks who is a [29:37] MIT professor who talks about uh [29:40] nobody's going to work here and where um [29:43] every department would be I native uh [29:45] what it tells you like in my next hire I [29:48] really want like people who are very [29:51] good uh has vertical like business skill [29:54] product skill has product mindset be [29:57] okay with autonomous [29:59] um uh autonomous teams which would be [30:03] like half hybrid and half people have a [30:05] good judgment when I have to like work [30:07] with autonomous agents and where I have [30:09] to work work with PE people. Um and then [30:13] lastly, I would just say they need to [30:15] make sure that that they are always [30:18] trying to focus on getting uh the [30:20] business problem done and get the [30:22] business outcome faster and that's where [30:24] the product management is going to move. [30:26] Uh it's going to be more uh business [30:29] focused and able to work in uh with a [30:32] hybrid teams. Yeah, I think there's one [30:34] thing that AI can't replace and that is [30:37] our judgment, our context. There's [30:41] there's so many scenarios where I'm I'm [30:43] working with AI and there's some context [30:45] that I forgot to give it. um whether [30:47] it's you know an example was we recently [30:51] started uh earlier this year actually we [30:53] we added a privacy policy um acceptance [30:57] box to our to our site and we have uh [31:02] claude connected to mix panel so we can [31:04] see all the data so I use it a lot for [31:06] analysis and you know how's traffic [31:08] doing are we down are we and you look at [31:10] it year-over-year and there is [31:13] definitely a discrepancy in the traffic [31:14] from this year to last here and the [31:17] discrepancy is about 15%. [31:19] Now the reason that is is that if you do [31:23] not accept that privacy policy, we can't [31:27] track you a mix panel. So I have to go [31:29] to then Google console to see you know [31:32] traffic. So so those are the types of [31:34] context that AI will never have [31:37] environmental context or maybe we [31:39] actually add in more training in a [31:41] certain area but the AI does not know [31:44] that. So there's a lot of context that [31:47] we have that you know you cannot put [31:49] everything into AI. So I do think that [31:52] there's always going to be a need for um [31:56] a human in the loop at some point [31:59] because it's not going to be perfect. [32:01] It'll get you, you know, [32:04] maybe 70 75% of the way there, but then [32:06] you have to do the rest. [32:08] as I'm hiring for a new product manager [32:11] in the next let's say year or a few [32:14] years, one of the things I do want to [32:16] make sure that person is is curious, [32:18] curious about AI, curious about learning [32:21] and constantly wanting to you know learn [32:23] and innovate and they're comfortable [32:26] with AI. um it's it's hard to ignore it [32:30] and and you know there are some people [32:32] who I think are very skeptical about it [32:34] but I think if you use it in the right [32:36] way if you use it more of as that [32:38] operational lever that helps you you [32:41] know get to the point where you can do [32:44] your best work which is providing that [32:46] judgment I think you know there's always [32:49] going to be need for those types of [32:50] product managers so those are the types [32:52] of people I think I would look for [32:54] curious curiosity and and you know [32:58] comfortable with AI [32:59] >> motivation like back to what Fei was [33:01] talking about this morning in their [33:03] absolutely [33:05] >> how about you Rowan in your startup [33:06] environment and yeah [33:08] >> how you guys are always cur like [33:10] motivated to save money and all cost and [33:13] all of that so especially in the startup [33:15] >> yeah I mean I think so like when we when [33:17] we were like hey like I mentioned like [33:19] you know six months ago like hey we're [33:20] going to just we're going to double um [33:22] the ambition of our product road map um [33:24] you know we basically said hey well we [33:25] need to double twice as many PMs and so [33:28] we are hiring more and more and the [33:30] ratios of PMs to end is actually [33:32] decreasing in the sense that we have [33:33] more PMs for every engineer um because [33:36] every engineer can be more more [33:38] effective more powerful um and and can [33:40] just basically produce more more output [33:43] um and so we're seeing that product is [33:44] becoming the bottleneck um generally [33:47] speaking we've seen kind of a shift of [33:49] like two types of of PMs um so the first [33:52] type is what we call like the builder PM [33:55] these are PM that are getting closer to [33:57] the code, closer to the solution, closer [33:59] to the technology. Um, you know, [34:01] sometimes they come from engineering [34:02] backgrounds that are really strong, uh, [34:05] uh, but really have good really product [34:06] sense. Um, and so these builder PMs, we [34:09] often pair them up with like one or two [34:12] engineers and they go and explore a new [34:15] B, new vertical, a new market. Um uh and [34:19] they can be very very incredibly [34:20] effective uh with a really small team [34:22] and it just because the PM is really [34:24] close to the technology they can kind of [34:26] like they have a really good grasp of [34:28] you know what direction to take take [34:30] that particular um particular um roadmap [34:32] item or or product. Um the other and so [34:35] that's like the builder PM and we're [34:36] seeing more of those and Nooks is hiring [34:38] more of those and um I think like you [34:41] know it behooves everyone to get more [34:42] technical um if you want to go that [34:44] route. Um the other route that we're [34:47] seeing is um kind of more what we think [34:49] we call it the business PM or the GMP [34:52] and uh these are people that are [34:54] actually getting more away from the tech [34:57] core technology and the idea is like hey [34:59] look you trust your engineers your [35:01] empowering your engineers to go come up [35:02] with the solutions kind of like you [35:04] always typically do um but what's really [35:07] important is how do you get and how do [35:09] you make sure and drive a sort of like [35:11] broader business outcome. So now these [35:13] PMs are thinking about staffing. They're [35:15] thinking about, hey, do we have the [35:16] right engineers working on the right [35:17] problems? Um where they're thinking [35:19] about um uh strategy, right? Um are we [35:23] exploring the right bets? Um they're [35:25] thinking about go to market. Do we have [35:27] the right salespeople? Um are the sales [35:28] people enabled? Um are we positioning [35:31] this correctly? And so they tend to have [35:32] a slightly different skill set. They'd [35:34] be better at go to market. Sometimes [35:35] they come from sales backgrounds. Um but [35:37] they obviously they still need good [35:38] product sense. They're really good at [35:40] people um and understanding uh people [35:43] and who's effective and who's not and [35:44] they can kind of think and say, "Hey, [35:46] this engineer would do really really [35:48] effective with this um product and then [35:50] you know maybe move this to this other [35:51] person." And so they're thinking about [35:53] how the overall org is staffed. Um and [35:55] so you have this GM business PM that I [35:58] think is incredibly important. Um and we [36:01] also are hiring more people like that. [36:03] And so I think you have those kind of [36:04] two sort of like types of PMs are kind [36:07] of being pushed in either direction and [36:08] people in the middle are going to get [36:09] squeezed because like the reality is [36:12] that stuff is like AI is like you know [36:14] drafting the you know a really simple [36:17] requirement is is like not the hard [36:18] part. The hard part is figuring out like [36:20] what bets you want him to take. Um what [36:22] are the key questions to be answering? [36:24] Those are the hard parts now. So um [36:26] that's how we've seen you know like I [36:28] said we're hiring even more than ever uh [36:30] on product. uh it's becoming the bigger [36:32] bottleneck. We have a thesis at Nooks [36:34] that [36:36] people I mean we we sell to other [36:37] sellers and so like you know we believe [36:39] that like you know there's this core [36:42] concept of selling and anything that [36:43] involves people is very fundamentally [36:44] human and you know AI can't replace that [36:47] human relationship and I think that also [36:49] applies to applies to sales people but [36:51] it also applies to product a lot of you [36:52] know if you think of running a really [36:54] good discovery you know interview with a [36:57] customer I mean picking up on the social [37:00] cues picking up on you know when they're [37:02] getting defensive about things. That's [37:04] something that AI is very very far from [37:06] and I think people that can build really [37:08] good relationships with customers and [37:10] glean out that insights are going to [37:11] always have a need for for um for a [37:13] product. So yeah, [37:14] >> so let me repeat the question. So the [37:16] question is what are what are the [37:17] metrics we use basically to measure [37:19] success or maybe ROI on use AI geni [37:23] generally um whoever wants to go first. [37:27] Rob I can I I can quickly add and uh [37:29] others can add too. Um we look into like [37:32] two or three things for AI productivity. [37:34] one is time to market or time to uh uh [37:37] outcomes right I think that's given [37:39] other is how fast we can do it uh [37:42] without causing rework right that's [37:44] important right like fast doesn't mean [37:46] that quality is there so that's [37:48] important third we are actually looking [37:50] into how much we are converting uh if we [37:54] are creating like reusable agents how [37:57] much are we converting into digital [37:58] workers and equivalent digital workers [38:01] and are they bringing productivity into [38:03] uh do that's like the third one. Third [38:05] one and fourth one is more about [38:08] compliance is the other thing in our [38:10] world that we look into. [38:11] >> Yeah, for us it's um typically time to [38:13] market is what we look at. Um we do a [38:17] lot of AB testing as well with those [38:18] ideas especially when it's when we're [38:20] using AI a lot. We AB test a lot of [38:22] things before we actually fully roll it [38:25] out. So we look at tight to market but [38:28] then also we look at does it add value [38:31] to the end customer like is it is it a [38:33] is it a sure bet is it a good bet I [38:36] guess. So a lot of times we actually try [38:38] to make sure that it's an experiment [38:40] first unless it's it's something that we [38:43] decided we were going to do a long time [38:44] ago and if it does that experiment does [38:47] win then we figure you know it was [38:49] successful but token usage is not [38:52] necessarily our our focus at all right [38:54] now. We're just trying to figure out how [38:56] can we um operate faster, get things to [39:00] value to our customers faster. [39:02] >> So the question is what's the impact of [39:05] AI in team structures? Who wants to [39:06] chime in? [39:09] >> Go ahead. I might some people [39:11] >> I I can I can share some of the you know [39:13] um perspective from a like we're like a [39:15] 300 person company. Um so we I mean we [39:19] generally have different pod structures [39:21] for different types of products. So if [39:23] you think of like a very like 0ero to [39:25] one product bet that we're taking I [39:27] think I mentioned this earlier it can be [39:28] like a PM and two engineers for example [39:31] like and they you know and the idea is [39:34] like you run experiments you move [39:35] quickly um you find design partners and [39:38] it either works or you kill it in like a [39:39] couple months. Um and you have to be [39:41] very willing to kill if you're going to [39:42] make like risky experimental bets. um [39:46] for broader like more traditional like [39:48] platform projects uh products um I think [39:52] like for us it hasn't changed [39:53] significantly in some ways like you know [39:57] um but like the ratios are just bigger [39:58] right so you have like one PM and a [40:00] designer for like you know 7 to 10 and [40:03] then sometimes even if it's like very [40:05] platformy and we think our engine [40:06] leaders are like really strong at doing [40:08] that we'll even have like we'll even [40:09] remove the product manager from there [40:10] we'll just effectively treat the like [40:12] you know EM as like the product leader [40:14] uh if it's like a very more like [40:16] platformy um type like product area. So [40:18] we very much pair it on like the type of [40:21] um uh uh type of product, but I think [40:24] before we just it wasn't even possible [40:26] to do these really small pods like and [40:28] so we've just seen our ability to run [40:30] these like very tight and small pods um [40:32] uh thanks to AI both on the engineering [40:34] and the product side. [40:36] >> How do you manage multiple pipelines for [40:38] them? Yeah. Like the are you talking [40:40] about like rollouts or what what part of [40:42] the project pipeline? [40:45] >> Yeah. I mean generally the product [40:47] owners like manage them, right? Right. [40:49] And so they're responsible for like [40:50] everything from like sourcing the [40:52] customers and design partners to running [40:54] them to uh there's typically a gate like [40:57] if you're running more experimental ones [40:58] of like before it goes to like GA and [41:01] then that's typically like where you [41:03] know they'll write up a proposal and [41:04] saying hey we ran this with like these [41:06] kind of customers and then we'll propose [41:08] but it does make it more hectic like cuz [41:09] like basically you have like you know [41:11] like three four times as many like [41:13] initiatives running at once. Um, but I [41:16] think with AI you can actually like as a [41:17] leader for example like it's actually [41:19] much easier to get in touch like you [41:20] know I have Claude looking through all [41:21] the Slack channels keeping me up to date [41:23] summarizing things. Um, so it's it's [41:26] easier from a more leadership [41:27] standpoint. [41:28] >> Thanks everyone for joining us. Uh, we [41:30] really appreciate it.