Phrase, Toptal Execs Warn AI Vibe Coding Fails Production Standards

Phrase与Toptal高管警告：AI“氛围编码”难达生产标准 浅层AI套壳应用和临时原型无法打造具备护城河的产品，除非团队将路线图建立在客观行为数据之上，并将自主智能体整合到人机协同工作流中。 一个由资深产品管理者组成的专家小组探讨了AI如何取代凭直觉的决策，转而促进客观的客户探索、持续的反馈整合以及快速的概念验证原型构建。演讲者剖析了现代产品经理所需拓展的技能体系，包括AI/ML素养、管理模型漂移，以及将探索性的“氛围编码”（vibe coding）与企业级生产软件区分开来。他们告诫不要构建同质化的套壳应用，并强调可防御的价值来自于深度的人机协同工作流与新兴的多智能体架构。会议最后分享了切实可行的策略，包括利用深度研究工具、优化内部产品运营（Product Ops），以及通过加快上市时间来衡量切实的投资回报率。 AI 推动产品管理从传统直觉转向基于客观数据的“痛点头脑风暴”，要求产品经理在商业、技术与分析三大传统支柱外，将模型漂移、算法可解释性与偏差识别等 AI/ML 素养作为第四核心支柱。 (3:33) 利用生成式 AI 自动化转录通话并对海量反馈进行聚类，帮助团队打破销售偏见与企业大客户的噪音干扰，挖掘出此前被忽略的高增长新兴用户群体。 (7:14) 专家指出企业落地 AI 的三大核心瓶颈：未经清洗的脏数据、认为“撒点 AI 仙女棒”就能解决一切的技术幻想，以及将几分钟做出的“vibe coding”原型误当作可靠可扩展的生产级代码。 (20:27) 易被底层基础模型快速商品化的浅层 AI 包装（wrapper apps）缺乏护城河，真正的防御性价值来自深度人机协同（human-in-the-loop）的业务流整合与周期压缩交付。 (26:10)

Product leaders argue that artificial intelligence is transforming product management from gut-feel decision-making into data-grounded discovery, requiring managers to balance rapid prototyping with domain oversight, technical fluency, and human-in-the-loop workflows. Technical Literacy and the Prototyping Gap (3:33) Devendra Singh Parmar observes that AI literacy—specifically managing model drift, training data bias, and algorithmic explainability—has joined business, engineering, and analytics as a mandatory fourth skill pillar for product managers. Susana Selles Valls notes that tools like Cursor enable "vibe coding," allowing managers to build immediate functional proofs of concept rather than static wireframes to validate ideas directly after customer conversations. However, Selles Valls warns against confusing fast prototypes with production-ready software. Quick demos built in an afternoon lack the security, error handling, latency controls, and compliance required for scalable production. While grassroots experimentation helps teams reach initial insights, sustainable product development requires strict architectural simplicity and data readiness, as models inevitably amplify underlying data quality flaws. Grounding Product Discovery in Objective Data (7:14) Semih Altinay and Paul Timmermann emphasize that automated conversational intelligence eliminates internal organizational bias, which historically rewarded the loudest sales representatives or largest legacy clients. At Toptal, integrating Google Gemini across a "feedback river" of support tickets and Net Promoter Score comments enabled continuous synthesis of unstructured input, uncovering nuanced relationship dynamics between enterprise clients and freelancers rather than trivial interface complaints. Objective behavioral data also reveals unexpected user segments. When Phrase launched an upstream content creation product in June, legacy localization managers vocally demanded complex governance controls. Behavioral analysis revealed that non-technical personnel in HR and marketing represented a major underserved market eager to bypass translation bottlenecks. By analyzing high-performing accounts across industries, the team established a data-driven "North Star" profile instead of designing solely around legacy requests. Domain Judgment in High-Stakes Environments (16:27) Because machine learning models rely strictly on historical inputs, they exhibit systemic blind spots regarding rapid macroeconomic shifts, evolving consumer sentiment, and geopolitical changes. Altinay cautions that when models lack internal product documentation, they generate incomplete conclusions; product managers must step in with domain expertise rather than forcing prompts to confirm pre-existing assumptions. Parmar notes that this interpretive responsibility is acute in heavily regulated sectors like banking and healthcare, where unverified AI outputs carry severe risk. Rather than deploying untracked models, enterprises must combine automated outputs with strategic human oversight and utilize structured environments, such as regulatory sandboxes established in North Carolina, to safely validate experimental systems before production deployment. Defensible Workflows and Agentic Execution (29:39) Timmermann warns that basic "wrapper apps" that merely automate simple tasks will be quickly commoditized by advancements in foundational models. Long-term competitive moats come from integrating AI into internal processes to augment human teams, balancing automation with intentional human-in-the-loop review points. Looking ahead, Altinay points to autonomous multi-agent systems as the next paradigm shift. In Phrase's agentic content workflow, specialized agents sequentially generate regional copy, adapt channel formatting, and actively remediate errors rather than merely flagging them. Evaluating the business impact of these systems, Altinay highlights cycle-time compression as the clearest measure of return on investment, noting an enterprise medical device client where deployment delays cost $1,000,000 per day. The panel concludes that while autonomous agents and rapid prototyping accelerate innovation, product organizations must institute clear governance frameworks to avoid accumulating unvetted proofs of concept, ensuring AI investments solve core user problems rather than chasing novel infrastructure.

来自Discover、Phrase、Suffolk与Toptal的产品领导者指出，人工智能正在将产品管理从依赖直觉和流程协调的传统模式，重塑为融合技术理解、客观数据洞察与第一性原理的工程学科。产品经理的核心职责正向技术落地、模型治理及多智能体工作流协同演进，但在跨越从原型到生产级交付的鸿沟时，团队仍须恪守业务本质与人机协同。 核心能力重塑与原型验证变革 (3:33) Susana Selles Valls与Devendra Singh Parmar提出，AI时代的产品经理正大幅超越需求收集与路线图规划等传统行政协调职责，向深度技术执行推进。在以往商业敏锐度、技术素养与分析能力三项传统支柱之上，“AI与机器学习素养”已成为强制性的第四支柱。产品经理必须深入理解模型漂移（model drift）、算法可解释性（algorithmic explainability）以及训练数据偏见，主动监测模型表现随市场与用户行为变化而出现的衰减，并能向用户清晰解释系统判断逻辑。 技术门槛的显著降低也彻底颠覆了产品发现与验证的节奏。以往产品经理在访谈用户后需要等待工程团队的排期来搭建线框图或测试界面；如今借助新型AI工具，产品经理能够在访谈后迅速自行搭建出具备实际功能的概念验证（POC）。这种在向软件工程师移交需求前完成自验的能力，大幅缩短了从构想到验证的反馈周期，推动团队能够更低成本地快速试错与迭代。 客观数据分析与需求优先级去噪 (7:14) 在需求发现和优先级排序上，产品团队长期受困于销售人员或客户成功经理等内部代理人带来的主观偏见，容易将单一甚至情绪化的诉求误判为全局痛点。Semih Altinay指出，Phrase通过引入AI对客户沟通进行全量转录与对话分析，有效剥离了组织内部的情感噪音；Paul Timmermann亦介绍，Toptal利用Google Gemini对其整合了NPS评论、客户工单和通话记录的Slack“反馈之河”进行实时聚合提炼，填补了以往受限于分析人力而错失的宏观洞察，进而发掘出自由职业者与企业客户之间关于信任机制的深层动态。 客观行为数据同时改变了产品路线图的决断逻辑。Semih举例指出，在Phrase的产品迭代中，尽管有九成传统本地化经理客户强烈要求保留或复刻繁琐的传统系统控制功能，但数据揭示出市场和人力资源等新兴跨职能用户迫切希望绕过传统流程瓶颈。产品团队因此得以跳出大客户主观声音的绑架，基于高价值采用模式构建功能。然而专家们同样提醒，AI模型因缺乏最新上下文容易出现判断漏洞，产品经理必须结合宏观商业背景与领域直觉加以干预校准，切忌为了迎合既定偏见而强行诱导模型。 警惕原型陷阱与克服生产级落地瓶颈 (20:27) Susana Selles Valls总结了AI产品落地过程中普遍存在的三大现实障碍。首先是数据就绪度缺陷，底层数据的脏乱不仅会破坏用户对系统的信任，还会直接放大智能体的执行偏差；其次是利益相关者普遍抱有“撒上AI魔法粉末”就能化解一切产品问题的虚高预期，忽视了简单架构在维护性上的优势；第三则是轻度原型开发（vibe coding）带来的认知错位，用户误以为利用ChatGPT或Claude在短时间内生成的Demo即代表了成熟软件，因而对正式系统长达数个季度的交付周期产生质疑。 Susana明确强调，快速原型制作绝不等于交付可靠、安全的生产级软件。处理未建模的边界情况、保障数据安全、优化延迟以及维持系统稳定性，依然是工程转化中最艰难的环节。Devendra补充强调，在医疗、银行与保险等受严格监管的行业中，模型对突发地缘政治事件、制裁政策及宏观监管调整存在天然盲区；企业必须通过行业监管沙盒等安全隔离环境进行受控测试，依托产品经理的宏观战略解读来规避系统性合规风险。 目标导向智能体与人机协同壁垒 (29:39) 展望技术前沿，Semih Altinay认为产品研发的重心正在转向目标导向的自主AI智能体。以Phrase在6月推出的智能内容管理系统为例，用户只需输入高阶目标、品牌规范、预算与受众画像，系统即可串联专门的生成、格式转换以及能够自主排查并修复错误的QA智能体。尽管如此，这也带来了企业客户对端到端自治与黑盒不可控之间的设计冲突。Devendra亦指出，随着模型上下文协议（MCP）和智能体通信标准的快速更迭，产品经理必须持续维持架构的灵活适配度。 面对生成式AI同质化的风险，Paul Timmermann告诫产品团队切忌陷入过度工程化，避免耗费重金自研基础模型或开发缺乏防御力的浅层套壳（wrapper）应用。长效的竞争壁垒必须建立在将AI与业务深度绑定的工作流中，通过设计高信任度的人机协同（Human-in-the-loop / Human-on-the-loop）机制将员工转化为各岗位的超级个体。而在ROI的衡量维度上，除客户侧的实际提效外，Semih指出内部将复杂的研发上市周期从两年大幅压缩至三到六个月，在医疗器械等每日延误损失高达百万美元的场景中，正是最直观且具防御性的商业回报。 研讨会揭示出AI时代产品管理的核心趋势：脱离追逐模型噱头的浅层创新，回归以解决用户本质痛点为中心的第一性原理。尽管多智能体协同与快速原型生成极大地提升了研发探索效率，但如何解决底层数据质量短板、在智能体黑盒决策中保留有效的人工介入机制，并跨越从即兴原型到企业级高可靠性交付的鸿沟，仍是当前产品组织亟待解决的关键课题。

## Background 该内容由 Ai4 于 2025 年 8 月 27 日发布在 YouTube 平台上。视频记录了一场关于“AI 时代产品经理如何驱动创新与优化产品开发”的专题圆桌讨论，汇聚了来自 Amazon、Discover、Suffolk、Phrase 和 Toptal 的多位资深产品领袖，深入探讨 AI 在客户需求洞察、功能优先级排序、敏捷原型构建及业务目标对齐等维度的深度应用与落地挑战。 ## TL;DR 讨论指出 AI 正在从根本上重塑产品经理（PM）的工作范式，倒逼 PM 掌握模型可解释性等 AI 专业素养，并通过客观的数据流剥离组织内部的主观偏见与噪音。嘉宾们强调“氛围编码”（Vibe Coding）与 AI 原型工具极大加速了快速试错与方向修正，但从 PoC 走向生产级交付仍需严谨的数据治理与预期管理。面对未来向目标导向演进的智能体（AI Agent）系统，产品团队需在自主执行与客户控制权之间取得平衡，并将 AI 的核心价值聚焦于解决真实业务痛点与大幅压缩研发周期上。 ## Atmosphere & style 整场对话在轻松幽默的破冰互动中拉开序幕，随后迅速切入高信息密度的专业探讨，兼具前瞻视野与务实底色。多位来自不同领域的资深产品总监各抒己见、节奏紧凑，既有对多智能体等前沿技术潜力的兴奋展望，也有对盲目堆砌技术、忽视第一性原理等落地误区的冷静反思。嘉宾们结合大量一线踩坑案例娓娓道来，整体氛围专业干练、直击痛点，给听众带来极具启发性与实战指导意义的沉浸式体验。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=4s) AI 时代对产品经理（PM）提出了保持“纪律性”（Discipline）的要求，在海量工具和信息过载面前必须坚持第一性原理，专注于解决实际客户痛点而非盲目跟风。 - [03:33](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=213s) PM 可利用 AI 工具大幅降低技术门槛，在向工程团队排期前快速构建可运行的概念验证原型（PoC），从而重塑产品验证与跨团队协同流程。 - [05:22](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=322s) 除了传统的业务、技术与分析能力，PM 必须建立第四大支柱——AI/ML 专业素养，重点应对模型可解释性、固有偏差消除以及模型漂移等核心挑战。 - [07:20](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=440s) 通过 AI 对客户全渠道通话及使用数据进行客观转录与自然语言分析，能够剥离内部人员转述的主观情绪偏见，实现周级别的快速纠偏与敏捷迭代。 - [13:04](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=784s) 需求优先级排序常被“嗓门大”的内部角色或传统核心用户误导，PM 需依托全量行为数据看清真正高价值的用户诉求与未来趋势，而非盲从局部群体意见。 - [19:37](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1177s) 在金融、医疗等强监管领域，AI 极易遗漏瞬息万变的宏观趋势、地缘政治与监管变化，绝不能盲从模型输出，必须结合 PM 的商业直觉与严格的人工审计。 - [20:27](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1227s) 企业落地 AI 面临三大严峻现实：数据清洗治理成本高昂、终端用户对 AI 抱有不切实际的“魔法”预期，以及原型与生产级高可用/安全性之间存在巨大鸿沟。 - [25:11](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1511s) “氛围编码”（Vibe Coding）的核心价值在于以极低成本让 PM 快速验证想法、敏捷试错与转向，但绝不能等同于交付可靠且安全的生产级产品。 - [25:58](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1558s) 薄封装的“套壳应用”易被同质化淘汰，企业的核心壁垒在于将人机协同（Human-in-the-loop）与独特的组织工作流深度融合，打造员工的“超级版本”。 - [29:37](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1777s) AI 正从单点任务执行向目标导向的自主智能体（Agent）演进，但实际落地中产品经理必须在“智能体完全自主”与“企业级客户对控制权和确定性的刚性诉求”之间取得平衡。 - [30:53](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1853s) 多智能体内容管理系统正从单纯抛出“QA 异常红旗”迈向“端到端自动修复错误”，直接交付符合企业品牌调性与受众约束的最终成果。 - [32:48](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1968s) PM 应当通过深度调研智能体（Deep Research Agents）及将 AI 融入产品运营（Product Operations）等方式躬身实践，以解决自身真实问题来探索最佳交互模式。 - [38:54](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2334s) 企业从海量 POC 走向规模化投产需建立完善的治理体系（Governance），并在监管沙盒等受控环境中贯彻“快速测试、快速失败、快速转向”的敏捷机制。 - [41:00](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2460s) 衡量 AI 投资回报率（ROI）不仅看外部商业价值赋能，更体现在合规保障下将原本两年的研发交付周期压缩至三到六个月，从而大幅削减每日数十万至百万美元的产品延期损失。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=4s) 开场破冰与嘉宾背景介绍 圆桌讨论在轻松幽默的氛围中拉开序幕。主持人 Abhai Pratap Singh（来自 Amazon Alexa 的高级技术产品经理）首先向现场观众互动提问：“场内有多少产品经理（PM）？请欢呼一下！”紧接着他又询问有多少怀揣抱负的准 PM，甚至半开玩笑地问道：“场内有 AI Agent（智能体）吗？”在确认全场都是“纯人类”后，他引出了当天的主题——AI 如何重塑产品管理，并邀请四位来自不同行业的资深产品领袖进行自我介绍，分享 AI 给自身工作带来的冲击。 Discover 的首席产品负责人 Devendra Singh Parmar 首先发言。他拥有长达 16 年的银行业产品经验，直言 AI 对产品管理的影响极其深远。在他看来，传统 PM 往往花费大量时间在泛泛的“头脑风暴”上，而现在 AI 正在帮助 PM 更加精准地切入客户痛点（Pain Brainstorming），以更聪明、更高效的方式代入客户视角解决实际问题。随后，语言技术公司 Phrase 的 AI 解决方案副总裁 Semih Altinay 指出，AI 正在彻底改变 PM 洞察客户的方式——过去 PM 很大程度上依赖直觉，而如今 PM 必须依靠真实客观的数据来构建叙事，并针对企业内不同利益相关者的诉求定制传递信息。 紧接着，Suffolk 的职能 AI 总监 Susana Selles Valls 介绍了自己的背景。她近期加入 Suffolk，此前曾就职于 Moderna 和 Amazon，长期处于产品战略与前沿 AI 的交汇点。Susana 专注于利用 AI 推动企业内部职能部门（如财务、法务、市场营销、业务拓展等）的数字化与工作流变革。最后，全球自由职业者人才平台 Toptal 的产品副总裁 Paul Timmermann 分享了他的感悟。Paul 表示，Toptal 一直在全公司及人才网络中探索 AI 的多样化应用；面对如今打开 LinkedIn 就能看到海量 AI 资讯的信息过载环境，AI 时代带给产品经理最重要的考验反而是“纪律性”（Discipline）——即如何在工具井喷的浪潮中坚守第一性原理（First Principles），不迷失产品经理的核心价值。 - **Devendra Singh Parmar**：Discover 首席产品负责人，拥有 16 年银行业产品经验，强调 AI 赋能客户痛点挖掘。 - **Semih Altinay**：Phrase AI 解决方案副总裁，聚焦数据驱动叙事与客户真实需求洞察。 - **Susana Selles Valls**：Suffolk 职能 AI 总监（前 Moderna、Amazon），主导企业中后台职能部门的 AI 转型。 - **Paul Timmermann**：Toptal 产品副总裁，倡导在技术爆炸时代坚持第一性原理与产品纪律。 --- ### [03:33](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=213s) AI 时代产品经理角色的演进与技术赋能 主持人 Abhai 将话题引向行业核心关切：“AI 绝非仅仅是 PM 工具箱里的又一个新工具，它正在从根本上重塑我们的工作方式。”他向 Susana 提问，作为 Suffolk 的 AI 领头人，她如何看待 PM 职责的深层演进。 Susana 分析道，传统意义上的产品经理职责——收集用户需求、充当客户代言人、把控项目排期、制定产品路线图（Roadmap）——其核心本质并不会消失，反而会在 AI 工具的加持下大幅提升执行效率。然而，AI 带来的关键转折在于它将 PM 推出了以往固化的业务框架，进入了更深层次的“技术腹地”。现存的新一代 AI 工具大幅降低了技术门槛，使得 PM 能够更自如地理解底层模型能力与系统集成架构。 这种技术能力的提升直接重塑了产品验证流程。Susana 分享了自己的亲身实践：过去 PM 在与客户交流产生新想法后，通常需要撰写繁冗的需求文档并流转给工程团队；而现在，PM 可以在找工程师排期之前，利用 AI 快速搭建出可运行的概念验证原型（PoC, Proof of Concept）。这不仅让 PM 在技术可行性上胸有成竹，还能以直观的原型为媒介，与开发人员以及客户展开更精准、高维度的协同共创。 ```mermaid flowchart TD accTitle: 传统与 AI 时代 PM 原型验证流程对比 subgraph Traditional["传统 PM 协作链路"] A1["收集用户需求"] --> B1["编写 PRD 文档"] B1 --> C1["排期等待研发"] C1 --> D1["工程团队开发 Demo"] end subgraph AI_Era["AI 赋能下的现代 PM 链路"] A2["捕获用户痛点与创意"] --> B2["PM 利用 AI 快速生成 PoC"] B2 --> C2["与客户直接交互验证"] B2 --> D2["与研发精准技术对接"] end ``` --- ### [05:22](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=322s) AI 时代 PM 核心技能重塑与模型可解释性挑战 在探讨技能演变时，主持人询问 Devendra：在当前浪潮下，究竟哪些技能属于新兴必备，哪些传统能力正在面临淘汰？ Devendra 总结道，传统产品经理通常依赖三大核心支柱能力：业务能力（Business）、技术能力（Technology）与分析能力（Analytical）。然而，随着 AI 深度融入产品创新、开发流程甚至成为产品本身的核心组件，行业正倒逼 PM 必须掌握第四项新型核心能力——AI 与机器学习（Machine Learning）专业素养。 Devendra 特别强调了“AI 作为产品”时 PM 所面临的全新挑战： 1. **模型可解释性与偏差消除（Explainability & Bias）**：PM 必须深刻理解算法处理数据的底层逻辑，能够向客户和业务方清晰解释 AI 决策过程，并敏锐排查模型是否存在固有偏差。 2. **应对模型漂移（Model Drift）**：AI 模型的训练基于历史数据，但用户的实际偏好与市场动态是时刻变化的。PM 必须监控模型的时效性，及时发现模型漂移问题。 3. **跨职能专业对话**：PM 必须掌握 AI/ML 的专业术语与逻辑体系，从而能够与数据科学家、高级分析团队实现无缝且平等的深度沟通。 针对这一论点，Abhai 幽默地补刀总结：“正如大家所知，模型总有知识截止日期（Cut-off Date），但人类没有，优秀的 PM 恰恰擅长不断进化。” --- ### [07:20](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=440s) 客户洞察的范式转移：打破内部主观偏见与敏捷迭代 Abhai 随后引出了亚马逊经典的产品方法论——“从客户出发，逆向工作”（Working Backwards），并向 Semih 请教 Phrase 在捕捉客户真实诉求方面经历了怎样的进化。 Semih 坦言，过去公司高度依赖组织内部人员转述的“二手洞察”。然而，内部视角往往充满主观偏见——销售人员或客户成功经理（CSM）在反馈产品缺陷或功能抱怨时，常常掺杂个人情绪或片面认知，导致产品团队无法看清真实全貌。作为一家全方位追踪数据的语言技术公司，Phrase 开始利用 AI 赋能客户倾听体系：通过对客户通话进行完整录音、自动语音转录，并利用 AI 生成深度通话分析。 通过这种方式，团队震惊地发现了显著的认知分歧：内部人员“自以为是的客户诉求”与“客户在对话中实际表达的痛点”经常大相径庭。Semih 直言：“AI 的好处在于它没有人类的情感包袱，它是完全客观的。”团队将 AI 提取的真实客户原声与系统使用行为数据（Usage Data）结合，呈现在直观的仪表盘上，彻底消除了组织内部的臆断与偏见。 在谈到这一转变对产品路线图（Roadmap）的影响时，Semih 指出最根本的质变在于“掉头转向的速度”（Pivot Faster）。以往产品团队可能需要耗费数个季度甚至半年时间去推进一个规划，最后才发现南辕北辙，白白耗费巨额资金、时间与精力；而在 AI 领域，竞争对手迭代极快，慢一步就意味着落后出局。现在，客观精准的客户洞察让产品团队身轻如燕，能够根据即时反馈迅速纠偏，给客户提供他们真正需要的产品。 Paul 也对此深表共鸣，并介绍了 Toptal 内部运行已久的机制——“反馈之河”（Feedback River）。Toptal 将全渠道的客户反馈实时汇总至专用的 Slack 频道中，涵盖 NPS（净推荐值）评分反馈、客服通话数据等多维信息流，帮助产品团队时刻把脉全球客户的真实情绪与动态。 - **客观性与去偏见**：AI 转录与自然语言分析剥离了内部转述者的情绪滤镜，直击第一手客户真实声音。 - **敏捷迭代与止损**：摆脱长达数个季度的盲目交付周期，依托实时数据看板实现周级别的路线图灵活调整。 - **全渠道原声汇流**：如 Toptal 的“反馈之河”，将多触点客户交互数据集成至统一协作空间，实现实时的客户脉搏监测。 ### [10:14](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=614s) 利用 AI 迭代客户洞察与深度关系挖掘 在处理客户反馈时，产品经理以往往往只能获得模糊的直觉共鸣，很难精准把握真实的宏观趋势。如今，借助 Google Sheets 中的 Gemini 等 AI 工具，产品经理可以直接对收集到的客户反馈进行多维度的实时聚类和快速打标。在以往，这种迭代过程需要反复与数据科学团队沟通协作并耗费数个开发周期；而现在，PM 能够实时调整分类标准并即时生成可视化看板与仪表盘。这种能力不仅加速了日常工作流，也改变了撰写产品需求文档（PRD）和战略宣讲的方式——虽然最终方案仍需人工把关，但 AI 可以在时间紧迫时快速补足市场调研与用户画像的初稿。 更深层的突破在于洞察维度的升级。以 Toptal 的自由职业者平台为例，传统的反馈分析多停留在浅层功能体验（例如某个按钮是否报错）；而 AI 能够处理复杂庞大的人际互动数据，挖掘客户与人才之间深层关系的成功密码，识别促成长期合作的隐性驱动因素。 ```mermaid flowchart LR accTitle: AI驱动的客户反馈分析工作流 subgraph Input["数据接入"] A["多渠道客户原始反馈"] --> B["电子表格 / 数据仓库"] end subgraph AIProcess["AI 实时洞察"] B --> C["Gemini / LLM 自动摘要与打标"] C --> D["多维度分类与实时迭代"] end subgraph Output["战略交付"] D --> E["可视化看板 / 趋势监控"] D --> F["PRD 洞察与用户画像初稿"] end ``` ### [13:04](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=784s) 消除主观噪音：从“谁嗓门大听谁的”到数据驱动优先级 功能优先级排序一直被称为产品经理最大的“痛点”。在传统的协作模式下，需求优先级往往被“嗓门最大的销售”或是宣称自身客户最重要的客户成功经理（CSM）所主导，抑或过度倾斜给当前体量最大的头部客户，忽略了那些具备巨大成长潜力的小客户。AI 的引入有效清除了组织内部的利益噪音，使团队得以依托全量行为数据看清用户的真实诉求。 以语言技术公司 Phrase 于 6 月推出的新产品为例，传统上公司主要服务于专业的本地化/语言管理者，这部分核心用户倾向于要求增加复杂的控制功能；但通过 AI 对全平台反馈的穿透分析，团队惊喜地发现大量企业内部的营销人员、HR 等非专业人员正涌入新产品。他们无需掌握复杂的翻译管理系统（TMS）即可直接推进内容生成。团队进而通过全量数据构建了涵盖跨平台使用的“北极星用户画像”，并将其最佳实践复制推广到旅游、生命科学等不同垂直行业，借助同侪竞争意识（FOMO）驱动更多客户采用自动化与 AI 工具。 - **去中心化语言处理**：非本地化专业背景的普通员工也能直接使用工具，打破了必须通过语言经理集中处理的传统流程瓶颈。 - **北极星客户复刻**：通过追踪高活跃客户的使用路径，提炼出跨行业通用的 AI 与自动化应用方案，帮助销售团队用真实数据讲好产品故事。 ### [17:02](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1022s) 避免盲信与“强行削足适履”：专家判断与监管边界 虽然 AI 极大地提升了分析效率，但在关键产品决策上依然不能完全放弃行业直觉与人工判断。AI 生成的结果严重依赖于输入数据的完整度与时效性。例如在评估某个新功能是否算得上“AI Agent”时，如果喂给大模型的产品信息不全面，模型可能会直接判定其不符合定义。此时必须依靠产品经理的专业知识补充上下文并重新调优，而不是盲目顺从模型结论。 与此同时，产品团队更要警惕“为了得到预设答案而强行诱导模型”（削足适履）的操纵行为。Devendra Singh Parmar 进一步指出，在医疗保健（如 AI 代写处方、辅助诊断）以及 Discover 所处的银行与保险等强监管行业中，模型并未覆盖所有边界场景与异常数据点。在这些高合规要求的领域，AI 建议必须与严格的人工审计和行业专业判断深度结合，才能确保决策的安全与可靠。 - **直觉与数据的平衡**：AI 无法掌握全部商业背景，专家经验是校验模型盲区、补全最新业务信息的关键防线。 - **防止模型操纵**：严禁通过修改 Prompt 或篡改分析数据去迎合既有的主观偏见。 - **强监管场景约束**：在金融、医疗等容错率极低的领域，模型能力仍在演进中，不可直接将未经人工复核的 AI 结论投入关键流程。 ### [19:37](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1177s) 宏观视角的缺失与产品经理的直觉把控 在银行或保险等强监管行业中，单纯依赖 AI 的输出是极具风险的。AI 模型本质上只能基于喂给它的既有数据进行计算与推理，极易错失更宏观的全局上下文。这些上下文包括快速转变的客户偏好、行业市场趋势的剧烈波动，以及日益频繁的地缘政治冲突与多国监管机构动态实施的制裁政策。模型在训练和构建时无法将这些瞬息万变的外部变量实时完全囊括，因此产品经理（PM）必须时刻戴好把控全局的“帽子”，在审核 AI 工具的输出结果时，融入人类独有的敏锐直觉与全面的综合视角。 ```mermaid flowchart TD accTitle: AI 局限性与 PM 上下文补充机制 subgraph S1["AI 模型输入与局限"] A["历史与既有训练数据"] --> B["AI 分析与模型推理"] end subgraph S2["动态外部上下文 (模型常缺失)"] C1["客户偏好转移"] C2["市场趋势震荡"] C3["地缘冲突与监管制裁"] end subgraph S3["综合决策输出"] B --> D["初步模型输出"] C1 & C2 & C3 -.-> E["PM 商业直觉与全局研判"] D & E --> F["可靠、合规的业务决策"] end ``` --- ### [20:27](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1227s) 落地 AI 的三大现实挑战：数据基建、期望管理与概念验证的鸿沟 在产品生命周期中深度整合 AI 绝非一片坦途或单纯追求效率提升。Susana Selles Valls 坦诚分享了她在行业内反复目睹的三大严峻挑战： 1. **数据准备与清洗**：如果企业底层数据没有为 AI 做好治理与准备，最终产出的结果必然达不到预期。随着人机协同进化到与 AI Agent 并肩协作的阶段，用户对模型输出的信任完全建立在干净可靠的数据基础之上，但很多客户并不理解清洗数据所需付出的巨大代价。 2. **不切实际的“魔法”预期**：许多终端用户误以为只要在产品中“撒上一点 AI”，就能瞬间解决所有问题。产品经理的关键职责在于开展用户教育，引导大家认识到“技术上的极简（Technical Simplicity）往往才是最佳方案”。PM 自身必须先与技术团队和行业前沿深度沟通，理解复杂模型与实际落地之间的权衡，再向终端用户解释为什么选择最适配而非最复杂的方案。 3. **“氛围编码”（Vibe Coding）与生产级扩展的脱节**：如今借助 ChatGPT、Claude 等大模型工具，任何人都可以在几分钟内快速拼装出一个原型，这导致终端用户产生一种错觉——“既然我用 ChatGPT 几分钟就能做出来，为什么你们的路线图还要排到明年？”然而，从原型试运行（Pilots）走向生产环境（Production）是产品周期中最艰难的一环，因为原型完全无法保证模型在未知边缘场景中的可靠性、安全性与系统可扩展性。 > “氛围编码（Vibe coding）并不等于交付一个可靠且安全的产品。” —— Susana Selles Valls --- ### [25:11](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1511s) 氛围编码（Vibe Coding）的真正价值：快速失败与高效转向 针对当下被频繁讨论甚至过度滥用的“氛围编码”（Vibe Coding），拥有软件工程背景的 Susana Selles Valls 明确指出了 PM 应如何正确借力这一趋势。她认为，氛围编码的核心价值绝不是直接替代严谨的工程落地，而是赋能 PM 以极低的成本去**测试想法（Testing ideas）**。 - **快速试错（Fail Faster）**：在投入昂贵的技术研发资源前，PM 能以分钟级的速度验证概念是否可行。 - **高效转向（Pivot Faster）**：在发现某个功能或构想无法成立时立即调整方向，避免在注定失败的方案上浪费数周或数月的时间。 - **提升 PM 自身生产力**：技术背景与 AI 原型工具的结合，让 PM 在探索阶段获得了前所未有的敏捷度与自主性。 --- ### [25:58](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1558s) 回归核心产品思维：避免为技术而技术与赋能人类员工 来自 Toptal 的 Paul Timmermann 回顾了团队在 AI 整合过程中的心路历程。他坦言在技术浪潮刚掀起时，团队很容易被外界的狂热所裹挟，一头扎进构建复杂的自有 AI 基础设施或训练自研模型中，却忽视了“我们究竟在为终端用户解决什么问题”。直到过去六个月中，团队重新回归核心产品原则，把重点拉回到深挖客户痛点、将 AI 视为赋能工具而非技术噱头之后，产品进展反而远超以往。 ```mermaid flowchart LR accTitle: AI 整合理念的转变 subgraph Wrong["初期误区"] W1["盲目跟风狂热"] --> W2["堆砌自研 AI 复杂架构"] W2 --> W3["脱离实际用户痛点"] end subgraph Right["回归正轨"] R1["深入理解客户本质需求"] --> R2["敏捷原型验证与快速试错"] R2 --> R3["AI 深度赋能员工 (Human-in-the-loop)"] end ``` 在谈到“如果能重来一次会做何改变”时，Paul 驳斥了“AI 全面取代各岗位人类员工”的流行叙事。他强调： - **打造超级个体（Superpowering Roles）**：AI 的真正价值在于将各岗位员工转变为“超级版本的自己”，深度重塑工作流程。 - **构筑护城河对抗套壳应用**：市面上的薄封装“套壳应用（Wrapper apps）”很快会被同质化抹平，而将“人机协同（Human-in-the-loop / Human-on-the-loop）”与企业独特的组织工作流深度融合，才是难以复制的核心竞争壁垒。 --- ### [29:37](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1777s) 未来展望：从任务执行到目标导向的 AI Agent 与用户的掌控权诉求 当被问及未来最令产品经理兴奋的 AI 趋势时，来自 Phrase 的 Semih Altinay 毫不犹豫地选择了 **AI Agent（智能体）**。他回顾道，过去（哪怕就在不久之前）AI 工具主要用于单点执行特定的孤立任务；而现在，用户或企业只需向 AI Agent 下达一个宏观的“整体目标（Entire goal）”，Agent 就能自主拆解步骤、规划路径、调用所需工具并执行各项任务，直至最终达成目标。 然而，Semih 也分享了一个在实际落地中遇到的现实反馈：当 Phrase 推出其基于 Agent 的 AI 工作流后，一位核心大客户虽高度赞赏其前瞻愿景，但也明确提出了一个关键需求：**“我们想要更多控制权（We want more control）。”** 这表明在完全自主的 Agent 愿景与企业级客户对确定性、透明度和可控性的刚性诉求之间，仍然存在着需要产品经理去精细平衡的产品设计空间。 ### [30:53](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1853s) 智能体内容系统的落地与“自动修复”进阶 Semih Altinay 探讨了当前 Agentic AI（智能体 AI）发展中的分水岭与权衡。一方面，多智能体协同（Multi-agent systems）能够帮助企业完成前所未有的宏大目标；但另一方面，部分客户仍渴望拥有控制权，不希望 AI 自行决定调用哪些底层工具，而是希望明确指定“使用特定工具完成特定环节”。Phrase 采取的智能体方案是让客户首先创建内容画像并设定约束条件——包括预算、数据限制、地区偏好以及品牌调性（Brand Voice）。系统将这些上下文输入 AI，AI 即可精准达成目标，例如向“阿根廷 35 岁以上女性”这一特定画像受众，以符合品牌声调的本地化语言进行触达。 谈及智能体的具体部署，Semih 澄清了普通 AI 工具与真正智能体的区别，并透露 Phrase 已于 6 月上线了首个落地成果，称之为“智能体内容管理系统”（Agentic Content Management System）。该系统底层由多个协同运作的智能体支撑，致力于将多种 AI 产品组合起来直接交付客户所需的结果。下一阶段的演进方向是让智能体自动进行针对性改造，并能自主发现 QA 质检问题后直接完成修复，而不是仅仅向用户抛出红旗警报，“不要只列出问题，直接帮我修好它”正是客户最迫切的诉求。 ```mermaid flowchart TD accTitle: Agentic CMS Architecture and Workflow subgraph Inputs["1. 客户画像与约束配置"] A["内容画像 & 品牌调性"] B["约束设定 (预算/地域/受众)"] end subgraph AgenticCore["2. 智能体内容管理系统 (Phrase Agentic CMS)"] C["首发智能体 (组合多款 AI 工具产出端到端内容)"] D["转化智能体 (针对特定场景/渠道形态变换)"] E["自动 QA 修复智能体 (发现错误并直接自动修复)"] end subgraph Output["3. 精准交付"] F["高精度本地化内容 (如特定区域细分受众)"] end Inputs --> C C --> D D --> E E --> F ``` - **部署时间节点**：首个多智能体系统于 2025 年 6 月正式投产部署。 - **核心演进目标**：从单纯提示“QA 异常红旗”（Red Flags）转变为端到端“自动化修复错误”。 --- ### [32:48](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1968s) 专家面板建言：产品经理如何躬身入局与拥抱前沿 在讨论产品经理（PM）应当如何开始探索 AI 时，多位嘉宾分享了切实可行的学习路径与思维方式： Devendra Singh Parmar 强调了多模态智能体（Multimodal Agents）的重要性，它们能够跨文本、图像等多种格式融合信息，获取过去产品经理无法触达的数据智能。同时他提醒大家，AI 技术仍处于高速演进且尚未完全成熟的阶段。智能体是当下的核心焦点，但诸如模型上下文协议（MCP，Model Context Protocol）、Agent-to-Agent 交互协议、安全护栏（Guardrails）与开发框架每隔几个月都在快速迭代，PM 必须紧密追踪底层架构演进，因为技术成熟度将直接重塑应用场景。 Semih Altinay 与 Susana Selles Valls 呼吁大家“弄脏双手”（Get your hands dirty），摆脱单一的大语言模型对话框思维。Semih 建议 PM 积极尝试 AI 幻灯片生成器、电子表格深度分析工具以及基于 Cursor 的“氛围编码”（Vibe Coding），在亲身体验优秀交互模式的过程中反哺自身产品的 UX 设计灵感。Susana 则着重推荐了两项利器：一是深度调研智能体（Deep Research Agents），这将彻底革新 PM 的调研效率；二是利用 AI 提出人类未曾设想过的问题，打破思维盲区，让技术激发出真正的创新流动。 Paul Timmermann 从实际落地角度补充，建议将 AI 深度融入产品运营（Product Operations）流程。产品运营通常因日常繁杂事务而被边缘化，但 PM 恰恰可以通过搭建轻量级智能体或利用 Vibe Coding 编写内部小应用来优化这些工作流。这样做不仅让 PM 自身成为产品的终端用户，还能在团队内树立标杆案例，以“为自己和组织解决真实问题”的方式加速掌握 AI 技能。主持人总结道：“产品经理绝没有消亡”，当下最关键的就是躬身实践。 --- ### [36:30](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2190s) 需求优先级的反思：9/10 客户反馈 vs 行业未来方向 在现场问答环节，有观众提问在 AI 时代如何科学地进行功能优先级排序。Semih Altinay 分享了一个关于“用户口头反馈与实际数据相悖”的经典案例。Phrase 团队在全流程中深度追踪用户沟通数据，每次客户访谈均有 AI 助手自动加入、转录并生成分析，甚至能精确量化客户与销售人员各自的发言时长比例及提及的产品特性。 分析显示，十位本地化经理中有九位都在反复要求某项特定功能。然而，当结合多模态、大规模多语言内容消费的宏观趋势，并与企业内部其他利益相关者的数据进行交叉比对后，团队发现满足这“十分之九”的本地化经理可能会导致产品走回头路。Phrase 最终选择坚持面向多模态内容规模化扩展的新方向，将受众拓宽至整个组织，而非局限于传统的本地化经理群体。Semih 指出，优先级决策的前提是看清数据背后到底代表了哪类角色的声音，决定权往往取决于“你想把公司带向何方”。 - **通话分析维度**：AI 自动记录会议转录、客户与销售的对话时长比、特定产品关键词提及频次。 - **战略抉择**：不盲从局部群体的传统诉求，优先押注多模态与全组织协同的未来趋势。 --- ### [38:54](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2334s) 告别盲目自嗨：从海量 POC 走向具备业务治理的生产落地 针对现场观众提出的“当前企业内部大量沉溺于 Vibe Coding 和概念验证（POC），真正上线产生实际价值的应用却寥寥无几，面临选择过剩（Problem of Plenty）”的问题，嘉宾们探讨了从实验过渡到规模化投产的方法论。 Susana Selles Valls 指出，这种现象取决于企业在 AI 采用上的成熟度阶段。初期让员工自由实验、摸索并迎来各自的“Aha moment”是非常健康且必要的；但随着组织成熟，企业必须迅速建立战略定位、治理规范（Governance）与清晰的优先级体系，将技术探索严格锚定回要解决的核心业务痛点上。 Devendra Singh Parmar 结合银行与金融科技背景补充道，创新不仅发生在企业内部，也正在监管层面被积极推动。例如北卡罗来纳州的金融监管机构专门搭建了沙盒环境（Sandbox Environment），主动邀请银行业机构在受控环境中测试 AI 产品与解决方案。他总结道，企业在推进 AI 落地时应当贯彻“快速测试、快速失败、快速转向”（Test fast, fail fast, and pivot fast）的敏捷机制，在自由探索与严格治理之间取得平衡。 ### [40:37](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2437s) AI 技术的学习曲线与持续试错 在谈及组织对人工智能的适应过程时，嘉宾们强调团队目前正处于一条持续上升的学习曲线中，必须不断学习才能跟上技术演进的步伐。当前的 AI 技术远未达到其发展的巅峰状态，作为一门仍在走向成熟的技术，企业不能指望一蹴而就。在实际落地中，团队必须保持开放的实验心态，做好直面技术陷阱与挫折的准备，并通过敏捷迭代逐步克服这些挑战。 - **技术成熟度**：当前的 AI 并非最终形态，技术仍处于高速演进和成熟期。 - **团队应对策略**：接受探索中的试错成本，将实验与跨职能团队的持续复盘作为推进 AI 项目的常态。 ### [41:00](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2460s) 衡量 AI 投资回报率（ROI）与成功指标 关于如何评估 AI 是否能带来足够的投资回报率（ROI）以证明前期投入的合理性，以及该用哪些核心参数来判定项目成功，现场引发了热烈讨论。嘉宾指出，评估 ROI 首先要厘清价值创造的指向：究竟是面向客户的外部产品价值，还是面向企业内部的研发与运营能效。对于外部客户，关键在于深入了解客户的组织目标与核心痛点，评估 AI 产品能否显著加速其商业目标的达成；而对于企业内部，ROI 则直接体现在产品研发与交付周期的极致压缩上。 有效的内部 AI 赋能绝非仅仅开发一层浅显的模型外壳（wrapper），而是要在建立完善的风控、合规护栏、前期调研以及严格的质量控制（QC）的前提下，将原本长达两年的产品交付周期大幅缩减至三到六个月。这种交付速率的飞跃本身就是极其可观的 ROI。嘉宾分享了一个真实案例：一家医疗器械公司曾坦言，新产品推向市场每延误一天，公司每天就会承受高达 100 万美元的巨额损失，而当时该项目已经延误了整整三个月。在这种高昂的时间成本场景下，如果 AI 能够帮助团队将上线节点提前，就能直接为企业挽回数以千万计的真金白银。尽管不同企业的业务形态各异，但只要聚焦于时间压缩与成本止损，组织内部都能找到切实的量化指标来衡量 AI 的投资价值。 ```mermaid flowchart TD accTitle: AI 投资回报率（ROI）评估维度 subgraph External["外部 ROI（客户价值赋能）"] E1["厘清客户组织目标与痛点"] --> E2["利用 AI 产品加速目标达成"] end subgraph Internal["内部 ROI（组织能效提升）"] I1["建立风控与质量控制 (QC) 护栏"] --> I2["缩短产品上市时间 (2年 ➔ 3-6个月)"] I2 --> I3["避免每日巨额延期损失与机会成本"] end ``` - **外部衡量维度**：紧扣客户的实际组织目标，量化 AI 产品为客户加速业务流与创造增量价值的程度。 - **内部衡量维度**：在合规与质量保障到位的前提下，将产品研发上线周期从 2 年缩短至 3 至 6 个月。 - **延误成本案例**：医疗器械领域产品未能按期上市，每延误一天导致的损失高达 100 万美元（$1,000,000/天）；挽回延期时间即是创造直接经济效益。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=4s) 场内有多少产品经理（PM）？请欢呼一下！场内有 AI Agent（智能体）吗？ — Abhai Pratap Singh > [05:22](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=322s) 正如大家所知，模型总有知识截止日期（Cut-off Date），但人类没有，优秀的 PM 恰恰擅长不断进化。 — Abhai Pratap Singh > [07:20](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=440s) AI 的好处在于它没有人类的情感包袱，它是完全客观的。 — Semih Altinay > [20:27](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1227s) 氛围编码（Vibe coding）并不等于交付一个可靠且安全的产品。 — Susana Selles Valls > [20:27](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1227s) 既然我用 ChatGPT 几分钟就能做出来，为什么你们的路线图还要排到明年？ — Susana Selles Valls > [29:37](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1777s) 我们想要更多控制权（We want more control）。 — Semih Altinay > [30:53](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1853s) 不要只列出问题，直接帮我修好它 — Semih Altinay > [32:48](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=1968s) 产品经理绝没有消亡 — Abhai Pratap Singh > [38:54](https://www.youtube.com/watch?v=m-dIiXPqGBM&t=2334s) 快速测试、快速失败、快速转向（Test fast, fail fast, and pivot fast） — Devendra Singh Parmar ## Mentioned Devendra Singh Parmar, Susana Selles Valls, Abhai Pratap Singh, Semih Altinay, Paul Timmermann, Discover, Suffolk, Amazon, Amazon Alexa, Moderna, Phrase, Toptal, LinkedIn, Gemini, Google Sheets, ChatGPT, Claude, Cursor, Slack, Agentic Content Management System (Phrase), Model Context Protocol (MCP) ## Video description Devendra Singh Parmar, Principal Product Owner, Discover Susana Selles Valls, Director Functional AI, Suffolk Abhai Pratap Singh, Senior Product Manager-Technical, Amazon Semih Altinay, VP AI Solutions, Phrase Paul Timmermann, VP of Product, Toptal Abstracts This session will explore how AI is empowering product managers to make more informed decisions, accelerate product development, and enhance customer experiences. Panelists will discuss how AI can be used to analyze customer feedback, predict market trends, and optimize feature prioritization. Learn how leading product managers are leveraging AI to streamline workflows, improve product roadmaps, and drive innovation. The discussion will also cover the challenges of integrating AI into the product lifecycle and how to ensure alignment between AI capabilities and business goals. [Source](https://www.youtube.com/watch?v=m-dIiXPqGBM)

[0:04] Abhai Pratap Singh: 场内有多少产品经理（PM）？请欢呼一下！场内有 AI Agent（智能体）吗？ [5:22] Abhai Pratap Singh: 正如大家所知，模型总有知识截止日期（Cut-off Date），但人类没有，优秀的 PM 恰恰擅长不断进化。 [7:20] Semih Altinay: AI 的好处在于它没有人类的情感包袱，它是完全客观的。 [20:27] Susana Selles Valls: 氛围编码（Vibe coding）并不等于交付一个可靠且安全的产品。 [20:27] Susana Selles Valls: 既然我用 ChatGPT 几分钟就能做出来，为什么你们的路线图还要排到明年？ [29:37] Semih Altinay: 我们想要更多控制权（We want more control）。 [30:53] Semih Altinay: 不要只列出问题，直接帮我修好它 [32:48] Abhai Pratap Singh: 产品经理绝没有消亡 [38:54] Devendra Singh Parmar: 快速测试、快速失败、快速转向（Test fast, fail fast, and pivot fast）

[0:04] Paul Timmermann: How do we continue to think about first principles when it comes to product thinking and apply all the great new tools out there, but not lose sight of what makes us great product managers? [7:14] Semih Altinay: AI doesn't have feelings. Maybe sometimes we think it should or we wish it should, but it's a good thing because now it's kind of separating some of the misconceptions within the organization. [12:56] Semih Altinay: Everybody else that we talk to is saying, 'Finally, there's a product for me, the marketer within the organization or an HR person... and I don't have to funnel everything through a localization manager.' [16:27] Semih Altinay: don't try to fit a square into a round hole [16:27] Semih Altinay: Don't coerce the model to do something that you've already wanted to do. [20:27] Susana Selles Valls: vibe coding is not shipping a reliable and safe product. [34:05] get your hands dirty

[0:04] Good morning everyone. Um, welcome to [0:07] what I think is going to be one of the [0:08] most practical sessions that you'll [0:10] attend today. Um, how many PMs in the [0:13] room? PM give me a cheer like [0:17] how many aspiring PMs in the room? [0:21] How many agents in the room? Are there [0:23] any? No. Okay, we're good. All humans. [0:27] Um, so I we we have a incredible lineup [0:30] of speakers here. Uh, and we're going to [0:33] be talking about AI in product [0:35] management. Um, we heard Jeffrey speak [0:38] this morning about how AI is reshaping [0:41] and his views. I think let's hear from [0:43] the gentleman's and ladies here. Um so [0:47] guys um I want you to do a quick [0:50] introduction about yourselves and so [0:53] your name your introduction and then [0:55] something around how AI is changing [0:59] product management for you [1:02] there. All right. Yeah. Hi. Hi everyone. [1:05] Uh my name is Dendra. I'm uh working as [1:07] a principal product owner uh at discover [1:09] and having 16 years of banking industry [1:12] experience and uh this is one of the [1:14] exciting topics very close to my heart [1:16] because it has impacted almost everybody [1:18] including product management and one of [1:20] the ways I think uh it has impacted us [1:23] is or or I see how it's getting [1:25] impacting this industry is as a PM I [1:29] think you are focusing more on pain [1:31] brainstorming right instead of [1:33] brainstorming you are always trying to [1:35] uh step into the customer's shoes and [1:37] trying to kind of figure out the problem [1:38] and I think AI is coming up with some [1:40] smarter cooler ways to help us uh in in [1:43] our day-to-day life in terms of solving [1:45] problems for the customer. [1:48] >> Hi there. Um Sami Alton with phrase VPAI [1:51] solutions and I think the way that AI [1:54] has been kind of shifting product [1:56] management is really get to know the [1:58] customer, really understand them. It's [2:00] not so much about instinct anymore, but [2:02] really um let's get the actual data, but [2:05] then let's be able to tell a story and [2:06] then craft that message depending on the [2:08] stakeholder within the company. I am [2:11] Susanna um I'm the director of [2:13] functional I uh at SUFFK. I recently [2:16] joined a company and previously I was at [2:19] Mona and Amazon and I am really at the [2:23] intersection of product strategy and AI [2:27] and I focus on the transformation of how [2:30] AI is going to change the way that we [2:33] work and in the corporate functions uh [2:37] aspects. So for example, think about [2:39] finance, legal, marketing, business [2:41] development and I'm really excited to be [2:43] here. Thank you so much for having me. [2:46] Hi everyone. I'm Paul Timberman. I'm the [2:48] VP of product at Toptal. To toptal is a [2:51] freelance marketplace that connects uh [2:53] clients with our freelancers all over [2:55] the world. And I'm super excited to be [2:58] on this panel today. We certainly have [2:59] been exploring a lot of different [3:01] applications with AI throughout our [3:03] organization, throughout our talent [3:05] network. Uh and I think one of the [3:07] biggest things uh that AI is teaching me [3:10] as a product manager, me as a product [3:12] leader, uh is is discipline. There's so [3:15] much happening uh every single day. Um I [3:17] don't know about you, but uh scrolling [3:19] through LinkedIn certainly is uh quite a [3:21] chore these days to kind of go through [3:23] everything, but like how do we continue [3:24] to think about first principles when it [3:26] comes to product thinking and apply uh [3:28] every all the great new tools out there, [3:30] but not lose sight of what makes us [3:31] great product managers? [3:33] >> Awesome. Yeah, first principles [3:34] thinking, that's what Pming is all [3:36] about. Um I I quickly introduce myself [3:39] as well. I'm a product leader at Amazon [3:41] Alexa and I'm going to be a moderator [3:43] for today. We're going to be covering [3:45] every aspect of product management. So, [3:48] here we have with me. Um, all right, [3:50] let's address the elephant in the room. [3:53] AI isn't just another tool for us PMs. [3:56] It's fundamentally changing how we work [3:59] as PMs. Um, Susanna, you're leading um [4:03] AI at Suffukk. how's AI started to [4:06] reshape what you think about the PM role [4:08] and how how that is evolving. [4:11] >> So I think traditionally we've [4:14] understood uh product management as um [4:18] gathering the user requirements being [4:20] the voice of the customer you know [4:22] managing those timelines and putting [4:23] everything into a road map. I don't [4:25] think that's going to change. I think [4:27] we're going to get much more productive [4:29] at what we do. But I do think that AI [4:33] has pushed us outside of like that set [4:36] box into more of the technical realm of [4:40] product management. So really there are [4:42] tools now that allow us to um become [4:45] more fluent in technical matters like [4:47] understanding models like integrations [4:49] and such. And I think that that's going [4:53] to become we're going to see it more as [4:55] a trend in the industry. And [4:58] specifically in my case what has really [5:00] helped me has been um bringing that [5:03] aspect of oh I have an idea I talk to [5:05] the customer. What about uh just [5:08] creating a proof of concept very quickly [5:10] before even going to engineering like [5:12] understanding those ideas how those can [5:14] shape on a technical aspect and being [5:16] able to collaborate that further with [5:19] the developers and also with the [5:20] customer as well. [5:22] Nicera [5:24] um from your perspective at discover [5:27] what's what's skills are becoming um new [5:32] of sorts in the AI era and what is [5:35] becoming obsolete. [5:36] >> Yeah, I think that's a great question. I [5:38] think as we know like traditionally PMS [5:41] are expected to have three key skills [5:43] which is which I classify as business, [5:46] technology and analytical. So I think [5:49] these have been the core skills that are [5:51] required in any PM job. But as AI has [5:53] stepped in like in terms of the the way [5:56] it has transformed and it the the kind [5:58] of potential it is bringing in in terms [6:00] of product innovation, product [6:02] development and even your product [6:04] solution could have AI within it right [6:07] uh now it's kind of pushing product [6:09] managers to upskill and create a new set [6:12] of skill set which I call it as AI and [6:15] you know uh machine learning and it's [6:17] important reason being the if you're [6:20] using AI as a product Right? You need to [6:22] be able to kind of explain what it is [6:25] kind of processing. You should be able [6:27] to talk to your customer and explain [6:28] those explanability in terms of whether [6:30] this product is having some biases [6:32] inherent biases as a result of which it [6:35] it's been kind of created or it should [6:38] also be kind of uh helping you uh uh you [6:41] know you as as a PM you should be able [6:44] to kind of understand and explain are [6:47] there any model drifts like in terms of [6:49] how how relevant your AI product is [6:51] relevant because it might have been [6:52] trained on the past historical customer [6:54] data and the customer preference. [6:56] references and market trends might have [6:57] changed and as a result there is a need [6:59] for you to be able to talk that language [7:01] with a cross functional team especially [7:03] an analytical teams data scientist teams [7:06] uh so it's important that uh uh from a [7:08] skill set perspective we upscale [7:10] ourselves continuously as new and new [7:12] things are emerging within AI space [7:14] >> awesome yeah models have a cut off date [7:17] humans don't PMs are great at that um [7:20] let's get a little more into the product [7:23] life cycle of sorts so at Amazon we have [7:25] this fundamental belief that you always [7:27] work backwards from the customer and [7:29] customer is always at the center of the [7:31] product that you build. Um Sammy, how's [7:35] how's product management evolved for you [7:38] in terms of looking at customer [7:40] insights, understanding what the [7:42] customers are really demanding and and [7:44] how have you been able to shape up with [7:46] that? [7:48] I think originally we kind of relied a [7:51] lot on insights that people had within [7:54] the organization. Now one of the key [7:56] things is really really understanding [7:58] what the customer is talking about like [8:00] without any bias without any kind of um [8:04] sometimes we we're not just very [8:05] objective right like internally. So you [8:07] may have a salesperson you may have a [8:09] customer success manager that's [8:10] complaining about something with the [8:11] product but sometimes you you really [8:13] don't get the real story. So the way and [8:16] by the way at phrase we track [8:17] everything. We're a language technology [8:18] company. I neglected that mention that [8:20] earlier but um we track everything all [8:24] the data points. So for us now listening [8:27] to the customers actually um recording [8:29] the conversation getting the transcript [8:31] getting analytics from that call we've [8:34] noticed there's there's a misalignment. [8:35] There's sometimes a divergence between [8:37] what we think is right and what the [8:38] customer is actually saying. And again [8:40] it's objective because AI doesn't have [8:42] feelings. Maybe sometimes we we think it [8:45] should or we wish it should, but it's a [8:48] good thing because now it's kind of [8:49] separating some of the misconceptions [8:51] within the organization. It's just given [8:53] us the actual data. Then we combine it [8:55] with the usage data and some of the [8:57] other analytics we have. Put it on [8:58] beautiful dashboards again tell the [9:00] story. I think it really really helped [9:02] us get to know the customer um and and [9:04] again eliminating those misconceptions [9:06] that we had initially. [9:07] >> Awesome. uh what impact has that had on [9:10] your product strategy roadmap and how [9:13] you've been able to incorporate that? [9:15] >> One of the key things is we can now [9:16] pivot so much faster so we're not [9:18] spending quarters and in half years and [9:20] whatever else and then realizing it [9:22] didn't work because now you just lost so [9:24] much money time and effort and in the [9:26] world of AI if especially if you're [9:28] producing AI products you're already [9:30] behind right like your competitors are [9:31] going to catch up. So now we're just so [9:33] much faster on our feet so to speak and [9:35] we can immediately pivot and we can give [9:37] the customers exactly what they want. [9:40] >> Excellent. Uh Paul, do you want to jump [9:42] in as well? [9:42] >> Yes. Uh certainly identify a lot with [9:44] the the customer feedback piece. Uh and [9:46] and something that we have been using a [9:48] lot at our organization kind of well [9:50] before uh AI we call our our feedback [9:53] river. Um it it is a set of feedback [9:56] that comes in from all of our our [9:59] customers and goes into a Slack channel. [10:01] It's good to have a pulse on just what [10:04] is happening, what our customers are [10:05] feeling, and it's coming from all [10:07] different sources. It's coming from our [10:08] NPS feedback. It's coming from call [10:10] data, and and it just helps us [10:12] understand how our customers are [10:14] feeling. But certainly, the piece that [10:16] we had a struggle with like threading [10:17] the needle on was what are the actual [10:19] trends of this? Like I I I feel that I [10:22] have like the empathy of what's going on [10:23] in this channel, but I don't necessarily [10:25] know like what are the trends and and [10:27] how should I interpret that? And a very [10:29] practical thing that you can do now [10:31] easily is as all this feedback ends up [10:33] in a spreadsheet, you can use the the [10:36] Gemini function to uh summarize that [10:39] that data and so and play with different [10:41] categorizations and just a lot of [10:43] iterations that you would have [10:44] previously gone to a data science team [10:47] to and that would have taken um a few [10:49] cycles and now we can play with our data [10:51] kind of in real time. And so uh that [10:53] fast labeling, categorization, being [10:55] able to instantly take that and uh [10:58] create visualizations, dashboards, [11:00] things like that is certainly uh [11:01] gamechanging in that way. Um and as far [11:04] as like how that has impacted our [11:06] strategy, our road map. Um we still uh [11:09] do our our best PRDs that we can. We [11:12] still kind of put together those [11:13] business strategy decks. Um, but I think [11:16] that we're all we're all busy product [11:17] people and certainly you might uh cut a [11:20] little corner on uh getting time to do [11:23] the market research, getting time to [11:24] really think about the customer personas [11:27] and things like of that nature. And now [11:29] you can certainly like make sure that [11:30] you're at least trying your first pass [11:32] at that uh through AI tools and and [11:35] certainly like that doesn't mean that [11:37] you should outsource all of that, but [11:38] it's a good way to kind of fill in those [11:40] details that maybe we didn't have the [11:42] time to fill in previously. Super super [11:45] nice. Um I'm curious about one [11:48] particular thing like once you've [11:50] started using AI or Gemini in your case [11:53] of analyzing these customer insights, [11:55] has there been anything which is which [11:58] has been a customer insight that has [11:59] actually surprised you uh which has come [12:03] out from AI? [12:04] >> Um I I think that the [12:07] we're we're understanding the the so in [12:10] our case we're connecting our our [12:12] clients and our talent. Um, and [12:13] certainly like the things that I'm uh [12:16] more aware of as a as a product manager [12:18] are is this feature working correctly. [12:21] Um, but we're we're understanding that [12:22] the nature of of the relationship [12:24] between our talents and our clients in a [12:26] whole new way and and what really makes [12:28] a successful relationship which just [12:30] requires looking at a lot of information [12:32] because human relationships by nature [12:34] are very complicated. And so being able [12:36] to kind of get into the depth of that [12:38] relationship, what is actually making [12:40] that human successful is where there's [12:42] really a magic sweet spot because I [12:44] wouldn't have been able to look at that [12:45] before. Um it' have been much more like [12:46] oh this button doesn't work right but [12:48] which is the more obvious surface level [12:50] things. You can really go a much level [12:52] much lower level and and much more depth [12:54] than you could before. [12:56] >> Awesome. Um in the interest of time [12:59] let's actually move to something which [13:01] is a PM's headache prioritization. [13:04] uh every time you have somebody uh a [13:08] customer walk up to you or somebody in [13:09] the team walk up to you it's like hey [13:11] why don't we build this um so Sammy I'm [13:15] I'm going to jump to you [13:18] so prioritization or like feature [13:21] prioritization has always involved data [13:23] and then scouring through customer [13:25] insights and then take those decisions [13:28] with AI as Paul was just mentioning [13:30] adding this layer of you know summarized [13:33] insights [13:34] How is prioritization changing for you? [13:37] >> So, I sit between business and product, [13:39] which is kind of cool. I work a lot with [13:41] sales. I look I work a lot with product. [13:44] It seems like priority goes to the [13:46] salesperson who's the loudest because [13:48] everybody's customer is the number one [13:50] customer in the organization. Every [13:51] customer success manager says, "My [13:53] customer deserves the best of the best [13:55] and we need to go there." Of course, you [13:56] can also look at the size of the [13:58] customer and say, "Well, this is the [14:00] biggest customer or biggest five [14:01] customers we have, whatever." Sure. But [14:03] how many of those smaller customers that [14:05] started small and become massive over [14:07] the years, right? They grew with you. So [14:09] I think the way AI helped us is it [14:13] really kind of cleared out the noise so [14:16] to speak. We can now see the real data [14:19] and um understand the customer better. [14:22] kind of like what you were saying Paul [14:23] like for example [14:26] we think that this is what the customers [14:29] need but then that's one customer [14:31] profile that we worked with [14:32] traditionally and we just put out a [14:33] brand new product we actually this is [14:35] real by the way we just put out a new [14:36] product in June and then this product [14:38] was our attempt to kind of shift left [14:41] and go at the content creation again [14:43] we're a language technology company [14:44] which traditionally worked with language [14:46] managers so somebody at your [14:48] organization that deals with [14:49] translations and those People want the [14:52] product to be this way. They look at [14:54] this thing and they say okay but can you [14:56] put those other features that you had in [14:57] the platform into this thing because we [14:59] want that control but everybody else [15:01] that we talk to is saying finally [15:05] finally there's a product for me the [15:06] marketer within the organization or an [15:08] HR person or somebody else and I don't [15:11] have to funnel everything through a [15:12] localization manager. Look they're still [15:14] in charge they still manage the assets [15:16] but now I can work with this thing. I [15:19] don't have to have a degree in [15:20] translation or I don't know like um [15:22] really know how to use a transition [15:24] management system. This is behemoth of a [15:27] workflow management tool. So um [15:30] initially we were only listening to this [15:32] one customer and now looking at the data [15:34] we're seeing more and more users are [15:36] coming in from this other area. Again [15:38] it's cleaning out the noise a little bit [15:40] and when we look at certain customers [15:42] and how they're using our product again [15:44] I said you know we track everything. We [15:46] now created a northstar, so to speak. [15:48] Here's our ideal customer that uses all [15:50] kinds of different parts of our [15:51] platform. That's the ideal customer. So, [15:54] the industry doesn't matter. How can we [15:57] get the other customers to be like that [15:58] customer? We look at all the usage data, [16:00] all the information they're giving us, [16:02] all the feedback they're giving us, and [16:04] now we can replicate that success [16:05] through others by applying the same [16:08] principles. We can now educate the other [16:09] customers and say if it's travel [16:11] industry, we say here's what what how [16:13] that customer is using it. If it's life [16:15] sciences, here's how this customer is [16:17] using all these automation tools and all [16:19] these AI tools to achieve these goals. [16:21] Now, the other customers are also saying [16:23] this is, you know, fear of missing out. [16:24] It's like if you're um I don't know if [16:27] you're Visa, you don't want to miss out [16:29] on an opportunity that discover is [16:31] grabbing, right? So, it's it's it's a [16:33] real thing. And when you educate the [16:35] customers on how people are using your [16:36] platform, using the real data, using the [16:39] insights, it tells the the story so much [16:42] better. It's no longer no longer [16:44] instinct. It's an actual data. [16:46] >> Now, that's that's that's super cool. [16:50] Once once you get these insights from [16:53] AI, like have you had like a CSM or an [16:55] account manager come up to you and say, [16:57] I don't trust this uh or some curious [17:00] insight in there. [17:02] >> Yeah, look, sometimes we still have to [17:04] go with the gut feeling. Um I I do [17:06] understand that AI doesn't have the full [17:08] story. Sometimes we get out of it what [17:11] we feed into it. Um an example I got [17:14] into with one of the product managers [17:15] about what's an AI agent is this thing [17:17] that we produced is that a real agent or [17:19] not. Right? And if you just throw that [17:22] question into AI, it comes back and says [17:24] because of these reasons, it's not an [17:25] agent, but it doesn't really know the [17:27] product. We just didn't fit all the [17:28] right information or the latest [17:30] information. So sometimes you do have to [17:31] go with your gut feeling and say look [17:33] here's what it means to be an agent. [17:35] This thing just didn't quite understand [17:37] it. let's rework it. Let's give it all [17:39] the information that we need because we [17:40] really want to extract the right [17:41] information from our AI tools. Um, so [17:43] sometimes you do have to go with your [17:44] gut, but that doesn't mean you ignore [17:46] the data. It is critical that you kind [17:49] of bridge the gap. Um, I think we're all [17:51] experts in the room for a reason. So you [17:53] need to use that expertise. Don't just [17:55] blindly trust AI, but don't um try to [17:59] fit a square into a round hole. It's [18:01] like um if that's the answer you want to [18:04] get, don't try to force AI and analytics [18:06] and manipulate the data to get the [18:08] answer that you want, which is I think a [18:09] great thing that came out of the keynote [18:11] this morning as well and yesterday. [18:14] Yeah, that's so neat. Don't coers the [18:15] model to do something that you've [18:17] already wanted to do. Um thera I'm going [18:20] to uh lead into you try and understand [18:24] how is it how is this different like now [18:26] that we have these AI models which can [18:29] turn out insights how much are you at [18:32] discovered using these AI generated [18:34] insights are you still trusting your gut [18:37] and your judgment so to speak [18:40] >> yeah uh that that's a great uh question [18:42] and I think like before before jumping [18:44] on to the examples from my own industry [18:46] I think I would like to take something [18:48] which is more generic and relatable. uh [18:50] we know like uh all the large healthcare [18:53] u you know companies or especially [18:55] hospitals if you talk about right in US [18:57] they are all experimenting or using [18:59] trying to use the AI you know as a as a [19:02] application maybe it's writing some [19:04] prescriptions on behalf of the doctor or [19:06] it's kind of providing some health [19:08] diagnostic but to what degree do we want [19:11] to rely on those right the the [19:13] underlying thing that we need to be kind [19:15] of aware of is while uh we have lot of [19:18] fancy AI tools that can do some job for [19:21] you but they do not have gone through [19:24] all the scenarios and data points right [19:26] there the the details are still kind of [19:29] being evolving as you know things are [19:31] progressing so similarly likewise if you [19:34] talk about you know banking or insurance [19:35] sector right these are heavily regulated [19:37] industries or uh you know as as a bank [19:40] or insurance but do we want all the [19:43] outcomes that are coming out of AI to be [19:45] solely rely on that because sometimes AI [19:48] just will look in terms of the data [19:50] items has been fed. It might miss out on [19:52] the overall context. The context could [19:54] involve customer shifts in the you know [19:57] their preferences the the market trend [20:00] shifting the ge geopolitical changes [20:02] that are happening very regularly uh you [20:04] know regulators are sanctioning some [20:06] countries like because of the wars that [20:08] are ongoing right so all these data [20:10] cannot be fed into the model when when [20:12] the models were built right and as a [20:13] result of which I think it's important [20:15] that PM wear that hat and provide that [20:18] instinct and provide that comprehensive [20:20] uh view when you are kind of uh you [20:23] reviewing the output or outcome of your [20:25] AI tool. [20:27] >> Okay, super cool. Um, okay, time for [20:31] some honesty. I think integrating AI [20:34] into products isn't all success. And [20:37] Rosie Gardens, um, it's it's not just [20:40] about efficiency gain. So, I want to [20:43] hear from you Susanna and Semi is like, [20:45] what's been your biggest this is harder [20:47] than I thought moment when you [20:49] integrated AI in your product life [20:51] cycle? [20:53] Susanna, do you want to go first? [20:55] >> Yes. [20:57] Here comes the honesty. Um, I think I [21:00] would like to speak to three kind of [21:02] challenges that I've been seeing coming [21:05] up over and over again. Um, as well as [21:08] what I've seen in the industry. So, the [21:11] first thing is data. Um, if your data is [21:16] not prepared and ready for AI, your [21:19] outcomes are not going to be the ones [21:21] that you would expect. And I know this [21:25] has been repeated all over like times. [21:27] And I don't think that's just specific [21:28] to AI, but with AI matters even more [21:32] because you are going to trust what that [21:35] model says. And we're getting to a point [21:37] where we want to work with this agent [21:39] side by side. So like being able to have [21:41] that clean and prepared data is very [21:44] important and customers don't always [21:47] understand that. Um the second thing [21:50] that I've seen is that there is this [21:53] like I don't know if you've also like [21:55] seen it around but there is this [21:57] unrealistic expectation that you [22:00] sprinkle AI into every product and [22:02] suddenly like miracles are going to [22:04] happen especially from end users right [22:07] it's like um yeah now we have a [22:10] supercharged AI product like let's save [22:12] the world um I think our role as as [22:16] product managers is to provide that [22:19] education, that education and that [22:22] understanding to end users that like [22:25] sometimes the best solutions are the [22:29] most simple ones. You know technical [22:31] simplicity is something that not many [22:34] people like grasp um because we think [22:37] about like you know we have all these [22:38] complicated models there must be so [22:41] good. Um and I think part of like a PM [22:44] experience now is really like getting [22:46] first ourselves educated working with [22:48] the technical teams working like with [22:51] the industry itself to understand um [22:54] where are we going and then making sure [22:56] that we bring back to the customer to [22:58] the end user I they understand why we're [23:01] building the best for them and then a [23:05] third thing that I'm seeing a lot is you [23:08] know coding or well actually I should [23:11] say vibe coding is everywhere. It's so [23:15] widely accessible right now and it's so [23:17] cool. I mean like at the beginning I [23:19] said that's one of the things that I [23:20] love about like how PM role are [23:23] transforming because we can become more [23:25] technical. We can upskill ourselves but [23:28] that's also true from every single [23:31] person in the world who has access to [23:33] chipd cloud etc. Now you can spin up [23:37] products or like prototypes in minutes [23:40] not like hours, weeks, minutes. So like [23:42] the expectation [23:44] of the end user of like oh this is so [23:46] easy. What is your road map talking [23:48] about like next year when I can just do [23:51] this with chat GPT and here I got it. So [23:54] I think it's again like the education [23:57] and really making sure that you're [23:59] bringing your customer from you know the [24:01] the actual product discovery um to to [24:05] all like the journey when it comes to [24:07] like the launch and specifically like [24:10] bringing pilots to production is one of [24:14] the most difficult things in my opinion [24:17] for you know the the product cycle and I [24:20] think it's because we are missing that [24:22] scalability concept and really thinking [24:25] that these models like work daytoday and [24:28] they're going to work for every single [24:30] scenario that we have not imagined yet. [24:33] So I think I would h if I would have to [24:37] highlight is like VIP coding is not [24:41] shipping a reliable and safe product and [24:44] then education as part of PMS and [24:47] education to ourselves as well is going [24:49] to become more and more important. you [24:51] touch a really interesting aspect of one [24:55] of the PM trends which is VIP coding um [24:59] a lot of been this word is also overly [25:01] abused so can you can you talk to a [25:04] little bit about how you've been using [25:06] VIP coding and what do you recommend the [25:08] PMs in the room to actually use that for [25:11] >> yes so I have a background also in [25:15] software engineering so like to me I [25:18] came to the PM world having that [25:21] technical um understanding as well. [25:24] However, um vibe coding for me is about [25:28] testing ideas and what I really like [25:32] about that is that I can fail faster [25:36] and even better I can pivot faster [25:39] before like spending too much time on [25:42] maybe a feature of or a concept that you [25:45] know it's not going to work. So I would [25:48] say like speed is something that we've [25:50] gained and our own productivity SPMs [25:53] like I think that's what vibe coding can [25:55] bring to us. [25:56] >> Super cool. Uh I think a lot of VIP [25:58] coders in the room as well. Paul um at [26:02] at your experience in toptal uh what's [26:05] been the hardest thing that you've [26:06] discovered in terms of integrating AI [26:08] into your products? [26:10] >> Sure. I I kind of opened up about this [26:12] idea of of discipline um and and how [26:15] everything is changing so quickly and [26:17] certainly like I I want to be as as [26:19] involved and understanding of all the [26:21] the new nuances and uh all the [26:24] excitement kind of that is happening. [26:26] Um, and it's easy to get wrapped up in [26:29] that and start uh building out more [26:32] complicated AI infrastructure to start [26:36] uh working on your own kind of in-house [26:38] uh models and and again missing like [26:41] what are we actually doing for our end [26:43] user uh and that getting back to uh kind [26:47] of core product principles uh thinking [26:50] um and and in our particular case we've [26:52] made way more progress over the last six [26:56] months or so um with our AI products [26:59] than we did well before that because we [27:02] start we started just uh understanding [27:05] those customer problems much better and [27:06] understanding how is this a a technology [27:09] that helps enhance that rather than how [27:11] do we do AI things just for the purposes [27:13] of AI. Um, I'm going to say a slightly [27:16] contradictory point here though as well. [27:18] Um, which is I think that there is a a a [27:23] level that we just have to try things as [27:25] well and kind of fail fast. to your [27:27] point and I I think that that's kind of [27:30] the the beauty of where this kind of [27:31] comes to a true confluence if you will [27:34] that uh yes like let's uh think about [27:37] those core user problems that we need to [27:39] solve but at the at the same time um how [27:43] how can we do quick prototypes and how [27:45] do we get comfortable just trying new [27:47] things to see if we can solve those uh [27:49] problems in in new and unique ways. Um [27:52] because sometimes it it's just a matter [27:54] of starting and and you're kind of [27:55] surprised by how far you can get. But [27:57] it's about starting on the right foot, [27:58] starting with that customer problem [27:59] you're really trying to solve, right? So [28:02] if you had the luxury to go back in time [28:05] and go back to the time when you're [28:07] starting this AI uh integration into [28:10] your product, what would you do [28:11] differently? [28:14] I think um something we're not really [28:17] talking about enough wi with uh AI, [28:20] there's all the the hype of like AI is [28:22] replacing this role and replacing that [28:24] role. Um and I think that that's the the [28:28] wrong narrative personally. I think it's [28:29] about how how does it help all of us [28:32] become uh super versions of ourselves? [28:35] Um and how does it help superpower every [28:37] single role? And I think that there's [28:39] something incredibly unique in how we uh [28:43] deeply integrate AI into our processes [28:46] where that the handoff to humans and how [28:48] we treat human in the loop um is is [28:51] super interesting, super successful and [28:53] honestly like that helps you uh beat the [28:56] the rapper apps that are out there [28:57] because everyone's going to catch up to [28:58] that. But if you find a unique way to [29:00] put your your workforce to to work and [29:03] superpower them in an entirely new way, [29:05] that that's the sweet spot and where [29:07] there's a lot of strengths in your [29:08] organization. And so I think that yeah, [29:11] you might get caught away with saying [29:13] like how can we just replace all this [29:15] with AI today? Um, but that might [29:17] actually again kind of stirring away [29:18] from the the customer problem there and [29:20] instead how do we uh really make our [29:23] organization a better version of our [29:24] organization today and help AI and use [29:27] AI to help enhance that rather than the [29:29] the replacement theory. [29:31] >> That's that's so true. Humans in the [29:34] loop and humans on the loop all times. [29:37] Um, so crystal ball time. Let's let's [29:39] look forward and think about [29:43] and I'll go with you Sammy. Um, what's [29:47] one CA AI capability or trend that has [29:50] genuinely excited you about product [29:53] management or you think is going to be [29:54] the next big thing for product managers? [29:58] I'll definitely say agents I'll explain, [30:02] you know, uh, [30:04] and and this is how I explain it too. [30:06] It's it's like back in the day back in [30:08] the day like yesterday, but back in the [30:10] day AI just accomplished tasks and you [30:13] had different AI tools that accomplish [30:14] different tasks. But now I as a user can [30:18] give an entire goal or I as an [30:21] organization can give an entire goal to [30:24] an AI agent and it'll figure out what [30:27] tools to use, what tasks to execute and [30:30] get that goal accomplished. That's what [30:33] excites me the most. But I do have one [30:35] caveat there. Recently, we we launched [30:38] um some a aentic AI workflows and [30:40] things. One key customer, one of our top [30:42] customers actually came back and said, [30:45] "That's great. We love your vision. [30:47] We're super excited about where this is [30:48] going, but we want more control." That [30:51] was very interesting to me for me to [30:53] hear. So, I'm kind of in that point now. [30:56] It's like the fork in the road. I love [30:59] the Aantic AI and what it can do. And [31:01] imagine multiple agents accomplishing [31:03] even greater goals for organizations. [31:05] That's fantastic. But what about those [31:07] customers that still want some control? [31:08] They don't want AI to go figure out what [31:10] tools to use. Maybe they say, "I want [31:12] you to use these these these tools for [31:14] this." But um the way that we're using [31:16] agents is really customer creates a [31:19] profile of content and tell us the [31:24] restraints so to speak budget data [31:26] locality um what to use what's my brand [31:29] voice for example and we take all that [31:32] information feed it to AI and now it can [31:34] go accomplish the goal I want to reach [31:36] out to uh millennials in this region [31:39] particularly maybe it's women living in [31:41] Argentina over the age of 35 that's a [31:43] very specific demographic, but if that's [31:46] where the customer wants to reach in [31:47] their language with their brand voice, I [31:49] can have AI accomplish that. Now, all I [31:52] need to do is give the right information [31:54] and it'll just do that for me. Nice. Um, [31:57] have you already deployed a bunch of [31:59] these agents like single agents, multi- [32:02] aents? There's like a lot of terms going [32:05] around with agents. Can you can you just [32:07] for the benefit of the room talk a [32:08] little bit more about it? Yeah, we [32:10] deployed our first one in June and and [32:12] right now it what we call it is is [32:14] really agentic content management [32:15] system. So it's going to have bunch of [32:18] agents underneath. We only did the first [32:19] one and again I'm not calling a regular [32:21] AI tool an agent, right? It is not an [32:23] agent. So this is our first attempt at [32:26] combining multiple AI products that [32:28] produce the end result that the customer [32:30] wants. And the next one is going to take [32:32] that and maybe transform it for a very [32:34] specific purpose. the next one is going [32:36] to find all the QA issues and fix them [32:39] automatically because a lot of customers [32:41] are telling us don't just throw the red [32:43] flags and say these are all the issues [32:45] fix them for me what a concept [32:48] u in the interest of time I think [32:50] everybody's in the room who's thinking [32:52] that I need to start somewhere um what's [32:56] your advice and I'll go around the [32:57] parallel what's your advice for what [33:00] these PM should try and what are the [33:02] things that they should learn [33:03] >> dendra let's start with you Yeah. Uh I [33:06] think one of the cool stuff with with AI [33:09] coming in picture is like it can process [33:12] the data in different formats right. So [33:13] multimodel uh uh agents right they can [33:17] process data from you know text or [33:19] images and all those stuff and they can [33:21] pro assimilate the information or [33:23] intelligence that was not available [33:24] earlier as a PM right so that's one [33:27] interesting thing and I think that that [33:29] we should be kind of looking out for and [33:30] it's going to kind of transform the way [33:33] we solve problems for our customer but [33:35] at the same time I want to call out a [33:37] caveat AI as a technology it's still [33:41] evolving it has not matured Right? If [33:44] you look at like in the recent time, [33:45] agentic AI is the current talk of the [33:47] town or it's it's the recent trend. But [33:49] if if you if you are closely following [33:50] the AI industry, every couple of months [33:53] you have MCP protocols or agent to agent [33:56] protocols in terms of what are the [33:57] guardrails, how the agents are going to [33:58] be talking to each other, what are the [34:00] different frameworks, how the agents are [34:02] going to be built, right? So there is a [34:03] lot of talks and you know innovation [34:05] going in that space. So it's going to be [34:07] you know a great like you know as a PM [34:09] we keep an eye on those kind of [34:11] innovations because though once the [34:13] technology matures that will also change [34:16] the kind of use cases we are using this [34:18] technology over a period [34:21] so that would be my advice for uh the [34:23] PMs or aspiring PMs. Yeah [34:27] >> I'll keep it super short. Yeah just get [34:29] your hands dirty. Just try these tools. [34:31] There's so many out there. Don't just [34:32] get, you know, stuck with chat GPT or [34:34] whatever your favorite uh chat chat uh [34:37] based um transformer based uh technology [34:40] is. But if you want to build decks, we [34:42] were talking about that earlier. Um you [34:44] know, use some of these really cool [34:45] tools that build decks based on the [34:47] information you give. Or if you want to [34:49] analyze spreadsheets, you know, use [34:51] those tools. If you want to do vibe [34:52] coding, use cursor, whatever it might [34:54] be. But um get your hands dirty because [34:56] then it also gives you ideas about your [34:58] product. Oh my gosh, the other day I was [35:00] doing this thing and what a cool user [35:02] experience that was. That's how our our [35:04] product should be. It really gives you [35:05] insights. [35:09] >> I just want to say two things for me. [35:11] The first one is deep research agents. I [35:14] think that's gamechanging especially for [35:16] PMs. And the second one is um using AI [35:22] to ask questions that they were never [35:25] asked before. So like really asking the [35:28] eye to go beyond like human curiosity [35:31] and just like spur some ideas and like [35:34] make sure that you know that innovation [35:36] flows back. [35:38] >> I see Paul maybe a last thought that is [35:41] a little bit more uh practical. Uh we we [35:46] uh should probably invest more in [35:48] general in like product ops and [35:50] everything to do with product [35:51] operations. Uh, and I know that it's [35:53] kind of always the the thing that is put [35:55] to the to the side because we have so [35:57] much other things to do. And so I would [35:59] certainly encourage all of you to to [36:00] take a look at your product operations [36:02] processes. I'm sure that there's one [36:04] that you can try out today uh and plug [36:06] into either um some sort of agent, plug [36:10] into uh some vibe coded app, just try [36:13] something out because it's always good. [36:14] Uh you're you're the actual end user at [36:17] that at that point. um you become an [36:19] advocate to your team on how they should [36:20] start using it more so they get more [36:22] realistic examples and uh it's the the [36:25] best way to learn is doing something [36:27] useful for for yourself and your [36:28] organization. [36:30] >> Sim, thank you so much. Um I think the [36:32] things that I'm taking away from the [36:34] panel is product management is [36:35] definitely not dead uh contrary to [36:38] popular belief. Um we need to get our [36:41] hands dirty. We need to try a lot of [36:43] tools. Uh, and there's a ton of cool [36:46] things that I'll go and try out. I think [36:48] we have a couple of minutes, so I'll [36:50] open it up to questions. Uh, yes sir, [36:54] please. [36:55] >> Yeah, I'll repeat the question so people [36:56] can hear. But yeah, so um, in terms of [36:59] prioritization, how do we do that? Like [37:01] do do we have tools that we use? Um, so [37:05] you know, we had this thing with a [37:07] product that, you know, we thought this [37:09] is what the customer wanted, but then, [37:11] you know, it's kind of funny. The data [37:13] did not exactly show what we were [37:15] hearing from the customers. So nine out [37:18] of 10 customers wanted this one feature, [37:19] they just kept bringing it up over and [37:21] over. However, when we looked at the [37:24] data more carefully, when we analyzed [37:27] it, and it could be really customer [37:30] interviews, for example, again, we we [37:32] track everything. There's a, you know, [37:34] little helper that joins the calls, [37:35] transcribes everything for us. We have [37:37] analytics. We even have analytics on who [37:39] talked more, the customer or our [37:41] salesperson, for example. What products [37:43] were mentioned in that particular call? [37:44] We track everything. Um, what we saw was [37:47] that divergent point where if we talk to [37:50] a localization manager in an [37:52] organization, we get one feedback and [37:54] it's nine out of 10 customers are asking [37:55] for those features. everybody else which [37:58] might be a better future for us as a [38:00] company actually likes this thing that [38:04] we built. So now do we go this way kind [38:08] of backwards so we can satisfy those [38:10] nine out of 10 customers or do we stand [38:13] our ground and change the industry which [38:16] is sort of where we're going to be [38:17] honest with you. We're seeing this [38:19] multimodal, multilingual content and [38:22] it's at scale. That means we need to [38:24] bring more people from the organizations [38:26] not just lo managers. Not to say lo [38:28] managers are dead, right? Like program [38:30] managers are not product managers are [38:32] not dead. We still need those people. [38:35] But that's that's basically how we use [38:36] that data really to look deeper and [38:38] deeper and let's understand who we're [38:40] talking to first because that determines [38:43] the decision that we're going to make [38:44] and in what direction we want to go. [38:46] maybe we don't want to talk to these [38:48] nine out of 10 customers or or this [38:50] particular feedback. Um so that's one we [38:52] what we I think took from that. Uh one [38:54] observation that I'm seeing across our [38:56] organization is that there's a lot of [38:57] playing around happening with AI tools. [38:59] Wipe coding PC's getting made. Very few [39:02] actually going into production and [39:03] actually solving user problems. Uh [39:05] there's a problem of plenty right now. [39:07] Um like is that okay uh in your how are [39:10] you observing it and should that [39:12] continue or is there a framework that [39:13] can be applied to say hey some things [39:15] that we need to you know absolutely you [39:17] know put our head down not get [39:18] distracted by all the updates that are [39:20] happening and get them uh to actual [39:22] impact [39:25] >> I can start I think it depends on like [39:28] the the degree of maturity of AI that [39:32] you have in your organization I've seen [39:34] that at the beginning it works really [39:37] Well, pe with people experimenting [39:39] themselves until they get to that that [39:41] aha moment. We all had it, right? Um, as [39:45] you mature as an organization, I think [39:46] it's very important to have strategy, [39:49] governance, and prioritization of where [39:52] you want to go and link back to the [39:55] actual uh business problems that you're [39:57] trying to solve. [39:59] I'll just add on uh just just from a [40:01] banking context perspective as part of [40:04] one of my previous organizations that I [40:06] working for right um uh I happened to [40:08] meet with one of the regulators from [40:10] North Carolina right and they have set [40:12] up a a sandbox environment inviting [40:15] banks to kind of come and play out with [40:18] all their AI products and solutions [40:20] right so it's not just that innovation [40:22] is being encouraged within the [40:23] organization it's also happening at the [40:25] industry level and different players are [40:27] contributing to that so it's about like [40:29] as Susan said like you know you know [40:32] test fast fail fast and also pivot fast [40:35] right it's it's important that you are [40:37] continuously in that learning curve to [40:39] be able to kind of catch up and as said [40:41] like this whole AI as a technology is [40:44] still to mature it's not that it's a [40:47] peak of the technology that we are [40:48] seeing today so you know we need to kind [40:51] of experiment we will have some pitfalls [40:54] that we have to kind of overcome as a [40:56] team but yeah I think uh [40:58] uh that that's the way to go. [41:00] >> How do you evaluate whether EI will [41:03] provide enough ROI to justify the [41:07] investment and uh how do you measure the [41:11] EI success? What are the parameters [41:13] you're looking to see like you know okay [41:16] so I have done the product and I've [41:17] implemented [41:19] is it success [41:20] >> what are the parameters [41:22] >> success always always the burning [41:24] question for [41:24] >> my question back is is it internal ROI [41:26] or is it external is it using AI to or [41:29] incorporate AI into your products for [41:31] customers or is it implementing AI [41:33] enterprisewide what type ROI yeah I mean [41:36] look there are so many studies out there [41:38] I think like both internally and [41:39] externally externally you have customers [41:41] is asking them the question, what are [41:44] you trying to do? What is the organ [41:45] organizational goal? And then see if we [41:48] can speed that up with our AI products. [41:50] But internally, hey, how fast can we put [41:52] out products? Like given all the [41:54] guardrails that you put in place, not [41:55] just rappers, but really try trying to [41:58] do proper research, risk management, all [42:01] these things, QC of course, put out the [42:04] product. I'm now going to reduce that [42:05] time frame to say 6 months, 3 months, [42:08] where it used to be two years. That [42:10] already is your ROI right there. There's [42:12] a medical device company that told me [42:13] one time, every day that we miss our [42:16] timeline for hitting the market, we're [42:18] losing a million dollars every single [42:20] day. And they were already delayed by 3 [42:22] months. So if I can help them, you know, [42:24] bring that back like closer. They're not [42:27] losing that money. Um so I think every [42:29] organization is different, but you can [42:31] definitely find these types of metrics [42:33] within the organization. [42:34] >> Uh I think we are at time. Thank you so [42:36] much panelists. I think this is a [42:38] wonderful discussion.