Pendo, Citizens Bank Shift AI Product Metrics to Task Resolution Rates

Pendo与Citizens Bank将AI产品指标转向任务解决率 产品负责人指出，自主AI智能体使传统的软件参与度指标不再适用，迫使工程团队严格将端到端任务完成率作为衡量成功的基准。 该专家讨论探讨了构建AI产品时所需的运营和战略转变，强调从衡量用户软件参与度转向评估自主任务完成率。来自受监管企业和面向消费者企业的负责人详细介绍了分层治理模型、持续评估基准闭环，以及在规模与数据敏感性之间权衡的自研与采购决策框架。他们还分析了诸如“针对否定场景设计”（designing for the no）等新兴AI交互模式、自动化工程工作流的兴起，以及现代AI产品经理角色如何通过快速原型制作和多模态反馈闭环被重新定义。 为AI智能体提供历史与实时的行为遥测数据如同共享屏幕，可将自主解决率提升高达30%。 (3:41) 受监管行业需要基于风险分级的准入与治理模型而非一刀切的审查，以区分低风险的摘要生成器与高风险的自主决策智能体。 (8:05) 现代智能体系统需要前置的故障模式工程化设计，为最坏情况设定不可妥协的通过/失败关卡以及明确的回退边界。 (14:05) 核心产品指标已从软件采纳率和参与度，转向自主任务完成率与端到端解决率。 (18:24) 持续改进依赖于跨产品、数据科学和工程团队进行基准对齐的严格评估套件，并将实时生产环境故障转化复用为新的测试用例。 (19:53)

Industry leaders at Ai4 argue that managing AI products requires abandoning traditional software development life cycles in favor of autonomous task-completion metrics, risk-tiered governance, and continuous evaluation loops. Across regulated and consumer domains, product success hinges on embedding behavioral telemetry, failure-mode safeguards, and disciplined evals from day one. Outcome Metrics and Continuous Evaluation Loops (18:24) Pierce Healy emphasizes that agentic AI shifts product evaluation from surface-level engagement to end-to-end task completion. In automated customer support (e.g., Intercom Fin, Sierra, Decagon), autonomous resolution rate serves as the primary metric. When companies bill clients per resolution, product performance directly mirrors business revenue. To systematically advance toward outcome goals, teams run "evals"—benchmarks pairing sample inputs with verified outputs and tool calls. Product managers define use-case boundaries, data scientists score agent runs, and software engineers modify models, prompts, and harnesses. Production monitoring detects live failure modes, recycling edge cases back into eval benchmark suites for subsequent development sprints. Risk-Tiered Governance and Non-Negotiable Guardrails (8:05) Devendra Parmar highlights the tension between shipping speed and organizational risk, citing a BCG-MIT study showing a 50% year-on-year increase in customer-impacting AI incidents alongside historic MIT findings that only roughly 5% of enterprise AI projects succeed. In response to heightened scrutiny—such as a tenfold expansion in OCC and FDIC regulatory publications since 2020—organizations must apply risk-based decisioning. Low-risk applications like internal summarizers follow lightweight paths, whereas autonomous agents executing high-risk financial or operational decisions trigger rigorous validation. Rachel Maness explains that Align Technology enforces this balance using a centralized intake framework that logs dynamic risk scores, approvals, and testing records. For non-deterministic agents, Anil Mathew Roy and Parmar note that "do no harm" guardrails—such as Fair Housing compliance at Zillow and anti-fraud protections in banking—serve as mandatory pass/fail gates that cannot be compromised for speed, cost, or latency. Build Versus Buy Economics and Telemetry (25:13) Deciding whether to build proprietary models or buy commercial APIs rests on task specificity, data sensitivity, and unit economics. While commercial APIs offer quick, affordable prototyping for generic tasks, serving tens of millions of users causes per-token costs to compound, justifying dedicated fine-tuned models. In banking, Parmar highlights divergent enterprise strategies: Capital One acquired Brex in 2026 for rapid agentic servicing, whereas JPMorgan Chase allocates roughly $18 billion annually to maintain proprietary infrastructure for sensitive fraud detection and anti-money laundering engines. At the integration layer, Healy notes that supplying agents with Pendo's historical behavioral telemetry rather than simple in-session clicks provides essential context, driving an average 30% increase in autonomous support resolution rates. Constrained UX Design and Feedback Diagnostics (31:49) As agentic tool harnesses reduce raw hallucinations, user friction increasingly stems from open-ended chat interfaces lacking structural affordances. To prevent dead ends, teams deploy hybrid interfaces combining chat with suggested prompt chips, starter templates, and structured selectors. Teams also "design for the no," replacing generic error messages with viable alternative actions or routing back to traditional interfaces. Cobus Kok and Roy observe that conversational logs function as real-time demand surveys. Because negative sentiment often stems from enforcing business constraints rather than technical failure—such as Priceline enforcing non-refundable hotel cancellations—teams pair binary feedback with qualitative root causes to isolate policy explanations from hallucinations or missing context. Critical systems retain seamless human-in-the-loop escalation paths when automation fails. Compressed PM Lifecycles and Autonomous Engineering (38:24) Team structures and workflows are fundamentally shifting. Parmar and Roy state that multi-month cycles of writing Product Requirement Documents and awaiting design hand-offs are being replaced by AI-assisted scoping and "vibe coding." Using tools like v0, product managers can build and test working prototypes with customers in as few as five days to secure stakeholder buy-in. Concurrently, engineering teams are trialing the "dark factory" model in low-risk internal environments, using autonomous agent squads to write code, conduct peer reviews, and manage deployments. This shift reframes software engineers from hands-on coders into managers who codify technical knowledge, set domain boundaries, and oversee modular agentic workflows. While AI-assisted tooling accelerates prototyping and execution, scaling autonomous agents remains bounded by dynamic failure modes and regulatory constraints. Product viability depends on whether organizations can maintain rigorous evaluation pipelines and clear escalation paths without eroding the economic benefits of automation.

在 Ai4 大会上，来自金融、医疗和科技行业的产品主管指出，人工智能产品管理正在从关注界面停留时间转向衡量自主任务完成率；构建智能体需要将全流程评估、分级治理与严格的安全底线深度结合。 核心指标从软件使用转向任务闭环 (18:24) Pierce Healy 指出，传统 SaaS 产品主要为人工操作提供交互界面，而智能体（Agentic AI）的核心转变为代表用户自主执行端到端任务。因此，团队绩效评估不再以软件使用时长或界面互动为中心，而是以任务完成度为核心依据。在客户支持等成熟应用场景中，核心北极星指标已转变为自主解决率（autonomous resolution rate），即无需人工客服介入即可成功解决问题的比例。当企业按解决结果向客户收费时，产品指标与商业收入直接挂钩。为了让智能体拥有充足的上下文，Pendo Agent Toolkit 将历史及同类群组行为遥测数据实时接入前端支持平台（如 Intercom Fin、Sierra 和 Decagon），在实际部署中将自主解决率平均提升了 30%。 评估基准与多维度指标管理体系 (19:53) 非确定性的人工智能系统无法仅依靠单一指标衡量，必须根据功能模块建立多维度指标组合：摘要任务关注真实性与完整性，搜索意图强调相关性，内容生成则衡量语气、风格与事实准确度。产品团队采用被称为“evals”的评测体系，由产品经理界定业务场景，数据科学团队构建基准测试集并打分，软件工程师根据评分调整模型、提示词、工具调用与编排框架。生产环境中检测到的真实失败案例会被持续回流，转化为新的评测套件以驱动工程迭代。此外，在涉及合规与隐私的领域，严禁跨越的防范风险（do no harm）指标属于上线前的非妥协门槛，必须优先于成本或延迟等优化考量。 分级准入机制平衡研发速度与合规风险 (8:05) Devendra Parmar 强调，过度追求交付速度会带来显著风险，并引用 BCG 与 MIT 的联合研究指出，对客户造成实质影响的人工智能事故同比激增了 50%。为防范偏见、幻觉和数据泄露，组织不能盲目套用传统软件研发周期（SDLC），而需推行基于风险权重的决策流程。Rachel Maness 介绍了 Align Technology 的统一准入框架：系统通过动态问卷评估项目风险，低风险应用（如内部文档摘要）进入轻量级验证，而高风险应用（如直接驱动业务决策的外部自主智能体）则必须通过端到端安全和合规审计，相关审批、归属与测试记录统一归档至可追溯的系统平台。Anil Mathew Roy 补充说明，团队应提前定义最坏情景作为不可逾越的淘汰关卡，在模型无法达成时执行严格的兜底限制。 自研与采购的权衡及交互体验演进 (25:13) 企业在自研专属模型或采购商用 API 之间，主要依据任务独特性、数据敏感性及规模化单体经济学三个维度进行权衡。对于通用的起草或分类任务，商用基础模型在早期具备高性价比，但在达到数千万用户量级时，微小 Token 成本差异会放大为数百万美元的开销，从而促使团队转向专用的小型微调模型；金融机构在反洗钱和欺诈检测等核心资产上则必须依靠多维内部数据自建专有模型。在交互设计层面，空白对话框往往容易引发超范围使用，团队正在通过预设提示词标签、交互式模版及混合 UI 引导用户预期。当智能体无法完成请求时，应采用“拒绝设计”（Design for the No）主动推荐合规替代方案，并将这类失败查询作为实时需求数据反馈给产品规划。 产品经理职能重塑与工程组织形态转变 (35:48) 人工智能工具正在显著缩短产品探索周期，产品经理不再依赖撰写冗长的 PRD 或是耗费数月等待设计交接。利用 v0 等生成式工具与即兴编码（vibe coding），产品经理可在短短五天内将创意转化为可测试的交互原型，以此获取业务方的确认。Devendra Parmar 指出，这一趋势推动了产品职能的去中心化，使深刻理解业务痛点的人才能够直接主导解决方案。在工程侧，团队开始低风险试水“黑灯工厂”（dark factory）模式，由自主智能体完成编写代码、同行审查到自动化部署的多阶段流程；工程师的角色因而从单纯的代码编写者转变为智能体资源的管理者，核心任务在于沉淀自身专业知识并规范智能体在模块化流水线中的执行边界。 与会者总结认为，随着生成式工具将产品原型周期压缩至数天并推动工程走向自动化编排，产品管理的本质挑战不再是技术交付本身，而是在业务规则限制引发负面情绪时清晰传达逻辑，并建立捕获深层归因的反馈闭环与可靠的人工升级兜底路径。

## Background 该视频由行业会议频道 Ai4 于 2026 年 9 月 8 日发布在 YouTube 平台。内容为一场关于“AI 产品管理”的专题圆桌讨论，汇聚了来自 Priceline、Zillow、Pendo、Align Technology 及 Citizens Bank 的多位资深产品与技术高管，共同探讨从需求规划、幻觉管理、评估体系搭建到平衡风控与创新速度的实战策略。 ## TL;DR 来自不同行业顶尖企业的嘉宾深入剖析了 AI 时代产品经理角色的深刻演进与落地方法论。核心议题涵盖了从传统功能交付向基于产出（如工单解决率）考核的范式转移，以及如何通过内嵌合规治理与 Evals 评测闭环来应对非确定性风险。针对自研与外购权衡、全新交互体验设计（如超越传统空白聊天框）以及监控用户挫败感与人机协同兜底，嘉宾们结合金融、房产与企业 SaaS 的真实案例提供了高密度的实操框架。 ## Atmosphere & style 整场圆桌讨论节奏紧凑、干货极多且充满高度实战色彩，既有消费级大厂在交互与原型验证上的敏捷反思，也有强监管金融与医疗企业在风控与治理上的严谨把控。主持人 Cobus Kok 幽默风趣地穿针引线，嘉宾们在观点抛接与相互补充中展现出深厚的行业一线经验与默契。现场互动热烈、切中痛点，真实还原了当下顶尖产品团队在探索 AI 前沿技术与业务变现平衡点时的紧迫感与务实探索氛围。 ## Key takeaways - [03:34](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=214s) 企业级 AI 平台落地需将合规与治理深度内嵌（Baked in）于底层架构中，而非作为事后补丁，以在安全可控的前提下持续集成最新模型。 - [03:34](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=214s) Pendo Agent Toolkit 将用户的实时行为数据转化为供给 AI Agent 的上下文与触发器，使客服工单平均解决率提升约 30%。 - [08:05](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=485s) 强监管行业中过度抢跑上线 AI 导致事故频发，企业应建立基于风险分级的准入与治理架构，平衡创新速度与合规安全。 - [16:35](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=995s) AI 风险应对逻辑需从防御已知问题转向最坏场景防御（Worst-case scenarios），将极端风险设为产品准入与熔断门禁。 - [18:16](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1096s) AI 时代产品评估范式从传统的工具使用（如采纳率、CSAT）转向任务结果达成（如工单解决率），并直接驱动按结果付费的商业模式。 - [19:53](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1193s) Evals（评测集）成为串联 PM、数据科学与工程的北极星指标，生产环境中的失败模式会反向沉淀为新的评测数据集以形成迭代闭环。 - [22:58](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1378s) 消费级与强监管 AI 产品需建立一票否决的“不作恶指标”（Guardrail metrics），在通用 LLM 与垂直微调自研模型间权衡敏感度与规模经济。 - [28:28](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1708s) 自研还是采购取决于业务壁垒：通用低风险场景优先采购第三方方案，反欺诈等涉及第一方专有数据与核心竞争力的业务则必须自研。 - [31:49](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1909s) AI 交互应走出单一空白聊天框，通过结构化控件（如 @工具、下拉选项）以及“为拒绝而设计”的引导来明确系统能力边界。 - [35:48](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2148s) 软件研发正探索无人值守的“黑灯工厂模式”，工程师角色从一线敲代码转变为调度与管理专业 AI Agent 团队的资源管理者。 - [41:35](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2495s) 产品经理借助 AI 编码与原型工具实现了从“写 PRD 需求文档”到“5 天自建原型并完成用户验证”的敏捷端到端闭环。 - [42:58](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2578s) 衡量用户挫败感依赖在线离线评估与多模态全渠道反馈聚合，结合点踩原因二次下钻及平滑的人工介入通道（Human in the Loop）。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=4s) 圆桌开场与嘉宾背景介绍 主持人 Cobus Kok（Priceline 产品管理副总裁）首先向观众致谢，并调侃道大家大老远坐自动扶梯上来专门探讨 AI 产品管理；如果问在场的五位嘉宾平时和 AI 交流的时间多还是和人类交流的时间多，得到的答案恐怕会和普通人大不相同。随后，各位嘉宾依次作了自我介绍： * **Anil Mathew Roy**：Zillow 负责 Agentic AI（智能体 AI）的高级产品经理，主要参与打造了 Zillow AI mode，通过对话式引导帮助用户更快找房，专注领域涵盖 Agentic Search（智能体搜索）以及驱动用户智能的个性化与情境模型。 * **Pierce Healy**：Pendo 负责 AI 产品的副总裁（曾创立客户情报平台 Zelda AI，后被 Pendo 收购）。他今年的核心重点是推广名为 Pendo Agent Toolkit 的新产品，帮助 Agent 引入客户情境（期间麦克风一度出现小故障，引来简短互动）。 * **Rachel Maness**：Align Technology（爱齐科技，以隐形矫治器产品 Invisalign 隐适美闻名）的 AI 赋能高级项目经理，专注于企业级 AI 战略与落地推广。 * **Devendra Parmar**：Citizens Bank（美国同胞银行）欺诈检测与告警管理产品高级副总裁，此前曾在 Capital One、Discover、State Street 及 HSBC 等多家知名银行积累了深厚的金融风控经验。 --- ### [03:34](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=214s) 实践案例：企业内部 AI 平台与上下文驱动的智能体工具箱 针对“近期构建了哪些 AI 产品、解决什么痛点及成效如何”的问题，Rachel Maness 与 Pierce Healy 分别分享了各自的落地案例。 Rachel 介绍了 Align Technology 面向内部全员搭建的企业级 AI 平台。该产品旨在解决两大核心痛点：一是现有的外部供应商软件许可（Licensing）无法跟上内部爆发的 AI 需求；二是员工渴望体验市场上最新涌现却未被企业内网引入的主流大模型。为此，团队打造了一个统一的内部交互界面，让员工能够安全合规地接入顶尖供应商模型。Rachel 强调，该平台的独特之处在于**将治理与合规组件深度内嵌（Baked in）于底层，而非事后补丁**，从而在满足合规底线的同时，持续根据员工反馈迭代集成前沿能力。 ```mermaid flowchart LR accTitle: Pendo Agent Toolkit 工作流 subgraph S1["用户行为采集"] A["点击与页面浏览"] B["历史全生命周期行为"] C["相似用户行为特征"] end subgraph S2["Pendo Agent Toolkit"] D["实时上下文与触发器引擎"] end subgraph S3["AI 客服产品"] E["Intercom Finn / Sierra / Decagon"] F["问题解决率提升 30%"] end A --> D B --> D C --> D D --> E E --> F ``` 随后，Pierce Healy 介绍了 Pendo 推出的 **Pendo Agent Toolkit**。Pendo 传统上用于追踪软件产品的行为数据（如点击、页面访问、定性反馈等），过去这些数据仅用于效果复盘或排查产品摩擦点；但在 AI 时代，行为数据转变成了供给 AI Agent 的“上下文（Context）和触发器（Triggers）”。以集成 Intercom Finn、Sierra、Decagon 等 AI 客服产品为例，AI 代替真人客服的最大瓶颈在于无法实时获知用户在应用内的具体操作。Pendo Agent Toolkit 充当了实时上下文的桥梁——就像给 AI 客服开启了“屏幕共享”： * **全维度行为洞察**：AI 不仅能看到用户当前会话的操作和遇到的报错，还能调取该用户乃至同类画像用户的全部历史行为轨迹。 * **显著业务回报（ROI）**：实际推向市场后，该方案使客户支持问题的平均解决率（Resolution Rate）提升了约 **30%**，直接大幅减少了升级转人工的工单量，为企业带来了切实的利润贡献。 --- ### [08:05](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=485s) 速度与风控的博弈：强监管行业中的 AI 治理与业务价值 面对“AI 虽能极速推进软件开发与创意验证，但强监管行业该如何平衡速度、品牌声誉与安全风险”的提问，Devendra Parmar 指出这正是银行业与医疗等领域当前的核心矛盾。 Devendra 援引 BCG 与 MIT 的联合研究指出，2025 年企业因 AI 事故导致客户负面影响的案例同比激增了 **50%**，反映出行业内一味抢跑上线 AI 所带来的代价。从产品领导者的视角来看，"客户会原谅你晚一点交付一款有价值的产品，但绝不会原谅你泄露隐私数据，或者在欺诈拦截时错误阻断合法交易、破坏正常体验"。自 2020 年至今，来自 OCC、FDIC 等监管机构针对银行业的规范要求激增了 **10 倍（10x）**。因此，传统 SDLC 流程不再适用，AI 产品需要全生命周期内嵌控制措施。 ```mermaid flowchart TD accTitle: 基于风险分级的 AI 需求准入与治理架构 A["业务提出 AI 需求<br>(Intake 流程)"] --> B{"自动化风险定级评估"} B -- "低风险<br>(如内部信息摘要/内部问答机器人)" --> C["简化治理与轻量验证<br>(即使宕机也不伤及核心业务)"] B -- "高风险<br>(如面向客户/涉及信贷与风控决策)" --> D["严格验证与合规审查<br>(幻觉检测、偏差评估、审计存证)"] C --> E["发布上线与全生命周期监控"] D --> E E --> F["统一中央记录系统<br>(审批链、所有权、测试与维护记录)"] ``` Devendra 进一步强调，组织在推进时需做好三维度的平衡： 1. **基于风险的分级决策（Risk-based decisioning）**：避免对所有场景一刀切。诸如简单的文本摘要或内部推荐等低风险用例无需背负过重的严苛治理包袱，而高自主性的 AI Agent 则必须从零开始筑牢防线。 2. **避免陷入纯技术/算法误区**：AI 落地不是单纯的数据科学或技术工程难题，归根结底是业务价值变现的问题。MIT 此前的报告显示仅有约 **5%** 的 AI 用例最终取得成功，主要原因往往在于团队过度纠结于治理或交付速度，却脱离了底层真正要解决的业务痛点。 3. **内嵌治理替代事后评估**：必须在立项之初就规划好防幻觉与防偏见机制，而非交付前夕才仓促检查。 针对该话题，Rachel 补充了 Align Technology 的做法。他们将“创新速度”与“风险管理”视为协同并进的抓手，建立了标准化 **Intake（准入评估）流程** 作为中央记录系统（System of record）。系统根据员工提交的应用形态（无论是内部自研还是集成第三方）自动判定风险等级： * **低风险场景**：如内部员工查询知识库的聊天机器人，哪怕下线也不会瘫痪业务，走极简流程快速上线； * **高风险场景**：涉及对外输出或直接驱动商业决策的 Agent，必须完成完整的模板化验证与合规审批。 所有权归属、审批记录、验证测试与后续维护全部在一个中心化平台留痕，既避免合规团队沦为业务研发的“拦路瓶颈”，又消除了数据孤岛。 --- ### [16:35](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=995s) AI 风险演进：从固定挑战到最坏场景防御 讨论延伸到近几年 AI 风险格局的变迁。Cobus 提到三年前 ChatGPT 刚问世时，企业面临的问题相对有限且单一，如今则复杂得多。 Anil Mathew Roy 结合 Zillow 的实践进行了剖析。他指出，过去行业的痛点往往是可预测的固定问题，而当下衍生出了无数未知的边缘挑战（N challenges）。现代 AI 产品经理的应对逻辑应当是：**从用户视角出发，穷举可能对品牌与用户造成毁灭性打击的最坏场景（Worst-case scenarios），并将这些最坏情况直接设定为产品的测试用例与准入/熔断门禁（Pass/fail gates）**。 在架构设计上，除了保障核心路径必须通过安全门禁外，产品必须为 AI 设立防御性兜底标准。面对自身无法明确把握、超出安全边界的意图时，模型应直接明确表达“抱歉，我无法协助处理该问题”，从而有效收敛未知风险。 --- ### [18:16](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1096s) 重塑评估体系：从工具交付转向结果任务达成 当谈及跨职能团队的协同与 KPI 制定时，Pierce Healy 明确指出，AI 时代对产品团队的考核机制正在经历根本性范式转移。 传统的评估维度（如用户采纳率 Adoption、留存率 Retention、CSAT 客户满意度等）依然重要，但根本区别在于：**以往软件只是提供工具辅助员工干活，而如今的 AI 往往是在直接代替人类交付工作成果**。因此，衡量产品团队成功与否的标准，直接变成了底层任务的最终完成度（Outcome completion）。以成熟的 Agentic 赛道（如 AI 智能客服 Finn、Sierra、Decagon 等）为例，行业的核心衡量锚点已经全面统一转向了任务的“解决率”（Resolution Rate）。 ### [19:53](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1193s) 产品 KPI、定价与评估体系（Evals）闭环 Pierce Healy 展开探讨了以产出（Outcomes）为导向的衡量方式，并举了一个极具启发性的案例：在某些前沿客户服务产品中，团队的核心产品 KPI 既不是传统的采纳率（Adoption），也不是用户交互体验（UX），而是非常纯粹的一个指标——“AI 智能体是否彻底解决了该工单（Did the agent solve the ticket）”。更为罕见的是，该指标直接与商业模式绑定：他们按照每个解决的工单（Per resolution basis）向客户收费。这是软件领域极少数产品核心 KPI 与商业定价模型完全重合的典范，使整个团队在统一的目标下高效协作。 为确保 AI 智能体稳定达成预期产出，业界正在凝聚一种全新的流程标准——模型评测（Evals）。所谓 Evals，核心在于确定一套代表性的输入用例集合（例如用户在聊天界面中可能输入的各种问题），并对应制定公认的标准正确输出或智能体应当采取的行动步骤。产品团队在对模型、工具调用、上下文窗口（Context）或外围编排脚手架（Harness）进行任何改动时，都必须以此基准集进行测试，将 Evals 得分的提升作为衡量改进的唯一标尺。 在组织协同层面，Evals 成为拉通产品、数据科学与工程团队的“北极星”： - **产品经理（PM）**：负责界定属于评测范围内的具体业务场景与用例（Use cases in scope）； - **数据科学团队（Data Science）**：负责构建具体的 Evals 数据集，并制定打分评估机制； - **工程团队（Engineering）**：负责调整代码、提示词与系统架构，全力拉升评估分值。 Pendo 正在将长期积累的软件使用分析能力扩展至 AI 领域，推出了“智能体分析（Agent Analytics）”产品。其核心机制是在生产环境中实时监控智能体是否完成了预定任务（Job to be done），一旦出现失效或报错，系统能精准归纳失败模式与原因。这一线上监控环节并非终点，而是与 Evals 形成正向反馈闭环：生产环境中暴露的盲区与失败案例，会直接反哺转化为全新的评测集，供工程团队作为下一个迭代周期的攻坚目标。 ```mermaid flowchart TD accTitle: AI Agent 评估与生产监控闭环 subgraph 定义与评测阶段 A["产品经理 (PM)<br/>定义业务范围与用例"] --> B["数据科学团队 (DS)<br/>构建评测集与打分机制"] B --> C["工程团队 (Eng)<br/>迭代模型与工具提升跑分"] end subgraph 生产运行阶段 C --> D["生产环境部署<br/>面向最终用户运行"] D --> E["Agent Analytics 监控<br/>追踪任务完成度与失败模式"] end E -->|"提取失败场景与未覆盖需求"| B ``` --- ### [22:58](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1378s) 消费级 AI 指标组合与“不作恶”红线护栏 针对 Pierce 的分享，Anil Mathew Roy（来自 Zillow）从消费级 AI 产品的视角补充了不同维度。他指出，面对聊天智能体等具备非确定性（Nondeterministic）特征的产品，很难依赖单一的北极星指标，团队需要管理的是一个“指标组合（Portfolio of metrics）”。不同的任务类型需要度量完全不同的维度： - **文本摘要（Summarization）**：侧重评估真实性/保真度（Faithfulness）与完整性（Completeness）； - **搜索意图识别（Search Intent）**：侧重评估搜索结果的相关性（Relevance）以及对用户意图的精准捕捉； - **内容起草（Drafting）**：侧重把控语言语气（Tone）与输出准确性（Accuracy）。 针对不同垂直行业的敏感程度，现场进行了举手互动，发现大量参会者来自医疗、金融或房地产等强监管敏感行业。Anil 强调，敏感行业的产品必须建立一层不可妥协的护栏指标——“不作恶指标（Do no harm metrics / Guardrail metrics）”。以房地产领域为例，系统除了要严格防范隐私泄露外，还必须严格遵守《公平住房法》（Fair Housing Act），防止算法在推荐或回答中产生歧视性偏见。这类护栏指标在团队中拥有绝对的一票否决权：无论其他业务指标表现多好，一旦触犯护栏便无法上线。 在自研与采购的权衡上，Anil 给出了一个实用的决策框架。他建议产品经理从三个核心维度进行评估： 1. **用例专用性（Use Case Specificity）**：业务需求究竟是通用的还是行业垂直深度的？ 2. **数据敏感度（Data Sensitivity）**：交互涉及的数据是否包含高度机密或强监管的隐私信息？ 3. **规模化经济效益（Economics at Scale）**：在用户量达到数百万级别时，调用通用 API 与自建专有模型之间的成本差异是否会放大到数百万美元？ 对于起草、摘要等通用能力，现成的商业前沿 LLM 在成本和体验上均有明显优势；但针对敏感数据、强监管场景，或在超大规模调用下，构建垂直微调模型才能同时兼顾合规、数据安全与单位经济效益。Anil 总结道，产品经理的直觉只能用来决定“先测什么”，最终决策绝不能依赖直觉，必须通过严谨的 A/B 测试，在成本、延迟与效果指标组合之间做出数据驱动的取舍。 --- ### [28:28](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1708s) 金融与强监管行业的自研与外购抉择 Devendra Parmar（来自 Citizens Bank）从金融体系的角度深入探讨了自研还是外购（Build vs. Buy）的深层考量。他指出，这一选择本质上是由企业整体战略和技术预算格局驱动的。行业中存在截然不同却均获成功的路线： - **并购整合路线**：例如 Capital One 在 2026 年收购 Brex，通过资本运作直接获取了成熟的商业银行 AI 智能体服务能力； - **自主重度研发路线**：摩根大通（JPMorgan Chase）每年拥有高达约 180 亿美元的技术预算，其中重金用于自建 AI 基础设施与专有模型平台，以谋求极致的自主控制力。 在具体业务落地中，用例的差异决定了技术路径： - **通用低风险用例**：若目标是快速验证概念与缩短上市时间（Time to market），企业应优先采购成熟的商业 API 或第三方厂商方案，快速摸清“可能性边界（Art of possibility）”； - **高壁垒专有业务（如反欺诈与反洗钱）**：以欺诈检测（Fraud Detection）模型为例，系统必须深度融合第一方的客户画像数据、实时交易数据、商户信息以及金融网络层面的底层特征。这种高度组织专有且复杂的智能逻辑无法依靠通用基础模型解决，必须全力自研，这也是银行构建差异化竞争护城河（Competitive edge）的核心资产。 Devendra 提醒大家不要低估自研模型的真实代价：自研绝非仅仅是写代码训练一个模型，它涵盖了企业内部合规数据集的搭建与清洗、持续的模型训练调优、端到端集成测试，以及与整个遗留应用生态系统的无缝兼容和长期运维支持。 --- ### [31:49](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1909s) AI 交互体验设计：走出聊天框与“为拒绝而设计” 谈及 AI 幻觉（Hallucination）时，Pierce 直言，随着当前应用逐步引入复合智能体架构（Agentic steps）和严密的校验流，纯粹的模型胡言乱语在技术层面已大幅缓解。当前更严重的交互痛点在于：用户常常对 AI 提出它根本无法处理、或者设计之初就未包含的诉求，进而产生“AI 给了错误答案”或“AI 不听指挥”的挫败感。 Pierce 指出，传统的图形用户界面（GUI）通过层级菜单、下拉列表等结构化元素清晰传达了系统的能力边界，而开放式聊天界面（Chat Interface）只留给用户一个完全空白的输入框，这种极简表面下隐藏着巨大的信息不对称——用户完全不知道这个智能体的能力起点与终点在哪里。为了解决这一设计缺陷，团队需要从以下几个方向优化 UX： - **主动提供建议（Suggested Prompts）**：在空状态时提供显式的 Prompt 选项或功能使用手册（Recipe book），降低用户的探索门槛； - **引入结构化控件（Structured Input）**：不能完全依赖纯文本。Pierce 称赞了 Claude 桌面端的交互演进，其支持使用 `@` 唤出工具列表、结合下拉选项器以及直观的选项选择器，将自然语言与确定性 UI 有机融合； - **为拒绝而设计（Designing for the no）**：当系统必须拒绝用户时，直接冷冰冰地回复“我无法协助”是最差的体验。优雅的做法是主动建议系统能完成的替代操作，或是平滑地将用户导流至应用中能够解决该问题的传统功能模块。 在总结聊天界面的价值时，Pierce 提出了一个独特的视角：聊天框本质上是一个“全年 365 天、每天 24 小时无休的用户调研通道”。用户在输入框中打出的每一个超出预期的请求，都在真实反映他们期望产品具备的能力。这不仅是用于防御滥用的监控日志，更是产品经理挖掘下一代产品路线图（Roadmap）的无价宝库。 --- ### [35:48](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2148s) 观众提问：AI 时代的产品研发团队架构与“黑灯工厂” 在观众提问环节，一位参会者提出了一个尖锐的组织架构问题：在 AI 产品的冲击下，原先“1 位产品经理配备 12 位工程师”的经典研发配比是否正在被颠覆？团队规模是在向 1:1 急剧收缩，还是在衍生出全新的形态？ Rachel Maness（来自 Align Technology）结合团队实践给出了深刻的洞察。她提到了工程界近期兴起的一个新概念——“黑灯工厂模式（Dark Factory Model）”： - **定义与探索**：借鉴制造业无人值守的黑灯工厂理念，在低风险的内部场景中，尝试将原本由人类承担的代码开发、代码质量审查（Quality Review）以及自动化部署流水线，全流程交付给一组协同工作的“智能体团队（Agent teams）”来运转； - **工程师角色的本质转变**：工程师的角色正从原先“双手紧贴键盘敲代码（Hands on the keyboard）”的传统独立贡献者（IC），向“资源管理者（Resource Managers）”转型。而他们所管理的下属资源，正是一个个专业 AI 智能体； - **组织进化的现实**：企业当前的重组重心不是立刻裁减或取代工程师，而是对资深工程师进行技能重塑——培训他们如何将高超的工程技艺、业务理解与质量标准沉淀下来，教导并指挥这些垂直领域的智能体协同工作； - **架构最佳实践**：在实践中，最有效的组织形式是角色职责严格隔离（Segregating roles），让多个智能体在各自明确定义的业务边界内深耕互联，而不是寄希望于一个全知全能的超级模型。 ### [38:24](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2304s) 问答环节：AI 时代产品经理角色与团队配置的演进 在 Rachel 阐述完资源管理与人效提升后，现场观众提出了一个核心问题：在各大企业所推进的 AI 项目中，究竟配备了多少名产品经理（PM）？这一角色究竟仍是传统的产品管理职能，还是已经发生了实质性转变？ ```mermaid flowchart TD accTitle: AI Product Management Evolution subgraph Traditional["传统产品流程 (耗时数周/数月)"] A["撰写繁杂 PRD 规格文档"] --> B["移交设计团队出交互图"] B --> C["等待工程排期研发"] C --> D["耗时长周期的外部用户测试"] end subgraph AIEra["AI 赋能下的端到端新模式 (5天极速验证)"] E["AI 工具辅助捕获需求与构想"] --> F["PM 利用 AI/轻代码直接生成原型"] F --> G["快速导入真实用户测试与验证"] G --> H["全团队获得高确定性并快速推进"] end ``` 针对这一问题，Devendra Parmar 结合自己在 Capital One 和 Discover 等金融机构的经历率先分享。他直截了当地指出，AI 产品管理本身正在快速演化，绝对不再是传统的产品管理模式。它不仅要求 PM 具备专业技能去深入理解 AI 与算法模型，还要充当纽带，将工程研发、数据科学/分析团队与客户的实际痛点紧密串联起来。 更为深刻的是，整个 AI 产品的生命周期自身也在被 AI 重塑。以往产品经理需要耗费大量时间撰写繁杂的产品需求文档（PRD）并等待漫长的概念验证（PoC），而在当下，PM 可以直接借助各种前沿 AI 工具自动化梳理需求、搭建 PoC 并展开高频试验，把过去动辄数月甚至更久的探索周期压缩到极致。Devendra 进一步强调，如今组织中的每个人都可以戴上“产品经理的帽子”——即使完全不会写代码，只要能够精准洞察业务目标与用户痛点，就能借助现成的 AI 工具发挥产品领导者的作用。 - **技能维度的专业化**：AI PM 需要跨越工程、数据科学和算法建模之间的壁垒，深度结合模型能力与用户场景。 - **全民产品思维**：AI 降低了技术门槛，使得非技术背景的业务人员也能借助智能工具快速探索产品解决方案。 --- ### [41:35](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2495s) 从“写文档”到“做原型”：产品构建者的敏捷闭环 Anil Mathew Roy 随后补充了一个独特的产品人视角。他认为，虽然产品构建者的根本使命（即定义核心问题）始终未变，但执行该使命的具体方式却发生了翻天覆地的变化。 以往的产品经理往往停留在文字层面，写出大篇幅的规格说明书（Specs/PRD）后移交给设计团队去出图；而现在，借助各类 AI 编码与原型生成工具，产品经理完全有能力在没有外部协助的情况下，把一个想法在极短时间内直接落地为可交互的原型。Anil 分享了自己团队的真实工作流：从最初的创意萌芽，到直接构建出原型并投入真实用户研究测试，整个闭环仅仅用了 5 天时间。这种快速拿到真实反馈并完成验证的做法，为跨职能团队带来了巨大的信心与认同感。产品经理已经从单纯的“需求定义者”，转变为能够独立端到端（End-to-End）闭环推进的高效产品构建者。 --- ### [42:58](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2578s) 衡量用户挫败感与消费级 Agent 的可观测性挑战 在随后的提问中，来自 Y Combinator（专注于构建 Agent 意图与挫败感分析工具）的创始人 Trishala 抛出了一个直击 C 端 Agent 痛点的问题：面对复杂的对话场景，像 Priceline 和 Zillow 这样打造消费级 AI Agent 的企业，究竟是如何衡量用户的“挫败感（Frustration）”的？现有的大模型分析与可观测性（Observability）工具是否足以还原用户卡壳、迷茫或犹豫的全貌？ ```mermaid flowchart LR accTitle: User Frustration Detection and Escalation Loop subgraph MultiChannelData["多源反馈数据输入"] A["多模态/全渠道数据 (呼叫中心、社媒等)"] B["会话评估 (Sentiment 变化分析)"] C["显式反馈 (Thumbs Down + 具体原因)"] end subgraph AIEngine["智能化洞察与闭环"] D["AI 聚合与聚类分析 (提炼最大痛点)"] end subgraph Resolution["解决与兜底路径"] E["产品功能与 Prompt 优先迭代"] F["人工介入通道 (Human in the Loop)"] end MultiChannelData --> D D --> E D --> F ``` Cobus Kok 首先从 Priceline 的实践切入，指出**离线与在线评估（Evaluations）**是核心抓手。对话交互蕴藏着海量上下文，团队必须持续评估这些对话轨迹，精准捕捉系统行为与用户预期不一致的拐点，并实时监测用户的“情绪（Sentiment）”在对话进程中是如何发生波动的。 Cobus 还分享了在线旅游业务中的一个经典矛盾场景：Agent 经常被要求去取消“不可退款的特价酒店房间”。从业务规则来看，AI 无法违规退款，因此对于用户来说结果必然是负向的；此时 Agent 的职责不仅是拒绝，更要在对话中清晰、有条理地解释政策原委，让用户在被合规限制的同时理解原因，从而将负面情绪降至最低。 - **评估驱动（Evaluations）**：从海量会话日志中挖掘模型回答不及预期的长尾案例，监控整场对话的情绪走向。 - **合规与体验的平衡**：在面对“不可退款房型取消”等天然负向场景时，Agent 必须以清晰易懂的沟通履行政策解释职责。 --- ### [44:43](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2683s) 多模态全渠道归因、点踩闭环与人工兜底安全网 针对挫败感的捕获与归因，Devendra Parmar 和 Anil Mathew Roy 展开了更深层次的方案拆解： 1. **多模态与全渠道情报捕获（Multimodal Feedback Loop）**：Devendra 提出，现代多模态大模型能够摄取多种格式的数据。用户其实无时无刻不在提供反馈——包括客服呼叫中心的录音、社交媒体（如 LinkedIn、Facebook）上的吐槽或点赞。企业应当建立自动化智能管道，将多渠道数据汇聚成结构化反馈，持续驱动产品迭代。 2. **显式反馈与二次原因下钻**：Anil 指出，传统的“点赞/点踩（Thumbs Up / Thumbs Down）”依然是大规模收集负反馈最轻量有效的手段。但在实践中，单纯的“点踩”并不能直接定位病因。因此他们设计了追问机制，探寻用户挫败的具体原因：“究竟是因为缺乏个性化推荐？还是没有正确理解上下文？亦或是模型出现了幻觉（Hallucinated）？”随后，团队再使用 AI 对收集到的用户评论进行聚类摘要，提炼出排在最前列的共性痛点，按优先级逐个击破。 3. **人工兜底通道（Human in the Loop）**：Cobus 补充了最后的安全防线。当 AI Agent 无法满足用户需求或用户情绪极度恶化时，系统必须提供平滑升级（Escalation）给人工客服的通道。建立这种人机协同的安全网，对于保障消费级 AI 产品的信任度至关重要。 ## Notable quotes > [03:34](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=214s) 将治理与合规组件深度内嵌（Baked in）于底层，而非事后补丁 — Rachel Maness > [08:05](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=485s) 客户会原谅你晚一点交付一款有价值的产品，但绝不会原谅你泄露隐私数据，或者在欺诈拦截时错误阻断合法交易、破坏正常体验 — Devendra Parmar > [16:35](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=995s) 从用户视角出发，穷举可能对品牌与用户造成毁灭性打击的最坏场景（Worst-case scenarios），并将这些最坏情况直接设定为产品的测试用例与准入/熔断门禁（Pass/fail gates） — Anil Mathew Roy > [18:16](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1096s) 以往软件只是提供工具辅助员工干活，而如今的 AI 往往是在直接代替人类交付工作成果 — Pierce Healy > [19:53](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1193s) AI 智能体是否彻底解决了该工单（Did the agent solve the ticket） — Pierce Healy > [22:58](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1378s) 产品经理的直觉只能用来决定“先测什么”，最终决策绝不能依赖直觉，必须通过严谨的 A/B 测试，在成本、延迟与效果指标组合之间做出数据驱动的取舍 — Anil Mathew Roy > [31:49](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=1909s) 聊天框本质上是一个“全年 365 天、每天 24 小时无休的用户调研通道”。用户在输入框中打出的每一个超出预期的请求，都在真实反映他们期望产品具备的能力 — Pierce Healy > [35:48](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2148s) 工程师的角色正从原先“双手紧贴键盘敲代码（Hands on the keyboard）”的传统独立贡献者（IC），向“资源管理者（Resource Managers）”转型。而他们所管理的下属资源，正是一个个专业 AI 智能体 — Rachel Maness > [38:24](https://www.youtube.com/watch?v=tv3a8lvAZa8&t=2304s) 如今组织中的每个人都可以戴上“产品经理的帽子”——即使完全不会写代码，只要能够精准洞察业务目标与用户痛点，就能借助现成的 AI 工具发挥产品领导者的作用 — Devendra Parmar ## Mentioned Cobus Kok, Rachel Maness, Devendra Parmar, Anil Mathew Roy, Pierce Healy, Trishala, Priceline, Align Technology, Invisalign, Citizens Bank, Zillow, Pendo, Zelda AI, Pendo Agent Toolkit, Zillow AI mode, Intercom Finn, Sierra, Decagon, Capital One, Discover, State Street, HSBC, JPMorgan Chase, Brex, BCG, MIT, OCC, FDIC, Y Combinator, Claude, Fair Housing Act ## Video description Cobus Kok, Vice President, Product Management, Priceline Rachel Maness, Sr. Program Manager, AI Enablement, Align Technology Devendra Parmar, SVP, Head of Fraud Products, Citizens Bank Anil Mathew Roy, Senior Product Manager, Agentic AI, Zillow Group Inc. Pierce Healy, VP, AI Products, Pendo How do you build AI products most effectively? In this session, top PMs and technology leaders share how they navigate the unique challenges of AI product development—from the planning phase to managing hallucinations to balancing control and automation. Learn how experts are shaping user expectations, iterating on complex ML-driven systems, and aligning stakeholders behind responsible AI product strategy. You will leave this panel with a clear idea on how to build the next great AI powered product for your business! [Source](https://www.youtube.com/watch?v=tv3a8lvAZa8)

[3:34] Rachel Maness: 将治理与合规组件深度内嵌（Baked in）于底层，而非事后补丁 [8:05] Devendra Parmar: 客户会原谅你晚一点交付一款有价值的产品，但绝不会原谅你泄露隐私数据，或者在欺诈拦截时错误阻断合法交易、破坏正常体验 [16:35] Anil Mathew Roy: 从用户视角出发，穷举可能对品牌与用户造成毁灭性打击的最坏场景（Worst-case scenarios），并将这些最坏情况直接设定为产品的测试用例与准入/熔断门禁（Pass/fail gates） [18:16] Pierce Healy: 以往软件只是提供工具辅助员工干活，而如今的 AI 往往是在直接代替人类交付工作成果 [19:53] Pierce Healy: AI 智能体是否彻底解决了该工单（Did the agent solve the ticket） [22:58] Anil Mathew Roy: 产品经理的直觉只能用来决定“先测什么”，最终决策绝不能依赖直觉，必须通过严谨的 A/B 测试，在成本、延迟与效果指标组合之间做出数据驱动的取舍 [31:49] Pierce Healy: 聊天框本质上是一个“全年 365 天、每天 24 小时无休的用户调研通道”。用户在输入框中打出的每一个超出预期的请求，都在真实反映他们期望产品具备的能力 [35:48] Rachel Maness: 工程师的角色正从原先“双手紧贴键盘敲代码（Hands on the keyboard）”的传统独立贡献者（IC），向“资源管理者（Resource Managers）”转型。而他们所管理的下属资源，正是一个个专业 AI 智能体 [38:24] Devendra Parmar: 如今组织中的每个人都可以戴上“产品经理的帽子”——即使完全不会写代码，只要能够精准洞察业务目标与用户痛点，就能借助现成的 AI 工具发挥产品领导者的作用

[3:41] Pierce Healy: sharing a screen with a support person [14:05] Anil Mathew Roy: I can't help you with that [18:24] Pierce Healy: From Tool to Task Execution [22:58] do no harm [28:28] Devendra Parmar: art of the possible [31:49] Design for the No [31:49] 365, 24-hours-a-day survey [35:48] dark factory [35:48] hands on the keyboard

[0:04] Hi everyone. Thank you so much for joining us  today. Um and you know taking the trip up the [0:09] escalator to speak about product management,  AI product management specifically. Um we're [0:15] lucky to have a panel of experts with us  today who live and breathe this every day. [0:21] Um, you know, I'm sure if you ask the five of  us, how much time are we spending talking to [0:26] AI versus humans, you're going to get different  answers from, you know, the average person. Um, [0:33] we live and breathe this every day. And I think  we have a good perspective on what's happening [0:37] in the industry. And hopefully we'll be able to  share some of our um, experiences and learnings, [0:42] you know, as you know, in this very dynamic and  and quickly evolving landscape. So, first of all, [0:49] you know, I'm going to give each of our panelists  the opportunity to introduce themselves and tell [0:55] you a little bit about, you know, what they  do um and what they're um building. And then [1:00] we're going to, you know, jump into questions.  I would ask we're going to leave Q, you know, [1:05] time for Q&A um until the end. We're going to have  about 10 15 minutes of of Q&A. So, don't you know [1:11] um please hold your questions um until that time.  So to um kick us off, I'm going to you know ask [1:18] Anil to introduce himself. Hey everyone.  Um happy to be back at AI4. Uh I'm Anil, [1:25] a senior AI PM um on agentic AI at Zillow. Uh  I'm one of the product managers contributing [1:31] to the launch of Zillow AI mode. Uh kind of  our conversational guided way to get people [1:36] home faster and help them search for homes. uh  my focus is on like agendic search uh and like [1:42] building personalization and contextual models  that power the user intelligence that you guys [1:47] might have used while using Zillow. So uh happy to  be here. Fierce. Cool. Um hey everybody I'm Pierce [1:56] Cely CT VP of AI products at window which is a  product analytics platform for apps and agents. [2:04] Um I was previously vendor of Zelda AI which  is a customer intelligence platform acquired [2:09] by Pendo a few years back. Um and my top  priority this year is to grow a new product [2:16] we have called Pendo agent toolkit which I  spent some time talking about today but is [2:21] really about how to bring customer context to  improve the effectiveness. So I miss you Mike. [2:33] Okay. You hear me? No. [2:39] Okay. I'll tell you I'll fill you in more on  that later on. Hey everyone. Uh my name is [2:46] Rachel Manis. I am a senior program manager of AI  enablement at Align Technology. Uh if you haven't [2:54] heard of Align Technology, you probably heard  of our main product, Invisalign. Um I focus on [3:00] AI strategy and driving AI adoption for the  enterprise. All right. Uh very good evening [3:08] everyone. Uh my name is Dendra. I'm a senior vice  president at uh Citizens Bank. Uh I lead uh uh the [3:14] product development within the fraud detection  and alerts management space. Um I've worked [3:20] on multiple other banks including Capital One,  Discover, State Street and HSBC Bank and pretty [3:26] excited to be here and sharing u the knowledge  with Right. Well, thank you very much everyone. [3:34] Um we're going to start off with um a question  you know um to Rachel specifically. Um Rachel, [3:41] can you walk us through an AI powered product that  you built recently and you know tell us a little [3:47] bit about what's the customer problem you solved  or try to solve you know what metrics did you [3:52] uh set out to impact and you know how did  it go? M check. All right. So, one example [4:00] of how we're driving AI at scale is a product  that we created for our employees focused on [4:09] uh AI platform. Um so it's a platform for our  enterprise and it's really solving two problems. [4:16] The first is we found that current licensing  with different vendors wasn't keeping up with [4:23] AI demand. And we also found that employees were  wanting to use AI models that weren't currently [4:30] available within our enterprise. So we developed  a solution, an AI platform internally uh that [4:37] served as a user interface where employees could  safely and securely access multiple AI models, [4:43] the top leading providers that you all are  familiar with today. And we also baked in [4:49] additional functionalities and features and  capabilities that are on the market as well [4:55] as what's up and coming. So we're continuing to  improve and enhance this platform so people can [5:00] take advantage of the leading functionalities,  capabilities, and AI models that you see on the [5:05] market today. So this is a a really I think unique  product because the governance component is baked [5:15] into it. It's not an afterthought. So looking at  uh compliance, governance, guard rails, those were [5:23] all key requirements and honestly was one of the  issues that we were trying to solve when we built [5:33] this product. And we continue to take employee  feedback and what they're looking for, what they [5:41] see in the market, what we don't have available  to the enterprise, and we're incorporating that as [5:46] future enhancements. Great. Thank you for sharing  that, Rachel. Peter, you're in a very different [5:53] industry. Do you have an example you can share?  Yeah. Yeah. And um I don't know if anyone heard [5:57] me the first time, but we work for Pendo. It's  a product analytics platform. Um, and one of the [6:02] new releases this year we're particularly proud  of is something called Pendo agent toolkit. Um, [6:08] the idea here is so Pendo tracks um, basically  behavioral data from your your software products. [6:14] So all of the um, clicks and page views and all  the interactions that users are having with an app [6:19] um, in addition to any qualitative data they're  sharing with you. And we're used to thinking about [6:23] this data as something that we use for tracking  success or research or finding where there's [6:28] friction. Um, but in the new AI enabled world,  we found a really great new opportunity for data, [6:33] which is to actually act as contexts and  triggers for agents that you're building. Um, [6:39] so an an example to bring this to life is we  have integrations between Pendo and a lot of [6:44] the AI customer support products. So the likes  of Intercom Finn or Sierra or Decagon that you [6:49] might have heard of, these are those chat widgets  that you see that, you know, replace uh talking [6:54] to a human about support. And one of the blockers  that those products have with like fully replacing [7:00] a human is like visibility into what the user is  actually doing in an application. Um, so Pendo's [7:05] agent toolkit can act as that bridge. And you can  almost imagine this like sharing a screen with a [7:10] support person. Um, it's not just what you did in  this session. It's not just the issues or problems [7:15] you faced uh today. It's all of the behavior that  you've ever had and all the behavior other users [7:19] like you have had. Um and we found so far rolling  this out it boosts resolution rate by like 30% um [7:25] kind of on average which is really direct ROI for  businesses. If you are investing in an AI support [7:32] program and 30% less tickets you're getting  escalate to human that's direct impact on bottom [7:37] line. So we're pretty excited about that product.  Um support's only the first uh kind of wedge as [7:43] we think about it. We think that all agents need  customer context to really perform well and that's [7:48] something we're we're making a big bet on. So  excited about that. Great. Thank you. Um so we've [7:53] heard about two you know interesting products in  different industries. You know Rachel being in [7:58] in medical devices um you know Pierce working you  know actually for product managers and for teams. [8:05] Le let's talk a little bit about you know more  you know regulated or you know industries with [8:11] um different you know sort of guard rails and  and and um risks attached. um Deendra you know [8:20] there there's like massive pressure on companies  across industries to ship new products very you [8:26] know quickly and you know AI actually enables  one to do that um it sort of changes the rules [8:33] of the game in terms of how quickly you can you  know write software how quickly you can ideulate [8:39] on new ideas how quickly you know you can um test  new features how do you think about balancing you [8:47] know that you know the the the the brand and and  reputational and security risks that come along [8:52] with that with this you know um new ability to  move uh quickly from your perspective. Yeah, [9:00] I think that that's a great question and I think  this is a real defining tension uh in the banking [9:06] industry at the moment and any other industry  which is heavily regulated like healthcare. Uh so [9:12] uh last year there was a uh research that was done  by BCG and MIT jointly which published uh that [9:19] uh uh the number of incidents uh which has  impacted customers negatively uh to the [9:25] industries and companies they have been on rise.  I think there has been a 50% shoot up year on year [9:32] uh as in 2025. So that's definitely a reflection  of uh the focus of industry has been real in terms [9:39] of moving or shipping the AI products faster.  But in my view as a product leader I think [9:45] uh customers are going to forgive you if you are  delivering a valuable product little later. But [9:50] if you are messing around in terms of uh you  know the the private data being leaked out or [9:57] if you are in terms of fraud detection which  where I work closely in my current role and in [10:03] previous banks where we are detecting the fraud  and we are trying to kind of block a legitimate [10:08] transaction hampering the customer experience.  I think some of these are real moments where we [10:14] need to reflect on in terms of are we building  the things in the right way with the right guard [10:19] rails and controls. So in terms of speed I think  one of the key factors that we need to understand [10:26] uh as a product leader and as an organization is  building AI product is not just about building [10:33] any software. So traditionally we have used SDLC  life cycle but building an AI product itself is [10:41] a different life cycle altogether and it needs  to be embedded with the required guardrails and [10:46] controls right from the start not like when  you are shipping the product and then you are [10:50] evaluating hey is my product is going to elicinate  or it's going to be biased in terms of its output. [10:57] uh so being being thoughtful about how in terms of  uh at what pace you want to roll it out but at the [11:06] same time how do you balance about maintaining  the right governances and I think uh one of the [11:11] important key thing that I as a leader you know  proactively think about is more about how do I [11:19] embed the risk decision risk based decisioning  when I'm you know selecting a use case and [11:25] deciding on which use case need to go through what  level of governance because not every AI uh use [11:31] case requires uh a similar stringent governance  process to be followed because there are lot of [11:38] use cases which are very very much straightforward  which is like summarization and providing AI based [11:43] recommendation. So those are the things which  are all across the industry different use cases [11:48] available. So do we want to kind of go through  for each and every such use case a a kind of a [11:56] very rigorous uh governance process. So that's a  that's a food of thought for all organizations and [12:02] as a as a product leader. But at the same time as  we are maturing in our AI journey as an industry [12:09] right now there has been lot of focus in terms of  how AI agents uh are kind of going to be becoming [12:17] all self-driven in terms of taking the intelligent  decision and with limited human intervention. So [12:23] as those intelligence are catching up uh it's  important that we are thinking about some of [12:30] these guardrails right from the start and scratch  and also factoring in some of the regulations. [12:36] So if you talk about the number of regulations  especially coming from banking industry they have [12:42] increased 10 times the the number of regulations  you know published by OC FDIC uh since 2020 uh [12:50] till today I think there are 10x more regulations  available today. So as a leader I think we need [12:56] to ensure that we are doing a right balancing act  in terms of the speed and the governance but one [13:02] last point which I feel is even more important to  that is while we are looking at pace while we're [13:08] looking at controls are we building the right  product for our customer right because many [13:13] times developing AI product uh some sometime the  organizations feel that it's a technology problem [13:19] or it's a data science problem but in my view  it's not neither a technology nor a data science [13:24] problem it's problem for a business where business  want to invest and see realizing the value using [13:30] the latest AI technology and I think we have seen  or read lot of these reports like MIT reports [13:36] which were published couple of years back where  you know only 5% of the use cases succeed and [13:42] and the key reasons being we are focusing maybe on  too much on governance or too much on uh delivery [13:48] pace uh forgetting the underlying use case we are  trying to solve for so I think it's it's important [13:53] that we are we need to balance as a product leader  all the three dimensions. Thank you for that. And [13:59] Rachel, you're also in a a regulated industry.  Do you have a similar perspective or Yeah, [14:05] so for us it it's similar. Um the speed  at which you innovate and risk management, [14:13] we don't see those as competing priorities. Uh we  think those should go hand in hand. So um you know [14:20] before you build before you ship we have certain  best practices that we have our employees follow [14:28] whether that's for a new AI tool whether that's a  vendor or a custom solution or an AI agent again [14:36] whether that's a completely customuilt end to  end or whether that's built on a thirdparty [14:40] platform now based on how a user completes that  intake process a certain risk level is assigned [14:48] um based on that risk level. Then they  have certain things that they need to do [14:53] um as part of their validation before the uh tool  or agent is actually deployed into production. So [15:00] we try to alleviate the heavy lift for things  that are really low risk. You know that's going [15:06] to be your internal chat bots where if you know  we turn them off nothing is really impacted. Um [15:13] employees just have to go back to getting that  information to how they did previously. but [15:17] it's not going to impact our business. High-risk,  you know, anything that's externally facing that [15:23] business decisions are being used based on, you  know, this agent's performance or responses. Um, [15:29] and that really determines how we proceed and  this intake process also serves as a system [15:36] of record. So all of the approvals, all of  the ownership, the accountability, you know, [15:42] if there's any future maintenance, all of  the testing, validation, it's all stored [15:46] within this one centralized place, which makes  governing tools or agents a lot more simplified. [15:54] And then something we're also um really excited  about, you know, we're a very large enterprise. [16:01] Information is all over the place, but we're  trying to centralize that when it comes to AI, [16:06] including different AI tools and agents across the  business. So people have the resources they need, [16:13] they know what best practices they need to  follow, they have validation templates that [16:18] they can leverage. Um so we're trying to make  the information as available as possible. So, [16:23] we're not slowing people down. We're not becoming  a bottleneck, but we're giving them the tools that [16:28] they need to succeed. Fantastic. And I mean, it's  it's so important for companies to understand, [16:35] you know, how AI is being used inside the  organization, what the risks are to put the [16:40] right frameworks in place. Maybe an open question  to anybody on the panel. Have you like how do [16:44] you see that process changing as the technology  evolves? Because like 3 years ago when you know [16:51] companies had the chat GPT moment and everybody  had to figure out how to use AI the challenges [16:57] were very different than they are today. Any any  comment on that? Yeah I think I can add to that [17:03] a little bit. So I think like you said like few  years ago there were like fixer challenges there [17:08] were like very similar challenges that we all can  kind of predict but right now the situation is [17:14] that there are like end challenges and and the  way to do it is is basically like you have to [17:20] understand like from your customer standpoint what  is the worst use cases that can happen that will [17:26] become a problem for your brand and the user. So  think from the user shoes and make sure those are [17:32] your test cases and make sure that's kind of your  pass or fail gates and and if it's failing you [17:38] need to make sure the experience you're defining  is is passing that uh gate that you're defining. [17:44] So in in some sense today like you have to kind  of work for scaling but you have to basically [17:51] limit to the biggest worst possible scenarios and  make sure those are are defined and and then the [17:58] other things you'll have to just try to kind of  work based on what you kind of learn from what [18:03] you do and and also kind of have a situation where  the AI just has a standard to saying okay I can't [18:09] help you with that or somehow making sure you are  defining ing that aspects which you are not aware [18:16] of today. Right? So let's move on a little bit to  how um we build these systems. Right? It it's a [18:24] very different process to build AI products.  Um you need different skills. Teams need to [18:30] collaborate very closely. Traditional roles are  changing. Um and you know how we measure success [18:39] is is different. So let's talk a little bit about  how you know how do you figure out what KPIs to [18:46] you know uh build towards and how do you align  your teams your cross functional teams around [18:52] those KPIs and make sure they're set up for  success. Pierce this is your area of expertise. [18:57] Yes. Um this is this this is an important  one for all my product people in the room. So [19:01] uh I'm happy to take this. Um, so when it comes  to measuring success for agents or AI investments, [19:09] I think some things have stayed the same. We we  still care about adoption, retention, seesat, [19:14] all those classics. Um, but what's changed is, you  know, we're we're no longer just giving employees [19:20] a tool to use to do their job. In a lot of cases,  we're actually completing the job for them. Um, [19:25] and that that changes how you would grade  um a software building team or a product [19:29] team. um you know they're measured on actually  completing the task or the outcome. Um so it's [19:36] um it's interesting to see this kind of change  happen in practice like some of the examples that [19:41] point to again coming back to AI customer support  as like a really mature aentic category. Um if you [19:47] look at the likes of Finn or Sierra Decagon these  companies there they all use resolution rate as [19:53] their product KPI. So it's not adoption. It's not  something to do with UX. It is did the agent solve [20:00] the ticket. Um and what's interesting is that's  a KPI is so important to them. It's actually also [20:07] the way they uh it's also their pricing model. Um  they charge customers on a per resolution basis. [20:11] So this is one of the few examples as a in the  software world where you have your product KPI and [20:17] your business model is the same number. Um which  is a pretty interesting way to uh kind of get [20:21] everyone working towards the same goal. Um when it  comes to kind of okay we care about outcomes how [20:27] do we actually uh practically work towards that um  what everyone's kind of started to align around is [20:32] a a new process step called evals um which no  doubt some of you are familiar with with with [20:38] that in this room but um if you haven't um played  around with evals yet um it's basically agreeing [20:45] on a on a on a on a representative set of inputs  to an agent. So those are just like possible [20:49] things you might type in for example if it's a  chat interface. Um and then agreed upon correct [20:55] outputs from those inputs. Um so that could be  an answer or steps that an agent should take. And [21:01] basically then as a as a as a product team every  change to a model, change to add additional tools, [21:08] change to context, change to harness, whatever  you might be doing to improve the product, [21:13] you're benchmarking everything you do against  increasing on those EVOs. Um so interestingly [21:19] there uh the new measure of success is actually  kind of back to your alignment question is also [21:26] also the way you align teams because now you  have one number that product the data science [21:32] engineering can all agree on as the north star  and work towards and split up duties around that. [21:37] Typically speaking product are going to set the  use cases that are in scope for the eval set. data [21:42] science are going to actually create the eval set  and um do the do the grading against it and then [21:47] engineering are going to be kind of make trying to  make changes to make the number go up. Um so all [21:52] pretty interesting there. Um Pendo has always kind  of been in the business of helping companies to [21:59] um evaluate whether their software is working.  Um and the next chapter for us in this area [22:04] um is to do that with agents. Uh we have a product  called agent analytics which aims to do this is [22:09] basically to track whether the agents that you  have in your product are actually solving the [22:12] job to be done that you set out. Um and crucially  also uh when they are failing what are some of [22:19] the patterns or reasons why um and the new loop  for product teams in Pendo working on agents and [22:26] product teams kind of uh more broadly is that that  production monitoring of finding where your agents [22:32] falling over and not working um becomes your new  eval. So basically there's the full loop there [22:37] of tracking like where is it failing right now?  What are the use cases that we're not serving? [22:43] um let's create new evals around that and that  becomes the the data set for engineering to to [22:47] to bash against. So Neil, any any any similar  um you know processes or or focus areas from a [22:58] consumer AI perspective? Definitely um I can give  a slightly different but related to what Pier was [23:04] saying here. So for instance like take the case of  like nondeterministic products like a chat agent [23:10] or something like that right. Um I would say like  there is no one single northstar we work towards. [23:16] It's more like a portfolio of metrics that you are  working towards. For instance, take the example of [23:22] like say summarization. It we might be looking  at like faithfulness and completeness. But for [23:28] a search intent feature, we might be looking at  relevance of the search and what is the intent [23:33] of that search like that that user is doing.  And say for something like drafting, we might [23:39] be looking at tone and accuracy. So in a way like  as product builders we are defining what is good [23:46] looks like for that specific outcome you want  to look at. So it really depends on the product [23:52] you're looking at and say like for instance it  could be like very different for someone in the [23:59] health industry and it could be very different for  someone in the finance industry. Can I get a quick [24:04] raise of hands? How many of people here are from  the like a sensitive industry like healthcare, [24:09] finance or like fin like any real estate? Oh  yeah, I think a lot of folks here are like [24:15] from sensitive industry. for you guys. There's an  additional layer which is like do know how metrics [24:20] or guardrail metrics sometimes which basically  take care of like privacy loss because you might [24:26] be dealing with very sensitive information for the  user and for like housing specifically I'm looking [24:31] at fair housing laws because those are also really  important things we need to look at when we are [24:36] building AI features. So from an industry  sensitive standpoint there's an additional [24:40] layer you need to also take care of while you  are kind of uh looking at your portfolio metrics. [24:45] So coming back to the question of alignment um I  think the main thing is these do no harm metrics [24:51] are guard and guardrail metrics are non-negotiable  those are like something that you need to pass [24:57] uh and what what are be the other metric scores so  the alignment is these do know how metrics should [25:03] be very core to the product and then you align  on what are your portfolio metrics across your [25:08] uh product product engineering and data teams.  So another question for you on that note like [25:13] in terms of how you build these systems you  know obviously the current AI paradigm is large [25:18] language models who knows that may change in the  future um like how do you decide whether you use [25:24] an offtheshelf frontier model you know whether  you are using open source models there are lots [25:29] of those these days or you know something more  proprietary what's your um approach yeah I I think [25:35] there's also like I would say slightly like a  framework you can apply there too uh for instance [25:41] like for generic capabilities like I was saying  summarization drafting classification almost [25:47] always all the time you will see that generic APIs  or commercial LLMs are already good and giving you [25:55] a like a great experience at a cheaper rate. Um so  why kind of build another one uh when you already [26:02] have something which is already doing the right  job at a cheaper rate right? So you could look [26:06] at it like three ways. So a few things that you  can ask yourself is how specific is your use case [26:13] uh how sensitive is the data and like what are the  like uh what does economics look like at scale so [26:21] I think that's a great criteria on the other hand  like I was saying for generic capabilities if you [26:26] are looking at scale and if you are seeing that at  a scale when you are like scaling at like millions [26:32] of users if you are seeing that it's cheaper  to build your own models more say summarization [26:38] You might even actually do that because I mean  when you're doing at millions you're looking at [26:43] cost differences of other millions as well right  so those are one way where you're looking at [26:48] generic capabilities then as I was talking about  like very sensitive or specific industries like [26:54] healthare housing uh finance you are dealing with  very private and sensitive information in those [27:01] situations uh almost always all the time sometimes  you kind of build your own specialized models to [27:07] take care of the sensitive use cases and make sure  the your customers user data and information is [27:14] properly used and also shared with the relevant  systems. So in this case like for real estate [27:20] and housing specifically like we like I was  saying fair housing laws is very critical us to [27:25] make sure it's um passing and everything is fine  from a user perspective before we ship ship the [27:31] experience. So I think uh like for the specific  use cases you will always have to go forward with [27:39] um building your own experiences but again this  is just my instinct sometimes uh but you cannot go [27:46] based on instincts either but how I use instincts  is to kind of prioritize my what I test first but [27:52] I can't say for 100% this is the best approach  because I have to AB test it in a way like I [27:57] have to look at my portfolio of metrics that I  was saying and look at it cost latency and then [28:01] see what generally makes sense from my use case  and and customer perspective. So then you kind of [28:08] get to an understanding of like okay I want to go  forward with this like you're basically AB testing [28:13] your experience and understanding what's the right  approach here. Yeah you have to figure out what's [28:18] the right model for each job and that's not as  easy as it sounds. That's where evaluations come [28:23] in. Dear I'm curious to get your take on this from  a you know more regulated industry perspective. [28:28] Yeah, I think um and it'll cover quite a few  uh scenarios but uh ju just just to call out [28:35] like in terms of build versus buy right there  are two important parameters or functions that [28:42] sometime define which way uh is a right  way for the organization. Uh so the first [28:48] one is more around the organization strategy and  business strategy right. uh we have seen lot of [28:53] organizations they they are doing successful  in either scenarios in terms of building [28:58] uh the propriety models and you know hosting  it on their local platforms. uh but we have [29:04] also seen uh successful scenarios just to give an  example like Capital One Bank acquired Brex right [29:11] in 2026 and as a result of this acquisition they  got an AI agent capabilities of serving business [29:18] banking customer and and in in case of the first  scenario where you know the banks want to do [29:24] everything by itself and have more control uh JP  Morgan for example they have an annual uh budget [29:31] size technology budget size of around 18 billion  us right that they are spending on building AI [29:35] infrastructure. So it it it perfectly boils down  to sometime the organization strategy and also [29:41] layering it with what specific use case we  are trying to build. Right? If it is like a [29:45] very simple generic use cases uh maybe for you to  do a quick experimentation time to market maybe [29:53] you might rely mostly on the commercial APIs or  foundation models. But if you are in the industry [29:59] where you're trying to kind of build something  which is more proprietary and sophisticated just [30:04] like fraud detection models right which requires  you to input your customer data, transaction data, [30:09] merchant data and network data. So there is a  function of lot of these parameters that need [30:14] to go in to develop a specific intelligence for  your organization to detect fraud or similar case [30:20] for you know money laundering detection and  all the stuff. So in which case it makes more [30:24] sense for organizations to spend more time  building those propriety model because that [30:30] is going to give them a a differentiating  factor or competitive competition a edge [30:36] uh in respect to other competitors in  in better servicing their customers. [30:41] So while while these have been you know driven  by strategy but there are some other factors [30:47] uh sometime to market is an important factor right  if you're looking for a quick experimentation I [30:53] would always rely on you know using some vendor  solutions using some commercial APIs and and get [30:58] the output done just to give a a context in terms  of what's the art of possibility solution is uh [31:04] alongside that uh it it's important that we also  factor in the development effort. Let's not forget [31:13] building a propriety models takes a a good amount  of effort. Not just about uh developing a model, [31:20] but it's about getting the right data set up in  your organization, getting it trained and you know [31:25] developing the model and providing that end to end  testing and support with the entire application [31:30] ecosystem. So I think yeah some of these factors  they play a critical role in terms of how you [31:37] decide. So as Anil was uh alluding like there's  no right or wrong solution. It all depend upon the [31:43] use case and the business needs. Great. Um we're  almost going to move over to Q&A but you know an [31:49] AI panel without talking a bit about hallucination  would be a miss. This was a problem you know that [31:55] was probably the most prominent problem when AI  you know came out three four years ago. Is it [32:00] still a problem? And how do you design your user  experience around, you know, model creativity? [32:10] Yeah, I can jump in. Um, so yeah, if  you're if you're going to talk about AI UX, [32:15] uh, certainly you're going to you're going to have  hallucination come up. Um, I think, um, you know, [32:20] the reality is like by now it's it's definitely  significantly less of a problem than it was a [32:25] few years ago. um especially with you know most  of the applications we're building are not just [32:30] raw model they have various kind of agentic steps  included to try and root that kind of stuff out [32:37] um but if I think about the underlying kind of  uh uh user frustration there is not just Asia [32:42] it's like AI giving me a wrong answer or AI doing  some not doing the thing I asked right as kind of [32:48] the more broad problem and in particular with chat  interfaces which is you know the thing that we're [32:54] most familiar with um I think it's a bigger issue  with users asking the AI to do something that it [33:00] wasn't designed to do um or that it can't do  and there's a lot of design UX you need to [33:06] actually deal with that problem um which is still  present um so you know the the reality is with a [33:12] chat interface it's totally open-ended there's is  literally a blank screen you can type in whatever [33:16] you want and uh from the user's point of view uh  it's not clear to them where that starts and stops [33:22] um traditional UIs communicate a lot of structure  structure to you. You come in, there's a menu of [33:26] options. You click on a dropdown, tells you  what you can do there. It's very clear what [33:32] the workflows are, what the product can and can't  do. All that goes away when you have open end the [33:36] chat interface. So, in any of the products we've  built with a chat feature, we've had to kind of [33:41] be pretty intentional on UX to deal with that  problem. Um, some of the things that we've done [33:46] is being pretty explicit in suggesting options  to user up front. So they could be like suggested [33:54] prompts if it's a prompt interface or you have  some kind of kind of recipe book up front that [33:57] you're just going to look at to see kind of what's  the general kind of gist of what this is supposed [34:01] to help me with. Um it's helpful to add some  structure to the input where it makes sense. So, [34:09] I actually think Claude have done a really great  job with this recently with their with their [34:13] uh with their desktop product um where you  can atmention tools and there's drop- down [34:19] options where that makes sense. If you use their  skill builder, they have great little kind of [34:23] uh option pickers, these kind of things. So,  it's not all going to be chat. There's still [34:27] um a lot of room for kind of more intentional UI.  Um and then I think it's it's also about actually [34:33] being thinking about the the the designing for  the no. We kind of use that phrase internally [34:38] which is the situations in which you know people  are going to ask it to do something you can't do [34:42] like what what do you actually want the model to  respond with there saying I can't help is usually [34:46] not a great response and frustrating. Um better  alternatives are suggesting an an alternative [34:51] action that you can do or gracefully handing them  off to like another part of your application that [34:56] actually can deal with that. Um uh because those  are kind of key moments for basically keeping [35:00] users coming back. Um, and ultimately, you know,  those are all difficulties with a chat interface. [35:07] There's a lot of great benefits as well. Um, as  a product manager, chat interfaces is a basically [35:14] uh 365 24 hours a day survey for your users.  They're telling you what they want your product [35:21] to help them with. Um, and as much as you want  to monitor like that people are not using it [35:26] ways you haven't intended, you also want to look  at what they actually think it should do because [35:30] that can become your road map. So, um, this is  an incredible data set for anyone who's building [35:34] a product like this to go in and look at all  those those user asks and try and see if any [35:38] of those actually would make sense to become part  of your product. So, great. Thank you so much. So, [35:43] on that note, we would like to hand it over  to the audience if you have any questions. [35:48] So a lot of our teams have changed uh with  AI and the team structures for products, [35:53] but how do you see the teams uh changing even more  in the products that are AI products? Um you know, [36:01] has it gone from 12 engineers and 1 PM to a 1:1  ratio or uh everything else that's going along [36:08] with that in your AI products? Rachel, I wonder if  you have a view on this. spoke about you know this [36:14] whole idea of a dark factory which I don't know  who came up with that term but we need a new term [36:19] right the the point is engineering is changing and  you know we have to adapt to you know no longer [36:26] you know having necessarily the same processes  but still the same um certainty in the quality [36:32] of the product something we've been experimenting  and exploring is this concept of a dark factory [36:39] model I'm not sure if you guys have seen this buzz  word um it's something we didn't invent. It's just [36:46] uh what what's coming I think in the engineering  world. So looking at using agent teams to do [36:54] development work, quality review and deployment.  Um, so obviously this is something that we're just [37:02] exploring with lowrisk internal use cases and  thinking about to your point how do you know [37:10] these teams evolve with this new technology. We're  seeing a shift where you typically had engineers [37:19] that were hands on the keyboard, really strong  developers coding, and now they're having to sort [37:25] of upskill where they're now managing resources.  And those resources are agents. So, where they [37:33] were more individual contributors, really good in  their craft and their skills, we're now trying to [37:39] train them how to teach those skills to AI agents.  Um, so this is new. We're still learning. We're [37:49] still innovating these agents and trying to make  them more successful in their own domains. Um, [37:56] I think segregating their roles and having  multiple agents that work across teams and their [38:03] defined silos is really what we found to be the  most successful. Um, but it's a really interesting [38:10] approach and I think that's the way engineering is  moving and I don't think it's replacing engineers. [38:17] I think it's just a shift in how they think about  what they do and now they're overseeing resources. [38:24] Whereas before they were maybe, you know, hands  down doing the work themselves and now they can [38:30] move a lot quicker, they can do a lot more if  they can properly manage the resources to do that. [38:38] Thank you, Rachel. Hi. Yeah, those that was  a great explanation by the way. I appreciate [38:43] that. I have a question. So, for the panel, on  all the AI projects that you guys talked about, [38:49] how many product managers did you deploy to  support those? And was that product manager [38:54] role the same traditional or did it evolve? I I  can't touch on. So, it's the question, has the [39:00] role of product manager sort of changed? Yeah.  Has the role of product managers changed and how [39:05] many product managers do you typically have on a  team versus before? Yeah, I think I I think I can [39:11] I can make a start. Uh yeah just just just to talk  about I think I so what I've seen in the industry [39:16] and in in some of the recent organizations that  I've worked including capital one discover right [39:22] uh the AI product management itself is evolving  right it's not a traditional product management [39:27] so let's let's get the thing straight here and it  requires a specialized skill set which involves [39:32] understanding the AI world understanding  the modeling work and being able to kind of [39:36] communicate and tie together the the connections  between what engineering team is doing versus what [39:42] analytics data science team is doing and also  having a good understanding of what's customer [39:47] need and wants are because at the end of the day  as a product leader you're trying to solve the [39:51] customer painoint. So as AI uh product management  uh as a stream yes uh the different phases have [40:00] evolved and right now what I have seen in some  of the organizations like capital one and all [40:07] uh the entire AI product life cycle itself is  being kind of revamped using AI just for taking [40:13] an example like you know as a product leaders  you're required to prepare a product requirements [40:17] document right which is capturing what product  features you're you're ready to launch next uh [40:23] that will create value for your organization and  customer and can you use some cool AI tools that [40:29] are available which can help you capture some of  those needs and wants in a documented for fashion [40:34] and do some run some quick experimentation. So  there are so much of things that are available [40:39] at the moment which can help you accelerate your  ideas and PC activities which were in in in normal [40:47] use cases where we do not had a AI uh you used  to take months and ages. So now I think with AI [40:54] coming in picture and being embedded into the  entire AI product life cycle I think there are [41:00] opportunities to accelerate and experiment and as  as a product leader I think it's a learning for [41:06] entire organization. Uh and last point I would  like to talk about is like now everybody in the [41:12] organization can think and wear a product mindset  and product hat even if you don't know a coding [41:17] if you know what what business you're serving what  customers painoint you're looking to solve for and [41:23] if you want to step into that shoes I think there  the the opportunities are unlimited because now [41:28] you can use AI tools at your disposal to act as  that product leader and you know solve for it. I [41:35] can add a slightly different perspective as well.  I feel like as product builders the fundamental [41:40] job of product builders has not changed meaning  we are here to define the problem but the way [41:45] we do it has slightly changed to just to add  on to the is like I think we are instead of [41:50] writing also sometimes specs we can now show the  prototype very easily using white coding and plot [41:56] and like different types of tools so you can kind  of fundamentally show your cross functional teams [42:01] the product that you want to build at a smaller  scale very quickly I've gone from like my ideation [42:08] to actual prototype and testing even like in 5  days like actually testing with the user reser [42:13] customers and now I have actually even validated  that and now everyone is buy in because bought [42:19] in because like they already know this experience  works we got the user feedback and that's kind of [42:24] a great confidence for us to move forward with  this experience so I think the way we do it as [42:29] change meaning like you have to upskill yourself  to understand those processes the way we are doing [42:34] it now as PMs Because earlier we used to define  a big writing docks because and hand it over to [42:40] design team to design stuff but now we ourselves  can build our prototypes all the way till the end [42:46] from like to uh kind of getting the inputs from  research. So I think the process has changed a [42:52] little bit where product builders are able to go  end to end very quickly. I think we have time for [42:58] one more right Austin. Cool. Um I'm Trishala. Uh  just for a quick intro um I'm in Y cominator. I'm [43:06] building agent analytics uh that can measure user  intent and uh user frustration. So PICE I thought [43:12] you had a really incredible point around there.  Um I'm curious for Kobis and Unil who are building [43:17] consumerf facing agents. Um how do you actually  measure user frustration today and do you feel [43:23] like your analytics and observability tools tell  the full story around where users might get stuck, [43:29] lost or hesitate? Um so I I'll you know  take a stab at that evaluations is the [43:35] most important way to do that because you  have this wealth of information you know [43:40] um your customers are actually talking to you  are you know you just have to make sure you're [43:47] finding those examples where the system is  not working as expected. So you have to get [43:52] really good at evaluating those conversations,  making sure you understand if user sentiment is [43:56] changing over the course of a of a conversation.  What's the outcome of the conversation? Did the [44:02] AI agent actually do what the customer  wanted? I mean, often in our business, [44:07] the unfortunate thing is the customer may want  something, but the agent is not able to, you know, [44:13] cancel a non-refundable hotel room. That is you  know as you can imagine the number one uh thing [44:19] that you know an an agent in our business is is  being asked to do and then you have to make sure [44:26] the agent explains the policy to the user even  though you know from the user's perspective the [44:31] outcome unfortunately is negative but the agent is  like enforcing policy and clarifying making sure [44:37] the customer understands why. Yeah, I think I I'll  just add one point right like today I think we [44:44] have a power of multimodal uh you know LLMs right  which has an ability to ingest data in different [44:51] format and I think users are giving feedback all  the time like they're calling you our call center [44:55] uh they are posting certain things or reactions on  the LinkedIn or Facebook or any other social media [45:01] right now how do you use some of these smart tools  which can ingest data in different format and give [45:08] you the required feedback and intellig igence loop  that can help you improvise a product. So I think [45:13] that's another way to kind of how do you kind of  constantly refine taking the feedback coming from [45:18] different channels and different formats and be  able to kind of assimilate those information to to [45:23] be kind of able to kind of come up with a product  feature which so which serves the customer. Yeah. [45:29] And I guess sometimes you can also kind of have  like these traditional methods already applying [45:35] to this meaning like the thumbs up thumbs down  and and also kind of getting actual like not [45:40] feedback. You can kind of like send feedback. So  those kind of mechanisms are still remaining true [45:44] for understanding frustration because that's the  easiest way to do it at scale like hey I'm giving [45:49] you a thumbs down and tell me why like you give  that thumbs down. So we have that mechanisms in [45:54] some of the features I'm working on because  we need to know why because a thumbs down [45:57] doesn't tell me why you are frustrated. Is it  because it's not personalized? Is it because I [46:01] didn't use your context well or is it because I  hallucinated? Right? So I think that those sort [46:06] of actual feedback really helps. Then I use AI  to summarize hey what's the biggest frustration [46:10] for our users and then try to solve the top  one and go priority by priority. I I'll just [46:16] add one more thing. It is important also to have  a human in the loop channel where there can be [46:20] an escalation to a human if the AI you know if  the customer is not happy with the answer they [46:25] get from the AI. So I think that's a a safety net  to to have in place. Yeah that's super critical.