UTMB, Meridian Shift AI Funds to Cash Cycles as Pilot Failures Mount

试点失败率攀升 UTMB与Meridian将AI投资转向现金流周期 缺乏基准衡量及用户信任缺失导致大多数企业部署失败，各机构必须放弃投机性的人工替代试点，将AI扎根于收入周期自动化。 讨论嘉宾探讨了企业AI项目高失败率与巨大潜在回报并存的悖论，强调可持续的ROI取决于用户信任、数据就绪度以及同步推进的以人为本的变革管理。与会专家告诫切勿开展基于未经验证的“商业传闻”的项目，并敦促团队用挂钩收入运营的实用最小可行产品取代脆弱且依赖脚本的PoC演示。在医疗与金融服务领域，高影响力价值来自直接创收的工作流程（如自动化临床使用申诉与预测性客户咨询），而非理论上的劳动力节省。最后，专家们强调，随着智能体模型开始跨越传统系统，推进数据架构现代化对于避免技术债务成倍累积至关重要。 企业 AI 面临显著的落地悖论：超过 30% 的项目在概念验证（PoC）后因高昂算力成本与模糊的商业价值而停滞或被废弃，但成功实践却能带来 10%–15% 的生产力提升及高达 4 倍的投资回报。 (0:04) 用户采纳度是决定 AI ROI 的核心，约半数部署失败源于终端用户缺乏信任而非模型算法缺陷；企业必须将人员变革管理与底层技术数据准备并行推进。 (4:16) 缺乏实证基线指标的“企业传闻”项目注定无法量化成效，团队应平衡速赢项目（如 Meridian 推广代码助手带来 25% 的代码采纳率提升）与反洗钱（AML）等多年期战略转型投资。 (12:18) 落后的数据架构会引发指数级技术债务，企业还需建立双轨可观测性：对内严密监控推理与 Token 消耗以防账单爆表，对外紧盯开源进展以防自建方案迅速被商品化淘汰。 (18:26) 企业在衡量落地价值时需区分“工时节省假象”与实际创收，例如 UTMB 将 Agent 聚焦于床位状态核定与拒赔申诉等收入周期管理环节，直接将算法精度转化为真金白银的净现金流。 (26:41)

Enterprise AI initiatives frequently fail when driven by ungrounded hype, but organizations can secure sustainable returns by coupling foundational data architecture with parallel human change management and targeting measurable revenue workflows. Panelists across healthcare, banking, and technology emphasize that genuine ROI depends on user trust, baseline operational metrics, and direct financial impact rather than theoretical labor savings. Replacing Institutional Folklore with Measurable Baselines (8:23) Industry data from Gartner reveals that over 30% of AI projects stall before pilot stages, and more than 30% of generative AI projects are projected to be abandoned post-proof-of-concept by late 2025 due to spiraling costs and unclear value. Dr. Peter McCaffrey warns that many enterprises fall prey to institutional "folklore"—unsubstantiated claims of operational losses, such as unmeasured revenue cycle leakage. Pouring capital into unmeasured problems guarantees failure because organizations cannot empirically demonstrate value realization or baseline movement. This diagnostic gap is exacerbated by fragile prototypes. Ravi Venugopal cautions that scripted demonstrations running in tightly controlled, rules-like environments often trigger unrealistic expectations from leadership. He cites a client seeking to reduce a 700-person support workforce down to 200 based on a demo, only to face production model hallucinations. To mitigate this disconnect, Venugopal advises abandoning prolonged proofs of concept in favor of concrete minimum viable products (MVPs) tied directly to scalable revenue operations. Modernizing Fragmented Architecture to Avoid Compounding Debt (16:04) Healthcare and enterprise environments frequently struggle with legacy digital ecosystems. McCaffrey notes that health systems often maintain upwards of 1,300 disparate IT systems and static interfaces. While agentic generative AI has the potential to leapfrog historical workflows entirely rather than merely optimize them, fragmented data architecture throttles model integration. McCaffrey warns executives that because AI innovation accelerates exponentially, failing to resolve foundational integration friction results in compounding, exponential technical debt. Infrastructure sustainability also dictates financial viability. Venugopal stresses the need to calculate component-level consumption early to avoid bill shock, citing organizations that deployed costly foundational models for simple email classification. Furthermore, he recommends a two-tiered observability framework: tracking internal performance, query latency, model drift, and token expenditures, while actively monitoring the external market to avoid building bespoke tools that open-source releases could commoditize overnight. Driving Adoption Through Human-Centered Change Management (4:16) Technical efficacy does not guarantee business impact; Venugopal observes that approximately half of failed enterprise deployments falter because end users reject the software. User adoption depends fundamentally on trust, which forms the prerequisite for ROI. Jeremy Steinert argues that organizations mistakenly treat AI as a purely technical rollout. Because modern architectures require extensive preliminary data groundwork that yields no immediate business visibility, business teams must run parallel change management workstreams—addressing user expectations, process re-engineering, and workforce education before deployment. Workforce sentiment also shapes adoption velocity. Mona Hassanzadeh notes pervasive employee anxiety in financial institutions around metrics explicitly framed as "saving full-time equivalents (FTEs)." Leaders must counter displacement fears by emphasizing augmentation, skill empowerment, and operational risk mitigation. Steinert adds that addressing the AI talent and readiness gap requires proactive workforce enablement, including structured internal training programs and direct university hiring pipelines. Balancing Near-Term Wins Against High-Yield Workflows (12:18) Enterprises manage deployment risk by balancing quick operational wins with strategic architectural bets. Hassanzadeh explains that Meridian established internal credibility by rolling out developer coding assistants, which yielded a verifiable 25% increase in code acceptance rates. This early success earned executive trust, enabling the organization to fund multi-year transformations, including document intelligence for automated anti-money laundering and check fraud detection, vetted through an AI Steering Committee and Privacy Impact Assessments. In operational environments, organizations must differentiate between diffuse efficiency and net financial return. McCaffrey points out that making salaried staff, such as laboratory technicians, 10% faster rarely reduces headcount or generates net savings. Instead, UTMB focuses investment on agentic utilization management and revenue cycle tasks—such as converting patient bed status from observation to inpatient and automating payer denial appeals. These workflows counter insurer-automated claim denials and generate measurable "new money" returns directly tied to top-line cash flow. While the panel views AI investments as compounding interest where robust infrastructure unlocks progressively larger operational capabilities, success remains constrained by organizational readiness and technical debt. Defensibility will increasingly depend on whether enterprises can move past superficial prototypes, establish empirical baselines, and counter automated external pressures—such as algorithmic insurance denials—without overpromising workforce reductions.

面对AI项目失败率高出传统IT项目一倍且超过30%的项目停滞于试点阶段的行业现状，与会专家指出，实现高投资回报率（ROI）的关键在于摒弃脆弱的原型演示与缺乏基线度量的盲目尝试，转向以数据就绪、组织变革和端到端可观测性为核心的生产级部署。 用例甄别与基线度量：破除缺乏依据的业务传说 (8:23) 在确定资本配置时，德克萨斯大学医学分部（UTMB）首席AI官Peter McCaffrey警告企业必须警惕缺乏经验数据支撑的“民间传说”。例如在医院收入周期或利用率审查中，业务人员常宣称存在数百万美元的流失，却无法提供基线运营指标；若在缺乏基线的情况下投入数百万美元，企业将因无法证明业务改善而注定失败。McCaffrey指出，对于临床指南检索等广泛的辅助性工具，强求前置财务ROI如同“询问企业使用Outlook的ROI是多少”一样不切实际；但对于具体运营系统，缺乏基线指标则是必须叫停项目的危险信号。 医疗与金融机构正转向严格的验证机制。Meridian数据科学与创新总监Mona Hassanzadeh介绍，该机构设立了严格的准入流程，由集中式AI指导委员会联合跨职能团队开展隐私影响评估（PIA）、负责任AI审查以及端到端运营就绪度审计。在效益衡量上，McCaffrey强调必须区分理论上的劳动时间节省与直接产生现金流的“新创收入”。例如，让实验室技术员效率提升10%很难直接转化为净成本降低，因此UTMB优先投资于利用率管理流水线，通过算法自动裁定观察与住院状态并处理拒付申诉，直接将模型精度变现为可衡量的收入回报。 组织变革与全职缩减：以人为本的用户信任机制 (4:16) Giggso创始人兼首席执行官Ravi Venugopal指出，企业AI部署失败案例中约有一半并非源于算法缺陷，而是最终用户的抵触与拒绝；成熟运营模式的最终ROI取决于用户采纳，而采纳的核心建立在信任之上。Turnberry Solutions合伙人Jeremy Steinert认为，企业最大的误区是将AI纯粹视为技术部署，而非以人为本的变革管理。由于底层数据与架构梳理在短期内难以显现商业成果，业务团队若被动等待工程完成才启动准备，将导致系统上线后遭遇严重的文化排斥。企业必须同步推行员工教育、预期对齐与流程再造的双轨机制。 在人员编制与绩效考核方面，直接挂钩“削减全职员工（FTE）”往往引发广泛抵触。Hassanzadeh指出，金融机构管理者必须将叙事重心转向员工赋能与日常流程降险。Venugopal列举了真实反面案例：某拥有700名支持人员的企业在观看了理想环境下的脚本演示后，盲目设定在首年裁减至200至250人的目标，结果在生产环境中遭遇生成式AI的大面积幻觉，最终不得不大幅下调预期。专家强调，企业应放弃易碎的单点概念验证（PoC），转向紧扣收入运营（RevOps）的最小可行产品（MVP），并让一线用户尽早参与共创。 架构现代化与双重观测：遏制指数级技术债务 (16:04) 医疗健康等传统行业常年背负高度碎片化的数字化包袱。McCaffrey以医疗系统为例指出，许多机构运行着超过1300个独立的IT系统、自定义ETL管道以及仅用于固化历史流程的静态GUI软件。随着代理型AI和生成式系统的成熟，AI技术将直接跨越而非仅仅优化这些陈旧流程，使大量单点工具被淘汰；但跨越的前提是直接且统一的数据访问能力。如果企业因1300个遗留系统的摩擦而无法将AI置于数据之上，所承受的技术债务将呈指数级累积，因此底层架构现代化对长期生存的重要性远高于短期的季度回报。 在底层基础设施层面，企业必须防范算力成本失控与生态淘汰风险。Venugopal提醒，部分企业仅为处理简单的邮件分类而调用Azure OpenAI大语言模型，导致月度Token账单远超预算。他提出了双重可观测性框架：内部需持续监控推理成本、网络延迟和模型漂移，避免计费爆表和隐性失效；外部则需保持严密的技术生态监测，防止因开源工具更新而导致昂贵的定制化研发瞬间归零。例如网页抓取行业曾耗资数十万美元研发专有管道，随后微软发布开箱即用的开源工具集，使此类商业系统一夜之间失去价值。 投资组合与垂直深耕：从速赢示范迈向战略复利 (12:18) 平衡短期成效与长期战略转型是维系企业AI预算的核心策略。Meridian采取“摘取低垂果实”与多年度复杂架构并行的组合方式：首先在其技术团队内部推广代码辅助工具，实现代码接受率即时提升25%，以此建立高管层信任并积累组织资本；随后将资本投入多年度战略级项目，如基于AI视觉与文档智能的端到端反洗钱（AML）和支票欺诈检测平台。Steinert提醒，企业不可因外部案例盲目设定6个月收回投资的教条指标，各部门考核指标必须因地制宜，例如软件工程应考核交付速度与周期缩短，客户支持则应追踪工单分流率与自动化解决速度。 面向未来五年的高价值场景，垂直行业的深层重塑尤为突出。McCaffrey指出，医疗收入周期管理正演化为一场提供方与支付方的“算法军备竞赛”：鉴于保险支付方已普遍利用AI加速拒赔，医疗机构必须利用AI整合传真、语音留言与病历等孤岛数据，自动化处理合同审查、计费与申诉以维持财务平衡；同时AI驱动的患者导诊与访问工具也将彻底颠覆低效的就医体验。Hassanzadeh则指出，银行业的最高ROI将来自辅助人类决策的个性化服务，例如通过“客户下一步最佳行动”预测模型，在保持顾问参与的前提下定制生命周期金融建议。 与会者将企业AI的投资回报比作“复利积累”，其指数级跃升完全依赖于前期在数据治理、架构现代化与组织信任上投入的底层本金。企业当前正在经历从追求盲目裁员的幻觉演示向注重收入运营与合规落地的理性回归，但如何在防范技术生态商品化冲击的同时，将模型准确率持续转化为经得起财务审计的实质收益，依然是检验落地成效的核心命题。

## Background 本内容由 YouTube 频道 Ai4 于 2025 年 8 月 28 日发布。该圆桌研讨会汇集了来自 UTMB、Turnberry Solutions、Giggso、PayPal 和 Meridian 的跨行业高管，共同探讨企业如何在 AI 实施早期做出明智的战略决策以推动可观的投资回报率（ROI）。 ## TL;DR 研讨会深入剖析了高达 50% 以上 AI 项目走向失败的根本原因，指出缺乏明确量化基准和用户采纳受阻是最大诱因。嘉宾们提出了涵盖数据就绪度、合规与风险前置、以及可观测性建设的四维用例评估框架，主张放弃脆弱虚假的 PoC，转而以能够验证 RevOps 真实价值的 MVP 敏捷落地。与会专家强调应平衡短期速赢工具（如 Copilot）与长期核心业务改造（如医疗收入周期与金融风控），并指出扎实的基础设施与全员 AI 素养将产生强大的复利效应。 ## Atmosphere & style 整场研讨会节奏紧凑、直击痛点，充斥着一线从业者祛除 AI 炒作泡沫的务实与清醒。主持人提问犀利且直切商业本质，来自医疗、金融与咨询领域的嘉宾在坦诚分享真实教训（如高昂 API 账单、脆弱演示与落地鸿沟）时互动默契、金句频出。从开场的冷峻现实剖析，到中后期的架构深挖与幽默风趣的快问快答，观感既专业严谨又充满启发性。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=ILOy8povAoQ&t=4s) AI 项目失败率高达传统 IT 项目的两倍，但成功落地后可带来 10% 至 15% 的生产力提升、25% 至 30% 的运营成本缩减，平均每投入 1 美元可产生超 4 美元的回报。 - [04:16](https://www.youtube.com/watch?v=ILOy8povAoQ&t=256s) 成熟企业评估 AI 用例依靠四维框架：商业价值、数据就绪度、价值实现时间以及用户采纳与信任，近半数失败项目源于缺乏用户采纳与信任构建。 - [06:38](https://www.youtube.com/watch?v=ILOy8povAoQ&t=398s) 组织变革必须与技术架构同步推进，业务团队需在技术铺设底层基础设施时提前推进组织变革管理（OCM）与员工技能培训，防范预期脱节。 - [11:02](https://www.youtube.com/watch?v=ILOy8povAoQ&t=662s) AI 用例评估的最大红旗是缺乏量化基线指标，盲目改造连效果都无法衡量的业务场景注定失败；但高度泛化的基础协同工具可免于严苛的单点 ROI 计算。 - [12:46](https://www.youtube.com/watch?v=ILOy8povAoQ&t=766s) 平衡 AI 投资组合应结合短期速赢与长期战略：以推广 Copilot 等提效工具快速建立业务信任，并在 AI 指导委员会合规评估下推进反洗钱等长期核心业务。 - [16:07](https://www.youtube.com/watch?v=ILOy8povAoQ&t=967s) 传统架构积累的大量孤岛系统在 AI Agent 时代会转化为指数级爆发的技术债务，未来的生成式工具有望直接跨越中间繁琐工作流重构产品形态。 - [18:20](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1100s) 基础设施需兼顾内部运行可观测性与外部市场可观测性，防范因底层大模型厂商或开源生态迅速覆盖原生功能而导致自身商业模式顷刻瓦解。 - [21:42](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1302s) 企业在推进 AI 时不能单看硬性指标，还需提升全员 AI 素养（AI Literacy）与领导层 AI 优先思维，平衡财务、运营体验与负责任 AI（RAI）合规指标。 - [25:15](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1515s) 不同领域 KPI 差异显著：SDLC 聚焦交付速度与代码质量，客户支持看重工单分流率，而医疗业务关注病历听写调用及拒赔申诉成功率等硬性收入。 - [27:55](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1675s) 企业必须警惕表面效率陷阱：未能转化为实际人员优化或产能扩充的单点效率提升，在财务报表上往往体现为零成本节约。 - [32:09](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1929s) 金融等行业普遍忌惮将“缩减 FTE”作为量化 KPI，过分强调裁员会导致基层排斥抵触，当务之急是通过负责任 AI 引导员工转向人机协同共存。 - [34:41](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2081s) 坚决弃用脆弱易失真的 PoC 并全面转向敏捷交付的 MVP，避免被规则受控的完美演示误导，尽早引入真实用户共同打造真正切合痛点的产品。 - [38:30](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2310s) 未来五年高 ROI 赛道集中在医疗收入周期自动化与患者准入、制造业结合既有工业 3.0 的增效、以及金融合规流程自动化与“下一最佳行动”模型。 - [42:31](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2551s) AI 的投资回报率具有“复利效应”（Compounding Interest），前期的稳健基础与正确选型会在中后期持续产生呈指数级放大的跨越式回报。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=ILOy8povAoQ&t=4s) AI 项目的投资悖论与高昂回报潜力 圆桌研讨会由 PayPal 商户报告与洞察负责人 Jyothi Srirangam 主持开场。她一开场就抛出了一个尖锐的现实问题：“如果告诉你超过 50% 的 AI 项目最终都会失败，你还会继续投资吗？” 多项行业研究表明，AI 项目的失败率是传统 IT 项目的两倍左右。根据 Gartner 的统计，超过 30% 的 AI 项目根本无法跨越试点（Pilot）阶段；更为严峻的是，预计到 2025 年底，超过 30% 的生成式 AI（Generative AI）项目甚至无法走出概念验证（PoC）阶段。导致大面积停滞的主要原因在于成本不断攀升以及商业价值模糊不清。 然而，企业对 AI 的投资热情并未因此减退，关键在于一旦项目成功，其所释放的价值将极其可观： - 带来 10% 至 15% 的生产力提升； - 实现 25% 至 30% 的运营成本缩减； - 投资回报率显著：每投入 1 美元，平均可带来超过 4 美元的收益回报。 因此，本次研讨会的核心议题便是探究如何区分那些真正带来高投资回报率（ROI）的成功项目与沦为昂贵学费的失败案例。 --- ### [01:42](https://www.youtube.com/watch?v=ILOy8povAoQ&t=102s) 跨行业专家阵容介绍 来自医疗、咨询、金融及企业级 AI 解决方案领域的四位专家依次介绍了自身背景以及 AI ROI 在日常工作中的结合点： - **Peter McCaffrey**：德克萨斯大学医学部（UTMB）首席 AI 官（Chief AI Officer），同时担任德州大学系统（UT System）旗下所有医院 AI 协作委员会主席。作为一名接受过 AI 与数据科学专业训练的病理学医生（MD），他的核心职责就是评估并交付医疗体系内各类 AI 项目的 ROI（无论是财务还是临床层面），场景覆盖从院内导航（Wayfinding）、停车管理、账单计费到复杂的临床诊疗。 - **Jeremy Steinert**：Turnberry Solutions 合伙人，负责数字化、数据与 AI 业务实践，长期协助企业客户评估 AI 用例、落地实施解决方案以及选型匹配的技术工具与合作伙伴。 - **Mona Hassanzadeh**：来自多伦多的加拿大金融机构 Meridian 数据科学与创新总监，在机构的 AI 指导委员会（AI Steering Committee）中发挥核心作用，专注于在负责任 AI（Responsible AI）与隐私影响评估（PIA）合规框架下，将数据科学转化为可衡量的业务价值。 - **Ravi Venugopal**：Giggso 创始人兼 CEO，公司专注于为大型企业提供可信 AI（Trustworthy AI）解决方案（涵盖安全性、伦理与负责任 AI），并通过“驻留企业家（EIR）”项目扶持初创公司推进 AI 战略。 --- ### [04:16](https://www.youtube.com/watch?v=ILOy8povAoQ&t=256s) 评估 AI 用例价值的决策框架与用户信任 面对“如何区分值得推进的 AI 场景与纯粹炒作”这一问题，Ravi Venugopal 分析了企业在 AI 成熟度上的两类典型画像：一类是勇往直前的早期采用者团队，另一类是已深耕 AI 领域的成熟团队。成熟团队在决策预算分配时，通常依托四维评估标准： ```mermaid flowchart TD accTitle: AI 用例评估与投资决策框架 subgraph ValueAssessment["价值与可行性评估"] A["业务价值与商业影响力"] B["数据就绪度与数据战略"] C["价值实现时间 (Time-to-Value)<br/>优先选择生命周期较长的用例"] end subgraph ExecutionCore["落地关键纽带"] D["用户采纳与文化转变<br/>(User Adoption)"] E["可信度与安全性<br/>(Trust)"] F["最终商业 ROI"] end A --> D B --> D C --> D D --> E E --> F ``` Ravi 强调，当前 AI 产品的技术迭代极快、生命周期急剧缩短，企业必须甄别出生命周期相对较长、能产生持久价值的项目。此外，在失败的 AI 项目中，近半数的问题都出在**用户采纳（User Adoption）**上。AI 落地本质上是一场组织文化变革，而用户是否愿意采纳完全建立在对系统的信任之上——“用户采纳建立在信任之上，而信任最终转化为 ROI（Trust leads to ROI）”。 --- ### [06:38](https://www.youtube.com/watch?v=ILOy8povAoQ&t=398s) 规避组织变革断层与协同推进 针对企业常见的实施误区，Jeremy Steinert 指出，很多团队最容易忽视的是技术方案背后的业务背景以及最核心的“人”的因素。人的行为具有不可预测性，仅仅开发出一款出色的技术产品远远不够，必须为用户创造接纳并融入该工具的组织环境与文化土壤。 在实践中，技术团队与业务团队往往存在预期错位： 1. **基础设施铺设期的期望管理**：AI 属于前沿领域，缺乏现成开箱即用的成熟架构范式，技术团队在早期搭建数据管道和底层架构时，难以在短期内拿出具象的业务成果。 2. **业务团队必须同步开展地面工作**：业务方不能被动等待技术交付，而应在系统开发的同时同步推进培训计划、组织变革管理（OCM）及沟通宣导，提前让员工队伍具备使用 AI 的能力。 --- ### [08:23](https://www.youtube.com/watch?v=ILOy8povAoQ&t=503s) 金融合规视角下的准入三要素与医疗场景的“业务需求民间传说” Mona Hassanzadeh 介绍了 Meridian 评估用例的严格准入机制。在受监管的金融行业，一个值得投资的 AI 用例必须满足三大核心标准： | 评估维度 | 核心要求与执行机制 | |:--- |:--- | | **真实业务价值** | 必须解决明确且可衡量的业务痛点，杜绝盲目跟风“炫酷”技术。 | | **合规与风控前置** | 设立跨职能跨部门的 AI 指导委员会，在需求接入期（Intake）统一进行负责任 AI、隐私影响评估（PIA）及风险可行性预审。 | | **运营就绪度** | 确保数据就绪度、可扩展的基础设施以及员工具备相应的技术应用素养。 | Peter McCaffrey 随后结合医疗系统的情况指出，许多组织内部充斥着大量关于业务痛点的“民间传说（Folklore）”——许多人主观笃定某个环节存在严重问题且能省下大笔开支（例如利用率审查、收入周期管理），但一旦深究具体数据（如留院观察率、住院率、医保拒赔率），往往发现根本没有基准数据，甚至完全缺乏测量手段。 Peter 警告道：“如果你在一个连效果都无法衡量的业务场景上砸下数百万美元去改造它，那么从一开始你就已经注定失败了。” ### [11:02](https://www.youtube.com/watch?v=ILOy8povAoQ&t=662s) AI 用例评估的底线：指标缺失是最大红旗 Peter McCaffrey 强调，在评估 AI 项目时，必须在一开始就明确到底要拉动哪项业务指标；如果连想要改善的底层基线（Baseline）都搞不清楚，那么就不能将其立项，否则整个项目最终只会沦为“讲故事的叙事练习”（inherently narrative exercise）。不过，对于极少数高度泛化、带有广泛赋能属性的基础工具（例如面向临床指南检索的 OpenEvidence 或泛临床 Copilot），组织可以给予一定宽容，不必强求精确计算每分钱的 ROI。这正如在企业内部“没有人会去问 Outlook 邮箱的 ROI 是多少一样”。但对于特定垂直业务工具，缺乏量化指标就是最危险的红旗，必须果断放弃。 主持人 Jyothi Srirangam 对此表示赞同，指出在推出新产品时往往缺乏基准数据。挑选用例只是第一步，团队接下来面临的核心抉择是时间维度的平衡：究竟是投资能迅速见效的短期速赢项目（Short-term wins），还是押注于数年后才能重塑业务的长期战略布局（Long-term strategic bets），以及如何在管理层急于看到短期回报时构建一个合理的 AI 投资组合。 ```mermaid flowchart TD accTitle: AI 用例评估与分流逻辑 subgraph S1["用例准入评估"] A["评估 AI 业务需求"] --> B{"是否有明确基线与量化指标？"} end subgraph S2["分流处理"] B -- "否 (泛化工具如 Copilot)" --> C["基于共识免除严格 ROI 计算"] B -- "否 (垂直业务工具)" --> D["视为红旗：果断放弃/停止"] B -- "是" --> E["进入 AI 组合分流"] end subgraph S3["组合管理"] E --> F["短期速赢项目 (Low-hanging Fruit)"] E --> G["长期战略押注 (Deep Transformation)"] end ``` --- ### [12:46](https://www.youtube.com/watch?v=ILOy8povAoQ&t=766s) Meridian 的平衡投资组合与合规博弈 来自 Meridian 的 Mona Hassanzadeh 分享了他们在金融机构内部平衡 AI 投资组合的实践策略。为了建立业务线（Line of Business）之间的信任和认可，Meridian 优先攻克“唾手可得的果实”（Low-hanging fruit），例如在全公司推广 Microsoft Copilot。这项举措不仅带来了普适的生产力提升，还产出了明确的可量化成效——Meridian 团队在代码采纳率（Code acceptance rate）上提升了 25%。 对于长期战略用例，Mona 列举了基于 AI 计算机视觉与端到端文档智能（Document Intelligence）的反洗钱（AML）与欺诈支票扫描检测（Scan check）。此类深度嵌入每笔交易的用例需要经过长期的投入与验证，所有项目都必须呈报给 AI 指导委员会（AI Steering Committee）。在委员会的审查中，负责任 AI（Responsible AI/风险合规）与预期投资回报率（ROI）必须进行严密的权衡与取舍，确保在稳健合规的前提下实现长期价值最大化。 - **短期用例成效**：全员推行 Copilot 辅助开发，代码接受率提升 25%。 - **长期用例落地**：利用视觉与文档智能处理交易级别的反洗钱（AML）与欺诈防范。 - **治理机制**：由 AI 指导委员会统一权衡 Responsible AI 风险与 ROI 之间的平衡。 --- ### [14:40](https://www.youtube.com/watch?v=ILOy8povAoQ&t=880s) 避免盲目复制神话：契合企业风险偏好与短期见效点 Turnberry Solutions 的 Jeremy Steinert 探讨了企业客户频繁要求“6个月内看到 ROI”的痛点。他指出，虽然咨询顾问更看重长期投资价值，但在特定场景下确实可以通过短期项目实现快速见效。例如：呼叫中心自动化（Call center automation）或医疗场景下的预先授权核验（Pre-authorization checks）。这些场景的共同特征是已有明确的流程和数据沉淀、重度依赖人力且 AI 能高效复刻该流程。 然而，Jeremy 提醒企业绝不能被外界吸睛的 AI 成功案例冲昏头脑，盲目跟风复制。AI 投资必须严格契合自身的风险承受能力（Risk tolerance）与运营实际需求。企业应建立可量化的短期里程碑 KPI，先在小范围内建立用户信任并驱动采纳率（User adoption），再逐步扩展规模，切忌在信任与成效显现之前进行过度投资（Overindexing）。 --- ### [16:07](https://www.youtube.com/watch?v=ILOy8povAoQ&t=967s) 医疗系统的代际跃迁与指数级技术债 在探讨医疗等高监管、长周期的行业时，Peter McCaffrey 剖析了推动长期战略部署的根本驱动力：AI 越来越强的自主代理能力（Agentic capabilities）和通用性，给传统架构带来了生存级危机（Existential question）。为了追逐眼前结果，现代医疗系统往往集成了多达 1300 多个分散的 IT 孤岛系统。 Peter 警告道，随着生成式与 Agent 级工具的进化，“产品”本身的定义正被重新书写——以往不同产品之间的界限不过是工作流中不同时刻的图形用户界面（GUI）。未来的 AI 将直接跨越（Leapfrog）掉现有的中间工作流，彻底消除无谓的繁琐操作。如果企业因为背负 1300 个集成系统和复杂的 ETL 管道，导致无法让生成式 AI 工具直接接触数据，那么在底层技术呈指数级发展的时代，企业所积累的技术债务也将呈指数级爆发（"Your technical debt is exponential"）。因此，长期的就绪度（Readiness）和架构重构远比下一个季度的短期财报回报更为关键。 --- ### [18:20](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1100s) 基础设施的双支柱：可扩展性与双重可观测性 Giggso 创始人兼 CEO Ravi Venugopal 从基础设施建设角度剖析了 AI 项目的可持续落地路径。他将底层能力拆解为“可扩展性”（Scalability）与“可观测性”（Observability）两大关键支柱： | 维度 | 核心关注点与实施策略 | 典型风险与案例 | |:--- |:--- |:--- | | **可扩展性 (Scalability)** | 从最小可行产品 (MVP) 出发，优先采用无头架构 (Headless architecture) 或云/边缘弹性方案进行资源调配。 | 某客户使用 Azure OpenAI ChatGPT API 进行电子邮件分类，由于未提前规划 Token 消耗，月底收到了天价账单。 | | **内部可观测性 (System Observability)** | 持续监控已部署模型的后台吞吐量、响应指标、推理质量与成本开销。 | 缺乏模型运行监控会导致后期维护成本不可控并出现严重的性能衰减。 | | **市场可观测性 (Market Observability)** | 紧密跟踪开源生态和底层基础大模型厂商的技术演进，防范“地毯被抽走”的颠覆风险。 | 许多公司投入数十万美元搭建可扩展网页抓取商业模式，但微软开源 NL Web 等工具后，其技术壁垒与商业模式几乎在一夜之间被归零。 | Ravi 强调，不仅要观察自己的系统如何运转，更要观察外部市场生态的巨变。企业如果花费巨资死守某项容易被底层大模型厂商原生覆盖的技术，一旦像微软这样的巨头推出开箱即用的开源工具，原本的商业模式就会顷刻瓦解。因此，技术基础设施的敏捷度与外部市场敏锐度必须同步建设。 ### [21:42](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1302s) 打造“AI 准备就绪”的员工队伍与多维度 KPI 衡量 许多企业在推进 AI 时往往直接扑向量化指标，却忽略了最核心的因素——“人”。与单纯的硬性数据不同，用户采纳度（User Adoption）和用户行为难以简单量化。很多企业目前的员工并不具备应用 AI 的能力，既不理解提示词工程（Prompt Engineering），也不掌握基础工具、数据策略和数据治理规范。为了解决这一短板，构建一支“AI Ready”的未来员工队伍至关重要。这不仅需要通过企业内部培训来实现，还可以直接走进高校招聘刚毕业的年轻人进行系统性培养，确保团队从一开始就具备与 AI 产品良好交互的能力。 在平衡合规、风险与业务价值时，企业需要兼顾财务与非财务指标。财务指标包括直接的成本节约、收入增长和生产力提升；运营指标则涵盖流程自动化减少的手工劳动，以及会员体验（Member Experience）与客户体验的改善。同时，负责任的 AI（Responsible AI / RAI）和风险缓释本身也是 ROI 与业务价值的一部分。通过设立“数据与 AI 治理及文化”专项团队，提升全员的 AI 素养（AI Literacy），推动管理层具备“AI 优先思维（AI-First Thinking）”，不仅能让业务部门主动挖掘用例，还能有效平衡传统手工操作中潜藏的风险与自动化技术引入的新风险。 ```mermaid flowchart TD accTitle: AI Value and KPI Measurement Framework subgraph Governance["治理与组织文化"] A["AI 治理与文化建设"] --> B["领导层 AI 优先思维"] B --> C["AI 素养普及与技能培训"] end subgraph Evaluation["评估与衡量指标"] C --> D["财务价值: 增收 / 节支"] C --> E["运营成效: 自动化 / 体验提升"] C --> F["风险与合规: 负责任 AI (RAI)"] end ``` - **财务类 KPI**：成本节约、增量收入（Revenue Uplift）、生产效率提高。 - **运营与体验 KPI**：流程自动化节约的人工工时、会员/客户体验评分。 - **风险治理 KPI**：负责任 AI（RAI）合规率、风险缓释效果及数据安全评估。 --- ### [25:15](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1515s) 垂直领域 KPI 差异与软件开发生命周期（SDLC）实践 不同业务领域追踪的 KPI 差异巨大。以软件开发生命周期（SDLC）为例，这是除运营支持外全球落地最为广泛的 AI 场景之一。在 SDLC 中，企业核心关注的是交付速度、代码质量以及人力与时间的缩减；而在客户支持与运营领域，核心指标则集中在工单数量（Support Tickets）、服务请求分流率（Case Deflections）以及工单结案率等具体维度上。 在医疗健康领域（如 UTMB），KPI 的追踪分为多个层次。日常基础指标关注 AI 的消耗量与活跃度，例如环境智能听写（Ambient Dictation）系统每天生成多少份病历、不同专科及医生的使用频率差异，以及床位状态裁定等任务中 AI 的调用次数。更深层次的 KPI 则与业务核心流程深度绑定，例如在收入周期运营与利用率管理中，利用智能体工作流（Agentic Pipeline）进行床位状态裁定（观察期转住院期），追踪其拒赔率与上诉成功率，每一次百分比的提升都能直接带来“实打实的新增资金 ROI”。 - **研发（SDLC）领域核心指标**：交付周期（Speed）、代码与产品质量（Quality）、人工作业耗时（Hours Consumed）及资源开销。 - **运营支持核心指标**：工单总量、客户响应表现、工单直接分流率（Case Deflections）。 - **医疗业务核心指标**：日常听写生成量、AI 床位状态裁定转化率、医保拒赔与上诉成功率。 --- ### [27:55](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1675s) 表面效率陷阱与业务驱动的预期管理 企业在衡量降本增效时常常陷入误区，即把“理论上的效率提升”误当成“真正的成本节约”。例如，引入 AI 让实验室技术员效率提升 10%，如果企业并没有实际减少 10% 的人员编制，这种效率提升对最终成本往往没有任何实质性的缩减影响。相比这种虚幻的内部节省，能够直接带来新收入的用例（如识别诊疗缺口、拓展医疗手术量等）往往更具商业吸引力和 ROI 确定性。 为了让技术团队、业务团队与最终用户在 AI 投资上达成共识，组织必须树立“AI 优先”的坚定信念。正如 2000 年代初期有人唱衰互联网一样，如今拒绝 AI 同样是不切实际的，“无论你是否想去学、想去用，它都已经到来，你只能选择像布鲁斯·班纳那样化身为绿巨人去拥抱变化，没有逃避的可能”。 在跨部门协作的摩擦点治理上，必须明确由业务部门主导业务问题并定义成功指标，技术团队则负责评估可行性并夯实底层数据治理。在传统软件领域“垃圾进，垃圾出”（Garbage In, Garbage Out）的规律在 AI 时代被指数级放大，因此数据策略和基础治理是协作的前提。此外，业务与技术团队之间往往在里程碑设定与优先级上存在脱节，引入值得信赖的第三方合作伙伴协助梳理路线图与对齐预期，往往能有效化解跨部门摩擦。 - **效率不等于省钱**：未转化为人员优化或产能扩充的单点效率提升，在财务报表上往往表现为“零实际节约”。 - **拥抱技术演进**：AI 转型如同当年的互联网浪潮，全员必须从被动接受转向主动的“AI-First”思维。 - **治理与预期对齐**：业务团队负责提出问题与成功标准，技术团队评估技术可行性，双方必须以坚实的数据治理为基石，严格管理交付预期。 ### [32:09](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1929s) 负责任 AI、隐私评估与金融业对“缩减 FTE”的顾虑 Mona Hassanzadeh 强调，建立负责任 AI（Responsible AI）和隐私影响评估（Privacy Impact Assessment）是让 AI 从虚幻走向现实的基石，能有效弥合大众误将 AI 视作“万能灵药”（silver bullet）与实际落地之间的认知差距。她进一步指出了当前在衡量 KPI 时普遍存在的避讳心理：在金融行业中，无论是团队领导还是普通员工，大家对公开谈论“节省全职员工（saving FTEs）”感到非常忌惮。 企业领导层竭力回避使用裁员或缩减人头作为指标，因为他们既不希望因引入 AI 而流失宝贵的劳动力，更担忧过分强调缩减 FTE 会引发员工对失去生计的剧烈恐慌，从而导致基层在日常工作中极力排斥和抵制使用 AI。Mona 直言，尽管 AI 旨在帮助员工提升日常工作效能，但“节省人力”依然是全社会不得不面对的现实；我们无法确切预测 AI 完全替代某些岗位还需要多久，但当务之急是主动调整心态、培养“具备 AI 认知与准备的工作思维”（AI-ready thinking），避免因被动抵触而在变革中一无所有。 ```mermaid flowchart TD accTitle: 落地 AI 面临的认知与指标困境 subgraph MindsetGap["认知与指标困境"] A["将 AI 视作万能灵药<br/>(Silver Bullet)"] --> B["负责任 AI & 隐私影响评估<br/>拉回现实基准"] C["KPI 指标过度聚焦缩减 FTE"] --> D["基层员工产生恐惧与抵触<br/>阻碍日常 AI 工具采纳"] end subgraph SolutionPath["破局之道"] B --> E["赋能员工日常决策与效率提升"] D --> F["培养 AI-Ready 思维<br/>实现人机协同共存"] end ``` --- ### [33:33](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2013s) AI 项目失败的教训：内部落地的盲区与尽早引入用户 主持人 Jyothi Srirangam 指出，尽管规划周详，但 AI 项目失败或未达预期是常有的事，而这些失败经验恰恰是塑造未来实施路径的最宝贵财富。她邀请 Jeremy Steinert 分享具体的挫折案例。Jeremy 坦言作为咨询公司，他们在帮助客户交付项目时总是全力确保成功，但最大的挑战往往发生在内部——即构建自己的 AI 工具和产品时。 Jeremy 将这种内部困境形容为“吃自己的狗粮”（eating your own dog food）。咨询顾问在服务客户时强调的方法论，在自身团队内部推进时却容易被忽视，导致开发出的 AI 产品缺乏明确量化的业务成果。Jeremy 特别剖析了“收益/好处”（benefit）与“核心价值”（value）之间的关键区别： - **价值（Value）**：严格以营收运营（RevOps）为导向，表现为硬性的“增加收入”或“削减成本”。 - **收益（Benefit）**：多属于非财务指标，能够为产品迭代和未来路线图提供参考，但由于缺乏核心量化锚点，往往极难精准证明其实际产出。 Jyothi 对此深表赞同，并补充道，除了未能明确定义业务成果之外，导致 AI 项目折戟的另一个致命原因是“没有在早期阶段邀请最终用户参与”。尽早让用户进入全流程，有助于在全面铺开之前及时发现差距、灵活调整方案，并与用户共同打造（co-create）真正契合实际痛点的产品，避免做无用功。 --- ### [34:41](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2081s) 跨越 Demo 与生产落地的鸿沟：摒弃 PoC，直奔 MVP 面对 Jyothi 提出的“演示（Demo）与实际生产落地之间存在巨大落差”的难题，Ravi Venugopal 进行了生动而尖锐的剖析。他指出，许多 Demo 往往只是工程师在后台用“热熔胶和电线”（wiring and gorilla glues）临时拼凑出来的产物，在受限环境下演示得天衣无缝。然而这种展示会使业务部门陷入极度亢奋，中层管理人员为了争夺预算，也会把高层情绪煽动得像“热铁皮屋顶上的猫”（a cat on a hot tin roof）。 ```mermaid flowchart LR accTitle: 从虚假 Demo 到生产级 MVP 的转化路径 subgraph FalsePromise["脆弱的 Demo 陷阱"] A["后台胶水拼凑 Demo"] --> B["规则引擎高度受限"] B --> C["过度承诺: 如 700 人裁至 200 人"] end subgraph RealWorld["真实落地路径"] D["设定底线预期与幻觉管理"] --> E["数据治理 + 完备的可观测性"] E --> F["构建敏捷交付的 MVP<br/>(放弃脆弱的 PoC)"] F --> G["融入 RevOps 营收运营验证并规模化"] end ``` Ravi 分享了一个极具代表性的真实案例： - **客户背景与盲目期望**：某大型企业拥有约 700 名客服人员专门处理支持邮件。在看完一段展示了 3 个高度受控用例的 Demo 后，管理层极其兴奋，提出试图在第一年内将 700 人的运营团队直接削减至 200 到 250 人的激进目标。 - **现实打脸与本质差距**：Demo 之所以完美，是因为它本质上是个“规则引擎”，把环境限制得死死的，几乎不可能出错；但一旦切换到真实的通用 AI，大模型必然会在各种复杂边缘场景下“到处出现幻觉”（hallucinating left, right and center）。 为了避免落地失望，Ravi 提出了务实的落地框架： - **理性预期管理**：明确承认 AI 输出具有不一致性的客观事实。 - **完备的基础设施**：提前部署成熟的可观测性工具包（observability toolkits）、严格的伦理与治理机制，并确保底层数据就绪。 - **坚决弃用 PoC，全面转向 MVP**：由于 AI 技术的“保质期（shelf life）非常短暂”，传统概念验证（PoC）不仅拖沓且容易失真。企业必须以最小可行性产品（MVP）快速验证能够支撑 RevOps 的实质性业务成果，再逐步扩展规模。 --- ### [38:30](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2310s) 未来五年的高 ROI 赛道：医疗营收周期、制造业与金融个性化决策 在探讨未来五年最具投资回报率的 AI 应用场景以及企业当前应打下的基础时，三位嘉宾从各自所在的垂直行业给出了深刻的前瞻分析： ```mermaid flowchart TD accTitle: 关键行业未来五年高 ROI 布局 subgraph Healthcare["医疗健康领域 (Peter)"] H1["打破数据孤岛与遗留系统"] --> H2["RevOps/收入周期全流程自动化"] H3["赋能患者自主获取与解读医疗信息"] end subgraph Manufacturing["制造业与基础设施 (Jeremy)"] M1["拒绝跳步，夯实工业 3.0/基础数据"] --> M2["引入机器学习与 AI 产生净收益"] end subgraph Finance["金融服务领域 (Mona)"] F1["基于合规流程的高确定性自动化"] --> F2["客户下一最佳行动 (Next Best Action)"] end ``` #### 1. 医疗健康领域（Peter McCaffrey） - **收入周期运营（Revenue Cycle Operations）的深度自动化**：医疗数据长期被割裂在传真、语音留言、老旧系统等碎片化渠道中。通过 AI 打通合同审查、账单提交、拒赔申诉等全流程管道，能极大清除行政复杂度。鉴于支付方（保险机构等）已在激进地使用 AI 审核，医疗服务机构若无法实现对等自动化，将在博弈中陷入严重劣势。 - **患者医疗准入与服务体验改善（Improvement of Access）**：现代医疗模式运作数十年，用户体验依然糟糕。随着 GPT 等通用大模型普及，患者开始尝试直接输入活检报告寻求治疗建议。尽管医生群体会对边缘案例心存疑虑，但患者对透明、便捷医疗服务的渴求不可阻挡，医疗系统必须主动拥抱这一变革，否则终将被时代视作阻碍。 #### 2. 制造业与底层架构（Jeremy Steinert） - 多数行业如果在没有打牢底层基础的情况下直接强行上马 AI 用例，不仅注定失败，还会积累庞大的技术债务。 - 制造业存在特殊机遇：许多制造企业目前仍在补足“工业 3.0”阶段的基础自动化与机器学习，如果能在这些既有流程中巧妙嫁接 AI，有望迅速释放可观的净收益（net benefit）。 #### 3. 金融科技与客户决策（Mona Hassanzadeh） - **合规框架下的流程自动化**：金融业面临严格的法规遵从要求，基于完备的流程文档与清洗好的数据来搭建自动化流程，既能满足合规，又能快速收获落地价值。 - **赋能人类决策的“下一最佳行动”（Next Best Action）**：高 ROI 的核心在于增强人类决策能力。Meridian 正在落地“会员下一最佳行动”模型，无论是用于个性化理财建议、预测客户预期，还是提供贴合个人生活场景的精准交互，都在创造显著的商业回报。 ### [42:31](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2551s) 结语快问快答：AI相亲红线、哈利·波特魔法棒与投资复利效应 在圆桌讨论的尾声，主持人以一组轻松幽默的快问快答（Lightning Round）为整场研讨会画上句号。当被问及“如果将 AI 比作约会软件（Dating App）上的个人资料，最大的‘危险红旗’（Red Flag）会是什么”时，Meridian 的数据科学与创新总监 Mona Hassanzadeh 毫不犹豫地给出了干脆利落的回答——“隐私”（Privacy），直接点破了当下企业与用户在拥抱 AI 时最敏感也最核心的顾虑。 随后，主持人向 Giggso 创始人兼 CEO Ravi Venugopal 抛出假设：“如果你手中有一根魔法棒，你会如何去修复客户最常犯的错误？”Ravi 幽默地借用了《哈利·波特》中的遗忘咒（Obliviate）打趣道，他最想用魔法棒重置并矫正客户那些脱离实际的预期，让大家清空不切实际的幻想并“脚踏实地”（"get them to ground"），回归理性的商业与技术本质。在随后的半句填空题“AI 的投资回报率（ROI）就像……”中，与会嘉宾将其精辟地比作“复利”（Compounding Interest）——只要在起步阶段将基础步骤做扎实、路径走对，随着系统与数据的持续沉淀，后续每一次推进都会带来跨越式、呈指数级放大的成效与飞跃（"get farther and farther leaps every time you do it"）。 当谈及客户带着诉求前来咨询时最普遍的 AI 落地场景，嘉宾坦言其实很难单一归纳，因为应用场景极其繁多且分散，而这恰恰构成了衡量与实现 ROI 的核心挑战。不过，这些多样化诉求背后往往存在一个共通的底层逻辑，那就是营收运营（RevOps）与收入周期（Revenue Cycle）优化。无论是降低成本、加速交付还是提升转化，各行各业寻求 AI 赋能的终极驱动力，归根结底都紧密围绕着商业变现与营收增长。 - **AI 相亲个人档案的最大红旗**：Mona 指出是“隐私问题”（Privacy），企业与用户对数据合规与安全漏洞高度警惕。 - **魔法棒愿望**：Ravi 希望施展“一忘皆空咒”（Obliviate）来消除客户过高且不切实际的预期，让 AI 落地回归务实基础。 - **ROI 的本质属性**：AI ROI 具有“复利效应”（Compounding Interest），前期的稳健投入与正确选型会在中后期产生加速放大的回报。 - **最核心的高频场景**：跨行业 AI 需求的最大公约数是 RevOps（营收运营）与收入周期管理。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=ILOy8povAoQ&t=4s) 如果告诉你超过 50% 的 AI 项目最终都会失败，你还会继续投资吗？ — Jyothi Srirangam > [04:16](https://www.youtube.com/watch?v=ILOy8povAoQ&t=256s) 用户采纳建立在信任之上，而信任最终转化为 ROI — Ravi Venugopal > [08:23](https://www.youtube.com/watch?v=ILOy8povAoQ&t=503s) 如果你在一个连效果都无法衡量的业务场景上砸下数百万美元去改造它，那么从一开始你就已经注定失败了。 — Peter McCaffrey > [11:02](https://www.youtube.com/watch?v=ILOy8povAoQ&t=662s) 没有人会去问 Outlook 邮箱的 ROI 是多少一样 — Peter McCaffrey > [16:07](https://www.youtube.com/watch?v=ILOy8povAoQ&t=967s) Your technical debt is exponential — Peter McCaffrey > [27:55](https://www.youtube.com/watch?v=ILOy8povAoQ&t=1675s) 无论你是否想去学、想去用，它都已经到来，你只能选择像布鲁斯·班纳那样化身为绿巨人去拥抱变化，没有逃避的可能 > [33:33](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2013s) 吃自己的狗粮 — Jeremy Steinert > [34:41](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2081s) 热熔胶和电线 — Ravi Venugopal > [34:41](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2081s) 到处出现幻觉 — Ravi Venugopal > [42:31](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2551s) 隐私 — Mona Hassanzadeh > [42:31](https://www.youtube.com/watch?v=ILOy8povAoQ&t=2551s) get farther and farther leaps every time you do it ## Mentioned Ai4, Azure OpenAI, ChatGPT, Gartner, Giggso, Jeremy Steinert, Jyothi Srirangam, Meridian, Microsoft Copilot, Mona Hassanzadeh, NL Web, OpenEvidence, Outlook, PayPal, Peter McCaffrey, Ravi Venugopal, Turnberry Solutions, UT System, UTMB ## Video description Peter McCaffrey, Chief AI Officer, The University of Texas Medical Branch Jeremy Steinert, Solution Partner, Turnberry Solutions Ravi Venugopal, CEO and Founder, Giggso Jyothi Srirangam, Head of Product - Merchant Reporting & Insights, PayPal Mona Hassanzadeh, Director, Data Science and Innovation, Meridian Abstracts This session will focus on how organizations can make strategic decisions early in their AI journey to drive strong returns on investment (ROI). Panelists will discuss the importance of evaluating potential AI use cases upfront, identifying opportunities that align with business goals, and assessing the feasibility and impact of each initiative. Learn how to set clear objectives, define success criteria, and make informed choices that lead to successful AI implementations. The discussion will also cover best practices for measuring ROI throughout the project lifecycle, tracking key performance indicators (KPIs), and refining AI systems to ensure ongoing value creation. Discover how a thoughtful approach to selecting and evaluating AI use cases can maximize both immediate and long-term success. [Source](https://www.youtube.com/watch?v=ILOy8povAoQ)

[0:04] Jyothi Srirangam: 如果告诉你超过 50% 的 AI 项目最终都会失败，你还会继续投资吗？ [4:16] Ravi Venugopal: 用户采纳建立在信任之上，而信任最终转化为 ROI [8:23] Peter McCaffrey: 如果你在一个连效果都无法衡量的业务场景上砸下数百万美元去改造它，那么从一开始你就已经注定失败了。 [11:02] Peter McCaffrey: 没有人会去问 Outlook 邮箱的 ROI 是多少一样 [16:07] Peter McCaffrey: Your technical debt is exponential [27:55] 无论你是否想去学、想去用，它都已经到来，你只能选择像布鲁斯·班纳那样化身为绿巨人去拥抱变化，没有逃避的可能 [33:33] Jeremy Steinert: 吃自己的狗粮 [34:41] Ravi Venugopal: 热熔胶和电线 [34:41] Ravi Venugopal: 到处出现幻觉 [42:31] Mona Hassanzadeh: 隐私 [42:31] get farther and farther leaps every time you do it

[4:16] Ravi Venugopal: User adoption is based on trust. Trust leads to ROI. [8:23] Dr. Peter McCaffrey: If you dumped millions into fixing that and you can't measure it, you already have failed. It's impossible to prove that you touched the needle at all. [8:23] Dr. Peter McCaffrey: It'd be like if someone asked what's the ROI of Outlook? Nobody asks that in the enterprise. [16:04] Dr. Peter McCaffrey: your technical debt is exponential. [28:34] Ravi Venugopal: It's like going back to the 2000s and naysaying that the internet is not going to be a big success... You either become Bruce Banner becoming the Hulk, or you don't want to do that at all. There is no chance you can escape the face of AI. [28:34] Jeremy Steinert: eating your own dog food [36:07] Ravi Venugopal: gorilla glues [36:07] Ravi Venugopal: like a cat on a hot tin roof [42:27] Ravi Venugopal: Obliviate [42:27] compounding interest

[0:04] Good morning everyone. Welcome to the [0:06] driving AI ROI panel today. My name is [0:09] Joti Sridangum, head of merchant [0:11] reporting and insights at PayPal. And [0:14] today I'm going to be your moderator for [0:16] the next 45 minutes. So let me begin [0:19] with a question. If I tell you that more [0:22] than 50% of AI projects fail, would you [0:26] still invest in them? [0:31] research consistently shows that more [0:34] than um the failure rate of AI projects [0:37] is twice as high as traditional IT [0:40] projects. Gartner reports that more than [0:43] 30% do not make it beyond a pilot stage. [0:47] And here is the kicker. By end of 2025, [0:51] more than 30% of generative AI projects [0:54] do not make it beyond proof of concept [0:56] stage. Why? escalating costs and unclear [1:01] business value. But here is the good [1:04] news and looks like most of you already [1:05] know and that's why you're into it even [1:07] if it's the failure rate is high. When [1:10] AI projects succeed, they really [1:13] succeed. [1:15] More than 10 to 15% productivity gain, [1:18] 25 to 30% cost reductions, and for every [1:22] single dollar invested, more than $4 of [1:26] return investment. [1:28] So what separates these successful AI [1:31] projects from those costly learning [1:34] experiences? [1:36] That's exactly what we are going to [1:38] discuss here today. With that, I welcome [1:42] our distinguished panel members who [1:44] bring in diverse perspectives from [1:47] financial services, healthcare, [1:50] enterprise solutions and AI technology. [1:53] Please introduce yourself with your [1:55] name, company, and how AI ROI intersects [1:59] with your work today. [2:00] >> Absolutely. Hey everyone. Um, it's nice [2:02] to be able to talk to you all today. So, [2:04] I'm coming from Texas. I'm the chief AI [2:06] officer at the University of Texas [2:08] Medical Branch and I also, uh, chair our [2:10] AI collaborative across all of our UT [2:13] system hospitals. I'm a pathologist by [2:16] training, an MD with additional training [2:18] in AI and data science. And really my [2:20] job in a nutshell is to try to decide [2:23] how we deliver ROI for AI, whether it's [2:26] financial, clinical, whatever across [2:27] anything from like wayfinding, parking, [2:30] billing, clinical care, and so forth. [2:32] And so think about it every day. [2:35] >> Jeremy. [2:36] >> Hey everyone. My name is Jeremy Steiner. [2:38] I'm a partner at Turnberry Solutions. I [2:40] lead our digital and data and AI [2:42] practices. Um, you know, we work with [2:45] enterprise clients, you know, every day [2:48] helping them to evaluate our AI use [2:50] cases to implement AI solutions, uh, to [2:53] choose the right AI tool and and and [2:55] partners. Um, so we, you know, work with [2:58] it consistently, uh, every day. So, [3:01] great to be here talking to everybody. [3:03] Thank you. [3:05] >> Good morning, everyone. My name is Mona [3:07] Hassan and I'm the director of data [3:09] science and innovation at Meridian. I [3:12] follow from Toronto. So I'm Canadian in [3:16] US. Um uh as my role actually play a key [3:21] central in selecting the use cases and [3:24] also the ROI investigation as part of [3:27] the AIS committee. This is how we [3:30] translate data science into [3:33] measurable meaningful business impact [3:36] while considering responsible AI and [3:38] privacy impact assessment. Thank you. [3:41] You want to Hey everybody, I'm Ravi [3:43] Vinagopal, founder CEO of Gigzo [3:45] Incorporate. Uh we are from Michigan and [3:48] our company provides trustworthy AI [3:51] solutions for very large enterprises and [3:53] we also have a uh entrepreneur and [3:56] residence program where we help startups [3:58] see themselves and uh progress towards [4:01] AI initiatives. We focus purely [4:03] surrounding trustworthy AI which is [4:06] including security and ethical AI and [4:08] responsible AI and uh we help our [4:11] customers uh find ROI in AI in the [4:13] initiatives that they are on. Thank you. [4:16] >> Awesome. So let's dive right in. Let's [4:19] start with a fundamental question. Every [4:22] organization wants ROI on AI but not [4:25] every problem is worth solving for. So [4:28] what separates these AI use cases that [4:30] are worth pushing for from the ones that [4:33] are just hype? Ravi, I would start with [4:35] you. From the various clients you work [4:38] with, do you see a standard framework [4:41] that clients use to decide which AI use [4:44] cases they need to select? [4:46] >> Oh, that's a prodical question. It's a [4:48] it's it's probably what everybody is [4:50] leaning towards finding where what what [4:52] do I do and with my budgets that I have, [4:54] right? So there are two types of teams [4:56] that are there. one is a uh go-getter [4:58] teams or the the early adopter teams and [5:01] the other part of the teams are much [5:03] more mature. They have definitely been [5:05] on the world of AI and they do believe [5:07] in what you mentioned earlier. Uh even [5:09] if 50% projects only succeed still they [5:12] need to go forward. So how do they make [5:14] the decisions on which projects have [5:16] better ROI and which ones to invest [5:18] their cash on, right? So they look at [5:21] business value and business impact and [5:23] then they look at data uh readiness and [5:26] the data strategy that they have and the [5:28] third is time to value because AI if you [5:31] look at it right every single product [5:32] out there has got a shelf life that is [5:35] vanishing so fast. So they want to see [5:37] which products they can or projects that [5:40] they can invest on which has got a [5:42] longer shelf life considered [5:43] comparatively and see how their money [5:46] can go into those and finally the most [5:48] important part companies large medium [5:51] mid mid-market the way they look at AI [5:54] is how does the user adapt to this AI [5:57] because there's a cultural shift and [5:58] there is also a user adoption problem [6:01] almost 50%age of the projects that [6:04] actually fail in that if to take a a [6:07] clearer look at it. Most of them have [6:09] user adoption issues and the way a lot [6:13] of matured AI companies are looking at [6:15] are to see how user adoption plays an [6:18] important role towards the ROI. So user [6:21] adoption is based on trust. Trust leads [6:24] to ROI. That's how I would uh counter [6:27] that question. I really like that you [6:30] brought up longer shelf life uh trust [6:32] and adoption which are really important [6:35] in evaluating the use cases. Um Jeremy [6:38] in your work across different industries [6:41] do you see common pitfalls which um [6:44] organizations or companies do not [6:46] consider and how do you help them avoid [6:48] it? [6:49] >> Yeah, thank you. I I agree with a lot of [6:51] what Robbie said. You know a lot of the [6:53] challenges is not understanding the why [6:55] and the business context behind the AI [6:57] solution and you know not understanding [7:00] the human element. You know human [7:01] behavior is unpredictable and you know [7:04] the organizational change management and [7:05] the culture shift is is critically [7:08] important for AI adoption and and to [7:10] showcase ROI. Um you know it's not just [7:13] enough to build a great product and a [7:15] great tool. You have to create the [7:17] conditions for users to adopt and [7:18] embrace it. Um so that's one of the [7:20] biggest challenges and and we see you [7:23] know we one of the things we train and [7:25] coach our clients on is you know setting [7:27] realistic expectations and managing [7:29] those expectations between the technical [7:31] and business teams and you know a lot of [7:33] times the technical teams have a lot of [7:36] groundwork to lay for infrastructure and [7:38] data and other things which you know AI [7:41] being so cutting edge there's not a lot [7:43] of well- definfined patterns and and [7:44] architecture uh readym made out of the [7:47] box so it's sometimes hard to set firm [7:49] from milestones and timelines and you [7:52] know often times those you know initial [7:54] stages don't have a tangible business [7:56] outcome that the business team might be [7:58] looking for. So one thing we encourage [8:00] is the business team has groundwork to [8:02] do too. You know there's training [8:04] there's user you know adoption there's [8:06] organizational change management and [8:08] communication uh changes that need to [8:10] occur in in tandem with the roll out of [8:12] the solution. So you know don't just [8:14] wait for a technical you know solution [8:17] to be ready. Uh that's a time to get to [8:19] work and start to you know get your [8:21] workforce ready as well. [8:23] >> Interesting. Mona, you come from a [8:25] regulated financial services industry. [8:28] How do risk and compliance factor into [8:31] your evaluation of ROI and help you [8:34] select the right AI opportunities. So [8:37] every use cases worth pursuing if it has [8:40] three critical criteria in our [8:42] organization. The first one of course it [8:45] has to have meaningful measurable [8:47] business problem that we are solving. So [8:50] we have to have an actual business [8:52] problem. It's not just because this is [8:54] fancy or exciting use case let's go for [8:56] it. We go through the solving a problem [9:00] which business uh bring to us for that [9:03] purpose we're also looking at it through [9:05] the lens of responsible AI and PIA but [9:08] also we do have AI steer committee. So [9:12] one thing that my organization built and [9:14] uh emphasize on that is having this com [9:17] um having this committee and crossf [9:19] functional squad to help with the [9:22] standardized intake review of all of the [9:24] use cases. So making sure this is risk [9:27] aware feasibility study has been done [9:30] ahead of time when the intake coming [9:32] into the place. In addition to that of [9:35] course we have to ensure operational [9:36] readiness. So AI data ready as Ravi [9:39] mentioned um AI infrastructure ready [9:43] scalable system and the process as well [9:46] as of course the workforce readiness for [9:48] that to ensure that we have a [9:50] responsibility responsibly build um all [9:53] of the use cases that we have. [9:56] >> Fascinating excitement is good but I [9:58] think if it's not tied to the right [10:00] business outcomes like you mentioned [10:02] it's just experimentation. [10:04] Um, Peter, healthcare really raises the [10:07] stakes. So, how do you evaluate the [10:10] right AI uses? [10:12] >> Yeah, I uh want to echo what I think all [10:14] three of you said, which is that when we [10:17] look across our healthcare enterprise, [10:18] both our our health system and even [10:20] other UTL systems, there's a tremendous [10:22] amount of what I would say is folklore [10:25] around needs and business problems. A [10:28] lot of people think there's a problem [10:31] but can't actually quantify it. So for [10:34] example, if we sit there and we say, [10:35] "Oh, I'm I'm sure that utilization [10:38] review and revenue cycle is must be [10:40] worth a lot of money." Okay, well, how [10:43] much? What's our obs, impatient rate? [10:46] What's our denial rate? Do we know that [10:48] information? And we constantly find out [10:49] that we don't really know it or if we [10:52] know it, we have not a clear way to [10:53] measure it at all. And this is a problem [10:55] because if you dumped millions into [10:57] fixing that and you can't measure it, [11:00] you already have failed. it's impossible [11:02] to prove that you touched the needle at [11:05] all. And so we try to help and [11:08] evangelize the folks that if we don't [11:10] understand the metric we're trying to [11:12] move from the get-go, then we're not [11:16] even going to consider that as a use [11:18] case because it it will just be a [11:20] inherently narrative exercise. I think [11:23] the the one exception of some of this [11:25] comes to things like [11:28] you can have a high conviction on sort [11:30] of a narrative basis. So something like [11:32] you know open evidence or clinical [11:34] co-pilots getting better guideline [11:36] retrieval in the hands of the workforce. [11:37] Okay, fine. Um, but for some of these [11:40] things, we carve out a willingness to [11:43] not necessarily have a cleanly tabulated [11:46] ROI in a super diffuse use case like [11:48] co-pilots because it'd be like if [11:50] someone asked what's the ROI of Outlook, [11:53] nobody asked that in the enterprise. And [11:55] so it's like, okay, so some things we're [11:56] willing to kind of absolve, but for a [11:59] targeted tool, we constantly find we [12:02] just don't know the underlying metric to [12:04] begin with. And I think that's the [12:06] biggest red flag to just stop and move [12:08] on to something else. [12:10] >> Great point. We may not always have a [12:12] baseline for new products even when we [12:15] want to measure it exactly. Um so [12:18] selecting the use cases I would say is [12:20] just the first step. The next challenge [12:22] is timing. Do we invest in short-term [12:25] wins that show value fast or long-term [12:28] strategic bets that can transform the [12:31] business over the years? Um, how do you [12:35] create an AI portfolio of use cases? And [12:38] when your leadership is looking for [12:41] near-term results, how do we justify the [12:43] mix? Mona, I would like to start with [12:45] you. [12:46] >> Yeah, sure. So, in our organization, we [12:49] have a balanced portfolio. We always [12:52] call something like lowhanging fruit. So [12:54] we definitely wanted to tackle those get [12:57] the buy in and the trust from our you [12:59] know other lobs within the financial [13:02] organization. Um so one of those [13:05] lowhanging fruit was implementing or [13:07] using co-pilot within Meridian. So every [13:10] individual is using copilot which is [13:13] bringing productivity efficiency and so [13:16] on so forth. We were able to quantify um [13:20] actually the outcome of that. So we had [13:22] 25% [13:24] uh higher acceptance rate in terms of [13:26] our coding. So in that case yes we had [13:30] the measure of the success. So we try [13:33] every every use case comes to us come [13:36] back with the quantify measurable impact [13:39] and the success rate. Some of the use [13:42] cases will consider the long-term use [13:44] cases. Those are like an example in in [13:48] our organization is we are doing scan [13:51] check uh for the AML and fraud detection [13:54] using AI vision and document [13:56] intelligence end to end to each [13:58] intransactional [14:00] information that we have. these [14:02] information these use cases consider [14:05] long-term ROI uh which all of them will [14:08] go again through the AI steer committee [14:11] where the responsible AI on one hand the [14:14] ROI on the other hand should have a [14:17] tradeoff of everything so making sure it [14:20] has enough ROI so we will balance [14:22] everything out of that [14:25] very interesting Jeremy how frequently [14:28] does someone come up to you and say I [14:31] need ROI in next six months and how do [14:34] you help them balance set them up for [14:36] success both in the short term as well [14:38] as long term? [14:40] >> Yeah, it's a it's a challenge we see [14:41] often working with enterprise clients [14:43] and you know we value long-term [14:46] investment over the shortterm um you [14:49] know attempt at at at ROI and gains. Um [14:52] but you know I think that there's an [14:54] opportunity for the right use case to [14:56] provide you know short-term gains. You [14:59] know, some examples could be, you know, [15:00] call center automation or um [15:03] pre-authorization checks for healthcare [15:04] and and simple things like that where, [15:06] you know, humans are involved today [15:08] that, you know, the the data and and [15:10] process are well defined and and AI can [15:13] uh replicate those fairly efficiently. [15:16] Um, but we we really try to anchor [15:18] clients to making investments that are [15:20] aligned to their risk tolerance and [15:22] their operational needs and not just [15:24] trying to replicate success stories that [15:26] they see elsewhere. Um, and I think [15:28] that's a big challenge where, you know, [15:30] the the success stories are are are very [15:33] enticing and everybody wants to capture [15:36] the same kind of outcomes. Um, but, you [15:38] know, really making sure that we're able [15:40] to have quantifiable KPIs that we can [15:44] measure, you know, short-term milestones [15:46] versus a long-term strategic initiative. [15:48] Uh, build trust early and then scale [15:51] from there. You know, making sure that, [15:53] you know, we're not overindexing on [15:56] anything. uh before the the trust and [15:59] the user adoption starts to show [16:01] results. [16:04] >> That's really nice. Um [16:07] in healthcare um it takes a long time to [16:10] validate and roll out but the impact [16:12] could be lifesaving on the patients. So [16:15] how do you push for these longerterm [16:17] initiatives which may not show any [16:19] result in the near term? [16:21] >> Yeah, very good question. Um so I think [16:23] one of the biggest catalysts to pushing [16:25] for these to the deferred benefit of [16:27] longerterm solutions is that in [16:29] healthcare at least the disruption [16:32] positive disruption I think brought [16:34] about by the increasing agency and [16:37] generality of especially geni tools and [16:39] aentic tools raises like an existential [16:42] question that we have to face. So for [16:44] example, the pressure to have something [16:46] right now and show it is one of the [16:48] reasons why when you look across a [16:50] health system, there's like 1,300 IT [16:54] systems that are integrated. And what we [16:56] really kind of express a lot to our [16:58] leadership is if you continue to play [17:00] out the advancement and the generality [17:03] of co-pilot tools that can be more and [17:05] more like automated tools and agentic [17:08] tools, you begin to question like what [17:11] is a product actually? like what is the [17:13] the actual difference between this [17:16] product and that product if ultimately [17:18] they were a guey that fit into my [17:21] workflow at that moment in time. And the [17:23] way we kind of express to our leadership [17:26] to look at it is that rather than trying [17:28] to say the workflow is totally [17:30] sacrosanked and you have to be satisfied [17:33] and your workflow now is still helping [17:34] you now that we will probably leaprog [17:36] entire workflows that nobody will do [17:39] anymore. And that is an homogenizer of a [17:42] technology I think. However, if you are [17:46] impeded in your direct ability to deploy [17:49] the generative tools on the data because [17:51] you have 1300 different systems and [17:53] integrations and ETLs. Now, if the [17:56] technical pace is exponential and you [17:59] fall off of that, your technical debt is [18:01] exponential. And this is usually a [18:04] sobering message that has made people [18:07] realize okay we we have to understand [18:09] that readiness is more important even [18:12] than like next quarter return because if [18:14] we mess this up the damages are going to [18:16] be really severe and compounding over [18:18] many many years. [18:20] >> You brought up a very good point on tech [18:22] debt. So moving on to you Ravi. How [18:26] would you help organizations invest in [18:28] the infrastructure? Because it's not [18:31] just about rolling an AI project. It's [18:33] about observability, scalability that [18:35] can only give results in the future. Um, [18:38] what do you advise your clients and how [18:39] do you go about it? [18:41] >> So, uh, that's an interesting question, [18:43] right? You brought about observability [18:44] and scalability, right? So, there are [18:46] two two different components to that. [18:48] Let's talk scalability first, right? So [18:50] when you build uh a product or a [18:53] solution for finding ROI in a specific [18:57] project now you look at the minimum [18:59] viable project product and then scale [19:02] from there that's the normal process for [19:05] getting quick wins when you build your [19:07] minimum viable product. You always have [19:09] to look at a scalable term of how do you [19:12] build a scalable infrastructure mostly [19:15] using headless architecture or you're [19:17] using more uh cloud oriented or if it's [19:20] not a cloud if it's more edge how do you [19:22] scale every single component so that is [19:25] the most important part because you have [19:27] to plan and adjust your scalability [19:30] needs and based on that you provision [19:32] for infra like I know of customers who [19:34] have used uh Azure OpenAI chat GPT the [19:39] uh token is for classification of emails [19:42] and they did not like what they saw at [19:43] the end of the month when the bill came [19:45] out. So you have to plan ahead, you have [19:47] to think through every single component [19:49] and then look at the scalability factor. [19:51] But the other portion of it is the more [19:53] important part observability. So when [19:55] you talk about observability, right? So [19:56] you deploy a specific model or a [19:59] specific project, you've got to keep [20:01] constantly watching it. If you do not [20:03] watch it and you don't keep uh an [20:05] analysis and the metrics of what is [20:07] happening behind the screens, you're [20:09] going to be losing out on the longer run [20:11] and it's not observability. There are [20:12] two parts of observability that I would [20:14] say. One is the observability pertaining [20:16] to the project or the product that [20:17] you've built. And the other thing is [20:19] observing how the market is behaving for [20:22] that specific use case. Why I'm saying [20:24] that is I'll give you a simple example. [20:27] There are a lot of companies that did [20:29] web scraping projects, right? Lots of [20:31] them. They've made so much of uh you [20:33] know uh business models based on web [20:35] scraping and u scalable web scraping [20:38] stuff. Microsoft launched a product open [20:41] source toolkit called NL web about a few [20:44] weeks ago. It literally made all the [20:47] business models go away. And NL web is [20:50] out of the box super easy and does magic [20:53] literal magic. So every single company [20:56] that's business model that's been built [20:57] on if they have not been observing [20:59] what's happening in the market they're [21:01] dead. So the point is if companies have [21:03] invested thousands of dollars hundreds [21:05] of thousands of dollars trying to build [21:08] scalable systems banked on specific core [21:11] technologies and then Microsoft came in [21:13] with the monkey wrench inside they're [21:16] gone. So observability for how your [21:18] product is working how the market is [21:19] working as well those two points have to [21:21] be considered. [21:23] Great. So, um whether it's short-term [21:26] win or long-term bet, you still need to [21:28] prove the value and measure it. Um so, [21:32] Jeremy, in your work across industries, [21:34] when you help them set up a measurement [21:36] framework, what do you see clients [21:38] usually missing and how do you help [21:40] them? [21:42] >> Yeah, I mean, I think the number one [21:44] thing that's missing is the human side. [21:46] you know, we can quantify metrics and a [21:49] lot of organizations dive straight to [21:50] metrics. Um, but we can't quantify user [21:54] adoption and you know, user behavior as [21:57] easily. And so, you know, a lot of [22:00] organizations today uh don't have AI [22:03] ready workforces. They don't understand [22:05] prompt engineering. They don't [22:06] understand the basics tools and data [22:08] strategy and governance needed to really [22:10] get take advantage of these tools. And [22:13] so I think you know a big component is [22:15] making sure that you know building that [22:18] AI ready workforce to better have a [22:20] better understanding of how adoption and [22:23] how you know users will interact with [22:25] the the product is a key component that [22:27] is often missed. Um, so you know, we [22:31] often work with clients on training [22:33] programs or, you know, we actually [22:35] recruit out of universities directly and [22:38] train, you know, people straight out of [22:40] college to make sure that we're we're [22:41] building the workforce of tomorrow. Um, [22:44] so I I think that's something a lot of [22:45] organizations need to adopt a as well as [22:47] they start to roll out AI initiatives. [22:50] >> Completely agree there. Um, and Mona, [22:53] you have to balance compliance, risk, [22:56] and also business value. what are some [22:59] financial and non-financial outcomes and [23:01] KPIs you measure for your projects? [23:04] >> Um so financial KPIs as like you know [23:07] the costsaving uplift of the revenue, [23:11] productivity and so on so forth. We do [23:14] have operational um metrics uh such as [23:17] you know process automation saving you [23:20] know all of this manual work that we are [23:23] doing. On the other hand, we have the [23:25] operation outcomes which is like a [23:28] member experience or the client [23:30] experience. Um the risk mitigation um [23:34] all of this are improving the [23:36] compliance. So in addition to the [23:38] technology side of it or the revenue [23:41] side of it, we also looking at the risk [23:43] mitigation to ensure the RAI uh which we [23:47] call responsible AI is actually part of [23:50] the business value in the ROI that we [23:53] are building. For that purpose, we [23:55] actually have another team of the um so [23:58] similar to the data science and [24:00] innovation at Meridian, we have the data [24:04] and AI governance and culture. So [24:06] specifically investing on that scenario, [24:09] making sure we have a right data and AI [24:12] strategy, we have a right AI literacy, [24:16] empower and educate our leaders and the [24:18] workforce to ensure they leverage AI [24:21] every day and also they are AI first [24:25] thinking. So everything that they are [24:28] doing they are thinking about how I can [24:31] improve it. If the leaders think ahead [24:34] of time like that so they encourage [24:37] everybody within that lob to do the [24:40] same. So in that case they can bring up [24:42] with the use cases they ensure the risk [24:44] is mitigated for all of the job that [24:46] they are doing because I would say there [24:48] is a balance between all of this manual [24:51] work that we are doing which we bring a [24:54] lot of risk into the organization along [24:57] with the automation and AI which will [24:59] have its own risk but if we assess all [25:02] of those and ensure that we are [25:03] mitigating that it would have been a [25:06] better solution. Awesome. Is there [25:09] anything you want to add to that? What [25:11] are some KPIs that you see your clients [25:13] asking for consistently? [25:15] >> Yeah, KPIs depend on the business [25:17] domain, right? So, what specific [25:18] business domain they have and based on [25:20] the domain? So, financial KPIs are [25:23] little different what she had explained [25:24] was pretty much bang on target about the [25:26] numbers because ultimately all KPIs go [25:29] back to numbers. So, it's uh it's it's a [25:31] given. But from an SDLC point of view, [25:34] uh if you're looking at KPIs that people [25:36] look for in SDLC, which is probably the [25:39] most adopted use case across the globe, [25:41] uh outside of operation support, uh in [25:45] SDLC, people look for speed and quality [25:48] and delivery and they're looking to [25:49] reduce the number of people trying to [25:51] get the job done and the time in which [25:54] you can see value come out right away. [25:56] So that in SDLC I see a lot of uh [25:59] metrics surrounding hours and uh [26:03] resources that are being consumed to uh [26:05] deploy a certain project or a product. [26:08] So that's what I would say. I mean it [26:10] depends on the domain like the number of [26:12] support tickets for example in the [26:13] operation side of it. That's a very key [26:15] metric that people use for uh gauging [26:18] their ROI in a specific uh support [26:20] operations need. uh case deflections are [26:23] very important use cases uh and that is [26:26] a place where a lot of metrics are used [26:28] for seeing how customers are responding [26:31] how many tickets were closed those are [26:33] some of the metrics that are being used [26:34] basically it's split on business domain [26:36] so each domain has got different [26:37] variables to it and that's how I put it [26:41] >> and uh Peter are there any KPIs you [26:43] track daily [26:45] >> yeah so I think a couple of layers one [26:48] it's important is just your volume of [26:51] consumption of AI. So for example, if we [26:52] use we use ambient dictation as many do [26:54] at UTMB and so how many notes are [26:57] generated every day and what specialty [26:58] by whom? Who does it more who does it [27:00] less? This kind of simple thing how many [27:02] AI adjudications of a bed status do we [27:05] do every day just are we engaging? Can [27:08] it be an attributable driver but then [27:10] each example has its own use case. So [27:12] for example like um we've deployed a lot [27:14] of AI into our revenue cycle operations [27:17] especially in utilization management. So [27:19] bed status adjudication is an agentic [27:21] pipeline at UTME and we closely track [27:24] you know were did we convert from [27:26] observation inpatient was that denied [27:29] did we appeal it was it consistent can [27:31] we use that to train the model and that [27:33] that number we know if we move that [27:34] percentage there's a direct new money [27:37] ROI in and so these are appealing KPIs [27:40] some of the other kind of KPIs that are [27:43] important like outcomes are you know [27:45] notoriously difficult to actually [27:46] calculate healthcare would be in general [27:49] if it was easier to calculate those [27:50] things. So that is a difficult thing to [27:52] do well. Not that we shouldn't try. I [27:55] think one of the things that is commonly [27:57] misunderstood in the enterprise is that [27:58] savings KPIs [28:00] are not always that exciting actually [28:03] because for example if you say I spent [28:05] all this money to deploy an AI to make [28:07] my laboratory technician 10% more [28:09] efficient. I guess that's great if their [28:11] day is more enjoyable but did it did you [28:13] like not that you would want to do this [28:14] but do you have 10% fewer lab techs? No. [28:17] It often has zero material impact on [28:19] savings, theoretical savings with actual [28:22] cost. And so the thing is, can I show [28:24] you revenue generating opportunities? [28:26] Maybe it's gaps in care, procedures, and [28:28] so forth. And then we can kind of move [28:31] down the down the stack from that. [28:34] >> So churn reduction uh or savings may not [28:38] look as attention grabbing as revenue [28:40] growth. And I also think just from my [28:42] experience non-tangible metrics like [28:46] whether it's user adoption or customer [28:48] delight and experience are super [28:50] important just like written on [28:51] investment and revenue growth. Um so you [28:55] have selected the use cases and now we [28:57] have a measurement framework but the [28:59] project will not succeed till you bring [29:02] all your teams to align on a shared [29:04] vision and uh how do you do that in your [29:08] business where you're dealing with [29:09] technical teams business teams end users [29:12] and make sure they're part of the [29:14] journey and aligned on the investments [29:16] you're making on the AI projects. Ravi I [29:19] would go with you. That's a it's a it's [29:21] a very difficult question to answer but [29:23] it's so simple like what Mona was [29:24] mentioning you need to have the thought [29:26] process of AI first and whether people [29:28] like it or not AI is here we are already [29:30] here it's like going back to 2000s and [29:33] nay saying internet is not going to be a [29:35] big success right so you look back at it [29:38] and that is really ridiculous but today [29:40] we are here and it is a conscious [29:42] thought process everybody has to start [29:44] using it and everybody has to start [29:47] adapting to AI and So whether you not [29:50] whether you want to learn it or whether [29:52] you want to use it or not, it is coming [29:53] for you. So you either become Bruce [29:56] Banner becoming Hulk or you don't want [29:58] to do that at all. So it's there is no [30:00] chance that you can escape uh the face [30:02] of AI. So it is AI thinking first. [30:05] Everybody and everyone has to start [30:07] adopting to it. It is here to stay. So [30:09] that's how I'd say it. [30:11] Um Jeremy, do you see a common friction [30:13] point where there's misalignment between [30:16] the business and technical teams and how [30:18] do you help bridge those gaps? [30:20] >> Yeah, we we first want to understand, [30:22] you know, who owns the use case and [30:24] typically it is the business that owns [30:26] it. Um you know, I talked a little [30:29] earlier about the groundwork that needs [30:30] to be laid on the technical side to [30:32] really take advantage of solutions at [30:34] least when building more custom [30:35] solutions or implementing a larger tool. [30:38] Um, you know, I think the the old [30:42] software paradigm of of garbage in, [30:44] garbage out is just accelerated at scale [30:46] with AI. And so, we need to make sure [30:48] that there's a strong foundation of data [30:51] strategy and governance to to start and [30:53] a and a you know, strong process and and [30:56] you know, organization uh communication [30:59] to to build on. Um so you know a lot of [31:02] the friction points come with um you [31:06] know how are milestones being tracked [31:08] and how are expectations being set [31:11] between the business and the technical [31:12] teams. And a lot of times we find, you [31:15] know, choosing a trusted partner to help [31:18] with that process is a net benefit [31:22] simply because of, you know, the [31:24] difference in priority between the [31:26] technical and the business team and and [31:27] having, you know, someone that can kind [31:29] of cut through all of it to define the [31:32] the road map and milestones more [31:34] effectively might might be a benefit [31:36] overall. [31:38] >> Is there anything you want to add to [31:40] that? I'm sure you have compliance risk [31:42] users. business and technical teams. How [31:45] do you help bridge the gap and align [31:46] them? [31:47] >> For sure. So, uh before that I want to [31:49] echo what what Jeremy mentioned. So, [31:53] business leads uh the business problem [31:55] and they have to come up with the [31:57] success metrics while the data and [32:00] technology assess the physibility but [32:03] the whole point and the big problem here [32:05] is the setting the expectation. Um so in [32:09] that case we have responsible AI. So [32:12] that's a ground conversation become [32:14] reality. So we make sure responsible AI [32:17] and privacy impact assessment bridge [32:20] that gap between the reality and what [32:23] they consider as a silver bullet these [32:25] days for the AI. Um and one more thing [32:28] that I would like to add is regarding [32:32] regarding the KPI these days people are [32:35] afraid of t saying saving FTEES [32:39] specifically in financial industry [32:42] people leaders everybody they try to [32:45] stay away from that measure because of [32:48] um they do not wants to lose the [32:51] workforce over the AI although the AI [32:54] will help them to improve and assess has [32:57] better in terms of their job daytoday [33:00] but this makes them so much afraid about [33:04] not using AI in a day-to-day basis. Um [33:08] so saving FD although is a area that we [33:11] all should live with it and I'm not sure [33:14] how long we'll take to get rid of our [33:17] job. uh hope not uh but that's the [33:20] reality. We have to make sure to uh [33:22] adjust with the AI to ensure that we [33:25] have AI ready thinking in our day-to-day [33:28] job before losing everything over that. [33:33] So despite our best plans um not always [33:37] the projects are successful and do not [33:39] yield the results that we are expecting. [33:42] Um I would like to hear one example [33:45] where you have not succeeded in the AI [33:48] project. What are the learnings from it? [33:50] Because these are the learnings that are [33:51] most important in shaping the future [33:53] approach. Um Jeremy I would like to ask [33:56] you this first. [33:59] >> Um that's a tough one. I mean you know [34:01] as a consulting company we you know go [34:05] into a project with a client trying to [34:08] make sure we're successful first and [34:10] foremost. I think the area that we've [34:13] we've struggled the most is internally [34:15] um with our own AI initiatives and we [34:17] have some great AI tools and products [34:19] that we've been building but um you know [34:21] we sometimes it's like eating your own [34:24] dog food. You know we we have to follow [34:26] the same process that we teach our [34:28] clients sometimes and you know make sure [34:31] that we have a clear business outcome [34:32] for the tools that we're building. um [34:35] not just a you know we talk with our [34:37] clients a lot about the difference [34:38] between a benefit and a value with value [34:41] being more revops oriented you know [34:44] increased revenue decreased cost um you [34:47] know there's a lot of [34:49] a lot of benefit to uh you know [34:52] non-financial metrics as well and they [34:54] help inform you know the product [34:56] direction and you know the the future [34:59] roadmap um but you know it's it's harder [35:03] to quant quantify the real outcome as I [35:06] I think we've all kind of echoed here [35:08] without you know a key indicator that [35:11] you can you can anchor to. [35:14] Yeah. So in my experience not aligning [35:17] or defining the business outcome very [35:18] clearly could be setting us up for [35:21] failure. One of the other reason I saw [35:23] is not inviting the users as part of the [35:26] journey early. It helps really validate [35:29] uh before we um and and gives us an [35:32] opportunity to identify any gaps to make [35:34] and pivot and address the solution and [35:36] co-create the solution with them without [35:39] waiting till it's completely rolled out. [35:41] Um so Ravi when you work with your [35:44] clients there could be a huge difference [35:46] between the demo and actually the [35:49] production implementation. So how do you [35:51] set the right expectations to ensure [35:53] there is no disappointment? Yeah, it's a [35:56] it's often too standard question, right? [35:58] So is uh demos not uh demos when you do [36:02] it successfully. So you just do some [36:04] sort of wiring in the backdrop, put some [36:07] gorilla glues and make it work and show [36:08] it in front of a customer, it works. The [36:11] business gets too excited about it and [36:13] there is a layer in between where the [36:14] middle management definitely makes sure [36:16] that the business is excited and they [36:18] get the funds and now the real battle [36:20] begins. So the way we've been advocating [36:23] and we we've been working with the [36:25] customer is not to get the excitement [36:26] levels so much like a cat on a hot tin [36:29] roof, right? So we want to make them [36:31] grounded and understandable about the [36:33] metrics that we are going to be getting [36:35] at. So one of the customers that I met [36:37] with uh they have almost 700 operational [36:40] operators sitting and uh working on [36:43] email support emails. It's a very very [36:45] large company and their goal was to [36:47] reduce that 700 to say 200 or 250 or [36:50] something like that in the in the first [36:52] year. So that was an excitement that [36:54] they got through a demo that was done [36:57] and that demo showed three use cases [36:59] heavily tight environments where you're [37:02] you're completely contained and there is [37:04] no way you can fail in any of those [37:06] cases because it's pretty much like your [37:08] rule engine. So it'll work but the [37:10] moment you go real AI it's not going to [37:13] work. So the expectations had to be [37:15] reccalibrated, reset because there is no [37:17] way they can get rid of so many people [37:19] within a year to manage the same [37:21] workload of what's happening with an AI [37:24] that is hallucinating left, right and [37:26] center. It is always going to be doing [37:27] that. So to answer that question right [37:30] so demo to uh successful so what I would [37:32] say is set the expectations right uh [37:35] ground the facts that can happen that [37:37] cannot happen and know that AI is not [37:39] going to be consistent all the time. So [37:41] it's it's it's a fact that it will not [37:43] be consistent. Have your proper [37:45] observability toolkits in place. Have [37:47] your proper ethical and uh governance [37:49] measures in place. Have your data ready [37:51] at first. Then when you bring your MVP [37:54] in, I don't believe in doing PC's [37:56] anymore. Uh I believe in building MVPs. [37:59] So the concrete solid business outcomes [38:01] that you can prove within a smaller [38:03] value of time and then that scales up [38:06] into RevOps. So it had it has to fit [38:08] into the RevOps equation. So that's how [38:10] I would usually say it. You have your [38:13] PC's or you know proof points, demos. Uh [38:16] stop doing that. Advocate customers to [38:19] move more towards an MVP because the [38:20] shelf life of AI is very very small. So [38:23] you have to make sure that you're moving [38:24] into the MVP stage so that can actually [38:27] scale up. So that's how I would say [38:29] that. [38:30] >> So so far we talked about what is [38:32] working but looking ahead what future [38:35] holds? What what are some AI use cases [38:39] you think have most impact on return on [38:41] investment in the coming 5 years and [38:44] what can organizations and companies do [38:46] to lay the groundwork right now? Um I'll [38:49] start with you Peter. What excites you [38:51] most? [38:52] >> Yeah, I think um I'll have two. One [38:55] theme is certainly revenue cycle [38:56] operations. It's not maybe exciting to [38:58] everyone in the room there, but I think [38:59] it's a huge opportunity to automate and [39:02] I think a lot of the reasons why it's [39:04] difficult is simply that your data is [39:06] typically bulcanized across a bunch of [39:08] different data sources, fact streams, [39:10] voicemails, legacy systems, etc. A world [39:13] where we can homogenize that and create [39:15] pipelines for things like contractor [39:17] review, billing, submission appeals, and [39:18] so forth will clean up a lot of the [39:21] administrative complexity that happens [39:23] inside of Rev Cycle. uh and I think [39:25] they'll just be in as payers doing this [39:27] already aggressively with AI they'll [39:28] reach some kind of equilibrium and if [39:30] you're not able to automate you'll be [39:32] severely disadvantaged in the [39:33] equilibrium. The second piece though I [39:35] think that's you know better for [39:36] patients is improvement of access. we [39:38] saw with the GPT5 launch, you know, like [39:39] a segment devoted to someone saying, you [39:42] know, uh, I I gave my biopsy report, [39:45] helped me with my care journey. And as a [39:47] physician, you know, we all want to sit [39:48] back and say, well, hold on, there's [39:50] edge cases and there's but the reality [39:52] is we have been doing healthcare in a [39:55] modern sense for decades and decades and [39:58] it's not a good exper experience for [40:00] like anybody involved. We had a chance [40:02] to make it really good. We've never been [40:04] able to do that. So, frankly speaking, [40:06] what patients want and what they [40:08] deserve, they're going to get. And I [40:10] think we just have to find a way to deal [40:12] with it and make it work. Uh, otherwise, [40:15] we're in the way and we don't want to be [40:17] like that. So, I think that's a an [40:18] exciting future with pros and cons. [40:21] >> Um, in interest of time, Jeremy, uh, can [40:24] you share any AI use cases that are [40:26] currently being overlooked and you think [40:28] has huge potential? Maybe in 30 seconds [40:31] or a minute. Yeah, I mean I I think for [40:34] most industries, not laying the [40:36] foundation and trying to jump straight [40:37] to an AI use case is going to be a [40:39] recipe for failure and likely a tech [40:42] debt that creates a longer term path to, [40:45] you know, the outcomes you're looking [40:46] for. Um, you know, the the exception may [40:49] be manufacturing where, you know, [40:51] there's a lot of companies we work with [40:53] that are still trying to implement [40:54] industry 3.0 initiatives for, you know, [40:56] kind of automation and, you know, basic [40:58] machine learning. uh but there's [41:00] opportunity to you know implement AI [41:03] into some of those areas um and get a [41:06] net benefit out of it. So I think that's [41:07] one area where you know there could be [41:09] some interest and I you know I'm curious [41:11] what's going to happen over the next few [41:12] years. Mona there are many emerging use [41:15] cases fraud prevention customer [41:18] operational efficiency anything you are [41:20] excited about uh so the first thing as a [41:23] u one of the one of the thing that [41:26] beside from compliance regulatory that [41:28] make it easy for financial industry to [41:31] build and benefit from it definitely [41:33] would be automation because having you [41:36] know the documented process flow the [41:38] data ready and everything will help you [41:40] if uh if you have all of those to build [41:42] the automation [41:43] So you will be aligned with the [41:45] compliance and regulation while the AI [41:48] parties would be a little bit more [41:49] difficult with all the open AI and [41:51] everything specifically in financial [41:53] industry. But what I see as a high ROI [41:56] in financial industry these days is the [41:59] cases that augments human decision [42:02] making is a personalized ad financial [42:05] advice or the predicted customer [42:09] expectation or engagement. That's one of [42:12] the thing that uh my organization is [42:14] working on as a member next best action [42:17] to consider what is the next best action [42:19] for each of the clients that we have no [42:22] matter if this is financial or the [42:24] personalized for their lives. [42:27] Those are some really valuable insights [42:29] and we covered some many key concepts [42:31] here today. Let us end this with some [42:33] fun questions. So, if AI is a dating app [42:39] profile, uh what would be the biggest [42:41] red flag? Mona, [42:46] >> privacy. [42:48] Nice. If you have a magic wand, how [42:52] would you address or fix some of the [42:54] most common mistakes that are made by [42:56] clients? Ravi, [42:58] >> I'll definitely magic wand to fix common [43:02] expectations. So I said them uh you know [43:05] obliviate on the uh magic wand from [43:08] Harry Potter you know get them to [43:11] ground. [43:12] >> Um AI ROI is like [43:17] >> uh it's sort of like uh like compounding [43:19] interest like if you lay the the steps [43:21] right you will get farther and farther [43:22] and farther leaps every time you do it. [43:24] >> Very interesting. And uh what is the [43:28] common um use case that people come to [43:31] come up to you with? [43:34] Oh boy, I don't know that there is one [43:36] common use case. I think that's the [43:37] challenge with ROI is there's so many. [43:39] Um, you know, I think the the [43:41] commonality between them is RevOps. You [43:44] know, everything is about revenue cycle [43:46] in one way or another. Um, so that'd [43:48] probably be the most common that we see. [43:50] >> Awesome. And that's a rabbis.