Hospitals Slow AI Rollouts as Vendor Data Traps, Workflow Risks Mount

供应商数据陷阱与工作流程风险加剧，医院放缓AI部署步伐 由于未受监控的模型和第三方数据利用威胁到患者安全及医院收入，医疗行业高管在扩大AI应用规模前必须审查供应商合同并重构临床工作流程。 医疗管理高层与医学信息学专家探讨了部署临床AI的现实考量与经验教训，涵盖从环境感知病历生成系统（ambient scribes）、深度学习影像模型到乡村医院工作流转型等多元实践。专家小组强调，成功的应用落地取决于深度的工作流程重构、持续监测与严格的本地化验证，而非单纯依赖原始算法的准确率。讨论的核心议题包括应对自动化偏差、突破传统IT基础设施的带宽限制、缓解员工被替代的担忧，以及防范掠夺性供应商合同以保护医院数据资产。发言专家最终呼吁医疗决策者审慎把握部署节奏，将AI风险与人类医疗差错基线对照评估，在行业炒作浪潮中切实将患者护理置于首位。 Geisinger 全面推广环境感知听记（ambient scribing）技术，平均每份病历可为医生节省约 2 分钟记录时间，极大缓解了文档倦怠，证明 AI 的核心挑战在于工作流集成而非单纯算法研发。 (4:13) 在乡村医疗场景中，环境 AI 恢复了医患直视沟通并自动充实简陋病历，但激增的数据调用也迫使机构紧急升级老旧网络带宽，并严格重审供应商合同以防患者数据被商用滥用。 (7:51) 以“睡衣时间”等晚间登录指标衡量医生福祉容易误判个人作息选择；长期来看，“自动化偏见（automation bias）”风险更为突出，医生可能因过度依赖 AI 而削弱对细微医疗差错与幻觉的警惕。 (10:36) 未经精细工作流整合的 AI 易适得其反（如未配置好的 CT 结节检测导致放射科医生被迫双重阅片），而医疗 AI 的 ROI 难以单纯靠直接收益计算，更应关注人员转岗增效与降低高额的医生离职成本。 (18:15) 评估临床 AI 风险与幻觉率应以真实的人类医疗差错基线为参照而非苛求绝对完美；医院管理者应在盲目推动“AI 优先”前建立安全沙箱收编“影子 AI”，并严查底层数据权属。 (29:17)

Healthcare leaders across academic and rural institutions argue that clinical AI succeeds only when technical development is subordinated to workflow integration, rigorous local evaluation, and data sovereignty. They warn that model accuracy yields no operational benefit if implementations compound clinician friction, foster unmonitored automation bias, or compromise institutional data rights. Workflow Integration Over Algorithmic Precision (4:13) Panelists emphasize that developing algorithms is far simpler than embedding them into clinical environments. At Geisinger, an enterprise rollout of ambient scribing saved approximately two minutes per clinical note and generated visceral clinician enthusiasm, yet Dr. Aalpen Patel noted from early deep learning initiatives that data pipelines and workflow adaptation mattered far more than model math. Unintegrated tools introduce friction: an unannounced CT pulmonary nodule detection rollout doubled radiologist workloads by generating duplicate scan series, forcing clinicians to review every case twice. In rural settings, ambient documentation restores bedside eye contact and expands sparse, one-line entries into comprehensive charts. However, Boone County Health Center CEO Caleb Poore observed that this surge in API calls forced the hospital to upgrade its 20-year-old network bandwidth. Dr. Caroline Chung of MD Anderson added that acceptable algorithmic uncertainty must be context-dependent: an AI tool that assists rural generalists with referral triage provides immense clinical value even if its precision falls short of academic medical center standards. Automation Bias and Patient Safety Boundaries (10:36) While ambient tools offer immediate operational relief, speakers urge caution regarding superficial metrics and long-term behavioral drift. Dr. Chung noted that tracking EHR "pajama time" misinterprets intentional work-life scheduling, such as logging in after family dinners, as system dysfunction. Over time, documentation tools introduce automation bias: clinicians who scrutinize every line during pilots gradually cease verifying drafts by day 1,000, compounding risks if models drift or introduce subtle hallucinations. Evaluating AI risk requires benchmarking against baseline human performance. Patel cited landmark studies from the Institute of Medicine and Johns Hopkins attributing 98,000 to over 250,000 annual U.S. deaths to preventable medical errors, arguing that validated AI curbing baseline mistakes yields an immediate safety dividend. However, Dr. Morgan Jeffries warned that the safety buffer provided by expert clinicians breaks down in patient-facing AI. Because consumer healthcare tools must target a fifth-grade reading level, advising lay patients to verify complex chatbot outputs is contradictory and unsafe, particularly during acute crises such as undetected suicidal ideation. Staffing Reallocation and Economic Realities (12:59) Calculating healthcare AI return on investment (ROI) presents distinct operational hurdles. Revenue cycle management offers straightforward financial accounting, but indirect gains—such as curbing physician burnout to avoid replacement costs estimated at upwards of $1,000,000 per doctor—suffer from difficult attribution. At Geisinger, measuring authentic time savings required diverting engineering resources to construct bespoke internal dashboards because generic vendor usage metrics failed to reflect clinical reality. Rather than executing workforce reductions, rural providers leverage AI to absorb acute staffing shortages. Poore explained that Boone County restructured its staffing model by repurposing transcription personnel into high-speed chart auditing and clinical quality review. Furthermore, because ambient tools draft the History of Present Illness and visit reasons automatically, outpatient Registered Nurses can be reassigned to high-acuity inpatient units like Med-Surg, the Emergency Room, and Obstetrics. Patel emphasized that demystifying AI as understandable mathematics and directly engaging frontline staff prevents cultural alienation and resistance. Data Sovereignty and Vendor Contract Pitfalls (38:10) Data governance and contracting practices require aggressive executive scrutiny before deployment. Poore shared a cautionary example from Boone County, where an audit revealed a core vendor was "providing" sensitive operational data to an external repository without technically selling it. Major insurance payers subscribed to that repository and utilized the hospital's own data to benchmark and depress reimbursement rates against the facility. To halt third-party exploitation, rural health systems increasingly mandate formal written authorization before institutional data can be extracted or repurposed for model training. Internally, enterprise governance faces additional strain from unmonitored "shadow AI"—where staff utilize unapproved consumer tools like Claude—and from legacy software vendors quietly embedding unvetted AI capabilities into routine software updates, necessitating cross-departmental oversight to continuously monitor risk. The panel concludes that healthcare AI is a complex system-of-systems challenge requiring deliberate pacing rather than hype-driven adoption. Strategic success demands that health systems establish clear criteria for measurable clinical value, perform local demographic validation instead of relying on vendor-curated datasets, and construct cross-functional governance encompassing clinical, IT, legal, and operational leadership.

医疗系统领导者指出，医疗人工智能落地的核心瓶颈不在于孤立算法的模型精度，而在于深度的临床工作流程重构、对人机协作偏差的防范，以及对机构数据主权的严格法律防御。 环境智能的临床赋能与自动化偏差隐患 (4:13) 环境临床记录（ambient scribing）技术正在重塑一线诊疗体验。在Geisinger，该技术的全面推广使医生每份病历平均节省约2分钟，显著减轻了工作负荷；在内布拉斯加州的Boone County Health Center，移动端环境智能让医生得以脱离屏幕并恢复与患者的直接眼神交流，不仅将极简的敷衍记录扩充为完整病历，还使机构能将门诊护士释放并重新分配至急诊、内外科等高负荷病区。然而，MD Anderson Cancer Center的Caroline Chung博士提醒，不能仅凭夜间登录EHR的“睡衣时间”（pajama time）来衡量减负，因为这往往是医生为陪伴家人而主动选择的弹性作息。 更深层的患者安全隐患在于“自动化偏差”（automation bias）。Caroline Chung与Aalpen Patel博士警告，随着医护人员逐渐习惯AI生成的病历，最初严谨的逐行校对会逐渐演变为盲目信任，难以察觉模型漂移或细微幻觉引发的诊断遗漏。缺乏持续的运行时监控，技术退化与人类行为松懈将直接转化为医疗差错风险。 流程再造与投资回报率的多维核算 (12:59) Aalpen Patel博士以Geisinger早年利用2.3 PB影像和25年EMR数据研发脑出血检测模型的历程指出，算法开发仅占整体难度的极小部分，关键在于底层数据管道架构与接收模型推断的工作流重塑。若缺乏流程协同，AI反而会加剧临床阻力，例如某机构CT设备自动启用的肺结节检测功能迫使放射科医生重复阅览原始与标注两套扫描图，导致工作量凭空翻倍。 在投资回报率（ROI）核算上，Morgan Jeffries博士分析认为，优化账单与营收周期管理的财务ROI易于量化证明，但改善患者预后或避免医生职业倦怠带来的间接回报却难以进行因果归因——尽管单名医生的离职置换成本常高达100万美元左右。Caleb Poore分享的实践表明，基层机构应通过岗位转型实现真实ROI：将原有人工听写员重组为高效率的病历审核与质控专员，既缩短了文书时间，又保留了人工安全核查机制。 基础设施瓶颈与数据主权防御 (7:51) 环境AI的实时运转对传统医疗IT架构构成了突发性压力。Boone County Health Center在接入移动端环境监听的高并发API数据流后，因近二十年未升级的基础设施难以承载，被迫对外部网络物理带宽进行了整体扩容。 更为严峻的挑战在于数据主权与第三方合同漏洞。Caleb Poore披露，审计发现某核心系统供应商在未构成法理“销售”的情况下，向第三方数据仓库“提供”了该院运营数据；该仓库的主要订阅方恰恰是大型医疗保险公司，后者利用这些数据压低了对该医院的医保报销费率。这一案例促使基层医院全面收紧数据治理，要求严禁第三方供应商在未获书面授权的情况下抽取本土患者数据用于商业模型训练或转售，筑牢法务与资产防线。 人类错误基准与患者端应用的安全边界 (35:40) 在算法验证与错误率容忍度方面，Caroline Chung主张应将技术表现与现实的人类基准对照，而非苛求不切实际的零缺陷。Aalpen Patel援引约翰霍普金斯大学发表于《英国医学杂志》（BMJ）的研究指出，全美每年因医疗差错死亡超过25万人，累计医疗失误约1,200万起；美国医学研究所（IOM）报告亦显示每年有高达9.8万人死于可预防的医疗差错。因此，只要经验证的AI能系统性压低人为差错基准，就能带来明确的临床净效益。 然而，Morgan Jeffries强调这一容错逻辑仅适用于医护端辅助，在面向患者的直接交互场景中完全失效。专业医生具备捕获模型幻觉的责任与能力，而普通公众的平均阅读理解基准通常仅设定在五年级水平，无法承担“既信任又自主核验”的矛盾要求。面对自杀倾向等危机信号，面向患者的工具必须建立100%可靠的人工转接安全网。此外，针对员工私下使用Claude等消费级工具的“影子AI”现象，机构应通过部署受控的企业级沙箱（如企业版Copilot）进行合规疏导，避免敏感数据外泄。 与会专家一致认为，医疗AI落地是一项高度复杂的系统性工程，涉及法务、IT架构、流程与多学科监管。面对机构内部打造“AI优先”形象的激进压力，管理层必须放慢盲目扩张的节奏，将机器智能视作劳动力、资本与信息之后的第四生产要素进行重构，依据各科室实际承载力确立优先次序，而非追逐脱离临床实效的技术概念。

## Background 本内容由 Ai4 于 2026 年 8 月 21 日发布在 YouTube 平台上，题为《AI Powered Care Providers for a Better Tomorrow》。节目汇集了来自 Geisinger、MD Anderson 癌症中心、Boone County 医疗中心以及风险投资机构 BGV 的多位医疗高管、临床医生与投资人，共同探讨医疗机构如何负责任地部署 AI 技术以增强诊断精度、精简病历文书并减轻临床医护人员的行政负担。 ## TL;DR 圆桌论坛深入探讨了医疗 AI 从概念验证走向企业级全面落地的真实挑战与关键策略。嘉宾们结合环境智能听记（Ambient AI）与影像算法的实操案例，剖析了技术在缓解乡村医疗人力危机、重塑医患沟通及优化病历质量方面的巨大价值。同时，专家们警示了模型漂移、自动化偏差、隐形供应商功能渗透以及数据主权外泄等深层治理风险，强调面向患者侧 AI 存在的安全红线。与会者一致呼吁管理层必须摒弃盲目的‘AI First’冲动，紧扣一线临床真实痛点，建立跨学科持续监测与严格的本地化验证闭环。 ## Atmosphere & style 整场对谈呈现出极具专业深度且务实冷静的行业研讨氛围，发言者既有扎根基层的院长视角，也有顶尖癌症中心与大型医疗系统的首席专家洞见。现场对话直面尖锐矛盾，嘉宾们不仅毫不避讳地披露‘影子 AI’、合同漏洞导致数据被保险公司利用等血淋淋的教训，还以‘全美每年 25 万医疗差错致死’的沉重基准引发全场深思。各方在理性克制的技术审视与对医护福祉的人文关怀之间自如切换，散发出一种历经实战沉淀后的厚重感与敬畏心。 ## Key takeaways - [04:13](https://www.youtube.com/watch?v=UBFVn8oRy10&t=253s) 环境智能听记（Ambient Scribes）已跨越概念验证进入实际投产阶段，不仅平均每份病历可节省约 2 分钟，更获得了临床医生极高的情感认可与依赖。 - [04:13](https://www.youtube.com/watch?v=UBFVn8oRy10&t=253s) 医疗 AI 落地的真正瓶颈往往不在算法模型本身，而在临床工作流的工程转化与端到端数据流的无缝集成。 - [07:51](https://www.youtube.com/watch?v=UBFVn8oRy10&t=471s) 偏远乡村医疗引入环境智能 AI 能够重塑医患眼神交流并显著提升病历质量，但同时也倒逼了老旧 IT 基础设施与网络带宽的紧急升级。 - [10:36](https://www.youtube.com/watch?v=UBFVn8oRy10&t=636s) 评估临床 AI 不能仅盯“睡衣时间”或登录时长等表面指标，必须防范长期使用中出现的自动化偏差（Automation Bias）与模型漂移风险。 - [13:00](https://www.youtube.com/watch?v=UBFVn8oRy10&t=780s) AI 价值与 ROI 的评估高度依赖具体应用场景与上下文环境，不能单看纸面算法性能，而要权衡其实际带来的工作流阻力与临床效益。 - [15:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=910s) 医疗机构在引入自动化技术时需保持克制，明确“有能力做”并不等同于“应该做”，避免以牺牲医患人文关怀为代价盲目追求过度自动化。 - [17:28](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1048s) 改善医生福祉和减少离职等软性价值在财务归因上极难量化，不当的 AI 引入（如未经规划的影像自动检测）甚至可能导致医生审片工作量翻倍反噬工作流。 - [21:53](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1313s) AI 转型本质是组织变革，领导者需提升全员 AI 素养，为员工提供安全的实验环境，并以“为了他们而非强加于他们”的原则让一线团队深度参与方案设计。 - [27:39](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1659s) 面对员工自发使用外部大模型的“影子 AI”现象，管理层应避免单纯封禁扼杀创新，而应主动提供企业级合规平台与透明的知识共享环境。 - [32:06](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1926s) 评估 AI 安全性应对比医疗系统的现实风险基线（全美每年约 1200 万起医疗差错），只要 AI 能优于现有水平、减少现实错误即具采纳价值，而非盲目要求机器达到 100% 绝对准确。 - [33:41](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2021s) AI 模型进入临床前必须进行本地化验证（Local Validation），不能单纯迷信厂商提供的监管审核（FDA）数据集表现。 - [36:20](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2180s) “人机协同”（Human-in-the-loop）模式在专业医护端有效，但不能直接套用于面向患者（Patient-facing）的 AI 交互，后者在极端安全事件识别上存在重大伦理与生命风险。 - [38:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2290s) 医疗管理层在采购技术时必须严查供应商合同中的数据流向与使用条款，警惕临床与账单敏感数据被第三方数据仓库转售利用。 - [43:57](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2637s) 人工智能作为劳动力、资本与信息之外的第四生产要素（智能），正在从底层逻辑重构医疗机构的战略优先级、业务流程与组织治理架构。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=UBFVn8oRy10&t=4s) 嘉宾自我介绍与当前最关注的 AI 议题 论坛由来自风险投资机构 BGV（Benhamou Global Ventures）的合伙人 Anik Bose 主持。他开门见山，请四位来自医疗体系与产业前沿的嘉宾简短介绍自己，并分享目前各自办公桌上“最火热、优先级最高”的医疗 AI 议题： ```mermaid flowchart TD accTitle: 医疗 AI 落地面临的核心挑战矩阵 subgraph MDAnderson["MD Anderson 癌症中心"] A["Caroline Chung 博士"] --> B["评估基准与终局价值<br/>超出单纯的模型准确率"] end subgraph BooneCounty["Boone County 医疗中心"] C["Caleb Poore 首席执行官"] --> D["填补乡村人力短缺<br/>兼顾就医体验与时间质量"] end subgraph CasperGroup["Casper Group / 前 Geisinger"] E["Aalpen Patel 博士"] --> F["连接人与模型的中间层<br/>让护理人员高效服务患者"] end subgraph Geisinger["Geisinger 医疗系统"] G["Morgan Jeffries 博士"] --> H["规模化临床评估与监测<br/>医生体验的代际变革"] end ``` - **Caroline Chung 博士**（德克萨斯大学 MD Anderson 癌症中心放射肿瘤学及诊断影像学教授、肿瘤数据科学研究所联席主任、首任首席数据与分析官）：目前正与产业界、美国国家癌症研究所（NCI）及多家学术机构联合攻关核心命题——如何建立真正有意义的基准评估标准（Benchmarking）。她强调，行业不能只停留在“容易衡量的指标”或“单纯看模型准确率”，而必须思考：我们需要什么样的证据，才能确信某款工具真正改善了医疗照护质量并驱动了实质性的效率提升。 - **Caleb Poore**（内布拉斯加州 Boone County 医疗中心总裁兼首席执行官）：已有约 10 年的高管领导经验，所在的阿尔比恩（Albion）小镇常住人口仅约 1,800 人。乡村医疗面临严峻的人力短缺瓶颈，因此如何利用 AI 补足人力、弥补服务能力缺口是重中之重。同时，乡村医疗的核心优势在于医生不会像大城市医院那样赶流水线，能花大量时间与患者沟通；因此，AI 的引入绝不能以牺牲这份重在沟通的医疗质量为代价。 - **Aalpen Patel 博士**（介入放射学家、临床信息学家、“自愈中的电气工程师”）：曾在 Geisinger 担任放射科主任 13 年，并曾任该机构的 AI 医疗总监；随后赴 MedQuest Imaging 担任首席临床与创新官。目前正在孵化一家处于隐身模式（Stealth）的健康科技初创公司 Casper Group，专注于护理者（Caregiver）领域。他最关心的核心是介于“底层模型”与“终端人员”之间的中间交互层，探讨如何通过这一层最有效地赋能患者照料。 - **Morgan Jeffries 博士**（神经重症专科医生、Geisinger 现任 AI 医疗总监，接替 Aalpen Patel）：指出临床医生的数字化体验领域正迎来巨大创新浪潮，预计未来三年将彻底改变临床流程。眼下他最关注、认为最具挑战性的工作，是在医疗机构内部建立针对临床 AI 的“规模化评估与持续监测体系”（Scalable evaluations and monitoring）。 --- ### [04:13](https://www.youtube.com/watch?v=UBFVn8oRy10&t=253s) 跨越概念验证：Geisinger 的环境智能听记与早期影像落地 主持人 Anik 抛出核心问题：当前业界充斥着大量 AI 试点项目（Pilots），各位所在的组织有哪些项目真正成功跨越概念验证阶段、实现企业级全面投产（Scale to production）？ Morgan Jeffries 首先分享了 Geisinger 部署“环境智能语音抄写员”（Ambient Scribes）的案例。这项在他接任前便启动的举措，已在全院范围内正式上线大约两年，投入了大量工程与临床配套资源。投产后带来了极其明确的双重收获：一方面是硬性的客观效率收益，系统仪表盘显示每份病历平均可为临床医生节省约 2 分钟书写时间，整日累加效果十分惊人；另一方面则是前所未有的主观情感反响。在医疗信息化领域，新系统的上线往往伴随抵触与抱怨，但这次医生的反馈强烈到令人震撼，许多人直接恳求管理层：“求求你们千万不要把这个系统撤走，用上它之后我才意识到自己过去过得有多痛苦，如果关停我真不知道该怎么办了。”这种强烈的情感认可加上可测量的硬核收益，为系统全院铺开提供了坚实的依据。 接续 Morgan 的发言，Aalpen Patel 回顾了 Geisinger 早在 2014 至 2015 年间的探索。当时卷积神经网络（CNN）方兴未艾，而 Geisinger 凭借电子病历系统的早期应用积攒了 25 年的 EMR 数据，光影像数据就高达 2.3 PB。团队在先进应用研究所的主持下进入隐身研发状态，成功训练出了一套用于自动检测脑出血的算法模型。然而，真正的艰难旅程不在于模型构建本身，而在于临床工程落地： ```mermaid flowchart LR accTitle: 医疗 AI 临床转化落地完整路径 subgraph DataInfra["数据工程底座"] A["2.3 PB 影像数据<br/>+ 25 年 EMR 数据"] --> B["打通临床侧与<br/>科研侧数据通路"] end subgraph Deployment["工作流设计与落地"] B --> C["设计全套数据流<br/>与端到端沟通机制"] C --> D["嵌入医生诊断工作流<br/>（从以转化为起点）"] end subgraph Outcome["临床成果"] D --> E["脑出血早期自动筛查<br/>规模化稳定运行"] end ``` Aalpen 总结道：“模型本身并不是最重要的，它只是其中一部分；真正的关键在于如何把它无缝融入日常工作流程。”当年既没有打通临床数据与科研环境的管网，也缺乏端到端的通信机制，团队必须从零设计数据流与工作流架构，从立项第一天就“以转化为核心（Translation in mind）”进行倒推设计，最终才成功跨过鸿沟。 --- ### [07:51](https://www.youtube.com/watch?v=UBFVn8oRy10&t=471s) 乡村医院的数字化巨变：带宽告急、数据治理与全流程增效 Caleb Poore 结合 Boone County 医疗中心的实践指出，在仅有 1,800 名居民、辐射周边约 10,000 人的偏远乡村环境，与 EHR 厂商深度整合的环境智能 AI（Ambient AI）同样带来了翻天覆地的转变： - **重塑医患视线交流与就医信任**：过往医生进门后大多背对患者、面向电脑疯狂打字；如今只需把手机录音开启平放在桌上，医生便能与患者保持全流程的眼神接触。患者感到前所未有的被倾听与被重视。 - **病历质量从“一句话应付”到详尽充实**：Caleb 幽默地举例称，院内一位资历极深的老医生以往写病历是出了名的敷衍，经常草草写上一句“患者状况良好，下次复诊”就草草结案。AI 介入后，病历详实度实现了从“近乎零数据”到“信息量拉满”的质变，为患者全生命周期管理建立了扎实的档案支撑。 - **倒逼 20 年未修的基础设施全面升级**：海量语音与结构化病历通过 API 高频回传，让该院近 20 年没有大规模改造的旧 IT 架构不堪重负。医院必须紧急扩容物理网络带宽，“把接入机构的互联网水管硬生生拓粗”，才能承载涌入的数据流。 - **唤醒乡村医疗的数据主权与合同治理意识**：内布拉斯加州的一众关键准入医院（Critical Access Hospitals）猛然意识到，自身多年来一直坐在一座未被开发的“数据金矿”之上，学术机构与商业厂商纷至沓来索取数据。但院方回查后震惊地发现，以往签署的供应商软件合同中存在大量开放性、模糊化的数据使用条款。乡村医院正迅速采取防御性治理行动，重新定义合同边界，明文规定厂商使用任何临床数据必须事先取得院方的书面授权。 - **医保达标、接诊量与计费编码的连带跃升**：病历的极度规范直接助力医院顺利达到 MIPS（基于绩效的激励支付系统）和 ACO（责任医疗组织）等医疗质量指标考核；医生不必再被文书拖累，单日接诊能力显著提高；更直接贯通了从计费编码到收入循环管理（Revenue Cycle）的闭环。 --- ### [10:36](https://www.youtube.com/watch?v=UBFVn8oRy10&t=636s) 反思“睡衣时间”：量化指标与学术医生的真实诉求 Caroline Chung 在认同前述观点的同时，引入了学术医疗体系特有的深度反思。她指出，临床医生对环境监听工具的情感共鸣几乎普遍强烈，但在许多早期学术文献与宣传中，大家评估其成效时往往死死锚定在一个特定指标——**“睡衣时间”（Pajama Time）**。 所谓“睡衣时间”，是指无数临床医生在白天看诊结束后，被迫在晚上 10 点以后穿着睡衣坐在家中电脑前，耗费数小时补录白天所有病人的病历日志。这曾被视作医生职业倦怠（Burnout）的罪魁祸首。然而，当 MD Anderson 癌症中心向自身的学术型医生发起针对“压缩睡衣时间是否真对他们具有核心意义”的深访调研时，却挖掘出了远比单纯缩短晚上文书时间更加复杂、深刻的诉求。 ### [11:15](https://www.youtube.com/watch?v=UBFVn8oRy10&t=675s) 衡量真实临床指标、自动化偏差与模型漂移 在临床 AI 的实际部署中，仅仅记录电子病历（EHR）系统的登录时间并不能反映技术对医生的真正影响。医生可能会选择在下班后先陪伴家人、参与孩子的课后活动，然后再在晚上补写病历。如果管理层只机械地统计医生在 EHR 上的打卡或登录时长，就会完全错失真正重要的指标。因此，理解工具落地的具体场景和医生的真实工作习惯至关重要。 与此同时，行业目前普遍追求“效率”与“速度”，但对病历质量的持续评估却往往被忽视。随着使用时间的推移，医生可能会对 AI 生成的诊断建议或病历产生过度信任，从而引发“自动化偏差”（Automation Bias）。医护人员在第 1 天使用 AI 时的行为，与在第 1000 天或 2 年后的行为会截然不同。因此，医疗机构必须建立持续的监控机制，不仅要跟踪人员行为的变化，还要警惕“数据漂移”（Data Drift）引发的“模型漂移”（Model Drift）。如果模型脱离了预设的安全参数，最终可能会对患者造成实质性伤害。 ```mermaid flowchart TD accTitle: AI 临床监测闭环与风险控制 subgraph Behavior["人类行为演变"] B1["第 1 天：谨慎验证与审查"] --> B2["第 1000 天：依赖加深 / 产生自动化偏差"] end subgraph Drift["系统与数据漂移"] D1["真实临床数据漂移 (Data Drift)"] --> D2["AI 算法模型漂移 (Model Drift)"] end subgraph Monitor["持续监控与干预"] M1["设立安全参数与质检基线"] M2["主动监测工作流与医疗输出"] M3["防止患者潜在危害"] end B2 --> M2 D2 --> M2 M1 --> M2 M2 --> M3 ``` --- ### [13:00](https://www.youtube.com/watch?v=UBFVn8oRy10&t=780s) 超越模型本身：结合工作流与上下文评估真实 ROI Caroline Chung 深入探讨了如何科学评估 AI 的价值与投资回报率（ROI）。她指出，行业内并不缺乏对 AI 算法性能的基准测试，但一个在纸面指标上表现极其优异的模型，如果无法融入实际业务流程，依然无法带来实质性改变。如果引入 AI 后反而要求医生重复复核十次，那么这种部署不仅没有减轻负担，反而增加了额外的操作阻力。真正的价值评估必须结合实际应用场景中的终极目标——在提高效率的同时，绝不能以牺牲诊断准确率或医疗质量为代价。 评估标准还取决于机构对“不确定性”的容忍度以及具体的部署环境。例如在缺乏专科医生的偏远乡村地区，即使 AI 工具不够完美、准确率没有达到顶尖医疗中心的严苛标准，只要它能为基层转诊提供有效指引，就能切实推动医疗效果的改善。同样的工具在不同的场景下，其成败标准完全不同，因此“情境感知型评估”（Context-Aware Evaluation）是定义 AI 价值的关键。 --- ### [15:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=910s) 就业焦虑、职业转型与“能做”并不等于“该做” Caleb Poore 从基层与乡村医院的管理视角切入，讨论了 AI 引发的岗位替代焦虑。在实际操作中，医院并非直接裁员，而是将原本负责听写、转录的部分人员重新分配到审核等其他关键流程中。原先需要四个人处理的工作，现在由更少的人员负责复核即可，这在劳动力成本上确实能带来直观的 ROI 节约。医生们虽然对 AI 节省时间、让他们能够回归家庭感到由衷高兴，但紧随其后的普遍担忧是：“算法正在学习我的诊疗行为和临床思维，并将这些数据存入数据库，这在 10 到 15 年后会不会让我彻底失业？” 针对这种顾虑，Caleb 告诫团队不要盲目追求过度自动化的工具，医疗行业中护士、医生与患者之间的人性化关怀是 AI 无法替代的。医疗机构在评估 AI 时必须保持克制，进行深入对话，时刻反思：“仅仅因为我们‘有能力’上线某项功能，是否就意味着我们‘应该’去部署它？” --- ### [17:28](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1048s) 隐性价值量化、流程反噬与归因难题 Aalpen Patel 与 Morgan Jeffries 进一步剖析了 ROI 量化的两难困境。运营流程的提效很容易直接折算成财务数字，但改善医生福祉、减轻职业倦怠等软性价值却极难呈现在财务报表上。改善医生体验需要经历多层转化：医生满意度提升 $\rightarrow$ 离职率降低 $\rightarrow$ 机构接诊能力增强 $\rightarrow$ 最终体现在营收与留存成本的节约上。PATEL 分享了一个反面案例：在他之前的机构中，供应商在未经充分规划的情况下，擅自开启了 CT 扫描仪上的“自动肺结节检测”功能，导致放射科医生必须同时查看原始扫描图像与后处理图像，审片工作量直接翻倍，引发了医生的大量抗议，最终不得不紧急关闭该功能。 ```mermaid flowchart LR accTitle: 临床 AI 软性价值转化为硬性 ROI 的链路 A["上线环境感知型 AI (Ambient AI)"] --> B["减轻病历负担 / 改善医生福祉"] B --> C["降低医生离职率 (减少约 $1M 替换成本)"] C --> D["维持高接诊量与服务连续性"] D --> E["实现长期财务回报 (ROI)"] ``` 在量化临床收益时，还存在严重的“归因难题”（Attribution Issue）与高昂的自研成本： - **资金完全可替代性（Fungibility）：** 增加计费（RCM）等能直接带来现金流的项目最容易通过审批；但如果要衡量“拯救一条生命”的财务价值，在管理层和伦理层面极难达成共识。 - **离职替换成本高昂：** 行业内替换一名全职医生的成本估算高达 100 万美元。虽然环境感知型临床 AI（Ambient AI）极受医生欢迎，且大概率在两到三年内挽留了部分医生，但管理层很难精确证明某位医生的留任百分之百归功于该系统。 - **专有监控系统的资源消耗：** 为了客观证明记录病历节省的时间，医疗机构不得不投入大量工程与管理资源打造专用的数据看板，这导致相关资源无法投入到其他技术项目中。许多商业软件仅提供浅层的“使用量指标”，迫使医院只能自行承担沉重的评估负担。 ### [21:53](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1313s) 组织变革与 AI 素养：打破怀疑与建立共识 在探讨 AI 落地效益时，主持人 Anik Bose 指出，AI 转型在本质上并非技术问题，而是一场组织变革。面对临床医生、护士以及行政员工普遍存在的疑虑，领导层如何推动组织从“怀疑观望”走向“积极拥抱”？ Caroline Chung 对此提出了独特的组织群体画像。她认为机构内部并非全员怀疑，大致呈现分层结构：约有四分之一的人属于激进创新派，主张“快速行动并勇于试错”（break things），认为传统模式难以为继；另有约四分之一的人处于极度恐惧和怀疑之中，甚至抱有“AI 将毁灭世界”的悲观态度；剩下的半数员工则处于中间地带，持观望态度，只要看到实际益处便愿意加入。 推动转型的核心瓶颈之一在于全员的“AI 素养与认知流畅度”（literacy and fluency）。随着技术快速演进，不同人对 AI 的理解差异巨大——专业人士能细数不同算法分支，而多数员工则简单地将 AI 等同于 ChatGPT 或某种大语言模型（LLM）。管理者必须统一认知语言，为担忧者提供安全的实验环境以亲身体验技术益处，同时让激进创新者在系统出错时反思原因、总结经验。真正的变革关键在于培育组织文化，而非单纯堆砌操作流程；仅凭硬性规定往往会导致员工绕开监管，只有建立起共同的文化土壤，转型才能持久。 --- ### [24:06](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1446s) 医疗人力短缺与角色重塑：从常规门诊到重症护理 Caleb Poore 从基层与乡村医疗机构的视角分享了截然不同的观察。在其所在的医疗体系中，对 AI 抱有迫切需求的员工高达 90%，持怀疑态度和中立态度的各占 5%。现代医护人员面临的工作负荷和代际诉求与二十年前完全不同，绝大多数员工最渴望了解的是“技术如何让我的日常工作更轻松”。 这种技术的引入正在重塑传统的护理分工模式。以往诊所普遍配备注册护士（RN）负责测量生命体征、询问就诊原因并协助转诊电话；而如今，医生借助 AI 环境录音（ambient recorder）工具，即可自动将现病史（HPI）、就诊原因及诊断相关分组（DRG）要素分发录入相应模块。这促使管理者重新评估门诊的人员配置： - **门诊常规职能转移**：基础的数据录入与问询工作不再必须依赖高资质的 RN。 - **高风险复杂科室聚焦**：通过自然人员流动（attrition），将珍贵的注册护士资源重新调配至内外科病房（Med-Surg）、急诊室（ER）和产科（OB）等真正需要高阶临床护理的重症科室。 - **缓解行业人力危机**：全美目前正处于前所未有的急剧护士短缺期，尤其是乡村医院。AI 的出现恰逢其时，其作用在于弥补人力缺口、提供市场补充，而非剥夺就业岗位。 --- ### [26:18](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1578s) “为了他们，而非强加于他们”：一线驱动的 AI 实施法则 Aalpen Patel 总结了 AI 落地过程中至关重要的管理法则——“不要把 AI 强加给他们，而是为了他们去做 AI”（Don't do AI to them, do it for them）。他强调，任何 AI 项目的启动都必须将最终使用该工具的一线团队深度纳入设计与部署全流程。 ```mermaid flowchart TD accTitle: AI 临床落地与实施路径图 accDescr: 展示从定义临床问题到最终团队协同落地的关键决策流程 subgraph ProblemDef ["第一阶段：问题与业务定义"] A["提出 AI 项目想法"] --> B{"明确具体要解决的问题"} B --> C["假设拥有完美数据"] C --> D["明确运营流程或临床实践的具体改变"] end subgraph FrontlineEngagement ["第二阶段：一线深度参与"] D --> E{"是否已与执行变更的一线团队沟通？"} E -- 否 --> F["深入一线调研需求与吸收反馈"] E -- 是 --> G["协同推进技术落地与流程再造"] F --> G end ``` 当有人提出 AI 方案时，管理层必须首先追问三个底层问题： 1. **定义问题**：这项技术到底要解决什么具体的业务或临床痛点？ 2. **假设推演**：即使系统给出了 100% 完美的精准数据，临床运营或诊疗路径将做出怎样的具体改变？如果无法回答，就必须先回到业务本身理清逻辑。 3. **一线协同**：是否与真正执行这些流程改变的一线团队进行过沟通？只有让未来使用工具的人全程参与，才能获取真实的业务洞察，避免技术脱离实际。 --- ### [27:39](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1659s) 应对“影子 AI”：保护创新文化与合规引导 Morgan Jeffries 指出，在建立治理框架的同时，管理者需要特别注意保护组织的创新活力，避免在无意间“扼杀创新文化”。目前医疗系统内部普遍存在相当规模的“影子 AI”（Shadow AI）使用现象。 由于许多机构出于安全合规考虑在网络端封禁了各类外部 AI 工具，导致部分员工在日常工作中私下使用 Claude 等大语言模型辅助非涉密工作（不输入受保护健康信息 PHI 或商业敏感数据），却因担忧引来层层质询而选择隐瞒。 为扭转这一局面，领导层应主动识别员工自发使用且反响良好的实用工具，并设法将其引入企业级合规环境，或提供可靠的官方替代方案。通过部署功能强大的企业级 AI 临床助手，让医护人员能够公开、透明地与同事探讨“我如何使用 AI 解决了某个工作痛点”，从而在合规边界内建立积极正向的知识共享氛围。 --- ### [29:17](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1757s) 风险基准与人类容错：重新定义 AI 的安全与准确率标准 在谈及医疗行业对 AI 极高的安全与质量要求时，Anik Bose 向 Caroline Chung 提问：面对模型幻觉、治理缺陷与意外风险，医疗机构应如何设立评估标准？ Caroline Chung 直言，医疗领域的任何现有行为都伴随着一定程度的风险，人类决策本身也会出现失误。因此，评估 AI 不应凭空设定脱离实际的绝对指标： - **建立基准线**：评估的核心在于对比当前现状——机构当下实际测量的安全与性能基线是多少？新引入的 AI 工具是增加了风险、减少了风险，还是维持了同等水平？ - **摒弃单一数字执念**：不结合具体临床场景而盲目要求“99% 准确率”缺乏实际意义。部分诊所可能缺乏训练该模型所需的数据源，进而促使机构去采集以往缺失的数据。 - **业务与风险权衡**：允许承担多大程度的不确定性，本质上是一项由组织领导层拍板的商业与战略决策。 针对人类与机器容错率的讨论，Anik Bose 补充道：临床实践中人类自身的准确率可能只有 94%，要求机器必须达到 100% 准确率往往是一种认知偏差。 Caleb Poore 随后从院长的管理角度进行回应。他风趣地提到自己每到一家机构都会与 IT 团队成为挚友，因为 IT 部门幽默且至关重要。作为医院 CEO，必须在宏观上把控合规与安全边界。无论前端采用的是 ChatGPT、Claude 还是 Grok，核心底线在于所有技术必须集成在医疗机构受控、安全的私有生态系统（Ecosystem）之中，严格恪守 HIPAA 等医疗数据合规底线，在此基础上稳健管控风险。 ### [032:06] 医疗生态系统安全与医疗错误的现实基准 在探讨系统风险与数据合规时，Caleb Poore 指出最关键的底线在于：所有技术动作必须发生在你自己的生态系统内部，并且完全符合 HIPAA（健康保险可携性与问责法案）标准。以 Boone County Health Center 为例，他们使用的 Copilot 工具被严格隔离在机构自己的 Office 365 产品环境内。这意味着即便系统出现某些失误，这些错误也至少被封锁在自家的内部闭环中，不至于外泄。关于如何衡量 AI 犯错的严重性，Poore 提出了一个务实的对比：以前许多医生在病历系统中只潦草地写上一行字，与这种信息极度匮乏的传统顽疾相比，一份结构完整、信息详实但可能偶尔有一两个单词拼写错误的 AI 生成病历，其风险显然要低得多。引入 AI 带来的收益，总体上已显著压倒了过去的固有缺陷。 随后 Aalpen Patel 用震撼的数据揭示了现代医疗面临的残酷现状：全美每年大约发生 1200 万起医疗差错，由此直接或间接导致的死亡人数高达约 25 万人。Patel 直言“让这个数字在脑海里沉淀一下（Let that sink in a little bit）”，并指出尽管该数字在业界存在争议，但约翰霍普金斯大学 2016 年发表在《英国医学杂志》（The BMJ）上的重磅研究对此进行了充分论证；而更早之前，美国医学研究所（Institute of Medicine）也曾估算出每年因医疗差错死亡的人数多达 9.8 万人。在如此巨大的现实伤亡基准下，医疗界引入任何新工具的核心评价标准其实非常纯粹：只要 AI 的表现能够优于现有水平、能切实减少哪怕一部分医疗差错，就应当毫不犹豫地予以采用。 - 关键数据：美国每年医疗差错约 1200 万起，致死人数约 25 万（BMJ 2016 约翰霍普金斯团队论文）以及既往 IOM 报告的 9.8 万人。 - 核心权衡：AI 偶尔出现的文本瑕疵或微小失误，相比人类医生“只写一行病历”导致的严重信息缺失，风险等级截然不同。 --- ### [033:41] 本地化验证的必要性与规模化治理的困局 针对 Aalpen Patel 提出的“只要比现状好就应引入”，Caroline Chung 补充了一个不可忽视的前提：你必须在工具即将落地的具体临床语境中去衡量其性能表现。许多厂商和机构在宣传时喜欢引用提交给 FDA（美国食品药品监督管理局）的数据，但那些测试是在经过高度清洗、专门为了过审而挑选的数据集上完成的，根本无法代表该模型在特定医院、特定患者群体中的实际表现。因此，在将算法开箱即用前，必须先进行本地验证（Local Validation）与本地微调（Local Fine-tuning）。 Morgan Jeffries 从自身在 Geisinger 负责 AI 治理体系的实操经历出发，剖析了规模化监管面临的巨大摩擦： ```mermaid flowchart TD accTitle: AI治理与监控流程中的现实摩擦 subgraph S1["准入与评估"] A["捕获新工具<br/>(独立应用与隐形嵌入功能)"] --> B["组建跨学科专家团队"] B --> C["测算本地风险与制定缓释方案"] end subgraph S2["落地与运行瓶颈"] C --> D["持续跟踪归档与合规记录"] D --> E{"规模化监控机制"} E -- "人力密集型" --> F["全团队转型做厂商评测"] E -- "技术瓶颈" --> G["缺乏自动化监控工具"] end ``` - **从独立工具转向隐蔽功能**：医疗系统正在迅速告别“什么软件都带个.ai 域名”的初级阶段，大量传统老牌厂商开始直接在其系统内部悄然上线 AI 功能。这些功能不会大张旗鼓地自我声明，导致医院在源头捕捉和盘点新增功能变得极为困难。 - **风险测算的不可迁移性**：厂商提供的风险指标往往缺乏实质参考价值，即便数字漂亮，也无法直接套用到本地特有的患者群体上。 - **评测资源的极端倾斜**：目前行业内几乎没有哪家机构能够在不堆积大量人力的前提下，实现模型在规模化运行中的全量评估与监控。Jeffries 透露，某家大型医疗系统甚至不得不解散原本从事内部系统自主研发的整个技术团队，将所有人全盘转岗去专职评测外来厂商交付的系统。 --- ### [036:20] 人机协同的界限与面向患者侧 AI 的致命隐患 在探讨风控机制时，Morgan Jeffries 指出医疗系统目前高度依赖“人机协同（Human in the loop）”，但这套模式之所以能够勉强运转，是因为现阶段落地的产品绝大多数都是面向医护人员等临床专家的。这些临床医生受过多年的高强度专业训练，同时又是医院雇员，管理层可以在责任归属上对他们形成强约束，明确告知“最终把关责任在你身上，你必须核对算法输出”。然而，当 AI 工具转向直接面向患者（Patient-facing）时，这套逻辑便会彻底瓦解。 医疗系统面向患者编写的所有材料，通常默认受众只有小学五年级（Fifth grade）的阅读水平，绝不能预设他们具备医学专业鉴别力。如果开发一个面向患者的聊天机器人（Chatbot），一边告诉患者“相信我们的系统”，另一边又在免责声明里要求他们“请自行核实所有答案”，这在沟通上是完全自相矛盾且混乱的。更严峻的是极端事件的防范：若无法在规模化监控中彻底查明系统的各种失效模式（Failure Modes），医院就无法百分之百确信当患者在对话中流露出自杀倾向（Suicidal ideation）时，系统是否一定会将工单即时上报给人工团队。以往由真人负责回访时能兜底的生命红线，一旦被尚未成熟的面向患者 AI 遗漏，将引发灾难性后果。 --- ### [038:10] 给管理层的决策忠告：合同合规、团队共育与抵御跟风 在对谈的尾声，主持人提出最后一个问题：面对在场的医疗系统高管，在他们为 AI 投入下一笔预算之前，最想给出的一条核心建议是什么？四位嘉宾分别从法律数据主权、临床信任构建、战略节奏控制以及顶层系统设计四个维度给出了警示： | 嘉宾 | 核心忠告 | 关键案例与行动建议 | |:--- |:--- |:--- | | **Caleb Poore** | **深挖采购合同与数据流向** | 务必明确数据所有权、数据流向、访问权限以及转售/交易条款。Poore 分享了一个惊人的内部排查案例：某家支撑医院核心业务运转的大型供应商，利用合同漏洞避开“出售数据”字眼，将包含各保险公司实际报销金额的敏感账单数据输入给第三方数据仓库，而该仓库的核心付费订阅者正是各大商业保险公司。保险公司利用这些数据精准压低对医院的报销额度，导致医院的自身数据沦为对付自己的武器。必须由专业法务逐字审核数据保护条款。 | | **Aalpen Patel** | **全员教育与深度绑定临床** | “教育、教育、再教育”。在研发模型过程中，Patel 曾与临床团队反复开会沟通五次，才发现最初定义的业务痛点只是 X，而真正的临床痛点其实是 X+Y。必须让医生明白“AI 不是巫术（AI is not voodoo）”，其背后是严谨的数学逻辑与科学方法。只有让一线人员理解工具是如何被制造出来的，他们才敢在临床中真正接纳并使用它。 | | **Morgan Jeffries** | **放缓节奏，拒绝伪“AI First”** | 推进速度应慢于管理层的冲动预期。部署 AI 的难度和摩擦总是远超预判，必须预先测算能拿到的真实价值并保持高度怀疑审慎。除非董事会施加不可抗拒的下压阻力（即便如此也应适度抗辩），否则千万不要盲目陷入“以 AI 为先”的虚荣人设中，始终将注意力放在为患者创造实质临床价值上。 | | **Caroline Chung** | **顶层战略聚焦与分级排序** | 部署 AI 从来不是单点技术问题，而是一个极其复杂的“系统的系统（System of systems）”挑战。领导者必须将战略重心放在整体路径规划和优先级排序上，在推进过程中切实带领全体人员共同演进。 | ### [42:44](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2564s) 医疗AI落地的全方位协同、持续治理与优先级确立 在医疗体系中推进人工智能技术的落地，绝非单一技术团队的任务，而是一场涉及多方角色的系统性变革。这一过程不仅需要临床医护人员深度介入，还需要 IT 团队以及所有负责数据输入与维护的人员通力配合。将 AI 模型顺利嵌入现有系统，需要全面协调各方资源，尤其是必须从一开始就让合规与风控团队介入审核。更关键的是，AI 的部署绝不是“上线当天按下开关就万事大吉”，后续还需要建立常态化的持续监测机制。这代表着巨大的投入——不仅体现在资金预算上，更体现在整个机构运营模式与工作流的深刻重塑。 面对 AI 技术的广泛适用性，医疗机构必须保持清晰的战略定力，学会明确优先级并设定清晰的成功标准。决策者需要提前评估自身的风险承受能力，并确保拥有足够支撑落地的资源，因为医疗 AI 项目“绝不可能廉价搞定（You can't do this necessarily on the cheap）”。尽管 AI 展现出无处不在的应用潜力，但在第一天就试图全面铺开是完全不切实际的。各个机构应结合自身实际情况摸清核心诉求，向全员指明清晰的发展方向，并通过系统性培训、教育与科学的评估指标，带领全院上下稳步迈入智能化转型。 ```mermaid flowchart TD accTitle: 医疗AI落地实施框架 subgraph S1["前期准备与评估"] A["明确业务优先级与目标"] --> B["评估风险承受力与资源"] end subgraph S2["多方协同与部署"] C["临床 / IT / 数据团队联动"] --> D["合规与风控审查"] D --> E["系统集成与上线"] end subgraph S3["长期运营与维护"] F["持续监测与指标评估"] G["全员培训与能力建设"] end B --> C E --> F E --> G ``` - **全员参与协同**：涵盖临床一线医生、IT 架构团队以及数据流转各环节的操作人员，确保全流程无缝衔接。 - **全生命周期合规监管**：合规团队需常态化介入，从方案初期设计、上线审批到投产后的持续监控，防范长期算法漂移与安全风险。 - **合理设定预期与成本**：拒绝盲目铺摊子，优先在风险可控、资源充足且能明确定义“成功形态”的场景中率先突破。 --- ### [43:57](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2637s) 总结与思考：第四生产要素“智能”的崛起 在圆桌论坛的尾声，主持人对全体嘉宾的深度分享表达了感谢，并为现场与会者留下了一个关于未来产业变革的核心思考。过去一百多年来，传统工商业主要依托三大经典生产要素运转：劳动力（Labor）、资本（Capital）与信息（Information）。而人工智能技术的成熟与爆发，为现代产业带来了第四个全新的生产要素——“智能”（Intelligence）。 这一全新生产要素的注入，要求企业与医疗机构从底层逻辑出发，重新审视和构建自身的业务优先级、日常工作流以及组织治理架构。单纯带着赶时髦的心态盲目涌入，或是采用“简单复制粘贴（blindly cutting and pasting）”的方式照搬现成方案，注定无法发挥 AI 的真正价值，甚至会带来不可预测的风险。唯有深刻理解智能要素对生产力关系的重塑，企业才能在智能化时代建立起长久的竞争优势。 - **生产要素的历史演进**：传统时代的三大支柱（劳动力、资本、信息）迎来了由 AI 驱动的第四维度——“智能”。 - **企业重构的核心维度**： - **战略优先级**：重新界定业务价值与投入产出比； - **业务工作流**：打破原有工序，实现人机协同共生； - **组织治理**：建立适配智能要素的安全、合规与风控体系。 ## Notable quotes > [04:13](https://www.youtube.com/watch?v=UBFVn8oRy10&t=253s) 求求你们千万不要把这个系统撤走，用上它之后我才意识到自己过去过得有多痛苦，如果关停我真不知道该怎么办了。 — 临床医生（由 Morgan Jeffries 转述） > [04:13](https://www.youtube.com/watch?v=UBFVn8oRy10&t=253s) 模型本身并不是最重要的，它只是其中一部分；真正的关键在于如何把它无缝融入日常工作流程。 — Aalpen Patel > [15:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=910s) 算法正在学习我的诊疗行为和临床思维，并将这些数据存入数据库，这在 10 到 15 年后会不会让我彻底失业？ — 医生（由 Caleb Poore 转述） > [15:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=910s) 仅仅因为我们‘有能力’上线某项功能，是否就意味着我们‘应该’去部署它？ — Caleb Poore > [26:18](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1578s) 不要把 AI 强加给他们，而是为了他们去做 AI（Don't do AI to them, do it for them）。 — Aalpen Patel > [32:06](https://www.youtube.com/watch?v=UBFVn8oRy10&t=1926s) Let that sink in a little bit — Aalpen Patel > [38:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2290s) 教育、教育、再教育 — Aalpen Patel > [38:10](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2290s) AI 不是巫术（AI is not voodoo） — Aalpen Patel > [42:44](https://www.youtube.com/watch?v=UBFVn8oRy10&t=2564s) 绝不可能廉价搞定（You can't do this necessarily on the cheap） — Caroline Chung ## Mentioned Anik Bose, Caroline Chung, Caleb Poore, Aalpen Patel, Morgan Jeffries, Benhamou Global Ventures (BGV), MD Anderson 癌症中心, Boone County 医疗中心, Geisinger, MedQuest Imaging, Casper Group, 美国国家癌症研究所 (NCI), FDA (美国食品药品监督管理局), 约翰霍普金斯大学, 美国医学研究所 (IOM), The BMJ (英国医学杂志), EHR / EMR, Ambient Scribes / Ambient AI, Office 365 Copilot, ChatGPT, Claude, Grok, HIPAA, MIPS, ACO ## Video description Morgan Jeffries - Medical Director for AI, Geisinger Anik Bose - Managing Partner, Benhamou Global Ventures (BGV) Aalpen Patel - Former Chief Clinical and Innovation Officer, Former MedQuest Imaging Caroline Chung - Professor and Co-Director of the Institute for Data Science in Oncology, The University of Texas MD Anderson Cancer Center Caleb Poore - CEO, Boone County Health Center AI can augment the clinical workforce—if deployed responsibly. In this holistic panel, provider organizations share how they’re using AI to enhance diagnostic accuracy, streamline documentation, and reduce administrative burden. We’ll dive into deployment strategies, integration with EHRs, and how clinicians are being trained to work with AI at the bedside. Join leading hospital executives, technologists and physicians in this impactful conversation about the modernized future of healthcare. [Source](https://www.youtube.com/watch?v=UBFVn8oRy10)

[4:13] 临床医生（由 Morgan Jeffries 转述）: 求求你们千万不要把这个系统撤走，用上它之后我才意识到自己过去过得有多痛苦，如果关停我真不知道该怎么办了。 [4:13] Aalpen Patel: 模型本身并不是最重要的，它只是其中一部分；真正的关键在于如何把它无缝融入日常工作流程。 [15:10] 医生（由 Caleb Poore 转述）: 算法正在学习我的诊疗行为和临床思维，并将这些数据存入数据库，这在 10 到 15 年后会不会让我彻底失业？ [15:10] Caleb Poore: 仅仅因为我们‘有能力’上线某项功能，是否就意味着我们‘应该’去部署它？ [26:18] Aalpen Patel: 不要把 AI 强加给他们，而是为了他们去做 AI（Don't do AI to them, do it for them）。 [32:06] Aalpen Patel: Let that sink in a little bit [38:10] Aalpen Patel: 教育、教育、再教育 [38:10] Aalpen Patel: AI 不是巫术（AI is not voodoo） [42:44] Caroline Chung: 绝不可能廉价搞定（You can't do this necessarily on the cheap）

[4:13] Dr. Morgan Jeffries: please never take this away [4:13] Dr. Morgan Jeffries: I didn't realize how miserable I was until this was deployed [4:13] Dr. Morgan Jeffries: I cannot tell you what I would do if it were ever turned off. [7:51] Caleb Poore: the patient was good, next visit. [24:28] Dr. Aalpen Patel: Don't do AI to them, do it for them. [35:40] Dr. Morgan Jeffries: trust us, but also verify all these answers yourself [40:08] Dr. Aalpen Patel: AI is not voodoo; there is method behind it, there's mathematics behind it.

[0:04] I'm the moderator. My name is Anik. Uh I'm a  venture capitalist with BGV. So I'd like to [0:09] get started by just asking each of our panel  members to introduce themselves briefly and [0:14] talk a little bit about what's uh kind of really  hot top of the list on their AI on the desk right [0:19] now. Sure. Hello everyone. I'm Dr. Caroline  Chung. I'm a professor in radiation oncology [0:25] and diagnostic imaging, the co-director of  the institute for data science and oncology [0:29] and have served as the inaugural chief data and  analytics officer at MD Anderson Cancer Center. [0:34] I would say one of the things that I'm working on  with a collective community both industry NCI as [0:41] well as many academic organizations is really  how do we start to benchmark what's meaningful [0:47] not just measuring what's easy to measure but  really measuring what is going to drive that [0:54] needle forward and that goes beyond just accuracy  of the model which is sort of where we're at [0:58] at this point but really thinking about what  would we need to convince ourselves that this, [1:04] whatever tool it may be, is going to really  improve care, drive true efficiency, and are [1:10] we actually measuring what matters? Hi, I'm Caleb  Poor, uh, president and CEO of Boone County Health [1:16] Center in Nebraska. Um, worked as a CEO now and in  executive leadership for about 10 years now. Uh, [1:24] this the town I live in is about 1,800  people. I'd say there's a extreme interest [1:28] in rural right now just um a lot of different  needs in the rural communities around us and [1:34] um things that ensue there. You know, for us with  regard to AI, you know, we're pretty limited in [1:40] what we have in terms of staffing right now. So,  filling different uh capabilities, ways to use AI [1:45] in the rural setting to offset some of those labor  shortages and needs, I think, is paramount for us. [1:50] Um and also you know just by the sheer nature  of being in rural we are a lot different than [1:56] you know populist settings we we take a lot more  time with our patients so we we tout you know that [2:01] we offer a little bit more of a care product just  because we can't uh we don't push people through [2:06] quite as fast as they do in populace areas. So um  you know figuring out how AI can complement that [2:12] quality but not sacrifice the quality at the same  time. So happy to talk a little bit more about [2:16] this. Thanks for having me today. Hi everyone. I'm  Alpin Patel. I am an interventional radiologist [2:24] um a clinical informaticist as well as a  recovering uh electrical engineer. Um and [2:30] uh I have been in the in the recent years uh  the chair of radiology at um Guisinger Health [2:37] for about 13 years. um was also the medical  director for AI there for a little bit and [2:42] um since then I was at um Mequest Imaging as  a chief clinical and innovation officer and [2:48] more recently I am standing up a startup uh  in healthcare health technology space called [2:55] Casper Group uh it's still in stealth um um but  it's it's in caregiver space what I what I'm my [3:02] interests are is between the model and um and  the people the layer that's between that uh Um, [3:08] how do we use that most effectively to uh to take  better care of patients? Uh, hi, I'm Morgan Jeff. [3:17] I'm a neurohosist and I'm the medical director for  AI at Guisinger. I succeed at Alban in that role. [3:23] Um and uh I would say that the most important  things that we're focused that I'm or the biggest [3:30] priority for us right now um there are a lot of  there's a lot of innovation uh happening in the [3:39] clinician experience space. Um that is going  to really transform that in the next we'll say [3:45] well it's starting now but within the next uh  three years say um and we're starting to run [3:51] some pilots in that area. Um, however, broadly, I  would say the thing that I think is the the thing [3:59] that I've got that I'm focused on that I think is  is the biggest challenge is scalable evaluations [4:04] and monitoring for clinical AI. So, thank you.  So, one question maybe Morgan you can go first is [4:13] uh we've heard a lot about AI and the plethora of  pilots that are out there. would love to hear from [4:18] you and each of you in terms of uh an initiative  where AI has scaled from a pilot to production. [4:27] Sure. Um yeah, I I can go first on that. Um so  um the most obvious one uh that that has been [4:34] the most impactful actually uh this actually  preceded me um but our deployment of ambient [4:41] scribes uh was that was one that was a real big  success. Um, so we I'm going to say it's probably [4:50] been probably been about two years since we uh  since we went uh enterprisewide with that. Um, [4:57] and it's we we were able to first of all, we  had to devote a lot of resources to doing this, [5:03] but we were able to demonstrate uh significant  time time savings in notes uh for our clinical [5:09] users. I think it was about every time I look at  the dashboard it's different but it's something [5:14] like 2 minutes per note which really adds up over  the course of a day. Um so there was a there was [5:21] this hard objective finding and then there was  also the uh the more subjective feedback that [5:28] we got which the sentiment that we received is the  kind that you really never get in informatics when [5:35] you when like when you roll out a new initiative  nobody's ever this excited about it and like [5:40] people are saying things like please never take  this away I didn't realize how miserable I was [5:45] until this was deployed uh I cannot tell you what  I would do if it were ever turned off. Uh you know [5:51] that kind of I people are super excited about it.  Um so I think that combination of hey look our [5:58] gut is telling us that this is good and important  and we can measure at least one objective thing. [6:04] Um that and that that really enabled us to  scale and and to feel justified in doing so. [6:11] So I can tell you Max um couple of one of the one  of the ear most earliest projects we had was in [6:19] 2015 perhaps 2014 where we realized that CNN's  are coming online convolutional nets and we were [6:27] sitting on a massive amount of data. Gazing was  one of the early adopters of um of EMRs as well [6:33] as digital technology. uh so we were sitting on  2.3 pabytes of data just on in just in imaging [6:39] and about 25 years of um of um just EMR data so  massive amount of data so what do we do with it so [6:46] under the opaces of institute of dance application  we we went stealth and uh when we emerged we [6:52] actually had a product u and there was this was  early on it was it's nothing monumental but we [6:57] actually had a product that u that detected um  brain bleeds but to get to where we needed to get [7:04] to was the was the real journey because we didn't  have a way of getting the data from the clinical [7:09] side to the research side. We didn't have a way  of uh communicating. So we had to build all of [7:14] those things. Model is not the important thing. It  is part of it but how do you actually put it into [7:20] workflow and we had to design that workflow? We  had to design the data flow all of those things. [7:26] And that's that's something that we had to to  to plan for and not just not just do the model [7:31] itself. So we we we started with translation in  mind uh and and then then translated um and that [7:40] was that was one of the most exciting uh projects  we had uh that that translator from not just the [7:45] uh uh just doing just making the model itself but  thinking about how to deploy it to begin with was [7:51] the key. Yeah, I think that you know for Boone  County Health Center again town of 1800 people [7:56] serve a total population of around 10,000.  You know I'd have to agree with Morgan. We [8:00] we implemented ambient AI with our EHR vendor and  it's it's been amazing. You know, what it's really [8:05] done for us is our patients really get that,  you know, before you got a provider, you know, [8:09] back turned, patient talking, you know, now they  set a recording on their phone, set it down, and [8:15] they really engage with that patient one-on-one  eye contact. The patient feels a lot more heard, [8:20] feel valued from a standpoint of, you know, they  heard my concerns. They they saw it. you know, [8:26] you probably wouldn't mind telling me, but uh  one of our most, I will say, tenure doctors, [8:32] um he uh is notorious for not doing a lot with  his notes. You know, it's like one line like [8:37] the pat patient was good next next visit. So,  so uh these next uh um with this ambient AI, [8:45] it just really helps us build out the chart a  lot more and make a case for each one of our [8:49] patients. We went from like zero data to like  maximal data. Um, so it just led to a lot of [8:55] downstream issues that we had to work through  right away. Our IT infrastructure hadn't been [8:59] upgraded in like 20 years. And so the the  problem is when you sign up for Ambient AI, [9:04] there's so much data coming back through your API  interfaces that you literally have to increase [9:10] the tube of internet that you're bringing back  into your facility because it's that much data. [9:14] I think for myself and a lot of my critical  access hospital colleagues in Nebraska, you know, [9:19] you start to realize, oh my gosh, we were sitting  on a gold mine this entire time and we didn't [9:24] know it. And it's it's our data. There's a lot of  people who want that data for for many different [9:28] reasons. Research um vendors want that data, a  lot of people. So, we're really talking through, [9:33] you know, governance now and starting to rethink  how we actually set up our contracts with vendors. [9:39] What does data governance look like? We're  finding out that with a lot of our vendors, [9:42] you know, it's open-ended and they probably did  that for a reason. They can use that data for a [9:47] lot of different things and we're kind of trying  to take background a little bit and say, well, no, [9:51] you can't use it for this, this, and this, this.  We are the administrators of that. You got to get [9:55] written communication from us before you can  do that. So, it's just a lot for us as a rural [10:00] facility. It's been really good for our patients.  It's enhanced the the quality of care, built out [10:05] the charts more, helped us hit quality measures  more with our MIPs and acco um all while not [10:12] really um bogging down our providers. It actually  enhanced our providers to see more patients. So, [10:18] it's really been pretty rewarding for a hospital.  I would say from a revenue cycle perspective, [10:22] you know, from a billing and coding standpoint,  which closes the loop on revenue cycle, [10:26] it's been outstanding as well. So, I guess those  are probably some of the biggest areas that AMIA [10:31] AI has worked for us. There's there's other areas  too, but I'm sure we'll get into that. Caroline, [10:36] I'm curious because I I I agree that I think  that um the reception of clinicians for ambient [10:41] listening tools is is incredible and it is almost  an emotional reaction in many circumstances. Um I [10:48] am curious though that because many of the early  publications was really focusing on this thing [10:53] called pajama time. And so if for those of you  who are not clinicians or are not aware, many [10:58] clinicians will sit up at 10 p.m. at night and  put all their notes from the clinic that they just [11:03] spent their day spending time with. Um and so the  one of the questions that we asked our physicians [11:09] around pajama time was, you know, is this  meaningful to you? And being an academic health [11:15] center, I think that some of our physicians chose  to have dinner with their family and they said, [11:19] you know, if you're clocking and this is this goes  back to what are you actually measuring and is it [11:24] meaningful for your context in your setting and  so understanding who your providers are and what [11:30] their practices are. Some people chose to do  their notes at that time because they wanted to [11:35] spend time with their children doing after school  whatnot. Um, and they chose to spend that time. [11:40] But if you're just clocking on the EHR, when  is this person logged in? You may not actually [11:44] be getting the metrics that really matter. And so  understanding the context in which you're actually [11:50] deploying is really important. Um I guess the  other piece around this is that I know everyone [11:54] has been driving across efficiency and speed,  but how are we actually evaluating the quality [12:01] of the note and is someone actually checking  as carefully or are you starting to trust the [12:08] the note? And this happens in a lot of diagnostic  tools as well. Um I think that the automation bias [12:14] and understanding that human AI interface and what  that means and how it actually uh the behavior at [12:22] day one is going to be different than a behavior  at day 1000 or two years into the future. And so [12:28] how do you actually set up continuous measuring  to understand what the behavior change would be [12:34] is also very important. And not only the people  behavior but also the model behavior, right? [12:38] Model drift is a real thing. Exactly. Um and data  drift as well. Yeah. Then data drift and and data [12:44] drift leading to model drift. Right. So so all  of those things um you have to actively monitor [12:49] them and make sure that you're staying within  the uh defined parameters otherwise uh you can [12:54] potentially harm patients. Thank you Alvin. So uh  Caroline, you made a very good point about how do [13:00] you measure value? How do you measure ROI? and uh  we we'd love to kind of get your perspective and [13:06] all of yours in terms of what you've seen that  has worked and what has not worked in trying [13:10] to address that question. Yes. I think that  when we talk about evaluating AI, I think many [13:17] evaluations, it's not that we're not evaluating  AI models. I think many people are evaluating [13:23] AI models, but measuring just model performance.  You can have an excellently performing model that [13:30] still doesn't move the needle in terms of how you  how you would plan to use the tool. Um going back [13:36] to how have you implemented it in your workflow?  Have you reshaped your operational workflow [13:41] because you're actually introducing this? How do  you how have you actually mitigated the safeguards [13:46] or are you adding additional human burden because  people are needing to recheck everything 10 times [13:53] over? And so I think that thinking about what is  it that end goal that you're trying to move. So [13:58] if you're trying to improve efficiency, you want  to improve efficiency and I think all of us would [14:04] agree, let's say it's a diagnostic tool, you  may improve efficiency, but you would not want [14:08] to do that at the detriment of losing accuracy or  quality. you want to have both. And so making sure [14:15] that you're measuring both of those pieces to show  that you're truly showing value would matter. Um, [14:21] how much uncertainty are you willing to accept?  And that may depend on how you're planning to use [14:26] the tool, in what context, who's actually using  the this tool, and what's your current current [14:34] situation. And so if your current situation is  you're in a rural area with no specialists and [14:39] you may get some insight that will help someone  direct someone to get referrals appropriately, [14:44] perhaps it it may not be the perfect  tool. You may not have the same bar, [14:49] but it's still going to move the needle in terms  of how you're planning to use use this tool. And [14:54] that same tool in different contexts may mean that  you've either met your mark or not met your mark. [14:59] And I think that that contextual contextaware  evaluation is actually a really critical piece to [15:06] defining value. Thank you. Other other thoughts  on that question? Yeah, I would just say that [15:10] I think um generally when you're evaluating  the accuracy and things like that, you know, [15:17] a big scare I think in the in the workplace right  now with AI is is and it's rampant of, you know, [15:22] is it going to take my job? uh really and and  to Caroline's point, I mean, we just take those [15:27] staff and or or a fraction of those staff and  redirect them to other other areas to do to do [15:32] other things. One of which is just review the  documentation. Now, there is savings from a a [15:37] labor standpoint of um you know, you don't where  you had maybe four people that were uh dictating, [15:43] transcribing before, you don't need four people to  read notes. It goes a lot faster. So, there is an [15:49] ROI savings on that on that component of things  certainly. Um I think what I hear a lot from is [15:55] you know our doctors you know that are loving this  and eliciting you know emotional responses of that [16:00] they they do get to have time back with their  families and those things. You know the very [16:04] next question though comes is what does this mean  for me long term? I mean these AI algorithms are [16:10] are literally learning upon my medical behaviors  and how I treat my patients and that data is all [16:16] going to some database uh for somebody to look  at for how I treat my patients. And so is this [16:21] somehow going to ultimately put me out of the  job? So I think everybody knows certainly short [16:25] term that you know those probably aren't realistic  goals for the tech industry 10 15 years. I think [16:32] things probably look a little bit different.  Um so I think you know as as AI unrolls itself [16:37] and and we all learn more about it including  patients I think it'll just be interesting to [16:42] see how things evolve. I think everybody needs to  just take a step back is what I've told my staff [16:48] and you know uh don't be so quick to jump into  something that's going to make your job overly [16:53] easy. Certainly we'll always have a need for  clinicians and and nurses, providers, everybody, [16:59] but you know there certainly is the human  component that AI does not capture uh currently [17:04] and and that's something that we do in rural  work really well and I think really clinicians [17:08] outside of rural. So I just think you know as  we evaluate AI in the coming years at least [17:13] for our facility we're having a lot of dialogue  on just because we can implement something does [17:18] that really mean that we should [snorts] um so  I think um so so value and ROI really have to [17:28] be defined by the problem that you're trying to  solve. If you're trying to solve the operational [17:32] problem making operations more efficient  that's easy to translate to hard dollars. [17:38] But if you're trying to say that I'm going to make  life easier for clinicians, that's not so easy to [17:43] translate onto a spreadsheet. Uh so but but that  may require a couple of transformations, right? [17:48] So you have happier clinicians who are there's  going to be decreased turnover. You're going to [17:53] be able to see more patients and therefore you're  going to be able to then retain those clinicians [17:58] and then be able to see more patients and that  translates into it. Right? So so yes, there is [18:03] the value is going to be defined by the problem  that you try to solve. Uh so that's going to be a [18:08] little different and and to your point uh earlier  about about solving the wrong problem can really [18:15] cause problems. So in my former role we bought a  CT scan and somehow the vendor turned on that the [18:23] uh automatic nodule detection on a CT scanner.  Now the radiologist had two sets of scans the real [18:29] scans and then the post scans. So now they had  to look at it twice. So I didn't help them any. [18:34] I actually increased their work because it wasn't  it wasn't implemented right. It was not it was a [18:39] it was an unplanned implementation that happened  and I I increased their work. So I got a lot of [18:43] concerned phone calls uh to to turn that off and  do the do it right. So I think I think that's [18:48] um that's that's really the real question here.  Yeah. That what what is the problem? How are you [18:54] trying to solve? Yeah. Um so I I'd say there are  a lot of challenges as I said before in evaluating [19:01] and and monitoring AI but specifically for ROI.  Um you know we you hear a lot that we haven't [19:08] been able to demonstrate ROI in healthcare. I  think there are some applications where there's [19:13] like a direct financial ROI and you see there's  there's this thing where like every system keeps [19:20] evolving until it you know until it has something  that impacts RCM. uh because if you can increase [19:25] billing that's that's a net good. But why is  that? It's not necessarily that we only care [19:30] about money. But money is completely fungeible and  you're paying money for the system. So if you want [19:35] to demonstrate that you're getting your money's  worth, it's super easy to do that if you can say [19:39] like we got we paid this much, we got exactly  this much out, there's no question that that's [19:43] worth it. Conversely, like if you start to talk  about, you know, lives saved or whatever, I mean, [19:48] every life saved is valuable, but you know, a  hundred million dollars to save a single life [19:54] is not a sacrifice that we can make. How much  is saving one life worth? I don't know. And [20:00] that's part of the problem is that it's very  you actually have to think about that and come [20:03] to an agreement about it. And there might be  some disputes. That's not the case with money. [20:07] There's also an issue with attribution that could  be challenging in measuring some of these things. [20:11] So like for I I think even just from a financial  standpoint, the fact that we saved uh physicians [20:17] and and and you know a time uh and that they  loved uh you know ambient so much probably [20:24] translates into higher retention and retention  physician replacement costs are extremely high. [20:30] I've read estimates of like a million dollars.  Um, but how do you know that you know somebody [20:38] stayed because you had ambient around? I mean, I  think when so many people love it, it's plausible [20:44] to say like we might have we probably saved  one person. Like it's it's plausible that in [20:49] three years we save two three years we saved one  person from that doesn't seem like such a stretch, [20:54] but you know how high is that number? I have no  idea. Um and then I would say like a a third major [21:00] issue is just what we're get like I mentioned for  you know ambient that we that we were able to just [21:07] demonstrate this time savings in notes. Okay. Well  we built a custom dashboard for that. So there was [21:12] an organizational push that we're this is going  to be a priority. Uh we're going to measure the [21:17] system. We're going to measure that wasn't the  only thing that we measured but that was the one [21:20] objective that was the one objective measure that  we found something. Um, but we had to dedicate a [21:26] lot of resources to that that weren't then able  to do other things. And if every system you want [21:31] to deploy, you've got to roll your own. That's an  issue. I will say, I'm not going to name names, [21:36] but perhaps there are vendors out there that maybe  you can only see usage metrics and you have to [21:41] accept things like, well, I guess people are using  it, so they find it valuable. That's good, right? [21:46] How good is it? I don't know. I don't I don't know  how to value that. So, a lot of challenges there. [21:53] So you've all touched upon the issue that the  AI transformation is not really technical, [21:57] it's organizational, right? So what have you  seen given kind of the level of skepticism that [22:02] most employees have whether they're, you know,  clinicians or nurses or doctors around AI? What [22:09] are some leadership best practices you've seen  that actually helps move it from the issue of, [22:15] you know, the skepticism to actually embracing  the technology where it's doing good? I would [22:22] say that not everyone's a skeptic. I would say in  any organization, you've got about a quarter of [22:27] the institution that wants to go faster, do more.  Let's just get out there and break things because, [22:32] you know, we need to innovate because we can't  keep doing what we're going to do. There's [22:36] probably another bit quarter of the institution  that truly are completely frightened, skeptical, [22:42] neither area is going to destroy the world sort  of perspective. And then there's people in between [22:48] who are just like, well, I'm not so sure. um there  seems to be some benefit really if if that's the [22:53] direction we're going to go I'm happy to jump on  board and so I think you know moving the needle [22:58] to try to help educate the different groups with  their different needs is a really critical piece [23:03] and I would say that the literacy and fluency  is something that I think we're all struggling [23:08] with because the technology is also moving I  mean I think if you ask people what AI is today [23:14] um there's people who can are very well verssed  and can talk you through all the different kinds [23:18] of AI and many others who will say well let's chat  GPT right or whatever your name your favorite LLM [23:24] and that's that's that's AI by their definition  and so I think bringing that literacy and fluency [23:28] so everyone's talking the same language and  starting to understand helping people who are [23:35] concerned be able to experiment and start to see  some of the benefit while also helping people who [23:41] are wanting to innovate start to hear more or  even reflect on when something does break why [23:49] did that break so they can actually you know learn  the lessons and then bringing together a community [23:54] um it's really around the culture and not only  the operational pieces if we just put processes [23:59] in place I think that a lot of people will get  around those processes but it's really building [24:03] that culture around your organization yeah and  I think that you know for us um you know I agree [24:09] like actually the skepticism is probably like  I was just thinking like as you were mentioning [24:14] categories I'm like that's probably 90% for  us and that's like 5% of skepticism And then [24:19] 5% of the people in the middle, the overwhelming  majority of healthcare workers right now, I feel [24:23] like want AI. They they want to know how are you  going to make my job easier? And it goes to really [24:28] culture of employees, generational employees  are different today than they were, you know, [24:31] 20 years ago. The level of workload that people  expect is a heck of a lot different today than it [24:36] was 20 years ago. So, it's just it's different.  I would say though, you know, um things to look [24:42] out for, you know, for us historically, we've  always staffed our clinic with RNs. you know, [24:46] do we re, you know, now that our doctors have an a  recorder that automatically distributes, you know, [24:51] HPI and and reason for visit and, you know, some  of these DRGs into their respective components, do [24:56] we really need an RN in our clinic, you know, as  many as we have to to do to get a set of vitals, [25:03] a reason for visit and make phone calls for  referrals? Probably not. I mean, so, you know, [25:08] I I think you're going to need time though for  those things to change just because, you know, [25:12] I think people are under the understanding that  healthc care facilities are just like, okay, [25:16] let's let's get all the RNs out of here and go  go to this classification of employee. It just [25:20] doesn't work like that. Um, you know, a lot of  people probably end up doing things according [25:24] to attrition. You know, for our RNs, we we want  them and need them to be in our highly acute, [25:32] very complicated areas of the hospital. med  surge, ER, OB, you know, those areas that, [25:38] you know, we have really complicated patients that  come in that really do require the needs of an RN, [25:43] you know, uh refocusing those employees to toward  those departments. But, I mean, we live in the [25:49] the most acute nursing shortage ever really. I  mean, there are not nurses anywhere right now, [25:55] especially for rural rural hospitals. And I  would even contend that in the populace area, [25:59] it's the same way. Um, so I would say AI is  perfectly timed. I think there will be, you know, [26:06] I don't think it's going to take any jobs, so to  speak, because there's so many open jobs right [26:10] now. I think it'll probably actually supplement  the market. It's the way it's supposed to be. So, [26:15] well, I I um I agree um to your question regarding  the um what um what are the things that we need [26:23] to uh we need to look for and what what we have  learned. Um, one of the one of the biggest things [26:29] is the getting getting people involved during  the implementation of AI. It's super critical. [26:34] Don't do AI to them, do it for them. Um, and I  think that's going to be that's the real key. Um, [26:41] if you if you don't involve the team, so what  when when somebody comes to me and says, hey, [26:46] I have this AI project, my first question to them  is, what are you trying to solve? Let's define [26:51] the problem first. If you and if I give you the  perfect data, what are you going to do with it? [26:55] How are you going to change the operational um  uh how are you going to change the operations or [27:01] how are you going to change your clinical practice  in order to to use that data? And if if you don't [27:06] know how, then let's figure go figure that out  first. And once we once we know that, then do we [27:11] know the team that's going to actually implement  those changes? Have you talked to them? If you [27:15] haven't, let's talk to them because then we're  going to have a lot of learnings from them. So, [27:18] let's not do AI to them, but do it for them. Uh  I I can go on but I think um I think you get the [27:24] point. Um that involve the teams that are actually  going to use AI in implementing it. Um and [27:31] um the uh the yeah I think I think that's I'll  leave that for now. Um I would say I I agree [27:39] with what everyone else said. Um, one thing I  would add, I think, um, as much as possible, [27:45] trying to foster a culture of innovation or at  least trying not to kill a culture of innovation [27:52] is important. Um, so one challenge is that there's  a substantial amount of shadow AI AI use. Um, [28:00] and something that we've been trying to do is  identify um tools that people seem to be using [28:08] uh and trying to uh use those like we're trying  to bring those into the enterprise or find a [28:13] reasonable substitute for them. um because we we  don't we've had a situation where I think people [28:20] are using a lot of these things and they don't  feel comfortable talking about it because you [28:24] know we've blocked a lot of tools off and uh  you know so they're they're like well I'm not [28:29] supposed to tell anybody that I'm using you know  if if I I I mean I have colleagues that I know are [28:35] using claude for workrelated things but they're  not putting PHI in it uh or you know business [28:40] sensitive information but they don't necessarily  want to air that because they don't want a million [28:45] questions around it. Um, so you know, I think  tools like that, if you have like a really capable [28:52] uh AI assistant like that that can really do a  lot of things and that people can discuss those [28:58] things, uh, that's handy. We're starting to get  more like clinically facing tools that are like [29:02] that. Um, and I I think that's an important piece  because you want people to be able to discuss with [29:08] their colleagues, yeah, I used I used AI for this.  Here's how. Um so yeah great thanks Morgan. Um [29:17] question for you um Caroline when you think about  the healthcare industries the the standard for AI [29:22] is a lot higher than many other sectors right  so how do you think about um addressing some of [29:30] the unintended risks of AI you know you mentioned  governance uh there's issue does the product work [29:36] um are the hallucinations so how are you thinking  about addressing those kind of issues well I think [29:43] everything we do is associated with some level of  risk and everything we do is not without fault. [29:49] humans falter and so I think that again if you  think about how we're actually benchmarking [29:54] what is the current state what is the performance  what is the safety metric that you have measured [30:00] are you even measuring that for your current  operations today and is this tool introducing [30:07] more risk less risk the same risk and that'll give  you a different perspective to rather than just [30:14] saying oh it needs to be 99% accurate to what end  I mean I think that you know depends on the data [30:20] that's available to you and perhaps you're in a  situation or a certain clinic setting where that [30:26] data may not be always available and so that  becomes a giant unknown and so not having that [30:31] information or it may motivate a clinic to say  we need to start capturing this new data source [30:37] that we'd never done before because we want to use  that tool and I see you nodding away so I'm going [30:41] to let you comment but I think that you know it  really depends on where's your starting place and [30:46] what is your where's your goal of where you ultim  ultimately want to go or which way do you want to [30:50] move that needle and how much uncertainty are you  willing to take and that's a business decision as [30:56] an organization and the organizational leadership  needs to be able to decide that that is the level [31:01] of risk that they're willing to take and it's like  any other decision. Yeah. So maybe as as the you [31:06] know you rest of panelists answer the question  u you made a good point Caroline which is what [31:12] is the acceptable rate of error and humans make  errors too. So why should we expect machines to [31:17] be at 100% accuracy while humans are at 94%. So  when you think about those trade-offs, how can [31:23] you kind of comment on that? I I think you know  in every organization I've ever been in, I always [31:28] make best friends with the IT guys. They're the  funnest guys. You know, they always make me laugh. [31:32] They're they're a good group of guys wherever I've  gone. And you know what I've always tried to do [31:36] is is is probably probably part of how my brain  works. I just enjoy that part of of the job. Um, [31:42] and as CEO, you kind of have to be a little bit  of an expert in every single area. I think with [31:47] regard to it and making sure that it, you know,  is governed the right way, secured the right way, [31:51] the level of risk that you're willing to take. I  think the important thing is that it all happens [31:55] in your ecosystem, especially from an HIPPA  perspective. Um, you have to make sure, you know, [32:00] to the chat GBT or Claude or, you know, Grock  or whoever, you know, it really doesn't matter. [32:06] The biggest thing is you have to make sure it  happens in your own ecos ecosystem and that [32:10] it's HIPPA compliant. You know, C-pilot for us  is walled off with our Office 365 products. So, [32:16] it's it's it's our ecosystem. It's it's it's HIPPA  compliant. So, if you make mistakes, you know, [32:22] at least it happens in your own ecosystem. I  think for the broader issue of mistakes, you know, [32:28] how do you quantify that? You know, back to the  doctor who gives one line in his chart. Well, [32:32] I would argue that's a lot bigger of an issue  than, you know, having a fully built out chart [32:35] that maybe has a misspelling of a word. So, it's  just, you know, what's your level of risk? You [32:41] know, I think generally most people have learned  with Indian AI it significantly probably outweighs [32:46] the risk that versus what you had before. No,  I I I agree with both of you. I think um so [32:53] let me let me lay out the problem medicine for  you. We probably make about 12 million medical [32:59] errors a year. that that that's leads to about  a quarter of a million deaths due to medical [33:04] errors per year. Let that sink in a little bit.  The number is controversial but this is this is [33:09] well published in uh in British Medical Journal  and back in 2016 from Johns Hopkins. Uh before [33:15] that Institute of Medicine published up to 98,000  people die of medical errors. So the numbers [33:20] aren't small. Medical errors do happen. Anything  we do has to be better than that. And and that's [33:26] the that's the key thing here. uh where where  if AI does a better job to reduce some of those [33:31] errors then then you then you got to put that in  uh it's it's uh um perhaps it is that simple. I [33:41] guess I'll just add one thing though when we say  with if AI does better I think it goes back to are [33:47] you measuring that performance in the context  in which you're planning to use the tool. Many [33:52] people quote whatever has been submitted to the  FDA that was tested on a very specific data set [33:58] that's been curated for evaluating that model to  be for submission. That may not be the performance [34:04] of that model in your setting. And so that that  is an an added factor and yeah and that's that's [34:10] why the way you implement it, you know, local  validation, local fine-tuning and so on really [34:15] has to happen before you implement it, not just  just take it out of the box and implement it. [34:22] Uh yeah, this is a really this is a really  complicated question. So um I've I've spent [34:27] a lot of time uh working in our system on our  uh on our governance uh system. Uh and it's [34:36] really challenging. There are a lot of challenges  there. There's there are challenges in capturing [34:40] everything that's coming through. Um there's  there are challenges in evaluating everything [34:45] um and assembling the right experts for that. But  there are also challenges around just tracking [34:50] the documents involved and understanding like  well we said the risk was this much and we put a [34:55] mitigation plan in place that's this that plan is  filed here. Um and then that is the throughput for [35:03] that is going to have to increase because we're  you know I mean we've been in we've been in kind [35:07] of the dotcom era where everything is whatever.ai  AI and we're rapidly moving out of that where like [35:13] all of your legacy vendors are going to start  introducing uh you know AI features just because [35:18] it's practical and it solves the problem uh but  that they don't necessarily announce themselves [35:22] so you've got to do a better job of capturing that  uh but also that's a lot more to track. Um so it's [35:29] it's hard to do that at scale and there's another  piece of this specifically that's hard which is [35:34] figuring out what those risks are. Yeah. Again,  I mean, sometimes you get numbers from the vendor [35:40] uh that can that are meaningful around like what  are the risks of this system and sometimes less so [35:47] and they certainly don't necessarily apply to your  own uh to your own population. Um that is it's [35:54] really it's really hard. I don't know any systems  that are doing that are uh that are measuring that [36:00] are evaluating and monitoring models at scale uh  without throwing a ton of people at the problem. [36:05] Um, and I I don't know if they would want me to  name them, but I I will say that I've talked to [36:09] one system in particular where they took pretty  much their whole team that was doing internal [36:14] build and repurposed it for evaluating models  that were coming for evaluating systems that were [36:20] coming from vendors. Um, I think that there are  areas I a lot of what we do now we put a lot of we [36:27] lean pretty heavily on human in the loop because  we have a lot of solutions that have been rolled [36:31] out that are facing clinical experts who have gone  through years of training. they're acknowledged [36:35] experts in what they do. They're also employees  and you can lean on them and say like listen it's [36:40] on you. You got to check you know to make sure  that this is correct. But a lot of what we want to [36:44] do is patient facing uh and you you can't like you  can't make the same assumptions about them. I we [36:51] assume that they have a fifth grade reading level.  U you know we tailor you know what we the reading [36:56] material that we send them around that assumption.  You you're definitely not assuming that they are [37:01] experts in this. And it's also a weird mixed  message if you're like if they're asking you [37:05] questions and you know or they're asking this you  know like if you have a chatbot and they're asking [37:10] it questions and it's answering them and you're  saying hey trust us but also please check all [37:14] of these answers for yourselves. It's a very mixed  message. Um so I I think and I think if we haven't [37:20] had like a lot of high or I I can't think of  any like really highprofile cases where you know [37:25] you've had something disastrous happen with that.  um partly because I think those systems aren't at [37:30] that level of maturity yet where you're uh where  they're handling things. But I think it's going [37:33] to I think it's going to limit what we can do and  the and the vision that we can realize if we can't [37:39] implement evaluations of monitoring at scale.  If you don't know the different failure modes [37:43] for something that you're deploying that's patient  facing and you can't say yeah I can tell you with [37:48] confidence that if a patient ever like alludes  to any kind of suicidal ideation it's definitely [37:54] going to get escalated. uh we you know we test  this we have numbers on it. Um you know you don't [38:00] know if you don't know if that's happening that's  a that's really bad if that if that were to happen [38:05] with you know some patientf facing agent and it  wasn't escalated because a human would otherwise [38:10] have been the ones calling themself. So one  last question for all of you. So for any of the [38:17] healthcare executives in this audience, there's  one piece of, you know, advice you would give [38:22] them before they spend another dollar on AI, what  would it be? I can I can start. Um, here's what I [38:28] would suggest. Um, before you do anything, I think  identifying the problem is a common theme up here, [38:34] but I would say even more important than or  equally as at a minimum, make sure you're [38:42] reading your contracts. Make sure that you're  looking at who owns this data, where is it going, [38:48] who has access to it, who has the ability to  sell it, who has the ability to trade it. Uh, [38:54] real life example, no vendor names, no insurance  names, nothing. But, you know, we we did sort of [38:59] a a recovery process where we went through that  exercise to to look into that. And what we found [39:05] was one of our major vendors um that literally u  runs a lot of our hospital was providing data to [39:14] a third-party repository. So circumventing the  word sell by providing it to that repository. [39:20] Uh so by definition wasn't selling it. But then  when you look at the main subscribers to that repo [39:26] repository, it's all the major insurance companies  that want that data. Um that claim that that [39:31] uh contains how much each reimbursement is  for each payer. And that's very valuable [39:36] data and that's a very easy way to ratchet down  reimbursement particularly for hospitals when [39:41] you want to you know push out some of your lower  performing quality players. And so, you know, our [39:47] data was only being used against us to ultimately  achieve what we all know is is usually the case, [39:53] which is ratchet down of reimbursements and other  things. So, I would just say make sure you're [39:58] reading your contracts, have your attorney look at  it, and make sure that you have data provisions in [40:02] accordingly to protect you as a safeguard  from that. I would say educate, educate, [40:08] educate. Um, and and uh that's that's the first  step. The second is uh is make sure you involve [40:15] your team. And the reason I say educate is um one  of the one of the we were when we were actually [40:20] making one of the models I met with the with our  clinical team five times just to make sure that [40:27] u that we are solving the right problem. I thought  there was a problem is X but after you talking to [40:33] them the problem is X plus Y right so that you're  solving the right problem. uh so that involving [40:39] your team is important but but it I also use that  time to just let you know educate them that AI is [40:46] not voodoo uh it's uh it there is uh there is  method behind it there's there's mathematics [40:52] behind it and that was actually pretty pretty  important part as well because uh we as clinicians [40:58] if we don't understand it it's very difficult for  us to actually use it or actually accept it so [41:05] um uh so helping to understand how the tools are  made and we have to understand the tools we use [41:10] and how they how they are um um how they're made  and how we use it and so so education on top of [41:18] involving them in the process are the two things I  would say yeah uh I would say you probably should [41:27] go slower than you think or than you feel like  you need to go um you know there there are so [41:34] many there's so much there's such a push now to  be AI I forward do things with AI. Um, I think [41:40] you really need to think about what value you're  going to get out of a system because usually the [41:45] deployments are going to be challenging or at  least harder than you might have thought. Um, [41:50] and you need to consider so you need to consider  like what value you're going to get and how [41:54] you're going to measure that and depending on the  application you might need to consider, you know, [41:59] what risks are involved as well. So be, you know,  skeptical about that. Um, but yeah, I mean I would [42:04] say if you don't if you don't have if you don't  have like a board that's pressuring you to be AI [42:11] first, if it's not that pressure is not coming  to you from above and even if it is, maybe give [42:14] a little push back. I would say don't don't lean  into that identity of being AI first. Like just [42:20] try to do the right things by your patients. Try  and try and deliver things that have actual value. [42:26] Carolyn, I would say really focus on your overall  strategy and prioritize. As you've heard, any any [42:34] of these problems we're trying to solve becomes  a system of systems challenge when you're trying [42:40] to deploy AI. So, you're going to have to bring  the people along. And the people aren't just your [42:44] clinicians. It's going to be the IT teams. It's  going to be anyone who's touching the data that's [42:49] feeding in. Um, so it's and coordinating all of  that, integrating that into your systems, bringing [42:56] your compliance team along and making sure that  they're reviewing. And it's not just on day one [43:00] to turn it on, but it's going to be continued  monitoring. And so these are massive investments [43:06] in terms of not just money, but it's massive  investments in terms of how your operational work [43:12] is going to shift. And so learn to really think  about prioritization. define what your end goal of [43:19] what would success look like and how much risk are  you willing to take and then making sure that you [43:24] do actually have the resources to be successful.  You can't do this necessarily on the cheap and so [43:30] again it looks like AI can be used everywhere and  potentially there's a ubiquitous nature to it but [43:36] to use it everywhere on day one makes absolutely  no sense. And so I think that each organization [43:42] is going to have their own set of priorities  but really take the time to figure out what [43:46] the priorities are and then be very clear with  your organization of what direction and then you [43:51] need to bring everyone along with the education  evaluation metrics etc. So I want to thank all [43:57] of you as the panel for a great conversation. I'd  also like to leave you all with a thought here. [44:03] So you know the traditional industries we've had  for the past 100 plus years have it a few factors [44:09] of production labor capital information with AI  you have a fourth factor of production called [44:17] intelligence and it requires us to rethink  a lot of things in the enterprise priorities [44:23] workflow governance so just running into it and  blindly cutting and pasting is not a good idea