Esfahani Replaces Static Healthcare Dashboards With Live AI Tools

Realtime Operational Instrumentation in Healthcare: AI Analytics in Action — Ai4

Ai4

Traditional retrospective analytics fail because they evaluate static data after decisions occur, requiring embedded real-time AI instruments to drive immediate operational actions.

AI-assisted summary of Realtime Operational Instrumentation in Healthcare: AI Analytics in Action — original source. Check the original for full context.

Source published:

Detailed notes

## Background This talk, titled "Realtime Operational Instrumentation in Healthcare: AI Analytics in Action," was presented by Akbar Esfahani, Enterprise AI & Systems Architect at Central California Alliance for Health, and published on YouTube by Ai4 on August 19, 2026. The description highlights how a Medi-Cal health plan serving 420,000 members replaced passive dashboards with real-time operational instruments embedded directly into daily workflows to drastically cut processing times and enforce compliance.

## TL;DR Akbar Esfahani explains how traditional retrospective healthcare analytics functions like analyzing a "dead butterfly," failing to impact live clinical and operational decisions. To solve this, his team built over 70 real-time operational instruments, including a unified dispute resolution tool that reduced median close times from 57 to 24 days and slashed backlogs. Built in weeks using AI coding agents, these tools emphasize human-in-the-loop decision-making and have returned over 90,000 service hours to members.

## Atmosphere & style The presentation carries an urgent, pragmatic, and forward-thinking energy, driven by Esfahani's vivid metaphors and direct operational insights. Rather than focusing on theoretical AI concepts, the talk feels like a hands-on operating manual grounded in real-world healthcare challenges and rapid problem-solving. The atmosphere is clear, authoritative, and empowering, showcasing how small teams can achieve massive leverage under strict regulatory constraints.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=4s) Traditional enterprise analytics relies on retrospective data and static dashboards, analyzing past events rather than impacting decisions in real time. - [02:00](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=120s) The health plan adopted a design principle of replacing passive dashboards with active operational instruments embedded directly into daily workflows. - [03:13](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=193s) AI vendor tools allowed providers to automate dispute filings, creating a overwhelming backlog of 2,700 cases for a 12-person dispute team. - [05:31](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=331s) By unifying ten software systems into a single operational interface with embedded predictive models, the team reduced median dispute close times from 57 days to 24 days. - [08:31](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=511s) Real-time HL7 data feeds from the local Health Information Exchange were integrated into dashboards to allow clinicians to track errors and query patient status live. - [09:54](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=594s) The provider dispute instrument was created in three weeks by two PhD data scientists and cleared half of the backlogged cases by the fifth week. - [10:48](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=648s) Leveraging generative AI coding agents reduced code creation costs to near zero, allowing the deployment of over 70 operational instruments with turnaround times under an hour per tool. - [12:05](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=725s) The organization measures impact through member-centric metrics, returning over 90,000 total service hours (equivalent to 25 minutes per member and 88 FTEs saved). - [14:26](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=866s) Change management relies on co-ownership, where developers use AI agents to build working prototypes within an hour alongside clinicians instead of drafting long requirements documents. - [16:28](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=988s) Compliance, auditing, and immutable event logging are engineered directly into every instrument's architecture upfront in partnership with compliance leadership. - [18:03](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=1083s) Embedded machine learning models do not make autonomous decisions; they provide risk probabilities to prioritize workflows, keeping human judgment at the center of final determinations.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=4s) Rethinking Enterprise Analytics through the "Dead Butterfly" Metaphor

Akbar Esfahani introduces himself as the leader of a small data science and AI team at a local health plan in Central California. Serving a agricultural, rural region with 420,000 members and a lean workforce of 700 employees, the organization faces immense pressure to maximize operational efficiency while serving vulnerable, underserved, and homeless populations. To illustrate the core flaw in modern enterprise analytics, Esfahani uses a vivid biological metaphor: if a field biologist records observations of a live butterfly in a notebook, the notebook acts as an active instrument capturing real-time behavior. However, if the biologist kills the butterfly to measure its wingspan in a lab, they can no longer analyze its flight.

In healthcare, traditional enterprise analytics acts like that dead specimen. Organizations routinely rely on retrospective data, pulling static Tableau dashboards or Excel spreadsheets to review past events. By the time machine learning models analyze this historical data, the opportunity to impact patient outcomes in the moment has passed. To bridge this gap, Esfahani’s team advocates moving away from static reporting destinations toward dynamic, real-time operational instrumentation embedded directly into daily workflows.

- Central California Health Plan parameters: 420,000 members served by 700 employees facing upcoming organizational downsizing. - Flaw of traditional analytics: Heavy reliance on static Tableau dashboards and Excel files analyzing retrospective, "dead" data. - Core design principle: Replacing passive reporting destinations with active operational instruments that capture and analyze live operational data as work occurs.

---

[03:13](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=193s) Operationalizing Provider Disputes and Real-Time HL7 Processing

To demonstrate operational instrumentation in action, Esfahani highlights provider dispute resolution. When medical claims are denied, providers resubmit disputes, creating double work for both the plan and the provider while delaying clinical care payments. Recently, AI vendor tools enabled providers to automatically file disputes on every denied claim, overwhelming the plan's 12-person dispute team with 2,700 backlog cases. Instead of granting the department's request to hire 30 additional staff members, Esfahani’s team developed a real-time operational instrument.

Instead of navigating ten separate software systems and clicking through dozens of screens, workers perform all tasks inside a single unified web interface. Embedded machine learning models analyze incoming cases in real time, calculating the probability of a claim being overturned so staff can prioritize high-risk, time-sensitive cases. Applying this approach to HL7 data feeds from the local Health Information Exchange (HIE) allows clinicians to query patient status in real time and track data errors directly on live dashboards.

``` [ Incoming Claims / Disputes ] │ ▼ ┌─────────────────────────────────────────┐ │ Operational Instrument Interface │ │ - Single unified web workspace │ │ - Embedded ML overturn probabilities │ │ - Live error tracking & HIE integration│ └─────────────────────────────────────────┘ │ ▼ [ Real-Time Priority Action by Staff ] ```

- Dispute team efficiency baseline: 12 staff members facing 2,700 backlog cases driven by automated provider AI dispute tools. - Turnaround time drop: Reduced median close time from 57 days down to 24 days (and 23 days by the morning of the talk), well under the 30-day regulatory mandate. - Backlog reduction: Six months post-deployment, pending pass-through cases dropped from 2,700 to just 3 daily rolling cases. - Engagement & Speed: Achieved a 96–97% daily staff engagement rate with a 450x increase in processing workflow speed.

---

[09:54](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=594s) Rapid AI Deployment, Zero-Cost Code, and Impact Metrics

The provider dispute instrument was engineered by two PhD data scientists in just three weeks, deployed to production in the fourth week, and cleared half the department's backlog by the fifth week. By deploying reusable application development patterns via cloud-based coding agents, Esfahani’s team reduced production cycles from weeks to days, and ultimately down to less than an hour per instrument. Because generative AI has reduced the marginal cost of code virtually to zero, the team has deployed over 70 operational instruments into live production.

To ensure these tools deliver genuine organizational value, the team shifted away from traditional IT output metrics—such as number of dashboards created—in favor of a human-centric KPI: hours of service returned to each health plan member.

- Development velocity: Two PhD data scientists built the dispute instrument in 3 weeks; total production time per tool now takes under an hour. - Operational scale: 70 operational instruments currently operating live across staff workflows. - Member service impact: Returned 25 minutes of clinical/service time per member (equivalent to over 90,000 total service hours), projected to double by year-end. - Team leverage ratio: The AI/data science team achieves a 21x return ratio (21 hours of operational time returned for every 1 hour worked). - Capacity equivalent: The efficiency gains represent roughly 88 full-time equivalents (FTEs) saved, cushioning the health plan against administrative cutbacks required by regulatory shifts like HR1.

---

[14:26](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=866s) Q&A: Co-Ownership, Coding Agents, and Embedded AI Governance

During the audience Q&A session, Esfahani fields questions regarding change management, development processes, regulatory oversight, and model decision-making. When asked how clinicians are trained on new tools, Esfahani explains their philosophy of "co-ownership." Rather than gathering business requirements documents (BRDs) and building in isolation, developers feed a user's initial wish list into coding agents to generate a working prototype within an hour. Clinicians sit directly across from developers to refine the live application, eliminating traditional post-delivery training delays.

Regarding AI governance and regulatory compliance, Esfahani notes that as chair of the health plan's AI governance board, he works closely with the Chief Compliance Officer. Compliance, auditing, and immutable logging are baked into every instrument's architecture upfront rather than retrofitted later. When asked about FDA approval, Esfahani clarifies that as a health plan rather than a drug manufacturer, FDA oversight does not apply. Crucially, the machine learning models embedded in these instruments never make autonomous final determinations; they provide risk probabilities to assist human reviewers, leaving final decision-making entirely in human hands.

- "Every workflow that you have not instrumented is a butterfly you're studying that is already dead." - Development paradigm: Replaces traditional BRD requirements gathering by converting user wish lists into code via coding agents in under an hour. - Governance architecture: Implements immutable, auditable logging of every system click and data transaction upfront in close partnership with compliance leadership. - Human-in-the-loop control: Machine learning models only supply predictive probabilities to guide prioritization; final clinical and administrative decisions remain strictly human-driven.

## Notable quotes > [14:26](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=866s) Every workflow that you have not instrumented is a butterfly you're studying that is already dead. — Akbar Esfahani

## Mentioned Akbar Esfahani, Central California Alliance for Health, Ai4, Medi-Cal, Tableau, Excel, HL7, Health Information Exchange, FDA

## Video description Akbar Esfahani - Enterprise AI & Systems Architect, Central California Alliance for Health

What if the problem with healthcare analytics isn't the models, but the fact that someone has to go somewhere to look at them? Most organizations deliver intelligence as dashboards, reports you visit after the work is done, while the decisions that matter are made inside workflows the dashboard never touches. I will show how my team at a Medi-Cal health plan serving 420,000 members replaced this pattern with operational instruments: AI-driven systems embedded at the moment of work, from provider dispute resolution that cut median close time from 57 days to 24, to compliance audits where regulatory rules are enforced by the architecture itself. Attendees will learn the design principles that separate an instrument from a dashboard, how to firewall risk models from the decisions they inform, and how to build this inside a regulated environment without waiting for an AI strategy to be approved. If your organization is still debating AI while the work piles up, this talk is the operating manual for starting anyway.

[Source](https://www.youtube.com/watch?v=OBWTqzKVWFU)

中文简报

Esfahani以实时AI工具替代静态医疗仪表盘 传统回溯性分析因在决策做出后才评估静态数据而失效,需要嵌入式实时AI工具来驱动即时的运营行动。 Akbar Esfahani解释称,传统的回溯性医疗分析就如同分析一只“死蝴蝶”,无法对实时的临床和运营决策产生影响。为解决这一问题,他的团队构建了70多个实时业务工具,包括一款将中位结案时间从57天缩短至24天并大幅减少积压的统一争议解决工具。这些工具利用AI编码智能体在数周内开发完成,强调人机协同(human-in-the-loop)决策,并已为会员返还了超过90,000小时的服务时间。 针对传统医疗分析依赖静态仪表板与事后数据的弊端,团队提出“业务运行仪表化”(Operational Instrumentation)理念,将动态 AI 工具直接嵌入日常工作流以实时指导决策。 (0:04) 面对服务商利用 AI 自动提报导致的 2700 起争议案件积压,团队将 10 个软件系统整合进内嵌预测模型的统一界面,把案件解决中位时间从 57 天压缩至 24 天。 (3:13) 借助生成式 AI 编程智能体,单个运行工具的开发周期被压缩至 1 小时以内,团队已在生产环境中部署超过 70 个实时业务工具,使代码生成边际成本降至近零。 (9:54) 团队将核心考核指标转向人本价值,累计为会员节约超 9 万服务小时(相当于释放 88 个全职人力),并坚持合规内嵌与人机协同,确保模型仅提供风险概率且最终决策权始终在人。 (14:26)

Reading summary

Akbar Esfahani of a Central California health plan argues that healthcare organizations must replace retrospective, static dashboards with real-time operational instruments embedded directly into staff workflows. By coupling machine learning prioritization with rapid agent-based software development, the plan demonstrates significant efficiency gains while keeping human operators in control. The Dead Butterfly Flaw in Healthcare Analytics (0:04) Traditional healthcare business intelligence relies heavily on retrospective reports, such as static Tableau dashboards and Excel spreadsheets. Esfahani compares this approach to a biologist killing a butterfly to measure its wingspan in a laboratory: once the specimen is dead, its flight cannot be studied. Similarly, by the time machine learning models process historical enterprise data, the window to actively influence patient outcomes or immediate operations has passed. To counter this limitation, Esfahani's team advocates for dynamic operational instrumentation. Serving a rural Central California region of 420,000 members with a lean workforce of 700 employees facing downsizing, the organization required tools that observe and support active workflows. Rather than sending staff to passive reporting destinations, operational instruments capture, analyze, and optimize data in real time as daily tasks occur. Resolving Provider Disputes via Embedded ML (3:13) The operational instrument model was applied to the health plan's provider dispute process. After external healthcare providers began using automated AI vendor tools to contest every denied medical claim, the plan's 12-person dispute team was overwhelmed by a backlog of 2,700 cases. Instead of hiring 30 additional staff members, the data science team built a single unified web interface that consolidated ten separate systems and embedded machine learning models to calculate the probability of a claim overturn. The real-time interface allowed staff to prioritize high-risk, time-sensitive cases and integrated live HL7 feeds from the local Health Information Exchange to track errors. Six months post-deployment, the pending backlog fell from 2,700 cases to just three daily rolling cases. The median close time dropped from 57 days to 23–24 days—safely beneath the 30-day regulatory requirement—while staff engagement reached 96–97% alongside a reported 450-fold increase in processing speed. Rapid Delivery and Member-Centric Metrics (9:54) The initial dispute instrument was built by two PhD data scientists in three weeks, deployed in the fourth week, and cleared half the backlog by week five. Utilizing cloud-based coding agents and reusable application architecture patterns, the team subsequently reduced development cycles from weeks to under an hour per instrument, deploying over 70 instruments into live production. Rather than tracking software volume through traditional IT outputs like dashboard counts, the team evaluates success by hours of service returned to members. Esfahani reports that the deployments have returned 25 minutes of service time per member—totaling over 90,000 service hours, with expectations to double by year-end. The data science team operates at a calculated 21x leverage ratio, returning 21 operational hours for every hour worked and generating capacity equivalent to roughly 88 full-time employees, which buffers the plan against regulatory cutbacks. Collaborative Development and Governed Human Oversight (14:26) The health plan bypasses lengthy business requirements documents (BRDs) in favor of a co-ownership model. Developers translate user wish lists into functional prototypes via coding agents within an hour, sitting directly with clinicians to iterate on live tools and eliminating post-delivery training friction. To maintain regulatory compliance and trust, auditing and immutable logging of all clicks and transactions are integrated directly into each tool alongside the Chief Compliance Officer. Esfahani clarifies that FDA regulations do not apply because the health plan does not manufacture drugs. Furthermore, embedded predictive models do not make autonomous decisions; they provide risk scores to guide prioritization, leaving all final administrative and clinical determinations to human staff. The health plan's operational instrumentation model demonstrates that internal teams can clear severe administrative backlogs by pairing human-in-the-loop predictive AI with rapid application scaffolding. However, maintaining this velocity relies on robust compliance integration and user co-ownership to prevent automated workflows from obscuring human accountability.

精炼阅读摘要

加州中部某地方健康计划的人工智能团队负责人 Akbar Esfahani 认为,医疗机构应摒弃依赖静态报表分析历史数据的“死蝴蝶”模式,转而开发直接嵌入日常工作流程的实时运营工具,以在业务发生的当下提供辅助决策并大幅提升运营能效。 “死蝴蝶”隐喻与实时运营工具范式 (0:04) 该加州中部健康计划覆盖42万名会员,仅有700名员工,主要服务农村、农业区以及无家可归等弱势群体,且面临即将到来的组织缩减压力。Esfahani 指出,传统企业分析习惯依赖 Tableau 仪表板或 Excel 表格审查事后数据,这犹如生物学家为了测量翼展而在实验室杀死蝴蝶,导致其无法再观测蝴蝶的实时飞行;当机器学习模型对历史数据完成分析时,改善患者预后的时机早已流逝。 为此,该团队主张从被动的事后报表转向嵌入日常工作流的动态实时运营工具。其核心设计原则是消除孤立的汇报终点,改为在工作人员处理业务的同时实时捕获与分析活态数据,使数据系统成为指导即时操作的活跃仪器。 服务商申诉实战:化解案件积压危机 (3:13) 在医疗服务商申诉业务中,外部服务商近期利用 AI 供应商工具对所有被拒索赔发起自动申诉,导致该健康计划仅12人的申诉团队瞬间积压了2700起案件。业务部门原本申请增聘30名员工,但数据团队通过构建实时运营工具化解了危机:工作人员无需在10个独立软件和数十个界面间频繁跳转,所有操作均被整合至单一网页端工作台。 该工具内置机器学习模型,可实时计算索赔被推翻的概率,引导员工优先处理高风险与紧急案件;同时接入本地健康信息交换所(HIE)的 HL7 实时数据流,方便临床人员即时查询患者状态并在活动仪表板上追踪数据错误。工具上线6个月后,处理流程速度提升了450倍,中位结案时间从57天降至24天(演讲当天达23天),远低于30天的法规要求,积压案件从2700起骤降至每日仅3起滚动案件,员工日常系统使用率达到96%至97%。 极低代码成本驱动敏捷部署与价值衡量 (9:54) 上述申诉工具最初由两位数据科学博士耗时3周构建,第4周正式投产,第5周即清理了一半的积压案件。Esfahani 表示,团队通过云端编码智能体复用应用架构模式,已将单个工具的交付周期从数周缩减至数天乃至1小时以内;随着生成式 AI 将代码的边际成本降至接近于零,团队已在实际业务中上线了70多个实时运营工具。 该机构还将衡量 IT 价值的 KPI 从单纯的仪表板数量转变为以人为中心的指标——为每位会员争取回的服务时间。据 Esfahani 统计,这些工具已为每位会员平均返还25分钟的服务时间(累计超9万小时),并预计在年底前翻倍;AI 与数据科学团队由此实现了21倍的人效回报比(每投入1小时研发换取21小时运营工时),相当于节省了约88名全职员工的产能,有效缓解了 HR1 等法规变动带来的行政裁员冲击。 共同所有权开发机制与人机协同治理 (14:26) 在系统落地与人员培训方面,Esfahani 提出了“共同所有权”理念。开发团队不再耗费周期收集传统的业务需求说明书(BRD),而是将业务人员的心愿单输入编码智能体,在1小时内生成可用原型,随后由临床人员与开发人员面对面实时调整测试,从而规避了交付后的二次培训周期。 在合规监管方面,兼任机构 AI 治理委员会主席的 Esfahani 与首席合规官密切配合,将不可篡改的审计日志与合规审查预先嵌入系统架构,完整记录每次点击与数据交互。针对监管质疑,Esfahani 澄清健康计划不属于药品制造商,因此无需遵循 FDA 审批;更关键的是,嵌入工具的模型仅提供预测概率以协助分级,所有最终行政与临床裁定均严格由人工完成。 Esfahani 展示的实时运营工具体系证明了敏捷 AI 原型开发与嵌入式分析在医疗行政降本增效中的潜力。该模式在严控人员规模的背景下显著释放了人力,但其运作前提是将模型限定在辅助分流角色、保持人工最终决策权,并依赖严格的前置合规审计。

Detailed notes — zh

## Background 本内容由频道 Ai4 于 2026 年 8 月 19 日在 YouTube 平台发布,由加州中部健康联盟(Central California Alliance for Health)的企业 AI 与系统架构师 Akbar Esfahani 主讲。页面介绍指出,该演讲针对医疗数据分析脱离一线工作流的行业通病,展示了一个服务 42 万 Medi-Cal 会员的健康计划如何用嵌入式“运行仪器”(Operational Instruments)取代传统事后仪表盘,将服务商争议结案时间从 57 天压缩至 24 天,并阐述了在强监管环境下快速落地 AI 与合规架构的设计原则。 ## TL;DR 面对外部 AI 批量引发的医疗争议激增与人员缩减压力,团队放弃了滞后的静态报表,转而构建了直接内嵌于业务一线的“运行仪器”,实现了多系统融合与实时 ML 预测。这套方案将供应商争议处理中位时长从 57 天断崖式缩减至 24 天,历史积压清零,并成功推广复制出 70 多个生产级工具。团队重构了以“归还给会员的服务时间”为核心的北极星 KPI,累计释放了 90,000 个高价值工时。同时,通过 Coding Agent 极速原型开发、临床人员共创模式以及严格的人机决策防火墙,团队实现了零培训成本与前置内生的高标准 AI 治理。 ## Atmosphere & style 演讲整体呈现出一种务实、犀利且充满革新锐气的工程现场感。Akbar 摒弃了空洞的 AI 宏大叙事,通过“活体蝴蝶”与“死标本”的生动隐喻直击传统 BI 报表的要害,论述极具穿透力与批判性。在展示令人惊叹的数十倍提效数据时,他的叙述冷静自信且紧贴一线业务现实;问答环节与观众的互动幽默松弛,不仅在回应 FDA 监管质疑时引发会心笑声,更在讨论合规与人机防火墙时展现出深思熟虑的严密逻辑与极强的启发性。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=4s) 传统数据分析往往是在事后研究“死标本”——当数据呈现在 Excel 或 Tableau 仪表盘时流程早已结束,无法在当下做出实时干预。 - [03:13](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=193s) 提出“运行仪器(Operational Instruments)”新范式,将 AI 模型与预测指标直接嵌入原生工作流与单一操作界面中,实现流内即时决策与闭环。 - [03:13](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=193s) 面对外部 AI 批量发起的争议激增海啸,通过运行仪器接入实时 HL7 数据流与争议翻转概率预测,使 12 人原班团队化解了扩招 30 人的需求。 - [08:06](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=486s) 上线运行仪器后,争议案件结案周期中位数从 57 天违规延误压缩至 24 天,逾期积压案卷从 2,700 件锐减至仅剩 3 件,员工采纳率达 96%。 - [10:50](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=650s) 重构团队北极星 KPI 为“归还给每位会员的服务时间”,彻底废弃单纯考核模型数量、看板数量或算法准确率的传统指标。 - [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) 团队累计构建 70 个运行仪器,为 42 万名会员返还 9 万小时服务工时(相当于 88 个全职员工产能),研发效能杠杆比高达 1:21。 - [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) 践行“分析不是目的地,而是工作流内部的交付系统”的哲学,避免让一线员工在处理业务时离开工作界面专程去外部平台找数据。 - [15:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=904s) 彻底废除繁琐的业务需求文档(BRD),借助 Coding Agent 在 1 小时内将临床医生的“愿望清单”转化为功能原型,通过共创机制消除后续培训成本。 - [16:54](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=1014s) 将 AI 治理与可审计性从第一天起内置于系统架构中,每一次前端点击与决策行为均实时写入不可篡改的审计表中。 - [16:54](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=1014s) 建立严格的人机决策防火墙:机器学习模型仅提供概率与模式参考,所有最终裁决与行政责任完全由一线审核人员独立承担。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=4s) 医疗健康计划的现实困境与“活体蝴蝶”隐喻 Akbar 领导着加利福尼亚州中部一家地方健康保险计划(Health Plan)的小型数据科学与 AI 团队。该机构覆盖 42 万名会员(主要为 Medi-Cal 参保群体),员工总数约 700 人,且面临团队缩减的压力。服务区域主要是以农业和农田为主的偏远乡村,涵盖了大量无家可归者以及医疗资源严重匮乏甚至未被覆盖的边缘脆弱群体。在资源紧缩和需求迫切的双重压力下,团队的核心使命是通过 AI 分析彻底重构一线员工的工作流,极致提升运营效率。 为了阐明传统数据分析的根本缺陷,Akbar 提出了一个生动的生物学比喻:假设你是一位在野外研究蝴蝶的生物学家,拿着笔记本记录蝴蝶在花丛中飞舞的鲜活状态,这个笔记本就是你的测量仪器;但若为了深入研究而必须把蝴蝶杀死并带回实验室制成标本,此时你只能测量它静止的翼展,却再也无法测量它活生生飞翔时的真实姿态。 ```mermaid flowchart LR accTitle: 传统事后分析与实时运行仪器的对比 subgraph Traditional["传统事后分析(死标本)"] direction TB A["历史理赔/出院数据"] --> B["Excel / Tableau 报表"] B --> C["滞后分析与离线决策"] end subgraph Instrumentation["运行仪器模式(活体飞翔)"] direction TB D["实时业务事件流 / HL7"] --> E["一体化工作界面 + 嵌入式 ML"] E --> F["即时干预与实时闭环"] end ``` 在医疗健康与保险行业中,绝大多数分析工作都在做“解剖死蝴蝶”的无用功:团队始终在回顾数周甚至数月前的历史数据,并在死数据上训练机器学习模型。当分析结果最终呈现在 Tableau 仪表盘或 Excel 报表里时,患者的诊疗流程早已结束,一线人员根本无法做出实时干预。Akbar 强调:“现实中大部分企业分析交付的都是死标本(dead specimen)”,必须跳出事后静态看板的模式,转向伴随业务实时发生的动态运行仪器。 - **服务人群规模**:覆盖加州中部农业区 420,000 名会员,机构员工约 700 人(面临人员缩减压力)。 - **传统分析死结**:Tableau 看板与 Excel 往往是业务发生后的“终点站”,无法驱动当下的干预与改进。 --- ### [03:13](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=193s) 供应商争议激增与运行仪器(Operational Instruments)的设计革新 医疗保险运营中的一大痛点是服务提供方争议(Provider Disputes)。当医疗机构提交的理赔(Claims)被拒付后,他们需要重新核对、整理并再次提起争议仲裁。传统的 12 人争议审核团队原本处于超负荷运转状态,甚至提出需要额外招募 30 名员工才能应对堆积如山的案卷。这种争议激增的推手恰恰是外部 AI 工具的普及:许多第三方 AI 公司为医疗机构提供自动化申诉服务,使得医院能够一键对所有被拒赔的项目发起批量争议。 面对迎面而来的“AI 生成式争议海啸”,团队没有选择盲目扩充人力,而是彻底颠覆了系统交互模式。团队不再为业务部门交付孤立的 AI 模型或查看报表的仪表盘,而是构建了直接承载业务操作的实时 Web 应用程序。Akbar 将这种新型工具正式命名为“运行仪器”(Operational Instruments)。在这些仪器中,工作人员所有的点击、判定与流转都在同一个界面内闭环,系统实时采集行为数据,并在工作流的原位(in situ)呈现预测模型输出与异常指标。 ```mermaid flowchart TD accTitle: 供应商争议运行仪器处理流 A["外部机构 AI 批量发起争议"] --> B["运行仪器集成工作台 (单屏闭环)"] subgraph Instrument["运行仪器核心能力"] B1["实时 HL7 / HIE 数据流接入"] B2["ML 翻转概率预测 (Overturn Risk)"] B3["界面内实时幻觉与错误捕获"] end B --> Instrument Instrument --> C["12人团队高效办结 (取代30人扩招需求)"] ``` 在技术实现上,这套体系直接接入了来自健康信息交换网(HIE)的实时 HL7 临床数据流,使临床协调人员能够在患者住院期间实时提问、核验诊疗数据并监控数据传输错误。由于整个系统采用 AI 辅助编码(Cloud Code / 智能生成代码),团队开发新仪器的周期从过去的数周压缩到了数小时。开发团队自身也被纳入了这套仪器的监控之中:系统能够实时捕捉模型在工作流中产生的“幻觉”与未报告异常,使开发者与业务人员在同一个动态反馈环路中协同工作。 --- ### [08:06](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=486s) 突破性业务战果:从 57 天违规延误到 24 天快速结案 在部署运行仪器之前,该健康计划的争议结案周期中位数长达 57 天,严重违反了加州监管部门规定的 30 天硬性合规要求,逾期未决案卷高达 2,700 件。上线运行仪器 6 个月后,整个争议处理流程发生了质的飞跃: - **结案时效压缩**:争议解决的平均中位周期从 57 天断崖式下降至 24 天(演讲当天早晨最新数据已进一步降至 23 天),全面进入监管合规线以内。 - **积压案卷清零**:历史逾期积压从 2,700 件锐减至仅剩 3 件。系统中实际已不存在长期滞留的案卷,仅存的少量跨日个案均能在次日完成审核。 - **员工参与度与采纳率**:一线审核人员对系统的日常使用参与率(Engagement Rate)高达 96%。 - **极速投产与人效提升**:仅由 2 名拥有博士学位的数据科学家耗时 3 周完成开发,第 4 周正式上线投产,第 5 周便清理了半数历史积压。原班 12 人团队的工作流速度提升了约 97%(等效工作效能提升达 450 倍),完全化解了原本需要增聘 30 人的用工需求。 之所以能取得如此极端的效率飞跃,核心在于工作流摩擦力被彻底消除。过去,审核人员为了处理单笔争议需要在 10 个不同的遗留系统与界面之间反复跳转、反复比对;而运行仪器将所有患者背景信息、理赔上下文以及预测争议翻转概率(Overturn Probability)的机器学习模型直接收拢整合在“单一屏幕”(One Screen)内。工作人员无需在多系统间切换,便能优先处理高风险、高概率翻转的关键案卷。 --- ### [10:50](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=650s) 生产范式复制与以“归还服务时间”为核心的新 KPI 随着首个运行仪器的成功,团队提炼出了一套可高度复用的软件设计与架构模式。目前,该健康计划内部已有 70 个运行仪器在生产环境中全天候运行。随着代码生成成本趋近于零,开发交付周期经历了指数级压缩:从最初构建一个工具需要 3 周,逐步缩短到 2 周、1 周,如今熟练团队在数小时内即可根据一线业务需求快速装配并投产一个新的运行仪器。 伴随技术架构转变的,是团队绩效考核(KPI)的彻底重构。Akbar 团队废弃了诸如“构建了多少个算法模型”、“上线了多少个应用系统”或“模型运行速度有多快”等与最终业务价值脱节的传统技术指标,转而确立了一个全新的北极星指标——**归还给每位会员的服务时间(Hours of Service Returned to Each Member)**。 | 评估维度 | 传统数据团队 KPI | 运行仪器模式新 KPI | |:--- |:--- |:--- | | **核心考核导向** | 交付物数量(模型数、看板数、应用数) | 临床与运营业务一线被解放的时间价值 | | **价值衡量标准** | 算法准确率、代码运行速度、报表访问量 | 归还给每位患者/会员的直接服务时长 | | **团队心理感受** | 制造在后台静默运行的“冷冰冰程序” | 深刻感知技术对脆弱群体健康带来的真实改变 | 这个新 KPI 的落脚点在于:AI 自动化为一线临床和运营人员节省下来的每一分钟,都能转化为他们直接服务患者、支持诊所的真实精力。截至演讲时,该团队通过 70 多个运行仪器,已累计为覆盖的每位会员平均节省并返还了 25 分钟的服务时间。当乘以 420,000 名会员的总量时,这意味着全组织已累计释放了约 90,000 个小时的高价值临床与运营服务工时。数据科学家不再沉迷于后台构建冰冷的代码,而是真正感受到了数据技术对社会底层脆弱人群生命体验的切实赋能。 ### [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) 从报表工场到业务核心:节省9万小时与“蝴蝶”隐喻 团队在衡量自身工作价值的方式上发生了根本性转变。截至 8 月 5 日,团队构建的操作型工具已累计为组织节省并返还了 90,000 个工时;预计到年底,这一数字将翻倍。更惊人的是团队的人效比达到了约 21 比 1(近 20 倍效能),即团队每投入 1 小时研发,就能为一线业务换回 21 小时的高效服务时间。演讲者直言,传统数据团队整天纠结的指标根本无足轻重——“谁真的在乎你产出了多少个仪表板(Dashboard)、制作报表花了多长时间这类数据?”这种视角的转变,让数据科学团队从边缘的辅助汇报部门,直接转变为企业不可或缺的核心运转力量。节省下的时间相当于为公司挽回了约 88 个全职员工(FTE)的劳动产能。在当前医疗政策变动(如 HR1 导致的成员资格审查收缩与参保人数减少)的大背景下,预算缩减往往伴随着裁员压力,但业务量不仅不会减少反而持续攀升,唯一可行的破局之道就是通过内置工具让现有员工效率成倍提升。 演讲者在演讲尾声抛出了一句发人深省的核心金句:“任何未经插桩度量与赋能的工作流(workflow),都像是一只你正在研究却早已死去的蝴蝶。”传统的报表往往只是在事故或延误发生后进行的“验尸报告”,对当下毫无补救作用。他最后向全场发问,留给所有人深思:“你们的员工究竟在哪些环节不得不离开正在处理的工作流,大费周章地跑去别处寻找分析数据?”分析与智能绝对不应该是一个需要专程前往的目的地(destination),而应当是直接嵌入一线工作流之内的交付系统(delivery system)。 - **返还工时规模**:截至 8 月 5 日已返还 90,000 小时,年底预计翻倍至 180,000 小时。 - **人效投资回报比**:团队研发投入 1 小时,业务侧收回 21 小时工时(约 20x+ 杠杆效应)。 - **挽救人力产能**:折合约 88 个全职员工(FTE)的工作量。 - **核心哲学**:"Analytics is not a destination, but it should be a delivery system inside their workflow."(分析不是目的地,而应是工作流内部的交付系统)。 ```mermaid flowchart LR accTitle: 传统事后分析与嵌入式操作工具对比 subgraph Traditional["传统分析模式(事后目的地)"] direction TB A["业务在系统A中处理工作"] --> B["遇到决策断点/停滞"] B --> C["脱离工作流前往BI平台"] C --> D["查看滞后仪表板/数据死蝴蝶"] end subgraph Modern["操作型插桩模式(流内交付)"] direction TB E["业务在原生工作流操作"] --> F["实时插桩与AI决策辅助"] F --> G["流内即时完成闭环与合规记录"] end ``` --- ### [15:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=904s) 问答环节:消灭需求文档,让临床专家共同所有与 Coding Agent 极速构建 在问答环节,现场观众首先盛赞了演讲的精彩,并提出一个棘手的落地问题:搭建这些操作工具的技术门槛或许已被攻克,但如果要改变临床医生(clinicians)的工作流,对他们进行使用培训通常需要耗费多少时间?演讲者回答,他们彻底摒弃了传统软件工程的交付路线,引入了“共同所有权(Co-ownership)”机制。临床医生从工具搭建的第一天起就深度介入其中,团队与医生面对面坐在一起,工具在边开发的同时就直接放在所谓的准“生产环境”中供其试用。医生在体验过程中持续提出反馈,确保最终呈现在屏幕上的逻辑完全契合临床真实意愿。通过这种方式,培训在开发阶段就已同步完成,根本不需要等交付完成后再组织耗时费力且容易遭遇抵触的培训。 更具颠覆性的是,团队彻底取消了业务需求文档(BRD)和漫长的需求收集流程。整个开发模式变成了极速的原型循环:团队直接向临床医生询问“你们的愿望清单(Wish List)是什么”,随后将这些愿望输入内部的 Coding Agent(编码智能体)。该编码代理能够调用并学习组织内部积累的所有现有代码库、设计模式与架构规范,在短短 1 小时之内自动生成功能原型。随后工程师带着热气腾腾的原型再次坐到临床医生面前:“这是根据你的愿望清单做出来的,现在请帮我们微调和定义它,把它打磨成你真正想用的工具,而不是某种开发出来就被弃之不顾的鸡肋玩具。” - **去 BRD 化敏捷流程**:跳过繁琐的业务需求文档编写,直接以临床业务方的“愿望清单”作为输入。 - **Coding Agent 赋能**:代码代理复用内部现有的成熟架构模式,1 小时内即可生成工作原型。 - **实时共创(Co-ownership)**:业务方在构建阶段即参与测试与定义,实现“交付即熟练”,消除事后培训周期。 --- ### [16:54](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=1014s) AI 治理前置设计与机器学习的“人机决策防火墙” 针对现场关于热门话题“AI 治理(Governance)”的提问,演讲者披露了自己的另一重身份:他本人同时兼任公司内部 AI 治理委员会的主席。他强调,合规与治理绝不能在系统开发完成后作为补丁“事后追加”,而必须从第一天起就内嵌在每个操作工具的底层架构中。他们构建的每套工具都具备绝对的可审计性,前端的每一次点击、每一个决策行为与上下文数据,都会被完整捕获并实时写入不可篡改/追加写入的数据表(Immutable/Append-only Table)中。此外,团队与首席合规官(Chief Compliance Officer, CCO)保持紧密结盟。当团队开发的 AI 工具提交至治理委员会进行审查时,团队成员会主动回避表决,由合规与业务委员会成员对其进行最严苛的盘问与质询,确保完全满足合规风控的高标准。 在被问及系统中的机器学习模块如何通过美国 FDA(食品药品监督管理局)审批时,现场引发了一阵会心的笑声。演讲者幽默地澄清:“我们不是制药公司(Pharma),我们不需要跟 FDA 打交道!”工具只需通过内部严格的合规与法务审核即可。更关键的实质在于系统的决策边界划分:系统中的机器学习模型绝不擅越职权代替人类做出最终裁决。模型的作用仅限于提供概率支撑(例如给出“该争议案件有极高置信度应予维持原判”的分析参考),但所有案件的最终核准与责任判定,完全且不可分割地留给一线复核人员。机器学习模型永远不会替人类生成终局结论,从而在算法建议与法律责任之间筑牢了清晰的防火墙。 - **合规前置内生**:治理要求在架构初期实现代码级落地,全流程操作日志实时存入不可篡改审计表。 - **独立治理机制**:自身作为 AI 治理主席在自己团队的项目评审中主动回避投票,交由委员会与 CCO 严格审议。 - **非临床处方定位**:属于医疗保险健康计划(Health Plan)运营体系,非制药与医疗器械,不受 FDA 监管管辖。 - **人机决策防火墙(Human-in-the-loop)**:ML 模型仅输出置信度评分与模式识别参考,任何最终结论与行政裁决均由经办人独立签署并承担责任。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=4s) 现实中大部分企业分析交付的都是死标本(dead specimen) — Akbar Esfahani > [10:50](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=650s) Hours of Service Returned to Each Member — Akbar Esfahani > [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) 谁真的在乎你产出了多少个仪表板(Dashboard)、制作报表花了多长时间这类数据? — Akbar Esfahani > [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) 任何未经插桩度量与赋能的工作流(workflow),都像是一只你正在研究却早已死去的蝴蝶。 — Akbar Esfahani > [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) 你们的员工究竟在哪些环节不得不离开正在处理的工作流,大费周章地跑去别处寻找分析数据? — Akbar Esfahani > [12:55](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=775s) Analytics is not a destination, but it should be a delivery system inside their workflow. — Akbar Esfahani > [15:04](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=904s) 这是根据你的愿望清单做出来的,现在请帮我们微调和定义它,把它打磨成你真正想用的工具,而不是某种开发出来就被弃之不顾的鸡肋玩具。 — Akbar Esfahani > [16:54](https://www.youtube.com/watch?v=OBWTqzKVWFU&t=1014s) 我们不是制药公司(Pharma),我们不需要跟 FDA 打交道! — Akbar Esfahani ## Mentioned Akbar Esfahani, Central California Alliance for Health, Medi-Cal, Ai4, Tableau, Excel, HL7, Health Information Exchange (HIE), Cloud Code, Coding Agent, FDA (美国食品药品监督管理局), HR1 ## Video description Akbar Esfahani - Enterprise AI & Systems Architect, Central California Alliance for Health What if the problem with healthcare analytics isn't the models, but the fact that someone has to go somewhere to look at them? Most organizations deliver intelligence as dashboards, reports you visit after the work is done, while the decisions that matter are made inside workflows the dashboard never touches. I will show how my team at a Medi-Cal health plan serving 420,000 members replaced this pattern with operational instruments: AI-driven systems embedded at the moment of work, from provider dispute resolution that cut median close time from 57 days to 24, to compliance audits where regulatory rules are enforced by the architecture itself. Attendees will learn the design principles that separate an instrument from a dashboard, how to firewall risk models from the decisions they inform, and how to build this inside a regulated environment without waiting for an AI strategy to be approved. If your organization is still debating AI while the work piles up, this talk is the operating manual for starting anyway. [Source](https://www.youtube.com/watch?v=OBWTqzKVWFU)

Source quotes

[0:04] Akbar Esfahani: 现实中大部分企业分析交付的都是死标本(dead specimen) [10:50] Akbar Esfahani: Hours of Service Returned to Each Member [12:55] Akbar Esfahani: 谁真的在乎你产出了多少个仪表板(Dashboard)、制作报表花了多长时间这类数据? [12:55] Akbar Esfahani: 任何未经插桩度量与赋能的工作流(workflow),都像是一只你正在研究却早已死去的蝴蝶。 [12:55] Akbar Esfahani: 你们的员工究竟在哪些环节不得不离开正在处理的工作流,大费周章地跑去别处寻找分析数据? [12:55] Akbar Esfahani: Analytics is not a destination, but it should be a delivery system inside their workflow. [15:04] Akbar Esfahani: 这是根据你的愿望清单做出来的,现在请帮我们微调和定义它,把它打磨成你真正想用的工具,而不是某种开发出来就被弃之不顾的鸡肋玩具。 [16:54] Akbar Esfahani: 我们不是制药公司(Pharma),我们不需要跟 FDA 打交道!

Source quotes

[14:26] Akbar Esfahani: Every workflow that you have not instrumented is a butterfly you're studying that is already dead.

Original transcript

[0:04] Thank you so much. Um, so hey everyone, my name  is Akbar. I run a small data science and AI shop [0:11] for a local health plan in California. So this  is central California. We have rural areas, [0:18] mainly farmland. And so we have some of the most  hardest hit populations, a lot of homeless people [0:26] and underserved or not served at all, you know,  people that live in our area, service area. And so [0:33] my team's job is to develop the next level of AI  analytics and just transform how people work. Um, [0:43] and so, you know, we have 420,000 members. um  and we have 700 employees. Uh that's going to [0:52] be shrinking pretty soon. And so efficiency is one  of the big impacts that we have to work towards. [0:58] And so today I'm going to be talking to you about  the new way that we have been implementing AI and [1:04] data science and analytics in workflows. So  um I'm going to start the talk by talking to [1:12] you about butterflies. And so imagine you're  um studying butterflies. You're going to be a [1:17] b biologist and you're studying butterflies in  the field and you're taking notes. You have a [1:23] notebook and that's your instrument. And so you're  recording in that notebook what is happening with [1:29] that butterfly. Um and then you want to study the  butterfly further. So you have to kill it and take [1:35] it with you to your lab to study it further.  But at that moment when the butterfly is dead [1:42] then you cannot really be measuring when it was  alive. And so you don't know what is happening. [1:47] And so what we find today in healthcare a lot  is that you're always retroactively looking at [1:53] data for the patient for the members as we call  them in the health insurance industry. And so you [1:58] cannot really make impact on their life if you're  always retro retrospectively looking at it and [2:03] then trying to build machine learning models or  AI systems that act on that retrospective data. [2:10] So what my team is doing you know is trying to  avoid this specimen that arrives in the lab dead. [2:17] And so you know you can measure the wingspan of  a butterfly but you cannot measure it when it was [2:24] flying. And so those are two different aspects of  analytics and you know measuring something that is [2:31] in your lab versus something that was alive and in  the moment. And so the reality is that enterprise [2:40] analytics today is that dead specimen, right?  You're going to a destination. Most of the time [2:47] it's a Tableau dashboard. A lot of the time it's  an Excel file of something that has happened in [2:52] the past but you have no real you cannot take real  action on that dead data except maybe you know for [2:59] optimizing your work um optimizing some type of  operational work you know trying to work with [3:05] that data and so um I'm going to walk you through  a couple of examples that we have been working on [3:13] um you know we have my team has built a AI well  it's not really AI model. It's a operational [3:20] instrumentation for uh provider disputes. So a  lot of the time providers file claims with us. [3:28] Sometimes it gets denied and then the provider  has to go back and re Oh sorry. Um a lot of the [3:36] time the provider has to go back look at their  claims and then submit them again. And so now [3:42] the work has the work is twice as hard. uh we have  to look at it, they have to look at it and so the [3:50] provider disputes you know the provider is waiting  so that means that the patient is waiting at the [3:55] same time because he you know he wants to get  paid for the work that he has he has done and so [4:00] my team's responsibility was you know building an  instrumentation that allows this work to be much [4:06] more efficient. So this disputes um has a team of  about 12 people and when it came this project came [4:15] to us they wanted to hire about 30 more people  to handle the disputes overwhelming disputes that [4:22] are coming in and that is happening the disputes  are happening because of AI actually because now [4:28] providers are able to just dispute anything that  you know all the claims that they're getting AI [4:34] companies are providing them a service and they're  within their rights to use that service to file [4:39] claims against those uh file disputes against the  claims that they have been denied. So they will [4:45] just make claims against all their you know all  their disputes and so what we had to do was design [4:53] an instrument um that you know went further than  what they were doing before. So to give you an [5:02] idea an instrument you know um we have a dashboard  we build web applications um in real time and what [5:12] we do with this web web applications is that we  are changing how the staff works so we are not [5:18] providing them analytics we don't provide them AI  we provide them an interface where they do their [5:24] work inside of it all the data gets captured  the team sees what they are doing themselves [5:30] They see the machine learning models that  are implemented within that dashboard or web [5:36] application that it's deployed on. We don't call  them anymore applications or dashboards. We call [5:41] them operational instruments because that's  where the work is happening. It's capturing [5:45] their real- time data and is providing real-time  analysis to everybody else so that they can take [5:51] real-time actions on it. Um, so we have also been  developing this tool. Um I think I clicked f past [6:00] something that no okay so we have been building  these tools for a while now and we have actually a [6:09] develop developmental platform that we have built  for it and so one of them is what I'm showing on [6:17] screen right now um is for HL7 data um and so  we get the HL7 data from his hie and we have to [6:26] process those in real time because members are  patients are in the hospital, we are receiving [6:31] their data and so we want to take actions in real  time on that data and so we built this instrument [6:39] um that allows us to look at the data in real  time. It allows us it allows the clinicians to ask [6:45] questions of that data in real time. it measures  the errors of that data and it displays it live in [6:53] the in their dashboard so that they can see what  they're working on, who they're working with. Um, [6:58] and so that is part of the work that we're doing.  Um and then the fi the funny part is when we are [7:08] developing these we can measure our own error  rates and so we can find you know we have [7:13] instrumented in a way that the developers also get  to see what's wrong with the with the development [7:19] that they're using. So we use cloud code um to  build these applications and so we can develop [7:26] these within matter of hours now because we have  been doing them for so long and we have about 70 [7:31] of them in production in real time usage by the  staff and so we can see you know when there was an [7:39] error. So as an example, we were testing this um  use case last week and we found that you know it [7:45] made bunch of errors that went unreported. um as  um the previous guest speaker was talking about [7:52] you know hallucinations and AI hallucinating  we can capture that in the in the flow of the [7:58] work and so that's there and then going back to  that instrumentation for the provider disputes [8:06] um you know when we started the work had been  running late um on an average of 57 days and so [8:14] there's a regulatory requirement of 30 days and  so We actually got it down to 24 days and it's [8:21] decreasing as we speak. Um this morning it was at  23. Uh we have 90 96% engagement rate. Um and the [8:35] past two cases were 2700. That's why they needed  30 extra people to work on. And so now it's at [8:42] three. This is six months after the fact that we  put this tool into production and there's really [8:48] no actual pass through cases. It's just that every  day the new cases come in and as they're working [8:54] on it, some of them can be passed through, but  those get closed out the next day. So there's [8:59] really almost no pass through cases and the staff  is always engaged in this tool. And the reason for [9:06] that is that this is a new um new way of working,  right? because they get to see the analytics. So [9:14] they get to see what is happening with that  patient. They get to see all the information [9:18] they that they need in real time. Before they had  to go to 10 different tools, click through screens [9:26] to get to the to, you know, to close one of these  disputes. Now it's just one screen. It's all right [9:32] there. um we provided them a machine learning  model that tells them what the probability is [9:38] of this case being either overturned or not. And  so based on that they can speed up their workflow [9:46] and close the cases that they that they think  it's uh higher risk faster than they were before. [9:54] So you might ask, you know, how many people did it  take? uh two of my staff, two PhD data scientists [10:01] worked on this for three weeks and we had this  in production in the fourth week and we closed [10:07] half their cases in the fifth week. The rest were  you know cases that just took time to to adopt to [10:14] the tool for them and so they have been in this  tool for six months and as I mentioned you know [10:19] the speed is picking up they are about 97% uh  faster that's about 450x times what they were [10:27] doing before instead of hiring 30 new people  12 people are doing the same work so you know [10:34] the disputes didn't decrease but the workflow and  the speed that they're working at has increased. [10:40] just because they don't have to  click through anything anymore. [10:43] All the all the information they  need is right at their fingertips. [10:50] Um, so like I mentioned, we don't call them apps  anymore. We call them operational instruments. [10:55] It's semantics, but you know, there's a difference  between being a data scientist and being a IT [11:01] person. And so that's where we're trying to  get at. Um, and the pattern repeats. So once [11:08] we build one tool, we can use the same pattern  for the next tool. And so like I mentioned, [11:13] we have 70 of these and it's increasing the  rate of production for us. So before we took [11:20] us three weeks to build one, then it took  us uh two weeks, then it took us a week, [11:25] and now we can have if you really want to  produce one of these every hour. And so the [11:32] the speed of to production has gone to zero just  because cost of code now is basically zero due. [11:41] Um, one of the other aspects of this is we  changed the KPI how we measured the impact [11:47] that we are making on the business and so before  you know it's how many models did you produce, [11:54] how many applications did you produce, how many  you know how fast were you, what is it doing for [11:59] the staff we changed it to hours of service return  to each member or each patient and so now the KPI [12:06] is how much more time are we providing back to  clinical staff. that they can support the clinic, [12:13] the the members and the patients to be better  off. And so that's our new KPI which has led to, [12:21] you know, my team having feeling the impact that  they're making on the person that they're trying [12:26] to help instead of just having a cold application  that's running in the background. And to date, you [12:33] know, we have returned two 25 minutes of service  time to each member. That might not sound a lot, [12:39] but when you multiply the 25 minutes by 420,000  people, that's a lot of hours. And so that's [12:46] around, you know, 90,000. But in reality, that's  just until right now. So, you know, on August 4th, [12:55] um, August 5th, we have returned n 90,000 hours.  By the end of the year, that's going to be [13:01] doubling. And the team is 21, you know, it's about  20x um more efficient. So my team is returning for [13:12] every hour that they work, they return 21 hours  back to service. So that's the measures that we [13:19] are you know tracking instead of what was it done  before where you know who cares really about the [13:26] metric of how many dashboards did you produce,  how long did it take you um and all these other [13:32] aspects that go with you know analytics and data  science. So we changed the the perspective of the [13:38] business itself and we are now core part of the  business because we are helping them reduce the [13:44] workflow and part of this 25 minutes really if you  want to translate it it can come out up come out [13:51] to about 88 FTEES that we have saved the company.  And if you then put that on into perspective with [13:59] HR1 which is going to be reducing our membership  and with our redu reduct reduction in membership [14:08] that means that reduction in employment and so  we going we are going to have to be laying off [14:12] people but the work doesn't decrease the work  is actually increasing and so how can we make [14:17] the people more efficient to be doing their work  and so that's what my team is trying to achieve. [14:26] And then so basically you know the tagline is  every workflow that you have not instrumented is [14:32] a butterfly you're studying that is already  dead and I will leave you with you know [14:40] uh where are you where are your staff leaving  their work to find analytics right because [14:46] analytics is not a destination but it should  be a delivery system inside their workflow. [14:55] Um, I know I flew past everything, so if you have  any questions, feel free, but that was my talk. [15:04] Maybe one question. By the way, fantastic  presentation. Thank you. I appreciate that. [15:07] I know you can't see me. S. Sorry. I was just  curious like you know building the instruments. [15:13] I think that's something that you have really  mastered and you can like produce operational [15:17] dashboards quite quickly. I'm curious like as  it pertains to training the clinic clinicians [15:22] to use and change the workflows how much time  does that take to like you know implement and [15:28] get them so we introduced the idea of co-ownership  that means that the clinician actually is working [15:35] with us during the building of the tool so they  are responsible for responding to us with their [15:42] ideas as we are developing this we put this in  sort of like quote unquote production and so [15:47] they are respons responsible party to make sure  that the work that they are seeing is what they [15:52] actually want and so they get trained right away.  there is no more you know waiting until after the [15:59] fact that you have delivered something and then  for them to train on it because once you do that [16:03] then you're you know you're moving past um what  they actually wanted right and that's what an [16:10] very important aspect of this is there is no BRD  there is no requirement gathering we just say [16:16] what is your wish list we feed that into a coding  agent and then we produce you know based on every [16:23] everything else that we have developed internally  that's available to the coding agent as well. And [16:29] so it looks at the code at the patterns and then  we can repeat that and produce something in an [16:35] hour and then we sit across from the clinician  and say this is what you wanted. This was your [16:41] wish list. Now help us define it and make it  better and tune it to what you really want it [16:47] to be instead of just some random tool that you  might not use again. Thank you. Um, governance [16:54] is a big topic here at this show. Yes. How do you  handle governance with the work that you're doing? [16:59] I am the chair of AI governance at our company.  Uh, and so um, every instrument that we build [17:08] has compliance built into it from the start. So  we don't retroactively put it into governance. [17:15] We do it from up front. Everything that we do is  auditable. It's actually capturing every single [17:21] information every single click and it's putting  that into a indemnable table and so everything is [17:29] there for governance and I am closely partnered  with the chief compliance officer to make sure [17:36] that this happens and every tool that we produce  we put it through AI governance where you know we [17:42] abstain from voting but they get to see what it  is and they get to ask the hard questions and we [17:49] have to deliver what they what they really need.  Just one last one. You mentioned that there's a [17:55] machine learning part in it. And how did you get  across the FDA in that part? So we don't have to [18:02] deal with the FDA. We are not a drug company. So  [laughter] I think you know that that would be [18:07] the response. Uh we got it across u our compliance  team you know making sure that it's passing their [18:13] checks. But since we don't deal with the FDA, we  don't have to get it past the FDA. So, you know, [18:20] um I don't I don't know if that answers your  question, but it's just basically we are not [18:25] farmers, so we don't have to deal with FDA. No.  So, there's no conclusion. It's just saying that [18:31] there's we are very confident that this that  this will be this case will be upheld but [18:37] there nobody is the the decision to be made and  captured is left to the person that is reviewing [18:44] the case. Nobody the machine learning model  will never make the um conclusion for them.