GE HealthCare Deploys Handheld AI Ultrasound Under Strict Human Oversight
Ai4
Legal restrictions on non-human medical practice force healthcare tech developers to maintain human-in-the-loop safeguards to limit liability and accelerate product adoption.
## Background
This video was published on YouTube by Ai4 on August 22, 2026. The page description outlines an expert discussion between Ashley Casovan of IAPP and Lara Liss of GE HealthCare on how leading healthcare organizations implement AI governance to ensure systems remain safe, well managed, and capable of accelerating AI adoption.
## TL;DR
Ashley Casovan and Lara Liss explore the regulatory and legal landscape surrounding AI in healthcare, focusing on key themes like the authorized practice of medicine and the essential human-in-the-loop safeguard. Using concrete examples like GE HealthCare's Vscan Air handheld ultrasound, they illustrate how software as a medical device must balance technical innovation with strict standards for safety, privacy, and reliability. The speakers also highlight governance as an essential organizational infrastructure requiring cross-functional collaboration, multidisciplinary buy-in, and adaptation to agentic AI. Ultimately, they emphasize that robust AI governance serves as a critical business driver that accelerates product adoption and enhances enterprise value during acquisition or IPO due diligence.
## Atmosphere & style
The discussion carries a polished, professional, and highly collaborative tone, blending executive expertise with practical industry insight. Casovan and Liss maintain an engaging, conversational dynamic, seamlessly interweaving light personal anecdotes—such as humorous reflections on parenting and consumer technology—with serious policy and technical discussions. The overall feel is thoughtful and forward-looking, offering an authoritative yet accessible look inside high-level healthcare decision-making.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) As technology companies enter healthcare and healthcare providers adopt advanced tech, emerging lawsuits around interactive chatbots are bringing intense regulatory scrutiny to AI-delivered medical advice.
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) Under current U.S. law, only humans can be licensed medical practitioners, making human-in-the-loop safeguards an essential baseline for AI governance in healthcare.
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) Healthcare organizations build legal defenses by anchoring AI risk management to established standards like the EU AI Act, NIST AI RMF, ISO 4201, and OWASP.
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) GE HealthCare operates across 160 countries with over 130 AI-enabled medical devices on the market.
- [05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Portable AI tools like GE HealthCare's pocket-sized Vscan Air ultrasound demonstrate high-impact applications in rapid point-of-care triage and early disease detection.
- [05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Software as a Medical Device (SaMD) requires managing potential risks across key metrics: safety, validity, reliability, fairness, privacy, security, transparency, and explainability.
- [05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Medical device manufacturers face strict clearance through regulatory bodies like the FDA and MDR, contrasting with unregulated direct-to-consumer tech tools.
- [09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) Blending AI governance with IT infrastructure opens up significant funding, addressing a massive industry imbalance where AI development spending far outweighs governance spending.
- [09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) Effective AI governance requires mandatory buy-in from key technical leaders, such as Chief AI Officers and lead data scientists, rather than operating in legal or compliance silos.
- [09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) Building cross-functional governance teams—including technical engineers, policy experts, and bioethicists—is crucial for embedding ethical oversight throughout the product lifecycle.
- [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) The rise of agentic AI requires updating traditional organizational tools like RACI matrices to define responsibilities and oversight for non-human AI actors.
- [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) Organizations are leveraging AI agents directly within governance workflows to increase the speed and scale of internal compliance processes.
- [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) Robust AI governance acts as a key driver of market value and IPO/acquisition readiness, proving critical during buyer due diligence on training data and risk mitigations.
- [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) Rather than acting as a bottleneck, well-implemented AI governance serves as a business accelerator that helps healthcare organizations achieve goals faster and build trust.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) AI Governance, Lawsuits, and the Regulatory Landscape in Healthcare
Ashley Casovan and Lara Liss open the discussion by examining current trends in AI governance, particularly the practical application of governance frameworks in healthcare. They note a shifting ecosystem where technology firms are pushing into healthcare offerings while traditional healthcare companies are integrating advanced tech capabilities. This convergence has brought new legal and regulatory scrutiny, particularly regarding AI-powered medical advice delivered through interactive chatbots.
Liss highlights a fundamental legal principle currently tested by these lawsuits: the authorized practice of medicine. Under current U.S. law, only humans can be licensed medical practitioners. While legislative proposals at the state level have attempted to grant AI systems limited authority to practice medicine, none have passed to date. Consequently, keeping a human in the loop remains an essential policy baseline and governance safeguard, framing AI as an operational tool to assist clinicians rather than replace them.
```
+-------------------------------------------------------------------+
| AI Risk Management |
+---------------------------------+---------------------------------+
|
+------------------------+------------------------+
| |
v v
+------------------+ +------------------+
| Safety Standards | | Legal Standards |
| (ISO 4201, NIST) | | (EU AI Act, FDA)|
+--------+---------+ +--------+---------+
| |
+------------------------+------------------------+
|
v
+---------------------------+
| Human-in-the-Loop Safeguard|
+---------------------------+
```
To build resilient defenses against legal liabilities, organizations must establish structured risk management programs grounded in core technical and policy standards:
- **EU AI Act & NIST Standards:** Benchmarking operational risks against comprehensive regulatory and technical frameworks.
- **ISO/IEC 4201:** Certifying AI management systems for structured risk identification and mitigation.
- **OWASP Guidelines:** Incorporating cybersecurity standardizations alongside broader policy controls.
- **GE HealthCare Deployment:** Operating across 160 countries with over 130 AI-enabled medical devices currently in the market.
---
[05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Practical Applications and Product Safety: The Case of Portable AI Ultrasound
Transitioning to concrete product applications, the speakers discuss software as a medical device (SaMD) and how AI delivers clinical value today. AI tools provide significant opportunities for earlier disease detection and non-invasive treatments. To ground the theoretical aspects of governance, Liss shares the real-world example of GE HealthCare's Vscan Air—a pocket-sized, portable ultrasound system powered by AI.
The Vscan Air allows field technicians and emergency room clinicians to conduct point-of-care triage rapidly without requiring constant external power supplies. However, deploying such medical equipment directly into clinical environments demands adherence to core safety principles. Organizations must manage potential risks across key operational metrics: safety, validity, reliability, fairness, privacy, security, transparency, and explainability.
- **Vscan Air Utility:** Operates as a battery-powered handheld probe, providing immediate imaging capabilities in remote locations or crowded emergency rooms.
- **Target Audience:** Designed specifically for expert clinicians who can interpret scan results, rather than direct-to-consumer use.
- **Regulatory Oversight:** Subject to rigorous testing and safety standards set by agencies like the U.S. FDA and the European Medical Device Regulation (M### AI Lawsuits and the Practice of Medicine
Ashley Casovan opens the session by reflecting on broader themes from the AI governance track, transitioning to a discussion on practical applications within healthcare. She points to a surge in high-profile litigation—specifically tech companies being sued over medical advice delivered via conversational chatbots. This signals an ongoing convergence where tech companies act like healthcare providers and healthcare companies adopt tech infrastructures. In light of potential deregulation alongside heightened lawsuit risks, Casovan prompts Lara Liss to address where these regulatory and legal dynamics are heading.
Lara Liss introduces her role as Chief Privacy and Data Trust Officer at GE HealthCare, outlining the scale of their operation across 160 countries with over 130 AI-powered medical devices on the market. Reflecting on recent litigation, Liss identifies two core principles shaping the industry:
* **Authorized Practice of Medicine:** Under current U.S. law, only human beings are licensed to practice medicine. While state-level legislative proposals continuously advocate for granting AI licensed clinical authority in specific capacities, none have passed to date. Liss expects these bills to re-emerge annually.
* **Human-in-the-Loop Safeguards:** Legal and policy frameworks rely heavily on keeping humans in the loop, positioning AI purely as an enabling tool for clinicians to enhance efficiency and accelerate early diagnoses.
* **Foundational Governance Frameworks:** Defense against emerging lawsuits relies on well-documented risk management programs that align external regulatory standards with internal priorities:
* Mapping the EU AI Act to the NIST AI Risk Management Framework (AI RMF).
* Implementing ISO 4201 standards for AI risk management.
* Leveraging OWASP frameworks for technical risk controls.
* Prioritizing patient safety as the foundational "north star."
---
[05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Software as a Medical Device: Opportunities, Risks, and Standards
Casovan shifts the dialogue toward Software as a Medical Device (SaMD) and asks for concrete operational examples, including how GE HealthCare navigates associated risks and opportunities. Liss emphasizes that healthcare offers immediate, high-impact AI applications over the next 5 to 10 years, particularly in early disease detection and non-invasive treatment planning.
To ground these concepts, Liss highlights **Vscan Air**, a pocket-sized, wireless portable ultrasound system powered by AI.
Casovan shares a personal perspective as a mother of twins, joking that having such a portable device at home would have made her check on her babies constantly, leading her to ask if direct-to-consumer releases are on the horizon. Liss emphasizes that GE HealthCare keeps these tools exclusively in the hands of trained expert clinicians. This dynamic highlights a regulatory divide in healthcare tech:
* **Strict Healthcare Regulation:** Established medical device manufacturers undergo rigorous FDA clearance in the U.S. and Medical Device Regulation (MDR) compliance in Europe, adhering to strict safety, validation, and testing standards.
* **Direct-to-Consumer Tech Disruption:** Unregulated tech companies entering the space often bypass medical-grade governance frameworks, raising critical societal questions about patient safety and product transparency.
---
[09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) Governance as Infrastructure and Multidisciplinary Collaboration
Casovan introduces "governance as infrastructure" as a prominent conference theme, asking who currently sits at the governance table during product evaluations and who *should* be present. Liss initially expresses hesitation about replacing "governance" with "infrastructure," noting that governance fulfills a distinct oversight function. However, she acknowledges key strategic advantages to blending the concepts:
* **Tapping IT Budgets:** Integrating governance into IT infrastructure unlocks substantial funding from core technology budgets. Liss cites panelist Nina (from CTO.ai), who noted that current industry spending skews heavily toward development—estimating a **5,000 to 1** ratio of AI investment relative to governance spending.
* **Organizational Placement:** Addressing where governance should reside, Liss explains that 20 years ago as a privacy officer, she sought the single "correct" structural home for compliance. Today, she believes location matters less than two factors: having the right stakeholders at the table with genuine influence, and securing adequate budgetary resourcing.
* **Mandatory Technical Buy-in:** Governance programs must be co-led alongside Chief AI Officers, lead data scientists, and product heads rather than isolated in legal or compliance silos. Liss notes her strict rule when establishing AI governance programs: if key technical and product leaders cannot attend a governance meeting, the meeting is canceled and rescheduled.
Casovan adds that benchmarking data from the International Association of Privacy Professionals (IAPP) shows organizational maturity varies by company scale, international reach, and technical complexity. While there is no universal blueprint, top-performing organizations build active cross-functional partnerships between technical product engineers, policy experts, compliance leads, and bioethicists. Liss emphasizes that bioethicists, long essential in hospital settings for end-of-life care decisions, must be paired with operational leads so ethical oversight is seamlessly integrated across the entire product lifecycle.
[17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) Evolving AI Governance: Agentic AI, Updated RACI Models, and AI Training
Lara Liss emphasizes that to effectively conduct AI governance, practitioners must be willing to "roll up your sleeves and dig in." A foundational element of this work is understanding the underlying data flows, as robust data governance becomes critical—especially with the rise of agentic AI. As AI systems evolve, organizations need to update traditional organizational tools like RACI matrices (Responsible, Accountable, Consulted, Informed). An intriguing emerging idea is incorporating AI agents directly into RACI models: mapping out which parts of the workflow AI agents perform, establishing clear oversight mechanisms for them, and defining how human team members interact with these non-human actors.
Ashley Casovan builds on this point, sharing that a GE HealthCare team member in India highlighted using AI agents directly within their governance workflows to increase scale and speed. This prompts novel questions for organizations like the IAPP, which traditionally trains human professionals. Casovan lightheartedly notes that people are now asking whether and how training will need to be provided to AI agents—an active area of exploration that hasn't been solved yet.
- **Hands-on AI Governance:** Requires granular mapping and deep technical understanding of underlying data flows.
- **RACI Evolution:** AI agents are actively being integrated into RACI charts to clarify operational responsibility, executive oversight, and human-agent collaboration.
- **AI in Governance Operations:** Organizations are using AI agents to accelerate and scale their internal compliance and review processes.
- **Agent Training:** Emergence of questions around how training, standards, and compliance frameworks will apply directly to autonomous AI agents.
---
[19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) Governance as a Value Driver for Acquisition, IPO, and Market Acceleration
Addressing advice for startups and companies seeking vendor contracts, acquisition, or an IPO with large healthcare enterprises like GE HealthCare, Lara Liss stresses that governance is far from a "nice to have." Building strong AI governance directly increases the enterprise value of a product when going to market. Potential acquirers and buyers will perform rigorous due diligence on how models were trained, how data flows are structured, and how safety and compliance risks have been mitigated. Companies must work with experienced legal counsel and domain experts so they have a compelling, defensible story to tell during due diligence.
Ashley Casovan strongly agrees with this perspective, noting that even if a company is not preparing for sale, robust AI governance serves as a critical business accelerator rather than a bottleneck. Proper management and safety protocols enable organizations to achieve their strategic goals faster while building long-term trust.
- **Market Value & Due Diligence:** Strong AI governance increases product valuation and prevents deal friction during corporate acquisitions or IPOs.
- **Key Due Diligence Areas:** Buyers evaluate dataset origins, training methodology, risk mitigation strategies, and system safety controls.
- **Business Acceleration:** Governance acts as an enabler rather than an obstacle, helping companies execute on their strategic goals faster and more safely.
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) keeping a human in the loop remains an essential policy baseline and governance safeguard, framing AI as an operational tool to assist clinicians rather than replace them. — Lara Liss
> [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) Under current U.S. law, only human beings are licensed to practice medicine. — Lara Liss
> [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) Prioritizing patient safety as the foundational "north star."
> [05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Software as a Medical Device (SaMD) and how AI delivers clinical value today.
> [05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Strict Healthcare Regulation: Established medical device manufacturers undergo rigorous FDA clearance in the U.S. and Medical Device Regulation (MDR) compliance in Europe, adhering to strict safety, validation, and testing standards.
> [05:01](https://www.youtube.com/watch?v=9AweKgZhSQw&t=301s) Direct-to-Consumer Tech Disruption: Unregulated tech companies entering the space often bypass medical-grade governance frameworks, raising critical societal questions about patient safety and product transparency.
> [09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) estimating a 5,000 to 1 ratio of AI investment relative to governance spending. — Nina
> [09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) Mandatory Technical Buy-in: Governance programs must be co-led alongside Chief AI Officers, lead data scientists, and product heads rather than isolated in legal or compliance silos.
> [09:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=563s) if key technical and product leaders cannot attend a governance meeting, the meeting is canceled and rescheduled. — Lara Liss
> [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) roll up your sleeves and dig in. — Lara Liss
> [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) RACI Evolution: AI agents are actively being integrated into RACI charts to clarify operational responsibility, executive oversight, and human-agent collaboration.
> [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) Building strong AI governance directly increases the enterprise value of a product when going to market. — Lara Liss
> [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) robust AI governance serves as a critical business accelerator rather than a bottleneck. — Ashley Casovan
## Mentioned
Ashley Casovan, Lara Liss, Nina, IAPP, GE HealthCare, Vscan Air, Ai4, CTO.ai, FDA, European Medical Device Regulation, NIST, ISO/IEC 4201, OWASP
## Video description
Ashley Casovan - Managing Director, IAPP
Lara Liss - Chief Privacy and Data Trust Officer, GE HealthCare
The opportunities to improve healthcare with advance AI are immense. Leading healthcare companies have found that they are able to accelerate their adoption of AI by ensuring these systems are safe and well managed. Join this expert discussion to learn how AI governance is being implemented in GE HealthCare practices.
GE医疗在严格人工监督下部署手持AI超声
针对非人类行医的法律限制促使医疗科技开发者保留“人类在回路”安全机制,以限制法律责任并加速产品应用。
Ashley Casovan 与 Lara Liss 深入探讨了医疗人工智能领域的监管与法律环境,重点关注获得许可的医疗执业以及关键的“人类在回路”(human-in-the-loop)安全机制等核心议题。她们以 GE HealthCare 的 Vscan Air 手持超声等具体案例,阐明了作为医疗器械的软件必须在技术创新与安全、隐私及可靠性的严苛标准之间取得平衡。演讲者还指出,AI 治理是一项至关重要的组织基础设施,需要跨职能协作、多学科认可并适应智能体 AI。最终,她们强调完善的 AI 治理是一项关键的业务驱动力,既能加速产品落地采纳,又能在并购或 IPO 尽职调查中提升企业价值。
随着针对医疗对话式 Chatbot 的诉讼增加,AI 治理面临严格监管;依据现行美国法律,仅人类具备行医执照,这使得“人机协同”(Human-in-the-loop)成为医疗 AI 不可逾越的安全与合规底线。 (0:04)
GE HealthCare 在全球 160 个国家拥有超 130 款获批的 AI 医疗设备;其便携式无线超声系统 Vscan Air 展示了 AI 在即时诊疗与早期疾病筛查中的潜力,但此类医疗级产品必须接受严苛监管,专供专业临床医生使用。 (5:01)
将 AI 治理与 IT 基础设施结合有助于获取关键技术预算,以此应对当前行业在 AI 研发与治理之间高达 5000:1 的严重投入失衡,同时推动首席 AI 官与产品团队在全生命周期内协同履责。 (9:23)
面对 Agentic AI 的兴起,机构正探索将 AI Agent 纳入 RACI 权责模型,并将其直接用于扩展合规审查流程,同时也引发了对 Agent 本身如何进行合规训练的新课题。 (17:25)
完善的 AI 治理是推动业务落地的加速器,并在企业并购或 IPO 尽职调查中直接提升产品估值与可信度,已成为初创公司与大型医疗企业合作的核心竞争力。 (19:23)
Reading summary
Lara Liss of GE HealthCare and Ashley Casovan of the IAPP assert that structured AI governance serves as both an indispensable clinical safeguard and a core commercial accelerator. They argue that navigating high-stakes healthcare deployment requires maintaining strict human-in-the-loop baselines, securing multidisciplinary technical buy-in, and embedding compliance into core data architectures.
Legal Boundaries and Human-Centered Baselines (0:04)
Recent high-profile litigation over medical advice provided by conversational chatbots underscores an enduring legal constraint: under current U.S. law, only humans can hold medical practitioner licenses. While state legislative proposals periodically attempt to grant AI systems limited clinical authority, none have passed. Consequently, retaining human clinicians in the loop remains an essential policy baseline and legal defense, establishing that AI functions strictly as an assistive tool to accelerate diagnoses and improve operational efficiency rather than replace medical professionals.
To build structured defenses against liability, healthcare organizations must benchmark operational risk management against formal standards. Frameworks such as the EU AI Act, the NIST AI Risk Management Framework, ISO/IEC 4201 for AI management systems, and OWASP technical controls provide foundational guidance. For enterprise healthcare providers like GE HealthCare—operating across 160 countries with over 130 AI-enabled devices—anchoring governance around patient safety provides the principal standard for meeting international regulatory demands.
Clinical Standards Versus Consumer Tech Entry (5:01)
Within Software as a Medical Device (SaMD), AI presents immediate opportunities in earlier disease detection and non-invasive treatments. The deployment of GE HealthCare’s Vscan Air, a battery-powered, pocket-sized ultrasound system, illustrates point-of-care clinical utility for emergency triage and remote environments. However, deploying clinical AI requires rigorous risk controls across key dimensions: safety, validity, reliability, fairness, privacy, security, and explainability.
A significant regulatory divide separates established medical device manufacturers from direct-to-consumer technology entrants. While enterprise devices undergo rigorous clearance processes through the U.S. FDA and European Medical Device Regulation (MDR), Liss emphasizes that tools like Vscan Air are restricted to trained clinical experts. Conversely, unregulated technology companies entering healthcare often bypass medical-grade validation, raising profound societal concerns regarding product transparency and patient safety.
Bridging the Governance Resource Deficit (9:23)
Integrating AI governance into broader enterprise IT infrastructure offers a strategic mechanism to capture necessary funding from core technology budgets. Citing industry figures from CTO.ai, Liss notes that corporate spending skews heavily toward development over governance by an estimated 5,000-to-1 ratio. Securing sufficient IT budget allocations is essential to close this gap.
Organizational placement is secondary to stakeholder authority and financial backing. Liss maintains that AI governance must be co-led with Chief AI Officers, lead data scientists, and product heads, noting a personal rule of canceling governance meetings if core technical leaders cannot attend. Furthermore, mature programs complement technical engineering with specialized bioethicists—traditionally consulted for hospital end-of-life decisions—who work alongside operational leads throughout the entire product lifecycle.
Adapting Governance for Agentic Workflows (17:25)
The rapid emergence of agentic AI requires practitioners to conduct granular mapping of underlying data flows and modernize traditional organizational frameworks. Teams are adapting standard RACI (Responsible, Accountable, Consulted, Informed) models to incorporate AI agents directly, identifying which operational tasks non-human actors execute, establishing executive oversight mechanisms, and clarifying human-agent handoffs.
Simultaneously, compliance organizations are utilizing AI agents within internal review workflows to increase operational speed and scale, as demonstrated by GE HealthCare teams in India. Casovan notes that this operational shift raises new, unresolved questions for governance bodies regarding whether and how professional training, compliance standards, and ethical frameworks must be extended directly to autonomous agents.
Commercial Due Diligence and Market Value (19:23)
Far from functioning as an administrative bottleneck, mature AI governance directly enhances enterprise valuation for startups seeking vendor contracts, corporate acquisition, or public listings. Prospective buyers and enterprise healthcare acquirers subject targets to exhaustive due diligence evaluating dataset origins, training methodologies, and documented risk mitigations.
Both Liss and Casovan observe that transparent safety protocols accelerate business goals by building sustained stakeholder trust. Companies that engage experienced counsel and domain experts early establish defensible compliance histories, preventing deal friction and transforming governance into a measurable commercial asset.
While AI governance demonstrates clear commercial and clinical value, critical operational challenges remain unresolved—notably the practicalities of training and regulating autonomous agents, as well as the widening safety gap between regulated clinical devices and direct-to-consumer tech entrants.
## Background
该内容由 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台上。视频记录了国际隐私专业人员协会(IAPP)常务董事 Ashley Casovan 与 GE HealthCare 首席隐私与数据信任官 Lara Liss 的专家对话,重点探讨了在医疗健康领域如何通过实施合规、安全的 AI 治理体系来加速 AI 技术的落地与应用。
## TL;DR
对话围绕医疗 AI 领域面临的法律困境与治理实践展开。针对 AI“行医资格”诉讼及监管挑战,GE HealthCare 提出了坚持“人机协作”与筑牢风险管理体系两条防御准则,并在产品开发中贯彻“安全、有效、可靠、公平、隐私、安全、可解释”七大 Responsible AI 原则。此外,两位专家还讨论了将 AI 治理融入技术基础设施、构建跨学科双组长治理架构、在 Agentic AI 时代重构 RACI 矩阵,以及将良好的 AI 治理转化为企业估值与并购加速器的实操路径。
## Atmosphere & style
整场对话专业、务实且充满行业洞察力,既有对法律风险与监管博弈的严肃探讨,又不失轻松幽默的互动氛围。两位嘉宾凭借深厚的行业积淀,以平实严谨的语言将复杂的合规概念与具体产品实例(如 Vscan Air)巧妙结合,呈现出一场兼具前瞻视角与落地指导价值的深度对谈。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 随着 AI 在医疗领域的应用激增,科技公司因提供医疗建议而面临诉讼潮,凸显了传统医疗企业与科技巨头的跨界竞争与融合。
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 根据当前法律框架,仅有资质的人类才具备“行医资格”,目前全美尚无任何赋予 AI 许可的州级立法案获得通过。
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 企业应对 AI 医疗法律风险的两大黄金准则为:坚持“人机协作”(Human-in-the-Loop)定位以及建立扎实可留痕的风险管理机制。
- [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) GE HealthCare 在 160 个国家运营,拥有超 130 款搭载 AI 功能的医疗器械,是全球 AI 驱动型医疗设备数量最多的企业。
- [00:05](https://www.youtube.com/watch?v=9AweKgZhSQw&t=5s) GE HealthCare 的便携式无线超声诊断仪 Vscan Air 证明了 AI 在诊疗现场(Point of Care)和急诊分诊中的应用前景。
- [00:05](https://www.youtube.com/watch?v=9AweKgZhSQw&t=5s) 医疗器械软件(SaMD)的 Responsible AI 治理需遵循七大核心原则:安全、有效、可靠、公平、隐私、安全与可解释性。
- [00:09](https://www.youtube.com/watch?v=9AweKgZhSQw&t=9s) 将 AI 治理融入基础设施能够借力技术预算,解决目前行业中 AI 研发与 AI 治理资金投入比高达 5000:1 的严重失衡现象。
- [00:09](https://www.youtube.com/watch?v=9AweKgZhSQw&t=9s) AI 治理团队应采用“双组长制”(Co-lead),将治理人员与首席 AI 官、数据科学家和产品负责人深度绑定,确保技术与业务意图真正贯彻全流程。
- [00:09](https://www.youtube.com/watch?v=9AweKgZhSQw&t=9s) 引入生命伦理学家(Bioethicist)等跨学科专家进入决策层,能防止伦理讨论脱离现实,将道德风险控制无缝嵌入产品研发。
- [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) 进入智能体 AI(Agentic AI)时代,管理者需要将 AI 智能体也绘制到 traditional RACI 矩阵中,明确其职责、监管机制与人机协同模式。
- [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) 良好的 AI 治理能帮助企业顺利通过并购和上市(IPO)中的尽职调查,是提升企业估值与加速商业目标达成的核心资产。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 医疗 AI 诉讼激增与“行医资格”的政策困境
来自国际隐私专业人员协会(IAPP)的 Ashley Casovan 与 GE HealthCare 的首席隐私与数据信任官 Lara Liss 展开讨论。二人围绕 AI 治理在医疗领域的落地经验进行深谈。近来科技公司因聊天机器人向用户提供医疗建议而屡遭起诉,这类事件突显了医疗与科技两大领域的深度融合与跨界竞争:传统医疗企业日益技术化,而科技巨头也正强力渗入医疗赛道。在当前“去监管化”趋势与“诉讼风险”攀升并存的大环境下,AI 治疗与诊断的合法性边界正经受严峻考验。
Lara Liss 强调,解决这些诉讼纠纷必须回归到最根本的法律原则——“行医资格”(Practice of Medicine)。目前在美国法律框架下,仅有具备资质的自然人(人类)才被授予行医执照。虽然全美多个州已引入针对 AI 行医资格的立法倡议,主张在特定范畴内赋予 AI 许可,但截至目前尚未有任何相关法案获得通过。
面对潜在的法律纠纷,Lara 提出了企业构建 AI 治理体系的两条黄金防御准则:
- **坚持“人机协作”(Human-in-the-Loop):** 将 AI 严格定位为辅助临床医生提高效率、实现早期诊断的工具,而非替代医生独立行医。
- **筑牢风险治理基本盘:** 结合《欧盟人工智能法案》(EU AI Act)、NIST 标准及 ISO 4201(人工智能管理体系)等国际规范,建立扎实且可留痕的风险管理机制,这是应对未来诉讼的最佳防线。
在行业风险控制层面,除了 ISO 4201 等政策性治理框架,两位专家还提及了 OWASP 等网络安全与技术安全标准,倡导跨领域融合多重规范。
- **GE HealthCare 行业地位:** 业务覆盖 160 个国家,目前市场上拥有超过 130 款搭载 AI 功能的医疗器械,是全球 AI 驱动型医疗设备数量最多的企业。
- **立法动态:** 允许 AI 拥有某种程度行医许可的州级法案虽屡被提起,但均未正式通过,预计未来几年相关立法博弈仍将持续。
---
### [00:05](https://www.youtube.com/watch?v=9AweKgZhSQw&t=5s) 便携式超声设备与医疗 AI 的七大治理原则
医疗行业被公认为能够最快从 AI 技术的突破中获益的领域。通过 AI 算法,医生可以在疾病早期阶段完成精准诊断,从而采取更温和、更具性价比的治疗方案。
为了避免空谈理论,Lara Liss 以 GE HealthCare 的产品 **Vscan Air** 为例说明了 AI 的实际应用。这是一款仅有口袋大小的便携式无线超声诊断仪,医护人员可以随身携带至急诊室(ER)或无稳定电力供应的偏远野外,在诊疗现场(Point of Care)快速完成影像扫描与急诊分诊(Triage)。
类似 Vscan Air 这类医疗器械软件(SaMD)在享受技术红利的同时,必须受到严格的 Responsible AI 治理框架约束。GE HealthCare 在研发全流程中严格遵循了七大核心风险原则:
```
+-----------------------------------------------------------------+
| Responsible AI 7大原则 |
+-----------------------------------------------------------------+
| [安全] [有效] [可靠] [公平] [隐私] [安全] [透明] |
| Safety Validity Reliability Fairness Privacy Security Explai- |
| nability|
+-----------------------------------------------------------------+
```
在产品落地过程中,向临床医生、监管机构及患者清晰解释 AI 的工作原理(可解释性)是建立信任的关键。Ashley Casovan 以双胞胎母亲的身份幽默地提到,如果这种便携设备直接面向消费者(Direct-to-Consumer),她可能会因为过于焦虑而天天在家里给孩子做超声检查。
Lara Liss 补充道,目前该设备仅提供给具备专业资质的临床医生使用,以此保证“人机协作”中的专业把关。当前传统医疗器械企业普遍受 FDA(美国食品药品监督管理局)及欧洲 MDR(医疗器械法规)严格监管;然而,随着大量未受过严格医疗监管的科技公司跨界推出消费级医疗技术,社会亟需厘清这些新型产品是否满足同等的安全与有效性标准。
- **Vscan Air 产品特性:** 手持掌上大小、无线超声探头、支持高清晰度成像、可充电电池供电,专为急诊分诊与偏远地区医疗设计。
- **核心合规要求:** 医疗设备上市前必须经过高强度安全性测试与多轮临床验证。
---
### [00:09](https://www.youtube.com/watch?v=9AweKgZhSQw&t=9s) 治理即基础设施:组织架构与跨学科协同
当前行业内部正在经历一场关于“AI 治理”(AI Governance)与“AI 基础设施”(AI Infrastructure)概念融合的大讨论。许多企业正逐步将治理职能下沉并融合至技术基础设施中。
虽然一开始对这种概念转换持保留态度,但 Lara Liss 承认,将治理融入基础设施能带来极大的务实好处——**资金与资源支持**。企业内部绝大部分预算往往掌握在 IT 和技术团队手中,借力基础设施预算,能够大幅充实 AI 治理的资金池。在会议的另一场讨论中,来自 CTO.ai 的 Nina 指出,目前行业在 AI 开发与 AI 治理上的资金投入比例悬殊,高达 **5000:1**(即每在 AI 开发上投入 5000 美元,仅有 1 美元用于治理)。
关于“AI 治理部门应该挂靠在公司的哪个架构下”,Lara 结合自己 20 年前作为隐私官(Privacy Officer)的从业经验指出,不存在唯一的“正确答案”。相比于部门名称或汇报线,更决定性的因素在于:
1. **决策桌旁是否有对的人;**
2. **治理部门是否拥有充分的权力和足够的预算支持。**
为了防止 AI 治理沦为“纸上谈兵”的合规文件,Lara 在搭建 AI 治理项目时坚持采用**双组长制/共同领导制(Co-lead)**:治理团队必须与首席 AI 官(CAIO)、首席数据科学家以及产品负责人(Head of Product)深度绑定。她甚至制定了一项硬性规则:**如果核心技术专家或业务负责人因故缺席治理会议,该次会议将直接取消并重排**,以此确保技术与业务的真实意图能够贯彻到产品全生命周期中。
Ashley Casovan 补充道,IAPP 针对不同规模和市场(本土 vs 跨国)企业的调研显示,将技术、工程、合规、政策以及**伦理专家(Ethicist)**共同引入决策层的交叉架构,已被证明具有更高的成功率。
在医疗生态中,生命伦理学家(Bioethicist)长期在临终关怀等重大医疗决策中扮演关键角色。在 AI 时代,将伦理专家与具备运营落地能力的产品团队结合,能有效防止伦理讨论脱离实际,从而将道德风险控制无缝嵌入到产品的研发全流程中。
- **投融资失衡现状:** 当前行业 AI 技术研发与 AI 治理的投入比约为 5000:1。
- **跨学科治理架构:** 必须包含法务合规、数据科学、AI 研发、产品管理及生命伦理学等多方角色,缺一不可。
### [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) 卷起袖子深入数据治理与 RACI 矩阵的 AI 化演进
在谈及日常的 AI 治理工作时,深入底层的实操能力至关重要。对于从业者而言,做 AI 治理绝不能浮于表面,必须能够“卷起袖子深入研究”(roll up your sleeves and dig in)。其核心前提在于厘清并理解底层的数据流(data flows)。随着智能体 AI(Agentic AI)时代的到来,数据治理(Data Governance)再次成为热点;只有做好基础的数据治理,才能为未来更高级的 AI 应用与治理打下坚实基础。
除了底层数据流的梳理,另一个备受关注的前沿议题是对传统责任分配矩阵(RACI 矩阵)进行重新映射与升级。在引入智能体 AI 后,行业内开始重新审视“谁负责(Responsible)、谁最终受责(Accountable)、咨询谁(Consulted)、通知谁(Informed)”这一传统框架。当 AI 智能体直接参与到项目流程中时,管理者必须将这些 AI 智能体也绘制到 RACI 矩阵中,明确它们承担了哪一部分工作、对应的监管机制(oversight)是什么,以及它们与人类团队如何协同工作。
* **智能体参与治理与培训设想**:讨论中提到,已经有企业(如在印度 AI 峰会上交流的团队)开始将 AI 智能体直接引入到自身的治理流程中,以此提升治理效率并实现规模化扩展。
* **对 AI 智能体的“培训”探索**:作为提供专业培训的机构,IAPP(国际隐私专业人员协会)等组织甚至接到了关于“如何向 AI 智能体提供培训”的有趣咨询。虽然目前行业尚未完全厘清这一模式的具体落地路径,但这已成为正被认真思考的前沿课题。
---
### [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) 良好的 AI 治理:提升企业估值与并购加速器
对于计划通过商业合同、并购(Acquisition)或公开上市(IPO)将其产品推向市场的企业而言,AI 治理绝非可有可无的“饰品”,而是能够直接提升产品与公司市场价值的核心资产。当买方或投资方对产品进行尽职调查(Due Diligence)时,他们会严格审核模型的工作原理、训练数据的来源与质量,以及相关潜在风险是否得到了妥善缓解。
因此,对初创公司及各类企业而言,必须确保有专业的法律顾问和 AI 治理专家参与团队建设,形成一套清晰、严密且富有说服力的 AI 治理叙事。良好的 AI 治理不仅能通过尽职调查,更是加速实现商业目标和公司战略的强力引擎。
* **尽职调查核心关卡**:在并购或上市过程中,投资者和买方重点审查模型训练过程、工作机制以及风险规避措施。
* **商业价值转化**:良好的 AI 治理能够直接转化为产品溢价与市场竞争力,帮助企业在众多的竞争对手中脱颖而出。
* **总结寄语**:无论是否涉及公司出售,健全的 AI 治理都是推动业务加速落地和目标达成的强力助推器。
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 解决这些诉讼纠纷必须回归到最根本的法律原则——“行医资格”(Practice of Medicine)。 — Lara Liss
> [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 坚持“人机协作”(Human-in-the-Loop):将 AI 严格定位为辅助临床医生提高效率、实现早期诊断的工具,而非替代医生独立行医。 — Lara Liss
> [00:04](https://www.youtube.com/watch?v=9AweKgZhSQw&t=4s) 筑牢风险治理基本盘:结合《欧盟人工智能法案》(EU AI Act)、NIST 标准及 ISO 4201(人工智能管理体系)等国际规范,建立扎实且可留痕的风险管理机制,这是应对未来诉讼的最佳防线。 — Lara Liss
> [05:00](https://www.youtube.com/watch?v=9AweKgZhSQw&t=300s) 在产品落地过程中,向临床医生、监管机构及患者清晰解释 AI 的工作原理(可解释性)是建立信任的关键。 — Lara Liss
> [09:00](https://www.youtube.com/watch?v=9AweKgZhSQw&t=540s) 目前行业在 AI 开发与 AI 治理上的资金投入比例悬殊,高达 5000:1(即每在 AI 开发上投入 5000 美元,仅有 1 美元用于治理)。 — Nina
> [09:00](https://www.youtube.com/watch?v=9AweKgZhSQw&t=540s) 如果核心技术专家或业务负责人因故缺席治理会议,该次会议将直接取消并重排 — Lara Liss
> [09:00](https://www.youtube.com/watch?v=9AweKgZhSQw&t=540s) 在 AI 时代,将伦理专家与具备运营落地能力的产品团队结合,能有效防止伦理讨论脱离实际,从而将道德风险控制无缝嵌入到产品的研发全流程中。 — Ashley Casovan
> [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) 对于从业者而言,做 AI 治理绝不能浮于表面,必须能够“卷起袖子深入研究”(roll up your sleeves and dig in)。 — Lara Liss
> [17:25](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1045s) 当 AI 智能体直接参与到项目流程中时,管理者必须将这些 AI 智能体也绘制到 RACI 矩阵中,明确它们承担了哪一部分工作、对应的监管机制(oversight)是什么,以及它们与人类团队如何协同工作。 — Lara Liss
> [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) 对于计划通过商业合同、并购(Acquisition)或公开上市(IPO)将其产品推向市场的企业而言,AI 治理绝非可有可无的“饰品”,而是能够直接提升产品与公司市场价值的核心资产。 — Lara Liss
> [19:23](https://www.youtube.com/watch?v=9AweKgZhSQw&t=1163s) 良好的 AI 治理不仅能通过尽职调查,更是加速实现商业目标和公司战略的强力引擎。 — Lara Liss
## Mentioned
Ashley Casovan, Lara Liss, Nina, IAPP (国际隐私专业人员协会), GE HealthCare, CTO.ai, Vscan Air, FDA (美国食品药品监督管理局), MDR (医疗器械法规), EU AI Act (欧盟人工智能法案), NIST, ISO 4201, OWASP, RACI 矩阵
## Video description
Ashley Casovan - Managing Director, IAPP
Lara Liss - Chief Privacy and Data Trust Officer, GE HealthCare
The opportunities to improve healthcare with advance AI are immense. Leading healthcare companies have found that they are able to accelerate their adoption of AI by ensuring these systems are safe and well managed. Join this expert discussion to learn how AI governance is being implemented in GE HealthCare practices.
[Source](https://www.youtube.com/watch?v=9AweKgZhSQw)
Source quotes
[0:04] Lara Liss: 解决这些诉讼纠纷必须回归到最根本的法律原则——“行医资格”(Practice of Medicine)。
[0:04] Lara Liss: 坚持“人机协作”(Human-in-the-Loop):将 AI 严格定位为辅助临床医生提高效率、实现早期诊断的工具,而非替代医生独立行医。
[0:04] Lara Liss: 筑牢风险治理基本盘:结合《欧盟人工智能法案》(EU AI Act)、NIST 标准及 ISO 4201(人工智能管理体系)等国际规范,建立扎实且可留痕的风险管理机制,这是应对未来诉讼的最佳防线。
[5:00] Lara Liss: 在产品落地过程中,向临床医生、监管机构及患者清晰解释 AI 的工作原理(可解释性)是建立信任的关键。
[9:00] Nina: 目前行业在 AI 开发与 AI 治理上的资金投入比例悬殊,高达 5000:1(即每在 AI 开发上投入 5000 美元,仅有 1 美元用于治理)。
[9:00] Lara Liss: 如果核心技术专家或业务负责人因故缺席治理会议,该次会议将直接取消并重排
[9:00] Ashley Casovan: 在 AI 时代,将伦理专家与具备运营落地能力的产品团队结合,能有效防止伦理讨论脱离实际,从而将道德风险控制无缝嵌入到产品的研发全流程中。
[17:25] Lara Liss: 对于从业者而言,做 AI 治理绝不能浮于表面,必须能够“卷起袖子深入研究”(roll up your sleeves and dig in)。
[17:25] Lara Liss: 当 AI 智能体直接参与到项目流程中时,管理者必须将这些 AI 智能体也绘制到 RACI 矩阵中,明确它们承担了哪一部分工作、对应的监管机制(oversight)是什么,以及它们与人类团队如何协同工作。
[19:23] Lara Liss: 对于计划通过商业合同、并购(Acquisition)或公开上市(IPO)将其产品推向市场的企业而言,AI 治理绝非可有可无的“饰品”,而是能够直接提升产品与公司市场价值的核心资产。
[19:23] Lara Liss: 良好的 AI 治理不仅能通过尽职调查,更是加速实现商业目标和公司战略的强力引擎。
Source quotes
[0:04] Lara Liss: keeping a human in the loop remains an essential policy baseline and governance safeguard, framing AI as an operational tool to assist clinicians rather than replace them.
[0:04] Lara Liss: Under current U.S. law, only human beings are licensed to practice medicine.
[0:04] Prioritizing patient safety as the foundational "north star."
[5:01] Software as a Medical Device (SaMD) and how AI delivers clinical value today.
[5:01] Strict Healthcare Regulation: Established medical device manufacturers undergo rigorous FDA clearance in the U.S. and Medical Device Regulation (MDR) compliance in Europe, adhering to strict safety, validation, and testing standards.
[5:01] Direct-to-Consumer Tech Disruption: Unregulated tech companies entering the space often bypass medical-grade governance frameworks, raising critical societal questions about patient safety and product transparency.
[9:23] Nina: estimating a 5,000 to 1 ratio of AI investment relative to governance spending.
[9:23] Mandatory Technical Buy-in: Governance programs must be co-led alongside Chief AI Officers, lead data scientists, and product heads rather than isolated in legal or compliance silos.
[9:23] Lara Liss: if key technical and product leaders cannot attend a governance meeting, the meeting is canceled and rescheduled.
[17:25] Lara Liss: roll up your sleeves and dig in.
[17:25] RACI Evolution: AI agents are actively being integrated into RACI charts to clarify operational responsibility, executive oversight, and human-agent collaboration.
[19:23] Lara Liss: Building strong AI governance directly increases the enterprise value of a product when going to market.
[19:23] Ashley Casovan: robust AI governance serves as a critical business accelerator rather than a bottleneck.
Original transcript
[0:04] for the better part of today about uh some of the
observations that we've seen both here in this
[0:09] uh track which has been focused on different
aspects of AI governance and so we're really
[0:15] uh excited to I think dig into a more practical
view of what some of these themes that we've been
[0:21] hearing mean uh governance as infrastructure the
different types of people that you need to have
[0:26] at the table and really situate that in the
healthcare context text. So, um first of all,
[0:32] thanks for being here and uh having this
discussion. I always learn so much from
[0:37] uh our discussions and so hopefully everybody else
uh here can benefit from that as well. Maybe just
[0:43] uh starting with um something that I think has
been in the news a lot lately and I know I follow
[0:50] um are the various different lawsuits that have
been starting to come and we've seen this in the
[0:56] healthcare context. Uh recently we saw how um some
healthcare or some tech companies uh are being
[1:05] sued for advice that people are receiving through
chat bots, health advice that people are being
[1:10] receiving through chat bots. Um, and we're seeing
more of this, uh, I I would say blend of health
[1:20] companies, uh, turning more into tech companies
and tech companies, um, leaning more into health
[1:26] care. And so from this perspective of looking
at regulation, where are you seeing this going
[1:32] and what do you think that uh both healthcare
companies and tech companies should be thinking
[1:36] about um especially in this era of deregulation
but increased potential for lawsuits. Yeah,
[1:43] absolutely. So welcome everyone. I'll start just
by setting the stage. I am the chief privacy
[1:47] and data trust officer for GE Healthcare. GE
Healthcare is a large medical device and medical
[1:52] technology company. We operate in 160 countries
and we have the highest number of medical devices
[1:58] powered by AI in the market today with overund
130 medical devices out there that operate on AI
[2:04] in their systems. So this is an issue that is um
inherent in our company and what we do and I find
[2:10] these issues as I assume many of you do super
fascinating. So to get to your question about
[2:15] the recent lawsuits um I find these headlines
um remind us of a few important principles. The
[2:22] first is the practice of medicine. So today in
the United States, only humans are licensed to
[2:28] practice medicine. So as we look at these medical
technologies that are coming onto the market,
[2:32] I think one of the issues that you're going to see
in these lawsuits is who is authorized to practice
[2:36] medicine. If you follow some of the legislative
advocacy happening around the United States today,
[2:41] that's happening uh quite a bit at the state
level, you may have seen that there have been
[2:46] some laws introduced in states that are advocating
for AI to be allowed to have a license to practice
[2:52] medicine in certain capacities. And I think
that's a fascinating issue, one that we have not
[2:56] seen before. To my knowledge, none of those laws
have passed yet. Uh but we think they will come up
[3:01] again year after year. And as we look at some of
the protections that we have in the AI governance
[3:07] space, one of them is having a human in the loop
and being focused on uh from our perspective,
[3:12] we see AI's the tools that enable the healthcare
practitioners to do their jobs more efficiently
[3:17] um and to better diagnose patients earlier. So I
think there's a really interesting policy issue
[3:21] coming up that these lawsuits are highlighting of
who is authorized to practice medicine and should
[3:25] technology be authorized to practice medicine
in the future or not. Um the second issue I
[3:30] would highlight as we look at what's coming
for companies in this space is just a focus
[3:35] on going back to the basics of your AI governance
program. So like many healthcare companies we map
[3:44] the EU AI act to the NIS standards um looking also
very practically at the ISO certification in this
[3:50] space for um AI risk management. And as we look
at those, some themes come across in healthcare
[3:57] that I think resonate in other industries as
well. So as you look at things like um safety,
[4:03] safety is our northstar as a company. So depending
on the industry that you're in, looking at your
[4:08] northstar, what risk management do you have in
place around that that supports that, enables it,
[4:14] and when you have a good riskmanagement program,
in many ways that's your best defense to lawsuits
[4:19] that may come. uh because you've been thoughtful
about the way that you approach these issues
[4:23] and you've been able to address those risks and
document uh the ways that you have taken steps to
[4:28] um appropriately minimize and reduce risks. Yeah.
And maybe we'll just add one more framework to it
[4:33] that we were talking about earlier is OASP.
And so speaking of just looking to different
[4:39] industries for uh risk controls um in a policy
AI governance space, we often go directly to
[4:46] something like ISO 4201. Um but I think combining
those in addition to uh pre-existing industry uh
[4:56] standards or regulations that might be
uh in your space. So pivoting towards
[5:01] uh more concrete and specific examples. A lot of
the work that you do um I would say fits within
[5:08] the software as medical device space. There are
standards for that um and pre-existing use cases.
[5:14] Can you speak to some examples of um how you
are using AI and as a company um and then maybe
[5:21] some of the uh risks and opportunities that that
come with that? Sure. So I think when I talk to
[5:28] people across industries, one of the things that
resonates with most people is that healthcare has
[5:33] some of the greatest promise to see the benefits
of AI not just far in the future but today and
[5:39] certainly in the next 5 to 10 years. I believe
we have incredible power through AI to be able
[5:43] to diagnose disease earlier and to better treat
it um when the the treatments are less invasive
[5:48] and more effective for patients. And I think
that's one of the um for me one of the reasons
[5:53] that gets me up in the morning and one of the
reasons why I work in the space is I believe in
[5:56] that power and that difference that we can have.
So as we look at and I also find it helpful to
[6:01] talk about specific products and stories because
we can talk in theory all day long but when you
[6:05] give a specific example I find that resonates a
lot more with people. So, the example I wanted
[6:10] to share with you of one of our technologies we
have in the market today is Vscan Air, which is
[6:15] a portable ultrasound system. And if I had it here
today, it would fit in my pocket. I could pull it
[6:19] out and show it to you. Um, it allows technicians
in the field to be able to use an ultrasound probe
[6:25] um uh at the point of care wherever they are.
So, it allows us to be able to do triage more
[6:31] efficiently in ERS. Um, it allows for use of
this in remote field situations where you may
[6:37] not have a constant electricity supply. it can
be charged up and taken out into the field. Um,
[6:41] and it has good quality images and scans. So, I
think it's one of the greatest examples of how AI
[6:47] is powering innovation in the market today that we
can see. But as you look at any product like that,
[6:53] um, we do see that um, of course you have
to map and address the risks, right? So,
[6:59] as we talked about earlier with that EU AI act
and the NIST framework and the ISO standards, um,
[7:05] we look at our seven principles of risk. So just
like you all are doing in many of your programs,
[7:10] what we have done in our responsible AI program
is to map for safety, for validity, making sure
[7:16] that we have um you know reliability in these
products that we have on the market. Um we look
[7:22] at fairness, we look at privacy, uh security, you
know, transparency and explainability. So making
[7:28] sure that people can understand the technology
and how it works. And for us that's at the the
[7:32] clinician level and making sure we can explain
to clinicians uh regulators and patients how the
[7:38] technology works. So those are some of the things
that we work on as we look at risk mitigation in
[7:42] the field. And I think that's such a interesting
example. Uh, I know when I had the opportunity to
[7:48] foresee this ultrasound machine um as a mother
of twins, I was horrified [laughter] um that
[7:56] that might be something that I could use in my own
home uh because I know I would have been checking
[8:01] it all of the time. Um so is that something that
you foresee in the future being provided direct
[8:08] to consumer? So today our medical devices are
things that we put in the field with those expert
[8:13] clinicians who do that interpretation and provide
the services. Um and that's one of the pieces you
[8:19] know we talk about having a human in the loop and
our technology enables those medical providers
[8:24] um to be able to better treat those patients.
So that's how we have that product set up. I
[8:28] do think as you look, you know, whether it's in
the exhibit hall or you look at technology and
[8:32] how it's evolving today, you can quickly begin to
see um other contexts where people may do be doing
[8:38] more direct to consumer. And I think as you look
at that from a governance standpoint, those of
[8:43] us who are in the medical device industry, we are
regulated by the FDA, by the MDR in Europe. We go
[8:48] through high safety standards, lots of testing.
And I think one of the interesting issues from
[8:52] a governance standpoint today is if you have tech
companies that are not highly regulated healthcare
[8:57] companies who are coming up with similar
technologies, what standards are they held
[9:02] to and do patients or customers fully understand
what's involved in that governance framework? And
[9:07] I think we have an obligation as a society to
make sure that the um that the medical devices
[9:13] or the other medical technologies that people
are interacting with are safe, are reliable,
[9:17] are valid and meet all of those um governance
standards that we are held to as medical device
[9:22] companies. Yeah. And I that's such an interesting
point this distinction between as we said earlier
[9:30] uh tech companies that are getting into the space
uh and then the long-standing infrastructure that
[9:36] healthcare companies have had to follow. Um,
and I use infrastructure intently because that's
[9:43] something that uh I think has been a big takeaway
or theme for me today has been that this word is
[9:50] being used so much to almost replace the word
of governance and how we've been using it. Um,
[9:56] but then we've also seen even earlier in
this room governance as infrastructure.
[10:01] And one of the things that I can imagine you work
on the compliance side of of the house that you've
[10:07] had to start working um more closely with some of
the product teams that are doing the evaluations
[10:13] that you talk about. Can you speak to who's at the
table um currently in some of these uh governance
[10:23] qu uh conversations? Um are they the right people?
Who needs to be there? Sure. So I would say um a
[10:31] couple of things here. The first is you know
I heard a theme and sometimes when you go to
[10:36] these conferences I find you pick up on a theme
that that percolates through multiple sessions.
[10:40] One of the themes I've heard today is moving away
from AI governance and focusing more on this AI is
[10:45] infrastructure. And as we look at that my first
reaction to that was I'm not sure I like that. I
[10:50] you know I think that there's an important role
that governance serves and it's different from
[10:54] what the infrastructure is that you build. there
is some overlap but the more I thought about it I
[10:58] do think there are some advantages to this idea
of infrastructure fusing with governance one of
[11:04] those advantages is uh simply funding we know
that a lot of the budget within organizations
[11:09] sits within the IT teams and they have that um
infrastructure budget so if we can use that to
[11:14] tap into more resources to further expand and
fund governance I think that's an interesting
[11:18] and practical angle to it I know earlier today
in this room I heard a panel um and Nina from
[11:24] CTOAI was saying that Today the investment of
um if I heard her correctly the investment in
[11:29] AI to governance has been 5,000 to1. So for every
$5,000 spent on AI there's only a dollar spent on
[11:35] governance and if incorporating infrastructure
into that is a way that we can increase that
[11:39] funding and resourcing that's something that I'd
be interested in talking about. Um but to your
[11:43] second question one of the other things I heard
today is where's the right place for governance
[11:48] to sit? Exactly where should it be? And earlier
in my career I'll tell you I thought there was a
[11:53] right answer to that question. Um it actually
started out when I was a privacy officer and
[11:57] the question 20 years ago the question was where
should privacy sit and I did a lot of research
[12:01] and benchmarking about where's the right place for
it to sit but the longer I've been in the industry
[12:06] um to me I don't think there's any one right place
I think the more important factors are one who's
[12:11] at the table and two is it appropriately resourced
so if you have the right people at the table with
[12:16] the right influence in the organization
and you have the right budget to be able
[12:20] to do things effectively I think it can sit in
a number of locations within your organization
[12:25] And I want to talk about who are the right people
to have at that table. I will tell you I've set up
[12:29] two AI governance programs at different companies.
Um, and I always say it has to be done paired
[12:35] with your data scientists and your business teams,
your best technology experts. So it cannot be just
[12:41] sitting in legal or compliance or governance. It
won't work. Having policies on paper that do not
[12:46] come to life in your products is not a recipe for
success. So, as we look at this and I set up these
[12:52] programs, I always say we want to co-lead it with
our lead data scientist, our chief AI officer,
[12:57] our head of product, people like that. And I
also tell the teams as we're getting the program
[13:02] off the ground. If those people can't come to the
meetings, I cancel the meetings. And I tell them,
[13:07] we're going to reschedule this when you can
come because it's so important for you to be
[13:10] at the table. We need your perspective. We need
your buyin for this to work. We can't just do uh
[13:16] an AI governance program on paper that meets
legal requirements without incorporating the
[13:20] day-to-day of how this product actually works. So
my recommendation is you get those right people in
[13:25] the room together and make sure they understand
that they have to be involved every step of the
[13:29] way for it to be truly successful and built into
your um your product development and life cycle
[13:34] process. One thing I'd really love to build off
of what you just shared too. So as uh IP is a
[13:40] professional association and uh serving digital
governance professionals, I lead our AI governance
[13:46] efforts and um that's been a very recent evolution
for the organization that supported privacy
[13:52] professionals prior to that. And so I I think
it's not just privacy professionals um but this
[13:58] idea that Larara shared of trying to benchmark and
understand where certain roles should fit within
[14:04] the organization or what that work should be done
has actually been a benchmarking activity that
[14:09] we've pursued for a long time on on that side of
the house and within the AI governance space. It's
[14:15] so complex and there's so many different types of
roles involved in this as as we're hearing from
[14:21] some of these examples that it's been difficult
to track and to um navigate where organizationally
[14:27] that should fit but also who has responsibility
or authority. Um however, one of the things that's
[14:32] been interesting that's come out of our surveys
which are publicly available so feel free to check
[14:37] them out. um is that depending on the complexity
and size of organization and whether it is serving
[14:44] a domestic market or an international market
there are some trends in terms of the types
[14:50] of roles and the types of governance um that
do seem to be more successful um so agreed
[14:57] um there's not one right answer um and I
think that if you're looking for some guidance
[15:03] um people with expertise um like we have here
today. Uh and also some of this benchmarking work
[15:10] to just give you a start to have that conversation
in your organization um is is useful if you're
[15:16] looking for that. So I think really understanding
and again another theme from this is exactly what
[15:25] you've just said that we've heard today of having
technical product engineering roles partnered with
[15:31] um compliance type roles policy roles um even I
think one thing that we haven't heard much today
[15:39] is the role of an ethist but we've been seeing
um a lot of tech companies recently try and hire
[15:45] ethsists and bring them to the table. Do you think
it's also important that as you said, I really
[15:50] loved what you just said of if they're they can't
come to the meeting, then I'm just not going to
[15:54] have the meeting. Um do would you put an ethicist
in the same category as that when you're trying
[16:00] to think about some of these really difficult
problems with high-risk systems um that you're
[16:05] building with software as medical devices? Yeah.
Um a couple of things there. So when you mentioned
[16:10] the IEP's work, I do want to I should have said
at the beginning that the views I say today are
[16:14] my own. they don't represent GE healthcare and I
sit on the board of the AP with Ashley. Um but as
[16:20] we look at the ethsists, yes, absolutely what an
incredibly important role to have at the table and
[16:25] I will tell you in the healthcare ecosystem, uh
healthcare ethicists and bioethicists have been
[16:30] an important part of the conversation for a long
time before AI ever came on the scene. Right? So
[16:35] I spent six years as a hospital and health system
attorney and we had an ethicist who advised us on
[16:40] a regular basis on things like end of life care
and other decisions in the health care process. So
[16:44] I think those ethicists are a critical component
of decisions throughout healthcare companies life
[16:49] cycles including where AI technology is involved.
I think they should be a good voice at the table.
[16:55] I do think it's really important to pair them
with someone who's got operational capability
[17:00] um and making sure that they have the right
additional resources to support. So it's not
[17:03] just a theoretical decision but it's built into
that product life cycle. You're working with
[17:07] those product teams throughout and you've got
it um blended seamlessly in there. And are you
[17:13] finding that the language of these um compliance
discussions or governance boards are shifting to
[17:19] be more technical in nature or to accommodate
um the perspectives that are coming into those
[17:25] discussions? Yeah. So, a couple of thoughts
there. I don't know what you guys think in
[17:28] the audience if you do this work on a day-to-day
basis, but for me, I always say if you're going
[17:33] to do AI governance work, you need to be able to
roll up your sleeves and dig in and because it's
[17:38] all about one, understanding those data flows
that sit underneath, right? data governance is
[17:42] hot again with the because we all know that if
data is governed well particularly in the age
[17:46] of agentic AI that's a big help in your work
going forward. Um I also think as you look
[17:53] at this work one of the other interesting ideas
I've heard people talk about is um updating your
[17:58] raciy and I heard people say so who's responsible
accountable consulted informed where does AI fit
[18:05] into that and someone said to me you need to now
map the people who are who is going to be working
[18:11] on your pro project including those AI agents and
see where those AI agents fit in on your racing
[18:17] um what part of the work are they doing what is
the oversight ite for those agents and how does
[18:21] that all fit together? So, as you um go through
this journey that we're all on of AI governance,
[18:27] making sure that you are evolving for those latest
changes as the agentic AI piece becomes more
[18:31] and more important. I'm really happy you brought
that up because uh one of your teammates in India
[18:36] uh when we were there for the AI summit earlier
this year had mentioned that they're starting
[18:41] to use AI agents as part of their governance uh
process. So, I don't know if that's throughout um
[18:47] your entire company, but I think that mapping
that's such sage advice to think about where
[18:54] you're using agents within your governance process
and how that's going to help you to scale and move
[19:00] through that quicker and also figuring it out
in relation to humans. One of the questions we
[19:06] get because we provide uh training to humans is
if we're going to provide training to agents and
[19:11] how that will work. Uh we have not figured it out
yet. So, please don't ask me. But, uh, but it's
[19:17] it's a good question and something that we're
we're earnestly thinking about. Um, okay, last
[19:23] uh point of advice uh that I think um we're just
going to assume the composite of people here for
[19:30] sake of time. Um if there are companies that are
looking to come to GE healthcare for uh contracts
[19:39] or potential acquisitions, um what advice do
you have for for them in terms of thinking about
[19:48] uh how they can set themselves apart from others
from a a governance perspective? Yeah. So the
[19:54] last thing I'll say is regardless of what industry
you're in, regardless of whether it's healthcare
[19:57] or anything else, if you are building something
that you eventually want to sell to the market,
[20:01] including through acquisition, an IPO, things like
that, governance is not a nice to have. It will in
[20:08] good governance of AI will increase the value of
your product when you go to market because what
[20:13] is going to happen is there will be due diligence
on that product, how it works, how it was trained,
[20:17] and whether the risks have been mitigated. So
my advice to you is make sure that you have
[20:22] a good lawyer you're consulting with, a good
expert that you're bringing in on the teams,
[20:26] and you've got a story to tell on AI governance
because people are going to ask that question
[20:30] as you go to eventually bring that product to
market. And it's it's important that you have
[20:34] a good story to tell. Thank you so much. We say
the exact same thing for whether you're if you're
[20:39] not selling your company, governance can be an
accelerator for you to complete uh the goal or
[20:45] objective that you had. So with that, we're done.
goes fast. But thank you so so much [applause]