OpenAI deploys medical AI models across global clinics to cut errors

OpenAI在全球诊所部署医疗AI模型以减少差错 通过定制医生对话训练模型减少了临床诊断错误，并让该平台成为不可或缺的医疗安全网。 本期节目讲述了Nate Gross博士和Karan Singhal因弥合诊所技术差距与确保AI安全的使命而投身医疗AI的个人及职业历程。他们探讨了ChatGPT上每日庞大的健康信息需求，并介绍了“HealthBench”——这是一个与250位医生合作构建的大型评估数据集，旨在培养模型在沟通和不确定性管理等方面的关键临床技能。对话还重点展望了未来方向，包括多模态数据整合以及成功的实际应用部署，例如在肯尼亚内罗毕开展并显著减少了医疗差错的临床副驾驶（copilot）研究。最终，OpenAI旨在通过构建AI安全网为患者赋能，同时通过减轻行政负担为临床医生节省时间。 在 ChatGPT 每周 9 亿活跃用户中，每天约有 4000 万人使用该平台寻求健康信息；OpenAI 设立了严格的安全防护机制，承诺绝不使用用户的医疗健康数据进行模型训练。 (3:02) OpenAI 耗时一年与约 250 名医生紧密合作，构建了涵盖 4.9 万个评估维度的多轮对话评测基准“HealthBench”，重点衡量模型在模糊症状下的主动追问与临床安全能力。 (6:57) OpenAI 将医疗考量贯穿预训练、中训练至后训练的完整管线，依托 5000 多场定制临床对话重点提升模型的分级转诊、自适应沟通以及在不确定时主动承认并提供指引的能力。 (10:10) 在肯尼亚内罗毕约 20 家诊所开展的临床协作研究显示，AI 在后台监测电子病历并适时预警，能够显著减少临床医生的诊断和治疗失误，发挥类似“安全网”的保护作用。 (16:16) OpenAI 将医疗战略归纳为三大支柱：托底普及技术、减轻行政负担让医生重归临床，以及拓展医学能力上限；未来的研究核心是整合可穿戴设备与历史病历以实现预防性预测。 (25:33)

OpenAI health leaders Dr. Nate Gross and Karan Singhal argue that AI trained on rigorous medical rubrics can bridge systemic healthcare fragmentation, functioning as both an everyday contextual health assistant for patients and an active clinical safety net for medical providers. Training Models for Patient Safety and Uncertainty (6:57) Addressing consumer demand—which Dr. Gross reports accounts for roughly 40 million daily health-related queries out of ChatGPT's 900 million weekly users—requires specialized training frameworks. Singhal explains that OpenAI designed 'HealthBench' in collaboration with 250 physicians over a year, evaluating models across 49,000 dimensions of multi-turn conversational performance. Rather than generating premature diagnoses from vague complaints, models are evaluated on whether they request context and adjust literacy levels according to the user's background. Real-World Workflow Integration and Error Reduction (16:16) Beyond standalone benchmarks, OpenAI emphasizes real-world deployment. Singhal draws an analogy to autonomous vehicles, asserting that AI can provide a 'protective effect' similar to having a clinician in the family. In a clinical trial conducted with PandaHealth across approximately 20 clinics in Nairobi, Kenya, an AI co-pilot running in the background of electronic health records yielded a statistically significant reduction in diagnostic and treatment errors by alerting clinicians only when potential mistakes occurred. Singhal notes that the tool became so embedded in local clinical workflows that investigators resisted establishing an unassisted control group for follow-up studies, viewing the removal of the AI safety net as potentially dangerous. Singhal forecasts that healthcare AI will cross a broader adoption inflection point within the next six months. Connecting Fragmented Records and the 'Clipboard Spouse' (21:05) Healthcare delivery remains constrained by decentralized, analog, and non-cloud systems that leave patients unmonitored for long periods. Dr. Gross states that OpenAI addresses this by developing HIPAA-aligned environments, adopting national CMS standards for electronic health record syncing, and integrating consumer biosensors and wearables. OpenAI emphasizes that it maintains encrypted protections and will not train its foundation models on users' private healthcare data. Dr. Gross envisions AI acting as a 'clipboard spouse'—an attentive digital companion that tracks longitudinal records, monitors calendars, and removes administrative friction so that patients and overburdened caregivers can better navigate complex treatment plans. Strategic Pillars and Frontier Medical Discoveries (25:33) Dr. Gross outlines OpenAI's health strategy across three operational pillars: 'raising the floor' by democratizing access to medical intelligence globally, 'sweeping the floor' by automating administrative documentation to return time to clinicians, and 'raising the ceiling' by accelerating scientific research. On the frontier side, Gross notes that longer-running models analyzing multimodal datasets and large volumes of historical data are beginning to identify new therapeutic uses for existing, shelved medications and spot preventive indicators that human physicians might overlook. OpenAI frames healthcare AI as a transformative mechanism to mitigate medical errors and administrative burnout, though realization of this vision depends on sustained clinician trust, strict data privacy enforcement, and widespread interoperability across deeply fragmented legacy health systems.

OpenAI健康团队负责人Nate Gross博士与健康AI研究主管Karan Singhal指出，全球庞大的日常健康咨询需求正促使AI向临床纵深演进；通过跨训练流程的医学对齐、单向加密的数据隐私隔离，以及深度融入诊疗工作流，AI正在从问答工具转型为减少医疗差错、打破数据孤岛并拓展医学发现的底层基础设施。 庞大健康需求与严格的数据隐私屏障 (3:02) 在ChatGPT每周9亿活跃用户中，约有四分之一的查询与健康相关，相当于每天有4000万人寻求健康信息。Gross博士指出，传统医疗系统高度碎片化且缺乏连续性，患者在一年中的绝大部分时间都处于缺乏集中医疗支持的被动状态。为服务这一庞大人群，OpenAI构建了ChatGPT for Health框架。 针对医疗场景的敏感性，OpenAI设立了单向阀式加密保护机制，并承诺绝不利用用户的健康数据训练模型。该架构旨在克服传统搜索引擎“无记忆”和千篇一律的缺陷，允许用户安全地引入个人病历与体征背景，使每一次交互都能在保护隐私的前提下实现个性化与临床事实锚定。 HealthBench评测与全流程临床训练 (6:57) Singhal强调，医疗属于高风险领域，安全性和事实锚定必须作为研发的起点，以此弥合模型能力与实际临床应用之间的鸿沟。为此，OpenAI与约250名医生开展了为期一年的合作，开发出多轮对话基准数据集HealthBench，在49000个维度上评估模型表现。例如，面对患者提出的模糊症状，基准着重考核模型主动追问补充信息的能力，而非轻率给出诊断。 在模型训练上，OpenAI打破了仅依赖标准化选择题评测的做法，将健康领域的对齐融入预训练、中程训练和后训练全流程。依托5000多个医生编写的真实对话和48500条评分准则，模型重点训练了三项核心临床能力：在复杂情况下主动建议线下就医的分级转诊机制、动态匹配专业医师或低识字率普通患者的语言自适应能力，以及在面对不确定性时承认认知盲区以减少过度自信幻觉。 临床工作流中的实时安全网与实证 (16:16) Singhal借用自动驾驶技术（如Waymo）带来的安全信任感，以及家庭成员中有医生所产生的健康保护效应，提出让AI为医患双方充当全天候“安全网”。在实证研究中，OpenAI与PandaHealth在肯尼亚内罗毕的约20家诊所部署了临床辅助工具。该系统静默监测医生在电子健康记录（EHR）中输入的内容，仅在检测到潜在差错或严重隐患时主动干预。 试验结果显示，使用该工具的临床医生在诊断和治疗错误率上均取得了统计学意义上的显著降低。Singhal表示，这标志着评估重点从静态基准转向了部署后的真实临床工作流；在后续追踪中，当地医疗团队甚至认为为了对照实验而暂停提供该AI工具将对患者构成危险。 破除数据孤岛与“带写字板配偶”愿景 (21:05) Gross博士指出，当前医疗机构的数据普遍散落于各自独立、非云端的软硬件系统中，导致技术落地常沦为碎片化的单点方案。为了在专业层面建立信任，AI不仅需要依靠训练权重，还必须动态对接最新的医学文献、权威指南以及地方特定的诊疗规范，并在符合HIPAA标准的私密环境中运行。 在个人健康管理层面，OpenAI正配合相关部门与CMS推动国家电子病历同步标准，打通可穿戴设备、生物传感器与移动终端的数据链路。Gross将理想的AI助手比作“带着写字板、有医学背景且高度专注的配偶”：在获得用户授权后，AI能够长期追踪复杂病史，结合日常作息与体征提供日程管理与用药建议，协助患者在诊疗旅程中从被动应对转变为主动掌控。 三大战略支柱与拓展医学发现上限 (25:33) 针对临床医护人员最为匮乏的时间资源，OpenAI将医疗工作划分为三大支柱：“提升下限”，即降低前沿智能的使用门槛，向包括免费用户在内的全球医患普及基础医疗认知；“清扫地面”，即通过自动化繁冗的行政和文书流程，将医生从琐务中解放出来，增加与患者直接沟通的时间；以及“拉高上限”，即推动多模态模型与医学推理能力的跃升。 在大模型处理长上下文和多模态数据的能力持续提升下，技术开始展现出从0到1的突破潜力。Gross提及，AI已开始跨越数十年病史数据捕捉人类医生可能遗漏的预防性线索，并为历史上闲置在实验室货架上的老药重新发现临床适应症，加速开展人类科研人员难以独立应对的大规模实验。 对话表明，OpenAI将医疗AI定位为放大专业人员能力的辅助工具而非替代者；但在技术走向深化的过程中，如何安全整合非标准化且未上云的遗留医疗系统、应对因地区资源差异带来的诊疗规范分化，并让模型长期稳定地处理超长周期的多模态医疗档案，仍是实现广泛临床信任的关键考验。

[0:00] Nate Gross: mostly pissing off the IT department [0:00] Nate Gross: fax machine, clipboard, paper binder [3:02] Nate Gross: 364 days per year without the opportunity to engage [3:02] Nate Gross: one-way valve [3:02] OpenAI: will never train on users' healthcare data [6:57] Karan Singhal: The most helpful thing to do in that situation and the safest thing to do... is actually to ask for more context [10:10] Karan Singhal: every phase of model training from pre-training to mid-training to post-training... integrates health into every major stage [10:10] Nate Gross: better know when it doesn't know and say that [14:22] Karan Singhal: biking next to a Waymo, I actually feel safer than if I was biking next to a human driver [21:05] Nate Gross: Sometimes I think of two patients and one patient has to navigate the healthcare system by themselves. And the other patient maybe has a spouse come with them. And that spouse has a clipboard and used to work as a healthcare professional and is very attentive, if not neurotic, and can follow up on details and is connected to your personal calendar. [25:33] Nate Gross: Raise the floor [25:33] Nate Gross: Sweep the floor [25:33] Nate Gross: Raise the ceiling [26:45] Nate Gross: been sitting on a shelf [28:44] PandaHealth Team: dangerous

[0:00] Andrew Mayne: Hello, I'm Andrew Mayne, and this is the OpenAI Podcast. [0:03] Andrew Mayne: Today, we're talking to Dr. Nate Gross, [0:05] Andrew Mayne: head of health in Karan Singhal, [0:07] Andrew Mayne: who leads health AI research at OpenAI. [0:10] Andrew Mayne: We'll cover what went into training models [0:11] Andrew Mayne: to handle sensitive questions [0:13] Andrew Mayne: and how it's helping clinicians, patients, [0:15] Andrew Mayne: and healthcare systems. [0:16] Karan Singhal: We actually worked really closely with a group, [0:20] Karan Singhal: a cohort of around 250 physicians [0:22] Karan Singhal: across every stage of generation of this data. [0:25] Dr. Nate Gross: And we're starting to see medications [0:27] Dr. Nate Gross: that have been sitting on a shelf [0:28] Dr. Nate Gross: that all of a sudden AI has found ways for them to have direct value in patient lives. [0:38] Dr. Nate Gross: How did you find your way into healthcare? [0:40] Dr. Nate Gross: So what drew me to healthcare initially was health policy. [0:45] Dr. Nate Gross: I was very interested. [0:47] Dr. Nate Gross: This was before the first Obama election. [0:51] Dr. Nate Gross: Value-based care was first becoming a thing. [0:53] Dr. Nate Gross: I started studying different ways to make healthcare more accessible to more people. [1:00] Dr. Nate Gross: And then eventually went to Emory for medical school. [1:04] Dr. Nate Gross: And what drew me to that was a large public hospital, Grady Hospital, [1:10] Dr. Nate Gross: to make sure that you're taking advantage of every clinical hour you have. [1:13] Dr. Nate Gross: So what kind of things were you doing? [1:15] Dr. Nate Gross: So I was mostly pissing off the IT department. [1:19] Dr. Nate Gross: When I was in medical school, the news feed came out, the iPhone came out. [1:23] Dr. Nate Gross: Twitter came out, the App Store came out. And so comparing the technology that we had as doctors, [1:31] Dr. Nate Gross: which was fax machine, clipboard, paper binder, the beginnings of electronic health records to [1:36] Dr. Nate Gross: what my friends had or what the patients had in the waiting room was pretty profound. [1:40] Andrew Mayne: So you come at it from the point of view as an AI researcher, [1:43] Andrew Mayne: where did your interest in applying this to healthcare come from? [1:47] Karan Singhal: So I nerded out a lot when I was younger about things like philosophy of mind. And I thought a [1:51] Karan Singhal: about you know intelligence and how far could intelligence go and could machines be intelligent [1:58] Karan Singhal: and a lot of those explorations took me towards as i was learning about AI and starting to work [2:03] Karan Singhal: on my first AI projects thinking a lot about the ways in which AI could have a lot of impact on [2:08] Karan Singhal: humanity in the future and i thought something like i didn't i didn't predict the future or how [2:12] Karan Singhal: fast it would happen but i thought something like agi would happen within our lifetimes so then once [2:17] Karan Singhal: once i had that conviction i thought a lot about you know what are the ways in which i can have [2:21] Karan Singhal: either positive impact and hopefully make that a really large upside for humanity or think about [2:26] Karan Singhal: the ways in which we could avoid downside. So since then in my career, I've been thinking a lot about [2:31] Karan Singhal: both sides of that coin, thinking about that from the perspective as a safety researcher, [2:35] Karan Singhal: which is part of my background. And then really some of that work on safety and privacy that I was [2:39] Karan Singhal: working on previously, I started applying it in healthcare. And then I started being like, [2:43] Karan Singhal: whoa, there's a really massive opportunity to think about the application of this technology, [2:47] Karan Singhal: especially large language models in healthcare. And that's what took me to transitioning to it [2:52] Karan Singhal: full-time was just the size of that opportunity and the fact that I felt like the healthcare and [2:56] Karan Singhal: clinical AI world was kind of not fully aware of that gap. And so I just thought it was kind of a really amazing opportunity and responsibility to bring us there. [3:02] Andrew Mayne: I want to understand both the [3:07] Andrew Mayne: vision and actually how this is going to be implemented. [3:12] Dr. Nate Gross: So our mission at OpenAI is to ensure that AGI benefits all of humanity. And health is one of the places where I think that is [3:19] Dr. Nate Gross: not only most achievable, but is the clearest. So healthcare today, as everyone knows, is fragmented, [3:29] Dr. Nate Gross: care is missed, left and right. Patients are often left 364 days per year without the opportunity to [3:37] Dr. Nate Gross: engage with the organizations that have the information centralized. And doctors have [3:46] Dr. Nate Gross: extremely limited time when they do get that chance to engage with the patient to actually have [3:52] Dr. Nate Gross: a meaningful impact beyond a simple surgery or a simple reactive prescription. The system is [3:59] Dr. Nate Gross: more reactive than it is proactive today. And that leads to tremendous challenges in the system. [4:08] Dr. Nate Gross: It leads to tremendous gaps in care. It leads to leaving people behind in situations when they [4:14] Dr. Nate Gross: could be thriving. And one of the reasons that I joined OpenAI is because access has always been [4:22] Dr. Nate Gross: a through line in my life. Access to knowledge, first in medicine, then in building a product for [4:30] Dr. Nate Gross: doctors to access the latest medical literature, and then in supporting entrepreneurs as they [4:35] Dr. Nate Gross: were building healthcare tools. But OpenAI has the type of technology that can do that at scale [4:42] Dr. Nate Gross: for the entire ecosystem all at once, help patients, help healthcare professionals, [4:48] Dr. Nate Gross: and help incredible entrepreneurs who are building for all of the corners and edge cases and tough [4:55] Dr. Nate Gross: challenges that exist in each area of the health market. [5:01] Andrew Mayne: What is the strategy here? We know that people use chatbots all the time now for medical questions, but it seems like you're building and [5:06] Andrew Mayne: working towards something bigger and more comprehensive, not just for the patient side, [5:10] Andrew Mayne: but the clinician side. Could you talk about what your goals are? [5:16] Dr. Nate Gross: Patients are increasingly turning to tools like ChatGPT throughout the year. In fact, 900 million people now use ChatGPT per [5:23] Dr. Nate Gross: week. And if you look at how many are doing health related queries, it's about one in four [5:30] Dr. Nate Gross: in a given week. So that's 40 million people per day. And so our strategy in health is as much [5:38] Dr. Nate Gross: proactive as it is reactive and stepping up to the responsibility and the opportunity to do good [5:45] Dr. Nate Gross: that comes with that strong consumer demand. And so with ChatGPT Health, we have created a space [5:53] Dr. Nate Gross: keep these conversations not just secure, but empowered. So when I say secure, of course, [6:01] Dr. Nate Gross: encrypted with this essentially one-way valve protecting your conversations. So these extra [6:07] Dr. Nate Gross: security layers, these protections to make sure that we will never train on users' healthcare data, [6:13] Dr. Nate Gross: combined with empowerment, really. Search engines that people have used before to navigate health [6:20] Dr. Nate Gross: have amnesia. They're one size fits all. And I think context really matters in healthcare. And so [6:28] Dr. Nate Gross: building a series of features and technology hooks to help patients bring in their own context that [6:37] Dr. Nate Gross: they choose to, so that each time they choose to engage with AI, it's grounded in their own context [6:42] Dr. Nate Gross: is a key reason why we've built this ChatGPT for Health Foundation. [6:47] Andrew Mayne: So I understand the safeguards you put in place to keep the data separate and to make sure that you don't get leaked between there and to be able to undergo a very rigorous method of making sure that your data is secure. [6:57] Andrew Mayne: But when it comes to the model itself, what comes into training models that are capable of working with something like health care? [7:04] Andrew Mayne: It's kind of the most important thing in the world. [7:06] Karan Singhal: For sure. It's a high stakes domain. [7:08] Karan Singhal: And because of the use of that, that people are doing, it's super important that we get it right. [7:13] Karan Singhal: So we think a lot about a few things when we think about evaluation and training for healthcare. [7:17] Karan Singhal: And this is actually the foundation for the work at health at OpenAI. [7:21] Karan Singhal: When we were first starting to work on the health effort at OpenAI, we were thinking a lot about the safety and grounding motivation as an important part of what we were doing. [7:30] Karan Singhal: And so part of the thesis actually for starting work on health at OpenAI was thinking, this is an excellent way to ground our work in safety and alignment and provide kind of concrete incentives and feedback loop for researchers who think about this problem. [7:43] Karan Singhal: So the model improvements and the safety thinking here is not just an afterthought. [7:46] Karan Singhal: It's actually the beginning of our work here. [7:48] Karan Singhal: And so where we started really was thinking about evaluation. [7:51] Karan Singhal: So can we think about the ways in which models were already starting to become useful to people then? [7:56] Karan Singhal: And there's already starting to be this capability overhang between what the models could do and what people were using them for. [8:02] Karan Singhal: And so we started to navigate that problem and think about where do the models still have gaps today? [8:07] Karan Singhal: And so that's where our work on evaluation comes in. [8:10] Karan Singhal: And so we've taken a pretty methodologically interesting approach to that. [8:16] Karan Singhal: And a lot of that has reflected in our work in HealthBench, which is this kind of realistic evaluation of conversations between users who are either health professionals or consumers talking to models and measuring their performance and safety of the models in these situations, which are these kind of multi-turn conversations. [8:36] Karan Singhal: And the way we worked on this is we actually worked really closely with a group, a cohort of around 250 physicians that we worked with to kind of across every stage of generation of this data from thinking about the ways in which, you know, the areas that we would focus in for the evaluation and the areas that we thought about were going to be clinically relevant or impactful to the specific, you know, what are the specific things that are being graded in this evaluation? [9:04] Karan Singhal: So that's like a range of things from, you know, are you tailoring your response to a layperson versus a more technical health professional? [9:12] Karan Singhal: Are you thinking about the ways in which you should see context first before providing an initial response? [9:18] Karan Singhal: The models used to be significantly or are much better today at kind of seeking context when needed because users are typing in, you know, much less than the models often need to be able to provide information that's most helpful. [9:32] Andrew Mayne: It burns. [9:33] Karan Singhal: Exactly. [9:34] Karan Singhal: You know, if a user types in, it burns. [9:36] Karan Singhal: You know, how do you think about the right way to provide information? [9:38] Karan Singhal: You can provide some initial information potentially based on an impression you might have of what the user might be saying. [9:45] Karan Singhal: But the most helpful thing to do in that situation and the safest thing to do in that situation is actually to ask for more context. [9:50] Karan Singhal: So that's just one example of the many ways that we kind of measured performance in HealthBench. [9:54] Karan Singhal: And HealthBench in particular actually measured around 49,000 different dimensions of performance. [9:59] Karan Singhal: And that's just an example of one possible dimension of performance. [10:03] Karan Singhal: So it's a very multifaceted evaluation that we built kind of in concert with this cohort of 250 physicians over a long period of time. [10:10] Karan Singhal: And it took us about a year actually end to end to work on that evaluation and then release it. [10:15] Andrew Mayne: In kind of the model development cycle, it seems like sometimes some company gets a bit ahead and somebody comes up and catches up and whatnot. [10:24] Andrew Mayne: I've noticed a pattern with the OpenA health models. [10:26] Andrew Mayne: They've consistently been far ahead in HealthBench and other evals that like by a big margin. [10:32] Andrew Mayne: Why is that? [10:33] Karan Singhal: I think we have a pretty dedicated effort here and a pretty serious effort that is cross-functional and kind of across the stack. [10:41] Karan Singhal: Everything from kind of pre-deployment evals to like HealthBench to monitoring in production traffic and thinking about the ways in which we are ensuring safety in production traffic in a privacy-preserving way. [10:55] Karan Singhal: And working with physicians across every step of that process. [10:58] Karan Singhal: And so to my knowledge, OpenAI's models are the only major models where every phase of model training from pre-training to mid-training to post-training and every step in between really integrates health into every major stage. [11:12] Karan Singhal: And I think the result is that our models are pretty good, not just on our own benchmarks, but also the benchmarks that other people put together. [11:18] Dr. Nate Gross: I'd like to add a little to what Karan said about the model training, because I think when we spend time with the healthcare ecosystem, that's one of the things that is most important to them. [11:30] Dr. Nate Gross: So not only were these models trained in development with hundreds of physicians who created over 5,000 conversations and 48,500 rubric criteria through which to evaluate AI responses and score them and identify ways that we could improve the model, do additional data acquisition, do additional post-training, hone in on a particular subspecialty or particular area of the world. [12:00] Dr. Nate Gross: where users were telling us we could improve health or healthcare in that specific topic. [12:09] Dr. Nate Gross: But in addition, I think that close proximity to physicians really leads to calling out the [12:16] Dr. Nate Gross: most important parts that should be focused on in model development. So, [12:22] Dr. Nate Gross: you know, other places, sometimes I see how a model fared on a medical school exam or a board exam. [12:30] Dr. Nate Gross: And healthcare is not multiple choice. [12:33] Dr. Nate Gross: You know, patients are coming in with a tremendous amount of complexities and their own stories and nuance and context. [12:41] Dr. Nate Gross: And that's presented in many different ways. [12:44] Dr. Nate Gross: And part of the job of working in healthcare is being able to draw from those disparate sources, draw from experience, balance all that in your head. [12:52] Dr. Nate Gross: And so having a training mechanism that thinks about things like when to escalate and how to escalate and keep that always as the top priority or adaptive literacy. [13:04] Dr. Nate Gross: I mean, compare the one-size-fits-all handouts that people get when they visit the doctor today to a model that can respond differently when it knows you're an oncologist versus a primary care doctor versus a pharmacist in Kenya versus a patient at the 12th grade literacy level or the third grade literacy level is extremely important for not only making sure that accuracy and impact is maximized, [13:32] Dr. Nate Gross: but also just to make sure that everyone can maximally participate in their own care on the patient side. [13:39] Dr. Nate Gross: And then finally, uncertainty. [13:41] Dr. Nate Gross: If you go back a year and a half ago, many of the mistakes people would call out about AI models were overconfident hallucinations. [13:50] Dr. Nate Gross: And I think in such a high stakes field like healthcare, one of the most important things is that the model can be trained to better know when it doesn't know and say that. [14:03] Dr. Nate Gross: And in addition, suggest follow-up that can be dug into either by the patient in a referral to the healthcare system or by the doctor if the doctor is using the model, a test that they might run, additional pathways they may go down to make sure that the patient can be led to the best possible outcome. [14:22] Andrew Mayne: We've seen the cost of intelligence drop every year, and it's exciting because every year you're able to get better answers, medicine, everything, healthcare across the board. [14:33] Andrew Mayne: But what are the challenges? [14:34] Andrew Mayne: What are going to be the blockers or what are you looking at ahead to say that, okay, we have to solve for this? [14:39] Karan Singhal: The drop in cost of intelligence has been super exciting here because so much of what we think about and care about here is actually about access. [14:46] Karan Singhal: And so the more people have access to technology, the more people will benefit. [14:50] Karan Singhal: And that's why we're working on rolling out ChatGPT H ealth more widely to all free users. [14:57] Karan Singhal: And so that's incredibly exciting. [15:00] Karan Singhal: Another thing that we think about as researchers is like, where will the marginal gains intelligence compound the most? [15:05] Karan Singhal: Right. And so I think Nate mentioned this exciting thing, which is like there is more and more data that is being collected that is across different modalities. [15:12] Karan Singhal: How do you think about integrating that data across all the different ways that people use Chat GPT and all the different modalities and wearables and things like this that people are collecting? [15:20] Karan Singhal: lab tests, things like this. And that's one place where I think a lot of the intelligence will [15:24] Karan Singhal: compound and we'll start to see kind of new zero or one capabilities. Like a model looks at my entire [15:29] Karan Singhal: history over a decade and tells me a prediction that even a human couldn't have because it's just [15:35] Karan Singhal: the model has a higher context size. So thinking about those zero to one capabilities, I think are [15:39] Karan Singhal: going to be really cool. The other thing we keep in mind is just like, how are people thinking about [15:44] Karan Singhal: and using ChatGPT today? Can we measure that? Can we improve that? And I think we're kind of [15:50] Karan Singhal: this interesting point right now. I call this to our team, the transition where, you know, for [15:56] Karan Singhal: context, I bike to work and I bike to work. I wear my helmet. I worry about cars and things like this [16:02] Karan Singhal: next to me. I just reached the point here in SF, you know, in SF, we have a bunch of self-driving [16:07] Karan Singhal: cars, including Waymo's. I just reached the point where, you know, when I'm biking next to Waymo, [16:12] Karan Singhal: I actually feel safer than if I was biking next to a human driver, right? I don't worry about [16:16] Karan Singhal: whether I'm in their blind spot or not or anything like this. So I feel this protective effect by [16:21] Karan Singhal: being next to the Waymo. And I want everybody to have this protective effect, right? I want [16:25] Karan Singhal: everybody to have this protective effect with health AI. There are these studies showing that, [16:29] Karan Singhal: you know, if you have a doctor in your family, that adds a protective effect to your health as [16:33] Karan Singhal: well. And I want everybody, whether they're patient or health professional, to think about the ways [16:39] Karan Singhal: in which as a patient, you want to feel safer having this. [16:45] Karan Singhal: As a health professional, you want this to be a safety net [16:48] Karan Singhal: for the decisions that you're making. [16:50] Karan Singhal: So that's another frontier that I think we're going to cross [16:52] Karan Singhal: in the next six months or so, which is really exciting, [16:54] Karan Singhal: this kind of inflection point. [16:57] Karan Singhal: Another thing that we're thinking about is kind of the right ways [17:00] Karan Singhal: to think around post-deployment monitoring of certain workflows. [17:04] Karan Singhal: And I think a good example here that I'd love to talk about [17:07] Karan Singhal: is our AI clinical co-pilot study that we did with PandaHealth. [17:13] Karan Singhal: This was a study where we worked with these 20 or so clinics in Nairobi [17:18] Karan Singhal: and actually thought about the ways in which we can deploy a safety net [17:21] Karan Singhal: for clinicians in that context, [17:23] Karan Singhal: which is basically monitoring things that they type [17:26] Karan Singhal: into their electronic health record [17:28] Karan Singhal: and only interrupting their flow [17:30] Karan Singhal: when there's something potentially concerning that's going on [17:34] Karan Singhal: or a potential error or things like this. [17:36] Karan Singhal: And what we found is that when we deployed this to clinicians in the setting, that there was actually a statistically significant reduction in diagnostic and treatment errors for the clinicians who are using this tool versus not. [17:48] Karan Singhal: And I think this is a step in the direction of moving beyond kind of model evaluations and even monitoring of the ways in which people are thinking about using ChatGPT today to actually thinking about workflows in which these technologies can be deployed and the right ways to evaluate those workflows after deployment. [18:03] Karan Singhal: I think that's another frontier that we are really excited about and would love to see more from our partners. [18:08] Andrew Mayne: Nate, what do you think the challenges are going to be? [18:11] Dr. Nate Gross: I'll start with talking through some of the challenges that exist on the professional side. [18:17] Dr. Nate Gross: So each day when healthcare professionals use AI, they're looking for the ability to trust what they're seeing in the answer. [18:29] Dr. Nate Gross: And so a lot of our recent work has been making sure that answers that the AI is providing are not just grounded in what the model was training on, but are grounded in the latest medical literature, the latest guidelines. [18:43] Dr. Nate Gross: And sometimes the latest guidance from their own institution or their own region. [18:50] Dr. Nate Gross: Some conditions are treated differently in different areas of the country. [18:54] Dr. Nate Gross: Other times, different care settings have different levels of resources, different levels of specialists and additional services on hand. [19:04] Dr. Nate Gross: And it can be helpful as a health care professional to be able to quickly navigate that and come up with completely personalized care plans. [19:12] Dr. Nate Gross: And so building connectivity within ChatGPT to not only be HIPAA aligned and be used in [19:19] Dr. Nate Gross: these secure environments, but also be able to combine sensitive information with the [19:26] Dr. Nate Gross: latest medical knowledge, I think is a great path that we've started down and something [19:32] Dr. Nate Gross: that will continue to keep trust as the top priority between how healthcare professionals [19:40] Dr. Nate Gross: engage with AI. [19:42] Dr. Nate Gross: So I think one of the other challenges is that the systems themselves in healthcare are quite siloed, both at an organization level, but also at the tools that have to be used within each organization. [19:57] Dr. Nate Gross: AI thus far has been deployed on really a point solution basis in the technology industry, but increasingly the connectivity is becoming available to connect the dots between the hundreds of different systems, some analog, some digital, some structured, some unstructured, many decentralized, many not on the cloud. [20:19] Dr. Nate Gross: Being able to connect all of those through unified AI layers to actually make sure that patients and information isn't falling through the cracks and that the connectivity can be maximized to actually bring the greatest amount of impact. [20:36] Dr. Nate Gross: That's hard in healthcare and it's certainly not something that we can say is solved. [20:40] Dr. Nate Gross: But with many of our recent products ranging from ChatGPT for healthcare and its connectivity to apps and connectors, to the OpenAI API for healthcare, to our Frontier Foundation for models and agents, we think increasingly there's going to be an opportunity to really accelerate what is possible within the healthcare system and what agents can achieve. [21:05] Andrew Mayne: Part of this seems like it's very collaborative working with the healthcare industry. And I noticed when using the ChatGPT Health app, the first thing I did was able to put in my records and get all of that. And it looked like there was a lot of just cooperation working across the sort of ecosystem to do this. How has that come to be? Where is this headed? [21:24] Dr. Nate Gross: It's extremely important that all of the healthcare system has an equal chance to contribute and engage nationally and internationally with providing the context that will help empower patients to receive the best possible answers from ChatGPT. [21:46] Dr. Nate Gross: And so on the electronic health record side, this means working with the government and Centers for Medicare and Medicaid Services, adopting national standards for electronic health record syncing so that patients in just a few taps are able to bring in their context in consented ways. [22:06] Dr. Nate Gross: It's being able to tap into existing standards like mobile phones and the most popular consumer health products and the most popular biosensors and wearables to make sure, again, in just one or two taps, patients are able to not only bring in that information, but leverage it in thoughtful ways, in ways that may not have been possible without the combined set of data that can exist in this sort of ecosystem. [22:35] Dr. Nate Gross: So, for instance, being able to reference your recent exercise activity when making a plan of how to spend your evening or being able to even do things as simple as, you know, reference your overnight sleep and stress when your agent is helping you set your calendar for the next day and what tasks you take on first. [23:02] Andrew Mayne: It's very exciting. [23:03] Andrew Mayne: You know, I have, you know, we're a smart ring, a watch, whatever, but I get this data and all I kind of have in my apps are rings to look at and go like, OK, I guess it's doing something. [23:12] Andrew Mayne: Being able to plug in ChatGPT has been fantastic because now I'm able to ask those kinds of questions. [23:16] Andrew Mayne: But that's very exciting. What you talk about, too, is if you get a plan from your doctor suggestions is literally say, hey, I didn't walk enough yesterday. [23:24] Andrew Mayne: What should I do today? I've had to be really good at menu planning and literally go on this menu film, tell me what to order and whatnot. [23:30] Andrew Mayne: And so you're saying we're just going to get more of that and much better. [23:34] Dr. Nate Gross: Yeah. And that's why our partnerships, I believe, are so important, because in these instances, ChatGPT doesn't replace the incredible technology that our partners are building to go deep on health insights for a particular wearable. [23:50] Dr. Nate Gross: But our surface area, our opportunity to bring in that health information can now extend to the many different ways people use ChatGPT, such as what they're going to cook for dinner or how they're going to plan their afternoon. [24:07] Dr. Nate Gross: You know, sometimes I think of two patients and one patient has to navigate the healthcare system by themselves. [24:16] Dr. Nate Gross: And the other patient maybe has a spouse come with them. [24:19] Dr. Nate Gross: And that spouse has a clipboard and used to work as a healthcare professional and is very attentive, if not neurotic, and can follow up on details and is connected to your personal calendar. [24:31] Dr. Nate Gross: And the best aspects of that with consent for the patients that want to, I think, represents a future where we can make it easier and easier for patients to follow care plans, to play active, captain-like roles in their own health in partnership with their care teams and their physicians. [24:53] Dr. Nate Gross: And I think if we can remove a lot of the friction that historically exists between those processes, whether it's just information not following or there's a lot to keep track of or a lot of old information to parse and bring in, we can do a tremendous amount of good or we can help patients themselves be empowered to do a tremendous amount of good in their own care plans. [25:18] Andrew Mayne: And you know, as a physician, that it's hard to give as much time as you would like, because you're going to always have more patients you have to deal with than you have hours in the day. And it's interesting to see a kind of a technology that has infinite time, infinite patience to be able to do that as a complement to that. [25:33] Dr. Nate Gross: I mean, if there's one thing that healthcare professionals are short on, it's time. [25:37] Dr. Nate Gross: So when we think about our role internally at OpenAI, we often break down the work that we're doing into three buckets. [25:48] Dr. Nate Gross: Raise the floor. [25:49] Dr. Nate Gross: So make sure that AI and the benefits of AI are accessible to everyone. [25:54] Dr. Nate Gross: And that could be patients, that could be healthcare professionals and others working in health-related industries. [25:59] Dr. Nate Gross: sweep the floor, which means help doctors and help other health professionals save time from [26:07] Dr. Nate Gross: the tremendous administration and bureaucratic burdens that they have every day so that they can [26:12] Dr. Nate Gross: spend more time with their patients. And then thirdly, raise the ceiling. The [26:18] Dr. Nate Gross: impact that AI can have in healthcare, I think will allow us to look back on this space in a few [26:25] Dr. Nate Gross: years and say, wow, we have all accelerated together in a way that medicine is still in [26:33] Dr. Nate Gross: the driver's seat, but is also far more empowered than ever before. [26:37] Andrew Mayne: Yeah. [26:38] Andrew Mayne: I don't think anybody feels like their doctor spent too much time with them. [26:42] Andrew Mayne: So it looks like this is going to be helpful to solve for that. [26:45] Andrew Mayne: What has been your favorite aha or wow, or this is really cool moment in the intersection [26:50] Andrew Mayne: of AI and healthcare? [26:52] Karan Singhal: I'll answer your question in a non-standard way, which is I think the most amazing thing to see for me in the last year has been the rate of adoption of health, actually even beyond the ChatGPT Health product before we announced the ChatGPT Health product. [27:05] Karan Singhal: It's been one of our fastest growing use cases is kind of health and wellness questions. [27:09] Karan Singhal: And we shared that hundreds of millions of people a week are starting to use ChatGPT for health and wellness. [27:15] Karan Singhal: I think seeing that rapid growth, especially coming from a background of being motivated to work on this problem because it felt like healthcare and clinical AI world were not super aware of the potential of LLMs in healthcare and seeing how far we had come, I think has been a really special moment for me. [27:29] Dr. Nate Gross: There's no doubt that the adoption of this technology and the fact that it is increasingly collaborative with the healthcare system, it is increasingly driving feedback loops back to us to improve the models is the most meaningful thing and the most mission aligned thing. [27:44] Dr. Nate Gross: But what I also get excited about is what our research team is increasingly able to give back to them using that feedback. [27:53] Dr. Nate Gross: And not only is it the capabilities of the models, but it's what can be unlocked once those models are allowed to run longer and have more context. [28:05] Dr. Nate Gross: And we're starting to see discoveries of medications that have been sitting on a shelf that all of a sudden AI has found ways for them to have meaningful and direct value in patient lives. [28:19] Dr. Nate Gross: It is starting to scale experiments that we as individuals wouldn't have been able to juggle on our own. [28:27] Dr. Nate Gross: And that partnership combined with that increased capability to finally move from being interesting to being useful and increasingly to being transformative is, I think, what is the most exciting thing for us heading into this year. [28:44] Andrew Mayne: Now that you've been working on this for some time, you've been engaging with clinicians and talking to people helping deploy this. [28:50] Andrew Mayne: What has been some of the feedback you've seen? [28:51] Karan Singhal: I think the experience of flying to Nairobi and seeing the clinicians using the tool and the ways in which we did this thing, which we call active change management, where we worked really closely with these clinicians and flew to Kenya a couple of times to think about the ways that we could deepen their workflows using the AI tool and make it something that not only made sense to them, but actually became kind of something that was indispensable for them. [29:19] Karan Singhal: And so as we were concluding the study, the team was actually thinking about, the team at PandaHealth was thinking about potentially running another study. [29:28] Karan Singhal: And they actually had a lot of hesitance around running another study because that would have involved having some group of clinicians using AI and some group of clinicians not using AI. [29:37] Karan Singhal: They actually felt that it was dangerous to have a group of clinicians not using the AI. [29:40] Karan Singhal: And so that's the point at which I was like, wow, we have done something major here. [29:44] Dr. Nate Gross: I think that the stories that we get back from our members every day are one of the most meaningful parts of the job. And these are from caregivers that are increasingly under strain, taking care of family members, trying to navigate their own health at the same time. [30:02] Dr. Nate Gross: This is from doctors and nurses who are truly overloaded every day, and we can help them extend their expertise and, you know, compress the tough parts of their day a little bit more. [30:17] Dr. Nate Gross: And then sometimes, and this is more rare, but increasing, it's the miracle cases. [30:23] Dr. Nate Gross: It's the patient who had been bouncing around the system for years, the unsolved diagnosis, the emergency where information wasn't present. [30:33] Dr. Nate Gross: And suddenly being able to step in and assist and accelerate and bring people into the care that could really help is truly a privilege. [30:44] Andrew Mayne: It's exciting. [30:45] Andrew Mayne: It's an amplifier, and every doctor I know wants to be able to do more for their patients. [30:50] Andrew Mayne: Thank you very much. [30:51] Andrew Mayne: This has been very interesting, guys. [30:53] Dr. Nate Gross: Thank you.