Takeda Rebuilds GenAI with Persona Memory to Fix Clinical Search Flaws

武田制药以角色记忆重构生成式AI，修复临床检索缺陷 通用检索模型难以满足制药工作流需求，因标准搜索算法忽略业务操作语境，因而需要能够主动识别证据缺口并映射科学决策的推理循环。 Preethi Ramesh 阐述了医疗健康领域标准 RAG 流程的实际局限性，指出通用总结与简单的引用指标无法满足严苛的临床及监管标准。为此，她介绍了一种内部 Sensemaking AI 架构，该架构围绕由“观察、推理、建议与适应”组成的四步认知循环构建，并以角色、证据和轨迹记忆层以及 Databricks 评估流程为支撑。通过对样本量、试验阶段和人口学多样性等临床参数赋予权重，该系统从被动搜索引擎转变为主动的科学辅助系统，能够识别证据缺口并适应跨市场的文化差异。其最终愿景是将领域特定的 AI 协作伙伴嵌入企业工作流，以提供高度可信、具可操作性的决策支持。 武田制药的 Preethi Ramesh 指出当前 AI 生态分为两派：大科技公司聚焦算力与基准竞赛，而制药等企业则致力于构建理解领域情境与多模态数据的“感知推理 AI”（Sensemaking AI）。 (0:04) 传统的 RAG 检索管线由于缺乏情境感知，在强监管与患者至上的药企环境中容易输出脱离实际或幻觉内容，进而损害专业信任。 (2:14) 科学文献评估需从学术声望转向临床感知信号，例如优先考虑 IV 期研究、超 10 万人的样本量以及多国与人群多样性。 (5:15) 系统通过“画像卡片”（Persona Cards）将业务目标与决策标准直接注入“观察、推理、推荐、适应”的闭环，并以人物画像、证据和追踪三层记忆架构保障推理一致性与可追溯性。 (7:47) 工程落地采用分流架构降低成本并确保合规：轻量模型负责预处理，旗舰 LLM 负责最终综合，并利用 Databricks 平台进行安全防护与自动化评测。 (20:57)

Preethi Ramesh of Takeda Pharmaceuticals argues that artificial intelligence delivers genuine value in life sciences only when systems shift from raw foundation model compute toward domain-aware 'sensemaking' loops that replicate how clinicians and commercial leaders evaluate evidence. Replacing Generic RAG with Scientific Ranking (5:15) Standard Retrieval-Augmented Generation (RAG) pipelines rely on semantic similarity and keyword density, which often surface highly cited academic papers from elite institutions. When deploying early prototypes for gastrointestinal research, Ramesh found clinical teams rejected these outputs because academic citation counts do not reflect clinical relevance. Clinicians require real-world applicability rather than theoretical prestige. To bridge this gap, Takeda re-engineered its retrieval ranking to prioritize clinical trial metadata: Phase 4 post-marketing studies, sample cohorts of 100,000 or more patients, multi-country trial sites, and diverse demographic representation. Furthermore, to maintain regulatory compliance and stakeholder trust, the system is constrained to prevent hallucination; when participant figures are missing from a source text, the engine is explicitly programmed to label the cohort size as insufficient data rather than estimating. Encoding Strategic Priorities in Memory Layers (7:47) To adapt outputs to specific enterprise functions, Ramesh introduced 'Persona Cards' for roles spanning commercial strategy, medical communications, publication planning, and medical affairs. These cards define operational objectives, geographic scopes, and an individual's top three annual performance goals, aligning the model with corporate brand priorities. This contextual information is operationalized through three distinct memory layers: Persona Memory (tracking user roles, preferences, and market responsibilities across sessions), Evidence Memory (cataloging verified text chunks and study cohorts), and Trace Memory (auditing the reasoning process and system evaluations). This structure dynamically shifts reasoning based on product maturity. Inquiries for newly launched treatments focus on baseline mechanisms of action and trial endpoints, whereas queries for mature brands marketed for over a decade automatically pivot toward combination therapies, safety profiles, and lifecycle management. Cost-Optimized Routing and Multi-Agent Orchestration (20:57) Running complex multi-agent workflows in LangGraph introduces computational latency and high token expenses. To manage these operational costs while upholding compliance, the pipeline routes incoming traffic through the Databricks AI Gateway to filter personally identifiable information before persona-driven query rewriting occurs. Workloads are then programmatically segmented across models. Ingestion, chunking, and routine preprocessing run on cost-effective CPUs or smaller GPU instances, reserving flagship large language models strictly for high-impact final synthesis. Databricks synthetic generators create automated follow-up prompts to detect cross-organizational inquiry trends, while Databricks Mosaic AI acts as an automated judge evaluating question-and-answer pairs against clinician feedback. Proactive Gap Analysis and Cultural Roleplay (26:33) Rather than acting as passive summarizers, domain-aware models are engineered to serve as critical evidence coaches. In a simulated workflow for a medical strategy lead preparing for a conference, the reasoning engine retrieves relevant inflammatory bowel disease studies while actively warning the user about strategic gaps—such as the absence of data on specific commercial competitors, missing long-term safety endpoints, or early study termination risks. The architecture also incorporates cultural roleplay agents to tailor scientific communication to local markets. For example, in Japanese pharmaceutical engagement, effective messaging requires deep sensitivity to specific population cohorts. The reasoning engine evaluates scientific evidence against local norms, advising field representatives to highlight studies focusing on elderly demographics to drive meaningful discussions with healthcare professionals. Ramesh outlines a long-term roadmap that moves beyond standalone document generation by integrating AI directly into CRM pipelines and automated regulatory review workflows. Success will be evaluated not by traditional computing metrics, but by 'sensemaking KPIs' measuring synthesis speed, strategic decision quality, and end-to-end evidence traceability.

武田制药（Takeda Pharmaceuticals）全球数据、数字与技术总监 Preethi Ramesh 指出，制药领域的生成式 AI 竞争核心不在于堆叠底层算力与通用模型规模，而在于构建具备“意义建构”（Sensemaking）能力的领域认知推理闭环。该架构旨在超越传统的检索增强生成（RAG），使系统能够依据具体的临床情境、业务角色与战略目标主动分析证据并揭示数据盲区。 通用模型的语境缺陷与临床评价差异 (2:14) Ramesh 表示，当前人工智能生态分化为两大阵营：科技巨头投入大量资金竞逐更庞大的 GPU、CPU 及基准测试指标，而制药等行业实践者则需要从模型中挖掘具有真实业务价值的上下文意义。制药行业拥有成熟且跨越早期研发到商业化落地的庞大数据资产，获取数据不再是瓶颈，核心痛点在于通用大模型虽然掌握行业词汇，却缺乏对“提问者身份与业务情境”的感知。在高度合规与患者至上的制药场景中，传统生成式 AI 原型所产生的幻觉或错误结论会直接破坏专业信任，并带来严峻的合规与临床后果。 Ramesh 结合自身经验以餐厅点评筛选作比喻指出，普通的相似度或高引用量排序无法满足专业需求。在 2023 年构建的早期系统中，算法根据引用量优先检索出哈佛或麻省理工学院的高影响力文献，却遭到临床团队彻底否定。临床科学家在评估炎性肠病（IBD）等文献时，优先考量的是试验阶段（优先关注监测长期有效性与安全性的 IV 期研究）、样本规模（如包含 10 万名患者的大队列）以及多中心跨国与人群多样性。为此，检索排序引擎必须转向深度贴合临床逻辑的元数据加权。 角色画像卡与三层记忆架构 (7:47) 为解决不同业务职能关注点脱节的问题，团队借鉴商业营销中的用户画像方法，为商业策略、医学沟通、出版规划和医学事务等不同角色定制了结构化的“角色卡”（Persona Cards）。这些角色卡明确记录了各个岗位的业务目标、核心交付物、决策评价标准以及预期的商业影响。将这些定义作为语境直接注入推理系统后，AI 能够动态调整检索策略与证据评估逻辑，实现从静态文本概括向定制化领域推理的跨越。 在架构设计上，系统通过三层记忆模块维持多轮交互与长期任务的语境连贯性。角色记忆（Persona Memory）追踪用户的固定属性、跨会话偏好及所负责的地理市场，甚至能将品牌年度前三项核心战略目标内嵌其中，使推理保持长达 12 个月的战略对齐；证据记忆（Evidence Memory）集中管理和索引已验证的文献文本块、研究队列与内部资料，确保引用的准确一致；轨迹记忆（Trace Memory）则完整记录推理步骤与历史评估，为结论的形成提供可审计的溯源依据。 四步认知闭环与生产级系统工程 (9:07) 系统在底层 RAG 架构之上构建了包含“观察（Observe）、推理（Reason）、推荐（Recommend）、适应（Adapt）”的四步内部认知循环。提示词工程经历了从早期零样本摘要、单样本角色与引用规范注入、基于临床特征（分期、样本量、新颖性）排序，再到区分优劣范例的少样本引导等演进阶段。为应对前沿靶点（如 TL1A）六个月内快速变化的趋势，系统必须无需额外提示即可自主识别新兴热点，并在此基础上计划引入能够预先过滤不良响应的多轮看门人智能体（Gatekeeper Agents）。 在工程落地层面，系统通过结合 Databricks 平台实现合规过滤与成本控制。Databricks AI Gateway 充当安全护栏以阻断个人身份信息（PII）泄露；模型生成的质量由 Mosaic AI Agent Evaluation 充当“AI 法官”，并结合人工反馈打分持续迭代。针对 LangGraph 多智能体工作流的高昂开销，系统建立了流量路由机制：利用轻量模型与低成本算力处理数据预处理与文本分块，仅将关键的最终综合任务分配给高参数旗舰大模型，同时利用合成数据生成高频问答以压缩调用延迟。此外，若原始文献未明确记录患者样本量，提示词逻辑强制系统标注“数据不足”，坚决杜绝数值臆造。 证据空白识别与跨文化角色扮演 (24:24) 在具体的业务场景演练中，例如全球医学战略负责人“Karen 医生”为学术会议准备炎性肠病沟通材料时，推理闭环不再仅是被动罗列匹配文章，而是演进为能够主动指出盲区的领域教练。当检索到头对头临床试验时，系统会主动提醒该研究对比的是非核心跟踪竞品，或提示某项新兴诊断研究面临随访提前终止的潜在风险，甚至标注文献缺乏长期安全性数据。这种揭示证据局限性与实时行业动态的能力，显著提升了医学专业人员的采纳信任度。 针对日本等具有鲜明文化特性的市场，系统引入了文化感知角色扮演机制。面对相同的文献集，角色扮演引擎能够模拟与医疗保健专业人士（HCP）的互动场景，结合当地偏好推荐以老年群体为侧重点的研究材料，从而契合高共情沟通的本土规范。根据 Ramesh 规划的演进路线，该架构未来将进一步实现与企业 CRM 系统的打通，使自动生成的宣教材料直接接入法规合规审批流；同时在评估体系上，不仅衡量传统商业回报，还将全面跟踪意义建构速度、决策质量与证据可追溯性等核心指标。 Ramesh 总结认为，制药企业引入 AI 的终极目标不是让大模型凭空想象业务世界，而是让 AI 深度协同并模拟人类专家的专业思考方式，打造出具备临床洞察力的 AI 医学顾问与商业策略伙伴。不过，演讲也反映出该体系仍面临多项未完全解决的工程与业务挑战：如何确保跨轮次看门人智能体对快速演进医学热点的实时把控，如何在深入跨国本地化时维持跨文化模拟的精确度，以及在多智能体高度复杂化后持续平衡算力开销与响应时延。

## Background 该内容由 YouTube 频道 Ai4 于 2025 年 8 月 27 日发布，主讲人为武田制药（Takeda Pharmaceuticals）全球数据、数字化与技术（DD&T）胃肠道治疗领域合作伙伴总监 Preethi Ramesh。页面简介指出，本次演讲主题为“为制药行业构建领域感知推理闭环”，旨在探讨生成式 AI 如何从单纯的搜索与摘要，演进为模拟科学家思维的“意义构建（Sensemaking）”认知闭环以输出可信洞察。 ## TL;DR 演讲深入剖析了传统生成式 AI 和通用 RAG 在制药强监管与复杂业务场景下的局限性，指出模型缺乏角色语境与临床推理逻辑。武田制药提出了涵盖“观察、推理、建议、自适应”的 Sensemaking AI 闭环架构，通过提示词工程演进、多层记忆体系（画像、证据、轨迹）以及注入核心战略目标，使 AI 具备专家级的情境感知能力。系统借助 Databricks 平台实现了路由改写、幻觉兜底与自动评测，不仅能在医学策略制定中精准识别人群样本与研究盲区（Gap Analysis），还能通过角色扮演跨越本土文化隔阂。最终，演讲呼吁行业超越对原始算力的盲目追求，打造出能够与人类科学家和业务专家同频思考的专业协同伙伴。 ## Atmosphere & style 整场演讲呈现出一种严谨务实、直击工业界痛点的技术老兵风格，没有空泛的术语堆砌，而是充满了对真实业务挫折与认知转变的深刻复盘。讲者以举重若轻的生动类比（如餐厅评论、教导幼儿）串联起硬核的技术架构，叙事逻辑清晰连贯，语调坚定且极具说服力。观众在聆听时能清晰感受到一种从喧嚣的“算力军备竞赛”抽离出来、转向追求“沉静而深思熟虑”的理性力量与对生命科学的敬畏感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=4s) 当前 AI 领域存在两极分化：大型科技巨头聚焦于算力扩张与基准跑分，而制药垂直行业的核心挑战在于如何构建“意义构建 AI”（Sensemaking AI）以实现深层业务价值。 - [02:14](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=134s) 传统 GenAI 与 RAG 模式在医药场景中受挫，核心痛点在于模型虽懂专业术语却缺乏具体的业务语境、角色身份认知与动机对齐，极易引发合规风险与信任危机。 - [04:48](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=288s) 医药工程师必须转变范式“像科学家一样思考”：临床决策往往不看重理论引用量，而是优先考量样本规模（如 10 万+ 患者）、多中心多国试验及 Phase 4 真实世界研究。 - [07:47](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=467s) 武田制药引入业务画像卡（Persona Card），系统化捕捉商业、医学与科学传播等不同职能的业务目标、影响力和思维盲区，将其转化为模型个性化推理的基石。 - [09:07](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=547s) Sensemaking AI 包含“观察、推理、建议、自适应”四步认知闭环，由提示词演进、记忆分层、领域知识库与 Databricks 基础设施四大工程支柱支撑。 - [11:26](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=686s) 提示词工程从无语境的 Zero-shot 迭代至 Few-shot 阶段，确立了以临床分期、样本量与新颖性（如 TL1A 靶点）为核心的医学文献排序与句子级溯源逻辑。 - [16:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=992s) 多层记忆体系包含画像记忆、证据记忆与轨迹记忆，通过注入年度战略目标和治疗领域周期上下文，确保 AI 长期锚定正确的业务航向。 - [20:57](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1257s) 工程落地中结合意图重写、缺失数据兜底防幻觉指令，并利用 Databricks 平台实现安全护栏（AI Gateway）、自动化裁判（AI Judge）以及降本分流路由（LLM Routing）。 - [26:33](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1593s) Sensemaking AI 具备从文献中识别“研究盲区与空白”（Gap Identification）的能力，能够主动提示竞品覆盖缺失或长期安全性数据不足，扮演专业辅导角色。 - [27:33](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1653s) 引入跨文化角色扮演机制，系统能针对日本等特定市场的文化敏感度（如优先选用老年患者素材）为一线沟通提供差异化的高影响力建议。 - [28:19](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1699s) 医药 AI 未来需建立专属的 Sensemaking KPI 评估体系，重点衡量意义建构速度、决策支持质量与证据可追溯性，并打通与 CRM 和合规审批的自动化闭环。 - [29:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1772s) 告别盲目堆砌算力，制药工业的核心追求是构建懂人类思维与业务逻辑的 AI 医学/商业团队伙伴，实现从简单搜索摘要到行动决策的跨越。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=4s) AI 世界的两极分化与制药场景下的“意义构建”（Sensemaking） 武田制药（Takeda Pharmaceuticals）全球数据、数字化与技术（Data, Digital & Technology, DD&T）胃肠道（GI）治疗领域合作伙伴总监 Preethi Ramesh 开门见山地介绍了自己的双重角色：她不仅拥有深厚的技术背景，实际利用 LangGraph 等智能体架构（Agentic Framework）在内部自研 AI 工具，更关键的是她深度扎根于业务团队，直面一线业务需求。在剖析当前生成式人工智能的格局时，她将整个 AI 世界划分为泾渭分明的两端：一端是由资金雄厚的大型科技巨头（Big Tech）主导的“算力军备竞赛”，各大巨头疯狂堆砌 GPU、CPU 算力并刷新基准测试评分（Benchmarking）；而另一端则是如何让这些强大的模型在真实产业中产生实际价值——这正是她所聚焦的领域，即所谓的“意义构建 AI”（Sensemaking AI）。 为了让非医药背景的听众理解这一探索的初衷，Preethi 梳理了制药工业复杂的价值链条。整个制药流程跨度极大：从早期的前沿研究与知识空白检测（Gap Detection），到严谨的临床试验项目，再到将临床知识沉淀为面向内外部受众的培训文档；从与医疗专业人士（HCP）的学术互动（例如向医生传递最新的生命周期管理（LCM）研究成果，说明某款药物在孕妇临床试验中的疗效已提升至 80%），直至产品在全球各监管辖区（如美国与其他国家严苛法规）的上市策略。制药行业从来不缺少海量数据，也完全掌握获取数据的途径，真正的痛点在于如何跨越“拥有数据”与“理解数据”之间的鸿沟。 - 制药行业全流程涉及早期研发、多阶段临床试验、内外部知识沉淀、HCP 学术交流以及高度合规的全球上市策略。 - 行业的底层核心诉求不是获取更多数据，而是赋予已有数据深度语境与临床意义。 ```mermaid flowchart TD accTitle: AI的两大阵营对比与制药流程 subgraph S1["AI 世界的两极分化"] A["大型科技巨头 (Big Tech)"] -->|"争夺算力 / 堆叠 GPU / 刷榜"| B["算力与基准竞赛 (Compute & Benchmarking)"] C["垂直行业实践 (Takeda)"] -->|"业务场景结合 / 上下文理解"| D["意义构建 AI (Sensemaking AI)"] end subgraph S2["制药行业核心业务链条"] E["早期研究与空白检测"] --> F["临床试验推进 (Phase 1-4)"] F --> G["知识沉淀与合规培训"] G --> H["HCP 学术交流与 LCM 研究"] H --> I["全球多国合规上市"] end ``` --- ### [02:14](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=134s) 传统 GenAI 在制药领域的盲区：缺乏语境与信任危机 许多供应商常常向业务方推销其“下一代最强大的 AI 引擎”，宣称只要接入其服务就能立刻为商业化或内容创新输出洞察。然而现实情况是，主流大语言模型（LLM）并不缺乏词汇量，也不缺乏对基础本体论（Ontology）和制药术语的认知，“它们并不愚蠢，它们什么都知道”，但模型唯独缺少关键的“语境（Context）”——提问者的真实身份是谁？为什么要在这个节点提出这个问题？ GenAI 刚兴起时，制药行业迅速跟进，利用海量历史数据快速搭建了能够生成见解、总结提要和导出 PDF 报告的酷炫原型。然而，当技术团队试图利用标准的检索、提取、生成（RAG）管道，配合 Top-K 或混合搜索（Hybrid Search）进行规模化标准化推广时，致命的短板暴露了：系统缺乏“意义构建”能力。生成式 AI 的商业价值绝不仅仅是“更快地生成见解”，而是“在充分理解使用者特定情境与概念的前提下，输出真正切中要害的洞察”。在制药领域，任何生成错误都不仅是技术偏差，更是对用户信任的致命破坏。更关键的是，医药行业的一切决策都系于“患者至上（Patient First）”的生命红线，任何不合规或幻觉错误都会直接危及患者安全与监管合规底线。 - 供应商模型普遍掌握医药术语与通用知识，但完全剥离了使用者的角色背景与业务提问动机。 - 传统 RAG（Top-K / 混合检索）只能做到事实片段提取，无法替代深度的情境推理与语义对齐。 - 在强监管的医药场景下，AI 输出的准确性直接关乎患者预后（Patient Outcome）与法规合规，任何错误都会彻底打破用户信任。 --- ### [04:48](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=288s) 从技术思维转向科学家思维：餐厅评论与临床研究的类比 拥有 14 年技术老兵背景的 Preethi 坦言，自己曾退后一步深刻反思：与其执着于掌握最新的 LangGraph 语法或堆叠 Agent 框架，不如转变思维范式——“像科学家一样思考，像与我合作的商业负责人一样思考”。为了形象阐述何为“意义构建”，她以挑选餐厅评论为例：普通的通用 AI 往往只会抓取高频通用关键词生成诸如“环境优雅、干净卫生”的平庸摘要；但对于一个偏好辛辣食物的印度裔用户或带着孩子的家庭而言，即便数百万条好评夸赞环境，只要有一条评论提到“菜品偏辣”或“提供儿童餐”，那才是真正决定决策的关键信息。“意义构建”的核心正是识别并串联起这些因人而异、因场景而异的关键线索。 这一认知直接源于她在 2023 年遭遇的技术挫折。当时她为某款 GenAI 应用精心设计了关于炎症性肠病（IBD）的高级检索查询与提示词工程，满心欢喜地为业务团队返回了算法评分最高的排名前三篇文献。然而业务专家一眼便予以否定，原因是业务团队在实际决策中需要的根本不是来自哈佛、MIT 等名校或单纯引用量极高的理论论文，而是样本量超过 10 万人、跨多国且涵盖不同种族背景的真实世界第 4 期（Phase 4）临床研究。这次碰壁促使她彻底重构了系统的排序逻辑：优先识别临床试验分期（重点关注总结实际疗效与经验的 Phase 4）、入组样本规模与人群多样性，让模型的推理与检索机制真正与医学科学家的逻辑同频。 - 传统的文献检索倾向于按引用量、知名机构（如 Harvard/MIT）进行权重排序，严重脱离医药业务实际需求。 - 临床决策更看重研究规模（如 10 万+ 患者入组）、多国多中心试验以及第 4 期（Phase 4）真实世界研究数据。 - 技术的精妙不能代替领域认知，工程师必须学会以科学家和商业负责人的逻辑重构评估与检索逻辑。 --- ### [07:47](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=467s) 引入业务画像（Persona Card）：从理解需求到概念落地 为了系统化地捕捉不同职能部门的思维差异，Preethi 借鉴了医药营销活动中常用于描绘患者群体（例如老年患者特征）的“用户画像（Persona）”方法论，并将其迁移到内部 AI 系统的架构设计中。在与商业化团队、医学传播部门（Medical Communications）以及论文发表团队（Publication Team）等跨职能部门协同的过程中，她为每个角色建立专属的“业务画像卡（Persona Card）”。 这些画像卡巨细靡遗地记录了各个角色的业务目标、核心输出诉求、所追求的业务影响力瓶颈，以及他们日常分析问题时的思维盲区与知识缺口。对于负责开发底层系统的技术工程师而言，这些画像卡既是理解业务语境的知识库，也是系统进行个性化情境推理的规则基石。通过明确不同受众的具体痛点，技术团队得以将抽象的“意义构建”理念转化为可落地的工程架构。 - 业务画像卡（Persona Card）涵盖具体职能的业务目标、期望产出、核心影响力指标及核心关注点。 - 画像机制使底层系统能够识别提问者的职能角色，从而动态适配其专有的临床逻辑与评价标准。 ### [09:07](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=547s) 构建 Sensemaking AI 闭环的四步架构与核心要素 作为一名技术背景的从业者，最难克服的直觉往往是在没有深入理解全局前就直接扑向纯代码与工程实现。讲者选择按下暂停键，先在概念层面上梳理出一套由内向外的清晰蓝图。整个 Sensemaking 架构分为四层：外层依然是传统的检索增强生成（RAG），而内层则是真正构建“理解力与认知逻辑（Sensemaking）”的核心所在。这一闭环具体由四个步骤串联而成：**观察（Observe）**、**推理（Reason）**、**建议（Recommend）**与**自适应（Adapt）**。 ```mermaid flowchart TD accTitle: Sensemaking AI 闭环与工程实现架构 subgraph Loop["Sensemaking 认知闭环"] direction LR A["观察<br/>Observe"] --> B["推理<br/>Reason"] B --> C["推理/建议<br/>Recommend"] C --> D["自适应<br/>Adapt"] D --> A end subgraph Pillars["底层工程支撑体系"] direction LR P1["提示词演进<br/>Prompt Engineering"] P2["记忆分层<br/>Memory Layers"] P3["领域知识库<br/>Domain Knowledge"] P4["Databricks 平台<br/>评估与开箱即用指标"] end Loop --> Pillars ``` 在工程落地层面，系统围绕四大支柱展开： 1. **提示词工程（Prompt Engineering）**：精细调整上下文，让大模型准确抓取业务语境。 2. **记忆层（Memory Layers）**：深入掌握用户画像（Persona）。例如，武田制药（Takeda Pharmaceuticals）总部位于日本，即便某位商业负责人常驻美国，其交互也高度集中在日本市场的营销材料制作上，系统必须牢牢记住这一角色背景。 3. **领域知识库（Knowledge Base）**：无论行业内如何热炒 MCP（Model Context Protocol）或各类前沿智能体概念，技术本身在脱离了高质量内容后毫无意义。最根本的问题始终是：**你喂给模型学了什么？** 唯有建立深度的知识储备，LLM 才能像真正的医药科学家一样思考。 4. **Databricks 基础设施**：直接利用 Databricks 平台自带的评估指标和开箱即用功能，避免从零重复造轮子。 --- ### [11:26](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=686s) 提示词工程的迭代路径：从零样本到少样本与信任构建 提示词工程经历了一个从浅入深的演进过程。最初的 0.0 版本是纯粹的**零样本提示（Zero-shot Prompting）**，仅要求模型从标准 PDF 文档中提炼总结。讲者通过观察总结规律后，迅速迭代为**单样本提示（One-shot Prompting）**。在这一阶段，系统注入了明确的角色画像（如商业负责人、医学负责人或科学传播专员）与期望的输出格式，强制要求生成结构化的要点列表，并对每一句话进行严格溯源（无论是检索自 PubMed 文献还是内部文档），以打消业务部门的疑虑。 在随后收集业务反馈的过程中，讲者发现科学家与业务人员评估医学文献的逻辑极为特殊：面对上百篇文献，他们主要依据**临床分期（Phases）**、**样本量（N-size / 招募患者数）**以及最重要的**新颖性（Novelty）**来进行优先级排序。提取分期和样本量相对容易，但识别“新颖性”难度极高——6 个月前具备新颖性的研究在今天可能早已成为常识。因此，持续更新的领域知识库成为破局关键。当模型能够主动输出当前行业最前沿的靶点或趋势（例如在消化道及自免疫领域火热的 TL1A 靶点）时，业务人员的信任感便油然而生。 为了进一步让大模型理解复杂的价值判断，提示词演进到了**少样本提示（Few-shot Prompting）**阶段。讲者形象地将其比作教育刚学会走路的幼儿——明确告知模型“什么是优秀回答（Good job）”、“什么是劣质回答（Bad job）”。通过正负案例对比结合可溯源机制与角色适配，系统逐步形成了稳定可靠的推理习惯。 - **0.0 阶段（Zero-shot）**：无上下文，仅输入 PDF 要求生成摘要。 - **1.0 阶段（One-shot）**：注入角色设定（商业/医学/科学沟通）+ 格式要求（要点化输出）+ 全文句子级引用溯源（PubMed / 内部库）。 - **文献筛选核心三要素**：临床研究分期（Phases）、样本患者数量（N-size）、行业前沿新颖性（Novelty，如 TL1A 等热点概念）。 - **2.0 阶段（Few-shot）**：采用“幼儿式奖惩”正反例示范，明确规范回答标准。 --- ### [14:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=872s) 下一代提示演进：多轮对话门控与跨文化角色扮演 在现有版本的基础上，未来的演进方向锁定在**多轮对话（Multi-turn Conversation）**与**多智能体角色扮演（Role-play）**两大维度。在多轮对话机制中，系统将整合历史对话记录、点赞点踩的显式反馈以及隐式交互偏好，并引入一个“门控智能体（Gatekeeper Agent）”。该门控在面对新请求时能够做出判断，主动避开用户过去反感或明确否决过的回答模式。 而在医药全球化运营中，**角色扮演机制**更是打破文化隔阂的利器。讲者以面向日本市场的商业推广材料为例：合规与法律层面的要求固然可以通过标准规则解决，但更精细的“文化语境”往往决定了营销的成败。在日本的文化背景与患者画像中，宣传素材与学术文献如果重点关注**老年患者群体（Elderly Patients）**，往往比采用年轻人或儿童形象更容易引发深刻共鸣。 通过构建专业的多智能体协同链路，后台智能体负责生成与推理，而把守最后一关的门控智能体则专门负责代入特定市场的文化视角，提示创作者：“请牢记，如果选用老年患者相关的素材，本次营销战役的成功率将大幅提升。”这使得 AI 不仅掌握了医学专业知识，更具备了跨区域商业专家的敏锐洞察力。 --- ### [16:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=992s) 记忆体系的深度构建：消除交互断层与证据沉淀 构建 Sensemaking AI 的另一大核心基石是**多层记忆体系（Memory Layers）**，其终极目标是让智能体真正“认识并理解”与它交互的用户。讲者分享了自己前一天的真实生活经历：当她带着全家出行并询问 ChatGPT 适合带孩子去的游玩地点后，紧接着追问“附近推荐去哪家餐厅”，大模型却完全遗忘了上一轮对话中“带娃出行”的核心前提。 讲者强调，一个合格的智能体不能仅仅把聊天记录冷冰冰地堆在左侧侧边栏中，而必须具备从对话中主动提取记忆并在后续交互中复用的能力（例如主动回应：“我知道您上一轮提到了带孩子，这里为您推荐几家亲子友好型餐厅”）。在医药系统的工程架构中，记忆层被细分为多维组件： - **画像记忆（Persona Memory）**：记录用户的职能角色、关注领域、地理及市场偏好（如常驻美洲但深耕日本业务的商业负责人）。 - **证据记忆（Evidence Memory）**：将检索、切块（Chunks）及已确认事实统一归档沉淀，实时吸收新证据并建立关联。 - **轨迹记忆（Trace Memory）**：全面记录系统过往的推理路径、评估结果及历史执行表现，为后续的溯源与自我进化提供完整线索。 ### [17:46](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1066s) 记忆层的输入与“独门秘方”：用户画像与战略目标注入 在搭建完循环流动的记忆层架构后，核心挑战转变为：究竟应该向这些记忆层中输入什么内容。传统的做法仅停留在角色分工与岗位职责的静态文档，但这种方式根本无法让 AI 产生真正的领域理解。LangChain、向量记忆库等技术早已普及，技术本身并不是护城河，真正的“独门秘方（Secret Sauce）”在于**用户画像（Persona）**及其背后的**战略目标**。即便是当下最先进的通用大模型，也无法直接扮演某位特定的资深科学家、商业负责人或应用主管，因为只有业务人员自己才清楚其年度核心战略目标。 企业中的战略目标并不复杂，通常仅有清晰明确的 2 到 3 个（例如品牌定位、商业推广重点等）。将这些精简的战略目标注入记忆层后，AI 就能在整整一年的周期内始终锚定正确的业务航向。除战略目标外，情境意识（Situation Context）与治疗领域上下文（Therapeutic Area Context）同样决定了回答的维度： - **情境意识**：在面对学术会议、与医疗卫生专业人士（HCP）交流或撰写出版物等不同业务场景时，受众对专业深度、商业属性的理解截然不同，AI 给出的策略与措辞必须随情境动态切换。 - **治疗领域与产品周期**：若产品属于新药上市阶段，AI 的推理重心必须放在**疗效数据（Efficacy）**与市场破局上；若产品已在市场上成熟运作了 10 年，讨论重点则应转向前沿联合疗法或差异化拓展。 通过将内部文档、外部证据与精准的画像/战略记忆深度融合，AI 便不再是凭空想象业务场景，而是真正像专业业务专家一样思考。 ```mermaid flowchart TD accTitle: 领域感知记忆输入与上下文架构 subgraph ContextLayer["上下文注入层 (Context & Strategy)"] P["用户画像 (Persona)<br/>医学主管 / 商业负责人"] G["战略目标 (Top 3 Strategic Goals)<br/>年度重点 / 品牌定位"] S["场景上下文 (Situation Context)<br/>学术会议 / HCP交流 / 出版物"] TA["治疗领域生命周期 (TA Context)<br/>新药上市 (聚焦疗效) vs 成熟期 (前沿疗法)"] end subgraph MemoryEngine["记忆引擎与推理层"] M["多层记忆层 (Memory Layers)"] R["领域感知推理引擎 (Sensemaking Loop)"] end P --> M G --> M S --> M TA --> M M --> R R --> O["情境适配的高可信决策建议"] ``` --- ### [20:57](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1257s) 工程化落地：推理链优化、评估护栏与流量路由 为了让各个记忆层与模型组件高效协同，武田制药设计了一套完整的工程化落地方案，避免从零重复造轮子，充分利用了成熟的企业级基础设施（如 Databricks）： - **意图路由与查询改写（Query Rewrite）**：系统结合注入的画像信息与领域上下文重写用户 Prompt，并引导思维链（Chain of Thoughts, CoT）。推理链被严格约束：优先评估研究的新颖性（Novelty），核实临床分期是否为最新阶段，并验证是否为多中心/多国研究。 - **幻觉抑制策略**：在分析临床数据时，大模型曾因被强制要求提取样本量（N count）而臆造随机数字。工程团队对此增加了严谨的兜底指令——若文献未明确提及样本量 $N$，则直接默认标记为“数据不足（Insufficient Data）”，并继续后续参数推导。 - **安全护栏与模型裁判（AI Gateway & AI Judge）**：采用 Databricks AI Gateway 构筑安全护栏，实时过滤 PII（个人身份信息）及敏感数据；利用 Databricks Mosaic AI Agent Evaluation 充当“AI 裁判”，对问答质量进行自动化评分，并结合人类在环反馈（点赞、点踩及具体修改意见）持续迭代模型。 - **合成问题生成（Synthetic Agent Evaluation）**：利用 Databricks 内置能力根据底层数据源自动生成跟进问题（Follow-up Questions），既降低了延迟与 Token 成本，又能结合武田内部高频趋势问题，为用户主动推荐后续探索方向。 - **模型降级与流量路由（LLM Fallback & Routing）**：针对 Agent 与复杂图编排的高昂调用成本，架构采取分流策略——前置预处理、特征提取及过滤分块使用低成本的小型 CPU/GPU 模型，仅在最终生成高价值决策输出的核心阶段才调用最强大的主模型。 --- ### [24:24](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1464s) 双层认知体系与实战落地：以 Dr. Karen 的 IBD 医学策略为例 整个架构将 AI 世界划分为两个有机组成部分：底层依赖通识大模型对疾病机制、治疗方案与临床终点的深厚医学底座；上层则搭载领域感知推理（Sensemaking GenAI），结合专家画像与 Prompt 模板生成真正契合业务目标的洞察。 在现场展示的具体案例中，系统模拟了一位名为 Dr. Karen 的全球医学策略负责人（Global Medical Strategy Lead）： - **业务情境**：Dr. Karen 正准备参加一场行业会议，计划与 HCP 探讨炎症性肠病（IBD）领域的产品疗效，核心诉求是将武田的产品精准定位在 IBD 治疗格局中。 - **智能筛选与差距分析（Gap Analysis）**：系统不仅从海量文献中精准检索出 3 篇高度匹配的权威论文，更关键的是指出了文献中的**认知盲区与差距**。例如，第一篇论文虽然样本量充足且具备溯源依据，但系统主动提示：“该研究所开展的头对头试验对比的是其他药物，并非武田当前重点追踪的 Top 3 核心竞品。” 这种不仅提供答案、更能识别信息差距（Gap Identification）的推理能力，使医学策略专家在走上交流前线前便能洞悉全局，真正实现了从“搜索摘要”到“行动决策”的跨越。 ### [26:33](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1593s) 从信息检索到智能教练：洞察研究空白与多维情报 在医药领域的复杂信息研判中，大语言模型（LLM）的价值不仅在于总结现有文章，更在于像专业导师一样主动“辅导”（Coach）使用者，指出内容中隐藏的研究空白与局限。例如，模型在分析头对头对比研究时，会提示该文献并未涵盖用户所追踪的特定竞品；在分析新兴诊疗趋势和生物标志物（Biomarkers）时，若能接入社交媒体舆情等实时数据源，模型就能进一步捕捉到外部动态，提示“该临床试验存在可能被提前终止（Cut off）的前瞻性风险”。 此外，模型还能敏锐识别长期安全性（Long-term safety）数据的缺失。当使用者面对大量文献时，LLM 能够主动提示：“这些文章非常优秀，但请注意，我注意到研究中存在以下盲区……”通过主动输出超越用户预期的深层洞察，系统与业务利益相关者之间建立起了至关重要的信任基础。 ```mermaid flowchart TD accTitle: Sensemaking AI 从内容分析到业务落地的闭环架构 subgraph SENSE ["智能理解与辅导 (Sensemaking & Coaching)"] A["多源文献与实时舆情"] --> B["LLM 洞察推理引擎"] B --> C1["识别研究盲区与安全性缺失"] B --> C2["角色扮演与本土文化情境适配"] end subgraph ACTIVATE ["业务激活与进化 (Activation & Evolution)"] C1 --> D["自动化对接 CRM 与合规审批"] C2 --> D D --> E["Sensemaking KPI 评估体系<br>(速度 / 决策质量 / 证据溯源)"] end ``` ### [27:33](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1653s) 角色扮演机制：在跨文化敏感度中优化 HCP 沟通 为了实现更有影响力的业务输出，Preethi Ramesh 强调了未来引入“角色扮演（Role-play）”机制的必要性，并特别以日本等高文化敏感度市场为例进行了说明。由于不同国家在医疗实践、沟通礼仪和文化敏感性上存在巨大差异，AI 不能仅仅静态地呈现两篇论文让用户自行挑选。 引入角色扮演能力后，LLM 能够根据具体的医疗专业人士（HCP）特征和对话情境进行决策。例如，模型会主动建议：“如果您在与该 HCP 交流时重点展示探讨老年患者群体的文章，将对沟通产生更大的影响力。”这种结合具体沟通对象与文化情境的精准建议，将成为医药代表与医学事务团队沟通中的“颠覆性改变者”（Game changer）。 ### [28:19](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1699s) 医药 AI 的未来愿景：激活流程、知识赋能与 Sensemaking KPI 面向未来的规模化落地，Preethi Ramesh 提出了三大核心发展方向： 1. **全面激活（Activate）：** 目前系统已经能够自动生成 PDF 或演示幻灯片等摘要成果，下一步是直接将生成物与医药 CRM 系统无缝对接，自动化触发合规与医学审查流程，打通从内容生成到一键审批的全流程自动化闭环。 2. **知识赋能（Coach / Educate）：** 针对缺乏特定国家市场背景的使用者（例如面对印度的本土化医疗环境），AI 扮演教练角色，主动普及当地的文化敏感点与沟通规范。 3. **指标演进（Evolve）：** 在衡量 AI 应用价值与投资回报率（ROI）时，业界需要建立专门针对“意义建构”（Sensemaking）的 KPI 评估体系，而不仅仅关注传统模型指标。 - **意义建构响应速度（Speed of Sensemaking）：** 模型从复杂证据链中完成推理并形成洞察的敏捷程度。 - **决策支持质量（Decision Quality）：** 系统所给出的行动建议与业务决策方向的精准度。 - **证据可追溯性（Evidence Traceability）：** 所有推荐与推理结论能否清晰回溯至底层医学证据。 ### [29:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1772s) 告别盲目算力扩张：构建懂人类思维的领域专家 AI 在演讲结语中，Preethi Ramesh 对当前大模型领域的喧嚣进行了深刻反思。她指出，“原始算力的轰动效应确实很响亮，但我们真正追求的强大力量往往是沉静而深思熟虑的”（"The raw power is really loud, but maybe the power that I'm looking for is quiet"）。当前整个行业都在热衷于争相构建下一个超大规模基础模型，但业务端真正关心的问题只有一个：“你的 AI 是否能像我一样思考？” 医药行业真正需要的，不是一个脱离人类思维逻辑的通用算法，而是能够深度理解医药研发与商业化逻辑的专业协同伙伴——包括懂临床与合规的“AI 医学负责人”（AI Medical Lead）、懂市场准入与推广的“AI 商业负责人”（AI Commercial Lead）以及严谨的“AI 出版物科学家”（AI Publication Scientist）。唯有打造出能够与人类专家同频思考的 AI 团队伙伴，才能真正释放 GenAI 在生物医药领域的深远价值。 ## Notable quotes > [02:14](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=134s) 它们并不愚蠢，它们什么都知道 — Preethi Ramesh > [04:48](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=288s) 像科学家一样思考，像与我合作的商业负责人一样思考 — Preethi Ramesh > [09:07](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=547s) 你喂给模型学了什么？ — Preethi Ramesh > [14:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=872s) 请牢记，如果选用老年患者相关的素材，本次营销战役的成功率将大幅提升。 — Preethi Ramesh > [16:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=992s) 我知道您上一轮提到了带孩子，这里为您推荐几家亲子友好型餐厅 — Preethi Ramesh > [24:24](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1464s) 该研究所开展的头对头试验对比的是其他药物，并非武田当前重点追踪的 Top 3 核心竞品。 — Preethi Ramesh > [26:33](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1593s) 这些文章非常优秀，但请注意，我注意到研究中存在以下盲区…… — Preethi Ramesh > [27:33](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1653s) 如果您在与该 HCP 交流时重点展示探讨老年患者群体的文章，将对沟通产生更大的影响力。 — Preethi Ramesh > [29:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1772s) 原始算力的轰动效应确实很响亮，但我们真正追求的强大力量往往是沉静而深思熟虑的 — Preethi Ramesh > [29:32](https://www.youtube.com/watch?v=EzDYgj4RXSE&t=1772s) 你的 AI 是否能像我一样思考？ — Preethi Ramesh ## Mentioned Preethi Ramesh, 武田制药 (Takeda Pharmaceuticals), Ai4, LangGraph, LangChain, Databricks, Databricks Mosaic AI Agent Evaluation, Databricks AI Gateway, ChatGPT, PubMed, Dr. Karen, 哈佛大学 (Harvard University), 麻省理工学院 (MIT) ## Video description Preethi Ramesh, Director – Data Digital & Technology Partner – GI Therapeutic Area, Takeda Pharmaceuticals Abstracts Sub heading : Architecting a Domain-Aware Reasoning Loop for Pharma Why this topic matters.: As GenAI continues to evolve in pharma, the next frontier is not just generating answers — but generating understanding. This session explores a Sensemaking AI loop that mirrors how scientists think: observing patterns, reasoning across evidence, and recommending actions in context. Key Takeaways: 1. Understand why traditional GenAI flows fall short. 2. Discover a new model for sensemaking 3. See real-world applications in Pharma 4. Learn how to move from search and summarization to actionable, trusted insight. [Source](https://www.youtube.com/watch?v=EzDYgj4RXSE)

[2:14] Preethi Ramesh: 它们并不愚蠢，它们什么都知道 [4:48] Preethi Ramesh: 像科学家一样思考，像与我合作的商业负责人一样思考 [9:07] Preethi Ramesh: 你喂给模型学了什么？ [14:32] Preethi Ramesh: 请牢记，如果选用老年患者相关的素材，本次营销战役的成功率将大幅提升。 [16:32] Preethi Ramesh: 我知道您上一轮提到了带孩子，这里为您推荐几家亲子友好型餐厅 [24:24] Preethi Ramesh: 该研究所开展的头对头试验对比的是其他药物，并非武田当前重点追踪的 Top 3 核心竞品。 [26:33] Preethi Ramesh: 这些文章非常优秀，但请注意，我注意到研究中存在以下盲区…… [27:33] Preethi Ramesh: 如果您在与该 HCP 交流时重点展示探讨老年患者群体的文章，将对沟通产生更大的影响力。 [29:32] Preethi Ramesh: 原始算力的轰动效应确实很响亮，但我们真正追求的强大力量往往是沉静而深思熟虑的 [29:32] Preethi Ramesh: 你的 AI 是否能像我一样思考？

Preethi Ramesh: the next frontier is not just generating answers — but generating understanding. [2:14] Preethi Ramesh: Are you creating an impact? Is your generative AI application—what is your sales pitch? You're going to get in the insight faster, or you're going to get in the insights knowing your situation, knowing the concept? [9:07] Preethi Ramesh: What matters the most is what have you made it to learn? What is the knowledge base that you have kept? [11:26] Preethi Ramesh: I don't have a PhD, but somebody who's not have a PhD built something inhouse, I need to earn their trust, and that's how I was able to earn their trust: to give them something like a traceability and show them the words that they are looking for. [20:57] Preethi Ramesh: insufficient data [24:24] Preethi Ramesh: I don't want to let AI to imagine my world; I want to make sure I've engineered it in such a way to think like me. [26:33] Preethi Ramesh: This article covers everything you need, but there is some prospective to be cut off. [29:30] Preethi Ramesh: Does your AI think like me?

[0:04] My name is Prrii Romesh and I'm a [0:06] director of um DDNT global portfolio [0:08] division um helping the GI therapeutic [0:11] partners. So in my role uh I have [0:14] experience building AI tools [0:16] specifically agentic framework langraph [0:18] I've been building all that so I [0:20] represent the business team. So I sit [0:22] with a business team and I try to [0:24] understand what is their needs and I try [0:26] to build in house. So that's the role [0:27] that I play. But to start this uh [0:30] presentation today, I imagine the [0:32] picture on the left. So what I imagined [0:34] is the AI world could be divided into [0:36] two. So there is a part one uh headed by [0:40] the big tech with the deeper pockets [0:43] racing for the power, bigger GPUs, CPUs [0:47] or even uh benchmarking. But then there [0:50] is an other part of the world where we [0:52] are trying to make meaning out of all [0:53] these models. That's the part that I'll [0:55] be focusing today and I'm calling that [0:58] part as sensemaking AI. But before I [1:01] talk about sensemaking AI, I want to [1:03] give you a background about the pharma [1:05] world. So I'm sure not everybody's from [1:07] pharma industry. So I want to give a [1:09] little bit of pharma industry of how it [1:10] works. The reason I'm giving a context [1:12] for this is it gives you information of [1:15] why I took this journey. So when it [1:17] comes to the pharma world, there are [1:18] multiple functions. There is um early [1:21] research where they do gap detection and [1:23] then we do clinical trial programs and [1:25] then we try to document all these [1:28] knowledge for training purposes internal [1:30] and external audience and then we engage [1:32] with HCP we engage with HCP and not [1:35] educate them like for example we would [1:37] have done a recent LCM study on um [1:40] efficacy of a particular drug and we [1:42] want to engage the HCP we want to update [1:44] him saying this is what it says we have [1:47] done a clinical trial program on a [1:48] pregnant women and our efficacy is [1:50] improved to 80%. We wanted to [1:52] communicate those aspect and then our [1:54] launch strategy. So every single product [1:56] is launched right and it's a lot of [1:58] regulated when it comes to launching in [1:59] US versus a different uh countries. So [2:02] the reason I'm giving you a very [2:04] in-depth information about the pharma [2:05] industry is we are not short on data. We [2:08] all have data and we don't it's not that [2:10] we don't know how to access data. We [2:12] know how to access the data. But what [2:14] the problem in the reality is when we [2:16] try to partner with a stakeholder they [2:18] say that we have built the next biggest [2:20] engine and if you use our services [2:22] you're going to get the insights for the [2:24] commercial purposes or for the content [2:26] innovation but in the challenge is these [2:28] LLMs do know words it's not that they [2:31] don't know they it's not that they're [2:32] dumb they know they know everything they [2:34] know ontologies they understand the [2:35] pharma industry but what do we need what [2:38] do we need from them is the context who [2:40] is this person why is this person is [2:42] asking that questions Since that's a [2:44] context which I believe the current LLM [2:46] might be little gap behind. The reason [2:49] is when jai came to the picture pharma [2:52] industry was really on it. The reason is [2:54] we have so much of data that we have [2:56] been collecting at every faces and using [2:58] genai we could generate a very cool um [3:01] prototype in no time. We could build one [3:03] AI genai that could uh generate [3:06] insights. It we could gen summarizing [3:09] creating a PDF. We could do a lot of [3:10] things. But then what was needed is when [3:13] we trying to say that we're going to [3:14] streamline it using this retrieval um [3:16] extract and generate process and then we [3:19] try to bring standardize it. How do we [3:21] extracted? Okay, you you can use top K [3:23] you can use hybrid search but I felt [3:25] there is another part of this is still [3:27] missing. That's the missing is sense [3:29] making when you generate an output. Are [3:32] you creating an impact? Is your [3:34] generative AI application what is your [3:37] sales pitch? You're going to get in the [3:39] insight faster or you're going to get in [3:41] the insights knowing your situation, [3:44] knowing the concept. Here is what is [3:46] what is needed. That's the whole real [3:48] impact. It's not about building the next [3:50] big model. The biggest impact is when [3:52] you use to real use case. That's the [3:54] role I play. I sit with the business [3:56] stakeholders. I understand the use case [3:58] and when I try to use one of the models, [4:00] it might not look like how they think. [4:02] So I started thinking why is such a [4:04] thing very important? Why does it [4:06] matter? So imagine you have built one uh [4:08] jai application any chatbot and you give [4:11] it to your internal audience or an [4:12] external audience and then it's [4:14] generating incorrect information. When I [4:17] say incorrect information the biggest [4:20] problem right here is a trust. You're [4:22] breaking the trust with the audience and [4:24] moreover that when it comes to pharma [4:26] industry every single thing is patient [4:28] outcome right? So if you think about [4:29] patient first we keep them in the mind [4:31] and whatever we build we make sure we [4:33] protect them. We we we have so much of [4:35] regulation around it and the same thing [4:37] when we generate some something we have [4:39] to make sure we are not breaching their [4:41] trust we are not breaching any [4:42] compliance issues that's why it really [4:44] matters of what we are trying to propose [4:46] to the audience who's using this [4:48] application then I started thinking I [4:51] come come from a very technical [4:52] background I have 14 years of experience [4:54] being in this technical industry so what [4:56] I tried to say I I took a step back okay [4:59] it's not about me building the next big [5:01] thing it's not about me learning lang [5:02] graph or learning the agent It's not [5:04] about that. What is it? How what do I [5:06] need to think like an scientist? Think [5:08] like a commercial lead that I'm [5:10] partnering with. So then I started [5:12] thinking okay I need something that [5:13] totally makes sense. So now what is this [5:15] sense? What is this sense making? So I [5:18] what I started thinking is if I give you [5:19] an example of a very general application [5:21] if you're trying to build imagine you're [5:23] building a um a summary for reviews that [5:26] is for restaurants and then um what that [5:29] summary is doing is it's looking for [5:31] some key information good atmosphere [5:33] clean atmosphere and it's generating a [5:35] summary saying this restaurant is good [5:37] but what would I do what would I do is [5:39] when I'm looking at a restaurant I would [5:41] look for reviews who's like me who's [5:43] posted like me and who is me somebody [5:45] from an Asian country somebody's is from [5:48] Indian who likes spicy food even if [5:50] there is a one comment about the spicy [5:52] food I would like to know that even [5:54] there are millions of comments about [5:55] everything else I'm not interested but [5:57] something like okay it's is it a child [6:00] um I mean if it's a children meals are [6:02] there spicy food all these connections [6:04] is what I try to make so what I'm trying [6:06] to do here is I'm trying to make sense [6:08] of whatever I'm trying to do so this is [6:10] exactly what I started applying to my [6:12] world in the pharma industry then what I [6:15] started thinking is the first question [6:16] the generative AI application I built in [6:19] 2023. I started to look back when I'm [6:22] asking a question of give me articles uh [6:24] about IBD. So I crafted a really good [6:27] search query. I crafted a really good [6:29] prompt engineering. I did the best that [6:30] I could and then it gave me the top [6:32] three articles and then when I took it [6:34] to the stakeholders they said nope [6:36] that's not what I'm looking for. What [6:37] I'm looking for an article that has a a [6:40] bigger uh clinical recruitment program [6:42] maybe like 100,000 patients and then [6:44] multi-country study. So this is [6:46] something that I don't know because I [6:48] come from a technical background. I [6:49] would not know. So this is how this is [6:51] what I felt about okay what do I need to [6:53] know is how they think how do they [6:55] approach a problem. So I spent more time [6:58] understanding from their perspective and [7:00] I started building the next version of [7:02] my geni application and what it started [7:04] doing is now it knows to rank. it knows [7:07] to rank. Okay. So by phases, every [7:09] clinical trial program goes through [7:11] phases. Phase 1, 2, three, and four. But [7:13] then when it comes to the four, that's [7:15] when everybody pays attention. Okay. [7:16] What happened in this study? What did [7:18] they improve? What did they learn [7:19] through this study? So what I started [7:21] ranking is okay, if an article is [7:23] talking about phase four study and a [7:25] multi-country study and it's recruited a [7:28] lot of patients with a different racial [7:30] background, then that's the article that [7:32] I need to propose as my output. not an [7:35] article which has which is coming from [7:36] maybe coming from Harvard or MIT who's [7:39] coming from uh really good citations [7:41] that's not my approach so I started [7:43] looking into thinking like them and I [7:45] started querying in a little bit now [7:47] that I've understood um how what I [7:49] wanted to do the next step was know the [7:52] persona so the persona concept is you go [7:56] to any marketing slides or any slides [7:58] I've already I've noticed that they [8:00] speak like a persona they introduce a [8:02] persona that persona would be a patient [8:03] patient and they will say okay this is [8:05] the patient and this patient is an [8:07] elderly patient so that's how they kind [8:08] of communicate when it comes to [8:10] marketing campaigns and I thought okay [8:12] so that's what I believe is needed even [8:14] in my application so I started looking [8:16] at personas when I'm trying to interact [8:19] uh when I have an I I have a really good [8:21] chance to interact with commercial folks [8:23] communication publication team there are [8:25] so many multiple functions in medical [8:26] and I when I speak to them I look at [8:28] okay what is their goal what is the [8:30] output that they're targeting what is [8:32] the biggest impact that they're trying [8:33] to make. So that's my persona card. This [8:35] is just one example, but I started [8:37] building my own persona card. In that [8:39] persona card, I'm trying to feed as much [8:41] of information possible. This is for two [8:43] things. If I am the technical person [8:45] who's going to build this application, [8:47] then I need to know what that persona is [8:49] going to do. So I'm trying to learn as I [8:51] go and I had my own persona card to [8:53] understand. So in this step, now that [8:55] I've understood the persona, I [8:57] understood the gaps and needs for every [8:58] different person. Then the next thing I [9:00] did is all right. So sense making I need [9:03] to make a concept as sensem but [9:05] conceptually how do I do if you notice [9:07] I've still not started to engineer [9:09] anything because as a technical person [9:11] that's the hard thing to do is to stop [9:13] thinking in a technical level but I [9:14] wanted to do a pause before I jump into [9:17] solution I want to take a pause of how [9:18] do I want to engineer this whole thing [9:20] but in a conceptional way. So what I did [9:22] is I I put together a four step. The [9:25] first step is the observe layer. The [9:27] second step I wanted to reason. The [9:30] third step is I wanted to recommend and [9:31] then finally I wanted to adapt but my [9:33] outer layer remains the same. It's still [9:35] the rag but then my inner layer is where [9:38] I'm trying to make a difference. I'm [9:39] trying to engineer the sense making. So [9:41] once I've understood okay this is what I [9:43] want to do. Now the next step is how I [9:46] want to implement. So this is for a this [9:49] is how I implemented it. When you look [9:51] at this slide you might think you might [9:53] have different ways of implementing it. [9:56] This is how I approached it to solve the [9:58] problem that I was going behind. So the [10:00] first thing uh what I did was prompt [10:02] engineering. We can go back to our [10:04] prompts and we can tweak it in a little [10:05] bit so that it understands the context. [10:07] And then the memory layers it was so [10:09] important that the memory it needs to [10:11] know this particular person has always [10:13] been interacting me talking in Japan. So [10:16] although this person is living in US, [10:18] this person has been interacting with me [10:20] how do I do the marketing materials in [10:22] Japan. So it means that this person is a [10:24] head of commercial lead in the Japan. Uh [10:26] so because Teddada pharmaceutical the [10:28] main headquarters in Japan. So I started [10:30] to learn this persona even though they [10:32] are in US they do a lot lot of work even [10:34] in Japan as well. So I want this persona [10:36] to know the memory. So then comes the [10:38] engineering decisions. What are certain [10:40] engineering decisions that I have taken [10:42] in order to engineer the whole pipeline. [10:44] Then the knowledge base. So the [10:46] knowledge base I believe is the most [10:48] important aspect. You can talk about [10:50] agents, you can talk about MP, MCP, you [10:52] can talk about any anything in this [10:54] current technology space. You can keep [10:56] following LinkedIn or even X right to [10:58] know what's latest. But that doesn't [11:00] matter at all. What matters the most is [11:02] what have you made it to learn? What is [11:04] the knowledge base that you have kept? [11:06] So that's why I went back to the [11:07] knowledge. Okay, what else can I feed to [11:09] this LLM so that it knows the context to [11:11] think like an scientist. Then lastly, [11:13] data bricks. That's what I've been [11:15] using. I'm going to show you certain off [11:17] tag that I could just use in data bricks [11:19] like evaluation metrics that I could [11:20] just use it that I don't have to build [11:22] from scratch. So these are what I will [11:24] be covering. So let's start with the [11:26] prompt. So my first version the zero [11:30] prompt prompting which I didn't do here [11:31] is the first version which had nothing. [11:34] So that's when everything was launched. [11:35] I was new to so I thought okay I'm just [11:37] going to do the zero prompt and then I'm [11:39] going to tell the LLM to generate a [11:41] summary out of the stand PDF. And then I [11:43] started to observe the pattern. Okay, [11:44] what do I what else do I need to do? The [11:47] next step, what I did is oneshot [11:48] prompting. Now I'm trying to feed two [11:50] things. Number one, I'm trying to feed [11:52] persona information and then I'm trying [11:54] to say this is my desired output. So in [11:57] the persona, I'm trying to set a [11:58] context. I'm a commercial lead or I am a [12:00] medical lead or I'm a scientific [12:02] communication person. I'm setting a [12:04] context and then I'm trying to say a [12:05] desired output. My output is I want it [12:07] to be in a bullet format. But in order [12:09] for my uh customers to trust my [12:12] application, I want every single [12:14] statement to be cited. So wherever it is [12:16] retrieving from, if it's written from a [12:17] PubMed articles oring from an internal [12:20] documentation, I wanted to say where [12:21] exactly retrieved from. That's my first [12:23] version that I built. So once I started [12:25] getting that version and I had a chance [12:27] to work with the business stakeholder, [12:28] get their feedback and then I built one [12:30] small feedback loop where they can click [12:32] like, dislike and also comment. So [12:33] that's a very basic pilot and then once [12:35] I had some more feedback that's when I [12:37] learned to them that that that's not how [12:39] they look at the the way they really [12:40] they look at the articles is if they [12:42] have 100 articles the way they rank the [12:45] articles is by the faces by the end size [12:47] and most importantly novelty I could do [12:50] faces I it's very easy to find the [12:52] keywords and do it and the second one I [12:54] I can also do the um the end size how [12:56] many patients that was recruited but the [12:58] most difficult job was the novelty the [13:01] novelty that happened 6 months back [13:03] might not be the same today. So the the [13:05] knowledge base comes into the picture. [13:07] What can I keep on updating so that the [13:09] LLM that I'm generating in knows [13:10] whatever is happening today. That's when [13:12] I can start making a difference. So when [13:14] a stakeholder is asking a question even [13:17] if that question wasn't asked for [13:19] example TL1A I'm not sure if everybody's [13:20] from pharma industry but that's what has [13:22] been trending right now. And if the LM [13:25] was able to give that particular word, [13:27] it's a big win for me and also for the [13:29] stakeholder then I will start earning [13:30] trust from them because they've been [13:32] doing the same thing for so long [13:34] partnering with a vendor providing a [13:36] vendor solution but suddenly if I bring [13:38] them in a solution which is built [13:39] inhouse I I don't have a PhD but [13:42] somebody who's not have a PhD built [13:44] something inhouse I need to earn their [13:45] trust and that's how I was able to earn [13:47] their trust to give them something like [13:48] a traceability and show them the words [13:50] that they are looking for. The next step [13:53] I did was a few short prompting. For [13:55] this, I was imagining my own toddler. So [13:57] when I talk to my toddler, I tell him [13:58] this is good job and this is bad job. [14:00] That's when I started doing for the same [14:02] thing with my AI solution also. These [14:04] are some of the good answers and these [14:06] are the bad answers. So once I start [14:08] doing this, what this LLM will know is [14:11] number one, it knows the memory. Number [14:12] two, it knows what to do and what not to [14:14] do. And on top of this, I'm also giving [14:16] the traceability aspect. So now if I'm [14:19] able to produce this results to the [14:21] stakeholders, they're able to get the [14:22] whole thing that they're looking for. [14:23] They're able to see the novelty. It's [14:25] already been ranked by how they want to [14:27] see on top of it feedback. It's able to [14:29] adapt based on the persona. But where do [14:32] I like to go? This version right now is [14:34] great. But then I don't want to stop [14:35] here. Where do I like to go is the first [14:38] one is multi-turn conversation. for that [14:40] multi-turn conversation what I would [14:42] like to do is the the memory that I [14:45] already been storing based on what the [14:46] stakeholders are saying the questions [14:48] that's been asked the likes and dislikes [14:50] and all the responses that's been taken [14:52] take that on top of it I want to add all [14:55] the feedback that has been given and on [14:57] top of the new request what this agent [15:00] will do is it will then decide it's like [15:01] a gatekeeper agent that will decide okay [15:04] so this is my situation for this [15:06] particular situation I don't have to do [15:07] these answers because I know in the [15:10] asked this person has asked similar [15:11] question and I didn't did not like this [15:13] answer. So that's what I would like to [15:15] do like a multi-turn conversations and [15:17] then the second thing the biggest thing [15:18] I think I would it's going to be a game [15:20] changer is the role play. The role play [15:22] is um an example of um the commercial [15:25] lead who was trying to do a uh [15:28] commercial materials for the Japan. So [15:30] at that time if a role play a regulated [15:33] of course I can get the legal aspects of [15:35] what needs to be done right but what [15:36] about some of the cultural aspect what I [15:39] learned during this one is when it comes [15:41] to Japan they really like to empathize [15:43] on article that talks about elderly [15:46] patients empathize on the pictures that [15:48] represents elderly patient not the kids [15:50] or not the any other um um like uh [15:53] younger professionals it's more about [15:56] elderly patients where their culture [15:57] connects the most so I would like to [15:59] have that role play while doing all [16:01] these things. Imagine one agent just [16:03] sitting and watching how the other [16:04] agents are performing and the last agent [16:07] becomes a gatekeeper. That gatekeeper is [16:09] bringing a cultural aspect. Just keep in [16:11] mind that if you use a material where is [16:14] an elderly patient, that's going to be a [16:16] success in your marketing campaign. [16:18] That's something that now I'm making my [16:20] AI to think like the commercial person [16:22] who knows or not only the context who [16:25] also knows the the market that we are [16:27] going inside. This is where um the [16:29] prompt evolution that I have been doing. [16:32] The next part is the memory layers. The [16:34] most key important of the memory layer [16:36] is I want my AI agent to know me to know [16:40] the person that I'm interacting. Even [16:42] today um not today like yesterday. [16:44] Yesterday I've come with my family. So I [16:45] was trying to ask uh the chat GTP like [16:48] where do I go with with my kids around? [16:50] So it gave me some suggestions and then [16:52] my follow-up question was which [16:54] restaurant do I go? It didn't remember. [16:56] It did not remember that my previous [16:58] question that I asked was kids friendly [17:01] activities. What if the response of the [17:03] restaurant was something like I know [17:05] that you you your previous question was [17:07] about kids. So here are some kids [17:09] friendly restaurants. That would have [17:10] been a game changer. That's what I [17:12] believe it needs to know. It's good that [17:14] it's storing the memory all the [17:15] conversations where you can go to chat [17:17] between a seat towards the left. But how [17:18] about it uses those conversations and [17:21] using the memory and then giving you the [17:23] answer that you would think it's very [17:24] relevant. Then comes the evidence [17:27] memory. In my evidence memory, what I [17:28] have done is all the chunks and [17:30] everything that's been stored in [17:31] retreat, I store it in one place. And [17:33] anything new, of course, it starts to [17:36] understand. Now, with the persona memory [17:37] and evidence memory, I start to do the [17:39] trace memory, which I told you is the [17:41] future which I really want to trace is [17:43] trace back to how it was performed, what [17:46] questions was answered, who's this [17:47] particular persona who's asking me this [17:49] question. Then it becomes the whole [17:51] cycle of the loop and data always flows [17:53] back, right? So it's always a loop and [17:55] it flows back. Now I've completed this [17:57] memory layers. The next big part for me [18:00] is what is feeding inside this memory [18:02] layer. For this what I started to take a [18:04] step back and look at all the [18:05] documentation that I have in place to to [18:08] feed into this memory. So um the blue [18:10] ones were the existing memory in my [18:12] earlier versions and the and the maroon [18:14] ones are the ones that I included more. [18:16] So I want to focus on the first one of [18:18] the persona. I know it will look like [18:20] okay it's just documented there's roles [18:22] and responses but that's not the thing [18:23] so if you ask me what is the secret [18:25] source whatever I've been speaking so [18:27] far I'm not speaking anything new [18:28] everybody knows about lang chain [18:30] everybody knows about memories everybody [18:31] knows about it but what exactly the [18:33] secret sauce is actually the first part [18:35] the persona part that's the part nobody [18:38] could play even the chat GDP today [18:40] nobody could play Elon Musk nobody could [18:42] play that that role can only be played [18:44] for that particular person who's a [18:45] scientist who's that particular person [18:47] who's a commercial lead or a application [18:49] lead nobody can replace that person only [18:51] they know what is their strategic goal [18:54] so every like our um in my employment [18:57] right so we have a performance appraisal [18:58] and I have a strategic goal for next one [19:00] year what am I going to do what am I [19:02] going to prove so that my end of the [19:03] year goal uh when I have um a discussion [19:06] with my boss he's able to assess my [19:08] performance the same thing even every [19:10] single company would do have the the [19:12] same thing like a strategic goal what is [19:14] a strategic for the brand how do we [19:16] position this brand and that strategy [19:18] goal. Luckily, there are not thousand or [19:20] hundreds. It's only three. What are the [19:22] top three strategy goal? It's so easy to [19:24] insert into the memory saying this is [19:27] what you need to focus for one whole [19:28] year when it comes to commercial. So, [19:30] what I started to realize is there are [19:32] so much of easy information out there. [19:35] All I needed to know is how do I [19:37] engineer it? I'm not trying to bring [19:38] anything new, right? I'm not trying to [19:40] become the next big thing. what I'm [19:42] trying to do is with everything out [19:43] there, how do I place it in a such a way [19:45] that my AI is thinking like the person [19:48] that I wanted to think. So then I [19:50] started to giving a situation. The [19:52] situation again is so important. [19:54] Situation today I'm in a conference. So [19:56] the way that I want to talk is uh [19:58] keeping in mind that the audience might [20:00] not know pharma. The audience might not [20:02] be technical. The audience might not be [20:03] very business focused. So I need to keep [20:05] that in mind. Same thing even I wanted [20:07] my AI to do as well. What is the what is [20:09] the context? what is the situation of [20:11] this person asking a question in pharma [20:13] industry we go to conferences we engage [20:15] with HCPs even we do a lot of [20:17] publications so b depending on the [20:19] situation the answer always changes um [20:22] then the therapetic area context this is [20:24] also very important because we wanted to [20:26] know the level of the brand is it a new [20:28] launch if it's a new launch strategy [20:30] then the whole answer changes but if [20:32] it's an existing brand it's been in the [20:34] market for 10 years then I'm not talking [20:36] more about efficacy I'm actually talking [20:38] about new advanced therapies that's what [20:41] but if I'm coming new in the market my [20:43] main goal is the efficacy so depending [20:45] on the therapeutic area context the [20:46] answer changes so that's what I did with [20:48] the persona and everything else remains [20:50] the same right you have internal [20:51] documentation you have external [20:53] documentation you bring everything into [20:55] so once I have fed all these memory [20:57] layers the next thing is how do I [20:59] engineer them how do I make them to talk [21:01] to each other so these are some of the [21:03] examples that I've given so I believe [21:05] everybody knows about intend routing [21:07] query rewrite but what something changes [21:10] the domain context. Now with the persona [21:12] information the query will be written [21:14] the rewrite will be done based on that [21:16] persona then what happens is the chain [21:18] of thoughts chain of thoughts I made it [21:20] to place it like this first make sure [21:22] it's novelty then you make sure that [21:24] your that the faces is latest and then [21:26] it's a it's a multi-country study I'm [21:28] kind of doing that chain of thoughts [21:30] once that chain of thoughts is done then [21:31] I'm trying to there are of course there [21:33] are problem with hallucination one big [21:35] example what I noticed was because I [21:37] kept in the chain of thoughts to make [21:38] sure there is end count I started Ed [21:40] noticing that it was generating some [21:42] random numbers and then I started to put [21:44] some information saying if you don't see [21:46] an end then just default it to [21:48] insufficient data and then move on with [21:50] other parameters. So that's how I was [21:52] able to do at least for that example. [21:54] Then AI gateway data bricks already has [21:56] an AI gateway. It acts as a guardrails. [21:59] So that I can use that guardrails for [22:01] safety measures for PII data for making [22:04] sure no personal information is out. The [22:06] reason I put there and I'm um saying [22:07] about data bricks is you don't have to [22:09] build this certain things from [22:11] scratcher. You can always use what is [22:13] existing and you can certainly uh do [22:15] certain mode addition towards it. So [22:17] that's what I did here. Then the AI [22:19] judge again I'm not building anything [22:20] from scratch. Data bricks already has [22:22] this. So I used databicks AI just to [22:25] assess. So that is something called [22:26] mosaic AI agent evaluation. What it [22:29] evaluated based on the question asked [22:31] and answer generated what is the [22:32] evaluation score? um based on that score [22:35] what the AI will do is it starts [22:37] learning okay so my score was poor what [22:39] do I need to do better now comes the [22:41] feedback and comes the human in the loop [22:43] they're already giving the like button [22:44] they're already saying dislike they're [22:46] already giving you the comments b with [22:48] that what how do I evaluate my model to [22:50] be able to generate even more better [22:51] results then comes the synthetic agent [22:54] evaluation [22:55] this was really important for me and I [22:58] used datab bricks again so they already [23:00] have a synthetic questions generator so [23:02] what I started doing was when my um LLM [23:05] gives an output, I made their model as a [23:09] prompt and I made their model to [23:11] generate follow-up questions. So here I [23:13] am saving my latency. I'm saving my cost [23:15] and tokens. I'm only using what data [23:17] bricks already has. So what I did was I [23:19] connected to the data sources and I made [23:21] it to generate some questions. Along [23:23] with that, I also started to give some [23:25] questions that asked most frequently. So [23:28] most frequent questions and also some [23:30] synthetic generated questions. So by [23:32] this I'm making sure that this person [23:33] who's interacting knows about there are [23:36] some trends around these questions used [23:38] by other folks in TADA. There is also [23:40] some more proposed questions that I can [23:42] ask for a follow-up questions. Then [23:45] lastly comes the LLM fallback model. So [23:48] I'm sure everything is expensive right [23:50] when it comes to agent when it comes to [23:52] langraph everything is very expensive. [23:54] So what I did is I kept the main model [23:55] as the gate and everything started [23:57] traffic routing it just smaller chunks [24:00] GPU or CPU whatever you do but then [24:02] trying to smaller chunks pre-process use [24:04] this and then after you process do this [24:06] and then if you want to generate answer [24:07] use this model using that I'm trying to [24:09] generate I'm trying to make sure this [24:12] traffic is across and only using the the [24:14] model that I want to generate answer the [24:16] towards the last stage where I want to [24:18] create a very impactful output. So [24:20] that's the engineering decisions that I [24:22] have taken. [24:24] Then uh so one world. So for me if you [24:26] go back to the first slide AI world is [24:28] divided into two. So for me one world [24:31] looks like this. I'm going to continue [24:33] using my default LLM model which I [24:34] believe is really strong the way it is. [24:36] It knows cure. It knows diseases [24:38] treatment outcomes everything it knows. [24:40] But on top of it adding my sensemaking [24:43] genai which knows the persona which [24:46] knows the pro templates how to use for [24:48] who. then reasoning engine I'm able to [24:50] generate an output which is domain aware [24:52] persona aware so that's how I have [24:54] engineered the whole thing and what I [24:56] wanted to say in this is I don't want to [24:59] let AI to imagine my world I want to [25:01] make sure I've engineered in a such a [25:03] way to think like me so I don't want to [25:05] just imagine hallucinate the whole world [25:07] so how does everything come into the [25:09] picture right like the whole loop so [25:11] I've given Karen Dr. Karen um the [25:14] popular Karen name who always asked for [25:16] I want to speak to the manager so I use [25:18] that word Karen but then uh in this [25:20] particular use case she is a global uh [25:23] medical strategy lead so what she's [25:25] trying to do her context is she's [25:26] preparing for a conference and trying to [25:29] engage with an HCP and her goal is to [25:32] have an efficacy discussion if you see [25:34] it is very clear of why she wants so [25:36] this is situation awareness once the [25:38] situation awareness is there it's been [25:40] fed inside her strategical need her need [25:43] is to make sure that she wants to [25:44] position our teada brand in a such a [25:47] conversation in the IBD space. So once [25:49] this is there then all the behind the [25:51] scenes I have my rag I have the the [25:53] sensemaking loop uh then then my [25:55] technology stack were mentioned on the [25:57] top. So everything is talking to each [25:59] other and when you see the output it [26:01] selected three articles but what it also [26:04] did was it selected the article and also [26:06] giving you the gap in the article that [26:07] you need to know about. For example, the [26:09] first article, although the article [26:11] looks what she thinks is relevant to [26:13] her, the N numbers and it also has the [26:16] traceability, but there is a gap. It [26:18] does talk about head-to-head, but it [26:20] doesn't talk about the competitor that [26:21] you are tracking. It actually talks [26:23] about a different drug. It that's comp [26:25] the different drug, but it's not the [26:27] compet you're tracking because not [26:28] everybody would be tracking all the [26:30] competitors, right? We will have top [26:31] three competitors. It's talking about [26:33] head-to-head, but it's not talking about [26:34] your um the tracked competitor. Another [26:37] example is it is talking about an [26:40] emerging trend and emerging uh diagnosis [26:42] procedure and also biomarkers but then [26:44] but then keep in mind there is a [26:46] prospective of this particular study to [26:48] be cut off. So that's something how will [26:51] it know that this study could be cut off [26:53] only if I access some real-time data [26:55] like a social media insights there are [26:57] some insights going on around for this [26:58] particular study only then I'm able to [27:00] give an impactful output saying this [27:02] article covers everything you need but [27:04] there is some prospective to be a cut [27:05] off then the lastly is something a [27:07] long-term safety it talks about [27:09] everything that you want but I've [27:10] noticed a gap in the study it doesn't [27:12] talk about long-term safety so basically [27:14] what this L&M is starting to do starting [27:16] to coach is starting to coach the user [27:19] who's using this tool to say these are [27:21] really good but keep in mind these are [27:22] the gaps that I've observed from these [27:24] articles now is starting to give me more [27:26] than what I'm thinking so that's when we [27:28] start getting uh trust from the [27:30] stakeholder so this is the future right [27:33] so I don't have it right now but then [27:34] this is the future where I wanted to do [27:36] is the role play so I gave an example of [27:38] the HCP use case but in Japan I'm keep [27:41] talking about Japan because that's [27:42] something really hard to solve uh [27:44] because it it comes cultural sensitivity [27:46] there are so much of information that [27:48] goes side. So in this case I'm imagining [27:50] like this. So a a person is being [27:53] presented with two articles and then the [27:55] LM is making to decide which article [27:57] should this particular person need to [27:59] take. How will the element how will the [28:01] LLM decide which article only if the [28:03] role play comes in the picture the role [28:05] play will tell you if you highlight the [28:07] article that talks about elderly patient [28:09] that's when I believe your conversation [28:10] with HCP is going to become make a [28:12] bigger impact. That's the future I [28:14] believe um having would be a gamecher [28:16] and a very impactful output. So forward [28:19] vision. So as I told you three things [28:20] I'm f foc foc foc foc foc foc foc foc [28:21] foc foc focusing on the first one is to [28:23] activate what I have been doing so far [28:25] is once I generate a summary um that [28:28] output could be PDF or slides anything [28:30] but then what if I'm able to connect to [28:32] the CRM system I'm able to directly send [28:35] it for an approval and review process [28:37] because that becomes a problem to be [28:39] solved once this PDF has been generated. [28:42] I still need to make sure it's [28:43] regularity compliance. I still need to [28:44] make sure it's been approved. But what [28:46] if I connect to CRM system, the whole [28:47] pipeline goes through and somebody [28:49] clicks approved. So my whole process is [28:51] automated. The next step is how how come [28:53] it becomes like a coach is trying to [28:54] educate you. You're trying to do in [28:56] India and this is how the sensitivity is [28:58] in India. So try to educate the person [29:00] who who doesn't know about India. Then [29:02] lastly is to evolve right and everybody [29:04] talks about KPIs. When we build [29:06] applications, we want to know what is a [29:07] KPI. We want to know what is return on [29:09] investment. How about we think about [29:11] sense making is one of the KPI is that [29:13] sense making working? What in what speed [29:15] is the sense making performing? And what [29:17] decision uh is a sense making doing and [29:20] also what evidence is doing as a trans [29:22] traceability. So these are the three [29:23] KPIs maybe we could use as a sense [29:26] making KPI in every application we could [29:27] bring and KPI as well. So that's where I [29:30] believe um I would like to see the [29:32] future. The last slide is about um the [29:36] raw power is really loud but maybe the [29:38] power that I'm looking for is quiet but [29:41] still until things changes right I [29:43] believe everybody is starting to talk [29:44] about it starting to talk about that's [29:46] okay everybody is talking to build a [29:48] next big model but does your AI think [29:50] like me that's what everybody has been [29:52] talking about so um my proposal here is [29:55] uh that's great you want to build but [29:57] make sure that you build AI um medical [30:00] lead AI commercial lead and also AI [30:02] publication mentioned you know AI [30:03] scientist that's what we want we want a [30:05] partner as a team not uh like like a [30:08] scientist who doesn't understand how a [30:10] human thinks so I'd like to close my [30:12] presentations today so thank you so much [30:14] for [30:16] [Applause]