UIC Builds Discover RAG Platform to Prevent Health AI Hallucinations
Ai4
Deploying standalone large language models in healthcare risks dangerous hallucinations, requiring organizations to anchor conversational AI to vetted internal repositories with strict diagnostic refusal guardrails.
## Background
Published by the Ai4 channel on YouTube on August 18, 2026, this presentation features Runa Bhaumik, a Research Associate Professor at the University of Illinois Chicago. According to the video description, the session introduces Discover, a Retrieval-Augmented Generation (RAG) conversational AI system built to provide safe, evidence-based, and source-linked access to fragmented mental health and psychiatric rehabilitation resources.
## TL;DR
Runa Bhaumik presents Discover, a modular RAG-based platform developed to overcome the limitations of traditional keyword search across scattered mental health repositories. She outlines the operational and safety hazards of using standalone commercial LLMs in clinical contexts, contrasting them with Discover's decoupled offline indexing and online grounded retrieval pipelines. Supported by specialized modular engines—Enceladus, Qdrant, Phobos, and Deimos—the system enforces hard clinical guardrails that prohibit autonomous diagnosis or treatment advice. Concluding with findings from multidisciplinary expert evaluations, Bhaumik details how this retrieval architecture can be safely adapted to other high-stakes domains through human-in-the-loop governance.
## Atmosphere & style
The talk carries a pragmatic, scholarly conference tone, seamlessly bridging rigorous biostatistical engineering with deep public health empathy. Bhaumik speaks with grounded authority and accessible warmth, infusing collegial charm by playfully pitching her student software architect to industry employers in the audience. Watching the presentation feels reassuring and soberly realistic, balancing excitement for semantic search with an unyielding commitment to patient safety, source attribution, and clinician-guided boundaries.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) Developing effective clinical AI tools requires multidisciplinary collaboration between biostatistics, computer science engineering, and mental health domain researchers.
- [01:19](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=79s) Traditional keyword search fails for mental health support because valuable information is fragmented across disparate modalities and users naturally ask conversational questions rather than searching exact keywords.
- [03:42](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=222s) Standalone commercial LLMs pose severe risks in healthcare settings due to factual hallucinations, data privacy constraints, lack of source attribution, and temporal knowledge cutoffs.
- [05:20](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=320s) Discover resolves standalone model limitations through a decoupled RAG architecture that separates offline semantic document indexing from online query retrieval and grounded generation.
- [07:41](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=461s) Strict safety guardrails prevent the AI from offering medical diagnoses or prescribing treatments, automatically filtering out weak semantic retrievals to avoid speculative generation.
- [07:41](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=461s) Human-in-the-loop expert curation ensures that all ingested PDFs, toolkits, and webinars are vetted for clinical accuracy and enriched with standardized metadata for precise retrieval.
- [09:08](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=548s) The platform utilizes a modular technical stack: Enceladus for ingestion and semantic preprocessing, Qdrant for vector storage, Phobos for conversational RAG orchestration, and Deimos for the web UI.
- [10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) The underlying RAG architecture is domain-agnostic and can be repurposed across finance, law, or public health by swapping the knowledge base and configuring domain-specific refusal boundaries.
- [10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) Close collaboration between clinicians and AI engineers is essential for implementing nuanced refusal filters and crisis signposting when handling high-risk topics like suicidal ideation.
- [13:16](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=796s) Healthcare organizations should prioritize retrieval-augmented systems over raw foundation models, enforce strict data governance, and co-design interfaces directly around end-user and clinician workflows.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) Introduction and Team Overview
Runa Bhaumik introduces herself as a Research Associate Professor at the University of Illinois Chicago, working within the biostatistics and AI research program where her team develops statistical and artificial intelligence models for healthcare practitioners. Presenting work from a federally funded study, she highlights the multidisciplinary team behind the project, including Dr. Cook—Professor of Psychiatry and Director of the Center on Mental Health Services Research and Policy—and Jessica, the Associate Director.
She makes a special point to spotlight Lokesh, a master's student in computer science who serves as the senior software engineer and AI architect. Bhaumik humorously notes that she is actively promoting him to the industry professionals in the audience, emphasizing that "he has done most of this work" and is currently navigating the job market.
[01:19](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=79s) The Information Fragmentation Problem in Mental Health Resources
The primary motivation behind the project stems from the Center on Integrated Health Care and Self-Directed Recovery, an organization dedicated to advancing knowledge and utilization of innovative models promoting health, recovery, and employment for individuals living with mental health conditions. While the center hosts an extensive body of valuable, trusted information, it is heavily fragmented across multiple modalities and locations—spanning disparate websites, PDF documents, videos, training materials, research pages, policies, toolkits, webinars, and podcasts.
```mermaid
flowchart TD
accTitle: Information Fragmentation to Conversational Access
subgraph S1["Fragmented Trusted Resources"]
A["Center Knowledge Base"] --> B1["PDFs & Toolkits"]
A --> B2["Webinars & Videos"]
A --> B3["Research Pages & Policy"]
end
subgraph S2["Traditional Access Barrier"]
B1 & B2 & B3 --> C["Keyword Search Failure<br/>(Requires Exact File Names / Terminology)"]
end
subgraph S3["Discover AI Platform"]
C --> D["Discover RAG System"]
E["Natural Language User Query"] --> D
D --> F["Source-Linked Grounded Answer"]
end
```
Navigating these scattered repositories creates severe friction for end users. Individuals rarely know exact document titles, file names, or specialized clinical terminology, rendering traditional keyword search engines largely ineffective. When people search, they instinctively ask natural language questions such as "Who can I contact?", "Do you have resources about wellness?", or "What are the materials for families?" The Discover project was initiated specifically to alleviate this search burden by offering conversational, natural-language access grounded directly in trusted organizational materials.
- **Target Population:** Individuals with mental health conditions, families, clinicians, and researchers seeking evidence-based recovery and employment resources.
- **Root Cause of Search Failure:** Mismatch between rigid keyword matching engines and how real humans articulate inquiries in natural dialogue.
[03:42](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=222s) Limitations of Standalone LLMs in Healthcare
Deploying generic, standalone Large Language Models (LLMs) directly into healthcare settings presents serious clinical and operational risks. Standard LLMs operate by predicting probable word sequences, often delivering answers that sound remarkably confident and fluent but completely lack verifiable evidentiary backing. This hallucination risk is compounded by knowledge cutoff dates that exclude the latest clinical literature and training gaps regarding private or local institutional protocols.
Bhaumik emphasizes that general-purpose commercial chatbots like ChatGPT are unsuitable for proprietary healthcare knowledge due to critical data privacy constraints. Unconstrained foundational models also suffer from prompt ambiguity and lack systematic retrieval constraints that force models to cite verified sources, making direct reliance on off-the-shelf LLMs unacceptable for patient-facing or clinical decision-support scenarios.
- **Primary Failure Modes of Standalone LLMs:**
- Hallucinated text lacking empirical evidence.
- Inability to ingest private or sensitive institutional documents.
- Absence of forced source citations and attribution constraints.
- Temporal knowledge cutoffs rendering protocols outdated.
[05:20](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=320s) The Discover Architecture: Offline Indexing and Online Retrieval
To resolve the hallucination and privacy dilemmas, the team implemented a Retrieval-Augmented Generation (RAG) framework structured into two decoupled pipelines: an offline indexing stage and an online conversational query stage.
During offline indexing, organizational documents, toolkits, and PDFs undergo semantic chunking and are transformed via embedding models into numerical vectors before being stored alongside rich metadata in a private vector database. During the online query stage, a user's natural-language inquiry is converted into a query embedding, matched against the vector database via similarity search to retrieve relevant text chunks, augmented into a structured prompt, and passed to the language model to generate a strictly grounded, source-linked answer.
- **Offline Indexing:** Ingests internal PDFs, performs semantic chunking, computes dense vector representations, and populates vector storage with custom metadata.
- **Online Processing:** Encodes real-time user queries, conducts nearest-neighbor semantic search, constructs grounded context prompts, and streams transparent responses back to the user interface.
[07:41](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=461s) Safety Guardrails and Expert Curation
In high-stakes mental health environments, technological design must be governed by rigorous safety protocols. The Discover platform is explicitly architected to support, rather than replace, clinical and professional judgment. The system actively avoids formulating diagnostic assessments or generating treatment recommendations, filtering out weak retrievals to prevent speculative generation.
A core foundation of the platform's reliability is human-in-the-loop expert curation. Subject-matter experts and research collaborators meticulously vetted all ingested materials—including HTML pages, webinars, policy documents, and podcasts—for clinical accuracy and relevance. Each ingested resource was enriched with standardized metadata and domain-specific tags, allowing the retrieval mechanism to execute precise filtering and transparent source attribution.
- **Core Safety Mechanisms:**
- Hard guardrails against providing diagnoses or prescribing medical treatments.
- Automated thresholding to discard weak or ambiguous semantic retrievals.
- Mandatory source attribution referencing verified organizational repositories.
- System boundary constraints designed strictly for resource discovery rather than autonomous clinical intervention.
[09:08](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=548s) Modular Technical Stack and Expert Evaluation
The Discover architecture is organized into modular components designed for maintainability and technical reusability. The platform incorporates Enceladus for document ingestion and semantic preprocessing, a Qdrant vector database for high-performance vector storage and metadata filtering, Phobos for conversational retrieval logic and grounded prompt construction, and Deimos for the client-facing web chat interface.
The system was subjected to rigorous expert evaluation by mental health researchers and clinicians. Reviewers systematically evaluated factual accuracy, retrieval quality, completeness, clarity, metadata tagging precision, and conversational safety, ensuring that every generated response adhered strictly to evidence-based standards.
- **Platform Modules:**
- **Enceladus:** Automated document ingestion, parsing, chunking, and semantic preprocessing.
- **Qdrant:** Dense vector indexing and metadata tag filtering.
- **Phobos:** Conversational RAG pipeline, context augmentation, and LLM orchestration.
- **Deimos:** Interactive, accessible web interface for end users.
- **Evaluation Dimensions:** Retrieval precision, response factual fidelity, citation transparency, and linguistic clarity.
[10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) Cross-Domain Adaptability and Human-in-the-Loop Governance
Bhaumik emphasizes that the foundational design pattern—*ingest, embed, retrieve, augment, prompt, generate grounded response, and cite resources*—is inherently domain-agnostic. While the underlying computational framework remains identical, transitioning the system to adjacent sectors simply requires substituting the specialized knowledge base, adjusting metadata tagging schemas, configuring domain-specific refusal rules, and defining targeted evaluation benchmarks.
| Target Sector | Core Domain Ingestion | Key Safety & Refusal Boundaries |
|:--- |:--- |:--- |
| **Mental Health (Current)** | Psychiatric rehab toolkits, webinars, recovery guides | Prohibits diagnostic claims, prescreening crisis language (e.g., suicide terms) |
| **Financial Services** | Regulatory filings, compliance manuals, internal policies | Prohibits automated investment advice or unvetted financial recommendations |
| **Legal Systems** | Case law, statutory codes, jurisdictional guidelines | Prohibits unauthorized legal advice or speculative case outcomes |
| **Public Health & Government** | Municipal ordinances, benefit guidelines, clinic schedules | Restricts answers to officially ratified public health communications |
Bhaumik stresses that successful deployments require multidisciplinary human collaboration across the entire lifecycle. Data engineers and AI engineers build the retrieval pipelines, but domain experts are indispensable for verifying outputs and defining critical safety guardrails. When institutional funders raised concerns regarding sensitive topics such as suicidal ideation, domain experts collaborated with the engineering team to calibrate refusal filters and prioritize immediate safety-focused resource signposting.
[13:16](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=796s) Practical Recommendations for Trustworthy Healthcare AI
Concluding her talk, Bhaumik provides concrete recommendations for organizations aiming to deploy conversational AI in healthcare and related fields. She urges practitioners to adopt RAG architectures rather than relying on standalone LLMs, noting that grounding generation in vetted internal data preserves privacy while eliminating speculative hallucinations.
Furthermore, she argues that genuine AI safety extends far beyond selecting powerful foundation models. It demands strict data governance, curated source repositories, proactive user feedback integration, and co-designing tools directly around existing clinician and consumer workflows. Because inaccurate information in healthcare directly affects patient well-being, systems must be built to answer only when verifiable evidentiary support is present.
- **Primary Recommendations:**
- Default to retrieval-augmented architectures over raw foundation models to ensure data privacy and factual grounding.
- Enforce strict governance by maintaining curated, approved knowledge bases.
- Co-design interfaces with service providers, clinicians, and consumers to solve authentic workflow bottlenecks rather than simply demonstrating technology.
- Establish clear boundary limits where the chatbot declines to answer if retrieved evidence is insufficient.
## Notable quotes
> What if finding trusted mental health information was as easy as asking a question—without worrying whether the answer is accurate, evidence-based, or safe? — Runa Bhaumik
> [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) he has done most of this work — Runa Bhaumik
> [01:19](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=79s) Who can I contact?
> [01:19](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=79s) Do you have resources about wellness?
> [01:19](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=79s) What are the materials for families?
> In this talk, I will present Discover, a Retrieval-Augmented Generation (RAG)-based conversational AI platform developed to improve access to trusted mental health and psychiatric rehabilitation resources. — Runa Bhaumik
> where traditional keyword search frequently falls short. — Runa Bhaumik
> [10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) ingest, embed, retrieve, augment, prompt, generate grounded response, and cite resources — Runa Bhaumik
## Mentioned
Runa Bhaumik, University of Illinois Chicago, Dr. Cook, Jessica, Lokesh, Center on Mental Health Services Research and Policy, Center on Integrated Health Care and Self-Directed Recovery, Discover, Enceladus, Phobos, Deimos, Qdrant, ChatGPT, Ai4
## Video description
Runa Bhaumik, Research Associate Professor, University of Illinois Chicago
What if finding trusted mental health information was as easy as asking a question—without worrying whether the answer is accurate, evidence-based, or safe?
In this talk, I will present Discover, a Retrieval-Augmented Generation (RAG)-based conversational AI platform developed to improve access to trusted mental health and psychiatric rehabilitation resources. I will explain how we addressed the challenge of locating reliable information that is often scattered across websites, PDFs, videos, webinars, toolkits, and educational repositories, where traditional keyword search frequently falls short.
I will demonstrate how Discover combines semantic search, vector databases, large language models, and conversational AI to allow users to ask questions in natural language and receive grounded, source-linked responses from trusted organizational resources. The talk will illustrate the platform's modular architecture—including Enceladus for resource ingestion and semantic processing, Phobos for conversational retrieval and response generation, and Deimos for the web-based conversational interface—and discuss the safety guardrails that promote trustworthy, transparent, and responsible AI.
Finally, I will share lessons learned from expert evaluations conducted with mental health professionals and service researchers, highlighting how conversational AI can improve resource discovery, usability, and accessibility while providing practical guidance for organizations seeking to implement responsible, evidence-based AI solutions in healthcare and beyond.
Runa Bhaumik and her research team at the University of Illinois Chicago developed Discover, a retrieval-augmented generation (RAG) platform that provides conversational access to evidence-based mental health resources while mitigating the hallucination and privacy risks of generic large language models.
Resource Fragmentation and Keyword Search Limitations (1:19)
The Center on Integrated Health Care and Self-Directed Recovery curates extensive materials promoting health, recovery, and employment for individuals with mental health conditions. However, this knowledge base has historically been fragmented across varied media, including toolkits, research pages, webinars, policy briefs, and podcasts. Navigating these disparate repositories presents significant barriers to consumers, families, clinicians, and researchers.
Traditional keyword search engines consistently fail because users rarely know exact document titles, file names, or specialized clinical terminology. Instead, individuals naturally formulate conversational inquiries, asking questions such as "Who can I contact?" or "Do you have resources about wellness?" The Discover initiative was designed to bridge this gap, replacing rigid keyword matching with natural-language access directly grounded in trusted institutional literature.
Clinical and Privacy Risks of Standalone Models (3:42)
Bhaumik emphasizes that deploying unconstrained, off-the-shelf commercial language models like ChatGPT directly into healthcare environments introduces severe operational and clinical hazards. Standard models generate responses based on probable word sequences, which often yields fluent, authoritative-sounding text that lacks empirical foundation or verifiable evidentiary backing.
Beyond hallucination, commercial foundation models are restricted by temporal training cutoffs and cannot safely ingest proprietary, sensitive, or localized institutional protocols due to critical privacy constraints. Furthermore, because standalone models lack architectural mechanisms to compel source attribution, they are ill-suited for patient-facing or clinical decision-support settings where accuracy and provenance are mandatory.
Decoupled Architecture and Modular Technical Stack (5:20)
To overcome the limitations of raw foundation models, Discover implements a decoupled RAG architecture divided into offline indexing and online query processing. In the offline pipeline, curated documents undergo semantic chunking and dense vector embedding, followed by storage in a Qdrant vector database alongside standardized metadata tags. In the online pipeline, incoming user queries are embedded in real time, matched against vector storage via nearest-neighbor semantic search, augmented into a structured prompt, and processed by an LLM to generate source-attributed responses.
The system is engineered into modular components to support long-term maintainability: Enceladus handles automated document ingestion and semantic preprocessing; Qdrant manages vector storage and metadata filtering; Phobos orchestrates conversational retrieval and prompt engineering; and Deimos serves as the client-facing conversational interface.
Safety Guardrails and Multidisciplinary Human Governance (7:41)
Operating in mental health necessitates strict safety controls. Discover is architecturally constrained to function exclusively as an informational resource discovery tool rather than an autonomous clinical practitioner; it actively prohibits diagnostic formulations and treatment prescriptions. To prevent speculative generation, the platform applies automated thresholding to discard weak or ambiguous semantic retrievals.
System integrity relies on continuous human-in-the-loop governance. Mental health clinicians and researchers vetted all ingested materials and collaborated with AI engineers to calibrate safety boundaries. For high-risk inquiries, such as those involving suicidal ideation, the team established refusal filters that divert users toward immediate safety resources. Bhaumik notes that while this technical pipeline is domain-agnostic and adaptable to law, finance, or government, cross-disciplinary expert evaluation remains essential across the deployment lifecycle.
Bhaumik concludes that healthcare AI safety depends on using retrieval-augmented architectures grounded in vetted internal knowledge bases rather than unconstrained foundation models. She stresses that systems must be co-designed around actual clinical workflows, maintain strict human-in-the-loop oversight, and enforce explicit boundary limits that decline to answer when verified evidentiary support is absent.
精炼阅读摘要
伊利诺伊大学芝加哥分校的 Runa Bhaumik 及其跨学科团队开发了名为 Discover 的检索增强生成（RAG）聊天机器人，旨在解决心理健康资源严重碎片化的痛点，并通过严格的专家审核与安全护栏，规避通用大语言模型在医疗领域的幻觉与隐私安全风险。
心理健康资源碎片化与传统检索失效 (1:19)
综合健康护理与自主康复中心（Center on Integrated Health Care and Self-Directed Recovery）汇集了大量旨在促进精神健康康复与就业的可靠资料，但这些宝贵资源高度分散在不同网站、PDF 文档、视频、培训手册、研究页面、政策文件、研讨会录像及播客中。多模态且割裂的存储形态给寻求循证支持的患者、家庭、临床医护人员与科研人员带来了严重的信息获取阻碍。
传统的关键词搜索引擎要求用户输入精准的文件名或专业医学术语，而普通用户在搜索时往往习惯使用自然语言，例如提问“我可以联系谁？”、“是否有关于身心健康的资源？”或“有哪些面向家庭的材料？”。由于刚性关键词匹配机制与人类日常语言习惯存在根本性脱节，传统搜索常常失效，迫切需要引入能够理解自然语义并准确锚定权威材料的对话式检索方案。
独立大模型的临床应用缺陷与安全边界 (3:42)
Bhaumik 指出，在医疗健康场景中直接依赖通用的独立大语言模型（如 ChatGPT 等商用产品）存在巨大的临床与操作风险。通用大模型本质上基于词元概率进行预测，容易生成语气极其自信但缺乏事实依据的幻觉内容；同时，其知识库受限于训练截止日期，无法及时涵盖最新的临床指南，更无法直接摄入机构内部的专有与敏感数据。
为确保在敏感医疗环境中的可靠性，Discover 进行了明确的系统边界约束，其核心定位是辅助资源发现，坚决不替代医生的临床判断。系统通过硬性规则严密禁止提供任何疾病诊断或治疗建议；一旦检索系统返回的语义匹配置信度较弱或模糊，系统会自动执行过滤，拒绝给出推测性答复，防止产生误导性建议。
双阶段检索增强架构与模块化技术栈 (5:20)
Discover 架构由离线索引与在线查询两阶段构成。在离线阶段，专家筛选的机构内部资料、工具包和文档经过语义切片，通过向量模型转化为高维稠密向量，并附带领域元数据保存在专有向量数据库中；在在线阶段，系统将用户的自然语言问题实时编码为向量并发起相似度匹配，检索出最相关的文本块后将其整合进结构化提示词中，驱动模型输出严格基于原始来源且附带引文字符串的答复。
在底层实现上，系统采用了模块化工程设计：Enceladus 负责自动化文档摄入、解析与语义预处理；高性能向量数据库 Qdrant 承载向量存储与细粒度元数据过滤；Phobos 充当对话检索 API 引擎以协调上下文扩充与大模型调用；Deimos 则作为面向终端用户的无障碍网页交互界面。该系统已经过精神健康专家与临床医生的系统性评估，证实其在事实准确性、检索召回精度以及语言清晰度方面均达到专业标准。
人机协同治理与跨领域迁移机制 (10:36)
Discover 的安全性与可靠性高度依赖全周期的多学科协同治理。临床学科专家参与审核了所有摄入系统的 HTML、研讨会材料和政策文本，确保医疗事实无误，并制定了标准化元数据标签。针对自杀意念等危机情况的伦理关切，专家与工程师协同校准了安全拦截规则，确保系统遇到自杀危机相关表述时坚决拒绝随意发挥，并立即提示用户获取专业安全救助资源。
Bhaumik 强调，该系统所采用的“摄入、嵌入、检索、扩充、提示生成与源引标注”计算模式完全具备领域通用性。只要替换底层领域专有知识库、调整元数据标签结构并重新设定拒答安全红线，相同的架构便可直接迁移至金融服务（禁止自动化投资建议）、法律合规（防范未经许可的法律咨询）或公共卫生政务等高度要求事实严谨性的专业领域。
Bhaumik 总结强调，在医疗健康等高风险垂直领域部署 AI 时，开发者应优先选择 RAG 检索增强架构以确保数据私密性并消除模型幻觉，而非单纯依赖基础大模型；同时必须建立严密的人机协同治理机制与审核知识库，紧密贴合医患人员的真实工作流进行工具设计，并在缺乏可验证事实证据时强制执行拒答策略。
Detailed notes — zh
## Background
本内容由 Ai4 频道于 2026 年 8 月 18 日在 YouTube 上发布。视频记录了伊利诺伊大学芝加哥分校（UIC）研究副教授 Runa Bhowmick 的专题演讲，重点介绍了基于检索增强生成（RAG）的对话式 AI 平台 Discover，该平台旨在帮助用户在海量碎片化数据中快速、安全地检索受信任的循证心理健康与精神康复资源。
## TL;DR
针对传统关键词检索难以理解自然语言以及通用大语言模型（LLM）容易产生事实幻觉和安全隐患的痛点，UIC 跨学科团队开发了 Discover 平台。该系统通过离线索引与在线查询两阶段的 RAG 架构，将经过领域专家审核的 PDF、网页、多模态教学材料等转化为向量知识库，结合 Qdrant 和模块化组件实现精准语义召回。系统设计了严格的医疗安全护栏，杜绝独立 LLM 的盲目生成与医疗诊断行为，强制要求提供溯源引用并在缺乏证据或涉及高危敏感话题时安全拒答。演讲还强调了“人在回路”在数据治理和系统评估中的核心价值，为医疗及其他严肃行业落地可信 AI 提供了系统性的实践范式。
## Atmosphere & style
整个演讲洋溢着严谨、务实且兼具人文关怀的学术工程气质。演讲者 Runa Bhowmick 以沉稳清晰的叙事节奏展开，从实际患者与临床人员获取资源的痛点切入，逐步拆解深奥的技术架构与安全设计，条理极其分明。现场氛围专注而理性，既没有夸大 AI 能力的商业浮夸，而是直面大模型在医疗领域的幻觉与伦理风险，让听众深刻感受到工程严谨性与临床责任感并重的专业力量。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) 心理健康与精神康复领域的权威资源高度分散在网站、PDF、视频等多种媒介中，传统关键词搜索难以满足用户的自然语言检索需求。
- [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) 针对心理健康资源的碎片化痛点，团队研发了名为 Discover 的检索增强生成（RAG）对话式 AI 平台，以降低信息获取门槛。
- [03:42](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=222s) 通用大语言模型（LLM）由于训练数据过时、缺乏机构私有知识和证据约束，容易产生具有误导性的“幻觉”，在医疗领域直接使用存在安全隐患。
- [03:42](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=222s) 出于医疗健康场景的隐私与合规要求，机构专有数据无法直接上传至公开商业 LLM，必须依赖私有化 RAG 架构进行本地索引与检索。
- [03:42](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=222s) Discover 架构分为离线索引与在线查询两个阶段，通过文本分块、向量嵌入以及向量数据库相似度匹配，为大模型生成提供权威证据支持。
- [07:16](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=436s) 系统确立了严格的医疗安全护栏，每条回答必须严格附带原始出处链接，且严禁直接提供医疗诊断或处方，定位为专业医疗判断的辅助工具。
- [07:16](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=436s) Discover 采用了由资源摄取（Enceladus）、向量库（Qdrant）、RAG API（Phobos）和 Web 对话界面（Deimos）组成的解耦模块化架构。
- [10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) 该 RAG 技术框架具备通用可复用性，只需更换受信任知识库、重构过滤策略并定制行业安全边界，即可迁移至金融、法律等其他严肃领域。
- [10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) 构建可信 AI 必须实行“人在回路”（Human-in-the-Loop），从数据清洗、标签标注到评测验证的全生命周期均需领域专家深度参与。
- [12:15](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=735s) 心理健康 AI 对“自杀”、“自残”等高危敏感词设立了前置过滤与干预机制，确保不会脱离私有知识库随意发挥引发危机风险。
- [12:15](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=735s) 开发严肃领域的对话式 AI 需要数据工程师、AI 工程师与临床领域专家跨学科紧密协作，并通过真实用户反馈形成持续迭代闭环。
- [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 在医疗健康等高严谨度领域应坚决避免使用独立的、无约束的通用 LLM，必须通过 RAG 架构将大语言模型限制在可信知识库范围内。
- [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 打造医疗可信 AI 的核心在于建立严谨的治理体系，坚持在证据不足时安全拒答，确保输出结果绝不影响患者的治疗与护理安全。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) 团队背景与心理健康资源获取痛点
伊利诺伊大学芝加哥分校（University of Illinois Chicago）生物统计与 AI 研究项目的研究副教授 Runa Bhowmick 介绍了由联邦资助的最新研究项目。该跨学科团队由精神病学教授兼心理健康服务研究与政策中心主任 Cook 博士领导，Jessica 担任副主任，硕士研究生 Lokesh 担任资深软件工程师与 AI 架构师并承担了系统的大部分开发工作。
项目的核心合作方是综合医疗保健与自我导向康复中心（Center on Integrated Health Care and Self-Directed Recovery），该中心致力于推广促进心理健康患者康复、就业与健康的创新模式。然而，中心面临一个严峻的信息访问挑战：虽然拥有海量权威的循证资源，但这些内容分散在网站、PDF 文档、视频、培训材料、研究论文、政策文件和工具包等各种媒介中。
传统的关键词搜索在这一场景下频频失效。真实用户通常使用自然语言提问（例如“我能联系谁？”、“你们有关于健康的资源吗？”、“给家庭的资料在哪里？”），他们并不清楚具体的文件名、文章标题或专业术语。为此，团队启动了名为 Discover 的检索增强生成（RAG）对话式 AI 平台，旨在通过自然语言交互消除信息检索负担，使用户能够轻松发现受信任的心理健康资源。
- **核心研究团队**：Runa Bhowmick 教授、Cook 博士（中心主任）、Jessica（副主任）、Lokesh（AI 架构师 / 开发者）。
- **主要解决痛点**：心理健康与精神康复资源高度碎片化，传统关键词匹配难以理解自然语言查询。
---
### [03:42](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=222s) 大语言模型（LLM）的局限性与 RAG 解决方案
通用大语言模型（LLM）在专业医疗健康领域存在明显的局限性。LLM 往往会生成表面听起来自信但缺乏事实依据的“幻觉”（Hallucination），原因包括训练知识过时、存在知识截止日期（Cut-off）、缺乏机构私有知识以及未学习过本地医疗规范。此外，提示词的歧义性和缺乏强制引用证据的约束，导致直接使用通用模型存在重大安全风险。出于医疗健康领域的隐私合规考量，机构内部的专有文档和数据也无法直接上传至公开的商业 LLM（如 ChatGPT）。
为了解决这些问题，团队采用了检索增强生成（Retrieval-Augmented Generation, RAG）技术架构，将整个系统流程划分为**离线索引（Offline Indexing）**和**在线查询（Online Query）**两个主要阶段：
```mermaid
flowchart TD
accTitle: RAG架构离线与在线处理流程图
subgraph Offline["离线索引阶段 (Offline Indexing)"]
A["私有文档/PDF/网页"] --> B["文本分块 (Chunking)"]
B --> C["向量嵌入 (Embedding)"]
C --> D[("向量数据库 (Vector DB) & 元数据")]
end
subgraph Online["在线查询阶段 (Online Query)"]
E["用户自然语言提问"] --> F["查询向量化 (Query Embedding)"]
F --> G["相似度搜索与检索"]
D -.-> G
G --> H["增强提示词 (Augmented Prompt)"]
H --> I["LLM 生成有证据支持的回答"]
end
```
- **离线索引**：将本地 PDF 和各种文档进行分块（Chunking），转换为数值向量（Embeddings），连同元数据（Metadata）存入私有向量数据库，构建专有知识库。
- **在线查询**：用户输入自然语言问题后，系统将其转化为查询向量，在向量数据库中进行相似度检索，提取最相关的权威文档片段，拼接为提示词送入 LLM，生成具有严格证据溯源的回答。
---
### [07:16](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=436s) Discover 系统的模块化架构与医疗安全护栏
Discover 平台融合了语义搜索、向量数据库（如 Qdrant）、大语言模型、对话记忆和来源归因（Source Attribution）技术。在医疗健康场景中，安全设计（Safety Design）至关重要。平台设立了严格的安全护栏：限制仅在受信任的资源范围内检索生成；主动过滤低相关度的弱检索结果；每条回答均附带权威来源链接；严禁提供具体的医学诊断或治疗方案，定位为“辅助而非替代专业医疗判断”（"Support not replace the professional judgment"）。
系统底层由领域专家对多模态资源（HTML、PDF、视频、播客、网络研讨会等）进行精细化治理与质量审查，并为每个资源赋予标准化元数据和特定主题标签。在技术实现上，Discover 采用了高度解耦的模块化组件（如用于资源摄取与语义处理的 Enceladus、向量数据库 Qdrant、处理对话 RAG API 的 Phobos 以及 Web 聊天界面的 Deimos），形成了“摄取 $\to$ 嵌入 $\to$ 检索 $\to$ 增强 $\to$ 提示 $\to$ 生成溯源回答 $\to$ 展示来源”的标准复用流程。
| 评估维度 | 核心检验指标与要求 |
|:--- |:--- |
| **事实准确性 (Factual Accuracy)** | 回答必须严格忠实于检索到的权威文档，杜绝幻觉 |
| **检索质量 (Retrieval Quality)** | 确保能够精准召回最契合用户意图的高质量知识片段 |
| **来源归因 (Source Attribution)** | 回答必须提供透明、可跳转追溯的原始出处链接 |
| **标签与元数据质量 (Tagging Quality)** | 主题分类与元数据标注需经专家审核，确保规范统一 |
| **安全性与边界 (Safety & Guardrails)** | 严禁诊断与处方推荐，遇到超纲或危险提问安全拒答 |
---
### [10:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=636s) 跨领域复用性与专家全程参与
Discover 的技术架构具备通用跨领域迁移能力。当迁移到金融服务、法律系统、政府政务、公共卫生或制造业等新领域时，底层的 RAG 架构保持不变，只需针对新场景进行定制适配：
1. **更新受信任知识库**：替换为特定领域的高质量文档与数据源；
2. **重构过滤与引用策略**：根据行业规范制定信息筛选与证据呈现方式；
3. **定制安全边界与拒答规则**：定义聊天机器人在该行业必须拒绝回答的敏感边界；
4. **针对性构建测试用例**：由该行业专家设计评估基准与测试场景，并适配对应的交互界面。
Bhowmick 教授特别强调了“人在回路”（Human Involvement）在构建可信 AI 应用中的决定性作用。在医疗健康等高严谨度行业中，从最开始的内容摄取、数据清洗，到标签制定、规则设计和系统评测，整个生命周期的每一步都必须紧密依靠领域专家和临床专业人员的深度参与与把关。
### [12:15](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=735s) 跨学科团队构成、安全护栏与用户反馈迭代
开发一个面向心理健康领域的可靠 AI 系统，需要多个不同角色紧密协作。在技术实施层面，数据索引与检索流程需要数据工程师（Data Engineer）来打磨，团队中由硕士生和研究员共同承担这一工作；同时，算法架构的搭建离不开 AI 工程师（AI Engineer）。然而，仅有技术人员是远远不够的，领域专家（Domain Expert）的深度参与至关重要。心理健康资源的专业性极强，系统检索并生成的每一条结果是否具备医学与学术价值、是否真实可靠，必须由领域专家进行严格的审核与验证。
```mermaid
flowchart TD
accTitle: Discover 平台跨学科协作与安全机制
subgraph Team["跨学科协作角色"]
DE["数据与 AI 工程师<br/>(索引、检索与模型开发)"]
EXP["领域专家 / 临床学者<br/>(内容审核与价值验证)"]
end
subgraph Guardrails["安全与护栏机制"]
FILTER["关键词过滤与风险监控<br/>(自杀/自伤等敏感词防护)"]
GROUND["私有知识库约束<br/>(禁止脱离证据自由发挥)"]
end
subgraph Feedback["交付与评估闭环"]
APP["Discover 应用上线 / Demo 演示"]
USER["用户反馈收集<br/>(临床工作者、科研人员、终端用户)"]
end
Team --> Guardrails
Guardrails --> APP
APP --> USER
USER --> Team
```
在系统安全护栏（Safety and Guardrails）的设计上，团队面临着来自资助方及业界的直接关切。在心理健康领域，诸如“自杀”等高危敏感词汇极为敏感，如果大语言模型（LLM）在没有约束的情况下随意生成内容，可能会带来严重的心理危机甚至生命危险。为此，团队构建了严格的过滤与干预机制，确保对高危场景进行安全管控。目前，Discover 平台已经构建完成并上线了 Demo，团队正在通过展示二维码等方式积极收集来自一线真实用户的反馈数据，以便对聊天机器人进行持续的迭代与优化。
- **多角色协同开发**：数据工程师负责数据管道与索引检索，AI 工程师负责大模型与对话系统，领域专家负责结果真实性与临床价值审查。
- **高危风险控制**：针对自杀、自残等精神健康敏感词汇建立安全护栏与前置过滤机制，严防大模型幻觉与不当引导。
- **实战产品与闭环迭代**：系统已完成原型并进入实际用户反馈收集阶段，用于指导后续模型与界面的优化。
---
### [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 医疗健康领域落地可信 AI 的实践建议与治理准则
基于 Discover 项目的研发实践，研究团队为在医疗健康及其他严肃领域应用 AI 工具箱提出了明确的实践建议。最核心的准则是：**必须采用检索增强生成（RAG）架构，坚决避免使用孤立的、无约束的独立 LLM（Standalone LLMs）**。RAG 架构能够强制要求系统仅从经过审核的私有可信数据源中提取答案，回答必须严格基于提供的私有内容，而非依赖大模型自身参数生成的不可控幻觉内容。同时，团队强调必须将“通过治理建立信任”置于首位，并将 AI 系统的设计紧密融入用户的日常工作流中。
技术的目标绝不是单纯“炫技”展示 AI 应用，而是要真正解决临床医生、服务提供者、科研人员以及终端患者在获取信息时的痛点。构建可信赖的 AI（Trustworthy AI）远不止于调用一个强大的底层大语言模型，它必须依赖一整套严谨的治理体系：
1. **严格筛选知识源**：仅使用经过专业审核与认证的高质量知识库；
2. **透明溯源与引用**：提供清晰可查的原始出处链接；
3. **证据不足时拒答**：仅在具备充分证据支持的前提下作答；
4. **划定明确的专业领域边界**：不越界处理未经校验的外部问题。
在医疗与心理健康领域，任何一条不准确的信息都可能直接影响到患者的治疗与护理安全，因此这些安全防线与治理措施不仅是技术要求，更是不可逾越的底线。
## Notable quotes
> [00:00](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=0s) What if finding trusted mental health information was as easy as asking a question—without worrying whether the answer is accurate, evidence-based, or safe? — Runa Bhaumik
> [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) 我能联系谁？ — Runa Bhowmick
> [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) 你们有关于健康的资源吗？ — Runa Bhowmick
> [00:04](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=4s) 给家庭的资料在哪里？ — Runa Bhowmick
> [07:16](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=436s) Support not replace the professional judgment — Runa Bhowmick
> [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 必须采用检索增强生成（RAG）架构，坚决避免使用孤立的、无约束的独立 LLM（Standalone LLMs） — Runa Bhowmick
> [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 严格筛选知识源：仅使用经过专业审核与认证的高质量知识库 — Runa Bhowmick
> [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 透明溯源与引用：提供清晰可查的原始出处链接 — Runa Bhowmick
> [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 证据不足时拒答：仅在具备充分证据支持的前提下作答 — Runa Bhowmick
> [13:36](https://www.youtube.com/watch?v=b9l7ylt3ps0&t=816s) 划定明确的专业领域边界：不越界处理未经校验的外部问题 — Runa Bhowmick
## Mentioned
Discover, Runa Bhaumik (Runa Bhowmick), University of Illinois Chicago (伊利诺伊大学芝加哥分校), Center on Integrated Health Care and Self-Directed Recovery (综合医疗保健与自我导向康复中心), Cook 博士, Jessica, Lokesh, Enceladus, Phobos, Deimos, Qdrant, ChatGPT, Ai4
## Video description
Runa Bhaumik, Research Associate Professor, University of Illinois Chicago
What if finding trusted mental health information was as easy as asking a question—without worrying whether the answer is accurate, evidence-based, or safe?
In this talk, I will present Discover, a Retrieval-Augmented Generation (RAG)-based conversational AI platform developed to improve access to trusted mental health and psychiatric rehabilitation resources. I will explain how we addressed the challenge of locating reliable information that is often scattered across websites, PDFs, videos, webinars, toolkits, and educational repositories, where traditional keyword search frequently falls short.
I will demonstrate how Discover combines semantic search, vector databases, large language models, and conversational AI to allow users to ask questions in natural language and receive grounded, source-linked responses from trusted organizational resources. The talk will illustrate the platform's modular architecture—including Enceladus for resource ingestion and semantic processing, Phobos for conversational retrieval and response generation, and Deimos for the web-based conversational interface—and discuss the safety guardrails that promote trustworthy, transparent, and responsible AI.
Finally, I will share lessons learned from expert evaluations conducted with mental health professionals and service researchers, highlighting how conversational AI can improve resource discovery, usability, and accessibility while providing practical guidance for organizations seeking to implement responsible, evidence-based AI solutions in healthcare and beyond.
[Source](https://www.youtube.com/watch?v=b9l7ylt3ps0)
Source quotes
[0:00] Runa Bhaumik: What if finding trusted mental health information was as easy as asking a question—without worrying whether the answer is accurate, evidence-based, or safe?
[0:04] Runa Bhowmick: 我能联系谁？
[0:04] Runa Bhowmick: 你们有关于健康的资源吗？
[0:04] Runa Bhowmick: 给家庭的资料在哪里？
[7:16] Runa Bhowmick: Support not replace the professional judgment
[13:36] Runa Bhowmick: 必须采用检索增强生成（RAG）架构，坚决避免使用孤立的、无约束的独立 LLM（Standalone LLMs）
[13:36] Runa Bhowmick: 严格筛选知识源：仅使用经过专业审核与认证的高质量知识库
[13:36] Runa Bhowmick: 透明溯源与引用：提供清晰可查的原始出处链接
[13:36] Runa Bhowmick: 证据不足时拒答：仅在具备充分证据支持的前提下作答
[13:36] Runa Bhowmick: 划定明确的专业领域边界：不越界处理未经校验的外部问题
Source quotes
Runa Bhaumik: What if finding trusted mental health information was as easy as asking a question—without worrying whether the answer is accurate, evidence-based, or safe?
[0:04] Runa Bhaumik: he has done most of this work
[1:19] Who can I contact?
[1:19] Do you have resources about wellness?
[1:19] What are the materials for families?
Runa Bhaumik: In this talk, I will present Discover, a Retrieval-Augmented Generation (RAG)-based conversational AI platform developed to improve access to trusted mental health and psychiatric rehabilitation resources.
Runa Bhaumik: where traditional keyword search frequently falls short.
[10:36] Runa Bhaumik: ingest, embed, retrieve, augment, prompt, generate grounded response, and cite resources
Original transcript
[0:04] Okay, my name is Runa Bhowmick. I'm a research
associate professor at University of Illinois
[0:10] at Chicago. And I am in the program of
biostatistics and AI research program.
[0:17] We develop you know statistical models and AI
models for the health care practitioners. So
[0:26] I promise you that you know my talk will
not be that long, but Yeah, so I will
[0:32] I will today you know I will talk about that
the research project we are doing right now
[0:38] through a federally funded study. And this is
our team. Dr. Cook, he is actually the professor
[0:45] of psychiatry director of the center of mental
health services research and policy. And Jessica
[0:51] is associate director. And Lokesh is our you know
the MS student who is you know computer scientist,
[0:58] senior software engineer and AI architect. And
he actually that's the another reason that you
[1:04] know he is on the job market. So he has done most
of this work. So I am you know just promoting him
[1:10] you know his work through the the organization,
the industrial people that we have over here.
[1:19] So the problem was actually we have a
center on integrated health care and
[1:24] self-directed recovery you know that's the
name. So they are dedicated to advance the
[1:30] advancing the knowledge and utilization of
innovative models to promote health, recovery,
[1:36] and employment for people with mental health
conditions. So this is their site and we have also
[1:45] that QR code you can go to that site over here.
[1:50] So what's the problem now that the center site
has an enormous amount of valuable information
[1:57] and that those information are spread across
websites, PDF documents, videos, uh training
[2:04] materials, research pages, policies, and toolkits.
Although, you know, these resources are trusted,
[2:11] but they are often difficult to navigate. So,
users generally frequently, you know, they
[2:17] spend significant time searching through multiple
sources before finding the information they need.
[2:23] So, our project is called the Discover. It was,
you know, so we uh we are creating actually this
[2:29] is ongoing process project creating to solve this
information access problem using this AI tool.
[2:37] So, the original problem statement is
the the one that I told that, you know,
[2:42] that the mental health and uh psychiatric
rehabilitation resources are very valuable, but
[2:48] they're fragmented across formats and locations.
Users may not know uh uh exact titles, file names,
[2:57] or technical search terms, so keyword search
can fail over here. So, the traditional search
[3:04] engines, you know, why the keyword search fails,
that the traditional search engine precise terms
[3:10] say real users ask in the natural language, right?
So, who can I contact? Do you have resources
[3:17] about wellness? Where what are the materials
for the families and so on. But, the search
[3:23] engine right now that we have that, you know, that
we cannot talk like this, right? We can we can
[3:28] only search by keyword. So, basically, our the
problem that we are going to solve is to reduce
[3:35] the search burden and make trusted resources
discoverable through conversational access.
[3:42] So, when we have, you know, all those PDFs,
web pages, and everything is scattered,
[3:46] so now, you know, we are trying to um uh get them
information uh that will be evidence based. Now,
[3:54] most of you, you know that the large
language models, they hallucinate,
[3:57] right? Use your questions, then use your generate
LLM generate some probable the next words,
[4:04] then answer sounds confident, you know, maybe
right, but they lack the evidence. That is,
[4:09] you know, a universal that's a problem, right?
So, we know that's a problem. So, why? Because,
[4:15] you know, the LLMs have been, you know, learned
through the outdated knowledge. So, that there is
[4:21] a tearing cut off on the old documents, then also
the private knowledge gap, whatever you know the
[4:26] private knowledge that we have in healthcare or,
you know, some other industry, that is not up to
[4:31] there. So, and so the local protocols
were never in the training process.
[4:37] Ambiguous prompt, multiple possible, you
know, interpretations and LLM can generate.
[4:43] So, and no retrieval constraint that, you know,
how the model do not force to cite the evidence.
[4:51] Though, you know, nowadays, right? If you do the
ChatGPT, you can go to the some links. So, there
[4:56] are links, so they are trying that that model, you
know, the LLM model, but I work in the healthcare,
[5:02] and so there are lots of, you know, documents and
then things that, you know, that is, you know,
[5:07] the the privately owned, so we cannot actually
put that, you know, into the ChatGPT because,
[5:14] you know, we for the privacy concern. So, that is
you know one issue with the large language models.
[5:20] So, the our solution was, you know, so to my
collaborator that, you know, okay, yeah, there
[5:25] is a tools exist, and we can, you know, use that
tool, and that tool is, you know, that's the rag,
[5:32] you know, retrieval augmented generation tool.
Uh I think next 1 2 days you will hear a lot
[5:38] about this, uh agents and, you know, AI agents and
retrieval augmented generations and those things.
[5:46] So, here, you know, the this problem. Just to
give you an idea because we all work in this,
[5:51] you know, this kind of situation can happen to any
industry. It's not only the health care or in my
[5:58] my domain. So we have done, you know, documents.
So there are two processes in rag. One is called
[6:05] the offline indexing, another one is a online
query. In the offline, where we are using our
[6:11] own PDFs, you know, or all the documents. Then
there we are chunking it, you know, then we are
[6:18] converting into the numbers because everything
is you know, we need to convert it into a number,
[6:24] otherwise it cannot produce anything. Then we are
creating our database, vector database, and then
[6:29] whatever the metadata, other data we have, so we
are trying to combine. So now we know we have our
[6:36] database that is a privately owned, right? So
that is on LLM doesn't know anything about that,
[6:42] but we created this database. Then if you ask
some question, then the that question will
[6:49] be also transferred to a number, a vector, the
query embedding, and then it will be, you know,
[6:56] it starts for the similarity, that question, and
then it will search to my database. It retrieves
[7:02] all the similar, you know, answers, and then it's
a and then we just add the prompt, and then we get
[7:09] the result. So this is, you know, the basic the
architecture of retrieval augmented generation.
[7:16] So that's the rag architecture. So our discover,
what it does, it lets users ask the natural
[7:22] language questions and receive the grounded
responses that point back to trusted RTC sources.
[7:30] Sources. It combines semantic search, vector
databases, large language models, all the tools,
[7:36] you know, the source attribution, conversational
memory, and safety guardrails, and then so on.
[7:41] So here one thing is important that you know, in
health care that we are building is a we need to
[7:47] talk about the safety design, right? Whether you
know, whatever we are responding it is correct or
[7:51] not. Or you know, it is not harming anyone. So, we
restrict our responses to the trusted resources,
[7:59] filter all the weak retrieval, and then
show the source attribution, avoid that
[8:04] diagnosis or treatment recommendations, support
not replace the professional judgment. So,
[8:10] those are the things that you know, we
try to implement in our tool over here.
[8:16] So, then this was a very important part for any
developing any tools that the expert curation of
[8:22] the trusted resources. So, our domain experts are
the collaborators, they curated all the trusted,
[8:28] you know, the mental health resources, contents
from all PDFs, HTMLs pages, videos, webinars,
[8:35] podcast, and everywhere reviewed for quality and
then relevance. Then each resource is assigned
[8:42] standardized metadata and topic-specific tags.
So, we gave them some tags so that you know,
[8:47] we can build our own conversational AI tool to get
that information. And that curated knowledge base
[8:55] actually enables accurate retrieval, transparent
source attribution, and trustworthy AI generated
[9:01] responses. So, that this discover, you know,
this one actually we can use it as a you know,
[9:08] reusable architecture. We have some
layers, some tools we have used over here,
[9:14] and each tool has does, you know, each step of the
rack part. So, that's the that we are not going to
[9:20] do that because there are, you know, these are all
the tools exist. It's obvious, Qdrant, vector DB,
[9:27] Forneus, Demuos, and all of those, you know.
So, the first one it does the ingestion and
[9:32] semantic processing, second one does embeddings
and metadata, sorry, vector database creates,
[9:38] then the forms for the conversational rag API,
and then use of the web chat interface, and then
[9:43] so on. So, this is the technical side. So, the
but the reusable patterns, you know, exist here
[9:50] in just embed, retrieve, augment, prompt, and
generated grounded answer, and show resources.
[9:56] So, this is our, you know, the reusable one. And
we can I'm coming into the next slides, but now
[10:05] expert evaluation. This was also was done, and
that is also important in any of this research,
[10:11] that the factual accuracy and retrieval quality,
source attribution, tagging quality, whether,
[10:17] you know, whatever it is tagged, it is,
you know, quality-wise, it is perfect.
[10:21] Then completeness, clarity, safety. So, these are
all expert evaluation that we have done, you know,
[10:28] in this project, so that, you know, um um so that,
you know, we can navigate uh correctly this one.
[10:36] So, now then uh what changes when Discover moves
to a new domain? We can use our architecture
[10:42] as a base, but then, you know, just change your
domain domain's part knowledge base, right? So,
[10:48] which trusted uh documents are we are ingesting,
and then, you know, how should resources be
[10:53] filtered and the cited? So, then the what are
the safety rules we should do, right? What must
[10:59] be the chatbot, you know, diffuse or then to
answer? Evaluation cases, you know, of what
[11:06] um scenarios uh prove the system works. They're
all the user interface. What should be in the
[11:12] user interface, and so on. So, that is also one
lesson that, you know, when we built this one,
[11:18] that we can actually move it to, you know, another
domain, but, you know, about the architecture the
[11:24] remains the same. So, there are several, you know,
the such example that I'm just giving you. We can,
[11:28] you know, apply this to this cross-domain
applications of um and of our discover
[11:35] project. And so, financial services, you know,
legal system, government, and public health,
[11:41] manufacturing, and then so on. Now, the last one
is a human involvement in a trusted application.
[11:48] So, this is, you know, very much true, and I
always face it because I'm working with the health
[11:53] care practitioners. And so, that is unless, you
know, they are involved, we cannot move forward.
[12:00] So, you know, always we have we have to
ask them. We have to get their attachment,
[12:04] you know, whatever they're and from for every, you
know, aspect of our project, we had to ask them.
[12:10] Content ingestion, what content should be,
you know, we are going to prepare. And then,
[12:15] you know, indexing and retrieval, we
need a data engineer that, you know,
[12:19] that can do that, you know, our master student
or me, we are trying to. Then, AI engineer
[12:26] we need. Then, the domain expert, right,
for the review and the verification. That
[12:30] is also important part. Whatever the results
we are getting and whether, you know, that is a
[12:36] valuable or not. So, our domain experts
can say that. Then, safety and guardrails.
[12:41] When when one of the our funder questions us
that, you know, okay, you know, we know that
[12:47] in the mental health problems like this. So, the
suicide terms, they only heard about the suicide,
[12:52] right? So, then, you know, what will happen, you
know, if the LLM generates something like that.
[12:56] So, we are trying to, you know, manage those.
We are trying to filter those things and then,
[13:03] building our stuff. And then, the user delivery
and feedback. This is the time that, you know,
[13:09] we are collecting all the information and those
things. And I think Okay, yeah. So, there is a
[13:16] demo and then, our QR code. So, that's some that's
our app. It's already built, and we are, you know,
[13:23] improving it. And so, we are getting responses. We
are trying to get responses from the users so that
[13:29] you know, we can move forward and we can improve
our the chatbot. So, the practical recommendation
[13:36] is that you know, this kind of application of
the toolbox where we want our the you know,
[13:42] we do not want only the LLM but you know, we
want the conversational kind of the interface,
[13:48] then adopt the retrieval augmented generation
instead of standalone LLMs because then you know,
[13:55] we can avoid, right? So, whatever private source
we can get the get our answers from those content,
[14:01] not from the LLM what LLM generates. And then
prioritize trust through governance. Just you
[14:07] know, we heard it, right? Before me. And then the
design AI systems around the usual workflow. So,
[14:15] just you know, ask the evolution with clinician
service providers, researchers, and consumers to
[14:21] ensure technology solves real information access
problems rather than simply showcasing the AI
[14:26] application and so on. So, that's the last one
that the trustworthy AI is about more than using
[14:33] a powerful language model. It requires governance
using curated and approved knowledge sources,
[14:39] providing transparent citation, answering
only when there is sufficient evidence, and
[14:44] operating with clearly defined domain boundaries.
These safeguards are particularly important in
[14:50] health care and mental health where inaccurate
information can directly affect patient to care.
[14:56] So, that's all for you know, I wanted
to share our work. Okay, thank you.
