MongoDB's Tengyu Ma Automation Is Replacing Manual RAG Engineering

MongoDB 马腾宇：自动化正取代人工 RAG 工程 企业 AI 必须从手工构建的检索流水线转向自动化、指令引导的语境化模型，从而简化部署并消除人工分块。 马腾宇阐述了企业落地 AI 时在长上下文模型、微调与检索增强生成（RAG）之间的权衡取舍。他详细介绍了 MongoDB 如何同时兼任数据库与检索机制，从而简化复杂的企业 AI 架构。马腾宇重点指出了诸如语境化分块嵌入和基于指令的重排序等高级 RAG 技术，这些技术在实现流水线机制自动化的同时，还能针对动态领域偏好提供细粒度的可控性。最后，该演讲展望了一个繁琐的检索优化工作完全由智能模型自动完成的未来愿景。 企业将专有数据引入 AI 工作流面临技术路径权衡：长上下文处理成本高昂，模型微调存在数据更新与合规治理难题，而 RAG 提供了低成本、模块化且低幻觉的实用方案。 (0:44) MongoDB 在 RAG 架构中兼具数据库与智能检索机制，通过端到端统一数据存储与向量检索流程，有效简化了跨多供应商组件的复杂开发体系。 (5:50) 为解决文本切块导致的全局上下文丢失，上下文分块嵌入模型（Contextualized Chunk Embedding）能自动将文档级元数据注入切块向量，省去了人工编写清洗脚本的繁琐操作。 (11:34) 基于自然语言指令的重排序技术（如 Rerank 2.5）支持用户通过提示词动态定制检索相关性标准，无需准备自定义数据集微调即可精准控制检索结果。 (17:19)

Tengyu Ma argues that deploying enterprise AI effectively requires grounding pre-trained models in proprietary data via Retrieval-Augmented Generation (RAG) rather than through costly long-context prompts or rigid fine-tuning. To scale RAG efficiently, systems must replace manual pipeline engineering with automated chunk contextualization and instruction-guided re-ranking. Evaluating Long-Context, Fine-Tuning, and RAG (0:44) Enterprise deployment requires integrating internal proprietary data that off-the-shelf foundation models lack. Ma outlines three primary data ingestion paradigms: long-context models, fine-tuning, and RAG. Feeding entire corporate libraries into long-context prompts creates steep computational expenses and risks quality degradation due to superficial scanning. Conversely, fine-tuning alters model parameters directly, which introduces strict data governance challenges, complicates rapid updates, and makes selective unlearning or redaction difficult. RAG operates as a modular alternative by retrieving only the most relevant passages to feed into the context window alongside a query. Ma asserts that targeted retrieval significantly suppresses hallucinations while controlling operational costs, providing a practical balance between dynamic data governance and reliable generation. Unifying Storage and Retrieval Infrastructure (5:50) Standard RAG workflows often suffer from architectural fragmentation, forcing engineering teams to stitch together disjointed embedding models, rerankers, and independent vector databases. Ma positions MongoDB as an integrated platform that functions simultaneously as the underlying database and the search retrieval mechanism—analogous to being both the library and the librarian. Within this framework, the large language model operates strictly as an external reasoning engine, while the database manages domain-specific enterprise memory. Unifying transactional data storage and vector search inside a single environment is presented as a way to eliminate brittle multi-vendor enterprise configurations while preserving a clean, modular architecture. Automating Document Chunking and Global Context (11:34) Conventional retrieval precision relies on labor-intensive engineering heuristics, including query decomposition, chunk enrichment, and hybrid lexical-vector search. In complex materials such as 50-page legal contracts, critical entity definitions established on the opening page lose their meaning once documents are divided into smaller chunks. Developers historically spent months writing bespoke scripts to manually replicate metadata and global context across every subsequent paragraph. To remove this manual pipeline friction, Ma advocates for contextualized chunk embedding models. By analyzing all chunks within their broader document context simultaneously, these models automatically detect missing global metadata and synthesize it directly into each chunk's single-vector embedding. Ma reports that automated contextualization reduces sensitivity to arbitrary chunking boundaries and shortens enterprise development cycles. Instruction-Guided Re-Ranking for Controllable Relevance While automated pipelines handle technical mechanics, search relevance remains inherently subjective across diverse business use cases. For instance, a query for 'Jaguar' cannot automatically determine whether a user intends to find automotive specifications or wildlife research. Rather than resolving semantic ambiguity through custom data collection or model retraining, Ma highlights natural language instruction-following directly inside retrieval and re-ranking models, such as Rerank 2.5. By appending plain-English instruction prompts to incoming queries—mirroring system prompts in conversational LLMs—users can steer ranking priorities dynamically, such as prioritizing title matches over abstracts. Ma notes that this zero-fine-tuning approach demonstrated consistent accuracy gains across public benchmarks like MTEB and MS MARCO, as well as on private enterprise datasets. Ma concludes that the road forward for enterprise RAG involves automating tedious chunking and contextualization pipelines while preserving developer control through instruction-driven interfaces. However, the presentation does not examine the operational latency or computational overhead associated with running simultaneous multi-chunk contextual embedding models at scale.

Tengyu Ma 指出，通用大语言模型缺乏企业私有数据支持，而检索增强生成（RAG）相较于长上下文模型与微调是兼具成本效益与数据治理优势的接入路径；企业 RAG 的演进方向在于摆脱繁琐的手工工程，转向系统级自动化上下文注入与基于指令的可控重排。 企业知识接入的三种范式权衡 (0:44) 企业在部署 AI 时高度依赖专有数据与行业知识，但现成的大语言模型（LLM）与智能体系统主要基于公开数据训练。Tengyu Ma 指出，将企业内部数据注入 AI 系统的科学方法主要有三种：长上下文模型（Long-Context Models）、微调（Fine-Tuning）与检索增强生成（RAG）。长上下文方案试图将数十亿词元的整套资料库拼接入单个提示词中，相当于每次提问都快速翻阅整座图书馆，计算成本极其高昂，且往往需牺牲回答深度或质量；微调则是通过目标数据集调整模型参数以形成记忆，但面临严重的知识遗忘、更新困难以及无法选择性擦除敏感或过时数据的治理难题。 RAG 机制则采用按需检索的方式，仅提取与用户查询最相关的章节或段落输入模型上下文窗口。Ma 认为，RAG 相当于规范化利用图书馆查阅针对性资料，具备模块化、高响应速度、低成本和高可靠性的特征，并通过提供明确的事实依据显著降低了模型的幻觉风险。 统一数据存储与检索的系统架构 (5:50) 典型的 RAG 与智能体系统在工程实现上高度碎片化，开发人员通常需要集成嵌入模型、重排器和向量数据库等多个独立组件。Ma 将 LLM 视为主导推理的“大脑”，而将 MongoDB 定义为兼具“图书管理员与图书馆”双重角色的基础设施，既充当底层数据仓库，又提供 AI 驱动的搜索机制。 通过将数据存储、向量检索与数据管道整合至统一的内存与存储系统中，MongoDB 旨在消除多供应商拼凑带来的架构复杂性。该架构允许开发者在保持模块化解耦的同时，将任意选定的大语言模型直接连接到底层统一记忆库，从而简化企业级端到端检索流程。 上下文分块嵌入实现端到端自动化 (11:34) 尽管业内存在混合搜索、查询分解、递归 RAG 及 GraphRAG 等多种优化技巧，但 Ma 强调检索技术的未来必须走向高度自动化。他以法律合同为例指出，合同通常在首部定义“客户”或“服务商”等实体名称，而在后续正文中均以泛称指代；当长文档被切分为小块（chunks）时，后半部分的内容便会丢失全局上下文。以往工程师往往需要耗费数月编写特定脚本，将头部元数据手工复制拼接到每个后续数据块中。 为解决这一工程痛点，MongoDB 采用了上下文分块嵌入模型（Contextualized Chunk Embedding Models）。该模型能够同时处理包含全局上下文的所有切块，自动识别缺失的全局元数据并将其直接注入各切块的单向量表征中。基准测试表明，这种自动注入机制不仅降低了系统对分块策略的敏感度，而且性能优于手动复制元数据，大幅压缩了企业 AI 团队的开发周期。 指令驱动重排与检索相关性控制 自动化虽然简化了技术管道，但无法解决用户主观意图与语义歧义问题。Ma 援引实际案例指出，当用户检索“Jaguar”时，通用模型无法自动判断其意图是汽车品牌还是野生动物，同时不同用户对标题匹配与摘要权重的优先级偏好也各不相同。由于“相关性”本质上是高度主观且因人而异的，系统必须提供明确的控制手段。 针对这一问题，MongoDB 在检索与重排阶段引入了基于自然语言指令的微调替代方案。用户无需收集定制数据或重新训练模型，只需在查询前附加自然语言指令（其运作逻辑类似于 ChatGPT 或 Claude 的系统提示词），即可动态调整排序标准。这一机制已集成至 Rerank 2.5 等重排模型中，公开基准测试（如 MTEB 和 MS MARCO）以及企业私有数据集的评估结果均验证了其在复杂场景下对检索排序精度的提升。 Ma 总结指出，针对长文档的切块处理在当下仍是客观刚需，但行业演进的终极目标是将开发者从手工切块与上下文拼接中解放出来；未来的研发重点在于通过更智能的模型实现全流程自动化，同时利用自然语言指令保留企业对检索相关性的精准掌控力。

## Background 该视频由 Ai4 频道于 2025 年 8 月 24 日发布在 YouTube 平台，由 MongoDB 首席 AI 科学家马腾宇（Tengyu Ma）主讲。内容简介指出，本次演讲主要围绕大模型落地路径之争（RAG 对比微调与长上下文）、当前 RAG 的优势与工程挑战，以及未来由 AI 模型承担更多基础工作的 RAG 演进方向展开。 ## TL;DR 马腾宇通过形象的“图书馆”比喻，论证了 RAG 相较于高成本的长上下文和易遗忘的微调方案，是企业接入私有专有数据最可靠、高效的现代解法。他阐述了 MongoDB 兼具“记忆图书馆与智能图书管理员”的生态定位，直击当前 RAG 管道中分块切分脆弱、工程技巧爆炸的行业痛点。针对这些瓶颈，MongoDB 与 Voyage AI 研发了“上下文感知分块嵌入”以实现文档级上下文的自动融合，并推出了支持自然语言指令的重排模型（如 Rerank 2.5），在免微调的前提下实现业务相关性的灵活定制。整体分享明确了新一代 RAG 系统的核心演进趋势：通过模型底层能力实现极高的自动化，同时兼顾业务侧的强可控性。 ## Atmosphere & style 演讲风格扎实硬核且不失风趣，开场以“在打卡会场讲硬核技术”的幽默自嘲迅速拉近了与观众的距离。马腾宇善于运用通俗直观的生活化类比（如去图书馆查书对比死记硬背、检索美洲豹与捷豹的偏好冲突）拆解复杂的算法机制与工程取舍。整体节奏紧凑、逻辑严密，既有对当下开发者调参困境的敏锐共情，又展现了前沿架构化繁为简的技术自信，给听众带来清晰透彻且富有启发性的视听体验。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=4s) 通用基础大模型虽然具备强大的通用推理能力，但对企业内部专有数据一无所知，如何高效、安全地接入私有数据是企业落地 AI 的核心壁垒与首要课题。 - [01:44](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=104s) 长上下文（Long-Context）方案每次查询都试图通读全量数据，如同每次都略读整座图书馆，面临极其高昂的计算成本与召回精度牺牲。 - [01:44](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=104s) 微调（Fine-Tuning）方案试图强行记忆全部数据，不仅对语料清洗质量要求极高，而且在数据过期或合规删除时面临难以解决的“知识遗忘”难题。 - [04:40](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=280s) 检索增强生成（RAG）通过按需检索最相关切片输入大模型，具备低成本、低延迟、高可解释性等优势，并能有效抑制大模型“幻觉”。 - [05:50](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=350s) MongoDB 在 RAG 生态中将持久化存储与智能检索原生整合，为“AI 大脑”提供专属记忆库，帮助企业避免组装零散组件的架构复杂度。 - [08:25](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=505s) 当前 RAG 落地陷入了分块切分、元数据补充、混合检索与重排序等“工程技巧爆炸”的困局，行业急需将这些繁琐工作交由底层模型自动化完成。 - [11:34](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=694s) 上下文感知分块嵌入（Contextualized Chunk Embeddings）能够在生成向量时统观全文并自动注入全局实体与上下文，彻底降低对 Chunk 切分大小的敏感度，并支持无缝替换现存向量库。 - [16:20](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=980s) AI 无法凭空猜测用户的主观业务意图，因此系统在将底层语义融合做到极致自动化的同时，必须为开发者保留清晰可控的干预接口。 - [17:19](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=1039s) 通过引入支持指令（Instruction）的重排模型（如 Rerank 2.5），用户仅需在查询中添加自然语言指令即可定制相关性定义，无需繁琐的模型微调。 - [19:06](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=1146s) 下一代 RAG 系统的核心演进方向是在“高度自动化（减轻工程负担）”与“高度可控性（满足定制业务诉求）”之间达成平衡。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=4s) 企业落地大模型的核心困境：私有数据的接入需求 演讲伊始，MongoDB 首席 AI 科学家马腾宇（Tengyu Ma）幽默地表示，对于这样一个通常大家都在拍照打卡、体验新鲜好玩事物的会场来说，本次分享的内容在技术深度上可能相对更硬核一些。他指出，大语言模型（LLM）以及近来兴起的智能体系统（Agentic Systems），毫无疑问是近年来科学界最重大的突破，也全面驱动了本轮 AI 浪潮。然而，在企业级生产环境中，直接开箱即用（Off-the-shelf）的通用基础大模型却面临着明显的局限。 企业真正的核心竞争壁垒在于其内部积累的海量专有数据（Proprietary Data）与垂直行业纵深知识。通用的开源或闭源大模型虽然在公网公开数据集上表现得极具智慧，但它们对企业内部的专属数据一无所知。如果企业仅直接调用这些通用模型，将无法充分释放大模型在实际业务中的潜力。因此，如何将海量的私有专有数据高效、安全地喂给 AI，成为企业迈向智能化必须解决的首要课题。 --- ### [01:44](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=104s) 数据接入路径之争：长上下文（Long-Context）与微调（Fine-Tuning） 围绕如何让大模型掌握企业私有数据，学术界与工业界一直存在着多种路径的争论。马腾宇通过人类解决问题与使用图书馆的生动类比，详细剖析了长上下文与微调两种主流方案的技术原理与现实弊端： - **长上下文（Long-Context）方案**：从技术实现上看极为简单粗暴，即把企业所有的文本数据拼合成一段包含数百万乃至数十亿 Token 的超长字符串，直接塞进大模型的 Prompt 输入框中，再附上用户的问题让模型通读后作答。马腾宇将其类比为“一个人为了回答一个具体问题，不得不每次都把整座图书馆从头到尾快速略读一遍”。这种方式计算成本极其高昂；而为了压缩成本若采取粗略扫读，又会不可避免地牺牲回答质量与召回精度。 - **微调（Fine-Tuning）方案**：该方案试图将全部数据作为训练语料来更新模型的参数权重。这好比“强行训练大脑的神经突触去死记硬背下整座图书馆的所有内容，从而彻底摆脱对外部书籍的依赖”。该方法不仅极度依赖高清洗度的优质数据，而且违背了知识存储的客观规律——人类建立物理图书馆正是因为大脑无法记忆一切。此外，微调在工程落地中面临严重的“知识遗忘（Knowledge Forgetting）”难题：当遇到数据过期、合规审查或安全权限要求彻底删除某部分机密数据时，更新神经网络内部已固化的参数几乎难以做到。 --- ### [04:40](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=280s) 检索增强生成（RAG）：高效可靠的现代解法 作为第三种方案，检索增强生成（Retrieval-Augmented Generation, RAG）代表了目前最优雅、最符合直觉的范式。RAG 的核心逻辑在于：当用户提出问题时，系统先从庞大的知识库中精准检索出最相关的几个章节或段落，随后仅将这部分精简的上下文连同原始问题一起打包送入大模型生成答案。 ```mermaid flowchart LR accTitle: RAG 架构流程示意图 accDescr: 展示用户查询如何通过检索层获取相关文档并由大语言模型生成最终回答的流程 subgraph DataLayer ["存储与检索层 (MongoDB)"] A["企业私有知识库 / 文档"] --> B["向量与全文检索组件"] end subgraph PromptLayer ["上下文装配"] Q["用户查询 (Query)"] --> B B --> D["精准相关文档片段 (Chunks)"] Q --> E["精简提示词 (Prompt)"] D --> E end subgraph GenLayer ["推理生成层"] E --> LLM["大语言模型 (Brain)"] LLM --> Out["生成准确答案 / 执行动作"] end ``` 马腾宇强调，RAG 才是“以正确方式使用图书馆”。它不仅具备高可靠性、高模块化、低延迟与低成本等天然优势，更关键的是，通过为大模型注入确切的事实上下文，大幅抑制了模型的“幻觉（Hallucination）”，使得生成结果在准确度与可解释性上均达到企业级标准。 --- ### [05:50](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=350s) MongoDB 在 RAG 架构中的定位：AI 的记忆与专属图书馆 针对“为什么一家数据库公司会深入涉足 RAG 领域”的疑问，马腾宇解释了 MongoDB 的独特生态位。如果将大语言模型比作极具推理智慧的“AI 大脑”，那么 MongoDB 所扮演的角色就是为这个大脑提供无限扩展的“记忆库与专属图书馆”。 ```mermaid flowchart TD accTitle: LLM 与 MongoDB 的协同分工架构 accDescr: 表现大语言模型充当计算大脑与 MongoDB 充当外挂记忆/图书馆的协同关系 subgraph Brain ["计算与推理引擎"] LLM["大语言模型 (LLM)<br/>'AI 大脑'"] end subgraph MemoryLibrary ["MongoDB 统一数据底座"] direction TB Librarian["智能图书管理员<br/>(AI 驱动的高精度检索 / 向量 / 混合搜索)"] Library["现代化知识图书馆<br/>(文档数据库 / 结构化与非结构化记忆)"] Librarian <--> Library end LLM <-->|"按需调用上下文 / 赋予领域适应能力"| MemoryLibrary ``` 在传统的 RAG 架构落地中，开发者往往需要拼装大量孤立的基础组件，涉及嵌入模型（Embedding Models）、重排序器（Rerankers）、专用向量数据库等繁杂拼图。MongoDB 的核心价值在于将数据持久化存储与 AI 驱动的高精度检索原生整合，“既当图书馆，又当图书管理员”。企业只需将 MongoDB 与业务选定的大模型对接，即可获得完全模块化、易维护的端到端数据检索支持。 --- ### [08:25](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=505s) 提升检索精度的传统技术栈与工程痛点 检索精度是所有 AI 开发者最关切的核心指标。为了把非结构化文本转化为机器可理解的高维数值向量，业界引入了嵌入（Embeddings）技术。它犹如对图书编制的高维数字索引，不仅能标注书名与分类，还能深度刻画章节段落的语义细节。 为了持续榨取检索质量，业界演化出了繁杂的技术组合与优化手段： - **领域专用嵌入模型**：通用嵌入模型在垂直场景表现平平。马腾宇提到，MongoDB 旗下的 Voyage AI 团队针对特定垂直行业研发了专有嵌入模型（如专注于代码检索优化的 Code Embeddings，以及针对金融和法律等场景的高精度模型），在特定领域显著优于通用方案。 - **混合检索与重排序（Hybrid Search & Rerankers）**：结合传统的关键词检索（BM25/全文索引）与稠密向量语义检索，并通过 Cross-Encoder 等 Reranker 模型对初筛候选结果进行融合打分与重新排序。 - **查询分解与重构（Query Decomposition）**：对用户的模糊原始 Prompt 进行上下文扩充、子问题拆解或意图改写。 - **分块元数据丰富（Chunk Enrichment）**：将长文档切碎为小块（Chunks）后，手动为每个 Chunk 补齐文档标题、作者、创建日期、所属层级等全局元数据。 - **嵌入模型微调与复杂图检索**：包括微调特定领域的 Embedding 参数，以及引入诸如 Recursive RAG、Graph RAG、Golden Retriever 等层出不穷的工程技巧。 马腾宇指出，目前行业正陷入一种“技巧爆炸”的困局，开发者往往耗费数月时间在这些复杂的管道中不断试错调参。他坚信 AI 检索的未来不应如此沉重，而应当向高度自动化、简洁化的方向演进——正如当今大模型免去了过去繁琐的特征工程一样，检索层也应由模型自主完成脏活累活。 --- ### [11:34](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=694s) 走向自动化：上下文感知的分块嵌入（Contextualized Chunk Embeddings） 马腾宇以法律合同长文本的处理为例，深入阐释了传统 RAG 管道在分块（Chunking）上的痛点。在标准法律合同中，开头通常会定义主体名称（如“客户是指某某公司，服务提供方是指某某公司”），而在后续数十页的正文中，通篇只会使用“客户”与“服务提供方”这两个代称。 ```markdown | 方案类型 | 处理机制 | 开发者工作量 | 召回精度与鲁棒性 | |:--- |:--- |:--- |:--- | | **传统人工元数据补全** | 手动提取首段或标题，通过脚本硬编码复制拼接到每个切片块开头 | 极高（需数月调优切分规则与拼接策略） | 脆弱；对 Chunk 切割大小极度敏感，容易丢失语义细节 | | **上下文感知分块嵌入 (Contextualized)** | 模型前向传播时统观整篇文档，自动将全局上下文特征编码至局部向量 | 极低（即插即用，模型全自动处理） | 卓越；自动消除对切分策略的敏感性，显著超越手动增强效果 | ``` 在传统流程中，由于单次只能处理一页或几个段落的 Chunk，切分后的文本块往往丢失了全局实体定义，导致检索直接失效。以往工程师必须手动编写复杂的解析规则，将全局首段信息机械地“复制粘贴”到后续所有切片前。针对这一痛点，MongoDB / Voyage AI 研发了**上下文感知分块嵌入模型（Contextualized Chunk Embedding Model）**： - 该模型在生成向量时能够直接感知整篇文档的全局结构，模型内部自动判定哪些全局元数据和实体定义需要注入到具体的 Chunk 向量表征中。 - 输出形式依旧保持为“每个 Chunk 对应一个独立向量”，无需改变现有的向量库存储规范，实现了即插即用的无缝替换（Seamless Drop-in Replacement）。 - 评测结果表明，该模型大幅降低了 RAG 系统对分块切分大小（Chunk Size）的敏感度，彻底解决了困扰 AI 工程师多时的切分调优瓶颈。 --- ### [16:20](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=980s) 自动化与可控性（Controllability）的结合 马腾宇强调，在推动算法全自动化的同时，系统必须保留足够的“可控性（Controllability）”。因为 AI 永远无法凭空猜测带有强烈主观偏好或特定业务限制的意图。 他分享了一个真实的客户案例：某客户在使用检索系统搜索关键词“Jaguar”时，检索结果同时返回了美洲豹（动物）与捷豹（汽车）的相关内容。该客户抱怨道：“我只喜欢汽车，讨厌动物，为什么系统不能只给我汽车？”马腾宇对此回应：“如果模型事先不知道你的偏好，它不可能擅自剥夺其他人检索美洲豹的权利。”这个例子生动地揭示了自动化的边界——对于算法可以通用的语义理解与上下文融合，系统应做到极致的自动化；而对于业务规则与用户私有偏好，系统则必须提供简洁可控的干预接口，使开发者能够轻松引导模型的检索行为。 ### [17:19](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=1039s) 基于指令的自然语言检索偏好与定制化相关性 在实际业务场景中，不同用户和系统对“相关性”（Relevancy）的定义存在显著差异。例如，有些用户可能只关心汽车领域的特定信息，有些场景更看重标题层面的相关度而非摘要内容，还有些业务则侧重完全不同的维度。为了解决这一痛点，MongoDB 提出了一种创新的偏好表达方式：允许用户在查询（Query）前或后添加一段自然语言指令（Instruction）。这种机制与使用 ChatGPT 或 Claude 等大语言模型时的 System Prompt 非常相似，使得用户能够直接用日常语言定义检索模型的偏好，而无需进行繁琐的模型微调（Fine-tuning）或准备专门的训练数据集。 ```mermaid flowchart LR accTitle: 基于指令定制相关性的检索重排流程 subgraph InputStage["输入端"] A["用户查询 Query"] B["偏好指令 Instruction"] end subgraph ModelStage["模型处理"] C["指令感知检索 / 重排模型<br/>(如 Rerank 2.5)"] end subgraph OutputStage["输出端"] D["按定制偏好排序的结果"] end A --> C B --> C C --> D ``` 这种“即插即用”的自然语言控制方式在真实业务场景中带来了显著的性能提升。在公共基准测试（如 MIR / MTEB 等检索评测）以及 MongoDB 经过客户授权评估的真实业务内部数据集上，该方案均实现了全面的指标增长。更重要的是，它极大地简化了系统架构和开发运维流程，工程师无需再为如何微调检索模型、适配复杂的业务规则而苦恼。 - **免微调的偏好控制**：直接在查询前后拼接入自然语言指令，以最直观的方式让模型理解业务对相关性的特殊要求。 - **广泛的性能验证**：不仅在公开学术/行业 Benchmark 上表现优异，在 MongoDB 客户的真实生产数据集中也展现了显著的效果提升。 --- ### [19:06](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=1146s) 核心总结与未来展望：迈向自动化与可控的 RAG 系统 回顾整个分享，演讲者总结了构建新一代 RAG 系统的核心要点。首先，文档切块（Chunking）是不可或缺的环节；虽然“为切块补充上下文”是一个已被证明行之有效的技巧，但 MongoDB 的目标是将这一过程完全自动化，让开发者无需手动设计和维护繁琐的上下文注入流程。其次，MongoDB 推出了支持指令驱动的新模型（如 Rerank 2.5 等重排模型），允许在重排（Reranking）阶段动态注入业务指令，从而实现对“相关性”定义的灵活定制。 这一系列技术演进的核心方向，就是推动 RAG 系统朝着“更自动化”（Automated）与“更可控”（Controllable）的方向发展。随着底层研发的持续推进，MongoDB 未来将推出更多模型、算法与技术组件，让模型承担更多复杂的基础工作，为开发者提供开箱即用且高度可定制的下一代企业级 RAG 解决方案。 - **切块上下文自动化**：将手动补充 Chunk 上下文的繁琐技巧固化为系统的自动化能力。 - **指令驱动的重排模型（Rerank 2.5）**：在重排阶段支持 Instruction，赋予定制化相关性定义的能力。 - **核心演进主线**：以更强的自动化减轻工程负担，以更精准的控制力满足千行百业的定制化业务诉求。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=4s) 大语言模型（LLM）以及近来兴起的智能体系统（Agentic Systems），毫无疑问是近年来科学界最重大的突破，也全面驱动了本轮 AI 浪潮。 — Tengyu Ma > [00:04](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=4s) 企业真正的核心竞争壁垒在于其内部积累的海量专有数据（Proprietary Data）与垂直行业纵深知识。 — Tengyu Ma > [01:44](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=104s) 一个人为了回答一个具体问题，不得不每次都把整座图书馆从头到尾快速略读一遍。 — Tengyu Ma > [01:44](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=104s) 强行训练大脑的神经突触去死记硬背下整座图书馆的所有内容，从而彻底摆脱对外部书籍的依赖。 — Tengyu Ma > [01:44](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=104s) 人类建立物理图书馆正是因为大脑无法记忆一切。 — Tengyu Ma > [05:50](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=350s) 如果将大语言模型比作极具推理智慧的“AI 大脑”，那么 MongoDB 所扮演的角色就是为这个大脑提供无限扩展的“记忆库与专属图书馆”。 — Tengyu Ma > [05:50](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=350s) 既当图书馆，又当图书管理员。 — Tengyu Ma > [08:25](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=505s) 正如当今大模型免去了过去繁琐的特征工程一样，检索层也应由模型自主完成脏活累活。 — Tengyu Ma > [16:20](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=980s) 如果模型事先不知道你的偏好，它不可能擅自剥夺其他人检索美洲豹的权利。 — Tengyu Ma > [19:06](https://www.youtube.com/watch?v=GkpAcxwiAkI&t=1146s) 这一系列技术演进的核心方向，就是推动 RAG 系统朝着“更自动化”（Automated）与“更可控”（Controllable）的方向发展。 — Tengyu Ma ## Mentioned Tengyu Ma (马腾宇), MongoDB, Voyage AI, ChatGPT, Claude, Rerank 2.5, BM25, Cross-Encoder, Graph RAG, Recursive RAG, Golden Retriever, MIR, MTEB ## Video description Tengyu Ma, Chief AI Scientist, MongoDB Abstracts The talk will have three parts: 1) Roadmap debate: RAG vs. finetuning vs. long-context 2) RAG today: benefits, challenges, and current solutions 3) RAG tomorrow: AI models do more work [Source](https://www.youtube.com/watch?v=GkpAcxwiAkI)

[0:04] Tengyu Ma: 大语言模型（LLM）以及近来兴起的智能体系统（Agentic Systems），毫无疑问是近年来科学界最重大的突破，也全面驱动了本轮 AI 浪潮。 [0:04] Tengyu Ma: 企业真正的核心竞争壁垒在于其内部积累的海量专有数据（Proprietary Data）与垂直行业纵深知识。 [1:44] Tengyu Ma: 一个人为了回答一个具体问题，不得不每次都把整座图书馆从头到尾快速略读一遍。 [1:44] Tengyu Ma: 强行训练大脑的神经突触去死记硬背下整座图书馆的所有内容，从而彻底摆脱对外部书籍的依赖。 [1:44] Tengyu Ma: 人类建立物理图书馆正是因为大脑无法记忆一切。 [5:50] Tengyu Ma: 如果将大语言模型比作极具推理智慧的“AI 大脑”，那么 MongoDB 所扮演的角色就是为这个大脑提供无限扩展的“记忆库与专属图书馆”。 [5:50] Tengyu Ma: 既当图书馆，又当图书管理员。 [8:25] Tengyu Ma: 正如当今大模型免去了过去繁琐的特征工程一样，检索层也应由模型自主完成脏活累活。 [16:20] Tengyu Ma: 如果模型事先不知道你的偏好，它不可能擅自剥夺其他人检索美洲豹的权利。 [19:06] Tengyu Ma: 这一系列技术演进的核心方向，就是推动 RAG 系统朝着“更自动化”（Automated）与“更可控”（Controllable）的方向发展。

[5:50] Tengyu Ma: We are both the librarian and the library. [17:19] Tengyu Ma: relevancy is inherently subjective, and no single universal definition fits every user or use case.

[0:04] Thanks so much. Um um it's great to be [0:06] here. Um I guess this um topic is [0:11] probably a little bit more relatively [0:13] speaking technical for this arena which [0:15] I probably would have uh usually come [0:18] for taking pictures, watching other [0:20] people doing fun stuff. Um I guess in [0:23] the next 15 minutes I will explain to [0:24] you uh this title what it means you know [0:28] why it matters for you and what we are [0:30] doing to achieve uh some of the goals [0:32] here. [0:34] So I guess you know all know that large [0:37] language model and these days agentic [0:39] systems are probably the most popular [0:42] and the most important breakthroughs [0:44] from the scientific community. um and [0:47] and that drives the AI wave uh in the [0:49] last few years. What sometimes people [0:52] don't talk that much often about is that [0:55] you cannot just use this offtheshelf [0:58] large language models or agent directly [1:01] uh in enterprise setting. That's because [1:04] you know in all of your companies you [1:06] have so much data proprietary data or [1:09] specific data with depth that you really [1:12] have a edge you know over other people [1:16] and if you just use a large language [1:18] model agents off the shelf which doesn't [1:21] necessarily know this data they are [1:23] tuned on public data right so they are [1:25] very brilli brilliant very smart but [1:28] they don't know your data if you just [1:30] use them off the shelf you wouldn't [1:31] really necessarily maximize the the [1:34] benefit. So that's why using AI for [1:36] enterprises require ingesting massive [1:40] proprietary or specific data. [1:44] How do we do this? I guess there are few [1:47] approaches and there are a lot of [1:49] scientific debates on which approach is [1:52] um better. Of course, every approach has [1:55] its uh pros and cons and has their [1:58] particular use cases. Uh I guess I'll [2:00] spend a few minutes uh to walk through [2:02] there's a lot of text here uh some of [2:04] these kind of like uh uh some of these [2:06] approaches and explain them with analogy [2:10] with how humans are generally kind of [2:12] like solving problems with access to [2:15] data. Let's start with long context [2:17] which is probably the simplest one to [2:19] explain here. Technically speaking it's [2:22] very simple. you just concatenate all [2:25] your data into a long string of billions [2:28] of tokens possibly and put this long [2:30] string of tokens of words um uh long [2:34] string of words into uh the prompt of [2:37] the large language model. It's kind of [2:39] like you put all of them in a text box [2:41] of the chat GBT and then you have a [2:44] question and then the large language [2:47] model will go through your prompt which [2:50] is sometimes millions of words and then [2:53] uh look at your question and answer the [2:56] um question accordingly. In some sense, [2:59] if you make analogy in h how humans are [3:01] solving the questions, it's kind of like [3:04] one person given a question scheme the [3:07] entire library to solve this question. [3:10] And as you can see the kind of the [3:12] challenge is that it's enormously [3:14] costly. Um and sometimes if you have to [3:17] reduce the cost, you have to somehow [3:19] sacrifice a little bit to scheme the [3:20] library as opposed to going through the [3:22] entire library very carefully. And then [3:24] you may also suffer a little bit about [3:26] the quality. [3:28] And the second approach is fine-tuning [3:30] which is quite different. So here you [3:33] take all of this data and you know if [3:36] you make analogy with humans it's kind [3:38] of like you read through this data very [3:39] carefully. You train yourself to muscle [3:42] memorize the data. It's kind of like you [3:44] update your synapses and neurons in your [3:46] brain so that you can just memorize them [3:48] and then you don't need a library [3:49] anymore. Um in a technical world you [3:52] know language is called fine-tuning the [3:54] models the model parameters with the [3:56] data and as you can see often this [3:59] requires a lot of high quality data u [4:02] because you know like you do really need [4:03] high quality data to really understand [4:05] deeply what happens uh and uh uh but [4:08] it's very challenging because you know [4:10] really no if this is so easy then why we [4:13] need a library everyone would just [4:15] remember everything in the library in [4:16] some sense the library exists because [4:18] you cannot really muscle memorize with [4:20] everything in your brain. Um, and also [4:22] there are kind of like practical issues. [4:24] For example, forgetting the knowledge [4:26] that you have already burned into your [4:28] head is not that that easy and sometimes [4:31] you just do have to forget things [4:33] because of the security reasons because [4:35] the data out of date because of the data [4:37] governance reasons. The third one which [4:40] is as you can see uh which is what we [4:43] advocate for is called retrieval augment [4:45] generation. [4:47] It's kind of like you retrieve the most [4:49] relevant book chapters or paragraphs [4:52] from a gigantic library and then using [4:56] this small set of information uh and a [4:59] question you answer the question [5:01] accordingly and u technically speaking [5:04] it means that you use a approach to [5:07] search or retrieval retrieve relevant [5:09] information from a large corpus and then [5:12] you give this retrieve the data to a [5:14] large language model uh as prompts. And [5:17] then the prompt of the large language [5:19] model becomes shorter than the entire [5:20] you know corers. And then you give a [5:23] question to the large language model and [5:24] then the large model will answer the [5:26] question for you. And the benefit here [5:29] is that it's reliable and modular and [5:32] fast and also cheap. It's similar to how [5:34] humans are solving the problem. It's uh [5:36] kind of like really using the library in [5:38] the right way in some sense. Uh and also [5:40] it reduce the hallucination because you [5:42] give context uh to the large model. um [5:45] and it you know makes the quality higher [5:47] and reduce the hallucination. [5:50] So you may wonder why MongoDB as a [5:54] database company you know uh have [5:56] anything to do with this. Uh the reason [5:58] why we are here is that MongoDB is [6:00] uniquely positioned to uh uh contribute [6:03] to this uh uh retrieve augment a [6:06] augmented generation stack because you [6:09] know we are doing the database and also [6:11] we are enabling the accurate AI powered [6:14] retrieval from the database. It's kind [6:16] of like we are both the librarian and [6:18] the library. So um the kind of like the [6:22] figure just to reiterate you know uh [6:23] this visually. So you have a database [6:25] which is in MongoDB and you have a [6:28] search or retrieval component which uh [6:31] is integrated into MongoDB and then [6:33] given a query you can use this component [6:35] to retrieve relevant documents and you [6:38] give the relevant documents to large [6:39] language model uh and will generate [6:41] answers or sometimes take actions for [6:42] you [6:45] and in some sense um the kind of the way [6:47] I'm thinking about it is that large [6:49] language model is kind of the brain of [6:51] AI right it's very intelligent Um but [6:54] MongoDB is offering the memory and [6:57] library of AI so that the brain can have [7:00] you know extended memory and library to [7:03] be more intelligent and do to be more [7:06] kind of adaptive to your domains and [7:08] your use cases and also we do you know [7:11] this is you know simple but actually [7:13] under the hood we are doing a lot of [7:14] work to enable this because if you look [7:17] at um uh what other people typically do [7:20] you know with this retrieval argument [7:21] generation stack or with agentics [7:23] systems is actually a pretty complex [7:26] workflow. You have many different [7:27] components. You know, I can throw away a [7:29] I can throw out a few kind of technical [7:31] buzzword here like embedding models, [7:33] reanchers, vector database, so on and so [7:36] forth. And what MongoDB does is that we [7:39] um take care of all the uh uh issues [7:42] with memory and library so that you know [7:44] basically everything with the data and [7:47] the retrieval of the data will be [7:48] handled by MongoDB. And then the only [7:50] thing you need to do is to connect this [7:52] uh retrieval system with the database [7:55] together with a large brush model which [7:56] you can choose. Um and it's also [7:58] modularized because you can choose any [8:00] large model uh you wish. [8:04] Um that's the high level kind of like a [8:06] picture of how MongoDB uh is [8:08] contributing to this AI revolution. Um [8:12] and in the next nine minutes I will talk [8:14] a little bit more about uh some of the [8:16] slightly more technical part uh and also [8:19] how do we kind of like make this even [8:21] more uh automatic and controllable. Um [8:25] um so how do we retrieve the accuracy? [8:27] Because accuracy here is still pretty [8:28] much the top of demands for most of the [8:30] AI developers. You want a retrieval to [8:32] be as accurate as relevant as possible. [8:35] Um let me quickly go through some of [8:37] these bullet points which are probably a [8:39] bit technical. One of the best way is [8:41] you know I guess maybe one thing I [8:43] should define first is that there's a [8:45] concept called embeddings which are uh [8:48] representations of the data in numerical [8:51] numbers. It's kind of like advanced [8:53] version of how you index the books in [8:55] the library. When you index books you [8:58] have a few numbers but here you have a [9:00] lot of numbers and they not only capture [9:02] the titles and categories of the books [9:04] but also captures the detailed [9:06] information in the books or in the [9:08] chapters. Um and this is the key uh uh [9:12] uh AI innovation to make the retrieval [9:15] high quality semantic uh uh accurate. Uh [9:19] and you can use kind of like domain [9:21] specific embeddings. You know, MongoDB [9:24] or Voyage AI team um made a lot of [9:27] different domain specific embeddings. [9:29] For example, we have the code retrieval [9:31] uh uh uh embeddings u which is optimized [9:33] for code and it's much better than other [9:36] people on code. We also have finance [9:37] embeddings. We have legal embeddings and [9:39] we can you know def uh achieve you know [9:42] specific you know improvements on this [9:44] particular domains [9:46] and um we can you another approach is [9:49] called hybrid search and revankers which [9:51] is like you use the existing keyword [9:54] based search together with the more [9:55] vectorbased search and then use a [9:57] revankers to combine the results. Um, [10:01] another one which I will talk a little [10:02] more about is the the so-called query [10:04] decomposition where you refreeze and add [10:07] context to the queries. Um, and another [10:10] one which is a little more technical is [10:12] so-called trunk enrichment. So basically [10:14] what people do is that you first chunk [10:16] the data into pieces so that you don't [10:18] have to handle a 50 pages uh long [10:21] document all at once. You can handle [10:23] smaller trunks and then you also augment [10:26] the trunks with global contexts like [10:28] document titles, hiders, categories, [10:30] authors and dates. And sometimes you can [10:33] use AI to generate additional M [10:35] information for each of the trunks. Some [10:38] of this may sounds a little kind of [10:39] technical. You know, you you see why I'm [10:41] talking about some of these because, you [10:43] know, we'll try to automate some of this [10:46] a way so that you don't have to think [10:47] too much about it. Um maybe let me skip [10:50] this. So um and another one is you can [10:54] fine-tune embedding models. This is a [10:55] way to customize your embedding models [10:57] to make it more specific to your [10:59] domains. [11:01] Um so these are some of the common [11:03] approaches right now. Um and there are [11:06] also you know many other techniques [11:08] there these days. You know AI is very [11:10] popular. You know there are a lot of [11:12] like uh uh all kind of attempts. Uh you [11:15] can have like recursive rack uh graph [11:17] rack golden retriever so and so forth. [11:20] Um I guess you know probably have some [11:22] of you have never heard of this but this [11:24] is like all different kind of like [11:26] tricks you can apply on top of the [11:27] embedding models u to make uh um u the [11:31] models um more accurate. [11:34] Um but you as you can see you know this [11:36] become this list can even go on and on [11:38] you know if I spend like another 30 [11:39] minutes I can give you another 30 tricks [11:41] to make the retrieval quality higher and [11:43] higher. uh in my opinion I think the [11:46] future of the AI and the future of [11:48] retrieval specifically in this context [11:50] should be more automated and more kind [11:52] of like a uh simpler right in some sense [11:55] this whole wave of AI driven by large [11:57] language model is um kind of like the [12:01] the unique thing about it is that it's [12:03] so easy to use AI these days like five [12:06] 10 years ago when I was at um uh [12:07] Stanford when I teach the students you [12:10] know how to apply AI in enterprise we [12:12] have a long list of you steps uh to [12:16] collect data, trend the model, to do a [12:18] tin test split, define a problem, you [12:20] know, reiterate, you know, evaluation, [12:22] so on and so forth. And these days with [12:24] the offtheshelf large model, you get [12:26] probably 90% of the work done, maybe 70% [12:30] of the work done, except that you have [12:31] to also connect your own data, which is [12:33] what we're talking about. But the same [12:35] thing happens here as well, right? for [12:37] retrieval. It shouldn't be that it [12:39] requires you to go through this complex [12:42] lengthy kind of like a development steps [12:44] to use retrieval uh models. It should be [12:47] more automatic, simpler and AI should do [12:50] most of the work and you just connect it [12:51] to your data. [12:53] So um um so that's what I mean by [12:56] automation. So maybe let's take a look [12:58] at this kind of like particular trunk uh [13:01] uh uh context uh issue. So let me give [13:04] example here. Let's dive into this. [13:07] Suppose you have a long legal document. [13:08] For example, we all know that most cases [13:11] in the beginning of the legal document, [13:13] you define the entities. You say the [13:15] client is this entity and the service [13:17] provider is this entity, right? And then [13:19] you never talk about these two names of [13:22] the entity anymore in a long legal [13:23] document, right? You just refer them to [13:25] the client and the service provider. [13:28] uh and because the legal document has is [13:31] typically very long so you have to trunk [13:34] them into pieces to process them [13:35] separately and when when you trunk them [13:38] then often the case is that one trunk [13:41] only contains maybe one paragraph or [13:43] maybe one page and that page doesn't [13:46] contain the name of the client and the [13:49] name of the service provider. So this is [13:51] a problem we refer to as you know [13:53] missing context. you are missing a [13:55] global context in this trunk of um of [13:58] data [14:00] and what people typically do is that you [14:02] put the global context in the trunk. So [14:05] basically you copy paste the first [14:06] paragraph to each of the uh future pages [14:10] um and you do this manually and you [14:12] identify which paragraph needs to be [14:14] copy pasted um and and needs to be kind [14:16] of like uh repeated. So this is a very [14:20] manual process. You can have many [14:21] different options. The search space is [14:23] very big. Many AI developers spend [14:25] months to try to find the best way to [14:28] augment your trunks with the right type [14:30] of metadata. And what we do here is that [14:33] we just automate this process so that [14:35] you don't have to think about it. We'll [14:37] let the new works do it for you. So [14:40] specifically, you know, we pretty much [14:42] uh oh, this is what people do uh uh [14:45] right now. And what we do is that we [14:47] just say you give all of these trunks to [14:50] what we call contextualized trunk [14:52] embedding model. You don't have to [14:54] really understand you know exactly how [14:55] the model under the hood the model is [14:57] working. The only thing important is [14:58] that the model sees all the trunks and [15:00] then automatically it figures out what [15:04] uh uh are the metadata each of the [15:06] trunks. What other metadata that has to [15:08] be repeated uh um in the representations [15:10] of each of the trunk and you output the [15:13] vectors the same way as before. or you [15:14] all to one vector for each trunk. This [15:17] is called embeddings. These are the [15:18] representations of those trunks and each [15:20] of these vector automatically contains [15:23] the detailed information of that trunk [15:25] and also the global high level context [15:28] from other trunks. So pretty much we [15:31] just automate this augmentation trunk [15:33] augmentation uh metadata augmentation [15:36] steps by the embedding model. [15:40] Um and you know the it's aware of the [15:44] full document. It's a seamless drop in [15:46] replacement and is less sensitive to the [15:48] trunking strategy and it's also more [15:50] accurate than manually doing the [15:53] metadata augmentation even if the [15:55] metadata are available. You know if [15:57] they're not available this is much [15:59] better. So we have some evaluation [16:02] results. Actually we have a lot more [16:03] figures like this in our blog post. Uh I [16:05] probably wouldn't go through the details [16:07] here. Um and uh we have you know figures [16:11] showing that it's reduced the [16:12] sensitivity to trunking which is [16:14] actually one of the key um bottleneck [16:16] for AI developers uh in the in the [16:18] development cycle. [16:20] And let me spend two minutes you know on [16:22] another dimension which is called [16:24] controllable. I think this is kind of [16:26] like go hand inhand with automation [16:28] because in some sense you know we cannot [16:30] automate everything for you because [16:32] there are things that only you know but [16:34] we don't know at all right for example [16:36] uh one of the um uh customers ask us you [16:40] know they say oh when I search the word [16:42] jagger you know it gave me both animals [16:45] and the cars but I only want cars I'm [16:49] just really like cars I don't like [16:50] animals and and I said okay how come we [16:54] know you have these preferences, right? [16:55] So like a like like our models is there, [16:58] you know, everyone can use it. We cannot [16:59] just say we only care about cars so that [17:02] nobody, you know, care about animals. So [17:04] somehow you have to tell us your [17:06] preferences and that's the [17:08] controllability part, right? This is the [17:10] the one of the few places where we [17:13] cannot automate because this is your [17:14] preferences and and that's why we are [17:16] trying to make it easier for you to [17:19] express your preferences so that you [17:21] know it's still a very smooth uh smooth [17:24] uh process. So, so what we do is that [17:27] you know we um you know maybe just to [17:29] repeat the issue you know basically the [17:30] issue is that not everyone has the same [17:33] definition of relevancy right someone [17:36] just care about cars someone care about [17:38] the relevancy in the title but not the [17:40] relevancy in the abstract someone cares [17:42] about some other kind of dimensions so [17:44] we need a way to express these [17:47] preferences [17:48] and the way we do it is that we say you [17:51] can append or prepend an instruction [17:55] ction before the queries. Pretty much [17:57] the same way as you do with large [17:59] language model. When you do large [18:00] language model, you can have your [18:02] preferences your system prompt right in [18:04] chatbt or cloud and that give you a [18:06] preference a way to express the [18:08] preferences and the same way you can do [18:10] that for the retrieval model so that in [18:12] the future when you search you also have [18:14] your own preferences expressed in [18:16] natural language. Um you don't have to [18:18] fine-tune you don't have to kind of like [18:19] provide any data to us. you just have to [18:21] express your preferences in the most [18:24] natural way u possible. And we found [18:26] that this you know gives a significant [18:28] boost in many cases especially real [18:30] world cases where people do have their [18:32] own preferences and we have a lot more [18:34] data um um in our blog post as well. [18:39] So so you can see that you know we [18:41] pretty much just improved the [18:42] performance across the uh uh across the [18:45] board in many different cases. you know [18:48] M uh IR is a public public benchmark. We [18:52] also have many in-house data sets where [18:54] we have real world data from our [18:55] customers. We evaluate on them with [18:57] their permission uh and uh and we see uh [19:00] great performance improvement and also [19:02] simplification because they don't have [19:03] to worry about this anymore. [19:06] Okay, just a quick takeaway. So trunking [19:10] is necessary. So that's why we are [19:11] adding context to a trunks uh is a [19:13] useful trick but we want to automate [19:15] this trick so that you don't have to [19:17] worry about it. Um and we also have this [19:20] new model called reanker to rewrank 2.5 [19:23] which allows us to add instructions in [19:25] the reanking process allowing [19:27] customization of the definition of the [19:29] relevance. Um and these are the [19:31] approaches we kind of like takes towards [19:33] making it more automated and [19:35] controllable. we will have more uh uh [19:37] models and approaches technologies uh [19:40] going forward uh as we are developing [19:42] our research process. Um thanks so much. [19:46] [Applause]