Vespa.ai Tensors Target Multimodal Search as Standard RAG Architectures Fail

标准RAG架构遇阻，Vespa.ai张量瞄准多模态搜索 企业级RAG系统在生产环境中性能下降，是因为标准向量架构将检索与重排序割裂为互不相连的工具。 演讲者探讨了传统RAG架构在从原型走向生产环境时面临困境的原因，特别是系统碎片化、数据滞后以及仅支持文本的局限性。他们主张将混合检索、多阶段重排序和实时索引统一在单一基于张量的引擎中，而不是拼凑互不相连的单一用途系统。通过生命科学和企业应用案例，他们展示了将非结构化向量搜索、结构化标量过滤、词法匹配和张量运算相结合，如何能够提高准确性并降低延迟。最终，构建生产级AI需要超越基础的文档分块，转向灵活的多信号检索策略。 标准RAG系统通常是因为上下文质量管理不善而失败，而非由于模型的推理能力限制。 (2:32) 纯文本RAG架构难以处理PDF、PPT和图表中的特定领域视觉信息。 (3:00) 传统搜索架构在将独立的检索、重排序和执行系统拼凑在一起时，会触及扩展上限。 (5:03) 现代搜索依赖多阶段重排序对候选结果进行筛选，在将上下文传递给LLM之前精简出高度集中的头部子集。 (6:57) Vespa基于多维张量存储而非简单的一维向量存储运行，从而支持多模态张量逻辑。 (11:51)

Bonnie Chase and Harini Gopalakrishnan of Vespa.ai argue that overcoming the scaling ceiling and hallucination problems of enterprise RAG requires moving away from fragmented, vector-only pipelines toward unified tensor architectures. In their view, model scaling cannot resolve retrieval errors when pipelines fail to supply high-quality, fresh, and visually grounded context. Root Causes of Enterprise RAG Failures (2:32) Harini Gopalakrishnan asserts that enterprise chatbot and RAG failures cannot be resolved simply by fine-tuning models or expanding context windows to billions of tokens, because large language models reason strictly over the specific context provided to them. Instead, hallucinations stem from context deficiencies across three areas: text-only pipelines that fail to parse visual information in PDFs, slides, and charts; stale indexes unable to reflect real-time updates like dynamic e-commerce inventories; and basic vector similarity searches that overlook implicit user context. To prevent model hallucinations, Gopalakrishnan contends that retrieval architectures must embed contextual user signals and aggressively narrow results. Rather than dumping dozens of retrieved passages into an LLM context window, production pipelines should prune candidates down to the top three to six high-relevance fragments before answer generation. Tensor Stores Versus Fragmented Vector Pipelines (5:03) Bonnie Chase identifies system fragmentation as the primary operational bottleneck when scaling RAG applications. Traditional architectures stitch together separate tools for vector retrieval, structured filtering, and machine-learned ranking. Chase argues that the glue and joins connecting these disparate systems introduce latency, data drift, and compounding infrastructure costs, causing systems under heavy load to degrade silently rather than throw explicit errors. To eliminate multi-system overhead, Gopalakrishnan differentiates conventional vector stores from tensor stores. While a vector is a one-dimensional array representing a single text chunk, a tensor stores multi-dimensional arrays, allowing mathematical operations across multiple embedding models and modalities simultaneously. Platforms like Vespa integrate vector search, structured scalar filtering, lexical search, and ranking into a single runtime, powering enterprise workloads at AlphaSense across more than 500 million documents, as well as Perplexity, RavenPack, and Spotify's transcript search. Principles of Multi-Stage Hybrid Retrieval (6:57) Gopalakrishnan recommends that enterprise RAG adopt proven methodologies from large-scale search platforms rather than relying solely on cosine distance. Production retrieval should employ a multi-phase funnel: an initial lightweight hybrid stage combines lexical keyword matching and semantic vector similarity to cast a wide net across thousands or millions of records, followed by machine-learned ranking functions that apply contextual signals to narrow the pool down to four to ten candidates. For quality assurance, Gopalakrishnan emphasizes continuous evaluation against benchmark corpora using metrics like Normalized Discounted Cumulative Gain (NDCG). She notes that achieving a benchmark score of 60% to 70% represents a solid retrieval baseline, whereas attempting to force accuracy to 90% often results in model overfitting. In parallel, systems must replace batch reindexing with real-time streaming pipelines to keep dynamic data continuously retrievable. Multimodal and Multi-Constraint Query Execution (14:41) In technical domains like life sciences, critical data is frequently locked in visual formats. Gopalakrishnan illustrates this with a clinical slide where a 20% reduction in Major Adverse Cardiovascular Events (MACE) from semaglutide was contained entirely within an embedded graphical chart. A text-only query would miss the finding if the body text used alternative phrasing like 'cardiovascular events,' but indexing the slide simultaneously as an image and text allowed a unified visual and lexical search to extract the precise data point in a single pass. A further demonstration involved identifying an antibody by executing four distinct constraints simultaneously within one ranking function: an unstructured vector similarity match on protein structure, scalar range filters for experimental temperature and pressure variants, a lexical match for Non-Small Cell Lung Cancer (NSCLC), and a recency filter restricting results to data from the preceding three months. Combining these operations within a unified tensor function surfaced exact matches without cascading queries across separate databases. The speakers conclude that retrieval and ranking remain the most difficult architectural hurdles in AI, warning that automated hyperscaler plumbing cannot replace deliberate chunking, embedding, and ranking design. Production viability requires abandoning batch updates for real-time streaming, adopting flexible chunking strategies that combine granular text with page-level optical character recognition, and evaluating relevance directly within retrieval to avoid the failure points of stitched-together systems.

Vespa.ai 的 Bonnie Chase 与 Harini Gopalakrishnan 认为，企业级检索增强生成（RAG）面临的核心瓶颈并非模型参数规模或上下文窗口大小，而是上下文质量、多模态处理能力及碎片化系统架构。她们主张采用基于张量（Tensor）存储的统一计算运行时，通过融合多阶段排序、混合检索和实时流式更新，解决传统多系统拼接导致的延迟与隐性质量退化。 传统RAG的瓶颈与上下文质量缺陷 (2:32) Gopalakrishnan 指出，早期企业级聊天机器人和 RAG 系统出现幻觉或劣质回答时，业界常见的补救方案是微调模型或扩展上下文窗口（即便支持数十亿词元），但这并未从根本上解决幻觉，因为语言模型严格受限于输入给它的具体上下文。提升回答质量的关键在于精确控制和优化送入模型的上下文内容。 当前企业级检索普遍存在三项结构性缺陷：第一是视觉与混合信号的缺失，在医疗与生命科学等专业领域，关键情报往往嵌入在 PPT、PDF、图表或曲线中，常规的纯文本分块无法有效提取；第二是数据鲜度不足，如电商中鞋码与颜色的库存状态实时变动，静态索引会导致模型推理脱离实际；第三是上下文感知不足，仅基于用户 Prompt 文本进行向量相似度搜索，会遗漏用户部门、历史交互偏好等至关重要的隐式背景信号。 架构割裂的扩展上限与多阶段排序 (5:03) Chase 解释称，AI 团队的核心障碍已不再是从原型走向生产，而是在生产环境中维持规模化运行。传统架构将检索、重排和模型执行拆分在互不连通的独立系统之间，这种拼接和连接层会导致高延迟、数据漂移以及不断攀升的运维成本；更严重的是，这些孤立系统在超负荷时往往不会报错，而是发生静默的质量退化。 为了构建稳健的检索系统，Gopalakrishnan 建议借鉴现代搜索技术（例如 Perplexity 在网页级实现亚秒级检索与逐行归因的机制）：采用结合词法关键词、语义相似度与机器学习排序函数的混合检索模式。系统应先通过轻量化搜索建立宽泛的初筛候选集（覆盖数千至数百万结果），再利用机器学习模型与上下文信号将其修剪至排名前 4 到 10 个（或 3 到 6 个）高质量候选传递给大语言模型，从而避免模型因候选过多而产生幻觉。在基准评估上，她指出在 NDCG 等相关性指标中达到 60% 至 70% 的准确率即代表扎实的基线，强求 90% 往往会导致模型过拟合。 张量存储架构与多模态联合检索 (11:51) Gopalakrishnan 说明了张量存储与传统向量存储的本质区别：传统向量仅是单维浮点数组，仅能表征单个内容切片；张量则支持存储多维嵌入数组，允许系统对同一段文本同时应用多种不同的专用模型嵌入。这使系统能够直接在引擎内部跨多种模态执行并行数学运算，省去了繁琐的外部自定义工程，降低了计算基础设施成本。 在具体的生命科学演示中，一份展示司美格鲁肽（semaglutide）降低主要不良心血管事件（MACE）20% 的幻灯片图表，其“20%”数据完全存在于视觉图表内部。若仅靠文本检索，当正文使用完整短语而缺少“MACE”缩写时便会落空；通过将单页内容同时建立文本与图像索引，统一引擎即可在单次查询中完成视觉信号与关键词的匹配。在更复杂的蛋白质与抗体检索案例中，系统能在单一张量排序函数内，同时执行蛋白质结构向量相似度匹配、物理实验条件标量范围过滤、非小细胞肺癌（NSCLC）词法匹配以及最近 3 个月的时间限制过滤，免去了级联多系统查询的复杂性。 生产级落地原则与实时流式数据 (17:11) Chase 与 Gopalakrishnan 强调，现代高级 RAG 必须从粗粒度的文档级检索转向精确的文档片段级提取，摒弃传统批处理索引方式，转为使用实时流式数据以保障鲜度。这一架构已被规模化验证，例如 AlphaSense 利用 Vespa 检索超过 5 亿份企业文档，Perplexity 驱动全网实时搜索，Spotify 索引每期播客的每个单词以支持片段查询，RavenPack 则用于提供低延迟金融数据。 演讲者总结了构建生产级系统的核心原则：检索与排序依然是最复杂的架构难关，自动化云服务拼装工具无法解决分块、嵌入和排序的设计挑战；向量只是检索信号之一，系统需依赖结合多嵌入模型的混合检索策略；数据分块策略必须保持弹性，需结合段落级文本与页面级光学字符识别（OCR）以及局部摘要来解析复杂表格；此外，系统拼接所产生的接口断层是导致规模化失败的主要诱因。 Vespa.ai 提出的张量级一体化架构，旨在消除传统 RAG 管道中检索、过滤与重排组件分离所带来的性能衰减，为多模态解析、实时鲜度与高并发查询提供了明确的技术路径。然而，演讲主要聚焦于整体架构原则及标杆客户案例（如 Perplexity 与 Spotify），对于既有复杂系统向统一张量引擎迁移时的工程摩擦与改造代价，仍有待企业在具体实践中进一步验证。

## Background 本内容由 YouTube 频道 Ai4 于 2026 年 8 月 19 日发布。视频由 Vespa.ai 的产品营销总监 Bonnie Chase 与医疗生命科学技术战略负责人 Harini Gopalakrishnan 共同主讲，深入探讨了构建可扩展的多模态、实时与个性化企业级 RAG 系统的核心架构挑战与解决方案。 ## TL;DR 演讲指出决定 RAG 效果的核心在于上下文质量而非单纯的模型微调或扩大上下文窗口。随着系统走向生产环境，跨系统拼接会导致严重的延迟与性能衰减，因此需要借鉴成熟搜索引擎的经验，采用多阶段排序、实时索引与混合检索。通过 Vespa 的单引擎与张量存储架构，企业能将向量计算、词法检索、结构化过滤和多模态特征原生统一，从而在医疗科研、金融分析及电商推荐等复杂场景中实现高吞吐、低延迟且无幻觉的精准生成。 ## Atmosphere & style 整场演讲风格专业务实且兼具生动幽默，开场打趣澄清品牌名称迅速拉近了与观众的距离。两位讲师配合默契，在技术理论剖析与行业前沿用例之间自如切换，不仅条理清晰地解构了底层张量架构，更直击企业在 RAG 落地时的隐性痛点。听感兼具深度架构探讨与实践落地指南的扎实感，充满极客洞察与工程说服力。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=4s) 现代 RAG 与检索技术的演进聚焦于三大核心维度：实时性（Real-time）、多模态（Multimodal）与深度个性化（Personalized）。 - [01:32](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=92s) 决定 RAG 生成质量与抗幻觉能力的关键在于上下文的纯度与精准度，而非盲目微调模型或无限制扩大上下文窗口。 - [01:32](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=92s) 企业级 RAG 落地面临三大现实挑战：非结构化多模态数据的融合、高频业务数据的秒级时效性，以及用户隐性上下文的感知与提取。 - [05:03](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=303s) 将向量库、全文检索与重排模型割裂部署会带来严重的系统间联接（System Joins）内耗，导致数据漂移、延迟累加及隐性失效（Quiet failure）。 - [07:02](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=422s) 现代检索架构应吸收搜索引擎 20 年的成熟经验，采用“混合检索粗筛 + 机器学习重排（MLR）+ 实时索引”的多阶段排序机制。 - [07:02](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=422s) 构建 RAG 系统必须建立基于真实语料的基准测试集，通常 60%-70% 的基准评测得分是避免系统过拟合的理想平衡点。 - [09:23](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=563s) Vespa.ai 采用单一引擎平台方案，将向量搜索、关键词检索、结构化过滤和 MLR 深度整合，消除了跨服务通信瓶颈。 - [10:31](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=631s) 从传统一维向量存储（Vector Store）向多维张量存储（Tensor Store）演进，可在底层原生利用数学公式直接完成跨模态特征计算。 - [12:52](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=772s) 生产级检索必须执行受控上下文生成，严禁将数十条初筛结果全量塞给大模型，应仅喂入精排出的 Top 3 到 Top 6 高纯度片段以杜绝幻觉。 - [14:46](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=886s) 在生命科学等高门槛场景中，支持在单一排序函数内同时融合蛋白质序列向量、实验数值区间、医学术语与时效性权重打分。 - [17:26](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=1046s) 搭建生产级 RAG 需摒弃机械化的拼接方案，告别定时批处理索引，实现流式实时更新、灵活分块以及“检索即精排”的一体化架构。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=4s) 检索技术的未来趋势与行业现实挑战 本场演讲由 Vespa.ai 的产品营销总监 Bonnie Chase 与医疗生命科学领域的 GO-To-Market 及技术战略负责人 Harini Gopalakrishnan 共同主讲。Bonnie 首先阐述了当前检索技术（Retrieval）的发展脉络：尽管大众对 AI 的未来有着极为科幻的想象，但检索领域的未来发展路径必须脚踏实地。现代检索技术的核心演进方向被总结为三个关键维度：**实时性（Real-time）**、**多模态（Multimodal）** 与 **个性化（Personalized）**。 虽然许多企业已经推出了相关应用甚至将部分功能投产，但在通往完全成熟的道路上仍然存在诸多技术瓶颈。本次分享重点剖析企业在突破这些瓶颈时面临的具体难题，并探讨解决方案。 * **演化三大方向**：检索系统正从单一的静态文本查找，演变为支持实时更新、跨介质数据融合以及基于用户语境的个性化推荐。 * **主要突破目标**：解决系统从原型阶段迈向高并发、低延迟生产环境时的性能衰减与架构瓶颈。 --- ### [01:32](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=92s) 从“误解”到“重构”：为什么上下文控制决定了 RAG 的成败 Harini 首先幽默地打趣道，很多人听到“Vespa”以为他们是一家生产轻便摩托车（Scooter）的公司，但事实上，Vespa.ai 是一家处理张量（Tensor）计算的前沿搜索与检索平台。在探讨检索增强生成（RAG）系统时，当前技术路线普遍存在一个误区：当聊天机器人出现幻觉或回答失效时，人们往往倾向于微调模型（Fine-tuning）、更换更高版本的模型，或寄希望于具备超强推理能力的新模型。 然而，真正决定生成质量的关键在于**上下文（Context）的质量与精准度**。无论大语言模型（LLM）的上下文窗口拓展到几十亿还是上百亿 Token，模型始终只能基于被喂入的上下文进行推理。通过精细化控制和提供高质量的上下文，才能从根本上提升回答的准确性。 ```mermaid flowchart TD accTitle: RAG 上下文优化与推理流程图 subgraph Input ["数据与查询输入"] A["用户查询与隐性信号"] B["多源异构数据<br>(文本/图像/表格)"] end subgraph Core ["Vespa 精细化检索与重排"] C["多阶段混合检索<br>(词法 + 向量)"] D["张量与机器学习重排<br>(MLR / 用户上下文)"] C --> D end subgraph Generation ["模型生成"] E["高纯度精准上下文"] F["LLM 推理与回答生成"] E --> F end A --> C B --> C D --> E ``` 企业级 RAG 在实际落地中最常遭遇三大难以解决的痛点： * **多模态与混合信号（Multimodal & Hybrid Signals）**：对于单纯的文本数据，将文本切块（Chunking）并进行向量化即可搭建基础 RAG；但在医疗生命科学或金融等领域，大量关键情报沉淀在 PPT、PDF、图表及趋势图中，多模态数据会导致传统 RAG 系统在扩展性上急剧衰退。 * **数据时效性（Data Freshness）**：在电商等高频更新场景中，商品库存和价格时刻在变。如果用户询问“能否购买某双红鞋”，AI 不仅需要理解语义，还必须保证底层的索引能秒级实时更新，确认商品处于可售状态。 * **感知上下文的回答（Context-aware Responses）**：用户输入的提问往往非常简短，无法涵盖所有相关信号。系统不能简单地对提问做语义相似度搜索，而必须在检索与排序阶段主动提取未在提问中显式表达的隐性信号（如用户画像、历史行为等）。 --- ### [05:03](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=303s) 生产环境的“隐形崩溃”与系统联结瓶颈 Bonnie 接续分析指出，随着越来越多的团队将 RAG 系统推向生产环境，核心矛盾已从“如何完成从原型到生产（Prototype to Production）的跨越”，转变为“如何在数据量激增、用户增多和多模态并发的情况下，保持数据新鲜度、准确性与极速响应”。 在规模化扩展的过程中，系统通常不会直接抛出明显的错误报错信息，而是出现“隐性失效（Quiet failure）”——即检索质量悄然下降、数据发生漂移（Data drift）。导致这种现象的根本原因在于传统检索架构的割裂： * **系统间联接（System Joins）的内耗**：典型的检索流程通常将向量检索、文本检索、重排序（Re-ranking）模型部署在不同的独立系统中。 * **性能与成本的恶化**：跨系统通信和数据传输会导致延迟累加、信息丢失，使得检索质量下降，同时运维成本随着规模扩大而呈指数级上升。 --- ### [07:02](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=422s) 借力搜索技术 20 年积累：多阶段排序与实时索引 针对上述挑战，Harini 提出企业不应盲目重造轮子，而应吸取搜索引擎行业发展 20 年所积累的成熟经验。以 Perplexity 等新兴 AI 搜索引擎为例，它们能在亚秒级时间内扫描全网海量数据，准确给出包含网页出处引用的回答。搭建现代 Enterprise RAG 应该引入以下成熟的技术框架： ```mermaid flowchart LR accTitle: 搜索引擎级多阶段检索与排序架构 subgraph Stage1 ["粗筛阶段 (Recall)"] A["原始查询"] --> B["混合检索<br>(BM25 + 向量空间)"] B --> C["海量候选集<br>(数千/数百万结果)"] end subgraph Stage2 ["精排阶段 (Fine Ranking)"] C --> D["机器学习重排<br>(MLR 模型)"] E["隐性上下文/用户画像"] --> D D --> F[" Top K 优质结果<br>(Top 4 - 10)"] end ``` * **混合检索（Hybrid Retrieval）**：不再局限于单纯的余弦相似度或 L2 距离计算，而是结合词法关键词（Lexical）、语义相似度（Semantic）以及机器学习排序函数，多维度衡量答案与问题的匹配度。 * **多阶段排序（Multi-phase Ranking）**：首先使用轻量级的混合检索技术在大规模数据集中快速“撒网”，召回数千至数百万候选结果；随后采用机器学习重排（Machine Learned Ranking）并结合用户上下文等信号，精细化筛选出最核心的 Top 4 或 Top 10 结果。 * **实时索引（Real-time Indexing）**：突破传统的批量更新（Batch processing）限制，建立高吞吐、低延迟的数据流水线，确保新数据秒级可检索。 * **相关性基准评估（Relevance Evaluation）**：建立由真实用户验证过的测试基准数据集（Benchmarking Corpus）。在检索系统中，60%-70% 的基准得分通常是较为理想且不易过拟合的平衡点。 --- ### [09:23](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=563s) Vespa.ai 单一平台架构与应用场景 Bonnie 总结了 Vespa 在打通这些环节中的技术方案。Vespa.ai 作为一个可扩展的 AI 搜索平台，将向量检索、文本检索、结构化数据过滤以及机器学习重排（MLR）深度集成至**单一引擎（Single Engine）**中，消除了跨系统 Join 操作带来的延迟与成本开销。 * **多场景覆盖**：Vespa 不仅能用于基础搜索，还全面支持电商个性化推荐、Agent 智能体工作流（为 Agent 提供接地气的 Context 支撑）等场景。 * **全栈能力集成**：在一个引擎内部同时实现检索（Retrieval）、排序（Ranking）、推理（Reasoning）、实时流式更新（Streaming updates）与结果服务（Serving）。 * **代表性客户**：已成功赋能 Spotify、Perplexity 及 RavenPack 等高并发、大数据的实时 AI 平台。 ### [10:31](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=631s) 行业客户案例与张量存储（Tensor Store）的技术演进 在实际生产场景中，底层检索架构直接决定了 AI 系统的响应速度、运维复杂性与事实准确性。AlphaSense 利用 Vespa 处理超过 5 亿份文档；Perplexity 将互联网索引与实时检索全部交给 Vespa 驱动，"每当你在 Perplexity 上提问时，背后的检索全是由 Vespa 完成的"；Spotify 借助其索引每一集播客中的每一个单词，实现即便用户记不清节目名称也能精准找回音频的强大搜索与个性化推荐；RavenPack 则以此向用户分发海量金融数据。无论处于何种垂直行业，核心诉求都高度一致：在极低延迟下获取具备上下文准确性的实时答案。 传统 RAG 系统多依赖一维的向量存储（Vector Store），即将文本段落通过单一 Embedding 模型转化为浮点数向量。然而在高质量多模态检索中，Vespa 采用了更高维度的张量存储（Tensor Store）。张量能够将同一文档通过不同模型生成的多种向量表示统一存储在一个数组中。例如针对同一段落存储两种维度的表征，直接在底层运用数学公式跨模态计算相似度，无需按模态逐个调用检索并手动合并，从而大幅削减了基础设施成本、降低了检索延迟并省去了繁重的工程开销。 ```mermaid flowchart TD accTitle: 传统向量存储与多模态张量存储架构对比 subgraph Traditional["传统向量存储 (Vector Store)"] A["输入内容 (文本)"] --> B["单一 Embedding 模型"] B --> C["一维向量表示 [x1, x2,... xn]"] C --> D["单模态粗筛 -> 耗费二次工程合并"] end subgraph VespaTensor["Vespa 张量存储 (Tensor Store)"] E["输入内容 (文本/图像/数据)"] --> F["多模态/多模型 Embedding 阵列"] F --> G["多维张量表示 [[Embedding A], [Embedding B]]"] G --> H["底层原生数学公式统一计算跨模态相关性"] end ``` - **AlphaSense**：在单一平台上检索与分析超过 5 亿份复杂文档。 - **Perplexity**：依托 Vespa 进行全网数据索引与极速检索生成。 - **Spotify**：实现播客全量逐字索引与毫秒级个性化音频推荐。 - **RavenPack**：为金融级分析提供超低延迟的数据检索管道。 --- ### [12:52](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=772s) 构建生产级检索方案的五大核心模式 无论开发者选择何种平台构建检索系统，都必须落地五项基本设计模式，以确保 RAG 系统具备高召回率与抗幻觉能力： ```mermaid flowchart LR accTitle: 高质量检索系统的五大核心模式流程 P1["1. 上下文提取与向量化<br/>(用户部门/历史偏好)"] --> P2["2. 混合检索<br/>(关键词 + 语义向量)"] P2 --> P3["3. 动态精排<br/>(复杂业务逻辑打分)"] P3 --> P4["4. 精简上下文生成<br/>(仅向 LLM 喂入 Top 3~6)"] P4 --> P5["5. 体系化评估<br/>(nDCG / 基准测试集)"] ``` 1. **提取上下文并向量化（Extract context and embed）**：不仅要对文档、音频或视频进行 Embedding，更要在用户提问时提取用户上下文（例如所在部门、历史提问记录、甚至电商中偏好的颜色信号）并融入向量检索。 2. **混合检索（Hybrid Retrieval）**：绝不能仅依赖单一方式，系统必须至少融合词法关键词检索（Lexical Keyword）与语义向量检索（Cosine、L2 距离等）。 3. **复杂精排（Ranking）**：语义相似度仅能给出基础距离分值，系统需要叠加业务上下文执行更复杂的多维度逻辑重排。 4. **受控上下文生成（Generation）**：严禁将初筛出的前 50 条结果一股脑塞给大语言模型；必须只喂入精排出的 Top 3 到 Top 6 高分片段，否则大模型极易产生幻觉。 5. **持续评估（Evaluation）**：必须引入基准测试文档与标准评估指标（如 nDCG），建立完善的评测语料库，量化评估 RAG 效果。 --- ### [14:46](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=886s) 生命科学领域的图文多模态与多条件联合检索 在生命科学等高门槛领域，许多关键信息往往封存在 PDF 图表或图像内。例如，药企幻灯片中记载了某款司美格鲁肽（Semaglutide）药物能降低 20% 的主要心血管不良事件（MACE）。当医生或研究员提问“司美格鲁肽是否有降低 MACE 的效果”时，答案“20%”仅存在于图表像素中，文字并未显式提及。Vespa 通过将页面同时嵌入为文本向量和图像特征向量，在查询时同时匹配视觉信号与关键词，并执行联合排序，直接命中并返回该页面。如果用户换用“心血管事件”进行自然语言提问，语义检索能捕捉到缩写 MACE 的含义，弥补纯关键词检索的漏检。 更复杂的场景体现在抗体与蛋白质的多条件筛选上。科研人员往往需要在一道查询中同时满足四个不同维度的约束，Vespa 能在单一排序函数内完成这种跨模态融合： ```mermaid flowchart LR accTitle: 四模态融合检索流程 Q["复合查询: 匹配非小细胞肺癌研究中的特定抗体"] Q --> M1["1. 蛋白质序列向量相似度<br/>(非结构化数据)"] Q --> M2["2. 试验物理条件过滤<br/>(温度/压力变体数值)"] Q --> M3["3. 疾病关键词匹配<br/>(NSCLC / 肺癌研究)"] Q --> M4["4. 时效性衰减打分<br/>(限定近 3 个月最新成果)"] M1 & M2 & M3 & M4 --> RF["单一 Ranking 函数融合运算"] RF --> Out["输出最精准的目标抗体候选项"] ``` - **视觉与语义双路打分**：避免了翻阅几十页文档的低效交互，前一至三条命中结果即可定位核心图表。 - **多模态合一执行**：蛋白质向量计算、数值区间过滤、医学术语匹配与时效性权重在引擎底层一次性打分完成，彻底省去了在不同数据库之间折返拼接数据的延迟。 --- ### [17:26](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=1046s) RAG 的未来趋势与关键总结 从传统 RAG 走向现代生产级架构，技术演进聚焦于三个核心维度： - **多模态深入细粒度碎片（Multimodal）**：不仅停留在文档级匹配，而是深挖图表、图像与局部片段。 - **个性化即时打分（Personalization）**：摆脱宽泛的全局相似度，在“检索的同时完成精排（ranked as they're being retrieved）”，让检索出的就是最符合该用户上下文的最优解。 - **从批处理转向实时流式刷新（Real-Time）**：告别定时全量重建索引的批处理模式，通过流式更新保证索引数据与现实世界毫秒级同步。 演讲最后总结了构建现代 RAG 必须牢记的五大核心要点：第一，搭建 RAG 并非简单将各大云厂商的开箱即用服务拼凑连线，检索与排序才是最坚硬的技术壁垒；第二，向量只是一种信号，必须拥抱混合检索与多模态策略；第三，分块（Chunking）策略必须保持灵活，切忌机械限定在段落或单字，面对包含复杂表格的页面，结合整页 OCR 与页面级摘要的粗细粒度结合往往更有效；第四与第五，系统扩展时的架构接缝（Seams）是导致质量劣化与运维灾难的根源，唯有将向量、文本、过滤与打分深度统一，才能打造出高并发、低延迟的生产级 AI 系统。用户亦可访问 `vespa-demos.ai` 体验视频搜索、以图搜图、图文互搜及电商个性化等互动演示。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=4s) 现代检索技术的核心演进方向被总结为三个关键维度：实时性（Real-time）、多模态（Multimodal）与个性化（Personalized）。 — Bonnie Chase > [01:32](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=92s) 真正决定生成质量的关键在于上下文（Context）的质量与精准度。无论大语言模型（LLM）的上下文窗口拓展到几十亿还是上百亿 Token，模型始终只能基于被喂入的上下文进行推理。 — Harini Gopalakrishnan > [05:03](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=303s) 核心矛盾已从“如何完成从原型到生产（Prototype to Production）的跨越”，转变为“如何在数据量激增、用户增多和多模态并发的情况下，保持数据新鲜度、准确性与极速响应”。 — Bonnie Chase > [05:03](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=303s) 在规模化扩展的过程中，系统通常不会直接抛出明显的错误报错信息，而是出现“隐性失效（Quiet failure）”——即检索质量悄然下降、数据发生漂移（Data drift）。 — Bonnie Chase > [07:02](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=422s) 多阶段排序（Multi-phase Ranking）：首先使用轻量级的混合检索技术在大规模数据集中快速“撒网”，召回数千至数百万候选结果；随后采用机器学习重排（Machine Learned Ranking）并结合用户上下文等信号，精细化筛选出最核心的 Top 4 或 Top 10 结果。 — Harini Gopalakrishnan > [09:23](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=563s) Vespa.ai 作为一个可扩展的 AI 搜索平台，将向量检索、文本检索、结构化数据过滤以及机器学习重排（MLR）深度集成至单一引擎（Single Engine）中，消除了跨系统 Join 操作带来的延迟与成本开销。 — Bonnie Chase > [10:31](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=631s) 每当你在 Perplexity 上提问时，背后的检索全是由 Vespa 完成的 — Harini Gopalakrishnan > [12:52](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=772s) 严禁将初筛出的前 50 条结果一股脑塞给大语言模型；必须只喂入精排出的 Top 3 到 Top 6 高分片段，否则大模型极易产生幻觉。 — Harini Gopalakrishnan > [17:26](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=1046s) 摆脱宽泛的全局相似度，在“检索的同时完成精排（ranked as they're being retrieved）”，让检索出的就是最符合该用户上下文的最优解。 — Bonnie Chase > [17:26](https://www.youtube.com/watch?v=I2bbSzxwsaY&t=1046s) 搭建 RAG 并非简单将各大云厂商的开箱即用服务拼凑连线，检索与排序才是最坚硬的技术壁垒；第二，向量只是一种信号，必须拥抱混合检索与多模态策略。 — Bonnie Chase ## Mentioned Bonnie Chase, Harini Gopalakrishnan, Vespa.ai, Perplexity, Spotify, RavenPack, AlphaSense, BM25, MLR (Machine Learned Ranking), nDCG, Semaglutide (司美格鲁肽), vespa-demos.ai ## Video description Bonnie Chase - Director of Product Marketing, Vespa.ai Harini Gopalakrishnan - GTM & Tech strategy, Industry (Health & Lifesciences), Vespa.ai In this session, we’ll explore what it really takes to build scalable multimodal and personalized RAG systems for real-world applications. How do you search across images and text simultaneously to surface the most relevant answers in life sciences or financial services? How do leading e-commerce platforms combine semantic understanding, inventory, pricing, user behavior, and personalization in real time? We’ll break down the core architectural challenges behind modern retrieval systems, including latency, data freshness, hybrid search, ranking orchestration, and combining structured, unstructured, and vector-based data within a single query pipeline. The session will also demonstrate how Vespa powers large-scale AI applications by unifying retrieval, ranking, vector search, and serving in a single platform — enabling companies such as Spotify, Perplexity, and RavenPack to deliver real-time AI experiences at scale. Through real-world examples from life sciences, financial services, and e-commerce, you'll learn how organizations are moving beyond basic RAG architectures to build AI systems that are accurate, adaptive, and production-grade. [Source](https://www.youtube.com/watch?v=I2bbSzxwsaY)

[0:04] Bonnie Chase: 现代检索技术的核心演进方向被总结为三个关键维度：实时性（Real-time）、多模态（Multimodal）与个性化（Personalized）。 [1:32] Harini Gopalakrishnan: 真正决定生成质量的关键在于上下文（Context）的质量与精准度。无论大语言模型（LLM）的上下文窗口拓展到几十亿还是上百亿 Token，模型始终只能基于被喂入的上下文进行推理。 [5:03] Bonnie Chase: 核心矛盾已从“如何完成从原型到生产（Prototype to Production）的跨越”，转变为“如何在数据量激增、用户增多和多模态并发的情况下，保持数据新鲜度、准确性与极速响应”。 [5:03] Bonnie Chase: 在规模化扩展的过程中，系统通常不会直接抛出明显的错误报错信息，而是出现“隐性失效（Quiet failure）”——即检索质量悄然下降、数据发生漂移（Data drift）。 [7:02] Harini Gopalakrishnan: 多阶段排序（Multi-phase Ranking）：首先使用轻量级的混合检索技术在大规模数据集中快速“撒网”，召回数千至数百万候选结果；随后采用机器学习重排（Machine Learned Ranking）并结合用户上下文等信号，精细化筛选出最核心的 Top 4 或 Top 10 结果。 [9:23] Bonnie Chase: Vespa.ai 作为一个可扩展的 AI 搜索平台，将向量检索、文本检索、结构化数据过滤以及机器学习重排（MLR）深度集成至单一引擎（Single Engine）中，消除了跨系统 Join 操作带来的延迟与成本开销。 [10:31] Harini Gopalakrishnan: 每当你在 Perplexity 上提问时，背后的检索全是由 Vespa 完成的 [12:52] Harini Gopalakrishnan: 严禁将初筛出的前 50 条结果一股脑塞给大语言模型；必须只喂入精排出的 Top 3 到 Top 6 高分片段，否则大模型极易产生幻觉。 [17:26] Bonnie Chase: 摆脱宽泛的全局相似度，在“检索的同时完成精排（ranked as they're being retrieved）”，让检索出的就是最符合该用户上下文的最优解。 [17:26] Bonnie Chase: 搭建 RAG 并非简单将各大云厂商的开箱即用服务拼凑连线，检索与排序才是最坚硬的技术壁垒；第二，向量只是一种信号，必须拥抱混合检索与多模态策略。

[0:04] Bonnie Chase: while popular discussions around AI often center on futuristic and abstract concepts, Chase emphasizes that the future of retrieval must remain grounded in solving immediate, real-world operational challenges. [2:32] Harini Gopalakrishnan: expanding context windows to billions of tokens or using massive models does not fundamentally resolve hallucinations because models reason strictly over the specific context provided to them. [2:32] Harini Gopalakrishnan: in domain-specific fields like healthcare and life sciences, critical intelligence is often trapped in visual formats such as PowerPoints, PDFs, graphs, and charts, where standard text chunking fails to scale. [5:03] Bonnie Chase: The primary obstacle for AI teams is no longer moving from prototype to production, but scaling once in production. [5:03] Bonnie Chase: The glue and joins between these separate systems create latency, data drift, degraded response quality, and compounding operational costs. [5:03] Bonnie Chase: When these systems fail at scale, they do not throw explicit error messages; they simply degrade silently. [6:57] Harini Gopalakrishnan: A benchmark accuracy score of 60% to 70% indicates a solid retrieval baseline, whereas chasing 90% often leads to overfitting. [11:51] Harini Gopalakrishnan: Storing arrays of embeddings within a single tensor allows mathematical computations to run directly across multiple modalities simultaneously, avoiding complex custom engineering and saving compute infrastructure costs. [11:51] Harini Gopalakrishnan: Passing only the top 3 to 6 candidates to the LLM for answer generation rather than top 50, avoiding model hallucination. [17:11] Bonnie Chase: Personalization must go beyond generic vector similarity by applying tailored scoring and inline ranking—evaluating relevance during retrieval rather than performing a separate, delayed post-retrieval ranking step. [18:56] Harini Gopalakrishnan: retrieval and ranking remain the hardest problems to solve. [18:56] Harini Gopalakrishnan: Vectors are only one signal in search; systems require a hybrid strategy using multiple embeddings simultaneously to handle multimodal contexts effectively. [18:56] Bonnie Chase: scaling is where search quality typically degrades, and attempting to stitch separate single-purpose systems together creates failure points along the joins.

[0:04] Hello everyone. Thanks so much for joining our  session today. I hope you've had a great first [0:09] day. Um hopefully none of you got stuck in um  the registration lines for too long. Um it's [0:15] a bit crazy today. Um but yeah, so today we'll be  talking about the future of retrieval. And I think [0:20] when we we being the general public talk about  the future of AI, you know, there's a lot of cool [0:27] things happening and it's easy to think about, you  know, a lot of the futuristic stuff. When we're [0:32] talking about the future of retrieval, we're going  to stay pretty grounded and talk about, you know, [0:37] where we are going with retrieval and what those  challenges are that are preventing us from getting [0:42] there. So here, you know, future of retrieval  is real time, multimodal, and personalized. [0:48] These are all things that everyone is already  kind of working on and and um maybe has some in [0:54] production, but there's still some challenges  that exist that prevent us from going all the [0:59] way. Um so that's what we'll be talking about  today. So we'll start with u a quick introduction. [1:05] We'll talk about the problem, the solution, give  you some examples, um some real world examples, [1:11] and then we'll go ahead and conclude. We do have  a lot of content to share with you today. Um, so [1:18] we'll move pretty quickly. Um, and if you want to  dig in further to any of this, uh, we are at booth [1:24] 1119. Um, so we can dig in to any of these topics  with you. So, um, as was said, uh, I'm Bonnie [1:32] Chase. I am director of product marketing at Vespa  AI. Um, and I have about 20 years of experience [1:37] working in knowledge management, product  management, and product marketing. So um um [1:43] you know more of the technical questions will  go to Harini but um excited to be here with you [1:47] today. Harini. Yeah. Hi. Hi everyone. I'm Harini  and uh I deal with life sciences and healthcare [1:53] primarily but I'm the tech strategy and go to  market for westpa AI. Um been in this industry [1:58] for 20 plus years as well. Um with background in  working with Snowflake, Amazon and Pharma Sanophi. [2:03] So nice to be here and talk about how we bring  this to life especially in multimodality. [2:09] Um, so I guess uh while we begin I just want to  show off hands. How many of people have heard [2:13] of Vespa AI? Uh because a lot of people  think it's a scooter company. So it's not [2:17] for the rest of you out there. So definitely  it's not automobile getting into tech but it's [2:22] a bleeding edge search and retrieval platform that  deals with tensors and we'll talk more about why [2:26] tensors and what they are. Um just the previous  session talked a lot about rags. So I guess [2:32] the audience here are very familiar with designing  retrieval systems or rag or even using AI. [2:38] uh but most of us have seen that when you  ask questions after the initial craze of [2:42] chat bots things have failed the answer  comes back but there is hallucination [2:46] um so in the initial days when I was working  with customers implementing rack people used [2:51] to fine-tune the model or use a better version of  the model or think having a better reasoning model [2:55] might help solve the problem uh but now we talk we  overemphasize on context but context essentially [3:02] has existed in the world of AI search for a long  time so you can actually get better reasoning or [3:07] better answers by controlling what you call the  context that goes into what the model thinks. [3:12] And it doesn't matter if the model can think over  billion tokens, zillion tokens. If the model gets [3:16] too big, that's not important because it's always  going to reason over the context you provide. [3:21] The better and more fine-tuned context you  provide, the better your answers are going to be. [3:25] How do you get that is what we will see. Um,  and the second part of the problem that racks [3:30] generally struggle with is three. We I mean there  are many many problems, but three things that we [3:34] see haven't been solved pretty well. uh one is  visual and hybrid signals. Things work very well [3:41] when everything is textual. People have taken lot  of text, chunked it, overchunked it, engineered [3:46] the heck of it and you can build a rag. But the  when in healthcare life science, we have a lot [3:50] of intelligence that's in a PowerPoint, that's in  a PDF, that's in a graph, that's in a chart. The [3:55] minute you have multimodality, it doesn't scale  well or there is a problem. So that's one thing [3:59] that we try to address. The second is freshness.  Maybe it's not that important in uh enterprise [4:05] rag but the freshness of data is very important  in e-commerce when you have constant source data [4:10] that changes and you're having a system that is  searching it to give you an answer like can I [4:14] buy a red shoe can I buy a black shoe okay you can  but the shoe has to be available for you to buy it [4:20] so how do you keep your indexes updated you  know really fast while AI is doing reasoning or [4:25] summarization that's about the freshness of data  that your AI has to address and rack the third is [4:31] contextaware responses is people talk about uh  okay I asked a question you don't control how a [4:37] prompt can be written by a user but the reason  answer is always not in the question you can't [4:43] just take the question do a similarity search and  get an answer back and say hey this is your answer [4:47] there is a lot of hidden signals that's not in  the question and how do you extract that for [4:52] your ranking and retrieval is also extremely  important most of enterprise rag don't work [4:56] at this scale uh search companies do and we want  to bring that inspiration to your rag systems as [5:03] Yeah. So, you know, in addition to the challenges  that Harini mentioned, I think many of us already [5:08] have um some rag system in production.  And what we're finding is the challenges [5:16] aren't really about getting to production, right?  A couple of years ago, it was like how do we move [5:20] from prototype to production? Well, we're doing  that now, but now you're in production. How do you [5:25] scale while keeping your data fresh, accurate, and  fast, right? and the fact the response is quick. [5:31] So, um so that's really where we're digging  in um now and um you know the problem with [5:38] scaling after you get to production is once  things start kind of um falling apart like the [5:44] you know you have more users, you have more  data, you have more uh modalities um it's not [5:50] going to throw uh an error message and say that  something's broken. It's just going to quietly uh [5:59] quietly fail, right? So, it's not going to  continue to get better. And that's really what [6:03] we're what we're looking um at Vespa to solve.  So, why most AI team AI systems hit a ceiling? So, [6:11] when you think about the typical flow  of a retrieval system, so um you know, [6:17] you have your uh retrieval, you have your ranking,  you have um your models and typically all of these [6:26] are in different systems. So these joins between  the systems are where uh the the things start to [6:34] kind of fall apart, right? So you're they  may work really well on their own, but it's [6:39] the talking together that takes a lot of work to  make all of these different systems work together. [6:45] So um you know, when that happens, that's when it  gets slower. That's when the information starts [6:50] to drift and that's where um you know,  quality gets lower and cost compounds. [6:57] So let's talk a little bit about the solution. So  uh as we mentioned as I in the opening right the [7:02] rack system shared a lot with AI search. So we'll  come up with for example one of our customers is [7:07] perplexity. So if you look how perplexity answers  questions there's an attribution pretty much for [7:12] every line they can tell you which page it came  from but it also does it over web scale really [7:16] fast in multiseconds right so when we design  enterprise rag as customers here or even a [7:21] search solution for a legal tech or a health tech  or companies that are doing this at a big scale [7:26] remember things are not new maybe it's new for the  enterprise that's solving the problem but that's [7:32] a problem that's been solved in other industries  pretty well so the whole aim of this is to take [7:37] those inspirations and have that not reinvent  search and re-ranking and some of those terms [7:42] again but see how you can leverage them and make  your retrieval systems better. uh in fact there [7:48] have been 20 years of research on this topic right  and some of the terms you should know about are [7:53] hybrid retrieval it's not just L2 distance cosign  similarity there is a combination of keyword plus [7:58] your semantic similarity plus even more cooler  things you can do with machine learn ranking [8:03] functions that you can vain throw in any amount  of modalities to say how similar is my answer [8:09] to your question that's been asked right that's  a powerful tool multiface ranking you can [8:13] actually cast the netwide like when I've asked a  question first get millions millions of hits or [8:17] thousands of hits that match the question using  a very simple search uh paradigm like a semantic [8:22] if you want or a lexical plus semantic if you want  and then fine-tune the thousand to get your top 10 [8:28] or top four heads using things called a machine  learn ranking or having additional signals like [8:33] the user context to refine them right how do you  support that itself is uh phrase ranking approach [8:38] which we can talk about separately if you want  uh the third is real-time indexing very important [8:44] maybe in enterprises the batch data doesn't change  often But for other industries where it changes, [8:48] your pipeline has to be fed, indexed and  make it retrievable very easily. So that's [8:52] another thing you should consider. And last is  relevance evaluation. Have a benchmark of corpus [8:58] vetted by your users that you train this rack  system on. Have a benchmark score. 60 70% is a [9:04] good score to say your retrieval system is pretty  nice, right? We would all like 90% but then you're [9:08] overfitting maybe. So then use that benchmark  and that engineering principles to see where you [9:14] can increase relevancy by these different  techniques and then build your rag foron. [9:18] Well, but as I said, none of it is new. We want  you to take inspiration and uh we'll see how. [9:23] Yeah. Yeah. So, um you know, at Vespa, we we want  all of these things to work together. I'll just do [9:27] a couple of quick slides about us and then we'll  continue moving on because this is this isn't [9:32] a Vespa only solution, right? So um Vespa we are  the most scalable AI search platform for real-time [9:39] search personalization and recommendations.  Um we unify vector text structured data um [9:47] and machine learn ranking into one um one engine.  So this really helps you power the trustworthy [9:54] um scalable um data that you have. uh just  from a a platform perspective here. The first [10:00] line really shows like the different types of  applications that you can build on top of Vespa. [10:05] So it's more than search you can build um you know  a recommendation engine commerce personalization. [10:12] Um you can use Vespa for your aentic AI  workflows to keep the um the context grounded. [10:18] Um and again Vespa is the solution  that that does um retrieval, ranking, [10:23] reasoning, uh real time streaming updates and  serving um serving the answers for you. Um [10:31] yeah, and so just a few customers just to kind  of give you an example. So AlphaSense, you know, [10:37] over 500 million documents that they're working  with. Um Perplexity, I think most of you should [10:42] know Perplexity by now. uh you know we we index  the internet for them and um so anytime you [10:49] ask perplexity a question the retrieval behind the  scenes is done by Vespa. Um Spotify uses us for um [10:58] also search and personalization. So if you are  looking for a podcast that you maybe heard but [11:04] you don't remember what it was called um  you can just type in some words because [11:08] we've indexed every single word of every single  podcast for Spotify to make search a lot easier. [11:14] Um and then Ravenpack um is another example.  So this is more on finan financial data. [11:19] Um so they work with us for um getting a  lot of the financial data to their users. So [11:25] different industries one engine underneath um and  regardless of the industry uh you know it really [11:33] it it require it comes down to one thing you want  contextually accurate answers right now. So the [11:39] speed is important, the operational complexity is  important and the grounded accuracy is important. [11:46] So let's uh dig in um a little bit deeper into an  industry. So Harini will talk about uh health and [11:51] life sciences. Yep. And one thing I want to call  out is that one of the differentiating things that [11:55] um all of this is relevant for is how you embed  it. The embedding model is very important when [12:01] you're doing high quality rag as well. So one  of the idea here is that we use something called [12:05] a tensor store, not a vector store. a vector.  Consider it like a one-dimensional representation [12:10] of a document. Use an embedding. You simple thing.  Put a paragraph in, use an embedding model. You [12:16] get a floating point representation of what the  paragraph was. It's just called an embedding. [12:20] But in a tensor, you can have an array of  embeddings, right? So I can take the same [12:23] paragraph, embed it in one way, embed it another  way, and now I have two different embeddings and [12:28] that's in an array. That's called a tensor.  Why restoring it as a tensor is better. I mean [12:32] you can quickly apply math formulas to find what's  relevant across two modalities rather than doing [12:37] it one by one. Um so it saves infrastructure score  speed and you know other things. Um so and that's [12:44] one of the reasons why a lot of the solutions  that are looking for multimodal retrieval [12:48] would rather have a tensor because otherwise  you're just writing a lot of engineering time. [12:52] Um before I get into examples just for the I know  there's varied audience in the room. So some basic [12:57] concepts of what these five patterns are when you  do retrieval whether where whichever platform you [13:03] use for building your retrieval solution make sure  you have these five. First is extract the context [13:08] and embed right don't just embed you are always  going to embed all your documents or music or your [13:14] videos or any modality but when your user asks a  question see how you can extract the context and [13:20] also use that in your embedding search right and  the context can be what department was a user from [13:25] what were the past questions they asked very  common in e-commerce right when you go to [13:29] Amazon and buy a shoe they know which other shoe  to recommend because you probably like red color [13:33] the most that's a signal they had stored somewhere  so that when you look for a shoe they just don't [13:38] show you any shoe they apply the signal and find  the red shoe and that's what we mean by user [13:43] context store that as well second is retrieval we  talked about a lot of things but the bare minimum [13:49] of retrieval you should do is lexical keyword  and semantic which is cosign l2 distance etc [13:55] and using a combination is called hybrid third is  ranking every answer you get you can simply score [14:00] them based on a complex criteria we'll I'll  show you in a quick example of what they are [14:04] bare minimum uh semantic will already tell you  this is the closest match by a formula like a [14:09] number but you can apply more complex logic  to it as well four is u of course generation [14:15] once you got the top three hits feed your top five  or top six to the AI model to give an answer don't [14:20] feed your entire 50 top 50 answers it's going  to end up hallucinating anyway and last is then [14:26] always evaluate there are benchmarks document  that you can use like better to evaluate your rag [14:32] you can all create your own custom documents to  evaluate your rag and you can get a score that you [14:36] can pretty much I think the NGDC is a very common  metric. Have that ready so that you know that it [14:41] is a a wellbenchmarked corpus. uh why do we do  all this complex stuff right like my rag works [14:48] try asking a question that's a real question  that if you stop by a booth you can we can talk [14:51] more about how we demo this or how we implement  this that's a powerpoint from a life science uh [14:57] document a brand document that says uh by using a  certain class of drug called semaglutide we save [15:03] cardiovascular events by 20%. That's what maze  means, right? So when somebody asks a question, [15:08] has been a maze reduction with semagoglutide?  You're looking at the corpus and the answer 20% [15:13] is inside an image. It's not explicitly written  anywhere, but I still need the answer back so I [15:17] can have a meaningful conversation with the  doctor, right? And I need this really fast, [15:22] right? I don't want 50 documents I can click  through. I just want it in the first or second [15:25] or third hit. That's where it's powerful. So when  you embed this single page as text and as an image [15:31] and I ask a question, I'm doing two matches,  right? I'm looking at uh visual signal match and [15:37] uh keyword match and then ranking them together  and that's how I'm able to extract the exact [15:41] page and give you the answer. Uh we also can do  more cool stuff. There is semiglutide and there [15:47] is mace. Two words that we have to search for by  keyword. If I only used keyword and I had somebody [15:53] ask cardiovascular events, it's going to miss  miss the word that mace was there, right? Because [15:57] keyword is only matching mace versus mace. So  remember that using semantic and keyword matching [16:02] will give you a more likely hit and the visual  is going to give you additional uh data points. [16:07] Um this is a much cooler example but it's more  complex for the non-life science folks in the [16:12] audience. The only reason I have that up here  is to show you how powerful your rag can be. [16:17] What I've asked here is a question that tells  me to find a protein. An antibbody is a kind of [16:21] protein from assuming I have a knowledge base  of proteins that are stored and I'm asking [16:26] you to find me an exact antibbody that matches a  condition. In this case, I'm doing four kinds of [16:31] searches for the non-life sciences. Just follow  the bubbles there that says 1 2 3 4. What I'm [16:35] trying to retrieve is four different modalities  at the same time. I'm finding a similarity search [16:40] for the protein which is an unstructured data.  Then I'm finding conditions that I have to match [16:45] which is some temperature variant or pressure  variant which is just a filter value. Then I'm [16:50] finding uh the keyword match which is NSLC. I need  it only for a lung cancer study. And finally the [16:55] fourth one is for time recency. I only want the  answer from the last 3 months. So in a single [17:01] ranking function I'm able to merge four different  modalities to give you the most relevant answer. [17:05] These are some of the cooler things you can do uh  with using a very advanced retrieval technique. Um [17:11] I'm going to skip through this uh yeah in  time check. Uh we only have 3 minutes. So [17:16] um again this is an example of how you can build  it yourself. and another cool demo of how we [17:20] did a brain scan and EHR search with the Tensor  back end. But stop by the booth for all of that. [17:26] Yeah. And um as we close um you know obviously as  I said if you want to dig deeper into anything we [17:32] are at booth 1119 um so we can talk through any of  those use cases, customer stories um and examples. [17:39] Um you can also play with some demos if  you want to play around with this. Um just [17:45] go to vespa-demos.ai AI and you can um play with  different ones like video search, text to image, [17:51] e-commerce, reverse image. Um so so that's a our  hub for you to um these are interactive demos. [17:59] So let's go ahead and close uh with the future.  Now we talked about multimodal, personaliz, [18:04] and real time. But ultimately when we talk about  multimodal, it's more than just documents. We [18:09] really want to get into the fragments. And that's  the part that Harini was kind of digging into. [18:13] How do you get the exact information that you  need? Um, for a personaliz it's beyond generic [18:19] similarity. It's more about scoring that's yours.  This is where ranking is really important so that [18:25] you can um return results that are ranked as  they're being retrieved. So you're retrieving [18:30] the best results, not just retrieving results  and then ranking them. And then from batch to uh [18:37] refresh. So always staying fresh with your  data. Um and that means moving away from you [18:42] know reindexing and uh doing a batch refresh. This  is about streaming real time data so that you have [18:50] the most up-to-date answers um in your system and  uh yeah you want to take leave us with the five [18:56] takeaways. I think we rehashing the thing that we  mean the messages here. So whatever it is at the [19:01] end of the room when you leave I hope you remember  the five things. Uh it's not just plumbing [19:05] together different systems. Today you can do it  auto in autopilot way with different hyperscalers. [19:10] But retrieval and ranking is the hardest part.  Whatever autoplumbing you do, you have to define [19:15] what document and how what embedding models  you use and how you're ranking whatever you're [19:19] retrieving. Vectors are only one signal. So just  don't think of one embedding. You need to have a [19:24] strategy that's more than one embedding at a time.  So think of multimodal. Think of hybrid retrieval. [19:30] Uh chunking needs to be flexible. A lot of  conversations I used to have in the past with [19:34] life science healthcare customers were about what  is the right level of chunking. Is it a paragraph? [19:37] Is it a page? Is it every word? You need a  combination of both. So think of how you would [19:41] chunk. But be flexible. Don't restrict yourself  because sometimes your page level meaning might be [19:47] very important for a user where it's especially  if you are looking at images it has to or a table. [19:53] It is better to have a page OCR strategy that  looks at the whole table inside the page and have [19:57] some summary of the page itself. Yeah. And then  the last two points really about, you know, making [20:01] sure that you're paying attention when you scale.  That's where quality degrades and then those seams [20:05] joining those systems together is where you end up  with with more challenges. Um, so we are handing [20:12] it off now. Um, I can't get to the last slide,  but that's okay. Um, if you have any questions, [20:17] we are at booth 1119 and we'll uh we'll get out  of the way here. Thank you so much. Thank as much.