Deepset Engineer Says Context Failures Cause Most Enterprise AI Flops

Deepset工程师称上下文失效是多数企业AI失败的主因 企业级AI Agent难以实现规模化落地，根源在于模型缺乏对内部工具、数据和工作流的动态访问能力。 Mathis Lucka回顾了企业级AI历经三年的演变路径：从2023年的基础探索、2024年的原型构建，走向2025年以追求ROI为核心的规模化应用。他指出了Agent规模化落地的关键瓶颈，包括系统集成、领域知识、安全性以及较低的用户采用率。为应对这些挑战，他提出了超越简单提示工程的“上下文工程”（context engineering），即在运行时动态提供工具、协议和内部知识。最后他强调，要实现可规模化的商业价值，必须从单一模型的上下文工程转向全面的系统编排与复合AI架构。 企业 AI 的重心已从 2023 年的团队探索与 2024 年的原型构建，迈向 2025 年规模化部署并产生明确商业投资回报率（ROI）的阶段。 (0:04) Gartner 分析指出，AI Agent 必须与企业专属数据、现有业务工具和工作流程深度整合，才能真正创造组织价值，否则极易面临落地瓶颈。 (3:27) 深层系统演进正从单纯的 Prompt Engineering 转向 Context Engineering，核心在于构建让大语言模型在运行时能动态聚合全部必要上下文的底层环境。 (6:07) 借助模型上下文协议（MCP）、Agent 间协议（A2A）、高级 RAG 与持久化记忆，工程团队正有效打破知识孤岛并打通企业工具连接。 (9:14) 打造生产级企业 Agent 必须跳出单一模型思维，转向涵盖全系统编排的复合型 AI 系统（Compound AI Systems）。 (11:33)

Mathis Lucka, Lead AI Engineer at deepset, contends that achieving measurable business return on investment from enterprise AI agents requires moving beyond manual prompt engineering to dynamic context engineering and full-system orchestration. Barriers to Scaling Production-Grade Enterprise Agents (3:27) Reflecting on industry shifts from 2023 to 2025, Lucka observes that while organizations have successfully moved from basic exploration to deploying individual applications, very few operate generative AI at scale with proven business value. Citing Gartner analysis, Lucka highlights that current AI agents perform basic tool calls well but consistently struggle in complex corporate settings unless deeply integrated with organizational workflows, enterprise systems, and rigorous evaluation metrics. According to deepset's field experience, four primary obstacles prevent enterprise agents from scaling: technical friction when integrating with legacy software ecosystems; model ignorance regarding proprietary company knowledge and internal processes; enterprise security demands for authentication, guardrails, and audit logs; and low user adoption rates that undermine anticipated ROI. Context Engineering Replaces Manual Prompting (6:07) To bridge the gap between model capabilities and enterprise tasks, Lucka advocates for context engineering—a discipline framed by Shopify CEO Tobi Lütke and Andrej Karpathy as the architectural practice of dynamically supplying all requisite context so an LLM can plausibly solve an assigned task. This approach represents an evolution beyond prompt engineering, shifting context assembly from manual user input to system-level automation. Lucka contrasts the two paradigms using an email assistant: prompt engineering demands that a user manually find, copy, and paste an associate's email, calendar entries, and SharePoint notes into the chat window. In contrast, a context-engineered architecture automatically queries internal address books, retrieves meeting details from calendar APIs, and fetches documentation from internal repositories at runtime before generating the output. Protocols and Infrastructures for Dynamic Knowledge (9:14) Implementing context engineering at an enterprise level depends on standardized connectivity layers and hybrid knowledge retrieval. Lucka points to emerging open standards, such as the Model Context Protocol (MCP) for creating uniform interfaces between models and external software tools, alongside Agent-to-Agent (A2A) protocols that enable multi-agent task delegation. For knowledge ingestion, architectures must look past static context limits by implementing retrieval-augmented generation (RAG), text-to-SQL database connections, and persistent memory across user interactions. In addition to technical protocols, Lucka highlights domain-expert prompting—such as pairing AI developers with credit underwriters—to encode specialized institutional logic that pre-trained foundation models inherently lack. Lucka concludes that while context engineering solves data retrieval and system integration barriers, single models cannot resolve broader enterprise operational demands; organizations must ultimately build compound AI systems that orchestrate multiple specialized agents, tools, and workflows in concert.

deepset 首席 AI 工程师 Mathis Lucka 指出，企业生成式 AI 已从原型开发阶段迈向规模化与投资回报率（ROI）验证期，而突破落地瓶颈的关键在于从单纯的提示词工程转向上下文工程，并最终构建系统级的复合 AI 编排体系。 企业级智能体的规模化与落地瓶颈 (0:04) Mathis Lucka 回顾了企业级 AI 自 2023 年以来的演进路径：2023 年行业聚焦于数据科学团队转型与生成式空间探索；2024 年转向客户案例验证与单点投产；2025 年的核心议题则是如何规模化运行智能体并产生可衡量的 ROI。然而现场民调表明，尽管不少企业已投产单个生成式应用，但真正实现规模化并获得明确商业回报的企业寥寥无几。 Lucka 援引 Gartner 的分析指出，AI 智能体虽能完成基础工具调用，但在企业复杂环境中表现欠佳，必须与企业数据、工具和工作流深度定制才能创造价值。结合 deepset 的业务实践，Lucka 归纳了当前企业落地的四大核心障碍：传统软件生态与日常工作流难以打通（系统集成）、模型欠缺企业专有数据与业务细节（内部知识）、缺乏严格的防护栏与可审计日志（安全与审计），以及因用户实际采纳率低下导致的商业价值缺失（价值与 ROI）。 上下文工程：运行时动态装配上下文 (6:07) 为破解知识与集成困境，Lucka 阐释了由 Andrej Karpathy 与 Shopify 首席执行官 Tobi Lütke 倡导的“上下文工程”（Context Engineering）概念。Lütke 将其定义为“为模型提供全部必需上下文，使任务在逻辑上可被大模型解决的艺术”。该理念标志着行业从早期静态的提示词工程（Prompt Engineering）发生根本性演进。 Lucka 以邮件助手为例说明两者的本质差异：在提示词工程模式下，用户必须手动将收件人全名、邮箱、会议日程和原始纪要复制粘贴到提示输入框中；而在上下文工程架构下，系统通过配置动态工具环境，使智能体在接收到简单指令后，能够自主跨系统查询企业通讯录、日历以及 SharePoint 文档。上下文工程的核心本质是构建一种系统架构，使模型能够在运行时按需、自动化地组装完整业务上下文。 核心支撑组件与复合系统编排 (9:14) 在技术落地层面，解决集成与知识瓶颈依赖一系列协议与技术架构的配合。其中包括用于统一大模型与外部软件工具交互界面的模型上下文协议（MCP）、支持不同专职智能体间自主协作与委派的智能体间协议（A2A）、超越静态窗口并集成文本转 SQL 及持久记忆的检索增强生成（RAG），以及联合信贷核保师等业务专家共同设计提示流程以弥补模型通用数据的不足。 Lucka 强调，上下文工程虽然有效解决了数据调用和信息补全问题，但仅能解决企业应用的部分挑战。构建真正达到生产级水准的智能体体系，企业必须超越单模型思维，转向由多组件协调运作的全系统编排（System Orchestration），即构建复合型 AI 系统（Compound AI Systems）。 Lucka 总结认为，企业实现 AI 规模化价值的前提是通过上下文工程解决数据集成与私有知识供给，但长期的技术确定性与业务回报，最终仍取决于企业能否建立起跨模型的复合系统编排能力。

## Background 本内容由 Ai4 频道于 2025 年 8 月 27 日发布在 YouTube 平台上。视频记录了 deepset 首席 AI 工程师 Mathis Lucka 在峰会上的主题演讲，内容探讨了在企业级 AI 落地中语境工程（Context Engineering）、MCP 协议、进阶 RAG 与复合智能体系统编排的关键作用，旨在帮助企业打破从原型到规模化生产的瓶颈并实现真实的业务 ROI。 ## TL;DR Mathis Lucka 回顾了企业级生成式 AI 从团队构建、走出原型到 2025 年聚焦“规模化落地”的演进历程，并指出阻碍 Agent 规模化的四大核心瓶颈在于系统集成、知识鸿沟、安全合规与价值兑现。演讲深入阐释了从“提示词工程”向“语境工程”转变的必要性，强调系统应动态装配工具、通讯录、企业知识库与数据库以补齐推理上下文。此外，演讲强调单靠语境工程仍不足够，企业必须走向包含严格护栏与可观测性的“复合 AI 系统”与多智能体编排，并通过 YPulse 与 MANZ 两个高 ROI 的垂直落地案例，印证了系统级工程设计对创造实质商业价值的决定性作用。 ## Atmosphere & style 演讲整体呈现出专业沉稳、务实且洞察深刻的工业界工程风格。Mathis 通过现场互动开场，精准戳中企业在规模化应用中“看得多、落地少”的真实痛点，迅速建立起强烈的行业共鸣。整个叙事层层递进、逻辑严密，摒弃了空洞的概念炒作，以翔实的架构图解与客户实战案例为支撑，带给观众一种严谨、落地且极具指导意义的技术分享氛围。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=CbuMj3zSonI&t=4s) 生成式 AI 在企业端的演进经历了从 2023 年团队转型探索、2024 年走出原型进入生产，到 2025 年的核心命题转变为实现规模化（Scale）并证明实际业务 ROI。 - [03:27](https://www.youtube.com/watch?v=CbuMj3zSonI&t=207s) 企业级 Agent 在迈向规模化时普遍面临四大核心瓶颈：系统集成、企业专有知识鸿沟、安全合规与护栏机制，以及落地后因未解决真实痛点导致的内部采用率低下。 - [06:07](https://www.youtube.com/watch?v=CbuMj3zSonI&t=367s) 行业关注点正从“提示词工程（Prompt Engineering）”转向“语境工程（Context Engineering）”，后者更贴近 AI 工程师构建高可用生产系统时的核心工作。 - [06:07](https://www.youtube.com/watch?v=CbuMj3zSonI&t=367s) 语境工程的本质是在系统和代码层面预先构建接口与逻辑，在运行时为大模型动态拼装外部工具、数据和背景信息，使其具备在逻辑上合理解答任务的完整上下文。 - [09:14](https://www.youtube.com/watch?v=CbuMj3zSonI&t=554s) Anthropic 提出的模型上下文协议（MCP）及智能体间协议（A2A）正在成为大模型连接外部工具与系统集成的标准化通用接口。 - [09:14](https://www.youtube.com/watch?v=CbuMj3zSonI&t=554s) 企业级知识注入已超越单纯的文档检索，需结合进阶 RAG、跨会话长期记忆（Memory）、业务专家深度介入提示设计以及 Text-to-SQL 结构化数据接入等多维手段。 - [11:33](https://www.youtube.com/watch?v=CbuMj3zSonI&t=693s) 构建工业级 AI 系统需要跨越“模型思维”走向“复合 AI 系统（Compound AI Systems）与系统编排思维”，结合明确的 KPI、严格的安全护栏与全链路可观测性。 - [11:33](https://www.youtube.com/watch?v=CbuMj3zSonI&t=693s) 企业 IT 架构的未来图景不是单个孤立的超大 Agent，而是由具备垂直专业能力、内置合规防护并通过系统统一编排的多个智能体协同构成的蜂群（Swarm）。 - [13:31](https://www.youtube.com/watch?v=CbuMj3zSonI&t=811s) 青年消费洞察平台 YPulse 借助 deepset 构建多模态、多智能体协作系统，上线首年即实现了高达 5 倍的投资回报率（5x ROI）。 - [13:31](https://www.youtube.com/watch?v=CbuMj3zSonI&t=811s) 欧洲法律出版机构 MANZ 通过严密的安全护栏与垂直领域的语境工程设计，成功打造了高粘性的专业法律检索分析助理，超额完成商业化目标。 - [16:30](https://www.youtube.com/watch?v=CbuMj3zSonI&t=990s) 语境工程提供了关键的数据与工具拼图，但只有全系统编排（Orchestration）才能真正释放生成式 AI 的全部潜能，帮助企业跨越原型验证走向规模化生产。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=CbuMj3zSonI&t=4s) 三年回顾：从团队转型、走出原型到规模化落地 在 Ai4 峰会的演讲台上，deepset 的 AI 工程主管 Mathis Lucka 首先带领现场听众进行了一次回顾。这已经是他连续第三年在 Ai4 的舞台上面向企业决策者与工程师发表演讲。Mathis 感慨这三年间生成式 AI 领域发生的翻天覆地变化：2023 年，他重点探讨的是传统数据科学与机器学习团队的角色重塑，当时这些团队突然被要求去打造软件产品，而不仅仅是训练离线模型，核心在于如何构建最初的 AI 团队架构与探索路径；到了 2024 年，焦点转向了“从原型走向生产（Prototype to Production）”，deepset 与企业客户一同登台，拆解了从概念验证迈向线上部署的实际旅程与加速方法。 步入 2025 年，整个行业的核心命题已经演变为“规模化（Scale）”。Mathis 指出，如今的关注点不再只是让 AI 应用或 Agent 跑在生产环境里，而是它们能否在大规模运行中真正创造业务价值、带来实质性的投资回报率（ROI）。 现场的举手互动清晰地印证了这一行业现状。Mathis 邀请台下听众评估各自团队的成熟度阶段：在“探索阶段”和“已有原型但尚未上线”的环节，举手的人相当普遍；问到“已有至少一个生成式 AI 应用投入生产”时，举手的人明显变少，但依然有相当一部分；而当最终问及“谁正在大规模运行生成式 AI，并切实看到了业务价值与实质影响”时，全场举起的手寥寥无几。现场的气氛瞬间变得专注而共鸣——正如 Mathis 所言，规模化落地之所以极具挑战，是因为几乎所有企业都卡在同一个关口。 ### [03:27](https://www.youtube.com/watch?v=CbuMj3zSonI&t=207s) 企业级 Agent 规模化落地的四大核心瓶颈 根据 Gartner 对企业级智能体（Agent）现状的调研分析，当前的 Agent 固然已经能够调用工具、完成基础任务，但面对复杂多变的企业环境时仍然暴露出严重不足。Gartner 认为，只有当 AI Agent 能够针对企业私有数据、业务工具系统与既定流程进行深度定制与集成，并具备完善的评估与验证机制时，才能真正展现出超越炒作的商业价值。结合 deepset 跨行业赋能企业客户的实际经验，Mathis 将 Agent 规模化落地面临的阻碍提炼为四大核心维度： - **系统集成（Integration）**：Agent 若要有效采取行动或消费数据，就必须与企业内部现有的软件生态和业务流程无缝打通，而不是孤立存在。 - **知识鸿沟（Knowledge）**：除了偶有员工违规将敏感信息上传至公共服务的情况外，基础模型对企业的内部数据几乎一无所知；更关键的是，模型对企业独特的业务流转与运营规则缺乏理解，这些专有逻辑既不在公开网络上，也不在预训练语料中。 - **安全合规（Security）**：这是任何追求规模化部署的团队必须放在首位的防线。Agent 需要具备严格的安全护栏（Guardrails）、身份认证与授权机制，并且每一次自主操作都必须清晰可审计。 - **价值兑现（Value）**：许多企业投入大量资源构建并发布了 Agent，最终却面临极低的内部采用率（Low Adoption），由于没有解决实际痛点，系统沦为无人问津的摆设。 ### [06:07](https://www.youtube.com/watch?v=CbuMj3zSonI&t=367s) 语境工程：超越提示词工程的新范式 为了攻克这些瓶颈，技术圈近来热议的核心命题正在从“提示词工程（Prompt Engineering）”转向“语境工程（Context Engineering）”。Mathis 指出，像 Andrej Karpathy 这样的行业领袖都在积极倡导这一概念。2023 年业界曾大肆炒作“提示词工程师”这一职位的崛起，但现实中这一泡沫并未成真；相反，语境工程更准确地描绘了当下 AI 工程师在构建高可用系统时的实际工作。 关于语境工程的定义，Mathis 引述了 Shopify 首席执行官 Tobi Lütke 的深刻概括：语境工程是“为模型提供所有必需的上下文，以确保大语言模型能够在逻辑上合理解答该任务的艺术（the art of providing all of the context to the model so that the task is plausibly solvable by the LLM）”。 为了直观对比提示词工程与语境工程的本质差异，Mathis 举了一个内部邮件助手的例子： 假设用户对内部 Agent 下达指令：“请给 Joe 发封邮件，总结一下我们上次和客户 X 的会议纪要并同步最新进展。”如果单纯依赖提示词工程，这个任务几乎必然失败，因为模型缺乏必要的上下文信息：公司里可能叫 Joe 的人有十几个，模型不知道用户的具体所指，更没有 Joe 的邮箱地址和会议记录。在传统的提示词做法下，用户必须手动在 Prompt 里巨细靡遗地敲入：“叫这个名字的 Joe、其邮箱是……、会议纪要包含以下几点……”。 ``` 传统提示词工程 (Prompt Engineering): [ 用户手动输入 ] ──> 繁琐拼接(背景、邮箱、纪要) ──> [ LLM 执行发信 ] 现代语境工程 (Context Engineering): ┌──> [通讯录工具] ──> 检索对应 Joe 与邮箱 [ 用户简短指令 ] ──> ├──> [日历工具] ──> 检索与客户 X 的近期会议 ──> [运行时动态组装] ──> [准确执行发信] └──> [SharePoint] ──> 抓取会议纪要与议程 ``` 相比之下，语境工程则是系统架构师在代码层面预先构建的工程体系：赋予 Agent 访问公司通讯录检索邮箱的工具、连接内部日历查询与客户 X 最近会议排期的工具，以及连接企业知识库（如 SharePoint）抓取会议议程与笔记的接口。通过这种方式，系统在运行时为模型动态拼装出全部缺失的事实拼图，让看似模糊的指令在完备的语境支持下得以完美解决。 ### [09:14](https://www.youtube.com/watch?v=CbuMj3zSonI&t=554s) 突破集成与知识：MCP 协议与复合检索手段 语境工程为上述四大瓶颈中的“集成”与“知识”提供了行之有效的工程解法。在集成层面，大模型通过结构化参数调用外部工具并接收执行结果，已经成为最稳健的模式。当前，Anthropic 提出的模型上下文协议（Model Context Protocol，MCP）正逐渐成为大模型与外部工具交互的通用标准接口；同时，像智能体间互联协议（Agent-to-Agent / A2A）等前沿规范也在快速演进，整个互联互通的技术生态正在加速成型。 在知识供给层面，技术演进超越了初期简单的文档外挂： - **进阶 RAG（检索增强生成）**：从最初为了解决模型幻觉与知识滞后，逐步演化为企业级运行时动态注入知识的基础设施。 - **系统级记忆机制（Memory）**：在系统层面对用户偏好、过往交互进行跨会话收集与沉淀，以便在未来的交互中持续复用。 - **业务专家深度介入提示设计**：Mathis 坦言，作为软件工程师，自己并不了解信贷审批员（Credit Underwriter）在判断一个项目是否准予贷款时的具体验证标准与逻辑细节，因此必须让一线业务专家参与进 Agent 行为规范与提示逻辑的设计中。 - **结构化数据接入（Text-to-SQL）**：通过自然语言转 SQL 等技术，将企业核心业务数据库中的关系型数据顺畅接入 Agent 决策链。 ### [11:33](https://www.youtube.com/watch?v=CbuMj3zSonI&t=693s) 从单点语境到全系统编排：复合 AI 系统的未来图景 尽管语境工程解决了工具调用与知识获取，但 Mathis 明确强调，语境工程并不能解决全部问题。要构建工业级、生产级的 Agent，工程团队必须跨越单纯的“模型思维”，进入“系统工程思维”。一个严肃的企业级应用必须定义明确的业务目标与量化关键绩效指标（KPIs）来追踪真实价值；必须布置严格的安全护栏以防范越狱与注入；必须具备完善的全链路可观测性（Observability），确保系统的每一个推理与动作都有迹可循。此外，复杂的预处理与后置业务流转也要求 Agent 必须深度嵌入到现有的自动化工作流中。 业界通常将这种架构称为“编排（Orchestration）”或“复合 AI 系统（Compound AI Systems）”。回顾 Tobi Lütke 关于语境工程的定义——“为模型提供所需的一切语境”，Mathis 强调：“当系统做到这一步时，它已经不再只是一个大模型，而是一套完整的软件应用程序。” 展望未来，企业 IT 架构的终局绝不是单个孤立的超大 Agent，而是一个由众多紧密协作的智能体构成的“蜂群（Swarm）”。各个子智能体专注于特定领域，具备高稳健性、内置合规防护、熟练调用专业工具与数据库，并在统一的系统编排下协同完成跨部门的复杂业务决策。 ### [13:31](https://www.youtube.com/watch?v=CbuMj3zSonI&t=811s) 实战案例：市场洞察平台 YPulse 与法律引擎 MANZ 为了证明系统编排与语境工程结合所带来的实际商业成效，Mathis 分享了两个典型的客户落地案例： - **青年消费市场研究机构 YPulse**： YPulse 长期专注于 Z 世代（Gen Z）与千禧一代的消费洞察，为耐克（Nike）、奈飞（Netflix）等跨国消费巨头提供战略咨询。其平台沉淀了海量的一手调研数据、分析报告与访谈内容。为了帮助品牌营销人员快速提炼诸如“如何向 Z 世代销售跑鞋”的策略建议，他们联合 deepset 构建了一套多模态、多智能体的协作系统。该系统不仅支持文本，还能处理音视频素材，其底层包含专门理解内部数据库架构与复杂查询的专用子智能体。这套方案涉及深度的语境工程设计，并与原有 SaaS 产品深度集成。上线第一年，该功能便为 YPulse 带来了高达 5 倍的投资回报率（5x ROI）。目前，他们正在进一步扩展系统能力，让 Agent 能够基于分析结果自主生成交互式数据仪表板并发送汇报邮件。 - **法律数字化出版机构 MANZ**： MANZ 是欧洲知名的法律内容平台（地位类似于美国的 Westlaw 或 Casetext）。他们基于全量判决文书、法规判例以及实时网络搜索能力，为法律专业人士打造了一款检索与分析助理。法律领域具有高度严肃与敏感的特征，因此架构中设置了极其严密的护栏，特别针对 Prompt 注入与合规越界进行了严防死守。该产品的商业化爆发力甚至超出了客户团队自身的预期：产品上线后，仅用了短短时间便提前完成了预定目标，印证了深谙专业语境的垂直应用对高价值用户群体的巨大吸引力。 ### [16:30](https://www.youtube.com/watch?v=CbuMj3zSonI&t=990s) 核心启示与 deepset 的技术定位 在演讲的尾声，Mathis 总结了全场最核心的洞察：“语境工程能够帮你拼上业务拼图中的重要板块，但只有通过全系统的编排（Orchestration），才能真正释放生成式 AI 的全部潜能。”构建 AI 应用不能只停留在挑选更强的基础模型上，而必须将精力投向整套系统架构的严密设计。 针对那些在技术选型中感到迷茫或在落地过程中遇到瓶颈的团队，Mathis 介绍了 deepset 的定位与价值主张。deepset 同时提供企业级商业平台与开源框架（Haystack 生态），在现代技术栈中定位于底层基础云设施与上层终端业务应用之间。由于 deepset 长期深耕金融、法律等受到强监管的企业级场景，其架构赋予了工程师对数据流转、组件集成与调用编排极其细粒度的控制力，旨在帮助各类企业跨越概念验证的鸿沟，稳步走向生产级规模化部署。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=CbuMj3zSonI&t=4s) 如今的关注点不再只是让 AI 应用或 Agent 跑在生产环境里，而是它们能否在大规模运行中真正创造业务价值、带来实质性的投资回报率（ROI）。 — Mathis Lucka > [03:27](https://www.youtube.com/watch?v=CbuMj3zSonI&t=207s) 只有当 AI Agent 能够针对企业私有数据、业务工具系统与既定流程进行深度定制与集成，并具备完善的评估与验证机制时，才能真正展现出超越炒作的商业价值。 — Gartner > [06:07](https://www.youtube.com/watch?v=CbuMj3zSonI&t=367s) the art of providing all of the context to the model so that the task is plausibly solvable by the LLM（为模型提供所有必需的上下文，以确保大语言模型能够在逻辑上合理解答该任务的艺术） — Tobi Lütke > [06:07](https://www.youtube.com/watch?v=CbuMj3zSonI&t=367s) 请给 Joe 发封邮件，总结一下我们上次和客户 X 的会议纪要并同步最新进展。 — Mathis Lucka > [09:14](https://www.youtube.com/watch?v=CbuMj3zSonI&t=554s) 作为软件工程师，自己并不了解信贷审批员在判断一个项目是否准予贷款时的具体验证标准与逻辑细节，因此必须让一线业务专家参与进 Agent 行为规范与提示逻辑的设计中。 — Mathis Lucka > [11:33](https://www.youtube.com/watch?v=CbuMj3zSonI&t=693s) 当系统做到这一步时，它已经不再只是一个大模型，而是一套完整的软件应用程序。 — Mathis Lucka > [11:33](https://www.youtube.com/watch?v=CbuMj3zSonI&t=693s) 企业 IT 架构的终局绝不是单个孤立的超大 Agent，而是一个由众多紧密协作的智能体构成的‘蜂群’（Swarm）。 — Mathis Lucka > [16:30](https://www.youtube.com/watch?v=CbuMj3zSonI&t=990s) 语境工程能够帮你拼上业务拼图中的重要板块，但只有通过全系统的编排（Orchestration），才能真正释放生成式 AI 的全部潜能。 — Mathis Lucka ## Mentioned Mathis Lucka, deepset, Ai4, Gartner, Andrej Karpathy, Shopify, Tobi Lütke, Anthropic, Model Context Protocol (MCP), Agent-to-Agent (A2A), SharePoint, RAG, Text-to-SQL, YPulse, Nike, Netflix, MANZ, Westlaw, Casetext, Haystack ## Video description Mathis Lucka, Lead AI Engineer, deepset Abstracts As AI adoption grows in the enterprise, it's become clear that effectively leveraging data and context is key for developing applications that drive real business value. Most AI failures are not due to model limitations but rather context failures. This presentation explores the critical role of context engineering in making AI work for your organization. We'll dive into the latest trends shaping the AI landscape, such as Model Context Protocol (MCP) for connecting tools to LLMs, Advanced Retrieval Augmented Generation (RAG) for enhancing AI with external knowledge, and Agentic Workflows for orchestrating complex AI systems. We'll also cover how to incorporate business teams and stakeholders in system design for maximum impact. Through real-world case studies, we'll demonstrate how a well-designed approach to context engineering can help businesses achieve tangible results across knowledge management, process automation, and decision intelligence. Join us to learn practical strategies to apply context in your own AI projects and measure success. [Source](https://www.youtube.com/watch?v=CbuMj3zSonI)

[0:04] Mathis Lucka: 如今的关注点不再只是让 AI 应用或 Agent 跑在生产环境里，而是它们能否在大规模运行中真正创造业务价值、带来实质性的投资回报率（ROI）。 [3:27] Gartner: 只有当 AI Agent 能够针对企业私有数据、业务工具系统与既定流程进行深度定制与集成，并具备完善的评估与验证机制时，才能真正展现出超越炒作的商业价值。 [6:07] Tobi Lütke: the art of providing all of the context to the model so that the task is plausibly solvable by the LLM（为模型提供所有必需的上下文，以确保大语言模型能够在逻辑上合理解答该任务的艺术） [6:07] Mathis Lucka: 请给 Joe 发封邮件，总结一下我们上次和客户 X 的会议纪要并同步最新进展。 [9:14] Mathis Lucka: 作为软件工程师，自己并不了解信贷审批员在判断一个项目是否准予贷款时的具体验证标准与逻辑细节，因此必须让一线业务专家参与进 Agent 行为规范与提示逻辑的设计中。 [11:33] Mathis Lucka: 当系统做到这一步时，它已经不再只是一个大模型，而是一套完整的软件应用程序。 [11:33] Mathis Lucka: 企业 IT 架构的终局绝不是单个孤立的超大 Agent，而是一个由众多紧密协作的智能体构成的‘蜂群’（Swarm）。 [16:30] Mathis Lucka: 语境工程能够帮你拼上业务拼图中的重要板块，但只有通过全系统的编排（Orchestration），才能真正释放生成式 AI 的全部潜能。

[6:07] Tobi Lütke: the art of providing all of the context to the model so that the task is plausibly solvable by the LLM [6:07] Mathis Lucka: Hey, agent, can you email Joe about our last meeting with customer X and give him an update or summary?

[0:04] My name is Martas Luca. I'm uh leading [0:07] AI engineering at Deepset. [0:10] And today I want to talk about how you [0:14] can leverage context and context [0:17] engineering to make AI work for your [0:21] business. [0:23] But before I start with that, I want to [0:27] take a step back and kind of reflect [0:31] because this is actually the third year [0:33] that I'm standing here on this stage or [0:36] maybe some other stage in this building [0:38] at AI4 [0:40] and um I want to take a look with you at [0:44] what has changed over these last three [0:48] years because I think it's a lot. So in [0:51] the first year in 2023 [0:54] um I talked about the changing role of [0:58] traditional data science and AI teams or [1:01] data science and machine learning teams [1:03] back then and how all of a sudden they [1:07] were expected to build products not just [1:10] models. [1:12] It was really a talk about giving AI [1:15] forwardthinking organizations the right [1:17] ideas around building out their initial [1:19] AI teams setting up the right team [1:22] structures to go and explore this space. [1:26] In 2024 [1:28] I talked about getting your prototype or [1:32] getting from prototype to production. Um [1:36] we brought a customer. We jointly [1:38] presented how they went through that [1:40] journey and how others uh might go [1:43] through the journey as well and how they [1:45] might accelerate it. And now um this [1:49] year I want to talk about scale. [1:52] What does it take to bring your AI agent [1:55] or your AI application from a production [1:58] application to something that drives [2:02] impact at scale that delivers ROI? um [2:07] that gives you and your business real [2:10] value. [2:12] But before I get into that, I want to [2:15] ask a question that I actually ask every [2:17] year and this going to be interesting on [2:19] how that changes. Um if you look at that [2:22] journey uh exploration, prototyping, [2:26] production, scaling, [2:29] where would you place yourselves, your [2:31] team, your company? So we're going to do [2:33] this simple show of hands. [2:36] um who is just exploring the space. [2:41] All right. [2:43] Who is prototyping Gen AI applications [2:46] but they haven't got anything in [2:48] production yet? [2:50] Okay. Now, who has at least one AI [2:54] application or let's say generative AI [2:56] application in production? [2:59] Okay, congrats. That's a big step. And [3:02] now who is running a Gen AI application [3:05] at scale where it really drives value [3:08] and impact? [3:12] Okay, a lot fewer hands. [3:15] It seems like scaling Gen AI [3:18] applications are still very challenging [3:21] and I can tell you you're not alone. [3:27] uh Gardner looked at the state of agents [3:29] in uh in the enterprise and they say [3:32] yeah agents can do stuff they can to [3:35] call tools they can um complete basic uh [3:39] tasks. However, um they still fall short [3:45] of adapting to very complex enterprise [3:47] environments [3:49] and they believe that AI agents will [3:53] only demonstrate greater value [3:57] when they're effectively customized and [3:59] integrated with enterprise data tools [4:02] systems um with the processes when they [4:06] have welldesigned evaluation [4:09] verification ben [4:11] And this really [4:14] this really resonates because we're [4:17] working with many different clients [4:19] around across different industries on [4:22] building um their AI applications and [4:26] agents [4:28] and we see the same things. And if I had [4:31] to summarize what challenges we look at [4:35] um when we're talking about scaling [4:37] agents, [4:39] I would say the following. First is [4:42] integration. [4:43] For an agent to take action effectively [4:46] to consume data, it needs to be [4:48] integrated [4:50] with your enterprise software systems, [4:52] with your business processes. [4:55] Second up is knowledge. [4:57] An agent naturally has very little or [5:00] actually mostly no knowledge about your [5:03] inter internal uh company data unless [5:06] some of someone at your company leaked [5:08] some data to chat GPT but um naturally [5:12] they have little to no knowledge of your [5:15] data. Um they also have a very shallow [5:19] understanding of your business processes [5:21] and how your company works. That's just [5:23] not on the public internet. it's not in [5:25] their training data. [5:28] Next up is security. [5:30] That should be top of mind for any team [5:33] scaling an AI agent or application. Um, [5:38] agents need guardrails, they need [5:40] authentication, authorization. [5:43] Um, the actions should be auditable. [5:46] And last but not least, value. Uh, does [5:50] your agent actually drive ROI? Does it [5:52] drive impact? Um, and most agents don't. [5:56] Most have low adoption. People build it, [5:58] they release it, nobody uses it. That's [6:02] a problem. [6:04] Now, [6:07] how might we address these challenges? [6:11] Um, you might run into uh this trending [6:15] topic right now uh on all all corners of [6:18] the internet. Context engineering. It's [6:21] embraced by [6:23] industry thought leaders like Andre Kapi [6:27] and it's kind of the attempt to reframe [6:33] or better capture what prompt [6:35] engineering tried to capture in 2023. So [6:39] you remember like people talking about [6:41] the rise of the prompt engineer. That [6:44] didn't happen. But now we're starting to [6:47] find better words for what AI engineers [6:50] are actually doing when they're building [6:51] AI agents. [6:54] Now, [6:56] how might we define context engineering? [7:00] Um Toby Lutker, CEO of Shopify says it's [7:04] the art of providing all of the context [7:08] um to the model so that the task is sol [7:12] plausibly solvable by the LLM. [7:16] Now, that's a bit abstract maybe. What [7:19] do we mean by that? Imagine you've built [7:22] an internal email agent tool that lets [7:25] you email anyone at your uh company and [7:28] you say, "Hey, agent, can you can you [7:30] email Joe about our last meeting with [7:33] customer X and kind of give him an [7:35] update, a summary?" [7:37] And now, if you look at that request, [7:39] the LLM really doesn't have any of the [7:41] context it needs to plausibly solve the [7:44] task. There might be a dozen different [7:46] Joe's at your company. It doesn't know [7:49] anything about your latest meeting. It [7:52] doesn't have Joe's email address. So [7:55] really, you left it no chance of [7:57] completing the task. And you might type [8:00] all of that into the prompt. That would [8:02] be prompt engineering. Like, okay, it's [8:04] this Joe, this email address, uh these [8:07] are the meeting notes. Put it into a [8:09] prompt. and then the learn more the [8:11] agent uh which has an email tool will [8:15] email Joe about it and that's that's one [8:18] way of approaching it. The other way and [8:20] this is context engineering would be to [8:23] give the agents all the tools to give it [8:26] a tool to access your company address [8:30] book to pull email addresses and so on [8:33] to give it a tool to your calendar so [8:35] that it could query your latest meeting [8:37] with customer X. uh to give it a tool [8:40] with access to your shareepoint so that [8:42] it can get the meeting agenda and the [8:44] meeting notes [8:46] and this way the model or agent can kind [8:50] of dynamically assemble all the missing [8:53] pieces at runtime and this was built by [8:57] by an engineer before the user sat there [9:00] and said hey please email Joe so that's [9:02] context engineering it's creating the [9:05] system so that the agent can dynamically [9:07] assemble all of the context it needs uh [9:10] to solve a task. [9:14] Now let's look at how context [9:16] engineering addresses some of the [9:18] challenges that we talked about earlier. [9:22] First up is integration. [9:25] We found a really effective way to um [9:31] to integrate large language models um [9:34] with other systems through tools. [9:37] uh agents, LLMs can call tools with [9:41] different parameters and then get [9:43] outputs. Um MCP, the model context [9:47] protocol has emerged as a little bit of [9:51] a standard [9:53] um to uh kind of like have a universal [9:57] uh interface between models and tools. [10:01] But there are other uh protocols like [10:04] agent to agent a toa which are [10:06] interesting and you should watch. Um so [10:09] this space hasn't fully played out yet [10:11] but that's a possibility to solve the [10:13] integration problem. [10:16] Now [10:18] knowledge is I think what uh we kind of [10:21] addressed at first like right after the [10:24] chatbt launched people were like yeah [10:26] but it doesn't know who the president is [10:28] and like gives me an old answer and like [10:31] I don't know uh it hallucinates a bunch [10:33] of stuff and then people figured out oh [10:35] we need rag retrieval augmented [10:37] generation we just feed it the right [10:39] data at runtime um that's one way to [10:43] provide the knowledge um there are other [10:45] ways case uh memory is an emerging topic [10:49] where kind of the system collects uh [10:52] information about its users and can [10:54] reuse that later on. Um [10:58] prompting is actually really important [11:00] like developing with experts to gather [11:03] um how an agent should behave because [11:05] typically like I'm an engineer I [11:07] typically don't know how a credit [11:09] underwriter is going through a process [11:11] of verifying if this project should be [11:14] financed or not. [11:16] So I pull in that credit underwriter to [11:20] help me write the prompts. And then you [11:22] might bring in structured data with text [11:24] to SQL for example. So you can see there [11:28] are ways to address these challenges [11:30] with context engineering. [11:33] However, [11:37] context engineering really only solves [11:40] part of the problems. We looked at [11:41] integrations and knowledge. [11:44] When you want a production grade agent, [11:46] you actually need to define KPIs and [11:50] business goals to make sure that value [11:52] creation is tracked properly. Um, you [11:55] need to install guard rails. Uh, [11:58] observability so that [12:01] um the system is behaving in the way you [12:04] expect it to behave and you can verify [12:06] that and um make it secure. You might [12:10] need complex data processing. You might [12:12] want to embed your agent into a workflow [12:14] where something runs before it and after [12:16] it. So the whole thing is a lot more [12:19] complex than what we can just solve with [12:23] context engineering. And we like to um [12:27] to call this orchestration, [12:30] orchestrating your AI system. Some [12:33] people say compound AI system. Um really [12:37] building a full application, moving away [12:39] from the model. You saw Toby Lutka's [12:42] definition like it's providing all the [12:43] context to the LLM. That's not an LLM. [12:46] That's an application. [12:49] And then if you take that further, [12:52] many people actually, many of our [12:54] clients need to connect multiple agents [12:59] to build their whole application. And [13:02] each of them need to be robust and [13:04] secure and have knowledge and [13:05] integrations and tools. And they also [13:07] need to work together. And now if you [13:10] look a bit further out into the future [13:13] and you think about the the future [13:16] landscape of your IT architecture, it's [13:19] a swarm of well orchestrated agents that [13:22] work together, process data, uh take [13:25] action and all of that needs to be [13:27] robust, auditable and so on. [13:31] So how does that look like in practice? [13:34] I brought two examples um for you from [13:37] real customers. First one is Wiples. [13:41] Um they're a longtime customer and [13:45] they've [13:47] um they focus on on market research [13:50] specifically on Gen Z and millennials [13:52] and they work with consumer brands like [13:55] uh Nike or Netflix. Um, and marketing [13:59] professionals can use their platform to [14:02] develop strategies for how to sell [14:05] running shoes to Gen Z or something like [14:08] that. Um, do research. Wip has a lot of [14:12] data and articles. They do surveys, [14:14] first party research and so on. And they [14:16] built an agent [14:18] that aggregates all of that information. [14:21] And it's actually a multi-modal. [14:23] We have different modalities, text, [14:25] audio, video, uh structured data, multi- [14:29] aent. Um we bring structured data in [14:33] with sub aents that know everything [14:36] about their database and how to query it [14:38] and so on. Um [14:41] system and it's a lot of context [14:44] engineering that goes into that. But [14:46] then there's a lot of integration work [14:47] as well. It needs to integrate with [14:49] their um with their product. it needs to [14:52] align with what their customers want and [14:54] need. And they did a pretty good good [14:57] job and achieved a 5x ROI in year one. [15:02] Um they extending the whole thing right [15:04] now for the agent to take action to [15:06] generate dashboards to send out emails. [15:09] Um so that's a really interesting um use [15:12] case that drives a ton of ROI for them. [15:17] Next up, [15:19] um, is really a breakout success. Uh, [15:22] Mans, they're legal content provider. [15:25] Uh, something like a West Law or Case [15:27] Text you might know here in the US. Um, [15:31] and they build a an assistant, an agent [15:35] for legal professionals to browse all of [15:38] their data, court decisions, rulings, [15:40] and so on. [15:42] And um there's a lot of different data [15:45] sources, their first party content, web [15:48] search that goes into it. The challenge [15:50] here it's legal. It's pretty sensitive [15:54] to topics. So there are a lot of [15:56] guardrails, prompt injection, a lot of [15:59] stuff that prevents the agent from going [16:02] off the rails. and they actually were [16:06] surprised themselves because they booked [16:09] uh their projection for the year in the [16:12] first six years in revenue. Um so again [16:16] there's context engineering here making [16:18] all of the rag work uh the web search [16:21] and so on and there's broader [16:22] integration into the existing product [16:24] the existing data landscape. [16:30] What should we take away from this? [16:33] It's [16:36] not much I would say like uh what I want [16:39] to tell you is context engineering is [16:41] going to solve part of the puzzle but [16:44] orchestration unlocks the full [16:45] potential. You need to think about your [16:48] AI in terms of a whole system not just a [16:51] model. [16:53] If you're saying thanks Mattis that was [16:56] a great talk. Um, I still don't really [16:59] know where to start or I'm I already [17:03] started but we're kind of stuck and we [17:05] don't know where to go from here. Um, [17:07] swing by our booth. We're deep set. We [17:10] have a platform and an open-source [17:12] framework because we work with uh [17:14] companies and regulated industries and [17:16] we know how they need to have a much [17:19] more fine grained control over what [17:22] they're building. Um we sit in the stack [17:25] between basic infrastructure layer and [17:27] the application layer helping companies [17:30] figure out this AI application building [17:32] orchestration and yeah [17:36] visit us and thanks