Google Uses Offline Dream Pipelines to Slash AI Agent Token Burn

谷歌利用离线“做梦”管道大幅削减AI Agent的Token消耗 自主智能体需要专门的Token预算用于离线后台提炼，因为将原始遥测数据整合为持久记忆可将任务执行成本降低高达99%。 Jason Davenport 概述了一种端到端架构，旨在将 AI Agent 从临时执行脚本演进为持续学习、自我完善的系统。通过捕获丰富的执行遥测数据并将其流式传输至 Google BigQuery，由 Gemini 1.5 Flash-Lite 驱动的后台“做梦”提取管道将轨迹提炼为结构化事实和语义偏好。这些精选记忆被持久化存储在 Google Cloud Spanner 中，并在运行时通过快速、低延迟的 RAG 动态注入，从而消除跨会话的冗余 Token 消耗。Davenport 还展示了企业团队如何管理记忆生命周期、开展队列轨迹分析，并对 Agent 集群运行自动化回归评估。 长期记忆将智能体从静态执行脚本转变为自我完善的系统，通过保留先前的解决方案大幅降低重复任务上的 Token 消耗。 (1:00) 智能体“做梦”充当离线整合子系统，系统地解析原始交互追踪记录，修剪非必要噪声，并提炼出持久的洞察。 (3:20) 组织必须将其整体 Token 配额的相当一部分专门预算用于次级观察、提炼与做梦架构。 (7:21) 在数据仓库内部使用 Gemini 1.5 Flash-Lite 等快速、高性价比的模型进行做梦提取，可将受限上下文摘要的提取成本保持在较低水平。 (11:01) 持久化智能体存储需要一种混合架构，将用于确定性特征的强类型键值对与用于语义召回的向量嵌入结合起来。 (12:37)

Google Cloud's Jason Davenport argues that autonomous agent self-improvement depends on offline memory consolidation pipelines—termed "agent dreams"—that synthesize raw execution telemetry into durable long-term storage. Rather than continually bloating prompt context windows, scalable agent fleets require separating active working memory from structured, multi-session knowledge bases. Dividing Working Memory from Long-Term Knowledge (1:00) Davenport observes that enterprise agents frequently underperform because they lack the institutional reference materials available to human staff, a deficit he identifies in roughly 90% of diagnostic reviews. While active working memory manages immediate multi-turn state, relying exclusively on massive context windows—even those scaling to one million tokens—introduces severe latency penalties and compounding financial costs. By retaining learned operational patterns in long-term memory, an agent can reduce token consumption on recurring multi-step tasks from 1,000,000 tokens down to 10,000 tokens. To manage this trade-off, agent architectures must divide execution context across specialized agent boundaries rather than deploying a single monolithic agent tasked with handling entire problem spaces. Long-term memory supplies the initial, foundational knowledge at task inception, leaving the agent's short-term scratchpad free to perform real-time summarization and execution. Offline Dream Harvesting via Telemetry Pipelines (3:20) Drawing an analogy to human sleep consolidating daily experiences, Davenport defines agent "dreaming" as an offline background process that parses execution logs, prunes operational noise, and synthesizes persistent memories. In the demonstrated reference architecture, an agent's runtime interaction trajectories, tool invocations, and model inputs are continuously logged via a Python wrapper into Google BigQuery. Rather than deploying expensive frontier checkpoints like Gemini Pro or Claude Opus for post-processing, dream harvesting leverages SQL-embedded prompts powered by lightweight models like Gemini 1.5 Flash-Lite directly inside the data warehouse. Because all requisite context resides within the query row payload, this approach bounds extraction costs while quickly isolating explicit user facts, behavioral adaptations, and system instructions. Decoupling Analytical Warehouses from Low-Latency Serving (6:06) Because analytical queries in BigQuery take several seconds to execute, live agent loops require a decoupled operational storage layer to satisfy interactive sub-300-millisecond latency thresholds. Harvested memories are synchronized into Google Cloud Spanner, which supports a hybrid schema pairing strictly typed key-value pairs (such as user roles and programming preferences) with semantic vector embeddings for similarity lookups. To prevent agents from retrieving brittle, overly specific rules, Davenport highlights the need for query expansion techniques—including HyDE and hybrid keyword-vector approaches—alongside multiple distinct extraction pipelines. Running separate pipelines allows an organization to extract granular session facts independently from high-level operational principles, which can then be baked directly into deterministic orchestration code. Cohort Trajectory Auditing and Lifecycle Governance (21:51) Managing persistent memory across large agent fleets parallels enterprise data warehousing principles like Slowly Changing Dimensions. Instead of immediately hard-deleting outdated records, architectures should log access recency, mark superseded memories as inactive, and shift unaccessed entries into cold archives to prevent active context pollution. At the fleet level, supervisors must inspect telemetry across cohorts of 50 or more sessions in a manner analogous to agile sprint retrospectives. Reviewing multi-session trajectories clarifies why an efficient run resolved a task in 500,000 tokens while an unoptimized run thrashing in loops consumed 2,000,000 tokens. Candidate memories and updated instructions must undergo regression replays against benchmark "golden sessions" using synthetic data or LLM evaluators to confirm performance improvements before promoting updates to production. Davenport concludes that meaningful advances in agent autonomy stem far more from surrounding telemetry, memory consolidation, and automated evaluation architectures than from raw foundation model upgrades. While the demonstrated reference stack costs roughly $60 due to Cloud Spanner provisioning, the persistence and evaluation layers can be adapted across relational, vector, and synthetic testing frameworks to meet an organization's specific latency budgets and governance needs.

Google Cloud 区域技术主管 Jason Davenport 指出，构建具备自我改进能力的自主 AI Agent 不能仅依赖单体提示词或静态上下文窗口，而必须建立类似生物睡眠的后台“做梦”（Dreaming）管道，将运行时遥测数据持续转化为可调用的分层长期记忆。 记忆双轨制与上下文管理 (1:00) Davenport 认为，开发者往往将 Agent 误视作单次运行的代码片段，但成熟的架构必须明确区分短期会话持久性与长期记忆。短期记忆维系多轮对话的上下文状态，防止关键信息在交互中丢失；而长期记忆则旨在解决高昂的 Token 冗余消耗——面对重复处理过十次的工作流，成熟的 Agent 应当借助沉淀的历史经验，将单次任务消耗从 100 万级 Token 骤降至 1 万级。对于企业中 Agent 表现欠佳的普遍反馈，Davenport 诊断指出 90% 的根源在于 Agent 未能获得人类员工日常使用的同等工具与企业知识库权限。 他借助儿童算术的双轨认知模式说明了记忆提炼的价值：儿童能瞬间报出“2+2=4”依赖的是机械记忆（Rote Recall），但这并不妨碍其掌握加法技能以应对复杂笔算。Agent 同样需要构建双轨机制：将高频出现的行为模式沉淀为确定性的快速执行路径，将高成本的复杂动态推理留给未知任务，以此推动系统从“单体全能 Agent”走向协作型专业化多 Agent 架构。 做梦机制与遥测数据提炼 (11:01) 在架构定义上，Davenport 将“做梦”阐释为一种离线的数据转换与清洗管道，而非记忆本身。该机制的核心在于系统级遥测采集：通过轻量化 Python 类与 Google Agent Development Kit（ADK）插件，将 Agent 在交互过程中的内部思维链路、执行轨迹、工具调用及输入输出完整流式写入 Google BigQuery 数据仓库，防止数百万 Token 的会话上下文在会话终止后直接丢失。 在记忆提炼环节，系统利用 BigQuery 与 Vertex AI 的原生集成，通过内嵌提示词的 SQL 分析查询直接在数据仓库内调用大模型完成信息抽取，无需向外部应用搬运数据。Davenport 强调必须严格遵循“Token 经济学”：由于记忆提炼属于上下文边界明确的摘要任务，团队应采用轻量低成本的 Gemini 1.5 Flash-Lite，而非昂贵的 Gemini Pro 或 Claude Opus 等尖端前沿模型。此外，工程预算必须留出相当比例的 Token 专门供给这些负责观察与做梦的次级辅助系统。 分层存储与毫秒级记忆检索 (12:37) 提取出的记忆会被同步至 Google Cloud Spanner 等持久化存储层，形成混合记忆架构。针对用户身份、核心岗位、首选编程语言及偏好基础设施等硬性约束，系统采用强类型键值对存储以保障确定性提取；而对于细微的行为倾向与上下文特征，则转为非结构化文本及其向量嵌入。Davenport 强调存储层必须满足严格的延迟服务等级协议（SLA）：BigQuery 适用于耗时数秒的批量离线分析与做梦计算，而面向线上 Agent 的实时检索则必须依赖 Spanner、PostgreSQL（`pgvector`）或 Redis 等能够实现低于 300 毫秒响应的高性能数据库。 在线上交互中，系统利用动态检索增强生成（RAG）将记忆按需注入当前 Prompt。为了避免用户口语化提问与向量空间发生偏差导致检索失准，架构中引入了次级 Agent 负责查询重写与语义扩展（如结合 HyDE 或 BM25 混合检索）。对于经过多次验证的高效操作规则，Davenport 建议直接将其硬编码进 Agent 的底层编排循环代码中，而非完全依赖不可靠的软性系统提示词。 队列分析与记忆生命周期 (21:51) 在多 Agent 协同团队中，做梦子系统扮演着类似敏捷开发中“迭代复盘会”（Sprint Retros）的角色。后台监督机制能够横向比对数十个会话队列，诊断资源消耗差异——例如分析为何同类任务有的运行仅耗费 50 万 Token 即产出理想代码，而有的却因编排计划模糊或上下文交接冗余耗费了 200 万 Token。在记忆维护方面，系统借鉴企业数据湖中的缓慢变化维（SCD）理念，通过时间戳记录访问频次，将长达数月未被调用的过期记忆转为冷归档状态，避免过时或冲突信息污染上下文。 自我改进闭环的建立高度依赖严谨的回归评测。Davenport 强调，团队需维护包含成功与失败样本的“黄金会话”基准数据集。当做梦管道生成新的候选技能或全局指令时，评测系统需重放历史基准用例，排查是否存在陷入死循环调用等病态行为；唯有通过验证并确认无退化迹象的记忆，才能最终晋升并合入生产环境的长期存储中。 Davenport 总结认为，自主 Agent 的核心演进动力已从追求基础模型参数转向构建外围的记忆管理与评估工程。面对多样化的业务指标，团队无需固守单一评估范式，而应结合合成数据生成与大模型裁判（LLM-as-a-judge）等实用路径，以可控的算力与时间成本保障 Agent 记忆体系的持续迭代。

## Background 本视频由 Ai4 频道于 2026 年 8 月 18 日发布在 YouTube 平台。内容为 Google 区域技术主管（Area Technical Lead）Jason Davenport 的主题演讲，主要探讨如何围绕 AI 智能体构建会话与记忆管理系统，通过后台生成“梦境（Dreams）”与沉淀持久记忆来驱动智能体集群的自主进化与自我完善。 ## TL;DR 演讲系统性地拆解了 AI 智能体持久记忆架构，将记忆体系划分为短期工作记忆、长期演进记忆与领域知识库三大维度。Jason 详细介绍了后台异步运行的“做梦（Dreaming）”机制——利用 BigQuery 归档全量会话遥测轨迹，结合 Gemini Flash-Lite 等轻量模型低成本提炼用户偏好与交互规则，并持久化写入 Cloud Spanner 实现 300 毫秒级的即时 RAG 召回。分享还结合现场实操与问答，深入探讨了多智能体敏捷复盘、记忆修剪防污染、HyDE 高阶原则抽象以及基于合成数据的闭环评测与工程落地实践。 ## Atmosphere & style 整场演讲风格务实、硬核且不失幽默风趣，Jason Davenport 巧妙结合生活化比喻（如孩子做算术的直觉反应、敏捷开发站会）将复杂的记忆系统工程讲解得生动透彻。实操演示流畅直观，台下互动极为活跃，来自各一线大厂工程师的连环追问与痛点碰撞，使现场充满了浓厚的生产级技术探讨氛围与探索前沿的极客热情。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=noVd6-bMcF4&t=4s) 智能体系统不只是单体代码，必须由短期会话记忆、跨任务长期记忆以及领域知识库三大维度协同支撑，以解决信息不对称导致的执行失败问题。 - [03:20](https://www.youtube.com/watch?v=noVd6-bMcF4&t=200s) “做梦（Dreaming）”是智能体的后台记忆收割与提炼流水线，通过离线重放交互轨迹，将高频推理转化为直觉模式并沉淀通用技能。 - [07:25](https://www.youtube.com/watch?v=noVd6-bMcF4&t=445s) 在系统架构预算中，必须为可观测性及负责行为分析与做梦的后台辅助系统预留相当比例的 Token 预算。 - [10:18](https://www.youtube.com/watch?v=noVd6-bMcF4&t=618s) 利用 Token 经济学原则，在后台日志做梦提炼等上下文完全闭环的任务中，应采用低成本轻量模型（如 Gemini Flash-Lite）以兼顾速度与效益。 - [12:37](https://www.youtube.com/watch?v=noVd6-bMcF4&t=757s) 提示词本质上就是代码，在 Cloud Spanner 等持久化存储中可采用自由文本（向量匹配）与强类型键值并行的双模存储架构。 - [14:04](https://www.youtube.com/watch?v=noVd6-bMcF4&t=844s) 通过查询重写与精准 RAG 注入，系统可实现约 300 毫秒级的极速记忆召回，有效追踪长周期用户行为演变。 - [21:51](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1311s) 多智能体团队可采用类似敏捷开发“Sprint 站会回顾（Retro）”的复盘机制，借助离线做梦审视不同执行路径的 Token 消耗与规划优劣。 - [24:36](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1476s) 借鉴数据仓库（SCD 缓慢变化维）与数据湖的分层思维，通过使用模式与时间戳对记忆进行冷热分级与逻辑归档，防止陈旧数据污染上下文窗口。 - [30:41](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1841s) 即使大模型支持超长上下文，无节制填充历史仍会导致高昂成本与延迟，短期工作记忆实时压缩与长期记忆按需检索依然是最佳范式。 - [32:35](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1955s) 避免记忆陷入微观细节的有效手段包括引入 HyDE 多维检索重排，以及运行专门提炼宏观 SOP 准则的泛化做梦管道，甚至直接反哺重构编排代码。 - [36:09](https://www.youtube.com/watch?v=noVd6-bMcF4&t=2169s) 智能体存储架构需解耦：BigQuery 等数仓负责全量海量遥测与后台做梦分析，Cloud Spanner 等数据库承接亚秒级在线记忆点查与服务。 - [41:12](https://www.youtube.com/watch?v=noVd6-bMcF4&t=2472s) 利用合成数据（Synthetic Data）构建高质量验证集，结合 LLM-as-a-Judge 与强化反馈闭环，是低成本评估和驱动 Agent 进化的关键路径。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=noVd6-bMcF4&t=4s) 智能体是否会梦见电子羊：上下文与记忆系统概览 Google Cloud 区域技术主管（Area Technical Lead）Jason Davenport 在 Ai4 大会上登台致谢，并以科幻作家菲利普·K·迪克（Philip K. Dick）的经典名作《仿生人会梦见电子羊吗？》（*Do Androids Dream of Electric Sheep?*）作为引子，展开了关于“AI 智能体是否会做梦”以及如何管理智能体持久记忆的深度探讨。Jason 介绍了自己的日常工作，主要负责协助开发者社区构建覆盖数据、容器、DevOps 以及前沿 AI 的云端工作负载。他强调，在构建智能体系统时，大家通常最关注输入端（用户提示词与上下文注入）、上下文窗口（Context Window）的填充与维持，以及输出端（模型生成的内容是否具有扎实的事实依据即 Grounding、是否切题）。 然而，智能体绝非一段仅仅运行在容器或某处服务器中的独立代码，而是一套协同完成特定任务与结果的复杂系统网络。Jason 将注意力引向智能体架构中至关重要的右半部分——记忆与会话管理系统。他指出，智能体系统必须涵盖三大核心记忆维度：一是短期记忆（Short-term Memory），负责维护会话级上下文的持久性与稳定性，避免会话信息在窗口滑动中丢失；二是长期记忆（Long-term Memory），这是使智能体随着任务积累不断自我进化的“下一个前沿”——例如当智能体重复处理某种任务 10 次后，它应当学会沉淀经验，将单次任务的 Token 消耗从 100 万压缩至 1 万；三是知识库（Knowledge Base），它与长期记忆紧密结合，为智能体提供等同于人类专家解决问题所需的信息。 许多企业常抱怨“智能体没有按预期完成任务”，Jason 往往会反问：“智能体是否拥有与人类员工同等的完整信息访问权限？”在 90% 的情况下，答案是否定的。因此，构建能够被智能体高效索引并利用的记忆系统，是解决该痛点的关键。 ```mermaid flowchart TD accTitle: Agent Context and Memory Architecture subgraph ContextFlow["上下文交互层 (控制面)"] A["用户输入 / 提示词"] --> B["上下文窗口 Context Window"] B --> C["落地输出 Grounded Output"] end subgraph MemorySystems["记忆系统层 (演进面)"] D["短期记忆 / 会话管理<br/>(Session & 短期状态)"] E["长期记忆 / 经验沉淀<br/>(降低 Token / 跨会话学习)"] F["知识库 Knowledge Base<br/>(领域资料检索与索引)"] end ContextFlow <--> MemorySystems ``` - **短期记忆**：确保会话状态持久化（Durable Sessions），防止多轮交互中上下文遗失。 - **长期记忆演进**：通过历史经验将高昂的单任务 Token 消耗（如 100 万 Tokens）优化为轻量调用（如 1 万 Tokens）。 - **信息对齐准则**：智能体任务失败通常归因于知识获取不对称，必须建立高覆盖率的索引体系。 --- ### [03:20](https://www.youtube.com/watch?v=noVd6-bMcF4&t=200s) 什么是智能体的“梦境”机制与技能沉淀 现场调研显示约半数听众已了解智能体“做梦”（Dreams）的概念。Jason 解释道，“梦境”本质上是记忆架构中的一个子系统。就像人类在睡眠做梦时会对白天吸收的信息进行再分析、重新编目、整合甚至主动遗忘无关噪声一样，智能体的“梦境”系统会在后台分析其交互轨迹（Trajectories），据此生成新的持久记忆或对模型策略进行自我迭代与更新。 在实际业务场景中，“梦境”具备多重用途：它可以动态跟踪并更新用户随时间发生变化的偏好；当智能体在运行中出现非预期或次优行为时，“梦境”能够对上下文进行压缩提炼，进而优化系统提示词（Instructions）或作为检索增强生成（RAG）的外部知识库；此外，它还能将特定业务操作模式固化为组织内部的标准规范。 Jason 举了自己 10 岁儿子的生动案例来类比智能体的能力固化：孩子既掌握了死记硬背式的加法（Rote Addition），也掌握了加法运算规则本身。当被问及“2 + 2 等于几”时，孩子能不假思索地脱口而出“4”，无需在脑海中列竖式、写加号、画横线进行繁琐推导；同时他也具备通用加法技能，能够应对更长、更复杂的算术题。在智能体系统中，“做梦”正是用来将高频模式沉淀为直觉知识，并将复杂行为抽象为通用技能。Jason 再次强调，不要试图强求单个智能体包揽所有事务，而应构建由多个专业化智能体协同工作的系统。 - **动态偏好更新**：捕捉用户长周期的习惯演变，修正过时认知。 - **行为校准与压缩**：诊断非理想行为，压缩交互轨迹以反哺 Prompt 指令或 RAG 检索。 - **直觉模式（2+2=4） vs 规则技能**：通过梦境机制将高频推理转变为低成本的快速响应，兼顾效率与通用能力。 --- ### [06:06](https://www.youtube.com/watch?v=noVd6-bMcF4&t=366s) 架构设计：利用 BigQuery 与 Spanner 构建持久化记忆流水线 随后 Jason 展示了支持智能体“做梦”与记忆自演进的整体技术架构。他幽默地调侃如今凭借“Vibe Coding（氛围编程）”可以极其快速地搭建出任何应用的原型。在这套记忆更新流水线中，系统通过多轮提问与用户进行交互，并全程开启遥测监控（Telemetry），捕获包括输入、推理步骤与响应在内的完整轨迹数据。 收集到的遥测轨迹会被全量导入 Google Cloud 的 BigQuery 数据仓库中。随后，系统利用大语言模型（LLM）针对 BigQuery 中的日志执行 Prompt 提取，精准抽取用户的意图、关键属性与行为特征。抽取的结构化记忆随后被沉淀到全球分布式数据库 Cloud Spanner 中。当未来智能体需要处理相关任务或再次与用户接触时，即可直接从 Spanner 中高效召回特定记忆片段，实现无缝的个性化上下文注入。 ```mermaid flowchart LR accTitle: Agent Memory and Dream Pipeline subgraph Ingestion["交互与遥测采集"] A["用户交互 / 智能体"] -->|"遥测数据 / 轨迹"| B["BigQuery 遥测日志库"] end subgraph Synthesis["梦境提取与存储"] B -->|"LLM 提示词批处理分析"| C["意图与记忆提取器"] C -->|"写入结构化记忆"| D["Cloud Spanner (全局数据库)"] end subgraph Recall["记忆召回"] D -->|"检索增强 / 偏好注入"| A end ``` - **全量遥测记录**：将智能体的全部轨迹（Trajectories/Traces）完整记录，为离线分析提供数据基石。 - **离线提炼（梦境处理）**：利用 LLM 批量解析 BigQuery 日志，提取高价值偏好与意图标签。 - **全局持久化存储**：借助 Cloud Spanner 实现跨地域、跨会话的毫秒级记忆检索与复用。 --- ### [07:25](https://www.youtube.com/watch?v=noVd6-bMcF4&t=445s) 实操演示：采访智能体交互与遥测轨迹捕获 进入实操演示环节，Jason 展示了一个分为五个阶段的 Web 应用程序，涵盖偏好提取、遥测采集、记忆沉淀与偏好召回。第一步是一个使用 Google Agent Development Kit（ADK，最新 Release 2 版本）构建的“面试智能体”（Interview Agent），底层模型接入了 Gemini 系列轻量模型（Gemini 3.5 Light）。该智能体主动向 Jason 发起提问，收集他的姓名、职位、擅长的编程语言以及常用的基础设施工具。 Jason 在界面中打趣地回复：“我的名字是 Jason，我是一名工程师；当开发智能体时我喜欢用 Go 语言，但除此以外我都用 Python——因为我读得懂 Go，但我写不来。”这句自嘲引发了台下观众的会心一笑。接着，针对数据库与云工具的提问，Jason 录入了“我重度使用 Spanner 和 BigQuery”。他指出，在没有外挂记忆系统的情况下，单体智能体一旦结束会话便会彻底遗忘这些信息，因此第一步必须将对话流写入日志系统。 通过 ADK 的内置插件，所有交互轨迹被自动传输至专用的 BigQuery 实例中，记录了详细的 Trace 链路、用户提示词及智能体响应表。Jason 特别提醒在场架构师与工程师：在规划智能体工作流预算时，**务必为专门观测与分析智能体行为的辅助系统预留相当可观的 Token 预算**，因为这些后台分析与“梦境”系统正是保障智能体长期稳定运行与智能进化的核心动力。 - **开发框架与模型**：基于 Google Agent Development Kit (ADK) Release 2 与 Gemini 模型构建。 - **轨迹监控机制**：利用 ADK 插件无侵入式地将全量链路 Trace 与 Prompt/Response 写入 BigQuery。 - **Token 预算规划**：强调必须将相当比例的 Token 预算分配给“可观测性与后台记忆分析系统”，而非仅仅计算前端生成成本。 ### [10:18](https://www.youtube.com/watch?v=noVd6-bMcF4&t=618s) 使用 BigQuery 与轻量模型进行“做梦”与记忆提取 在长达 15 到 20 分钟的长程对话中，系统可能会产生多达数百万个 Token 的交互信息。演讲者强调千万不要把这些宝贵的数据白白扔掉，必须将其完整记录下来，因为这些日志正是未来决定组织或 Agent 能否取得成功的核心资产。在展示的实现中，系统通过一个极其简单的 Python 类捕获用户交互，并以结构化格式转储至 BigQuery，以便后续加工处理。 进入“做梦（Dreaming）”与记忆收割（Memory Harvesting）的核心环节，BigQuery 展现出了独特的架构优势：它原生集成了多种大语言模型（如 Google 自身的 Gemini，或 Vertex AI Model Garden 中的各种模型），允许开发者直接在日常编写的 SQL 查询中嵌入 AI 能力来提取用户偏好。这种方式极大地降低了门槛，使用户能够轻松对自己的历史数据进行分析并构建专属的记忆模型。在模型选型上，正如前一位演讲者提到的 Token 经济学（Tokenomics），对于总结和提取这类信息已经完全包含在上下文窗口中的任务，根本不需要动用 Opus、Pro 等昂贵的大模型，直接选用响应快且成本极低的 Gemini Flash-Lite 即可，在合理的地方保持低成本运作。 ```mermaid flowchart LR accTitle: BigQuery 驱动的记忆收割流程 subgraph LogCollection["对话日志收集"] A["长程交互会话<br/>(数百万 Tokens)"] --> B["Python 结构化日志"] B --> C[("BigQuery 日志存储")] end subgraph Harvesting["记忆收割 (Dream Harvesting)"] C --> D["SQL + 内嵌 AI 查询<br/>(调用 Gemini Flash-Lite)"] D --> E[("Cloud Spanner<br/>持久化记忆库")] end ``` - **单次提取成效**：演示中的收割批处理执行完毕后，一次性从历史日志中自动提炼出 7 条结构化记忆。 - **Token 经济学考量**：对于总结类任务，无需使用昂贵的前沿旗舰大模型，选择 Flash-Lite 即可兼顾性能与成本。 --- ### [12:37](https://www.youtube.com/watch?v=noVd6-bMcF4&t=757s) 提示词即代码与 Cloud Spanner 结构化记忆存储 演讲者展示了一个相对简洁的提示词模板，并指出：“归根结底，提示词虽然是英语，但它同时也是代码。”演示展示的代码构成了整个系统的骨架，而开发者根据自身业务目标想要从 Agent 行为中提取的特定维度，才是每个团队独特的“独门秘方（Secret Sauce）”。这种做梦收割机制既可以针对单次会话运行，也可以跨多个历史会话批量执行，核心在于明确做梦收割的业务价值以及未来如何赋能给下游 Agent。 随后，演讲者刷新了 Google Cloud Spanner 数据库以查看提取到的偏好数据。在 Spanner 数据表中，清晰地展示了多组键值对，例如用户的常用编程语言、工作角色、姓名、首选基础设施及开发工具。对于这类记忆存储，设计上可以分为两类模式：一种是自由文本（Free-form），便于后续在向量空间中进行语义模糊匹配；另一种则是强类型键（Strongly Typed Keys），用于明确记录用户或业务流程的确定性属性。在 Cloud Spanner 的同一张表中，可以轻松兼容这两种存储方式。 - **存储字段示例**：用户姓名、岗位角色、偏好编程语言、首选基础设施架构等。 - **双模存储架构**：支持用于向量相似度检索的自由文本字段，以及用于精确状态控制的强类型键值字段。 --- ### [14:04](https://www.youtube.com/watch?v=noVd6-bMcF4&t=844s) 实时召回演示：查询重写、300 毫秒极速检索与自我进化 将收割到的记忆重新应用回 Agent 系统主要有两种路径：第一种是直接根据“梦境”中暴露出的不良行为去修改 Agent 的系统指令或内部循环逻辑；第二种则是更为实用但常被低估的方式——借助 RAG（检索增强生成）理念在运行时动态注入相关记忆。演讲者现场演示了第二个专门用于交互的 Agent：当接收到用户提问时，该 Agent 首先会对问题进行查询重写（Query Rewriting），使其更精准地命中向量空间中的记忆，随后将检索结果注入上下文并生成回答。 在现场互动中，系统提示用户提问“问问我最喜欢的东西是什么”，演讲者调侃道：“你应该早就知道了对吧？你自己告诉我。”结果 Agent 几乎瞬间输出了答案，准确列出了演讲者的姓名、偏好基础设施、常用工具以及工作风格。整个数据库检索与记忆返回耗时仅约 300 毫秒，充分印证了在长程交互中 Flash-Lite 模型的高响应速度优势。当演讲者进一步追问“你还知道什么？”时，Agent 迅速翻查了以往测试会话沉淀的更早记忆。这种持续收割、定期压缩并追踪用户行为演变的设计范式，为构建稳定耐用的长程运行系统提供了标准化模板。 --- ### [16:54](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1014s) 架构总结与 Agent 进化的系统化思维 演讲者对整体架构进行了回顾与串联：首先利用 Google Agent Development Kit 构建产生交互数据的 Agent，随后利用 BigQuery 和 Agent 平台能力搭建“做梦系统”完成洞察提炼，最后将记忆集中持久化在 Google Cloud Spanner 中，使 Agent 所需的记忆数据与业务数据实现同构协同存储。 演讲者在屏幕上展示了代码仓库的二维码，并坦诚说明部署该示例由于包含 Cloud Spanner 实例大约需要 60 美元成本，但开发者完全可以让大语言模型将其改写并接入任何自选数据库。在总结 Agent 技术的演进脉络时，演讲者指出，记忆管理是当下改善长周期及短周期 Agent 表现最至关重要的抓手；借助“做梦”机制，Agent 能够在人类监督反馈（Human-in-the-loop）下实现自主的闭环自我进化。正如他所强调的：“这不仅仅关乎底层模型本身，更关乎我们围绕模型所构建的整个系统工程，只有系统才能驱动 Agent 真正达成我们预期的目标。” --- ### [18:43](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1123s) 现场问答：厘清“做梦”与“记忆”的关系及大规模知识冷启动 在问答环节，现场观众提出了深入的机制探讨： 首先，有观众询问“做梦收割（Dream Harvesting）”与日常在 ChatGPT 等编码助手中直接说“把这存入记忆”是否有本质区别？演讲者解释道，虽然人们常说不要把 AI 拟人化，但人类心智模型确实有助于理解这些概念：“可以将‘做梦’理解为生成与更新记忆的数据处理流水线（Data Pipeline），而‘记忆’则是这条流水线产出的最终数据制品。”单条命令式的保存只是单纯存储，而做梦是一个结构化萃取、提炼甚至修改旧记忆的后台异步处理过程。 随后，另一位观众就知识初始化提出了冷启动问题：演示中是通过一段实时聊天对话生成初始记忆，但如果企业面对的是包含大量文档、企业知识库的语料体系（Corpus），该如何利用这种机制进行大规模解析、构建键值对并持续迭代演进？该问题为下一阶段企业级多源知识的大规模记忆沉淀拉开了讨论序幕。 ### [20:31](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1231s) 从遥测日志与知识库起步构建智能体系统 在被问及如何真正迈出第一步、从 15 分钟的演示 Demo 走向生产级的全 Agent 系统时，Jason Davenport 指出首要任务就是记录所有的遥测数据（Telemetry）。无论是测试会话还是对抗性测试（Adversarial Testing），团队必须拥有至少 50 个会话记录来进行横向对比与分析，观察不同会话之间的具体差异。对于存放 Agent 必备常识的知识库，开发者必须重视“重写问题”（Rewriting the Question）的机制。Jason 分享了自己早期构建系统的踩坑经历：在向系统查询偏好语言时，由于输入问题与底层存储的 Embedding 在语义空间存在偏差，导致了“语义零匹配”（Zero Match）。 为了解决这类问题，必须持续监控检索命中率。当完整的会话遥测数据被持久化记录后，开发者可以直接将 5 组会话打包投递给 LLM，让大模型自主分析在特定向量或特定场景下哪些策略奏效、哪些失败，甚至结合 Token 消耗成本进行归因分析。这种通过日志驱动自我复盘的模式，是冷启动 Agent 系统最直接有效的方法。 ```mermaid flowchart LR accTitle: 遥测日志驱动的 Agent 优化流程 subgraph Logging["遥测采集"] A["Agent 对抗/测试会话 (50+ Sessions)"] B["Telemetry 日志归档"] end subgraph LLMAnalysis["LLM 复盘与评估"] C["LLM 多会话横向对比"] D["Token 成本与命中率分析"] end subgraph Feedback["策略迭代"] E["重写检索 Prompt / 知识库对齐"] F["行为与指令优化"] end A --> B --> C --> D --> E --> F ``` - **全量遥测记录**：将所有 Agent 交互会话持久化存储，积累可供横向比对的数据样本池。 - **语义空间对齐**：通过重写用户 Prompt 避免语义偏差导致的向量检索零命中。 - **LLM 自主诊断**：利用大模型横向比对多组会话日志，自动化评估成本开销与成功路径。 --- ### [21:51](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1311s) 多智能体团队与“敏捷回顾”式梦境收割 来自 Align Technology 的 Rachel Manis 提问：当企业采用类似于软件开发“黑灯工厂”（Dark Factory）的模式，让工程师管理完全由 Jira 工单和 PR 驱动、没有任何多余上下文偏置的 Agent 团队时，梦境收割（Dream Harvesting）和记忆机制该如何协同运作？ Jason Davenport 用敏捷开发中的“Sprint 站会回顾”（Sprint Retro）做类比：梦境机制就像是让一个子系统在多个 Agent 协同完成某项特定目标后，通盘审视整个执行链路，分析为什么某种路径能达成理想目标，而另一种路径却失败了。例如，在完成同等复杂度的任务时，为什么一组 Agent 消耗了 50 万 Token，而另一组却消耗了 200 万 Token？复盘结果可能会发现，正是负责统筹规划的“导演 Agent”（Director Agent）在两组实验中给出的初始计划质量差异巨大。 在多 Agent 团队中落地梦境机制时，开发者可以先利用 LLM 生成用于优化团队协作的初始 Prompt，进而调整指令或修剪各个 Agent 之间过度传递的上下文信息。梦境与评测（Eval）紧密结合：系统可以将表现极佳的“黄金会话”（Golden Session）作为评测基准，修改策略后让 Agent 团队重新执行相同任务，通过类似 A/B 实验的持续迭代，推动平台自主进化。 - **Sprint Retro 式复盘**：利用离线梦境审视多 Agent 协作过程中的路径差异与规划漏洞。 - **Token 效率归因**：排查极端消耗（如 50 万 vs 200 万 Token）背后的决策树与上下文冗余。 - **黄金会话回放**：以成功轨迹为 Benchmark 进行仿真重放，闭环验证指令改进效果。 --- ### [24:36](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1476s) 记忆修剪策略与数据仓库设计理念 针对现场关于“何时以及如何修剪（Prune）过时或不再相关的梦境记忆”的提问，Jason 指出这本质上更偏向业务定义而非纯技术难题。人类自身的记忆也会随着时间推移遗忘低频信息，Agent 系统同样没有绝对的对错标准。在执行具体任务检索时，评测系统应当确保始终优先弹出相关度最高的信息，同时可以引入最后访问时间戳（DateTime）进行排序。 ```mermaid flowchart TD accTitle: 记忆生命周期与归档判定流程 A["Agent 梦境记忆库"] --> B{"近期是否有访问？ (如 6 个月内)"} B -- "高频访问" --> C["保持活跃并参与 RAG 检索"] B -- "长期未访问" --> D{"该类型问题是否重复出现？"} D -- "从未出现" --> E["继续保留 (低检索权重)"] D -- "频繁出现但已废弃" --> F["标记为 Inactive / 归档出 Context Window"] F --> G["定期冷存储清理"] ``` 在 Agent 架构中，数据并不一定需要物理删除，核心在于“防止陈旧数据污染（Pollute/Taint）上下文窗口”。如果某条记忆在过去 6 个月内从未被检索，系统需要判断：是业务上从未遇到该场景，还是遇到相同场景时该记忆已不再适用？若是后者，则应将其状态标记为“非活动”（Inactive）或移入归档区。 Jason 强调，构建 Agent 梦境系统在底层逻辑上与数据工程中的“数据仓库”（Data Warehousing）和“数据湖”（Data Lake）如出一辙，就像处理缓慢变化维（Slowly Changing Dimensions, SCD）一样，记忆生命周期的管理本质上是基于业务事件与使用模式驱动的冷热分级。 - **防上下文污染**：区分物理删除与逻辑停用，避免低质陈旧记忆占用有限的 Context Window。 - **访问模式判定**：结合时间窗口与问题重现率，动态归档过期策略。 - **数仓思维映射**：借用 SCD 维度建模与冷热数据分层思想管理 Agent 记忆生命周期。 --- ### [26:52](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1612s) 谷歌内部的 Agent 轨迹分析与群体学习模型 在被问及谷歌内部如何维护梦境的长期有效性并分发给工程师团队时，Jason 透露谷歌内部具备专门的系统来追踪和记录开发者的 Agent 代码生成轨迹（Agent Trajectories）。工程师既可以分析个人的交互轨迹，也可以采用“群体模型”（Cohort Model）进行群体行为洞察。 Jason 将群体模型形象地比喻为“班级教学”：通过观察整个开发群体的行为，分析究竟是整体工作流存在系统性缺陷，还是个别“刺头”环节引发了异常，进而提炼出符合预期的最佳实践行为模式。谷歌内部的底层系统在结构上与本次分享展示的梦境架构高度相似——均是在离线阶段深度分析轨迹，提炼成经验后再按需重新注入（Reinject）到 Agent 中。 系统最终的胜负手依然落在检索增强生成（RAG）上。当使用现成的固定执行框架（Harness）而无法修改底层子系统时，关键就在于如何设计精准的 RAG 机制，在不耗尽上下文 Token 配额的前提下，将最相关、最关键的梦境记忆即时召回。 - **轨迹洞察系统**：捕获代码开发中的 Agent 行动轨迹，支持单人回溯与群体对比。 - **班级群体模型（Cohort Model）**：排查孤立异常与普遍性流程瓶颈，沉淀全局最佳模式。 - **动态经验重注入**：离线提取洞察，在线通过精准 RAG 注入运行时，严格把控 Token 预算。 --- ### [28:41](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1721s) 梦境上下文抽取跨度与长期记忆本质 关于“在初始化提取梦境数据时需要回溯多长时间的上下文”这一问题，Jason 表示这完全取决于提取目标。他以与配偶的日常沟通为例：“如果我和妻子聊天，有些刚讨论过的极其重要的事情我必须永远记住，这种情况下哪怕对话窗口只有 3 分钟，也需要立即固化为记忆”；而在另一种场景下，若要学习某个随时间演变的复杂业务流程，抽取系统可能需要汇总分析数周的数据，提炼出更高阶的聚合认知（Aggregate Learning）。 梦境系统的本质就是根据 Agent 的核心目标进行动态适配：无论是以长期行为演进为目标，还是以突发触发事件（Triggering Incident）为警示，梦境负责将散落的数据聚合成具备指导意义的知识并反哺 Agent。在回答人类长短期记忆与 Agent 记忆机制的对应关系时，Jason 明确指出：梦境系统在绝大多数场景下承担的都是生成和沉淀“跨周期的长期记忆”（Long-term Memories）的职责。 - **情境敏感的时间跨度**：即时关键指令可在 3 分钟内固化，宏观流程沉淀需汇总数周上下文。 - **事件驱动与演进聚合**：既能针对异常 Incident 形成避坑策略，也能针对通用模式形成全局知识。 - **定位长期记忆生成器**：短期上下文由运行时会话承载，梦境负责沉淀持久化、跨任务的长期认知。 ### [30:41](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1841s) 长短期记忆的分工与“做梦”系统的动态更新机制 在多智能体系统（Multi-agent System）的设计中，短期记忆通常被定位为“工作记忆”（Working Memory）。例如，当 Agent 正在执行一项具体的编程任务时，它需要实时在脑海中保留当前关键的上下文信息，以协调并管理两到三个并行的子流程。在这类实时交互场景下，可以通过内置的压缩系统（Compaction System）对信息进行实时摘要与缩减；而在任务离线或后台阶段，则通过“做梦”（Dream）系统来进行全局视角的整理与更新。 ```mermaid flowchart TD accTitle: Agent 记忆流转与做梦更新流程 subgraph Execution ["实时执行阶段"] LTM_Read["读取长期记忆 (Long-term Memory)"] --> Task["执行具体任务 / 编码"] Task <--> STM["读写短期工作记忆 (Short-term Memory)"] STM --> Compaction["实时上下文压缩 (Compaction)"] end subgraph OfflineDream ["做梦与沉淀阶段 (Dreaming)"] Task --> Log["记录完整轨迹与结果"] Log --> Dream["做梦系统分析：<br/>提取经验、评估收益"] Dream --> LTM_Write["更新/固化长期记忆 (Long-term Memory)"] end ``` 这两类记忆机制本质上是相辅相成的，核心在于开发者希望驱动 Agent 产生何种行为： - **长期记忆（Long-term Memory）**：在任务启动时最先被检索调取，用于回答“关于该用户、流程或系统我已经掌握了哪些背景知识”，进而判断这些知识如何辅助当前任务。 - **短期记忆（Short-term Memory）**：在任务执行过程中保持活跃状态，支持多步骤推理与即时决策。 - **做梦系统（Dream System）**：回顾整个任务轨迹中调用的长期记忆、生成或变更的短期记忆以及最终的执行结果，评估得失并反哺更新长期记忆库。 即使如今以 Google 为代表的前沿大模型已经支持高达 100 万甚至更高 Token 的超长上下文窗口，无节制地将所有历史塞入上下文依然面临两大挑战：一是推理成本极其昂贵，二是长链条运行会导致延迟过高。因此，通过 RAG（检索增强生成）技术按需向上下文窗口注入记忆，并将复杂任务拆解为多 Agent 团队各自维护专属上下文，依然是优化智能体行为和控制成本的核心范式。 --- ### [32:35](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1955s) 提炼高阶“品味”与原则：避免记忆陷入过度狭隘的具体细节 在现场问答环节，一位来自 Google YouTube Ads 团队的工程师提出了一个普遍存在的痛点：在构建辅助产品经理（PM）进行实验决策的数据科学 AI 时，系统往往记住了 Senior PM 极其狭隘、战术性的具体偏好（例如“当询问某种收入数据时采取 X 方式处理”），却无法抽象出高层次的战略“品味”（Taste）和通用原则。即便是 Claude 3.5 Pro 等顶级模型，在从微观细节提炼宏观偏好方面也往往表现不佳。 针对如何让记忆更具通用性而非过度微观化，Jason Davenport 从检索和记忆提取管道两个维度给出了应对策略： 1. **改进检索与提问方式（Query Rewriting）**： - 传统的单次向量检索容易因提问用词过于具体，直接命中相似度高达 95% 的狭隘片段。 - 建议引入 BM25 混合检索以及 **HyDE（Hypothetical Document Embeddings，假设性文档嵌入）** 技术：针对原始输入，在 Embedding 空间中生成 5 个语义不同但意图相关的平行问题，召回多维度候选结果后再通过重排模型（Re-ranker）筛选。 2. **构建多层级的做梦管道（Multi-pipeline Dreams）**： - “做梦”不应只有单一流程，而是可以并行运行多个处理管道。除了抓取具体细节的微观管道外，应建立专门总结宏观规律的泛化管道（分析“在整段轨迹中哪些做法奏效、哪些失败”）。 - 泛化提炼出的 5 条核心准则可以直接固化为 Agent 的标准操作流程（SOP），甚至直接沉淀为**编排代码（Orchestration Code）**。 正如 Jason Davenport 所强调的，优化 Agent 绝不仅仅局限于修改 Prompt，更可以通过做梦洞察直接重构代码循环（Loop）：“有时你需要实际修改 Agent 的 Python 调度代码，强制执行工作流约束——先做 A，再做 B，然后再自由发挥。” --- ### [36:09](https://www.youtube.com/watch?v=noVd6-bMcF4&t=2169s) 记忆基础设施选型：遥测数据仓库与高并发检索库的权衡 针对观众关于短期与长期记忆存储架构的提问，Jason 分享了 Google 内部及企业级实践中的典型技术选型方案。在演示架构中，系统将日志遥测（Telemetry）与在线检索存储进行了明确解耦： | 存储组件 | 典型技术选型 | 核心定位与适用场景 | |:--- |:--- |:--- | | **遥测与离线分析仓库 (Telemetry & Analytics)** | Google BigQuery | 存储海量会话日志、Token 消耗、交互轨迹等全量数据；运行后台“做梦”分析与批量知识抽取。不适合在线实时召回（查询延迟难以稳定在 300ms 以内）。 | | **在线记忆检索库 (Memory & Retrieval)** | Google Cloud Spanner / PostgreSQL (Cloud SQL) | 存储结构化长期记忆、向量索引及知识库，兼顾强一致性、复杂查询能力与亚秒级响应。Spanner 功能全面但成本较高。 | | **极速工作记忆 (Working Memory)** | Redis / Firestore / NoSQL | 适用于 Agent 高频读写、快速迭代的短期运行状态；Redis 具备极致的并发性能，Firestore 则提供轻量便捷的文档查询。 | 在选型时，团队必须基于业务 SLA（服务等级协议）进行考量。尽管现代数据仓库一直在降低查询延迟，但要在实时 Agent 交互中实现 300 毫秒以内的快速响应，必须为 Agent 构建专门的数据服务层，避免直接用分析型数仓承接在线点查。 --- ### [38:47](https://www.youtube.com/watch?v=noVd6-bMcF4&t=2327s) 验证集构建与闭环评估智能体（Validation & Eval Agents） 在讨论 Agent 系统的进化（Evolve）与自我迭代时，Jason 指出当前整个行业面临的最大瓶颈是**数据标注（Data Labeling）的缺失**。许多团队积累了数十次 Agent 运行会话，却从未投入精力进行系统化的评级与归因分析（例如“在 15 次会话中，明确标记出哪 3 次效果极佳、哪 12 次表现糟糕，并写明具体原因”）。 无论是人工标注还是借助 LLM 辅助初评，建立高质量的**验证集（Validation Set）**是 Agent 进化的先决条件。验证与评估智能体的主要运作机制包括： - **故障模式挖掘**：利用辅助子系统排查明显的异常调用（例如单次任务中意外向 MCP 服务器发起上百次冗余请求），同时捕捉由于记忆引入带来的细微行为偏移。 - **强化反馈闭环**：记忆与技能的沉淀本质上遵循强化学习（Reinforcement Learning）的逻辑。系统在注入新记忆或修改 Agent 行为后，会使用验证集中的相同 Prompt 重新运行基准测试。 - **回归与增益测试**：观察 Agent 团队的执行轨迹（Trajectory）是变好了还是变差了。若带来正向收益则将该记忆持久化沉淀；若引发下游意外异常，则果断回滚或废弃该经验，重新构建更优解。 ### [41:12](https://www.youtube.com/watch?v=noVd6-bMcF4&t=2472s) Agent 评估方案选择：Reward Model、LLM-as-a-Judge 与合成数据验证 关于评估 Agent 输出的技术路线，现场讨论了两种常见趋势：一种是训练专门的奖励模型（Reward Model）为 Agent 的每次生成打分；另一种则是当前非常流行的方式——利用大语言模型充当裁判（LLM-as-a-Judge）进行评估。针对究竟该选用哪种方案或是否应该结合两者的疑问，演讲者指出，具体选择完全取决于实际的业务场景与应用需求。 演讲者分享了自己的背景与经验，他最初是数据工程师出身，而在过去 12 个月里，他投入在合成数据（Synthetic Data）上的精力比以往任何时候都要多。如今开发者完全可以通过大模型合成生成高质量的验证集（Validation Sets）。因此，在评估方案的选择上目前并不存在绝对的“对与错”，核心在于找到一种最简便、且“不会在时间和资源上打破预算（without breaking the bank in both time and resources）”的落地方式。 在工程实践中，建议团队根据具体的业务目标挑选一到两种方法进行尝试，随后评估效果并根据需要采用混合方案。例如在 Google 内部，拥有多套不同的评估与评测平台已是常态，因为不同的业务所衡量的核心指标各不相同，各自优化的用户体验也千差万别，导致最终采用的 Eval 和验证体系形态各异，而这种多元化在实际研发中是完全合理且必要的。最后，演讲者向现场听众表达了感谢，圆满结束了问答环节与整场分享。 - **用例驱动评估**：Reward Model、LLM-as-a-Judge 以及基于规则的打分各有优劣，选型需紧密结合具体业务用例与资源预算。 - **合成验证集（Synthetic Validation Sets）**：利用大模型生成合成数据构建测试验证集，已成为近一年来极具性价比且高效的评估手段。 - **多元化 Eval 平台共存**：针对不同的优化目标与体验场景，团队应当灵活组合多种评测工具与标准，无需拘泥于单一固化的评测体系。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=noVd6-bMcF4&t=4s) 智能体是否拥有与人类员工同等的完整信息访问权限？ — Jason Davenport > [07:25](https://www.youtube.com/watch?v=noVd6-bMcF4&t=445s) 务必为专门观测与分析智能体行为的辅助系统预留相当可观的 Token 预算，因为这些后台分析与“梦境”系统正是保障智能体长期稳定运行与智能进化的核心动力。 — Jason Davenport > [07:25](https://www.youtube.com/watch?v=noVd6-bMcF4&t=445s) 我的名字是 Jason，我是一名工程师；当开发智能体时我喜欢用 Go 语言，但除此以外我都用 Python——因为我读得懂 Go，但我写不来。 — Jason Davenport > [12:37](https://www.youtube.com/watch?v=noVd6-bMcF4&t=757s) 归根结底，提示词虽然是英语，但它同时也是代码。 — Jason Davenport > [16:54](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1014s) 这不仅仅关乎底层模型本身，更关乎我们围绕模型所构建的整个系统工程，只有系统才能驱动 Agent 真正达成我们预期的目标。 — Jason Davenport > [18:43](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1123s) 可以将‘做梦’理解为生成与更新记忆的数据处理流水线（Data Pipeline），而‘记忆’则是这条流水线产出的最终数据制品。 — Jason Davenport > [28:41](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1721s) 如果我和妻子聊天，有些刚讨论过的极其重要的事情我必须永远记住，这种情况下哪怕对话窗口只有 3 分钟，也需要立即固化为记忆。 — Jason Davenport > [32:35](https://www.youtube.com/watch?v=noVd6-bMcF4&t=1955s) 有时你需要实际修改 Agent 的 Python 调度代码，强制执行工作流约束——先做 A，再做 B，然后再自由发挥。 — Jason Davenport > [41:12](https://www.youtube.com/watch?v=noVd6-bMcF4&t=2472s) 核心在于找到一种最简便、且“不会在时间和资源上打破预算”的落地方式。 — Jason Davenport ## Mentioned Jason Davenport, Google, Google Cloud, Philip K. Dick, 《仿生人会梦见电子羊吗？》（Do Androids Dream of Electric Sheep?）, Ai4, BigQuery, Cloud Spanner, Google Agent Development Kit (ADK), Gemini, Gemini Flash-Lite, Vertex AI Model Garden, Claude 3.5 Pro, Rachel Manis, Align Technology, YouTube Ads, Redis, Firestore, PostgreSQL, HyDE (Hypothetical Document Embeddings), MCP ## Video description Jason Davenport - Area Technical Lead, Google The systems around agents make them smarter and better equipped, with systems like memory and session management. Learn how to build an AI system that self-improves by generating, storing, and improving agent 'memories. You'll learn how to create dreams and use systems of agents to autonomously improve your agent fleet. [Source](https://www.youtube.com/watch?v=noVd6-bMcF4)

[0:04] Jason Davenport: 智能体是否拥有与人类员工同等的完整信息访问权限？ [7:25] Jason Davenport: 务必为专门观测与分析智能体行为的辅助系统预留相当可观的 Token 预算，因为这些后台分析与“梦境”系统正是保障智能体长期稳定运行与智能进化的核心动力。 [7:25] Jason Davenport: 我的名字是 Jason，我是一名工程师；当开发智能体时我喜欢用 Go 语言，但除此以外我都用 Python——因为我读得懂 Go，但我写不来。 [12:37] Jason Davenport: 归根结底，提示词虽然是英语，但它同时也是代码。 [16:54] Jason Davenport: 这不仅仅关乎底层模型本身，更关乎我们围绕模型所构建的整个系统工程，只有系统才能驱动 Agent 真正达成我们预期的目标。 [18:43] Jason Davenport: 可以将‘做梦’理解为生成与更新记忆的数据处理流水线（Data Pipeline），而‘记忆’则是这条流水线产出的最终数据制品。 [28:41] Jason Davenport: 如果我和妻子聊天，有些刚讨论过的极其重要的事情我必须永远记住，这种情况下哪怕对话窗口只有 3 分钟，也需要立即固化为记忆。 [32:35] Jason Davenport: 有时你需要实际修改 Agent 的 Python 调度代码，强制执行工作流约束——先做 A，再做 B，然后再自由发挥。 [41:12] Jason Davenport: 核心在于找到一种最简便、且“不会在时间和资源上打破预算”的落地方式。

[1:00] Jason Davenport: 90% of the time that answer is usually no [6:06] Jason Davenport: you can vibe code the crap out of anything these days [7:21] Jason Davenport: My name is Jason. I'm an engineer and I like Go when I use agents, but otherwise I use Python because I can read Go, but I cannot write it [10:18] Jason Davenport: Do not let that information go to waste and just throw it away. Make sure you're logging this stuff because this is ultimately what will make your organization or your agent successful in the future. [11:01] Jason Davenport: It's a summarization activity all the information is in the context window that it needs let's keep it cheap... where it makes sense to do it. [14:04] Jason Davenport: Ask me what your favorite things are. You should know this, right? You tell me [14:04] Jason Davenport: What else do you know? [18:38] save this to memory [32:35] when they ask about revenue in this specific way, do X [32:35] Jason Davenport: Over this entire sequence, what worked well and what didn't? [41:12] Jason Davenport: breaking the bank in both time and resources [41:12] Jason Davenport: and that's okay.

[0:04] Check. Check. There we go. All right. Uh thank  you so much uh everyone for attending today and a [0:09] big thank you to AI4 and all the sponsors for the  opportunity uh to talk here. So uh how many people [0:17] have ever read a novel by Philip Dick? Good show  of hands. uh writes a number of interesting um [0:24] you know science fiction nos and kind of not  dystopian but other types of things with that. So [0:31] I thought here you know let's let's frame our  discussion today um for AI agents and dreaming and [0:36] one of the the things that he asks is do robots  dream of of of electric sheep in that. So uh [0:44] nice to meet you all. My name is Jason Davenport.  I'm an area technical lead at Google Cloud. I work [0:48] with a number of developer communities on both  AI but then just you know how we do workloads [0:53] in the cloud from data to um containers to  to devops and all those fun things with it. [1:00] And today uh the premise that we'll show here in  a couple minutes is how we think about context [1:05] management uh in agentic systems. So here  I have just a couple of things that we talk [1:10] about. Um first off if you think of the things  on the left so these are really things if you've [1:15] built an agent before you know you're controlling  right? So you're thinking about, you know, what is [1:19] the user inputting to my to my agent as context.  How does that then get into a context window that [1:25] the agent is using and maybe filling over time uh  with different pieces of information or knowledge? [1:30] And then the output. So hey, what is the  agent giving you back? Is it something that [1:34] is grounded and relevant to the user's uh prompt  or question or maybe the task that was in it? [1:40] And then we have other systems, right? So,  um, agents are not actually just a piece of [1:44] code that we run, you know, in a container or  somewhere else. Uh, they're really a set of of [1:49] systems that we use in order to do, uh, tasks and  accomplish those outcomes. So, where I spend a lot [1:55] of my time thinking about things is, you know, how  do we do short-term memory in agents? So, how are [2:00] we managing, you know, session information over  time? Um, making sure that sessions are durable, [2:05] you know, that we're not, you know, losing, you  know, information in a context window for it. [2:09] Next is long-term memory. So, how are we  thinking about things that agents should [2:13] be learning over time from the tasks that they've  seen or, you know, the prompts or responses that [2:18] they've gotten? And if you think of, you know,  how we actually make agents better over time, [2:24] you know, long-term memory really is kind of that  that next frontier, right, of of how we can, you [2:29] know, do something where an agent's seen a task 10  times. Well, great. Don't use a million tokens per [2:34] task. Now that you've seen it, you should be able  to do that in 10,000. Uh, just as one example. [2:39] And then last but not least, you know, long-term  memory and knowledge bases really go hand in hand. [2:44] So how do agents have access to the same amount  of information that we would do in order to solve [2:48] a problem? You know, one of the things that  I actually hear a lot from organizations that [2:53] I talk to is, well, hey, the agent didn't do  what we wanted it to do, right? Everyone's had [2:57] that example. Well, my first question is, did the  agent have access to what you would have access to [3:03] in order to actually, you know, accomplish that  task? 90% of the time that answer is usually no. [3:09] But it is solvable, right? And so it's how do  we get a all the information into a place that [3:14] can be indexed and then used by our agents  to be successful. All right. So let's spend [3:20] most of our time today uh focusing on the  right side here which is the memory systems [3:25] and specifically let's talk about dreams. So  uh just quick show hands. How many people here [3:30] have heard about dreams and agents? Perfect. So  about half the room. Uh so we'll do a quick uh [3:36] quick definition here and and why it's useful and  then we'll get in and uh look at a demo for it. [3:42] So uh dreams are essentially a subsystem uh within  memory where what we do is we go through and we [3:48] analyze all of the information that we get from  an agent and then we create memories or we update [3:55] our agent based on the dreams and the things  that we learned. So not unlike human behavior, [4:00] right? When we dream, we're actually indexing all  the information uh that we've processed during the [4:04] day and we're recataloging it and sometimes we're  forgetting you know some something else that we [4:09] had. Uh dreams and agents work very similarly. So  you know sometimes in the simplest example is you [4:15] know maybe user states of preference. So we can  use a memory system to do that but maybe there [4:20] are preferences that change over time where we use  dreams to update those preferences. Another is, [4:25] you know, when agents undertake behaviors  that we didn't expect or that were non ideal, [4:29] uh we can use dreams to essentially compact uh  some of our our different contexts and then, you [4:34] know, use that to either improve the instructions  of our agent or to give that information to the [4:39] agent in the future in a form of of rag or  retrieval augment generation. And then last is [4:45] to make sure that you know as we see different  behaviors, maybe we want to have a specific [4:49] way that we we accomplish that as a part of our  organization or a company. And the most kind of [4:56] relevant example to me that I think about is  you I have a I have a 10-year-old at home. [5:00] Um he learned wrote addition, but he also knows  addition. So wrote addition here is if you ask [5:06] him what 2 plus two is, he just knows that the  answer is four. You know, he's not thinking about, [5:11] you know, hey, I'm going to do these pieces of  math and I'm going to, you know, put the two [5:14] on the top and then a two on the bottom and put a  plus sign next to it and, you know, draw the line. [5:19] He just knows, right? 2+ 2 equals 4. He also knows  how to do addition though which is essentially a [5:24] subsystem your or a behavior or a skill that  he can go and then you know do on on longer or [5:30] more complex things for it and these are things  you know if you think about dreams right we can [5:34] use this to actually design agent behavior uh  in the ways that we want to accomplish for it [5:40] and here the other thing that I'll just call out  is that ultimately if you think about agents right [5:45] we're building systems of agents that actually  accomplish a task so if we're trying to do all of [5:50] this with one agent Right? Probably likely that  that agent may be a specialist in one thing but [5:54] probably can't accomplish you know other things  that we have and here it's about the system and [5:59] building a system that works for us not just a  single agent for it. [snorts] All right I have [6:06] this here um just in terms of how we show we do  these but I also actually put this into the app [6:11] because you can v code the crap out of anything  these days which is super fun. Um but just here [6:17] you know what are we going to do? So, we're going  to have an agent where we're going to essentially [6:21] ask a couple questions. Uh, we've enabled  telemetry on that agent so we can get all the [6:26] trajectories and the other pieces of information  for it. Um, using that information uh that we'll [6:31] be loading into BigQuery, we can actually use LLM  prompts to extract information you about how the [6:37] LM worked and we can use that to get intents or  maybe to extract specific things that we want to [6:42] store. And then we can load that into in this case  a spanner instance which is just a global database [6:48] that then our agent will be able to retrieve when  we have specific things that are recalled within [6:53] memory for it. So uh here we'll we'll see the  system and then um I'll get a link at the end if [6:59] you want to go and try to deploy this or or use  the code on your on your own for it. All right. [7:07] All right. So here I am. Uh I just have  a simple web app that I've created. [7:15] And we're at this page here. And let me make  it a little bit larger just so we can see it. [7:21] All right. So, we have five steps that we're  going to accomplish here with with each of [7:25] our agent pieces. The first one is that we  have one agent, which is just a simple agent. [7:30] This agent is going to be able to give us, you  know, essentially extract our preferences or [7:35] maybe things that we ask of it. We'll see how  that telemetry comes through. We'll harvest it. [7:40] We'll see what exists in our actual memory  that our agent can use and then we'll look [7:44] at how that returns from our preferences. So,  first off, here I have an interview agent. So, [7:49] the agent is essentially just asking me some  different things that maybe I'd want to know. [7:54] And here I've used um Google's agent development  kit. Uh the newest version is release 2 [8:00] uh which has a lot of cool things in it. And  then we use I should have updated this to agent [8:03] platform but we're using a model off of that.  In this case, Gemini 3.5 light. And I could [8:09] I could say here, you know, my it's asking me  for my name, my role, and my favorite programming [8:14] language. So I'll say my name is Jason. I'm  an engineer and I like Go when I use agents, [8:22] but otherwise I use Python because I can  read Go, but I cannot write it. All right, [8:30] I did a message, right? Someone got  my joke. I appreciate that. All right. [8:35] So, first thing, right? So, we have this uh  we have our prompt. We can see that, you know, [8:39] we've recorded this information. Um there's other  things that maybe we could do. So, it's asking me [8:44] other questions. Which databases, infrastructure,  or cloud tools do I use? So, I'm going to say I [8:49] use a lot of Spanner and BigQuery. [snorts] And  we could continue to just get information, right? [8:56] So think of you know a travel agent or another  agent at your company maybe you know an HR agent [9:02] or a finance agent which is just asking users  questions you know here I'm getting information [9:07] right but if you think about just a single  agent well if I'm not storing this information [9:12] if I come back to the agent right it's not going  to know any of this stuff so the first thing we [9:17] want to do is actually get all these things into  logs that we can then use to start building dreams [9:22] off of and for that what we've done is we have  our logs here in BigQuery. So, um, using ADK, [9:29] there's a plugin that you can use that just  sends all of your telemetry data to your own [9:34] BigQuery instance. And what that allows you to do  is to see all of the trajectories of your agent. [9:38] Think of trajectories just very similar to traces,  which we've used in observability for a long time. [9:45] And you can see here that I have all my different,  you know, logs that are coming in. We have our [9:50] user prompts. We have our agent responses.  And I actually have the prompts and responses [9:56] here stored in this table. And this is  actually a really important part. So [10:00] you know, one of the things that I hear a  lot from users that are working with agents [10:05] is, you know, the cost right of of that agent  of that agentic workflow and I always tell [10:09] people to make sure that you're budgeting  a pretty significant portion of your tokens [10:13] to the systems that are actually observing your  agents behavior. And so think of this, you know, [10:18] when you get into these longunning, you know,  15, 20 minute discussions, you may be storing, [10:23] you know, upwards of of millions of tokens of  information. Do not let that information go to [10:28] waste and just throw it away. Make sure you're  logging this stuff because this is ultimately [10:32] what will make your your organization  or your agent successful in the future. [10:37] All right, so we have these different things  coming in. You know, here you can see we just have [10:41] a very simple Python class which we're logging.  So we have, you know, the different interactions [10:46] that we have and then we're just dumping this  to BigQuery uh in essentially some sort of [10:50] structured format to make it easy for us to use.  All right. So now let's talk about the fun part, [10:55] right? So how are we actually doing uh you know  harvesting our dreams or even our memories for it. [11:01] So um we're using uh BigQuery for this and  BigQuery actually directly integrates with a [11:06] number of of different large language models.  um obviously Gemini which is a Google one but [11:11] you can use any number of models from the agent  platform model garden and you can actually use [11:17] AI within BigQuery as a part of just your SQL  that you run and that makes it really easy to [11:22] do things like extract preferences. So  I'll go ahead and kick off a run here [11:28] and you can see here you know here's essentially  our queries that we're running. So, if you look at [11:32] some of these things, what's great you there's a  prompt field here, which um I'll see if I can get [11:36] the full prompt pulled up, but we're just using  SQL, right, to essentially get information out of [11:42] our data, which makes it really accessible if you  think about, you know, how are you letting users [11:46] have access to their own information, maybe to  get their own insights or build their own dreams [11:50] and memories off of things that they're trying  to accomplish. And here I'm using uh to the one [11:56] of the the presenters just before me is talking  about you know tokconomics you know for things [12:00] like summarization right like I don't need to use  you know the the pro model or opus or or or fable [12:07] for it I'm just using flashlight right it's it's  a summarization activity all the information is [12:12] in the context window that it needs let's keep it  cheap you know where where it makes sense to do it [12:18] all right so we can see that we have our  essentially our harvesting complete and so we have [12:23] seven memories that we were able to extract from  it. And then we'll see this in Cloud Spanner here [12:28] in a second for how we we got those in. All right.  And the other piece here, so you can see, [snorts] [12:37] you know, what's cool about this, right, is like  prompts are prompts are English at the end of the [12:41] day, but prompts are also code. You know, here  I have a a relatively simple prompt template, [12:45] but this is if it you think of what I'm showing  you as kind of the the the bones of a system. [12:51] You know, this is where it's like  what's your secret sauce? You know, [12:53] what are the things that you're trying to extract,  you know, out of your agentic behavior. You could [12:57] go and modify this to essentially be whatever,  you know, it is that you're trying to get out. [13:02] And you can even you can do this for single  sessions. You you can do this over many sessions. [13:06] It's really about what is is the value of  of the dream harvesting and then what are [13:10] you going to use that for uh in your agents uh  moving forward. All right. So let's uh just see [13:18] this to make sure that we actually have our our  preferences. So I'm going to go ahead and refresh. [13:24] So we can see here now in our spanner table  uh we have a number of different keys. So [13:30] we essentially you store things like what's the  user's favorite language, you know, what's the job [13:34] role, what's what's my name, you know, preferred  infrastructure. And what's what we want to do here [13:39] in some cases maybe we keep this free form. you  know, if there are, you know, intents where we're [13:44] just looking for, you know, vector matches to say  this is close enough. But there are other things, [13:48] right, where we do want, you know, something  that's, you know, kind of a strongly typed key [13:52] that we know about, you know, a user or a process  or something else, uh, to use in it. And we can [13:57] obviously do both of those in a table like this.  All right. So last thing, so let's actually see [14:04] this in action. So if you think where we're at  in our journey, so we've we have an agent which, [14:08] you know, is generating all this this information.  uh we've submitted this to essentially an agentic [14:14] system where we can then harvest that information  and have a you know dream uh to get that in. [14:19] But now we actually need to share that information  back with our users. So um there's a couple ways [14:26] here, right? I mentioned that you can change  instructions of your agent. So you know if [14:30] you look at a dream and you there's behaviors  that are not ideal in your agent, you can just [14:35] modify the actual instructions or maybe your your  loops within the agent to change that behavior. [14:40] But another thing here that's actually I think you  know probably one of the more underrated ways to [14:45] think about this is well how do you come back to  the concepts of rag to just inject information [14:50] at runtime that's relevant to what the agent is  trying to do and in this case I have a second [14:55] agent where the first thing that it does is it  sees you know a question from the user it rewrites [15:01] that question so if there's a match in the vector  space that it will find it and then it will give [15:05] that back to me as a result. So let's go ahead and  see that. So, I'm just gonna say it says, "Ask me [15:12] what your favorite things are." It's like, "You  should know this, right? You tell me." [snorts] [15:18] And we can see that we actually got our  information back. So, you know, obviously it [15:22] knows my name. So, it was able to extract that.  It was able to get my preferred infrastructure, [15:27] my preferred tools, and it also has some other  things here that it's extracted about, you know, [15:32] my working style and and other things  that we would like to maybe get from it. [15:36] And you can see here that you know flash light  is obviously a very fast model for demonstration [15:42] purposes. This is great but in in a long recall  or a long running example speed actually is very [15:47] important and we were able to get all this back  in roughly 300 milliseconds from our database. [15:54] So really cool right to see all these subsystems  come together and as a result of that now we can [15:59] think about how we start to build more durable  longunning operations uh in our processes [16:04] for them. So I can say you hey we have all  these other things is like what else do you know [16:12] [snorts] and we'll actually get back even more  information because it's just going back and [16:16] you know looking at some of the other test  sessions that I've run in the past to give [16:20] me more things that it it understands about  me and I could even go back and maybe use the [16:24] harvesting method to take this session and then  maybe compact other things or maybe you look at [16:30] you know changes of of behavior over time and then  store that information as a old. So, same pattern, [16:37] but you can use that pattern based on what your  business uh processes are and how you want to [16:41] make it the most useful in it. All right. So,  uh let's go ahead and we'll wrap it up here. [16:54] All right. Couple things. Um so, what did we  just talk about, right? So uh we use Google agent [16:59] development kit to essentially build a couple  you know agents where we can then you know get [17:03] data out of those agents to make it useful for our  processes. Uh we use Google cloud spanner which is [17:08] essentially our memory store. So you think of as  you know how do I colllocate the memories that my [17:14] agents may need with other data that my agents may  need. And then BigQuery and agent platform were [17:19] really the tools that we used in order to create  a dream system for our different processes in it. [17:26] So, um, last piece here that I'll maybe some of  and I'll put the QR code if you want to go and, [17:31] uh, look at the code or try it out. It costs  about $60 because of the Spanner instance, but [17:36] you could have an LM change it to any database,  uh, that you want to use in it. First off, uh, [17:41] memory management. If you think of, you know,  kind of tier things, the evolution of agents where [17:46] we're at, probably the most, um, important thing  that you want to do to actually improve longunning [17:52] or even shortrunning agent behavior. And then  agents can use that memory with dreaming to [17:57] actually self create a system of self-improvement.  Obviously uh you know with humans in the loop to [18:03] to validate things. But this is really where we  think about you know it's not necessarily just the [18:08] model. It's about the systems that we build around  the model uh to essentially get agents to do the [18:14] things that we want to do for it. All right,  let me go ahead and throw the QR code up here. [18:20] Um you can see here it's it's in this repo. Um,  but yeah, with that, I'd like to thank you all. [18:26] I really appreciate uh you coming out. I hope  this was useful and gave you kind of a a pattern [18:30] that you can follow or start off with. Uh, but  I'll open up to any questions you might have. [18:38] Perfect. [applause] [18:43] Thank you. Uh, great presentation. Firstly, uh, [18:46] I just wanted to understand the dream harvesting  piece again. Uh, if I can think of an analogy [18:53] when I'm working on with a coding assistant, is  can you hold the mic a little closer? Sorry. Uh, [18:58] if I think of an analogy, if I'm working with  a coding assistant, is it the same as save this [19:03] to memory? Is that what dream harvesting is?  Or is the dream concept something more deeper [19:09] is what I'm trying to Sorry, can  you repeat? I I could Sorry. Uh, [19:12] I am just trying to understand are dreams the  same as memories? Like if I'm working with [19:18] something like chat JPT and if I say save this  to memory, is it the same as a dream or is it [19:23] something more nuanced is what? Yeah. So think  of I tend to think of dreams as like the process [19:27] that creates memories. So you know if  you're dreaming right like you're not [19:33] like it it's funny we don't anthroorphicize AI but  it always comes back to like then concepts that [19:39] like we actually can like gro um you dreams help  us create memories or help us update memories. [19:45] So it's really you know there's a dream  which is you know maybe like a session [19:48] that you look at think of almost like a data  pipeline but what the artifacts of that are [19:52] the memories that were then storing storing  or updating or changing as a result of it. [19:59] Hey there. Uh you showed in your demo sort of  like a a chat to get the the initial memories [20:05] and sort of discussion going. How do you think  about sort of like a corpus of information going [20:10] into this that would kickstart the process?  So uploading a bunch of documents or sort of [20:15] company knowledge for example and then starting  to parse and build those key value pairs and then [20:22] starting to like iterate on that over time. How do  you think of a challenge like that at scale versus [20:26] sort of the the demo which is maybe more like one  conversation turning into these key value pairs? [20:31] Yeah. So first um so the question was like how  do we how do we kickstart these things right and [20:36] actually like create like fully agentic systems  because obviously like a 15-minute demo is not [20:41] a a production thing. First off is log all of  your telemetry. So like if you have an agent [20:47] right like even test sessions right and like  you know using like adversarial testing like [20:52] do you have 50 sessions that you can actually look  at together and say hey like what happened or or [20:57] what were differences within it. um you have a  knowledge base probably somewhere of like things [21:01] that the agent should know. Um the tough the  thing with knowledge base that you just have to [21:06] make sure of right is that you're always rewriting  the question. It's like a question that I ask you [21:12] when I was originally building it's like return  me like which language I like it found like a [21:16] a zero match in the semantic space because what I  asked versus the embedding that was it was stored [21:23] were actually like semantically different in  it. So you want to just make sure that you're [21:27] actually looking at hit rates of those things.  This is where when you have it logged though, [21:31] you can actually just give these things to an LLM  and say, "Hey, look at five sessions together. [21:35] Tell me what worked or what didn't like based on  the token cost maybe associated for it if there's [21:40] a particular vector." Um, but those are probably  the the easiest ways that I would get going. And [21:44] it usually just starts first off by making sure  that you're logging everything that you may have. [21:51] Hi. Um, name's Rachel Manis with Align Technology.  Um, I just had a question. We're starting to [21:58] explore and experiment with Agentic Teams.  Yep. So, having our engineers actually manage [22:03] Agentic Teams that do a full almost like dark  factory model for development. Um, in that space [22:12] right now, we're instructing these different  agents uh just based on like Jer tickets and PRs. [22:19] Yeah, they have no other context to eliminate any  bias that might be available within that context [22:24] window. How does this like dream harvesting  and memory uh work in parallel with that? [22:32] Yeah. So the question is how you if you think  of context management right with agent teams [22:36] like how do dreams and memories work within it?  Um the real world example I kind of think of an [22:42] agent team you know in dreams is like think  of like a sprint retro if you've ever used [22:47] agile like a sprint retro is is somewhat like a  dream. So you can have an agent team write or a [22:52] subsystem like look at the behaviors like of  many agents like over maybe a particular goal [22:57] and then say great like what worked or what didn't  or in one case like we got to the ideal outcome [23:02] and another case we didn't like why what what's  the difference between those and then a dream [23:08] system is really you know it's it's what you put  into the prompt in many aspects like in the the [23:13] things that you're trying to optimize for. So on  an agency team maybe what's the like one yielded [23:18] 500,000 tokens of usage and another yielded 2  million like what were the things that happened as [23:24] a result of you know the 2 million tokens versus  the 500,000 like and maybe it was that like the [23:29] director agent you know had a really good plan in  one example and the other one was a terrible plan. [23:35] So those are the probably the first things I would  do, you know, in agent teams is you can use the [23:39] same system. It's just like what are what's the  prompt look like? You know, use an LLM to build [23:44] you the first prompt that you're trying to maybe  optimize for in the team. And then you can use it [23:49] both to change the behaviors like of the agent  v in of the agent, sorry, with instructions. [23:54] You can also look and say, well, are we passing  off maybe too much context in different scenarios [23:59] and use the LM to just self-improve that platform.  The only other thing I'll add and this is where [24:05] um dreams and eval also work together is you want  the eval. So the eval in this case could be you [24:10] know your golden session that worked really well  and then you're going to replay that. So like hey [24:14] give the agent team that goal again. So you know  whatever that that state was you change things now [24:20] like did it actually improve and then a dream  you know is essentially you know running like [24:24] an experiment and then looking at the two and  comparing like did this work better or not and [24:28] then maybe and just continuing to iterate on that  improvement for it. Hope that helps to answer it. [24:36] Uh hello. So, we talked about how to create and  [music] update memories using the dreaming system, [24:44] but how do we determine when [music] to prune  those memories like when they're no longer [24:50] relevant or out of date? Yeah. So, the question  was like create and update are easy, but like how [24:57] do we think about you know deletion, right, of of  dreams and things you within that. So, there's no [25:03] there's no right or wrong answer. like even as  people it's like what I knew 5 years ago versus [25:08] what I know today like there are things that I've  forgotten just because it's things that haven't [25:12] shown up as much. Um, you know, a lot of this does  come back in some cases like when you do the the [25:17] lookups, you know, so I have a task or a problem  like what's the most relevant to solving that [25:23] and then just test, you know, having the eval  systems test to make sure the things that are [25:27] always the most relevant are popping up and  you can even order it, you know, say by like [25:31] a date time of when that last, you know,  field or piece of information was there. [25:36] Um, you know, what's nice with agents, right, is  you never really have to delete it, but you don't [25:40] want it to pollute or taint the context window. So  you can even say hey if if a particular you know [25:46] memory wasn't accessed in the past 6 months well  did we not see that problem in which case you know [25:52] we just maybe we keep it in there but or we have  seen the problem that used to be used for it [25:58] and now like that's no longer relevant  so I'm just going to archive it off or [26:01] maybe set it to inactive to come back to. It's  really like what's interesting to me is like [26:07] the dream system really does come back to like  data warehousing and data lake work at the end [26:11] of the day and like if you read up on those things  like this is it's really no different um because [26:16] like hey you know dimension like slowly changing  dimensions or information about people like we [26:22] we change those all the time also based on you  know particular like maybe business events that [26:26] happen or things in it. So I would I'd probably  start with those and just saying like hey what's [26:31] the usage patterns that I would expect you know if  it's not being accessed and that's okay then like [26:36] you know remove it or you know archive it for a  period of time and then remove to to pop it off. [26:41] It's a business I think it's honestly more  of a business question than like a behavior [26:45] question than it is like a technical problem to  solve in it. Yeah. I just uh was curious with [26:52] with dreams and like retaining them over time.  I think you kind of touched on a little bit, but [26:58] like how is a how is like Google managing like  the longevity of these dreams and making sure [27:06] that memories are always kind of relevant and then  how are you making them like available to your [27:13] engineers or your uh different people using these  agents? Like do you just have like a centralized [27:19] place where all these agents live and they have  their own dreams and memories and everything or [27:23] just kind of curious. Yeah. So one of um so the  question was like what does Google do right with [27:27] some of these things like we have systems right  where we can go and look at agent trajectories. [27:32] So our stuff is logged you that we're writing like  code with so I can go look at and say hey what [27:37] happens you and I can either do it like on my own  personal stuff or like we you can do like cohorts. [27:44] So I think like the cohort model is actually a  pretty good one in particular as you look at like [27:48] you know sets of behaviors that are happening.  So it's like it's kind of like a classroom. [27:53] You like is the whole classroom bad or is  it like you know one one one kid that's like [27:57] causing problems or you know are they all good  and like what's the behav the desired like things [28:02] that we're getting out of it. Um but we have you  know internal systems that are very actually like [28:06] structurally pretty similar to to what I showed  that are just like ways that we can go analyze [28:12] this and then reinject you know this into the  agent like when we when we want it. But it really [28:17] is about like the retrieval at the end of the day.  Like um you can in some cases you can change the [28:23] subsystem. You know if you're using a stamp like  an out of the box harness you can't. And so then [28:27] it's like well what's the rag system and how  do I get that information back in a way that's [28:31] that's the most relevant and pertinent without  just like blowing through tokens in the window. [28:41] when when you look at um capturing  the the the data initially, how much [28:48] context do you actually give to it? So, you know,  you had a conversation with the AI. Yeah. How [28:55] far back do you go in order to be able to  draw the the the the dream information out? [29:01] Yeah. So, the question was like how  far back would I need to go? Um, and I [29:06] the probably the most like accurate answer is it  depends on like what you're trying to extract. [29:11] You know, there are some things, right, where  it's like, you know, if I'm in a conversation [29:15] with my spouse, right? I'm like, I don't want  like I should remember this, right? And like I [29:19] will remember it forever because we just talked  about it and like it was very important. And so [29:24] like in that case like you know even my window is  like 3 minutes. But there may be things which like [29:29] I'm learning you about a process over time where  maybe it like that subsystem is actually like [29:34] you know weeks worth of information that I I then  summarize and then I give you know to essentially [29:39] create like the the aggregate learning off of. So  I would say a lot of it is like if you have agents [29:45] like what is the goal of the agent and then you  know what are the things that you're looking for [29:49] to to manage that like is it behavior over time  is it a specific thing that happened that like we [29:55] don't want to do again and it was you know there  was a triggering incident and then you use dreams [30:00] to essentially like aggregate that and share it  back to the agent that in a way it makes sense. [30:05] Yeah. Great talk by the way. Uh my question  is about like humans have long-term memory [30:10] and short-term memory. So in that case how  your agent is performing in terms of memory [30:16] and how you're deciding like you just mentioned  about the memory like 3 minutes back you can just [30:21] erase [music] those stuff. So in that case do  you update those steps or what are the process? [30:26] Yeah. So the question was like how you if you  think of short-term and long-term memory and [30:30] dreams like how do these things work? you know,  in most cases, I say dreams are probably more, [30:36] you know, we're generating probably longer term  memories like across the board. So, it's like, [30:41] hey, like what are things I should know over  time about, you know, a user or a processor or [30:46] a system for it? But you can always reinject it  into short-term memory. So I think of short-term [30:52] memory typically is like working memory like  hey you know I'm going to solve a problem or [30:56] I'm working on a task and I have you know I'm  doing a coding thing so I have to retain like just [31:01] this information like top of mind to manage like  three different processes like I can still use [31:07] you know a dream system to you know maybe update  those things but I maybe be using also just like a [31:13] compaction system within an agent like on in real  time to continuously like summarize and reduce. [31:19] So, I think they they go together. The  question is really like what are the behaviors, [31:23] you know, that I'm trying to drive? Like  long-term memory, if I was to start something, [31:27] I'm probably going to pull from long-term memory  first to say, okay, what what do I know, you know, [31:32] and then like how is that relevant to this task  and then go execute the task and then short-term [31:36] memory, I'm just going to keep, you know, active  as long as I'm going through the different things. [31:40] Um, but dreams, right, I want to look at,  you know, what were the long-term memories [31:44] that I used? what was the short-term memory that I  generated or updated or deleted and then what was [31:50] the result and then maybe I go back and actually  update some of the long-term memories that I had [31:55] you know as a result of that via a dream. So  I hope that kind of answers like where these [31:59] things fit you but that's generally how I split  them. Um, but you can use, you know, rag methods, [32:04] right, to inject anything that you want into  a into a context window basically at any time. [32:10] And I think that's a good way you to  really like start to tune agent behavior [32:15] because like you know even Google models  with a million tokens, right? One, [32:18] it's expensive and two, like it just is so  longunning that now we're talking about like [32:23] how do we build like agent teams and how  do we split context based on a specific [32:27] task and these patterns work for that also to  do analytics and to understand the behavior. [32:35] Hi, I work at Google on YouTube ads. I'm trying to  build a data science AI to help PMs make decisions [32:41] about experiments to run. Um a lot of the times  when I ask my AI multi- aent system to remember [32:47] these preferences of the senior PM it is very  it's too specific. It's like oh okay in this [32:51] specific case when they ask about revenue in this  way I will do X and it's like well that's not the [32:56] high level thing. It's like it's like there  it's a taste thing I want you to remember. [33:00] So the LM seems not very good.  Even 3.5 Pro seems very bad at like [33:05] understanding oh what's the higher level taste uh  to remember as opposed to oh very tactical detail. [33:09] Um I'm wondering how to make it more uh the  memories more uh uh applicable widely as [33:14] opposed to like too tactical and not very useful  in general. Yeah. So the question right is like [33:19] and this really gets like specifics in like in  memory systems and dream is like how do we know [33:24] when to make things specific versus when to like  make generalized like principles and all that. Um [33:31] obviously there's no like right answer on that.  What I would say though a couple things. one [33:38] if you're trying to retrieve the question. I  think sometimes like you know we think of rag [33:43] as like the user asks a question just go look at  the question and then like get an answer back and [33:48] in reality like we're probably like missing the  embedding space in a lot of cases. So like are if [33:54] you're retrieving something that's too specific  is like well was the question so specific and [33:58] then you just had like a match which was like 90  you know like 95% you know or or like on cosign [34:05] or whatever. So first thing I look at is like  actually like rewriting questions or like tasks [34:11] that you're trying to accomplish. So like I think  it's like is it BM25 or 24 like I always forget [34:17] but there's like that and and hide which is like  really focused on okay well if you have a question [34:22] ask five questions like in the embedding space  that are similar questions but semantically [34:27] different enough to then retrieve you know a list  of results that maybe you put through a reanker. [34:33] So that's one way I would kind of look at  is like you know are you asking the question [34:36] too specifically and then you're getting a two  specific result back. Um the other thing I think [34:41] about and this is where like you know dreams are  are pipelines right and you can you don't have to [34:45] have one pipeline you can have many you hey you  can have one that's looking for like specific [34:50] you know things that you know within that that  memory or within that session or trajectory that [34:56] you're trying to extract from it. You could  also have general ones that are saying, "Hey, [35:00] over this thing like what were what worked well  or what didn't, you know, and then maybe you have [35:05] a list of five things which become like the  process that an agent should follow and then [35:10] maybe that actually becomes code, right? Maybe  you're actually going to modify the loop that the [35:14] agent's operating in based on on those things for  it. So um what's great about is like because now [35:19] we can generate all these insights with dreams, we  can actually you know modify both the agents like [35:25] code itself or the agent behavior and we can also  selectively choose when we want to inject things [35:30] into you agents or other teams for it. So I hope  that gives you a few ideas of of where to start. [35:38] Think we have one more [35:51] sometimes like actually changing like the like  the py like whatever code your agent since it's [35:56] like hey like like you know enforcing a workflow  as example like do this do this do this and then [36:02] open season and so not just the prompt like the  actual like like orchestration code within it. [36:09] Hi um in in the example that you have given  you have used uh bigquery for logging and [36:15] uh spanner for the memory storage. So for the  long-term memory and short-term memory what do [36:22] you recommend? How do we use it? [snorts] Uh is  there going to be different um like uh storage [36:28] locations? How do we connect with them? And what  is the process for that? What do you recommend? [36:33] Yeah. So in this example like I use BigQuery for  storing like all my telemetry work. So that's you [36:39] know per that's where I'm performing all like my  dreams or my analytics. So that's you know kind [36:45] of helping me to create you know my knowledge  base and then my memories that we may like give [36:49] to an agent. Um I use spanner in this example but  you could use like you know Postgress or you know [36:55] essentially any database you really want for  retrieval. Um, and it really just depends on, [37:01] you know, somewhat like the agent speed, but  also what does the agent need to have scoped. [37:05] So, like I work with organizations that use  Reddus for like, you know, short-term memory just [37:08] because the agent's working, working, working,  working. It's like, I mean, it's lightning fast, [37:13] right? It's Reddus. Um, but maybe they use  Postgress for like initial retrieval or like, [37:18] you know, vector lookups. So there's not  like necessarily a right or wrong answer like [37:23] like I use spanner in this case because  it I mean it is pretty it's pretty good [37:27] at like all the you know pieces together.  Um that said it is also more expensive. Um [37:34] but you could use you know any number of the the  tools that I listed for it like we have you know [37:39] obviously all and cloud SQL you could use like  fire store if you just want a simple like you know [37:45] no like no SQL lookup system. Um, but it's  really more like it's it's about making sure [37:51] that we're building the systems and then like  you know having the capabilities. So like you do [37:55] have to have like a data warehouse for agents,  right? Like that's one of the key things here [37:59] and then making sure that data is available. Like  I probably don't want to serve it in BigQuery, you [38:03] know, unless it's like I just don't care about the  latency because like BigQuery is going to take, [38:08] you know, we've done a lot of work to improve  it, but it's not like 300 milliseconds, right, [38:13] to like get something back for an agent.  So it's really then dictated by what are [38:16] the business requirements that the agent  has and how do you meet those like SLAs's [38:29] all right this is going to be our last  question thank you so much can you talk [38:34] about evolve because uh I understand the memory  is quite important but thinking your slides you [38:40] also mentioned about uh evaluation agent right  can you share a little details about that. [38:47] Yes. So I caught part of it. So your me how does  evaluation fit in? Yeah. So um how how do you [38:54] how do you build your eval agent? Um any  valuation? Valuation. Oh validation. Yeah. [39:05] So you know one of the things that I like in  validation like we have to actually label our data [39:11] and like one of the things that we do not do  very well like collectively is label data. [39:16] Um, so what I mean by that is like, hey, you have  15 sessions with an agent. Like did you actually [39:20] go and rate them? You like did you did you take  the time and say, hey, these three were great. Um, [39:26] and here's why. These 12 were terrible and here's  why. Like we have to spend like time, right? Is is [39:33] is individuals or you know with LM systems helping  us to do the initial grading to then create [39:39] like validation sets off of? So once I have those  then I can say hey well you know based on these [39:44] things like what worked well in these and  that would be good for an agent to you know [39:49] because LMS usually are pretty good about  dissecting like pieces of information that may [39:53] be useful you know in a future future iteration um  but also like what didn't work and you know using [39:59] the subsystem to then like find what happened in  there and sometimes it's obvious right like you [40:04] accidentally tried to call an MCP server  like a hundred times and you shouldn't have [40:09] um but there's like more subtleh behaviors that  we're trying to use dreams to actually find and [40:13] then you'll figure out how we how we do that how  we do that better. We have to have that validation [40:19] set though because like think of like memories  like memories and skills that you have like [40:25] there's a reinforcement learning system there,  right? Which is okay like how did you get [40:30] how did you like actually know that that was  a memory that was useful? Well, if you have [40:34] a validation set and we have the initial prompt  then like we actually want to run test scenarios, [40:39] right? So give it the same you know modify the  agent or you know inject you know a new memory [40:44] same prompt like was the result the same or was  the trajectory of an agent team the same or better [40:49] as a result great like maybe let's store that was  it worse well like where did it break then did it [40:55] introduce a different behavior that we expected  downstream and now we're modifying or should we [40:59] just trash it and you know build something else  you as a result like that's what we want to get [41:04] to with the validation and evalu I guess you  you use uh reinforcement learning to train [41:12] reward model and then use it score the output from  one of the agents you have you build every time [41:21] I think there's another trending that people  use language model it's a charge who evaluate [41:28] which one are using or both I mean it's like all  all these things are like so use case specific [41:37] Um, you know, I've also like I've done more with  like synthetic data, you know, in the past 12 [41:42] months. I was I started off as a data engineer  like than I ever did before because I'm like you [41:46] can synthetically generate validation sets. So  I'm like I don't know that there's like right [41:52] now like any right or wrong one. It's really like  what's like what's like the easiest way for you [41:57] to like do this without you know really breaking  the bank and both like time and and resources. [42:04] So, I'd say, you know, pick one or two, try  them like based on what you're trying to do [42:08] and then like see which one you do and maybe  you use like multiple like ways to do the work. [42:13] Like we have multiple platforms for eval. I  don't think that would shock anyone but like [42:18] we're all measuring different things, right? And  trying to optimize for different experiences and [42:21] as a result like some of the evals and the  validations look different and that's okay. [42:30] Awesome. Thank you so much everyone. Really [42:31] appreciate um all the questions and  answer. Sure hope it was helpful.