Writer AI Chief Pushes Multi-Tier Memory Over Raw Context Expansion

Memory & LLMs: Rethinking Recall With WRITER — Ai4

Ai4

Expanding transformer context windows cannot solve enterprise AI memory due to quadratic attention bottlenecks, requiring systems to adopt decoupled latent and parametric storage tiers.

AI-assisted summary of Memory & LLMs: Rethinking Recall With WRITER — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published by Ai4 on YouTube on August 26, 2025, this presentation features Dan Bikel, Head of AI at Writer. According to the page description, the talk clarifies what 'memory' truly means in LLM-based systems—defining it as the storing and fetching of task-relevant information—while examining context window dynamics, online versus offline trade-offs, and implicit versus explicit representations.

## TL;DR Dan Bikel deconstructs the concept of LLM memory by categorizing it across orthogonal axes, including update cadence, storage tiers, representation formats, and target beneficiaries. He argues that relying solely on expanding context windows is computationally inefficient due to quadratic attention scaling and goldfish memory failures, necessitating decoupled ingestion and retrieval architectures. Drawing from prior industry research, Bikel surveys advanced paradigms like MemWalker, Memory Layers, and Explicit Working Memory alongside Writer's enterprise Palmyra models. He concludes by presenting Writer's 'Reflect, Retry, and Reward' framework, illustrating how memory functions on a continuum from explicit retrieval to parametric adaptation.

## Atmosphere & style The presentation carries the polished, pragmatic tone of an industry veteran, blending light self-deprecating humor about speaker headshots and pronoun adjustments with rigorous technical depth. Bikel delivers a brisk, structured lecture that balances cognitive science analogies—such as comparing human memory compression to brute-force token buffers—with concrete engineering diagrams. The resulting atmosphere feels authoritative and clear-headed, demystifying long-term AI memory for enterprise practitioners without leaning into marketing hype.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=4s) Dan Bikel brings long-term memory and NLP expertise from leading conversational memory at Meta GenAI and co-developing semantic parsing for Google Assistant to his role as Head of AI at Writer. - [01:54](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=114s) Writer differentiates its agentic AI platform by strictly optimizing for enterprise workflows, powered by its proprietary Palmyra foundation model family. - [04:47](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=287s) System memory must be disambiguated from static weight memorization during training and long-term distributional adaptation over time. - [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) Context windows cannot serve as a complete memory solution due to token overflow ('goldfish memory') and the quadratic O(N²) computational complexity of transformer self-attention. - [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) Even large 1-million-token context windows remain tiny compared to the vast, fragmented datasets and multi-source APIs enterprise systems must orchestrate. - [11:34](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=694s) Memory update mechanisms span a spectrum between online, real-time ingestion during active conversations and asynchronous, offline batch processing. - [12:06](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=726s) LLM memory mirrors traditional computing hierarchies, treating the context window as fast cache memory backed by external non-parametric storage and parametric model weights. - [12:06](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=726s) Memory architectures require intentional choices between raw, sparse text representations and compressed, continuous latent vectors. - [13:44](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=824s) Memory systems must be categorized by their intended scope, distinguishing between immediate conversational coherence, user personalization, multi-user groups, agent identity, and task-level state tracking. - [15:28](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=928s) Implementing long-term conversational memory requires a decoupled collector-storage-retriever pipeline paired with base LLM fine-tuning on memory-injected training data. - [16:32](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=992s) Advanced memory architectures like MemWalker, Memory Layers, and Explicit Working Memory emulate human cognition by compressing context into navigable trees or trainable latent buffers. - [19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) Writer's 'Reflect, Retry, and Reward' framework optimizes models using self-reflection preference pairs, calculating gradients strictly over reflection tokens to prevent overfitting. - [19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) Memory in AI agents should be conceptualized not as a discrete feature, but as a continuous continuum connecting dynamic retrieval with parametric adaptation.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=4s) Introduction and Dan Bikel's Background

Dan Bikel introduces himself with a lighthearted comment about his speaker photo on the screen, joking that while it might look airbrushed, it is at least "realish" and not AI-generated. Bikel recently joined Writer as their Head of AI, having started on July 21st—roughly three weeks prior to the talk. To establish his technical background, he outlines his prior career path across major industry labs. Most recently, he spent three years at Meta within the Generative AI organization, where he led the effort to build long-term conversational memory architectures for social chatbots, creating the underlying template later adopted by Meta AI and related agent platforms.

Prior to Meta, Bikel spent over a decade at Google Research (spanning 2010 to 2022), punctuated by a two-and-a-quarter-year stint at LinkedIn where he founded and led the company's core Natural Language Processing (NLP) team. During his tenure at Google, Bikel collaborated with Jakob Uszkoreit and Percy Liang to build the semantic parser that originally powered Android Voice Actions. That system eventually evolved into Google Assistant and served as core infrastructure for parsing Google Search queries over the following decade.

- **Current Role**: Head of AI at Writer (started July 21, 2025). - **Meta GenAI (3 years)**: Led long-term conversational memory for Meta's social chatbots and Meta AI. - **Google Research (2010–2022)**: Co-built the semantic parser for Android Voice Actions and Google Search with Jakob Uszkoreit and Percy Liang. - **LinkedIn (2.25 years)**: Founded and led the enterprise NLP team.

---

[01:54](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=114s) Enterprise Focus and the Palmyra Model Family

Bikel transitions to introducing Writer, describing it as an agentic AI platform company dedicated to enterprise-grade solutions. He humorously catches himself adjusting his pronouns from third-person "Writer" to first-person "we" and "our" as he settles into his new leadership role. He highlights that Writer's primary competitive advantage—its "secret sauce"—is a relentless, exclusive focus on the enterprise customer. Rather than trying to build a generic tool for every consumer use case, Writer designs its entire stack specifically for enterprise workloads, directly informing training methodologies, dataset curation, and application workflows.

Writer operates an end-to-end platform powered by its proprietary family of foundation models, known as the Palmyra series. Founded in 2020, Writer has steadily advanced its internal model development alongside published research milestones. Bikel points out that Palmyra models compete directly with flagship models from major tech companies, placing squarely in the mix when evaluating standard evaluation suites like HELM (Holistic Evaluation of Language Models), where Palmyra models match the top five average benchmark scores.

```mermaid flowchart TD accTitle: Writer Enterprise AI Stack Architecture subgraph Foundation ["Foundational Layer"] A["Palmyra Model Family\n(Enterprise-Trained LLMs)"] end subgraph Platform ["Platform & Tooling Layer"] B["Writer Agentic AI Platform"] C["Integration APIs & Custom Data Pipelines"] end subgraph Solution ["Application Layer"] D["Enterprise Agentic Solutions"] E["Customer Workflows & Partnerships"] end A --> B C --> B B --> D B --> E ```

---

[04:47](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=287s) Defining Memory: Training, Memorization, and Dynamic Adaptation

Turning to the core topic of the presentation, Bikel addresses the inherent ambiguity surrounding the term "memory" in machine learning. Because machine learning models naturally generalize from and retain information seen during pre-training, people often conflate parameter memorization with functional system memory. To establish a more precise taxonomy, Bikel separates the broader concept into three distinct mechanisms:

1. **Training / Memorization**: The passive retention and statistical generalization of static patterns from training examples baked directly into weights. 2. **Memory (Store & Fetch)**: The dynamic or semi-dynamic storing and fetching of task-relevant information at inference time. 3. **Adaptation**: The capacity of an LLM or agent system to adjust over time as data distributions shift in the real world.

Bikel proposes a working definition for system memory: *"Memory describes when we can explicitly and dynamically store and fetch information as part of some larger LLM-based system that we are building."* However, he immediately critiques this definition as incomplete, noting that it focuses purely on external store-and-fetch systems while failing to capture newer architectures where LLMs are equipped with integrated, differentiable memory parameters.

---

[07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) The Fundamental Drivers: Context Limits, Quadratic Scaling, and Information Needs

Bikel examines the core engineering and cognitive reasons why external and parametric memory mechanisms are necessary in LLM systems:

```mermaid flowchart TD accTitle: Pressures Driving Memory Architectures in LLMs subgraph LLM_Limits ["Transformer Architectural Limits"] A["Finite Context Window\n(History overflows / Goldfish memory)"] B["Quadratic Attention Complexity\n(O(N²) pairwise token overhead)"] end subgraph RealWorld_Needs ["Operational Information Demands"] C["Dynamic Task Needs\n(Immediate relevant state)"] D["Vast Scale of Data\n(1M tokens is tiny vs. storage hardware)"] E["Disparate Data Sources\n(Multiple external APIs & silos)"] end LLM_Limits --> F["Explicit & Integrated Memory Systems"] RealWorld_Needs --> F ```

The first pressure is the finite length of the transformer context window. Regardless of whether an agent has access to thousands, tens of thousands, or a million tokens, appending raw conversational history eventually leads to overflow. Without dedicated memory, the agent suffers from "goldfish memory", forgetting critical facts established earlier in a session or across multi-week user relationships.

Beyond raw token bounds, Bikel identifies a deeper architectural inefficiency in transformer self-attention. Full attention matrices scale quadratically with sequence length, attempting to encode all pairwise token relationships regardless of relevance. Drawing an analogy to human cognition, Bikel emphasizes that human brains do not compute exhaustive pairwise associations across all historical inputs; instead, humans continually compress, abstract, and store key information as they go. Treating the context window as a brute-force memory buffer is scientifically unsatisfying and computationally inefficient.

Finally, real-world enterprise applications present dynamic, vast, and fragmented information needs: - **Dynamic Relevance**: Tasks demand specific data points fetched precisely when required. - **Massive Scale**: Even a 1-million-token window represents a tiny speck compared to the gigabytes of data readily stored on an ordinary thumb drive. - **Multi-Source Silos**: Enterprise agents must orchestrate data across multiple heterogeneous systems, APIs, and data sources on demand.

---

[11:34](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=694s) Dimensions of Memory: Online vs. Offline Updates

Bikel urges the audience to pay close attention to the conceptual framework of memory, framing it as the central takeaway of the session. He begins mapping out the multi-dimensional design space of memory systems, starting with the spectrum of **when and how updates occur**:

- **Online Updates**: Memory is written, indexed, or integrated in real time during the interaction as new tokens and facts stream in. - **Offline Updates**: Information is processed, condensed, indexed, or written asynchronously outside the critical path of the active conversation.

[12:06](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=726s) The Orthogonal Dimensions and Memory Hierarchy of LLMs

Memory in large language models spans several independent, orthogonal axes, starting with the frequency of updates. System updates can occur in periodic offline batches, in near real-time streams, or as immediate, real-time updates triggered during live user-agent interactions. Alongside update cadence is the question of storage mechanism and representation format. Memory may reside directly within parametric model weights, exist in external non-parametric storage subsystems, or sit directly within the active context window.

```mermaid flowchart TD accTitle: Memory Hierarchy in LLM Systems accDescription: Analogy between traditional computing memory hierarchies and LLM memory storage layers.

subgraph LLM_Hierarchy ["LLM Memory Hierarchy"] CW["Context Window<br/>(Analogous to CPU Cache / Fast Working Memory)"] Weights["Model Parameters<br/>(Parametric Long-Term Memory)"] ExtStore["External Subsystem<br/>(Non-Parametric Storage / Vector & Document Stores)"] end

subgraph Formats ["Representation Formats"] Raw["Raw / Sparse Representation<br/>(Text, Discrete Tokens)"] Dense["Continuous / Compressed Latents<br/>(Embeddings, Compressed Trees)"] end

ExtStore --> CW Weights --> CW Formats -.-> ExtStore ```

Drawing an analogy to the classical computing memory hierarchy, Dan Bikel likens the LLM's context window to cache memory—fast, immediate, but limited. Below that layer lie external storage modules and the frozen parameters of the neural network itself. Furthermore, the information held across these layers can take the form of raw, sparse representations (such as discrete text chunks) or continuous, compressed latent embeddings. Although these design choices frequently intersect in deployed architectures, they represent fundamentally distinct axes of system design.

- **Update Frequencies:** Batch updates, near real-time ingestion, and interactive real-time turn-by-turn updates. - **Storage Modalities:** Model parameters (parametric), external retrieval databases (non-parametric), and in-context buffers. - **Representation Types:** Raw sparse tokens versus compressed continuous latent vectors.

---

[13:44](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=824s) Categorizing the Beneficiaries and Purposes of Memory

Beyond how memory is implemented, system designers must evaluate who or what the memory is intended to serve. Memory can support the immediate conversational interaction, an area Bikel focused on during his time at Meta. Alternatively, memory can center on user personalization, preserve context across multi-user group interactions with an agent, maintain the continuity of the agent itself as an enduring entity over time, or track progress across long-horizon downstream tasks.

A vital conceptual distinction separates memory required for the immediate next conversational turn from task-level memory. Remembering details that must appear in the very next response is direct and immediate, whereas tracking state across a multi-step task operates "one step removed, one step meta." Structuring memory systems requires matching the architectural retention strategy to the specific entity and scope being served.

- **Conversation-Level:** Maintaining coherence across dialogue turns within an active session. - **User-Level:** Long-term personalization and individual user preference tracking. - **Group-Level:** Shared contextual state across multi-party user interactions. - **Agent-Level:** Preserving agent persona, state, and identity over long lifespans. - **Task-Level:** State management and memory over complex, multi-step workflows.

---

[15:28](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=928s) Architectures for Long-Term Conversational Memory

Implementing production-grade conversational memory requires a decoupled pipeline operating between the user and the language model. A memory collector—implemented either as a rule-based function or a dedicated auxiliary model—monitors the stream of dialogue turns and extracts salient facts into a persistent backend storage layer. During subsequent interactions, a retrieval module queries the storage layer to inject the most relevant historical context back into the context window, supplying the primary model with the background needed to craft the next optimal response.

```mermaid flowchart LR accTitle: Conversational Memory System Architecture accDescription: Architectural pipeline showing dialogue collection, backend retrieval, and memory-conditioned LLM generation.

subgraph Ingestion ["Ingestion Pipeline"] UserTurn["User Interaction"] --> Collector["Memory Collector<br/>(Function / Model)"] Collector --> Storage[("Backend Memory<br/>Storage")] end

subgraph Generation ["Retrieval & Generation"] Storage --> Retriever["Retrieval System"] Retriever --> Context["Context Window Injection"] Context --> LLM["Main LLM Generator"] TrainData["Memory-Specific<br/>Training Data"] -.-> LLM LLM --> Response["Next Best Response"] end ```

Crucially, standard pre-trained LLMs do not inherently excel at handling synthetic memory injections without specialized tuning. The architecture depends heavily on fine-tuning the core language model using memory-specific training data. Training the base model on examples that explicitly feature injected memories teaches the network how to seamlessly integrate retrieved background facts into natural, coherent responses.

---

[16:32](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=992s) Advanced Memory Approaches: MemWalker, Memory Layers, and Working Memory

Bikel highlights several foundational research efforts from former colleagues that push beyond naive context stuffing by emulating human-like cognitive compression:

```mermaid flowchart TD accTitle: Advanced Memory Architectures Comparison accDescription: Overview of MemWalker, Memory Layers, Explicit Working Memory, and MemAgent paradigms.

subgraph StructuralComp ["Structural & Latent Compression"] MW["MemWalker<br/>Hierarchical Tree of LLM-Compressed Nodes"] ML["Memory Layers<br/>Trainable Parametric Key-Value Parameters"] end

subgraph DynamicRefinement ["Active Monitoring & Policy Tuning"] EWM["Explicit Working Memory<br/>Latent Buffers with Active Generation Pauses"] MA["MemAgent & DAPO<br/>Recursive Multi-Turn Context Compression"] end

MW -.-> EWM ML -.-> MA ```

*MemWalker* (developed by Jason Weston, Oleksandr Sobal, Ramakanth Pasunuru, and colleagues at Meta AI) addresses long contexts by organizing text into a hierarchical tree of summary nodes. The model iteratively summarizes chunks and navigates this tree structure either dynamically or semi-statically, allowing it to explore large document sets repeatedly without re-reading raw tokens.

*Memory Layers* (by Barlas Oğuz, Scott Yih, Luke Zettlemoyer, and collaborators) introduces dedicated, trainable Key-Value parameter sets that expand the model's parametric capacity at scale rather than relying solely on ephemeral context activations.

*Explicit Working Memory* populates a dedicated working memory buffer with latent representations of retrieved passages. During text generation, the system actively tracks its own partial output, periodically pausing generation to refresh working memory and correct reasoning errors on the fly.

Similarly, *MemAgent* builds upon policy optimization techniques such as DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization) to handle multi-conversation environments by repeatedly compressing dialogue histories into a shared latent space.

---

[19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) WRITER's "Reflect, Retry, and Reward" and the Adaptation Continuum

Transitioning to recent research at WRITER, Bikel highlights the *Reflect, Retry, and Reward* framework. This approach leverages the LLM itself to generate high-quality preference pairs for reinforcement learning. The model reflects upon its own candidate outputs, critiques errors, and attempts a retry.

To avoid degenerate overfitting, the training objective computes gradients strictly over the tokens associated with the self-reflection steps rather than the specific factual details of the prompt. This self-supervised reflection mechanism yields strong performance gains across complex real-world benchmarks tailored to enterprise customers.

Bikel concludes by emphasizing that memory is not a single feature or isolated module, but a continuous spectrum running from explicit retrieval to parametric adaptation. Building fully autonomous and capable AI agents requires unifying these orthogonal dimensions—update cadences, storage tiers, representation formats, and target beneficiaries—into comprehensive, all-encompassing architectures.

## Notable quotes > The term "memory" is rightly ambiguous, especially in the context of LLMs. — Dan Bikel

> Even training itself is, in some sense, related to "memory" due to how well we can get LLMs to remember information from training examples. — Dan Bikel

> storing and fetching of task-relevant information — Dan Bikel

> [00:04](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=4s) realish — Dan Bikel

> [01:54](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=114s) secret sauce — Dan Bikel

> [04:47](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=287s) Memory describes when we can explicitly and dynamically store and fetch information as part of some larger LLM-based system that we are building. — Dan Bikel

> [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) goldfish memory — Dan Bikel

> [13:44](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=824s) one step removed, one step meta — Dan Bikel

> [19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) Reflect, Retry, and Reward — Dan Bikel

## Mentioned Dan Bikel, Writer, Meta, Meta AI, Google Research, Google Assistant, Google Search, Android Voice Actions, Jakob Uszkoreit, Percy Liang, LinkedIn, Palmyra, HELM (Holistic Evaluation of Language Models), MemWalker, Jason Weston, Oleksandr Sobal, Ramakanth Pasunuru, Memory Layers, Barlas Oğuz, Scott Yih, Luke Zettlemoyer, Explicit Working Memory, MemAgent, DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization)

## Video description Dan Bikel, Head of AI, WRITER

Abstracts

The term "memory" is rightly ambiguous, especially in the context of LLMs. Even training itself is, in some sense, related to "memory" due to how well we can get LLMs to remember information from training examples. This talk will provide a clearer picture of what "memory" actually means for LLM-based systems, namely, that it is the storing and fetching of task-relevant information. Through this lens, we will explore recent advancements of how memory is implemented in LLMs, including aspects like context window usage, online vs. offline methods and trade-offs, memory-related training methods and explicit vs. implicit representations.

[Source](https://www.youtube.com/watch?v=i7L4Lc9wg2U)

中文简报

Writer AI 负责人主张采用多层记忆机制而非单纯扩展上下文 由于注意力机制存在二次方计算瓶颈，扩展 Transformer 上下文窗口无法解决企业级 AI 的记忆问题，系统需采用解耦的隐式与参数化存储层级。 Dan Bikel 从更新频率、存储层级、表示形式和目标受用者等正交维度切入，对 LLM 的记忆概念进行了拆解。他认为，受注意力机制的二次方扩展开销和“金鱼记忆”缺陷限制，单纯依赖扩展上下文窗口在计算上效率低下，必须采用解耦的摄取与检索架构。结合行业前沿研究，Bikel 调研了 MemWalker、Memory Layers 以及显式工作记忆（Explicit Working Memory）等先进范式，并结合了 Writer 的企业级 Palmyra 模型进行探讨。最后，他介绍了 Writer 的“反思、重试与奖励”（Reflect, Retry, and Reward）框架，展示了记忆如何在一个从显式检索到参数化自适应的连续谱系中协同运作。 Writer AI 负责人 Dan Bikel（前 Meta GenAI 对话记忆主管）指出，必须将大语言模型的系统记忆与训练期的静态权重记忆及长期分布适应严格区分开，系统记忆的核心在于推理时动态存储与检索任务相关信息。 (4:47) 尽管大模型上下文窗口已扩展至百万 Token，但受限于 Transformer 自注意力机制的 O(N²) 二次复杂度以及企业级海量异构数据的规模，单纯依靠上下文填充无法解决“金鱼记忆”问题，必须引入外部与集成式记忆系统。 (7:18) 大模型记忆体系映射了传统计算的存储层级，将上下文窗口作为快速缓存，背后依托外部非参数化存储和参数化模型权重，并在在线实时更新与离线异步处理、稀疏文本与压缩潜变量之间权衡设计。 (12:06) 在前沿记忆探索中，Bikel 分析了 MemWalker 树状压缩与显式工作记忆等架构，并展示了 Writer 的“反思、重试与奖励”（Reflect, Retry, and Reward）框架，该框架通过对自我反思步骤进行梯度计算来实现高效的强化学习与企业级任务适应。 (19:11)

Reading summary

Dan Bikel, Head of AI at Writer, argues that relying on transformer context windows as brute-force storage is computationally inefficient and structurally inadequate, requiring enterprise systems to adopt multi-tiered architectures that span explicit retrieval, hierarchical compression, and parametric adaptation. Architectural and Scalability Limits of Context Windows (7:18) Bikel outlines two core transformer limitations that make dedicated memory systems essential: finite context capacity and computational complexity. Regardless of context window size, continuously appending dialogue or enterprise history eventually causes overflow, producing conversational amnesia or "goldfish memory." More fundamentally, standard self-attention exhibits quadratic complexity, computing pairwise associations between all tokens regardless of relevance. Bikel contends this brute-force mechanism is cognitively unnatural—drawing an analogy to the human brain, which continuously abstracts, compresses, and extracts salient points rather than maintaining raw historical sequences. In addition to computational overhead, enterprise operational realities demand external memory. Even a context window of one million tokens remains negligible compared to standard physical storage or multi-gigabyte enterprise data stores. Furthermore, production agents must dynamically locate and integrate specific facts scattered across heterogeneous enterprise tools and API silos, which cannot simply be loaded entirely into a prompt. A Multi-Dimensional Taxonomy for System Memory (12:06) To eliminate ambiguity in how machine learning practitioners discuss memory, Bikel categorizes it into three distinct capabilities: static parameter memorization baked in during pre-training, dynamic store-and-fetch mechanisms used at inference time, and the systemic capacity to adapt over time as real-world data distributions drift. Bikel maps system memory across several orthogonal engineering axes analogous to classical computer architectures. In this hierarchy, the context window functions like a fast, limited CPU cache, backed by non-parametric external databases and long-term parametric model weights. Update frequencies vary across offline batches, near real-time ingestion, and live, interactive turn-by-turn modifications. Systems also vary by representation type—sparse text chunks versus continuous latent embeddings—and by their ultimate beneficiary, which ranges from immediate dialogue coherence and individual user personalization to long-horizon task tracking. Decoupled Pipelines and Latent Compression Architectures (15:28) Implementing long-term conversational memory in production requires decoupling data handling from text generation. A dedicated memory collector—configured as a deterministic rule or a specialized auxiliary model—monitors interactions, extracts salient facts, and deposits them into persistent storage. When subsequent prompts arrive, a retrieval subsystem injects relevant context into the context window. Bikel emphasizes that pre-trained language models do not automatically process injected synthetic context effectively; base models must undergo specialized fine-tuning on memory-augmented dialogue to learn how to weave retrieved background facts naturally into subsequent responses. Beyond basic retrieval pipelines, Bikel highlights research aimed at emulating cognitive compression. These include MemWalker, which organizes text into an LLM-summarized tree hierarchy to traverse long documents without re-reading raw tokens; Memory Layers, which supply dedicated, trainable Key-Value parameter sets; Explicit Working Memory, which pauses generation to dynamically refresh latent buffers; and MemAgent, which uses policy optimization (such as DAPO) to recursively compress multi-turn dialogues into a shared latent space. Parametric Adaptation and the Reflect-Retry-Reward Framework (19:11) Writer integrates memory principles into its enterprise-focused platform and proprietary Palmyra foundation models, which Bikel reports match top-five average benchmark scores on evaluation suites like HELM. To enable parametric adaptation without catastrophic forgetting, Writer developed the "Reflect, Retry, and Reward" framework. This framework uses the LLM itself to self-supervise and generate preference pairs for reinforcement learning by reflecting on initial attempts, critiquing errors, and retrying. To prevent degenerate overfitting to specific factual inputs, training gradients are applied strictly to the tokens generated during the self-reflection step rather than the prompt's underlying factual data. This mechanism allows the model to refine its internal reasoning strategies dynamically while preserving generalizability. Bikel concludes that system memory cannot be treated as an isolated, bolt-on retrieval tool. Building resilient enterprise agents requires unifying the entire spectrum—from external store-and-fetch pipelines and structured compression trees to continuous parametric adaptation—into a single coherent architecture.

精炼阅读摘要

Writer 人工智能主管 Dan Bikel 指出，单纯依靠扩大上下文窗口无法真正解决大语言模型的长程记忆缺失，必须在认知架构层面构建兼具显式检索与参数自适应的分层记忆体系。面对自注意力机制二次方复杂度的计算瓶颈与企业异构数据的规模需求，设计者需在更新时机、存储层次、表征形态与服务目标等多维正交空间中系统化推进系统构建。 上下文扩展瓶颈与企业记忆需求 (7:18) Dan Bikel 指出，在企业级智能体应用中，单纯拉长上下文窗口无法消除模型遗忘关键事实的“金鱼记忆”现象。从底层机制来看，标准 Transformer 的自注意力计算复杂度随序列长度呈二次方（O(N²)）增长，穷举所有词元间关联不仅带来高昂计算开销，在认知机理上也缺乏合理性。Bikel 以人类认知为对照强调，人脑并不会逐字穷举历史细节，而是持续执行动态压缩与抽象提取。此外，即使是百万词元级别的上下文窗口，相较于普通外部存储介质而言依然极其微小，难以独自承载企业分散在多系统与外部 API 中的海量动态数据。Writer 研发的 Palmyra 系列基座模型虽然在 HELM 基准测试中达到前五平均分水准，但仍需依赖专用外部记忆机制来支撑复杂的企业工作流。 记忆多维正交空间与计算分层类比 (12:06) 为消除机器学习中“记忆”一词的多义性，Bikel 将其细分为三大机制：预训练期固化在模型参数中的被动统计泛化（Training/Memorization）、推理期对任务相关信息的动态存取系统（Memory），以及模型应对真实世界分布漂移的长期调整能力（Adaptation）。他将经典计算机存储层级映射至大模型架构：活跃的上下文窗口相当于快速但容量受限的 CPU 缓存（工作记忆），其下层是非参数化外部存储子系统（如向量与文档数据库），底层则是冻结的网络参数本身。在此基础上，系统设计还受更新频率与表征形式等正交维度的制约。记忆既可以在交互过程中实时在线写入，也可以通过异步批处理在离线状态下凝练；而在信息表征上，则存在离散文本块构成的稀疏形式与连续隐空间向量构成的稠密压缩形式之间的技术权衡。 目标服务对象与解耦工程管线 (15:28) 记忆系统的工程架构必须与其服务的实体目标精准匹配。Bikel 分析，记忆可服务于单次对话连贯性、长期用户个性化偏好、多用户群组上下文、智能体自身人设一致性，以及多步骤工作流的任务状态追踪。其中，支撑即时交互的对话记忆与跟踪复杂执行流的任务级记忆处于不同抽象层级，后者需要更高级别的元状态控制。在工程落地层面，生产级对话记忆通常采用解耦管线：由规则或专用辅助模型构成的收集器（Memory Collector）监听对话流并将核心事实写入持久化存储；在后续交互中，检索模块按需抽取相关上下文并注入上下文窗口。Bikel 特别强调，基础大语言模型原生并不擅长消化人造的注入记忆，生产系统必须引入针对记忆注入样本的专用微调数据，训练主干模型将外部检索事实无缝融入自然回复。 前沿结构压缩与自省优化范式 (16:32) 为了超越原始文本的简单拼接，学术界与工业界探索了一系列借鉴人类认知压缩的前沿方案。例如，MemWalker 将文本构建为多层级摘要树，引导模型按需遍历结构化节点而无需重读原始全文；Memory Layers 通过引入可训练的键值参数在模型内部扩展容量；Explicit Working Memory 在文本生成中维护动态隐空间缓存并适时暂停生成以纠正推理偏差；MemAgent 则结合解耦裁剪与动态采样策略优化（DAPO）实现多轮历史向隐空间的递推压缩。在 Writer 自身的技术实践中，团队提出了“反思、重试与奖励”（Reflect, Retry, and Reward）框架，利用模型对自身生成结果的自我批判与重试来构建高质量强化学习偏好数据。为避免模型过拟合于提示词中的特定事实，该训练范式严格仅针对自省步骤关联的词元计算梯度，从而在企业级任务评估中实现了稳定的性能提升。 Bikel 总结认为，记忆不应被窄化为单一的功能特性或检索插件，而是一条涵盖显式存储、动态压缩与参数自适应的完整连续谱。未来构建高可靠企业级 AI Agent 的核心任务，在于统一调度更新节奏、存储层次、表征结构以及服务对象，打造兼具即时感知与长期演化能力的综合记忆架构。

Detailed notes — zh

## Background 该视频由 Ai4 频道于 2025 年 8 月 26 日发布在 YouTube 平台上。视频记录了 WRITER 的 AI 负责人 Dan Bikel 的主题演讲，简介指出演讲旨在厘清大模型语境下“记忆”的真实定义——即任务相关信息的存储与检索，并全景探讨上下文窗口利用、在线与离线权衡、记忆专项训练及显式与隐式表征等最新进展。 ## TL;DR Dan Bikel 结合自身在 Google 与 Meta 主导研发记忆系统的深厚经验，剖析了大模型仅靠长上下文窗口面临的“金鱼记忆”缺陷与注意力机制计算瓶颈，论证了引入专用记忆架构的必要性。他系统拆解了记忆更新频率、存储介质（参数化与非参数化）及表征形态等多维坐标，并划分了面向对话、用户、群组、Agent 与任务的记忆层级。此外，他详细阐述了主模型需经记忆专项微调的工程要求，并梳理了 MemWalker、Memory Layers 等前沿机制。最后，他分享了 WRITER 团队最新的 R3（Reflect, Retry, and Reward）反思训练技术，呼吁将记忆置于从存储检索到动态自适应的连续谱系中进行系统设计。 ## Atmosphere & style 演讲兼具资深架构师的沉稳工程洞见与前沿探索者的敏锐度，开场以自嘲非 AI 生成美颜头像的幽默迅速破冰，随后迅速切换到高密度的硬核技术输出。在拆解核心理论框架时，讲者语气热烈而笃定，甚至提醒听众“千万不要眨眼”，将复杂的计算机存储层级类比与认知机制讲得生动透彻。整体叙事节奏紧凑、逻辑层层递进，散发着务实严谨且极具启发性的技术研讨氛围。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=4s) 讲者 Dan Bikel 拥有深厚 NLP 与搜索背景，曾在 Google 参与打造 Android 语音语义解析器，并在 Meta 主导了社交聊天机器人与 Meta AI 的长期对话记忆体系研发。 - [01:54](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=114s) WRITER 专注于企业级智能体（Agentic AI）平台，依托针对企业场景高度优化的全栈架构与自研 Palmyra 系列前沿模型，在 HELM 等标准评测中展现出顶级性能。 - [04:47](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=287s) 大语言模型的“记忆”应与“训练”（静态参数内化）和“自适应”（长期分布调整）区分开来，其本质是在推理系统中对任务相关信息进行显式、动态的存储与调取能力。 - [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) 单纯依赖上下文拼接无法解决长程交互问题：窗口截断会导致“金鱼记忆”，注意力机制的二次方复杂度（O(N²)）带来严重计算浪费，且无法静态容纳外部异构系统的大规模数据。 - [11:34](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=694s) 记忆系统的更新机制分布在一个连续光谱上，涵盖纯实时在线更新、半实时流式同步与离线批处理更新，设计时需在延迟、资源消耗与一致性之间做深度权衡。 - [12:06](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=726s) 记忆系统具有多维正交设计空间：LLM 上下文窗口可类比为计算机架构中的 Cache 高速缓存，底层可结合参数化存储与外部非参数化数据库，形态覆盖离散文本与连续稠密向量。 - [13:44](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=824s) 记忆服务对象分为五大核心矩阵：维系多轮交互的对话记忆、沉淀画像的用户记忆、协同共享的群组记忆、记录演进轨迹的智能体记忆，以及维护工作流状态的任务记忆。 - [15:07](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=907s) 工业级长期对话记忆系统的关键不仅在于捕获与检索模块，还在于必须使用专项记忆数据对主 LLM 进行微调训练，使其真正具备在正确时机利用注入历史片段的能力。 - [16:32](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=992s) 前沿记忆架构正在借鉴人类认知机制演进，如树状层级遍历的 MemWalker、参数化存储的 Memory Layers、实时监控纠错的显式工作记忆，以及递归潜在压缩的 MemAgent。 - [19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) WRITER 提出的《Reflect, Retry, and Reward》(R3) 方法通过自我反思自主生成强化学习偏好对，并仅针对反思 Token 计算损失，有效防止对事实细节过拟合，大幅提升企业级泛化纠错能力。 - [20:21](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1221s) 构建高自主性 AI 智能体需要跳出单一的上下文拼接或检索框架，将记忆理解为从“存储检索”延伸至“动态自适应”的完整连续谱系。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=4s) 讲者背景与个人履历 演讲者 Dan Bikel 开场时幽默地指着大屏幕上的个人头像调侃道，虽然那张照片看起来做了一些美化修图（airbrushing），但绝非 AI 生成，而是货真价实的真人照。Dan 刚加入 WRITER 出任 AI 负责人（Head of AI）大约三周时间（于 7 月 21 日入职），并对团队的前景感到非常兴奋。为了让大家了解讲台上发言者的背景，他简要回顾了自己的职业生涯：在加入 WRITER 之前，他在 Meta 的生成式 AI（GenAI）部门工作了约三年，主要负责为聊天机器人研发大语言模型（LLM）；在此之前（2010 年至 2022 年），他长期在 Google Research 工作，期间曾休整两年多前往 LinkedIn，从零搭建并带领了 LinkedIn 的 NLP 团队。 在 Google 任职期间，Dan 曾与 Transformer 架构的共同发明人 Jakob Uszkoreit 以及斯坦福大学教授 Percy Liang 合作，共同构建了最早支撑 Android 语音操作（Voice Actions）的语义解析器（Semantic Parser），该系统后来演化为 Google Assistant，并承载了 Google 搜索的查询理解。过去十年来全球用户在 Google 进行的搜索，大部分底层都运行过他们编写的代码。他在 NLP 与搜索领域积累了深厚的实战经验，也奠定了他对大模型对话记忆机制的研究基础。 - **职业履历节点**：Meta GenAI 部门（主导社交聊天机器人与 Meta AI 的长程对话记忆）、Google Research（2010–2022）、LinkedIn（NLP 团队负责人兼创建者）、WRITER（现任 Head of AI）。 - **经典工程贡献**：与 Jakob Uszkoreit、Percy Liang 联合开发了 Android Voice Actions 及 Google 搜索底层的语义解析器。 --- ### [01:54](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=114s) WRITER 的企业级定位与 Palmyra 模型 Dan 介绍了 WRITER 的公司定位与核心竞争力。WRITER 是一家专注于智能体 AI（Agentic AI）的企业级平台公司，致力于为企业客户打造同类最佳的解决方案。Dan 坦言，自己虽然刚加入公司不久，但已经深刻体会到 WRITER 的“独门秘籍”（secret sauce）所在：对企业级客户场景的极度专注与执着。因为不需要迎合所有普通大众的泛化需求，WRITER 可以将所有工程与科研资源集中于企业任务，这种聚焦直接贯穿到了模型架构设计、训练方法以及数据筛选的全流程中。 WRITER 坚持全栈自研，不仅拥有端到端应用平台以及深厚的企业客户合作关系，还推出了自主研发的 Palmyra 系列前沿模型。自 2020 年成立以来，WRITER 在模型研发上取得了显著进展。为了客观评估模型能力，WRITER 在业界标准的 Helm 基准测试中对 Palmyra 进行了严格评测；在对比 Helm 前五大顶尖模型平均水平时，Palmyra 稳居前列，证明了其与行业巨头正面竞争的顶尖科研实力。 ```mermaid flowchart TD accTitle: WRITER Enterprise AI Platform Architecture accDescription: WRITER's end-to-end platform focusing on enterprise solutions, proprietary Palmyra models, and external benchmarks. subgraph CoreStrategy["核心战略聚焦"] A["专注企业级场景需求"] --> B["定制化训练数据与流程"] end subgraph ModelLayer["自研模型与平台"] B --> C["Palmyra 系列基础模型"] C --> D["端到端 Agentic AI 平台"] end subgraph Evaluation["基准评测与落地"] D --> E["企业级智能体解决方案"] C --> F["HELM 等第三方基准评测"] end ``` --- ### [04:47](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=287s) 澄清概念：什么是大语言模型中的“记忆”？ 进入本次演讲的核心主题——大语言模型与记忆机制（Memory）。Dan 在 Meta 期间曾主导为社交聊天机器人构建长期对话记忆系统，该系统后来成为了 Meta AI 及各类对话机器人的底层记忆模板。他指出，“记忆”（Memory）在 AI 领域是一个极为宽泛且容易引起歧义的术语，不同人在不同语境下对其理解大相径庭。为了厘清讨论边界，Dan 将其与几个高度相关的概念进行了区分： ```mermaid flowchart LR accTitle: Memory and Related Concepts Taxonomy accDescription: Comparison between Training, Memory, and Adaptation in LLM systems. subgraph Concepts["LLM 相关概念辨析"] T["模型训练 (Training)<br/>静态内化样本分布"] M["系统记忆 (Memory)<br/>动态/半动态存储与检索"] A["在线自适应 (Adaptation)<br/>跨时间适应分布漂移"] end T --- M M --- A ``` 1. **训练（Training）**：机器学习模型通过观察大量训练样本进行泛化，模型天然具备在参数中“记住”部分训练实例的能力，属于静态的知识内化。 2. **记忆（Memory）**：在推理阶段，系统能够以动态或半动态的方式，显式地存储并检索对当前任务有用的信息。 3. **自适应（Adaptation）**：模型随着时间推移，在真实数据分布发生变化时持续调整自身行为的能力。 Dan 给出了一个初步的操作性定义（working definition）：**“记忆是指我们在构建更大的基于 LLM 的系统时，能够显式且动态地存储和提取信息的能力。”** 不过他随即补充指出，这个定义依然不够完备，因为它无法完全涵盖那些在模型权重内部集成了特定“记忆参数”的新兴端到端架构。 --- ### [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) 为什么 LLM 必须引入记忆机制？ 针对技术听众，Dan 剖析了为什么我们在构建基于 LLM 的 Agent 时，绝不能仅依赖单纯的模型前向传播，而必须引入专门的记忆机制。根本原因来自于三个维度的现实瓶颈与认知差异： 第一，**有限上下文窗口与“金鱼记忆”问题**。尽管现代 LLM 的上下文窗口已从数千、数万拓展到上百万 token，但窗口长度终究是有限的。如果只是一味将所有历史对话塞入上下文，早期的关键信息终究会从窗口边缘滑落被截断。如果 Agent 连 10 分钟前或两周前聊过的重要内容都遗忘掉，就会表现得像只有“金鱼般的短暂记忆”，这对于需要长程交互的社交助理或企业智能体是完全不可接受的。 第二，**二次方注意力机制的内在低效性**。标准 Transformer 的注意力矩阵计算开销随输入序列长度呈二次方（$O(N^2)$）爆炸式增长，试图对所有 token 对之间的关系进行全局编码不仅极其昂贵，而且在认知科学上也是不合理的。人类认知在处理信息时绝不会对看到的所有词汇做全对全的无序关联，而是在流式感知中不断进行实时压缩与结构化提取。用穷举注意力硬塞全部原始数据，在科学逻辑上显得“既不优美又极其低效”。 第三，**动态与海量的信息需求**。在实际业务中，Agent 面临的信息需求具有强烈的动态性与海量特征。即便拥有百万 token 的上下文窗口，与普通 U 盘动辄存储的海量信息相比依然只是一沧海一粟。此外，Agent 运转所需的大量数据分散在多个外部异构系统与独立 API 中，必须依靠动态的记忆与检索机制按需调取。 - **长程交互挑战**：纯上下文拼接在长周期对话中注定遭遇截断丢弃。 - **计算瓶颈**：全对全 Attention 的 $O(N^2)$ 复杂度导致超长文本计算成本失控。 - **信息鸿沟**：百万级 Token 上下文相比外部物理存储依然极其微小，且无法静态囊括 API 数据。 --- ### [11:34](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=694s) 拆解记忆的多维空间：更新时机与机制 Dan 强调，接下来的内容是整场演讲最核心的理论框架，他呼吁现场观众“在接下来的两三张幻灯片中千万不要眨眼”，因为这将全景式解构大模型记忆系统设计的核心维度。 理解记忆系统的第一个关键维度，是**记忆更新的时机与机制（When and how updates are done）**。系统对信息的写入和更新并不存在单一的标准范式，而是分布在一个连续的光谱（continuum/spectrum）上，两端分别是在线实时更新（Online Updates）与离线批处理更新（Offline Updates）。不同的应用场景需要在写入延迟、计算资源消耗、信息一致性与检索准确率之间进行深度的工程权衡。 ### [12:06](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=726s) 记忆的多维正交坐标轴与计算层级类比 构建大语言模型（LLM）的记忆系统时，涉及多个相互交织但本质上正交的技术维度。首先是**更新频率（Update Frequency）**：系统可以是离线的批量更新（Batch Updates）、半实时或近实时更新（Semi/Near Real-time），或者是与用户和智能体交互即时同步的纯实时更新（Real-time Updates）。其次是**存储介质（Storage）**：信息既可以直接固化在模型的参数权重中（参数化存储，Parametric Storage），也可以存放于独立的外部子系统（非参数化存储，Non-parametric Storage）。在存入之前，系统往往还需要对原始信息进行不同程度的压缩与精炼。 在存储位置与表达形式上，可以将大模型的存储体系与经典计算机体系结构的“存储层级（Memory Hierarchy）”做直观类比。LLM 的上下文窗口（Context Window）在初看起来非常接近计算机架构中的**高速缓存（Cache Memory）**，具备极高的读取即时性但容量受限；而外部存储模块与模型参数则类似于主存与持久化磁盘。此外，记忆单元的**表示形式（Representation）**既可以是原始稀疏的符号文本（Raw/Sparse Representation），也可以是经过编码的高维连续向量（Continuous Representation）。这些维度共同构成了审视大模型记忆技术的多维设计空间。 - **更新机制**：批量离线处理、近实时流式写入、交互级实时响应。 - **存储架构**：参数化模型内存储 vs. 外部检索子系统；上下文窗口（类比 Cache）vs. 外部持久化数据库。 - **表示形态**：从离散的原始文本切片到紧凑的稠密向量表征。 --- ### [13:44](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=824s) 记忆的服务对象与应用场景矩阵 除了“如何存储与更新”，另一个核心命题是：**记忆究竟是为谁、为了什么而服务的？** 记忆在系统中的定位直接决定了其架构设计与生命周期管理方式。根据赋能主体的不同，主要划分为以下几种场景： 1. **服务于交互本身（Conversational Memory）**：维系单次或多轮对话的连贯上下文，这也是 Dan Bikel 此前在 Meta 团队期间的核心研究方向。 2. **服务于具体用户（User-Level Memory）**：沉淀用户的长期偏好、历史习惯与背景画像，直接对应个性化（Personalization）体验。 3. **服务于协作群体（Group Memory）**：处理多用户与一个或多个智能体共同交互的群组协同场景，管理复杂的跨主体共享上下文。 4. **服务于智能体自身（Agent Memory）**：记录智能体跨越时间跨度的行为轨迹、自我演进状态与经验沉淀，使其成为具备时间连续性的独立实体。 5. **服务于特定任务（Task Memory）**：维护工作流、状态机及多步骤任务的执行上下文与元信息。 在实际生产环境中，系统往往需要灵活组合上述多种记忆模式，以应对复杂的复合业务需求。 --- ### [15:07](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=907s) 长期对话记忆系统架构与特定数据训练 在处理即时对话与任务级记忆时，系统逻辑存在显著的层级差异：前者关注“为了生成下一句最佳回复必须立刻召回的具体事实”，后者则更偏向“脱离单点上下文的元任务状态（Meta-task State）”。针对长期对话记忆的工程落地，Dan Bikel 介绍了一套曾广泛应用于工业级生产环境的高层系统架构。 ```mermaid flowchart LR accTitle: Long-term Conversational Memory System Architecture subgraph Capture["交互捕获"] UA["用户与智能体交互"] --> MC["记忆收集器<br/>(Memory Collector)"] end subgraph Storage["存储与检索"] MC --> Backend[("后端存储模块<br/>(Backend Storage)")] Backend --> RS["检索系统<br/>(Retrieval System)"] end subgraph Generation["生成引擎"] RS --> Context["上下文注入<br/>(Context Window)"] TD["记忆专项训练数据<br/>(Memory-specific Data)"] --> BaseLLM["主 LLM 模型<br/>(Main LLM)"] Context --> BaseLLM BaseLLM --> Response["下一轮最优回复"] end ``` 在该体系中，记忆收集器（Memory Collector）负责从交互流中提取有价值的信息并写入后端存储，检索模块则根据当前对话状态动态召回高相关记忆并注入上下文窗口。但该方案能够高效运作的核心“隐性前提”在于：**主模型必须经过专项训练**。仅靠 Prompt 拼接往往无法让模型稳定消化注入的记忆片段；必须使用专门包含记忆注入格式的训练数据对主 LLM 进行微调，模型才能真正学会在恰当的时机精准利用检索到的历史片段生成最优响应。 --- ### [16:32](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=992s) 前沿记忆机制探索：MemWalker、Memory Layers 与显式工作记忆 为了突破简单拼接上下文的瓶颈，学界与工业界探索了一系列借鉴人类认知模式的前沿技术路线： * **MemWalker（树状层级压缩与动态遍历）**：由 Jason Weston、Oleksii Sliusar 及 Ramakanth Pasunuru 等人提出。该方法利用 LLM 将长文本分块逐级递归压缩为一棵“记忆树（Tree of Memory Bits）”。对于需要反复探索的文本集合，系统只需构建一次静态/半静态树结构，随后智能体便能根据查询动态地在树节点间穿梭遍历（Walk the tree），大幅削减长文本上下文开销。 * **Memory Layers（参数化记忆层）**：由 Barlas Oğuz、Scott Yih（Scott G）、Luke Zettlemoyer 等人提出。与仅在推理期保留静态 Activation 不同，该方案引入了超大规模的可训练键值参数（Trainable KV Parameters），直接在模型参数空间内实现可扩展的高容量参数化记忆。 * **显式工作记忆（Explicit Working Memory）**：该机制将最相关的段落编码为隐空间表征填入工作记忆区。在生成过程中，工作记忆模块会主动监控（Actively Monitor）生成流，适时暂停生成并刷新工作记忆，实时比对并修正部分已生成的输出片段，实现“边生成、边反思、边纠错”。 * **MemAgent**：基于策略优化算法 DAPO（Decoupled Clip and Dynamic Sampling Policy Optimization）扩展至多会话架构，通过持续将历史经验递归压缩至相同的潜在表征空间，高度模拟人类大脑的信息整合与长期记忆固化过程。 --- ### [19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) WRITER 最新成果：Reflect, Retry, and Reward 与企业级落地 针对企业级高价值场景对大模型推理与适应能力的要求，Dan Bikel 重点展示了 WRITER 团队的最新核心成果——论文 **《Reflect, Retry, and Reward》（R3）**。 该技术旨在利用模型自身的自我反思（Self-reflection）能力，自主生成高质量的偏好对（Preference Pairs），用于强化学习（RL）对齐训练。其核心创新点在于**损失计算的隔离机制**：模型在训练阶段**仅针对自我反思相关的 Token 计算梯度**，而对具体的事实细节 Token 进行掩码（Mask）。这种做法有效防止了模型过度拟合特定样本中的具体事实细节，从而学会通用的自我审视与纠错逻辑。 该方案在多项面向企业级客户的核心真实业务基准评测中表现出卓越的泛化能力与任务完成度，为垂直领域的精准任务适应提供了全新的无监督/弱监督优化范式。 --- ### [20:21](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1221s) 总结：从记忆到自适应的连续谱 记忆绝非单一维度的上下文拼接或向量召回，而是一个涵盖参数化沉淀、外部非参数化存储、动态压缩与多层级反思的完整技术体系。在推动 AI 智能体向更高自主性、更强适应能力迈进的过程中，行业必须构建“全包容型（All-encompassing）”的复合架构，将记忆置于从“存储检索”到“动态自适应（Memory-to-Adaptation Continuum）”的连续谱系中进行系统性考量。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=4s) 虽然那张照片看起来做了一些美化修图，但绝非 AI 生成，而是货真价实的真人照。 — Dan Bikel > [01:54](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=114s) 因为不需要迎合所有普通大众的泛化需求，WRITER 可以将所有工程与科研资源集中于企业任务，这种聚焦直接贯穿到了模型架构设计、训练方法以及数据筛选的全流程中。 — Dan Bikel > [04:47](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=287s) 记忆是指我们在构建更大的基于 LLM 的系统时，能够显式且动态地存储和提取信息的能力。 — Dan Bikel > [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) 如果 Agent 连 10 分钟前或两周前聊过的重要内容都遗忘掉，就会表现得像只有‘金鱼般的短暂记忆’，这对于需要长程交互的社交助理或企业智能体是完全不可接受的。 — Dan Bikel > [07:18](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=438s) 用穷举注意力硬塞全部原始数据，在科学逻辑上显得‘既不优美又极其低效’。 — Dan Bikel > [11:34](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=694s) 在接下来的两三张幻灯片中千万不要眨眼。 — Dan Bikel > [15:07](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=907s) 仅靠 Prompt 拼接往往无法让模型稳定消化注入的记忆片段；必须使用专门包含记忆注入格式的训练数据对主 LLM 进行微调，模型才能真正学会在恰当的时机精准利用检索到的历史片段生成最优响应。 — Dan Bikel > [19:11](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1151s) 模型在训练阶段仅针对自我反思相关的 Token 计算梯度，而对具体的事实细节 Token 进行掩码。 — Dan Bikel > [20:21](https://www.youtube.com/watch?v=i7L4Lc9wg2U&t=1221s) 记忆绝非单一维度的上下文拼接或向量召回，而是一个涵盖参数化沉淀、外部非参数化存储、动态压缩与多层级反思的完整技术体系。 — Dan Bikel ## Mentioned Dan Bikel, WRITER, Palmyra, Meta, Meta AI, Google Research, LinkedIn, Jakob Uszkoreit, Percy Liang, Android Voice Actions, Google Assistant, HELM, MemWalker, Jason Weston, Oleksii Sliusar, Ramakanth Pasunuru, Memory Layers, Barlas Oğuz, Scott Yih, Luke Zettlemoyer, MemAgent, DAPO, Reflect, Retry, and Reward (R3) ## Video description Dan Bikel, Head of AI, WRITER Abstracts The term "memory" is rightly ambiguous, especially in the context of LLMs. Even training itself is, in some sense, related to "memory" due to how well we can get LLMs to remember information from training examples. This talk will provide a clearer picture of what "memory" actually means for LLM-based systems, namely, that it is the storing and fetching of task-relevant information. Through this lens, we will explore recent advancements of how memory is implemented in LLMs, including aspects like context window usage, online vs. offline methods and trade-offs, memory-related training methods and explicit vs. implicit representations. [Source](https://www.youtube.com/watch?v=i7L4Lc9wg2U)

Source quotes

[0:04] Dan Bikel: 虽然那张照片看起来做了一些美化修图，但绝非 AI 生成，而是货真价实的真人照。 [1:54] Dan Bikel: 因为不需要迎合所有普通大众的泛化需求，WRITER 可以将所有工程与科研资源集中于企业任务，这种聚焦直接贯穿到了模型架构设计、训练方法以及数据筛选的全流程中。 [4:47] Dan Bikel: 记忆是指我们在构建更大的基于 LLM 的系统时，能够显式且动态地存储和提取信息的能力。 [7:18] Dan Bikel: 如果 Agent 连 10 分钟前或两周前聊过的重要内容都遗忘掉，就会表现得像只有‘金鱼般的短暂记忆’，这对于需要长程交互的社交助理或企业智能体是完全不可接受的。 [7:18] Dan Bikel: 用穷举注意力硬塞全部原始数据，在科学逻辑上显得‘既不优美又极其低效’。 [11:34] Dan Bikel: 在接下来的两三张幻灯片中千万不要眨眼。 [15:07] Dan Bikel: 仅靠 Prompt 拼接往往无法让模型稳定消化注入的记忆片段；必须使用专门包含记忆注入格式的训练数据对主 LLM 进行微调，模型才能真正学会在恰当的时机精准利用检索到的历史片段生成最优响应。 [19:11] Dan Bikel: 模型在训练阶段仅针对自我反思相关的 Token 计算梯度，而对具体的事实细节 Token 进行掩码。 [20:21] Dan Bikel: 记忆绝非单一维度的上下文拼接或向量召回，而是一个涵盖参数化沉淀、外部非参数化存储、动态压缩与多层级反思的完整技术体系。

Source quotes

Dan Bikel: The term "memory" is rightly ambiguous, especially in the context of LLMs. Dan Bikel: Even training itself is, in some sense, related to "memory" due to how well we can get LLMs to remember information from training examples. Dan Bikel: storing and fetching of task-relevant information [0:04] Dan Bikel: realish [1:54] Dan Bikel: secret sauce [4:47] Dan Bikel: Memory describes when we can explicitly and dynamically store and fetch information as part of some larger LLM-based system that we are building. [7:18] Dan Bikel: goldfish memory [13:44] Dan Bikel: one step removed, one step meta [19:11] Dan Bikel: Reflect, Retry, and Reward

Original transcript

[0:04] Hello everyone. All right. Thanks for [0:05] for coming to to this session and to my [0:08] talk. Um I I think that looks like me. [0:11] That's actually is a photo, but it's [0:13] it's not AI generated. It's it's [0:15] realish. [0:18] I think they did some some airbrushing. [0:20] But anyway, if if you think it looks [0:22] different, come come talk to me [0:23] afterwards. Um, so I have been at Writer [0:27] as the head of AI for all of about three [0:31] weeks. I started there July 21st. So I'm [0:34] very excited about it. Um, I just want [0:36] to tell you a little bit more of my my [0:39] history. So uh, so you know who is [0:42] standing up here trying to tell you [0:43] stuff. Um, before being at Writer, I was [0:48] at Meta for about three years in the [0:50] Genai org. um where I worked on building [0:54] LLMs for chat bots and I'll tell you a [0:58] little bit more about that. Um before [1:01] then from about 2010 to 2022 I was at [1:04] Google research worked on a lot of [1:06] things there. I took a two and a quarter [1:08] year break in the middle there where I [1:11] went to LinkedIn and um led and built [1:14] the the NLP team for for LinkedIn for [1:17] the company. Um and I've worked on a [1:20] whole bunch of stuff. Um, one of the [1:23] things I did at Google, uh, with two [1:24] other guys, uh, Yakaba Buskrite and [1:26] Percy Leang, we built the semantic [1:28] parser that originally lay behind [1:30] Android voice actions, uh, which became [1:32] the Google Assistant and also handles, [1:36] um, search queries. So, if you've used [1:39] Google Search in the last decade, [1:41] probably went through our stuff and um, [1:44] yeah, I've worked on a lot of different [1:45] things. So, happy to chat with any of [1:48] you afterwards um, about any of that. Um [1:51] so [1:54] wait a minute. All right let me just [1:57] start by telling you a little bit about [1:59] about writer. Writer is an agentic AI uh [2:03] platform company and uh it is building [2:08] best of breeds solutions for enterprise [2:10] customers and that is um if you ask me [2:15] and I'm still getting my feet wet part [2:17] of the secret sauce of writer is that it [2:19] has a relentless focus on the enterprise [2:21] customer. So it doesn't have to build [2:24] things for absolutely everyone. It just [2:27] has to build really good stuff for [2:29] enterprise and that informs everything [2:32] up until uh up into the the the way we [2:36] train our models, the data that we use, [2:38] everything. Um [2:41] all right. [2:43] So right how how writer is different. Um [2:47] we build our own models. There's an [2:49] entire series called pelaria. Uh we have [2:52] an an endto-end platform and we have [2:55] deep partnerships um with our customers. [2:58] Notice I'm using the wei and the hour. I [3:00] I'm kind of uh uh adjusting my pronouns, [3:03] right? Writer is sort of a third person [3:06] pronoun thing, but I'm trying to I'm [3:08] part of the team. So now it's a it's a [3:10] we. [3:12] Okay. And this uh uh this slide [3:14] demonstrates a bit about uh the the [3:17] progress over the the last several [3:20] years. writer formed um in 2020 and has [3:25] seen this incredible progress in its uh [3:29] ability to produce the Palmyra series of [3:31] models. And just to so that you know [3:34] that these models are performing [3:38] exceptionally well for their intended [3:40] purpose which once again is serving [3:42] enterprise customers building um agentic [3:46] AI solutions for them on the writer [3:49] platform [3:50] but writer has made sure to run external [3:55] benchmarks on its own models so that [3:58] there's a proper appreciation of where [4:00] they sit within in the entire space of [4:03] models. And in fact, writer models play [4:07] right up there with the big boys, which [4:09] is to say that on on a standard [4:12] benchmark like Helm, you know, if you [4:14] were to average the top five uh Helm [4:17] scores, Palmyra is right in the middle [4:20] there. And so it's it's just evidence of [4:23] the of the quality of of research. And [4:26] here's another another kind of [4:27] alternative view of the model series on [4:30] top and some important research papers [4:33] below uh on a on a rough timeline [4:37] that that demonstrates this this [4:39] commitment to to cutting edge work and [4:42] it was one of the many reasons why I [4:45] wanted to join the company. [4:47] Okay, so now let's get into the the meat [4:51] of the talk here. I'm I'm here to talk [4:53] to you about LLMs in memory. It's um [4:57] something I've I'm very passionate about [4:59] and in fact it's something I worked on [5:02] quite a bit uh at Meta in the last three [5:04] years. Um I led the effort to to build [5:08] long-term conversational memory for [5:11] Meta's social chat bots and it also [5:13] formed the template for Meta AI and [5:15] basically all the chat bots. [5:18] So um what is memory in the context of [5:23] LLMs? [5:25] I think the term memory is rather broad [5:28] and ambiguous and that's part of why I'm [5:31] giving this talk is that memory means [5:33] different things to different people at [5:34] different times. And I'd like to try to [5:36] disambiguate it a bit and talk about the [5:38] many aspects that that um come into play [5:41] when you're building things that look [5:43] like that fall under that umbrella term [5:46] with LLM based uh agents and systems. So [5:50] some of the the other related terms are [5:54] training right because training itself [5:56] is looking at examples and h and having [5:58] a model generalize from them a machine [5:59] learning model of any kind right so [6:01] there's some degree of of um remembering [6:05] the examples right models historically [6:08] have been very very good at at [6:10] remembering training instances and then [6:13] there's memory itself which I will try [6:15] and and distinguish a little further but [6:17] basically anytime time you need to store [6:20] and fetch information um in a [6:24] dynamic or semi-dnamic way to be useful [6:28] um at the time of inference. And then [6:30] finally there's adaptation [6:33] which is sort of related to to both of [6:35] the above but can we have a model that [6:38] successfully adapts over time uh as the [6:42] distribution changes [6:45] right and we'll talk uh I'll talk a [6:47] little bit more about that towards the [6:48] end of this so let's let's look at a [6:52] possible working definition [6:55] we could say that memory describes [6:58] when we can explicitly and dynamically [7:00] store and fetch information as part of [7:02] some larger LLMbased system that we are [7:04] building. [7:06] But this is not quite sufficient. This [7:10] doesn't adequately capture LLMs that are [7:12] equipped with integrated memory [7:13] parameters. So why is that? [7:18] So let's look at why why do we even talk [7:21] about memory at all when we're building [7:23] LLM based agents, right? So this this is [7:27] the technical track. So this should come [7:29] as no surprise to any of you. We have [7:31] this um in almost all cases a finite [7:34] length LLM context window, right? So [7:37] what are the many problems when you have [7:39] got a finite window? Well, if you're [7:42] building say a chatbot, an AI agent that [7:45] is engaging in an interaction with a a [7:47] user, [7:49] um let's imagine that you have I don't [7:52] know several thousand, tens of [7:53] thousands, even a million or more uh [7:56] tokens to play with. Eventually, [7:59] if you're just feeding the entire [8:01] history of an interaction, it's going to [8:03] fall off the edge of that window, right? [8:06] It's finite. So, this is sort of the f [8:09] very first reason that you might want to [8:11] think of, oh, we need some other [8:13] mechanism to encode important [8:16] information about, let's say, the [8:18] history of the interaction, the chat [8:20] history, literally, right? Otherwise, [8:22] it's like a goldfish memory, right? I [8:25] I'm chatting with a chatbot. it's an AI [8:27] agent, might be a social chatbot, [8:29] whatever it is, if if it doesn't [8:32] remember the thing that we were just [8:33] chatting about 10 minutes ago or two [8:34] weeks ago, that's that's bad. So that's [8:39] one of the very first instances where [8:40] you might think, oh, we need to [8:42] incorporate some additional mechanism. [8:44] But there are other things other [8:45] fundamental limitations that might make [8:47] us think about memory. So [8:50] all of you in this room probably or most [8:52] of you know about um that we have these [8:56] these large attention matrices whenever [8:59] we have transformers and they are big [9:03] and they explode quadratically with the [9:04] lengthy input typically. Um and that [9:08] there are fundamental there's a [9:10] fundamental inefficiency of trying to [9:12] encode the relationships between all [9:14] token pairs. [9:16] Um [9:18] there's sort of an um I like to draw an [9:20] analogy here to human cognition. [9:23] We know um or or at least we strongly [9:26] suspect based on work in cognitive [9:30] science this is not how we humans seem [9:33] to process information. We're not uh [9:36] trying to encode relationships between [9:39] all pairs of tokens um in the sort of [9:41] arbitrary I don't know what's what way. [9:45] rather we have um a way of of storing [9:49] information that's that's quite [9:50] important so that we can um uh use it [9:54] later but we we compress as we go and [9:57] we're going to see that later on in some [9:58] of the approaches here. So, um, this is [10:01] this is sort of it'll get the job done [10:04] if if you're encoding absolutely [10:06] everything with your attention [10:07] parameters, but it it it's clear that [10:12] this is like in some sense [10:15] scientifically unsatisfying and [10:17] seemingly wildly inefficient. [10:20] Okay, other things that that come into [10:23] play here, uh, we have information [10:24] needs, right? Why are we building AI [10:27] agents in the first place? Well, we [10:29] often have information needs that that [10:31] come into play that also lead us to [10:32] think we need memory in incorporating [10:34] these systems. So, um our information [10:36] needs are often dynamic. We need stuff [10:38] now at this point in the interaction, [10:40] right? What is the most relevant and [10:41] important information right now? Um the [10:44] information is vast. I mentioned, you [10:45] know, LLM context windows, right? So, [10:48] let's say for the sake of argument, you [10:50] have a million tokens, a million subword [10:53] units. [10:54] Well, okay, that's tiny. [10:58] That's a speck relative to the vast [11:02] quantities of information you can stick [11:04] on a thumb drive for example. Right? So [11:07] um and then finally um the information [11:12] that our AI agent might require uh is [11:16] often available only via uh is often um [11:20] uh available via multiple sources and [11:23] unique uh potentially unique APIs. And [11:27] so another argument for oh we need a [11:29] dynamic way of accessing such [11:31] information. [11:34] Okay. All right. This is kind of the [11:35] heart of the talk. So so try not to [11:38] blink for the next two or three slides [11:41] because this is kind of what I want you [11:43] to to to take away from this talk if you [11:46] can. I want to tease apart the various [11:50] dimensions of memory. [11:52] All right. So the first dimension is [11:55] when and how we do updates. [11:58] So [12:00] there's sort of this continuum or [12:02] spectrum between online and offline [12:04] updates. [12:06] Uh we might be doing batch updates, we [12:09] might have semi or near realtime [12:12] uh updates and finally real-time updates [12:14] that have to do with the interactions [12:17] uh between say user and an agent. Then [12:20] there's a storage. [12:22] Uh we might have storage directly in the [12:25] model's parameters. [12:27] Um bonus points for anyone after the [12:29] talk who can tell me where this famous [12:32] diagram comes from. Uh we might have an [12:35] external subsystem that's doing the the [12:38] storage. Um [12:41] uh and and we might have so this is [12:43] nonparametric storage. [12:46] we might compress the items that we're [12:49] interested in storing. [12:51] And then finally, there's uh oh well, [12:52] not finally, but but there's where do we [12:54] store stuff so we can put things [12:57] directly in the context window. We can [12:58] sort of think of them an analogy to the [13:00] memory hierarchy that we're used to from [13:02] uh from computing, right? The LLM [13:04] context window uh sort of kind of at [13:07] first blush looks like cache memory in [13:10] some sense, right? [13:12] Um [13:14] there's also the the external module or [13:16] the parameter the model parameters [13:18] themselves right and then finally there [13:20] what what about these the items that [13:21] we're we're storing we might have a raw [13:24] or sparse representation we have to talk [13:25] about their representation or some [13:27] continuous representation these are [13:29] overlapping and intersecting dimensions [13:32] but they're largely orthogonal [13:35] and I wanted you to to consider these um [13:40] this is like one of the the the main [13:42] messages from from this talk. [13:44] Okay. And then here's one of the other [13:47] main messages of this talk is who or [13:49] what is this memory for anyway? [13:53] So it might serve to help the [13:57] interaction itself. It might be [13:59] conversational memory and like I [14:00] mentioned earlier or alluded to earlier, [14:02] this is the main type of thing that I [14:05] was working on when I was at Meta. um it [14:08] might be intended to serve the user, [14:12] right? And so in this case, it it [14:14] directly relates to personalization. [14:17] It might serve a group of users because [14:20] they're instances where you have groups [14:22] of users interacting with each other and [14:24] an agent. [14:26] It might serve the agent itself as its [14:29] own entity and and existing over some [14:33] some span of time. And then there are [14:36] the tasks that we're asking of the [14:38] agent, right? That's what these icons [14:40] are meant to depict. [14:42] And then finally, combinations thereof. [14:46] Okay, so this isn't exactly exhaustive, [14:50] but it's [14:52] fairly representative of the the things [14:55] that we might use memory for. [14:58] Okay, I'm going to move a little quickly [15:00] now because we don't have much time. All [15:03] right. So, um [15:07] there's a bit of a difference in [15:10] remembering and using information that [15:12] is particular to the very next response [15:14] and and perhaps even needs to appear in [15:16] the very next response versus memory [15:18] that's related to task. It's sort of one [15:20] step removed, one step meta if you'll [15:22] excuse the term. Um and this is kind of [15:26] an essential difference. So, I want to [15:28] present very very quickly. This is a [15:30] exceptionally high level diagram that's [15:33] in fact rather representative of the [15:35] design uh that we used for for [15:38] implementing long-term conversational [15:39] memory. There's basically some uh memory [15:42] collector which is any kind of function [15:45] could be a model that operates on the [15:47] interaction between user and agent and [15:49] then some kind of retrieval system. uh [15:51] there's a backend storage um module and [15:55] we're trying to retrieve the most [15:57] relevant memory at any given point in [16:00] the interaction to make the next best [16:02] response. [16:04] And then crucially in all this and this [16:06] is it's sort of hidden in here uh a [16:09] little bit uh you see their memory [16:11] specific training data going into the [16:13] main LLM. This approach assumes or at [16:17] least works well when you specifically [16:20] train your main LLM with examples that [16:25] that contain the types of information [16:28] that we're inserting into the LLM [16:30] context window. Okay, I'm going to have [16:32] to go exceptionally fast. Um, we'll have [16:34] to talk about this afterwards. Um, I [16:37] mentioned how humans tend to compress [16:38] things. There's an a great paper by some [16:40] of my URSTW colleagues um at U Metai uh [16:46] notably Jason Weston and Oley Silicas um [16:50] uh and uh Ramicon uh Pasanuru um which [16:54] has to do with this approach I've called [16:56] Memalker which involves constructing a [16:59] tree of of memory bits where an LLM is [17:02] itself doing the compression of entire [17:05] chunks so that it can then walk the tree [17:07] dynamically. um either dynamically or [17:10] semi-statically. And there's a there's a [17:12] trick to this because oftentimes we have [17:13] to explore a set of of information um [17:16] repeatedly. And so you can sort of build [17:18] this tree once for each uh set of [17:21] information and it it's it's basically a [17:23] way of compressing information. Um [17:26] there's another really cool piece of [17:28] work by my Urswell colleague uh Barlas [17:30] Ogus um also with uh Scott G and Luke [17:34] Zetto and others called memory layers. [17:38] Um and I'm just citing uh other people's [17:40] work. The the work that I did went [17:42] straight to production and so I I [17:43] alluded to that earlier. Um in this work [17:47] it's all about um exploiting parametric [17:51] memory specific sets of parameters KV [17:53] trainable parameters that uh and that's [17:56] the the really crucial bit. These aren't [17:57] just activations they're trainable KV [17:59] parameters that um also operate on a [18:02] much larger scale. All right, I'm going [18:04] at hyper warp fast speed here. Um [18:08] there's there's another piece of work [18:09] called explicit working memory um where [18:13] uh crucially memory is populated with [18:15] these latent representations of the most [18:16] relevant passages and then generation [18:19] begins and the explicit working memory [18:21] actively monitors the generation [18:23] occasionally pausing to refresh the [18:25] working memory and check the partial [18:26] output um kind of fixing things as it [18:29] goes. And this is this is really cool [18:30] and it's and it's again sort of uh [18:33] loosely inspired by human cognition. [18:36] And then finally um let me just mention [18:38] MEM agent which is more recent work. Um [18:41] it sort of extends some previous work uh [18:43] of yet another the alphabet soup of [18:46] policy optimization algorithms called [18:47] DAPO decoupled clip and dynamic sampling [18:50] policy optimization. Um it extends this [18:53] to multi conversational uh setup and um [18:57] crucially does this repeated kind of [18:59] compressing into the same space. Again [19:02] very reminiscent in a lot of ways to the [19:04] to human cognition. I think we can we [19:06] can do well when we get at least [19:08] inspired by human cognition. [19:11] All right. And then I would be remiss [19:13] hopefully I'm yes I'm presenting some of [19:15] the most recent work from a writer which [19:18] is this excellent paper called well I [19:20] think it's excellent. Okay, I'm I'm I'm [19:22] boasting about the new company I just [19:24] joined, but it's called Reflect, Retry, [19:26] and Reward. And it is about um making [19:30] use of the LLM itself to produce [19:33] excellent uh preference pairs for RL [19:36] training. And it does this by reflecting [19:38] on its own output and crucially training [19:40] only on the the um uh the tokens that [19:44] had to do with the self-reflection so [19:47] that it's not over [19:49] um uh training on the specifics [19:54] information that um that was the subject [19:56] of reflection. Okay. And it and it um on [20:01] top of everything I presented there, it [20:03] does exceptionally well on a bunch of [20:06] different real world tasks that are very [20:09] meaningful to enterprise customers which [20:11] like once again sort of the bookend [20:13] things those are the customers that that [20:15] writer in particular is concerned with. [20:19] So okay [20:21] I I did give it all away by telling you [20:24] what I wanted you to focus on which were [20:26] the various largely orthogonal axes [20:29] dimensions of memory. [20:31] There is no one kind of memory and we [20:34] need to build all-encompassing types of [20:36] solutions if we want to to capture all [20:39] the all the ways that we want AI agents [20:41] to behave well um in the future and to [20:44] be more capable more autonomous. So um [20:48] along this memory to adaptation [20:50] continuum and that's it. Thank you very [20:53] much everybody.