Granica Deploys Myelin Gateway to Cut Enterprise AI Agent Costs 90%
Ai4
Treating AI agents as persistent infrastructure rather than ephemeral tasks prevents context degradation and locks in low-cost prompt caching across enterprise workflows.
## Background
Published on YouTube by Ai4 on August 19, 2026, this presentation features Radhika Ravirala and Tarang Vaish from Granica.ai discussing the economics of AI agents. The talk addresses why treating agents as disposable tasks leads to skyrocketing costs and declining quality, advocating instead for treating them as persistent, learned infrastructure.
## TL;DR
Tarang Vaish and Radhika Ravirala explain how standard AI agent implementations waste resources by repeatedly starting from scratch or bloating context with naive memory systems. They outline the key architectural pillars needed for enterprise agents—cost, quality, governance, and portability—and detail how improper prompt cache management inflates token spend. Granica demonstrates its Myelin system, showing how experiential learning and cache shaping can achieve up to 99% cached reads and drastically reduce operational overhead.
## Atmosphere & style
The session carries a clear, pragmatic engineering tone, cutting through industry hype to focus on hard financial and technical realities. The speakers engage the audience with relatable metaphors, comparing unoptimized agents to forgetful interns and sharing candid failure modes from real-world production. It feels like an authoritative, data-driven masterclass that shifts from identifying core enterprise pain points to delivering a grounded solution.
## Key takeaways
- [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) Granica focuses on bringing intelligence to enterprise data, aiming to reduce storage costs, lower query latency, and optimize agentic workflows for higher ROI.
- [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) AI agents are not inherently smart; their capability depends heavily on tool calling and the operational context provided to them.
- [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) Without baseline operational context, simple agent debugging tasks can balloon up to 71 turns, wasting effort and frequently leading to hallucinations.
- [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) Treating agents like 'forgetful helpful interns' forces them to repeat the same sequence of mistakes across identical future tasks.
- [04:11](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=251s) Naive memory systems cause context bloat over time, inflating prompt payloads and causing agents to operate on stale or conflicting facts.
- [04:11](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=251s) Memory degradation often happens silently, leading developers to incorrectly attribute performance drops to model degradation.
- [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) Unmanaged memory leads to severe financial penalties due to massive price variations between reused cache reads ($0.50/1M tokens) and uncached writes ($6.25–$10/1M tokens).
- [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) Enterprise agentic systems require balancing four key architectural pillars: Cost, Quality, Governance, and Portability.
- [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) While Gartner projects a 40% cancellation rate for agentic AI projects, Granica predicts up to 80% will fail due to ad-hoc infrastructure.
- [11:12](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=672s) Granica's 'experiential learning' approach continuously measures, audits, traces, and prunes context to optimize both cost and response quality.
- [11:12](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=672s) Granica's Myelin system achieves a 99% cached read rate in production traffic, ensuring minimal new writing overhead per session.
- [11:12](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=672s) Maintaining stable prompt prefixes through 'cache shaping' accounts for 78% of token savings by locking in lower cache pricing rates.
## Detailed walkthrough
Here is a detailed, faithful walkthrough of the transcript:
[00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) Introduction to Granica and the Problem with AI Agents
Tarang Vaish, CTO and co-founder of Granica, opens the session alongside Radhika Ravirala, Senior Product Manager at Granica. He introduces Granica as a research and development company focused on bringing intelligence to data—helping big data enterprises processing tens or hundreds of petabytes lower storage costs, accelerate queries, and optimize agentic workflows to maximize ROI. Moving into the main topic, Tarang strips away the hype surrounding AI agents, noting that they are not inherently "super smart." Instead, an agent's true capability relies on two core elements: calling tools and reasoning based on the operational context provided to it.
To illustrate how current agents operate, Tarang presents a common scenario where an agent is tasked with diagnosing why a nightly billing report has been failing since Tuesday. Starting with a completely blank canvas, the agent must undergo a tedious sequence of tool calls just to figure out basic context: what the report is, when it runs, and what changes occurred. As models advance, they perform extensive self-verification loops, running through dozens of turns to double-check their own work. Yet, despite these elaborate cycles, agents frequently miss the mark or hallucinate because they lack human judgment and unwritten operational context. A human operator usually has to step in to correct facts or refine preferences.
By the end of this trial-and-error process, a single task might consume 71 turns. Worse yet, if the exact same issue arises again, the agent starts from scratch with a clean slate, repeating the identical sequence of mistakes. Tarang compares this setup to managing a "forgetful helpful intern" who wastes team effort by repeatedly failing to retain what it learned.
- **Granica's Core Focus:** Data intelligence, reducing token/storage costs, and accelerating query latency for enterprise big data.
- **Agent Fundamentals:** Agents excel at tool calling and reasoning, but depend heavily on provided context.
- **The Core Issue:** Tasks like debugging a failing billing report often take upwards of 71 turns due to lack of baseline operational knowledge.
[04:11](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=251s) The Pitfalls of Naive Memory Systems in Production
To prevent agents from repeating mistakes, developers typically introduce memory by placing past session logs and learned operational context at the front of new agent sessions. Initially, this setup seems effective: the agent acts with greater authority, avoids past errors, and completes tasks faster. However, Tarang warns that in production environments, naive memory implementations quickly become a major source of system failure.
Over time, memory continuously grows as past debugging steps, preferences, and multi-user interactions accumulate. Passing months-old instructions—such as how a developer debugged a billing system three months prior—into every new turn inflates the prompt payload. This massive context window causes the agent to lose focus, rendering the memory bloated and useless. Furthermore, memory becomes stale and misleading. Tarang shares a personal example where an agent persistently attempted testing against a three-month-old release build because its stored memory falsely dictated that as the current baseline environment. When team members input conflicting directions, the memory layer becomes polluted, leaving it no better than a clean slate.
Because memory degrades silently, debugging these failures is extremely frustrating. Developers are left wondering whether the underlying model (like Claude 3.5 Sonnet or Opus) suddenly degraded, when in reality, the issue lies entirely within their own memory infrastructure.
- **Memory Growth:** Accumulates unnecessary historical context, inflating token usage across every turn.
- **Stale Context:** Leads agents to operate on outdated facts (e.g., attempting testing on a release from months prior).
- **Silent Failure:** Memory issues manifest as sudden drops in output quality, often incorrectly blamed on model degradation.
[07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) The Economics of Tokens and Architectural Pillars for Agents
Tarang breaks down the financial impact of memory bloat using Anthropic's Claude 3 Opus pricing model for a million tokens. Newly created tokens requested with prompt caching cost $5 per million, while re-using cached tokens drops the cost down to $0.50 per million (a 90% savings). However, writing new tokens that require explicit cache setup can cost between $6.25 and $10 per million. Because of this 20x price variation, an inefficient or poorly managed memory structure can severely punish an organization's budget instead of saving money.
To build sustainable agentic systems, enterprise engineering leaders must balance four key architectural pillars:
1. **Cost:** Keeping token spend predictable rather than letting it balloon uncontrollably.
2. **Quality:** Ensuring developers get optimal outputs by leveraging the best available models without performance degrading due to context bloat.
3. **Governance:** Enforcing security, keeping interactions within internal environments (using Model Context Protocol / MCPs), and adhering to corporate rules.
4. **Portability:** Building flexible infrastructure that allows teams to swap agent harnesses or underlying models without breaking the system.
Tarang highlights a Gartner projection stating that 40% of agentic AI projects are expected to be canceled, though he personally predicts the failure rate will be closer to 80% as teams rush to "move fast and break things." To survive this cycle, enterprises need stable memory and gateway infrastructure rather than ad-hoc setups tied to specific model versions.
- **Anthropic Claude 3 Opus Pricing Variations (per 1M tokens):**
- **Reused Cache Reads:** $0.50
- **Fresh Prompt Cache Creation:** $5.00
- **Uncached / Custom Cache Writes:** $6.25 – $10.00
- **The Four Pillars:** Cost, Quality, Governance, and Portability.
- **Industry Outlook:** Gartner estimates a 40% project cancellation rate for agentic AI, while Tarang anticipates up to 80%.
[011:12] Experiential Learning and Granica’s Myelin System
To address memory failure, Granica developed the concept of *experiential learning*, which it tested with design partners over six months. Unlike traditional black-box memory, experiential learning continuously measures, audits, traces, and tunes the impact of stored context against cost and quality goals. If a new model version makes certain legacy memories obsolete, the system actively prunes them to eliminate payload overhead.
Radhika Ravirala takes over to demonstrate Granica’s system, named **Myelin**, using internal engineering workflow data. She references the 71-turn session scenario where a human inputs a single prompt token while the system re-sends thousands of context tokens on every turn. In the Myelin dashboard, Granica’s production traffic shows that **99% of read requests** were retrieved directly from stored state, with only 1% representing genuinely new work written to the system.
Analyzing where token savings originate, Radhika explains that **78% (roughly three-quarters) of savings come from "cache shaping"**—a process that keeps the prompt prefix stable across turns so that reads stay at the cheap $0.50 rate rather than sliding toward $10. Context compression accounts for another 8% of the savings. By standardizing prefixes and periodically learning from multi-session patterns, Myelin delivers significant token reduction, faster execution speeds, and cross-model portability for complex coding and development workloads.
- **Myelin System Dashboard Metrics:**
- **99% Cached Reads:** 99% of session reads fetched from pre-stored state; only 1% new writes.
- **78% Cache Shaping Savings:** Achieved by maintaining stable prompt prefixes across turns to lock in low-cost cache pricing.
- **8% Compression Savings:** Derived from payload optimization and context compression.
- **Key Takeaways:** Continuous learning across sessions reduces token burn, improves response speed, and stabilizes agent quality across shifting model environments.
## Notable quotes
> Most enterprises treat AI agents as tasks: fire one off, take the answer, discard everything it learned.
> When work and memory survive across sessions, models, and people, the economics flip: you lower spend, raise quality, and move faster at once, not as tradeoffs.
Most enterprises treat AI agents as tasks: fire one off, take the answer, discard everything it learned. That's why cost explodes and quality slips: every session starts cold, rebuilding context it already had and paying for it again. This talk makes the case for treating agents as learned infrastructure instead: persistent, collaborative, continually improving, and compounding in value the more your teams use them. When work and memory survive across sessions, models, and people, the economics flip: you lower spend, raise quality, and move faster at once, not as tradeoffs.
Granica 部署 Myelin 网关，将企业 AI Agent 成本削减 90%
将 AI Agent 视为持久基础设施而非临时任务，可防止上下文退化，并在企业工作流中实现低成本的 Prompt 缓存锁定。
Tarang Vaish 和 Radhika Ravirala 解释了标准的 AI Agent 实现方式如何因反复从头开始或使用简陋记忆系统导致上下文膨胀而浪费资源。他们概述了企业级 Agent 所需的关键架构支柱——成本、质量、治理与可移植性，并详细阐述了不当的 Prompt 缓存管理如何推高 Token 开销。Granica 展示了其 Myelin 系统，指明经验学习与缓存整形如何实现高达 99% 的缓存读取，并大幅降低运营开销。
Granica 专注于为企业数据赋予智能，旨在降低存储成本、减少查询延迟，并优化 Agent 工作流以实现更高的投资回报率。 (0:08)
AI Agent 本身并不聪明；其能力在很大程度上取决于工具调用以及提供给它们的运行上下文。 (0:08)
在缺乏基准运行上下文的情况下，简单的 Agent 调试任务可能会激增至 71 轮交互，既浪费精力又频繁引发幻觉。 (0:08)
将 Agent 视为“健忘且乐于助人的实习生”，会迫使它们在未来相同的任务中重复出现同样的一连串错误。 (0:08)
简陋的记忆系统会随着时间的推移导致上下文膨胀，增大 Prompt 载荷，并导致 Agent 基于陈旧或冲突的事实运行。 (4:11)
Reading summary
Granica co-founder Tarang Vaish and product manager Radhika Ravirala argue that production AI agents fail when burdened by naive memory systems, requiring specialized infrastructure that actively manages context bloat and leverages token caching economics.
Operational Amnesia in AI Agent Workflows (0:08)
Vaish contends that AI agents are not inherently intelligent, but instead depend fundamentally on tool execution and the operational context supplied to them. In complex diagnostic tasks, such as determining why an automated billing report failed, an agent lacking institutional memory must undergo extensive tool-calling sequences and self-verification loops simply to establish baseline facts.
This lack of initial grounding can extend routine debugging sessions to dozens of turns—reaching up to 71 turns in Granica's illustrative scenario. When human intervention resolves the issue, the agent retains none of the operational preferences or discoveries. If the same issue recurs, the agent starts anew with a clean slate, repeatedly committing identical errors like a forgetful intern.
The Silent Failures of Naive Memory Implementations (4:11)
To prevent agents from repeating mistakes, engineering teams frequently prepend past session logs and historical notes directly into prompt prefixes. Vaish warns that while this approach initially seems successful, unmanaged context injection rapidly degrades in enterprise production.
Over time, accumulated multi-user notes and outdated operational instructions inflate prompt payloads. This bloat dilutes the model's focus and introduces stale assumptions, such as an agent persistently testing against a three-month-old release environment. Because this context pollution degrades outputs silently, engineering teams often incorrectly suspect that underlying foundation models—such as Claude 3.5 Sonnet or Opus—have suffered quality regressions.
Token Economics and Architectural Requirements (7:15)
Context bloat carries direct financial consequences due to model pricing tiers. Using Anthropic's Claude 3 Opus pricing as an example, Vaish highlights that reading reused cached tokens costs $0.50 per million, whereas creating fresh prompt cache tokens costs $5.00, and uncached or custom cache writes range between $6.25 and $10.00 per million. Poorly structured prompts trigger a 20-fold cost penalty when cache hits fail.
To build viable agent infrastructure, organizations must address four architectural pillars: cost stability, output quality, governance via secured interaction protocols, and portability across different harnesses and models. Vaish notes a Gartner forecast that 40% of agentic AI initiatives will be canceled, while offering his own estimate that project failure could reach 80% if teams deploy brittle, ad-hoc memory layers.
Experiential Learning and Cache Shaping via Myelin
To replace static memory dumps, Granica developed "experiential learning," an architectural approach that continuously tracks, evaluates, and prunes stored context as model environments change. Demonstrating the company's Myelin system on internal engineering workloads, Ravirala reports that 99% of request reads were successfully served from pre-stored state, with only 1% requiring net-new writes.
Ravirala attributes 78% of the platform's token savings to "cache shaping"—the practice of stabilizing prompt prefixes across sequential turns so queries qualify for discounted cache read rates rather than higher write rates. Context compression delivers an additional 8% in savings, demonstrating that deliberate prefix management can maintain context continuity without compounding prompt costs.
Granica maintains that sustaining enterprise agents requires shifting from raw context accumulation to managed, learned infrastructure. Without systematic prefix management and continuous context pruning, enterprises risk project abandonment driven by compounding token expenses and context pollution.
## Background
本视频由 Ai4 频道于 2026 年 8 月 19 日在 YouTube 平台发布。内容由 Granica.ai 联合创始人兼 CTO Tarang Vaish 与高级产品经理 Radhika Ravirala 共同主讲，重点探讨了企业将 AI Agent 仅当做单次‘任务’执行而导致成本失控与质量下滑的痛点，并倡导将其升级为具备持久记忆与复利价值的‘习得性基础设施’。
## TL;DR
演讲深入剖析了传统 Agent 因缺乏隐性运营上下文而沦为‘健忘实习生’的困境，以及在生产环境中引入记忆时常见的膨胀、陈旧与污染陷阱。针对同等规模 Token 在缓存命中与冷启动间高达 20 倍的成本差距，演讲提出了涵盖成本、质量、治理与可移植性的四大架构支柱。Granica 进而展示了其实践经验学习架构的产品 Myelin，通过前缀缓存整形、上下文压缩与全生命周期审计，实现了高达 99% 的状态读取复用与近 80% 的成本大幅削减。
## Atmosphere & style
整场演讲呈现出典型研发硬核、直面痛点且不落俗套的技术风格。CTO Tarang Vaish 的表达犀利幽默，以‘健忘实习生’和‘静默损坏’直击行业落地痛点，敢于直言实际失败率可能高达 80%；随后产品负责人 Radhika 的切入则沉稳严谨，用详实的数据和产品图谱为降本逻辑提供坚实支撑。两位讲者配合默契，叙事节奏紧凑，在拆解复杂底层经济学的同时展现出开放诚恳的极客交流氛围。
## Key takeaways
- [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) Granica 专注于数据智能化与 Agent 成本优化，旨在降低存储查询延迟与 Token 开支以提升 ROI。
- [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) AI Agent 的核心能力仅在于工具调用与逻辑推理，高度依赖输入上下文，面对空白任务时极易偏离目标与产生幻觉。
- [03:04](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=184s) 缺乏经验记忆的 Agent 宛如健忘的实习生，每次面对相同问题都必须从零排查并消耗大量多轮交互与人工校准。
- [04:22](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=262s) 生产环境中的朴素记忆机制容易静默崩塌，引发记忆无限膨胀、信息陈旧过时以及多工作流交叉污染等严重问题。
- [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) 同等规模 Token 在冷启动全量生成与命中缓存读取之间存在高达 20 倍的成本差距，缺乏治理的记忆会导致严重的 Token 滥用。
- [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) 企业级 Agent 架构必须平衡成本、质量、治理与可移植性四大支柱，避免因过度硬编码导致无法适配新模型和新框架。
- [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) 针对 Gartner 对 Agent 项目 40% 失败率的预测，Granica 认为在浮躁迭代风气下实际失败率可能高达 80%。
- [11:12](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=672s) 经验学习机制要求记忆必须可度量、可审计与可追踪，并在模型升级时自动清理边际效益归零的过载上下文。
- [12:19](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=739s) Granica 的 Myelin 架构通过保持会话前缀稳定实现高频状态复用，其中读操作命中已存状态比例高达 99%。
- [12:19](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=739s) 在系统开销优化中，缓存整形贡献了约 78% 的成本节省，配合数据压缩可将昂贵长上下文调用锁定在低廉成本区间。
- [16:49](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=1009s) 通过将 Agent 视为持续学习的基础设施而非一次性任务，企业能够打破成本与质量的妥协，实现跨模型复用与响应加速。
## Detailed walkthrough
### [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) Granica 的定位与 AI Agent 的现实工作机制
作为当天会议的最后一场压轴演讲，Granica 联合创始人兼 CTO Tarang Vaish 与产品负责人 Radhika Ravirala 共同登台，直奔主题探讨“为什么记忆机制正在让你的 Agent 走向失败”。Tarang 介绍，Granica 是一家专注于为数据注入智能的研发型公司，长期服务于处理数十至数百 PB 级大数据的企业客户，业务涵盖降低存储成本、加速查询延迟，以及优化 Agent 工作流中的 Token 成本以最大化数据投资回报率（ROI）。
谈及 Agent 的运作本质，Tarang 试图打破行业滤镜将其还原为基本逻辑：Agent 本身并没有无所不能的超级智能，其核心强项仅在于两点——“调用工具（Tool Calling）”与“逻辑推理（Reasoning）”。Agent 的表现完全取决于输入上下文的优劣。他举了一个典型的企业运维场景：当被指派去调查“为什么周二以来的夜间账单报表一直失败”时，Agent 面对的是一张完全空白的画布。它必须发起一系列工具调用去艰难探索：“你到底在说什么报表？”、“这份报表到底代表什么？”、“周二之后发生了什么变更？”。即使借助前沿模型的自我验证机制（Self-verification）或开发者搭建的各种 Slash 循环与目标推进逻辑，几轮交互下来，Agent 往往依然偏离靶心、产生幻觉，无法真正完成任务。
```mermaid
flowchart TD
accTitle: 传统无记忆 Agent 的执行循环
subgraph S1["冷启动与盲目探索"]
A["任务输入: 夜间账单失败"] --> B["空白画布 (Blank Canvas)"]
B --> C["连续工具调用与探索"]
end
subgraph S2["多轮消耗与偏离"]
C --> D["自我验证与循环逻辑"]
D --> E{"是否偏离目标 / 产生幻觉?"}
E -- "是 (高达71轮交互)" --> F["人工介入校正事实与偏好"]
E -- "否" --> G["任务勉强完成"]
end
subgraph S3["状态重置"]
F --> G
G --> H["会话结束: 遗忘所有经验"]
H -. "下次遇相同问题".-> B
end
```
---
### [03:04](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=184s) “健忘的实习生”困境与经验记忆的引入
在任务执行遭遇偏差时，系统往往不得不依赖人类介入来校准基调、偏好乃至业务事实。Tarang 指出，这正是因为 Agent 缺失了企业内部沉淀的“运营上下文（Operational Context）”与“人类判断力（Human Judgment）”。这些隐性知识通常未被整理成书面文档供 Agent 检索，必须在实际执行任务的过程中不断学习。
这种机制导致了极大的资源浪费：为了解决一个具体问题，团队可能耗费了整整 71 轮（Turns）交互；而当下一次有人提出完全相同的问题时，Agent 又会从零开始重复这一漫长痛苦的过程。Tarang 将其生动地比作公司里“一个既热心但又极度健忘的实习生（forgetful helpful intern）”，你必须不厌其烦地一遍遍重新教它。为了根治这一痛点，除了优化 Prompt 和选用更擅长工具调用的模型外，核心方案是为 Agent 赋予“经验记忆（Experience）”——将 Agent 在执行任务中踩过的坑、纠正的错误路线以及隐性业务知识持久化沉淀下来，避免重蹈覆辙。
- **核心矛盾**：Agent 具备推理与工具调用能力，但缺乏未被显式编码的企业隐性运营知识与人类判断。
- **冷启动成本**：缺乏记忆的系统在单次排错中可能消耗多达 71 轮模型调用，且知识无法在后续会话中复用。
- **经验资产化**：将排错路径、失败尝试与人工干预转化为结构化经验并存入记忆，是避免重复教学的关键。
---
### [04:22](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=262s) 生产环境中记忆机制的静默崩塌与污染
为 Agent 引入记忆看似完美，但在实际生产落地中却布满陷阱。Tarang 直言：“任何在生产环境中使用过记忆机制的人都知道，这简直就是灾难不断。”在引入记忆的第一个月，系统表现往往十分惊艳——历史经验被置于会话前部，Agent 能够权威地依据既往判断做出精准操作。然而，随着时间推移，记忆机制开始遭遇严重的系统性崩塌：
1. **记忆无限膨胀**：系统不断累积历史判断，导致 3 个月前排查账单的具体指令在每一次后续交互中都被全量塞入 Prompt。Agent 面对庞大冗杂的上下文，无法辨别哪些内容与当前场景相关，最终导致记忆整体失效。
2. **记忆陈旧与过时（Stale Memory）**：过时的信息会被 Agent 奉为圭臬。Tarang 分享了自身的亲历痛点：Agent 牢牢记住了 3 个月前的一个软件版本号，导致后续每次执行测试时都固执地退回到该旧版本，令排查过程陷入极度混乱。
3. **记忆交叉污染**：在调试、开发以及多成员协作等不同工作流中，不同人员对 Agent 输入的偏好与指令相互冲突，导致记忆库被严重污染。
当记忆系统失灵时，它往往是“静默损坏”的。开发者无法直观定位原因，往往在困惑中误以为是底层大模型的能力出现了神秘衰退：“昨天还好好的，难道今天 Claude Opus 莫名其妙降级了？”面对这种混乱，业内甚至有人无奈提出“每隔 6 个月就彻底清空所有累积上下文与记忆，重新开始”的极端做法。
---
### [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) 20 倍 Token 成本悬殊与企业 Agent 架构的四大支柱
记忆管理的混乱直接反映在企业的账单上。Tarang 深入剖析了 Token 调用的经济学模型。以 Claude Opus 为例，同样是 100 万 Token（1M Tokens），在不同调用机制下成本差异巨大：
| 调用模式 | 计费水平（每 1M Tokens） | 场景说明 |
|:--- |:--- |:--- |
| **全量标准生成/输入** | **$5.00** | 每次会话冷启动，全新生成或传入完整上下文 |
| **Prompt 缓存命中读取** | **$0.50** | 长期复用已缓存的高频静态上下文 |
| **缓存显式写入与更新** | **$6.25 - $10.00** | 向模型提供方显式写入新的缓存内容 |
从 $0.50 到 $10.00，同等规模的 Token 调用存在高达 **20 倍** 的成本波动。缺乏治理的记忆系统不仅无法利用缓存降本，反而会导致严重的“Token 滥用（Token Maxing）”，极大地破坏投资回报率（ROI）。
```mermaid
flowchart LR
accTitle: CTO 评估 Agent 系统的四大支柱
subgraph Pillars["企业级 Agent 架构四大支柱"]
direction TB
P1["1. 成本 (Cost)<br/>避免预算无序膨胀，保证支出可预测"]
P2["2. 质量 (Quality)<br/>兼顾记忆沉淀与快速迁移至最新前沿模型"]
P3["3. 治理 (Governance)<br/>内部环境管控、安全边界、MCP与工具合规"]
P4["4. 可移植性 (Portability)<br/>解耦硬编码，从容应对每周迭代的 Agent 框架"]
end
```
面向技术决策者，Tarang 提出了设计 Agent 系统必须权衡的四大支柱：**成本（Cost）**、**质量（Quality）**、**治理（Governance，包括 MCP 工具调用与企业安全边界）**以及最重要的**可移植性（Portability）**。过往团队为了控制前三者往往会将系统过度硬编码（Harden），导致底层出现新的 Agent 框架或模型时无法平滑迁移。针对 Gartner 预测“40% 的 Agent AI 项目将以流产告终”，Tarang 认为该预测甚至过于乐观，在当前“小步快跑、不惜犯错（Move fast and break things）”的浮躁风气下，**实际失败率可能高达 80%**。企业真正需要的是独立于单一模型、能够跨越技术演进周期并经受住时间检验的稳定记忆与网关基础设施。
### [11:12](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=672s) 引入经验学习机制：度量、审计与消除“记忆膨胀”
在当前的 AI 架构中，Agent 的记忆机制往往非常临时且黑盒化，企业很难评估它究竟在多大程度上带来了业务价值。为了打破这种盲目性，Granica 团队与设计合作伙伴在过去六个月中深入探索并推出了“经验学习”（Experiential Learning）概念。其核心逻辑在于：记忆必须能够被精确度量，并且对既定的核心目标产生可衡量的积极影响。如果企业将底层模型升级为更新版本的 Claude Opus，而早期积累的旧记忆已经无法提供有效助益，那么系统就必须果断清理这些失效记忆。
通过对所有记忆进行端到端的追踪（Tracing）、审计（Auditing）与调优，系统可以大幅削减因“记忆膨胀”（Memory Bloat）而引发的昂贵载荷税（Payload Tax），从而实现最大化的成本效益比。
- **记忆的黑盒破局**：摒弃传统 Ad-hoc、缺乏能见度的记忆调用方式，全面量化记忆对系统性能与成本的实际贡献。
- **动态清理与模型适配**：在升级更先进的基础模型时，自动识别并清除边际效益归零的过载上下文，防止无谓的 Token 浪费。
- **精细化审计与调优**：通过全生命周期追踪优化，确保每一份驻留记忆都能在成本与收益之间取得最佳平衡。
```mermaid
flowchart TD
accTitle: Myelin 经验学习与缓存优化架构
subgraph Input["会话输入与状态管理"]
A["多轮 Agent 会话 (如 71 轮重度编码)"] --> B["Myelin 智能记忆层"]
end
subgraph Core["经验学习与优化引擎"]
B --> C["审计与追踪 (清理过载/失效记忆)"]
B --> D["缓存整形 (保持 Prefix 字节稳定)"]
B --> E["上下文压缩"]
end
subgraph Output["降本增效成果"]
D --> F["78% 成本节省 (维持 $0.50 低价读取)"]
E --> G["8% 压缩节省"]
C --> H["99% 读操作命中已存状态 (仅 1% 写入)"]
end
```
### [12:19](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=739s) Myelin 实战解析：99% 状态复用与缓存整形的降本经济学
在典型的 71 轮长会话中，人类工程师往往每轮仅键入 1 个 Token，却被迫在每一轮中重复发送其余 9 个 Token 作为背景上下文。为了解决这一痛点，Granica 的高级产品经理 Radhika Ravirala 展示了其实际工程工作流中的产品 **Myelin**。Myelin 落地了经验学习架构，随着会话次数的增加，系统能够持续学习并降低 Token 消耗与综合成本。
在 Myelin 流经的流量数据中，高达 **99% 的读操作都是直接从已存储的状态中重新读取**，仅有 **1% 是真正写入的新增工作**。这一数据有力证明了 Myelin 能够自主从会话模式中提炼核心上下文，并通过在多轮交互中保持稳定的前缀（Prefix），实现精准的高效召回。从节省来源的分布来看：
- **缓存整形（Cache Shaping）贡献了 78%（约四分之三）的开销节省**：通过保持前缀字节稳定，防止长上下文读取滑向高达 $10 的昂贵支出，将其牢牢锁定在 $0.50 的低廉成本区间。
- **数据压缩（Compression）贡献了 8% 的开销节省**：作为缓存整形的有效补充，进一步削减了载荷体积。
- **经验学习在实际业务中的三大印记**：
1. **超高读复用率**：99% 的复用率证实记忆层在真实运行中生效，而非流于表面的概念包装。
2. **跨模型与跨 Agent 泛化**：无论是不同的底层大模型、各类复杂会话还是不同职责的 Agent，均获得了显著的成本降低与智能增强。
3. **随时间推移的开销优化**：在重度编码（Heavy Coding）等长周期工作流中，基线之上的 Token 开销随复用效率大幅下降，直接转化为会话响应速度的飞跃与显著的财务节约。
### [16:49](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=1009s) 总结与交流互动
Radhika 总结指出，通过深度复用上下文并释放基础设施的学习能力，企业不仅能大幅压降 Token 开销，更能显著提高 Agent 会话的响应敏捷度。Granica 团队最后向与会者开放提问与交流，热忱欢迎业界提供各种维度的真实反馈——“好的、坏的、糟糕的，我们统统收下”（"good, bad, ugly, everything. Remember, we catch them"），期待与开发者共同打磨下一代 Agent 基础设施。
## Notable quotes
> [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) 为什么记忆机制正在让你的 Agent 走向失败 — Tarang Vaish
> [00:08](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=8s) 你到底在说什么报表？ — Tarang Vaish
> [03:04](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=184s) 一个既热心但又极度健忘的实习生（forgetful helpful intern） — Tarang Vaish
> [04:22](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=262s) 任何在生产环境中使用过记忆机制的人都知道，这简直就是灾难不断。 — Tarang Vaish
> [04:22](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=262s) 昨天还好好的，难道今天 Claude Opus 莫名其妙降级了？ — Tarang Vaish
> [04:22](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=262s) 每隔 6 个月就彻底清空所有累积上下文与记忆，重新开始 — Tarang Vaish
> [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) 40% 的 Agent AI 项目将以流产告终 — Gartner
> [07:15](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=435s) 小步快跑、不惜犯错（Move fast and break things） — Tarang Vaish
> [16:49](https://www.youtube.com/watch?v=aPu5p7t3NYM&t=1009s) good, bad, ugly, everything. Remember, we catch them — Radhika Ravirala
## Mentioned
Granica.ai, Tarang Vaish, Radhika Ravirala, Claude Opus, Gartner, Myelin, MCP
## Video description
Radhika Ravirala - Senior Product Manager, Granica.ai
Tarang Vaish - Co-Founder & CTO, Granica.ai
Most enterprises treat AI agents as tasks: fire one off, take the answer, discard everything it learned. That's why cost explodes and quality slips: every session starts cold, rebuilding context it already had and paying for it again. This talk makes the case for treating agents as learned infrastructure instead: persistent, collaborative, continually improving, and compounding in value the more your teams use them. When work and memory survive across sessions, models, and people, the economics flip: you lower spend, raise quality, and move faster at once, not as tradeoffs.
[Source](https://www.youtube.com/watch?v=aPu5p7t3NYM)
Source quotes
[0:08] Tarang Vaish: 为什么记忆机制正在让你的 Agent 走向失败
[0:08] Tarang Vaish: 你到底在说什么报表？
[3:04] Tarang Vaish: 一个既热心但又极度健忘的实习生（forgetful helpful intern）
[4:22] Tarang Vaish: 任何在生产环境中使用过记忆机制的人都知道，这简直就是灾难不断。
[4:22] Tarang Vaish: 昨天还好好的，难道今天 Claude Opus 莫名其妙降级了？
[4:22] Tarang Vaish: 每隔 6 个月就彻底清空所有累积上下文与记忆，重新开始
[7:15] Gartner: 40% 的 Agent AI 项目将以流产告终
[7:15] Tarang Vaish: 小步快跑、不惜犯错（Move fast and break things）
[16:49] Radhika Ravirala: good, bad, ugly, everything. Remember, we catch them
Source quotes
Most enterprises treat AI agents as tasks: fire one off, take the answer, discard everything it learned.
When work and memory survive across sessions, models, and people, the economics flip: you lower spend, raise quality, and move faster at once, not as tradeoffs.
[0:08] Tarang Vaish: forgetful helpful intern
[7:15] Tarang Vaish: move fast and break things
Original transcript
[0:08] Hey, thank you everyone. You know, last session of
the day, we'll make it very exciting and you know,
[0:13] we'll introduce Granica. Our talk is about
how memory is failing your agents and to I'm
[0:20] accompanied with Radhika. She is heading product
for Granica. Myself, I'm the CTO and co-founder at
[0:26] Granica. Uh briefly about us, we are a research
and development company. We bring intelligence
[0:33] to your data. You know, we want to make your
data more intelligent for the applications,
[0:37] the use cases that you are driving through that.
U so we work with big data companies processing
[0:43] tens or hundreds of pabytes of data, bring down
your storage cost, improve the query latency,
[0:49] query uh acceleration. uh we help with your
agentic workflows, cut down your token cost
[0:56] and just generally improve the ROI you know that
you want to get. You want to maximize the ROI
[1:01] from your own data for different applications that
you may have. Okay, so let's begin with you know
[1:09] how agents work and how you accomplish tasks
with your agents. Let's dump it down, right?
[1:15] An agent is not super smart. What it is good at
is it's good at calling tools and it is good at
[1:22] reasoning. Right? Those are two essential things.
The right context that you pass into your agent,
[1:27] the better it is able to do. So I'll give an
example where you know I want to accomplish a task
[1:33] that my nightly billing report has been failing
and since Tuesday and I want to find out why,
[1:39] right? So the agent has to start with a blank
canvas here. It has to discover with a series of
[1:45] tool calls. What the hell are you talking about?
Right? Uh what does this report even mean? Uh what
[1:51] is happened since Tuesday? You know, when do I run
it? Do I need to verify whatever I'm doing? The
[1:57] more and more you are using advanced models, they
are trying to do self-verification in this process
[2:02] to actually understand, you know, I am giving it
route right results or not. people have built like
[2:08] slash loops and you know lot of different goal
mechanics to actually make sure sufficient turns
[2:14] are taken with the model to move to the next step.
But guess what? After all that happens, you know,
[2:20] you realize that uh the agent has completely
missed the mark. It is hallucinating or it is
[2:26] not accomplishing the task that you needed it to
do. And a human intervention is often needed to
[2:32] actually set the tone or the set the preferences
or set even the facts correct. Why? You know if I
[2:38] wonder the reason is that these agents are lacking
the operational context that you have and the
[2:44] human judgment that you have and the intelligence
that your whole corporation you know your
[2:49] organization has built in. You know that's where
humans are still relevant. They have the judgment.
[2:53] They have the operational context that is not
written somewhere that the agent can just discover
[2:58] always right. So these are things that the agent
has to learn through the journey of accomplishing
[3:04] a task. So what happens in the end like you know
you get the task done but you may have incurred
[3:10] 71 turns in getting there and the next time if you
start with the same exact question the same exact
[3:17] stuff it's going to repeat the same process again.
Right? So why is that? because the agents lack any
[3:23] memory. You know, they start with a clean slate
and you're going to waste your effort in teaching,
[3:28] you know, this forgetful helpful intern that you
have in know in your organization. So, what is
[3:34] a common solution? You can improve your context.
You can use better models which can do better tool
[3:40] choices and judgment. But essentially what you
want to do is build some memory into your give the
[3:46] agent some memory that's going to continue to fix
the all the wrong turns it took all the mistakes
[3:53] it did and all the operational knowledge that you
know you could not just write it somewhere that
[3:58] you had to do when the agent actually accomplishes
a task. So we call it experience that you know you
[4:04] have to actually store the experience that the
agent had while accomplishing tasks into memory.
[4:11] Okay. Okay, so great. Memory solves a very vital
problem where a simple task can take a long time
[4:16] because that's how agents are designed under
the covers. Okay, so you gave the agents memory
[4:22] then what happened over time, right? So in the
first month the agents are working well. They
[4:27] are accomplishing this by actually putting
the memory in the front of the session. So
[4:32] they are more authoritative about what's going
to happen next and all your previous judgment,
[4:37] your experiences have been stored in your session.
So it can continue to use that to do better better
[4:44] uh you know operations. But what happens there is
that over time memory gets into a lot of trouble.
[4:51] Anybody who is using memory in production
knows that it is just full of trouble and
[4:56] uh I will name some of those. You know one is that
memory is continuously growing over time. So you
[5:02] know all your judgments are accumulating and you
have to send this entire instruction set of how
[5:09] I debugged my you know billing system one day uh 3
months back still is being passed on to every turn
[5:16] that your agents are doing because it doesn't
know what's relevant and what it's not right.
[5:20] So your memory becomes so big that the agents
start actually only focusing on parts of the
[5:25] memory. So your memory also becomes useless over
time because the agent has just too much context
[5:31] of memory that it's looking at. Very uh like more
troubling things start happening where the memory
[5:38] starts getting stale and it starts remembering
things that are no longer valid and because it
[5:43] thinks that it is more authoritative it can take
directions which are no longer valid. Uh one of
[5:49] the things I experienced is that the agent thought
that we are on a certain release which happened
[5:54] three months back and it would always go back and
try to use that release you know to do its testing
[6:00] and you know I was left in utter confusion like
why is the agent thinking that you know we are in
[6:06] this time frame even though a lot of context has
moved. So debugging these challenges with memory
[6:11] is very hard. Over time uh more troubling things
happen that it is leading to like mixture of uh
[6:19] judgments that happen in different workflows like
a debugging or a developer or your team members
[6:24] told uh your agents different things and it gets
polluted right so in the end memory actually
[6:31] becomes no better than what you started with.
And you know some people have said that actually
[6:37] just delete whatever you know context and memory
that you have accumulated in your last months 6
[6:42] months because it's actually not going to help and
it's better to you know start with a clean slate.
[6:47] So these are interesting challenges and but the
trouble is that memory always breaks silently you
[6:54] know when it's not working you know you are just
left in utter confusion like you know this thing
[6:58] was working yesterday what changed today did opus
just magically degrade itself and you know I would
[7:04] blame them most of the time but turns out okay
you know it's your own memory infrastructure that
[7:08] has been giving you all this trouble um so let's
dive deeper into the economics of this trouble
[7:15] um let's talk about token usage message, right?
So, this is the same million tokens that Opus
[7:21] publishes and they have three different price
points. There is a price point where you just
[7:26] have the tokens freshly created and you are asking
uh Anthropic to cash those tokens, it's going to
[7:33] cost you $5. But if you keep reusing those tokens,
it's going to only cost you 50. But if you want to
[7:40] uh write new uh tokens where you want anthropy to
specifically cash, it can be $6.25 $25 to $10. So
[7:48] the same million tokens can have a 20x variation
in how much it costs you. And memory could have
[7:55] really helped here, but it could also cost you
very dearly because it's very workflow dependent.
[8:00] Right? So there is a lot of room for optimization
here, but there is a lot of problems that we just
[8:07] talked about. Right? So moving on you know as a
CTO as a business leader you know we all have to
[8:15] think about how do we design our systems you know
correctly because you can end up with token maxing
[8:20] and you know getting worst ROI and you need to
think through you know what problems you are
[8:25] trying to solve that you know you you are here
for right so uh so to me like these are these
[8:30] four pillars you know first is cost right you want
the cost to not not balloon you know you're you're
[8:36] happy to give uh AI some budget and you know
continue to increase based on the productivity
[8:41] you are getting but you also don't want it to be
unpredictable right the second thing is quality
[8:46] you know this memory uh can really improve
your quality but can also really degrade your
[8:52] quality you want to make sure your developers are
getting the best quality so you know you're always
[8:56] switching to the le the next model the moment it
shows up this is you know because everybody wants
[9:02] the best quality you know and it goes better than
cost in most of the scenarios The next thing is
[9:08] governance. There is a lot of security aspect to
it. But it is also an aspect of keeping the system
[9:14] within its own environment. Uh like you know you
want to make sure the internal tools are used,
[9:18] the MCPS are used. You want to have the internal
context and rules and regulations baked into what
[9:25] your agents need to do. And finally, I think the
most important is portability because the moment
[9:31] you start thinking about the left three, uh, you
know, you start hardening your agentic systems
[9:35] and the moment you start hardening your agentic
systems, it becomes very hard to move to a new,
[9:40] you know, like a agentic harness
that might be showing up, you know,
[9:44] uh, uh, uh, in the news that day. So what it means
is that you know you want to make sure your agents
[9:51] are agentic systems are flexible enough that they
can go through the test of time which is almost
[9:57] every week you know in this world. So so how do
we do that right? So what do we ideally need is
[10:04] not something that is uh specifically optimizing
certain turns or it is specifically optimizing
[10:11] uh using a Kim K3 model. Right? You want something
that can work cohesively and give you a memory and
[10:18] a gateway infrastructure altogether that can stay
steady even if you know your models are changing
[10:24] your governance is increasing or you know you are
trying to optimize your cost u so I would also add
[10:31] like Gartner you know specifically because people
don't care about all these aspects or are not able
[10:36] to tackle all of these when they jump into you
know their agentic journey uh you know 40% of
[10:42] the agentic AI projects are still expected to be
cancelled and I would say they are very generous.
[10:47] I would expect 80% to get cancelled you know
personally everybody's in the mode of like move
[10:52] fast break things right in this world so I would
say 80% of the AI projects are supposed to get
[10:58] cancelled but what you want is a steady hand of
memory and infrastructure that can ride along with
[11:05] you in this journey and that can stay through the
test of time you know of how things are evolving.
[11:12] So this is where we introduced the concept of
experiential learning. This has been something
[11:17] that you know Granica internally has been working
on the last 6 months and we have been working with
[11:22] our design partners on this. What we mean here
is that typically memory is very ad hoc. It is
[11:28] very blackbox and you don't really know how it's
working for you. uh granica works on actually
[11:34] measuring the impact of memory that is you know
that is being leveraged by your agents means that
[11:40] it has to actually have a measurable impact
on the four goals that I talked about so for
[11:45] example if you move to a newer model of opus and
your previous memories are no longer uh helping
[11:51] you it's better to get rid of them you know so the
payload tax that you're giving uh getting because
[11:57] you have so much memory bloat is is dramatically
reduced if you start measuring the impact act of
[12:02] all your memories. You start tracing them. You
start auditing them and you're also a optimizing
[12:08] and tuning them for the maximum cost benefit that
you can get out of it. So to show this in action,
[12:14] you know, I will invite Radhika to take over
and show you, you know, how this impacts real
[12:19] world scenario. Thank you, Tarang. So, Tarang has
just showed you what a 71 turn session looks like
[12:26] and where a human types just one token whereas the
other nine are actually resent for every turn. So,
[12:37] I want to show you what this looks like in
uh in practice when you don't have to pay for
[12:43] the recent context. So this is a dashboard of our
engineering workflows at uh Granica where u uh we
[12:51] have a product called Milin which uh essentially
implements the experiential learning and helps you
[12:58] uh save cost on your token consumption as well
as uh um decreasing your u cost overall cost and
[13:09] um also improving this experiential learning as
you go through several sessions. So if we look at
[13:18] uh the dashboard uh let's start with uh the the
cache of all the traffic that is flowing through
[13:26] milein you will notice that 99% of the reads
were reread from our stored state and only 1% was
[13:36] genuinely new work that was written and what this
shows you is that we have enabled milin to learn
[13:46] from the sessions, store the most relevant context
that is uh important [clears throat] uh for the
[13:54] session and retrieve it by keeping the prefix
uh uh stable across many terms. And so if you
[14:03] look at the distribution of where the savings are
coming from, you'll notice that 3/4 of the savings
[14:09] are coming from cash shaping. This is where we
keep uh the prefix stable across turns so that
[14:17] the 50 cent reads that Tarang mentioned earlier
stays 50 cents and you're not sliding towards the
[14:24] $10 uh cost that you will incur. So uh by keeping
the bite stable uh and uh keeping the relevant
[14:33] context we are able to achieve uh 78% of uh the
cash shape shaping uh savings through myelin uh
[14:42] compression is a a small slice of that 8% of it
is coming from compression uh and so what what uh
[14:50] the key takeaway from from this uh uh dashboard
is that myelin is periodically learning from the
[15:00] sessions and the patterns that it sees across your
sessions and it is improving the intelligence of
[15:08] the memory layer and uh if you if I if we talk
about the experiential learning in reality it
[15:15] is manifesting its fingerprints in in three
areas. So if you look at uh the usage again, it
[15:22] is manifesting as the percentage of uh the reads
which is 99%. What this is telling us is this is
[15:30] not a memory portrayal. It is the memory that is
uh actually working in reality. And uh if you look
[15:41] at the models and agents uh that are actually uh
consuming these uh saved state and benefiting you
[15:51] will see that across the board across the
models across the sessions and across the
[15:56] agents there is a significant uh cost reduction
as well as the improvements that are coming from
[16:03] this intelligence that we have built. um and uh in
addition to that uh your spend versus the baseline
[16:11] over time will decrease depending upon the
workloads. So what we have noticed in reality is
[16:19] that as you engage uh your agents in heavy coding
sessions, you will see a significant reduction in
[16:27] in terms of how the tokens are consumed by
uh very effectively reusing the context and
[16:35] uh leveraging that intelligence that we are
building in order to uh realize the byproducts
[16:41] of of uh this efficiency which is which is cost
savings. and uh fastness uh of your sessions. So
[16:49] that's really the glimpse and and the demo of
what we are building at uh Granica. We welcome
[16:55] your questions, your comments, your feedback,
uh good, bad, ugly, everything. Remember,
[17:01] we catch them. So please send it our way and uh
we look forward to hearing from you. Thank you.
