Rasa CTO Warns 50% of Enterprise AI Agent Code Expires Every 18 Months

Palm Reading and Other Skills: Agent Architectures in 2027 With Rasa — Ai4

Ai4

Enterprises get trapped in perpetual rewrites because foundation models rapidly subsume runtime scaffolding, requiring organizations to shift focus to reusable velocity layers and federated authoring to scale production deployments.

AI-assisted summary of Palm Reading and Other Skills: Agent Architectures in 2027 With Rasa — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by Ai4 on August 15, 2026, this presentation features Alan Nichol, CTO and co-founder of Rasa. According to the description, the session explores the architectural choices made during the first 90 days of building enterprise AI agents, breaking down which architectural bets pay off and what separates resilient production deployments from doomed initiatives in regulated industries.

## TL;DR Alan Nichol addresses the central tension in enterprise AI: engineering teams are proud of shipping complex scaffolding, while business stakeholders remain frustrated by slow delivery velocity. He explains that foundational models continuously 'eat the stack,' giving custom runtime code an 18-month half-life and rendering low-level scaffolding obsolete. To build durable systems, Nichol proposes a three-tier architecture that shifts focus away from ephemeral model wrappers and undifferentiated compliance toward a high-ROI 'Velocity Layer.' This layer empowers domain experts through federated development, composable skills, reasoning-level observability, and an autonomous flywheel that simulates fixes on real production conversation logs.

## Atmosphere & style Pragmatic, experienced, and wryly humorous, Nichol delivers a grounded reality check to enterprise AI hype, opening with fortune-teller anecdotes and the relatable metaphor of the messy 'first pancake.' The talk balances technical architectural critique with sharp organizational empathy, charting the evolution from single-token context windows to modern agent tool-calling. Listening feels like receiving candid, hard-won advice from an industry veteran who has seen chatbot and LLM paradigms come and go, offering a clear blueprint to escape the maintenance hamster wheel.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) A central tension in enterprise AI exists between engineering teams proud of complex infrastructure and business leaders frustrated by slow use case delivery. - [01:46](https://www.youtube.com/watch?v=2hO01YhyWCo&t=106s) Foundational models consistently 'eat the stack' over time, natively subsuming custom code, state tracking, and dialogue management layers previously built by hand. - [01:46](https://www.youtube.com/watch?v=2hO01YhyWCo&t=106s) Much of today's custom agent scaffolding will be rendered obsolete by future model iterations, mirroring past architectural shifts like the waning indispensability of vector search. - [04:37](https://www.youtube.com/watch?v=2hO01YhyWCo&t=277s) Enterprise AI stacks operate with an 18-month half-life, meaning roughly half of custom runtime plumbing becomes redundant within a year and a half. - [04:37](https://www.youtube.com/watch?v=2hO01YhyWCo&t=277s) Focusing solely on low-level runtime capabilities traps engineering teams in a continuous cycle of rewriting scaffolding rather than driving business value. - [04:37](https://www.youtube.com/watch?v=2hO01YhyWCo&t=277s) An enterprise agent platform is structured across three tiers: the rapidly deprecated Capabilities Layer, the table-stakes Enterprise Readiness Layer, and the high-ROI Velocity Layer. - [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) Sustainable engineering requires composable skill catalogs, permanent structural fixes over prompt-and-pray tweaks, and reasoning-level counterfactual observability. - [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) Business stakeholders require data-driven roadmaps and bi-directional metric drilldowns without being blocked by Jira tickets for conversational adjustments. - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) A federated development model empowers non-coding domain experts to directly shape and refine agent behaviors. - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) A self-improving flywheel automatically detects production coverage gaps, suggests fixes, and verifies them via historical conversation simulations before human deployment. - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) True platform success is measured by the velocity of scaling beyond the messy 'first pancake' agent to making each subsequent use case exponentially faster and cheaper.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) Mystic Mona, the Enterprise Dragon, and the Delivery Tension

Alan Nichol opens his talk with an anecdote about Mystic Mona, a fortune teller stationed at the Rasa booth. Mona shared two predictions with him earlier that morning: first, that every single person in the room is fighting the exact same dragon; second, that his presentation was destined to bomb. As the founder and CTO of Rasa, a platform used by some of the world's most demanding enterprises to build customer- and employee-facing AI agents, Nichol notes that Rasa has been building in this space "since well before ChatGPT" and doing chatbots "since before they were good."

Through his regular conversations with IT leadership—including CTOs, CIOs, and Chief AI Officers—Nichol observes that the industry has moved far beyond early proofs of concept and experimentation into establishing internal agent platforms. However, this progress has triggered a sharp structural tension within organizations:

- **The Engineering Perspective:** Technical teams are rightfully proud of the sophisticated architecture, tool-calling pipelines, and agents they have engineered and shipped. - **The Business Perspective:** Leadership and business stakeholders remain chronically dissatisfied, consistently complaining that use cases and new production agents are not going live fast enough.

```mermaid flowchart TD accTitle: The Enterprise AI Agent Organizational Tension subgraph S1["Engineering Reality"] A["Custom Agent Scaffolding"] --> B["Pride in Shipping Complex Infra"] end subgraph S2["Organizational Friction"] B -.->|"The Dragon"| C["Delivery Velocity Bottleneck"] end subgraph S3["Business Pressure"] D["Need for Fast Production Use Cases"] --> C C --> E["Tension: 'Why are we not in production faster?'"] end ```

---

[01:46](https://www.youtube.com/watch?v=2hO01YhyWCo&t=106s) How Models Eat the Stack: From Single Tokens to Tool Calling

To illustrate why this organizational friction exists, Nichol poses a seemingly absurd question: is it possible to build an AI agent with a one-token context window? When Rasa was founded a decade ago, that was the literal reality. Early language models could only inspect a single token at a time to predict a specific attribute. Everything required to make that model function as an interactive agent—handling multi-turn context, dealing with follow-up queries, tracking state, invoking backend APIs, and applying business logic—demanded massive amounts of custom, handwritten code. The model's sole responsibility was matching user utterances into one of a few hundred predefined buckets called "intents" (e.g., greetings, affirmatives, refund requests).

Over the subsequent years, foundational models did not merely grow in size; they aggressively began "eating the stack" around them. When BERT arrived—treated at the time as a large language model—it allowed systems to evaluate entire sentences in context, natively resolving complex linguistic structures like negation and instantly rendering large swaths of manual parsing code obsolete. Next, GPT-3 expanded context windows across entire transcripts, and subsequent generations introduced native tool-calling capabilities. One by one, components that developers spent years writing by hand—state tracking, dialogue management policies, and predefined response templates—were subsumed by the underlying models.

Despite modern agents appearing to be nothing more than an LLM invoking tools in a loop, production deployments today are still surrounded by fragile, handwritten scaffolding. Nichol warns that history will repeat itself: foundational models will inevitably consume these bespoke layers in upcoming iterations. A prime example is the frenzy around vector databases and vector search, once marketed as an indispensable pillar for RAG and agent architectures, only for teams to discover that conventional search methods perform just as well or better for most domains, including code search.

---

[04:37](https://www.youtube.com/watch?v=2hO01YhyWCo&t=277s) The 18-Month Half-Life and the Three-Tier Agent Platform

Because the pace of model innovation has accelerated from multi-year cycles to a matter of months, Nichol formulates an operating rule for engineering leaders: "Your AI stack has a half-life of 18 months." Roughly 50% of the custom runtime scaffolding built today will become completely unnecessary in a year and a half, yet engineering teams cannot predict which half will survive. Focusing solely on immediate runtime plumbing locks organizations into an exhausting "hamster wheel," forcing them to perpetually rewrite components just to stay aligned with newer model releases.

To break this cycle, Nichol conceptualizes an enterprise agent platform as a three-tiered cake:

```mermaid flowchart TD accTitle: Three-Tier Enterprise Agent Platform Architecture subgraph T3["Top: Velocity Layer (High ROI / Differentiator)"] V1["Reusable, Composable Skills"] V2["Autonomous Learning Flywheel"] V3["Domain Expert Authoring (No Jira)"] V4["Causal & Counterfactual Observability"] end subgraph T2["Middle: Enterprise Readiness Layer (Table Stakes)"] E1["Security, Compliance & Guardrails"] E2["Role-Based Access & Governance"] E3["Non-Functional Requirements"] end subgraph T1["Bottom: Capabilities Layer (Rapidly Deprecated)"] C1["Runtime Model Wrappers & Scaffolding"] C2["Handwritten State Tracking & Dialogue Logic"] C3["Prompt Engineering Workarounds"] end

T1 --> T2 --> T3 ```

- **Tier 1: The Capabilities Layer (Bottom):** Encompasses everything in the immediate vicinity of the model at runtime (prompt wrappers, state glue, tool orchestration). It provides short-term functionality but ages poorly and is repeatedly eaten by model updates. - **Tier 2: The Enterprise Readiness Layer (Middle):** Encompasses governance, compliance, security, and non-functional requirements. While essential for enterprise production, it is completely undifferentiated. - **Tier 3: The Velocity Layer (Top):** The critical layer that multiplies enterprise ROI, ensuring that building the 10th agent or use case is exponentially faster and cheaper than building the first.

---

[06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) Engineering and Business Requirements for a High-Velocity Layer

A well-architected velocity layer addresses the distinct day-to-day requirements of both engineering teams and business stakeholders, transforming how agents are maintained and diagnosed:

#### For Engineering Teams - **Composable, Reusable Skills:** Access to an enterprise-wide, pre-vetted catalog of capabilities (such as authentication, transactional workflows, or even palm reading) that can be plugged across multiple agents with guaranteed compatibility. - **Permanent Fixes vs. "Prompt and Pray":** Naive implementations tweak prompts to patch single edge cases, inadvertently creating regressions elsewhere. Robust architectures ensure that when a bug is fixed once, it stays fixed. - **Reasoning-Level Observability:** Standard low-level LLM tracing (inspecting raw input and output strings) provides excessive micro-detail without actionable insight. Teams need structural observability into *how* the agent reasoned—specifically understanding why a decision was made and what counterfactual condition would have altered the trajectory.

#### For Business Stakeholders - **Data-Driven Roadmaps:** Roadmaps grounded in real production conversations rather than speculation, complete with a continuous learning flywheel. - **Bi-Directional Metric Drilldowns:** The capacity to look top-down from high-level business KPIs into the exact interactions driving them, as well as bottom-up from an isolated anomalous conversation to determine if it represents a widespread pattern. - **Bypassing the Jira Bottleneck:** Transitioning away from legacy workflows where every minor conversational tweak or business logic adjustment requires filing an engineering ticket in Jira.

---

[08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) Federated Development, Self-Improving Workflows, and Long-Term ROI

To achieve true organizational velocity, enterprises must adopt a federated development model. Rather than forcing all agent requests through a centralized IT bottleneck, domain experts—who understand business processes intimately but do not write code—must be empowered to shape agent behaviors directly.

A modern velocity layer acts as an autonomous feedback loop that continuously processes live production logs through a three-stage lifecycle: 1. **Proactive Gap Detection:** Analyzing incoming enterprise conversations to automatically highlight unhandled user intents and failure points. 2. **Suggested Remediation:** Drafting concrete architectural or logic adjustments to address the identified gap without starting from scratch. 3. **Historical Simulation:** Running rigorous simulations against past conversation datasets to prove the change resolves the issue without introducing regressions, presenting a verified recommendation for human review.

```mermaid flowchart LR accTitle: Autonomous Learning and Simulation Flywheel A["Live Production Logs"] --> B["Detect Coverage Gaps"] B --> C["Generate Proposed Fix"] C --> D["Run Simulations on Historical Data"] D --> E["Domain Expert Review & One-Click Deploy"] E --> A ```

Nichol emphasizes that Rasa’s platform architecture was intentionally designed to respect the substantial investments enterprises have already made over recent years. Instead of demanding a destructive "rip and replace," the platform allows teams to "color outside the lines" and customize or override individual components.

He notes that the first enterprise agent is always like "the first pancake"—slow, messy, and experimental. Long-term success is measured by how quickly subsequent use cases reach production. When built on real customer data, vetted by business experts, and hardened from day one, stakeholder sentiment fundamentally flips from *"Why are you so slow?"* to *"We love this. Can we have more? How do we scale this out?"* Nichol concludes with a tongue-in-cheek nod to Mona, summarizing the audience's AI fortune in five words: *"Find me at the booth."*

## Notable quotes > [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) since well before ChatGPT — Alan Nichol

> [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) since before they were good. — Alan Nichol

> [01:46](https://www.youtube.com/watch?v=2hO01YhyWCo&t=106s) eating the stack — Alan Nichol

> [04:37](https://www.youtube.com/watch?v=2hO01YhyWCo&t=277s) Your AI stack has a half-life of 18 months. — Alan Nichol

> [04:37](https://www.youtube.com/watch?v=2hO01YhyWCo&t=277s) hamster wheel — Alan Nichol

> [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) Prompt and Pray — Alan Nichol

> [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) the first pancake — Alan Nichol

> [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) color outside the lines — Alan Nichol

> [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) Why are you so slow? — Alan Nichol

> [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) We love this. Can we have more? How do we scale this out? — Alan Nichol

> [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) Find me at the booth. — Alan Nichol

## Mentioned Alan Nichol, Rasa, Mystic Mona, Ai4, ChatGPT, BERT, GPT-3, Jira

## Video description Alan Nichol - CTO & Co-founder, Rasa

Most enterprise AI agents being built right now won't make it to durable production. Not because the models aren't good enough, but because the architectural choices made in the first 90 days quietly seal the project's fate.

Alan Nichol, founder of Rasa, has spent over a decade reading the leaves: watching enterprise agents get built through the chatbot era, the LLM shift, and now the agent era. In this session, he'll share the pattern recognition: which architectural bets pay off, which ones are cursed from the start, and what separates the agents running at the world's largest banks, telcos, and healthcare companies from those that never made it out of the cards.

If you're building or operating AI agents in a regulated enterprise, this talk will tell you with uncomfortable specificity what your agent's future holds — and how to change it.

[Source](https://www.youtube.com/watch?v=2hO01YhyWCo)

中文简报

Rasa CTO警告：50%的企业AI Agent代码每18个月就会失效 由于基础模型正迅速吞噬运行时脚手架，企业容易陷入无休止重写的困境；若要实现生产级规模化部署，组织必须将重心转向可复用的速度层与联邦化开发。 Alan Nichol 剖析了企业 AI 领域的核心矛盾：工程团队热衷于交付复杂的底层脚手架，而业务干系人却对缓慢的交付速度倍感沮丧。他解释称，基础模型正持续“吞噬技术栈”，导致自定义运行时代码的半衰期仅有 18 个月，并使底层脚手架迅速被淘汰。为了构建具备持久生命力的系统，Nichol 提出了一套三层架构，主张将重心从短暂易逝的模型包装器和同质化的合规层，转移到高投资回报率的“速度层”（Velocity Layer）。该层级通过联邦化开发、可组合技能、推理级可观测性，以及能够利用真实生产对话日志模拟修复的自主飞轮，全面赋能业务领域专家。 企业级 AI 当前普遍存在工程团队以自建复杂基础设施为傲，而业务高管则对用例上线速度迟缓极为不满的结构性矛盾。 (0:04) 基座模型持续“吞噬”外围技术栈，使得大量手动编写的状态跟踪与对话管理层被原生能力取代，企业 AI 技术栈的半衰期仅为 18 个月。 (1:46) 现代企业级 Agent 平台划分为三层架构：快速被模型淘汰的能力层、作为准入门槛的合规就绪层，以及能倍增商业 ROI 的敏捷交付速度层。 (4:37) 实现可持续工程落地需要可组合的技能库与推理级可观测性，同时摆脱事事提交 Jira 工单的桎梏，赋能领域专家基于真实数据持续迭代优化。 (6:52)

Reading summary

Rasa founder and CTO Alan Nichol argues that enterprise AI agent initiatives stall because engineering teams over-invest in runtime plumbing that advancing foundation models quickly render obsolete. To achieve sustainable deployment velocity, he contends organizations must build agent platforms around an enduring layer of reusable skills, causal observability, and autonomous learning loops driven by domain experts. The Enterprise Delivery Bottleneck and Scaffolding Friction (0:04) As enterprises advance from isolated proofs of concept to centralized agent platforms, an organizational divide emerges between technical teams and executive leadership. Engineering teams often celebrate the deployment of complex, bespoke agent architectures and tool-calling pipelines. In contrast, IT executives and business stakeholders express ongoing frustration with the slow pace at which new production use cases and agents go live. Nichol identifies this delivery bottleneck as a shared enterprise struggle. Technical organizations find themselves constrained by custom runtime scaffolding, creating friction where technical pride in architectural complexity conflicts with executive demands for rapid business-facing delivery. Model Evolution and the 18-Month Stack Half-Life (1:46) Historically, foundational models have systematically absorbed the handwritten code surrounding them. A decade ago, models evaluated only a single token at a time to predict predefined intent buckets, requiring massive manual codebases to manage dialogue state and backend orchestration. Successive breakthroughs—such as BERT resolving sentence-level structures like negation, GPT-3 widening transcript context, and newer models adopting native tool-calling—continually rendered custom parsing and state tracking obsolete. Because model capabilities advance on compressed schedules, Nichol formulates an operating rule: enterprise AI stacks have a half-life of roughly 18 months. Approximately 50 percent of the custom runtime scaffolding built today will become redundant in a year and a half, yet engineering teams cannot predict which half will survive. He cites the market frenzy around dedicated vector search—which many teams discovered performed no better than conventional search methods—as evidence that over-indexing on temporary architectural layers traps teams in a perpetual rewrite cycle. Structuring Platforms Around an Enduring Velocity Layer (4:37) To escape the cycle of rewriting runtime plumbing, Nichol proposes separating an enterprise agent platform into three distinct tiers. At the bottom lies the Capabilities Layer, encompassing model wrappers, prompt engineering, and state orchestration; this layer provides immediate utility but suffers from rapid deprecation. The middle Enterprise Readiness Layer provides necessary table stakes—governance, compliance, security, and non-functional requirements—which are essential but offer little competitive differentiation. The top tier, designated the Velocity Layer, serves as the primary multiplier of enterprise return on investment. Rather than reinventing infrastructure for every use case, this layer ensures that building subsequent agents is exponentially faster than building the first. It relies on composable, reusable skill catalogs—spanning transactional workflows to niche capabilities—with guaranteed compatibility across different enterprise deployments. Federated Authoring and the Autonomous Learning Flywheel (8:40) A viable velocity layer replaces fragile engineering habits, such as reactive prompt tweaks that create unforeseen regressions, with reasoning-level observability. Instead of inspecting raw token strings, developers require structural insight into why an agent executed a specific decision and what counterfactual condition would have shifted that path. Concurrently, business leaders need bi-directional metric visibility to trace high-level business indicators directly down to specific customer interactions. To bypass centralized engineering backlogs, the architecture must support federated authoring, enabling non-coding domain experts to adjust conversational logic without filing Jira tickets. This operates through a continuous, three-stage feedback loop: automatically detecting coverage gaps in live production logs, drafting concrete remediations, and running simulations against historical transcripts to verify fixes before domain experts deploy them. Nichol stresses that platforms must support overrides and component customization, preserving past enterprise investments rather than demanding destructive platform replacements. Nichol concludes that while an enterprise's first agent is inevitably slow and messy, long-term success hinges on flipping stakeholder sentiment from frustration over delivery delays to demand for broader rollouts. This shift relies on maintaining an architecture that insulates core business logic from rapid underlying model deprecation.

精炼阅读摘要

Rasa 创始人兼首席技术官 Alan Nichol 指出，企业在推进 AI Agent 时普遍遭遇工程架构与业务交付速度脱节的矛盾；由于底层基础模型持续吞噬上层定制代码，企业 AI 技术栈的半衰期仅有 18 个月，组织必须将技术重心转向支持快速复用与业务专家参与的“速度层”。 交付张力与基础模型对技术栈的吞噬 (0:04) Nichol 指出，当前企业已从早期的概念验证转向构建内部 Agent 平台，技术团队往往为其搭建的高级架构和工具调用管线感到自豪，但高管与业务部门却持续抱怨新用例与生产 Agent 的上线速度严重滞后。这种组织摩擦的核心原因在于定制化运行时代码的快速贬值。十年前 Rasa 成立时，语言模型仅有单 Token 的上下文窗口，系统处理多轮对话、状态追踪与后端 API 调用等所有交互逻辑完全依赖手工编写的代码，模型仅负责将意图分类到预设类别中。 随着技术演进，基础模型开始迅速“吞噬技术栈”。BERT 的出现让模型具备就地理解完整句子的能力并原生解决否定句等复杂句式，使大段手写解析代码瞬间失效；GPT-3 与后续模型进一步扩展了上下文并原生集成了工具调用能力，逐步取代了原本需要人工编写的状态机和响应模板。Nichol 警告称，当前围绕模型构建的大量手工粘合代码和提示词脚手架同样难逃被后续版本淘汰的命运，正如过去被过度热炒的向量数据库与向量搜索，最终被不少团队发现其表现并不优于传统检索方式。 18 个月半衰期与三层平台架构 (4:37) 由于模型创新周期已缩短至数月，Nichol 为工程管理者提出了“AI 技术栈半衰期为 18 个月”的经验法则：企业当前自研的运行时脚手架中，约有 50% 会在一年半内变得完全多余，且工程团队无法预判究竟是哪一半会被淘汰。过度专注于底层运行时的拼接，会导致技术团队陷入无休止重写组件以适配新模型的被动循环中。 为打破这一循环，Nichol 主张将企业 Agent 平台划分为三层蛋糕架构：底层是围绕模型运行时的“能力层”（Capabilities Layer），包含提示词包装器和工具编排，该层折旧极快且易被模型迭代吞噬；中间层是涵盖合规、安全和访问控制的“企业就绪层”（Enterprise Readiness Layer），这是投产的必要门槛但无法形成业务差异化；顶层则是作为核心投资回报杠杆的“速度层”（Velocity Layer），其核心目标是确保企业构建第十个 Agent 的速度和成本相较第一个实现指数级优化。 面向工程与业务需求的高速度层规范 (6:52) 有效构建的速度层必须同时解决工程研发与业务运营两套截然不同的诉求。在工程维度，平台需要提供经过验证、可在多 Agent 间无缝复用的企业级技能库（涵盖从身份验证到交易流程等功能模块），彻底摒弃针对特定边缘案例反复微调提示词的“祈祷式修补”（Prompt and Pray），确保系统性修复不会引发其他退化。同时，团队需要深入决策机理的“推理级可观测性”，不仅追踪输入输出字符，更能掌握 Agent 的决策逻辑以及改变其决策轨迹的反事实条件。 在业务维度，产品路线图必须扎根于真实的生产交互数据而非主观推测，形成持续学习的正向飞轮。业务人员需要具备双向指标钻取能力：既能从高层业务 KPI 穿透至底层驱动该指标的具体会话，也能从孤立的异常对话向上判定其是否代表全局性趋势。更为关键的是，速度层必须打破每一次业务微调都需要向技术团队提交 Jira 工单的传统研发瓶颈。 联邦式开发与自愈型仿真闭环 (8:40) 为了真正提升交付速度，企业需采用联邦式开发模式，赋予深度理解业务流程但不会编写代码的领域专家直接定义和调整 Agent 行为的权限。现代速度层应当基于真实生产日志构建自动化的三阶段闭环：首先主动分析线上对话，自动定位未处理的用户意图与故障断点；其次自动起草具体的逻辑修正方案；最后在历史对话数据集上执行仿真测试，验证修改在彻底修复缺陷的同时不会导致功能回归，最终交由业务专家完成一键部署。 Nichol 强调，平台架构应兼容企业过去数年的既有资产，支持在预设框架外进行定制与组件重写，而非推倒重来。他借用“第一块煎饼往往做坏”的比喻指出，企业首个 Agent 的交付难免杂乱且缓慢，但长期价值取决于后续能力复制的敏捷度。当 Agent 能够依托真实业务数据自愈并由业务人员驱动时，管理层的态度才会从质疑研发进度缓慢转变为积极寻求规模化复制。 Nichol 总结认为，企业 Agent 的长期成败取决于构建后续用例的速度飞轮，而非执着于自研极易被模型迭代所吞噬的底层运行时脚手架。不过，在基础模型持续快速演进的背景下，企业如何在保障合规与防范回归风险的前提下，真正将控制权安全下放给无代码经验的业务领域专家，仍是这一架构在各行业深度落地时需直面的核心工程课题。

Detailed notes — zh

## Background 本视频由 Ai4 于 2026 年 8 月 15 日发布在 YouTube 平台。内容记录了 Rasa 联合创始人兼 CTO Alan Nichol 的主题演讲，聚焦企业级 AI Agent 为何难以走向持久生产，并深入探讨项目前 90 天的架构决策如何左右智能体在受监管大型企业中的最终命运。 ## TL;DR Alan Nichol 剖析了工程交付与业务诉求脱节的核心痛点，指出大模型正在不断吞噬外围胶水代码，导致 AI 技术栈面临仅有 18 个月的半衰期。为了打破深陷底层重构的“仓鼠滚轮”，他提出了“三层蛋糕模型”，呼吁企业从易过时的基础能力层转向构建以敏捷复用为核心的“速度层”。速度层能彻底终结“改提示词并祈祷”的脆弱调试模式，并提供深度的决策推理可观测性。最终通过联邦开发模式与生产闭环仿真赋能业务专家，让企业交付第 10 个用例时实现十倍级的敏捷与稳健。 ## Atmosphere & style 演讲以幽默自嘲的“算命师预言”轻巧开场，随后迅速切入刺痛当前企业的工程现实，整体基调兼具技术老兵的洞察敏锐与从容幽默。Alan 善用诸如“仓鼠滚轮”、“打地鼠”和“煎第一块松饼”等生动隐喻，将枯燥严肃的系统架构与合规治理讲得妙趣横生。整场分享由浅入深，在唤起技术栈快速贬值的危机感后，又为在场者指明了一条清晰笃定的破局架构路径。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) 企业 AI Agent 当前的核心矛盾在于：工程团队对交付成果倍感自豪，业务部门却始终抱怨新用例落地周期过长、迭代太慢。 - [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 大模型的演进规律表明，模型正在不断“吞噬”外围的技术栈，手写胶水代码、规则策略和工具编排正逐步被模型内建能力取代。 - [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 向量数据库的前车之鉴证明，过度押注单一特定基础设施存在风险，随着模型演进与经典搜索的结合，专用组件极易贬值。 - [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 现代 AI 技术栈的半衰期仅有 18 个月，今天搭建的代码组件有一半在一年半后将失去存在价值，且无法预知具体是哪一部分。 - [05:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=340s) Agent 平台的三层蛋糕模型包含基础能力层、企业就绪层和速度层，其中只有速度层能真正充当业务 ROI 的乘数放大器。 - [05:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=340s) 工程团队极易被困在底层运行时组件的“仓鼠滚轮”中，陷入不断重构代码的死循环，无暇沉淀具有差异化竞争力的业务资产。 - [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) 速度层通过沉淀可复用技能库，终结了“修改 Prompt 然后祈祷”的脆弱维护方式，避免修复单点问题时引发未知连锁回归。 - [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) 高阶可观测性应当超越底层 Token/Trace 级日志，清晰呈现 Agent 做出特定决策的推理链路与假设改变后的替代分支。 - [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) 业务团队需要直接基于真实生产对话建立反馈飞轮，实现宏观业务指标与微观对话实例的下钻对齐，缩短上线链条。 - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 联邦开发模式（Federated Development Model）将 Agent 交付能力下放给领域业务专家，打破由中央 IT 团队全权包办的交付瓶颈。 - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 自动化的生产学习闭环能够自主识别对话盲区、生成处理策略并基于历史数据运行沙盒仿真，在确保无回归后推向生产。 - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 遵循“第一块松饼”规律，首个 Agent 用例探索往往耗时且不完美，架构的核心目标是保证构建第 10 个用例时能以极快速度稳健上线。 - [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) Agent 架构必须尊重企业现有的技术资产沉淀，避免推倒重来（Rip and Replace），提供高度开放和可解构定制的能力。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) 算命师的预言与企业 AI Agent 的核心矛盾 Alan Nichol 以展位上的占卜师 Mystic Mona 开场。当天早晨 Mona 为他算了命并给出两点预言：第一，在场的所有人都在对抗同一条“恶龙”（the same dragon）；第二，他这场演讲会彻底搞砸（bomb）。Alan 笑称希望她只猜对了一半。作为 Rasa 的创始人兼 CTO，Alan 介绍 Rasa 是一个专门为企业构建面向客户与员工的 AI Agent 平台。他们早在 ChatGPT 问世之前就已经深耕此领域，与全球诸多要求严苛的顶尖机构合作，可以说是“在聊天机器人好用之前就在做聊天机器人了”。 在频繁接触各大企业的 IT 高管（CTO、CIO、Chief AI Officer）过程中，Alan 发现了一个普遍存在的紧张关系，这就是大家正在共同搏斗的“恶龙”。AI Agent 早已越过概念验证（POC）和实验阶段，企业内部搭建了自己的 Agent 平台，并交付了面向客服或员工支持的智能体。然而，矛盾随之显现：一方面，工程团队对自己交付的优秀成果倍感自豪；另一方面，业务端却永远不满意，抱怨上线新用例太慢、将新 Agent 推向生产环境的周期太长。 - **核心矛盾**：工程团队认为技术交付成果显著，业务部门却始终抱怨应用落地与迭代速度不足。 - **业务诉求**：跨越原型阶段后，如何高频、敏捷地交付面向生产环境的真实场景。 --- ### [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 大模型正在吞噬技术栈：从单 Token 到全栈 Agent Alan 抛出一个问题：“如果上下文窗口只有一个 Token，你能否构建出一个 Agent？”在 Rasa 刚写下第一行代码的早期阶段，语言模型的能力极其微弱，单次只能观察并分析一个 Token。当时为了将模型包装成可交互、可调 API、可完成任务的完整 Agent，开发者必须在模型外围编写海量的手写胶水代码。当时模型的唯一任务是将用户输入匹配到数百个预定义分类中的一个（即 Intent，意图识别），例如判断是打招呼、回答“是/否”、道别还是询问退款。所有分支判断、上下文管理、多轮对话衔接和 API 调用编排，全都依赖开发者或平台硬编码的逻辑。 伴随技术演进，大模型不仅变得更强大，还在不断“吞噬”外围的技术栈。BERT 的出现让模型能够处理整句话并理解上下文与否定句，使原本需要手动处理的一大批规则直接作废；随后 GPT-3 能够通读整个对话历史并自行调用工具，状态跟踪（State Tracking）、对话策略（Dialogue Policies）乃至手写模板回复都被模型原生能力取代。如今业内看似只需“模型在循环中调用工具”即可构成 Agent，但现实生产中周围依然堆叠了大量用于落地的手写外围代码。Alan 强调，历史规律表明这些外围手写逻辑迟早也会在未来一两个版本中被模型自身吃掉。 ```mermaid flowchart TD accTitle: 模型演进吞噬外围技术栈架构图 subgraph S1["早期架构：手写代码驱动"] direction TB M1["单 Token 模型\n(仅识别预设 Intent)"] C1["外围手写胶水逻辑\n(状态跟踪 / 策略规则 / 模板生成)"] M1 --- C1 end subgraph S2["现代/未来架构：能力内聚"] direction TB M2["现代大模型\n(内建上下文记忆 / 工具调用 / 自然语言生成)"] C2["待被吞噬的外围手写代码\n(当前临时补丁)"] M2 -.逐步取代.-> C2 end ``` - **历史例证**：从单 Token 分类到句子级理解（BERT），再到对话历史感知与自主工具调用（GPT 系列），外围专用模块被逐步淘汰。 - **向量数据库的前车之鉴**：向量搜索曾被视为 RAG 和 Agent 绝对不可或缺的基础设施，但随后实践证明传统的经典搜索在绝大多数场景下表现更好或旗鼓相当，导致相关专门投入迅速贬值。 - **AI 栈的半衰期**：当前构建 AI Agent 所依赖的技术组件，其“半衰期仅有 18 个月”；今天搭建的模块，有一半在 18 个月后将彻底失去存在价值，且无法预知是哪一半。 --- ### [05:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=340s) Agent 平台的三层蛋糕模型 针对技术快速过时带来的挫败感，Alan 提出了评估 Agent 平台的三层架构模型（Three Tiers of the Cake）。大多数工程团队沉溺并引以为豪的部分，往往集中在最底层，从而陷入不断重构的“仓鼠滚轮”（hamster wheel）。 ```mermaid flowchart TD accTitle: Agent 平台三层蛋糕模型 L3["速度层 (Velocity Layer)\n复用资产 / 敏捷交付 / 业务赋能 / 价值倍增器"] L2["企业就绪层 (Enterprise Readiness)\n安全合规 / 权限治理 / 非功能性硬指标 (无差异化)"] L1["基础能力层 (Capabilities Layer)\n运行时组件 / 提示词工程 (半衰期 18 个月，易被模型吞噬)"] L3 --- L2 L2 --- L1 ``` 1. **基础能力层（Capabilities Layer）**：围绕模型运行时的微观组件与胶水代码。这部分组件更新迭代最快、最容易被模型新能力吞噬，导致团队陷入不断推翻重构的死循环。 2. **企业就绪层（Enterprise Readiness Layer）**：落地生产必不可少的非功能性需求（如安全合规、权限治理等），属于刚性门槛，但不具备业务差异化竞争力。 3. **速度层（Velocity Layer）**：让团队成为业务英雄的关键层级。它能保证团队在构建第 10 个 Agent 或第 10 个用例时，交付速度远超第 1 个用例，是撬动业务投资回报率（ROI）的真正杠杆。 - **核心陷阱**：底层的两层架构吞噬了团队几乎所有的时间与精力，使技术人员困在重复造轮子的滚轮上，无暇构建能够让业务 ROI 产生乘数效应的资产。 --- ### [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) 速度层对工程与业务的核心价值 一个健全的“速度层”能彻底重塑工程师和业务人员的协作范式： 对于工程团队，速度层意味着拥有经过验证、可自由组合的“可复用技能库”（例如占卜技能 Palm Reading）。更重要的是，它终结了“调整 Prompt 然后祈祷”（prompt and pray）的盲目维护模式——即在一个地方改提示词修复缺陷，却在另外三个地方引发未知回归。速度层能确保“一处修复，永久生效”。此外，它提供了超越基础微观 Trace（仅展示 LLM 输入输出）的抽象可观测性，能够让开发者清晰审视 Agent 的整体推理逻辑，明确“它为何做出该决策”以及“条件改变后会做出何种替代分支”。 对于业务团队，速度层让产品路线图建立在真实的生产对话数据之上，构建学习飞轮。业务人员能够自上而下将驱动商业回报的宏观指标逐级下钻到具体的对话实例，也能自下而上判断单次异常对话是偶发特例还是系统性趋势。更关键的是，它打通了“从想法直达生产”的路径，无需凡事都走漫长繁琐的 Jira 工单流程。 - **打地鼠困境**：缺乏结构化保障时，基于 Prompt 的补丁常常演变成修一个引发三个 Regression 的打地鼠游戏。 - **高阶可观测性**：摆脱底层 Token/Trace 级日志的干扰，聚焦在业务决策链与推理归因。 --- ### [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 联邦开发模式与生产闭环学习 高效的速度层依托于“联邦开发模型”（Federated Development Model），打破依赖单一中央 IT 团队交付所有用例的瓶颈，直接赋能业务领域的专家。业务专家无需具备手写代码的能力，系统应当以主动、自动化的方式协助他们完成闭环演进： ```mermaid flowchart LR accTitle: 自动化生产学习飞轮 D1["生产真实对话流"] --> D2["自动洞察 Gap\n(识别无法处理的新诉求)"] D2 --> D3["智能生成策略建议\n(提供 Agent 处理方案)"] D3 --> D4["历史数据沙盒仿真\n(验证无回归后推送到业务端确认)"] D4 -.上线应用.-> D1 ``` Alan 强调，新版 Rasa 在设计哲学上高度尊重企业已有的技术投资，绝不强制企业“推倒重来”（rip and replace）。平台必须允许用户自由打破边界（color outside the lines），随意解构并按自身方式定制组件。 首个用例的上线周期往往漫长，如同“煎第一块松饼”（the first pancake）难免不尽如人意。企业真正应该追求的是：构建第 10 个用例时，核心价值能够由懂业务的业务专家利用真实数据驱动上线，且从第一天起就具备生产级健壮性。当这种体系建立后，业务端对工程团队的态度也会从质疑性的“为什么你们这么慢？”转变为积极支持的“这个成果太棒了，我们怎样才能继续扩大规模？” 演讲尾声，Alan 开玩笑地回应开场的占卜师 Mona，称大家未来 AI 命运的答案就藏在五个字里：“来我们展位找我”（Find me at the booth），并邀请在场听众扫码获取个性化的占卜签文与资料。 - **自主进化的三步飞轮**：检测生产语料盲区 $\to$ 给出修改建议 $\to$ 基于历史数据跑仿真测试，验证通过后由业务专家一键合并。 - **架构哲学**：避免推倒重来，兼顾对现有架构的兼容性与开箱可定制性。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) 在场的所有人都在对抗同一条“恶龙”。 — Mystic Mona > [00:04](https://www.youtube.com/watch?v=2hO01YhyWCo&t=4s) 在聊天机器人好用之前就在做聊天机器人了。 — Alan Nichol > [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 如果上下文窗口只有一个 Token，你能否构建出一个 Agent？ — Alan Nichol > [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 当前构建 AI Agent 所依赖的技术组件，其半衰期仅有 18 个月。 — Alan Nichol > [01:42](https://www.youtube.com/watch?v=2hO01YhyWCo&t=102s) 今天搭建的模块，有一半在 18 个月后将彻底失去存在价值，且无法预知是哪一半。 — Alan Nichol > [05:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=340s) 大多数工程团队沉溺并引以为豪的部分，往往集中在最底层，从而陷入不断重构的“仓鼠滚轮”。 — Alan Nichol > [05:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=340s) 速度层能保证团队在构建第 10 个 Agent 或第 10 个用例时，交付速度远超第 1 个用例，是撬动业务投资回报率的真正杠杆。 — Alan Nichol > [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) 调整 Prompt 然后祈祷（prompt and pray）。 — Alan Nichol > [06:52](https://www.youtube.com/watch?v=2hO01YhyWCo&t=412s) 一处修复，永久生效。 — Alan Nichol > [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 平台必须允许用户自由打破边界，随意解构并按自身方式定制组件。 — Alan Nichol > [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 首个用例的上线周期往往漫长，如同“煎第一块松饼”（the first pancake）难免不尽如人意。 — Alan Nichol > [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 业务端对工程团队的态度也会从质疑性的“为什么你们这么慢？”转变为积极支持的“这个成果太棒了，我们怎样才能继续扩大规模？” — Alan Nichol > [08:40](https://www.youtube.com/watch?v=2hO01YhyWCo&t=520s) 来我们展位找我（Find me at the booth）。 — Alan Nichol ## Mentioned Alan Nichol, Rasa, Mystic Mona, ChatGPT, BERT, GPT-3, Jira ## Video description Alan Nichol - CTO & Co-founder, Rasa Most enterprise AI agents being built right now won't make it to durable production. Not because the models aren't good enough, but because the architectural choices made in the first 90 days quietly seal the project's fate. Alan Nichol, founder of Rasa, has spent over a decade reading the leaves: watching enterprise agents get built through the chatbot era, the LLM shift, and now the agent era. In this session, he'll share the pattern recognition: which architectural bets pay off, which ones are cursed from the start, and what separates the agents running at the world's largest banks, telcos, and healthcare companies from those that never made it out of the cards. If you're building or operating AI agents in a regulated enterprise, this talk will tell you with uncomfortable specificity what your agent's future holds — and how to change it. [Source](https://www.youtube.com/watch?v=2hO01YhyWCo)

Source quotes

[0:04] Mystic Mona: 在场的所有人都在对抗同一条“恶龙”。 [0:04] Alan Nichol: 在聊天机器人好用之前就在做聊天机器人了。 [1:42] Alan Nichol: 如果上下文窗口只有一个 Token，你能否构建出一个 Agent？ [1:42] Alan Nichol: 当前构建 AI Agent 所依赖的技术组件，其半衰期仅有 18 个月。 [1:42] Alan Nichol: 今天搭建的模块，有一半在 18 个月后将彻底失去存在价值，且无法预知是哪一半。 [5:40] Alan Nichol: 大多数工程团队沉溺并引以为豪的部分，往往集中在最底层，从而陷入不断重构的“仓鼠滚轮”。 [5:40] Alan Nichol: 速度层能保证团队在构建第 10 个 Agent 或第 10 个用例时，交付速度远超第 1 个用例，是撬动业务投资回报率的真正杠杆。 [6:52] Alan Nichol: 调整 Prompt 然后祈祷（prompt and pray）。 [6:52] Alan Nichol: 一处修复，永久生效。 [8:40] Alan Nichol: 平台必须允许用户自由打破边界，随意解构并按自身方式定制组件。 [8:40] Alan Nichol: 首个用例的上线周期往往漫长，如同“煎第一块松饼”（the first pancake）难免不尽如人意。 [8:40] Alan Nichol: 业务端对工程团队的态度也会从质疑性的“为什么你们这么慢？”转变为积极支持的“这个成果太棒了，我们怎样才能继续扩大规模？” [8:40] Alan Nichol: 来我们展位找我（Find me at the booth）。

Source quotes

[0:04] Alan Nichol: since well before ChatGPT [0:04] Alan Nichol: since before they were good. [1:46] Alan Nichol: eating the stack [4:37] Alan Nichol: Your AI stack has a half-life of 18 months. [4:37] Alan Nichol: hamster wheel [6:52] Alan Nichol: Prompt and Pray [8:40] Alan Nichol: the first pancake [8:40] Alan Nichol: color outside the lines [8:40] Alan Nichol: Why are you so slow? [8:40] Alan Nichol: We love this. Can we have more? How do we scale this out? [8:40] Alan Nichol: Find me at the booth.

Original transcript

[0:04] We have a fortune teller at our  booth. And her name is Mystic Mona. [0:09] And I met her this morning and we had a chat and  she read my fortune. And she told me two things. [0:14] She told me that everyone in this  room is fighting the same dragon. [0:20] She also told me that my talk was going to  bomb. So, let's hope she's only half right. [0:24] By way of introduction, I'm Alan, founder of  Rasa. We're a platform for enterprises to build [0:31] customer-facing and employee-facing AI agents. And  we've been doing that since well before ChatGPT. [0:38] We did that with some of the most sophisticated  and demanding organizations in the world. [0:43] And as I like to say, we've been doing  chatbots since before they were good. So, [0:48] I spend a lot of time with IT leadership in these  organizations. CTOs, CIOs, chief AI officers. And [0:56] a very consistent theme I've been seeing is this  tension. This is the dragon people are fighting. [1:03] AI agents have been the biggest topic in the  world for the last few years. So, we're well past [1:08] POCs and experimentation. These teams are building  up and setting up their agent platforms internally [1:14] and they're delivering agents on top of them.  Whether that's for customer service or for [1:18] employee assistance or whatever else  that might be. And on the one hand, [1:22] you have an engineering team who are rightly proud  of all the great things that they're shipping. [1:26] But on the other hand, the business is never  satisfied. We're not bringing use cases live [1:30] fast enough. We're not shipping new agents  and bringing them to production fast enough. [1:36] And I think I've finally figured  out where this tension comes from. [1:42] But before we go, I'm going to  start and ask you one question. [1:46] Do you think you could build an agent  with a one-token context window? [1:52] It sounds bananas, right? If your LLM  could only see one word, one token, [1:55] how on earth would you build an agent that  way? Well, that's how things used to work. [1:59] When I wrote the first lines of code for Rasa, we  had language models that could look at one token [2:04] and tell you something about that one token.  And everything else around that, that turned [2:09] that into an agent, into something that could hold  a conversation, interact with systems and APIs, [2:14] and complete tasks on behalf of a user, was a  whole bunch of handwritten code around this model. [2:21] The model had one tiny job, which was just to  take what the user said and match it to one [2:26] of a few hundred predefined categories. We called  these intents at the time. So, you would just say, [2:31] you know, is this user greeting us, are they  saying yes or no, or goodbye, or they asking [2:34] about a refund? Just one of these predetermined  cases. And everything else, all the if statements, [2:39] all the logic, how we interact, how we deal  with follow-up questions, how we deal with the [2:43] context of a multi-turn conversation, how we call  APIs and use that information to feed back in, [2:48] that was all a whole bunch of code. Some of  it your platform wrote, some of it you wrote. [2:53] And what kept happening over the years is  that models didn't just get bigger and better, [2:57] they started eating the stack around them. So,  first there was BERT, which at the time we called [3:02] a large language model, and it meant that you  could look not just at one token at a time, but [3:07] you could look at a whole sentence, and understand  what people were saying in context. You could [3:10] understand things like negation. We had a whole  set of things that just dropped off. You didn't [3:15] need to do anymore. You didn't need to do manually  anymore because the model took over the job. [3:19] Right? Then, of course, GPT-3 came along, and  you started building models that looked at the [3:25] whole transcript. Beyond that, models could call  tools themselves. So, every time the models got [3:32] bigger and better, they started eating all  the components that we'd built around them. [3:38] So, now you look at an agent implementation,  and you think, well, I've got a model. [3:44] State tracking, gone. The model just  keeps track of the whole conversation. [3:50] Right? Um dealing with dialogue policies,  well, the model just calls tools itself. [3:56] Handwritten responses, well, the  model just generates the answer. [3:59] You kind of look at this and you go, well,  what's what's actually left to build? [4:02] Is there anything left to do? We've got a  model calling tools in a loop. That's an agent. [4:08] Of course, in reality, if you look at any real  implementation, there's still a whole bunch of [4:12] stuff. There's a whole bunch of components with  handwritten code sitting around that model, [4:17] sitting in the immediate vicinity of that  model, to turn that model into a useful agent. [4:22] And what I'm telling you is that the model is  going to eat that stack, too. So, everything [4:26] you've been building there is also going to get  eaten by the model. And it might be in the next [4:30] iteration, might be the iteration after that.  But that's the lesson we can learn from history. [4:37] Good example, vector databases and vector  search. Huge topic. A whole category of [4:42] products and companies, and was considered  to be an absolutely essential element [4:46] to building rag or building any kind of agent.  Turns out, actually good old-fashioned search [4:51] for almost every use case works better or just  as well. Code code doesn't use vector search. [4:57] So, whole set of investments and things, they kind  of became irrelevant. This keeps happening. And in [5:04] fact, it's happening much faster than it ever did  before. I've been working on Rasa for 10 years, [5:09] right? We used to knock out these components one  every couple of years. Now, it's more like every [5:12] couple of months. So, that's why I feel confident  in telling you that your AI stack has a half-life [5:18] of 18 months. All these things that you're  building, you need them to have an agent today. [5:24] About half of them you won't need in 18 months.  And unfortunately, I can't tell you which half. [5:28] So, that's that could be very frustrating if  you're an engineering leader or you're leading a [5:31] team and you're building a bunch of stuff. So, how  do you work around this? How do you think about [5:35] this? It could potentially be very depressing, but  that's if you're only looking at one slice of the [5:40] problem. So, the way I think about agent platforms  is that there are three tiers of the cake. [5:47] And everything we've talked about so far, and  everything that that engineering team was proud [5:51] of having shipped, sits in that bottom layer,  the capabilities layer. Everything that is in [5:55] and around the model at runtime that turns that  LLM into an agent. That stuff doesn't age well. [6:04] And so you're on this hamster wheel where you're  constantly reinventing these components because [6:08] you want to take advantage of  the latest and greatest models, [6:10] which means that a bunch of things  that you built before become redundant. [6:14] On top of that, you have the enterprise readiness  stuff, absolutely essential non-functional things [6:19] that are required for you to build this into  production. But completely undifferentiated. [6:25] And then on top of that, you have the thing that  really makes you the hero, the velocity layer, the [6:30] thing that means that the 10th agent or the 10th  use case gets built faster than the first one. [6:37] And the bottom two layers are eating up  everybody's time and everybody's attention, [6:41] and you're on this hamster wheel reinventing a  bunch of stuff, and you're not getting around [6:44] to building the things that are really going to  melt melt multiply the ROI for your business. [6:52] So what does it mean if you have a really good,  really strong velocity layer? Well, for engineers, [6:59] it means that you have a library of proven,  reusable skills. Palm reading might be one [7:04] of them. That you can integrate into  all the agents that you're building, [7:08] and they've been vetted, and you know that they  compose together, and you know that they work. [7:12] It means that when you fix something  once, you know that it stays fixed. [7:16] And that's definitely not the case for a  naive agent implementation. I like to call [7:20] this prompt and pray, right? You tweak some  prompts here to fix a problem that you saw. [7:24] Guess what? You introduced three more regressions  elsewhere. You need a system that you know you can [7:28] reliably fix things in, so you're not chasing the  tail or playing whack-a-mole. You need the ability [7:34] to see the root cause of why your agent decided  to do something at any given point in time. [7:39] People talk about observability, and they tell you  that if you had LLM traces and you can see exactly [7:44] what went into the LLM and what came out that  that's the thing you need. That's not what you [7:47] need. That's like the micro detail. Nobody needs  to operate at that level. You need something more [7:51] abstract. You need a level about how your agent  is reasoning about things. And you need need the [7:55] ability to say, "This is why it did what it did."  And if these conditions had changed, it would have [8:00] done this thing instead. That's the level of  observability you need on your agent system. [8:06] For the business, it means that you're building  your roadmap on real conversations and you have [8:09] a flywheel of learning from those conversations  in production. It means that you can look at [8:14] the metrics that are driving the business case and  you can drill down to the individual conversations [8:19] that are moving the needle. Similarly, you can  look down at an individual conversation and say, [8:23] "Is this a pattern or is this a one-off?" You've  got to go from top-down and from bottom-up. [8:29] And then finally, you've got to be able to go from  idea to production without going through Jira. [8:34] That was okay 10 years ago. It's not  now. To make that a little less abstract, [8:40] what kind of good velocity layer look like?  It empowers people across the business [8:47] in a federated development model where you're  not relying on the central single central IT [8:52] resource to ship every use case and every agent.  And you empower that person who is the expert, [8:57] the domain expert, to define what it is. But  they're not an engineer. They're not building [9:03] things from scratch. That's not their job. A good  velocity layer should be proactively saying, "Hey, [9:07] look. We have a whole bunch of production  conversations coming in. And actually, [9:12] we found a gap. People are asking about  this and you don't know how to handle it." [9:18] Second, I made a suggested change to  how your agent is going to handle this. [9:23] And then third, I ran a bunch of simulations  based on historical data. And actually, this [9:28] thing checks out. It works pretty well. Here's  the suggestion. Would you like to incorporate it? [9:34] You need a machine that helps you learn almost  automatically from production conversations. [9:42] And when I asked the team to build the new version  of Rasa, I told them we have to build this in a [9:46] way that respects the investments that enterprises  have already made. Because again, this isn't [9:51] 2 months after chat GPT. People have been  investing. People have been building things. [9:56] We wanted to be able to bring the things that  they've done and that they do do uniquely well. [10:00] And that's a very important philosophical point  about Rasa is that you don't have to live with [10:04] our assumptions. You can color outside  the lines. You can break open any piece [10:08] and say, "Well, actually we want to do that  this way." Right? So, how do we make sure [10:11] we're not telling everyone they have to rip and  replace their agent platform and start from zero? [10:15] How do we complement the things  that people have already built? [10:22] So, that by the time you're building the  10th use case or the 10th agent, that's where [10:27] you're seeing the ROI and the value. It's very  tempting to focus on and stress out about how [10:32] long it took you to get that first use case in  production. But, that's like the first pancake. [10:38] What matters is that by the time you're shipping  real value, it's done by the people who know the [10:42] business. It's informed by real data. And  it's hardened in production from day one. [10:48] So, then the conversation that you start  to hear, it's the same as the one before, [10:53] but just with a very different  underlying sentiment. It's not, [10:56] "Why are you so slow?" It's, "We love this.  Can we have more? How do we scale this out?" [11:04] So, back to Mona. And a little bit of  fortune telling. I'll give you your AI future [11:10] in five words. Find me at the booth. [laughter]  And I do recommend that you scan the QR code. [11:18] You'll get a personalized fortune and  a few other helpful things. Otherwise, [11:23] thank you all very much for listening and I'll  be in the booth for the rest of the afternoon.