Elastic Ditches Monolithic AI Agents as Specialist Workflows Cut Costs

Multi-Agent Orchestration with Elastic Workflows and Agent Builder With Elasticsearch — Ai4

Ai4

Telemetry across 32,000 real-world triage runs shows monolithic multi-tool agents balloon latency and token overhead by evaluating every tool schema on each turn, whereas deterministic workflows of specialized domain models prevent context poisoning and slash compute costs.

AI-assisted summary of Multi-Agent Orchestration with Elastic Workflows and Agent Builder With Elasticsearch — original source. Check the original for full context.

Source published: 2026-08-18

Detailed notes

## Background Published by Ai4 on YouTube in August 2026, this presentation features Tristan Ma, Manager of Solutions Architecture at Elasticsearch. According to the page description, the session explores how to scale multi-agent intelligence without ballooning costs or latency by combining Elastic Workflows, Agent Builder, Liquid Agent patterns, and granular token tracking.

## TL;DR Tristan Ma outlines why coordinating modular, specialized AI agents through deterministic workflows consistently outperforms single monolithic 'super agents' in production environments. Using an incident triage and root cause analysis demo, he illustrates how Elastic serves as both a contextual grounding layer and an OpenTelemetry trace repository. By adopting declarative YAML workflows, Liquid templating to pre-filter payloads, and indexed trace logging, teams can empirically optimize performance—including selectively switching from Claude Sonnet to Haiku to slash latency while maintaining diagnostic accuracy.

## Atmosphere & style The presentation is pragmatic, engineering-focused, and refreshingly empirical, stripping away generative AI hype in favor of hard telemetry and real-world system architecture. Tristan delivers his insights with accessible clarity and relatable industry humor, engaging the audience with live console walkthroughs and candid observations about customer trust in LLMs. The talk feels like an insider masterclass in observability and cost control, empowering practitioners to move past proof-of-concept guesswork into rigorous production tuning.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=4s) Enterprise automation is evolving from single-turn RAG to multi-agent architectures that mirror human team collaboration across specialized domains. - [02:02](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=122s) Deterministic AI workflows provide auditable, repeatable rails for standard runbooks, while autonomous agents excel at open-ended, exploratory problem-solving. - [02:02](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=122s) Coordinating modular, specialized agents consistently outperforms monolithic 'super agents' by preventing context poisoning and lowering token costs during production scaling. - [04:33](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=273s) Elasticsearch functions simultaneously as the contextual grounding source and the OpenTelemetry (OTel) trace store for end-to-end agent observability. - [04:33](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=273s) Assigning sub-agents domain-specific biases (infrastructure, application telemetry, search) prevents noisy context from overwhelming LLMs before evidence synthesis. - [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) Declarative YAML orchestration allows both human engineers and LLM workflow builders to reliably construct, inspect, and modify multi-step agent chains. - [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) Enterprise adoption requires embedding direct, clickable links to raw telemetry and logs in LLM outputs to allow subject matter experts to verify findings independently. - [11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) Applying Liquid templating natively in Elasticsearch extracts only necessary fields from JSON payloads, eliminating redundant MCP roundtrips and slashing token consumption. - [11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) Strategically swapping heavy frontier models for lighter models like Claude Haiku in targeted agent steps achieves a 15% latency reduction and lowers inference costs. - [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) Telemetry from over 32,000 internal triage iterations reveals that loading single agents with 14+ tools creates massive per-turn token overhead compared to deterministic workflows. - [16:11](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=971s) Native MCP endpoints with version control and import/export capabilities enable teams to systematically track and update tool schemas as tools evolve. - [18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) Granular execution trace logging allows teams to empirically evaluate model trade-offs and assess whether lighter models provide sufficient fidelity for downstream agent chains.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=4s) The Evolution of AI Automation: From Rules and RAG to Agentic Systems

Opening with a quick show of hands across the audience to gauge familiarity with the ELK stack, Tristan Ma introduces Elastic as not only an open-source full-text search engine, but a robust analytics platform powering search, observability, and security. Given Elastic's wide operational footprint across thousands of enterprise environments, the team has had a front-row seat to how organizations are adopting artificial intelligence. Beyond the natural enthusiasm present at an AI conference, Tristan acknowledges the intense executive and board-level mandate leaders face to embed AI directly into core business workflows.

This corporate appetite reflects a broader maturation in how industry approaches automation. Early enterprise automation relied on strict, rule-based systems: you defined a rigid logical process, executed it, and hoped it held together, frequently wishing "if only it could be smarter" or "if only it could just catch that small nuance." The industry then advanced into Retrieval-Augmented Generation (RAG). RAG established a reliable production architecture that combined LLM reasoning with enterprise data grounding, allowing systems to provide cited, verifiable answers without requiring models to be trained directly on private data.

Yet, single-turn RAG has distinct limitations when addressing complex, multifaceted enterprise challenges. Real-world troubleshooting rarely mirrors a single query to a single individual; instead, humans naturally consult multiple colleagues across different teams, each offering distinct perspectives and specialized domain expertise. To replicate that dynamic in software, the industry has transitioned into agentic AI, prompting an architectural distinction between structured workflows and autonomous agents.

[02:02](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=122s) AI Workflows vs. AI Agents: Super Agents vs. Specialized Domain Experts

Tristan distinguishes between an AI workflow and an autonomous AI agent, noting that while both utilize agentic capabilities, they serve fundamentally different operational purposes: - **AI Workflows (Deterministic Paths):** Designed for known procedures and standard operating runbooks. The LLM reasons through pre-defined stages sequentially, providing strict rails, deterministic execution, and end-to-end auditability, trackability, and logging. - **AI Agents (Exploratory Reasoning):** Equipped with diverse tools and skills to reason independently. These agents are ideal for open-ended problem exploration where the exact path or destination cannot be predetermined, though tracing their exact decision path can prove more complex.

```mermaid flowchart TD accTitle: AI Workflows vs Autonomous AI Agents subgraph Workflow ["AI Workflow (Deterministic Rails)"] W1["Standard Runbook Step 1"] --> W2["LLM Guided Step 2"] W2 --> W3["Auditable Output & Logs"] end subgraph Agent ["Autonomous AI Agent (Exploratory)"] A1["User Goal / Query"] --> A2["LLM Tool Selection & Reasoning"] A2 --> A3["Dynamic Tool Execution"] A3 --> A2 end ```

Beyond workflows versus agents lies a critical architectural debate: should organizations deploy a single, monolithic "super agent" equipped with dozens of tools, or coordinate a fleet of modular, specialized agents? Tristan emphasizes that this distinction becomes paramount on "Day 2"—when systems move past internal proof-of-concept demos and scale into production environments where latency and inference costs multiply. Based on Elastic's internal testing—including data from their InfoSec security team analyzing over 32,000 real-world conversations—specialized agents consistently outperform single super agents, delivering superior answer accuracy while significantly reducing overall token consumption and context poisoning.

[04:33](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=273s) Architecture of a Specialist Multi-Agent System on Elastic

To demonstrate this specialist philosophy, Tristan presents an incident triage and root cause analysis (RCA) architecture built entirely on Elasticsearch. In this system, Elasticsearch serves a dual purpose: acting as the contextual grounding source (knowledge base and logs) and functioning as the OpenTelemetry (OTel) telemetry store for deep system observability.

```mermaid flowchart TD accTitle: Multi-Agent RCA Architecture on Elastic subgraph Coordination ["Triage & Routing"] A["Triage Agent (Atlas)"] end subgraph DomainAgents ["Biased Domain Specialists"] P["Pulse (Infrastructure)"] L["Lens (Application Telemetry)"] S["Scout (Search & Relevance)"] end subgraph SynthesisLayer ["Synthesis & Incident Management"] SYN["RCA Synthesizer Agent"] REP["RCA Reporter Agent"] KIB["Kibana Case Management"] end subgraph StorageLayer ["Elasticsearch Core Platform"] ES[("Elasticsearch: Grounding Data & OTel Store")] end

A --> P & L & S P & L & S --> SYN SYN --> REP REP --> KIB P & L & S -. Grounding & Telemetry Lookups.-> ES SYN -. Telemetry & Trace Logging.-> ES ```

The multi-agent workflow operates through specialized stages: - **Atlas (Triage Agent):** Receives the incoming incident or alert and coordinates delegated tasks out to specialized sub-agents. - **Specialist Sub-Agents (Pulse, Lens, Scout):** Each sub-agent is intentionally given domain-specific instructions and strong bias toward its respective area (Pulse for infrastructure, Lens for application logs/telemetry, Scout for search relevance). This domain isolation prevents "context poisoning," ensuring individual LLMs remain focused without being overwhelmed by noisy, irrelevant telemetry. - **RCA Synthesizer:** Aggregates findings and evidence submitted exclusively by the specialist agents, acting as an impartial judge to determine the true root cause based on domain inputs. - **RCA Reporter & Case Logging:** Compiles a finalized investigation report and automatically logs it directly into a Kibana Observability case (or external case management platform) for trackable post-incident review.

The agents are configured using Elastic Agent Builder via API or UI, integrating system instructions, custom capabilities, and an "RCA on Rails" workflow definition while streaming live OTel execution traces back into Elasticsearch.

[07:37](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=457s) Live Investigation: Diagnosing a Complex 15% Conversion Drop

Tristan demonstrates the architecture in action by presenting the system with a realistic, multifaceted production incident: *"Search conversion dropped 15% overnight—is this a relevance problem (i.e., are my search results bad) or is this an infrastructure problem?"*

Single-agent systems and traditional RAG pipelines often struggle with questions spanning multiple distinct failure modes. As the live run executes: - The **Pulse** infrastructure agent inspects its specialized metrics and flags an initial signal pointing toward a potential Redis connection pool exhaustion. However, isolated on its own, it cannot verify whether that resource constraint directly caused the top-line conversion drop or if search relevance degraded independently. - The **Atlas** triage agent and the overarching Elastic Workflow orchestrate concurrent investigations across the remaining domain agents. - Within the Kibana interface, operators can watch the agent execution steps live, inspecting the exact inputs, prompt instructions, system references, and raw outputs generated by each specialist throughout the workflow.

- **Observed Scenario:** An overnight 15% drop in search conversion evaluated across infrastructure health vs. relevance degradation. - **Infrastructure Signal Detected:** Redis connection pool exhaustion identified by the Pulse sub-agent. - **Platform Inspection:** Full multi-agent execution visible within the Elastic Agent Builder and Kibana Workflow console, tracking discrete agent IDs, tool parameters, and intermediate reasoning steps.

[09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) Declarative Workflows, Full Auditability, and Root Cause Synthesis

As the execution progresses, the backend synthesizes a complete Root Cause Analysis (RCA). Every stage of the execution chain—the user prompts, intermediate agent thoughts, tool inputs, and raw outputs—is automatically written into an Elasticsearch index. Because this data is indexed natively, teams can query execution runs independently and even have an LLM run secondary meta-analyses over past execution traces to inspect chains and evaluate system performance.

```mermaid flowchart TD accTitle: Multi-Agent RCA Workflow Architecture in Elastic

subgraph Trigger ["1. Trigger & Parsing"] A["Incoming Alert / APM Error"] --> B["Declarative YAML Workflow"] B --> C["Specialized Pre-LLM Filters"] end

subgraph Execution ["2. Grounding & Liquid Templating"] C --> D["Elasticsearch Native Query"] D --> E["Liquid Templating (Field Extraction)"] E --> F["Specialized LLM Agent (Sonnet / Haiku)"] end

subgraph Observability ["3. Audit & Verification"] F --> G["Workflow Execution Index"] G --> H["Grounded RCA Output + Deep Links"] H --> I["Human SME / Security Analyst"] end ```

The orchestration layer relies entirely on declarative YAML workflows. Tristan explains that this was a deliberate architectural decision because "LLMs love YAML. Who knows why? But it's very structured. It kind of knows how to write it." This structure allows both engineers and LLM workflow-builder skills to generate and modify multi-step chains reliably. When the sample execution finishes, the system successfully pinpoints that what looked like a relevance failure is actually two distinct incidents driven by an underlying infrastructure error. Crucially, the final synthesis includes direct, clickable links into Elastic APM inventory and log views. This raw provenance is essential for enterprise adoption, as Tristan notes from customer feedback:

> "What we're finding in our conversation with customers is most of our customers do not trust the LLM output by itself, right? Quick show of hands... We trust it, but it's like, yeah, is that right? It's much better for us to be able to go through if we want to have a subject matter expert if we need to go in and actually check the raw evidence."

- Workflows can be triggered automatically ahead of time for alert triaging, establishing an investigative path and gathering raw evidence before an analyst even opens the ticket. - Full trace auditability in Elasticsearch indexes allows organizations to inspect inputs, outputs, and agent transitions over time.

---

[11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) Performance Tuning: Liquid Templating and Model Switching

Having full visibility into execution traces transforms agent performance optimization from guesswork into an empirical science. When an RCA run takes two minutes, teams need verifiable levers to reduce latency without sacrificing output quality. Two primary optimization techniques address this: Liquid templating and model switching.

Liquid templating takes advantage of Elasticsearch acting as the grounding layer. Instead of forcing an LLM to parse and filter through an entire raw JSON payload, Liquid templates extract only the exact fields required before passing context to the model. Because the data is called and formatted natively within Elastic, it eliminates unnecessary Model Context Protocol (MCP) roundtrips, slashes context window bloat, and materially reduces latency.

Model switching provides a complementary optimization path. By swapping Anthropic's Claude 3.5 Sonnet with Claude 3 Haiku for specific specialized agent steps, the system achieves a 15% latency reduction alongside lower token costs. However, because both runs are indexed side-by-side in Elastic, teams can perform nuanced comparative evaluations between Sonnet and Haiku to determine whether slight variations in the generated answers remain acceptable for production environments.

---

[13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) Real-World Lessons from InfoSec: Monolithic Agents vs. Deterministic Specialized Workflows

Elastic's internal Information Security team serves as "Customer Zero" for these multi-agent capabilities. With roughly 3,000 employees company-wide, Elastic does not maintain a massive global security operations center, requiring the security team to rely heavily on automation. Over the course of running more than 32,000 triage iterations, the team analyzed their workflow telemetry to resolve persistent reliability and cost bottlenecks.

Tristan emphasizes the core operational takeaway: "You can only fix what you can see." Loading a single generalist agent with 14 tools forces the model to evaluate every tool definition on every single turn before taking action, creating immense token overhead and latency penalties. For high-volume alert triaging where processes are well-understood, routing events through deterministic, specialized agent workflows slashes costs while preserving fidelity.

- The architecture remains fully open and supports the Agent2Agent (A2A) protocol, allowing external tools and agents to integrate without vendor lock-in. - Advanced implementations can branch out into asynchronous workflows when an agent needs to diverge from a linear chain to explore an investigative lead.

---

[16:11](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=971s) Audience Q&A: Deterministic Workflows, MCP Versioning, and Model Output Quality

During the closing question-and-answer session, attendees dig into practical architectural trade-offs:

1. **Independent Specialist Agents vs. Monolithic "Deep" Agents:** An audience member asks whether Elastic advocates for a suite of independent agents or a single deep agent overseeing everything. Tristan clarifies that for high-volume operational tasks like alert triage, deterministic workflows with specialized agents are far more effective at cutting out noise and controlling token budgets. However, Elastic also supports asynchronous workflows where agents can fork off into non-linear, exploratory sub-tasks when needed. 2. **Managing MCP Tool Drift and Schema Changes:** Another attendee asks how to manage tool definitions when underlying MCP tools change over time. Tristan explains that Elastic provides native MCP endpoints equipped with version control and import/export capabilities so teams can track, rename, and update tool schemas methodically. 3. **Evaluating Model Swaps (Sonnet vs. Haiku):** A participant asks whether Haiku actually reproduced Sonnet's output quality during the 15% latency improvement tests. Tristan confirms that while Haiku ran noticeably faster and cheaper, differences in the responses did appear, underlining why continuous indexed trace logging is critical for measuring acceptable quality thresholds.

[18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) Evaluating Model Trade-Offs: Claude Sonnet vs. Claude Haiku in Multi-Agent Workflows

Evaluating model performance step-by-step revealed that while the depth and level of supporting evidence differed between models, the underlying conclusion remained identical. Sonnet brought richer supporting detail, whereas Haiku arrived at the exact same conclusion and correctly flagged the issue as a "high risk" rather than a "critical risk." Having granular auditability shifts the engineering debate away from abstract assumptions about model capabilities toward empirical proof: asking "it's faster and it's cheaper—is that answer good enough?" rather than asserting outright whether Haiku or Sonnet can handle a specific task.

```mermaid flowchart TD accTitle: Single-Step vs Downstream Chaining Trade-off subgraph StepEvaluation["Single Step: RCA Synthesizer"] Haiku["Claude Haiku Faster & Cheaper Same Conclusion, Less Depth"] Sonnet["Claude Sonnet Higher Cost & Latency Richer Detail & Evidence"] end

subgraph DownstreamImpact["Downstream Agent Chain"] NextStep["Next Agent Input / Output"] FinalOutcome["Subsequent Pipeline Quality"] end

Haiku -->|"Sufficient single step?"| NextStep Sonnet -->|"Better context?"| NextStep NextStep --> FinalOutcome ```

Haiku demonstrated surprisingly strong capability in the role of the root cause analysis (RCA) synthesizer—a stage where intuition might suggest a heavier, more advanced frontier model is strictly required. For that specific synthesis task, a lighter model produced the same core outcome with slightly less depth, giving system architects and auditors the flexibility to decide what level of detail is acceptable. However, evaluating quality in isolation at a single step does not tell the whole story; richer output from one agent might provide higher-quality context into the next step of the pipeline, potentially compounding downstream accuracy across subsequent agents. Because the workflow logs every input and output at each hop, teams maintain the full audit trail needed to investigate these chained effects across the entire lifecycle.

- **Outcome Parity:** Haiku matched Sonnet's core risk classification (flagging the issue as high risk rather than critical), differing primarily in the amount of depth and granular evidence provided. - **RCA Synthesizer Viability:** Lighter, cost-efficient models like Haiku proved fully capable of handling complex synthesis tasks previously assumed to require heavier models. - **Pipeline-Wide Evaluation:** While single-step verification confirmed Haiku was sufficient for that standalone phase, full multi-agent evaluation requires analyzing how output fidelity cascades into subsequent downstream agent inputs. - **Full Traceability:** Complete historical records of inputs and outputs for every workflow stage allow teams to audit and fine-tune model selection across each node of the multi-agent chain.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=4s) if only it could be smarter — Tristan Ma

> [00:04](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=4s) if only it could just catch that small nuance. — Tristan Ma

> [07:37](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=457s) Search conversion dropped 15% overnight—is this a relevance problem (i.e., are my search results bad) or is this an infrastructure problem? — Tristan Ma

> [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) LLMs love YAML. Who knows why? But it's very structured. It kind of knows how to write it. — Tristan Ma

> [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) What we're finding in our conversation with customers is most of our customers do not trust the LLM output by itself, right? — Tristan Ma

> [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) We trust it, but it's like, yeah, is that right? It's much better for us to be able to go through if we want to have a subject matter expert if we need to go in and actually check the raw evidence. — Tristan Ma

> [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) You can only fix what you can see. — Tristan Ma

> [18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) it's faster and it's cheaper—is that answer good enough? — Tristan Ma

## Mentioned Tristan Ma, Elastic, Elasticsearch, ELK stack, Elastic Agent Builder, Elastic Workflows, Kibana, Kibana Observability, Elastic APM, OpenTelemetry (OTel), Anthropic, Claude 3.5 Sonnet, Claude 3 Haiku, Redis, Model Context Protocol (MCP), Agent2Agent (A2A) protocol, Liquid, YAML

## Video description Tristan Ma - Manager, Solutions Architecture, Elasticsearch

How do you scale multi-agent intelligence without ballooning latency or costs? We built a system on Elastic using specialist agents coordinated through Workflows and Agent Builder.. We refined performance by adopting "Liquid Agent" patterns—offloading lookups to Workflows to slash token usage—and using a "measure-don’t-guess" methodology to cut latency. We also implemented granular token usage tracking within workflow indexes to enable deep historical analysis. Join us to learn how these strategies can optimize your own agent deployments for superior efficiency and observability.

[Source](https://www.youtube.com/watch?v=wJrKm_Rz75Y)

中文简报

Elastic 弃用单体 AI Agent：专业化工作流有效削减成本对 32,000 次真实分诊运行的遥测数据显示，单体多工具 Agent 因在每一轮评估所有工具 schema 而导致延迟和 Token 开销激增；相比之下，由专业领域模型构成的确定性工作流能防止上下文污染并大幅削减计算成本。 Tristan Ma 阐述了为何在生产环境中，通过确定性工作流协调模块化、专业化的 AI Agent，其表现始终优于单一的单体“超级 Agent”。他通过事件分诊与根因分析的演示，说明了 Elastic 如何同时充当上下文基础层与 OpenTelemetry 追踪存储库。通过采用声明式 YAML 工作流、使用 Liquid 模板预过滤负载以及实施索引化追踪日志记录，团队可以通过实证优化性能——包括针对性地从 Claude Sonnet 切换至 Haiku，在保持诊断准确性的同时大幅降低延迟。企业自动化正从单轮 RAG 演进为跨专业领域模拟人类团队协作的多 Agent 架构。 (0:04) 确定性 AI 工作流为标准运行手册提供了可审计且可复现的轨道，而自主 Agent 则擅长解决开放式、探索性的问题。 (2:02) 在生产规模扩展期间，协调模块化、专业化的 Agent 能防止上下文污染并降低 Token 成本，其表现始终优于单体“超级 Agent”。 (2:02) Elasticsearch 同时用作上下文基础源与 OpenTelemetry (OTel) 追踪存储，以实现端到端的 Agent 可观测性。 (4:33) 为子 Agent 分配特定领域的倾向（基础设施、应用遥测、搜索），可在综合证据之前防止噪声上下文干扰或压垮 LLM。 (4:33)

Reading summary

Elastic's Tristan Ma argues that moving enterprise automation from proof-of-concept into production requires replacing monolithic 'super agents' with deterministic, specialized multi-agent workflows. Using Elasticsearch as both a contextual knowledge base and an OpenTelemetry data store, teams can mitigate context poisoning, reduce latency, and provide fully auditable evidence trails. Specialization Over Monolithic Super Agents (2:02) While single-turn Retrieval-Augmented Generation (RAG) established reliable grounding for enterprise data, Ma notes it falters when faced with complex, multi-domain troubleshooting that typically requires cross-functional human expertise. To solve this, organizations must choose between building a single monolithic agent loaded with dozens of tools or orchestrating a fleet of specialized sub-agents. Elastic's internal Information Security team tested both approaches across more than 32,000 real-world triage iterations. Loading a single agent with 14 general tools forced the model to parse every tool definition on every turn, driving up token consumption and execution latency. By contrast, deploying targeted agents with isolated domain prompts reduced baseline token overhead, avoided 'context poisoning,' and delivered higher answer accuracy. Domain-Isolated Multi-Agent Incident Triage (4:33) To demonstrate specialized orchestration, Ma presented an incident triage and root cause analysis (RCA) pipeline executed through declarative YAML workflows in Elastic Agent Builder. The setup coordinates distinct specialist agents: Atlas handles initial routing, while Pulse (infrastructure), Lens (application telemetry), and Scout (search relevance) conduct isolated investigations using Elasticsearch as their contextual grounding source. In a live run investigating an overnight 15% drop in search conversions, Pulse identified Redis connection pool exhaustion, allowing an impartial RCA Synthesizer agent to separate the infrastructure fault from search relevance anomalies. The resulting Kibana report included direct, clickable links back to raw APM inventory and log views. Ma emphasized that raw provenance is mandatory for enterprise adoption, as practitioners rarely accept synthetic LLM answers without direct access to supporting telemetry. Latency Levers: Liquid Templating and Model Swaps (11:26) Because every intermediate thought, tool parameter, and output is indexed as an OpenTelemetry trace in Elasticsearch, engineers can systematically diagnose multi-agent latency bottlenecks. Ma highlighted Liquid templating and model switching as two practical levers for optimizing execution speed. Liquid templating extracts and formats only the specific fields required before passing context to the LLM, eliminating context window bloat and avoiding extra Model Context Protocol (MCP) roundtrips. Concurrently, swapping Anthropic's Claude 3.5 Sonnet for Claude 3 Haiku at targeted agent steps yielded a 15% latency reduction along with lower token costs, while indexing execution traces side-by-side enabled empirical quality comparisons. Evaluating Model Trade-Offs in Cascading Pipelines (18:24) Ma cautioned that evaluating model downgrades requires measuring downstream pipeline effects rather than isolated steps. In the RCA synthesizer role, Claude 3 Haiku matched Claude 3.5 Sonnet's core conclusion—both classified the incident as high risk rather than critical—though Haiku supplied less detailed supporting context. While Haiku proved viable for standalone synthesis tasks previously presumed to require frontier models, Ma noted that richer narrative context from one node may improve reasoning in downstream agents. Full trace logging across every stage allows system architects to empirically evaluate whether cost and latency gains compromise the accuracy of subsequent pipeline steps. While deterministic multi-agent workflows effectively cut costs and noise for standard operational runbooks like alert triage, open architectural questions remain regarding how best to support asynchronous, exploratory agents when an incident requires diverging from predefined paths.

精炼阅读摘要

Elastic的Tristan Ma指出，企业在将AI嵌入核心业务流程时，单一体庞大的“超级智能体”在生产环境中普遍面临严重的Token消耗、延迟激增与上下文污染；通过结合Elasticsearch底层检索与遥测能力，采用声明式确定性工作流编排多个专业化小模型智能体，能够在复杂的故障排查中实现更高的诊断精度、更低的运行成本与端到端的可审计性。确定性工作流与专业智能体选型 (2:02) Tristan Ma指出，单轮检索增强生成（RAG）难以应对需要跨团队协同排查的复杂企业难题。在系统架构演进中，企业必须厘清遵循固定标准作业程序的“AI工作流”与具备自主推理能力的“AI智能体”的职责边界：前者依托预定义步骤提供强约束的确定性执行与严密审计轨迹，后者则适用于探索未知路径的开放式问题。面向生产级规模化部署，Ma强调采用模块化专业智能体集群显著优于依赖单一搭载数十种工具的通用型“超级智能体”。这一架构路线得到了Elastic内部信息安全团队的实证支持。作为公司内部的“零号客户”，该团队在约3000名员工的组织规模下，依托遥测数据分析了超过32,000次警报分诊运行记录。结果表明，为单一智能体加载14种通用工具会导致模型在每个对话轮次都必须评估所有工具定义，产生巨大的基线Token开销与延迟，并引发上下文污染；而将任务拆解给限定职责的专用工作流，能在高吞吐告警场景下大幅压低推理成本并保障准确率。隔离偏置驱动的根因分析架构 (4:33) 为验证专业化多智能体理念，Elastic构建了以Elasticsearch为核心的根因分析（RCA）架构。在该体系中，Elasticsearch同时承担上下文知识库与OpenTelemetry（OTel）遥测存储库的双重角色。系统通过Atlas分诊智能体接收告警并派发任务，下设针对特定领域具备强偏置的子智能体：Pulse负责基础设施、Lens分析应用遥测、Scout核验搜索相关性。这种领域隔离指令有效阻断了无关遥测噪声造成的上下文污染，确保各专业模型保持高度专注。在一次模拟“搜索转化率隔夜下降15%”的复合故障排查中，Pulse智能体率先检测到Redis连接池耗尽迹象，而其他智能体并行排查应用层与搜索效果。随后，RCA合成智能体汇总各领域证据，客观判定该问题并非搜索质量劣化，而是底层基础设施资源瓶颈诱发的两起独立故障。所有调查结果由报告智能体沉淀至Kibana Case中，并附带直达APM日志与指标的交互式深层链接。Ma援引客户反馈强调，鉴于企业用户普遍对未经校验的LLM生成内容持审慎态度，提供可一键跳转核验的原始数据凭证是打消信任顾虑、实现企业级采纳的关键支撑。模板提取与模型降级降本增效 (11:26) 当一次根因分析执行耗时达到数分钟时，系统调优必须依托可量化的工程杠杆。Elastic提出了两项关键优化措施：Liquid模板字段提取与阶段性模型降级替换。传统架构常将完整的原始JSON载荷直接输入大语言模型，不仅造成上下文窗口膨胀，还带来额外的MCP往返延迟；借助Elasticsearch原生集成的Liquid模板，系统能在检索层就地抽取目标字段后再输入模型，有效减免了无谓的网络交互与数据载荷。另一项调优手段是针对特定智能体节点实施模型分级替换。在多智能体流水线的部分环节中，团队测试使用轻量化的Claude 3 Haiku替代高规格的Claude 3.5 Sonnet，实现了15%的延迟下降与更低的Token开销。得益于所有运行轨迹以原生方式在Elasticsearch中进行索引留存，工程团队能够针对不同模型的输出开展并排比对与差异化评估，将以往模糊的提示词调试转变为有据可查的经验性工程调优。全链路索引审计与级联质量评估 (18:24) 该多智能体系统的编排层完全依托声明式YAML工作流构建。Ma指出，选择YAML是由于大语言模型在结构化语法解析与代码生成上表现优异，工程人员与LLM均能稳定构建与修改流水线。在执行过程中，系统将用户输入、中间推理步骤、工具调用参数与原始输出实时写入Elasticsearch专用索引，这不仅提供了事后审查途径，还允许二次调遣LLM对历史执行轨迹开展元分析，以评估整体链路性能。在对Claude 3.5 Sonnet与Claude 3 Haiku的深入评测中，数据表明Haiku在承担RCA合成任务时表现出足够的实用性：它同样准确得出了故障属于“高风险”而非“严重风险”的核心结论，差异仅在于提供的支持论据详细程度较低。然而Ma强调，评估智能体质量不能局限于单个独立步骤，因为某一节点输出的上下文丰富度会直接作为下游节点的输入凭据，进而产生级联影响。借助全流程细粒度索引记录，系统架构师能够在确认输出质量达标的前提下，精确衡量速度提升、成本缩减与下游上下文密度之间的平衡点。 Tristan Ma总结指出，企业级多智能体系统的落地核心在于摆脱对单一全能型智能体的依赖，通过声明式工作流、开源协议及原生遥测索引打造透明可控的确定性架构。然而在实际工程中，轻量化模型在复杂多跳管线中的局部精简是否会因信息密度下降而对下游产生隐性负面级联，仍是一个需要企业依据持续索引的遥测轨迹进行长期实证评估的关键考量。

Detailed notes — zh

## Background 该视频由 YouTube 频道“Ai4”于 2026 年 8 月 18 日发布，主讲人为 Elasticsearch 解决方案架构经理 Tristan Ma。视频简介指出，本次演讲主要探讨如何利用 Elastic Workflows 与 Agent Builder 构建和扩展多智能体协同系统，并借助“Liquid Agent”模式、精准可观测度量以及细粒度 Token 追踪技术，在保证系统高可用性的同时大幅削减延迟与推理成本。 ## TL;DR 演讲系统性地梳理了企业 AI 从规则引擎、RAG 走向 Agentic AI 的演进路径，并深入剖析了确定性工作流（Workflow）与自主探索型 Agent 的架构权衡。结合 Elastic 内部基于 32,000 多次真实会话的实战评测，主讲人证明了“专业化小 Agent 矩阵”相比单体“超级智能体”在 Token 成本与准确率上的显著优势。现场不仅演示了基于 Elasticsearch 全链路审计与 YAML 编排排查“15% 转化率骤降”的根因分析流程，还重点分享了利用 Liquid 模板与高低阶模型（Sonnet 对比 Haiku）动态切换来兼顾低延迟与高精度的优化策略。 ## Atmosphere & style 整场演讲呈现出浓厚的一线工程实战风格与务实严谨的技术基调。主讲人通过现场举手互动与幽默生动的语言迅速拉近与观众的距离，随后借助清晰的系统拓扑图和真实的 Kibana 控制台演示逐步展开硬核内容。演讲全程贯穿着“用数据说话、拒绝凭空猜测”的工程师精神，既有面对多维故障排查时的敏锐剖析，也有对模型成本权衡与工具治理挑战的坦诚反思，给听众带来扎实、可信且极具实操指导意义的现场体验。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=4s) 企业自动化正从单次检索的 RAG 迈向模拟跨部门协作的 Agentic AI，以应对多视角、多领域的复杂业务排障。 - [02:09](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=129s) 确定性 AI 工作流（Workflow）提供严格可审计的执行轨道，而自主 AI Agent 适合目标发散的探索任务，两者在生产中有明确的边界划分。 - [02:09](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=129s) Elastic 内部基于 32,000+ 次真实会话测试，证实“专业化小 Agent 矩阵”在输出质量和 Token 成本控制上显著优于“单体全能超级 Agent”。 - [04:37](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=277s) 基于 Elasticsearch 的多 Agent RCA 架构由分诊 Agent（Atlas）将问题分发给专精基础设施、应用和日志的垂直子 Agent，避免全局大上下文导致的投毒与混淆。 - [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) 工作流采用声明式 YAML 语法定义，执行过程中的所有元数据、调用链和输入输出均原生索引至 Elasticsearch，支持全链路回溯与二次分析。 - [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) AI 自动化系统生成的 RCA 报告会附带指向 APM 和原始日志的深层链接，为人类专家（SME）核验提供事实依据，解决对黑盒 LLM 的信任问题。 - [11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) 采用 Liquid 模板化可以直接在结构化数据中提取所需字段，无需将海量 JSON 全量喂给大模型，显著降低 MCP 调用开销与推理延迟。 - [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) 单体全能 Agent 在处理任务前需在 Prompt 中加载所有工具定义，产生严重的上下文膨胀惩罚；而确定性工作流通过按需加载工具实现极致降本。 - [18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) 在根因分析综合（RCA Synthesizer）环节，轻量模型（如 Claude Haiku）在核心定性上与高阶模型（如 Sonnet）基本一致，主要差异在于论述详尽程度，但能大幅缩减成本与延迟。 - [18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) 多 Agent 协同具有级联效应，上游节点的输出细节会直接影响下游 Prompt 上下文，端到端索引化的可观测性让团队能够客观评估模型切换带来的全链路连锁反应。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=4s) AI 自动化的演进：从规则引擎、RAG 到 Agentic AI 演讲开场时，主讲人通过现场举手互动调查了观众对 Elastic、Elasticsearch 以及 ELK 技术栈的认知。Elastic 不仅是一个开源的全文搜索引擎，更是一个强大的分析引擎，广泛支撑着企业的搜索、可观测性（Observability）与安全（Security）三大核心场景。正因为其平台覆盖面的深度与广度，Elastic 团队得以近距离观察大量企业客户在引入人工智能时的真实演进路径。如今，各行各业的研发团队都承受着来自管理层和董事会的巨大压力，被要求将 AI 快速整合进日常工作流中，这也推动了自动化诉求的不断成熟。回顾自动化的发展历程，最初是大家熟知的“基于规则的自动化”，按既定逻辑执行虽然可靠，但人们总会抱怨系统不够聪明，只要遇到一丝微小的细微差别就会失效。随后 RAG（检索增强生成）应运而生，并在生产环境中展现出巨大的价值——它不仅保留了私有数据的安全性，还通过外挂企业私有数据为大语言模型（LLM）提供了充分的上下文支撑，使答案更具推理依据且能够追溯引用来源。然而，面对日常复杂的自动化业务，RAG 这种“单次检索、单次作答”的模式逐渐显现出局限性。在现实团队协作中，人们排查问题通常不会只咨询一个人，而是会跨多个部门与拥有不同视角的同事讨论。为了在 AI 体系中复刻这种协作方式，行业正式迈入了 Agentic AI（智能体 AI）的新阶段。 ### [02:09](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=129s) 确定性工作流 vs. 探索型 Agent，以及专业化 Agent 的优势在 Agentic AI 的具体落地中，存在两种关键形态：AI 工作流（AI Workflow）与自主 AI 智能体（AI Agent）。AI 工作流非常适合“确定性路径”，最典型的例子就是标准运维手册（Runbook）。它遵循已知的排查步骤，由 LLM 进行有序的逻辑推理，更为关键的是，它提供了严格的轨道（Rails），具备极高的可审计性、可追踪性与操作日志记录能力。相反，自主 AI Agent 则更具探索性，系统为其配备丰富的工具和技能由其自由决策，适合那些终点并不明确的探索型任务，但其代价就是答案的溯源与审计难度大幅上升。围绕 Agent 的架构设计，业界目前存在着激烈的技术选型讨论：究竟是构建一个集成所有工具技能的“超级智能体”（Super Agent），还是采用多个各司其职的“专业化智能体”（Specialized Agents）协作？主讲人强调，在面对生产环境的“Day 2”（次日运维与规模化运行）阶段时，控制调用成本与延迟至关重要。Elastic 内部的信息安全（InfoSec）团队在经过超过 32,000 次真实会话的严格评测后发现：采用专业化 Agent 的方案不仅在生成质量上更胜一筹，在 Token 消耗与成本控制上也展现出明显优势。与其彻底放手让模型无约束地“自主思考”（"think for yourself"），不如通过带有约束的工作流结合领域专长，打造出兼具严密推理与确定性结果的落地方案。 - **AI Workflow 核心价值**：步骤明确可控、符合运维规程、具备极强的合规审计与执行轨迹追踪能力。 - **AI Agent 适用边界**：工具自主调度、探索未知路径，适合非线性、目标发散的复杂分析场景。 - **规模化评测数据**：Elastic 安全团队基于 32,000+ 场实际会话测试，验证了专业化 Agent 在准确率与 Token 成本控制上的双重优势。 ### [04:37](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=277s) 基于 Elasticsearch 的多 Agent 协同架构与 RCA 诊断流主讲人通过现场架构图拆解了整体技术实现方案。该系统完全构建在 Elasticsearch 之上，Elasticsearch 在其中承担着双重核心角色：一方面作为底层知识库，为智能体提供事实接地（Grounding Back End）；另一方面作为 OpenTelemetry（OTel）遥测数据的持久化存储，全面收集会话与执行轨迹，以满足系统迭代所需的可观测性。系统入口由一个名为 `Atlas` 的分流与分诊智能体（Triage Agent）把守，它根据问题将任务分发给三个高度专业化的垂直子智能体：负责基础设施的 `Pulse`、负责相关性与应用的 `Lens` 以及负责安全/日志探索的 `Scout`。 ```mermaid flowchart TD accTitle: Multi-Agent RCA Architecture on Elastic accDescription: Triage agent Atlas distributes work to specialized domain agents, aggregating in RCA Synthesizer and logging to Kibana. subgraph Entry ["分流与接入层"] Q["用户复杂排障请求"] --> Atlas["分诊总控 Agent: Atlas"] end subgraph Specialists ["专业化子 Agent 矩阵"] Atlas --> Pulse["基础设施专家: Pulse"] Atlas --> Lens["应用与相关性专家: Lens"] Atlas --> Scout["日志与检索专家: Scout"] end subgraph Analysis ["根因分析与输出层"] Pulse --> Synthesizer["根因合成 Agent: RCA Synthesizer"] Lens --> Synthesizer Scout --> Synthesizer Synthesizer --> Reporter["报告生成器: RCA Reporter"] Reporter --> KibanaCase["Kibana / 外部工单案例系统"] end subgraph DataStore ["统一存储与遥测层 (Elasticsearch)"] Atlas -. 遥测跟踪.-> ES[("Elasticsearch: 知识接地 + OTel 数据源")] Pulse -. 检索事实.-> ES Synthesizer -. 记录审计.-> ES end ``` 在设计这些子 Agent 时，团队有意让每个子智能体对其所属领域保持高度“偏见”（Biased towards their point of view）。这种设计高度还原了跨部门协同的真实流程——每个领域的专家（SME）都优先从自身领域审视问题，不仅能大幅降低全局大上下文带来的“上下文投毒”（Context Poisoning）与模型认知混淆，还能让每个模型专注提供最精准的证据。随后，所有证据被汇总至 `RCA Synthesizer`（根因合成智能体），该智能体仅专注于比对分析各领域专家提交的客观证据并做出最终定性。最后由 `RCA Reporter` 生成完整的根因分析报告，并自动沉淀至 Kibana Observability 的案例管理模块（亦可对接外部任意工单系统）。整个智能体集群由 Elastic Agent Builder 统一构建（支持 API 与 UI 两种配置途径），并通过预设工作流实现所谓的“轨道化 RCA 诊断”（RCA on Rails）。 ### [07:39](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=459s) 实战演示：15% 转化率骤降的多维根因排查在现场 Demo 环节，主讲人抛出了一个生产中常见的复杂多维故障场景：“搜索转化率在一夜之间断崖式下跌了 15%，这究竟是搜索结果变差导致的‘相关性问题’，还是底层的‘基础设施故障’？”面对这种涉及多部门边界的模糊问题，传统的单领域智能体或标准 RAG 系统往往会顾此失彼。在演示界面左侧，专精于基础设施监控的 `Pulse` 智能体首先给出了局部探测结果：检测到了 Redis 连接池耗尽（Redis connection pool exhaustion）的异常信号。然而，`Pulse` 无法单凭这一指标下定论，它只能提供基础设施层面的可疑线索。与此同时，总控智能体 `Atlas` 依托 Elastic Agent Builder 启动全链路推理，并在 Kibana 的 Elastic Workflow 页面上实时展开逐步调度。控制台清晰展示了每一个子 Agent 的输入上下文、思考轨迹与独立输出，工程师可以完整追踪诊断的推导链条。在 Elastic Agent Builder 中，开发者可以为各 Agent 细致配置系统指令、Agent ID 以及专属的 Elastic 检索与分析能力，使大模型既能发挥其强大的语义理解力，又时刻运行在可观测、可审计的确定性工作流轨道之中。 ### [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) YAML 声明式工作流与 Elasticsearch 根因分析审计在演示工作流执行的过程中，系统正在后台针对警报进行根因分析（RCA, Root Cause Analysis）的综合推理。所有执行过程中的元数据、输入输出以及调用链都会完整写入 Elasticsearch 索引中。这意味着所有步骤都可以被独立查询，用户甚至可以让大语言模型对自身历史执行的输入输出链进行二次分析与复盘。在工作流定义方面，团队特意选用了基于 YAML 的声明式语法。谈及这一架构决策时，Tristan 幽默地调侃道：“LLM 偏爱 YAML，天晓得为什么，但它的确结构严谨且易于模型生成。”这种设计使得工程师既能通过明确的规范定义工作流，也能直接借助 LLM 工作流构建工具（Workflow Builder Skill）自动生成流程。 ```mermaid flowchart TD accTitle: 基于 Elasticsearch 的告警分诊与根因分析可观测架构 accDescription: 告警进入系统后触发基于 YAML 的确定性工作流，调用专用 Agent 分析并存储至 Elasticsearch 索引，提供证据链接。 subgraph Trigger["告警触发层"] A["监控与安全告警流入"] --> B["YAML 声明式工作流引擎"] end subgraph Execution["专用 Agent 编排层"] B --> C1["专用 Agent A: 指标分析"] B --> C2["专用 Agent B: 日志比对"] C1 --> D["根因分析 (RCA) 综合推理"] C2 --> D end subgraph Grounding["Elasticsearch 数据与证据层"] D --> E["写入 ES 索引 (全链路审计)"] D --> F["生成 APM / 错误实例原始跳转链接"] E --> G["分析师复核 & SME 原文核验"] F --> G end ``` 执行完成后，左侧面板展示了完整的 RCA 诊断结果：系统精准判断出当前并非简单的相关性问题，而是由两个互不相同的独立故障引发的基础设施级根因。更关键的是，系统生成了指向 Elasticsearch APM 资产库的深层链接（Deep Link），分析师可以直接点击跳转至产生异常的具体代码和环境。Tristan 现场向观众提问并指出，大部分企业客户在实际业务中“根本无法完全信任大模型独立给出的结论”——这也是当下业界的普遍共识。在真实的警报分诊（Alert Triaging）场景中，预先自动触发的工作流能够为分析师快速规划排查路径，并直接附带模型进行事实锁定的原始证据（Grounding Evidence），既保留了人类领域专家（SME）的核查能力，又大幅压缩了排障响应时间。 - **全链路索引化**：所有 Agent 执行步骤、输入输出上下文均原生存储于 Elasticsearch 索引中，支持即时搜索与审计。 - **声明式 YAML 规范**：降低 LLM 生成与修改工作流的门槛，兼顾机器可读性与人类工程化维护。 - **事实锁定与追溯**：自动关联 APM 和底层日志链接，解决企业对 LLM 幻觉和“黑盒决策”的不信任痛点。 --- ### [11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) 延迟与成本调优：Liquid 模板化与模型切换（Sonnet 对比 Haiku）工作流全链路审计的核心价值在于为性能调优提供坚实的数据支撑，避免盲目猜测。针对现场演示中耗时约 2 分钟的执行流，Tristan 提出了两套具体的优化手段：“Liquid 模板化（Liquid Templating）”与“动态模型切换（Model Switching）”。 Liquid 模板化的核心逻辑在于利用 Elasticsearch 自身作为数据对齐与检索层（Grounding Agent）的能力。传统做法是将查询返回的海量完整 JSON 数据直接丢给 LLM 解析，这不仅极度消耗上下文窗口，还会显著拉高推理延迟。而借助 Liquid 模板，系统可以直接指示工作流在已完成解析的结构化输出中精准提取指定字段，通过 Elastic 原生调用完成数据组装，从而直接减少一次 MCP（Model Context Protocol）调用，避免在上下文之间传递冗余数据。在模型切换方面，Tristan 展示了将高阶模型（如 Claude Sonnet）替换为轻量级模型（如 Claude Haiku）的基准对比实验。尽管这看似显而易见——“因为 Haiku 是个更便宜的模型，理所当然跑得更快”，但在真实工程中，核心问题在于回答“速度提升是否伴随着质量滑坡”。通过对比索引中记录的每次 Agent 输入输出，团队测出在小样本测试中 Haiku 带来了约 15% 的延迟降低，但输出结论存在细微差异。借助这些审计数据，工程团队才能基于可复现的结果展开细致的技术评审，权衡是否接受轻量模型带来的差异。 --- ### [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) 内部安全团队的实战数据：单体大 Agent 与专用小 Agent 的代币成本权衡随后展示的进阶工作流源自团队内部的一线实践：该流程完全由外部安全告警触发，自动切入 RCA 分诊链路，并对输入和输出 Token 消耗进行精确计量。Elastic 自身的 InfoSec（信息安全）团队充当了安全产品的“零号客户”（Customer Zero）。作为一个约 3,000 人规模的中型企业，Elastic 并没有庞大的全球安全运维大军来人工处理每一起安全事件，因此内部团队基于该系统累计跑了超过 32,000 次真实迭代。 ```mermaid flowchart LR accTitle: 单体全能 Agent 与确定性专用 Agent 架构对比 accDescription: 展示多工具单体 Agent 在上下文选择时的 Token 开销，与确定性工作流中专用 Agent 高效执行的对比。 subgraph Monolithic["模式 A: 单体全能 Agent (14 Tools)"] direction TB M1["接收告警"] --> M2["全量载入 14 个工具定义"] M2 --> M3["LLM 反复遍历评估工具匹配度"] M3 --> M4["高 Token 开销 & 高上下文膨胀"] end subgraph Specialized["模式 B: 确定性多 Agent 工作流"] direction TB S1["接收告警"] --> S2["确定性编排路径"] S2 --> S3["专用 Agent 1 (仅加载必要工具)"] S3 --> S4["专用 Agent 2 (专项深入)"] S4 --> S5["极低 Token 消耗 & 高吞吐"] end ``` Tristan 强调了一句关键原则：“你只能修复你能看见的问题（You can only fix what you can see）。”在缺乏端到端指标监控时，团队往往只能凭空争论架构优劣；而通过工作流索引中的实测数据，Elastic 证明了“单体全能 Agent（One giant agent with many skills）”与“确定性专用小 Agent（Specialized agents）”之间的巨大经济性差异： - **单体 Agent 的上下文惩罚**：如果为一个庞大的通用 Agent 配备 14 个工具，模型在做出任何实际动作之前，都必须在 Prompt 中加载这 14 个工具的完整定义与元数据，并逐一评估“选这个、选那个还是都不选”。这会导致每次交互都伴随着巨大的无效 Token 开销与延迟。 - **确定性工作流的降本优势**：对于具备明确业务 SOP 的告警分诊场景，采用确定性流水线调度专用小 Agent，可以精准隔离上下文并按需加载工具，极大压缩 Token 开销。 - **研发与生产环境的权衡**：在 Dev 阶段验证原型时单体 Agent 尚可接受，但一旦推向生产环境面对海量告警，上下文膨胀带来的成本惩罚将变得无法承受。 --- ### [15:42](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=942s) 开放生态与现场问答：工具演进、异步编排与模型输出差异在总结发言中，Tristan 强调 Elastic 秉持开源与开放生态理念，全面支持 A2A（Agent-to-Agent）等开放协议与工具导入，用户绝不会被锁定在单一闭环系统内（并现场邀请观众会后前往 311 展台深入交流）。随后，演讲进入观众现场问答环节： - **关于“单体通用 Agent”与“专用独立 Agent 套件”的技术演进路线**：现场观众询问系统是在向单一统领全局的“深度 Agent”迁移，还是坚持独立 Agent 组合。Tristan 澄清，这本质上是“AI Agent”与“AI Workflow”的定位抉择。在告警分诊这种以高吞吐、降噪为核心诉求的场景中，确定性工作流编排专用小 Agent 是目前控制 Token 成本和提升准确率的最优解。同时，平台不仅支持单线串行链路，也完全支持异步分支（Async Workflows）探索，只是为了本次现场演示的清晰度而展示了确定性串行流。 - **关于 MCP 工具随时间迭代与变更的治理难题**：针对观众关于“MCP 工具接口发生版本变动时该如何应对”的提问，Tristan 表示这同样依赖于平台的审计与版本控制机制。Elastic 提供了专属的 MCP 端点服务，支持工具的导入、导出、重命名和版本追踪。尽管工具治理本身仍是一个处于快速演进中的行业课题，但平台已具备了基础的版本管控能力。 - **关于模型切换（Haiku vs. Sonnet）时的输出一致性细节**：有观众追问 Haiku 是否真的能够输出与 Sonnet 完全一致的诊断结果。Tristan 明确回应两者在细节表达与分析深度上确实存在差异，并准备切回对比界面为观众具体展示两者在相同上下文下的回答异同。这也再次印证了对全链路推理结果进行持久化索引和可观测评估的必要性。 ### [18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) 轻量级模型在 RCA 场景的实测表现与多 Agent 链式影响在对比 Claude 3.5 Sonnet 与轻量级模型 Claude 3 Haiku 的实际表现时，演讲者指出，两者在最终结论上并没有本质差异，主要差距体现在支撑结论的证据深度与细节丰富度上。例如在风险评估中，Sonnet 给出的论据更为详实充分，但 Haiku 同样准确识别出了核心问题并标记为高风险。这种基于 Elastic Workflows 完整审计追踪能力得出的量化结果，改变了团队的技术选型讨论方式——从空泛地争论“Haiku 能不能做、Sonnet 有多强”，转变为基于真实数据的客观评估：“Haiku 响应更快且成本极低，这样的输出质量对当前业务而言是否已经足够好？让我们用数据说话（Let's prove it）。” 演讲者坦言，Haiku 在根因分析综合器（RCA Synthesizer）环节的表现令人惊喜。通常人们直觉上认为综合多方信息做归因分析必须依赖大参数量的高阶模型，但实测证明轻量级模型完全可以胜任，只是在论述深度上有所精简，呈现出不同的风格与侧重点。这使得用户或系统审计人员能够灵活权衡：是追求极致性价比与低延迟，还是为关键业务环节采购更高质量与深度的输出。 ```mermaid flowchart TD accTitle: Multi-Agent Model Tradeoff and Chaining Effect subgraph SingleStep["单节点模型权衡 (以 RCA Synthesizer 为例)"] Sonnet["Claude 3.5 Sonnet • 证据充分详实 • 延迟与成本较高"] Haiku["Claude 3 Haiku • 结论一致、深度略减 • 极低延迟与推理成本"] end subgraph ChainingEffect["多 Agent 链式级联影响"] CurrentStep["当前步骤输出 (Output)"] NextStep["下游 Agent 输入 (Input)"] FinalOutcome["端到端最终决策质量"] CurrentStep -->|"更丰富的上下文"| NextStep NextStep --> FinalOutcome end SingleStep -.->|"评估是否足以支撑下游"| CurrentStep ``` 同时，这种模型选型还会对多 Agent 系统的级联链路产生潜在影响。演讲者在本次 Demo 中主要聚焦于单个处理节点的模型对比，但他也特别指出，在复杂的 Multi-Agent 编排中，上游 Agent 输出的详细程度直接决定了下游 Agent 接收到的 Prompt 上下文质量，从而可能对后续整个链路的输出产生链式连锁反应。得益于系统在工作流索引中完整记录了每一步骤的输入（Input）与输出（Output），架构师既可以孤立测试单个节点，也能全局审视全链路的综合表现，为后续系统调优提供了坚实的可观测性与审计基础。 - **结论一致性 vs 细节深度**：Haiku 在 RCA 场景下能够得出与高阶模型相同的核心结论，差异仅在于输出证据的详尽程度。 - **轻量模型的可用性验证**：打破了复杂归因与信息综合必须使用最顶级大模型的固有认知，为大幅削减推理 Token 与延迟提供了可行路径。 - **端到端链式审计能力**：Elastic Workflows 记录了链路中每一步的 Input 和 Output，支持开发者既评估局部节点的性价比，也能全局追踪上下文级联对整个 Agent 系统的影响。 ## Notable quotes > [02:09](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=129s) think for yourself — Tristan Ma > [04:37](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=277s) Biased towards their point of view — Tristan Ma > [04:37](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=277s) RCA on Rails — Tristan Ma > [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) LLM 偏爱 YAML，天晓得为什么，但它的确结构严谨且易于模型生成。 — Tristan Ma > [09:05](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=545s) 根本无法完全信任大模型独立给出的结论 — Tristan Ma > [11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) Liquid Templating — Tristan Ma > [11:26](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=686s) 因为 Haiku 是个更便宜的模型，理所当然跑得更快 — Tristan Ma > [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) Customer Zero — Tristan Ma > [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) You can only fix what you can see — Tristan Ma > [13:28](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=808s) One giant agent with many skills — Tristan Ma > [18:24](https://www.youtube.com/watch?v=wJrKm_Rz75Y&t=1104s) Haiku 响应更快且成本极低，这样的输出质量对当前业务而言是否已经足够好？让我们用数据说话 — Tristan Ma ## Mentioned Tristan Ma, Elastic, Elasticsearch, ELK, Kibana, Elastic Agent Builder, Elastic Workflows, OpenTelemetry (OTel), Atlas, Pulse, Lens, Scout, RCA Synthesizer, RCA Reporter, Claude 3.5 Sonnet, Claude 3 Haiku, Liquid, MCP (Model Context Protocol), A2A (Agent-to-Agent) ## Video description Tristan Ma - Manager, Solutions Architecture, Elasticsearch How do you scale multi-agent intelligence without ballooning latency or costs? We built a system on Elastic using specialist agents coordinated through Workflows and Agent Builder.. We refined performance by adopting "Liquid Agent" patterns—offloading lookups to Workflows to slash token usage—and using a "measure-don’t-guess" methodology to cut latency. We also implemented granular token usage tracking within workflow indexes to enable deep historical analysis. Join us to learn how these strategies can optimize your own agent deployments for superior efficiency and observability. [Source](https://www.youtube.com/watch?v=wJrKm_Rz75Y)

Source quotes

[2:09] Tristan Ma: think for yourself [4:37] Tristan Ma: Biased towards their point of view [4:37] Tristan Ma: RCA on Rails [9:05] Tristan Ma: LLM 偏爱 YAML，天晓得为什么，但它的确结构严谨且易于模型生成。 [9:05] Tristan Ma: 根本无法完全信任大模型独立给出的结论 [11:26] Tristan Ma: Liquid Templating [11:26] Tristan Ma: 因为 Haiku 是个更便宜的模型，理所当然跑得更快 [13:28] Tristan Ma: Customer Zero [13:28] Tristan Ma: You can only fix what you can see [13:28] Tristan Ma: One giant agent with many skills [18:24] Tristan Ma: Haiku 响应更快且成本极低，这样的输出质量对当前业务而言是否已经足够好？让我们用数据说话

Source quotes

[0:04] Tristan Ma: if only it could be smarter [0:04] Tristan Ma: if only it could just catch that small nuance. [7:37] Tristan Ma: Search conversion dropped 15% overnight—is this a relevance problem (i.e., are my search results bad) or is this an infrastructure problem? [9:05] Tristan Ma: LLMs love YAML. Who knows why? But it's very structured. It kind of knows how to write it. [9:05] Tristan Ma: What we're finding in our conversation with customers is most of our customers do not trust the LLM output by itself, right? [9:05] Tristan Ma: We trust it, but it's like, yeah, is that right? It's much better for us to be able to go through if we want to have a subject matter expert if we need to go in and actually check the raw evidence. [13:28] Tristan Ma: You can only fix what you can see. [18:24] Tristan Ma: it's faster and it's cheaper—is that answer good enough?

Original transcript

[0:04] My talk today is about multi-agent orchestration with elastic. Um, now [0:08] quick show of hands. When I say the word elastic, if I were to say elk, [0:11] elastic search, elkstack, quick show of hands on who may recognize that. All right, cool. So, [0:17] partially um for those of you who don't recognize it, elastic is an open source full text search [0:21] engine um as long as as well as an analytics engine and that will play a part in my talk today. [0:27] So because of Elastic's wide breath of what we can do as a platform supporting both search observably [0:33] and security use cases, we've actually been able to see quite a bit of our customers in their [0:37] adoption of AI. Now we're all an AI conference, so I can understand we're all very pro AI, [0:41] but there's quite a bit of executive pressure and board pressure across the board to institute AI [0:46] into the daily workflows. Everyone's heard that a couple times, maybe a lot. Um, and so we're kind [0:51] of seeing this maturation of this desire around automation, right? So of course most of us are [0:57] familiar with the rule-based kind of automation. Makes sense. You have a process rule goes awesome, [1:02] right? And in those days we always wished if only it could be smarter, right? If only it could just [1:06] catch that small nuance and it would have worked. Well, inter rag, right? Rag was an excellent [1:11] iteration and still is a wonderful production architecture for retrieval augmented generation to [1:16] allow the LMS to have improved reasoning but still ground it in the retrieval and augmented element [1:21] of your own private data. Ideally, you shouldn't be trained on your data, but it is what it [1:25] is. um with the private data element here, right? Rag proved to be a wonderful addition to make it [1:29] so that way the LMS had more context, gave you the answer, and we could trust a little bit more or at [1:33] the very least we could site our sources on where the LLM used for their answers. But that pushed [1:40] us all further. Right now we're at the point now where it's like rag is awesome, but the initial [1:44] retrieval and just one-time answer doesn't work for a lot of our day-to-day automations. Right? [1:49] Often times, you think about how you talk to your own peers. We're not just asking one person. [1:53] if we're lucky, we can, but most of the time we're talking to multiple different folks with their own [1:57] opinions that come into it. So to mimic that in the AI world, we've now kind of all been in this [2:02] new agentic AI insert the marketing phrase here kind of era. But the way that I think about it is [2:09] we're actually kind of at this point where there's this concept of an AI workflow versus an AI agent, [2:14] right? And both can be agentic in nature, right? But there's a different place for each one, [2:19] right? When we look at something like an AI workflow, right? My thought around this is [2:24] if it's a deterministic path, right? If you have a runbook as an example, an excellent example of [2:28] an AI workflow, right? It's a known procedure, we can have the LLM reason through it and it works. [2:33] And more importantly, for those of you who need to prove auditability, uh, trackability, logging, [2:38] this is all in a sequence where you can actually see these are the steps it went through because [2:42] these are the rails that I provided it. Now, on the other edge, we have AI agents. [2:46] also an excellent implementation of time, but this is where you're going to give it variety of tools, [2:50] skills for it to reason through on its own. And these are best for when you don't necessarily [2:53] know where you want it to land, right? It's going to be a bit more explorative in nature, [2:58] but by contrast, that means when you need to prove that you that where it got its answer from, [3:02] it can be a little bit more challenging. But that doesn't necessarily mean in any way that's [3:06] that one architecture is better than the other. But what I'll be going over today [3:10] is this other concept, right? And I know many of us have opinions and I don't mean to start [3:15] any tribal wars. Um, but in general there's this general concept of should we have one I'm gonna [3:20] call it a super agent if you will with multiple skills and tools or should we have specialized [3:24] agents. Right now again I'm not here arguing one way or the other which way we should go [3:30] but there is a real kind of conversation around this specifically when you look at day two. [3:35] Right? We're not talking here about oh when I want to dev something or show something off [3:39] to my internal team. We're talking about when we need to scale to production, how can we actually [3:42] control a lot of the inherent costs that come with scaling these kind of AI workflows, right? [3:47] And having kind of specialized agents, what we found in our own kind of experience, and [3:52] I actually have some slides from our own infosc team on their testing over 32,000 conversations, [3:57] we're finding that the specialized agents path tends to be one that proves a little bit more [4:01] beneficial both from a answer standpoint, but also from a token standpoint as well. So [4:06] that's what I'll be presenting on today uh for my demo today. We'll be switching here in a moment. [4:11] Um but the idea is I'm not declaring that this is the beall endall path, but this is a way that I [4:15] believe that more of our customers and people in general can start adding AI into their workflows, [4:21] pun intended, in order to allow us to actually have a provable method of this is how we're [4:25] using the LM through its reasoning and context. But that way we don't have to necessally unleash [4:29] it all right and say, "Hey, think for yourself, right? We're able to provide our own expertise [4:33] with this." All right. Uh, if you could switch to the HDMI. Awesome. All right. So, what we've built [4:39] today, and this is just a quick architecture overview. This is all built on elastic search, [4:43] right? Which is going to function both as our grounding source as well as our hotel store [4:48] as well because the most important element around building any new application is we need to be able [4:51] to observe it because how else are we going to iterate around this? Now, this become a principal [4:55] point of today's talk. But basically, the general consensus is I have an triage agent called Atlas [5:00] that will then call out to specific sub aents that are specialized. So we have pulse, lens and scout. [5:06] Each of them have their own domain expertise. Oh uh yeah sorry um [5:17] better good sorry about that. All right so we have pulse lens and scout. All of these have [5:23] their own specific domain that we're targeting, right? And the idea around this is I want my sub [5:28] agents to be as biased as possible towards their point of view because it mimics how we [5:32] interact today with our own sub teams, right? Think about it when you're triaging an issue. [5:35] You're always asking for another team or a subject matter expert, if you will, to provide their lens [5:40] on a problem. So, they're naturally going to be biased towards it. And in fact, this allows us [5:44] to avoid things like context poisoning. Right? I'm not saying this the solve for it, but this allows [5:48] the LM to really focus on what you're asking for and not necessarily get kind of confused with too [5:52] much information. Right? One of the negatives of having a holistic agent with many skills. [5:57] The next thing it does is it sends all of its findings, these three separate agents [6:01] to an RCA synthesizer. So the only job that this RCA synthesizer agent has is to look at [6:06] all the evidence presented by the domain experts and then make a call based on that information [6:11] alone. So we know we have good information for the RCA to look at and it's purely focused on [6:15] answering the question. The next thing we do and this is more for reportability down the line. [6:20] We can write out an entire RCA reporter and then for our case I went ahead and uh logged it all in [6:25] a Cabana observability case. For those you don't know Cabana is our visualization layer. Um but [6:30] again this could technically in theory be pushed out to any case management system you have but [6:33] for the sake of the demo and for simplicity that's kind of what we went with today. Now in terms of [6:38] some building elements so this agent creation. So Atlas and all of this was built on elastic agent [6:43] builder. Now um I know most of us probably more fans of APIs than UIs, but so I used the AI API [6:49] example here on how to create its instructions, subscriptions, and the tools it has access to. [6:53] The most important to pay attention to today is this workflow, and we'll go into that in a moment. [6:58] But we can also do this via UI as well. I know we say no UI required, but sometimes UIs are nice. [7:03] The next thing we're going to look through is that multi- aent workflow I just mentioned. [7:06] So this is my kind of RCA on Rails, if you will, right? where I'm defining for it, hey, [7:11] this is the steps. Step one, two, defining which agent uh AI agent uh sorry, AI model, excuse me, [7:17] to detect the issue. And we'll go through all this a little bit more in detail. And then lastly, [7:21] this is the hotel instrumentation example here as well. Um and then this just down below just [7:25] to show you these are all liveel feeds. And again, this is all feeding into elastic search. So again, [7:30] elastic search is operating both as our grounding back end for our agents, but also as our data [7:34] store for all the subsequent telemetry for these conversations. All right. So let's actually get to [7:41] arena and I apologize I might have to zoom out a little bit to fit this in but the premise around [7:46] this is we want to be able to ask a multiaceted question right and this is where things like rags [7:52] example or single domain agents can fall over right so in this scenario what I'm asking it [7:57] is search conversion dropped 15% overnight is this a relevance problem i.e are my search results bad [8:02] or is this an infrastructure problem? Now on the left hand side, what you are actually seeing is [8:06] a UI representation of what the pulse agent is capturing, right? That's our infrastructure only [8:10] agent. So it's able to tell me, hey, looks to be a Reddispool connection exhaustion, [8:14] but it doesn't know much beyond that and it's not necessary definitive proof, right? It just says, [8:18] hey, it's maybe the infrastructure I have a signal versus on the Atlas side [8:23] as it thinks through this reasoning, right? And this is all exposed for our Asian builder UI here. [8:28] you're able to see how it's reasoning, but more importantly, we're able to follow along on the [8:32] execution of this in our elastic workflow page. So again, we're here in Cabana, and the wonderful [8:37] part about this is as we're able to walk through the investigation, we can actually see the inputs [8:42] given to each AI agent and their subsequent outputs, right? So, we're we're going to give [8:47] this a moment to finish, but I just want to go ahead and showcase on the UI element, these are [8:51] all of my defined agents within the elastic agent builder. And within here I'm able to define system [8:56] references, agent ID, custom instructions, adding in additional elastic capabilities, right, where [9:01] it knows how to use elastic more efficiently. And then lastly, we have some other elements [9:05] around uh different workflows that I can use before the a before the LM is actually called. So [9:12] as we continue through this, we can see that it's synthesizing on the RCA, right? And this will all [9:18] end up showing up here on our lefth hand side as it finishes. But this is just to kind of show [9:22] folks um kind of some of this backend elements. And what is very important about this is if we [9:27] go to a successful execution here that I ran a couple hours ago, all this information is simply [9:33] in an index in elastic. So all of this can be queried independently. You can even have the [9:37] LLMs kind of run its own set of analysis if you'd like to look over, hey, these are the inputs and [9:42] outputs. These are the chains. And you can run your own analysis around this, especially as [9:47] all of this is directly queryable through Alaska search. But more importantly, if you look on the [9:50] workflow side here, you'll see that it's all YAML declarative. That was a deliberate choice on our [9:54] part. LLMs love YAML. Who knows why? But it's very structured. It kind of knows how to write [9:59] it. And more importantly, it allows us to have a more definitive way to define our workflows, [10:03] but more importantly allows us to have LLM create this for us. So I made this all just with our own [10:08] workflow builder skill. Um, but the per that was weird. Um, but the idea around this again [10:13] is showing the ability to kind of audit this as it goes through its various iterations. So [10:19] if I go back to executions now, it should be done. Perfect. All right. So we can go back [10:23] and see we actually have the full answer here, right? It's able to tell us it's actually two [10:30] distinct incidents. It's not a relevance problem. Has infrastructure cause, right? And it's able to [10:34] give us all the various elements. And again, because we're storing this all in elastic, [10:38] we can actually jump to these specific uh links here for us to look at. So, if we were to click [10:43] on the APM inventory link, we can go ahead to that exact air because what we're finding in [10:47] our conversation with customers is most of our customers do not trust the LLM output by itself, [10:52] right? Quick show of hands, right? In general, we're probably yeah, a little bit different, [10:56] right? We we trust it, but it's like, yeah, is that right? It's much better for us to be able to [11:00] go through if we want to have a subject matter expert if we need to go in and actually check [11:04] the raw evidence. But as we think about this, the workflows can be fired automatically ahead of time [11:09] for alert triaging as well. So if you think about that element, we're actually able to discern very [11:14] quickly an investigative path for our analysts who need to rise to that level to address the alert, [11:19] but we're able to give them the evidence where the LLM was grounding. All right. So [11:26] what does all of this audibility actually give us? Well, what it gives us is the ability for tuning, [11:32] right? The often question is how can we improve things like latency? Right? That took me 2 [11:36] minutes. 2 minutes may be fast enough for me. It may not be, right? But there should be decisions [11:40] that I can make from the auditability of the data to improve it. Right? I need to be able to show [11:44] proof of what I'm doing. So I'm going to show two concepts today. One is liquid templating, right? [11:49] And the other is model switching. model switching pretty self-explanatory but liquid templating [11:53] since elastic is operating as our grounding agent we actually don't need for the LLM to parse out [12:00] the entire JSON dump of the output right we can actually just tell it look for these specific [12:05] fields in the output because it's already parsed out and we're calling elastic natively [12:10] that's one less MCP call less context moving around and it's much faster on the latency side [12:16] and then the other element is around model switching right that's song. Love the demo gods. [12:21] All right, so this is a really basic one, right? But the idea around this is how can I make it [12:26] faster? One of the easiest ways I could think of is well, I'm using Sonnet today. Not saying this [12:31] is the be all end all. We can use OpenAI or what or any other models you can think of, but I want [12:35] to be able to prove that I can actually replace Sonnet with Haiku. It's faster. It's cheaper. [12:41] That's better, right? Well, in general, I'm sure most of you are looking at this, seeing my limited [12:45] sample size. You're like, "Well, Tristan, of course, Haiku is going to run faster because [12:49] it's a cheaper model. That's the intention." That's correct, right? I can track that latency, [12:53] but more so because I'm able to see again if we go back to that input and outputs the agents. [12:57] I can actually compare the answers that it was given. Now, again, this is all we're have to scale [13:02] this out this that the third LM are great for this kind of analysis, but you're actually able to see, [13:07] okay, we were able to improve in our limited test runs latency by 15%. But there was a difference in [13:13] answer. Is that acceptable to us? That's a great question, right? But this allows us to have a more [13:19] nuanced discussion on, hey, this is how we set it up. These were the responses and these are [13:22] the tweaks and tunings we can do and these are the repeatable outputs that we can have. [13:28] So, a little bit more on a more advanced um one and this is actually from a member of my team. [13:33] So, in his workflow, this is all fired based on an alert coming in, right? And this is the [13:38] nice part again since Elastic also operates as a observability security system. We can go ahead [13:43] and actually have these alerts feed into the RCA triage and go through much more advanced [13:48] um kind of workflow here. And you can also see the input and output token tracking as well. [13:54] All right. So what I'm listing here right are examples of what our infosc was able to find [13:59] through the traceability and observe and workflow execution from their own examples of running their [14:04] own test. Right? You know, this was over 32,000 kind of iterations. And for reference, our infosc [14:08] team is not a massive team, right? Elastic is a company of about 3,000 people. Subsequently, [14:12] we do not have a global national team that's going to handle all these incidents. So, we have to be [14:17] smart. And the wonderful part is infosc actually operates as our customer zero for security. [14:21] And as you kind of notice overall, they ran into problems that I'm sure everybody in this room has [14:26] gone for, right? But you can only fix what you can see, right? We can all hypothesize that this is [14:31] the reason, that is the reason, we can argue to it blue in the face. But where this comes in on the [14:35] ability to have all this information indexed, it allows for us to have these conversations [14:40] to in order to leverage these fixes and more importantly you can see in the far right hand side [14:45] we can actually fix this all within elastic itself if we wanted to for on the workflow side and agent [14:49] builder side as well and the numbers kind of prove it. So again this goes back to the concept of one [14:55] giant agent with many skills versus specialized agents. Now again this is not to say that one [15:01] one agent with many skills is the wrong option but we can kind of see in this case [15:04] at least for alert triage at scale where there's a deterministic process there is an inherent cost [15:09] penalty that comes with it right because what ends up happening if you end up giving the LLM [15:13] in this example 14 tools it's going to look through each of these tools to say ah [15:18] yes no maybe so but that's a lot of context and a lot of tokens to load all for it to do [15:24] nothing until it decides what tools to use versus if you have it on a deterministic path, [15:28] it can end up being quite a bit cheaper, especially from a token side. Now, again, this [15:32] doesn't necessarily matter as much when you're in dev mode, you're trying to prove a concept, but [15:35] it does very much matter as we look to scale this out to actually handle production scale workloads. [15:42] And last slide here, um, really quickly, why this works, right? We're able to have [15:46] specialized agents. We do have the open element around the A2A protocol if needed. Happy to talk [15:50] about this later. Um, oh, quick plug. Our booth is 311 if anyone wants to come by. Um, happy to [15:55] talk about this more. I didn't necessarily get to show the A2A side, but you don't have to have [15:58] this all within Elastic. We are not a closed ecosystem by nature. We're open source. So, you [16:02] can naturally call these and have these uh agents and tools imported um to whatever it is you need. [16:08] Um but anyways, on that point, um just want to say thank you and I believe I have three [16:11] minutes for questions if anyone has any. Uh I had a a question about whether you [16:17] created like a suite of independent agents or you're migrating to like a deep agent that kind [16:24] of oversees everything. Yeah. Is that kind of the path you're you're taking? Yeah, I think and it [16:30] goes back to that concept of like are we do are we are we looking to implement an AI workflow or an [16:34] AI agent, right? I think with this deterministic side, it's definitely on the workflow side. So [16:37] having these specialized agents knowing that we can acutely focus them to the task we want them [16:41] to handle, I think serves this point and we kind of see that on the token side of the house. But [16:45] again, not the beall endall, but definitely what we're seeing at least from a volume at least want [16:49] to treat it from an alert triage standpoint and many folks are looking to do that, right? How can [16:52] I eliminate all this noise and just distill down? That's where this kind of specialized agent multi- [16:57] aent workflow kind of works. And mind you, I did everything in this example in one definitive flow. [17:02] We do have the ability to have async workflows kind of parts outcome as well. So there may be [17:06] parts where you don't want to chain it. You want to go off and explore perfectly fine. I just for [17:10] the sake of the demo, for simplicity, I didn't choose to do that, but happy to talk about it. [17:14] Mhm. Oh, sure. Please. these deterministic workflows and you're using tools like MCP to MCP [17:23] tools, how do you deal with the MCP tools changing over time? Yeah, it's a great question. Um, and [17:30] so this kind of all goes back to audibility and changes. And so for us on the tool side, we can [17:34] actually we have our own uh MCB endpoint. So you want to go ahead and version control um import [17:39] them, reimport them, rename them. I'm not saying we have a complete solve for it, [17:42] but we have the ability to import, export, and have version control within the platform. [17:45] So you can kind of keep track of those um as well. So not perfect, but we're getting there. [17:52] All right, got about 50 seconds. Oh, [18:01] fast. Thank you. Hey, when you switched from the different models from like Hyudus on it, Yeah. [18:06] did you see a difference in the output? Yeah. Oh, actually I can hold on. So that was kind of the [18:14] ending part and I apologize if I moved too fast on that. I was just curious if the haiku was able to [18:19] produce the same output as sonnet would. Exactly. Um so I'm happy to show it here but like at the [18:24] end we actually did see a difference in answer but not in the conclusion. Right. So the level of [18:29] evidence that sonnet brought better fine but haiku came to the same answer was able to flag it as a [18:34] high risk not a critical risk. So that's where having this audibility of like hey it's faster [18:38] and it's cheaper. Is that answer good enough? And I think that's a better conversation to have than [18:42] oh Haiku can do this sonic can. It's like let's prove it. But I did I was actually very very [18:47] impressed with Haiku's ability. I'm not going to lie for the RCA synthesizer which one would think [18:51] you need some a more advanced model to synthesize. Haiku does a wonderful job of it. So again lighter [18:55] models can work. So same same answer just less depth. Yeah. It's kind of a different different [19:00] flavor of it than you as the user or you know the auditor if you will can decide is this good [19:04] enough? Do I want a higher quality answer? How does this then chain in lower events? [19:08] Right. Right? Because what I did is I just looked at one uh section. But you could also argue well [19:13] if the input if the output is more detailed that means the input in the next chain could be better [19:18] and then maybe the subsequent agents produce better answers down the line. I just looked purely [19:22] at one step but in theory you could actually look at it holistically across. So my own conclusion [19:26] on my quick demo was hey for this one step we're good. But in terms of the chain I didn't actually [19:31] dig further into that. I could because you see all the input and output for each step but [19:34] that's probably a larger conversation. But all the auditability is there. Thank you. Yeah, of course.