Solve the AI Trust Gap with Agent Observability With Cisco

Solve the AI Trust Gap with Agent Observability With Cisco — Ai4

Ai4

AI-assisted summary of Solve the AI Trust Gap with Agent Observability With Cisco — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by Ai4, this presentation features Paul Lacey, Product Marketing Leader at Cisco, addressing how enterprise autonomous agents stall when guardrails cannot be trusted. According to the description, the session showcases how Splunk Agent Observability allows teams to turn high-accuracy evaluations into live runtime guardrails using purpose-built small language models (SLMs) that evaluate production traffic with sub-200ms latency at up to 98% lower cost.

## TL;DR Paul Lacey breaks down the primary obstacles undermining enterprise AI deployments: unpredictable model behavior, prohibitive latency, and runaway token expenses. He illustrates that standard off-the-shelf LLM judges achieve only ~70% accuracy, leading to frequent guardrail failures and operational blind spots. To resolve this, the presentation details a closed-loop data science workflow that incorporates subject matter expert annotations to raise eval accuracy above 90%, followed by distilling those evals into lightweight SLMs operating under 200 milliseconds. Through an interactive multi-agent support bot demo, Lacey demonstrates live trace inspection, anomaly clustering, automated eval authoring, and active circuit-breaker guardrails that block unauthorized tool calls in real time.

## Atmosphere & style The session has the polished, data-backed cadence of a major enterprise tech conference presentation, grounded by pragmatic engineering realities rather than speculative hype. Lacey speaks with an authoritative, conversational, and self-aware delivery, laughing off minor live-demo hiccups while cleanly navigating complex telemetry dashboards. The atmosphere feels focused and practical, transitioning smoothly from alarming industry failure statistics into technical, step-by-step remediation workflows.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) Enterprise adoption of autonomous AI agents faces a trilemma: unpredictable model outputs, unaffordable token costs, and extreme difficulty in governance at scale. - [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) Market research reveals that 85% of organizations experience production AI issues within six months of deployment, with 62% of incidents caused by missing or defeated guardrails. - [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) Default out-of-the-box guardrails fail under stress, with nearly 100% bypassed in simulated attack scenarios. - [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) A guardrail functions as an evaluation signal driving a circuit breaker; if the underlying evaluation is inaccurate, the guardrail inevitably fails. - [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) First-approximation LLM-as-a-judge implementations top out around a 70% F1 accuracy score compared to human experts, creating a 30% error margin in safety checks. - [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) Frontier LLM evaluators introduce up to 4 seconds of latency and massive context token costs, making them impractical for inline, synchronous guardrail execution. - [07:03](https://www.youtube.com/watch?v=y_FAUWYALd8&t=423s) Integrating Subject Matter Experts (SMEs) directly into trace annotation workflows boosts evaluation accuracy from 70% into the 90%+ range by injecting domain-specific edge cases. - [07:03](https://www.youtube.com/watch?v=y_FAUWYALd8&t=423s) Distilling high-accuracy LLM evaluations into custom fine-tuned Small Language Models (SLMs) achieves sub-200ms latency and reduces runtime evaluation costs by up to 98%. - [11:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=702s) Because model hallucinations and policy violations often occur in less than 1% of transactions, catching needle-in-a-haystack edge cases requires 100% full-stream trace observability rather than sampling. - [13:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=822s) Multi-agent observability instruments granular tool invocations, vector searches, and system prompts, allowing engineers to pinpoint specific failures like missing instruction parameters. - [16:24](https://www.youtube.com/watch?v=y_FAUWYALd8&t=984s) Automated telemetry clustering identifies systemic failure patterns across thousands of traces and automatically generates contextual evaluation prompts using real failure examples. - [16:24](https://www.youtube.com/watch?v=y_FAUWYALd8&t=984s) Active SLM runtime guardrails can intercept invalid tool calls mid-execution, blocking unauthorized transactions and steering agents back into correction loops before users see an error. - [19:05](https://www.youtube.com/watch?v=y_FAUWYALd8&t=1145s) Detailed token and resource tracking across entire agent fleets prevents runaway infrastructure costs from recursive loops and redundant prompt context.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) The AI Trust Gap and the Reality of Missing Guardrails

Paul Lacey opens the session by addressing the central dilemma facing modern AI deployments: controlling autonomous agent behavior reliably at scale. Before diving into the technical realities, he shares the standard SEC safe-harbor reminder that enterprise buying decisions should always be grounded in currently available products rather than future roadmap items. He outlines that enterprise struggle with AI fundamentally clusters around three interconnected vectors: AI is inherently unpredictable, it becomes economically unaffordable without strict constraints, and it is categorically difficult to govern at enterprise scale.

Even state-of-the-art frontier models struggle with basic operational safety. Lacey points out that OpenAI was explicitly transparent during the launch of its newest flagship models regarding their tendency to take unwanted, destructive actions if given autonomy without guardrails. Yet across the industry, engineers routinely hook these models directly into execution environments with zero safeguards in place. The resulting fallout is widespread across early enterprise adopters.

```mermaid flowchart TD accTitle: The AI Reliability Crisis Breakdown accDescription: Breakdown of the causes and statistics behind enterprise agent failures in production.

subgraph Core_Struggles["The AI Trilemma"] A["Unpredictable Outputs"] B["Unaffordable Token Costs"] C["Ungovernable at Scale"] end

subgraph Production_Impact["Production Reality (Survey Data)"] D["85% of Orgs Experience AI Issues<br/>(Within First 6 Months)"] E["62% of Incidents Caused by<br/>Missing or Defeated Guardrails"] F["~100% of Default Guardrails<br/>Defeated in Attack Simulations"] end

Core_Struggles --> Production_Impact ```

- **Market Incident Rate**: A survey conducted across the market revealed that 85% of organizations deploying AI or autonomous agents publicly experience critical production issues within the first six months of launch. - **Guardrail Vulnerability**: 62% of those reported production incidents were traced directly to missing, misconfigured, or defeated guardrails. - **Simulation Failure**: In simulated attack scenarios and stress tests, nearly 100% of out-of-the-box, default guardrails were bypassed or defeated. - **The Core Triad of Roadblocks**: The failure to safely deploy agents stems from three root causes: inaccurate evaluation signals, extreme time-to-market pressure, and punishing cost/latency bottlenecks.

---

[03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) The Limitations of LLM-as-a-Judge and the High Cost of Skipping Evals

To build effective guardrails, engineering teams rely on automated evaluations—primarily the LLM-as-a-judge paradigm, where a secondary language model inspects unstructured interaction feeds and classifies agent behavior. Lacey clarifies the fundamental architecture: "A guardrail is essentially just an eval with a circuit breaker that it drives." Consequently, if the underlying evaluation signal is inaccurate, the guardrail itself is inherently broken.

Getting these evaluators to perform reliably is notoriously difficult due to the subtle, nuanced nature of agent behavior. Referencing foundational research including the Galileo ChainPoll benchmark, Lacey emphasizes that even a reasonably well-tuned, first-approximation LLM judge tops out at around a 70% F1 score when measured against human domain experts. Having an evaluator that is wrong roughly one out of every three times creates a dangerous blind spot in both observability telemetry and safety enforcement. Under aggressive delivery deadlines, teams frequently succumb to the temptation of skipping rigorous evaluation cycles for paths they deem low risk after witnessing a few successful ad-hoc runs.

Furthermore, frontier LLM judges impose untenable performance and cost penalties. A large model acting as a judge can introduce up to 4 seconds of latency—making it completely unusable in the critical synchronous execution path of a user-facing agent, especially when multiple guardrail checks must execute in sequence. From a cost perspective, passing full multi-turn conversational sessions and execution traces with millions of context tokens to detect issues like recursive infinite loops causes API bills to explode across thousands or millions of daily production transactions.

```mermaid flowchart LR accTitle: LLM-as-a-Judge Production Bottleneck accDescription: Flow showing how frontier LLMs in the critical path introduce unacceptable latency and cost.

subgraph Critical_Path["User Critical Path"] User["User Request"] --> Agent["Autonomous Agent"] Agent --> Judge{"Frontier LLM Judge<br/>(4s Latency)"} Judge -->|"Millions of Tokens<br/>($$$ Explosion)"| Circuit{"Circuit Breaker<br/>(Only ~70% Accurate)"} end

Circuit -->|"High False Positive/Negative"| Response["Delayed User Response"] ```

- **Accuracy Ceiling**: Off-the-shelf, first-pass LLM-as-a-judge implementations average an F1 accuracy score of ~70% relative to human evaluators, leaving a 30% error margin. - **Skipped Evals Penalty**: Self-reported industry data shows teams that bypass evaluation on perceived low-risk behaviors suffer 2.3 times more production incidents than teams with thorough eval coverage. - **Latency Drag**: Frontier LLMs introduce response times of up to 4,000 ms, rendering them unsuitable for inline runtime safety checks. - **Context Token Explosion**: Full trace and loop detection requires passing extensive session histories totaling millions of tokens per evaluation, driving massive operational costs at scale.

---

[07:03](https://www.youtube.com/watch?v=y_FAUWYALd8&t=423s) A Closed-Loop Tuning Process: SME Annotations and Custom SLMs

To break through the accuracy and latency barriers, leading engineering organizations employ a disciplined, closed-loop data science workflow for evaluation engineering. The process begins with standard LLM-as-a-judge setups, where candidate evaluation models are A/B tested for cost-versus-accuracy trade-offs against a curated ground-truth dataset and system instruction prompts. However, rather than pushing raw prompts directly into production, the critical breakthrough occurs by integrating Subject Matter Experts (SMEs) directly into the observability workflow.

Domain experts inspect live execution traces and score the evaluator’s decisions in real time. This annotation pipeline generates a high-quality feedback dataset used to automatically optimize judge prompts and inject highly specific, hard-to-find edge cases as few-shot examples into the template. Operating this iterative feedback loop raises evaluator accuracy from the baseline 70% level into the 90% to high-90% range.

```mermaid flowchart TD accTitle: Closed-Loop Agent Eval and SLM Guardrail Architecture accDescription: Workflow showing how SME annotations tune LLM judges, which are distilled into sub-200ms SLMs for runtime agent guardrailing.

subgraph Eval_Tuning_Loop["1. Continuous SME Tuning Loop"] Base["Base LLM Judge & Prompts"] --> Bench["Ground Truth Benchmark"] Bench --> SME["SME Trace Review & Annotations"] SME -->|"Targeted Few-Shot Edge Cases"| TunedJudge["High-Accuracy Judge (>90% F1)"] TunedJudge -.->|"Continuous Feedback"| Base end

subgraph SLM_Distillation["2. Model Distillation & Deployment"] TunedJudge --> Distill["Fine-Tune Custom SLMs"] Distill --> Deploy["Deploy on Hardware / Cloud<br/>(<200ms Latency, Low Cost)"] end

subgraph Runtime_Enforcement["3. Inline Production Guardrail"] Traffic["Production Agent Traffic (100%)"] --> Deploy Deploy -->|"Circuit Breaker"| KillSwitch["Machine-Speed Kill Switch"] Deploy --> Telemetry["Observability & Drift Detection"] Telemetry -.->|"New Injection / Drift Patterns"| Bench end ```

Once high evaluation accuracy is locked in, the next step solves the cost and latency problem: distilling that tuned evaluation capability into custom, fine-tuned Small Language Models (SLMs). These purpose-built SLMs can be hosted flexibly on proprietary enterprise hardware or in the cloud at a fraction of frontier model costs, returning verdicts in under 200 milliseconds. This sub-200ms threshold transforms the evaluator into a true inline circuit breaker capable of terminating rogue agent actions at machine speed. Because user prompts and adversarial injection techniques evolve weekly, the continuous telemetry feed constantly captures new drift patterns and feeds them back through the tuning cycle—a core capability delivered through platforms like Splunk Agent Observability.

- **Human-in-the-Loop Accuracy Gains**: Continuous SME trace review and automated few-shot prompt injection elevate evaluation accuracy from ~70% to well above 90–98%. - **Sub-200ms Execution**: Fine-tuned custom SLMs reduce eval latency down to under 200 milliseconds, unlocking true real-time, inline circuit breaking. - **Economic Scalability**: SLM distillation slashes evaluation inference costs down to "pennies on the dollar" (up to 98% lower cost), allowing 100% coverage across production traces. - **Continuous Drift Defense**: Real-time observability feeds novel adversarial patterns and behavioral drift back into the SME benchmark dataset to perpetually retrain guardrails.

[11:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=702s) Live Demo: The E-Commerce Support Chatbot and the Rare Hallucination Problem

To ground the concepts of continuous evaluation and runtime control, the presentation transitions into a live interactive demonstration centered on a mock online retail storefront. The demo features an e-commerce assistant designed to handle post-purchase customer inquiries. The scenario begins from the customer's point of view: a shopper asks the chatbot for a receipt for a pair of Bluetooth headphones. The agent acknowledges the request, narrates its steps, and produces the receipt. The shopper then tests a return scenario, claiming buyer's remorse after purchasing two pairs and requesting a full refund. The assistant initiates backend processing and issues a $700 refund.

When the initial demo execution surfaces an unexpected outcome, it triggers a knowing laugh from the stage: "This is why you should always set up for your demos, folks, and run through." As it turns out, the system had already engaged the live guardrail at the tail end of the sequence. Running through the interaction again highlights an essential operational reality of modern generative AI: model hallucinations and policy violations are rarely constant. They frequently manifest at an incident rate well under 1% of total enterprise traffic. Because these failures are intermittent needle-in-a-haystack events, catching them cannot rely on intermittent spot-checks or passive logging—it demands 100% visibility over live user sessions.

- The demonstration uses an online retail customer support assistant handling receipt generation and refund requests. - Edge-case hallucinations and unauthorized agent actions often occur in less than 1% of production transactions, requiring full-stream observability rather than sample-based auditing.

[13:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=822s) Multi-Agent Tracing and Trace Annotation in Splunk Observability

Deploying an autonomous agent into production without instrumentation renders the entire architecture a complete black box. Once data streams directly into Splunk Observability, operators gain clear end-to-end visibility into what is actually an intricate, multi-agent system. The front-facing support agent does not operate in isolation; it orchestrates multiple backend worker agents tasked with distinct responsibilities—retrieving product return policies, classifying customer intent, determining refund eligibility, and synthesizing the final response.

```mermaid flowchart TD accTitle: Splunk Multi-Agent Observability and Evaluation Architecture subgraph MultiAgentSystem["Multi-Agent Runtime"] A["User / Front-End Agent"] --> B["Intent Classifier Agent"] B --> C["Vector Search / Policy Retrieval"] C --> D["Decision & Response Agent"] end subgraph ObservabilityLayer["Splunk Agent Observability"] D --> E["Trace Inspection & Tool Calls"] E --> F["LLM-as-a-Judge Evaluations"] end subgraph ContinuousImprovement["Feedback & Auto-Tune"] F --> G{"Instruction Adherence Check"} G -- "Flagged Failure" --> H["Human Trace Annotation"] H --> I["Auto-Tune Queue / SLM Training"] end ```

Drilling into individual execution traces reveals the granular anatomy of each transaction. Engineers can inspect the exact tool invocations, the vector search queries returning store policies, and the raw system prompts and completions of the decision-making model. Alongside raw traces, automated LLM-as-a-judge evaluators assess whether each step complied with enterprise policies. In one trace, an evaluator flags `instruction adherence` as `false`. Hovering over the metric shows the exact failure: the model omitted a mandatory refund request ID stipulated in its prompt. Rather than leaving teams stuck, the platform provides direct remediation paths: an engineer can update the evaluation criteria to create a new revision if requirements have changed, or annotate the trace directly by reinforcing or correcting the score and recording the underlying rationale. This feedback directly populates the auto-tune queue to retrain and refine evaluators at scale.

- Multi-agent architectures route single customer prompts across multiple specialized services, including policy lookup via vector search and intent classification. - Traces capture every intermediate tool call, context injection, and prompt/response payload. - LLM-as-a-judge evaluators check specific criteria such as instruction adherence, pinpointing missing parameters like refund request IDs. - Direct trace annotation allows operators to reinforce or correct judge decisions, piping human feedback straight into the automated tuning pipeline.

[16:24](https://www.youtube.com/watch?v=y_FAUWYALd8&t=984s) Automated Eval Generation and Deploying SLM Runtime Guardrails

Managing massive telemetry streams introduces a common scaling hurdle: how to detect systemic anomalies across thousands of traces without manually inspecting each one. Splunk addresses this by turning analytical AI back onto the telemetry feeds. The system automatically clusters anomalies, flagging patterns where hallucinating agents used external tools to issue unauthorized or mathematically inaccurate refunds. The interface visualizes the exact frequency of these errors and offers a single-click workflow to generate a brand-new evaluator. This generated evaluator comes pre-populated with a contextual prompt containing real trace examples culled directly from production failures, eliminating cold-start prompt authoring.

Once an evaluator is validated and tuned for accuracy, teams can convert it directly into an active runtime guardrail. While teams may prototype their evaluations using general-purpose LLM-as-a-judge setups, deploying high-throughput guardrails cost-effectively requires moving beyond large foundational models. Using Splunk's data science studio, teams train dedicated Small Language Models (SLMs) over their annotated data across training epochs, register them, and execute them inline. The live trace demonstration shows the guardrail firing mid-execution: the instant an agent attempts to emit an invalid refund, the guardrail intercepts the payload, blocks the transaction, and steers the agent back into the calculation loop to correct the error before the end customer ever sees an improper response.

- Automated AI telemetry scanning surfaces recurring failure modes, such as inaccurate refund calculations across tool calls. - One-click evaluator synthesis automatically injects real production failure traces into pre-formed evaluation prompts. - Prototyping begins with LLM-as-a-judge before compiling and training lightweight SLM judges in the data science studio to minimize GPU overhead and latency. - Active guardrails run synchronously within the execution trace, halting bad tool calls and re-steering agents toward valid outcomes.

[19:05](https://www.youtube.com/watch?v=y_FAUWYALd8&t=1145s) Fleet Token Tracking and Cost Observability Preview

The session concludes by addressing the financial dimension of enterprise agent fleets: granular token and resource tracking. Every tool invocation, intermediate agent handoff, and trace records precise token consumption metrics, revealing latent inefficiencies, runaway recursive loops, and redundant prompt context across the architecture.

Announcing an upcoming product launch focused specifically on fleet-wide cost observability, the session frames continuous tracking as the key to scaling agentic systems without runaway infrastructure bills. Attendees are invited to sign up for early preview access and meet offline for technical follow-ups: "One of the products that we'll be talking about that we'll be launching very soon is about helping you achieve cost visibility across your entire fleet. And with that, I'd like to welcome you to sign up for an early preview of this new product, and I'll be taking questions offline."

- Token tracking attached to every trace and sub-action exposes resource waste across complex agent pipelines. - Cisco and Splunk are launching a dedicated fleet-wide cost visibility product aimed at monitoring and optimizing production AI expenditures. - The session wraps with an invitation to join the early preview program and transition to direct Q&A.

## Notable quotes > [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) A guardrail is essentially just an eval with a circuit breaker that it drives. — Paul Lacey

> [11:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=702s) This is why you should always set up for your demos, folks, and run through. — Paul Lacey

> [19:05](https://www.youtube.com/watch?v=y_FAUWYALd8&t=1145s) One of the products that we'll be talking about that we'll be launching very soon is about helping you achieve cost visibility across your entire fleet. — Paul Lacey

> [19:05](https://www.youtube.com/watch?v=y_FAUWYALd8&t=1145s) And with that, I'd like to welcome you to sign up for an early preview of this new product, and I'll be taking questions offline. — Paul Lacey

## Mentioned Paul Lacey, Cisco, Splunk, Splunk Agent Observability, Ai4, OpenAI, Galileo, Galileo ChainPoll, SEC

## Video description Paul Lacey - Product Marketing Leader, Cisco

Enterprise agents stall when their guardrails can't be trusted — 1 in 3 LLM-as-judge eval scores are wrong without proper tuning. Splunk Agent Observability helps teams create and deploy high-accuracy evals as live runtime guardrails, using purpose-built small language models that evaluate 100% of production traffic at up to 98% lower cost. See a live demo across evaluation, observability, and agent control.

[Source](https://www.youtube.com/watch?v=y_FAUWYALd8)

中文简报

借助 Cisco 的智能体可观测性弥合 AI 信任差距 Paul Lacey 剖析了削弱企业级 AI 部署的主要障碍：不可预测的模型行为、令人望而却步的延迟以及失控的 Token 开销。他指出，标准的现成 LLM 裁判准确率仅约 70%，导致护栏频繁失效并形成运营盲区。为解决该问题，演示详细介绍了一种闭环数据科学工作流，该工作流结合领域专家标注将评估准确率提升至 90% 以上，随后将这些评估提炼为运行时间低于 200 毫秒的轻量级 SLM。通过交互式多智能体客服机器人的演示，Lacey 展示了实时链路追踪检查、异常聚类、自动化评估编写，以及能够实时阻止未授权工具调用的主动熔断护栏。 企业采用自主 AI 智能体面临三重困境：不可预测的模型输出、难以承受的 Token 成本以及大规模治理的极高难度。 (0:04) 市场调研显示，85% 的组织在部署后六个月内遇到了生产环境中的 AI 问题，其中 62% 的事件是由护栏缺失或失效造成的。 (0:04) 默认开箱即用的护栏在受压时会失效，在模拟攻击场景中几乎 100% 被绕过。 (0:04) 护栏的作用相当于驱动熔断机制的评估信号；如果底层评估不准确，护栏必然会失效。 (3:28) 与人类专家相比，初步实现的 LLM 裁判的 F1 准确率最高仅达 70% 左右，在安全检查中留下了 30% 的误差幅度。 (3:28)

Reading summary

According to Cisco's Paul Lacey, enterprise deployments of autonomous AI agents face an operational crisis driven by uncalibrated evaluators and vulnerable guardrails. Lacey argues that closing this trust gap requires closed-loop agent observability, domain expert annotations, and distilling large evaluators into specialized Small Language Models for real-time enforcement. The AI Trilemma and Guardrail Vulnerabilities (0:04) Enterprise adoption of autonomous AI struggles across three interconnected vectors: outputs are unpredictable, token consumption becomes economically unsustainable, and systems are difficult to govern at scale. Lacey cites market survey findings showing that 85% of organizations deploying autonomous agents experience critical production issues within six months of launch. Furthermore, 62% of reported production incidents stem directly from missing, misconfigured, or bypassed guardrails, with simulated stress tests defeating default out-of-the-box guardrails nearly 100% of the time. Despite warnings from model providers regarding the destructive potential of unconstrained autonomy, engineering teams frequently connect models directly to operational environments. Driven by time-to-market pressures and inaccurate evaluation signals, organizations often skip rigorous testing on workflows assumed to be low-risk, compounding systemic failure rates across early adopters. Latency and Accuracy Limits of LLM Judges (3:28) Safety guardrails operate fundamentally as automated evaluations tied to circuit breakers, making system safety entirely dependent on evaluation precision. Lacey notes that off-the-shelf LLM-as-a-judge implementations—where a secondary model reviews agent outputs—reach an F1 accuracy ceiling of approximately 70% compared to human experts on benchmarks such as Galileo ChainPoll. This leaves a 30% error margin in safety telemetry. Self-reported industry data shows that teams skipping evaluations on perceived low-risk paths encounter 2.3 times more production incidents. Frontier models also introduce prohibitive operational friction when used inline. Large models can introduce up to 4,000 milliseconds of latency, making them unusable in synchronous user-facing paths requiring multiple sequential checks. Additionally, evaluating long multi-turn sessions and detecting recursive loops requires passing traces spanning millions of tokens, leading to unsustainable API costs at enterprise transaction volumes. SME Tuning and Small Model Distillation (7:03) To overcome the accuracy ceiling, Lacey describes a closed-loop engineering process that integrates Subject Matter Experts (SMEs) directly into trace inspection. When domain experts review live execution telemetry and score evaluator decisions, their feedback generates high-fidelity datasets that inject targeted edge cases into judge prompts. This human-in-the-loop workflow lifts evaluation accuracy from the 70% baseline to between 90% and 98%. Once tuned, the evaluation logic is distilled into custom, fine-tuned Small Language Models (SLMs). These lightweight models can be deployed on proprietary infrastructure or in the cloud, reducing inference costs by up to 98% compared to frontier models. Crucially, distilled SLMs return verdicts in under 200 milliseconds, converting slow offline evaluators into inline circuit breakers capable of halting unauthorized agent actions at machine speed. Full-Stream Tracing and Inline Circuit Breaking (13:42) Because production hallucinations and policy violations often manifest in less than 1% of enterprise traffic, intermittent audits fail to catch intermittent anomalies. In a demonstration of Splunk Agent Observability using a retail refund assistant, Lacey demonstrates how front-end requests route through multiple specialized agents, including intent classifiers and policy retrieval systems. Automated evaluators inspect these execution traces to flag compliance failures, such as missing refund identifiers. The platform clusters anomalies across telemetry streams to detect systemic failures, such as unauthorized financial calculations, and automatically synthesizes new evaluators populated with production failure examples. Trained into inline SLMs, these guardrails synchronously intercept erroneous payloads mid-trace, blocking unauthorized tool executions and steering the agent to recalculate before the end user receives an incorrect output. While multi-agent observability and distilled SLMs resolve latency and accuracy bottlenecks, enterprise fleets still face compounding expenses from recursive loops and context bloat. Lacey previews an upcoming Cisco and Splunk product focused on fleet-wide token and cost observability, though its real-world effectiveness remains unproven ahead of its general commercial release.

精炼阅读摘要

思科（Cisco）与 Splunk 的 Paul Lacey 指出，企业规模化落地自主智能体（AI Agent）普遍面临不可预测性、成本失控与治理困难的三重困境，单纯依赖开箱即用的默认防护栏或基础大模型评估器无法确保生产安全；企业必须通过全链路会话追踪、领域专家标注闭环以及轻量化小语言模型（SLM）构建毫秒级断路器，方能填补智能体信任鸿沟并抵御行为漂移。 智能体部署信任危机与防护栏失效 (0:04) 在企业推动自主智能体落地的进程中，缺乏可靠的防护栏构成了核心运营风险。Lacey 引用的行业调研数据显示，在公开部署人工智能或自主智能体的企业中，多达 85% 在上线前 6 个月内遭遇了关键生产事故。调查进一步表明，62% 的生产事故直接归因于防护栏缺失、配置错误或被突破；在模拟攻击与压力测试中，几乎 100% 开箱即用的默认防护栏均被轻易攻破。这些事故的根本诱因集中于三点：评估信号失准、严苛的交付时间压力，以及安全检查带来的成本与延迟瓶颈。 面对业务交付压力，工程团队常在观察到少量临时测试成功后，主观判定某些调用路径为低风险并跳过严格评估。然而行业统计显示，选择跳过低风险路径评估的团队，其遭遇生产事故的频率是拥有全面评估覆盖团队的 2.3 倍。即便是业界顶尖的前沿旗舰模型，在缺乏严格外部约束时也会展现出执行非预期破坏性动作的倾向，若直接接入企业生产环境，极易造成系统性破坏。 大模型裁判的准确率瓶颈与延迟成本 (3:28) 为约束智能体行为，业界普遍采用“大语言模型作为裁判（LLM-as-a-judge）”的自动化评估范式，由第二模型审查交互日志并判定行为合规性。然而 Lacey 强调，防护栏的本质是驱动断路器的评估机制，若底层评估信号失真，防护机制便形同虚设。参考 Galileo ChainPoll 等基准测试，未经深度调优的首版大模型裁判与人类领域专家相比，其 F1 准确率最高仅约 70%。这意味着评估器每三次判定就会出现一次误判，在遥测可观测性与安全合规上留下了巨大的盲区。 此外，前沿大模型裁判在执行效率与经济成本上带来了难以承受的负担。在面向用户的同步关键路径上，大型前沿模型担任裁判会引入高达 4 秒（4000 毫秒）的延迟，若按顺序执行多项检查，用户响应体验将被彻底破坏。从成本维度来看，为了识别死循环等复杂失控行为，系统必须传递包含多轮会话记录与执行链路的数百万上下文令牌（tokens），导致生产环境下数以万计甚至百万计的日常交互成本迅速膨胀。 专家闭环调优与小语言模型断路机制 (7:03) 为了突破准确率与性能的双重瓶颈，领先的工程实践引入了数据科学闭环调优流程。该流程首先对候选大模型裁判开展 A/B 测试以权衡成本与精度，随后核心步骤在于将领域专家（SME）直接接入可观测性工作流。业务专家实时审查生产执行链路并纠正裁判判定，这些标注数据作为针对性极强的小样本（few-shot）罕见边界案例反哺提示词模板，可将裁判准确率由基线约 70% 提升至 90% 甚至 98% 以上的高位区间。 当高精度评估确立后，架构设计将调优后的评估能力蒸馏至微调的小语言模型（SLM）中。此类专用 SLM 能够灵活部署在企业私有硬件或云端，推断成本大幅降低高达 98%（降至每笔极低开销），并可在 200 毫秒以内返回判定结果。低于 200 毫秒的时延使评估器由事后分析转变为在线机器级断路器，能够实时拦截异常动作，并配合持续遥测捕获每周演进的对抗性注入与行为漂移，维持安全模型的周期性迭代训练。 多智能体全链路追踪与运行时干预 (11:42) 现代生产环境中的前端交互往往涉及多智能体协同，由负责意图分类、知识库检索与业务决策的后端多个工作智能体分工配合。演示表明，智能体幻觉与越权违规具有极高的隐蔽性，发生概率往往低于生产总流量的 1%，这类“大海捞针”式的偶发异常无法依赖抽样审计，必须依托对实时会话的 100% 全量追踪。在 Splunk 可观测性体系中，工程师可下钻查看每个中间工具调用、向量检索返回的退货政策以及提示词输入输出，精确捕获违规细节，例如在演示中模型因遗漏提示词强制要求的“退款申请 ID”而被评估器标记为违背指令。 面对海量遥测日志，系统利用分析型 AI 对异常调用实施自动聚类，展示未授权退款或金额算术错误的发生频次，并提供一键式评估器生成工作流，自动嵌入真实的生产故障样本以消除冷启动提示词编写成本。将调优后的评估器转换为运行期防护栏后，系统能够在智能体通过外部工具尝试发放错误退款的瞬间介入，就地阻断负载并强制智能体返回计算循环进行修正，确保异常结果在触达最终用户之前被纠正完毕。 随着企业智能体向复杂多智能体系统演进，针对每次工具调用与智能体交接的令牌追踪已成为预防基础设施费用失控的关键手段。思科与 Splunk 预告了针对全系统机群级成本可观测性的新产品规划，以实现生产环境支出的精细化归因与治理；然而，在对抗注入手段持续变种与偶发性幻觉难以根除的现实下，企业如何在保障超低延迟的同时平衡全量追踪的算力消耗，仍是规模化落地需要长期攻克的工程难题。

Detailed notes — zh

## Background 该视频由频道 Ai4 于 2026 年 8 月 15 日在 YouTube 平台发布，由思科（Cisco）产品营销负责人 Paul Lacey 主讲。内容聚焦于企业智能体因缺乏可靠护栏而停滞不前的困境，介绍并演示了 Splunk Agent Observability 如何通过专用小语言模型（SLM）部署高精度运行时护栏，以极低成本对 100% 生产流量实施可观测性与实时控制。 ## TL;DR 演讲深入剖析了企业落地自主智能体时面临的失控、高成本与治理三大痛点，揭示了初阶 LLM-as-a-Judge 高达三分之一误判率及延迟高昂的现实缺陷。针对这些瓶颈，思科提出了引入领域专家标注的自动化微调飞轮，将评估准确率从 70% 提升至 90% 以上，并进一步蒸馏至专有小语言模型（SLM），实现延迟低于 200 毫秒、成本降低多达 98% 的内联实时熔断护栏。现场还通过电商多智能体退款系统的实机演示，展示了 Splunk Agent Observability 在全链路追踪、异常自动挖掘、评估器即时生成以及全舰队 Token 成本治理上的完整能力闭环。 ## Atmosphere & style 整场演讲风格专业沉稳且富有浓厚的工程落地色彩，讲者台风亲和幽默，无论是开场轻松化解 SEC 合规声明，还是在演示中护栏意外触发时机智调侃，都显得游刃有余。叙事逻辑从企业级生产事故的扎实数据痛点层层递进，直至深入后端多智能体链路的调用细节与向量检索追踪，节奏张弛有度。在严谨剖析大模型幻觉与安全漏洞的同时，通过行云流水的产品实操展现了破解信任鸿沟的工业级方案，传递出务实而坚定的技术掌控力。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) 企业在规模化落地 AI 智能体时面临不可预测性、高昂成本与规模化治理困难三大核心痛点，大模型在缺少防护机制的情况下裸跑极易执行非预期破坏性操作。 - [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) 调研显示 85% 发布 AI 应用的企业在上线 6 个月内遭遇生产故障，且 62% 归因于护栏缺失或被击穿；缺乏准确评估信号、上线时间压力以及评估的高成本与延迟是护栏部署的核心阻碍。 - [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) 未经精细调优的 LLM-as-a-Judge 评估器 F1 准确率往往卡在 70% 左右（即三分之一的评估存在误判），无法为实时熔断护栏提供高置信度的判定信号。 - [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) 因交付压力跳过 Eval 行为测试的团队在生产中遭遇事故的概率高出 2.3 倍，而传统前沿大模型单次判定高达 4 秒的延迟与数百万 Token 的 Trace 审查使生产级实时评估难以承受。 - [07:03](https://www.youtube.com/watch?v=y_FAUWYALd8&t=423s) 引入领域专家（SME）对生产 Trace 进行实时标注与校验，沉淀长尾边缘案例并注入 Few-shot 自动化微调，能将评估器准确率从 70% 显著提升至 90% 以上。 - [09:51](https://www.youtube.com/watch?v=y_FAUWYALd8&t=591s) 将黄金基准知识蒸馏微调至专有小语言模型（SLM）作为运行时护栏，可将单次评估延迟压缩至 200 毫秒以内，并将运行成本降低最高达 98%。 - [09:51](https://www.youtube.com/watch?v=y_FAUWYALd8&t=591s) 面对模型漂移与新型 Prompt 注入攻击，企业需要将可观测性、数据标注与 SLM 内联护栏整合为动态演进的数据飞轮，以实现毫秒级紧急击杀与防护。 - [13:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=822s) Splunk Agent Observability 支持对多智能体协作网络进行全链路追踪（Trace），透明化呈现向量检索、工具调用细节，并支持对指令遵循度（Instruction Adherence）等指标进行在线标注与自动微调入队。 - [16:24](https://www.youtube.com/watch?v=y_FAUWYALd8&t=984s) 系统具备 AI 自动扫描海量链路并挖掘异常流量（如错误退款与幻觉工具滥用）的能力，可一键注入样本生成专用评估器，并在离线验证达标后部署为拦截并纠偏重算的生产级 SLM 护栏。 - [19:05](https://www.youtube.com/watch?v=y_FAUWYALd8&t=1145s) 平台提供细粒度的单操作与全链路 Token 消耗度量，精准消除系统中的冗余调用，并将治理能力拓展至企业级全智能体机队（Agent Fleet）的统一成本监控与管控。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) 弥合企业 AI 信任鸿沟与规模化控制挑战 在演讲开场，演讲者首先依照惯例提到了美国证券交易委员会（SEC）的安全港合规声明，幽默地提醒观众做出采购决策时应基于当前市场上已有的产品，尽管接下来的分享既会涵盖当下可用的功能，也会展望未来的产品路线图。进入正题后，核心议题聚焦于企业在推进智能体（Agent）规模化落地时普遍面临的“失控”现象——当 AI 智能体发生偏离预期的恶性行为时究竟意味着什么、缺少护栏（Guardrails）会带来多大的破坏性后果，以及为什么为自主智能体构建可靠的护栏在技术和工程上异常艰难。 企业在采用 AI 时往往在三个核心痛点中反复挣扎：首先是不可预测性（Unpredictable），几乎所有落地团队都亲身经历过模型产生出乎意料的幻觉或偏离；其次是不可承受的高昂成本（Unaffordable），特别是在没有防护机制约束 token 消耗的情况下；最后则是极其难以在规模化场景下进行有效治理（Govern at scale）。即便业界最顶尖的前沿大模型也无法幸免——OpenAI 在发布其最新旗舰模型时就曾非常坦诚地披露，该模型在特定场景下存在执行破坏性、非预期操作的倾向。然而令人担忧的是，大量企业用户依然直接在生产机器上裸跑这些模型，完全没有任何护栏拦截机制。 根据思科针对市场进行的深入调研数据显示，这种防护缺失正在引发大规模的生产事故： - 85% 公开发布 AI 或 Agent 应用的企业在产品上线后的 6 个月内遭遇过生产级故障。 - 62% 的故障直接归因于护栏缺失或护栏被绕过/击穿（Defeated guardrails）。 - 在多项针对现有默认护栏的对抗模拟测试中，几乎 100% 的现成防护措施均被成功击穿。 演讲者指出，阻碍企业成功部署护栏的根本原因可以拆解为必须按顺序攻克的三个递进瓶颈：第一是缺乏能够精准识别智能体行为的“准确信号”（Inaccurate signals）；第二是业务端急于上线的“时间压力”；第三则是大模型评估带来的“高延迟与高昂成本挑战”。 --- ### [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) LLM-as-a-Judge 的三大困境：准确率瓶颈、发布压力与性能开销 为了看清智能体的行为轨迹，行业目前最普遍采用的方法是“大模型作为裁判”（LLM-as-a-Judge），即通过提示词让大模型对非结构化的交互数据流进行分类和评估，进而将其作为护栏触发的基础信号。从本质上讲，护栏就是一个连接了熔断机制（Circuit breaker）的评估器（Eval）。然而，如果评估信号本身不够准确，护栏就形同虚设。学术界的多篇论文（包括 Galileo 著名的 ChainPoll 研究）表明，未经精细调优的初阶 LLM 裁判对标人类专业评估者时，其 F1 准确率得分往往卡在 70% 左右的上限。这意味着每三次评估就有一次是误判（"1 in 3 eval scores are wrong"），在可观测性和实时防护中留下了巨大的盲区。 ```mermaid flowchart TD accTitle: LLM-as-a-Judge 落地生产的三大核心瓶颈 subgraph Bottlenecks["LLM-as-a-Judge 核心困境"] A["准确率瓶颈<br/>(F1 约 70% 上限 / 1/3 误判)"] B["上线时间压力<br/>(跳过评估测试 -> 事故率激增 2.3 倍)"] C["成本与延迟爆炸<br/>(响应超 4 秒 / 每次 Trace 消耗数百万 Token)"] end Bottlenecks --> D["生产盲区与护栏失效"] ``` 除了信号不准，快速交付的市场压力促使许多工程团队在低风险或受控环境下看到几次成功运行后，就抱有侥幸心理跳过严谨的 Eval 环节直接上线。调研显示，因赶进度而跳过未测试行为评估的团队，在生产环境中遭遇的 AI 事故数量是严格测试团队的 2.3 倍之多。 第三大现实阻碍则是 LLM 裁判极其昂贵且响应缓慢。在生产环境中，大型前沿模型（如 GPT 级别）进行单次判定可能需要等待长达 4 秒。如果在用户请求的关键返回路径上串联多个需要规模化运行的护栏，这种延迟是完全不可接受的。与此同时，如果要对完整的调用追踪（Trace）或多轮会话进行审查以防止死循环，单次评估就可能向大模型灌入数百万 token 的上下文；在每天成千上万乃至数百万次交互的企业级规模下，推理成本将直接失控爆炸。 --- ### [07:03](https://www.youtube.com/watch?v=y_FAUWYALd8&t=423s) 高精度评估微调闭环与领域专家标注 为了破解上述困局，演讲者分享了一套源自成熟客户实践的高精度评估微调流程。该体系从传统的 LLM-as-a-Judge 架构出发，由三个核心支柱构成：一是用于评估的模型底座（可针对不同候选模型进行 A/B 测试以平衡性价比与精度）；二是系统提示词（指引裁判在数据流中寻找何种具体行为）；三是最为关键的真实基准数据集（Ground Truth dataset）。这个基准数据集是贯穿智能体整个生命周期、最具持久价值的数字资产。 ```mermaid flowchart LR accTitle: 结合专家标注的高精度 Eval 自动化微调闭环 subgraph Loop["高精度微调飞轮"] direction TB A["生产/测试 Trace 数据"] --> B["LLM 裁判初审"] B --> C["领域专家 (SME) 实时校验标注"] C --> D["捕获长尾边缘用例 & 生成 Few-shot"] D --> E["自动迭代微调 Prompt 与裁判模型"] E --> B end ``` 针对评估准确率会随时间严重漂移的痛点，最关键的工程举措是引入领域专家（Subject Matter Expert, SME）的主动标注。让精通业务上下文的业务人员直接审阅调用追踪与裁判给出的评分，实时校验判定是否正确。这一闭环能够沉淀出极高价值的长尾边缘案例（Edge cases），并将其作为靶向 Few-shot 示例注入提示词模板中，从而实现裁判系统的自动化微调。实践表明，通过构建这种持续捕获数据与重训的闭环飞轮，团队能够将评估器的准确率从 70% 显著提升至 90% 以上，部分垂直数据集甚至可以达到 90% 后段的极高水准。 --- ### [09:51](https://www.youtube.com/watch?v=y_FAUWYALd8&t=591s) 定制 SLM 运行时护栏与 Splunk Agent Observability 在建立高精度的黄金基准之后，解决线上高延迟与高成本的终极方案是将知识蒸馏并微调到专用的小语言模型（Small Language Models, SLMs）中。企业可以根据自身合规要求将这些轻量级 SLM 部署在私有硬件或任意云端环境，运行成本仅为大模型的几十分之一（"pennies on the dollar"），成本降幅最高可达 98%。更重要的是，微调后的 SLM 能够将判断延迟大幅压缩至 200 毫秒以内。 这一性能突破使得将评估逻辑作为内联（In-line）熔断器部署进生产关键路径成为可能。当后台 Agent 试图执行破坏性指令、偏离合规轨道或陷入恶性死循环时，系统能够在毫秒级的机器速度下触发“紧急击杀开关”（Kill switch），实现真正值得信赖的自主智能体防护。 考虑到模型性能会漂移，且新型 Prompt 注入攻击手段层出不穷（“今天存在的提示词注入技术在几周前甚至还未出现”），这套体系必须演进为类似数据科学训练的动态飞轮——持续捕获线上异常流量并回流至调优流水线中。演讲者指出，能够将全链路可观测性、数据专家标注闭环与轻量化 SLM 运行时护栏完美整合的代表性工业平台正是 Splunk Agent Observability。 ### [11:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=702s) 演示开场：电商 AI 智能体的退款场景与护栏机制 为了让观众更直观地理解在大规模数据下如何管理智能体，演讲者将演示切换至一个销售前沿产品的在线电商平台。平台内置了一个 AI 助手（聊天机器人），演示者模拟顾客提出退款请求。首先，顾客询问能否获取蓝牙耳机的收据，AI 助手在后台处理后迅速提供了收据。随后，演示者模拟“购买了两件商品后产生悔意”的情境，要求智能体退款。智能体在后端执行了一系列操作，并成功发放了一笔 700 美元的退款。 在演示过程中，系统一度直接触发了实时护栏，演讲者幽默地打趣道：“朋友们，这就是为什么你必须提前做好演示准备并完整彩排一遍。”但这也恰好展示了实时护栏生效时的状态。演讲者强调，虽然大模型幻觉的发生率通常远低于 1%，属于罕见故障，但正是因为发生率极低且破坏力大，实时拦截才显得至关重要，这也是对生产流量保持 100% 可观测性的根本原因。 ```mermaid flowchart TD accTitle: 电商多智能体退款处理流程图 subgraph Frontend["前端交互"] U["用户请求退款"] --> A["前端客服 Agent"] end subgraph MultiAgent["多智能体后端协作与追踪"] A --> B["意图分类 Agent"] B --> C["向量检索退款政策"] C --> D["决策与工具调用 Agent"] end subgraph Observability["Splunk 可观测性与治理"] D --> E["LLM as a Judge 评估指标"] E --> F{"是否符合规范?"} F -- 否 --> G["人工标注 / 加入自动调优队列"] F -- 是 --> H["放行并输出结果"] end ``` --- ### [13:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=822s) 深入后端全链路追踪与 LLM-as-a-Judge 评估体系 如果不引入可观测性体系，直接将智能体推向生产环境就如同面对一个完全不透明的“黑盒”。通过将所有数据接入 Splunk Agent Observability，团队可以洞察整个系统的架构——这实际上是一个复杂的后端多智能体系统。前台负责与用户交互的 Agent 会调度多个后端 Agent 进行分类、调取企业政策、识别操作意图，并最终决定是否批准退款。 在 Trace（链路追踪）视图中，工程师可以单步回溯单次交互的完整执行路径，包括内部系统调用、向量搜索获取特定产品退款政策的 Tool Call、以及大语言模型最终决策的完整输入与输出。此外，系统还会并行运行“LLM as a Judge”评估指标，实时判断 Agent 是否合规。例如，在当前演示的一条链路中，系统显示“指令遵循（instruction adherence）”评分为 false，悬停查看后发现是因为模型未按照 Prompt 要求生成退款请求 ID（Refund Request ID）。针对这类问题，开发者可以直接修改评估器生成新版本，或者直接在链路中添加反馈进行标注，修改或强化评估判定并记录原因。这些标注数据会自动进入“Auto Tune Queue”（自动微调队列），用于后续在大规模场景下持续训练评估模型。 - **多智能体协作**：前台接待、意图分类、策略检索与决策执行分工明确。 - **全链路追踪能力**：完整呈现 Vector Search、Tool Calls 以及 LLM 的原始输入与输出。 - **即时评估与标注**：支持针对 Instruction Adherence 等指标进行在线审核、微调打分和提交复核理由。 --- ### [16:24](https://www.youtube.com/watch?v=y_FAUWYALd8&t=984s) 智能异常挖掘、评估器生成与 SLM 护栏闭环 针对前序环节中“如何在海量数据中高效发现异常”的行业痛点，演讲者介绍了利用 AI 反哺分析数据流的方法。系统自动扫描海量链路后，主动标记出多起退款金额不准确、出现幻觉并滥用退款工具的异常实例，并清晰展示其发生频率。运维人员只需点击按钮，即可一键生成专用评估器。该功能会在 Prompt 中自动注入被标记的真实异常样本，让评估器从一开始就具备丰富的上下文背景。 评估器经过离线测试和精准度迭代达到预期置信度后，便可无缝转化为实时生产护栏（Guardrail）。在原型验证阶段，团队可以采用成本较低的“LLM-as-a-Judge”模式；一旦准备上线，便可在 Data Science Studio 中训练专有的小语言模型（SLM），并将其注册至生产环境以极低成本对 100% 流量进行毫秒级把关。当护栏拦截到错误退款时，会打断异常执行链路并重新引导（re-steer）智能体重新计算，从而在不影响用户体验的前提下交付正确的结果。 ```mermaid flowchart LR accTitle: 评估器构建与 SLM 护栏落地生命周期 subgraph Discovery["1. 异常挖掘"] D1["AI 自动扫描链路"] --> D2["标记退款金额异常"] end subgraph BuildEval["2. 评估器构建"] D2 --> E1["一键生成 Prompt (注入样本)"] E1 --> E2["LLM-as-a-Judge 离线测试"] end subgraph Production["3. 护栏部署与执行"] E2 --> P1["Data Science Studio 训练 SLM"] P1 --> P2["注册为运行时实时 Guardrail"] P2 --> P3["拦截错误并纠偏重算"] end ``` --- ### [19:05](https://www.youtube.com/watch?v=y_FAUWYALd8&t=1145s) Token 成本洞察与全舰队治理展望 在演示尾声，演讲者补充了 Splunk Agent Observability 在成本控制方面的能力。系统能够细粒度统计每次操作和单条 Trace 所消耗的 Token 数量，帮助团队快速定位多智能体协同网络中的资源浪费与冗余调用。这项能力同样适用于企业内部部署的全部智能体机队（Agent Fleet）。演讲者透露，思科/Splunk 即将正式推出专注于“全舰队成本可观测性”（Fleet-wide Cost Visibility）的全新产品，并邀请在场观众申请早期预览版体验，随后宣布进入会后面对面答疑环节。 - **Token 消耗透明化**：精准度量每个 Tool Call 与推理步骤的 Token 开销，消灭系统浪费。 - **全机队治理扩展**：支持跨团队、跨 Agent 架构的统一成本监控与可观测性覆盖。 - **新产品预览**：即将上线专项成本管控工具，支持企业级规模的降本增效。 ## Notable quotes > 1 in 3 LLM-as-judge eval scores are wrong without proper tuning. — Paul Lacey > Splunk Agent Observability helps teams create and deploy high-accuracy evals as live runtime guardrails, using purpose-built small language models that evaluate 100% of production traffic at up to 98% lower cost. — Paul Lacey > [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) 85% 公开发布 AI 或 Agent 应用的企业在产品上线后的 6 个月内遭遇过生产级故障。 — Paul Lacey > [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) 62% 的故障直接归因于护栏缺失或护栏被绕过/击穿（Defeated guardrails）。 — Paul Lacey > [00:04](https://www.youtube.com/watch?v=y_FAUWYALd8&t=4s) 在多项针对现有默认护栏的对抗模拟测试中，几乎 100% 的现成防护措施均被成功击穿。 — Paul Lacey > [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) 从本质上讲，护栏就是一个连接了熔断机制（Circuit breaker）的评估器（Eval）。 — Paul Lacey > [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) 未经精细调优的初阶 LLM 裁判对标人类专业评估者时，其 F1 准确率得分往往卡在 70% 左右的上限。这意味着每三次评估就有一次是误判（"1 in 3 eval scores are wrong"）。 — Paul Lacey > [03:28](https://www.youtube.com/watch?v=y_FAUWYALd8&t=208s) 因赶进度而跳过未测试行为评估的团队，在生产环境中遭遇的 AI 事故数量是严格测试团队的 2.3 倍之多。 — Paul Lacey > [09:51](https://www.youtube.com/watch?v=y_FAUWYALd8&t=591s) 企业可以根据自身合规要求将这些轻量级 SLM 部署在私有硬件或任意云端环境，运行成本仅为大模型的几十分之一（"pennies on the dollar"），成本降幅最高可达 98%。 — Paul Lacey > [09:51](https://www.youtube.com/watch?v=y_FAUWYALd8&t=591s) 今天存在的提示词注入技术在几周前甚至还未出现。 — Paul Lacey > [11:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=702s) 朋友们，这就是为什么你必须提前做好演示准备并完整彩排一遍。 — Paul Lacey > [11:42](https://www.youtube.com/watch?v=y_FAUWYALd8&t=702s) 虽然大模型幻觉的发生率通常远低于 1%，属于罕见故障，但正是因为发生率极低且破坏力大，实时拦截才显得至关重要，这也是对生产流量保持 100% 可观测性的根本原因。 — Paul Lacey ## Mentioned Paul Lacey, Cisco, Splunk, Splunk Agent Observability, Ai4, OpenAI, Galileo, ChainPoll, SEC (美国证券交易委员会), Data Science Studio ## Video description Paul Lacey - Product Marketing Leader, Cisco Enterprise agents stall when their guardrails can't be trusted — 1 in 3 LLM-as-judge eval scores are wrong without proper tuning. Splunk Agent Observability helps teams create and deploy high-accuracy evals as live runtime guardrails, using purpose-built small language models that evaluate 100% of production traffic at up to 98% lower cost. See a live demo across evaluation, observability, and agent control. [Source](https://www.youtube.com/watch?v=y_FAUWYALd8)

Source quotes

Paul Lacey: 1 in 3 LLM-as-judge eval scores are wrong without proper tuning. Paul Lacey: Splunk Agent Observability helps teams create and deploy high-accuracy evals as live runtime guardrails, using purpose-built small language models that evaluate 100% of production traffic at up to 98% lower cost. [0:04] Paul Lacey: 85% 公开发布 AI 或 Agent 应用的企业在产品上线后的 6 个月内遭遇过生产级故障。 [0:04] Paul Lacey: 62% 的故障直接归因于护栏缺失或护栏被绕过/击穿（Defeated guardrails）。 [0:04] Paul Lacey: 在多项针对现有默认护栏的对抗模拟测试中，几乎 100% 的现成防护措施均被成功击穿。 [3:28] Paul Lacey: 从本质上讲，护栏就是一个连接了熔断机制（Circuit breaker）的评估器（Eval）。 [3:28] Paul Lacey: 未经精细调优的初阶 LLM 裁判对标人类专业评估者时，其 F1 准确率得分往往卡在 70% 左右的上限。这意味着每三次评估就有一次是误判（"1 in 3 eval scores are wrong"）。 [3:28] Paul Lacey: 因赶进度而跳过未测试行为评估的团队，在生产环境中遭遇的 AI 事故数量是严格测试团队的 2.3 倍之多。 [9:51] Paul Lacey: 企业可以根据自身合规要求将这些轻量级 SLM 部署在私有硬件或任意云端环境，运行成本仅为大模型的几十分之一（"pennies on the dollar"），成本降幅最高可达 98%。 [9:51] Paul Lacey: 今天存在的提示词注入技术在几周前甚至还未出现。 [11:42] Paul Lacey: 朋友们，这就是为什么你必须提前做好演示准备并完整彩排一遍。 [11:42] Paul Lacey: 虽然大模型幻觉的发生率通常远低于 1%，属于罕见故障，但正是因为发生率极低且破坏力大，实时拦截才显得至关重要，这也是对生产流量保持 100% 可观测性的根本原因。

Source quotes

[3:28] Paul Lacey: A guardrail is essentially just an eval with a circuit breaker that it drives. [11:42] Paul Lacey: This is why you should always set up for your demos, folks, and run through. [19:05] Paul Lacey: One of the products that we'll be talking about that we'll be launching very soon is about helping you achieve cost visibility across your entire fleet. [19:05] Paul Lacey: And with that, I'd like to welcome you to sign up for an early preview of this new product, and I'll be taking questions offline.

Original transcript

[0:04] Today, we're going to talk about solving  the AI trust gap, and more specifically, [0:08] we're going to talk about controlling agent  behavior at scale. So, let's dive into this. [0:15] This First of all, we would like to the SEC  would like to remind you to always make your [0:19] purchase decisions based on products that  are in the market, and we will talk about [0:22] products that are available for purchase  today, as well as future road map items. [0:28] But, we're going to talk more specifically  about what it looks like when agents go awry. [0:34] We're going to look at why that happens, and  what the impact is of missing guardrails. [0:38] We're going to talk about why it's actually really  challenging to get guardrails around our agents, [0:44] and then we're going to talk about some of the  the techniques and the process that we've learned [0:49] from a lot of our customers about how we can  get this accurate and get this right at scale. [0:55] And then finally, we're going to jump into a  little bit of a demo. So, let's get get going. [0:59] We got a some ground to cover here. First, we're  going to talk about AI is really um that we we [1:06] struggle with AI in three three areas. We struggle  with the fact that AI is unpredictable, and [1:11] everybody in this room, I'm sure, has experienced  that. We struggle with the fact that AI can be [1:15] unaffordable, especially without guardrails around  it. And it and almost categorically, uh this makes [1:23] AI extremely hard to govern at scale. However,  there's a lot of benefits to AI, as well. And so, [1:30] we need to figure out a way to get our arms around  these three challenges if we want to launch these [1:35] products. The the challenge is, though, that that  a lot of things conspire against that. And even [1:41] some of the most sophisticated, state-of-the-art  models are still having challenges. People are [1:46] still experiencing issues. Um OpenAI was very  transparent when they launched their newest [1:53] flagship model that this model does have a  tendency to take unwanted destructive actions. [1:59] People are still using it on their machines  with no guardrails. Why is that? Well, there [2:05] it really comes down to the fact that there there  are three core issues that that um that that that [2:15] drive this. And it comes down to the fact that  AI across the board is driving lots and lots of [2:22] problems at scale. We we ran a survey in the  market earlier this year. We We found out that 85% [2:29] of organizations that launch AI or agents publicly  experience issues within 6 months of that launch. [2:36] 62% of the those issues were due to missing or  defeated guardrails. And over a hundred almost [2:43] 100% of provided guardrails have been defeated in  a number of simulations across the board. I think [2:51] it's cutting off some of our of the the links  there, but I can send you this research if you [2:56] want to hit me up after. So, this comes down  to three core areas that we we want to we need [3:04] to tackle in sequence. The first is that we have  inaccurate signals to be able to understand what [3:11] our agents are doing and create these guardrails.  The second are the time pressures involved with [3:17] launching AI and I know everybody in this room is  trying to get AI out into the market as quickly [3:21] as possible. And finally, we have a cost and  latency challenge that we'll dive into as well. [3:28] But starting with inaccurate signals, one of the  best ways to understand what's happening with your [3:34] agent is using a technique known as LLM as judge.  I'm sure lots and lots of people in this room have [3:40] developed an LLM as judge. They and we've deployed  it. We're looking at these unstructured data feeds [3:46] and we're trying to classify them and we're trying  to classify them in specific ways so that we can [3:51] understand what the agent is doing at scale and  ultimately so we can turn this into a guardrail. [3:58] And a guardrail is essentially just an eval  with a circuit breaker that it drives. So, [4:04] if we if we don't have an accurate eval, we don't  have an accurate guardrail. Now, the challenge is [4:10] that lots of these evals are extremely difficult  to tune. The behavior that we're trying to monitor [4:16] is extremely nuanced. And there's some research  here, a couple of papers that that that were [4:22] launched including the original Galileo chain  pull paper that demonstrated that even the the [4:29] even a reasonably well-tuned first approximation  eval taps out at around 70% accuracy F1 score to [4:37] a human evaluator. So, if we have an evaluator  that's only right two out of three times, [4:43] that means that we've got a significant blind spot  in our observability and we've got a blind spot in [4:49] our guardrails as well. And this is not something  that we can feel confident launching at scale. [4:56] The second challenge is that teams oftentimes  are moving quickly and it is tempting to skip the [5:03] eval step. It's tempting to just launch something  especially if you've seen it working a few times [5:09] in a controlled environment. Another another  thing that we learned from talking to the market [5:15] was that those that people that launch AI that  with untested behaviors have experienced two and [5:23] a half or 2.3 times more incidents in production  with that AI product than folks that that did not [5:31] report skipping those low-risk behaviors. And this  is this is self-reported data on a study that was [5:37] run earlier this year. The last challenge is that  LLM judge is both expensive and it is time it [5:45] it takes a lot of time for the model response to  come back. So, these are the this is looking at a [5:53] can't see it there at the bottom, but this is  looking at a GPT model and comparing it against [5:59] a small language model, which we're going  to talk about a little bit in a second. But, [6:02] as you can see if we're waiting up to 4 seconds  at at production speed, this is not something [6:09] that we're going to feel confident putting in the  critical path of a response coming back to a user, [6:14] especially if we've got to run several guardrails  at scale. The second challenge is simply the cost. [6:21] These things can be extremely expensive. And  if you're evaluating it a full trace, if you're [6:26] evaluating a full session to understand whether  or not the the the agent should stop looping, [6:31] that can get extremely expensive. You can be  passing millions of tokens of context into [6:37] the evaluator to be judged, and that the cost can  explode at when you're when you're thinking about [6:44] thousands to to millions of interactions a day,  depending on the scale of your application. So, [6:49] all three of these things are are pushing us away  from deploying these guardrails and these these [6:56] evals at scale. This is the reason why so many  of us are still experiencing these challenges. [7:03] Now, how do we get around this? Well, when  talking to a lot of the folks that we work with, [7:08] we've learned that there's actually  a really simple process to go through [7:13] to tune the performance of an eval and then make  it performant in production such that you can [7:19] you can deploy it and trust it as as your  observability signal and your guardrail. [7:26] And it starts with a traditional LMS judge  eval. So, we're going to start with the LLM [7:31] model that we want to use, and we can AB test  this across a number of different candidates. So, [7:36] we figure out which one hits the right price  point and the right accuracy. We have a prompt, [7:43] which is essentially the system prompt or the  instruction set to the LLM to say, "I'm about [7:47] to send you some data. Here's what I want you to  look for in this data feed." And then we have the [7:52] data set itself, and this is so important.  This data set should be our ground truth. [7:58] And this is what we test against, this is what we  benchmark against. This is the durable asset that [8:03] we build throughout all of our observability  throughout the life cycle of this agent. Now, [8:09] a lot of people do that and they launch it into  production, and that's fine. That's a great [8:13] first step. But they find over time that the the  accuracy of these things can drift significantly. [8:20] The one of the most essential additions to this  process is the subject matter expert annotations [8:27] over these evals. So, we find that the teams  that have the most success actually sit down [8:33] with the people who know the domain of the agent,  and they allow them to read the trace and look at [8:39] the evaluator score, and in real time score  whether or not that evaluator was accurate. [8:44] That generates a very rich data set, which we can  use to auto tune the evaluator. We can use that [8:51] to correct the LLM's judge prompt. We can use that  to provide very targeted few-shot examples at the [8:58] bottom of that prompt because we can we can search  for the most um the rarest edge cases to put [9:05] into that prompt template. And so, all of these  things help us make this more accurate at scale. [9:12] And if you have the ability to build a loop around  this such that you're constantly capturing this [9:18] information, constantly retuning the LLM's judge  evaluator, this is how we get to an extremely [9:24] high-accuracy judge. And we've seen teams achieve  accuracy levels above 90% and and in some cases [9:32] into the high 90% of accuracy on the data sets  that they are that they tune around using this [9:38] method. So, this method is going to give us our  first step in the journey, which is we have true [9:44] accurate visibility into what these agents are  doing at scale. Now, from there, what we found [9:51] is that the only way to solve the latency and  cost challenge is to create our own custom SLMs, [9:59] fine-tune our own small language models that we  can deploy anywhere we want on our own hardware, [10:05] we can deploy it in the cloud, we can run them  at pennies on the dollar, and we can we can then [10:11] achieve both high coverage observability across  all the agent all the behaviors our agents take, [10:18] as well as we can think about putting these things  in line with our systems because we can see things [10:24] like less than 200 milliseconds response time,  so we can actually have this as a circuit breaker [10:29] while the agent is going about its its regular  tasks. And finally, this enables us to guardrail [10:35] the agent, and and that that's where we all want  to be, right? We all want agents in production [10:41] that act we actually can trust because we know  that when they try to do something they shouldn't, [10:46] we're going to be watching, and we can  actually hit the kill switch at machine speed. [10:51] Um but it doesn't stop there because what we're  seeing is that models drift and user behavior [10:57] also drifts. Prompt injection techniques that  exist today didn't exist even a few weeks ago, [11:04] and so we're constantly having to keep up with  what's going on and what's coming into our agents. [11:09] This is where having again full observability  gives us a lot of power to be able to take all [11:14] this and and put this back into the process, rerun  the tuning cycle again. And if this looks like a [11:20] data science process, that's exactly what it is.  We've learned this from the the same way that we [11:26] we train models, we're just doing it with  a different target. The target is how do we [11:31] evaluate our agent? So, now let's take a look at a  Oh, and I would be remiss without mentioning that [11:37] one of the platforms that does this extremely well  is Splunk Agent Observability, and it's been built [11:42] around the needs of collecting this data at scale  and enabling you to run this process across your [11:51] agents both from development all the way through  to production. All right, now let's go ahead and [11:57] switch over if you don't mind. I I'd like to  show you what this looks like in action so we [12:01] can make it a little bit more real. Here we have  a online store where we sell lots of cutting-edge [12:08] projects products, and I'm going to go ahead and  open up the AI assistant that comes with this [12:14] store. It's a a little chatbot here, and  I'm going to ask it to as as a customer, [12:20] can I get a receipt for my Bluetooth headphones?  It's going to go through, it's going to [12:25] it's going to narrate a little bit, and then  it's going to give me the the receipt that I [12:29] was looking for. Okay, this is great now, but I I  bought two of them, and so maybe I have a little [12:34] buyer's remorse. I'd like to send them back.  So, I'm going to say, can you issue me a refund? [12:39] Now, the agent's going to go back, it's going  to do a bunch of things on the back end, [12:43] and then it issued it issues a refund. And  in this case, the refund is is it's $700, [12:51] so it's very similar to what we were looking at  before, but let's let's take a look over here. [12:59] Yep. This is why you should always set up  for your demos, folks, and run through. [13:04] Let's go ahead This was actually a great We jumped  to the end, but this is a great illustration of [13:09] what when the guardrail is actually live. Let's  go ahead and run through that again and and see [13:16] Often times, these things will happen on a  incident rate of of well under 1% of all of [13:22] your traffic, right? These these hallucinations  are are are rare, which is why catching them in [13:27] the moment is so important. And this is  also again why getting 100% visibility. [13:33] All right. In this case, our agent  seems to be extremely performant. So, [13:37] it's it's giving me the correct refund every  time. But let's go ahead and take a look. Let's [13:42] Let's look on the back end and see. Now, if I was  just at launching this into production without [13:46] observability, of course, this would be a complete  back black box. But now that I have all of this [13:52] data flowing into Splunk Observability, I can  come and I can take a look at this entire system, [14:00] and I can see I can go in and see that this is  actually a multi-agent system on the back end. [14:05] This is an agent that interacts with the user, and  then it calls out to other agents on the back end [14:11] that are doing things like classifying um they're  pulling the policy, they're classifying the action [14:17] intent, and uh ultimately generating a response  and a decision of whether or not that eval or [14:24] whether or not that that refund is um is honored.  And if I go into um some traces here, for example, [14:32] I can see that um I can see the entire trace, the  entire interaction. I can step through the trace [14:38] and understand the things that were pulled from  the internal system that went into the LLM. So, [14:44] I can see all the different tool calls  along the way. I can see the vector search [14:48] that returned the policy on this particular  product and device. And ultimately, I can see um [14:56] the the actual final output, the input and output  of the LLM that made that final decision. So, [15:03] I have I have a a lot of of uh deep observability  here. And I can also see over here a number of [15:10] metrics. These These are the actual LLM as judge  evals running, telling me whether or not the [15:16] agent was in compliance as it was going through  this flow. And for instance here, I can see that [15:22] instruction adherence was false. I can hover over  that and I can see that it's actually saying that [15:28] it didn't include a refund request ID, which is  something that's in the prompt that I wanted to [15:33] make sure that it always does. So, I can actually  go in and I can do a number of things here. I can [15:39] I can either go in and update this eval and  say, "Actually, that's not behavior that is [15:44] required anymore." And so, I can create another  rev of that eval, or I can add feedback here. [15:51] And this is what we talk about actually  annotating the trace. I can go in here and I can [15:56] I can actually I can correct the value or I  can reinforce the value and I can give it a [16:01] reason why I'm reinforcing this value. Maybe  the reasoning that it came up with for this [16:06] is not exactly accurate and there's another  reason why I feel that this is in compliant [16:12] uh with with what it should be doing.  This then goes into the auto tune queue [16:17] and and goes back into a uh training the evaluator  at scale. Now, I'm going to go ahead and and go [16:24] back to my agent graph here for a second. Somebody  was talking in the previous session about how do [16:30] we process all this data at scale? How can we  find the things that are happening when when when [16:36] our system is collecting information but we might  not necessarily be running over all those traces. [16:41] And one of the ways that we do that is by actually  turning AI onto these data feeds. It's a great way [16:49] to see what could be happening across our  agents. And in this case, I see that there are [16:55] fi- there are several instances that it's flagged  for me where the refund amount is not accurate. [17:01] There's hallucinations and and the system has  been using tools to actually grant refunds across [17:07] the board and it actually highlights for me how  often this is happening and it gives me the option [17:13] to create an evaluator right here. And this is a  this is a pre-formed prompt that has been designed [17:21] with some of these examples injected into the body  of the prompt so that the evaluator immediately [17:27] starts out with rich context on what I'm looking  for and what I want to be tracking at scale. [17:34] I can save this evaluator. I can test this  evaluator for accuracy and I can go through the [17:39] entire process that we're talking about as many  times as I want until I feel that it's confident. [17:44] And then I can step into the the last step which  is creating actually creating a guardrail off [17:51] of this. So, I'm going to go back to the  the evaluator stream. And in this case, [17:58] the control the control is on and this  is where I can actually come through [18:04] and take these evaluators. I can use them in LLM  as judge mode if I want to prototype this before I [18:10] actually go through all that the effort of running  the epics on my own GPUs and training these [18:16] these small language models. When I'm ready  to actually step this into production, [18:21] I can take this into the studio the data science  studio and create these LLMs these SLM judges [18:27] and then I can register them and and execute them  at scale. Now, we've already seen a few examples [18:33] of this working. We've seen how it actually stops  things from happening and let's just go take a [18:39] look at one of these traces. We can see that when  we actually run this with the guardrail in place, [18:45] the guardrail fires at this point in the trace to  actually block that that bad refund from happening [18:53] and restirs the agent to redo the calculation  so that the agent gets it right and then the [18:59] the user gets the outcome that they  were looking for in the first place. [19:05] So, we'll go ahead and go back to the slides. Now,  um so, all of this is is available as part of the [19:12] Splunk agent observability. The last thing  we'll we'll touch on just briefly is the the [19:18] the other thing that we didn't look at was the  ability to see the number of tokens that are [19:22] associated with each one of these actions and each  one of these traces, which is going to give us the [19:27] visibility that we need to see where where there's  waste in our system. And this this also applies [19:34] to all the other agents in your fleet. So, one  of the products that we'll be talking about that [19:38] we'll be launching very soon is about helping you  achieve cost visibility across your entire fleet. [19:45] And with that, I'd like to welcome you to sign  up for for an early preview of this new product, [19:53] and I'll be taking questions offline. I'll be  over here in the corner. Thank you very much.