American Airlines Trims Complex AI Agents to Slash Token Costs 80%

美国航空精简复杂AI Agent，削减80%的Token成本 企业工程主管警告称，多智能体框架往往会带来不必要的脆弱性与开支，并认为生产系统需要确定性护栏、外部化状态存储以及短期供应商承诺。 专家组探讨了从单智能体实验向高韧性企业级多智能体系统的架构演进。与会嘉宾主张将对话状态从模型上下文窗口解耦并转移至外部执行引擎，将非确定性的意图分类与受到严格约束的确定性子智能体相结合。他们强调了实时评估网关、分布式OpenTelemetry追踪以及影子模式部署对建立企业信任的重要性。最后，发言者强调应基于无状态MCP等开放协议进行构建，并采用短期供应商合同，以在快速变化的生态中保持敏捷。 多 Agent 系统必须引入阶段门控与执行熔断等传统工程规范，且许多复杂管道实可重新设计为单 Agent 或零样本系统，例如美国航空以此在两周内将某应用 Token 成本骤降 80%。 (4:08) 企业架构日益推崇分层模式：在前置入口使用大模型进行非确定性意图分类，后续交由受约束的确定性子 Agent 执行，在合理架构下运行小模型其表现往往能超越粗放使用的大模型。 (7:35) 将交互记忆解耦至外部持久化状态引擎可避免上下文窗口饱和并在故障时实现确定性恢复，同时在 MCP 协议旁并行部署标准 REST 端点能兼顾可控性与执行算力成本。 (13:08) 为解决通用大模型在专业场景下引发的“语义意图分歧”，针对特定业务本体微调的小语言模型（SLM）展现出更高可靠性，而分散在各工具中的企业记忆也在倒逼组织治理历史数据与权限体系。 (21:05) 企业通过让流水线在真实流量中运行但暂不触发实际操作的“影子模式”来逐步建立信任，同时在技术快速迭代下采取合同最长为一年的采购策略以规避供应商锁定风险。 (25:39)

Enterprise adoption of agentic AI requires tempering multi-agent hype with classical software engineering discipline, separating non-deterministic intent classification from deterministic execution layers while anchoring systems in externalized state and formal ontologies. Enforcing Deterministic Boundaries Over Multi-Agent Hype (4:08) Bradley Criqui argues that multi-agent systems must be governed by classical software engineering rigor, utilizing strict stage gates, isolation zones, and execution circuit breakers rather than unconstrained autonomous chatter. Henna Davar notes that American Airlines initially rushed a multi-agent system into production, only to replace it six months later with a single-agent, zero-shot setup built in two weeks that reduced token consumption by 80%. Consequently, American Airlines structures workflows into two tiers: non-deterministic frontier models deployed strictly at the intake boundary to parse ambiguous human intent, followed by deterministic sub-agents executing rigid business rules and policy guardrails. George Miranda and Criqui emphasize that when tasks have established remediation paths or standard REST endpoints, systems should default to deterministic scripts and traditional APIs. Bypassing generative execution for known operational workflows eliminates the need for expensive LLM-as-a-judge evaluation layers and prevents runtime cost inflation. Context Limits, Memory Silos, and Semantic Divergence (16:34) Context window saturation is a primary catalyst driving architectures toward multi-agent splits, yet corporate memory remains fragmented across tools like GitHub, Slack, and BambooHR. Vira Tkachenko explains that consolidating data across an organization's history surfaces substantial contradictory and stale information, turning agent deployments into a forcing function for internal data hygiene. Unifying access also forces enterprises to reconstruct role-based access control and privacy perimeters across departmental boundaries. Davar warns of "semantic intent divergence," where simple user inputs—such as asking to book an earlier flight—can mean rebooking, creating an additional reservation, or upgrading seats. To prevent catastrophic errors caused by frontier models misinterpreting business ontology, panelists advocate decoupling state into persistent external stores and using targeted small language models (SLMs) fine-tuned on specific domain logic rather than passing raw conversational transcripts through massive foundation models. Distributed Tracing and Evals-Driven Development (7:35) Miranda draws parallels between multi-agent orchestration and Site Reliability Engineering's historical shift toward microservices: while individual components are deterministic, multi-agent execution paths become probabilistic. Traditional flat text logging is insufficient for production accountability; engineering teams must adopt distributed tracing frameworks like OpenTelemetry to log exact prompts, retrieved context, tool calls, latencies, and probabilistic verification scores within each execution span. The panel agrees that enterprise development is shifting toward Evals-Driven Development (EDD). At InsightFinder, real-time LLM gateways evaluate inputs and outputs dynamically, halting transactions immediately if prompt injections or data leaks occur. Criqui similarly advocates using secondary models as validation checksums, noting that evaluating an agent's output against an independent model frequently reveals hallucinations that primary agents report as successful. Building Stakeholder Trust and Avoiding Vendor Lock-In (25:39) To build operational trust in highly regulated industries, American Airlines deploys an internal "Agent Operating System" utilizing a shadow mode architecture. Agent pipelines run against live production data and call tools, but their decisions are recorded in internal audit logs and reviewed on stakeholder dashboards rather than executed directly, transitioning to autonomous execution only after reliability is proven. Because rapid foundational shifts can render proprietary tooling obsolete within months, Davar emphasizes adopting open, interoperable standards like stateless Model Context Protocol (MCP) and independent semantic abstraction layers. To prevent catastrophic vendor lock-in, American Airlines has abandoned historical multi-year agreements in favor of one-year contract caps, maintaining an enterprise procurement posture focused on buying modular capabilities without long-term lock-in. While emerging standards like stateless MCP and real-time evaluation gateways improve operational scalability, the panel underscores that enterprise memory architectures remain fundamentally unresolved, with the industry still lacking consensus on how to define, secure, and federate contextual memory across organizations.

在Ai4研讨会上，来自航空、投资管理、可观测性与软件研发领域的工程负责人指出，落地企业级智能体架构的核心不在于盲目堆叠前沿大模型或无序的多智能体通信，而在于以经典工程纪律隔离不确定性，将状态外置并依托评估驱动开发与开放标准建立可靠系统。 架构解耦与防范多智能体过度设计 (4:08) 行业对多智能体架构的过度追捧在企业实际落地中面临现实检验。American Airlines的人工智能工程与治理负责人Henna Davar透露，其团队此前曾仓促上线一套复杂的多智能体工作流，但在6个月后发现该场景使用单智能体或零样本提示即可更稳定地解决，最终团队耗时两周重构应用，将Token消耗成本直接降低了80%。CBRE Investment Management全球工程负责人Bradley Criqui强调，多智能体协同不能依赖自由发散的智能体对话，而必须回归传统软件工程纪律，设置严格的阶段门禁、隔离区和形式化分界点，利用事件驱动的离散过渡、访问控制与熔断机制阻止失控循环。 针对确需多个智能体协同的场景，Davar建议将对话状态从大模型易失的上下文窗口中彻底抽离，移入外部持久化状态存储或工作流执行引擎。状态与运行时解耦后，不仅在执行步骤意外中断时支持确定性恢复，也避免了子智能体向编排器回传全部原始对话记录，大幅削减了上下文开销，同时沉淀下不可篡改的动态审计日志。 确定性执行与双轨语义防护 (7:35) 将系统划分为非确定性与确定性双层架构是与会者的一致共识。Davar阐述了American Airlines的设计原则：前沿大模型仅部署在系统最前端的输入边界，专门负责解析模糊的人类自然语言并准确分类意图；一旦意图明确，任务便直接分发给受严格业务逻辑、策略护栏与语义数据层约束的领域子智能体。InsightFinder市场营销副总裁George Miranda以其SRE智能体Rei为例说明，虽然异常诊断需要概率化的大模型推理，但实际故障自愈往往只需执行既有的确定性脚本，直接调用已知脚本可规避昂贵的大模型裁判层与运行成本膨胀。 此外，自然语言的歧义性极易导致“语义意图分歧”。例如旅客提出“想订更早的航班”，系统逻辑可能涵盖改签、新建预订或升舱。为了防止通用模型在细分业务领域出现灾难性偏差，小组建议采用在特定业务本体上微调的小型语言模型（SLM）。MacPaw首席技术与创新官Vira Tkachenko表示，出于数据隐私与低延迟考虑，MacPaw在macOS和iOS设备端大力投资微调本地小模型，专注于工具调用等单一职责。Bradley还补充指出，CBRE采用双模型交叉校验机制，利用独立的次级模型校验首个智能体的输出，以排查幻觉与关键遗漏。 记忆割裂与上下文工程瓶颈 (16:34) 上下文记忆被Tkachenko列为当前智能体基础设施中最核心的未决挑战。现代企业的组织知识高度碎片化地分散在GitHub代码库、Markdown文档、BambooHR、Confluence及Slack沟通记录中。尽管将扁平文件直接灌入上下文窗口是早期快捷做法，但暴力填充在企业规模下财务成本极高。MacPaw在整合18年历史数据的实验中发现，跨系统整合暴露出海量陈旧和相互矛盾的信息，这迫使企业必须先治理底层数据卫生，才可能将其转化为有效资产。 上下文容量与状态传递同样成为限制单智能体表现的主要瓶颈，迫使团队将复杂流程拆解为细粒度管线。与此同时，业界对智能体记忆的定义依然混乱，针对状态管理与短期、长期或程序性记忆的划分标准缺乏共识，统一的集中式共享存储也带来了角色权限控制（RBAC）与数据隔离边界的重构难题。对此，CBRE将研发重点转向与内部业务流深度集成的“上下文引擎”，力求使大模型具备精确的作业上下文。 影子模式与评估驱动开发 (25:39) 在民航等强监管环境中部署自主系统，核心在于建立业务利益相关方的信任。American Airlines开发了集成了安全护栏、成本控制与链路追踪的“Agent OS”，并严格实行“影子模式”（Shadow Mode）：端到端智能体管线直接接收真实生产数据运行并输出推荐，但该结果仅写入内部审计库，不直接触发生效或面向旅客展示；业务人员通过仪表盘长期核验智能体的推理准确性，待建立充分信心后才逐步放行流量。 系统运行防护需要结合实时网关与评估驱动开发（EDD）。Miranda介绍，InsightFinder通过LLM网关对输入输出进行实时评估，一旦探测到提示词注入便立即阻断，若输出端出现敏感数据或个人身份信息（PII）泄露也会即时拦截并报错。与会专家一致认为被动的“点赞/点踩”按钮反馈过于滞后，行业正加速从测试驱动开发（TDD）转向评估驱动开发（EDD）。同时，智能体系统的可观测性不能停留在普通文本日志，而必须依托OpenTelemetry等分布式追踪框架捕获每个执行跨度（Span）内的提示词、记忆、工具调用及评估打分，实现深层归因。 开放协议与规避技术锁定的采购策略 (32:50) 面对日新月异的技术迭代，企业基础设施应避免用单体厂商方案取代既有架构。小组主张在现有API端点之上，通过Anthropic的模型上下文协议（MCP）构建可复用的组件。Davar特别强调，MCP向无状态模式的演进消除了横向扩展的瓶颈，配合业务知识图谱与语义层，能让智能体在不重构老旧核心库的前提下安全访问企业数据。此外，Bradley提及CBRE在每个MCP服务旁均标准化部署传统REST端点，对于可预测任务优先走REST路由以节约算力。 由于底层技术演进极其迅速，与会者警告采用专有技术栈将带来极高的锁定风险。Davar表示，American Airlines已彻底调整采购策略，摒弃了以往3年、5年甚至10年的长期合同，全面转向“保持约会而非结婚”（Dating, Not Marrying）的采购模式，将企业软件合同严格限制在一年以内，从而保留在更优模型或框架涌现时随时替换底层组件的灵活性。 当前智能体软件架构正处于从概念验证向工程落地的过渡期。虽然无状态MCP与分布式追踪为老旧系统的智能化接入提供了路径，但跨部门的记忆碎片化治理、企业级权限隔离以及对“记忆”定义的行业分歧，仍是阻碍全自主智能体大规模落地的关键技术阻碍。

## Background 本内容由 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台。页面简介表明，本次圆桌研讨汇聚了来自 Team8 VC、美国航空（American Airlines）、MacPaw、InsightFinder AI 和 CBRE 投资管理的多位技术高管与工程领袖，深入探讨了 Agentic AI 在企业级落地中所必需的编排层软件、状态管理、防护栏、工具适配与基础设施架构演进。 ## TL;DR 圆桌深入剖析了企业构建 Agentic AI 编排系统时的关键工程权衡。嘉宾们一致主张在非确定性的意图分类与确定性的受约束工作流之间建立清晰的分界点，并推崇将内存状态外置于持久化工作流中，以大幅降低 Token 成本并规避失控任务。同时，讨论重点强调了利用 MCP 协议和分布式追踪（如 OpenTelemetry）打通传统遗留资产，提出从 TDD/BDD 迈向“评估驱动开发”（EDD），并通过“影子模式”与“只恋爱不结婚”的一年期采购策略破除供应商锁定与安全风险。 ## Atmosphere & style 整场研讨气氛务实、幽默且极具工程实战底色，没有浮夸的 AI 概念炒作，充满资深技术骨干在真实业务泥潭中摸爬滚打出的冷峻洞察。开场嘉宾们分享修图合影、短信斗骗子、猫咪数据追踪等生活趣事，迅速拉近距离；随后话题直切要害，现场频繁迸发出“半夜被报警电话叫醒的是工程师而非产品经理”和“只恋爱不结婚”等生动金句，在严谨的技术架构思辨与会心一笑的行话共鸣间保持着紧凑而高密度的信息张力。 ## Key takeaways - [04:08](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=248s) 多智能体系统的协作复杂性呈指数级上升，必须借鉴传统软件工程思维，设置阶段关卡（Stage Gates）、分界点与事件驱动隔离机制，以防止失控任务与幽灵进程。 - [05:10](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=310s) 架构选型切忌盲目追求多智能体过度设计；将内存状态外置到外部持久化存储（Durable Workflow）不仅支持故障断点恢复，还能大幅降低高达 80% 的 Token 冗余开销。 - [07:41](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=461s) 借鉴云原生分布式系统经验，智能体的执行路径具有概率性，唯有建立端到端全链路追踪（End-to-End Tracing）向业务方解释因果关系，才能在企业内部建立起对 AI 的信任。 - [09:20](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=560s) 多智能体协作的核心价值在于共享上下文的语义化通信，以及通过引入独立模型进行交叉校验与对抗审查（Model Debrief），有效破除单一智能体的“确认偏差”。 - [11:10](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=670s) 架构设计远比底层模型本身更关键：采用顶级 SOTA 模型处理非确定性的意图分类，后续路由交给受护栏约束的子智能体调用标准 REST API 或既有脚本，小模型也能胜任全栈应用。 - [14:33](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=873s) 在编排器与工作智能体架构中，将对话历史、工具调用结果与监控日志集中落盘至外部持久化存储，能防止上下文窗口爆炸，并通过离线轨迹挖掘持续优化协作瓶颈。 - [16:34](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=994s) 企业将碎片化数据整合为集中式智能体内存时面临严峻治理瓶颈，不仅需要全盘重构权限隔离机制（ACL），还倒逼企业必须彻底清洗多年积累的历史陈旧与冲突数据。 - [20:01](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1201s) 业务场景中普遍存在“语义意图分歧”（如航班预订场景中改签、新建还是升舱的多义性），缺乏垂直业务上下文与本体定义的智能体极易产生误判或相互矛盾的决策。 - [21:55](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1315s) 针对特定垂直任务微调小语言模型（SLM）践行“少即是多”原则，剥离多余的世界知识并注入专有业务逻辑，能显著压制意图分歧并带来高确定性的执行输出。 - [22:50](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1370s) 在端侧设备运行高度聚焦单一用例的小模型兼具隐私保护与低成本；从生产环境中实时捕获未预料到的边缘异常行为，是持续微调专用模型的最佳数据飞轮。 - [24:38](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1478s) “上下文就是国王”，通用大模型在缺乏业务上下文时如同无米之炊，企业实现数字化转型的核心突破口在于构建能吃透复杂业务流程的底层“上下文引擎”。 - [25:40](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1540s) 美国航空构建企业级 Agent OS，采用强制“熔断开关”与传统规则引擎兜底；在面向核心业务时推行“影子模式”（Shadow Mode）让智能体全链路空转审核建议，以渐进式放量建立安全信任。 - [28:18](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1698s) 抛弃线上无意义的事后点赞机制，InsightFinder 在 LLM 网关层串联实时裁判（LLM Judges），在输入端就地阻断 Prompt 恶意注入，在输出端实时拦截 PII 敏感信息泄露。 - [30:37](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1837s) 智能体生成的逻辑在线上运行前往往未经过人类完整阅读，工程范式因此从 TDD 迈向“评估驱动开发”（EDD），自动化评估体系已成为全生命周期不可妥协的工程底线。 - [33:01](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1981s) 传统资产与智能体融合宜采取自研核心编排层的策略以掌握完全控制权，通过在既有 API 端点上封装 MCP 服务器，低成本将遗留系统资产转化为标准化智能体工具。 - [34:30](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2070s) 监控智能体必须以“分布式追踪”替代传统的“打日志”，结合 OpenTelemetry 标准并在链路跨度（Span）中注入实时评估逻辑，以捕获并解释概率性大模型调用的全流程上下文。 - [36:18](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2178s) 构建基于知识图谱与本体（Ontology）的语义层是智能体通信与数据调用的基石，MCP 协议向无状态（Stateless）模式演进有效解决了高并发下的扩展瓶颈。 - [37:48](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2268s) 技术迭代以周为单位快速翻新，企业采购策略已转向“只恋爱，不结婚”（合同期不超过一年），依托开源开放标准解耦架构，彻底防范昂贵的供应商锁定风险。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=4s) 嘉宾介绍与个人最喜爱的 AI 趣事 Nitay Joffe（Team8 风险投资合伙人）作为主持人开场，欢迎大家参加这场关于 Agentic AI 编排层（Orchestration Layer）的研讨，并邀请四位嘉宾简要介绍自己以及各自在日常生活中最喜欢的 AI 个人用法： - **Henna Davar**（美国航空 / American Airlines AI 工程与 AI 治理负责人）：分享了自己从去年 12 月开始的日常用法。由于她的大多数家人都住在澳大利亚，她经常错过家里的许多庆祝聚会。随着图像生成和编辑模型的飞跃，她现在经常将自己的照片天衣无缝地“P”进家人的聚会合影中，并打趣道自己通常会特意选一张把自己修得最好看的照片放进去。 - **George Miranda**（InsightFinder 营销副总裁，O'Reilly《可观测性工程》合著者）：分享了一个反欺诈短信用途。他以前常收到诸如“我是你们 CEO，正在参加活动，赶紧帮我买几张礼品卡”之类的诈骗短信。他自己去回复会浪费大量时间，于是专门写了一个短信 Agent 来替他周旋，并给 Agent 下达了一堆指令，让它表现得“极其笨拙、对技术一窍不通”。看着骗子和 Agent 之间啼笑皆非的对话让他乐此不疲，而最近这类骚扰短信也明显减少了。 - **Vira Tkachenko**（MacPaw 首席技术与创新官 CTIO / 联合创始人，专程从乌克兰赶来）：MacPaw 旗下拥有 CleanMyMac、Setapp 等知名 B2C 软件。她最有趣的 AI 用途是“可观测性猫咪成长追踪”。她收养了一只名叫 Shusha 的混血小猫，由于品种未知，她在云端建了一个项目，把尾巴长度等各种生理指标记录进去，让 AI 生成图表、与正常猫咪成长曲线对比，并尝试推断预测它的具体品种。 - **Bradley Criqui**（CBRE Investment Management 全球工程负责人）：对他而言，最兴奋的用法是在工程端加速软件构建，通过搭建 Agentic 工程流水线和编排系统，以极高的速度交付新功能和新应用。 - **Nitay Joffe**（主持人）：分享了自己带孩子去游乐园的经历。他用 Claude 生成全套行程规划，模型自动检索了所有游乐设施的限制并询问孩子的身高。令他万万没想到的是，模型贴心地提示：“我注意到你的孩子比所有最好玩的过山车限高刚好矮了 1 英寸，这里推荐几款鞋垫链接供你选购。”Nitay 笑着表示自己照做了，效果非常棒。 --- ### [04:08](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=248s) 多 Agent 协作的复杂性与工程控制 主持人 Nitay Joffe 将话题切入核心：从单一 Agent 迈向多 Agent（Multi-agent）系统时，系统需要彼此协作与通信，复杂性呈指数级飙升。面对多个自主运行的系统，如何管理协调并防止它们互相踩脚甚至造成灾难？ Bradley Criqui 指出，即便进入 Agent 时代，传统的软件工程思维依然必不可少。必须建立清晰的分界点（Demarcation Points）与阶段关卡（Stage Gates）。在实践中，他们通过基于事件驱动（Event-driven）的机制来实现 Agent 之间的工作交接，在各层之间配置控制层、完备的安全策略以及遥测监控（Telemetry），确保系统的隔离性与控制力，彻底遏制失控任务与幽灵进程（Runaway tasks）。 ```mermaid flowchart TD accTitle: Agentic Engineering Control and Demarcation Flow subgraph MultiAgentExecution["多 Agent 执行与交接"] A["Agent A (上游任务)"] -->|"触发交接事件"| B["阶段关卡 / 分界点 (Stage Gates)"] B --> C["Agent B (下游子任务)"] end subgraph ControlGovernance["控制与持久化治理"] B -.-> D["安全与隔离控制 (Isolation & Security)"] A <--> E["持久化工作流与外部状态 (Durable Workflow)"] C <--> E E -.-> F["全链路追踪与可观测性 (Tracing & Telemetry)"] end ``` --- ### [05:10](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=310s) 单 Agent 与多 Agent 的架构取舍：状态外置与成本优化 Henna Davar 结合美国航空的落地实践补充道，企业必须准确评估实际业务场景，切莫过度设计。很多最初被设想为复杂多 Agent 的场景，深入评估后往往发现只需单个 Agent 甚至单次零样本提示词（Zero-shot Prompting）就能搞定。 Henna 直言，在她 20 年的 IT 生涯中，第一次在短短 6 个月内就把一个系统称为“遗留系统（Legacy App）”——团队此前迅速上线了一套多 Agent 架构，但 6 个月后发现完全可以简化为单 Agent，于是仅耗时 2 周就完成了重构。这次架构重构直接节约了约 80% 的 Token 成本。 对于确实需要多 Agent 架构的系统，Henna 分享了关键设计原则： - **内存状态外置化（Externalizing In-memory State）**：将 Agent 内部维护的内存会话转移到外部持久化存储或持久化工作流执行引擎（Durable Workflow Execution Engine）中。 - **故障弹性恢复**：一旦 Agent 工作流出现网络异常或步骤失败，不需要从头重跑，因为持久化存储记录了历史所有步骤，随时可以无缝恢复。 - **大幅降低 Token 开销**：Token 成本是编排层首要考虑的指标。有了持久化共享状态，子 Agent 向主编排 Agent 汇报时无需将冗长的完整上下文逐字传递，只需按需索引外部状态。 - **提供确定性审计链路**：状态外置化让动态决策与推荐过程“白纸黑字”落地成卷，便于追踪复盘模型做出的每一次决定。 --- ### [07:41](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=461s) 借鉴分布式系统：端到端可观测性与信任建立 George Miranda 结合自己深厚的 SRE 和分布式系统背景指出，当前 Agent 编排面临的问题在传统工程领域早有先例。一年前的 AI4 峰会上大家还在讨论单体式的 LLM 应用（类似于早期的单体服务器架构），如今一跃进入跨成百上千节点的分布式多 Agent 架构。 在传统的云原生分布式系统中，虽然业务代码本身是确定性的（Deterministic），但执行路径却是概率性的（Probabilistic）——两个看似完全一样的请求，底层的路由和执行链路可能截然不同。而在多 Agent 系统中，这种复杂性更甚。George 强调，目前许多团队过度执着于工作流编排和状态管理，却忽视了**全链路追踪（End-to-End Tracing）**。如果无法向业务方清晰解释系统的每一步执行细节与因果关系，就永远无法在企业利益相关者之间建立起对 AI 系统的信任。 --- ### [09:20](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=560s) 语义化通信与多 Agent 的实际应用边界 Vira Tkachenko 认为，从底层来看，Agent 与传统工程系统并无二致，依然可以通过标准 API 或 MCP（Model Context Protocol）协议进行交互。但 Agent 带来的真正革新在于**以共享上下文和共享记忆为核心的语义化通信**。 Agent 兼具核心智能，介于人类与传统确定性系统之间。除了结构化 API，它还能像人类一样更新上下文，甚至直接使用 Jira 提工单，或通过 Slack 消息来进行跨系统协调。不过 Vira 也坦言，在 MacPaw 内部，绝大多数通用业务场景目前仍以单 Agent 为主；真正的多 Agent 复杂协作系统目前主要被高阶软件工程师用于解决极具复杂度的研发任务。 Bradley Criqui 随后补充了多 Agent 的另一个关键不可替代场景：**模型交叉校验与对抗决策（Model Debrief / Multi-agent Evaluation）**。单个 Agent 往往会产生“确认偏差”，自信地认为自己已经完美完成了任务；而引入第二个经过验证的独立模型进行对照审查时，往往能立刻揭露第一个模型遗漏或误判的隐蔽缺陷。 ### [11:10](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=670s) 架构设计优先：确定性与非确定性工作流的拆分与成本优化 在多智能体系统的构建中，清晰划分系统中的“确定性”（deterministic）与“非确定性”（non-deterministic）组件是架构设计的核心前提。Henna Davar 指出，并非所有业务场景都适合交给智能体自由发挥。在实际落地中，智能体主要被重度用于“意图分类”（intent classification），因为这是系统直接接收终端用户和客户输入的第一道关口，需要调用最先进的 SOTA（State-of-the-Art）模型进行精准解析。一旦意图被准确分类，系统就会将其移交给各个领域的专业子智能体，而这些子智能体绝大多数都在执行确定性的工作流。 ```mermaid flowchart TD accTitle: 确定性与非确定性智能体分流架构 subgraph Ingestion["输入与意图解析 (非确定性)"] A["用户输入 / 客户请求"] --> B["SOTA 顶级模型<br/>(意图分类 Router)"] end subgraph Execution["专业子智能体执行 (确定性)"] B --> C["受约束智能体 A<br/>(规则/策略/护栏)"] B --> D["受约束智能体 B<br/>(领域专家工作流)"] C --> E["语义层 & 真实接地数据 (Grounded Data)"] D --> F["标准 REST API / 自动化修复脚本"] end ``` 这种模式在业内被称为“受约束智能体”（constrained agent），即智能体必须在既定的策略、护栏（guardrails）和业务规则约束下运行，直接调用底层语义层并对接真实接地数据（grounded data）以返回确定性结果。Henna Davar 强调，企业当下的首要业务诉求是在加速生成式 AI 投产的同时实现极致的 Token 优化与成本控制。实践证明，系统架构的设计远比底层模型本身更重要——"只要架构设计得当，使用小型模型也能构建出功能完备且复杂的全栈应用；反之，即便拥有最顶级的 SOTA 模型，若缺乏优秀的架构，最终做出来的可能连平庸的 Web 应用都不如。" George Miranda 随后结合 InsightFinder 的实践对此表示赞同。InsightFinder 旗下有一款名为 REI 的 SRE 智能体。在可靠性运维工作流中，虽然部分环节需要概率性的推理与解释，但后续的故障修复（remediation）通常只需要执行一段现成的脚本，完全没有必要调用外部大模型。在复杂的业务流程中，必须敏锐辨别哪些环节真正需要生成式 AI，哪些环节应沿用经过验证的成熟方案。这种拆分不仅能大幅压降 Token 成本，还省去了在概率性环节上方部署外部验证模型或 LLM 裁判（Judge LLM）的开销。同时，为每个 MCP（Model Context Protocol）服务器提供标准 REST API 接口，只要能走确定性的 REST API 路径，就能以更低的成本获得高置信度的确定性结果。 - **意图分类路由**：入口端使用最强 SOTA 模型解析多变的用户意图，后端移交给领域专家子智能体。 - **受约束智能体定义**：严格在策略和护栏内运行，通过语义层对接 Grounded 业务数据。 - **架构 vs 模型**：优秀的架构能让轻量级的小模型发挥出复杂全栈系统的效能，降低对昂贵通用大模型的依赖。 - **运维与脚本落地**：在 SRE 等场景中，推理可采用概率模型，但修复执行应直接调用成熟脚本或 REST API。 --- ### [14:33](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=873s) 多智能体协作与状态管理：共享内存的外部化落地 当业务复杂度上升到必须依赖多智能体协同（Multi-agent Orchestration）时，多智能体之间的上下文协作与“共享内存”（shared memory）机制便成为决胜关键。针对主持人 Nitay Joffe 关于多智能体协作中共享内存具体形态的提问，Henna Davar 分享了其正在落地的“编排器-工作智能体”（Orchestrator-Worker / Sub-agent）架构模式。 在多智能体持续交互的过程中，编排器智能体的上下文窗口（context window）很容易被大量的对话历史迅速填满。为了防止生产环境中出现上下文爆炸和 Token 浪费，团队采取了“状态管理外部化”的策略。他们引入持久化存储（persistent storage）作为统一的状态与内存底座： 1. **对话全流程记录**：智能体之间每一次跨节点交互与对话，均实时写入外部持久化存储。 2. **中间工具调用追踪**：子智能体发起的每一次工具调用（tool calls）及其中间返回结果，全部记录在案。 3. **成本与审计日志**：Token 消耗明细、系统运行追踪日志统一同步落盘。 Henna Davar 指出，外部化状态管理不仅解决了上下文膨胀和 Token 成本问题，更带来了额外的核心价值：通过对持久化存储中的交互轨迹和调用数据进行离线挖掘与模式识别（pattern identification），团队能够洞察智能体协作中的瓶颈，进而反哺和优化整个多智能体系统的运转逻辑，让智能体协同变得更加高效和聪明。 --- ### [16:34](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=994s) 内存层碎片化与陈旧数据挑战：企业数据的治理困境 Vira Tkachenko 将“内存层”（memory layer）定义为当前企业乃至整个 AI 行业亟待攻克的下一个核心前沿。目前企业内部的数据与记忆处于严重碎片化（fragmented）的状态：研发团队的数据散落在 GitHub 代码仓库中，通常以 Markdown 文件的形式作为上下文，虽能应付现阶段需求，但在成本优化上仍显粗放；而人力资源等业务部门的数据则分布在 Confluence、内部知识库以及 Slack 等即时通信工具中。 ```mermaid flowchart LR accTitle: 企业碎片化数据向集中式智能体内存汇聚面临的挑战 subgraph Sources["碎片化数据源"] S1["GitHub (代码/Markdown)"] S2["Confluence / 知识库"] S3["Slack / 即时通讯"] end subgraph IngestionChallenge["整合与治理难题"] Sources --> M["集中式企业共享内存<br/>(Centralized Storage)"] M -.-> C1["访问控制与权限重构 (ACL)"] M -.-> C2["18年历史陈旧数据 (Stale Data)"] M -.-> C3["内部冲突矛盾数据 (Controversial Data)"] end subgraph Agents["智能体消费"] M ==> A["多智能体系统 / 业务 Agent"] end ``` 为了让各类智能体都能无缝调取所需上下文，MacPaw 正在探索将这些分散的数据统一汇集到一个中心化的存储层中。然而，这种集中化实践面临着极其严峻的数据治理挑战： - **权限与隔离机制**：过去在各孤立系统中已经解决的权限控制问题，在向智能体开放全局上下文时必须全盘重构。 - **陈旧与矛盾数据**：企业积累了长达 18 年的历史资产，其中充斥着过时、互相矛盾的数据（stale and controversial data）。 - **倒逼数据治理**：智能体的落地反向倒逼企业必须彻底清洗底层资产，因为“数据只有被清洗干净，才能真正成为智能体可用的核心资产”。 George Miranda 补充道，上下文窗口的物理限制正是迫使系统向多智能体架构演进的核心推手。即便在处理某项概率性推理任务时，智能体若要高质量完成工作，往往需要喂入庞大的背景上下文。受限于上下文长度，原本可以在单个智能体或单一进程中处理的任务，最终被迫进一步拆解为多个细分步骤，由多个子智能体协同承接。 --- ### [20:01](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1201s) 内存定义的行业困惑与上下文中的语义意图分歧 行业内部对于“内存”（Memory）这一概念的定义依然存在极大的分歧与模糊。Henna Davar 提到，在与 Strands 框架团队交流以及同 MongoDB 的 Nina 探讨时，不同专家对状态管理与内存类型的理解各不相同——短时内存（short-term memory）、过程内存（procedural memory）与长期存储之间究竟该如何划分？企业能否用统一的内存架构支撑所有场景，同时确保数据安全不受侵害？这些问题在当前的 AI 基础设施领域尚无明确的标准答案。 在谈及上下文管理时，Henna Davar 提出了“语义意图分歧”（semantic intent divergence）的概念。虽然业内常说“上下文为王”，但实际落地中业务短语的多义性对决策准确度构成了巨大考验。以美航的机票预订场景为例：当用户输入“我想订早一点的航班”（I want to book an earlier flight）时，该请求在业务领域内可能对应截然不同的操作意图： - 改签现有的已有预订； - 重新发起一笔全新的机票预订； - 因为想要升舱而选择重新订票。 ```mermaid flowchart TD accTitle: 航空领域语义意图分歧示例 U["用户输入: '我想订早一点的航班'"] --> R{"语义意图分歧解析"} R -->|分支 1| A1["改签现有航班 (Rebooking)"] R -->|分支 2| A2["新建独立订单 (New Booking)"] R -->|分支 3| A3["升舱换票 (Upgrade & Rebook)"] ``` 如果智能体无法依托精准的业务上下文对这些意图进行优先级排序与层级判定，就会做出错误的业务决策。Henna Davar 还引述了 Andrej Karpathy 使用 AI 智能体控制智能家居的案例来做类比：如果 Karpathy 拖着疲惫的身体回家说了一句“今天真是杀人般劳累的一天，快把开关全灭了/干掉，我要休息”（I've had a killer day and let's kill the switches and relax），负责安防的智能体若无法理解特定语境下的语义分歧，就极有可能对“kill”产生危险的误判。因此，深度结合业务领域的精准上下文与意图解歧，是智能体能够做出正确决策的基石。 ### [21:55](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1315s) 语义意图分歧与小语言模型的垂直领域微调 当智能体没有经过特定训练，无法透彻理解业务领域的专业术语、行业概念以及上下文层（Context Layer）之上的本体结构（Ontology）时，系统就会出现所谓的“语义意图分歧”（Semantic Intent Divergence）。这种分歧会导致不同智能体之间产生相互矛盾的输出或不可预测的结果。现场嘉宾指出，这恰恰是需要针对特定垂直领域微调小语言模型（SLM）的核心原因。举例来说，对于航空公司而言，“重新订票”（Rebooking a flight）有着极其严苛的业务流程与规章，而对于演出场馆老板来说，“预订演出”（Booking acts）则是完全不同的另一回事。如果盲目将这类细分任务丢给前沿基础大模型（Frontier Model），输出结果往往不可控，"天知道你会得到什么"。因此，在业务逻辑极为关键的场景下，针对特定任务拆分并采用小模型，往往能取得“少即是多”（Less is more）的效果。 ```mermaid flowchart TD accTitle: 领域专用小模型应对语义分歧 subgraph Divergence["前沿大模型的不确定性"] A["多义业务词汇 (如: Rebook / Book)"] --> B["前沿模型 Frontier Model"] B --> C["语义意图分歧 (Semantic Intent Divergence)"] C --> D["结果矛盾与不可控"] end subgraph SLMApproach["小模型微调方案"] E["领域本体与业务逻辑 (Ontology)"] --> F["领域微调小模型 (Fine-tuned SLM)"] F --> G["精准工具调用与确定性输出"] end ``` - **语义意图分歧（Semantic Intent Divergence）**：通用大模型难以区分不同业务场景下的同名或相似概念，容易造成跨智能体协作时的决策冲突。 - **少即是多（Less is more）**：剥离多余的世界知识与冗长上下文，只给小模型灌输专有业务逻辑，能显著提升执行精度。 --- ### [22:50](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1370s) 端侧 AI 与生产环境真实数据的飞轮效应 针对小模型的落地，MacPaw 团队将重心投向了端侧 AI（On-device AI）。在消费级设备（如 iPhone 和 MacBook）上直接运行小型专用模型，不仅大幅降低了推理成本，更满足了将用户隐私置于首位的严苛要求。端侧模型的秘诀在于对用例极尽克制：模型被训练用来执行高度聚焦的单一任务（One task / One job to be done），例如专属的工具调用（Tool Calling），这样既能达到顶级回答水准，又兼顾了隐私与经济性。与此同时，现场讨论也揭示了一个常被忽视的环节——生产环境实时捕获能力的价值。团队在会议室里对业务逻辑（例如退改签）的预设再完美，一旦软件推向真实世界，用户总能以意想不到的方式将其搞崩。唯有实时捕捉生产环境中的异常行为与边缘用例，才能获取最宝贵的微调语料，进而训练更精准、更可控的小模型，并借助顶层的裁判机制（LLM Judge）来评估其是否真正达成了业务目标。 --- ### [24:38](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1478s) 上下文引擎：重塑企业业务流程的基石 在谈及模型能力之后，讨论迅速聚焦到了“上下文”（Context）的决定性地位上。现场嘉宾直言“上下文就是国王，或者是女王”（Context is king or queen）。当下行业虽然已经触及了通用级智能的门槛，但如果无法提供恰当且精准的上下文，模型就如同无米之炊；反之，一旦给足了契合业务的上下文，想象空间将是无穷的。企业数字化转型的真正突破口，就在于深入业务流程肌理、挖掘并构建底层的“上下文引擎”（Context Engine）。只有打造出能够吃透复杂业务流程的上下文引擎，企业才能跳出打补丁式的微调，真正重构并开创全新的工作流程与思考方式。 --- ### [25:40](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1540s) 美国航空的企业级 Agent OS、安全底线与影子模式 随着探讨深入到全自主系统的编排，主持人抛出了核心痛点：谁来负责防护栏？如何才能让人在夜里安心入睡？美国航空（American Airlines）的技术主管详细分享了在强监管行业构建利益相关者信任的企业级实战方案。美国航空搭建了一套中心化的“智能体操作系统”（Agent Operating System），供全企业所有团队调用。该系统整合了开源智能体框架、第三方的安全防护解决方案、全链路追踪与可观测性系统，同时对成本与延迟设置了硬性阈值。更为关键的是两道安全底线：一是所有智能体系统必须强制配备“熔断开关”（Kill Switches）；二是生产环境中始终并行运行着传统的基于规则的引擎（Rule-based engine），一旦智能体出现偏差，系统能够无缝回滚降级到成熟稳定的基准规则上。 ```mermaid flowchart LR accTitle: 美国航空影子模式与防护架构 subgraph Gateway["Agent OS 核心控制"] Input["业务/用户输入"] --> Guard["安全/成本/延迟防护栏"] Guard --> Agent["开源智能体管道"] end subgraph ShadowSystem["影子模式验证 (Shadow Mode)"] Agent --> Decision["生成观察与决策建议"] Decision -.->|不直接下发终端| DB[("拦截并存入数据库")] DB --> Dashboard["业务人员仪表盘审核"] end subgraph Fallback["生产兜底保障"] Agent -->|异常/触发熔断| Rule["传统规则基准引擎"] Rule --> Output["安全保底输出"] end ``` 针对核心与面向客户的高危业务，美国航空普遍采用“影子模式”（Shadow Mode）来逐步建立信任： - **全链路空转**：从数据摄取、工具调用到决策建议，完整的智能体链路端到端运行，但输出结果绝不直接发给客户，而是静默记录在数据库中。 - **业务回溯与渐进放量**：业务人员通过可视化仪表盘审查智能体的决策逻辑与推荐建议，直到全员对其稳定性和准确性达成共识，才逐步推向生产环境灰度放量。 --- ### [28:18](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1698s) 确定性阶段门、隔离控制与网关级实时 LLM 裁判 在传统企业控制与防护方面，除了保留人机协同（Human-in-the-loop）外，更关键的是设置“阶段门”（Stage Gates）与事件分界点。当智能体完成阶段性工作并落地为一个事件（Event）时，必须通过更确定性的程序化规则进行风险校验。同时，严格推行权限隔离控制（Isolating controls），严禁向任何单一智能体开放全局访问权限，确保其权限边界局限在“知其所需”的最小范围内。 InsightFinder 的实践则将防御推向了网关层面的实时自动化评估。行业往往习惯于在开发阶段做评估或打磨提示词，但在线上却仅靠页面底部一个不痛不痒的“点赞/点踩”（Thumbs up/down）按钮，既无人复盘，也无法阻止已经造成的损害。InsightFinder 在 LLM 网关中串联了实时 LLM 裁判（LLM Judges）： - **输入层直接截断**：在请求被路由给底层大模型之前，裁判若识别出恶意 Prompt 注入或恶意攻击，直接就地阻断（Kill it at the source），根本不调用大模型。 - **输出层实时止损**：若模型输出涉及 PII（个人身份信息）泄露或越轨行为，网关会拦截输出，宁可给终端用户返回通用的报错提示，也绝不泄露敏感数据，并在后台实时触发告警推送到应急工单中。 --- ### [30:37](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1837s) 监控回滚实战与“评估驱动开发”（EDD）的演进 预防策略固然重要，但现场嘉宾强调，线上实时监控与快速应急响应能力在 AI 时代变得空前重要。由于智能体生成的代码与执行逻辑在进入生产环境之前可能没有任何人类完整阅读过，开发团队永远无法 100% 确定其行为。这就要求工程团队必须具备专门的训练与敏捷技能，能够在指标异动的第一时间迅速执行回滚并深入排查根因。 在谈到“如何才能在夜里睡个好觉”时，现场分享了一个生动的内部博弈趣事： > 产品经理往往急不可耐地想把那些直接用网页代码拼凑出来的新功能赶紧推向生产环境。面对催促，工程师祭出的杀手锏是："半夜被报警电话叫醒的人可不是你们产品经理，而是我们工程师！所以请给我们留出足够的时间把它打磨完善。" 话音一落，全场莞尔，而这招往往能让产品团队意识到严肃性并妥协让步。 讨论最后升华至工程范式的历史演进：从早期的业务驱动开发（BDD）、领域驱动开发（DDD）到测试驱动开发（TDD），当下智能体开发已经正式迈入**评估驱动开发（Evals-Driven Development, EDD）**时代。在智能体产品的生命周期中，无论是前期的开发闭环还是上线的实时监控，自动化评估体系（Evals）已经成为了不可逾越、不可妥协的工程底线。 ### [33:01](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1981s) 传统系统与智能体编排的融合：组件复用与自研控制 主持人将讨论引向系统构建的最佳实践：企业普遍拥有大量仍在发挥关键业务价值的传统遗留系统（Legacy Systems），如何在这类资产之上平滑接入新兴的智能体（Agentic）框架，化解新旧架构间的不匹配与断层，是当前落地的核心瓶颈。 ```mermaid flowchart LR accTitle: Legacy Systems Integration with Agentic AI subgraph Legacy["传统资产层 (Legacy Assets)"] A["现有 API 端点"] B["核心业务数据库"] end subgraph Protocol["协议与语义抽象层"] C["MCP 服务器 (Model Context Protocol)"] D["语义层 / 知识图谱 (Semantic Layer)"] end subgraph Agentic["智能体编排层 (Orchestration)"] E["内部自研编排引擎"] F["多智能体协作网络 (A2A)"] end A --> C B --> D C --> E D --> E E --> F ``` 针对这一问题，Bradley Criqui 指出应对策略的核心在于构建专用且可复用的组件。为了保持对系统的完全掌控力并最大限度复用既有系统，团队更倾向于自主构建编排层的核心部分，而非直接把全部功能托管给单一第三方厂商。通过在现有的 API 端点之上搭建 MCP（Model Context Protocol）服务器，既能避免因平台迁移引发的庞大集成工程，又能随时根据业务需求向外扩展，将传统接口低成本转化为智能体可直接调用的标准化工具。 - **核心原则**：优先构建专用、可复用的内部组件，避免将整个编排堆栈绑定到单一供应商。 - **实施路径**：在既有 API 端点上封装 MCP 服务器，由内部编排层统一调度，按需灵活扩展。 --- ### [34:30](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2070s) 可靠性与可观测性：分布式追踪替代传统日志 从系统可靠性与运维角度，George Miranda 分享了在展区与从业者交流时的普遍误区：许多团队仍在尝试用传统“打日志”（Logging）的方式来监控智能体，但在复杂的智能体系统中，"Distributed tracing is your friend. You are not logging, you are tracing what your agents do."（分布式追踪才是你的好伙伴，你不是在打日志，而是在追踪智能体的行为）。 分布式追踪能够捕获智能体在每一步执行中的全部上下文，并将各个调用链路串联起来，从而为端到端的执行链路提供完整的可解释性（Explainability）。面对概率性的 LLM 输出，系统还需要在追踪链路中嵌入实时评估机制，判断模型输出是否合法；若不合法，必须记录下原因并作为上下文数据保存。结合成熟的 OpenTelemetry 等行业标准，只需在传统分布式追踪中注入 AI 可观测性（AI Observability）层，为 LLM 的特定调用跨度（Span）补充深层上下文，即可实现确定性监控与非确定性大模型调用的可靠融合。 - **追踪优于日志**：利用分布式追踪（Distributed Tracing）完整还原智能体多步决策与调用的端到端上下文。 - **实时评估注入**：在追踪 Span 中加入实时评估逻辑，动态捕获并解释 LLM 概率性输出的异常原因。 - **标准协议支持**：采用 OpenTelemetry 作为底层监控标准，以保障系统的通用性与向后兼容性。 --- ### [36:18](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2178s) 标准协议、语义层与 MCP 无状态演进 Henna Davar 阐述了标准化协议与数据语义层在智能体落地中的基石地位。智能体间的通信（A2A）以及智能体对企业数据的调用，高度依赖标准互操作协议。团队在设计任何生产级智能体系统之前，首要任务就是确认是否存在一个包含本体（Ontology）或知识图谱（Knowledge Graph）的语义层（Semantic Layer），以此清晰定义业务领域内的实体与其关联关系。 在数据访问层面，开发团队不再需要编写专用的旧式数据调用代码，而是将传统系统数据统一封装并通过 MCP 暴露给智能体，同时在接口上统一构建安全与鉴权机制。此外，Henna 特别提到近期 MCP 协议开始支持无状态（Stateless）模式，这一重大改进有效解决了此前的扩展瓶颈，大幅提升了大规模并发场景下的扩展能力。 --- ### [37:48](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2268s) 破除供应商锁定：“只恋爱，不结婚”的技术选型策略 针对技术日新月异带来的架构淘汰风险，嘉宾们一致强调了避免供应商锁定（Vendor Lock-in）的紧迫性。George 幽默地指出，在今年的 AI4 大会上，大家讨论的内容与去年相比已经彻底换血；在当前的技术演进速度下，"I go to sleep Friday and like log out of work and when I come back on Monday, the approaches that I knew on Friday are completely different."（周五下班打卡睡觉，周一回到工位时，周五掌握的方案已经完全过时了）。因此，任何非标准、定制化的专有绑定都可能让企业陷入被动。 ```mermaid flowchart TD accTitle: Vendor Procurement Strategy Shift subgraph Past["过去：长期深度绑定"] P1["签订 3~10 年长期合同"] --> P2["高昂的定制化与迁移成本"] end subgraph Present["现在：敏捷解耦策略"] C1["单年合同 (Maximum 1 Year)"] --> C2["'只恋爱，不结婚' (Dating, not marrying)"] C2 --> C3["依托 MCP / OpenTelemetry 等标准协议随时平替"] end ``` Henna 赞同这一观点，并分享了美国航空（American Airlines）在采购哲学上的剧烈转变：过去企业习惯与厂商签订 3 至 5 年甚至长达 10 年的长期合作合同，而现在则采取“只恋爱、不结婚（Buy not marry / just dating for one year）”的原则，合同期限严格限制在最多一年。这种策略让企业能够始终保持敏捷架构，一旦行业涌现出更优方案即可快速切换，从根本上化解技术迭代周期中的供应商锁定风险。 在圆桌会议尾声，主持人赞叹本场讨论堪称架构模式与落地的“金矿”，并在向台下观众征集提问时打趣道，如果现场没有提问，他自己随身还准备了整整 57 个问题。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=4s) 我注意到你的孩子比所有最好玩的过山车限高刚好矮了 1 英寸，这里推荐几款鞋垫链接供你选购。 — Claude (引用自 Nitay Joffe) > [11:10](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=670s) 只要架构设计得当，使用小型模型也能构建出功能完备且复杂的全栈应用；反之，即便拥有最顶级的 SOTA 模型，若缺乏优秀的架构，最终做出来的可能连平庸的 Web 应用都不如。 — Henna Davar > [16:34](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=994s) 数据只有被清洗干净，才能真正成为智能体可用的核心资产。 — Vira Tkachenko > [20:01](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1201s) I've had a killer day and let's kill the switches and relax — Andrej Karpathy (引用自 Henna Davar) > [21:55](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1315s) 天知道你会得到什么 — 现场嘉宾 > [21:55](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1315s) 少即是多 — 现场嘉宾 > [24:38](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1478s) 上下文就是国王，或者是女王 — 现场嘉宾 > [30:37](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=1837s) 半夜被报警电话叫醒的人可不是你们产品经理，而是我们工程师！所以请给我们留出足够的时间把它打磨完善。 — 现场嘉宾 > [34:30](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2070s) Distributed tracing is your friend. You are not logging, you are tracing what your agents do. — George Miranda > [37:48](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2268s) I go to sleep Friday and like log out of work and when I come back on Monday, the approaches that I knew on Friday are completely different. — George Miranda > [37:48](https://www.youtube.com/watch?v=KDgvi9AjgT4&t=2268s) 只恋爱，不结婚 — Henna Davar ## Mentioned Nitay Joffe, Henna Davar, George Miranda, Vira Tkachenko, Bradley Criqui, Team8, American Airlines, InsightFinder, MacPaw, CBRE Investment Management, Claude, CleanMyMac, Setapp, Jira, Slack, REI, MCP (Model Context Protocol), GitHub, Confluence, Strands, MongoDB, Andrej Karpathy, OpenTelemetry, O'Reilly ## Video description Nitay Joffe — Operating Partner, Team8 VC Henna Davar — AI Engineering Leader, American Airlines Vira Tkachenko — Chief Technology & Innovation Officer (CTIO), MacPaw George Miranda — VP of Marketing, InsightFinder AI Bradley Criqui — Global Head of Engineering, CBRE Investment Management Agentic AI won’t scale in the enterprise without a robust orchestration layer, the connective tissue that manages tasks, memory, tool usage, and multi-agent coordination. This panel will take a closer look at how orchestration software is evolving, from lightweight frameworks to enterprise-grade platforms that resemble an operating system for agents. Panelists will discuss how to evaluate orchestration tools, what features matter most (state management, guardrails, tool adapters, observability), and how to integrate these systems with existing enterprise tech stacks. The conversation will also cover emerging standards, vendor approaches, and trade-offs between building in-house vs. adopting commercial platforms. Attendees will walk away with a grounded understanding of what an effective orchestration layer should do and how to choose the right path for their organization. [Source](https://www.youtube.com/watch?v=KDgvi9AjgT4)

[0:04] Claude (引用自 Nitay Joffe): 我注意到你的孩子比所有最好玩的过山车限高刚好矮了 1 英寸，这里推荐几款鞋垫链接供你选购。 [11:10] Henna Davar: 只要架构设计得当，使用小型模型也能构建出功能完备且复杂的全栈应用；反之，即便拥有最顶级的 SOTA 模型，若缺乏优秀的架构，最终做出来的可能连平庸的 Web 应用都不如。 [16:34] Vira Tkachenko: 数据只有被清洗干净，才能真正成为智能体可用的核心资产。 [20:01] Andrej Karpathy (引用自 Henna Davar): I've had a killer day and let's kill the switches and relax [21:55] 现场嘉宾: 天知道你会得到什么 [21:55] 现场嘉宾: 少即是多 [24:38] 现场嘉宾: 上下文就是国王，或者是女王 [30:37] 现场嘉宾: 半夜被报警电话叫醒的人可不是你们产品经理，而是我们工程师！所以请给我们留出足够的时间把它打磨完善。 [34:30] George Miranda: Distributed tracing is your friend. You are not logging, you are tracing what your agents do. [37:48] George Miranda: I go to sleep Friday and like log out of work and when I come back on Monday, the approaches that I knew on Friday are completely different. [37:48] Henna Davar: 只恋爱，不结婚

[4:08] Henna Davar: legacy app within 6 months [21:05] Henna Davar: If Andrej was to walk into his home one day and say, 'I've had a killer day, let's kill the switches and relax'—does the security agent now actually kill the switches and turn on the alarms? [21:05] Henna Davar: less is more [28:28] Vira Tkachenko: My last argument is: it won't be you who will wake up at night—it will be engineers. So please give us time; we need to make it better. [37:48] shooting yourself in the foot incredibly fast [37:48] Henna Davar: dating [37:48] Henna Davar: buy, not marry [37:48] Nitay Joffe: only 57 more

[0:04] Thank you all for joining us today. I'm really  excited. We have an esteemed panel here to cover [0:07] this very exciting topic. Uh so I thought  we'd kick it off with start with give us [0:12] a few sentences each of you about yourselves  and tell us your favorite personal use of AI. [0:18] Uh sure I can go first. Um I'm Hannah. I uh lead  the AI engineering and AI governance at American [0:25] Airlines. Um very excited to be here today with  an esteemed panel. Um the fun news that I use AI [0:31] for maybe on a daily basis started last December  actually. Um there so all of my family lives in [0:37] Australia most of them and I kind of miss out on  the events a lot of celebrations that happen there [0:42] and our image models have really gone better  and become really nice. Uh now it uh creating [0:48] new images or inserting people in different  images. So when whenever I miss out on any event [0:53] uh in my family it's very easy for me to insert  my picture in there. So, and I typically try to [0:58] make myself look the best. So, that is the way I  use uh AI for it's about me. Happy to be here. I [1:04] love that. Um, hi, I'm George Miranda, uh, VP of  marketing at a company called Insightfinder. Uh, [1:11] also co-author of the O'Reilly book observability  engineering. Um, my probably funnest use of AI [1:19] personally is I have a text messaging agent uh  that I use for um like spammy and fraudulent [1:26] messages. Like I used to get a lot of those, hey,  I'm your CEO. I'm at this event like send me gift [1:31] cards type messages. And I would reply to those  people and then realize I'm wasting a bunch of [1:36] time too. Like I could just get an agent to do  this. And so it has a bunch of directives to be [1:41] really incompetent and like bad at technology. And  just seeing these conversations I find endlessly [1:46] amusing. Um, but I don't get a lot of those text  messages anymore. So I think the agent probably [1:51] had something to do with that. Hi everyone, I'm  Vira. I came here from Ukraine so far away. Quite [1:58] quite a trip. I'm a CTO and co-ounder of MCPO. If  you use Mac, you might know our products as clean [2:04] setup and mostly B2C software. The funniest use of  AI for me it's kind of observability of the growth [2:11] of my cat Shusha. I adopted a small cat and I was  curious how it grow and to track all the details [2:20] like length of the tail and other parameters. So  I created a project in uh cloud and just tracked [2:29] all the data and asking create me some graphs and  analyze if it's like how it's compared to normal [2:36] and try to match the breed because it's adopted  cat so unknown breed so it was maybe the finest [2:42] use of of [laughter] AI personal and keep track  of growth. Thank you. Hi uh Brad Kriki um head of [2:51] engineering at CBRE investment management um for  me it's all about building the applications at [2:58] uh greater speed so building out uh agentic  engineering pipelines and orchestration systems [3:05] and uh just the speed at which you can develop new  features and applications for me that's best capes [3:13] amazing thank y'all right uh and I'm Nai Jaffy  I'll be your moderator I'm a partner ner at a firm [3:19] called team8 for a global fund early stage infrai  type of stuff. Uh I'll give you my personal story [3:24] and then we'll dive in which is uh the other day  I was planning a trip to do to take my kids to an [3:30] amusement park and so naturally I put it in cloud  code. I said okay build me the whole itinerary the [3:34] whole plan. It did what you would expect. It went  it found all the rides all the restrictions. It [3:39] asked me my kids heights. The one part I did not  expect is at one point it told me I noticed your [3:44] kid is one inch under all the best rides. Here's  links to a few shoe inserts that you can get. And [3:51] naturally, of course, I got it and it made the  day. It was amazing. Um, okay. So, the topic [3:56] for today is orchestrating AI. Uh, and the way I  want to have this conversation is let's make it as [4:02] casual and fun. We're five friends talking. Let's  not go, you know, one by one. Feel free chime in. [4:08] Uh, make it organic. So the first question I  want to ask you all is you know when we move [4:13] from single to multi- aent systems that need to  computer to to collaborate and and uh communicate [4:18] with each other the complexity naturally  skyrockets and so how do you think about managing [4:23] that coordination ensuring the multiple autonomous  systems are not stepping on each other and causing [4:28] a lot of you know havoc? Yeah I can speak to that.  Um you know I think of it from a you still want to [4:36] keep a traditional engineering uh sense in place.  So demarcation points, stage gates. Um the way [4:43] we're approaching it is having still you know  creating events right where one agent will hand [4:50] it off to another right to have that demarcation  point to have control layers um the right security [4:56] in place, the right telemetry in place and making  sure that um yeah that you have isolation and [5:04] control mechanisms so that you don't have runaway  tasks and processes. Um, yeah, that that's a real [5:10] important aspect of it. Yeah, maybe I can add uh  some uh on top of that what Brad just said. Uh [5:19] for us, I think um we've learned over time that  it is important for us to gauge exactly what the [5:24] use case is. So sometimes we think of a use case  being a multi- aent use case and turns out it is [5:29] a really a single agent or a zerosot prompt call  use case. uh we've learned it. Um I've I think in [5:35] my entire uh technology career of 20 years is the  first time I've called an application a legacy app [5:40] within 6 months. Uh because we did we were very  quick to jump into a multi-agent architecture [5:45] uh and quickly realized within 6 months that  it could easily be solved as a single agent [5:49] architecture and um we built it uh again within 2  weeks because of the pace and the nature at which [5:54] the applications can be built. Now uh we saving  about 80% of token cost at this point with the [5:59] single agent architecture. Having said that, I  think when you design a multi-archchitecture, [6:04] multi- aent architecture, some of the things that  we are realizing now is um the ability to move [6:10] the in-memory conversations outside into um an  externalized state or a durable workflow execution [6:16] engine. Uh what that helps with is anytime your uh  agentic workflow kind of fails uh and it needs to [6:21] recover you don't have to now uh it's not it's  recoverable because it is a persistent storage [6:26] that has all the information all the all the  steps that were taken by the agent. Uh so now [6:31] you're able to recover and um additionally it  also helps with again token cost is one of the [6:36] number one orchestration priorities for me.  Uh so it also helps with reducing the number [6:41] of tokens that it needs uh for a sub agent to go  back to an orchestrator agent and communicate the [6:47] conversation uh whatever has happened on to pass  the context back. So now it can just tap into this [6:52] externalized state and and get that information  if need be. Um that is one of the uh ways we have [6:57] tried to identify how to um uh work through our  multi- aent um systems and uh one of the other [7:03] things is of course tracing and observability  is very important and uh kind of having a [7:07] externalized state also helps with building that  tracing ability and to be able to showcase that [7:12] uh the agent decisioning the dynamic decisioning  and the recommendation that it's taking is [7:17] actually now set in stone you can see and trace  back to whatever decisions that were made uh by [7:22] the agent. So I think those are some of the things  and uh during the talk if I come across anything [7:26] else that comes to my mind I will share with  everyone I think uh for now I think one of the [7:30] best things that you've learned is externalizing  the state management to a a different store [7:35] which is durable execution. I want to build on the  tracing and observability uh part of that which is [7:41] um I feel like we're repeating or going down  the path of an already solved problem. So my [7:47] background is uh coming from the S sur side of  the world. Uh and in that you know we sort of saw [7:55] a similar evolution where a year ago at AI4 what  we were talking about was LLM based applications [8:01] right or like say like essentially like monolithic  applications and the same thing happened in [8:07] uh traditional deterministic applications. We  went from single monolith servers to suddenly [8:11] distributed systems that were running across  thousands of global nodes. Right? And so when [8:15] you think about a multi- aent architecture and  that complexity explosion, absolutely tracing [8:20] is 100% key. You need to understand the entire  endto-end transaction that occurred inside of [8:27] your workflow. Uh in traditional deterministic  systems, right, when we started using cloud, [8:32] the applications themselves were deterministic,  but the execution paths were very probabilistic [8:37] and you never really knew exactly what was  happening between any two given requests that [8:41] should look identical. They were being executed  in radically different ways. And that's where [8:45] the problems showed up. And so I think one of the  things that I don't hear a lot when it comes to [8:51] orchestration and multi-agent architectures is how  we trace things end to end to have that sort of [8:58] visibility in place. I know we're all like really  focused on just like the workflow and the state [9:02] management and some of the execution bits, but if  you can't explain what is happening and you can't [9:07] trace back all of those steps, you're never going  to get the kind of trust that you need from your [9:12] stakeholders to make those uh workflow initiatives  successful. I agree that um agents it's nothing [9:20] new that typical engineering system. So at some  it's the same system they need to communicate. So [9:26] uh for multi- aents we use typical APIs MCP  which is the same API so it's kind of the same [9:33] and but what's new is communication v context we  shared memory it's something new we've never seen [9:41] before used before so because agents what what's  different from like typical engineering system [9:45] it's has an intelligence in core so it's more it's  between a human and some like more deterministic [9:52] system so it can update that context it can use  Jira for ticketing ticketing. So it can uh for [9:59] synchronization it can it can use conversational  um people conversational um like me methods of [10:08] use to like Slack or or J. So it's like addition  to typical API communication but to say the truth [10:16] in our company mostly use a single agent for  most of the use cases. So for multi- aents like [10:23] real multi- aent systems are mostly used for by  software engineers because they're more advanced [10:29] and tasks they solve with AI is more complicated.  So I I see multi- aent system like real the real [10:37] value mostly in software engineering and and for  everyone else is single agent to dec to say yeah [10:45] one one uh scenario where you really want to  use multiple agents is to make sure that you're [10:51] weighing one agent's decision versus another one  model's decision versus another. um you one agent [10:58] can be completely convinced that it's done the  full job and then if you just weigh another you [11:04] know validated model against that a lot of times  it'll uncover um pieces that the first model [11:10] didn't see. So that's one thing to keep in mind  as you go through these loops and processes that [11:15] just having that additional check sum really goes  a long way to improve your quality. Yep. Um can I [11:22] add another point just came to my mind. So in um  multi- aent and building on what George just said [11:28] um we are very mindful of actually breaking down  our architecture into figuring out what is going [11:32] to be non-deterministic in this architecture  and what's going to be very deterministic. So [11:37] not everything is a use case for an agent. Uh we  heavily use agents for classification of intents [11:42] because that is the place where we actually uh  consume information from our users or end users [11:46] and customers and in order to classify it intently  we use the best state-of-the-art model there. [11:52] uh because once the intent is classified  correctly, we hand it over to all our agent [11:56] sub agents which are domain specialists and they  mostly work off of deterministic workflows. So [12:01] there's a term that we use within the company  and maybe it is an industrywide term. I'm not [12:05] sure but it's called a constrained agent and  that what that means is that it'll work with [12:09] the constraints and policies and guardrails  that we provide and typically that means it's [12:12] it's a deterministic workflow taps into the real  grounded data uh and brings back the results that [12:18] is working off of our semantic layer. So um  very good combination again my theme is going [12:22] to be how do we save cost make it cost effective  architectures token optimize because that is the [12:27] number one priority from our business today how  do we accelerate our um production u roll out of [12:33] aic systems but do it in a very cost-ffective  manner and one of the important things we've [12:37] learned and we've discovered is the architecture  is extremely important more important than the [12:42] model and and the fact that um somebody said  that you can actually have a full stack complex [12:48] application built out with a small model if you  design the architecture well is so true and you [12:53] can have the best state-of-the-art model and not a  great architecture and you can still not produce a [12:58] very average web app. So it's very important how  we work on the architecture and determine those [13:02] components as deterministic and non-deterministic  very early on in the architecture design. Yeah, [13:08] I want to talk about cost as well, but in uh to  your point, when it comes to figuring out where [13:14] you use deterministic or probabilistic approaches,  uh so at Insightfinder, we have uh an AI sur agent [13:20] called REI. And what we discovered is at least  in a like a reliability workflow. There are [13:25] parts of the workflow that need probabilistic  interpretation, but a lot of things like for [13:28] example remediation is a script typically that  would run, right? And so there's no need to, [13:34] you know, use an external model for that. And  so really a lot of our use case and not just [13:39] software engineering but like any reasonably  complex workflow is having a real eye for what [13:45] actually needs generative AI behind it and like  what's tried and true right and so just breaking [13:50] that down uh also removes the need for having like  that external validator or judge LLM on top which [13:56] you really need on the probabilistic side but  if you have something known and tried and true [14:01] just stick to it right and that's uh that will  also help drive down cost but we'll get there [14:05] Nice. Yeah. One of the things we do is we try and  have a standard REST API for every MCP server. So [14:13] we can take either route and if if you can take  the REST API route, it's you know you have much [14:19] you have higher degree of confidence that you're  going to get a specific result, right? So and it's [14:25] more cost effective. Um yeah, cool. So you guys  mentioned a lot of uh great topics there and in [14:33] particular I I really like the use cases you  mentioned for how do you differentiate between [14:37] when you use single agent versus multi- aents we  talked about um kind of this multiple decision [14:42] points this this routing kind of layer this  uh kind of planning um classification uh thing [14:49] um I want to jump off one of the points you guys  mentioned a few of you mentioned around the shared [14:52] memory so when you do have those use cases that go  into the okay this truly needs multiple agents how [14:58] do you actually get that kind of collaboration  effect that that shared memory piece that a few [15:02] of you highlighted. How does that how does that  work? How does what does good look like there? [15:07] Um I can take uh so for me uh the one project um  or the genty system that we are currently building [15:13] is an orchestrator sub agent or orchestrator  worker agent kind of an architectural pattern [15:18] and uh we've realized that uh once uh an intent  is classified by the orchestrator and it hands [15:24] it over to the sub agents to work on those  tasks. uh very quickly the conversational uh [15:29] memory kind of fills up the context uh of the  orchestrator agent and um because uh we do not [15:36] want that to happen in in production of course  uh we started to uh look into externalizing the [15:41] state management options and to your uh question  uh that was the memory a persistent storage that [15:46] we wanted to depend on. So every time there is a  conversation that happens between the agents uh [15:51] it gets logged in that memory in that persistent  storage and um every time there is a tool call an [15:57] intermediate tool call uh we also log in uh that  stuff and anything that's related to token spend [16:02] logs or anything else that we need to it all  goes into that persistent storage and one of [16:06] the things that we've uh realized over time  now as we building this uh product with uh [16:11] the state management and memory um is that not  only can this data really help with saving on [16:17] tokens and context uh providing the entire context  but it also helps with uh identifying patterns [16:22] in which we can make our agents work better or  smarter because now we have the data and we can [16:27] gain some intelligence and insight into this data  and make our entire agentic system work better. [16:34] What else? Yeah, memory is a is a really a huge  topic. It's for us the next frontier and not [16:42] just for us for the whole industry to to solve  and for now it's all is really fragmented. So [16:50] each function of the company stores used to  store their data in their system. So it logs [16:57] inside the system. So for engineer it mostly in  GitHub in repository. So which is okay. So mostly [17:04] context is plain markdown file. It's for current  stage is enough and I guess the next stage is [17:11] uh we need cost efficiency so it's it need to be  gets better more smarter don't just read all the [17:19] files and get it as a context because it's will  be to not so cost effic but it's just the next [17:24] uh stage but for let's say HR the all data logged  in like for us in hop in confluence and and you [17:33] how is it storage memory layer looks like and  all agents needs to be connected uh to to those [17:39] systems and like some data in Slack a lot of data  in Slack by the way in some messengers. So current [17:46] stage is so fragmented and uh we are working on  to solve it and have this more centralized memory [17:53] which is still for me is is a question should it  be like centralized for all the function because [17:59] access it should be like more restricted the same  problem was already solved in previous system but [18:05] now it needs to be resolved again this new idea  to make exit con context available to all the [18:13] agents that needs to have data. So we have this  experiment in trying to get all the data from that [18:20] fragmented system to one huge storage and like  to to try uh make it cost efficient and really [18:30] get the right information. But it's not an easy  task because what what we see is a problem of [18:34] stale data and controversial data because people  are people and we ended up with not the great [18:42] greatest data um in our company for 18 years.  So it's our current challenge to fix this stale [18:52] data pro problem to clean everything up and like  agents kind of push us to do it to to use data [19:01] as a real asset of your company and real asset  for your agent. So it's it's current stage what [19:09] we have now in company. You know what I'll I'll  build on that and add that the other driver for [19:14] multi-agent workflows is context right and even  uh you know so earlier I was saying right like [19:21] uh probabilistic or uh deterministic functions  within your workflow even for those probabilistic [19:26] uh bits of workflow tasks that we need to  carry out what we found was our real limiting [19:31] factor was how much context we need to give this  agent in order for it to complete its tasks well [19:37] right and so for us a lot of tasks that should  live within a single process or a single agent [19:43] ended up having to get broken down even further  because of those same challenges with providing [19:49] memory and providing all the context in order  to like drive the process forward. And so yeah, [19:54] I just wanted to mention that that's the other  thing that really drove us down this path. Yeah. [19:58] Yeah. Actually to add on to Vera's point, uh the  the concept of memory is really very confusing. I [20:04] spoke with the the panel who presented um strands  uh framework yesterday had a different answer [20:10] about what state management is. I spoke with Nina  call out for Nina from MongoDB. She presented a [20:15] talk yesterday. I had a conversation with her  trying to understand it's like this is memory [20:19] ha but what you're asking is a different kind  of memory. So I think the the confusion around [20:23] memory is real. We all trying to figure out our  way. And one of my questions was can I use for [20:29] uh short-term memory memory procedural memory for  everything else just one memory and do we do we [20:34] have any issues with security then um and I don't  think I've got an answer yet from all the folks [20:39] that I've spoken with so uh spoken to so I think  the confusion is real there and we're all trying [20:44] to figure out our ways around it and coming to the  context point um I think uh context is definitely [20:50] they say context is king but I feel context is  queen um but that's a [laughter] conversation. [20:56] So one of the things one of the things I think  uh in context uh is very important for us when [21:01] we build our products is um the the notion of  semantic intent divergence. So uh I'll tell you [21:07] an example for somebody who's trying to book  a flight and uh comes up with a phrase saying [21:12] I want to book an earlier flight. Now what does  that intention of that customer mean to us? Uh is [21:17] it booking rebooking an existing flight? Is it uh  making a making a new booking at this point? Is it [21:23] I want to upgrade that's why I want to rebook.  So the definition of phrases within a business [21:28] domain and how they mean in the kind of order  and the hierarchy or the prioritization that they [21:33] take is very important for agents to uh make some  decisions. And so for example a couple of uh weeks [21:38] ago uh Andre Karpati in his interview said that he  uses AI agents to control his home devices. And uh [21:45] I would like to understand if Andre was to walk in  into his home one day and say uh I've had a killer [21:50] day and let's kill the switches and relax. Does  the security agent now actually kill the switches [21:55] and uh you know turn on the alarms and because  if the if the agent is not trained to understand [22:01] those phrases and the business domain concepts  and knowledge with the kind of ontology that [22:05] you need on top of that context layer um there  is a concept of semantic intent divergence that [22:10] can produce inconsistent results between agents  and contradictory outcomes. And that's exactly [22:14] the place where you need a small language model  tuned to that particular domain, right? Because [22:19] rebooking a flight is significantly different than  let's say you're a venue owner and you're booking [22:24] acts, right? Like two different ways to interpret  that. And if you use a frontier model for it, [22:29] who knows what you're going to get, right? And  so that is what we found like those bits where [22:34] business logic is absolutely essential and this  means a particular thing in your domain not only [22:39] mattered for us and our S sur agent but then when  customers take that into their environment they [22:44] also need to break out specific tasks. Exactly.  For those reasons because with with small models [22:50] sometimes small less is more. Yes. Yes. So less  context less knowledge is better and we invest a [22:57] lot in on device CI so in smaller models because  with mark privacy is number one priority and we [23:05] want we invest in the same way. So not use just  frontier model and it's for privacy and cost [23:10] efficiency but create on device model for iPhone  and MacBook and like do the same like best answer [23:17] but the private way and u like all the context and  powerful enough for for the task you want to do [23:24] but to achieve this for sure you need to be uh to  really understand what's your use cases to train [23:31] it just for that set of use cases say for us is a  tool calling just one task ask one job to be done [23:38] and you can use small model and benefiting on in  everything cost and privacy. So it's it's a great [23:46] way and and to that end I think one of the things  that I don't hear a lot in these conversations is [23:51] the value of real time uh I guess detection of  what is happening in production because we may [24:00] have a good idea of what like rebooking a ticket  means as we're having this conversation in a room [24:05] and as soon as we put it out into the world our  users are going to break that in ways that we [24:10] never anticipated. So being able to capture those  outputs in production is the best source of being [24:16] able to create fine-tuning data for those small  customized models that don't know a lot like [24:22] we can get very granular about what they need to  know to control it. And then to your earlier point [24:26] having uh like a judge process on top of that to  see like well did that make sense for the overall [24:31] task that we're trying to accomplish is uh is yeah  it's like real world data. Yeah. Great. Yeah. I [24:38] want to add one thing on the context. I think it's  I do think context is king or or queen. I think [24:44] it's the next the next uh it's the next level  right I mean you can we've achieved general level [24:50] intelligence but if you give it the right context  if you don't you can't it doesn't do a lot but [24:56] if you do the sky is the limit and I think as you  start to look at the transformation opportunities [25:03] in the enterprise and in processes as you start  to dive into the context you start to identify [25:11] more and more of those opportunities where there's  new ways of working, new new ways of thinking and [25:17] then getting into that context layer. What is the  context engine? So that's like one of the things [25:22] we're really working on now is creating a context  engine that goes deep into the business processes, [25:29] right? And then once you get there, then you  can start to reimag how things work entirely [25:34] in new ways. So I think context is the key to  doing that. I want to jump off this point you [25:40] guys are making because we've heard kind of this  theme a few times around the right kind of context [25:44] divergence different hallucinations small models  that are very fine-tuned and so as you're trying [25:49] to get to a point where you're orchestrating a  fully autonomous system how do you build in that [25:54] trust who's responsible for the guard rails how  do you get it to a point where you can actually [25:59] say I I trust this I can sleep at night yeah  so um at American Airlines and for any other [26:06] regulated industries I think Building trust with  stakeholders is extremely important. Uh for us [26:11] uh we kind of have a centralized agent operating  system uh that any of the enterprise teams can [26:17] make use of in order to build their agentic  systems. And what this operating system gives [26:21] them is a centralized place of uh tapping  into an agent framework which for us is [26:25] an open source framework. Um security uh  guardrails and uh security uh solution which [26:31] we use a vendor for. Of course observability  as well for tracing mechanisms. Um and we have [26:36] um kill switches are mandatory for any agentic  systems that we build. Uh we always have a [26:41] baseline rule-based engine running in production  just in case we need to have uh to fall back on [26:46] something that has always been working and working  well for us. Uh so we we make all these and and of [26:53] course we do have to introduce now with Agentic  system the cost and latency guardrails as well. [26:58] So as long as we have these operational controls  baked in uh with security, observability, tracing, [27:03] cost and latency and uh making sure that we're  tracing every decision and uh every recommendation [27:09] the agency is making making I think it it really  builds trust and uh one of the um one of the [27:15] other ways I've seen uh in critical applications  especially the ones that are customerf facing and [27:19] uh probably very important for the business uh  we've also tried to build trust using uh agentic [27:24] systems in shadow mode and what that does is it  It um builds the entire agentic system pipeline [27:30] uh from A to Z and including uh from like whatever  data we using data to tools and calling and use [27:38] making use of the of course the agent operating  system and we uh the agents are able to observe [27:44] they're able to decide they're able to recommend  but at the end we don't send out the output to [27:49] um the customers and we just log it in a database.  What that does is it lets the business users [27:54] kind of look at the data, look at the dashboards,  look at the decisioning process and once they are [27:58] comfortable with everything that the agents are  doing, that is when we actually start gradually [28:02] rolling it out to production. So that pattern  has really worked well for building trust with [28:07] the business stakeholders at American Airlines.  And I think um anything we can do to keep our [28:11] systems resilient and secure and compliant is is  what uh we like to go ahead with. I want to take [28:18] that a step further but I want to hear if there  are other experiences we want to share first along [28:22] those lines. Yeah. Um we keep a human in the loop  for most processes right still. Um and then also [28:30] like I mentioned earlier having demarcation points  stage gates where you know if you land if if you [28:37] land the work of an agent on an event then you can  evaluate that through more deterministic processes [28:44] as needed and and to do to mitigate the risk.  And so those are the two fundamental principles [28:51] and then also isolating controls right you don't  want to give your agent access to everything. uh [28:57] you really want to make sure just like you would  for any process or person that they're isolated to [29:02] what they need to know and only that. So the the  way I wanted to take that forward a bit is that [29:08] insidefinder what we do is we focus on real-time  evaluations, right? And so often we we're like [29:14] we think about eval production or sorry in uh  development and better like doing better prompt [29:19] engineering, right? But we use uh like LLM judges  uh to look at inputs and outputs in real time and [29:27] uh use those in conjunction with an LLM gateway  so that if we detect uh like a bad response, [29:32] right? or like even bad inputs that are I don't  know doing like malicious prompt engineering [29:36] we can just we detect that within what's being  submitted don't even route it to the underlying [29:41] LLM just kill it at the source right and then  similarly like if that happens to get through [29:46] and I don't know like PII gets leaked or whatever  uh the LLM is judge process will go ahead pick [29:52] that up also based on policies that we've set can  just stop that and it's better to give your user [29:59] like a generic error than to actually leak any of  that information and to go ahead and uh flag that [30:06] as an erroneous condition that somebody should  look at immediately, right? And trying to set up [30:10] that real-time feedback loop. And what gets me is  that in a lot of these conversations, what I see [30:16] as the feedback mechanism is like a thumbs up or a  thumbs down at the end of the process, right? And [30:21] like who looks at that? How often? And is that too  late to actually do anything about it? And I think [30:27] a real component that we need to be thinking about  is that real-time evaluation layer. It's not just [30:32] for development but you need it in production  as well. Yeah, it's key to the evolution this [30:37] system is overall the improving architecture.  Definitely. And and our approach is for security [30:44] it's kind of like it was before. So you have two  ways and you need to have both. It's prevention [30:49] and monitoring and quick response time and you can  solve almost everything. So for prevention we have [30:56] this pop policies, guardrails and like other stuff  like some checkers and stuff like that. But that [31:04] monitoring life monitoring and quick response time  become more and more important because besides the [31:09] rules yeah we allow like code that no one had  uh read on production and so you you can never [31:17] be be sure how it works. So if it's okay. So you  need to monitor and be really quick in response [31:24] how to roll it back, how to and because it's a  special skill and for your engineering team they [31:30] need to be really trained and skilled to to do it  and to then investigate what what was wrong how [31:37] how to fix it. So actually it's that skill become  more and more important. Um and we do it that way [31:46] about like sleeping well at night. I use this uh  metaphor for product managers because they really [31:55] want to like push to production their [laughter]  webcoded stuff and when we discuss about it my [32:02] last uh like uh arguments it won't be you who  will wake up at night it will be engineers s [32:11] so please give us time we need to make it better  so because it won't be you it's not that on it's [32:19] getting serious and usually it works and yeah we  understand yes so do your job [laughter] yeah so [32:26] works yeah well I think I would uh add in  engineering we've seen u businessdriven [32:32] development domain driven development testdriven  development now I think we are focusing on evals [32:38] driven development so eval are a non-negotiable  for any agent products eval so very important uh [32:43] as a part of the development process and of course  as for ongoing monitoring as extremely sorry. And [32:51] it seems like there's um a few things you guys  have highlighted where there's kind of you know [32:56] the old patterns that we end up forgetting that  that actually still are useful aroundability [33:01] around kind of best practices of building systems.  And so how how do we take these legacy systems we [33:07] all have that are clearly serving a good purpose  and kind of bolt on top of them these new agentic [33:12] frameworks? How do you deal with that kind of  um emptiness mismatch if you will of layering [33:18] that in? What's the biggest hurdles you  run into and how do you think about it? [33:27] Yeah, I think it's all about making sure that  you have, you know, purpose-built components, [33:32] reusable components. So um like this comes down  to one of the things that we are leaning more [33:41] towards like building portions of you know of the  orchestration layer internally rather than just [33:47] pushing everything into one vendor or another  to make sure that we have full control and we [33:54] can reuse our existing uh systems in place. So  like reusing existing API endpoints where we can [34:02] building EMCP servers on top of those and then  not having to do massive integration projects [34:08] for by porting everything into one vendor  or another. And so I think you know if you [34:15] boil it down to reusable components then you just  extend where you need to. So when you need an MCP [34:22] server you build that up and then integrate  that uh through your orchestration layer. [34:30] I'll take it from the reliability perspective.  And so when you were asking about things that are [34:34] tried and true and how we bolt that on top, uh,  you know, I'm, uh, I'm down in the expo floor and, [34:40] uh, so many of the conversations that I have  with folks in the booth are about how they log [34:44] what their agents are doing. And distributed  tracing is your friend. You are not logging, [34:50] you are tracing what your agents do. And from uh  an observability and a reliability perspective, [34:57] right, what you are trying to do is capture all of  the context that happened in a specific step and [35:02] interconnect that so that you have explainability  for the entire process end to end. And uh there [35:08] are additional mechanisms that you need in that  right which is uh especially when you're you're [35:13] dealing with a probabilistic system or an LLM  output uh you need an additional layer to evaluate [35:19] whether or not that output was valid and if it  wasn't why and include that as part of the context [35:25] and so I think really the approach is looking  at like what are the practices that are tried [35:30] and true like monitoring real-time evaluation in  production right and then fuse that with bits that [35:37] we already know work well, right? And so, uh,  distributed tracing for large complex systems [35:43] is tried and true. And all you need is an AI  observability layer to take those particular spans [35:49] for LLM based work and add the right bits of very  deep context to understand what's happening. So, I [35:56] think it's it's sort of like what we were talking  about before, right? There are places where tried [36:00] and true deterministic approaches make sense and  then there are places to um insert LLM. So I think [36:05] the same thing applies and the uh reliability  approach. Yeah. Like using open telemetry [36:11] absolutely will get you get you there and then  systems you know that use open telemetry as well. [36:18] Yeah. And yeah no I was just going to add that um  I think making use of interoperable protocols like [36:24] A2A um agents work really well if we expose our  data using MCP. So uh a lot of our legacy systems [36:31] uh they shouldn't really worry about how to invoke  that data using the legacy methods of calling into [36:37] that data. So we've tried to whenever we want to  use um um data as we use uh we expose the data [36:44] using MCP for our agents to communicate and get  access to that data and of course build security [36:48] on it as well. Um and we've seen that pattern  work really well. Um and uh the idea of having [36:54] a semantic layer before we dive into any agentic  AI production system is very important for us. So [36:59] whenever we start thinking about an architecture  of okay this is the production agentic system [37:04] that we're going to build the first thing we do  is do we have a semantic layer for it and that [37:08] means that do we have standard connectors that  we'll use to access that data that's required [37:12] with the security measures. U also making sure  that the agents are able to communicate using [37:16] the standard protocols. Um well MCP just went  stateless uh recently. So now hopefully we can [37:22] scale it better as well because that was one of  the limitations of using MCP. So um I think using [37:28] uh interoperable protocols, standard protocols,  making sure that the legacy data is exposed via [37:34] MCP has really helped us and making that semantic  layer with the ontology or the knowledge graph [37:39] that uh makes you understand how different  entities and relationships are connected within [37:43] a business domain. Uh that has really helped  us build a robust uh agentic systems. Yeah, [37:48] MCP AAA open telemetry, right? Uh what I will  point out is, you know, this year at AI4, again, [37:55] like I said earlier, we're not talking about any  of the things that we were talking about last [37:58] year. So any bespoke choice that you make that  leads you down the path of vendor lock in is just [38:05] really going to shoot you in the foot incredibly  fast, right? Especially in an era where I go to [38:10] sleep Friday and like log out of work and when I  come back on Monday, the approaches that I knew on [38:15] Friday are completely different. So yeah, anything  that you can do to have that and or that semantic [38:20] layer and define it up front absolutely. So I mean  this is a philosophy where we uh you buy not marry [38:26] kind of a thing where we we're just dating for  one year. Initially American Airlines would sign [38:30] contracts three to five 10 years sometimes there  used to be relationships with vendors now it's [38:35] maximum one year. So I don't know where this  is going to go but that is how we approaching [38:39] vendor login just to avoid the vendor login to  to be honest because you don't know what's going [38:43] to come around as a next best option. Um I want to  be mindful of time here. This this has been a gold [38:49] mine of architectural patterns, use cases, things  that we've discussed here. Um I want to be mindful [38:53] of time. Does this audience have any questions?  Otherwise, I have only 57 more. I'm nice.