United Airlines Runs 24/7 AI Agents on Delay Alerts Under Human Gate

Flying High with AI Agents in Operation at United: Kushal Pasari, United Airlines — Ai4

Ai4

Enterprise AI breaks down over fragmented operational policies rather than code flaws, forcing airlines to prioritize contextual grounding over labor replacement.

AI-assisted summary of Flying High with AI Agents in Operation at United: Kushal Pasari, United Airlines — original source. Check the original for full context.

Source published: 2026-08-19

Detailed notes

## Background Published on YouTube by Ai4 on August 19, 2026, this presentation features Kushal Pasari, Director of AI Engineering at United Airlines. The session pulls back the curtain on how United deploys governed AI agents operating 24/7 to communicate transparent, real-time operational updates to customers.

## TL;DR Kushal Pasari details how United Airlines deploys specialized multi-agent AI systems to transform complex, real-time flight disruptions into transparent, contextual explanations for passengers. Rather than relying on unconstrained generation, the airline integrates human-in-the-loop 'storytellers,' strict domain guardrails, simulation models, and safe fallback paths across customer touchpoints like mobile alerts and airport signboards. Pasari underscores that enterprise AI failures stem from fragmented operational context rather than basic coding limitations, emphasizing that systems must prioritize customer trust, predictable governance, and human-machine collaboration over pure execution speed.

## Atmosphere & style The talk carries a pragmatic, engineering-focused, and authoritative tone, deliberately stripping away speculative AI hype in favor of operational reality. Pasari delivers a thoughtful, structured presentation that combines technical rigor with genuine empathy for passenger stress during travel delays. Watching the session feels like a candid, behind-the-scenes engineering debrief, balancing technical architectural detail with a strong, safety-first philosophy centered on human judgment.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=4s) Generative AI carries an estimated $3 billion economic impact for travel and transportation, where airlines must navigate real-time operational volatility across weather, maintenance, and air traffic control. - [00:04](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=4s) Customer frustration during flight disruptions is driven primarily by emotional uncertainty and lack of information rather than the lost time alone. - [03:50](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=230s) Radical transparency transforms the passenger experience by answering three vital questions: what is happening, why it is happening, and what the immediate next steps are. - [03:50](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=230s) Grammatical fluency does not guarantee operational accuracy, and fine-tuning can cause multi-dimensional regression where optimizing for one scenario degrades messaging in another. - [06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) United deploys specialized agent squads with clear separation of responsibilities across analyst, architect, and security review roles rather than relying on monolithic prompts. - [06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) Human storytellers remain actively in the loop to review and approve agent-drafted messages using real-time operational context and risk signals. - [09:15](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=555s) World models and simulation environments enable AI agents to test and evaluate complex multi-leg disruption messaging prior to live operational review. - [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) Operational AI messaging must be strictly grounded in real-time data and equipped with automated fallback paths for ambiguous or high-risk edge cases. - [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) Agent-assisted contextual messaging extends beyond mobile app alerts to physical passenger touchpoints like airport gate signs and digital flight boards. - [14:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=848s) Enterprise AI failures are primarily caused by fragmented organizational policies, loose decision rights, and ambiguous interfaces rather than model synthesis limitations. - [15:47](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=947s) Strict metric gating blocks underperforming or unverified AI agents from entering live production until domain-specific benchmarks are satisfied. - [17:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=1028s) Enterprise AI strategy should focus on combining machine intelligence, organizational context, and human judgment rather than aiming to eliminate human workers.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=4s) AI Disruption in Airline Operations and the Communication Problem

Kushal Pasari, Director of AI Engineering at United Airlines, introduces how United is moving beyond theoretical AI discussions to implement autonomous, governed AI agents operating 24/7 in real-world airline operations. Pointing to a McKinsey study highlighting an estimated $3 billion impact of generative AI on the travel and transportation industry, Pasari explains that airlines operate in constant real-time flux. Systems must continually adapt to changing weather patterns, aircraft maintenance schedules, airport conditions, FAA regulatory policies, and millions of customer decisions made daily.

To illustrate how United addresses these operational challenges, Pasari frames the discussion around a deceptively simple yet emotionally charged scenario: explaining a flight delay. Disruptions—whether caused by weather, airport construction, or maintenance inspections—trigger massive frustration not just because of the delay itself, but because of the deep uncertainty customers experience. Whether traveling for a family reunion, a first business trip, or a long-awaited vacation, passengers facing disruptions often find themselves anxious and demanding answers at customer service desks.

```mermaid flowchart TD accTitle: Real-Time Operational Inputs Driving Disruption Communication subgraph Inputs["Operational Data Sources"] W["Weather Patterns & Radar"] M["Aircraft Maintenance & Inspections"] A["Airport Conditions & Construction"] R["FAA Regulations & Air Traffic Control"] end subgraph United["United AI Agent Infrastructure"] Engine["Agent Reasoning & Context Engine"] end subgraph Output["Passenger Experience"] Msg["Transparent, Customized Narrative"] end Inputs --> Engine Engine --> Output ```

- A McKinsey report projects nearly a $3 billion economic impact from AI technologies across the travel and transportation sector. - Flight disruptions stem from multidimensional operational constraints, including severe weather, airport construction, unscheduled maintenance, and FAA flow programs. - Disruption affects passengers emotionally through uncertainty rather than the time lost alone.

---

[03:50](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=230s) Transforming Operational Complexity into Transparent Narratives

United recognized an opportunity to transform complex operational data into customized, transparent narratives for impacted passengers. Customers do not expect an airline to control the weather or eliminate every mechanical delay, but they expect complete transparency regarding what the airline knows, what actions are underway, and what happens next in their journey. As Pasari highlights, "Transparency does not eliminate the disruption. It actually changes how the disruption is experienced by the customers."

To execute this at scale, United tested their messaging approach across multiple cohorts, including internal storytelling teams, customer advisory boards, and employee focus groups, while deploying statistical evaluation models behind the scenes. A crucial finding emerged: a grammatically flawless message can still be operationally incorrect, insensitive, or contextually inappropriate. Because AI in production exhibits multi-dimensional regression—where fine-tuning a model to improve weather explanations might inadvertently degrade aircraft maintenance updates—the team broke down message quality into strict, measurable evaluation criteria.

- Multi-lens evaluation processes incorporate feedback from customer advisory boards, frontline employees, operations experts, and statistical quality models. - Operational quality requires decomposing message attributes into clear metrics rather than relying on aggregate or average model performance. - Transparent updates focus on three customer needs: what is happening, why it is happening, and what the immediate next steps are.

---

[06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) Multi-Agent Architecture and Governed Workflows

Deploying AI to craft critical customer communications required moving far beyond simple AI-assisted coding or basic prompt engineering. United established deep agent squads featuring distinct, specialized roles such as analysts, architects, quality reviewers, and security reviewers. Rather than maximizing the raw count of agents, the architecture emphasizes explicit separation of responsibilities, task delegation, and persistent context management.

Human oversight remains integral to the execution loop rather than an external checkpoint. United’s human storytellers review and validate agent-generated messages before they are dispatched to customers. The system translates invisible model processes into concrete operational signals, surfacing elapsed flight times, operational risks, and key context so storytellers can intervene effectively.

```mermaid flowchart LR accTitle: Governed Multi-Agent Storytelling Pipeline subgraph Agents["Specialized Agent Squad"] direction TB An["Analyst Agent"] Ar["Architect Agent"] Sec["Security & Policy Reviewer"] An --> Ar --> Sec end subgraph Governance["Human-in-the-Loop Governance"] Signal["Operational Signal & Context Interface"] Human["Human Storyteller Review"] Signal --> Human end Agents --> Signal Human --> Comm["Final Customer Notification"] ```

- Specialized agent roles include task decomposition across analyst, architecture, and security review agents. - Enterprise agents combine core LLM reasoning with surrounding system context, memory persistence, tool calling, explicit boundaries, and policy guardrails. - Human storytellers remain directly in the loop, using operational signals (stakes, risk, elapsed time, cost) to guide final approval.

---

[09:15](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=555s) Simulation, World Models, and Context-Rich Flight Messaging

Looking to the future of operational AI, United is actively piloting world models and simulation environments. These platforms allow AI agents to simulate complex scenarios—such as multi-leg journeys and partner airline connections—evaluating potential messaging combinations in simulated environments before deploying them to live production review queues. Furthermore, United is exploring pre-trained and fine-tuned retrieval agents alongside reinforcement learning and feedback loops, enabling systems to self-correct and refine narratives autonomously over time.

Pasari contrasts legacy automated notifications with the rich context delivered by AI agents today. Historically, a customer received a bare-bones alert indicating a flight number, gate, and departure time. Today's agent-generated communications provide comprehensive context; for instance, explaining that severe weather along an en-route flight path is causing a delay, even if the skies appear completely clear at the departure airport in Newark or Chicago O'Hare. Every communication is evaluated across three core operational pillars: customer comprehension, operational trust, and system-wide delivery consistency at scale.

- Future workflows incorporate world models and simulations to test complex, multi-leg flight scenarios prior to human review. - Specialized fine-tuned agents leverage reinforcement learning and domain retrieval to enable autonomous self-correction and healing. - Success is measured across three primary dimensions: "Does the customer understand it? Could the operations trust it? And then could the systems deliver it consistently at scale?"

[12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) Grounding Personalization: Safe Fallback Paths and Airport Signboards

Speed without quality is simply rework, and autonomy without governance creates immense enterprise risk. In an operational environment like United Airlines—where physical and operational security is the top priority—AI agents must be integrated into the workforce under strict controls rather than left to generate text freely. Personalization at scale does not mean letting a model improvise without boundaries; every message delivered to a passenger must be grounded, verified against real-time operational context, and aligned with company policies. This messaging framework has expanded across passenger touchpoints, including airport signboards and gate information displays that travelers have seen rolling out over the past year.

To maintain system reliability under pressure, safe operational paths must be the easiest paths to execute. When operational risks spike or edge cases arise, the system relies on predefined fallback paths to ensure that communication remains accurate and safe. As United's CIO Jason Birnbaum highlighted, flight delays and cancellations inevitably frustrate travelers, but providing honest, contextual explanations softens the impact: customers are far less frustrated when they understand the specific reasons behind an operational change.

```mermaid flowchart TD accTitle: United AI Agent Operational Governance Workflow subgraph Inputs["Operational Data & Constraints"] A["Flight Disruption Event"] --> B["Policy & Operational Context Engine"] end subgraph AgentCore["Agent Processing & Safety"] B --> C["AI Agent Processing"] C --> D{"Guardrail & Context Evaluation"} D -- "High Risk / Ambiguous Context" --> E["Safe Fallback Path"] D -- "Within Defined Guardrails" --> F["Human Storyteller Review"] end subgraph Delivery["Customer Touchpoints"] E --> G["Multi-Channel Output"] F --> G G --> H["Airport Signage & Mobile App Alerts"] end ```

- **Operational Grounding:** Output is strictly anchored to live flight data, weather conditions, and regulatory policies rather than open-ended model generation. - **Airport Signage Integration:** Grounded AI-assisted messaging now directly powers public-facing airport displays and digital flight boards. - **Controlled Fallback Paths:** High-risk or ambiguous situations automatically trigger fail-safe communication templates to prevent hallucinations or misleading guidance.

---

[14:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=848s) The Bridge Between Code and Enterprise Context

While the initiative received notable media coverage and industry attention, the underlying driver was improving customer experience through disciplined engineering. AI agents do not operate in a vacuum; they function through a coordinated architecture combining structured data, rigid safety guardrails, intentional UX design, and internal human storytellers who review critical messages before they reach customers.

The biggest lesson from scaling these capabilities across United is that enterprise AI failures are rarely caused by a model's inability to write code or process text. Instead, failures occur when agents lack deep organizational context, clear operational standards, or unified policies. In large enterprises, policies and operational rules are heavily fragmented and federated across disparate departments. Bridging this gap requires far more than generic coding assistants; it demands comprehensive specifications, purpose-built tooling, enforceable guardrails, and agentic reinforcement learning feedback loops.

- **Human-in-the-Loop Review:** Dedicated operational communicators ("storytellers") continue to validate and review sensitive messaging before distribution. - **The Enterprise Context Gap:** LLMs struggle primarily with fragmented policies, ambiguous interfaces, and incomplete upstream specifications rather than baseline code generation. - **System Architecture Essentials:** Scalability relies on concrete specifications, agentic reinforcement learning loops, and rigid API interfaces rather than pure prompt engineering.

---

[15:47](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=947s) Measuring Success Across Trust, Confidence, and Leverage

When failures occur, post-mortems consistently trace the issue back to ambiguous system interfaces, incomplete context feeds, or unclear decision rights granted to the agent. Because of these dynamics, deploying operational agents requires evaluating the entire end-to-end workflow rather than isolated model benchmarks.

``` +------------------------+-------------------------------------------------------------+ | Success Dimension | Core Focus & Operational Requirement | +------------------------+-------------------------------------------------------------+ | Customer Trust | Delivering accurate, transparent, and grounded explanations | | Operations Confidence | Proving predictability and safety to frontline teams | | Engineering Leverage | Scaling developer and operational productivity safely | +------------------------+-------------------------------------------------------------+ ```

To prevent unearned confidence, United enforces strict identity governance and metric gates for all agents. If an agent does not meet predefined, approved evaluation metrics across its target domain, it remains strictly gated in a pilot environment and cannot graduate to live production. This discipline prevents inflated performance claims and guarantees that agents demonstrate measurable precision before taking operational actions.

- **Root Causes of Agent Errors:** Ambiguous API contracts, missing context, and loosely defined decision rights. - **Production Gating Rules:** Unapproved or under-evaluated agent metrics block deployment; production access is strictly restricted to proven, governed workflows. - **Three-Dimensional Evaluation:** Performance is measured against customer trust, internal operational confidence, and engineering efficiency.

---

[17:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=1028s) Trust Over Speed: Leading with Context and Human Judgment

Every single flight in United's network carries a human story, and every AI agent deployed in the operation is developing a story of its own. As engineering leaders, the primary mandate is not simply to build agents that execute tasks quickly, but to ensure that every action they take is thoroughly trustworthy. Gaining the trust of frontline operational teams and end customers cannot happen overnight; it requires sustained validation, transparent evaluation metrics, and steady operational performance over time.

Organizations should avoid framing AI initiatives around the goal of eliminating human labor. Instead, engineering and business teams must evaluate where machine intelligence, deep enterprise context, and refined human judgment can combine to produce better operational outcomes.

- **Earning Operational Adoption:** Internal trust from frontline station managers and flight operations requires historical reliability and transparent metrics rather than immediate full automation. - **Core Guiding Principle:** "Please do not start by saying like where you can remove humans in the process. Do start by asking where the intelligence, where the context and the judgment can actually replace a better outcome."

## Notable quotes > [00:04](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=4s) Ever wondered how one of the biggest airlines enables messages to our customers? We will raise the curtain and take you through our journey in enabling Agents running 24x7 to make it happen. — Kushal Pasari

> [03:50](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=230s) Transparency does not eliminate the disruption. It actually changes how the disruption is experienced by the customers. — Kushal Pasari

> [09:15](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=555s) Does the customer understand it? Could the operations trust it? And then could the systems deliver it consistently at scale? — Kushal Pasari

> [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) Speed without quality is simply rework, and autonomy without governance creates immense enterprise risk. — Kushal Pasari

> [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) Personalization at scale does not mean letting a model improvise without boundaries; every message delivered to a passenger must be grounded, verified against real-time operational context, and aligned with company policies. — Kushal Pasari

> [14:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=848s) The biggest lesson from scaling these capabilities across United is that enterprise AI failures are rarely caused by a model's inability to write code or process text. Instead, failures occur when agents lack deep organizational context, clear operational standards, or unified policies. — Kushal Pasari

> [15:47](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=947s) When failures occur, post-mortems consistently trace the issue back to ambiguous system interfaces, incomplete context feeds, or unclear decision rights granted to the agent. — Kushal Pasari

> [17:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=1028s) Every single flight in United's network carries a human story, and every AI agent deployed in the operation is developing a story of its own. — Kushal Pasari

> [17:08](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=1028s) Please do not start by saying like where you can remove humans in the process. Do start by asking where the intelligence, where the context and the judgment can actually replace a better outcome. — Kushal Pasari

## Mentioned Kushal Pasari, United Airlines, Ai4, McKinsey & Company, Federal Aviation Administration (FAA), Newark Liberty International Airport, Chicago O'Hare International Airport, Jason Birnbaum

## Video description Kushal Pasari - Director - AI Engineering, United Airlines

Ever wondered how one of the biggest airlines enables messages to our customers? We will raise the curtain and take you through our journey in enabling Agents running 24x7 to make it happen.

[Source](https://www.youtube.com/watch?v=DzzfIsl-Ids)

中文简报

美联航在人工把关下全天候运行AI智能体发布延误预警企业级AI的失效多源于碎片化的运营规则而非代码缺陷，迫使航空公司优先考虑上下文语境对齐而非单纯替代人力。 Kushal Pasari详细介绍了美联航如何部署专用的多智能体AI系统，将复杂实时的航班延误情况转化为面向旅客的透明且具备上下文语境的解释。该航空公司并没有依赖不受约束的内容生成，而是在移动端推送提醒、机场航显屏等客户触点中，融合了“人机协同叙述员”、严格的业务领域护栏、仿真模型以及安全回退路径。Pasari强调，企业级AI的失败根源在于碎片化的运营语境，而非基础代码层面的局限；系统必须优先保障客户信任、可预测的治理以及人机协作，而非一味追求纯粹的执行速度。生成式AI预计将为旅游和交通运输业带来约30亿美元的经济效益，航空公司必须在天气、维修和空中交通管制等各环节的实时运营波动中有效应对。 (0:04) 航班延误或取消期间，旅客的沮丧情绪主要是由心理上的不确定性和信息缺失造成的，而非仅仅是因为损失了时间。 (0:04) 极致的透明度通过回答三个关键问题重塑旅客体验：正在发生什么、为何发生，以及接下来的应对步骤是什么。 (3:50) 语法通顺并不代表运营层面的准确性，模型微调可能会引发多维退化，即针对某一场景优化却导致另一场景下的信息质量下降。 (3:50) 美联航部署了分工明确的专业智能体小组，分别承担分析师、架构师和安全审查等角色，而非依赖单一庞杂的提示词。 (6:37)

Reading summary

United Airlines has operationalized autonomous, governed AI agents to transform multidimensional flight disruption data into transparent, customer-facing narratives. Speaking for the airline, AI Engineering Director Kushal Pasari argues that enterprise AI success depends not on raw automation or coding benchmarks, but on grounding models in operational context, human-in-the-loop oversight, and strict metric gating. Contextual Transparency in Managing Flight Disruptions (0:04) Airlines navigate continuous operational variables, including severe weather, unscheduled aircraft maintenance, airport construction, and FAA flow programs. Pasari notes that customer frustration during delays stems primarily from uncertainty rather than just lost time. While legacy systems transmit minimal data points like flight numbers, gates, and departure times, United's AI agents synthesize broader operational context to explain underlying root causes—such as adverse weather along an en-route flight path when skies at departure hubs like Newark or Chicago O'Hare are completely clear. Citing an estimated $3 billion economic impact of generative AI on travel and transportation from a McKinsey study, Pasari emphasizes that transparency cannot eliminate disruptions, but it alters how passengers experience them. Evaluations across customer advisory boards, frontline employees, and statistical models revealed that grammatically accurate text can still fail if it lacks operational correctness or sensitivity, requiring messages to address three specific passenger questions: what is occurring, why it is happening, and what the immediate next steps are. Multi-Agent Architectures and Human Governance (6:37) United structured its deployment around specialized agent squads rather than maximizing agent counts or relying on simple prompt engineering. The architecture separates functional responsibilities across dedicated analyst, architect, and security reviewer agents, backed by persistent memory, tool calling, and defined policy guardrails. In production, these agents output not only proposed messaging but also concrete operational signals, such as elapsed flight time and risk levels, directly to human communicators. Human storytellers remain inside the execution loop to review and validate generated notifications before customer dispatch. To safeguard live operations across passenger touchpoints—including mobile alerts and public-facing airport gate signboards—the platform enforces predefined safe fallback paths. If operational risks spike, context is ambiguous, or models encounter edge cases, the system routes communication to approved safe templates to prevent hallucinations or erroneous customer guidance. Simulation Environments and Production Metric Gates (9:15) To evaluate complex operational dynamics without risking live disruptions, United pilots world models and simulation platforms. These environments test multi-agent messaging across multi-leg journeys and partner airline connections before drafts enter production review queues. The architecture also integrates domain retrieval and reinforcement learning feedback loops to help agents self-correct over time. Deployments are held to strict metric gating based on three core dimensions: customer comprehension, operational trust, and delivery consistency at scale. Post-mortems indicate that agent failures rarely stem from poor baseline code or text generation; instead, they trace back to fragmented enterprise policies, ambiguous API contracts, and missing upstream context. Under United's identity governance rules, agents that fail predefined domain performance metrics remain restricted to pilot environments and are blocked from production access. Prioritizing Trust and Judgment Over Automation (17:08) Pasari cautions against framing enterprise AI as an exercise in labor elimination. Frontline adoption by station managers and flight operations requires earning institutional trust over time through predictable metrics and sustained validation rather than rapid, ungoverned speed. United's guiding implementation principle advises engineering teams to identify where machine intelligence, enterprise context, and human judgment intersect to improve operational outcomes. System failures in complex environments are prevented by clear specifications and unified policies rather than unconstrained model autonomy. United's operational deployment demonstrates that autonomous messaging agents can reduce passenger anxiety and handle complex airport communications when bounded by strict fallback paths and human review. However, sustaining these workflows requires overcoming fragmented organizational policies and unresolved edge cases across multi-leg flights, leaving the eventual balance between human validation and full agentic autonomy an ongoing engineering effort.

精炼阅读摘要

美国联合航空公司（United Airlines）AI工程总监Kushal Pasari指出，美联航正在将自主受控的AI智能体全面部署至全天候运营环境中，重点将高度复杂的航班延误数据重构为透明、可信的旅客沟通叙事。该实践表明，企业级AI落地的核心并非单纯追求代码生成或盲目取代人工，而是依托严密的多智能体分工、人在回路治理与安全兜底机制，在极高动态的运营网络中重建旅客信任。破解延误焦虑：从机械通知到透明叙事 (0:04) Pasari援引麦肯锡（McKinsey）的研究预测指出，生成式AI预计将为旅游与运输行业带来近30亿美元的经济影响。然而在航空公司的日常实际运行中，突发天气、机场跑道施工、非计划维护与联邦航空管理局（FAA）流量调控等动态制约因素交织，导致航班调整频繁发生。Pasari强调，航班延误给旅客带来的核心痛苦不仅在于损失的时间，更在于未知信息引发的情绪焦虑与不确定性。传统的自动化系统往往仅发送登机口或预计离港时间等冰冷数据，难以缓解旅客在柜台前的不安与疑惑。为了改善旅客体验，美联航利用智能体将后台复杂的运营数据加工为针对“正在发生什么、具体成因是什么、后续步骤如何”的定制化透明叙事。例如，即便纽瓦克或芝加哥奥黑尔机场的天空万里无云，智能体也能清晰向旅客说明延误由航路沿线的恶劣天气所致。美联航在跨部门测试中发现，语法无误的文本仍可能在业务逻辑或沟通语境上出现偏差，且AI在生产中存在多维回归风险——针对天气叙事的调优可能意外破坏机务维护叙事的准确性。因此，团队联合客户咨询委员会、一线员工与统计评估模型，将消息质量拆解为多项可量化指标，摒弃了依赖模型平均表现的评估方式。专业多智能体架构与人机协同 (6:37) 在系统架构层面，美联航超越了基础的代码辅助或简单提示词工程，建立了包含分析智能体（Analyst）、架构智能体（Architect）与安全合规审查智能体（Security & Policy Reviewer）的专业化智能体编组。该架构不追求智能体数量的无序堆叠，而是聚焦于严格的职责分离、任务委托、持久上下文管理与安全护栏设定，将大语言模型（LLM）的通用推理能力与企业系统上下文及工具调用紧密绑定。人类专家依然牢固地处于决策执行回路中。美联航内部的专业叙事团队（human storytellers）在最终通知发出前进行逐条审核与把关。系统将不可见的模型运算转化为具体的运营信号，向审核人员展示已飞行时间、潜在风险、综合成本等关键上下文，辅助人类完成精准校验。Pasari指出，企业应用AI受挫的主因往往并非模型自身生成代码或文本的能力不足，而是企业内部政策割裂、上游规范模糊所造成的深层业务上下文缺失。确定性安全兜底与世界模型仿真 (12:14) 民航运营对物理安全与系统可靠性有着严苛要求，美联航始终秉持“缺乏治理的自主性将带来巨大企业风险”的工程原则。目前，经过AI辅助生成的动态叙事已不仅用于移动端推送，还直接接入了机场航显大屏与登机口信息牌。为防止大模型在边缘场景下产生幻觉，系统内置了高度可靠的安全兜底路径（fallback paths）：一旦遇到高风险态势或上下文模糊场景，系统将自动触发经验证的安全通信模板，杜绝误导性信息的产生。面向未来，美联航正开展世界模型（world models）与仿真环境的试点。在信息推送到人工审核队列前，智能体系统可在模拟环境中对多航段行程、跨航司合作联程等复杂航况进行组合演练与预先评估。与此同时，美联航也在探索微调检索智能体与强化学习反馈回路，使系统在严格规则约束下逐步具备自主纠错与自我修复的能力。生产指标门禁与三维价值评估 (15:47) Pasari指出，缺乏严格约束的速度本质上只会制造返工。根据美联航的故障复盘，智能体在执行中的偏差几乎均可归因于模糊的API契约、不完整的上下文供给或定义不清的决策权限。为此，美联航实行了严格的指标门禁（metric gates）与身份治理机制：任何未能通过特定业务域指标考核的智能体，都将被严格锁定在试验环境中，绝对禁止直接进入生产环境发布操作。对于智能体系统的成效评判，美联航建立了三维评估框架：其一是旅客是否充分理解并建立信任（Customer Trust），其二是一线运营团队是否获得预测性与安全信心（Operations Confidence），其三则是工程系统能否在规模化交付中稳定提升整体效率（Engineering Leverage）。Pasari强调，技术团队在推进AI时不应以“能在哪些流程中剔除人工”为出发点，而应深入探索如何将机器智能、深层企业上下文与人类经验判断有机结合，从而促成更稳健的运营结果。美联航的实践印证了在强监管与高动态的复杂业务体系中，生成式AI的应用重心正从概念验证转向高确定性的系统工程。通过专业化多智能体分工、人在回路审查与严格的安全兜底机制，企业能够在确保运营安全的前提下重塑用户信任。不过，如何进一步跨越跨部门策略割裂与接口规范模糊带来的上下文壁垒，仍是运营级AI规模化演进中需要持续攻克的关键课题。

Detailed notes — zh

## Background 该视频由频道 Ai4 于 2026 年 8 月 19 日发布在 YouTube 平台上。视频记录了美联航（United Airlines）AI 工程总监 Kushal Pasari 的主题演讲，揭秘美联航如何通过 7x24 小时全天候运行的自主 AI Agent 体系，为乘客实时生成精准透明的航班动态与延误说明。 ## TL;DR 演讲详细阐述了美联航如何利用多智能体协作集群（Agent Squads）破解航班延误叙事这一经典运营难题，将气象、维保与空管等多源复杂信号转化为兼具同理心与准确性的定制化通知。面对提示词优化引发的“多维回归”等工程挑战，美联航构建了涵盖分析、架构与合规审查的深度协作分工，并将人类专家深度嵌入主流程进行把关与安全回退。此外，团队正积极探索世界模型仿真与智能体强化学习，以支撑更长周期的复杂任务推演。美联航确立了以客户信任、运营信心与工程杠杆为核心的成功评估标准，强调企业级 AI 落地的关键在于厘清上下文与接口规范，而非单纯替代人类。 ## Atmosphere & style 演讲风格务实干练且兼具技术深度，讲者开场幽默风趣，迅速将听众带入航空运营的真实挑战中。整体叙事充满了对旅客出行痛点的真切共情与严谨克制的工程思维，既有对前沿 Agent 架构与世界模型的大胆展望，又有对人机在环治理和兜底机制的清醒反思。整个分享呈现出一种在极端复杂物理世界中依靠系统工程解决现实问题的从容感与说服力。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=4s) AI 预计将为全球交通与旅游业带来近 30 亿美元的商业价值，美联航正通过 7x24 小时运行的自主 AI Agent 应对气象、维保与监管等高复杂度动态运营场景。 - [01:39](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=99s) 航班延误带给旅客的核心痛点是“不确定性”而非单纯的时间损失；透明化虽然无法消除突发延误，但能够从根本上重塑旅客对延误的感知与体验。 - [04:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=277s) 语法完美的生成内容在民航运营中可能是完全错误的；AI 评估存在“多维回归”特征，优化单一提示词可能导致其他业务维度退化，必须依赖多维度量化评估体系。 - [06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) 美联航构建了分工明确的深度 Agent 协作集群（涵盖分析师、架构师、合规审查与安全审查角色），核心价值在于显式划分职责边界与精准任务委派。 - [06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) 人机协同采用“流程之中”（in the flow）模式，人类叙事专家作为放行关卡介入主流程，结合可视化风险与成本信号确保生成意图对齐业务目标。 - [08:45](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=525s) 迈向下一代企业级 Agent 需要引入世界模型（World Models）与数字沙箱仿真，推演长周期复杂场景，并结合强化学习（RLHF）赋予检索与决策 Agent 自我修复能力。 - [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) 大规模个性化消息绝非放任模型自由发挥，必须保证内容深度扎根（Grounded）于真实运营数据，并在高风险情境下具备明确的安全回退路径（Fallback paths）。 - [13:47](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=827s) 运营透明度极大缓解了旅客的焦虑情绪，美联航将 AI 生成的精准解释同步至短信与机场看板，通过有同理心且准确的叙事重建旅客信任。 - [15:10](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=910s) 企业级 AI 落地失败的主因并非模型能力不足，而是缺乏业务上下文、统一的技术规范以及清晰的决策权限，单纯辅助写代码无法跨越业务鸿沟。 - [16:27](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=987s) 美联航从客户信任（Customer Trust）、运营信心（Operations Confidence）与工程杠杆（Engineering Leverage）三大维度严格把控 AI 智能体项目的成败与投产门槛。 - [16:27](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=987s) 技术落地的核心不应是一上来就考虑从流程中剔除人类，而应思考如何在关键节点引入智能、上下文与人类判断力以达成最佳业务成效。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=4s) 航空公司运营的复杂性与 AI 带来的巨大商业变革在本次演讲中，美联航（United Airlines）AI、机器学习及平台工程负责人 Kushal Pasari 探讨了美联航如何通过 7x24 小时运行的自主 AI Agent 实现运营创新。尽管作为午餐前为数不多的演讲者之一，他幽默地表示会加快节奏，但内容直击核心：将技术前沿的 AI Agent 真正落地到真实世界的航空运营中，从乘客通知叙事到受控的自主交付体系。航空业正在经历 AI 技术的深度重塑。根据麦肯锡此前发布的研究报告，AI 预计将为全球交通与旅游行业带来近 30 亿美元（$3 Billion）的商业价值。航空运营系统每时每刻都在经历动态变化，系统需要实时权衡多维度制约因素：气象变化路径、飞机实时状态、机场跑道及地面施工、FAA 监管政策以及数以百万计的乘客出行决策。美联航正是利用 AI 技术，深入到这些复杂的运营节点中进行智能化重构。 - **行业潜在价值**：麦肯锡报告评估 AI 将为交通与旅游业创造约 30 亿美元的经济效益。 - **动态决策要素**：涵盖恶劣天气形态、飞机检修与维护、机场施工、FAA 规章政策及数百万乘客的实时衔接。 ### [01:39](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=99s) 航班延误中的核心痛点：化解乘客的“不确定性” Kushal 提出了一个看似简单却极具欺骗性的经典难题——如何向乘客解释航班延误。几乎所有旅客都曾经历过在服务台前因航班延误或取消而焦躁愤怒的时刻。这些航班背后可能是一场期盼已久的家庭团聚、一趟至关重要的首次商务出行，或是规划已久的度假旅行。当行程受阻时，旅客体验到的不仅仅是时间上的推迟，更是一种深层的“不确定性”。延误的原因极其繁杂，可能交织着航路天气、机场地面施工、突发的机械安全检查或例行维护。过去，美联航后台的“叙事团队”（Customer Storytelling Teams）需要穿梭于各类内部业务系统之间，拼凑信息来向旅客解释原因。美联航由此看到了巨大的变革契机：两年前团队开始探索，能否针对每位乘客所搭乘的具体航班，自动生成精准、定制且极度透明的延误说明。正如 Kushal 所强调的核心理念：“透明化并不能消除突发延误，但它彻底改变了乘客体验这种突发延误的方式。”（"Transparency does not eliminate the disruption. It actually changes how the disruption is experienced by the customers."） - **旅客核心痛点**：延误引发的“不确定性”远比时间损失更具破坏性。 - **业务核心目标**：将极度复杂的底层运营现状，在大规模并发下转化为清晰、得体、透明的叙事。 ### [04:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=277s) 质量解构与多维评估：从语法正确到运营准确为了把运营复杂性转化为高质量的乘客叙事，美联航采取了多管齐下的测试与评估策略。团队不仅在内部叙事团队和员工中进行测试，还引入了客户咨询委员会（Customer Advisory Boards），并构建了底层的统计评估模型。AI 生成文本的关键挑战在于：一条在语法上完美无瑕的消息，在实际民航运营中可能是完全错误的，甚至可能显得缺乏同理心或脱离实际情境。 ```mermaid flowchart TD accTitle: 美联航多Agent协同与人机在环消息生成流程 subgraph Signals["多源运营信号输入"] A["运营数据源 (气象雷达 / FAA管制 / 机械维保 / 机场施工)"] end subgraph AgentSquad["深度 Agent 协作集群 (Deep Agents)"] B["分析 Agent (Context Analyst)"] --> C["生成与编排 Agent (Architect / Narrative Engine)"] C --> D["多维合规审查 Agent (Safety & Policy Reviewer)"] end subgraph HumanGate["人机在环管控 (In-the-Flow)"] E["后台叙事专家终审 (Storyteller Review Gate)"] end subgraph Delivery["触达与交付"] F["定制化透明延误消息 (Mobile App / 短信通知)"] end A --> B D --> E E --> F ``` 在生产环境中落地 AI 时，模型评估呈现出明显的“多维回归”（Multi-dimensional Regression）特征。优化一个维度的提示词或参数可能会带来意外的副作用——例如，当气象延误消息的质量得到提升时，关于飞机机械维保的解释质量却可能同时出现滑坡。因此，美联航总结出的关键工程经验是：必须将消息质量拆解为可量化的多维度指标，而不能仅依赖单一的平均指标。 - **评估主体**：内部专业叙事人员、员工、客户咨询委员会以及自动化统计评估模型。 - **生产环境回归现象**：多维系统下的全局优化往往伴随局部退化，要求模型具备精细化的兜底机制与严格的验收标准。 ### [06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) 深度 Agent 协作架构与人机协同治理（Human-in-the-Loop）美联航构建的企业级 AI 体系超越了单纯的辅助代码生成（AI-assisted coding），演进为涵盖上下文记忆、工具调用、边界护栏和严格治理的深度 Agent 协作集群（Agent Squads）。系统将复杂任务明确拆解给具备专门职责的 Agent 角色，包括分析师（Analyst）、架构师（Architect）、内容审查员（Reviewer）以及安全合规审查员（Security Reviewer）。价值的核心不在于创建了多少个 Agent，而在于职责边界的显式划分以及任务在 Agent 之间的精准委派。值得注意的是，美联航将人类专家的介入设计为“流程之中”（in the flow）而非“流程之外”（outside the workflow）。人类叙事专家作为决策关卡，在消息正式发送给旅客前完成把关。交互界面将后台隐蔽的模型行为转化为清晰的运营信号，包括风险等级、航班耗时、干预成本及上下文信息，确保每一次生成都对齐最初的业务意图。 - **角色分工明确**：涵盖分析、生成架构、合规审查与安全验证等专业 Agent。 - **人机在环机制**：人类专家处于主流程的关键节点，基于可视化运营信号进行最终放行与修正。 ### [08:45](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=525s) 世界模型仿真与三大成功维度：迈向下一代企业级 Agent 真正的企业级 Agent 绝非仅靠单一的大语言模型（LLM）编写 Prompt，而是具备推理、规划、工具调用、长期状态记忆与治理能力的多步工作流系统。展望未来，美联航正在试点引入世界模型（World Models）与模拟仿真环境（Simulations），让 Agent 能够在连接真实世界前，在数字沙箱中测试海量推演组合，模拟联程航班或跨航司合作等长周期复杂场景（Long-horizon tasks）。同时，团队正在结合微调与强化学习反馈机制（RLHF），赋予检索端专属 Agent 自我修复与纠错的能力。从实际效果来看，过去乘客收到的往往只是一条冰冷的固定格式通知，如“您的航班 X 将于某时间在某登机口起飞”；而如今在 Agent 的支持下，系统能够提供丰富的背景上下文。例如，从芝加哥奥黑尔机场（O'Hare）或纽瓦克机场（Newark）出发的旅客虽然在当地看到晴空万里，但 Agent 能够洞察整条飞行航路上的气象颠簸与管制，并提前向乘客清晰透明地解释延误原因。美联航以此确立了衡量消息生成的“三大成功维度”： - **乘客维度**：乘客能否真正理解延误的具体缘由？（"Does the customer understand it?"） - **运营维度**：业务与一线运营团队能否充分信任生成的内容？（"Could the operations trust it?"） - **系统维度**：底层系统能否在大规模高并发下实现持续稳定的交付？（"Could the systems deliver it consistently at scale?"） ### [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) 规模化治理与扎根式个性化：从模型输出到安全回退在组织内部衡量 AI 的价值时，绝不能仅仅局限于模型本身的输出（例如调用 Claude 或 Codex 的生成结果），更关键的是要衡量整个工作流的运转效率。没有质量保证的速度只是在制造返工，会消耗团队海量的时间与精力。尤其是当智能体开始自主学习业务流程时，缺乏治理的自主性将演变为巨大的系统性风险。对于美联航（United Airlines）而言，安全性始终是第一要务，如今 AI 智能体也已被纳入整体员工体系（Workforce）中接受同等严格的安全审视。所谓的“大规模个性化消息传递”（Personalization at scale），并不意味着放任模型天马行空地自由发挥，而是必须保证所有消息都是“有据可依、深度扎根”（Grounded）的。例如在过去一年左右的时间里，美联航已经开始将这些由 AI 生成的消息同步投放到机场的电子标牌和信息看板上。这意味着每一条向旅客展示的信息，都经过了实际运营上下文的校验，严格贯彻了企业政策与质量标准；在面临极高运营风险时，系统还必须具备清晰的回退路径（Fallback paths）。当安全路径同时成为最易执行的路径时，整套智能体系统才真正具备了规模化落地的可行性。 ```mermaid flowchart TD accTitle: 智能体消息生成与安全防护流程图 accDescr: 展示从运营上下文输入到受治理的客户消息输出的端到端流程 subgraph ContextInput["输入与上下文接入"] A["航班运营上下文与事件数据"] B["企业政策与业务标准规范"] end subgraph AgentProcessing["智能体处理与安全护栏"] C["AI 智能体编排与生成"] D{"护栏评估与风险判定"} E["安全回退路径 (Fallback Path)"] end subgraph VerificationOutput["人机协同与终端发布"] F["内部叙事者/人工审核"] G["机场看板 & 旅客端精准消息推送"] end A --> C B --> C C --> D D -- "高风险 / 不合规" --> E D -- "通过校验" --> F E --> F F --> G ``` ### [13:47](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=827s) 提升客户体验与透明度：CIO 的洞察与人类叙事者的角色智能体赋能消息传递带来的最大改变，在于极大缓解了旅客在异常情况下的焦虑感。美联航首席信息官（CIO）杰森·伯恩鲍姆（Jason Birnbaum）曾精辟地总结道：“当航班延误或取消时，旅客固然会感到沮丧；但现在他们至少没那么沮丧了，因为他们终于清楚地知道了延误或取消的具体原因。”这一举措不仅获得了媒体的广泛关注与正面报道，更实现了最核心的目标——提供更优质的客户体验。这种卓越体验的背后，凝聚着美联航在数据整合、系统设计、安全护栏以及人类判断力维度的严格工程纪律。整个流程并非完全放手给算法，美联航内部的“专业叙事者”（Storytellers）在信息正式对外发布前依然会介入把关。这是一套隐于幕后的工程体系，通过人机协同的方式确保对外传递的每一句话都既具同理心又精确无误。 - **核心价值导向**：AI 带来的不仅是自动化，更是通过运营透明度重建旅客对航司的信任。 - **人机协同审核**：组织内部的专业人员依然扮演最后一道质量防线，确保信息发布万无一失。 ### [15:10](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=910s) 落地企业的关键教训：规范、上下文与智能体强化学习对于希望将 AI 引入自身业务的企业，美联航提炼出了几条极为深刻的实战经验。当前 AI 辅助工具在代码生成和文本输出上已经十分强大，但企业级落地之所以会失败，往往不是因为模型本身“不会写代码”或“无法生成内容”，而是因为智能体缺乏上下文，缺乏统一的标准与政策约束。在大型企业中，这些政策与规则往往高度碎片化、分散在不同的业务孤岛中，极难集中管控。单纯的编码辅助无法跨越这一鸿沟，企业真正需要的桥梁是清晰的技术规范、配套工具、安全护栏以及持续的反馈回路——尤其是智能体强化学习（Agentic Reinforcement Learning）。复盘过往的失败案例，问题几乎总是出在模糊的接口定义、不完整的规格说明以及未厘清的决策权限上。因此，让智能体理解运营之外的业务“故事”与上下文背景，是决定系统成败的关键。 ### [16:27](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=987s) 成功的三个维度与构建可信赖的 AI 智能体美联航从三个核心维度来衡量 AI 智能体项目的成败： 1. **客户信任（Customer Trust）**：旅客是否认可航司提供的信息并感到安心； 2. **运营信心（Operations Confidence）**：一线调度与运营团队是否敢于依赖智能体提供的支持； 3. **工程杠杆（Engineering Leverage）**：技术团队是否通过智能体获得了实质性的生产力放大。为了确保系统严谨性，所有智能体在获得组织内部指标的正式批准前，必须严格停留在试点（Pilot）阶段，绝不允许直接推向生产环境。工程师与技术领导者的核心使命，是让智能体的每一次行动都“值得信赖”（Trustworthy）。这种信任无法一蹴而就，需要通过详实的数据评估和长期的稳定运行逐步建立。在演讲的最后，讲者强调，本次分享始于“一次航班延误的处置故事”，而如今美联航的每一个航班、每一个正在运行的智能体都在书写属于自己的故事。他向所有技术领导者呼吁：“不要一上来就思考如何在流程中剔除人类，而应该去思考，在哪些环节引入智能、上下文与人类判断力，能够创造出更美好的业务结果。” ## Notable quotes > [01:39](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=99s) 透明化并不能消除突发延误，但它彻底改变了乘客体验这种突发延误的方式。 — Kushal Pasari > [04:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=277s) 一条在语法上完美无瑕的消息，在实际民航运营中可能是完全错误的，甚至可能显得缺乏同理心或脱离实际情境。 — Kushal Pasari > [06:37](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=397s) 价值的核心不在于创建了多少个 Agent，而在于职责边界的显式划分以及任务在 Agent 之间的精准委派。 — Kushal Pasari > [08:45](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=525s) 乘客能否真正理解延误的具体缘由？ — Kushal Pasari > [08:45](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=525s) 业务与一线运营团队能否充分信任生成的内容？ — Kushal Pasari > [08:45](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=525s) 底层系统能否在大规模高并发下实现持续稳定的交付？ — Kushal Pasari > [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) 没有质量保证的速度只是在制造返工，会消耗团队海量的时间与精力。 — Kushal Pasari > [12:14](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=734s) 当安全路径同时成为最易执行的路径时，整套智能体系统才真正具备了规模化落地的可行性。 — Kushal Pasari > [13:47](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=827s) 当航班延误或取消时，旅客固然会感到沮丧；但现在他们至少没那么沮丧了，因为他们终于清楚地知道了延误或取消的具体原因。 — Jason Birnbaum > [15:10](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=910s) 企业级落地之所以会失败，往往不是因为模型本身‘不会写代码’或‘无法生成内容’，而是因为智能体缺乏上下文，缺乏统一的标准与政策约束。 — Kushal Pasari > [16:27](https://www.youtube.com/watch?v=DzzfIsl-Ids&t=987s) 不要一上来就思考如何在流程中剔除人类，而应该去思考，在哪些环节引入智能、上下文与人类判断力，能够创造出更美好的业务结果。 — Kushal Pasari ## Mentioned Kushal Pasari, 杰森·伯恩鲍姆 (Jason Birnbaum), 美联航 (United Airlines), 麦肯锡 (McKinsey & Company), 美国联邦航空管理局 (FAA), 芝加哥奥黑尔国际机场 (O'Hare), 纽瓦克自由国际机场 (Newark), Claude, Codex, Ai4 ## Video description Kushal Pasari - Director - AI Engineering, United Airlines Ever wondered how one of the biggest airlines enables messages to our customers? We will raise the curtain and take you through our journey in enabling Agents running 24x7 to make it happen. [Source](https://www.youtube.com/watch?v=DzzfIsl-Ids)

Source quotes

[1:39] Kushal Pasari: 透明化并不能消除突发延误，但它彻底改变了乘客体验这种突发延误的方式。 [4:37] Kushal Pasari: 一条在语法上完美无瑕的消息，在实际民航运营中可能是完全错误的，甚至可能显得缺乏同理心或脱离实际情境。 [6:37] Kushal Pasari: 价值的核心不在于创建了多少个 Agent，而在于职责边界的显式划分以及任务在 Agent 之间的精准委派。 [8:45] Kushal Pasari: 乘客能否真正理解延误的具体缘由？ [8:45] Kushal Pasari: 业务与一线运营团队能否充分信任生成的内容？ [8:45] Kushal Pasari: 底层系统能否在大规模高并发下实现持续稳定的交付？ [12:14] Kushal Pasari: 没有质量保证的速度只是在制造返工，会消耗团队海量的时间与精力。 [12:14] Kushal Pasari: 当安全路径同时成为最易执行的路径时，整套智能体系统才真正具备了规模化落地的可行性。 [13:47] Jason Birnbaum: 当航班延误或取消时，旅客固然会感到沮丧；但现在他们至少没那么沮丧了，因为他们终于清楚地知道了延误或取消的具体原因。 [15:10] Kushal Pasari: 企业级落地之所以会失败，往往不是因为模型本身‘不会写代码’或‘无法生成内容’，而是因为智能体缺乏上下文，缺乏统一的标准与政策约束。 [16:27] Kushal Pasari: 不要一上来就思考如何在流程中剔除人类，而应该去思考，在哪些环节引入智能、上下文与人类判断力，能够创造出更美好的业务结果。

Source quotes

[0:04] Kushal Pasari: Ever wondered how one of the biggest airlines enables messages to our customers? We will raise the curtain and take you through our journey in enabling Agents running 24x7 to make it happen. [3:50] Kushal Pasari: Transparency does not eliminate the disruption. It actually changes how the disruption is experienced by the customers. [9:15] Kushal Pasari: Does the customer understand it? Could the operations trust it? And then could the systems deliver it consistently at scale? [12:14] Kushal Pasari: Speed without quality is simply rework, and autonomy without governance creates immense enterprise risk. [12:14] Kushal Pasari: Personalization at scale does not mean letting a model improvise without boundaries; every message delivered to a passenger must be grounded, verified against real-time operational context, and aligned with company policies. [14:08] Kushal Pasari: The biggest lesson from scaling these capabilities across United is that enterprise AI failures are rarely caused by a model's inability to write code or process text. Instead, failures occur when agents lack deep organizational context, clear operational standards, or unified policies. [15:47] Kushal Pasari: When failures occur, post-mortems consistently trace the issue back to ambiguous system interfaces, incomplete context feeds, or unclear decision rights granted to the agent. [17:08] Kushal Pasari: Every single flight in United's network carries a human story, and every AI agent deployed in the operation is developing a story of its own. [17:08] Kushal Pasari: Please do not start by saying like where you can remove humans in the process. Do start by asking where the intelligence, where the context and the judgment can actually replace a better outcome.

Original transcript

[0:04] I'm Kushal Pasari. I am uh leading the AI and machine learning uh and platforms engineering [0:10] at United. Uh I'm told that I'm the next one or maybe one more session before the launch. So I'll [0:18] try to wrap it up soon. U what I'm going to talk about is how do we at United uh do innovation with [0:26] AI agents? And right before this session, there was a technical talk uh that says like all the [0:33] all the cool stuff that we can do now with AI agents, we are actually now implementing that [0:38] in reality and the real world examples that you will see uh today. Uh from a customer story uh to [0:45] act governing uh governed uh autonomous delivery at United is what our AI agents are are doing [0:52] right now. And a as airlines are really really disrupted by AI technologies there is a continuous [1:02] change that happens in the real time systems and this was a McKenzie study that was done uh about [1:08] a year or so ago where uh it's supposed to have like almost three billion uh dollar impact on the [1:14] travel and transportation industry itself. And so as we look at the weather patterns, the aircrafts [1:20] uh the airports uh even the applications the policies that are governing uh FAA regulations uh [1:26] millions of customers decisions that are needed to be made uh we are really like doing fantastic in [1:32] every facets uh using AI technologies at United. So what we will cover I will start with one simple [1:39] deceptively pro deceptive problem explaining a flight delay right we all have been in that space [1:47] where either our flight is delayed canled whatever right u then I'll unpack how we actually use [1:55] uh intelligence uh required behind that experience and how we actually change and and better [2:02] communicate with our customers uh with with some of those uh messages and then finally I'll show [2:07] you what are those patterns uh that we learned across the journey uh when we started to implement [2:13] uh some of these agents uh in the operations and and as we evaluated the tools as we planned these [2:18] tools uh with human judgment uh that's where like we're changing uh but the way that we communicate [2:24] with our customers so as as I mentioned I'm sure like everybody here in the audience would have at [2:32] least had this moment in their lives where they are at a customer service agent's desk screaming [2:40] as to why their flights are delayed. Think about it, right? A family reunion, uh a first business [2:46] trip, or maybe a long planned vacation that you're actually taking. And when that journey is [2:52] disrupted, the customer does not just experience a delay, they experience uncertainty. that's the [2:59] most impactful uh uh for for your for your travel and and that experience at that uncertainty that [3:06] simple delay can involve weathers uh that could involve uh airport constructions that [3:13] could involve uh inspections maintenance that we do on our planes as well. And so all these [3:19] constraints do impact uh some of the delays and cancels that that we do uh across the across the [3:25] airlines I would say. Right. But at the same time, our customer storytelling teams are like basically [3:32] behind the scenes telling that story to the customers that like this is what's happening and [3:37] and and by the time they get to this they get go through a lot of like applications uh showcasing [3:45] uh to tell the better story uh to the customers. That's where we saw the opportunity. This was like [3:50] two years ago uh where we thought like what if we could actually send specific message to our [3:57] customers customized to the flight that they're taking, right? And and as transparently as we [4:04] can because customers do not expect us to actually manage uh or control the the weather or the delay, [4:12] but they want us to manage like what we know and what we can communicate transparently. [4:19] and then what happens next with their journey. That's the most important communicating part. So [4:25] transparency does not eliminate the disruption. It actually changes how the disruption is experienced [4:31] by the customers. And that's the critical part in this whole experience. And so our opportunity was [4:37] to convert these operational complexity into a clear appropriate narrative at scale. Right? And [4:45] so we started this we we looked at multipronged approaches in terms of looking at how we how we [4:52] uh tell the story. Uh we we tested that with our own storytelling team internally at United. Uh we [4:58] tested it with our customers customer advisory boards. Uh uh uh we also tested that with our [5:05] employees. And then uh there were statistical models that we had created behind the scenes to [5:10] evaluate these messages because a grammatically uh perfect message could also be like operationally [5:16] wrong. It could be insensitive or it could be unclear or inappropriate in terms of the situation [5:22] that's happened and the disruption that has happened with the customers. So we evaluated these [5:27] messages through multiple lenses. the customers, employees, our experts and then of course like our [5:32] daily reviews that that our operations teams do. So the lesson is uh important here because uh the [5:39] quality must be like decomposed into measurable outcomes and that was the most key element uh that [5:46] we found out uh across the journey. uh some of the average results were not enough like when we were [5:53] evaluating these messages uh we were looking at how the a change in weather could actually improve [6:00] uh the message but at the same time it could also impact the maintenance message that we are sending [6:06] out to the customers. So in production AI is definitely a regression in multi-dimensional [6:12] right a model can become better overall but becoming worse exactly at the same time could [6:17] also be a possibility right so our fallback was uh the product uh that that we created and you'll see [6:24] that uh in terms of uh the the messages that come out of this uh process were very clear [6:31] concise that we can uh send very quickly to our customers during the time of disruption and So [6:37] we also looking at uh AI was never a problem like I think AI coding AI assisted coding has always [6:45] existed uh for some time now I mean at least 3 years now the customers saw a better message [6:51] uh but at the same time the message that was being crafted by our coding agents uh were not really [6:59] just AI assisted uh engineering it was more than just agents co creating the code it was looking [7:06] at the whole system, the context, the memory, uh the the guardrails that it's supposed to do, [7:11] every decision was governed, uh validating the quality of the outcome as well as the same [7:17] challenge is now shaping how we write software uh behind the scenes as well. And so uh at the [7:25] uh as we looked at the squads of agents that were created uh deep agents I would say that that have [7:31] been created now planning uh file system context uh the task delegation that happens behind the [7:38] scenes the work is now decomposed uh into uh specialized roles like an analyst or or [7:46] uh an architect or a reviewer or even a security reviewer that's reviewing these messages before it [7:51] actually is sent out to the customers. The value is not how many number of agents that we create. [7:57] It's actually in terms of the explicit separation of uh the responsibilities in this process as well [8:04] as the outcomes that these agents are delegating to to each other and then the outcome that it [8:09] it creates. So the human gate is not outside the workflow in this process. it's in the flow where [8:15] the humans are taking decisions in our case in this case like the storytellers uh review the [8:20] messages uh before it goes out to the customers as well. So the interface turns uh invisible [8:25] model activity uh that happens behind the scenes uh into an operational signal like think of it [8:30] as the stake the risk uh even the elapse time of a flight uh cost uh and the required information [8:38] that it needs to uh intervent uh in the process. Uh the point is not that again AI agents writing [8:46] the code better. The point is that like every artifact becomes uh connected to the original [8:51] intent and that intent is explicit uh criteria for acceptance. Uh this is where the coding agents are [8:58] evolving into workflows. Uh an enterprise agent is not just simply like an LLM writing a prompt. [9:04] uh the model provides reasoning capability uh to to the context uh surrounding systems like [9:11] uh including the memory the tools the boundaries and then the accountability [9:15] that's the most critical part in the flow uh planning does support multi-step workflow [9:23] uh the memory does preserve the state in this process uh tools does connect the reasoning [9:29] behind the scenes uh and then evaluation does detect the errors is as it detects the errors [9:35] uh those governance rules does get applied uh policies and the procedures and standards do [9:41] apply for uh these agents and then the future uh of this flow will involve like the world models [9:48] and simulations that that's what we're piloting now uh where uh the agents can actually simulate [9:53] these actions in the real world using real world scenarios uh where agents can test the possible [9:59] combinations u and before affecting a live uh uh message or or uh making it available for our [10:07] storytellers to review. Uh one of the things that we are also now exploring is pre-training some of [10:14] the capabilities uh where we can fine-tune and and provide retrieval capabilities specifically for uh [10:20] reinforcement learning for example or feedback as these agents learn more info on how the messages [10:27] are being crafted. uh they will be actually able to self-correct, self-heal some of the outcomes [10:33] that that they're creating. So we are specializing uh some fine-tuned agents uh that will be uh [10:39] specializing specifically on the retrieval side of the house. And then these simulation behaviors for [10:43] the long horizon task like complex tasks if there are let's say connecting flights and so on as our [10:49] partner airlines that we work with uh there could be more complex scenarios that it could simulate [10:53] as well. And so in a nutshell like as we looked at the the technology the outcome of this was [11:02] really amazing right? Think of like the the before the AI agents were crafting the stories you would [11:09] just get a message saying that your flight X is departing at this time and this gate. Now with [11:15] the story that we have crafted it gives you a lot more context. It tells you if there are weather [11:22] related delays. You may not see that during the path because you're let's say if you're in in [11:27] O'Hare or uh Newark and you're taking that flight uh you may not see specifically like there are uh [11:35] weather patterns that may impact your path uh to uh Chicago to hair but uh the agent sees that and [11:42] it lets the customers know that there might be a delay uh because of x reason right so the success [11:49] does did have the three dimensions that we were evaluating uh these messages is does the customer [11:54] understand it? Uh could the operations trust it and then could the systems deliver it consistently [12:01] at scale and these agents are doing a fantastic job so far in in delivering these messages to our [12:07] customers. Um we are applying the same discipline now where as as more and more agent factories are [12:14] created within the organization and and measuring the workflow uh not just merely the output of the [12:20] model uh that we provide to cloud or codeex or so right and so that's really key because speed uh [12:27] without the quality is really a rework for us like it it just takes a lot of time on on enabling uh [12:34] these capabilities but autonomy u once once Once the agents are uh learning the process uh without [12:41] that governance, it will definitely be a bigger risk. Um that that uh and and as most of you may [12:48] know, security is one of our number one priority for United and even agents are are part of that [12:54] workforce now as well. And so as we look at uh the the messages personalization at scale does [13:02] not just mean by allowing a model to uh improve freely as well. it does mean those messages are [13:09] grounded. U and now we have started to like look at the sign billboards and sign boards [13:15] at the airports as well where you would see these messages have been popping up for about a year or [13:19] so. Uh it means these these messages have been grounded uh verified in the operational context [13:26] applying the policies uh quality uh as well as the fallback paths when the risks are really [13:32] really high. the systems does become scalable when the safe path is definitely easy path as well. [13:39] but within the guardrails within the uh controls that the agents are expected to to behave in. And [13:47] then you see like u it was it was really cool like our CIO Jason Bernbon had uh said like our [13:55] customers were frustrated uh when there when there is a delay or a cancellation but at least they're [14:01] less frustrated because now they know why there there is a delay or a cancel as well. And so uh [14:08] as we looked at uh the external validation I think the uh it was really gratifying to see like all [14:14] these uh mentions in in uh really good uh uh uh media attention. Uh but again the outcome was a [14:22] better customer experience. That's the important concept uh that I want you guys to take away with [14:27] uh created through like the discipline that we integrated with these AI agents uh in terms of [14:33] the data uh the design uh the guardrails and the human judgment because our our storytellers [14:40] within the organization are still reviewing uh these messages uh before it goes out the door as [14:45] well. So the invisible engineering behind this amazing uh work uh plus a lot of my colleagues [14:52] uh at United uh there is there is a real story that uh we implemented with these AI agents uh [14:59] at United but uh there are a couple of lessons that we have learned uh and and these are really [15:05] important for everybody to think about as they take these learnings back to their organizations. [15:10] AI can produce code that's really awesome. like we can we can feed data uh to these AI assisted [15:17] tools but enterprise like us fail uh when the agents lack the context when the agents lack the [15:26] standards and the policies and it's really hard to control these standards and policies because [15:29] they are very fragmented federated at multiple places as well and so the bridge uh is not just [15:36] coding assistance but the bridge will be the specs the tools uh the guardrails and the feedback loop [15:42] especially the reinforcement learning or agentic uh reinforcement learning uh if you will our [15:48] failures were rarely caused by the models actually and and it was uh unable to or it was not able to [15:55] generate the code behind the scenes. It was always the context uh the ambiguous interfaces that it it [16:02] had to deal with uh the incomplete specifications that we had to power uh power it with or decision [16:10] uh rights uh especially like there were that were unclear uh to the to the organ uh to the [16:15] uh agents as well and that is why the story does matter beyond the operations. I think that's where [16:20] the key messages uh that I would want you guys to take back. Uh and then the success was not just [16:27] uh as I said right uh there were three dimensions that we were measuring our success the customer [16:32] trust uh the operations confidence and then the engineering leverage uh that we want to uh use [16:39] uh some of these outcomes with. So your discipline is making sure that you're labeling your results. [16:45] How do you actually evaluate and govern uh the identity of the agents? Uh if the metric is not [16:51] approved uh like within the organization uh it should be actually in the pilot state. You [16:56] cannot take it to production. And so the goal is to get the credibility to these agents uh so that [17:02] they're not just inflated uh with the precision that it can actually create as an outcome. And so [17:08] I'll leave you with like one last thought, right? We began this presentation with just one flight. [17:15] and then one disruption. Uh every flight within the organization today has a story [17:22] and then every agent now is getting its story as well. Uh whether it's taking actions, [17:28] whether it's uh responsibility for the engineers to actually produce some of these actions. But our [17:34] responsibility as leaders as uh engineers at heart is to make actions trustworthy. [17:41] I think the AI agents are really great at executing stuff but we have to make them trust [17:47] uh and humans should be trusting this and that trust will come with time. It will that adoption [17:53] will come with time as well. So it's not going to be just instantly that the agents will be trusted [17:58] uh by the operations teams within our our environment as well. It took us uh a good fair [18:04] amount of time for us to enable and then get the trust with the numbers with the evaluations that [18:09] that we had conducted. So please do not start uh by saying like where you can remove humans [18:16] uh in the process. Do start by asking where the intelligence where the context and the judgment [18:23] can actually replace a better outcome. Right? With that thank you. [applause]