Wayfair Adds Deterministic Orchestration to Run Autonomous AI Systems

The Missing Layer: AI Orchestration Director With Wayfair — Ai4

Ai4

Enterprise AI systems fail in critical operations because probabilistic models require a deterministic orchestration layer to enforce policy gates, validate outputs, and manage failure recovery.

AI-assisted summary of The Missing Layer: AI Orchestration Director With Wayfair — original source. Check the original for full context.

Source published: 2026-08-18

Detailed notes

## Background Published on YouTube by the Ai4 channel in August 2026, this presentation features Nirmal Jingar, Senior Engineering Manager at Wayfair. The session explores why enterprise multi-agent deployments fail without a dedicated coordination layer, presenting architectural frameworks for governance, deterministic workflows, and production-grade execution.

## TL;DR Wayfair engineering leader Nirmal Jingar explains that bridging the gap between advisory chatbots and autonomous production systems requires a deterministic orchestration director to oversee probabilistic LLMs. Walking through enterprise supply chain scenarios, reverse logistics optimizations, and automated root cause analysis, he demonstrates how to enforce policy gates, right-size models, and validate decisions before execution. The talk highlights key principles—including strict contracts, LLM judges, human escalation, and deep observability—culminating in a phased 90-day roadmap for rolling out resilient agentic workflows.

## Atmosphere & style The talk has a pragmatic, engineering-centric keynote energy that cuts through AI hype with grounded operational realities. Jingar engages the audience by drawing on visceral production stakes—from a broken-down delivery truck jeopardizing a promised sofa delivery to protecting engineers from 2:00 a.m. on-call alerts. The delivery feels structured, authoritative, and instructional, steadily moving from fundamental architectural challenges to concrete enterprise execution blueprints.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=4s) Despite widespread enterprise adoption of advisory LLM tools, fully autonomous agentic AI remains virtually absent in tier-one mission-critical systems due to a lack of governance and coordination layers. - [02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) LLMs are inherently probabilistic, creating unacceptable operational and financial risks when deployed in core enterprise workflows that require 100% deterministic reliability. - [02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) An orchestration director functions as a deterministic backbone that bounds model behavior, validates outputs, and ensures evidence-backed verification before actions execute in production. - [04:36](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=276s) Multi-agent orchestration can automatically resolve complex operational failures, such as transit breakdown logistics, by balancing constraints and coordinating resources without manual dispatch triage. - [07:52](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=472s) Intelligent routing reduces latency and compute costs by delegating standard business logic to deterministic APIs and matching cognitive tasks to appropriately sized models, from SLMs to frontier LLMs. - [10:15](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=615s) Core orchestration mechanics, state management, and software design patterns remain universal across industries, requiring only adjustments to domain-specific compliance rules and policies. - [10:15](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=615s) Upfront deterministic policy gates should evaluate baseline business constraints—such as eligibility and return windows—before invoking any probabilistic or expensive model logic. - [12:37](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=757s) AI orchestration delivers high enterprise ROI by simultaneously optimizing multi-variable trade-offs, such as weighing reverse freight expenses against customer lifetime value to issue instant concessions. - [15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) An autonomous Root Cause Analysis (RCA) Director can ingest cross-system telemetry across logs, deployments, and Jira to assess blast radius and generate pull requests for engineering review. - [15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) Secondary background LLM judges validate diagnostic assertions and code fixes, paving the way for self-healing systems that handle routine production incidents and reduce on-call alert fatigue. - [19:24](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1164s) Enterprise-ready orchestration relies on codified tribal knowledge, strict input/output contract validation, full-lifecycle observability, and mandatory human-in-the-loop escalation gates. - [20:43](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1243s) A structured 90-day implementation strategy reduces adoption risk by dedicating 30 days to defining a single workflow, 30 days to hardening it, and the final 30 days to scaling across adjacent domains.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=4s) The Production Gap in Enterprise AI

Opening the session by polling the room, Nirmal Jingar discovers a 50/50 split between technical engineering attendees and non-technical leaders, prompting him to dive deep into the technical architecture of enterprise orchestration. Setting the stage with his background as a Senior Engineering Manager at Wayfair specializing in AI supply chain systems, a TEDx speaker, and a Forbes Technology Council member, Jingar points out a stark disconnect in the industry. While millions of users and enterprises have adopted tools like ChatGPT for reasoning, drafting, and recommendations, very few systems in production operate with true governance and orchestration.

When Jingar asks the audience if anyone is running fully autonomous, end-to-end agentic AI across tier-one mission-critical workloads—such as healthcare, banking, financial systems, or core supply chain infrastructure—not a single hand goes up. Current AI deployments remain largely advisory, leaving humans to take the final actions because production-grade safety mechanisms are missing. The agenda bridges this gap by establishing the role of the orchestration director and walking through two concrete enterprise architectures: reverse logistics returns and automated root cause analysis.

```mermaid flowchart TD accTitle: Enterprise AI Orchestration Director Architecture subgraph Ingestion ["1. Request Ingestion & Governance"] REQ["Incoming Enterprise Request"] --> OD["Orchestration Director"] OD --> POL["Policy & Eligibility Gate"] end

subgraph Routing ["2. Intelligent Routing & Execution"] POL --> DEC{"Reasoning Required?"} DEC -- "No (Deterministic)" --> API["Traditional APIs / Microservices"] DEC -- "Yes (Probabilistic)" --> MODEL_SEL{"Model Selector"} MODEL_SEL --> SLM["Small Language Model (SLM)"] MODEL_SEL --> LLM["Large Language Model (LLM)"] end

subgraph Guardrails ["3. Verification & Observability"] API --> VAL["Validation & Evidence Check"] SLM --> VAL LLM --> VAL VAL --> OBS["End-to-End Observability & Audit Log"] VAL --> ESC{"Confidence / Policy Breached?"} ESC -- "Yes" --> HITL["Human-in-the-Loop Escalation"] ESC -- "No" --> OUT["Executed Production Action"] end ```

[02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) Probabilistic Models vs. Deterministic Production Demands

Standard large language models excel at synthesizing information, generating summaries, and drafting communications, but they fall short when tasked with autonomous operational execution. The fundamental tension lies in model behavior: "LLM by default is probabilistic. We all know. It's not going to be 100% deterministic, and we cannot put a probabilistic system on production which doesn't give you the right information and doesn't make the right decision." In a tier-one supply chain or financial ledger, deploying an unguarded probabilistic model risks causing millions of dollars in financial losses within seconds.

To close this vulnerability, enterprise architectures require a dedicated "orchestration director." This layer acts as the deterministic backbone that oversees the probabilistic model. It guarantees that every decision is strictly validated, bounded by governance rules, and backed by verifiable evidence before any action touches production databases or physical infrastructure.

- **Probabilistic vs. Deterministic Risk**: LLMs output likelihood-based predictions, whereas enterprise operations demand 100% reliable, deterministic execution. - **Evidence-Backed Validation**: Decisions cannot rely solely on model intuition; every output must cross-reference verifiable enterprise data. - **Financial Stakes**: Unorchestrated model failures in core operational systems can instantly compound into multi-million-dollar outages or liabilities.

[04:36](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=276s) Supply Chain Scenario: Resolving In-Transit Exceptions

To illustrate the necessity of autonomous orchestration, Jingar walks through a high-friction last-mile delivery failure. Consider a scenario where a customer is promised a heavy sofa delivery by 3:00 PM. The customer adjusts their personal schedule, takes a half-day off work, and waits at home. Meanwhile, the delivery truck carrying 100 boxes breaks down mid-route:

> "The truck broke. Now, what do you do? I told customer, you are going to get your product at 3:00 p.m. It's a very large heavy sofa. Customer might have taken half day or made some adjustment in their personal life and they are sitting at home waiting for their sofa thinking that they are going to get the sofa at 3:00 p.m."

In a traditional manual workflow, the driver must stop and call dispatch. A human coordinator is then forced to manually triage the breakdown: determining which nearby drivers have capacity, contacting third-party logistics providers, weighing competing pricing options, and trying to preserve the delivery window under immense time pressure.

In an orchestrated agentic architecture, the orchestration director handles the disruption automatically. Operating within strict policy constraints, the agent detects the vehicle breakdown, evaluates real-time capacity across nearby fleets, balances provider reliability against operational cost thresholds, and books an alternate truck. It automatically coordinates Driver A and Driver B to transfer the cargo, preserving the 3:00 PM delivery window without requiring stressful, manual human intervention.

[07:52](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=472s) Core Responsibilities of the Orchestration Director

The orchestration director functions as the brain and safety harness for multi-agent workflows. It begins by formulating an end-to-end execution plan and determining the most cost-effective, reliable compute path. Not every task warrants an LLM; standard business logic belongs in deterministic systems, APIs, and event streams. When cognitive reasoning is required, the director routes the sub-task to the appropriate model tier—selecting a lightweight Small Language Model (SLM) for targeted classifications or a larger, more expensive model for complex reasoning.

Beyond routing, the orchestration layer enforces continuous input/output validation, human-in-the-loop escalation, and comprehensive system-wide observability:

- **Intelligent Routing**: Segregates tasks between traditional deterministic microservice APIs and generative models, preventing unnecessary token costs and latency. - **Model Right-Sizing**: Matches cognitive complexity to the cheapest capable model, reserving large frontier models exclusively for high-reasoning tasks. - **End-to-End Observability**: Captures exhaustive audit logs and telemetry for every agent interaction. "There is no way we are going to have the agentic solution running in production without any logging and without any tracking," because when systems fail, executive leadership requires immediate root-cause evidence. - **Dynamic Human Escalation**: Automatically halts execution and passes context to human operators when an agent encounters ambiguous edge cases or exceeds pre-set risk thresholds.

[10:15](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=615s) Reverse Logistics: Building an Orchestrated Return Flow

The architectural principles governing supply chain agents apply universally across finance, healthcare, and other regulated domains. Software design patterns, state management, and orchestration mechanics remain consistent; only domain-specific policies and compliance guardrails change.

To demonstrate an implementation, Jingar breaks down an e-commerce reverse logistics workflow. When a customer initiates a return for a single item or an entire order, the request enters the system and is immediately intercepted by the orchestration director. Before triggering shipping labels, refunds, or warehouse routing agents, the orchestrator passes the request through an eligibility policy gate. This gate deterministically checks hard business constraints—such as whether the item falls within the standard 30-day return window and whether the specific SKU is classified as returnable—instantly rejecting ineligible requests before invoking any probabilistic agent logic.

[12:37](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=757s) Policy Enforcement, Cost Optimization, and Automated Return Routing

Once an inbound return request passes its baseline policy evaluations and configuration checks, the orchestration director moves to cost optimization and customer lifetime value calculations. Many consumers encounter situations with online e-commerce retailers where customer service simply states, "Okay, you didn't like the product, you can keep it and we'll give you 100% return." The orchestrator handles this trade-off deterministically by weighing the expense of reverse logistics against customer value. If returning the item is uneconomical, the system offers an immediate resolution—such as telling the customer to keep the item for a full refund, or extending a partial concession such as 20% to 30% off to retain the product and close the ticket instantly.

If physical recovery is warranted, the orchestrator calculates the most cost-efficient recovery method and assigns the optimal destination facility. To prevent margin erosion, routing logic prevents cross-country shipping anomalies, ensuring an East Coast customer does not ship a returned item across the country to a California facility. While individual deterministic rules could execute these steps in isolation, AI orchestration provides the primary return on investment by coordinating dozens of intersecting variables across eligibility, concessions, reverse logistics costs, and warehouse routing in a single automated pipeline without requiring human intervention.

```mermaid flowchart TD accTitle: Automated E-Commerce Return Orchestration Pipeline subgraph INTAKE["1. Policy & Validation"] A["Return Request"] --> B["Policy & Eligibility Gate"] end subgraph DECISION["2. Optimization Director"] B --> C{"Cost vs. Lifetime Value"} C -->|"Low Margin / High Cost"| D["Concession / Keep Item Offer"] C -->|"Physical Return Required"| E["Cost-Optimal Recovery Engine"] end subgraph ROUTING["3. Execution & Settlement"] D --> F["Instant Refund / Closure"] E --> G["Proximity-Based Facility Routing"] end ```

- **Policy gatekeeping**: Evaluates baseline configuration, product return windows, and operational constraints prior to financial calculations. - **Dynamic concession handling**: Automatically calculates whether to issue a keep-it refund or offer a 20% to 30% discount to eliminate reverse freight costs. - **Geographic optimization**: Selects the nearest fulfillment or liquidation facility relative to the customer's location to minimize transport expenses.

---

[15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) Root Cause Analysis Director and Self-Healing Production Systems

Engineering operations present an ideal domain for AI orchestration through an autonomous Root Cause Analysis (RCA) Director. When high-severity production incidents occur, engineers expend substantial effort sifting through logs, stack traces, deployment histories, GitHub repositories, and Jira tickets. The RCA Director automates this diagnostic phase by aggregating telemetry across these disparate systems, identifying the blast radius, validating diagnostic hypotheses, and recommending targeted remediation steps. In mature implementations, the director synthesizes the corrective code change, allowing an on-call engineer to review and approve the pull request directly.

As the orchestration director matures, systems can transition into self-healing architectures. By embedding background "LLM judges" that continuously evaluate model outputs for accuracy and deterministic consistency, organizations can automate low-risk remediations end-to-end without manual intervention. This automation is not aimed at replacing human engineers, but rather at protecting operational quality of life and eliminating alert fatigue on tier-one critical platforms: "Some of the companies have 24/7 on-call support. If anything breaks, they need to wake up at 2:00 a.m. I don't want my engineer to wake up every week [at] 2:00 a.m." Complex, unprecedented failures that exceed automated confidence thresholds are escalated directly to engineers with fully compiled diagnostic context.

```mermaid flowchart LR accTitle: Root Cause Analysis Director and Self-Healing Architecture subgraph DATA["Telemetry Ingestion"] A["Logs & Traces"] B["GitHub & Deployments"] C["Jira Systems"] end subgraph RCA["RCA Director Engine"] A & B & C --> D["Diagnostic & Blast Radius Engine"] D --> E["Patch Generation & Hypothesis Testing"] E --> F{"LLM Judges Validation"} end subgraph RESOLUTION["Remediation Path"] F -->|"High Confidence"| G["Self-Healing Deployment"] F -->|"Low Confidence / Critical"| H["Human Escalation & PR Review"] end ```

- **Cross-system ingestion**: Aggregates disparate diagnostic telemetry spanning application logs, stack traces, deployment pipelines, GitHub commits, and Jira issues. - **Automated triage and repair**: Measures blast radius, generates candidate patches, and supplies ready-to-merge remediation code. - **LLM judge validation**: Secondary LLM evaluation layers verify that diagnostic assertions and code fixes meet strict determinism standards before triggering autonomous execution.

---

[19:24](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1164s) Core Operating Principles for Enterprise AI Orchestration

Designing scalable orchestration layers requires strict adherence to architectural and governance principles. First, systems require rigid contract definitions where every input and output is explicitly validated. Second, implicit organizational and tribal knowledge must be extracted from individual engineers' heads and formalized into codified, machine-readable policies.

Third, end-to-end observability, logging, and tracking must span every agentic transition to maintain operational oversight. Fourth, high-risk or destructive actions must enforce explicit human gates, ensuring the director automatically halts execution and escalates to a human operator when confidence boundaries are breached. Finally, comprehensive auditability ensures engineers can trace every decision point, model prompt, and system action back to its source whenever a component fails.

- **Strict interface contracts**: Explicit validation schemas for all agent inputs, outputs, and system transitions. - **Codified policies**: Formalized organizational rules that eliminate reliance on unwritten institutional memory. - **Comprehensive observability**: Continuous tracing and structured telemetry across the entire multi-agent lifecycle. - **Mandatory human gates**: Configurable escalation triggers that interrupt autonomous processing for high-risk operations. - **Granular auditability**: Full replayability and event logging to trace root causes behind any unexpected system behavior.

---

[20:43](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1243s) The 90-Day Implementation Roadmap and Conclusion

Implementing an enterprise orchestration architecture should follow a phased 90-day roadmap rather than attempting immediate broad adoption. During the first 30 days, teams must focus exclusively on defining a single workflow, documenting every validation check, evaluation standard, business rule, and policy boundary. The subsequent 30 days are dedicated to rigorous testing and hardening, running that initial workflow end-to-end until it operates deterministically. During the final 30-day phase, the validated patterns, judges, and governance framework are replicated across subsequent workflows, systematically expanding enterprise AI capability into fully authentic, production-grade automation.

- **Days 1–30 (Define)**: Scope a single business workflow; author all evaluation criteria, interface contracts, and policy guardrails. - **Days 31–60 (Validate)**: Conduct comprehensive end-to-end operational testing and hardening on the initial workflow. - **Days 61–90 (Scale)**: Replicate the hardened orchestration engine across adjacent organizational workflows and enterprise use cases.

## Notable quotes > [02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) LLM by default is probabilistic. We all know. It's not going to be 100% deterministic, and we cannot put a probabilistic system on production which doesn't give you the right information and doesn't make the right decision. — Nirmal Jingar

> [04:36](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=276s) The truck broke. Now, what do you do? I told customer, you are going to get your product at 3:00 p.m. It's a very large heavy sofa. Customer might have taken half day or made some adjustment in their personal life and they are sitting at home waiting for their sofa thinking that they are going to get the sofa at 3:00 p.m. — Nirmal Jingar

> [07:52](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=472s) There is no way we are going to have the agentic solution running in production without any logging and without any tracking — Nirmal Jingar

> [12:37](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=757s) Okay, you didn't like the product, you can keep it and we'll give you 100% return. — Nirmal Jingar

> [15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) Some of the companies have 24/7 on-call support. If anything breaks, they need to wake up at 2:00 a.m. I don't want my engineer to wake up every week [at] 2:00 a.m. — Nirmal Jingar

## Mentioned Nirmal Jingar, Wayfair, Ai4, ChatGPT, TEDx, Forbes Technology Council, GitHub, Jira

## Video description Nirmal Jingar - Sr. Engineering Manager, Wayfair

Most AI systems fail not because the models are weak but because the coordination layer is missing. This talk will explain why multi agent systems need an orchestration director that manages planning, validation, governance, and execution across agents. I will walk through real architectural patterns for deterministic workflows, agent communication, escalation handling, and failure recovery. Attendees will learn how to design scalable and trustworthy AI systems that avoid chaos, hallucinations, and inconsistent decisions. This session is for engineers and leaders building production grade AI platforms beyond simple chatbot experiences.

[Source](https://www.youtube.com/watch?v=NTOgU-lJ_Wc)

中文简报

Wayfair引入确定性编排以运行自主AI系统企业AI系统在关键业务中面临失效风险，因概率型模型需要确定性编排层来执行策略门禁、验证输出并管理故障恢复。 Wayfair工程负责人Nirmal Jingar指出，要弥合顾问型聊天机器人与自主生产系统之间的鸿沟，需要引入确定性编排层来监督概率型LLM。通过剖析企业供应链场景、逆向物流优化和自动化根因分析，他演示了如何在动作执行前落实策略门禁、适配模型规模并对决策进行验证。演讲重点强调了严格契约、LLM裁判、人工升级机制以及深度可观测性等核心原则，并最终给出了分阶段落地高弹性智能体工作流的90天路线图。尽管大语言模型已被企业广泛采用，但在核心关键任务系统中端到端自主的 Agent 仍几乎处于空白状态，其本质上的概率性行为难以满足生产环境 100% 确定性执行的要求。 (2:41) 企业架构亟需引入“编排主管”（Orchestration Director）作为确定性骨干，在调用模型前通过前置策略网关核验基础业务约束，并在传统 API、小语言模型（SLM）与大型 LLM 间智能路由以控制成本与延迟。 (7:52) 在逆向物流等场景中，AI 编排通过同时权衡逆向运费与客户终生价值等多维变量，能够自动做出保留商品退款或就近退运决策，显著提升投资回报率。 (12:37) 在系统运维中，根因分析（RCA）主管可跨日志、代码和工单系统生成诊断与代码补丁，依托背景“LLM 裁判”实现低风险故障自愈，并在超出置信度时强制触发人工介入。 (15:59)

Reading summary

Nirmal Jingar, Senior Engineering Manager at Wayfair, argues that enterprise adoption of autonomous generative AI remains stalled because probabilistic models lack the deterministic guardrails required for mission-critical production systems. Jingar proposes inserting an "orchestration director" layer to govern model routing, enforce strict policy gates, and maintain end-to-end auditability. The Gap Between Advisory Tools and Deterministic Execution (0:04) While tools like ChatGPT are widely used across enterprises for drafting and synthesis, Jingar observes that autonomous agentic deployments remain virtually absent from tier-one infrastructure such as healthcare, banking, and core supply chains. Because large language models are inherently probabilistic rather than deterministic, deploying them unguarded into production databases or physical logistics risks causing millions of dollars in financial liability from incorrect decisions. To bridge this gap, Jingar advocates for an orchestration director acting as a deterministic supervisory backbone. This layer enforces that operational decisions are bounded by strict business policies and corroborated by verifiable enterprise data before any state change occurs, preventing probabilistic uncertainty from triggering operational failures. Responsibilities of the Orchestration Director (7:52) The orchestration director plans end-to-end execution paths and minimizes operational costs by avoiding unnecessary model usage. Standard deterministic microservice APIs handle routine business logic, while cognitive tasks are routed to the most cost-effective tier—delegating simpler tasks to Small Language Models (SLMs) and reserving expensive Large Language Models (LLMs) strictly for complex reasoning. Beyond cost-aware routing, the orchestration layer continuously enforces safety guardrails. It establishes strict interface contracts, tracks comprehensive audit logs, and monitors telemetry across every agent interaction. When an agent encounters ambiguous edge cases or exceeds defined risk thresholds, the system halts execution and escalates the context to a human operator. Applying Orchestration to Supply Chain and Operations (10:15) Jingar demonstrates orchestration mechanics through e-commerce reverse logistics and in-transit delivery disruptions. In a delivery truck breakdown, an orchestrated system detects the issue, analyzes capacity across nearby fleets against cost thresholds, coordinates cross-loading between drivers, and preserves promised delivery windows without manual dispatch triage. In reverse logistics, inbound return requests first encounter a deterministic policy gate that evaluates hard constraints like return windows and SKU eligibility before invoking generative models. For eligible returns, the director balances freight costs against customer lifetime value. If physical recovery is uneconomical, it automatically triggers instant concessions, such as issuing a full refund while allowing the customer to keep the item or providing a 20% to 30% discount. When physical return is necessary, proximity-based routing ensures items are directed to the nearest suitable facility rather than incurring cross-country transit fees. Autonomous Diagnostics and Self-Healing Deployments (15:59) In engineering operations, an autonomous Root Cause Analysis (RCA) Director aggregates diagnostic telemetry across logs, stack traces, deployment pipelines, GitHub commits, and Jira tickets. The engine assesses the incident's blast radius, validates diagnostic hypotheses, and synthesizes candidate code patches for engineer review, significantly mitigating on-call fatigue during off-hours. As confidence in the system matures, secondary background "LLM judges" can continuously evaluate model outputs against deterministic standards. This setup allows low-risk remediations to be deployed automatically as self-healing operational actions, while unprecedented or high-severity anomalies continue to be routed directly to human engineers with pre-compiled diagnostic context. To adopt this architecture without excessive risk, Jingar outlines a structured 90-day implementation roadmap: the first 30 days are dedicated to codifying business policies and defining interface contracts for a single workflow, days 31–60 focus on end-to-end hardening and testing, and days 61–90 replicate the validated orchestration patterns across adjacent enterprise use cases.

精炼阅读摘要

Wayfair高级工程经理Nirmal Jingar指出，当前大语言模型因其固有的概率性特征，难以直接独立承担核心生产级任务；企业必须引入具备确定性验证与治理能力的“编排调度器”（Orchestration Director），才能实现真正端到端的智能体自主自动化。概率性模型与确定性生产需求的矛盾 (2:41) 虽然ChatGPT等工具被广泛用于推理、文本起草与建议，但在医疗、银行或供应链等关键任务级工作负载中，几乎没有企业在完全自主运行端到端的智能体系统。Jingar强调，大语言模型在本质上是概率性的，无法保证100%确定性的输出。若直接将未受约束的概率模型部署到核心业务或财务系统中，一旦发生失误，可能在数秒内引发数百万美元的损失。因此，目前的AI部署大多停留在辅助咨询阶段，最终操作仍需依赖人工完成。编排调度器的架构职能与模型分流 (7:52) 为了在概率性模型之上提供确定性保障，企业架构需要设立专职的编排调度器。该组件充当多智能体工作流的中枢大脑与安全控制层，首先制定端到端执行计划，并确定成本最高效的计算路径。常规业务逻辑直接分流至传统的确定性微服务API，无需动用模型；在需要推理时，调度器根据任务复杂度选择轻量化小语言模型（SLM）或高成本的大语言模型（LLM），避免不必要的Token开销与延迟。调度器同时负责端到端的可观测性、输入输出校验以及人机协同（HITL）升级机制。系统必须记录详尽的审计日志与遥测数据，以满足故障发生时管理层对根本原因追溯的严格要求。一旦智能体遇到歧义边界情况或超出预设的风险阈值，调度器会自动暂停自主执行，并将完整上下文移交人工接管。业务落地验证：逆向物流与在途异常协同 (10:15) 在干线运输异常场景中，若运送沉重沙发的货车中途抛锚，传统人工调度需耗费大量时间协调司机与第三方运力，极易延误预定交付窗口；而编排调度器能在策略约束下实时评估邻近车队运力与成本，自动派单并协调换车倒装，保障下午3点的交付承诺。在电商逆向物流流程中，退货请求接入后首先进入确定性策略网关，核验30天退换期及SKU属性等刚性规则，直接拦截无效请求。通过验证后，系统综合权衡物流成本与客户终身价值：针对退运成本过高的低利润单，直接发放全额退款并建议客户保留商品，或提供20%至30%的折扣折让以立即结案；若需实物回收，系统则基于地理就近原则分流至最近的履行或清算设施，防止跨国跨区的高额运费损耗。自愈式根因分析与工程运维减负 (15:59) 在工程运维领域，自主根因分析（RCA）调度器可汇总应用日志、堆栈跟踪、发布历史、GitHub提交与Jira工单，自动化评估故障爆炸半径、验证诊断假设并推荐修复步骤。在成熟阶段，系统甚至能直接生成修复代码补丁并提交PR供值班工程师审核。引入后台“LLM裁判”（LLM Judges）持续评估输出准确性与确定性后，低风险故障可实现端到端自愈。Jingar指出，此举的目的不是取代工程师，而是消除核心平台运维人员在凌晨两点被告警唤醒的负担，解决7×24小时值班带来的疲劳问题；超出置信度阈值的复杂未知故障，则依然保留人工审核防线。企业编排落地的核心原则与90天路径 (19:24) 构建企业级AI编排架构必须遵循五大核心原则：明确验证所有输入输出的严格接口契约、将工程师头脑中的隐性组织经验转化为机器可读的代码化策略、贯穿智能体全生命周期的端到端可观测性、高危操作的强制人工介入卡点，以及全链路决策可复现的细粒度审计追踪。 Jingar建议企业采取分阶段的90天落地路线图：前30天专注于圈定单一业务流程，完成规则、契约与评估标准的定义；中间30天对该单一流程开展端到端强化测试与确定性验证；最后30天将已固化的编排引擎、评估裁判与治理模式横向复制到其他企业业务场景中。该方案的核心逻辑在于通过确定性的工程治理包容概率性的AI模型，但企业在实际落地时，能否将零散的组织隐性知识完全代码化，并在兼顾成本的同时维持高水准的规则维护，仍是长期部署中需要面对的现实挑战。

Detailed notes — zh

## Background 本内容由频道 Ai4 于 2026 年 8 月 18 日在 YouTube 平台发布，主讲人为 Wayfair 高级工程经理 Nirmal Jingar。视频围绕企业级多智能体系统展开，重点探讨为何大多数 AI 系统的失败源于协调层缺失，并深入讲解了如何通过构建“编排指挥官”（Orchestration Director）来实现规划、校验、治理与执行，从而打造可扩展且值得信赖的生产级 AI 架构。 ## TL;DR Nirmal Jingar 指出当前企业级 Tier-1 核心系统落地 Agentic AI 的最大断层在于缺乏将概率性模型转化为确定性行动的编排层。演讲通过物流配送异常动态调度、电商逆向物流全自动退货及成本优化、以及研发运维自动化根因分析（RCA）与系统自愈等实战案例，系统拆解了编排指挥官在智能路由、多模型调度、可观测性与合规治理中的核心机制。最后，他提炼了包含契约定义、策略显性化与人工门禁在内的五大运营原则，并给出了从单点突破到规模化复制的 90 天企业级实施路线图。 ## Atmosphere & style 演讲整体呈现出务实、严谨且极具一线工程实战色彩的技术布道风格。主讲人通过尖锐的现场互动开局，直击行业中“Demo 繁荣与生产落地脱节”的痛点，随后以条理清晰的架构图和生动的业务案例层层推进。整场分享节奏紧凑、干货密集，既有面向高可用架构的硬核技术推演，又充满了帮助研发团队摆脱“凌晨告警疲劳”的工程师共情。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=4s) 大多数 AI 系统在生产环境中失败的核心原因不是模型能力不足，而是缺少了关键的协调编排层（Orchestration Director）。 - [02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) 大语言模型本质上是概率性的，直接用于资金流转或一级核心系统（Tier 1）容易带来巨大风险，必须通过编排层转化为 100% 确定且有事实证据支撑的决策。 - [07:52](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=472s) 编排指挥官负责动态智能路由：传统确定性逻辑直连 API，简单任务派发给小模型（SLM），仅复杂推理才调用昂贵的前沿大模型（LLM）。 - [04:36](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=276s) 在异常物流场景中，Agentic 编排系统能在司机抛锚时自动计算运力、成本与 SLA，毫秒级调度转运并消除传统人工沟通摩擦。 - [10:15](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=615s) 企业级编排架构具有跨行业通用性，不同业务领域（如供应链、金融、医疗）仅在具体业务规则与合规控制层上有所差异。 - [12:37](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=757s) 逆向物流编排不仅校验退货资格，还能综合计算物流成本与客户终身价值（CLV），自动决策全额退款、折价保留或就近仓储路由。 - [15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) 根因分析（RCA）指挥官整合日志、堆栈跟踪与 GitHub/Jira 数据，自动化排障并生成修复代码，配合 LLM 裁判逐步演进为系统自愈能力。 - [19:24](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1164s) 多智能体落地的五大核心运营原则包括：严格的契约定义、隐性知识策略显性化、全链路可观测性、人工把关门禁（HITL）以及可追溯审计能力。 - [20:43](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1243s) 企业推进 AI 编排应采用 90 天路线图：前 30 天定义单个工作流，中 30 天闭环测试调优，后 30 天横向复制扩展至更多业务场景。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=4s) 生产级企业 AI 的断层：从概率性模型到确定性编排在技术演讲的开场中，Wayfair 高级工程经理 Nirmal Jingar 向现场观众致意，并迅速做了一个现场背景调查。现场观众的技术与工程背景占比大约各占一半（50/50）。Jingar 指出，当前大多数人对 AI 的使用仍停留在类似 ChatGPT 的通用工具层面——人们提出问题，模型进行推理并给出答案或建议，再由人类根据建议采取行动。然而，在真正的企业级生产环境中，尤其是对容错率极低的一级核心系统（Tier 1 systems）而言，能够实现严格治理与完整编排的系统凤毛麟角。 Jingar 结合自身在 AI 供应链领域的深厚积累（包括 TED/TEDx 演讲者、福布斯技术委员会成员及多家机构顾问董事的经历），向在场的技术同行提出了一个尖锐的问题：现场是否有人已经在医疗、银行、金融或一级供应链等 Tier 1 核心系统上运行全流程的端到端 Agentic AI？看到全场没有一个人举手，他直言“这正是我们今天齐聚于此的原因”。他强调，真正的企业级落地必须跨越从模型输出到生产级决策的断层，并宣布本次分享将重点围绕架构体系以及两个具体场景展开：逆向物流退货编排与自动化根因分析（Root Cause Analysis, RCA）。 - **企业级 Tier 1 系统的苛刻要求**：涵盖金融、医疗及大型供应链等关键业务线，要求系统绝不能出现不可控的决策偏差。 - **全场调研现状**：目前业内在核心生产系统中真正实现端到端自主 Agentic AI 闭环的企业仍旧极少。 --- ### [02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) 编排指挥官的核心使命：确定性、治理与证据链大语言模型（LLM）的输出本质上擅长预测、提炼摘要以及撰写邮件草稿，但当进入企业级关键生产环境时，系统最致命的缺失正是“编排指挥官”（Orchestration Director）。LLM 天生是概率性的（probabilistic），无法保证每一次输出都是 100% 确定（deterministic）。如果在管理着数百万资金流转的一级供应链系统中直接部署概率性系统，“模型一旦出现误判，我们可能在短短几秒钟内损失数百万美元”。编排层的职责，正是作为守门人来保证系统上线后决策的 100% 正确性。它不仅要负责全流程的合规治理，还必须对模型的所有输出进行严格校验，确保每一条决策都具有“可追溯的事实证据支撑”（backed by evidence）。编排器将底层的概率性推理转化为可预测、可审计且符合业务规则的确定性行动，从而让企业敢于将自主决策权赋予生产系统。 ```mermaid flowchart TD accTitle: AI Orchestration Director Core Architecture subgraph Client ["输入与触发"] REQ["业务请求 / 异常事件"] end subgraph Orchestrator ["Orchestration Director 编排指挥层"] ROUTER{"路由决策"} VAL["输入/输出校验与策略治理"] OBS["端到端可观测性与审计日志"] ESC["人工升级处理机制 (HITL)"] end subgraph Execution ["执行与模型层"] API["确定性传统 API / 事件"] SLM["轻量级/低成本模型 (SLM)"] LLM["复杂推理前沿大模型 (LLM)"] end REQ --> ROUTER ROUTER -->|"无需模型推理"| API ROUTER -->|"简单分类/抽取"| SLM ROUTER -->|"复杂逻辑推理"| LLM API & SLM & LLM --> VAL VAL --> OBS VAL -->|"超出阈值/低置信度"| ESC ``` --- ### [04:36](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=276s) 供应链实战场景：货车抛锚与 Agentic 动态调度为了具象化展示编排层的价值，Jingar 分享了一个极具现场感的大件物流场景：某位消费者订购了一件大型沉重沙发，系统承诺在两天后的下午 3:00 送达。为了等待这次大件配送，客户特意请了半天假或专门调整了个人的生活作息。然而在派送当天，负责该区域的 1 号货车突发机械故障抛锚，车上装载着包括该沙发在内的 100 个包裹。在没有 Agentic AI 的传统人工模式下，流程极其繁琐且充满摩擦： - 抛锚司机必须手动致电调度员。 - 调度员需要手忙脚乱地排查周边是否有其他空闲的货车与司机。 - 若内部运力不足，调度员还要紧急联络第三方运力服务商，在多达 10 种报价方案中人工权衡“哪家更便宜、哪家更可靠、谁能在下午 3:00 前准时将沙发送到”。而在具备成熟编排治理层的 Agentic 架构下，整个流程可以全自动无缝运转：编排系统依据预设的合规政策与成本阈值，自动评估可用运力，智能选定最具性价比且时效可靠的备用司机或第三方服务商。系统自动完成两名司机的对接与路线调度，引导双方完成货物转驳（cross-docking/transfer），最终确保客户依然能在约定的下午 3:00 准时签收沙发，整个过程无需漫长的人工介入与协调摩擦。 - **消除人工摩擦点**：供应链、金融和医疗领域充斥着类似的大量手动触点，编排驱动的 Agent 能够实现跨主体、跨系统的毫秒级运筹。 - **履约承诺保障**：在异常事件（如车辆抛锚、运力中断）发生时，自动在成本阈值与 SLA 约束之间计算最优解。 --- ### [07:52](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=472s) 编排器的全生命周期职能：路由、多模型选择与可观测性编排指挥官绝不是简单的提示词转发器，而是承担着复杂架构职责的控制中枢。首先是**端到端工作流的规划与执行**；其次是**智能路由决策**。Jingar 强调，绝非所有业务请求都需要调用大语言模型——对于已有成熟规则和确定性逻辑的业务，系统应当直接路由给传统的 API 或事件驱动系统；只有真正需要推理能力的环节，才会被派发给 AI 模型。在选择模型时，编排器会根据任务的难度与成本敏感度，动态选择使用轻量级小语言模型（SLM）、低成本模型还是昂贵的高性能前沿模型。另一个不可妥协的基石是**端到端的可观测性（Observability）**。在生产环境中运行 Agentic 方案，绝不允许存在任何缺乏跟踪与日志记录的“黑盒”环节。Jingar 指出，一旦系统在生产中出错并导致企业蒙受经济损失，董事会高管必然会追问：“到底发生了什么？我们在哪一步出了差错？”因此每一个决策触点都必须保留完整的证据链。此外，编排器还掌管着**人工升级（Escalation Handling）**机制，针对处于自动化早期阶段或置信度低于阈值的复杂边缘情况，及时将控制权交还给人工处理。 - **动态模型分流**：根据业务复杂度和成本预算，在传统 API、小模型（SLM）与大模型（LLM）之间灵活调度。 - **全链路证据追踪**：对 Agent 的输入、上下文、推理步骤及工具调用实施 100% 日志化与证据捕获。 - **人机协同（Human-in-the-Loop）**：依据企业的自动化成熟度，提供平滑的人工介入与决策接管通道。 --- ### [10:15](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=615s) 逆向物流案例剖析：编排器与规则治理层谈及企业架构设计，Jingar 指出，无论是在供应链、金融还是医疗行业，软件架构、设计理念与底层编码逻辑是完全通用的。不同行业之间的主要区别仅在于特定业务规则、合规性要求以及受监管行业（Regulated Industry）所需的额外控制层，基础编排架构完全可以跨行业复用。以电商中的**逆向物流（Reverse Logistics / Returns）**为例，当用户对购买的商品不满意并发起退货时，退货请求可能涉及单件商品或包含多件商品的整个订单。此时，位于顶层的编排指挥官将全程接管从客户发起退货到最终退款清算的全生命周期。其介入的第一道防线就是**退货资格策略层（Eligibility Policy）**： - 系统首先核验时间窗口（例如是否在平台规定的 30 天退货期内）。 - 校验该类目商品是否属于支持退货的品类。 - 只有严格通过合规规则校验的请求才会被编排器推向后续的逆向调度阶段，不符合资格的请求则在治理层被直接拦截驳回，防止无效诉求进入昂贵的下游业务链路。 ### [12:37](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=757s) 电商退货编排与成本优化全流程自动化在电商售后场景中，编排器（Orchestrator）的核心价值在于将业务策略、成本优化与履约决策无缝串联。当一笔退货请求通过基础合规与策略校验后，编排器会立即进入成本与客户终身价值（Customer Lifetime Value, CLV）的多维度核算阶段。很多消费者在电商平台退货时都经历过客服直接告知“商品你留着，我们全额退款”的情况，这正是系统权衡逆向物流成本与客户价值后的决策结果。编排器会精确计算将商品召回所需的物流开销；如果逆向运输不划算，系统会自动向用户提供替代方案，例如建议用户保留商品并给予 20% 至 30% 的折扣。若用户接受，该工单即刻关闭；若用户坚持退货，编排器则会评估最具成本效益的取件方式与路由策略，并就近指定仓储退货点，绝不会出现让美东用户的退货包裹跨越整个美国寄往加州仓库的高昂履约损耗。整个退货决策链路涵盖策略校验、退款资格判定、金额核算、逆向物流优化及目标仓库路由，全部由编排器自动化驱动完成。很多人可能会质疑“这种确定性流程是否根本不需要 AI”，但 AI 在此处的真正价值在于处理海量变量组合下的智能决策——当数以百计的动态条件、履约参数与客户画像交织在一起时，AI 能够实现高投资回报率（ROI）的最优解，实现端到端的全自动化闭环，彻底免去人工客服介入的繁琐流程。 - **成本权衡决策**：自动比对逆向物流费用与商品残值，灵活触发“全额退款无需寄回”或“部分折扣保留商品”的协商策略。 - **履约路由优化**：根据用户物理位置就近匹配接收仓，严格杜绝跨区域（如美东至加州）的高昂运输损耗。 - **多变量综合决策**：借助 AI 在多维复杂变量组合中寻找最优解，真正释放端到端自动化的业务 ROI。 --- ### [15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) 根因分析（RCA）指挥官与自愈系统架构对于一线研发团队而言，系统故障排查向来是耗时最长的痛点。为此，Wayfair 探索了面向工程运维的“根因分析（RCA）指挥官”体系。该系统作为多智能体中枢，持续从应用程序日志、堆栈跟踪（Stack Traces）、发布部署记录以及 GitHub、Jira 等工程工作流工具中汇聚全量上下文数据。RCA 指挥官能够自动化推演故障根因、测算故障影响面（Blast Radius）、验证核心假设，并生成包含修复代码（PR）的针对性建议。研发人员只需审查生成的代码并确认上线，大幅缩短了平均修复时间（MTTR）。 ```mermaid flowchart TD accTitle: Root Cause Analysis Director and Self-Healing Workflow subgraph Ingestion["数据收集层"] A["系统日志与堆栈跟踪"] B["部署记录 / GitHub / Jira"] end subgraph Orchestration["RCA 指挥官分析"] C["根因分析指挥官"] D["影响范围评估 & 假设验证"] E["自动生成修复代码与建议"] end subgraph Validation["治理与执行层"] F["LLM 裁判验证确定性"] G{"复杂度与风险评估"} H["自动化系统自愈上线"] I["工程师人工审查 (Tier-1 兜底)"] end A --> C B --> C C --> D D --> E E --> F F --> G G -- "高置信度/低风险" --> H G -- "复杂异常/高风险" --> I ``` 随着编排体系的成熟，系统引入了多维度的“大模型裁判”（LLM Judges）机制，在后台对大模型生成的每一步响应与修复方案进行严格的确定性与正确性校验。当系统置信度达到一定阈值后，特定场景的故障即可演进为完全无需人工介入的“自愈系统（Self-Healing Systems）”。这种架构并不是为了替代工程师，而是为了将核心研发人员从高压的 24/7 待命状态中解救出来。特别是在支撑 Tier-1 核心生产系统时，“我不希望我的工程师每周都在凌晨两点被告警叫醒”，唯有将复杂低阶的排障与修复自动化，工程师才能聚焦于极少数最复杂、高风险的架构难题。 - **多源数据融合**：打通 Logs、Traces、Deployments、GitHub 与 Jira，构建全局排障上下文。 - **LLM 裁判机制**：在后端对模型推理与输出做确定性交叉校验，确保自愈操作安全合规。 - **改善研发体验**：接管常规 Tier-1 告警处理，终结运维工程师“凌晨两点被叫醒”的疲劳作战状态。 --- ### [19:24](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1164s) 编排层落地的五大核心运营原则在设计企业级 AI 编排层与各类场景指挥官时，必须确立严谨的运营设计原则，防止多智能体系统在生产环境中失控。第一是**契约定义（Contract Definition）**，必须严格明确系统的输入与输出规范，对每一个流入流出的数据载荷进行严格校验；第二是**策略显性化（Policies）**，当前大多数企业中存在海量未被记录的“隐性组织知识”，只停留在员工大脑中，必须将其全面沉淀并文档化为规则库与配置项。 ``` ┌────────────────────────┐ │ 编排层五大核心运营原则 │ └───────────┬────────────┘ ┌──────────────┬───────┴───────┬──────────────┐ ▼ ▼ ▼ ▼ 【契约定义】【策略显性化】【全链路可观测】【审计追溯性】严格校验 I/O 载荷隐形知识转文档 Logging/Tracing 全触点回溯定位 │ ▼ 【人工把关门禁】高风险动作自动升级 ``` 第三是**全链路可观测性（Observability）**，贯穿系统运行的每一个节点，提供详尽的分布式追踪（Tracing）与日志记录；第四是**人工把关门禁（Human Gates）**，对于任何涉及资金、核心数据或高风险的操作，编排层必须设计自动升级机制（Escalation），交由人类专家裁决；第五是**全触点审计能力（Auditability）**，当系统发生异常或决策偏差时，技术团队必须能够沿调用链逐层回溯，精确定位故障节点与决策诱因。 - **契约定义**：建立严格的 API 契约，对所有模型的输入和输出做强类型与格式校验。 - **知识显性化**：将团队内部口口相传的组织策略与业务逻辑转化为可执行的配置与规则。 - **风险隔离**：设置自动化的人工介入（Human-in-the-loop）门禁，高危动作强制人工审批。 - **审计追踪**：记录每一个智能体交互的执行触点，确保事后复盘可追溯、可归因。 --- ### [20:43](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1243s) 企业级落地 90 天实施路线图与总结针对大型企业引入 AI 编排层的落地路径，Nirmal Jingar 提出了务实的“90 天阶段式实施计划”。面对庞大复杂的企业业务，切忌一上来就试图全面铺开，而应聚焦于单点突破。在第一个 30 天内，团队应当集中精力定义并梳理出“单个核心工作流”，将该流程涉及的所有评估指标（Evals）、检查点、业务规则、安全配置与策略约束彻底文档化并完成编排实现。进入第二个 30 天阶段后，团队的工作重心转向对该单点工作流的全面测试与调优，确保该流程在端到端各环节均能达到极其严密的确定性与稳定性。在完成前 60 天的单点验证后，团队在最后的 30 天便可以这套经过验证的编排范式为标准模板，快速横向复制到第二个、第三个乃至更多复杂的工作流中，最终实现整个企业 AI 系统由初级实验向真正可靠、规模化的生产级 AI 转型。演讲最后，Nirmal 感谢了参会观众的聆听，并鼓励大家通过现场二维码在 LinkedIn 上建立联系，共同探讨多智能体编排与工程落地的实践经验。 - **第 1–30 天（定义单点流程）**：选择单一高价值业务流，彻底理清所有策略、规则配置与评估指标。 - **第 31–60 天（闭环测试调优）**：对选定流程进行极限测试与端到端跑通，实现高稳定性的确定性交付。 - **第 61–90 天（横向复制扩展）**：以标准编排模板为基座，向更多业务场景快速横向推广，实现规模化落地。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=4s) 现场是否有人已经在医疗、银行、金融或一级供应链等 Tier 1 核心系统上运行全流程的端到端 Agentic AI？ — Nirmal Jingar > [00:04](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=4s) 这正是我们今天齐聚于此的原因 — Nirmal Jingar > [02:41](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=161s) 模型一旦出现误判，我们可能在短短几秒钟内损失数百万美元 — Nirmal Jingar > [07:52](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=472s) 到底发生了什么？我们在哪一步出了差错？ — Nirmal Jingar > [12:37](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=757s) 商品你留着，我们全额退款 — Nirmal Jingar > [15:59](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=959s) 我不希望我的工程师每周都在凌晨两点被告警叫醒 — Nirmal Jingar > [20:43](https://www.youtube.com/watch?v=NTOgU-lJ_Wc&t=1243s) 单个核心工作流 — Nirmal Jingar ## Mentioned Nirmal Jingar, Wayfair, Ai4, ChatGPT, GitHub, Jira, LinkedIn, TED, TEDx, Forbes Technology Council ## Video description Nirmal Jingar - Sr. Engineering Manager, Wayfair Most AI systems fail not because the models are weak but because the coordination layer is missing. This talk will explain why multi agent systems need an orchestration director that manages planning, validation, governance, and execution across agents. I will walk through real architectural patterns for deterministic workflows, agent communication, escalation handling, and failure recovery. Attendees will learn how to design scalable and trustworthy AI systems that avoid chaos, hallucinations, and inconsistent decisions. This session is for engineers and leaders building production grade AI platforms beyond simple chatbot experiences. [Source](https://www.youtube.com/watch?v=NTOgU-lJ_Wc)

Source quotes

[0:04] Nirmal Jingar: 现场是否有人已经在医疗、银行、金融或一级供应链等 Tier 1 核心系统上运行全流程的端到端 Agentic AI？ [0:04] Nirmal Jingar: 这正是我们今天齐聚于此的原因 [2:41] Nirmal Jingar: 模型一旦出现误判，我们可能在短短几秒钟内损失数百万美元 [7:52] Nirmal Jingar: 到底发生了什么？我们在哪一步出了差错？ [12:37] Nirmal Jingar: 商品你留着，我们全额退款 [15:59] Nirmal Jingar: 我不希望我的工程师每周都在凌晨两点被告警叫醒 [20:43] Nirmal Jingar: 单个核心工作流

Source quotes

[2:41] Nirmal Jingar: LLM by default is probabilistic. We all know. It's not going to be 100% deterministic, and we cannot put a probabilistic system on production which doesn't give you the right information and doesn't make the right decision. [4:36] Nirmal Jingar: The truck broke. Now, what do you do? I told customer, you are going to get your product at 3:00 p.m. It's a very large heavy sofa. Customer might have taken half day or made some adjustment in their personal life and they are sitting at home waiting for their sofa thinking that they are going to get the sofa at 3:00 p.m. [7:52] Nirmal Jingar: There is no way we are going to have the agentic solution running in production without any logging and without any tracking [12:37] Nirmal Jingar: Okay, you didn't like the product, you can keep it and we'll give you 100% return. [15:59] Nirmal Jingar: Some of the companies have 24/7 on-call support. If anything breaks, they need to wake up at 2:00 a.m. I don't want my engineer to wake up every week [at] 2:00 a.m.

Original transcript

[0:04] Really appreciate sticking around with me and this session. If you are in the previous session, [0:10] I think this perfectly aligns. Like we are going to talk about the infrastructure of AI [0:16] and specifically about the orchestration layer and how do we build it, right? [0:20] How many people are like technical how engineering background? Okay, cool. Half. Okay, so 50/50. [0:28] I might try to go a little bit more technical because because this is related to like how do you [0:33] build the orchestration layer and what are the things you need to take care about it. Few things [0:40] about my background. I'm a senior technology leader in AI supply chain. I'm a TED TEDx speaker, [0:47] Forbes Technology Council member. I'm also a board member and board of advisor for few organizations, [0:52] right? So, um we have been using AI from some time from couple of years now, right? Um everybody's [1:00] using chat GPT, bunch of other tools which you ask your question and give you the answer, right? [1:06] It's more of a like the way we are using is like it's give you the reasonings, right? It's reason. [1:13] And it's make some recommendations, right? And you make some action based on those recommendations, [1:19] right? Um but there are very few system even at the enterprise level which are properly governed, [1:27] which are properly orchestrated, right? Because when I say enterprise level, I want to see [1:33] the AI system, the system that that I'm putting on production is making the right decision. [1:41] Right? So, when I say enterprise level AI, I'm talking about the tier one supply chain systems. [1:47] I'm talking about the health care system and the financial systems, right? Um has anyone anybody [1:53] from the audience running fully end-to-end agentic AI on tier one system like health care banking [2:01] finance or supply chain. No? I don't see any heads. So, yeah, that's why we are here, right? [2:10] Cool. So, I'm going to talk about like starting from the production gap [2:16] and then we'll have few things and finally we'll talk about two specific examples. [2:21] I have the supply chain background so I'm going to talk about like one example from returns like [2:28] how do we build the orchestration for returns and the another example I'm going to talk about [2:34] root cause analysis, right? How do we build the authentic root cause analysis? [2:41] So, let's move on. Cool. So, I talked little bit about the model output, [2:47] right? So, model output can predict give you the summaries, right? And [2:56] maybe help you in making some draft email draft which all of us have been using it, right? And uh [3:03] uh if we are using at the enterprise level, the missing layer is the orchestration layer. [3:09] The real orchestrator director, right? The real orchestrator director, [3:16] the system which is responsible for making sure we have the 100% right decision on production, [3:26] right? Your orchestration layer is taking care of all the governance. It validates the output, [3:33] right? It makes sure it backed by the evidence. So, whatever response you are getting, [3:39] it has to be backed by evidence, right? And it's make and we need to also make sure [3:45] the response from the LLM model is deterministic, right? It's very very important because [3:51] LLM by default is probabilistic. We all know. Right? It's not going to be 100% deterministic [3:59] and we cannot put a probabilistic system on production which doesn't give you the [4:04] right information and doesn't make the right decision. Right? If I put the probabilistic [4:10] system on tier one supply chain application on production and it might make a mistake and we [4:16] might lose millions of dollar within few seconds. So, that's very risky. Right? So, what does the [4:25] this missing layer does other than what I have already told you? It makes sure we are [4:30] making the right operational decisions. If I talk about this specifically supply chain [4:36] examples, right? So, let's talk about uh let's say I have a customer, right? [4:43] And I promised the customer, you are going to get your package in 2 days. Right? And then let's say [4:52] we got the package, everything is fine. The whole supply chain worked end to end and now the package [4:59] is in the truck and the driver is going to deliver to the customer address. Right? [5:05] During that day, we had five driver drivers. Everyone is like going to their planned routes and [5:12] customer X had a product and driver one but they the truck broke. Now, what do you do? [5:22] Right? I told customer, you are going to get your product at 3:00 p.m. It's a very large heavy sofa. [5:29] Right? Customer might have taken half day or made some adjustment in their personal life [5:35] and they are sitting at home waiting for their sofa thinking that they are going to get the [5:40] sofa at 3:00 p.m. But I have problem with the truck, right? So, let's say we don't have any [5:45] AI right now. Right? What that driver is going to do? The driver is going to call somebody manually, [5:51] right? And we need a person, human, who decide like, "Okay, oh my god, this [5:57] truck has like 100 boxes, and now I need to figure it out like which other driver is available, [6:03] which other truck is available on the spot, right?" And the human need to make a decision, [6:09] um either they are going to contact some other third-party systems who provide these kind of [6:14] services, right? And uh the human also need to make a decision how expensive is that, right? [6:20] They might have like 10 different solutions, but they also need to see uh which one is cheaper, [6:24] which one is reliable, is going to get the product on time at 3:00 p.m., right? So, there are so many [6:30] manual processes. Now, think about think it about like you have a pure agentic system um in that [6:36] place. We have this orchestrator layer already, which is taking care of all the governance, [6:42] all the policies, all the regulations, everything is all set, perfect world, and then agentic [6:48] solution automatically figure it out uh which is the next service provider or driver is available, [6:56] and your agent automatically decide based on the cost and threshold like which is the reliable, [7:03] and we are also not paying a lot of money to that company or driver, whoever is providing [7:08] that service, right? So, it automatically figures it out, find out the driver, connect both driver, [7:13] they both driver get connected, the the stuff from truck A to truck B gets transferred, and [7:19] customer gets product on time, right? So, why I'm sha- sharing this? There are so many use cases, [7:27] uh not only in supply chain, in other enterprise solutions, right? Which can be solved through [7:33] agentic systems, right? Where we can reduce the human friction as we we have as of now. Like, [7:39] there are so many manual touch points in supply chain, there are so many manual touch points [7:44] in health care finance and banking, right? Uh So this is where uh the orchestrator help us. [7:52] Well, um let's now quickly talk about this uh orchestrator. What's the role of the orchestrator? [7:58] Orchestrator need to make sure um we have proper plan, how we are going to execute [8:06] the agentic workflow end-to-end, right? And it's going to route the request to the right model. [8:14] When I say model, it can be LLM model, or it can be non model non LLM models also. Those are like [8:21] your deterministic systems, right? Which we already have. All of us are using API, right? [8:26] Not every request need to go to LLM, right? So your orchestrator decide like do I really need [8:32] an LLM for this request, for this problem? If we don't need, we go to the traditional route, we use [8:38] the API events, whatever this, we use them without LLM. And wherever we require reasoning, only those [8:45] requests goes to the models. And we also decide like do I need to use small language model? [8:51] Do I need to use cheaper model? Or do I need to use expensive model based on the problem whatever [8:57] we are solving, right? It also validates all the inputs and outputs, right? It also observe. [9:04] This is very very important topic. The role of the orchestrator is making sure it is able to observe [9:11] the whole pipeline end-to-end. What is happening in the whole agentic system, right? Because [9:18] we need evidence for every touch point. There is no way we are going to have the agentic solution [9:24] running in production without any logging and without any tracking. Why? Because when something [9:30] break, um if you are losing money, your executive board is going to ask you question what happened. [9:36] Where we went wrong. [clears throat] Right? So, that's your observability. Um whatever [9:42] it requires to escalate to human, right? So, not every decision is for agent, right? Every company [9:49] is different, every problem is different. So, we are not saying we are not automating everything. [9:54] You might be uh in the earlier phase of automation or maybe some companies or some systems are in the [10:00] uh higher phase of the automation. So, based on that, wherever you are, uh we are going to [10:04] escalate to human um and the human gets into the picture and uh solves that problem. Cool. [10:11] I'm not going to repeat this. These are the same thing with that we just talked about. [10:15] This is the one of the interesting slide. Uh let's talk about the real example. [10:20] Right? Um as I said, I'm from supply chain, so I'm going to talk a lot about supply chain, but um [10:27] you can use the same framework, same methodology in any enterprise problem. [10:34] Right? Why? Because the software architecture, the design, and the principle is same everywhere. [10:43] I don't see any major difference. It's the same coding, same orchestrator. The way you [10:48] are designing the orchestrator for supply chain, it will be the same orchestrator everywhere. [10:53] Maybe the policies and rules, regulations, and maybe you have to build something [10:58] additional if you are working on the regulated industry. Other than that, [11:02] you can use the same base everywhere in every industry. [clears throat] So, let's talk about the [11:07] uh reverse logistic return example. Look, what um uh let's say we have an e-commerce business. [11:14] Um I bought something, I didn't like it. Um I want to return it. Right? [11:19] So, we return we start the return process. Um it can be the whole order, uh combination of multiple [11:25] items, or it can be just one item, right? Then, uh let's say we have this end-to-end return process, [11:32] and on top of that if you see the uh the this section we have the orchestrator director. [11:39] What orchestrator director is doing here it's it is orchestrating the whole return workflow [11:46] since it start from the customer side and until the return is completed finally, right? [11:51] So the next step is your eligibility policy. So this is the example of the policies [11:58] that we have for any systems, any companies, right? So we are talking about return. [12:03] In return we have the return eligibility, right? So when I say return eligibility lies, okay, [12:09] is this customer returning within 30 days because we have 30-day return policy, right? Is this item [12:14] returnable, right? So all all the policies, all the rules that you want to put into place [12:22] that goes here and your orchestrator make sure everything goes through this layer and if it is [12:30] not eligible, it is going to reject it. Right? So you can use this policy example in any other [12:37] system as well. Because every system should have the policies, right? And the right configuration, [12:44] what is allowed, what is not allowed. Cool. Let's say we pass through the policies. All policies [12:51] looks good. I'm going to continue processing. Next step is we are going to optimize for cost, [12:58] right? Um all of us know when we call to the e-commerce company online, sometime they say, [13:07] "Okay, you didn't like the product, you can keep it and we'll give you 100% return, right?" [13:13] Why is that? Right? Because they are thinking about the cost and also customer lifetime value, [13:21] right? Um we'll talk about scam separately, but let's focus on non-scam right now. [13:26] Uh so I'm going to this orchestrator is going to take care of about the cost like, okay, [13:33] um, is this worth getting this product back? How much money I'm going to spend on it, right? [13:38] If it is not worth, I'm going to tell customer, okay, you can keep it or we can have some other [13:43] rules and policies and we'll say customer like, okay, you take 20% 30% off and you can keep the [13:49] product. If customer is happy, customer will say and we'll close the discussion. That's all. If no, [13:56] then we'll start the recovery, um, and we'll define what is the right way to get that product. [14:03] Right? What is the most cost-efficient way to get that product back, [14:07] right? So that's another rule. [clears throat] Right. Um, then, [14:12] uh, we need to make sure once that orchestrator decide the the eligibility is passed, the routing [14:19] optimization cost and recovery is decided, we need to decide the destination for the product, right? [14:25] What is the closest destination facility which can accept this product? Right? So that has to be [14:34] close up the customer. I don't want to return an item from East Coast and the customer is sitting [14:40] in East Coast and have a facility return facility in California. That's too expensive for me. [14:45] Right? So we need to decide. [clears throat] Cool. Um, so that's the outcome of the orchestrator, [14:52] right? The role of the orchestrator here here orchestrator is making sure all the policies, [14:59] all the business decisions, all the operational decisions has been passed through this [15:04] and then it automatically decide whether we should refund, not refund, right? How much we [15:10] should refund? And it's also going to take care of cost automatically and will decide the destination [15:16] automatically. So everything all whole end-to-end pipeline automatic. Right? Uh, now we can say [15:24] I don't need AI for this and you are right. Right? But how AI is making how AI is helping here is [15:32] making making sure we have the smart decision when you have so many variables in combination. [15:40] Right? When you have all of these variable into combination and then [15:43] you get the real value, real ROI out of it and then you make sure [15:48] like I can process everything end-to-end automated and we don't need to involve human anywhere. [15:55] Like nobody need to call a human and say I'm going to return this product. [15:59] Cool. Um another example. This is very interesting and this is very close to my heart, right? Uh [16:06] my background is engineer, so I care a lot about root cause analysis. When something breaks, my [16:13] engineers has to spend a lot of time figuring it out. What is the root cause? What is the problem? [16:20] How I'm going to fix it? Right? So how do we solve that problem? [clears throat] So you have the [16:25] root cause analysis director, right? Which is collecting all the data from logs, stack traces, [16:33] deployments, your GitHub, Jira, all the systems that you use for your engineering workflows [16:40] is going to get all the data from that. Right? And that it going to try to find out [16:46] what is the actual root cause, what is the blast radius, how big is the problem. [16:51] Right? And it's going to validate the key hypothesis and will suggest fixes automatically. [16:59] Will come up with the recommendations like these are the the recommendation and this is how you [17:03] are going to fix it. And it might be able to generate a code for you. Like here's the problem, [17:10] here's the solution, I know how to fix it, here's the code for you. The code base is ready and now [17:16] that goes to engineer and engineer can just review the code and deploy it to production. Right. [17:24] If we are more confident as we continue working on this process, as we are making our orchestrator [17:31] director more and more mature, right? And we reached up to the point where certain cases [17:38] doesn't require human review, we can create the self-healing systems. [17:45] Right? When I say the self-healing systems, means all of these processes for root cause [17:49] analysis happening automatically. Your director is responsible for observability, logging, tracking, [17:55] everything. We also have the LLM judges in the back end, right? When I say LLM judges, [18:01] the the role of the LLM judges is um making sure they are validating every response of the LLM, [18:10] and making sure that response is deterministic, and that's response is correct and valid, right? [18:16] Once we reach to that level, we can say certain amount of cases we can fully automate it, [18:22] and we can have the system which is self-healing, and we don't require to involve human in the loop, [18:27] right? The human is only important for the most critical things, right? Where the [18:34] the problem is too complex, right? And your RCA director is not able to resolve it, [18:41] not able to orchestrate through this workflow, then it goes to human, right? [18:46] I'm not saying here we don't need human, and we are replacing human, right? That's not the problem [18:51] here. Think it Think it about the engineers' life. We have engineers working on tier one systems, so [18:58] much under pressure. They have to deliver on time. They need to make sure they keep the lights on for [19:04] all the production systems. Some of the engineers, some of the companies have 24/7 on-call support. [19:11] If anything breaks, they need to wake up at 2:00 a.m. [19:15] I don't want my engineer to wake up every week 2:00 a.m., right? So, that's why we need this. [19:24] Some of the operating principle that we need to make sure when we are designing this orchestrator [19:29] layer and we are designing the RCA director or any use case director for your company, [19:35] we have to make sure we have the proper contract defined. What is allowed? [19:42] Validating each and every input. Validating all the output and making sure we have the proper [19:48] contract defined, right? [clears throat] Policies. We need to make sure all the policies are written. [19:56] Right? We have as of today in any organization, we have so much of organizational knowledge [20:02] which is not written anywhere and it's in people's head. So, we need to make sure [20:08] we have the policy documented. Observability, we talked about it. We need to make sure we have an [20:13] end-to-end observability logging tracking so we can track everything. Um human gates. [20:19] For any high risk actions, we need to make sure we have human gates and the system automatically [20:24] escalate to human, human get involved into it, right? And the last thing is the is the a- [20:30] auditability, right? We should be able to audit every touch point. If something breaks, if [20:37] something is not working in the system, I should be able to go back and figure it out why it break. [20:43] Um I'm on time so I'm going to rush through this quickly. Um this is your uh 90-day plan. Again, [20:52] this is not the fixed timeline. You can reduce it based on your problem, but I'm taking and like [20:57] really large enterprise level problems, right? So, on your first 30 day, make sure you define at [21:03] least one workflow. Just one workflow. Right? And you need to make sure you write down everything, [21:10] all the checks, all the e-vals, all the rules, configuration, policies, everything, right? And [21:16] in next 30 days, you start testing that system. Make sure that one workflow work end to end [21:25] perfectly. Once you have that, then you can replicate that system, that workflow, [21:31] keep on building on another workflow, another workflow, another workflow, [21:35] right? And that's how you take your enterprise AI to like fully authentic AI. Cool. So, that's all. [21:41] This is Nirmal Chingar again. Um would love to connect on uh LinkedIn. Again, scan this QR code. [21:48] I would leave it here for a second. Uh but thank you so much for listening to me and hanging around [21:54] even after 4:30. I will be outside this door if anyone wants to talk to me. Thank you so much.