JPMorgan, Wayfair AI Executives Mandate Kill Switches, Hard API Rules

摩根大通与Wayfair AI高管要求推行紧急关停机制与硬性API规则 企业AI部署之所以未能产生实质性ROI，是因为企业在部署未经管理的智能体时，未将安全性、Token限制和确定性控制直接嵌入软件SDK中。 来自Wayfair、Actualize AI、DigitalNet.ai、摩根大通（JPMorgan Chase）和M Ventures的行业领袖探讨了编排层在现代企业AI部署中的关键作用。他们详细介绍了任务路由、动态模型选择、成本治理以及硬编码安全护栏等实用技术，以维持系统的可靠性与控制力。专家组还深入探讨了多智能体设计模式、自研与采购编排系统的权衡取舍，以及弥合个人生产力提升与企业真正ROI之间差距的策略。 企业 AI 战略重心已从早期的试验试点全面转向生产落地，核心在于通过集中解耦的编排控制层实现全流程信任验证、权限控制与运行时的动态模型调度。 (0:04) 处理金融与供应链等高风险业务时，系统应优先采用确定性解析与轻量分类器，仅在必要时调用昂贵的多模态模型，以防止数值幻觉并平衡 Token 支出与业务风险。 (1:11) 自主 Agent 最适用于无法预先穷举规则的非线性任务；团队可在沙盒环境中观察其行为并提炼为确定性规则，同时须将治理护栏和紧急终止开关直接硬编码至 SDK 和 API 中。 (21:29) 行业调查显示 87% 的数字化员工因使用 AI 每周平均节省 11 小时，但仅 13% 的企业切实实现了业务回报；企业需紧扣成本节省与收入增长等硬性财务指标以弥合价值鸿沟。 (40:03)

As enterprise AI shifts from initial pilots to production environments, engineering leaders emphasize that orchestration layers must enforce deterministic controls, dynamic cost governance, and programmatic safety to convert raw technical capabilities into verifiable business returns. Deterministic Routing and Selective Agent Autonomy (0:04) Panelists argue that reliable orchestration prioritizes deterministic logic over generative models. Nirmal Jingar of Wayfair stresses that non-AI microservices and legacy APIs should always execute tasks first, invoking language models strictly when natural language synthesis or complex reasoning is required. To prevent hallucinations in specialized domains, systems isolate structured data pipelines before engaging generative capabilities. Ajay Swamy illustrates this through FundLens.ai, where financial balance sheets and tables are processed using deterministic parsers, reserving small classifier models and multimodal vision endpoints for complex layouts. Swamy reports this pipeline achieves an accuracy rate exceeding 95% when identifying Quality of Earnings discrepancies. Autonomous agent workflows are deployed only when deterministic decision trees fail to cover edge cases, such as uncovering unrecorded balance sheet obligations. In contrast, Allen Badeau of DigitalNet.ai details the use of dynamic multi-agent planners, like his platform's Zeus orchestrator, which profiles dynamic skill sets to assign non-linear tasks across 10 to 15 coordinated sub-agents. Programmatic Guardrails and Shadow AI Prevention (1:17) Enterprise governance must move beyond passive documentation to technical enforcement embedded directly inside software-defined SDKs, APIs, and role-based access controls. Swamy notes that without automated, low-friction integration paths, employees bypass security frameworks and deploy untracked 'Shadow AI' across enterprise environments. Automated control planes must include an enterprise kill switch capable of instantly terminating runaway or non-compliant autonomous processes. Jingar compares contemporary AI governance to secure software development, arguing that developers must write safety and compliance logic into application code from inception rather than delegating oversight to external security teams. Additionally, panels stress that full auditing visibility—including state-change tracking and execution replayability—is necessary to diagnose and correct anomalous model behaviors before autonomous actions take effect in live production workflows. Dynamic Tokenomics and Model Portability Abstraction (1:11) Managing model economics requires real-time runtime selection rather than static hardcoding during development. Jingar notes that while runtime cost optimization is critical, it must be weighed against business impact; paying premium token fees for a frontier model is justified when preserving critical consumer guarantees, such as two-day logistics deliveries. Swamy reports that API and token expenditures on FundLens.ai run between $2.30 and $5.60 per private equity deal, demonstrating that viable operations depend on predictable per-unit unit economics. To counter ecosystem lock-in from frontier providers, Rafi Khardalian and Badeau advocate for centralized, decoupled orchestration layers that ensemble outputs across dozens of model endpoints via standard APIs. Jingar proposes establishing an internal 'Organizational Layer' to act as contextual middleware. This broker encodes institutional knowledge and translates intents before routing queries, allowing enterprises to swap underlying model providers without rewriting core prompts or enterprise application code. The Enterprise Value Gap and Build Disciplines A critical tension exists between widespread individual tool adoption and measurable enterprise return on investment. Moderator VJ Narasimhan cites a Work AI Institute survey of 6,000 digital workers conducted by Glean, which revealed that while 87% of employees use AI—saving an average of 11 hours per week (~25% of their time)—only 13% of organizations report realizing tangible business value. To bridge this divide, Swamy describes how JPMorgan Chase links generative AI projects to top-down return metrics—specifically tracking cost reduction, cost avoidance, and revenue gains on a quarterly basis. Panelists also warn against the psychological trap of building custom orchestration tools internally merely because AI code assistants make development faster. Instead, Khardalian and Swamy advise teams to buy or use open-source frameworks for standard scaffolding, avoiding custom model training that risks obsolescence within months, and building proprietary software only where it yields unique competitive differentiation. While AI tooling has dramatically shortened engineering cycles—reducing tasks that took three months to a single week—the panelists conclude that technology speed outpaces operational absorption. Long-term enterprise success hinges on objective financial tracking and disciplined use-case selection rather than deploying AI for vanity.

在Ai4专题研讨中，来自Wayfair、Actualize AI、DigitalNet.ai及JPMorgan与FundLens的技术领袖指出，企业AI正从概念验证转向生产级落地，其关键在于通过编排层实现模型路由、确定性控制与成本治理；同时需通过代码化规则与严格的量化指标，弥合员工效率提升与企业实际商业价值之间的脱节。 确定性优先与多模态动态任务路由 (0:04) 多位专家指出，在企业生产环境中，编排层必须优先调用传统API与确定性代码，而非无差别依赖大语言模型（LLM）。Wayfair高级技术主管Nirmal Jingar强调，微服务和传统系统应当拥有最高优先级，只有在严格需要推理或自然语言生成的环节才引入LLM，并应借助基于LLM的评估裁决器（LLM judges）来验证输出，使结果逼近确定性系统的可靠标准。 FundLens.ai创始人兼JPMorgan高级总监Ajay Swamy以金融文档摄入管线为例说明了分层路由架构：对于复杂的PDF、路演PPT与Excel表格，系统先由确定性解析器与轻量级分类模型处理基础结构与固定表格，对财务数据执行确定性计算以根除幻觉；仅在评估指标或文档复杂度触发阈值时，才升级调用多模态视觉模型。该机制使得收益质量（QofE）等核心指标的跨数据源匹配准确率超过95%。 运行时成本优化与集中解耦控制 (1:11) 随着行业从早期的资源补贴转向严格的按Token计量计费，控制运行成本成为编排层的核心诉求。Actualize AI首席执行官Rafi Khardalian指出，企业必须将模型选择逻辑与业务应用解耦并集中到控制层，以便在无需重构业务代码的情况下快速平替底层模型、接入低成本开源权重或根据性价比阈值即时调优。Ajay Swamy披露其实际基准数据指出，处理单笔私募股权交易的API与Token支出稳定在2.30美元至5.60美元区间，在此基准下模型可动态替换但计算准确率底线不可牺牲。 Nirmal Jingar补充表示，模型选型需在运行时动态完成，且成本控制不能脱离商业影响孤立审视；例如在关乎两天送达时效承诺的供应链决策中，调用高成本顶级前沿模型是合理的。DigitalNet.ai首席AI官Allen Badeau提出了“最高编排器”（Supreme Orchestrator）方案，通过确定性控制平面依循既定章程统筹智能体团队，把人机协同设计为独立执行节点，同时利用模型集成（Ensemble）与覆盖上百个端点的统一抽象API平抑Token成本突增风险。 代理自治边界与状态可重放性 (21:29) 关于智能体（Agent）的适用场景，研讨嘉宾界定了明确边界：仅当流程无法被预先穷举为确定性决策树时，才应交付具备探索能力的自主智能体。Ajay Swamy指出，当财务数据中出现未入账负债等异常时，FundLens.ai会派出智能体自主抓取监管报表之外的外部新闻与脚注以构建解释框架。Allen Badeau介绍称，其编排器“Zeus”通过量化Myers-Briggs性格与领导力特征，动态组建由10至15个差异化智能体构成的非线性协同方案，以应对实验偏差等意外输出。 然而自治系统必须纳入严密监管。Rafi Khardalian倡导混合演进路径，建议先在沙盒环境中观察智能体的动态决策，再将其可预测的规律固化为确定性代码。Nirmal Jingar则强调全流程可观测性的必要性，企业必须建立完整的审计日志，以支持异常发生时对执行步骤与系统状态进行精确的反向追溯与事件重放（Replay）。 代码化治理防范影子AI蔓延 (1:17) 针对企业内部自发式创新引发的风险，被动的安全手册和繁琐的手动审批已难以为继。M Ventures技术投资总监VJ Narasimhan提到，企业内部沙盒曾迅速衍生出超过900个用户自建的微型智能体。对此，Rafi Khardalian与Ajay Swamy强调，清晰的审查协议是提升组织敏捷度的前提；治理机制必须直接硬编码至软件开发工具包（SDK）、系统API与角色访问权限（RBAC）中，并配置能够即时终止失控进程的企业级“紧急开关”（Kill switch），以防员工因审批壁垒而转向不可控的“影子AI”（Shadow AI）。 Nirmal Jingar提出应当将软件工程传统的代码安全规范前置到AI开发首日，并在企业内部构建“组织层”（Organizational Layer）。该中间层汇聚了会议讨论、暗默知识与跨部门规则等上下文，在请求分发至底层模型前先行完成语义解析与规则校验，从而有效化解前沿模型厂商通过专业套件引发的生态锁定风险。 自研外购陷阱与企业价值鸿沟 (34:32) 在架构选型上，专家警示了“因AI能辅助写代码便盲目自研”的认知陷阱。Ajay Swamy指出，从零微调或训练定制模型成本过高且极易在6个月内遭技术迭代淘汰，应采取“自研与外购并举”（Build and Buy）的策略，在成熟云原生底座（如AWS）上构筑轻量编排层。Rafi Khardalian建议仅将工程资源倾注在具备企业独特竞争力的核心逻辑上，通用编排架构应优先采用开源或商业方案。 研讨最后剖析了Rebecca Hynes在Glean/Work AI Institute开展的6000名数字化员工调研所揭示的鸿沟：87%的员工使用AI且人均每周节省约11小时（约占工作时间25%），但仅有13%的企业组织报告获得了可体现在财务底线的实际商业价值。Ajay Swamy介绍称，JPMorgan要求对投行、零售与财富管理等业务场景建立自上而下的量化ROI追踪，按季度严格监控成本规避、成本削减或获客增益等财务指标，杜绝用于品牌公关的虚荣项目；专家一致认为，在工程开发周期已被AI工具从3个月压缩至1周的当下，弥合鸿沟的关键在于业务端能否有效跟进并承接高价值业务流。 本场研讨表明，编排层的成熟标志着企业AI正从概念验证期的技术狂热转向以确定性、解耦架构与严格审计为特征的工程化管理阶段。然而，87%的员工效率提升与仅13%的企业价值转化之间的显著脱节，说明单纯的技术工具普及并不等同于商业成效；如何避免将模型当作单纯的搜索增强工具，并在受控治理框架下让工程交付与核心财务ROI精准对齐，仍是企业全面落地AI的根本挑战。

## Background 该视频由 Ai4 频道于 2026 年 8 月 22 日发布在 YouTube 上。内容为 M Ventures 科技投资总监 Vijay Narasimhan 主持，汇聚了来自 Wayfair、Actualyze AI、摩根大通（JPMorganChase/FundLens.ai）及 DigitalNet.ai 的技术领袖圆桌论坛。圆桌围绕企业级 AI 编排层（Orchestration Layer）展开，探讨其如何作为核心控制平面来协调模型、工具与工作流，涵盖任务动态路由、状态管理、护栏治理以及自研与外购权衡等关键议题。 ## TL;DR 论坛深入探讨了企业 AI 从概念验证（PoC）转向生产级信任、确定性决策与 Token 成本管控的范式演进。与会专家指出，编排层的核心职责在于将通用大模型调用解耦，推行“确定性与 API 优先”以及轻量专精小模型的运行时动态路由，同时通过代码和 SDK 内嵌治理以根治“影子 AI”。针对 Agent 的落地边界，嘉宾明确提出 Agent 本质上是非线性探索的部署机制，可穷举的业务应坚决采用传统确定性代码，并通过构建企业组织上下文层避免单一供应商生态锁定。最后，面对“个人效率提升与企业回报感知严重脱节”的生产力悖论，讨论强调自研应严格聚焦于业务核心差异化，自顶向下绑定真实的财务 ROI 与业务影响力。 ## Atmosphere & style 整场研讨氛围务实、严谨且极具一线工程质感，彻底褪去了大模型初期的盲目狂热，充满了对架构复杂性与商业价值转化的冷静审视。主持人的提问直击要害，嘉宾们在坦诚分享真实数据（如单笔交易推理成本、多模型分流策略）的同时，不乏“自砸饭碗”、“工程师做太快等业务”等幽默犀利的行业洞见。多方在集中式解耦还是松散编排、全自主 Agent 还是严格确定性规则等架构哲学上的激烈碰撞，让现场洋溢着极高密度的实战智识与思辨张力。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=4s) 企业 AI 战略正从 PoC 实验阶段转向生产级控制与信任验证，核心诉求在于确认 AI 是否能直接做出可靠的业务决策，而非仅仅执行逻辑推理。 - [00:04](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=4s) 随着 AI 合同全面转向按 Token 与使用量计费，企业需要建立全局治理底线，包括跨组织追踪 Agent 清单、调用路径以及随时可切断故障调用的熔断开关（Kill Switch）。 - [04:32](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=272s) 在处理非结构化数据时切忌全量依赖大语言模型，应采用多阶段分级解析流水线：结构化表格走确定性解析，文档由轻量模型分类，仅在复杂图表时升级至高算力视觉模型。 - [10:02](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=602s) 模型选型不能在编译或设计阶段硬编码，编排控制层必须在“运行时”根据任务难度、延迟、上下文长度与成本预算进行动态路由分流。 - [11:41](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=701s) Token 经济学不能脱离业务价值孤立看待，必须将调用成本与业务履约风险相结合，避免因盲目使用廉价模型导致严重业务损失。 - [17:11](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1031s) 企业 AI 治理无法依靠纸面政策推行，必须将其内嵌到 SDK、API、角色权限（IAM）与网络控制等工程基础设施中，以杜绝“影子 AI”蔓延。 - [23:37](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1417s) Agent 的本质是一种部署机制，若业务逻辑可通过流程图或时序图清晰穷举，应直接采用确定性代码构建；只有在无法穷举路径且需要自主探索时才调度 Agent。 - [28:45](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1725s) 为了避免被特定前沿大模型厂商生态锁定，企业应当构建独立于 LLM 的组织上下文层（Organizational Layer），将底层模型降级为纯粹的无状态推理引擎。 - [35:21](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2121s) 自研与外购的黄金法则是“只为你业务中最具差异化价值的核心用例写代码”，通用基础设施和样板组件应积极购买或采用开源方案，并保持编排层的轻量与松散。 - [40:03](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2403s) 企业正面临“87% 员工使用 AI 提效 25%，但仅 13% 企业看到实际商业价值”的生产力悖论，必须自顶向下将 AI 投入与避免成本、降低支出或增量营收等硬性财务指标严格绑定。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=4s) 圆桌开场：从试点探索到生产级信任与治理的范式转移 会议由 M Ventures 科技投资与合伙人业务总监 Vijay Narasimhan 主持。Vijay 开场指出，编排层（Orchestration Layer）已经成为企业级 AI 落地的核心控制平面，在座专家既包括向客户提供编排架构服务的服务商，也包括在大型金融与零售机构内部亲自构建落地的工程负责人。他向全场抛出首个核心议题：在当下，各企业 AI 战略议程中最首要的任务是什么？与 12 个月前相比发生了哪些本质变化？ ```mermaid flowchart TD accTitle: AI Strategic Evolution from Pilot to Production Control subgraph S1["12 个月前：实验与探索"] direction TB A1["PoC 概念验证 / 实验项目"] A2["默认全量调用 Frontier 前沿模型"] A3["供应商早期补贴的 Token 经济"] end subgraph S2["当前阶段：生产、控制与确定性"] direction TB B1["生产级业务决策与信任验证"] B2["运行时动态路由与小模型专精化"] B3["按量计费管控、安全审计与熔断机制 (Kill Switch)"] end S1 ==> S2 ``` 来自 Wayfair 的资深技术工程负责人 Nirmal Jingar 率先发言，他掌管着 Wayfair 物流与供应链的大量 Tier-1 核心系统。Nirmal 指出，过去一到两年全行业的焦点是从“试点走向生产”（pilot to production），而现在的诉求已经升级为：不仅要在生产环境中上线，更要证明“我能否在生产环境中真正信任 AI？AI 是否能够直接做出业务决策，而不仅仅是做逻辑推理”（"Show me whether I can trust the AI in production and can AI make decision on production and not just reason"）。 随后，刚在当周推出新公司的 Actualyze AI 联合创始人兼 CEO Rafi Khardalian 从商业与经济学角度补充，AI 的使用成本结构正在发生急剧转变——行业正在告别大模型早期由供应商补贴代币的阶段，全面进入“按实际使用量与 Token 计费”（per use / billed by token）的商业合同阶段。伴随着企业规模化落地的加速，信息安全（InfoSec）与网络安全维度的审视变得前所未有的严苛。DigitalNet.ai 首席 AI 官（CAIO）Allen Badeau 则强调，其核心聚焦在终端用户与 AI 的交互界面上，致力于提供完全可信、具备全面问责性（accountable）并能与用户协同持续进化的解决方案。摩根大通（JPMorganChase）GenAI 平台与治理资深执行总监、同时也是 FundLens.ai 创始人的 Ajay Swamy 总结道，今年的企业头等大事是“全局控制”：摸清整个企业内到底在使用哪些 Agent、调用了什么模型、底层的 Token 经济账如何计算，以及至关重要的——当系统偏离预期时，如何确保有一套随时可触发的“紧急熔断机制”（"make sure that I have a kill switch engaged when something goes awry"）。 - **战略阶段演进**：从单纯的 PoC 验证与推理测试，转向生产环境下的自动决策授权与信任验证。 - **Token 经济学转型**：企业合同全面转向按 Token 计费，促使成本管控与投资回报率（ROI）计算进入深水区。 - **治理与安全底线**：企业需要跨组织掌控 Agent 清单、模型调用路径以及具备随时切断故障调用的熔断开关（Kill Switch）。 --- ### [04:32](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=272s) 实战工作流：FundLens.ai 的多阶段文档解析与确定性保障 主持人 Vijay 引导讨论进入具体业务场景，邀请嘉宾分享各机构内部依赖编排层实现的核心工作流。Ajay Swamy 以其创立的私募股权与股票投研平台 FundLens.ai 为例，深入拆解了高价值金融文档的自动化摄取与解析流水线。在私募股权（PE）尽职调查中，系统需要处理海量高度异构的非结构化与半结构化文件，包括 PDF、PowerPoint 以及密集的 Excel 财务底稿。 ```mermaid flowchart LR accTitle: FundLens.ai Multi-Stage Document Parsing and Routing Pipeline subgraph Ingest["文档输入"] D1["异构输入: PDF / PPT / Excel"] end subgraph Routing["编排与分流决策"] direction TB R1{"文档类型判断"} R2["确定性解析器 (Deterministic Parser)"] R3["轻量小模型 (Small LLM) 进行结构分类"] R4["版面感知解析器 (Layout-Aware Parsers)"] R5["高阶 Vision 视觉大模型"] end subgraph Output["业务验证与输出"] direction TB O1["对比 Golden Corpus 黄金评估语料"] O2["95%+ 确定性抽取精度"] O3["收益质量 (QoE) 校验与红旗预警"] end D1 --> R1 R1 -- "Excel 结构化表格" --> R2 R1 -- "PDF / PPT 文档" --> R3 R3 --> R4 R4 -- "版面评估难度上升" --> R5 R2 & R5 --> O1 O1 --> O2 --> O3 ``` Ajay 强调，编排层的首要法则在于“切忌把所有东西一股脑扔给通用大语言模型（LLM）”。对于 Excel 表格，编排系统会直接将其分流至传统的确定性解析器（Deterministic Parser）；对于 PDF 或 PPT，系统首先调度轻量化小型 LLM 对文档进行分类并分析其排版结构，接着将其传送给多个版面感知（layout-aware）的专业解析器；只有当文档复杂度与解析评估指标上升、确实遇到复杂图表时，编排层才会将请求升级路由至高算力的视觉大模型（Vision Model）。整个处理流程必须持续对照预设的“黄金评估语料库”（golden corpus of eval documents）进行校验，从而确保非结构化抽取精度稳定在 95% 以上。 该工作流的核心业务产出是辅助投资人进行敏锐的洞察挖掘与风险拦截。例如在核对标的企业的“收益质量”（Quality of Earnings, QoE）指标时，同一项财务数据往往分散在管理层报告、审计底稿等多个不同文档中。通过这套确定性优先的编排链路，所有提取和计算出的数值都是绝对可靠、可追溯且杜绝了 LLM 幻觉的。系统能够自动比对多方来源的一致性，一旦发现数值冲突，立即向 PE 投资合伙人标出“红旗预警”（red flags），辅助专业团队锁定真实的数据源头（Source of Truth）。 - **分级解析流水线**：Excel 采用确定性解析；PDF/PPT 由轻量 LLM 分类后再由版面感知解析器处理；复杂图表动态升级到 Vision 模型。 - **质量基准**：依赖 Golden Corpus 自动化评测基准，将抽取准确度维持在 95% 以上。 - **杜绝财务幻觉**：确保跨文档比对的数值计算具备绝对确定性，保障并购尽职调查中的 QoE 交叉校验。 --- ### [06:26](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=386s) 编排层的架构分歧：小模型专精化、多 Agent 发现与 API 优先 围绕 Ajay 的分享，其他嘉宾展开了多维度的延伸讨论。Rafi Khardalian 指出，行业早期倾向于“无论任务大小一律采用 Frontier 前沿大模型”，但当前的显著趋势是转向针对特定任务微调的专用小模型（purpose-built, fine-tuned models）。然而，企业内部的普通业务人员往往不具备甄别各种模型特性的专业知识。因此，编排层必须承担起抽象复杂性的职责，在后台全自动完成质量、成本与业务结果之间的权衡与模型选型。 Allen Badeau 从政府与跨组织系统集成的角度提出了不同视角。在联邦机构与大型政企场景中，系统往往需要接入大量来自第三方的现成 Agent，面临极为棘手的“Agent 发现”（Agent Discovery）与互操作难题：当外部引入的 Agent 分别基于五到十种不同的底层模型甚至完全不同的方法论（如确定性逻辑与生成式 LLM 混杂）构建时，全局编排层如何统一调度并实施安全治理？特别是在网络安全（Cybersecurity）层面，编排层甚至首先需要具备“检测并识别环境中是否存在未知 Agent”的基础能力，随后才能对其进行合规管控与模型规范。 | 专家视角 | 核心所属机构 | 编排层核心关切 | 落地架构策略 | |:--- |:--- |:--- |:--- | | **Ajay Swamy** | JPMorganChase / FundLens.ai | 异构数据确定性提取、金融级防幻觉、全局熔断 | 确定性解析与小模型先行，动态升级至视觉模型，Golden Corpus 持续评测 | | **Rafi Khardalian** | Actualyze AI | Token 经济学、消除企业选型认知门槛 | 推动专精小模型微调，编排层抽象模型分流以平衡成本与产出 | | **Allen Badeau** | DigitalNet.ai | 第三方 Agent 发现、异构系统互联、政企安全治理 | 识别未知 Agent 资产，跨多供应商规范治理策略与确定性/LLM 混合调用 | | **Nirmal Jingar** | Wayfair | 供应链 API 复用、减少 LLM 依赖、输出确定性 | **API 优先原则**：能用现有 API/确定性逻辑绝不用 LLM；前置 LLM-as-a-Judge 裁决 | Nirmal Jingar 则针对响应结果的稳定性提出了“API 优先”与“前置判决”的设计哲学。他直言，以目前的技术水平，想要从纯 LLM 端获得 100% 确定性的响应依然不切实际。因此，Wayfair 在架构设计上高度强调： 1. **前置大模型裁判（LLM Judges）**：在系统返回最终响应给业务端之前，执行严密的验证与裁判逻辑，尽可能逼近确定性结果； 2. **非 AI / 确定性 API 兜底**：企业本身沉淀了大量成熟的业务 API 与传统确定性软件系统，编排层接收到请求的第一步，是优先判断是否存在现成的 API 或非 LLM 方案能满足需求，只有在不可替代的场景下才将请求下发给 LLM。 --- ### [10:02](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=602s) 编排决策演进：从设计时硬编码到运行时动态路由 针对四位嘉宾共同提及的“模型选型”与“经济性约束”，主持人 Vijay 进一步深入提问。他指出，普通个人用户使用 AI 时往往只需登录 ChatGPT 默认窗口直接输入 Prompt，而企业级编排远比这复杂得多：企业如何在编排层设计动态路由？当构建一个多级升级路径（例如先调用低成本模型，未达到阈值再升级至昂贵大模型）时，系统如何避免层层累加的试错成本，并随着时间推移让调用链路越来越高效？ Nirmal 立即回应并提出了关键架构原则：**模型选型决不能在系统设计或编译阶段（design level）硬编码写死，而必须由编排层在“运行时”（runtime）根据传入请求的动态上下文即时决策**。当请求到达编排层下方的控制层时，系统需要动态评估负载、成本、上下文长度与精度要求，以毫秒级决策决定将该次交互路由至现有业务微服务 API、垂直小模型还是前沿多模态大模型。 ### [11:41](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=701s) 成本与业务价值的权衡：为什么不能只优化 Token 支出 围绕模型调度的经济性与输出质量，嘉宾们展开了深入探讨。Nirmal Jingar 明确指出，在考量 Token 经济学和成本控制时，绝不能脱离业务实际孤立地削减开支，必须将“成本”与“业务影响力”（Business Impact）结合起来共同优化。他以电商与供应链场景举例：假设企业向客户做出了“2天内送达”的承诺，此时供应链后端的编排层在面临模型选择时，到底是调用廉价模型还是高价模型？如果使用廉价模型会导致订单延误或交付失败，那么即便前沿大模型单价更高，企业也必须毫不犹豫地为其买单。 ```mermaid flowchart TD accTitle: Model Selection Trade-off A["业务触发事件<br/>（如客户承诺 2 日内送达）"] --> B{"编排控制层评估"} B -->|"选择最廉价模型"| C["潜在准确度风险<br/>供应链延误 / 违约交付"] B -->|"选择前沿昂贵模型"| D["交付准确保障<br/>维护履约承诺与客户满意度"] C --> E["业务损失大于 Token 节约"] D --> F["Token 成本被商业价值覆盖"] ``` 随后，Ajay Swamy 透露了其所在平台的实际运行成本基线：在处理单笔交易（Per deal processed）时，推理费用最低为 2.30 美元，最高可达 5.60 美元。底层的大模型固然可以替换以降低这几美元的开销，但系统绝对无法容忍业务输出的准确性受到损害。为此，必须建立完备的意图检测（Intent Detection）、Prompt 校验与持续评测（Evals）机制。在业内普遍质疑“Token 消耗是否真能带来投资回报（ROI）”的背景下，团队必须在成本与业务成效之间找到严谨的平衡点。 - **核心考量基准**：评估大模型选择不能只看 API 标价，更要看错误输出对终端业务造成的沉没成本。 - **单笔交易处理成本范围**：Ajay Swamy 平台的实际运行数据显示，单笔交易模型成本在 $2.30 至 $5.60 之间波动。 - **保障准确性的防线**：依赖意图识别（Intent Detection）、提示词防护以及严格的评测系统（Evals），确保更换模型时不牺牲产出质量。 --- ### [13:06](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=786s) 确定性架构与“人类在环”工作流的治理实践 Allen Badeau 介绍了其生产环境中落地的确定性（Deterministic）Agent 解决方案。该架构的显著特点在于：Agent 编排并非由不可控的黑盒 LLM 自主发散驱动，而是由一个被称为“最高编排器”（Supreme Orchestrator）的系统像人类项目经理一样进行统一调度与严格控制。在这一框架内，每个 Agent 都有其独立的宪章（Constitution）、治理规则与执行层，具备极高的确定性与灵活性。 同时，这套系统把“人类在环”（Human-in-the-loop）无缝嵌入到自动化流程中。在该架构下，人类员工的地位类似于一个特定 Agent——他们同样拥有具体的交付物、提交物与职责节点，并作为工作流的一个环节被统一管理。系统会端到端监控整体流水线的人力与计算成本。Allen 幽默地指出了许多企业部署 Agent 后的常见痛点：“很多人抱怨 Agent 运行太慢、甚至瘫痪，排查后才发现任务其实是卡在某位员工的电子邮箱里无人处理”，从而引发连锁延误和预算超支。通过这种全程可见的跟踪机制，企业可以清晰掌握用户在何处花费了预算、做出了何种选择。 - **确定性编排哲学**：拒绝将调度权完全交给单一 LLM，改由类似项目经理的最高编排器按规则协调各 Agent。 - **Agent 宪章体系**：每个微智能体均拥有独立的宪法、治理准则和权限边界。 - **人类与 Agent 同构**：把人类专家抽象为具备特定输入、输出与交付职责的节点，便于量化整体流程时延与资金消耗。 --- ### [15:22](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=922s) 集中式智能模型选型与合规挑战 Rafi Khardalian 阐述了其平台在处理模型调度时的核心架构思路。他主张将模型的选择逻辑从单体应用或孤立 Agent 中剥离出来，提升为一个独立的解耦控制层（Control Plane）。该层根据 Prompt 内容实时判断任务难度，并在成本与质量容差之间动态权衡： 1. **运行时动态选择（Runtime Selection）**：根据输入上下文对多样性或确定性的要求，实时分流至合适模型。 2. **多模型与开源权重快速适配**：面对快速迭代的开源权重模型与各类托管服务商，编排层必须能够无缝切换并持续度量性价比。 ```mermaid flowchart LR accTitle: Centralized Orchestration Architecture subgraph ClientSide ["应用与代理层"] App1["业务应用 A"] App2["Agent B"] end subgraph ControlPlane ["集中式控制层（解耦治理）"] Router["运行时模型选择器<br/>（平衡成本、质量与意图）"] Metric["度量与评估引擎"] end subgraph ModelPool ["模型与基础设施提供商"] M1["前沿商业模型<br/>（高合规 / 高成本）"] M2["开源权重托管平台<br/>（Kimmy K3 / 字节等）"] end App1 --> Router App2 --> Router Router --> Metric Router --> M1 Router --> M2 ``` Ajay Swamy 随即对低成本模型提出了严厉的合规预警。尽管市场上涌现出诸如 Kimmy K3、字节跳动发布的无限 OCR（Unlimited OCR）等极其便宜的模型工具，但在强监管行业（Regulated Industries，如金融、医疗）中，这类模型目前几乎不可能通过安全合规审批。因此，受规管企业为了满足审计要求，往往必须“硬吃”前沿合规模型的高昂成本溢价。Rafi 也对此表示赞同，强调在采用开源权重模型时，托管服务商的资质信誉与模型本身的安全系数同等关键。 --- ### [17:11](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1031s) 规避“影子 AI”：用代码治理取代文本条规 讨论转向了企业内部 AI 治理的核心矛盾。主持人提到，一年前企业关注的是推动采纳与实验，但随后试点爆炸式蔓延——例如其母公司内部允许员工在“围墙花园”（Walled Garden）中自建助手，短时间内就自发涌现了超过 900 个 Agent，此时治理层若仅靠“一刀切地拒绝（Say No）”或关闭模型权限，将严重扼杀员工的创新活力。 | 治理方式 | 实施手段 | 企业后果 | 敏捷度 | |:--- |:--- |:--- |:--- | | **传统行政式治理** | 纸面政策文件（Policy Documents）、人工审批审批流 | 审批流程冗长；催生大量“影子 AI”（Shadow AI），扩散未受控的数据外泄与合规风险 | 极低 | | **工程嵌入式治理** | 统一 SDK、API 访问控制、基于角色/身份的权限管理（IAM） | 明确数据源边界与紧急拔线（Pull the plug）机制；在合规防线内快速放行（Say Yes） | 极高 | Ajay Swamy 强调了一句关键洞见：“治理绝不可能依靠政策文件来推行，你必须将治理内嵌到 SDK、API、身份与角色权限之中。”审批时间拖得越久，内部就越容易滋生规避监管的“影子 AI”（Shadow AI），将企业核心数据暴露于巨大风险中。Nirmal Jingar 补充道，这与传统的软件安全同理：就像软件工程师不会指望把所有代码安全都留给安全团队事后排查，而是从写代码第一天起就内置防护一样，AI 应用开发也必须在工程层面自底向上实现安全合规。 --- ### [21:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1294s) 突破线性流水线：为科学探索 Agent 注入人格与动态反馈 主持人结合母公司（覆盖化学材料与生物制药领域）研发“AI 联合科学家”（AI Co-scientist）的实践，提出了一个关于工作流编排的技术难题：早期的编排框架非常线性化（Agent A 的输出直接喂给 Agent B），一旦科学家遇到出乎意料的湿实验或计算结果，静态的串行流水线便无法自发调整研究路线或展开深度调查。在高度强调确定性的系统里，该如何保留足够的“探索创造力”以支持自主应变？ Allen Badeau 揭示了其最高编排器“宙斯”（Zeus）的破局机制：为每个 Agent 赋予独一无二的“人格画像”（Personalities）与“技能集特征”（Skill Sets）。系统甚至引入了类似人类职场中的量化性格测试工具，例如迈尔斯-布里格斯类型指标（Myers-Briggs / MBTI）及领导力评分。通过这些评估工具，用户可以为每一个智能体生成专属的行为风格配置。这使得整个团队由各具专长的 Agent 组成，面对不可预测的突发实验数据时，不同画像的 Agent 会表现出差异化的反应、评估与纠偏策略，从而从根本上跳出静态大模型设定的刻板反应，为非线性探索提供底层自治动力。 - **线性工作流局限**：预先固化的输入输出链条无法应对反常的科学实验结果，缺乏自主回溯与分支探索能力。 - **人格驱动架构**：每个 Agent 绑定具体的 MBTI / 领导力人格评估结果，以此塑造差异化的推理倾向与协作风格。 - **最高编排器“宙斯”（Zeus）**：根据不同 Agent 的技能配置与个性特征动态构建复杂工作流，在受控的确定性与适应性创造力之间达成平衡。 ### [23:37](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1417s) Agent 的本质：非线性工作流与部署哲学 探讨智能体（Agent）工作流时，构建过程通常极具动态性且绝非线性。面对复杂的全局目标，编排系统往往会动态衍生出 10 到 15 个专有 Agent，并为其分配配套的工具集。在大型语言模型爆发前，AI 就已经存在，但如今 Agent 的“个性设定”以及它们调用工具的具体策略，已成为决定实际工作流形态的核心驱动力。 对此，Ajay Swamy 提出了他对 Agent 的独到哲学：**Agent 纯粹是一种部署机制（Deployment Mechanism）**。如果业务逻辑可以通过流程图（Flowchart）清晰绘制，或者能用时序图（Sequence Diagram）明确表达，那就直接编写确定性的代码，根本不需要引入 Agent。Agent 真正适用的场景，是当工程师“无法穷举所有确定性路径”，且系统必须具备自主探索与自我引导能力的时候。 ```mermaid flowchart TD accTitle: 确定性逻辑与 Agent 适用边界决策图 accDescription: 展示何时应当使用传统确定性工作流，何时应调度自主 Agent subgraph Evaluation["场景评估"] A["业务需求与任务目标"] --> B{"能否穷举确定性执行路径？<br/>（绘制流程图 / 时序图）"} end subgraph Deterministic["确定性构建"] B -- "是 (可穷举)" --> C["直接构建程序化/规则工作流<br/>（无需 Agent）"] end subgraph Agentic["自主探索与回溯"] B -- "否 (不可穷举)" --> D["调度自主 Agent 探索与推理"] D --> E["跨多源数据挖掘<br/>（法规外财报附注/新闻/研报）"] E --> F["输出假设并设置边界约束"] end ``` 以金融分析平台 FundLens 为例，当系统自动识别出某家公司资产负债表上存在一笔不明原因的表外未注资债务时，分析师往往需要深入探究。此时平台会派出 Agent，主动穿透到法定披露文件之外的新闻报道、非结构化文件和深层财务附注中进行跨源检索，最终整合出该隐秘科目为何被隐藏的深度分析报告。Ajay 强调，构建 Agent 时必须保持克制与清醒，务必设立明确的边界约束，以便在系统偏离预期时能够及时制动与回撤。 --- ### [25:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1534s) 确定性与自主性的混合架构：可观测性、审计与重放 针对 Agent 与传统确定性逻辑的关系，业界达成了高度共识：绝大多数业务场景依然应当优先依赖程序化、确定性的实现方式。然而，二者之间存在强大的混合架构（Hybrid Approach）空间——团队可以先部署 Agent 去探索未知任务并观察其自主决策路径，随后依据 Agent 在复杂场景中的表现，反向沉淀并构建出高效的确定性架构。 在 Agent 的全生命周期治理中，端到端的可观测性（Observability）与可审计性（Auditability）是企业落地的生命线： - **全链路追踪（Traceability）**：必须对 Agent 的每一个思考步骤、工具调用和子任务路由提供完整的执行日志，实现向后回溯。 - **事件重放（Event Replay）**：支持对 Agentic 工作流的历史事件进行精确重放，以便定位幻觉根因或在合规检查中还原决策现场。 --- ### [27:06](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1626s) 模型供应商锁定 vs. 企业级可移植性 当讨论从工作流确定性转向“回答结果的确定性”时，企业 AI 领域正在上演一场激烈的力量博弈。头部前沿模型供应商（如 OpenAI、Anthropic 等）倾向于构建强大的生态壁垒，引导企业深度依赖其特定的交互范式与 Prompt 响应风格。现场互动调查显示，在使用多种生成式 AI 工具的与会者中，对于“面对不同模型时是否会改变 Prompt 提问方式”，受众群体呈现出势均力敌的分歧。 ```mermaid flowchart LR accTitle: 供应商生态锁定与企业可移植性的博弈 accDescription: 展现前沿模型厂商构建生态引力与企业追求模型可替换性之间的对立拉锯 subgraph Providers["模型厂商引力 (Lock-in)"] P1["专属行业应用<br/>(Claude Legal / Design)"] --> P2["特定 Prompt 响应依赖"] P2 --> P3["提高企业迁移与议价成本"] end subgraph Enterprise["企业级诉求 (Portability)"] E1["通用组织上下文层 / SDK"] --> E2["降低模型切换成本"] E2 --> E3["保持跨模型输出与指标一致性"] end Providers <-->|"商业利益博弈"| Enterprise ``` 对企业而言，最忌讳的莫过于被单一供应商深度锁定（Vendor Lock-in），从而丧失商务谈判与价格博弈的筹码；但供应商天然希望提高迁移成本。为了避免模型切换时出现“业务中断、答案偏差、原有 System Prompt 全面失效”的困境，企业在架构设计上必须建立自主掌控的解耦机制。 --- ### [28:45](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1725s) 企业组织上下文层（Organizational Layer） 针对模型切换的鲁棒性问题，Nirmal Jingar 提出了构建独立于 LLM 的**企业组织层（Organizational Layer）**的核心设想。企业级 AI 的最大痛点在于，大量至关重要的上下文信息并没有沉淀在代码或数据库中，而是零散存在于员工的大脑、日常会议记录以及两名员工偶遇时的咖啡闲聊（Coffee Chat）之中。 当前许多人幻想在短短两三个月内就能通过 Agent 让企业系统全自动、99% 确定性地运转，这完全脱离了现实。金融、医疗、供应链等大型核心系统经历了 10 到 20 年的演进，极其错综复杂，根本不具备“即插即用加个 AI 就能运转”的数字化底座。 因此，企业必须构建统一的组织上下文层，使所有发往底层模型的请求都先经过该层做上下文补全、意图识别与安全对齐。在此架构下，底层的 LLM 仅作为纯粹的计算推理引擎，企业随时可以无缝替换模型，而不会破坏上层业务逻辑。 --- ### [30:45](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1845s) 推理层商品化、模型集成（Ensemble）与 API 抽象 围绕模型层的解耦与架构选型，嘉宾们进一步展开了深度碰撞： - **警惕供应商的“垂直应用引力”**：Anthropic 等厂商不断推出如 Claude Legal、Claude Design 等开箱即用的垂直功能。这些工具在短期内看似业务加速器，实则是加深生态绑定的引力陷阱。企业应当像招聘最优秀的人才一样看待模型选型，组建多模型供应商组合（Portfolio of Providers），并持续利用最新模型进行并行测试与灰度验证。 - **多模型集成（Model Ensemble）与 Token 经济学**：Allen Badeau 指出，其系统不依赖任何单一 LLM，支持通过 URI 和 API Key 即插即用接入超过 100 种大语言模型。通过对多个模型的返回结果进行集成（Ensemble），不仅能提升答案的综合上限，更能根据任务复杂度分流，避免无脑使用最贵模型导致 Token 成本失控。 - **通用 API/SDK 与跨层级评估**：Ajay Swamy 强调，具备通用的 API 或 SDK 仅仅是基础门槛（Table Stakes）。真正的难点在于如何确保模型切换后，企业的业务输出、洞察深度和准确率依然稳定。这就需要依靠组织层的意图检测（Intent Detection）、Prompt 规范化管理以及跨模型层级的系统化评估（Eval）体系。 --- ### [34:39](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2079s) 编排层的自研与外购抉择（Build vs. Buy） 随着讨论的深入，Vijay Narasimhan 引用了当天上午吴恩达（Andrew Ng）在演讲中提到的经典命题——**自研与外购的抉择（Build vs. Buy）**。 在复杂的企业 AI 架构中，编排（Orchestration）已经演变成一个涵盖智能路由、模型选取、状态控制、策略治理及安全防护等众多环节的庞大概念。自研与外购并非简单的非此即彼（二元对立），而往往是一个根据不同功能模块进行渐进式组装的细分决策。企业如何在编排层的各个关键切面上权衡掌控力、落地速度与总体拥有成本，成为了全场关注的新焦点。 ### [35:21](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2121s) 自研还是外购：大模型时代的权衡与架构控制权 小组讨论切入了一个企业技术决策的核心矛盾：“自研还是外购”（Build vs. Buy）。台上嘉宾指出，在当前环境下，这其实不是非此即彼的选择，而是演变成了“边自研边采购”（Build and Buy）。然而，在模型层面上，自研和微调的成本极其高昂。如果企业本身不是从事前沿大模型研发的公司（Frontier Model Company），那么自己从头训练出来的模型往往“在六个月内就会被送进坟墓”（graveyard）。因此，企业绝不应该盲目自研底层基础模型。 真正的分歧在于编排与应用层——例如，当团队想要将各种组件串联起来时，是使用像 LangChain 这样的开源框架自行搭建，还是直接采购现成的商业解决方案？ ```mermaid flowchart TD accTitle: Build vs Buy Decision Framework accDescription: Decision tree for choosing between building, buying, or open source in AI enterprise architecture. subgraph Evaluation["决策评估维度"] D1{"是否构成业务的核心差异化？"} D2{"是否有工程产能与维护意愿？"} end subgraph Strategy["实施路径"] S1["【自研 Build】: 仅聚焦核心差异化业务逻辑与松散编排"] S2["【开源/外购 Buy】: 采用标准化基础设施、模型与现成工具"] S3["【渐进演进】: 先外购快速占领市场，再视资源并行自研替代"] end D1 -- "是" --> S1 D1 -- "否 (样板代码/通用能力)" --> S2 D2 -- "缺乏长期维护团队" --> S2 D2 -- "具备充沛工程产能" --> S3 ``` 针对这一抉择，来自 JPMorganChase 的 Ajay Swamy 表达了相对审慎的工程态度。他倾向于在云厂商（如 AWS）的原生基础服务（primitive services）之上构建，并采用非常“松散”（loose）的编排层。他的核心逻辑在于：编排层做得越深、越重、定制链路越长，引入的系统复杂性就呈指数级上升，后续排查安全漏洞、系统 Bug 和故障的难度就会剧增；保持编排层轻量松散，才能保留足够的灵活性和控制面（control surface）。 - **前沿模型残酷迭代**：非头部大模型厂商自建的定制模型往往在半年内便全面落后淘汰。 - **架构复杂度陷阱**：过深过长的自研编排中间件会大幅吞噬运维精力，必须保持控制面的极简与透明。 --- ### [36:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2194s) 赋能最终用户还是绑定技术团队：AI 基础设施的定位之争 DigitalNet.ai 的首席 AI 官 Allen Badeau 提出了截然不同的视角。他认为企业必须首先回答一个本质问题：“你是希望你的专业 AI 团队来做所有的集成对接，还是希望企业的实际业务终端用户（actual users）能够直接上手使用？”他笑称，自己的终极目标在很多事情上是“砸掉自己的饭碗”（put myself out of a job）。在企业日常运作中，不应该每次要做个电子表格数据分析，都必须调动专门的 AI 专家团队来编写、部署一套智能体（Agents）。 他以自身平台 Zeus 为例，指出平台应当引导用户逐步完成任务，将控制权和易用性交还给业务一线，而无需技术人员充当“中间人”。 紧接着，Actualyze AI 的首席执行官 Rafi Khardalian 指出，技术革命中历史总在押着相同的韵脚重复。当前行业正面临一个典型痛点：很多 AI 团队为了推进项目，早期搭建了庞大繁重的定制基础设施，但随着技术演进，他们日后根本不愿、也无法长期维护这套“技术债务包袱”。他的黄金准则非常明确：**只为你业务中最具差异化价值的核心用例写代码进行自研（build only what differentiates your business）；凡是通用的、样板性质的组件（boilerplate），一律购买或采用开源方案**。 Wayfair 的高级工程经理 Nirmal Jingar 对此表示完全赞同。他认为“自研还是外购”是软件工程几十年来不变的老议题，AI 没有颠覆这一商业逻辑： - 如果某个外部产品能让你快速进入市场、建立竞争壁垒并抢占市场份额，那就果断外购； - 若后续有战略需要，团队完全可以在内部并行自研，逐步替代、废弃（deprecate）原有的商业现成工具； - 决定权最终取决于业务问题本身、团队可用产能（capacity）以及工程资源分配。 此时，台上有嘉宾插话提醒了一个现代团队极易陷入的认知误区：“如今工程师往往产生一种错觉——‘我拥有强大的 AI 编程工具辅助，所以我自己什么都能造’，这成了极易陷进去的自研陷阱。”同时，低代码与“平民开发者”（citizen developers）在企业内部遍地开花，这也对 IT 部门的管控（governance）与权限合规提出了前所未有的挑战。 --- ### [40:03](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2403s) 生产力悖论：企业价值转化的巨大断层 主持人随后抛出了圆桌最后的关键议题——价值落地与“生产力鸿沟”。他分享了在大会首日参加 Glean 公司 Rebecca Hines 演讲时听到的权威数据。由 Work AI Institute 发布的《Work AI Index》调研了 6,000 名数字化办公人员： - **员工层面的极高渗透率**：高达 **87%** 的受访员工表示在日常工作中使用 AI； - **个人提效显著**：平均每人每周可节省约 **11 小时** 的工作时间（约占每周总工时的 25%）； - **组织层面的严重脱节**：仅有 **13%** 的受访企业组织真正看到了这些 AI 实践转化为实际的商业价值。 面对“个人效率飙升 25%，企业回报感知仅 13%”的巨大断层，主持人向全场提问：编排层乃至整个系统架构，究竟该如何充当桥梁，让局部的原型验证（pilot）与纵向业务线收益真正扩散为企业级价值？ | 指标项目 | 调查数据表现 | 暴露出的核心矛盾 | |:--- |:--- |:--- | | **员工 AI 使用率** | **87%** 的数字化员工在工作中使用 AI | 基层工具探索极为活跃，使用意愿普及 | | **单兵时间节省** | 平均每人每周节省 **11 小时** (~25%) | 员工单点效率获得实质性释放 | | **组织价值感知率** | 仅 **13%** 的企业看到实际商业价值产出 | **巨大的组织断层**：零散效率未映射为公司财报 ROI | --- ### [41:07](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2467s) 总结陈词：重塑 ROI、消除虚荣指标与跟上工程速度 在各自的总结陈词中，四位嘉宾从治理、预算、认知及业务协同四个角度展开了收官剖析： 首先，Ajay Swamy 结合摩根大通（JPMorganChase）的内部实践进行了拆解。在投行（Investment Banking）、零售银行（Consumer Banking）或资产与财富管理（Asset/Wealth Management）等各条业务线中，必须自顶向下（top-down）建立极度严苛的 ROI 评估模型。在决定让 AI 介入前，必须与切实的财务指标强行绑定——究竟是**避免成本（cost avoidance）**、**降低支出（cost reduction）**，还是**带来增量营收（revenue uplift）**？企业必须在具体的业务流程和获客指标中持续监控，按季度（quarterly）进行复盘与调优，唯有如此，企业才能刺破泡沫，看清真实的价值兑现情况。 Allen Badeau 随即提醒大家谨防“为了做 AI 而做 AI”的面子工程。他指出，AI 近来被捧得太高、获得了过多特殊优待，导致大家忽视了企业预算池中其他同样重要的技术选型。企业必须建立统一的创新管线管理机制，把有限的资本砸在真正能产出最高价值的刀刃上；若盲目为了对外宣发“我们正在用 AI”而跟风立项，长远来看对公司反而是毁灭性的伤害。 Rafi Khardalian 进一步将 AI 定位为企业工具箱里的一件常规工具。他观察到行业仍处于采纳的早期阶段，绝大部分基层员工依然仅仅把 ChatGPT 当作“高级版网页搜索”（glorified web search）。要打破 13% 的价值天花板，企业还需要花费大量精力在全员教育上，让组织深入理解在何种场景下动用 AI 才是真正恰当且具有生产力的。 最后发言的 Nirmal Jingar 将视角拉回现实的研发与业务脱节问题。他强调价值鸿沟不能全甩锅给编排层，底层还涉及 AI 素养（AI literacy）、模型选择意识、调用成本控制以及组织内“人均成本上限”（cap per human）的设定。他指出从战略制定第一天起，核心关注点就必须是“业务影响”（Impact），而非技术本身： > “现在的核心问题反倒出在业务侧：工程师的交付速度太快了，业务部门的演进却根本跟不上。过去需要 3 个月的项目，工程师借助 AI 工具 1 周就做完了，做完后只能坐在那里干等业务方提出下一个真正有价值的业务命题。” 圆桌在对工程师提效与业务响应错配的深度思考中步入尾声，主持人在全场掌声中宣布圆桌研讨圆满闭幕。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=4s) Show me whether I can trust the AI in production and can AI make decision on production and not just reason — Nirmal Jingar > [00:04](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=4s) make sure that I have a kill switch engaged when something goes awry — Ajay Swamy > [17:11](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1031s) 治理绝不可能依靠政策文件来推行，你必须将治理内嵌到 SDK、API、身份与角色权限之中。 — Ajay Swamy > [23:37](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=1417s) Agent 纯粹是一种部署机制（Deployment Mechanism）。如果业务逻辑可以通过流程图清晰绘制，或者能用时序图明确表达，那就直接编写确定性的代码，根本不需要引入 Agent。 — Ajay Swamy > [35:21](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2121s) 在六个月内就会被送进坟墓（graveyard） — Ajay Swamy > [36:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2194s) 你是希望你的专业 AI 团队来做所有的集成对接，还是希望企业的实际业务终端用户能够直接上手使用？ — Allen Badeau > [36:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2194s) 砸掉自己的饭碗（put myself out of a job） — Allen Badeau > [36:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2194s) 只为你业务中最具差异化价值的核心用例写代码进行自研；凡是通用的、样板性质的组件，一律购买或采用开源方案。 — Rafi Khardalian > [36:34](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2194s) 如今工程师往往产生一种错觉——‘我拥有强大的 AI 编程工具辅助，所以我自己什么都能造’，这成了极易陷进去的自研陷阱。 > [41:07](https://www.youtube.com/watch?v=RR62nq0k3HQ&t=2467s) 现在的核心问题反倒出在业务侧：工程师的交付速度太快了，业务部门的演进却根本跟不上。过去需要 3 个月的项目，工程师借助 AI 工具 1 周就做完了，做完后只能坐在那里干等业务方提出下一个真正有价值的业务命题。 — Nirmal Jingar ## Mentioned Rafi Khardalian, Actualyze AI, Nirmal Jingar, Wayfair, Vijay Narasimhan, M Ventures, Ajay Swamy, JPMorganChase, FundLens.ai, Allen Badeau, DigitalNet.ai, LangChain, Haystack, Ray Serve, ChatGPT, Kimmy K3, ByteDance, Zeus, MBTI, OpenAI, Anthropic, Claude Legal, Claude Design, Andrew Ng, AWS, Glean, Rebecca Hines, Work AI Institute ## Video description Rafi Khardalian - CEO, Actualyze AI Nirmal Jingar - Sr. Engineering Manager, Wayfair Vijay Narasimhan - Director, Technology Investments and Partnerships, M Ventures Ajay Swamy - Senior Executive Director - GenAI and Governance | Founder of FundLens.AI, JPMorganChase | FundLens.ai Allen Badeau - CAIO, DigitalNet.ai As enterprises scale their AI programs, orchestration has become the critical control plane for managing complexity. This panel will cover how orchestration frameworks handle task routing, state management, workflow sequencing, and integration with a growing ecosystem of tools. Panelists will compare approaches, from open-source frameworks like LangChain, Haystack, and Ray Serve, to enterprise-grade middleware and cloud-native orchestration services. Key themes will include workflow reliability, modularity, extensibility, and guardrail enforcement, as well as how orchestration connects AI systems with APIs, data pipelines, and business processes. Learn what capabilities matter most in choosing an orchestration strategy, and how cutting-edge platforms are evolving into the “operating system” for enterprise AI. [Source](https://www.youtube.com/watch?v=RR62nq0k3HQ)

[0:04] Nirmal Jingar: Show me whether I can trust the AI in production and can AI make decision on production and not just reason [0:04] Ajay Swamy: make sure that I have a kill switch engaged when something goes awry [17:11] Ajay Swamy: 治理绝不可能依靠政策文件来推行，你必须将治理内嵌到 SDK、API、身份与角色权限之中。 [23:37] Ajay Swamy: Agent 纯粹是一种部署机制（Deployment Mechanism）。如果业务逻辑可以通过流程图清晰绘制，或者能用时序图明确表达，那就直接编写确定性的代码，根本不需要引入 Agent。 [35:21] Ajay Swamy: 在六个月内就会被送进坟墓（graveyard） [36:34] Allen Badeau: 你是希望你的专业 AI 团队来做所有的集成对接，还是希望企业的实际业务终端用户能够直接上手使用？ [36:34] Allen Badeau: 砸掉自己的饭碗（put myself out of a job） [36:34] Rafi Khardalian: 只为你业务中最具差异化价值的核心用例写代码进行自研；凡是通用的、样板性质的组件，一律购买或采用开源方案。 [36:34] 如今工程师往往产生一种错觉——‘我拥有强大的 AI 编程工具辅助，所以我自己什么都能造’，这成了极易陷进去的自研陷阱。 [41:07] Nirmal Jingar: 现在的核心问题反倒出在业务侧：工程师的交付速度太快了，业务部门的演进却根本跟不上。过去需要 3 个月的项目，工程师借助 AI 工具 1 周就做完了，做完后只能坐在那里干等业务方提出下一个真正有价值的业务命题。

[4:00] Nirmal Jingar: pilot to production [68:30] Allen Badeau: Supreme Orchestrator [34:32] Rafi Khardalian: build only what differentiates your business [30:48] Ajay Swamy: 80% less context or guidance [34:32] Ajay Swamy: model graveyard in 6 months [39:16] Panelists: Oh, I can build it because I have AI to help me build it. [41:07] Panelists: doing AI because... so that you can get your name out there [41:07] Panelists: glorified web search

[0:04] Well, thank you everyone for coming and I  hope that you had an inspiring morning and [0:07] a couple of days at the conference so far. I'm  VJ Nera Simhan. I'm the director of technology [0:12] investments and partnerships at M Ventures  and I'm thrilled to be moderating this panel [0:16] today. I've had a lot of good conversations with  these folks over the last couple of weeks. Um, [0:21] and know that they're they're deep into this from  sort of two sides of the equation. both providing [0:25] services to clients on the orchestration  layer and implementing them themselves in [0:29] the firm. So really excited to to get this  going today. So I think we'll start and [0:34] um maybe we'll go from this end down. I'll ask  each of you to introduce yourself briefly and [0:38] the opening question is what is at the top of  your AI strategic agenda now and how is that [0:45] different than it was 12 months ago and you  can either talk about again within your own [0:50] firm or what your clients are telling  you. So yeah n go ahead. Oh, actually [0:59] hello everyone. Thank you for joining. Uh this is  Nirmal Jinger. I'm a senior technology leader in [1:05] uh Wfair um owning bunch of tier one supply  chain systems in logistic and supply chain and [1:12] uh bunch of other things. Yeah. Um so to answer  your question, I think uh what's different now [1:18] compared to last one to two years? I think uh  I moved from pilot to production and the answer [1:24] is now like not using the AI or not just like  uh using in production. Show me whether I can [1:31] trust the AI in production and can AI make  decision on production and not just reason. [1:41] Hello everyone. Uh I'm Rafi Kardallian, CEO and  co-founder of Actualize AI. We just launched [1:47] uh this week. So top of mind for for me and us  and what we're hearing from our customers is how [1:54] the economics of AI use are changing dramatically  and rapidly. Uh we're moving past the subsidized [2:02] phase of token economics into the per use phase.  You've seen a lot of that now with enterprise [2:08] deals starting to transition from including any  amount of usage to being buil by the token and [2:15] subsequently um the security angle there is is  always incredibly important. It's becoming much [2:21] more pronounced uh now that the you know the  push for adoption has changed the adoption is [2:28] is gaining momentum. So now there's a lot more  focus on the real side of this from a security [2:33] angle and in infosc. Hi everyone, I'm Alen Bedau.  I'm the chief AI officer for digital.ai and really [2:42] our primary focus is on the user interaction with  AI and trying to make sure that we're providing a [2:50] trustworthy solution. we're providing a solution  that is fully accountable and then also looking [2:55] at that relationship to see how our AI can  continue to learn with the user. Good morning [3:02] everybody. Uh my name is AJ Swami. I'm a senior  director at JP Morgan Chase where I manage Genai [3:09] platform and governance. I'm also the founder of  fund lens.ai. It's institutionalgrade research [3:15] for private equity and equity research. So um  what's top of mind this there's a few things [3:22] right like as Nurmal mentioned you know last  year was going from pilot to production this [3:27] year it's around how do I control across the  firm enterprise what agents are in use what [3:34] models are being used what is the tokconomics  behind it and how do I make sure that I have a [3:40] kill switch engaged when something goes arai so  those are the kinds of things that are top of [3:48] Yeah, really interesting. And you can see how  that spans the gamut all the way from users [3:52] interacting at that very first layer through to  the deepest levels of control at the back layer, [3:57] right? Um so I think to make this a lot more  practical it would be helpful if um each of you [4:03] could talk about a specific workflow that you're  using um in your organizations or that you've had [4:11] exposure to and you know what decisions does that  orchestration layer that you're using enable um [4:18] and what business outcome did that drive you  know this idea of uh not just going from pilot [4:22] to production but really seeing you know the  active things and um you can feel free to jump [4:27] in you know in any in any order. So AJ you look  like we're ready to go. Yeah. So so so I'll use [4:32] an example VJ from um from my startup fund lens.  So from like an orchestration perspective right [4:37] like we ingest a ton of documents private equity  lots of heterogeneous documents PDFs powerpoints [4:44] Excel spreadsheets. Now you have to be careful  not to send them all through an LLM. If it's [4:50] an Excel spreadsheet orchestrated it so it goes  through a deterministic parser. Same thing when [4:56] you have a PDF or a PowerPoint. I use a small  LLM to understand what the document is, classify [5:03] the document, understand its structure, then send  it through several different PDF layout aware uh [5:11] parsers and then eventually when we think based on  the evality of the document is rising then maybe [5:20] routed to a vision model. So start deterministic  and then route through the vision models and make [5:28] sure you always test it against your golden corpus  of eval documents. So that way you know where [5:35] the breaks can happen and you make sure you get  almost 95% plus extraction accuracy. Could could [5:42] you comment a little bit on the business outcome  there? So what are people using that workflow for? [5:46] Yeah. So so so listen this this this is around  insight generation, right? So when you're looking [5:51] at a deal in a private equity space, you want to  make sure that for example quality of earnings [5:58] that is reported across different documents,  they can all tee up and match together. So you [6:04] have to make sure the figures that are computed  and stored. They're deterministic. They're never [6:09] hallucinated by an LLM. So you can compare them  and the and then you can raise those red flags [6:15] to your P partners and say hey this number is  different from this document and this number [6:21] is different from this document. Give it a  look understand and figure out which is the [6:26] real source of truth. Cool. Yeah. So, I mean to to  sort of uh uh support some of what AJ said there, [6:36] we're seeing quite a bit of that as well. Um just  in model selection, using the appropriate models [6:41] for the task. Um to this point, it had been pretty  straightforward to say, I'll always use a frontier [6:47] model. Um but there's much more of an adoption  as well of smaller models. We're still in the [6:54] early days of that. But where we see the direction  going is more purpose-built, finely tuned models [7:00] for specific tasks and jobs. And one of the  challenges that we're seeing is well, how do [7:06] you make the decision on which model to to choose?  Not everyone is as versed as say people that are [7:12] attending this conference in understanding the  different capabilities of each model. So, how [7:17] do you make that simpler, more straightforward for  your enterprise to make those those decisions and [7:24] actually remove as much of that as you can? This  is both for quality and from a cost perspective, [7:31] just being conscious of it and as well as the the  business outcomes that are associated to it. So, [7:38] I'll uh I agree with AJ and and and the things  that you were saying. So, I I'll I'll go at a [7:44] little bit of a different uh angle. So for  instance, we're looking at orchestrating [7:50] agents that come from third parties and agent  discovery and trying to figure out, you know, [7:56] if one agent is using one type of model, how are  we going to orchestrate and integrate that into a [8:00] global system that may use five, six, 10 different  types of models or even methodologies. If one is [8:07] using more of a deterministic solution, how is  that going to integrate with one that is using, [8:12] you know, just traditional LLMs? And that is not  a trivial challenge to be able to do. And so as [8:19] we are looking at those, as we are discovering  those as the baseline, especially from a cyber [8:23] security perspective, the importance of that and  being able to identify those and making sure that [8:28] they're using the appropriate model, making  sure that they are governed uh uh you know, [8:34] appropriately and just in some cases it's just  being able to identify that there's an agent in [8:39] there so that you can orchestrate it is one of our  uh big focus areas, especially on the federal side [8:45] as they're bringing in so many different products.  and so many different uh types of agents. It's [8:50] becoming more and more important to be able to do  that. Yeah. From my side, similar to AJ and what [8:56] you said on top of that, um I'm also focusing  a lot on the making the response deterministic, [9:03] right? Um having a lot of like LLM judges before  even we get the response from model. O making sure [9:11] I'm always validating the response and trying to  make it as close to as deterministic like it's not [9:17] going to be possible to have a 100% deterministic  uh uh response from the LLM as of today. Uh we'll [9:23] see what happen in future. Um other than that I'm  also focusing on like not every request required [9:31] LLM. All of us have the APIs already. All of us  already have the determinist deterministic system [9:41] and that works. You don't have to replace each and  everything right. So when my orchestration layer [9:47] hits first I will check do I have the API exist.  Do I have the nonAI non LLM model solution exist? [9:54] I go to that and the request only goes to LLM when  it is really required here. So I'm going to pull [10:02] on a couple of threads in the the different  conversations because a very common theme was [10:06] model selection. And I heard that from all four of  your responses. Um, and so let's dive a little bit [10:11] more deeply. I think that some of you have already  alluded to this into the economics of of running [10:16] these models, the subsidies, so to speak, that  have gone away, um, and the implications of that, [10:22] but also sort of thinking about, you know, the  the practicality. So the the easiest solution [10:28] obviously is that you deal with it the way that  you would as a personal user, right? You log into [10:31] chat GBT, it's the default model, and you start  typing like that happens. Um and then we obviously [10:37] want to get more complicated and and more complex  and nuanced than that in what we are doing. Um how [10:44] does the orchestration layer actually decide where  to send the task? How do you make sure that the [10:52] response that comes back is of good enough quality  and maybe is standardized and these different [10:57] things? And you know, if you have to do it in this  kind of escalation path where you're running cheap [11:02] models that still cost money and then eventually  have to go to the larger models, how do you make [11:08] sure that that process gets more efficient over  time? So there's lots of sub questions. You don't [11:12] have to answer all of them, but if you have any  thoughts about any of them, please chime in. Yeah, [11:15] probably I can start. I will build upon what  I was saying earlier like using um APIs, LLMs, [11:22] etc. So we have to do that and I think the model  selection is not at the design level. The model [11:29] selection should happen at runtime right. So below  your orchestration layer you should be able to [11:35] decide at runtime when you are getting the request  uh from your system like whether it should go to [11:41] the small language model any other model frontier  model right. Um the what was the second part of [11:47] your question sorry. Oh yeah, it was is basically  about like you know how do you make sure that the [11:52] economics are working out and then you know you  mentioned judges so that sort of helps with the [11:56] quality but so I I think um when it comes to token  economics and cost um we have to optimize for cost [12:07] and also for the business impact at the same  time. You cannot just optimize for cost because [12:14] let's let's say um I'm just trying to think of an  example uh the real example um let's say I have [12:24] like customer delivery I promised uh the customer  is going to get the product in next 2 days right [12:31] somewhere in the system in supply chain I have the  I have this orchestration layer which is trying to [12:36] define like okay what should I do should I use the  cheapest model should I use the uh the expensive [12:42] model right so um if it is going to affect the  customer delivery no matter what happen I'm ready [12:49] to pay more for LLM right so I think it's also  depend on not just optimizing for cost optimizing [12:58] for uh cost business impact and for your customer  yeah and I'll I'll go next with our platform that [13:06] we have we are using a deterministic solution  and it is running in a production environment [13:10] and one of the things that we take a lot of  pride in is that each agent that we have has [13:15] its own constitution. It has its own governance.  It has its own layers. It has its own, you know, [13:20] personalities. And you know, orchestrating  those isn't driven by a large language model. [13:26] It's driven by a supreme orchestrator that we  call it that acts as a program manager just as [13:33] a human would. And so that allows us to a lot of  flexibility. It allows us a lot of control that a [13:39] human would have as they're trying to execute  their projects just as our agents do. And the [13:44] other benefit to that is is that we can put a  human in the loop anywhere that we want to. And [13:49] that human acts just like an agent in some cases  that they have deliverables, they have things that [13:55] they have to uh submit and they act uh just as  part of the workflow and we're monitoring the [14:02] entire end toend cost of that. And so I, you know,  all of us have heard that we have agents that are [14:09] running and oh, it's not working. It's so slow and  come to find out it's stuck in some human's inbox [14:16] and that's causing delays and that's causing cost  runs, right? And so we're monitoring that every [14:21] step of the way, which also helps with the token  monitoriization because we can see what the users [14:27] spending their money on and what they're choosing  and how they're doing that. If I could just jump [14:33] in. Um, on a per deal processed on my platform,  the cheapest is $2.30 and it goes up to $5.60. [14:43] Now, I've measured that thoroughly and you could  replace models underneath that. However, what we [14:50] cannot compromise on is the business output and  the accuracy of that business output, right? [14:56] And this is why things like intent detection,  checking the prompts, checking your evals and [15:02] making sure the accuracy remains is paramount. So  while there is a huge sort of you know u pause on [15:12] like hey are my tokens actually returning um good  ROI? You have to take that in balance. So a couple [15:22] different parts of this I really want to touch  on. So we have quite a few opinions on this and [15:27] they're encapsulated in our our platform. So the  first is runtime selection based on what's in the [15:34] prompt. The criteria that you set between cost  and quality. You might want more variability or [15:41] be open to more variability in some cases and not  others. So that's one dimension. Another dimension [15:47] is there are continuously new models available.  How quickly are you able to adapt to them? new [15:53] providers that make the openw weight models  available. How soon how quickly are you able to [15:58] to leverage those? So when we're talking about the  the cost quality dimensions, a lot of that plays [16:04] into which model you're choosing, when you're  choosing it, and whether you have the ability [16:10] to make those adjustments and measure it so that  you can adjust your optimizations accordingly. [16:16] Now our view is that this shouldn't necessarily be  done at the from the standpoint of each and every [16:22] application as in the intelligent model selection  is one layer removed from each individual app [16:29] or agent so that all of this intelligence can  be captured and measured in a single place. I [16:35] I'll just add one more thing uh Rafie um there's  cheaper models out there. There's Kimmy K3 bite [16:42] just released unlimited OCR. Now think through  some of the security and the risk implications if [16:49] cost is prohibitive to you. Can this actually work  in regulated industries? And right now the answer [16:56] is no. So you will have to take some sort of a hit  in terms of cost when you're using frontier models [17:04] that are actually approved by the regulatory  authorities. Yeah, that's really interesting. Um [17:11] thinking about sort of that that control layer. So  I think something that came up in the discussions [17:15] that we had earlier was a year ago it was all  about AI adoption like just like getting people to [17:22] use the thing right to experiment with it and then  experiments proliferated pilots proliferated um at [17:29] at my company for example we have the ability for  people to design um agents and assistants within [17:36] our our walled garden framework and you more  than 900 of them have now emerged that people are [17:41] sharing and you know trying to actually get into  wider circulation. And so when you have a control [17:46] layer that comes in, how do you handle the  nuance of not just saying no to everything, [17:53] not just turning off models entirely and actually  sort of taking advantage of that energy and that [18:00] creativity, but maybe directing and steering it in  a more productive way to realize enterprise value. [18:06] So So maybe you can talk about the control plane  there. Um Rafie, I know you have some thoughts [18:10] that I do. Yeah, I definitely have thoughts.  Um, so I'll I'll start us off on this one. [18:15] So I I think the governance layer, while it's not  always the the most exciting thing to talk about, [18:20] the view the view is or I strongly believe that  having the right degree of governance is what [18:27] allows you to say yes, more readily, more rapidly.  If you have the appropriate degree of control, [18:33] you're able to do the right amount of vetting,  whether it be the provider or the model itself, it [18:39] provides you that degree of flexibility to be able  to facilitate and say yes to the organization. So [18:45] the nice thing about the openw weight models for  example like we were talk talking about Kimmy K3, [18:50] the providers are really going to matter,  right? Um so the degree of trust that you [18:55] have the controls that you apply will be based  on how much you trust the provider as much as [19:02] how much how much you trust the model. Yeah. And  and one thing I'll add to that is um governance [19:10] cannot be enforced through policy documents.  you have to build governance through your SDKs, [19:17] through your APIs, through your um identity and  role permissions. And the reason why I say that is [19:25] because the longer it takes for you to say yes to  something like an AI initiative or an AI project, [19:33] um the more there will be shadow AIS.  Exactly right. Exactly right. And look, [19:40] that's just going to proliferate risk across the  enterprise. So be very thoughtful in terms of who [19:48] you assign responsibility to, who's using it,  what data sources is touching, can I pull the [19:54] plug on it, all of those great things, and make  sure we have a clear view across the enterprise. [20:02] Yeah. And I would add also that from our  platform's perspective because we're focused [20:08] on the user and the AI relationship you know the  governance naturally flows through the humans and [20:14] because we're using a deterministic solution to  drive a lot of that then the benefit to that is [20:19] going to be that accountability is still going  to be at the human level and we can have the AI [20:25] do the other stuff. The the great thing that  we've run into is that the adoption everybody [20:30] was trying to to build on oh let's use AI let's  use AI but now from a platform perspective we [20:36] are inheriting a lot of technical debt they have a  lot of projects they have a lot of knowledge with [20:43] a specific vendor or a specific tool set and then  we have to bring that in and then also run through [20:49] the governance pieces with that and so there  are conflicting uh challenges when it comes to [20:54] allowing the human to bring their stuff in versus  what we're what we're trying to optimize. Just to [21:01] add add on that I think um want to talk about  the the stuff uh we used to do earlier right we [21:08] used to have a security team we used to have the  engineers when your engineers were writing code [21:13] they were not leaving everything to security team  right so the engineer were making so sure they are [21:20] writing the code which is secure already so it's  the same thing with the AI so this um there's a [21:29] few threads to unravel and I'm going to pull on  the determinism one a little bit first. So Alan, [21:34] I'll start with you on this one and then others  can chime in. So when we are in the process of [21:39] building our AI co-scientist, right, I I our  parent company is a chemical materials and and [21:43] biotech/farma company. And so we really want to  deploy an agent that can act as a co-scientist [21:50] that will be a teammate/ a tool that our  scientists can use to improve their productivity. [21:55] And a lot of the early orchestration kind of  workflows felt very what I might call linear, [22:01] right? As in, you know, you start with running  this task and the output from that agent goes [22:06] into this other agent and things are very well  controlled and all of that is kind of set up [22:10] um as we mentioned at runtime, but it's like  it's set and it isn't very reactive and so when [22:16] you get an experimental result that's not what  you expected, how do you change course? How do [22:21] you do deeper investigations, etc. So Alan tell  me in your system right where you're doing this [22:26] deterministic thing which is great from the  economics perspective the control perspective [22:30] and even from the reliability perspective um how  do you engineer enough let's say creativity into [22:36] that system where these agents that you've created  are actually able to run somewhat autonomously [22:43] uh in ways that may not have been predictable when  you first start. So the first thing that we do is [22:48] we are assigning personalities and skill sets to  each agent. So each agent will have a personality [22:54] skill set profile. They'll have a uh something  that we can can quantify whether it is your [23:00] MyersBrigg scores, whether it's your leadership  scores. Really, it's a almost an assessment, [23:05] a personality assessment that the user can provide  to that uh that individual agent. So if you have a [23:11] team of agents, each one has its own personality  profile, which means it's going to behave, it's [23:16] going to act, it's going to adjust differently  than what a a static large language model managed [23:23] person, you know, persona or agent would look  like. And so by doing that, then that gives us a [23:29] head start. When Zeus is our supreme orchestrator,  right? When Zeus creates the workflow, then it's [23:37] looking at all of those things. It's looking at  all those pieces of that goal and how is it going [23:41] to provide an optimized solution and it's usually  not linear at all. you know we and we will have [23:48] in some cases 10 12 15 agents that it will create  and then the tool sets that follow from that and [23:54] like we were talking about earlier you know we all  know that uh AI didn't exist before large language [23:59] models right and so we forget about all that  other stuff that's out there but personality and [24:05] how they use those tools is a huge driver in what  that workflow actually looks like very interesting [24:14] so I have a philosophy When it comes to agents,  um I think agents are a deployment mechanism, [24:21] right? So if you can map out a flowchart or build  a sequence diagram, just build it. You don't need [24:29] an agent for that. Agent is for when you cannot  enumerate the amount of deterministic paths and [24:38] you need it to self-guide itself, right? So  I'll give you an example, right? like on my [24:43] platform fund lens let's assume um our platform  recognizes that for this company there is some [24:50] sort of an unfounded balance sheet obligation and  maybe the analyst wants to double click on that [24:58] right and that's when I would dispatch an agent  and go look at data sources news articles filings [25:07] etc that are outside the scope of the regulated  filings and then go search and figure out and [25:14] come back with the thesis as to why this  line item has been so obscure and it's in [25:20] the footnotes and it's not readily visible.  So be thoughtful about why you build agents [25:26] and make sure you always have constraints  around it so you know when to pull back. [25:34] Yeah, I think it's an incredibly important point  to make the decision as to where it's appropriate [25:39] to to deploy an agent. I think most of us and  what we're emphasizing is there's still a place [25:45] for a large place for procedural deterministic uh  implementations. Now there's hybrids of this as [25:53] well that are important to keep in mind in that  you could deploy an agent, observe its behavior [25:59] and build the uh deterministic structure based  on what the what decisions the agents are making. [26:06] But I think what was said best was if you can't  straight uh from a straightforward perspective [26:13] quantify how to build something in a deterministic  manner, that's when you can put an agent there [26:18] and to navigate some of that uncertainty or  complexity. Yeah. Um I think it's also important [26:27] to have end to end visibility into what your agent  is doing. You should be able to audit each and [26:35] everything what your agentic workflow is doing,  right? And you should be able to um trace backward [26:44] what happened. You should be able to audit  and you should be able to replay those events. [26:52] So yeah, that was that was really interesting in  talking about the determinism of the workflow and [26:57] I think you beautifully illustrated that like  the case for agents is exactly when you can't [27:01] have a deterministic workflow which is great.  So now we're going to circle back to what you [27:06] talked about Nurmal at the beginning which is  determinism in the answer right. So this was [27:12] a really interesting conversation that we had  about two opposite forces um that are happening [27:18] in enterprise AI. So you have frontier model  providers and other sorts of providers that [27:24] want to guide you, you know, honestly into their  ecosystems, right? So that you understand the way [27:31] that they work and the reactions that they're  going to give to specific prompts and you learn [27:36] to like that. Like I feel like for most of  us who are using AI in a commonplace way, [27:40] you kind of like the way some systems answer  and not others. You like the way you're able [27:43] to interact with certain systems. who uses more  than one like AI tool let's say of the generative [27:50] AI variety right now okay a lot of people in the  room right of those people how many of you change [27:56] the way that you speak or chat with or code with  these different systems or are you talking to both [28:03] the same way so if you're talking differently to  the systems if you're talking the same way to the [28:10] systems okay so you know a more even split  there. And so this is a really interesting [28:16] question about portability. So what an enterprise  hates more than anything is to get locked in, [28:22] right? To get locked into some ecosystem where you  can't negotiate on price anymore and you're stuck. [28:28] And the opposite end of that is total portability,  which is something that the providers don't want [28:32] because it makes switching costs very very  low, right? And they lose their competitive [28:35] advantage that way. So let's say that you're  engineering for the enterprise. How, you know, [28:41] and you can speak a little bit to this cuz you've  talked about it. How do you get it into a state [28:45] where if tomorrow you needed to switch your  model out, how do you make it so that things [28:50] in the middle don't break and the answer is not  dramatically different and all the prompts and [28:54] the system prompts that you design don't fail?  Yeah. So the way I think uh personally is every [29:01] enterprise should have its own organizational  layer. Right? When I say organizational layer, [29:09] that organizational layer contains the context of  everything, right? So it's not about the request [29:16] that we are sending to LLM. Of course, that's a  part of that. Right? There is a lot of context [29:21] at the enterprise level which is in people's head.  It's not written anywhere. It's not in the system. [29:27] Right? So if we are talking about like we need an  agentic system and everything will be magically [29:33] work and everything will be 99% deterministic. How  is that that going to happen without having that [29:38] missing organizational context which happens in  meetings and uh which happens between two people [29:44] uh in a coffee chat and there are so many things  at the enterprise level so many teams so many [29:49] connection it's really really hard right there's  no way we are going to have in like next couple [29:53] of months like all the system connected and  everything magically working agentically right [29:58] these uh systems like finance system healthcare  system supply chain is like really complex system [30:03] um we we took like 10 to 20 years to reach to this  stage right and we don't have that infrastructure [30:10] ready where is like oh all of my systems are plug  and play let me add AI it's not going to work like [30:15] that right so again going back to the question  uh I think the way it should work is like every [30:21] enterprise every company should have their own  separate uh organizational layer whatever you call [30:27] it um every request goes through that right and uh  so under the under that layer you should be able [30:35] to switch to any model. Right? So I'm not going  to go into too technical uh just keep it high [30:41] level. So yeah. So there's a couple things you you  touched on there VJ. So the the first one I wanted [30:48] to start off with was commodit commoditization of  the inference layer. I think from the standpoint [30:55] of what organizations need to see is that  commoditization. Well the providers are pulling [31:01] in the opposite direction. You see, especially  Anthropic, each and every day they have their own [31:07] purpose-built application that's being delivered.  Claude legal, Claude design, you name it, [31:13] there's a new one and you gravitate towards that  and it creates that gravity towards anthropic. So, [31:20] it could look like it's an accelerator and  maybe it is in the short term, but you are [31:24] over time becoming more and more locked in as an  organization. the more you gravitate towards those [31:31] uh uh provider delivered capabilities. So it's  important to have a portfolio of providers uh [31:39] available to your organization and then as you're  building your agents, your applications, it's [31:44] important to design them and test them utilizing  as many models as you conceivably can. Um you will [31:53] want to leverage the latest models as quickly  as you can. If you figure your competitors are [31:58] leveraging them, um it's just like hiring the best  people. Wouldn't you always want to have the best [32:03] people possible to hire? Well, the model layer  is very much that. So, setting yourself up to be [32:10] able to leverage the changes that are happening  almost on a daily basis is incredibly important. [32:16] Okay. So, one of the things that we're doing is is  that uh we don't we don't care what large language [32:23] models being used or what the organization is  using because ours is plug-and-play. We can [32:28] take over a 100 different large language models  and a user if they've got a URI and an API key, [32:33] they can plug it in. The thing that we're doing is  is though we are ensembling those responses that [32:38] we get back from those large language models that  allow us to do a lot more with those responses [32:43] than you can do in a traditional sense. And  so by creating that ensemble capability, [32:49] then how we are integrating with the user becomes  the important piece, not oh there's a brand new [32:55] model that's been released and we all want to use  this and we all want to use that and then all of [33:00] a sudden your costs go through the roof because of  well tokconomics, right? And so it's one of those [33:07] it's one of those key drivers that allow us really  to just, you know, continue to make sure we're [33:13] using the large language model where it should be  used versus just having it do everything. Yeah, [33:20] just just to piggyback um to me it's around the  table stakes are having a universal API or an SDK [33:29] where you can switch out models. That is key. Now  that's not easy and done. What you need to ensure [33:38] is that your business output, your insights, your  accuracy still continues across these different [33:44] model tiers, right? So that goes back to what  Nurmal was talking about, right? that organization [33:50] layer, your intent detection, your prompts, your  eval across these different tiers and see how it [33:58] works out and see what's actually returning  in terms of outputs and the accuracy of the [34:03] responses. That's critical, right? So the other  question VJ that you asked the audience was a [34:10] good one in terms of like hey has your behavior  changed when you're using claude or chat GPT and [34:17] are you actually modifying your prompts and I can  tell you when I use Fable I literally provided you [34:26] know 80% less context or guidance because it's  smart enough. Now you can do the same thing with [34:32] the business application. Very interesting. So  thinking about, you know, this even things that [34:40] Andrewing was talking about this morning that  we've been talking about all conference long is [34:43] this constant build versus buy decision. Um and  and these may be different decisions and nuance [34:49] decisions for different parts of orchestration  because orchestration is a big term that involves [34:54] a little bit of routing and a little bit of  you know selection and control and governance [34:58] and all of those pieces. What do you think are  the most critical pieces that I mean this might [35:04] even not be the right question but like how does  an enterprise go about making that build versus [35:11] buy decision and it's not really a dichotomy. It  might even be more of a peace meal thing but if [35:15] you have any thoughts about that I'd be curious  for the whole panel. Yeah. Um it's it's never [35:21] really build versus buy. It's kind of build and  buy. You can't just do one or the other, right? [35:29] um if you're building or if you're fine-tuning  models or creating models from scratch, that's [35:34] a huge huge amount of cost, right? And by the way,  if you're not a frontier model company, your model [35:42] will be in the graveyard in 6 months. So don't  build instead model. But say for example, people [35:49] want to use lang chain versus buying the solution,  right? To sort of chain different things together. [35:55] What would you all advise them to do? You know,  how to make that decision? Yeah, ju just ju just [35:59] just two quick sentences on that. Um I tend more  conservative in the sense that I will build on [36:06] top of primitive services AWS for example. Um and  I'll tend to use a very loose orchestration layer [36:13] on top because that gives me more flexibility  and control surface in being able to troubleshoot [36:20] whether it's security or bugs or whatnot. the  deeper and longer your specific orchestration [36:28] layer becomes that there's more complexity that  you induce and the harder it becomes to manage. [36:34] So I would say we we look at the enterprise  a little bit different because we ask the [36:40] question do you want your AI team to integrate  this and do all the things that you expect or [36:48] do you want your actual users to be able to use it  at the enterprise level? Those are very different [36:53] models. I mean I tell everybody that I'm trying to  put myself out of a job for a lot of these things. [36:58] you shouldn't have to have an AI team come in  and deploy uh a team of agents to do spreadsheet [37:06] analysis or those kind of things. And that's  where that's where we are today. And as we, [37:11] you know, start to again shift that focus to  the user as opposed to, you know, just your your [37:18] AI team or your software developers or whatever  that is going to look like. Then foundationally, [37:23] you start to think a little bit different. You  start to ask your platform to behave a little bit [37:27] different. you put different checks and balances  in there uh and then you know that that usability [37:32] will become a lot easier because for instance  Zeus guides the users through the process so [37:38] they don't have to have an AI team to do some  of this thing and so that's where that's where [37:42] our primary focus is yeah sorry really quickly  here yeah so um we've seen this cycle time and [37:50] time again with technology iterations revolutions  we're encountering this quite a bit right now we [37:57] run into the AI team that's built a whole lot of  say infrastructure that they're likely not going [38:02] to want to maintain long term. They might have had  to do it to get to this point. So the way I always [38:08] like to think about it is to build only what  differentiates your business and your use case [38:14] and buy or leverage open source where uh where  it's not differentiated where it's boilerplate. [38:21] Right? So it's there's those cycles can obviously  be utilized more effectively and more efficiently [38:29] if you're you're building towards your business's  differentiation. Yeah. I think in my mind it's [38:36] it's not a new problem. Right. Right. We have  been having this discussion even before AI. I [38:44] don't think so there is anything changed with AI.  It's the same discussion. Right. Let's talk about [38:49] the basics. Right? when you when you will build  versus buy, right? Um you want to buy something [38:56] uh which is like really available and you can  be competitive in the market and gain the market [39:00] share. Yep. Go ahead and do it. That's your  temporary. Maybe you gain the market share and [39:05] then you can inhouse keep building in parallel and  like uh deplicate that uh off-the-shelf product [39:10] and continue building your own. Every problem is  different, right? So we need to think about the [39:16] problem whether this problem is like you should  uh build internally. Do you have capacity? Do you [39:21] have engineering resources? Right. So, um again,  um it's it's all the same questions and answer. I [39:27] don't think so anything changed with AI here. So,  oh, go ahead. The one thing I would add is there's [39:34] much more of an inclination now to say, "Oh, I can  build it because I have AI to help me build it." [39:40] So, that's that's a trap that's easy to to get  caught into as well. Yeah. Yeah. Yeah. But by the [39:46] way, citizen developers are abound. So you have  to be able to govern those and control those folks [39:54] as well. So maybe in our final few minutes here,  I'm curious about sort of the where this winds up [40:03] translating into enterprise value. So there was  a talk I went to on um Monday when when I first [40:10] arrived by um Rebecca Hines at Glean. So she's  part of this uh work AI index which is released [40:16] by the work AI institute. So they surveyed 6,000  digital workers and they found that 87% of those [40:23] workers are using AI at work and it saves those  workers about 11 hours a week. So that's a lot [40:29] right? I mean you know depending on how many hours  you work a week it's up to you know 25% or so of [40:34] their time. However, those same people were saying  that only about 13% of their organizations are [40:41] seeing the value come out from these activities.  So there's this huge gap, this huge divide. [40:48] So maybe each of you can in your concluding  remarks talk about how the orchestration [40:54] layer specifically can help close that divide can  help these individual pilots, these individual [41:00] verticals that are benefiting within your business  translate more widely. So um yeah, you look like [41:07] you're ready to go. So we'll go this way. Yeah.  So so I'll give I'll give you an example from um [41:11] from my current role at uh JP Morgan. every single  use case across a line of business whether it's [41:18] investment banking, consumer banking or asset or  wealth management. You sort of look at it from a [41:24] top- down ROI perspective and then you think about  is this going to be valuable for me for AI to step [41:31] in and then you attach financial benefits to  that use case whether it's whether it's in [41:37] terms of cost avoidance cost reduction u revenue  uplift etc. And then you monitor that whether [41:45] it's across a business process or whether it's in  terms of customer acquisition or what have you, [41:52] right? And then when you do that and you  get a feel for that and you optimize for [41:57] that on a quarterly basis, you get to the real  truth in terms of what is my value realization. [42:06] Yeah, I would say really quick, you know, AI has  been treated very special recently and oftentimes [42:14] we forget about all the other technologies that  are out there that we have to budget against. [42:18] And so if you have a way that you're managing an  innovation pipeline, then you're going to put the [42:24] resources where the best value is going to be  for your enterprise. And if AI fits into that, [42:29] that's great. But don't do AI for the sake of  just doing AI so that you can get your name out [42:35] there. Oh, we're doing AI because in the long run  it's actually more detrimental to your business. [42:41] And so just focus on the innovation piece and  you'll find the right technology to be able to [42:46] integrate. And maybe AI can help and maybe it  won't, but at least you're putting the dollars [42:50] where it needs to it needs to go. I to sort of  further that point, I see it as a tool uh just [42:58] like anything else. The accessibility of that tool  to an organization is a huge part of it. You know, [43:04] it's there. You're able to reach for it when it's  appropriate. Something we were talking about just [43:09] before we got on stage here was how we're still at  the very early days of AI adoption and how much of [43:15] the user base is still treating it like, oh, I go  to chat GPT and use it as a glorified web search. [43:22] So there's a lot of education as well that's still  happening within the enterprise to think through [43:27] some of what's possible and what's appropriate  in terms of AI use and I I think we'll see a lot [43:32] of that change over the coming years. Yeah.  So coming back to your original question, [43:37] I think this problem is not only related to  orchestration layer, right? There's so many other [43:43] parameters like AI literacy, making sure people  understand how to use it, when to use what model, [43:49] uh they understand about the cost and uh you  have a cap per human in your organization, [43:55] right? Um from day one, your focus should not be  AI, your focus should be impact. What impact you [44:03] are going to make, right? It's same thing we  have been doing this in our road map planning [44:08] and initiatives whenever we are planning for a  year 6 month or quarter we have to follow the [44:13] same thing it's just and like you said like AI is  another tool so when you are solving a business [44:18] problem can this tool help me moving in faster  right um the engineers has been moving much much [44:25] more faster now because this is where we see the  most impact of the AI tooling right I think the [44:30] problem is on the business side the engineers  are moving so fast the business is not moving [44:35] so fast. The engineer is done in uh one week  instead of like finishing the 3 months project [44:40] and the engineer is waiting like what is the  next business impact I should be making. Yeah, [44:45] really interesting. Well, I if you  have any other questions for our panel, [44:49] I'm sure we'll be able to hang out for a few  minutes outside, but thank you all. This is [44:52] a wonderful conversation. Um so, thanks all for  joining. Give them a round of applause. Thank you.