Gallagher, Asset Managers Treat AI Models as Disposable Utilities

Gallagher与资产管理机构将AI模型视作可随时替代的基础工具 当企业AI落地受制于基础模型供应商而非专有工作流元数据时往往会陷入停滞，这要求企业将模型视为可替换的通用商品，并将财务责任直接转移至业务部门。 企业技术高管深入分析了AI从实验性沙盒向可衡量的生产部署转型的过程，强调专有上下文、元数据和运营工作流程构成了企业的真正竞争护城河，而底层基础模型正逐渐变成可互换的通用商品。专家小组主张通过设定实施前的运营基准并采用成本明示/分摊（showback/chargeback）模式，将投资回报率（ROI）问责权从IT部门转移至财务和业务职能主管。为保持性能并控制推理成本，高管们重点提及了提示词缓存、特定领域的小型语言模型（SLM）、防止产生“AI垃圾内容（AI slop）”的持续上下文健康度评估，以及为实现最小可行治理而采用务实的辐射状（hub-and-spoke）架构。 企业AI的优先级正迅速从内部生产力实验转向规模化投产能够直接推动营收增长的用例。 (0:08) 高管对投资回报率（ROI）的讨论应聚焦于净业务产出而非原始Token成本，并通过企业级可观测性底盘和提示词缓存来遏制冗余支出。 (5:07) 智能体（Agent）从沙盒过渡到生产环境时的失败，主要源于上下文工程不良、缺乏依据的检索以及工作流元数据缺失，而非基础模型的局限性。 (12:12) 实现真正的企业自主性，要求在部署自动化之前预先梳理基准运营模型和决策逻辑，将机构专有上下文规范化。 (17:24) 为消除部门疑虑并确保可信度，应由公司财务部门而非IT部门对照实施前的运营基准，独立审计并验证AI的投资回报率。 (20:42)

Enterprise leaders are pivoting from model-centric experimentation to durable infrastructure, grounding AI agents in proprietary context and business-led accountability to generate measurable P&L returns. As foundation models commoditize, panelists argue that sustainable enterprise value depends on structured workflow metadata, independent finance-led ROI validation, and minimum viable governance frameworks. Decoupling Disposable Models from Durable Enterprise Context (29:37) Mark Bloom and Geetesh Iyer argue that foundation models from providers like Anthropic, OpenAI, and xAI are rapidly becoming interchangeable, disposable utilities that organizations can hot-swap as price and performance evolve. The enduring enterprise moat resides within the durable layer: institutional context, proprietary business logic, and structured workflow metadata. Amit Patel emphasizes that identical models yield radically different production results depending on the context wrappers and infrastructure built around them. Organizations that scale successfully invest in mapping daily workflows and process metadata before deploying agentic pipelines. Left unmonitored, enterprise context layers degrade quickly. Iyer warns that repositories accumulate 'AI slop'—stale documentation, redundant entries, and conflicting schema definitions—within two to three weeks. In practice, ambiguous prompts can cause agents to pull conflicting metrics, such as divergent Annual Recurring Revenue definitions between sales and finance departments. While the underlying arithmetic may be flawless, the output becomes operationally incorrect, underscoring the necessity of continuous evaluation harnesses and automated freshness checks. Measuring P&L Value Through Finance-Led Validation (20:42) As organizations abandon unconstrained token usage in favor of fiscal discipline, panelists caution against relying on vanity metrics like login counts or raw token spend. Mark Bloom asserts that technology leaders should not calculate their own AI returns, as self-reported figures invite internal skepticism regarding whether savings impact the general ledger. At Arthur J. Gallagher, IT partners with corporate finance, operations, and legal teams to establish rigorous pre-implementation baselines; finance then independently evaluates operational telemetry to validate true P&L impacts. Demonstrating hard economic value is also reshaping organizational design. Rob DiNuzzo highlights the emergence of 'marketing AI engineers' who orchestrate autonomous agents, allowing firms to calculate ROI by comparing infrastructure costs against avoided traditional headcount. To align divergent corporate priorities, Iyer proposes a composite Impact Score that balances executive adoption depth, engineering technical reliability, and audited financial savings according to organizational maturity. Domain Ownership and FinOps-Style Chargeback Mechanisms (24:52) Tarun Sood draws a direct historical parallel between agentic AI scaling and public cloud migrations. Early adoption requires upfront capital expenditure before organizations achieve efficiency through prompt caching, smaller context windows, and targeted small language models (SLMs). For example, American Century Investments deploys SLMs rather than frontier models for conversational text-to-SQL business intelligence, achieving high conversion by meticulously mapping KPI context and metadata with business sponsors. For agentic initiatives to scale, functional business units must take direct ownership of outcomes. While IT owns the productivity metrics for internal developer tooling, commercial leaders in sales, marketing, and underwriting must own the business KPIs for client-facing workflows. Sood cites industry findings indicating that business-owned AI projects are three times more likely to scale successfully than IT-isolated efforts. Furthermore, panelists anticipate that enterprises will adopt cloud FinOps practices, implementing showback and chargeback systems that bill token and API usage directly to the consuming unit's P&L. Minimum Viable Governance and Hub-and-Spoke Execution (40:14) The panel cautions against governance paralysis, advocating instead for 'minimum viable governance' that establishes essential regulatory, audit, and security standards without stalling early momentum. Patel outlines a hub-and-spoke model utilized by institutions like JPMorgan Chase and Walmart: a centralized AI Center of Excellence provides baseline infrastructure, data platforms, and risk guardrails, while decentralized business spokes execute rapid domain-specific experimentation and pass validated patterns back to the center for enterprise-wide scaling. To prevent agent sprawl and redundant engineering, Gallagher deployed an automated intake agent. Any employee seeking to build an AI agent must complete an interrogation regarding data privacy, PII handling, and compliance, while the system scans existing enterprise repositories to recommend established tools before authorizing duplicate development. Sood adds that production deployments inevitably generate technical debt, meaning long-term reliability depends on cataloging metadata and enforcing data hygiene rather than relying on out-of-the-box model checkpoints. While enterprises increasingly deploy stateful context harnesses and domain-specific models to control costs, scaling remains fundamentally bottlenecked by data readiness. Panelists emphasize that frontier models cannot compensate for undocumented human workflows, internal data territoriality, or fragmented legacy architectures.

企业部署AI智能体（Agent）正从前期的无节制Token消耗与沙盒概念验证，转向严密可控的商业投资回报（ROI）与工程化落地。多位行业高管与投资人指出，成功的企业战略必须将基础模型视为可随时替换的商品化组件，转而聚焦于建设私有持久上下文层、流程元数据以及“中心辐射式”的最小可行治理体系。 算清Token成本与财务主导的ROI (5:07) Amit Patel指出，早期的生成式AI探索普遍采用“最大Token化”策略，但如今AT&T等企业正面临推理预算严重超支，特斯拉等公司甚至对员工设定了每月200美元的使用硬上限。面对成本压力，Arthur J. Gallagher全球首席信息官Mark Bloom强调，IT领导层必须部署全局可观测底座与Prompt缓存层以杜绝冗余计费，但在向业务汇报时应聚焦业务产出而非纠结于Token账单本身；若一项产出能带来50万至60万美元的收益，10万美元的Token成本就具备合理性。 针对财富500强企业中普遍存在的“AI用例投产超半数却无法量化ROI”的矛盾，Mark Bloom提出由企业财务团队独立全权主导ROI评估，IT仅在上线前后提供业务基准与遥测数据，以避免自评自夸引发的跨部门质疑。American Century Investments首席数据官Tarun Sood补充道，这一演进类似早期的云迁移，未来各业务线必须像遵循云FinOps模型一样，承担Token计量与内部转账计费（Chargeback）；由业务单元负责人直接主导的AI项目，其规模化投产成功率在行业数据中高出纯IT项目三倍。 持久上下文层与可替换模型的解耦 (29:37) 针对模型技术快速迭代带来的技术债务隐患，Mark Bloom提出了“持久层”（Durable Layer）与“消耗层”（Disposable Layer）解耦的架构准则。他指出，无论来自Anthropic、OpenAI还是xAI的模型，本质上都在经历激烈的价格性能竞争并迅速商品化，属于可随时热插拔替换的消耗品；而企业真正的核心护城河与持久资产，完全在于组织内部沉淀的上下文、企业元数据、专有业务逻辑与权限体系。 WisdomAI创始产品经理Geetesh Iyer和Nimble的Rob DiNuzzo从生产环境故障点分析指出，智能体从沙盒走向生产的瓶颈已非基础模型的能力缺陷，而在上下文工程与防范“AI废料”（AI slop）。若缺乏动态上下文维护，企业知识库往往在两到三周内就会因陈旧文档和冲突定义导致模型表现崩塌。例如销售与财务部门对年经常性收入（ARR）的统计口径不一致，会导致模型即便算术正确却输出完全错误的业务结论。因此，企业必须构建持续评估体系与内部检索索引，避免智能体陷入重复爬取与逻辑失真。 流程元数据沉淀与专用小模型落地 (17:24) 行业讨论往往聚焦于通用人工智能（AGI）或全自主智能体，但与会者普遍认为，即使AGI明日降临，多数企业仍无从使用，根本症结在于关键的业务决策背景与流程元数据依然被隔离在人工孤岛中。例如摩根士丹利在构建核心自动化工作流时，第一步并非选择或微调大语言模型，而是借助Skan等流程智能平台对其整个运营模型进行全面流程发现与测绘，以提取执行瓶颈、单步耗时、成本及人工决策倾向等隐性上下文。 在底层模型选型上，Tarun Sood以American Century Investments的实践证明，对话式商业智能（如Text-to-SQL）完全不必依赖昂贵的尖端前沿模型，通过业务协同对KPI上下文及数据关联进行精细标注，专用小型语言模型（SLM）即可实现大幅转化提升并显著规避Token开销。此外，企业必须破除内部数据割据，如Gallagher正推进将多达800个遗留数据仓库归集至Snowflake统一实例，配合原生基于角色的访问权限，为模型应用提供坚实的数据源保障。 准入智能体与中心辐射式敏捷治理 (40:14) 为应对治理滞后或过度防范导致的创新瘫痪，论坛倡导推行“最小可行治理”（Minimum Viable Governance）。Tarun Sood警示，任何代码投入生产环境的第一天起便成为技术债务，严密的治理元数据并非行政负担，而是保证智能体免于幻觉与合规风险的性能加速器。在实操层面，Gallagher通过部署专用的“准入智能体”（Intake Agent）对新项目进行自动化前置审查，强制核验PII数据与合规策略，并扫描现有智能体资产库以主动推荐复用，杜绝重复开发引发的智能体蔓延。 针对跨业务部门协同，Amit Patel结合摩根大通与沃尔玛的实践，提炼出“中心辐射”（Hub-and-Spoke）治理范式。中央AI卓越中心（Hub）仅负责制定基线风险容忍度、数据平台规范与基础设施，严禁微观干预具体用例；业务单元（Spoke）则负责快速试验与业务适配，经验证成熟的智能体模式再交由中心向全企业推广放大。Geetesh Iyer进一步提出，企业应引入跨越底层模型与上下文引擎的“四分卫智能体”（Quarterback Agent）及评测框架，实现分析状态的持久化并平衡准确率、延迟与调用成本。 企业智能体战略的成熟标志，是从追逐模型参数与盲目概念验证，转向以业务收益为导向的工程与治理深耕。尽管前沿模型仍在持续降本增效，但企业能否建立系统化的上下文清理机制、理清复杂的组织数据政治，以及将不可避免的合规风控内化为业务自觉，仍是当前智能体实现规模化商业变现的核心未决挑战。

## Background 该视频由 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台上。视频记录了一场关于企业级 AI 战略落地的圆桌研讨，汇集了来自 Plug and Play Ventures、Gallagher、American Century Investments、WisdomAI 及 Nimble 的高管与专家，重点探讨如何将自主 AI Agent 深度融入企业战略、跨越试点阶段并在全组织内实现规模化运营。 ## TL;DR 论坛深入剖析了企业推动 AI Agent 走向生产环境的核心路径，提出“租用模型但必须自研并拥有上下文智能层”的关键架构理念，将底层基础模型解耦为可灵活插拔的大宗商品。针对 Token 成本与算力挑战，嘉宾们倡导推行严格的成本纪律、Prompt 缓存与上下文工程，并强调 ROI 应由业务部门与财务团队协同核算并承担主导责任。此外，讨论还提出了以“准入 Agent”防范工具蔓延、采用 Hub-and-Spoke 轴辐模式推进“最小可行治理”，以及构建“四分卫 Agent”统筹调度有状态上下文的完整落地体系。 ## Atmosphere & style 整场研讨气氛热烈且极具实战锋芒，嘉宾们开场便以“排队长龙堪比泰勒·斯威夫特演唱会”等幽默风趣的调侃拉近与观众的距离。对话完全摒弃了浮躁的技术炒作，五位来自投资、金融经纪、资管与技术底座领域的领袖在务实、直率的互动中频繁碰撞思想火花。观众问答环节直击“数据政治”与“治理博弈”等敏感痛点，全场节奏紧凑、干货密集，展现出技术老兵与业务操盘手在面对技术剧变时的沉着与远见。 ## Key takeaways - [00:08](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=8s) 告别 PoC 试点与内部玩具：企业 AI 战略重心正从单纯提升内部生产力，全面转向构建能够直接驱动业务营收、具备可衡量 ROI 的规模化生产级 AI 平台与 Agent 体系。 - [05:07](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=307s) Token 经济学与成本纪律：行业心态从早期追求“最大化 Token 消耗”快速摆向“模型效率与精细化管理”，企业必须通过 Prompt 缓存和上下文工程大幅优化推理开销。 - [10:40](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=640s) 租用模型但掌控智能层：基础大模型正逐步沦为可替换的大宗商品，企业必须构建多模型架构并自主掌控专有的上下文、元数据和智能层，以沉淀自身核心知识产权（IP）。 - [12:12](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=732s) 跨越沙盒到生产的断层：Agent 在真实生产环境中崩溃的根本原因在于缺乏专有索引与上下文治理，精准的上下文生态能够同时兼顾准确率、成本与信任。 - [15:55](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=955s) 运营元数据映射是 Agent 自主性的基石：大多数企业无法落地 AGI 的核心阻碍在于未将底层业务流程中的决策原因、瓶颈和耗时等细粒度元数据进行上下文结构化映射。 - [18:35](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1115s) 从单点执行者转型为 AI 编排者：ROI 的量化应直接体现于人力编制成本与业务产出的对比，初级员工职能正在从基础战术操作者全面进化为 Agent 工作流的编排与管理者（Orchestrators）。 - [20:42](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1242s) ROI 权责划分与财务结盟：CIO/CDO 不应亲自背负财务回报算账的责任，而应在项目初期与财务团队共建基准线，由财务独立核算并出面倡导收益，确保客观公信力。 - [24:52](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1492s) 业务主管亲自为 ROI 买单：由业务部门主导并承担 KPI 与 ROI 的 AI 项目，其成功规模化落地的概率高出整整 3 倍，能推动普通 IT 项目真正演变为深度的业务转型。 - [27:26](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1646s) 风险治理是 ROI 的底线保障：生产级 Agent 必须具备严格的审计追踪与访问护栏，忽视治理导致的数据泄露或法律诉讼会使一切账面投资回报瞬间归零。 - [29:37](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1777s) “持久层”与“一次性层”的架构解耦：企业应将业务工作流、核心逻辑与上下文沉淀为“持久层”，将处于外部激烈竞争中的基础模型视作随时可插拔替换的“一次性层”，以此化解技术过时与技术债务。 - [31:07](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1867s) 治理“AI 垃圾”与上下文漂移：企业必须建立持续的 Eval 评测体系与上下文健康检查机制，及时监测并消除跨部门语义冲突（如 ARR 计算口径）与陈旧数据。 - [33:18](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1998s) 推行最小可行治理（MVG）：避免因追求绝对完美的治理框架而陷入停滞，在守住基本合规底线的前提下，依托高质量统一数据底座小步快跑推进部署。 - [35:45](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2145s) 以 Agent 治理 Agent：Gallagher 引入“准入 Agent（Intake Agent）”对新 Agent 的构建需求进行合规与 PII 审查，并通过匹配现有 Agent 资产遏制企业内部的野蛮生长与重复造轮子。 - [36:42](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2202s) 领域小模型（SLM）赋能特定场景：对于对话式 BI 等 Text-to-SQL 场景，只要具备严密定义的元数据与语义上下文，轻量级领域小模型完全可以替代高成本的通用旗舰大模型。 - [42:30](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2550s) Hub-and-Spoke 组织落地模式：中央 AI COE 负责提供最小可行基础设施与统一风险基准，各业务线 Spoke 拥有自主探索场景的敏捷性，并能将跑通的成功范式反哺全企业沉淀。 - [44:40](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2680s) 有状态上下文与“四分卫”治理架构：现代企业级 Agent 必须具备有状态记忆，通过统筹基础模型、上下文引擎与评估矩阵的“四分卫 Agent”，构建兼顾精度、成本可控与稳健治理的落地生态。 ## Detailed walkthrough ### [00:08](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=8s) 嘉宾开场介绍与企业核心 AI 目标 论坛主持人、Plug and Play Ventures 合伙人 Amit Patel 首先向现场观众致谢，并调侃场外排队长龙的夸张程度。他介绍 Plug and Play 作为全球大型创业投资机构，拥有广泛的企业创新生态网络，长期与 Nike、Walmart、Mercedes 等跨国巨头合作，帮助他们对接全球前沿初创公司以推动业务增长与 AI 创新，同时基于这些产业互动数据进行投资决策。机构投资十余年间已孕育出约 33 家独角兽企业。随后，Amit 邀请台上的四位嘉宾逐一自我介绍，并抛出开场问题：**各家企业当前排在第一位的核心 AI 目标是什么？** ```mermaid flowchart TD accTitle: 企业核心 AI 目标与演进阶段 subgraph S1["技术与基础设施提供方"] direction TB N["Nimble<br/>Rob DiNuzzo"] --> N_G["核心：实时 Web 数据可信度、可审计性与 Token 效率"] W["WisdomAI<br/>Geetesh Iyer"] --> W_G["核心：Agentic 分析上下文引擎、兼顾算力与 Token 成本"] end subgraph S2["大型企业应用与业务方"] direction TB G["Gallagher<br/>Mark Bloom"] --> G_G["核心：跨越生产力工具，转向驱动销售与创收的用例"] A["American Century<br/>Tarun Sood"] --> A_G["核心：告别 PoC 试点，构建规模化运营级 AI 平台与 ROI"] end ``` Nimble 营销副总裁 Rob DiNuzzo 首先发言。Nimble 专注于为企业级 AI Agent 构建实时网络智能层（Live Web Intelligence Layer），聚焦于治理管控、Token 使用效率与回答准确性。他指出，当前行业已充分认知到 AI 的能力边界，但要真正将其业务化、工程化落地，核心症结在于“对 AI 的信任（Trust of AI）”——即企业能否信赖 Agent 从公开网络抓取的数据、获取过程是否完全可审计，以及在产出精准答案的同时能否保持极高的 Token 成本效率。 Arthur J. Gallagher 全球首席信息官 Mark Bloom 接着发言，风趣地调侃入场队伍“比泰勒·斯威夫特（Taylor Swift）演唱会的排队长龙还要夸张”。作为全球顶尖的保险经纪集团之一，Gallagher 拥有约 7 万名员工、业务遍布 65 个国家、市值约 600 亿美元。Mark 介绍，公司内部已将 50 多个用例成功推向生产环境，在内部员工生产力提升上成效显著；但由于经纪业务的本质依赖销售与市场，公司当下的首要目标是“调转方向盘（flip the script）”，全力推进能直接带来营收增长的业务用例（Revenue-producing use cases），并通过大规模员工培训和推广来加速落地。 WisdomAI 创始产品经理 Geetesh Iyer 介绍了其专注的 Agentic 数据分析平台。他表示，传统商业分析是一项成熟技能，但在 AI 时代需要全新的“肌肉记忆”与运转节奏。与传统确定性分析不同，Agentic 系统的很大一部分本质上是概率性的（Probabilistic）。WisdomAI 当前最重要的目标是充当上下文引擎（Context Engine），在底层数据平台收取的计算费用（Compute costs）与云厂商/模型巨头（Hyperscalers）收取的 Token 费用之间寻找最优解，在提供高度可解释、值得信赖的分析结果的同时，大幅优化企业的基础设施开销。 American Century Investments（美盛投资）首席数据官 Tarun Sood 随后介绍了这家管理约 3500 亿美元资产（AUM）的资产管理公司。针对 AI 战略，Tarun 强调其头号任务是“跨越概念验证（PoC）与试点阶段（Moving beyond the pilot phases）”。行业已经证明了 AI 技术本身的强大能力，现在的核心挑战在于如何建立可扩展的底座平台以支撑 Agent 和业务用例的大规模运营化落地（Operationalizing AI at scale），并确保每一个落地场景都能带来扎实的投资回报率（ROI）。 - **Plug and Play Ventures**：投资超 10 年，被投组合中包含约 33 家独角兽企业。 - **Arthur J. Gallagher**：市值约 600 亿美元，70,000 名员工，业务覆盖 65 个国家，已有超 50 个 AI 用例进入生产环境。 - **American Century Investments**：资产管理规模（AUM）约为 3500 亿美元。 --- ### [05:07](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=307s) Token 经济学与成本纪律：从盲目消耗到产出 ROI 驱动 Amit Patel 引导讨论转向企业在大模型推理成本与 Token 预算上的剧烈变化。早期的心态往往是“不惜消耗尽可能多的 Token 直到算出完美答案”，但如今像 AT&T 等大企业正面临推理预算严重超支的困境，甚至有传闻称特斯拉等公司开始为员工设定每人 200 美元的 Token 使用上限。Amit 向 Mark Bloom 和 Tarun Sood 提问：在数万名员工的大型企业体量下，真正的“成本纪律（Discipline at scale）”究竟该如何执行？ ```mermaid flowchart LR accTitle: 基础设施演进与成本优化类比 subgraph Cloud["上云迁移历程 (Cloud Migration)"] direction TB C1["本地机房上云初期<br/>(算力成本阶段性飙升)"] --> C2["退役旧数据中心<br/>(基础平台就绪)"] --> C3["云架构精细化调优<br/>(降本增效)"] end subgraph AI["AI Agent 演进历程 (Agentic Infrastructure)"] direction TB A1["早期盲目追求 Max Token<br/>(模型厂商主导指标)"] --> A2["搭建企业统一 AI 底座<br/>(用例规模化接入)"] --> A3["上下文工程与 Prompt 缓存<br/>(最小 Context 获取智能答案)"] end ``` Mark Bloom 指出，建立纪律的第一步是建立可度量的技术底座（Backplane），“无法测量就无法管理”。Gallagher 目前已经具备统一的底层控制平面，能够实时监控系统中有多少 Agent 正在运行、Token 的流向与消耗速率以及整体利用率。但 Mark 强调，Token 经济学仍处于非常早期的阶段，企业在与业务部门沟通时不应过分纠结于单一的 Token 账单，而必须聚焦于业务产出（Outputs）。 > "如果你花 10 万美元的 Token 能换来 50 万或 60 万美元的业务营收，根本没有人会在意那些 Token 成本。" Tarun Sood 深入剖析了行业心理的钟摆效应：大约半年前，模型提供商还在极力鼓吹“最大化 Token 消耗（Max Tokenization）”，甚至将其当作衡量生产力释放的标尺；如今市场迅速荡向另一极端，全行业都在高呼“模型效率（Model Efficiency）与低成本”。Tarun 将此过程与当年“企业上云（Cloud Migration）”的历史进行了深度类比： 1. **基础设施构建期**：从本地机房向云端迁移初期，由于平台尚未完全成熟，企业在承担云端算力成本的同时还要逐步退役本地旧数据中心，前期支出必然上升； 2. **用例赋能期**：企业必须先搭建一套可扩展的企业级 AI 平台底座，将 Agent 和各类用例平稳接入； 3. **架构优化期**：正如云上工作负载稳定后企业会重构架构降本一样，企业接下来将进入针对 Agent 基础设施的精细化迭代——利用上下文工程（Context Engineering）以最小的上下文窗口获取高质量的智能回答，从而大幅削减冗余 Token。 在技术实现细节上，Mark Bloom 补充指出，除了模型轮换（Model rotation）和精细化智能上下文之外，架构设计中必须强制引入 **Prompt 缓存（Prompt Caching）**。正如传统 Web 浏览器通过客户端缓存减少网络请求一样，Prompt Caching 可以避免向大模型云厂商支付重复上下文的高昂处理费用。 最后，Amit Patel 总结指出，此前业界对 Token 消耗的无序追逐也暴露出了一个关键隐患：企业对单一模型供应商的严重过度依赖（Over-reliance on a single vendor）。如果企业的关键业务系统全盘绑定在某一家模型提供商身上，将面临巨大的商业与系统风险，因此在技术架构层面构建“多模型架构（Multi-model Architecture）”已成为当下大型企业的必然共识。 ### [10:40](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=640s) 多模型架构与企业自研私有智能层 在探讨模型所有权与架构演进时，嘉宾们提到关于大语言模型（LLM）概念的提出与发展。企业界逐渐达成共识：仅采用多模型架构（Multi-model architecture）是不够的，构建专属的定制化 AI 模型正在成为一种现实需求。除了成本考量外，企业必须保护并沉淀自身的知识产权（IP）。近期行业中流行一种理念——“你可以租用模型，但必须拥有智能层”（You may rent the model, but own the intelligence layer）。要降低代码与 Token 消耗成本，建立自研模型与自有层是必由之路。 过去，企业架构主管常抱怨缺乏运营管理多模型架构的资源与管理开销，但当下的技术生态正在迅速降低这一门槛。例如在大会应用上亮相的初创平台（如 Ume AI 等），正在赋能顶级一级银行及大型金融机构，使它们无需庞大的底层基础设施或海量人力，即可无缝训练并部署定制化模型。 ```mermaid flowchart TD accTitle: Enterprise AI Intelligence Layer Architecture accDescription: Flow showing how enterprises rent foundation models while owning the proprietary context, indexing, and intelligence layer. subgraph FoundationModels["底层基础模型 (租用 / 接入)"] M1["开源/商用大模型"] M2["领域定制模型"] end subgraph ProprietaryLayer["企业专有智能层 (自主掌控 IP)"] direction TB ContextEngine["上下文引擎 & 元数据映射"] PropIndex["专有实时索引 & 智能缓存"] AgentEngine["Agent 编排与工具套件"] end subgraph EnterpriseOps["企业业务终端与落地场景"] DataTeams["数据团队 / 系统管理员"] EndUsers["业务分析师 / 终端用户"] Workflows["核心业务工作流 (生产环境)"] end FoundationModels --> ProprietaryLayer ProprietaryLayer --> EnterpriseOps ``` --- ### [12:12](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=732s) 沙盒到生产的断层：专有索引与上下文赋能 企业在落地 AI Agent 时面临普遍的痛点：Agent 在沙盒（Sandbox）隔离环境中表现完美，但一进入真实的生产环境往往就会崩溃瓦解。针对这种从沙盒扩展到生产环境的阻碍，嘉宾们探讨了其背后的核心断裂点。 Rob DiNuzzo 指出，核心瓶颈在于**规模化信任（Trust at scale）**以及对**专有索引（Proprietary index）**的构建： - **Token 成本与信任危机**：以私募股权（PE）或二级市场分析为例，若全球多名分析师每天反复查询最新股价或财报，会导致 Token 成本呈指数级膨胀，最终因预算壁垒导致 AI 被停用或失去信任。 - **实时网络检索与去重共享**：当 Agent 接入实时互联网时，不能每次查询都重复调用外部搜索，必须建立专有索引和缓存，将获取的信息沉淀并共享给后续的其他人员或 Agent。 WisdomAI 的 Geetesh Iyer 进一步强调，当前的技术环境已经跨过了“模型能力不足”的阶段（特别是在 Opus 等前沿版本发布后），问题转向了**如何注入精准的上下文**： - **打破不可能三角**：许多客户误以为准确率（Accuracy）、成本（Cost）与信任（Trust）三者无法兼得，要保证信任就必须牺牲成本或准确性。Geetesh 坚决反对这一观点，认为只要构建好上下文生态与底层智能层，三者完全可以兼顾。 - **优先赋能数据与管理团队**：在与某《财富》前十强（Fortune 10）企业的合作中，他们发现先为数据工程师和系统管理员构建上下文治理工具链，而非一开始就扑向终端提问用户，是项目落地的关键分水岭。 --- ### [15:55](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=955s) 运营元数据映射：决定 Agent 自主性的隐形支柱 讨论深入到企业决策流程的本质。相同的模型置于不同的上下文基础设施中，其输出结果和性能会有天壤之别。企业界数十年来一直在谈论“数据碎片化”，如今虽然换了新词汇，但根本痛点并未改变。 首席转型官（CTO）或首席运营官（COO）常宣称要在 AI 时代重构工作流，但一旦追问业务流程背后的**元数据（Metadata）**与底层逻辑，实际情况往往支离破碎： - **决策细粒度的缺失**：管理者通常只能描述宏观流程，却无法说明每一步决策的具体原因、考量要素、各环节耗时、瓶颈所在以及对应的成本开销。 - **AGI 降临与应用脱节**：“即便通用人工智能（AGI）明天就到来，绝大多数企业也根本不知道该如何使用它”，因为企业从未捕获并结构化自身的业务决策上下文。 - **流程映射先驱案例**：如致力于流程智能的企业（例如与摩根士丹利等顶尖金融机构合作的 Skan 等），其核心工作就是将整个底层运营模型在深度上下文中完整映射出来，为上层自动化铺平道路。 --- ### [18:35](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1115s) 衡量真实 ROI：从虚幻效率到损益表（P&L）与编排者转型 针对《财富》500 强高管的调研显示，虽然三分之二的企业已在生产环境中部署了 AI，但超过半数的高管表示**无法准确衡量其投资回报率（ROI）**。对 CEO、CFO 和董事会而言，“提高生产力”和“提升效率”往往是务虚的模糊指标（Fluffy metrics），他们真正关心的是损益表（P&L）上的硬指标——究竟是拉升了营收上限（Top-line），还是削减了运营支出（Opex）。 Rob DiNuzzo 从自身带领营销团队的经验出发，分享了如何将 ROI 具象化为硬性的用工成本控制与人员结构演进： - **初级岗位职能质变**：在四五年前，同等业务规模下需要招聘 5 到 6 名初级员工执行战术性任务；如今团队招聘的应届毕业生不再亲自执行基础操作，而是转型为 **“AI 工程师 / 编排者”（AI Engineers / Orchestrators）**。 - **编排者取代纯执行者**：从营销职能到“氛围编程”（Vibe coding）等技术领域，初级角色正从单点战术贡献者（IC）全面进化为 Agent 工作流的编排与管理者。 - **清晰的 ROI 锚点**：ROI 直接体现为**节约的人力编制成本（Headcount cost）**与 Agent 系统运行成本的差值，同时确保交付产出的质量与数量持平甚至远超原有多人团队的标准。 ### [20:42](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1242s) ROI 的归属与衡量：技术与财务的权责划分 在探讨 AI 代理的投资回报率（ROI）时，马克·布鲁姆（Mark Bloom）提出了一个非常务实且略带反差的观点：作为首席信息官（CIO），他根本不想亲自去计算或背负 ROI 的衡量责任，而是主张将这项问责权完全移交给财务团队。他直言，如果技术主管自己去算账，往往会陷入业务与财务层面的无休止争论，诸如“为什么在损益表（P&L）上看不到这笔收益”等质疑会层出不穷。因此，最佳策略是在项目启动之初就与财务、法务等部门紧密结盟，在动手前共同确立运营基准线（Baseline）。 一旦系统上线运行，技术团队负责提供实际运行数据，让财务团队依据既定基准去核算产出并出面倡导收益成果。当双方对测算结果存在分歧时，再通过迭代沟通达成妥协。“让财务团队做他们的专业工作，让技术团队做技术的工作”，CIO 作为合作伙伴协同推进，既避免了自说自话的尴尬，也确保了财务数据的客观公信力。 ```mermaid flowchart LR accTitle: AI ROI 测算与权责协同流程 subgraph S1["项目启动阶段"] A["技术团队"] --> C["确立基准线 Baseline"] B["财务与法务团队"] --> C end subgraph S2["项目落地与实施"] C --> D["落地 AI 代理与用例"] D --> E["交付运行数据"] end subgraph S3["核算与结果倡导"] E --> F["财务团队独立核算 ROI"] F --> G["财务出面倡导成果 / 迭代对齐"] end ``` --- ### [21:54](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1314s) 衡量 AI 影响力的三大核心利益相关者与综合影响力评分 吉特什·艾耶尔（Geetesh Iyer）将 ROI 的衡量视角进一步拓宽，指出在企业引入 AI 代理生态时，必须满足三类核心角色（Personas）的诉求，并以此构建一套综合的“影响力评分”（Impact Score）： ```mermaid flowchart TD accTitle: AI 实施的三大核心角色与影响力评分架构 subgraph Personas["核心利益相关者评估维度"] P1["业务 / 技术领导者<br/>(Business Leader)"] -->|"核心诉求"| M1["采纳深度 (Adoption Depth)<br/>真实创造价值的人数比例"] P2["一线管理与技术负责人<br/>(Admin / Tech Leader)"] -->|"核心诉求"| M2["准确率与省力程度<br/>(Accuracy & Effort Saved)"] P3["财务与损益团队<br/>(Finance / P&L Team)"] -->|"核心诉求"| M3["工时节省与财务回报<br/>(Person-Hours & P&L Impact)"] end M1 --> IS["综合影响力评分<br/>(Impact Score)"] M2 --> IS M3 --> IS IS --> Decision["综合决策与权衡<br/>(结合 Token 消耗与运行成本)"] ``` 在实际落地中，这套影响力评分的各项权重会根据企业规模进行调整。针对“最终由谁负责生成并主导该评分”的问题，吉特什指出，通常应由业务主管或技术决策者主导。因为引入 AI 的本质是优化工作量、缩短获取答案和执行任务的时间，对这些业务成果最在意的正是业务部门本身。管理者在掌握成本、Token 消耗等全局视图后，依据这一综合评分作出关键决策，从而有效避免陷入不切实际的空谈或无效指标中。 - **三大关键角色与关注指标**： - **业务/技术领导者**：关注“采纳深度（Adoption Depth）”，不仅看有多少人在用，更看重有多少人真正从中获得了业务价值。 - **技术与实施管理员**：关注系统的“准确率（Accuracy）”以及能否真正为一线员工减轻工作负担，提供可背书的技术保障。 - **财务与 P&L 团队**：关注实打实的“工时节省（Person-Hours Saved）”与底线财务收益。 --- ### [24:52](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1492s) 混合所有制与业务部门主导：让业务为 KPI 和 ROI 买单 塔伦·苏德（Tarun Sood）强调，ROI 的所有权不应是非黑即白的二选一，而是一种“混合模式（Hybrid Model）”，其归属取决于用例在何处赋能。例如在 IT 内部，团队引入编程代理模型来辅助软件工程任务，此时 IT 部门自然全权对该用例的 ROI 负责；但如果是为销售或营销部门启用的用例，则必须由相应职能部门的主管对 KPI 和 ROI 负责。技术与数据高管（如 CDO/CIO）扮演的是赋能者角色，负责提供模型与平台支持，但不应背负全公司所有 AI 项目的商业回报责任。 塔伦回顾了数据看板与云计算的发展历程，指出以前业务指标始终由业务端定义，随着云资源普及，企业逐步建立了内部费用分摊与可见性机制（Chargeback / Showback），AI 代理的 ROI 演变也将遵循完全相同的轨迹。阿米特·帕特尔（Amit Patel）对此高度认同并补充了关键行业数据：由业务部门主管亲自对其领域内 ROI 负责的 AI 项目，其成功规模化落地的概率高出整整三倍（3x）。因为一旦由技术部门包揽，它就沦为了一个普通的“IT 项目”；而由业务部门主导时，它才会真正转变为深刻的“业务转型（Business Transformation）”。 --- ### [27:26](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1646s) 风险治理与基准线：没有控制与底线就没有真正的 ROI 吉特什·艾耶尔进一步指出，ROI 测算中绝不可忽略的重要组成部分是“风险（Risk）”。生产级 AI 代理必须具备严格的治理与管控能力，例如建立完整的审计追踪（Audit Trail），明确解释代理为何生成特定回答、调取了哪些数据源，并支持屏蔽非美国本土源或不可信网站。他警示道，哪怕账面计算出的 ROI 再高，一旦发生数据泄露或因生成不受控、无法审计的内容而招致诉讼，一切回报都将瞬间归零。 塔伦与马克补充强调，治理往往容易被视作沉重的管理负担，但如果 ROI 权责由业务端持有，业务发起人便有了切身利益（Skin in the Game），进而由业务团队自发推动组织变革管理（Change Management），而非由 IT 团队强推。同时，这一切都建立在项目初期双方达成共识的数据基准之上。阿米特分享了 Guardian Life（卫报人寿）AI 负责人的深刻见解：“我没有所谓的 AI 战略，我只有业务战略，我需要搞清楚的是 AI 能否真正帮助我实现业务战略。”盲目为了做 AI 而做 AI 是失败的核心根源；现场嘉宾还打趣道，这周安全人员都跑去参加 Black Hat（黑帽大会）了，因此在场的每位管理者都必须自行在系统设计中扛起风险责任。 --- ### [29:37](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1777s) 应对技术快速过时：“持久层”与“一次性层”的架构解耦 面对技术日新月异、今天部署的方案明天就可能沦为技术债务（Technical Debt）与遗留系统的行业焦虑，马克·布鲁姆提出了极具前瞻性的架构设计理念：将“持久层（Durable Layer）”与“一次性/可丢弃层（Disposable Layer）”彻底解耦。 ```mermaid flowchart TD accTitle: 持久层与一次性层的架构解耦设计 subgraph Durable["持久层 Durable Layer (企业核心资产)"] CTX["上下文层 Context Layer"] WF["业务工作流 Workflow"] BL["核心业务逻辑 Business Logic"] end subgraph Disposable["一次性层 Disposable Layer (外部竞争演进)"] M1["Claude"] M2["OpenAI / GPT"] M3["Grok / 其他开源模型"] end Durable -->|"灵活插拔与无缝切换"| Disposable ``` 在马克看来，底层基础模型本身终将沦为一种“大宗商品（Commodity）”，各大模型厂商会持续展开激烈的商业与技术竞争。对于企业而言，模型是随时可以替换的“一次性消耗品”——今天喜欢 Claude，明天切换到 OpenAI，后天换成 Grok 都可以；而企业真正需要掌控并沉淀的，是包含上下文层（Context Layer）、业务工作流以及核心业务逻辑的“持久层”。这种将精力聚焦于上下文架构、静观模型厂商竞争并随时择优取用的策略，正是企业在瞬息万变的技术浪潮中抵御技术债务、保持系统长久生命力的关键解法。 ### [31:07](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1867s) 上下文健康检查与“AI 垃圾”治理 Geetesh Iyer 顺着 Mark Bloom 的观点进一步探讨了上下文管理的长期演进。他指出，如果放眼未来三到五年，上下文管理工具链中必须包含一个关键环节——对上下文进行定期的“健康检查”（Health Check）、新鲜度/陈旧度评估（Staleness Measurement）以及持续的评测（Eval）。如今很多企业热衷于接入各种模型、快速构建大量应用，但如果缺乏监控机制，这些随手塞进系统的“AI 垃圾”（AI Slop）在几年后不仅会失去意义，更会从资产演变成企业的负债。这种上下文的漂移甚至不需要等上几年，在短短两三周内，随着新数据的不断注入，同样的提问就可能产生完全不同的检索行为。 Geetesh 举了一个生产环境中极为典型的业务指标冲突案例：年度经常性收入（ARR）。销售团队对 ARR 的计算口径通常与财务团队截然不同。当销售人员向 AI 发问时，系统如果抓取了财务部门定义的表格或逻辑，就会套用错误的公式并输出看似正确实则不符合预期的结果。随着时间推移，如果销售团队废弃了某张旧表而其他部门接管了该数据空间，AI Agent 在调用底层信息时就极易产生混淆。因此，企业必须构建一个强大的评测生态系统（Eval Ecosystem）和动态护栏，确保 AI Agent 每次调用都能精准命中最新、最正确的业务上下文。 - **上下文陈旧度监测**：防止过期、废弃的数据表或定义被 Agent 误用，避免输出失真。 - **业务语义冲突消歧**：通过上下文引擎明确跨部门同名术语（如销售口径 vs. 财务口径的 ARR）的差异。 - **评测驱动治理**：建立自动化 Eval 机制，确保注入系统的每一项上下文持续产生正向价值而非增加技术债务。 --- ### [33:18](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1998s) 最小可行治理与模型商品化趋势 Amit Patel 随即指出治理过程中的一个常见误区：很多企业为了追求完美的治理框架和风险护栏，反而陷入了“治理惯性”（Governance Inertia），导致项目迟迟无法落地。他主张企业应建立一种“最小可行治理”（Minimum Viable Governance）和轻量级风险协议，在保障基本合规与安全的前提下快速推进部署，因为“过分追求完美会扼杀任何已经建立的势头”。 ```mermaid flowchart TD accTitle: 最小可行治理与数据底座架构 subgraph Governance["治理与接入控制"] A["业务需求提出"] --> B["Intake Agent 准入审查<br/>(合规/隐私PII/排重)"] B --> C{"是否已有成熟Agent?"} C -- 是 --> D["Agent Store 复用现有资产"] C -- 否 --> E["最小可行治理准入 (MVG)"] end subgraph DataFoundation["企业上下文与数据底座"] E --> F["AI-Ready 数据湖/仓<br/>(如 Snowflake 统一数据源)"] F --> G["上下文与元数据层<br/>(指标定义、关联关系、健康检查)"] end subgraph Execution["模型解耦与应用执行"] G --> H["Text-to-SQL / 对话式 BI"] G --> I["自动化业务 Agent 协作"] H & I -.灵活插拔.-> J["底层模型层<br/>(开源/闭源/SLM/LLM)"] end ``` Tarun Sood 对此表达了赞同，并坦言任何系统一旦推向生产环境，从那一刻起本质上就成了需要日后偿还的技术债务。但他强调，只要企业拥有严谨的工程规范并打造了“AI-Ready”的高质量数据底座，底层的模型本身就会变成纯粹的“大宗商品”（Commodity）。目前市场上关于开源模型与闭源模型的激烈争论在长期来看往往是个伪命题，因为模型供应商宣称的千亿参数在劣质、割裂的数据面前毫无意义。技术人员与业务主管的核心精力应当放在架构与上下文层的建设上，以保持未来根据成本和效率自由替换模型的技术灵活性。 --- ### [35:45](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2145s) 以 Agent 治理 Agent：Gallagher 的准入代理实践 针对企业内部容易出现的“Agent 野蛮生长”（Agent Sprawl）和上下文碎片化问题，Mark Bloom 分享了 Gallagher 独创的一套控制机制。为了在全公司范围内维持统一的上下文标准，他们专门开发了一个“准入 Agent”（Intake Agent）。在 Gallagher，任何团队或员工想要构建新的 Agent，都必须先通过这个准入 Agent 的全套“问询流程”。 这个 Intake Agent 会详细审查申请者的业务目标、是否涉及个人身份信息（PII）以及潜在的数据安全风险，从源头落实负责任 AI 治理。更巧妙的是，Intake Agent 还会实时检索企业现有的“Agent 商店”（Agent Store）；一旦发现已有相似功能的工具，系统就会直接建议申请者复用现有资产，坚决遏制重复造轮子和应用蔓延。这种“用 Agent 来构建和把关 Agent”的创新方案不仅在内部收获了员工的极佳反馈，也为大型企业解决 Agent 治理难题提供了极具实操性的范例。 --- ### [36:42](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2202s) 现场问答：化解数据政治、小模型应用与统一数据策略 在随后的现场观众问答环节中，一位观众提问：企业在构建统一的企业级数据环境时，该如何化解内部的数据政治阻力？能否通过领域小语言模型（SLM）来自主掌控数据捕获与配置环境，从而摆脱对外部 Token 计费的依赖并加速开发？ ```mermaid flowchart LR accTitle: 对话式 BI 与 Text-to-SQL 落地路径 A["业务 KPI 语义上下文<br/>& 元数据关系定义"] --> B["领域小模型 SLM<br/>(Text-to-SQL 转换)"] B --> C["企业业务数据库"] C --> D["精准指标输出<br/>替代传统 BI 看板"] ``` Tarun Sood 首先结合 American Century Investments 的实际经验作答。他指出，对于很多诸如“用对话式 BI / 问答取代传统 BI 看板”的场景，企业根本不需要调用最新、最昂贵的云端超大模型（如 Claude 最新旗舰），因为本质上这只是一个 Text-to-SQL 问题，轻量级的小模型（SLM）完全能够胜任。然而，SLM 能够跑通的前提依然是严密定义的上下文。他们在概念验证（PoC）阶段与业务伙伴紧密协作，不仅梳理了模型所需的元数据，还明确了关键 KPI 之间的关联逻辑，从而大幅提升了 SQL 生成与查询的转化准确率。Tarun 直言，只要有组织存在就必然存在“数据政治”，但化解这一难题的最有效武器是**强有力的核心业务赞助人（Business Sponsor）**，由业务端主导变革管理。 Mark Bloom 随后补充了 Gallagher 的全球化经验。他强调企业文化是打破数据孤岛的决定性力量。依托高度协同的文化，Gallagher 推行了全球统一的数据战略，致力于打造唯一的真实数据源（Single Source of Truth）。他们正在将全球多达 800 个分散的数据仓库和存储库逐步淘汰，统一迁移至 Snowflake 实例中。在权限管理上，他们直接沿用业务系统现有的访问权限规则，从根本上兼顾了数据流通与安全隔离。 --- ### [40:14](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2414s) 现场问答：多方博弈下的治理落地与结果导向 另一位现场观众提出了一个直击痛点的问题：当 IT 基础设施、负责任 AI 委员会、信息安全、数据团队等多个部门各自持有不同的诉求与利益关切时，在一个充满未知的技术探索期，企业究竟该如何平衡各方利益以确立所谓的“最小可行治理”？ Tarun Sood 坦率地指出：“治理是全行业最难啃的硬骨头，因为人人都高喊需要治理，却没人愿意真正动手去承担治理工作。”对于身处强监管环境的金融服务机构而言，合规与治理不是可选项，而是必须内嵌在每一个业务用例中的刚性底线。 为了说服持怀疑态度的技术与业务伙伴，Tarun 的团队选择用实际效果说话： - **治理完备的数据资产**：指标定义清晰、元数据编目完整的系统，AI Agent 能够快速理解上下文并精准执行指令。 - **缺乏治理的混乱系统**：面对模糊的数据，Agent 只能在原地打转、“原地空转”，根本无法给出有效答案。 正是通过这种直观的对比实验，他们向各部门证明了数据治理并不是阻碍业务的繁文缛节，而是让 Agent 变得更加聪明的核心前提。Amit Patel 总结道，打破部门间治理博弈的最佳路径就是坚持“结果导向”（Outcome-based），通过设定清晰的业务成果目标并在小步快跑的测试中验证成效，进而倒逼出最合理的治理尺度。 ### [42:30](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2550s) 敏捷治理与 Hub-and-Spoke 落地架构 在探讨企业级 AI 的治理时，嘉宾们指出“最小可行治理”（Minimum Viable Governance）在不同团队之间的定义截然不同——对于塔伦（Tarun Sood）的数据团队、马克（Mark Bloom）的 IT 团队或是创业团队而言，治理的颗粒度与侧重点各有差异。要实现有效的治理，首先需要明确业务问题的类型，并确立该场景所追求的“最低预期结果”。嘉宾们强调，业内根本不存在所谓绝对“正确”或一步到位的治理标准，企业不应因追求完美治理而陷入停滞，而应先设立底线并躬身入局；一旦在具体业务中验证了某种机制行之有效（例如塔伦团队的成功落地案例），便可以以此为范本在全公司推广沉淀。这种由下至上证明“此路径可行且合规”的过程，构成了治理体系中最坚实的基础层。 ```mermaid flowchart TD accTitle: Hub and Spoke Enterprise AI Governance Model subgraph Hub["中央层：AI 卓越中心 (AI COE)"] H1["最小可行基础设施与数据平台"] H2["企业风险容忍度与基准规范"] H3["全企业推广与能力沉淀"] end subgraph Spokes["业务单元 (Business Units)"] BU1["业务线 A：场景与边缘案例探索"] BU2["业务线 B：快速试点与 PoC 验证"] BU3["业务线 C：高价值成果输出"] end H1 --> BU1 H1 --> BU2 H1 --> BU3 H2 --> BU1 H2 --> BU2 H2 --> BU3 BU3 -.->|"成功经验沉淀放大"| H3 H3 -.->|"赋能其他业务线"| BU1 H3 -.->|"赋能其他业务线"| BU2 ``` 从大型企业的成熟实践来看，摩根大通（JPMorgan）、沃尔玛（Walmart）以及已将 50 多个用例投入生产环境的加拉格尔（Gallagher）等行业领军者，无一例外采用了“轴辐模式”（Hub-and-Spoke）。在此架构中： - **中央枢纽（Hub / AI COE）**：负责搭建最小可行基础设施（MVI）、统一数据平台并界定风险容忍度与合规底线，确保基础设施层足够敏捷。 - **业务辐射端（Spoke / 业务部门）**：拥有充分的自主权，自主决定具体业务用例及边缘场景的部署，确保探索步伐不会因繁琐的审批流程而落后于市场竞争对手。 - **双向赋能闭环**：单个业务单元中跑通的成功方案，会迅速通过中央枢纽沉淀并扩展为整个企业的通用能力平台。 --- ### [44:40](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2680s) 有状态上下文与“四分卫”Agent 治理架构体系 针对模型运行时的状态与上下文获取问题，嘉宾们一致指出，现代企业级 Agent 必须是“有状态的”（Stateful）。智能体必须在知识图谱和历史交互的上下文环境中保留并提取关键信息，才能进行准确的业务分析与决策推演。然而，单纯依赖模型本身的 Token 处理能力远远不够，企业必须在上下文层之上构建一套稳固的“测试与运行封装架构”（Harness）。 ```mermaid flowchart LR accTitle: Quarterback Agent and Context Harness Architecture subgraph ContextHarness["统一治理封装层 (Context Harness)"] QB["四分卫 Agent (Quarterback Agent)"] M1["模型层 (Model Access / Tokenization)"] C1["上下文引擎 (Context Engine / 知识库)"] E1["综合评估矩阵 (精度、成本、个性化)"] QB <--> M1 QB <--> C1 QB <--> E1 end QB -->|"全域视角统筹输出"| Enterprise["企业各业务部门与应用场景"] ``` 为了打破模型与上下文相互割裂的局面，系统需要一个具备全局视野的“四分卫智能体”（Quarterback Agent）作为统一纽带。这个核心枢纽承担着多维度的协调职责： - **全域监控与评估**：四分卫 Agent 同时审视基础模型与动态上下文，并在全局范围内对输出精度、计算成本、业务风险等多项指标进行实时评估与权衡。 - **应对模型商品化**：即便底层大模型日益 Token 化或被快速迭代替换，企业依然需要围绕每一种模型构筑专属的治理 Harness；随着开发工具的演进，构建此类 Harness 的效率正在大幅提升。 - **最终总结（TL;DR）**：构建企业级 Agent 的核心要义在于打造这一套四分卫生态系统——以强劲的上下文引擎驱动知识库与个性化模型访问，实现兼顾灵活性、成本可控与稳健治理的智能体落地架构。 在热烈的讨论中，主持人提醒时间已到，全场以掌声向嘉宾们分享的实战见解致谢，圆满结束了本次关于企业级 AI Agent 战略落地的专题研讨。 ## Notable quotes > [05:07](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=307s) 无法测量就无法管理 — Mark Bloom > [05:07](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=307s) 如果你花 10 万美元的 Token 能换来 50 万或 60 万美元的业务营收，根本没有人会在意那些 Token 成本。 — Mark Bloom > [10:40](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=640s) 你可以租用模型，但必须拥有智能层 — Amit Patel > [15:55](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=955s) 即便通用人工智能（AGI）明天就到来，绝大多数企业也根本不知道该如何使用它 > [20:42](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1242s) 让财务团队做他们的专业工作，让技术团队做技术的工作 — Mark Bloom > [27:26](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1646s) 我没有所谓的 AI 战略，我只有业务战略，我需要搞清楚的是 AI 能否真正帮助我实现业务战略。 — Guardian Life AI 负责人 > [33:18](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=1998s) 过分追求完美会扼杀任何已经建立的势头 — Amit Patel > [40:14](https://www.youtube.com/watch?v=Grj8hf1ojZc&t=2414s) 治理是全行业最难啃的硬骨头，因为人人都高喊需要治理，却没人愿意真正动手去承担治理工作。 — Tarun Sood ## Mentioned Tarun Sood, Amit Patel, Geetesh Iyer, Mark Bloom, Rob DiNuzzo, American Century Investments, Plug and Play Ventures, WisdomAI, Arthur J. Gallagher, Nimble, Nike, Walmart, Mercedes, AT&T, Tesla, Ume AI, Skan, Morgan Stanley, Guardian Life, Snowflake, JPMorgan, Claude, OpenAI, Grok ## Video description Tarun Sood - Chief Data Officer, American Century Investments Amit Patel - Partner, Plug and Play Ventures Geetesh Iyer - Founding Product Manager, WisdomAI Mark Bloom - Global Chief Information Officer, Gallagher Rob DiNuzzo - VP of Marketing, Nimble In order to reach the end goal of deploying AI agents at scale, organizations must first align clear business objectives with technological capabilities. This cross-industry panel dives into the structural and strategic foundations necessary to ensure autonomous systems push the enterprise forward. Learn how forward-thinking leaders are selecting use cases, navigating change management, redefining human workflows, and establishing key performance indicators for agentic operations. Panelists will discuss strategies for securing executive buy-in and building a resilient organizational culture ready for an AI-augmented workforce. Walk away with tactical insights on integrating AI agents seamlessly into your long-term corporate strategy. [Source](https://www.youtube.com/watch?v=Grj8hf1ojZc)

[5:07] Mark Bloom: 无法测量就无法管理 [5:07] Mark Bloom: 如果你花 10 万美元的 Token 能换来 50 万或 60 万美元的业务营收，根本没有人会在意那些 Token 成本。 [10:40] Amit Patel: 你可以租用模型，但必须拥有智能层 [15:55] 即便通用人工智能（AGI）明天就到来，绝大多数企业也根本不知道该如何使用它 [20:42] Mark Bloom: 让财务团队做他们的专业工作，让技术团队做技术的工作 [27:26] Guardian Life AI 负责人: 我没有所谓的 AI 战略，我只有业务战略，我需要搞清楚的是 AI 能否真正帮助我实现业务战略。 [33:18] Amit Patel: 过分追求完美会扼杀任何已经建立的势头 [40:14] Tarun Sood: 治理是全行业最难啃的硬骨头，因为人人都高喊需要治理，却没人愿意真正动手去承担治理工作。

[0:08] Mark Bloom: that line is the longest line I think I've ever seen anywhere. Makes Taylor Swift look silly. [0:08] Mark Bloom: flip the script [5:07] Mark Bloom: As you're talking to your business partners, focus on the outputs rather than the token cost, right? Because again, if you're making $500,000 or $600,000 and spending $100,000 in tokens, nobody's going to care. [17:24] autonomy could—AGI could arrive tomorrow and most enterprises wouldn't know how to use it [18:35] fluffy metrics [18:35] Rob DiNuzzo: marketing AI engineers [27:26] Head of AI at Guardian Life: I don't have an AI strategy, I have a business strategy and then I got to figure out like if AI can truly help me with that business strategy. [31:07] Geetesh Iyer: AI slop [33:18] minimum viable governance [33:18] Tarun Sood: The moment you put [anything] in production, it is technical debt. [40:14] Tarun Sood: governance is the hardest nut to crack because everyone wants governance, but no one wants to do it.

[0:08] Thank you all for being here. I know the  line is really crazy outside, so hopefully [0:13] uh you had someone to talk to while while you're  in that queue. Uh but uh as quick, you know, [0:19] as we kick things off, I'll introduce myself. Um  my name is Emit. I'm a partner at Plug-and-Play [0:24] Adventures. Uh we're effectively a global  venture firm with the world's largest corporate [0:28] uh innovation networks. We work with firms like  um Nike, Walmart, Mercedes uh and our goal is to [0:34] kind of help them connect with emerging companies  from around the world uh to support their growth, [0:39] innovation and AI activities. Um and we use the  interactions uh um or the data points from these [0:47] interactions to make investment decisions. So  we've been investing for over a decade and we have [0:53] um about 33 unicorns in our portfolio now. Um, and  I would love for the panel to introduce himself [0:59] as well. Um, and as you introduce yourself, it  would be great if you can highlight what is the [1:05] number one AI objective that you have at your  firm today. Awesome. Thank you, Amit. My name [1:11] is Rob Denuzo. I'm the VP of marketing at Nimble  and we built the live web intelligence layer for [1:15] enterprise AI agents, uh, focusing on governance  and control, uh, token efficiency and accuracy. [1:22] uh our number one focus uh both internally  and for the customers that we build for [1:28] uh is is trust of AI, right? Uh I think we're to  the point now where we all know AI's capability, [1:33] but the ability to operationalize AI comes down  to trust. Can you trust what it's sourcing from [1:38] the live web? Can you audit what it has gotten  you? And at the end of the day, can you make sure [1:42] that what you're getting are not only accurate  answers, but token efficient answers. And that's [1:46] what we're focused on both internally and for our  uh and for our customers. Morning everybody. I'm [1:52] I'm Mark Bloom. I'm the global chief information  officer at Arthur J. Gallagher. Um first of all, [1:57] thank you guys very much for burying that  line. That line is the longest line I think [2:00] I've ever seen in anywhere. Makes Taylor Swift  look silly, right? That's crazy line. Um so, [2:05] so uh Gallagher is one of the world's largest  insurance brokerages. Um about 70,000 people, [2:11] about 65 countries, about a $60 billion market  cap. So we'll talk a lot about some global aspects [2:16] today. Um I I think is why I think about my number  one objective for the year. We've done over 50 use [2:22] cases now in production. So we've done a really  good job relative to productivity. Uh but because [2:27] we are a brokerage, right? It's that for us it's  about sales and marketing and so now it's time [2:31] to flip the script to more revenue producing  use cases. How can we drive more revenue with [2:35] our use cases? So we're pushing adoption, we're  pushing training and therefore pushing lots more [2:39] revenue use cases. So thank you guys very much  for being here. Look forward to the discussion [2:43] today. Awesome. Um good morning everybody. My  name is Gesh. I'm the founding product manager [2:49] at Wisdom AI. Uh so for folks who don't know,  wisdom AI is an agentic analytics platform. [2:54] So analytics is something that we've all done.  But in the age of AI, you require a different [3:00] rhythm. You need a different muscle to be able  to do that. And wisdom AI is one of the best [3:04] platforms out there to be able to do that. And  uh one of the things that separates you know [3:10] u an agentic platform from others is a large part  of it is probabilistic in nature. So how one of [3:16] the largest things that one of the things that we  spend a lot of time uh solving is a the ability [3:22] to solve the answers which are highly explainable  and trustworthy and something that is top of mind [3:28] for many people is like we have the data platforms  that want you to spend on them for compute. You [3:34] have the hyperscalers who want to spend on them  on tokens. How do we make sure that we sit outside [3:39] of that and optimize for both of them while at  the same time focusing on trust and accuracy. [3:44] So trying to solve that whole eight trifecta uh  as a context engine uh for the asentic analytics [3:50] world. Well hi everyone my name is Tun Sud and I'm  the uh you know chief data officer for American [3:58] Century Investments. American Central Investments  is an asset manager. Uh we have about you know 350 [4:05] odd billion in AUM. uh our focus um you know is  basically helping uh you know customers whether [4:15] they are institutional clients or retail clients  uh to achieve their financial outcomes. Um so [4:22] that's a little bit of our company. Uh what I do  and what my biggest priority from it when it comes [4:28] to AI is moving beyond the pilot phases. Right?  We've already shown that AI is a very capable [4:36] uh you know technology. How do we leverage it  to drive productivity? Uh as Mark was mentioning [4:41] we've also put a lot of use cases in production.  But as we start thinking about operational AI [4:47] operationalizing AI at scale how do we do that?  and how do we have a scalable platform that can [4:53] enable the use cases? Uh how do you make sure that  um you know there's better ROI for your use cases? [5:00] Those are the things that we are focusing on.  So I'll I'll kick off with with a question and [5:07] um this is something that we've we've been seeing  across our enterprise ecosystem um as well as in [5:14] public right there. There's a real shift happening  in terms of kind of how enterprises think about [5:19] uh token usage, right? Before it was, you know,  like uh use as many tokens as you want until [5:25] you get the the the perfect answer. Uh but that  that kind of framework is shifting because you, [5:31] you know, you're seeing large enterprises who  are blowing past their budgets like inference [5:36] budgets like AT&T and many others. Um, so I  guess I I'll start this question off with you, [5:42] Mark and Taran. You know, what does discipline  look like at a scale? I mean, you mentioned you, [5:49] you know, you're a big company with so many  employees. I mean, I I heard Elon Musk, you know, [5:54] they li Tesla's now limiting what, $200 per  employee to usage. So, Mark, at your scale, [6:00] what does discipline look like for you when  it comes to kind of the usage? Uh, well, first [6:04] first of all, you got to get the the backpoint.  You got to be able to measure, right? So if you [6:08] can't get the data to have your baselines, you  can't measure it. So first and foremost, you got [6:12] to have some form of a back plane to know what the  heck's going on. So we have a back plane today. I [6:16] can see how many agents are going on. I can see  what people are doing with tokens. I can see the [6:20] utilization. Uh so that part's that part's good.  Uh I would also say we're an early stage, right? [6:25] This whole concept of tokenization is only a few  months old. So we're still experimenting with [6:30] the business model. We'll see what happens in the  near future. But one of the things that I think is [6:34] super super interesting and probably one of the  important takeaway points I would offer to you [6:38] is as you're talking to your business partners,  focus on the outputs rather than the token cost, [6:44] right? Because again, if if you're making5 or  $600,000 and spending $100,000 in tokens, nobody's [6:50] going to care, right? And so again, to me, the  conversation should and we're starting to see [6:55] it shift from people talking about, oh, how much  is this costing to actually what am I getting, [7:00] right? what's the output from what I'm getting  from the from the token utilization and all of [7:04] a sudden that stuff's going to start dissipating  but again early stage we'll see how we'll see how [7:08] it continues to evolve. Yeah. uh you know in my  case like what I'll say is see tokenization when [7:16] and the max tokenization was a very good concept  and and it was really good for the you know model [7:24] providers right it's it's really good for them if  you think about at least 6 months back everyone [7:29] was talking about max tokenization because that  was how they were measuring productivity now we [7:34] are shifting completely and on the other spectrum  and all we're now talking about model efficiency [7:40] how Can we make models more efficient and  cheaper? So I think there two sides of [7:45] that spectrum and somewhere in between is where  most of the enterprises are going to lie, right? [7:50] Um so an apps comparison that I think of is if  you go back to the cloud computing days when we [7:57] were starting to build cloud you know move our  workloads to the cloud uh there was a lot of [8:03] like compute costs that you're starting to acrew  because you're maturing right we were maturing [8:07] from an on-prem to a cloud platform and the to  and the compute was starting to gain here but then [8:14] you're starting to retire your legacy uh you know  legacy onrem data centers. So I think similarly [8:22] uh as we start utilizing more and more AI the  initial cost is going to be there like you have to [8:28] have a platform you have to have a platform that  can enable use cases as well as that can enable [8:36] you know your agents. So once you build that out  then what Mark was referring to makes ton of sense [8:42] right you can go and enable these use cases on to  this platform. So there are those upfront costs [8:48] that you have to you will you'll have. But then  as you start building and as you start looking at [8:54] your utilizing that platform this is where things  get interesting because you're now looking at all [9:00] right what is a context how can you minimize the  context so that you get an intelligent answer [9:06] and that is where you start becoming more and  more efficient. Um but I think it will be the [9:11] next iterations uh as similar to what we did with  cloud as we start and we moved a lot of workloads [9:18] to the cloud then we start thinking about okay  what is that optimal architecture for our cloud [9:23] to be efficient similarly we'll come back to our  agentic infrastructure and say this is the most [9:29] optimal context window for us context engineering  for us so that we are minimizing tokens uh and [9:36] I think that will be our next iteration Yeah, as  can I just add one so as he talked about the model [9:41] rotation and the smart context which I completely  agree with. One of the other things you should [9:45] make sure of with you know within the model itself  or within the way you're architected is is prompt [9:51] caching. So if we think about the way people  have always cached right up in a browser right [9:55] you're now getting into prompt caching. So are  they are the prompts actually getting cached to [10:00] reduce all those cycles that the hyperscalers take  advantage of. So something else you want to be [10:04] thinking about. So just to add on to the technical  context. Yeah. And one thing I would mention say [10:08] as well like one thing in my opinion that the  that the token maxing has brought to light is [10:14] um the over reliance on a single vendor quite  frankly speaking right think about this like [10:20] if all of your critical applications rely on  a single model provider I mean I don't have [10:25] to explain why that is you need to like really  consider like and so we've actually within our [10:30] ecosystem we've been talking about like uh the  idea of like companies need to be thinking about [10:35] building a mult multimodel architecture and this  is not something I came up with. I mean it was a [10:40] palunteer not palenteer CTO he introduced the idea  of KLM's uh about 3 years ago actually over three [10:46] years ago at that point. Uh and so I think there  is a real kind of uh necessity for enterprises [10:52] to look at even creating not just a multimodel  architecture but looking at developing their own [10:58] custom AI models. I mean outside of the cost aside  you know enterprises also need to think about um [11:05] you know how are they going to build out their own  IP right we're hearing this idea about at least in [11:10] the last month about owning the you may rent the  model but own the intelligence layer and I think [11:15] the only way you can actually do that and even  reduce like the code token cost is thinking about [11:21] uh building out your own models now when I used to  have conversations with like heads of enterprise [11:25] architects uh in many of these enterprises you  know the number one thing I always used to get [11:29] is that well I don't have the operational overhead  or the resources to be managed such a you know a [11:35] multimodel architecture but I think that landscape  is changing I mean there are companies out there [11:40] that I know there's one company that I actually  saw on the AI4 app um UMI UI that is what they're [11:46] building actually they're actually working with  some top tier one banks within our ecosystem and [11:50] network where because and a lot of our major  like top banking partners financial services [11:55] partners are vigorously thinking about like  building out their models and right now some [12:01] of the platforms like Umei that are coming  out they help you do it pretty seamlessly [12:05] without needing such a large infrastructure um  or resources. Um but I I also want to kind of [12:12] uh you know talk to you Rob and and Gesh you  know one of the things I constantly hear for [12:17] example from our ecosystem is while agents work  great in a sandbox environment they tend to fall [12:23] apart as soon as you get into production and  I and I've heard this consistently. I mean the [12:27] fact that you have 50 use cases in production  that's a that's a hats off to you because [12:31] uh it's it's not something that's very very common  uh mark but so as kind of you know as you know you [12:39] know with experience of working with different  customers what is I'm curious what is the single [12:45] biggest breaking point that you know that prevents  scaling from from a sandbox to to production yeah [12:53] um for me I'll go back to my previous statement  around trust right we We know AI is capable, but [12:58] can you trust it at scale? And part of this, and  I'm going to double down on what Mark just said, [13:02] beyond just caching is is building a proprietary  index, right? If you have many people accessing [13:07] this throughout the world, um let's say you are  working in PE or something to do with the live [13:12] markets and you have multiple people every day  querying what is the current price of what are the [13:16] earnings reports of. This builds up token costs at  the long scale which at the end of the day drives [13:22] both mistrust and basically disuse of of AI if  you're hitting barriers of of token cost. So [13:29] this at the end of the day I think we're doing web  search right. So we're enabling agents to connect [13:32] to the live web. Um you need to have a way that  you're not just repeating yourself or your agents [13:37] just aren't repeating themselves every time you  query something. You have to ground them, but you [13:41] also have to make sure that they are sharing  their information and storing that somewhere [13:45] proprietary for your other people to access or  your other agents to access down the road. Yeah, [13:50] I I think caching and u you know indexing as  a as a concept I think is is pretty critical. [13:57] But I think another thing that we see a lot of our  customers now ours is an agent analytics platform. [14:01] So essentially what you need to do is you provide  a lot of context. We have a context engine where [14:05] you build a lot of that context. the barrier to  entry for AI to make sense is pretty large. uh [14:13] because the when chachd came in 3 and a half years  ago, everyone saw thought that hey you know what [14:18] it's going to solve world hunger you know what  you pass everything to the model the model will [14:22] generate something and that's it let's go home but  I think more and more organizations are realizing [14:26] that hey in order for us to give it the right kind  of it's not a model problem anymore in the sense [14:31] that as you said you could use different models  for different things now they're like hey now I [14:35] know that the models do a great job and especially  after you know January the January version of Opus [14:40] it's a groundbreakingly different world we live  in now. So now the models are not the problem. [14:45] How do we infuse the right kind of context and  how the barrier to entry into this ecosystem is [14:51] lowered? What is the tooling we have? Uh and then  along with that how can we optimize for that holy [14:58] triactor of accuracy, cost and trust. Yeah, I  often hear a lot of our customers say that hey [15:03] you know what if you have to optimize for trust  the other two have to be compromised. I refuse [15:08] to believe that. uh it is a hard problem out of  the box if we think that model is the only thing [15:13] that's going to run. If we build that intelligence  layer, if we build that context ecosystem and make [15:19] it easy for the users or the data admins or the  end users to be able to use it, uh the entire [15:26] framework essentially becomes easy. We recently  did something with a fortune 10 company and one [15:32] of the first things that they said that hey you  know what the fact that you focused a lot on the [15:37] data teams or the admin teams uh and not first on  the end user who's going to ask questions on data [15:44] uh kind of made a lot of the difference uh and  I think that's where indexing trust and creating [15:49] that toolkit that engine that empowers all of this  I think what you mentioned that toolkit around [15:55] like the context around the models it's extremely  critical because it's actually shown and known [16:00] that you can have same model in two different  settings and the performance entirely depends [16:06] on what what is the infrastructure around it like  what does that built look like so the output is [16:11] going to defer from what the layer that you've  built around the model that that is significantly [16:15] going to influence the output now the other thing  you mentioned enterprise like you mentioned the [16:20] context layer right this is kind of a the idea of  like hey you need to be able to like map out your [16:25] entire enterprise decision making and all the data  points. We've been talking about this. I mean, [16:31] it's it's got a word, it's got a label assigned  to it, but this is not a new concept. Uh right, [16:36] like we've been talking about data  fragmentation for for for decades at this point. [16:41] Um uh uh and one of the critical things that  I see like with with enterprises is you know I [16:48] have conversations with like chief transformation  officers and COOs and like oh yeah we want to we [16:53] want to redesign our workflows or we want to  reimagine our workflows in the age of AI but [16:57] the the reality I always see is that whenever I  ask them okay what does your workflow what does [17:02] your process look like? Yes, they can tell me  at the high level, right? But if once you get [17:07] into the meta data behind their operating model,  that's where kind of things kind of fall apart. [17:12] And so I think you know where we've seen success  especially when it comes to AI adoption is folks [17:18] who've actually spent considerable amount  of time in building out that context like [17:24] what do you want to call like just like having  that meta data uh behind their operating model, [17:29] right? like which workflows are being used, you  know, who's using it, the bottlenecks, you know, [17:34] how much is costing you, how long, and especially  where it becomes important is the nuances, [17:40] right? If a decision is made, you know, why was  that decision made? What factors were considered [17:46] you? I mean, we talk about autonomy like agents  becoming autonomous. Um, uh, but frankly speaking, [17:52] I mean, autonomy could AGI could arrive tomorrow  and most enterprises wouldn't know how to use it. [17:59] uh uh because that that to your point that  context they're in the captured there is a company [18:04] actually um called scan SK an uh they actually  have a booth here too uh one of our you probably [18:10] know them I don't I don't know them oh you never  heard of them okay I think they may have met Gala [18:16] back a few years ago but they actually work with  several of the top tier financial institutions [18:20] um and that is what exactly what they do one of  the workflows that Morgan Stanley for example was [18:25] building a critical workflow one of the first  thing that they did was actually like map out [18:29] the entire operating model in that context which  was super important. But let me let me uh jump [18:35] gear a little bit too. Um we actually conducted a  survey uh with the fortune like fortune 500 senior [18:41] leaders and our survey found that threethirds uh  of the partners have AI in production. So similar [18:47] to what you mentioned Mark like you've got 50  use case in production but over half of them [18:51] say that they couldn't measure ROI. uh and you  you mentioned earlier as well like it's really [18:56] important to recognize understand the output you  know not just like how many tokens are being used [19:02] um so I'll start with you uh um guys like you  know how are you measuring the output or how [19:09] are you measuring the ROI because anytime I've  had a conversation with the CEO CFO or bed board [19:14] members you know metrics like productivity and  efficiency these are fluffy metrics like they want [19:19] to know what is the impact on their P&L like is it  improving my top line or is at reducing my opex. [19:24] So how how do you measure ROI that can tangibly  be tied into the P&L? Yeah. So from my perspective [19:30] leading a marketing team uh in years past uh even  as recently as probably four or five years ago, [19:36] we would have had to hire at our scale five to six  very junior employees to do very tactical things. [19:41] Um so way I can measure is is what type of people  are you hiring? Call it on the junior level. You [19:47] still need managers, you still need directors,  but uh for example, we've hired three recent [19:51] college graduates and they are what we call uh  uh engineers. They are AI engineers on marketing [19:56] teams that are building agents and managing agents  versus doing the tactical work themselves. So [20:02] measuring this as an ROI is is a function of how  much are you spending on this agentic workflow or [20:07] these agentic workflows uh that you would have  had to invest in in people costs. And I'm not [20:11] saying we are not hiring anymore. We in fact  very much are. But the types of people we're [20:15] hiring are orchestrators. You hear this all over  organizations, not certainly just in marketing u [20:20] especially in vibe coding especially. So it's it's  it's an interesting sort of uh evolution of what [20:26] this sort of junior role looks like and it's an  orchestrator versus an individual contributor at [20:31] a tactical level. So we measured that ROI in in in  saved headcount cost uh and making sure that we're [20:36] also having the output that we would expect if  not better than having many more people on board. [20:42] So, so, so kind of an interesting perspective  on this. I don't I don't want to measure the [20:47] ROI. Meaning I as the CIO don't want to  measure the ROI. I want my finance team [20:52] to take the accountability. Honestly, I want my  finance team to take that because otherwise you [20:55] get into these battles of I can't see it on  the P&L or I can't see it here. Right? So, [21:00] what we have done is we continue to work in  partnership with our finance teams, legal teams, [21:05] etc. I I referenced before establishing a baseline  relative to some of the metrics I talked about [21:10] before. We also get baselines before we start,  right? So what is our baseline that we're [21:15] operating off of? We get the right agreements and  then when we implement it, then we can measure it [21:19] and we get the finance team to advocate it rather  than me. Sure, I'm a partner in it. But again, [21:24] this is something that you know from an ROI  standpoint, let the finance guys do their job. [21:28] Let the tech guys do their job. So who would you  who would own producing it then? Because if you're [21:32] the technology leader, would you exactly right?  So what we do is we again we'll we've implemented [21:37] these use cases. We give the data to finance  and let them provide the results and if there's [21:42] a challenge we will go back and we'll iterate  through it to to actually get to a compromise and [21:47] sometimes we don't agree but generally we agree on  most of it. Okay. U I I look at it a little more [21:54] broadly. First uh I think there are three major  stakeholders and I'm broad I'm doing some broad [22:00] strokes here Mark give me some leeway here. Uh the  broadstrokes are there are three major players. [22:06] The first bear when you are introducing AI  into agentic ecos into your agentic into your [22:11] ecosystem the first person is the technology  leader or the business leader who's evaluating [22:17] it. So what does what matters most to them and we  feel the most critical thing for them is adoption [22:23] depth. This is like hey how many people are using  it's not just how many people are using it how [22:28] many people actually drew value out of it. So the  most critical ROI for that person is like adoption [22:34] depth. The second person is the technical you  know the leaders a lot of people over here who are [22:40] essentially going and these are the people who  are making it come up come come alive. Now what [22:44] matters most to them the thing that matters most  to them is it accurate uh is it saving uh effort [22:51] of the people. So making sure that accuracy and  something that they can actually vouch for for [22:58] that data leaders or that admin persona. And  the third persona is the finance and P&L team. [23:03] That's where and that is where we're talking  about you know person hours saved. So at a very [23:09] high level we call it an impact score and we work  with our customers on creating that impact score [23:14] internally. So you are addressing three personas  as part of any AI implementation. It's the leader, [23:21] it's the admin who's actually on the ground  doing tactical work and third is the P&L team [23:25] who's making the decision. How do we create  and the impact score is a combination of these [23:31] three and depending on the size of the company  the weightage for each of these is different. [23:36] So yeah and in your scenario who would who  would be accountable like in generating that [23:42] when you have three different personas  like that's fantastic answer typically [23:46] uh it is the cos the technical leader or the  business leader because at the end of the day the [23:52] value is something what is what are we doing with  AI we are trying to optimize effort we are trying [23:58] to reduce time to get to an answer we are reduce  time to do a task and if that has to be done who [24:04] cares the most it's the business leader. Now we  give that the highest priority. This is what we [24:11] see being the uh you know the most prevalent mode  of operation but then giving them viewpoints into [24:18] the cost the token usage and everything giving  them everything but at the end of the day making [24:22] sure that these three personas are addressed with  one person essentially saying that hey you know [24:27] what I I look at everything now I'm going to take  a call based on this impact score. So that we've [24:32] seen has worked. we've kind of worked I mean this  is as you pointed out this is an evolving industry [24:38] this is going to get better but I think to avoid  the fluff it's very important to start somewhere [24:44] and then get to the you know uh you know global  maxima what I what I would say is it's a hybrid [24:52] right you cannot just say it's a business owning  the metrics or the IT owning the metrics I sit in [24:58] IT uh so I think it all depends upon where the  use case is enabled Let's take for example in [25:04] IT we've enabled several use cases whereby we're  now seeing the software engineering tasks you know [25:12] uh we're using a lot of uh you know let's call  it um model for coding right so there's a lot [25:19] of coding agents and whatn not that we're using  so in that case we are owning the ROI for that [25:25] right yeah similarly if it is a use case that is  enabled for sales or marketing the responsible [25:31] leader of that functional unit should own the KPI  and should own the ROI. So there's an enablement [25:39] function of it. Yes, me being the CO, we enable  the function and the model and use cases, [25:46] but I should not be the one accountable for all  ROI. Yeah. Should be the sponsor, the business, [25:52] the functional unit. If you're doing internal,  if you're doing external, then the use case, [25:57] what is a P&L tied to that use case? So I think  that is how ROI will evolve. Of course uh similar [26:04] like again if you look at again I like to draw  historical parallels right if you go back to [26:09] dashboards and data and uh investment in data the  KPIs were always measured by the business define [26:19] what the KPIs are uh and then the teams work  towards enabling that uh and I think that is what [26:26] we'll be doing here as well which is are like you  know there will be a chargeback at showback coming [26:32] similar to how we did with the cloud cloud usage.  So I think that is how the ROI will be measured [26:38] going forward. Uh but I think there is still time  to scale. Yeah. And once we get to that scale uh I [26:44] think that is how the it will shift. Yeah. I think  one thing you mentioned around like um like making [26:51] the functional and the business unit leaders I  think that is critical. I mean we've we've I've [26:55] seen data that actually says that business unit  leaders who who are responsible for the ROI in [27:01] their domain are three times more likely to scale  it. I mean it makes sense right cuz if it's owned [27:06] by the technology team it becomes an IT like a  project but if if it's owned by the business it [27:11] becomes a business transformation and so I think  they can trickle down into your point like you [27:16] know should be and they can understand like the  actual usage and and the changes and the actual [27:22] work. So you should sell beyond by them. You  have Yeah. And and one other point we haven't [27:26] mentioned. A part of ROI is risk, right? Uh it's  it's not calculated necessarily, but I I do think [27:31] you need to take it into account. Um so what you  do and what your agents do has to be governed and [27:36] it has to be controlled. For example, you need an  audit trail to understand why an answer was given [27:41] to an agent or why why that agent fetched that  answer for you. You need to be able to control [27:45] saying you don't want to pull from sources that  are maybe based outside of the US or or from a [27:50] particular uh type of website. So the governance  and control aspect of any production AI agent that [27:56] you're building is very critical when assessing  risk and which is very connected to ROI because [28:00] you could have a great ROI but if you have a  breach or if you have uh a lawsuit against you [28:05] uh because an agent gave you an answer that you  can't control or you can't audit this is a major [28:09] factor and your your ROI is not going to exist.  Yeah. And and to add to that right I mean whenever [28:15] you introduce governance or add governance to  anything it becomes an overhead our head goes [28:20] to oh yeah it's an overhead function but if it  is owned by the business the ROI is owned by the [28:26] business let's call it then there is a skin in  game for the business you know for your business [28:32] sponsors and that is how change management is  enabled cuz now you don't like you're you're not [28:39] the IT team is not going and driving the change  management to happen the business themselves are [28:45] driving the change management function. I think  that's very critical and remember all of this [28:51] predicated on having a baseline of data to start  from. All of this stuff kind of falls apart unless [28:56] you have an agreed upon start point. So that's  where you got to start. Yeah. And I I think that [29:01] the ownership from the business unit leader I  think that perspective Matt uh last month I was [29:06] talking to the head of AI guardian life and and  he said to me was that you know I don't have an [29:10] AI strategy I have a business strategy and then  I got to figure out like if AI can truly help me [29:15] with that business strategy right I think we got  to go business fast and like I think just doing AI [29:20] for the sake of doing AI seems like we're that's  where things have been falling apart. It's funny [29:24] and a running joke I keep hearing here already  is that all the security people are down at [29:28] Black Hat this week. So, everyone here does have  to take some level of responsibility for this. [29:37] Shifting gears, you know, we're seeing technology  move so fast, incredibly fast that, you know, [29:44] whatever you end up deploying today ends up may  end up actually being technical debt or legacy [29:49] uh tomorrow. So a question like you know how are  you making these like you talk about changing [29:57] architecture and like you know like how are you  making these decisions to make sure that whatever [30:01] you deploy today doesn't become technical  debt tomorrow like so so um we've talked [30:09] a lot about the context versus the model and you  were actually talking a little bit about the the [30:12] model and I think about your own model to me the  model is almost going to become a commodity right [30:17] the these guys are going to continue to compete  with one another and the thing you control is [30:21] that context layer. So as you think about the  context layer, that's the piece that will be [30:26] durable over time. So an expression I like to say  is the durable layer versus the disposable layer, [30:31] right? You got the disposable layer or the are the  models because you can switch swap out models, but [30:36] that durable layer, there's where your workflow  is, there's where your business logic and that's [30:40] going that's going to persist, right? That's not  changing overnight. So again, that's why the focus [30:44] on context that we talked about so much and let  these guys fight it out from a business from [30:48] a from a model standpoint, right? um you know,  somebody likes Claude today, they'll like OpenAI [30:52] tomorrow, they'll like Rock the next day. Great.  And we can all take advantage of that. So again, [30:56] that's that's the way I I think about dealing with  the the pace of change. We'll see. I mean, it is [31:01] early that it's changing almost every day, right?  So it's crazy. And I think uh please go ahead. So [31:07] I think to Mark's point uh the managing context  also especially when it comes to if we think [31:15] about what happens like 5 years from now it's also  very important as part of the context management [31:22] toolkit I think it's also very important we  have a very strong space for the health check [31:27] of your context the stalness of your context the  evaluation of your context I know that it's a very [31:33] these are all terminologies that are thrown around  a lot But the most critical thing is let's say you [31:38] you add a lot of AI slop today everything is you  know as Mark said bring in any model uh you know [31:44] build a bunch of things with this 3 years from  now a lot of this doesn't make sense but how is it [31:49] that you know that everything that you have added  contributes to some kind of value you need that [31:55] regular health check on top of your context uh  regular health check on top of anything that you [32:00] do for that matter and making sure that you're  measuring staleness making sure that you have [32:05] a very strong eval ecosystem that is uh powering  the context engine is going to be like absolutely [32:11] critical and it's not just for the future it's  also something that helps in a more uh you know [32:18] acceptable time frame of let's say 2 weeks 3 weeks  we see that as and how you keep adding context the [32:24] same query doesn't operate the same way now  uh ARR for example this is something a very [32:30] common example that we see in production ARR is  something the sales team defines differently and [32:34] the finance team defines differently. When someone  from the sales team asks something, it uses the [32:40] wrong formula and you get a different output.  The answer is not wrong. It's just that it's [32:45] been defined in a way which is ambiguous. So that  is where make and making and as time progresses, [32:50] let's say the sales team stops using that table.  Someone else is using that space. We want to make [32:55] sure that whenever someone's asking a question  on AI or an agent is requesting something, [33:00] it's picking up the right things. So,  TLDDR having a very strong eval ecosystem [33:07] uh and the right guard rails that are set up  on top of your context ecosystem is absolutely [33:13] critical in making sure the AI slop doesn't you  know become a liability than an asset. One thing I [33:18] would would probably quickly mention like setting  up these guardrails and governance I've also seen [33:22] it act become an inertia where pe folks trying  to get the perfect governance framework and the [33:28] guard and and there isn't anything like yeah  so we've we I mean I think there needs to be [33:33] kind of a a a minimal viable governance like  what is the minimal viable governance protocol [33:40] risk protocol that you need to yeah for you to be  able to deploy so it doesn't cause that slowdown [33:46] uh because uh you What is it? Uh progress will  you know like trying to get perfection will slow [33:53] down any sort of momentum that you can build.  But Darren, do you have any opinions? Well, [33:58] like you know I believe anything that you put  in production the moment you put in production [34:03] it is technical right let's just be very honest  right because uh you start counting the moment you [34:10] still put something in production uh it is like  technical debt that you have to resolve down the [34:15] road. Um so but if you have disciplines and if you  have let's say good processes like if your data is [34:22] AI ready you can swap models right uh what what do  you every every model provider will tell you that [34:29] oh yeah this model has so many billion parameters  best model out of the you know out of the box but [34:35] if your data is really crappy and it's not you  know stitched well together then all these models [34:41] are not going to scale properly. So uh similarly  if your architecture is not sound uh and the [34:48] context layer is not sound then you won't get the  answers you need. So once you have these I will [34:53] say foundational elements in place then I totally  agree models are going to be a commodity right and [35:01] uh down the road like now there's a big debate  happening in the marketplace around open source [35:06] versus closed models right so tomorrow you can use  anything and the efficiency is what will should [35:13] drive your use case and the model selection right  but I feel that is a mood point What should we [35:21] really what we should focus as both technologies  technologists and business leaders is how do you [35:27] make sure that you have the disciplines in place  the foundation in place the architecture in place [35:32] such that the you have the flexibility uh to  switch models you have the flexibility to switch [35:39] technologies and still drive outcomes you know can  I can I just add to one of the things is around [35:45] this this context and thinking about enterprise  context and I've referenced before about the size [35:50] of our And one of the things we've done to try to  keep the context at an enterprise level and not [35:54] have agent spraw and context sprawl is we actually  built an agent for an intake. So, so nobody in our [36:00] company can build an agent without going through  an intake agent. And so they go through basically [36:04] an interrogation process of what are you going  to do with it? Are you going to use PII data? [36:09] Those sort of things which helps with the whole  responsible and the governance aspects. And then [36:13] we also check what else is out there to reduce  duplication, reduce agent spraw. It's just an [36:17] it's an interesting idea that we got a lot of  great feedback from our from our employees and [36:22] just think about using an agent to build an agent.  So I just does it make recommendation like hey [36:26] there's already something that exists. Absolutely.  It makes a recommendation on something is out [36:30] there in the agent store. You go use that rather  than again create more agent. I haven't anyone do [36:34] that. So that's interesting. Yeah. Um I I actually  want to open up for for audience Q&A as well. Um, [36:42] so yeah, you both explicitly referenced the  need to create a context layer or or or a [36:49] data foundation uh which I I think everybody would  agree with. Um, how are you solving the politics [36:56] around data within your organizations to a address  creating an enterprise common environment and is a [37:05] possibility to trying to solve that problem using  small domain or enterprise domain models so that [37:12] you own essentially your own data provisioning  environment, data capture environment into your [37:18] ever evolving model to then free yourself from  tokenization. Sorry guys. Uh but also enable [37:26] more development first. Sure. Uh I can go first.  I can give you an example. Right. So absolutely [37:35] like what we realized was um an SLM would work  real like I'll give you a use case, right? [37:41] If you're looking for a BI use case to be solved  like replacing BI dashboards or whatnot with [37:48] conversational AI or conversational BI however you  want to address it you don't need a you know the [37:54] latest cloud model right for solving that problem  cuz it's a it's a text to SQL problem right [38:00] uh so you basically want SLM for that and that's  a use case that you would use for for that now but [38:07] to but for it to work properly you need context  defined Clearly and we actually proved that in a [38:14] PC right we did a PC and we said okay here's  what in order for us to define this and work [38:20] we should work with our business partners not only  define the metadata needed for the models but also [38:27] define the relationships and define you know the  context around the key KPIs and once we did that [38:36] uh we saw a huge uptake in the conver for the  conversion there is always like let's see as long [38:44] as organizations will have been there there's  always going to be politics right but this is [38:49] where I think a true business sponsorship  helps because when there is a key business [38:54] sponsor behind a use case the uh you know the  change management aspect of it streamlines [39:04] so so you know one of the things we really  haven't spoken about as you think about [39:08] deployments or anything like that on our global  scale is culture and one of the things that's [39:12] unique about Gallagher is its culture right it is  this absolutely differentiating feature and that [39:18] culture has allowed us to actually have a singular  global data strategy so so when you think about [39:23] the way to control data if you have an agreed  upon data strategy and you get to one source of [39:28] truth for your data which is incredibly hard to  do as I'm sure all of us know that is the journey [39:32] we are on and as we continue to do that we just  let our data access privileges that are inherent [39:37] to whatever that system and application is That's  what we are using. So I feel very fortunate that [39:42] we have an agreed upon global data strategy. We're  putting it all into these snowflake instances. Uh [39:46] we're retiring 800 up to 800 different different  data repositories. Long long journey. Um but [39:53] that's where we're on to try to address exactly  that sort of problem. So again start with the [39:57] data. When you can start with the data and get  that data to a place where you know what it is, [40:01] you can trust it and then you can have your  own data access privileges that those systems [40:05] applications have around it. Let that control the  the data sprawl. Yeah, good question. I don't I [40:14] have a question. You said minimal viable  governance. How do you balance different [40:20] departmental needs? For example, it will have a  concern say operational infrastructurally you have [40:26] responsible AI interests in the long term of what  they would like to see. Then you have security [40:31] concerns, evaluations, data concerns. when each  of these persons responsible those divisions have [40:38] a horse in that race of that conversation. How  do you establish minimum viable governance when [40:43] we don't know what's happening yet and you're  going through that? Yeah. Yeah. So I'll say [40:50] that I actually didn't say that. I said I I but  it's all good. But what I but I would answer the [40:58] question in this way, right? Um so see governance  like everywhere governance is the hardest not to [41:08] crack because everyone wants governance but  no one wants to do it right I mean that's the [41:13] biggest problem that you'll have uh so but but you  have to have like see there the financial services [41:20] industry we are in financial services industry so  we need governance because we otherwise you know [41:27] there multiple reasons we are regulated uh and  whatnot right so in in for that reason we have [41:35] put in any use case that we bring to the market we  have governance embedded in it and what we've seen [41:41] is the use case that I was just explaining about  texture SQL right if you have a well-governed [41:48] product which is well defined well cataloged it is  very easy for an agent to understand the context [41:56] and actually the agent becomes that much more  smarter. If you do not have that governance or [42:01] the minimum viable governance that we've defined,  then the agent just spins and doesn't know how to [42:08] answer the question that you're asking. So, we've  actually shown this and proven it. Hence, we've [42:13] changed the minds of our business partners as well  as our tech partners uh so that the necessary evil [42:20] is addressed. I think it could be outcome based.  I think it's a great point you pointed out. it [42:25] it's very I mean one way to kind of look at it is  just create an outcome and test for that outcome. [42:30] So the definition of minimum viable governance  would be different for Tarun's team versus Mark's [42:34] team versus our team. So in order for us to get  there uh first identify the problem type and if it [42:42] uh uh happens to be in that problem type what's  the out what's the minimum outcome that you are [42:48] looking for. So and if you kind of tie everything  to that try to be as objective as possible we'll [42:54] never be right in the first as saying there  is nothing called correct governance we have [42:58] to start somewhere but I think we can meander  around it once we create an outcome and then say [43:03] that hey you know what look this worked like in  Taran's case it worked perfectly we can actually [43:07] go and tell the company that this is this is the  way we do it so there's multiple layers to that [43:12] governance the lowest layer is it answering hey  look this worked now this is why we should do this [43:18] um So I said that but I'll tell you what  I meant when when I said a minimum viable [43:23] gunners framework. So the the firms that I've  seen who are very successful like JP Morgan in [43:27] our ecosystem Walmart who've like done a fantastic  job sounds like Mark as well with 50 use cases in [43:33] production but at least like the the common  factor that I've seen is that a lot of these [43:37] enterprises have set up a hub and spoke model. So  in this case the hub is your centralized layer and [43:42] the job of the centralized layer you can call it  AICOE or whatever but the hub of the centralized [43:48] layer is to create a minimum viable infrastructure  your data platforms your your you know um uh risk [43:55] tolerance all of that like they set the minimum  benchmark uh but they let them let the business [44:00] units decide the use cases and and and the edge  cases of where they want to deploy um effectively. [44:06] Okay. So what is a minimum viable infrastructure  that these business units need to be able to [44:11] uh um work with the different use cases and then  so that's the what I meant and then whatever [44:16] works in the single business unit that's the spoke  right the spoke in this case is the business unit [44:21] whatever works in the business unit and then it  then it gets scaled across the enterprise platform [44:25] to through the kind of the central AICE or  centralized layer. Uh but you have to create kind [44:30] of that minimal viable framework to not prevent  you know to make sure that the business units [44:35] are not being slowed down because your your peers  and your competitors certainly aren't the model [44:40] stateless or stateful and where do they get the  context the context yeah they got to be stateful [44:46] right so so no they're definitely stateful so so  yeah because that's how you that's the context [44:51] right so they have to have that information right  as part of the graph to to to do the analysis [44:59] So tokenization that's a different that's a  different point. Yeah. So I think the harness [45:03] has to be built with that. So I think to Mark's  point the context layer needs to have that harness [45:08] which has a quarterback kind of a quarterback  agent which looks at your model. It looks at [45:14] your context and it has that equal view to other  parts of the business organization where you're [45:20] looking at accuracy cost evaluates everything.  So these can't be treated as two separate [45:25] layers until there is some unifying function  and that unifying function is the hardness. [45:33] The hardness and the loop you build around  it. Yes, absolutely. So even though models [45:37] become tokenized, you will still have  to build I mean if you have five models, [45:41] you'll have to build the hardness around each of  them for it to work for your use case. That has to [45:45] happen. The thing is the the technology is getting  better that we can build harnesses faster now. [45:52] Uh and yes so TLDDR I know we're getting uh  kicked out but we need to have that quarterback [45:59] uh ecosystem that context engine for you  to power the context layer which is the [46:03] reposiitoized version and the model access  and every personalization etc. Well we have to [46:09] wrap it up. Sorry guys. Um thank you so much.  So, let's give our panel a round of applause.