Airbus, eXp Realty Limit Enterprise AI Scaling Failures to 20% Tooling Gap

空客与 eXp Realty：企业 AI 规模化困境仅 20% 源于工具差距 企业级 AI 推行受阻主要源于人为抵触与静态基础设施，而非技术短板，这需要持续的运营资金投入以及与工作流的深度融合。 该专家小组深入探讨了将 AI 试点转向企业级规模化部署所面临的运营、战略及架构现实。与会嘉宾强调，AI 的成功规模化主要是一项涉及人员与变革管理的挑战，而非纯粹的技术难题，需要直接契合业务需求并建立用户信任。核心讨论议题包括评估“自研还是外购”决策、保持模块化架构以防止供应商锁定、结合全民开发优化 Token 成本，以及通过运营支出（OpEx）财务模型应对硬件快速淘汰的问题。专家们最终建议，应通过内部倡导者构建自下而上的推动力，接纳较短的软件生命周期，并将 AI 直接嵌入员工现有的工作流程中。 企业 AI 试点的失败主要源于“技术先行”的路径，而非未能达成技术 KPI；实现成功的规模化需要解决具体的业务痛点以建立用户信任。 (3:06) 企业 AI 的规模化推广有 80% 取决于变革管理、行为转变和落地执行，仅有 20% 与实际工具本身相关。 (3:06) 企业必须严格评估“外购、自研还是观望”的决策，在利用商业平台构建底层基础设施的同时，将安全关键环节与核心知识产权保留在内部。 (11:43) 遵循第一性原理构建模块化系统架构有助于防止供应商锁定，并能在外部技术成熟时无缝替换次优组件。 (19:14) 部署智能体 AI（Agentic AI）的企业中仅有 25% 达到了预期的投资回报率，这促使企业探索全民开发等自筹资金机制，以收回传统的 SaaS 支出。 (23:01)

Enterprise AI scaling depends fundamentally on organizational change management, end-user trust, and architectural modularity rather than raw technological capability, according to industry practitioners convened by Ai4. While consultancies report that 99% of executives feel unready and IBM finds that only 25% of enterprises realize their intended AI return on investment, successful scaling requires embedding tools directly into established workflows and managing rapid operational shifts. Prioritizing Trust and Native Workflow Integration (3:06) Panelists argue that enterprise AI initiatives stall when pursued technology-first rather than anchoring on concrete business pain points. Tracy St.Dic notes that scaling AI is 80% people, behaviors, and change management and only 20% tooling, leading Zapier to appoint its Chief People Officer as its AI Transformation Officer. Raghu Bala identifies trust as the central operational bottleneck, explaining that shifting employees from deterministic software to probabilistic AI outputs requires active human-in-the-loop controls, continuous feedback mechanisms, and explicit cost-benefit validation. Adoption also falters when organizations force employees into disparate standalone applications. Because workers build significant expertise and comfort with incumbent platforms like Salesforce, replacing familiar software triggers immediate friction. The panel advises infusing AI directly into existing workflows via embedded widgets and focusing evaluations on efficiency gains, output quality, and employee experience. Fostering internal champions generates bottom-up pull, which scales technology far more effectively than top-down mandates. Modular Architecture and Selective Sourcing Strategies (11:43) Navigating the decision to buy, build, or wait requires distinguishing proprietary core competencies from commodity infrastructure. Reena Solomon outlines Airbus's strict delineation: passenger safety-critical capabilities and core intellectual property are engineered in-house or with close partners, while standardized orchestration layers and enterprise infrastructure are purchased from dedicated tech providers. Seth Siegler warns against waiting due to competitive obsolescence, emphasizing that eXp Realty builds clean-sheet pilots quickly but remains willing to decommission internal tools when public cloud providers like GCP release managed native features. Architecting deployments from first principles ensures modularity, enabling organizations to swap out individual components as superior third-party models or safety suites emerge. Retaining control over overarching orchestration prevents enterprises from becoming captive to single-vendor roadmaps. Furthermore, incorporating standardized guardrails and compliance layers from the start avoids the costly technical debt of retrofitting governance into monolithic architectures. Token Financial Discipline and Infrastructure Shifts With IBM benchmarking that only one in four companies currently realizes target ROI from agentic AI deployments, financial rigor has become paramount. Organizations maintain fiscal discipline by using automated agents to audit token consumption, routing non-complex tasks from expensive models like Claude Opus to lighter options like Claude Haiku and converting recursive agent loops into deterministic routines. To contextualize expenses, business units benchmark annualized token expenditures directly against full-time employee (FTE) headcount costs to evaluate genuine capacity expansion. Simultaneously, AI hardware lifecycles are disrupting traditional IT budgeting. While conventional enterprise hardware depreciates over three to five years, rapid architectural leaps cause AI compute infrastructure to experience functional obsolescence every 6 to 12 months. Panelists stress that IT and finance leaders must abandon rigid multi-year Capital Expenditure (CapEx) schedules in favor of dynamic Operational Expenditure (OpEx) models to continuously access faster, cheaper compute without being trapped by outdated physical assets. Controlled Citizen Development and Ephemeral Software Enterprises increasingly rely on citizen development programs to self-fund AI initiatives, allowing frontline workers to build custom automations and niche applications that displace costly third-party SaaS subscriptions. However, unmonitored creation risks sprawling shadow IT and compliance liabilities. To mitigate this, organizations enforce centralized shields comprising universal model routers, pre-configured safety guardrails, and unified data access controls. A robust knowledge management structure ensures successful local pilots are recognized, hardened, and scaled into enterprise best practices without duplicating effort across units. In parallel, leadership must foster acceptance of short software lifecycles. In an accelerating AI market, decommissioning an internal tool after a few months in favor of a superior alternative represents operational agility and cost recovery rather than wasted investment. The panel emphasizes that moving from pilot projects to enterprise-wide adoption requires treating deployment as an iterative continuum rather than a binary rollout. However, maintaining the balance between grassroots citizen innovation and centralized governance remains an ongoing operational challenge, particularly as three-quarters of enterprises continue to struggle with realizing measurable financial returns.

行业专家在 Ai4 研讨会上指出，企业实现人工智能从试点走向全面落地的瓶颈主要不在于底层技术，而在于组织变革管理、业务价值验证以及以人为本的流程整合。 以业务需求与信任为先的评估体系 (3:06) Airbus 人工智能合作负责人 Reena Solomon 指出，许多技术试点停滞的根本原因在于盲目采取“技术优先”策略，而脱离了具体的业务与客户痛点。由于 AI 能力迭代速度快于业务单元的实际需求，立项必须从验证用户真实问题切入。AI Squared 的 Raghu Bala 强调，将员工从确定性传统软件引导至具有概率特征的 AI 输出，首要障碍在于信任，必须通过人机协同、反馈机制及精确的投入产出比分析来化解疑虑。Zapier 全球人才副总裁 Tracy St.Dic 表示，企业 AI 转型的成败中约 80% 取决于人员、行为和变革管理，工具本身仅占 20%，其评估核心应聚焦于效率提升、产出质量以及员工日常体验的改善。 第一性原理架构与自研采购权衡 (11:43) 在基础设施与系统架构规划上，专家建议采用第一性原理对系统进行模块化解耦，明确区分核心专有资产与通用功能。Reena Solomon 解释，在对安全要求极高的航空制造领域，涉及核心知识产权与关键安全的功能坚持自主研发，而底层编排与基础软件设施则采购成熟科技巨头的方案。eXp Realty 首席创新官 Seth Siegler 警告，在当前竞争环境下选择观望将带来严重的淘汰风险；当公有云提供原生功能时，企业应果断淘汰自建工具以减轻维护负担。保持系统架构的模块化与可插拔性，不仅能防止单点供应商锁定，还能避免日后在非模块化单体系统中追加合规安全功能所产生的技术债务。 财务回报困境与成本精细化管控 根据 IBM 的行业基准数据，目前仅有约 25% 的企业在部署智能体 AI 系统后实现了预期的投资回报率（ROI）。为应对严苛的预算压力，企业正推行平民开发者（Citizen Development）计划，使一线非技术员工能通过低代码或无代码工具构建垂直内部应用，从而替代高昂且不完全契合需求的第三方 SaaS 订阅，直接收回软件许可成本并为核心 AI 项目提供自筹资金。在日常运营层面，企业通过自动化监测智能体对组织范围内的 Token 消耗进行精细化管理，例如将非复杂任务从高成本模型（如 Claude Opus）降级路由至轻量级模型（如 Claude Haiku），并消除不必要的循环推理。将部门月度 Token 开销与全职员工（FTE）雇佣成本进行对照，也为评估实际产能增量提供了客观基准。 平民开发治理与算力支出模式转型 缺乏规范的平民开发极易引发应用泛滥、“影子 IT”与沉重的维护债务。专家建议在启动初期构建集中化防护屏障，统一部署通用模型网关、预置安全合规防线及标准化数据访问权限，并配合知识管理系统，以防不同业务单元重复开发相同功能。与此同时，企业在硬件基础设施战略上面临根本性转变。不同于传统 IT 设备 3 至 5 年的折旧周期，AI 算力硬件在 6 到 12 个月内便可能因性能跃升和成本下降而面临功能性淘汰。企业财务必须从周期性的固定资产投资（CapEx）转向持续性的运营支出（OpEx）模型，以适应硬件架构的动态迭代。 无感嵌入工作流与自下而上的推广路径 推动全员规模化落地的核心在于降低系统摩擦，避免强制要求数千乃至上万名员工脱离熟悉的现有业务系统去使用全新的独立应用。员工多年积累的使用经验会形成心理依恋与壁垒，生硬切换往往会引发抵触。更有效的策略是将 AI 能力以插件或小组件（Widgets）的形式无缝嵌入现有工作环境（如主流 CRM 或生产力软件中），实现渐进式升级。在组织推广路径上，应通过扶持业务线内的积极探索者打造标杆案例，利用同行示范效应形成“自下而上”的需求拉动，而非单纯依赖自上而下的行政命令，并将系统迭代视作持续演进的过程，坦然接受并庆祝以更优方案替代旧软件的短生命周期常态。 本场讨论的共识在于，企业 AI 的全量推广不是单次交付的技术切换，而是围绕业务实效、成本效益和人员习惯展开的长期组织演进。如何建立标准化的治理网关并衡量 Token 开销的真实产能替代价值，仍是管理者在推进规模化落地过程中需要持续探索的核心课题。

## Background 该视频由 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台上。内容围绕企业如何将 AI 智能体（AI Agents）从早期单点试点推进到全企业级规模化落地展开，汇集了来自航空、地产、自动化平台与金融等领域的行业高管，探讨高价值用例识别、基础设施与预算管理、化解组织文化阻力以及设定关键 KPI 等实战策略。 ## TL;DR 圆桌嘉宾一致指出，AI 试点陷入泥潭的核心原因在于“技术先行”，企业必须从真实业务痛点出发，让可量化的业务价值和 KPI 自然显现。在自研与采购决策上，应聚焦核心差异化壁垒并保持模块化架构，同时借助全民开发者（Citizen Development）削减冗余 SaaS 支出以实现 AI 投资的自造血，并适应 AI 基础设施从固定 Capex 向高频迭代 Opex 的财务转型。最终，企业级规模化的成败有 80% 取决于人与变革管理，成功的关键在于通过早期拥护者自下而上拉动需求、拥抱快速迭代，并将 AI 润物细无声地直接注入员工现有的工作流与存量工具中。 ## Atmosphere & style 整场圆桌讨论务实坦诚、节奏明快，兼具工业制造巨头的严谨防线与敏捷科技企业的创新锐气。主持人的敏锐引导与四位嘉宾的多维碰撞形成了极具张力的互补，既有直面“75% 企业未获预期 ROI”等现实困境的冷峻剖析，也有对赋能一线员工和自下而上变革的激情倡导。聆听过程充满落地启示，给人一种拨开技术炒作迷雾、深入企业数字化转型真实驾驶舱的沉浸感与获得感。 ## Key takeaways - [03:06](https://www.youtube.com/watch?v=RtsBFew-bf0&t=186s) 切莫采取“技术先行”模式，成功的 AI 规模化必须从用户和业务部门的核心痛点出发，让可量化的业务影响与 KPI 自然显现。 - [04:50](https://www.youtube.com/watch?v=RtsBFew-bf0&t=290s) 从确定性软件转向概率性 Agentic 系统，跨越“信任鸿沟”是走向生产的关键，需要引入人机协同（Human-in-the-Loop）与精确的 ROI 度量。 - [07:20](https://www.youtube.com/watch?v=RtsBFew-bf0&t=440s) AI 变革成功的 80% 取决于人、行为与变革管理，只有 20% 取决于工具；评估试点应立足效率提升、成果质量与员工体验三大维度。 - [09:32](https://www.youtube.com/watch?v=RtsBFew-bf0&t=572s) 从第一性原理出发白纸重构业务流程，全企业规模化上线决策应设定极高的“无可争议的大满贯”（Grand Slam）门槛。 - [11:43](https://www.youtube.com/watch?v=RtsBFew-bf0&t=703s) 不要被动等待技术成熟而产生决策瘫痪，企业应保持敏捷，将自研与成熟云服务有机结合，随时准备快速替换更优的云原生方案。 - [14:29](https://www.youtube.com/watch?v=RtsBFew-bf0&t=869s) 推行“公民开发者”（Citizen Development）模式，赋能贴近业务的一线非技术人员自主构建定制化工具，摆脱漫长的 IT 排期队列。 - [16:37](https://www.youtube.com/watch?v=RtsBFew-bf0&t=997s) 划清自研与采购的战略边界：传统企业在通用编排与底层大模型上采购成熟方案，将核心资源集中在涉及业务壁垒与安全生命线的领域。 - [18:45](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1125s) 采用解耦的模块化架构，在初期将安全护栏（Guardrails）与合规机制预设为标准插槽，以支持后续无缝热替换更先进的模型与服务。 - [24:15](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1455s) 一线员工自研的微应用可替代高昂的第三方通用 SaaS，为企业节省大量支出并反哺后续企业级 Agent 试点的自造血循环。 - [25:42](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1542s) 精细化监控全局 Token 消耗，通过将昂贵大模型降级为轻量模型或转为确定性工作流来降低成本，并将 Token 支出对标全职人力成本进行多维价值核算。 - [27:51](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1671s) 警惕全民开发的生命周期失控风险，必须通过统一的模型路由层、平台级安全护栏与知识共享体系进行集中治理与防线把控。 - [30:56](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1856s) 由于硬件迭代周期缩短至 6-12 个月，AI 基础设施预算必须从传统的一次性资本支出（Capex）转变为持续动态刷新的运营成本（Opex）。 - [34:02](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2042s) 化解组织阻力的最佳方式是扶持早期采纳者成为“内部拥护者”，用成效激发其他团队主动寻求接入的自下而上“拉动式”（Pull）推广。 - [34:57](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2097s) 打破对软件生命周期的固有执念，若构建的系统短期内被更新、更便宜、更高效的方案所淘汰，应视其为敏捷迭代的积极进展。 - [35:50](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2150s) 试点到全企业推广是渐进式的迭代光谱而非二元开关，员工的大多数抵触源于缺乏明确指引与考核预期，必须狠抓变革管理。 - [37:13](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2233s) 避免强推独立新软件造成“工具疲劳”，明智的策略是将 AI 能力直接以微件（Widget）或插件形式无缝注入员工既有的工作流和日常工具中。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=RtsBFew-bf0&t=4s) 圆桌开场与嘉宾背景介绍 主持人 Alina Rivilis 首先对大家表示欢迎，并开启了本场关于“从试点项目到全企业范围规模化落地”（Scaling from Pilots to Enterprise-Wide Adoption）的圆桌讨论。Alina 担任加拿大第七大银行 Fairstone Bank of Canada 的数据科学与 AI 总监，同时也是 AI Future Leaders 及 AI Leadership Academy 的创始人兼 CEO。 随后，现场四位来自不同行业的重量级嘉宾依次介绍了各自的背景与组织概况： * **Reena Solomon**：空中客车（Airbus）AI 合作伙伴负责人。空客作为欧洲航空航天制造巨头，正在积极探索前沿 AI 技术的工业化应用。 * **Seth Siegler**：eXp Realty 首席创新官（CIO）。eXp Realty 是全球规模最大的独立房地产经纪公司（纳斯达克上市公司），在全球 29 个国家和地区拥有超过 80,000 名房地产经纪人。 * **Tracy St.Dic**：Zapier 全球人才副总裁。Zapier 定位于连接最广泛的 AI 编排平台以及自动化与 Agent 的治理行动层。Tracy 指出自己是台上唯一代表“人才与 HR”视角的嘉宾，她坚信推动 AI 在企业内全面落地的关键在于人、行为、人才与文化，而不仅仅是底层技术基础设施。 * **Raghu**：AI Squared 的 Field CTO。AI Squared 为企业提供将 AI 应用与资产从构想孵化一路推向生产环境的“控制平面”（Control Plane）。 在破冰之后，Alina 抛出了核心议题：如今大量企业都在开展生成式 AI 系统的试点，也有部分企业将其推向了生产环境或建立了运营模型；然而根据麦肯锡（McKinsey）和 Gartner 的调研，仅有极少数高管（甚至约 1% 的高管明确表示）认为自己做好了规模化落地的准备，并拥有完备的基础设施。与此同时，也有行业先锋（如 AT&T）宣布在超过 10 万名员工中成功部署了 71 个生成式 AI Agent，创造了显著的业务价值与投资回报率（ROI）。面对这种巨大的落差，企业究竟如何判断“何时该规模化”？有哪些具体的 KPI、业务信号或风险阈值，标志着一个项目已经准备好从局部试点走向全企业级的生产系统？ --- ### [03:06](https://www.youtube.com/watch?v=RtsBFew-bf0&t=186s) 空客经验：切莫“技术先行”，围绕业务痛点自然沉淀 KPI 针对“是否存在某个标志着可以规模化的特定 KPI”这一问题，空中客车的 Reena Solomon 直言不讳地指出，在她的实际经验中，并不存在某个单一的指标能直接给出答案。相反，企业中存在大量“卡在试点阶段（Stuck in Pilots）”、永远无法走向规模化的项目。 ```mermaid flowchart TD accTitle: AI 试点走向规模化的两种路径对比 subgraph BadPath ["技术先行模式（容易受阻）"] A1["追踪最新 AI 能力与技术"] --> B1["自上而下强推技术方案"] B1 --> C1["忽视业务单元与真实用户痛点"] C1 --> D1["陷入 Pilot Purgatory（试点泥潭）"] end subgraph GoodPath ["业务为本模式（成功规模化）"] A2["识别用户/业务部门核心痛点"] --> B2["审视 AI 是否为必要且最佳解法"] B2 --> C2["形成明确业务影响与自然 KPI"] C2 --> D2["各业务线主动快速采纳并规模化"] end ``` Reena 剖析了试点项目夭折的根本症结：**很多项目是从“技术先行”（Technology-First）出发构建的**。当前 AI 技术和模型能力迭代飞快，但客户与业务部门的实际需求演进却没有那么激进。如果只因为某项技术很酷就强行寻找应用场景，规模化必然受阻。 空客采取的推进方式完全反其道而行之： 1. **以用户、客户和业务部门为核心**：在动手构建任何解决方案之前，必须先深入挖掘其真正的业务痛点和核心诉求。 2. **严肃审视必要性**：明确评估解决这一问题是否真的非 AI 不可。 3. **让 KPI 水到渠成**：当业务痛点明确、方案切实奏效后，可量化的业务影响和 KPI 自然会显现。当一个业务部门确立了显著成效并兑现了 KPI，其他部门就会自发、快速地跟进采纳，规模化推广便会顺理成章。 --- ### [04:50](https://www.youtube.com/watch?v=RtsBFew-bf0&t=290s) AI Squared 视角：跨越确定性到概率性的“信任鸿沟”与严格的 ROI 度量 AI Squared 的 Field CTO Raghu 对 Reena 的观点深表认同。根据 AI Squared 服务大量联邦政府机构和大型企业的实践经验，他指出阻止试点走向生产和规模化的最大阻碍是**“信任”（Trust）**。这种信任挑战需要从终端用户与业务负责人两个维度来审视： * **终端用户维度的思维范式转变**：在传统软件环境中，系统输出是高度确定性（Software-Defined / Predictive）的；而在 Agentic 系统中，AI 的输出具有固有的概率性（Probabilistic）。对于日常工作依赖这些结果的员工来说，这种心智转变极具挑战。为了打破阻碍，平台必须内置支持“人机协同”（Human-in-the-Loop）与主动反馈（Active Feedback）的工具链。 * **业务负责人维度的量化回报诉求**：企业决策者不能仅凭主观感受做决定。Raghu 提到，曾有企业在系统上投入了数百万美元，最终却只能得到“这挺不错”或“那不太好”的琐碎轶事反馈，这在商业上是无法接受的。走向规模化必须构建全套成本分析与效能度量工具，精准追踪系统花费、实际业务收益以及受影响的人群规模。 Raghu 强调，如果无法建立用户的信任体系，也无法提供清晰的 ROI 度量工具，再出色的项目也必然会困在试点阶段。 --- ### [07:20](https://www.youtube.com/watch?v=RtsBFew-bf0&t=440s) Zapier 的三维评估法：效率、质量与员工体验 Zapier 的全球人才副总裁 Tracy St.Dic 分享了组织层面的独特视角：在 Zapier，负责推动全公司 AI 变革的最高负责人（AI Transformation Officer）正是首席人才官（Chief People Officer）。Zapier 的核心理念认为：**AI 转型与全企业规模化落地的成功要素中，80% 取决于人、行为习惯、落地执行与变革管理（Change Management），只有 20% 取决于底层工具**。 工具的更迭极其迅速，但真正拖慢全企业推广速度的，永远是那些产生抵触情绪、或是由于缺乏认知而不知道该如何操作的员工。 ```mermaid flowchart LR accTitle: Zapier 评估 AI 规模化落地的三维框架 subgraph ZapierPillars ["Zapier 试点到规模化评估框架"] P1["效率提升<br/>(Efficiency)<br/>速度加快 10%~50%"] P2["成果质量<br/>(Quality)<br/>改善利益相关方/候选人体验"] P3["员工体验<br/>(Employee Experience)<br/>消除苦工，赋能员工"] end P1 --> Scale["规模化推广落地"] P2 --> Scale P3 --> Scale ``` 当 Zapier 评估一个试点项目是否适合推广到全公司时，会基于三大核心维度进行综合考量： 1. **效率（Efficiency）**：将 AI 叠加到既有工作流上，使流程提速 10%、20% 乃至 50%。这通常被称为 AI 采纳（AI Adoption），是大多数试点的起点。 2. **质量（Quality）**：产出结果是否切实变好。对人才团队而言，这意味着为求职候选人、招聘经理和面试官提供显著改善的交互体验。 3. **员工体验（Employee Experience）**：这是最关键却常被忽视的维度。如果员工没有真切感受到工作体验的实质性改善，任何试点都很难在全公司铺开。AI 必须切实帮员工“减少繁重枯燥的苦工（Toil）”，让他们能将精力投入到更具影响力和人性温度的工作中。 > “除非你的员工深刻感受到 AI 是在为他们服务（Working for them），而不是强加在他们身上的负担（Happening to them），否则你很难推进真正的变革管理与落地执行。” —— Tracy St.Dic --- ### [09:32](https://www.youtube.com/watch?v=RtsBFew-bf0&t=572s) eXp Realty 的第一性原理：全面重构业务流程与规模化决策 eXp Realty 的首席创新官 Seth Siegler 结合公司业务特性进行了深度补充。作为拥有 8 万多名经纪人的庞大平台，经纪人就是 eXp 的客户。公司日常需要处理海量繁琐的交易审批、入职流程以及经纪人政策咨询等人工业务，存在极大的 AI 优化空间，比如消除跨多部门沟通的摩擦、缩减等待时间、填补深夜无人响应的空缺。 ```mermaid flowchart TD accTitle: eXp Realty 从白纸重构到规模化决策流程 subgraph Inception ["1. 第一性原理设计"] A["摒弃'AI 版人工替代'思维"] --> B["从零白纸重构业务流程"] end subgraph Validation ["2. 双轨并行验证"] B --> C["小范围客户群双轨并行测试"] C --> D["合规、安全、隐私审查入场"] end subgraph Decision ["3. 规模化敲定（Grand Slam 门槛）"] D --> E{"全方位大幅提升<br/>NPS 提升 / 成本优化"} E -- 是 --> F["重构运营模式 / 替换 SaaS / 调整人员"] end ``` 在推进 AI 试点与规模化时，eXp Realty 遵循以下核心准则： * **从第一性原理（First Principles）出发彻底重塑**：不满足于仅仅做出“某岗位员工的 AI 替代版本”，而是拿出一张白纸从零构思——“如果我们根本没有这些现存团队，我们会如何重新构建这套体系？” * **双轨并行试点验证**：选择一部分核心经纪人群体进行并行灰度测试，观察系统是否能够全天候稳定交付，是否在一致性、响应速度与可用性上全方位超越旧模式。 * **上市公司合规与治理底线**：作为上市公司，审计、安全（Security）与隐私（Privacy）合规属于准入门槛（Table Stakes），必须协助相关职能团队深刻理解新方案的合规性与价值。 * **规模化决策必须是“无可争议的大满贯”（Grand Slam）**：做出全面上线的决策意味着重大的组织变革——包括公司运营方式的转变、既有 SaaS 软件合同的替换甚至团队人员编制的重新调配。因此，决策门槛必须极高：在 NPS（净推荐值）、响应速度、一致性等多项指标上必须形成毫无悬念的碾压优势（“No-brainer”），才能正式拍板规模化部署。 ### [11:43](https://www.youtube.com/watch?v=RtsBFew-bf0&t=703s) 构建还是购买（Buy vs. Build）的战略抉择 来自金融领域的讨论主持嘉宾（Alina Rivilis）指出，在银行等强监管行业中，合规、治理和风险控制是绝对的红线；如果没有健全的制衡与审核机制，企业根本不可能着手构建智能体系统（Agentic Systems）。为了推动 AI 智能体从试点阶段快速走向规模化落地，企业必须建立成熟的 AI 运营模型，涵盖工具技术、人员流程以及底层基础设施支撑。由此，讨论引出了核心议题：在落地智能体解决方案时，企业究竟应该选择自研（Build）还是采购（Buy）？甚至在某些场景下，是否应该选择“等待”技术进一步成熟？ 对于“等待”这一选项，eXp Realty 的 Seth Siegler 明确表达了反对态度。他直言自己绝不愿意被动等待技术的到来，因为“你可能会永远等下去，而最终只会被某家具有颠覆性的初创竞争对手瞬间消灭（vaporized）”。当前的技术环境下，构建软件和智能体的速度比以往任何时候都快。Seth 分享了团队的真实经历：他们经常快速自研出某些功能，而当产品接近上线时，Google 突然在 GCP 上推出了同类的云服务。对于这种情况，团队完全不会感到沮丧，反而觉得是重大利好——他们毫不留恋地将自研系统下线（sunset），因为自研过程既没花费数月漫长时间，也没耗费数百万美元，如今直接采用云服务反而卸下了维护负担。他认为，虽然在选择智能体底层平台时可能会让人产生决策瘫痪，但只要保持灵活性，将自研与成熟云服务相结合，就能极大降低试错成本。 ```mermaid flowchart TD accTitle: 自研与采购决策路径 subgraph CoreDecision["决策依据 (Core vs Non-Core)"] A["评估业务场景与核心优势"] --> B{"是否关乎核心壁垒 / 差异化？"} end subgraph BuildPath["自研 / 深度控制路径"] B -- "是 (如飞行安全/核心流程)" --> C["自研核心业务逻辑与流程"] C --> D["依托公民开发者与业务一线快速搭建"] end subgraph BuyPath["采购 / 平台集成路径"] B -- "否 (如底层编排/通用云服务)" --> E["采购成熟技术栈与第三方服务"] E --> F["模块化插拔：当更优云原生服务上线时随时替换"] end D --> G["统一编排与安全合规治理"] F --> G ``` --- ### [14:29](https://www.youtube.com/watch?v=RtsBFew-bf0&t=869s) Zapier 的“公民开发者”模式与一线员工赋能 Zapier 的 Tracy St.Dic 分享了他们内部鲜明的自研文化。在 Zapier，有一条不成文的规定：“在提出采购外部工具之前，你必须先尝试自己构建它”。当然，对于一些非核心能力的工具（例如人才招聘团队所使用的 AI 语音面试智能体），Zapier 并不会从零自研底层语音技术，而是围绕该工具进行外围定制开发，确保工作流无缝贴合内部员工的实际工作习惯。Tracy 强调，这从来不是非黑即白的“买或建”，而是两者的有机融合。 Zapier 本身作为无代码/低代码的自动化与智能体编排平台，天然具备工具无关性（Tool-agnostic），能够自由集成和替换各类底层模型与第三方 SaaS，充当工作流的“超级粘合剂”。基于这一优势，Tracy 提出了驱动创新的关键理念： - **一线人员主导解决问题**：最贴近具体业务痛点的人，才是解决自身问题的最佳人选，而不是事事依赖 IT 部门或等待工程师排期。 - **大力推行公民开发（Citizen Development）**：企业如果想通过自建来削减成本并获得高定制化方案，就必须全方位赋能非技术员工，让他们安全、合规、低门槛地成为构建者。 - **摆脱冗长 IT 队列**：如果企业依然坚持所有系统只能由小型 IT 团队统一开发，导致每个需求都要排队等上 9 个月，那么企业最终只能被迫采购大量死板的商业软件；反之，赋能全员开发能够让每个岗位拥有量身定制的高效工作流。 --- ### [16:37](https://www.youtube.com/watch?v=RtsBFew-bf0&t=997s) 空中客车（Airbus）的边界：核心竞争力与安全生命线 Airbus 的 Reena Solomon 从大型工业制造巨头的视角阐述了清晰的战略边界。空客的核心业务与立身之本是飞机的工程设计与制造，而不是去跟 Google、Microsoft 等科技巨头竞争开发企业级 AI 智能体编排层。科技巨头投入数十亿美元去攻克协议、底层安全与治理机制，传统企业在这些非核心领域直接采购现成方案显然是更安全、更明智的选择。 然而，一旦涉及航空制造的核心领域、直接关系到未来飞行竞争优势或产品安全的智能体系统，空客就会毫不犹豫地投入资源全力自研并保持绝对控制权。Reena 幽默而严肃地指出：“如果你今天乘飞机来到拉斯维加斯，你有 50% 的概率坐的是波音或空客”。由于空客必须对最终产品和乘客的生命安全承担绝对责任，因此只要关乎安全，空客就必须在牢牢掌控核心的前提下，携手生态伙伴协同构建，坚决不盲目重复造轮子。 --- ### [18:45](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1125s) 第一性原理与模块化解耦架构 来自 AI Squared 的嘉宾结合平台经验，探讨了从第一性原理（First Principles）设计系统的巨大价值。在过往二十年的技术周期中，架构师很少拥有“推倒重来、彻底重构”的奢侈机会，大多数时候都是在老旧系统上修修补补；但在当前的智能体时代，企业终于可以从第一性原理出发，通盘重构技术架构、人员组织与业务流程。 这一阶段的关键在于进行清晰的商业决策：将通用组件交给专业厂商（例如 AI Squared 专注于提供控制面编排系统与生产级落地能力，而非替空客写飞机安全系统），自身则专注沉淀核心 IP。在系统搭建之初，必须将安全护栏（Guardrails）、人在回路（Human-in-the-loop）和合规机制作为模块化插槽预先设计进去。即使某些环节暂时采用次优方案，只要预留了模块化接口，未来当科技大厂推出更优秀的成熟云服务时，就能实现无缝热替换（Swap out）。反之，如果前期忽视了底层的安全与合规架构，后期想要推倒重构去补丁安全时，整个系统就会变成一场“灾难性的噩梦”，导致用户体验与质量严重下滑。 Tracy 对此深表赞同，将其形象地比喻为“拆解整条流水线并重新设计工厂”。企业必须对自身的工作流保持绝对控制权，不能受制于外部数百家供应商的产品路线图，而是要在清晰的模块化愿景下，随时将最先进的 AI 技术按需接入自身的体系中。 ### [22:49](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1369s) AI 投资与 ROI 困境：如何合理分配预算？ 在确立了架构模式、安全护栏（Guardrails）以及针对非标准架构的风险治理流程后，讨论转向了当下企业面临的核心痛点：AI 投资预算与投资回报率（ROI）。主持人向现场观众和小组嘉宾抛出一个现实问题：现场许多企业都在持续增加 AI 预算，并背负着交付强劲 ROI 的巨大压力；然而 IBM 近期发布的一份报告显示，无论是自研还是采购 Agent 系统，目前仅有 25%（即四分之一）的企业能够真正实现预期的 ROI。 针对“在 AI 预算分配中是否‘投入越多就产出越多’”这一疑问，嘉宾们指出了不同企业环境下的破局之道。高层的战略定力固然关键（例如某些企业高层预先认知到前期必须承受投入，后续才会迎来规模化收益），但对于大多数面临严格财务审视的企业而言，寻找能实现“自造血（Self-funding）”的路径才是推动 AI 持续发展的核心前提。 ```mermaid flowchart TD accTitle: AI投资与自造血循环 subgraph Core["ROI 挑战与破局路径"] A["传统高额 SaaS 采购 / 外部软件支出"] --> B["赋能全民开发者 (Citizen Developers)"] B --> C["构建定制化微应用与自动化流"] C --> D["削减 SaaS 支出 & 替换高昂工具"] D --> E["释放预算 (Self-Funding)"] E --> F["反哺企业级智能体 (Agentic Vision) 试点"] end ``` --- ### [24:15](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1455s) 全民开发者模式：用业务自研软件节省数百万 SaaS 支出 Seth Siegler 分享了 eXp Realty 如何通过全民开发者（Citizen Development）策略实现“以战养战”式的自给自足。几年前，团队联合 Lovable 工具平台，全面赋能了数百名非技术类的一线运营人员与计时员工。这些最了解业务痛点的员工主动自学，针对本团队具体的工作流构建定制化解决方案。 目前，这些员工已上线了超过 50 个组合了后端自动化工作流的生产级 Web 应用程序。所有应用均严格经过工程加固（Hardening）和代码审查。由于这些内部自研工具比市面上通用的 SaaS 更加适配实际业务，公司借此砍掉了大量臃肿昂贵的第三方软件，节约了数百万美元的 SaaS 支出。更重要的是，这笔省下来的资金直接为公司后续推进更庞大的企业级 Agent 试点提供了启动资本。 --- ### [25:42](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1542s) Zapier 的 ROI 衡量标准：Token 支出追踪与多维价值核算 Tracy St.Dic 盛赞了全民开发者的价值，并分享了 Zapier 在 ROI 评估与成本控制方面的三项实用机制： 1. **淘汰第三方采购**：让业务人员直接解决自身痛点，构建更契合自身流程的定制工具，从而削减大量外部订阅开销。 2. **微观 Token 监控与智能降本**：Zapier 对员工使用 Token 持开放态度，但会按月追踪全局 Token 消耗。团队在 Agent Harness 中运行专门的自动化技能：“显示我的所有 Token 消耗，并给出降低支出的优化建议”。系统会建议何时将昂贵的大模型（如 Claude Opus）降级为轻量模型（如 Claude Haiku），或者何时将开放式 Agent 工作流重构成确定性（Deterministic）自动化流程，从而在保证相同交付质量的前提下大幅降低成本。 3. **Token 支出与全职人力（FTE）等效对比**：以人力资源（People）团队为例，团队会将每月的 Token 支出按年度折算，并与团队一名全职员工的平均雇佣成本进行对标，直观评估这笔 Token 预算是否合理，进而从“替代工具成本”、“优化人力结构”以及“同等成本更高产出”三个维度建立多层次的 ROI 叙事。 --- ### [27:51](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1671s) 全民开发的暗面：软件生命周期失控与治理防线 现场嘉宾提出了不同维度的实践观察：在大型政企和超大规模组织中，全民开发模式如果缺乏集中约束，极易引发预算黑洞与运维灾难。 一线业务人员构建的往往只是概念验证（PoC）或最简可行产品（MVP）。当这些应用在组织内被广泛使用后，其后续的维护、更新与全生命周期管理都需要持续注入资金。在很多大型组织中，此类自研应用多达数百个，且员工对自己的成果存在极强的情感依附；一旦企业试图将它们纳入统一的企业合规框架，治理成本和预算便会急剧膨胀。 为了防止全民开发拖垮预算，企业必须在开放的同时预设严格的受控框架： - **统一底层基础设施**：禁止各团队随意挑选 LLM，必须通过企业统一的模型路由层（Model Router）进行集中控制与流量分发。 - **内置标准化安全护栏**：统一由平台层提供开箱即用的 Guardrails，而非让业务人员自行摸索或各行其是。 - **知识管理与最佳实践沉淀**：建立高效的知识共享体系，避免 5 个不同的业务部门各自耗费巨资重复造轮子；当某个自研工具被验证为最优解时，应及时将其收归为企业级标准方案。 --- ### [30:56](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1856s) AI 基础设施演进：从 Capex（资本支出）转向 Opex（运营成本） 在数据基础、员工培训之外，AI 专用硬件基础设施的成本管理是另一个巨大挑战。特别是在数据主权（Data Sovereignty）诉求日益增长的背景下，越来越多的组织倾向于采用本地化（On-Prem）方案。 传统 IT 硬件通常采用 3 到 5 年的折旧周期；然而 AI 硬件的生命周期急剧缩短至 6 到 12 个月。市面上每隔几个月就会出现算力提升 10 倍、推理成本降低 5 倍的新型硬件架构。AI 硬件的“过时（Obsolescence）”并非源于设备物理损坏，而是由于更新、更便宜、更高效的算力与模型发布导致其相对落后。 因此，企业必须将 AI 基础设施从传统的**一次性资本支出（Capex）**转变为**持续性的运营成本（Opex）**： - 预算规划不再是一个静态的财务专案，而是一个必须定期进行技术迭代与硬件刷新（Refreshment Cycle）的持续性支出。 - IT 与 AI 团队需与业务利益相关方紧密协同，按周期评估实际算力需求与更新节奏，从根本上重塑 AI 硬件与算力的财务预算逻辑。 ```mermaid flowchart LR accTitle: 传统IT与AI硬件预算模式对比 subgraph Legacy["传统 IT 硬件预算模式 (Capex)"] direction TB L1["一次性大额采购"] --> L2["固定折旧 (3-5 年周期)"] L2 --> L3["物理损坏后淘汰替换"] end subgraph Modern["AI 基础设施预算模式 (Opex)"] direction TB M1["持续性运营支出预算"] --> M2["高频动态迭代 (6-12 个月周期)"] M2 --> M3["更高性价比算力/模型出现即触发替换"] end ``` --- ### [32:59](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1979s) 圆桌总结：从试点迈向企业级规模化部署的关键法则 主持人总结指出，AI Agent 从前期的单点试点走向企业级受控规模化应用，是一项横跨多维度的系统工程：它不仅要求技术架构与算力基础设施的动态演进、持续性的组织变革管理与全员技能培训，更需要清晰的权责归属（Ownership）、全方位的合规治理以及可量化的业务价值产出。在进入小组讨论的最后环节前，主持人邀请各位嘉宾分享各自关于“如何成功将 Agent AI 扩展至全企业级落地”的核心实战建议。 ### [34:02](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2042s) 克服组织阻力：通过早期采纳者建立自下而上的“拉动式”推广 面对新技术的引入，组织内部往往不仅存在对未知的恐惧，更存在对“现有业务系统被破坏”的深切担忧。要化解这种阻力，最有效的策略并不是依靠高管层自上而下地强行推行，而是在各个业务部门中寻找愿意尝试的早期采纳者（Early Adopters）。通过协助他们率先试用特定解决方案，一旦这批先锋团队取得了切实的成效并创造了业务影响力，他们便会自然而然地成为该应用的“内部拥护者”（Champions）。 当其他部门看到这些先行团队因采用新工具而大幅提升了运营效率，甚至在领导层面前获得了更高的可见度与认可时，整个组织的推广逻辑就会发生质的转变。项目不再需要自上而下地硬性推销（Push），而是演变为其他部门主动前来寻求解决方案的自下而上“拉动模式”（Pull Approach），从而使规模化推广变得更加顺畅和高效。 ```mermaid flowchart TD accTitle: 自下而上的AI推广路径 subgraph Phase1["试点与验证"] A["寻找各部门早期采纳者"] --> B["协助测试特定AI方案并验证成效"] end subgraph Phase2["影响力辐射"] B --> C["早期采纳者转化为内部拥护者"] C --> D["展现运营效率与管理层可见度"] end subgraph Phase3["自下而上拉动"] D --> E["其他团队主动寻求应用 (Pull)"] E --> F["高效实现全企业规模化扩展"] end ``` --- ### [34:57](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2097s) 拥抱速朽与快速迭代：切勿对当前构建的软件产生过度执念 在当今 AI 技术日新月异的发展环境下，企业必须彻底打破对传统软件生命周期的固有思维。过去企业构建一套软件系统，往往默认其会稳定运行数年之久；但在当前阶段，技术迭代速度极快，但这绝不能成为犹豫不决或按兵不动的借口。 企业文化中应当树立并推崇一种新的认知：如果团队构建的某个系统只服役了很短的时间，随后就被当年晚些时候出现的更新、更便宜、更容易构建且性能更优的方案所替代，这绝非失败，而是一件“值得庆祝的事”。拥抱这种现实，最核心的心态就是“不要产生过度执念”（Don't get too attached to it），始终保持敏捷以拥抱最优解。 --- ### [35:50](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2150s) 跨越落地鸿沟：将规模化视为持续迭代的光谱与变革管理 从初步试点到全企业级推广之间存在着一个宽广的光谱，切忌将其看作非黑即白的“二元开关”。如果企业将落地策略简化为“先做几个试点，紧接着直接全员上线”，将会承担巨大的系统性风险。真正健康的路径应是一个由浅入深的迭代路线图：从微型试点出发，演进到更大规模的实验，再逐步上升为正式的业务倡议，最终迈向全企业范围的应用。 即便某项技术已经实现了全员推广，迭代与“快速失败”（Fail Fast）的文化依然不能停滞。在这一过程中，决定成败的关键往往在于人员与变革管理（Change Management）： - **阻力的本质**：在实际落地中，员工产生的抵触情绪极少是出于纯粹的“哲学或观念对抗”。 - **阻力的根源**：绝大多数阻力源于缺乏清晰的指导规范（Lack of clear guidelines）、不知道何为合格标准（What good is），以及不明确预期的考核与责任归属。 - **应对之策**：必须把核心精力放在变革管理与人员预期引导上，否则技术方案再完美也会陷入停滞。 --- ### [37:13](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2233s) 融入既有工作流：将 AI 植入存量工具而非强行更换软件 在大规模推行 AI 的过程中，一个极其严峻却常被忽视的痛点是用户交互方式与“工具疲劳”。市场上几乎每一家新供应商或技术开发者都会推出一套全新的独立应用程序，并期望拥有数千甚至数万人规模的大型企业能够全盘切换并适应新软件。这种要求员工抛弃熟悉工具（例如 Salesforce 等核心系统）去学习新界面的做法，往往会在实际执行中彻底碰壁。 对于一线员工而言，他们可能花费了数年时间才彻底摸透现有工具的每一个功能角落，这种专业熟练度本身就是他们工作效能与自信的来源。如果企业每隔两个月就强行要求他们更换一套全新工具，必然会引发强烈的抵制。因此，规模化落地的明智做法是： - **无缝嵌入**：不要剥夺人们习惯使用的工具，而是将 AI 能力直接“注入”（Infuse）现有的业务场景中。 - **微件化赋能**：通过打造嵌入式微件（Widget）或插件，直接在员工当前的办公界面中提供增量价值，避免员工暴露在数以千计的新独立软件中。 - **平滑演进**：在现有工作流内逐步渗透，随着时间的推移稳步实现长远的智能化重塑愿景。 --- ### [39:24](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2364s) 圆桌总结：AI 规模化落地的核心在于“人” 在圆桌会议的尾声，主持人向各位嘉宾——空中客车（Airbus）的 Reena Solomon、eXp Realty 的 Seth Siegler、Zapier 的 Tracy St.Dic 以及 Raghu 表达了感谢，并对整场讨论的核心洞察进行了升华提炼。 从技术验证到全企业级扩展，最为关键的共识在于：**绝对不能忽视人性的一面（Human side）**。AI 战略的成功不仅仅取决于技术本身的先进性，更取决于企业如何管理变化、引导预期并赋能员工。变革管理绝非次要辅助工作，而是决定 AI 能否真正生根发芽的底层支柱。 ## Notable quotes > [07:20](https://www.youtube.com/watch?v=RtsBFew-bf0&t=440s) 除非你的员工深刻感受到 AI 是在为他们服务（Working for them），而不是强加在他们身上的负担（Happening to them），否则你很难推进真正的变革管理与落地执行。 — Tracy St.Dic > [11:43](https://www.youtube.com/watch?v=RtsBFew-bf0&t=703s) 你可能会永远等下去，而最终只会被某家具有颠覆性的初创竞争对手瞬间消灭（vaporized）。 — Seth Siegler > [14:29](https://www.youtube.com/watch?v=RtsBFew-bf0&t=869s) 在提出采购外部工具之前，你必须先尝试自己构建它。 — Tracy St.Dic > [16:37](https://www.youtube.com/watch?v=RtsBFew-bf0&t=997s) 如果你今天乘飞机来到拉斯维加斯，你有 50% 的概率坐的是波音或空客。 — Reena Solomon > [18:45](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1125s) 拆解整条流水线并重新设计工厂。 — Tracy St.Dic > [25:42](https://www.youtube.com/watch?v=RtsBFew-bf0&t=1542s) 显示我的所有 Token 消耗，并给出降低支出的优化建议。 — Tracy St.Dic > [34:57](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2097s) 不要产生过度执念（Don't get too attached to it）。 — Seth Siegler > [39:24](https://www.youtube.com/watch?v=RtsBFew-bf0&t=2364s) 绝对不能忽视人性的一面（Human side）。 — Alina Rivilis ## Mentioned Reena Solomon, Seth Siegler, Tracy St.Dic, Alina Rivilis, Raghu, Airbus, eXp Realty, Zapier, Fairstone Bank of Canada, AI Squared, AI Future Leaders, AI Leadership Academy, McKinsey, Gartner, AT&T, Google, GCP, Microsoft, Boeing, IBM, Lovable, Claude Opus, Claude Haiku, Salesforce ## Video description Reena Solomon — Head of AI Partnerships, Airbus Seth Siegler — Chief Innovation Officer, eXp Realty Tracy St.Dic — Global Head of Talent, Zapier Alina Rivilis — Director, Data Science & AI, Fairstone Bank of Canada Scaling AI agents from initial pilots to full enterprise adoption requires deliberate strategy and careful execution. This panel will highlight successful approaches organizations have taken to expand AI agent deployments effectively. Topics will include identifying scalable use cases, managing resources and infrastructure, addressing cultural and operational resistance, and tracking progress with meaningful KPIs. Attendees will gain insights into replicable strategies and common pitfalls to avoid in scaling AI solutions. [Source](https://www.youtube.com/watch?v=RtsBFew-bf0)

[7:20] Tracy St.Dic: 除非你的员工深刻感受到 AI 是在为他们服务（Working for them），而不是强加在他们身上的负担（Happening to them），否则你很难推进真正的变革管理与落地执行。 [11:43] Seth Siegler: 你可能会永远等下去，而最终只会被某家具有颠覆性的初创竞争对手瞬间消灭（vaporized）。 [14:29] Tracy St.Dic: 在提出采购外部工具之前，你必须先尝试自己构建它。 [16:37] Reena Solomon: 如果你今天乘飞机来到拉斯维加斯，你有 50% 的概率坐的是波音或空客。 [18:45] Tracy St.Dic: 拆解整条流水线并重新设计工厂。 [25:42] Tracy St.Dic: 显示我的所有 Token 消耗，并给出降低支出的优化建议。 [34:57] Seth Siegler: 不要产生过度执念（Don't get too attached to it）。 [39:24] Alina Rivilis: 绝对不能忽视人性的一面（Human side）。

[3:06] Tracy St.Dic: AI transformation and scaling enterprise-wide adoption is actually 80% the people, the behaviors, the implementation, the change management, and 20% the tooling. [3:06] Seth Siegler: For it to go live, for you to make the decision to scale it, it should just be a grand slam, a no-brainer. [11:43] Tracy St.Dic: If you enable more citizen development within your company and your organization, you might be able to build a lot more and have tailored solutions for every individual role. [11:43] Reena Solomon: If you have taken an aircraft today to come here to Las Vegas, it's highly likely a 50/50% chance whether it's a Boeing or an Airbus... safety is primary concern for us.

[0:04] Hello everyone. We have an exciting panel  for you today about scaling from pilots [0:10] uh to enterprisewide adoption. My name is Alina  Villis and the moderator for the panel. I am a [0:18] director of data science and AI at first Bank of  Canada. We are based in Toronto in Montreal and [0:26] we are the seventh largest bank in Canada. I'm  also the CEO and founder at AI Future Leaders [0:33] and AI Leadership Academy and I'm excited to be  here. Uh Reena, hi. Um I'm Reena Solomon. I lead [0:42] AI partnerships at Airbus, which is the European  aerospace manufacturer. I'm excited to be here and [0:49] looking forward to the discussion. Yeah, great.  So, uh Seth Seagler, the chief innovation officer, [0:55] uh at a company called EXP Realy. It's the largest  single real estate brokerage in the world. It's [1:02] uh publicly traded on NASDAQ. We have a little  over 80,000 real estate agents in 29 countries. [1:08] Um and yeah, also pretty pumped out to be here.  Hi everyone, my name is Tracy Sek. I'm the VP of [1:15] global talent at Zapier. Zapier is the most  connected AI orchestration platform in the [1:20] govern action layer for your automations and your  agents. Um, my role in this and why I'm so excited [1:25] for the panel is that I think I'm the only one  who represents the the people HR perspective [1:30] here and my role is to scale to enterprisewide  adoption through people and behavior, talent [1:35] and culture and not just infrastructure. Hi, I'm  Raguate. I'm the field CTO at AI Squared. We at AI [1:43] Squared provide the control plane for taking AI  applications and AI assets right from inception [1:49] all the way to production. So this is perfect  uh uh panel uh to be on taking pilots and taking [1:56] them into production and scaling them. Uh looking  forward to the discussion. So uh thank you. Now a [2:02] Gent AI systems many of us are running pilots many  of us some of us are deploying them to production [2:10] some of us have got operating models but according  to McKenzie according to Gartner very few actually [2:17] it's around almost 1% of executives that are  saying that they are not ready for that scale [2:24] and they don't have the right infrastructure and  um some companies are quoting especially AT just [2:32] mentioned they deployed 71 generative AI agents  across uh more than 100,000 employees driving [2:40] significant value and ROI. So my question  for us is how do we know when do we scale? [2:48] what specific KPIs or business signals or maybe  risks um are those thresholds that tell you that [2:56] we are actually ready to go from the pilot to a  production grade system across the enterprise. [3:06] I can start with the this um if you ask me for a  particular KPI or signal that shows that a pilot [3:14] is ready for scaling honestly in my experience  that hasn't happened but we have had a lot of [3:21] pilots that just gets stuck and doesn't reach  scale right and when we examine what was the [3:27] reason for that usually it's because uh it's built  technology first if you think about AI I and the [3:36] capabilities it's moving at like really fast pace  but if you think about the customer and the needs [3:42] of the business unit not so fast right so when  you think about uh uh use cases that are developed [3:51] with technology first in mind scaling is always  become a challenge. So the way we approach these [3:58] pilots is we think about the user, the customer  and the business unit. First we identify what is [4:05] the their pain point and what their need is before  going and building a solution. Um and when we do [4:12] that the KPI is is a natural result of it. We also  really think whether AI is required to solve that [4:20] problem. And once we answer all these questions,  the impact becomes much more clear. And when the [4:27] business unit is able to establish this impact  and deliver the KPI, then others in the company [4:35] other organizations are willing to adopt it at  a much faster pace and scaling becomes easier. [4:41] So my recommendation is think about the customer,  think about the business unit and focus less on [4:49] the KPI first. I kind of agree with you like uh  we serve a lot of very large organizations like we [4:57] are mainly in the federal government uh and also  larger enterprises. One of the things that we find [5:04] uh that is a blocker for actually going from  pilots all the way into production and then [5:10] scaling is the trust factor. People need to  trust and especially like what is happening [5:15] is that people I mean you have to look at it from  two spectrums right like one is from a user who is [5:22] the end user of whatever is the system that this  agentic harness or agentic system is supposed [5:29] to deliver and they need to like actually go in  and figure out whether I need to like change my [5:36] work to go from a very predictive softwaredefed  software uh results to a almost probabilistic [5:45] uh AI kind of a results and that shift is not an  easy shift especially if it is your daily work [5:51] and something that you deliver depends on it. So  building in those things, the tooling uh that can [5:59] measure that, building in uh uh items that are  like human in the loop or like active feedback, [6:06] uh all of those things are very very essential  if you want to take your system to production [6:12] and to scale. That's the first portion. But if you  flip that thing and look at uh the business owner, [6:18] then the business owner actually wants to see the  ROI. I have had like cases where like people have [6:24] invested millions of dollars into a system just  to get anecdotal things that this is good this [6:30] is bad and that is not good enough. So one of  the other things is like you do need to build [6:35] in tooling which can give you a real cost analysis  of like what how much is it going to cost me what [6:42] is the actual benefit of this how many people is  it impacting those are all the kind of things that [6:49] you need to like build into the system without  these things it's a great project it will get [6:54] stuck in pilots I can guarantee I have seen this  so many times it is going to get stuck in pilots [6:59] if you are not able to build the trust from the  user and the measurements and all the things that [7:04] will give you the ROI metrics that are like really  really important. But at the end of the day, I do [7:10] agree with Reena that like we are still very early  in the game. So if you ask me what is that one KPI [7:16] or what are the top five KPIs, I don't think I  have an answer for that too. I think there's a [7:21] reason why at Zapier our AI transformation  officer was our chief people officer. And [7:27] one of the things that we believe at Zapier,  and we can we can debate the the percentages, [7:30] but we believe that AI transformation and  scaling enterprisewide adoption is actually 80% [7:37] the people, the behaviors, the implementation,  the change management, and 20% the tooling. To [7:42] Reena's point, like that that tooling is going to  continue to change very very rapidly. But what is [7:47] going to slow down your enterprisewide adoption  are the people who may be actually resistant or [7:52] seem resistant not understanding how to do the  thing that you're asking them to do. And so when [7:57] I take a step back at Zapier when we think about  scaling from a pilot to enterprisewide we look at [8:03] three different things and there can be different  KPIs depending on your department but we look at [8:07] efficiency quality and employee experience and  I think that third one is really important and [8:12] often very missed. Everyone's looking for ways to  become more efficient. We call that AI adoption. [8:17] You're just throwing AI or your pilots on top of  things you already do and you're maybe getting 10% [8:23] 20% 50% faster. That's great. That's a great  place to start. Most pilots maybe start that [8:28] way. There are some metrics attached to that, but  there's also metrics attached to quality. Are you [8:33] actually doing something that's better for the  customer, better for whoever your stakeholders [8:37] are? Obviously, for me and talent, that's often  candidates and hiring managers and interviewers, [8:41] right? And then the last one though which  I think is such an important way to think [8:45] about KPIs attached to this is the experience  of the employee. If your employees are not [8:51] experiencing their job in a marketkedly better  way, it is very difficult to scale AI adoption [8:59] and AI transformation in any pilot across  your company because they have to feel like [9:04] what they are doing is reducing their toil,  allowing them to have a much stronger impact, [9:09] more of the human things that they actually  like. And so that's why we think about those [9:13] three things. We can be more efficient. We can  have better quality. But unless your employees [9:17] are feeling like AI is working for them and AI  is not something that's happening to them, then [9:23] it's going to be really hard to get the change  management, the implementation that you actually [9:26] want. Yeah. And then so I I think I agree with  everything that's been said here so far and and um [9:33] at EXP we just do a lot of manual work. Um we're  a company where our customer is the real estate [9:38] agent. So we process all of their transactions  and handle their onboarding to the company, [9:43] handle their questions that they have about  for their broker, all that kind of stuff. Um, [9:47] and so there's a huge opportunity there to  make that better with AI. Uh, and just reduce [9:53] the number of different people that they have  to talk to or things that take a long time or [9:58] things that are not available in the middle of  the night. All of that sort of stuff. So we have [10:02] just a a process where we think about things  and we invest in pilots and we think about it [10:07] from a total reinvention standpoint not just like  here's the AI version of what that person does. [10:13] um and think about it as let's think about it from  first principles from a a clean sheet of paper and [10:19] if we don't even have all of these different teams  what would we build and then we do invest in it [10:25] and we do what we do is we run it in parallel uh  for a select set of our customers our real estate [10:30] agents and we just see you know does it uh does  it deliver and then there's the the whole flip [10:36] side of this being publicly traded um got a lot  of compliance and different things that need to [10:41] come along with that all of those things are table  stakes. Won't even dwell on it too much. They have [10:45] to be done and you have to help um some of those  folks along, your auditors, your security folks, [10:50] your privacy folks, all that kind of stuff. Make  sure that they see the benefit there. But what we [10:55] aim for is for it to be just better in every way.  For it to just be more consistent, faster, always [11:01] available. Um that we'll end up with a better NPS  score from our our users at the end of the day. [11:08] Um, and then if that all goes well, you have to  make the conscious decision to deploy it and scale [11:15] it. And that means changes in the way that your  company operates, changes in the SAS contracts [11:20] that is replacing, reassigning or or figuring out  if that affects headcount and stuff like that. Um, [11:26] and that's sort of the final step though. But at  the end of the day, uh, in my eyes, it, you know, [11:31] shouldn't even be close. you know, if um for it to  go live, for you to make the decision to scale it, [11:36] it should just be a grand slam, a no-brainer. It  checks many different KPIs. Um and you end up with [11:43] a better experience in the end. That makes sense.  Um I come actually from a financial uh sector. Um [11:50] and we also have to look at that compliance,  the governance, those risks. Without those [11:57] checks and balances, we don't even take Yeah. the  agentic system even we don't even start getting [12:02] into it or building it and uh the AI operating  model becomes very important as you do that to [12:10] speed up those agentic systems going from pilot to  scale we find that we need that AI operating model [12:18] that has got the tools and technology the people  process that uh Tracy just mentioned we also need [12:26] um the right infrastructure and the right support  which brings me to the question uh buy versus [12:33] build. How much of this do we build inhouse?  Uh and how much of this do we uh do we buy? And [12:42] there is another uh category maybe sometimes  we wait maybe we don't even go after certain [12:48] uh solutions. So how do we approach that buy  versus build when it comes to agentic solutions? [12:57] Um, I could I could pick uh just pick this  up. Um, so that's just changing all the time, [13:03] you know. Uh, the nice thing is that um it's  quicker to build than it ever has been. And [13:10] so we've seen a lot of times where I think waiting  that's one I just I don't want to do. I don't want [13:16] to wait. You could wait forever, you know, for  uh for, you know, the technology to come to you, [13:22] but you'll miss out, right? and then you may end  up just getting disrupted and and vaporized by [13:26] some competition startup or whatever. But um we've  built a lot of different things only to be close [13:33] to to rolling it out and then something like  Google comes out now it's a service in GCP. So [13:39] it's like that's actually great news. We're just  going to sunset this thing. We're not attached [13:43] to it. Um it didn't take a ton of time to build.  It didn't cost millions of dollars to get done. [13:48] Uh and now we don't have to own that. So for  that's I like when that happens you know um [13:54] the the tricky thing is the allincclusive or sort  of like what platform do you decide to build on [14:00] uh if you're thinking about like an agentic system  that's been something that can that can paralyze [14:06] uh the best of us I think um but uh but there's  a lot of choices and now I think a lot of them [14:11] are quite mature um and easy to use and so um I  think it's it's a combination of things at the [14:17] end of the day but um a lot of a lot of building  is just much more feasible than it was before. [14:23] And if there's bits and pieces that end up just  being a cloud service, awesome. Yeah, I would [14:29] just agree with that. I mean, we try to build as  much as possible before we buy. In fact, there's [14:35] actually a unspoken rule at Zapier is you have to  try to build it before you can even ask to buy it [14:40] to buy a tool. Now, of course, there's going to be  tools that we are just not well positioned to do. [14:44] Like in in my talent team, we use voice agentic  AI for interviewers. We're not going to build [14:51] that at Zapier, but we're going to build around it  to ensure that it's customized to how we want that [14:56] workflow to look and where we're going to be able  to meet our employees in their flow of work. And [15:01] so I don't think it's buy versus build. I think  it's a combination. Um, again I will admit that [15:05] we have a little bit of wind at our back at Zapier  because we are a platform where you can build, [15:10] you know, your orchestrations, your automations,  your agents and it's tool agnostic. So you can [15:15] build around any tool or platform that you use.  You can swap them out, you can swap out models, [15:20] but Zapier is really the glue that brings  it all together. And so because of that, [15:24] we could bring in any thirdparty platform and then  we build it to the workflows and the ideal that we [15:31] really want that works for us at Zapier. I just  truly believe that the people closest to the work [15:36] are the people who are going to solve their own  problems, not like shipping it out to it or asking [15:40] an engineer to build it for you. And so I think  what's important is if you're going to take the [15:44] approach of we want to build more than buy to save  costs or to make it more personal to us, you have [15:50] to then equip your workforce to also be builders.  You have to have a lot of citizen development that [15:56] encourages the people the closest to the problem  to be the ones building their own solutions. [16:01] And then you have to make it as easy as possible  for them to do that and to do that safely with [16:05] compliance and governance. Um, and so I think it  depends on what your stance is too as a company. [16:10] If you are a company who wants everything built  only by your small IT department and everyone has [16:15] to wait in a nine-month queue to build something,  then you probably will end up buying more tools. [16:20] But if you enable more citizen development  within your company and your organization, [16:26] you might be able to build a lot more and have  tailored solutions for every individual role and [16:30] workflow in a way that will actually save cost  and also empower more of your workforce too. I [16:37] go for it. I agree with every everything that has  been mentioned. So it technically depends, right? [16:43] Uh but the way uh we approach it at Airbus is we  think about what is our competitive advantage and [16:49] what is our expertise. Uh we build we engineer  and build aircrafts. That's our bed bread and [16:55] butter. Do we build integ enterprisegrade AI agent  tech systems orchestration layers? That's not our [17:03] our key expertise right uh and it's hard for us  to compete with a Google or Microsoft in that [17:10] um direction. The reason being they spend  billions of dollars thinking about the protocols, [17:15] the safety, governance, everything around it and  it's going to be hard for us to compete with them [17:21] in that category. So in things that where the  it's not our key expertise then we decide to [17:28] bite and that makes total sense for us and it's  a much more safe option for us. But if there are [17:34] agentic systems that are core for us, that's  when we put in the resources to build it. If [17:42] it's for our if it it's if it's a differentiator  and it matters to our competitive advantage for [17:48] the future of flying, then we want to make sure  we have control around it. That's when we decide [17:54] to build. But is that build done in isolation?  Not necessary. We work with a lot of partners, [18:00] bring in expertise as needed, but we want to make  sure we still have control over the final product [18:06] or service that's delivered because in the end,  we are the ones who have to guarantee the safety [18:12] of the product itself, right? And if you have  taken an aircraft today to come here to Las Vegas, [18:17] it's highly likely a 50/50% chance whether it's  a Boeing or an Airbus, right? So we want to make [18:23] sure safety is primary concern for us and in order  to deliver that that and for products and services [18:30] where safety matters we make sure that uh we  build but definitely in conjunction with the [18:36] ecosystem and not reinventing the wheel ourselves.  Thanks for that the safety part. It matters. It [18:45] does. It does. I appreciate I do agree with uh uh  what Reena is talking about in terms of not being [18:51] able to compete with the people who do this for a  living, right? Like the companies that exist just [18:57] to offer an orchestration system. So we at AI SQU  offer that control plane orchestration system. How [19:03] do you build not just about like agentic harnesses  that give you an agent but how do you take it to [19:09] production? But I want to go back to what set  was talking about uh earlier when you you said [19:14] like when you were starting to think about how to  build you are starting from the first principles [19:21] and that's that I think that is a luxury like  we are having currently that I have not seen in [19:27] the past two decades of like what I'm working when  you are starting for the first principles then you [19:33] can lay out your architecture not just in terms  of the technology but also people and processes [19:38] and everything and Then figure out what are the  pieces that are not available uh in the market. [19:44] Sometimes they are right like you are talking  about guardrails, you are talking about human [19:48] in the loops, you're talking about owning the IP  that you want and being able to distribute it to [19:53] the people. You you are now able to like look at  a much larger picture and figure out am I going [20:00] to build all of these things and make a business  decision at that point. Right? Most of the time [20:05] it is a business decision of like if there is a  tool that is outside which will exactly do what [20:11] I want to do in my system. There is no reason for  anyone to go and build it in house unless they [20:17] want to keep that IP for whatever reason. Right?  There are people who are doing this and they have [20:23] a lot they have the DNA. Our DNA is not if uh  you ask me hey can AI squared go and build me a [20:32] safety system for my Airbus. No, we are not that  right. Like if you have people who are building [20:38] that and if you want to orchestrate it and if you  want to like actually take it to scale then that [20:43] is what we are right. So looking at that picture  and making a business decision is very important. [20:48] But starting from first principles if there are  pieces that are not available today then either [20:53] partnering or slotting it in even if it is like  a suboptimal thing having it slotted in is really [21:02] really important because tomorrow when Google  comes in and like gives you that option you can [21:07] swap it out but if you are building a very large  agentic system or any system for that matter and [21:12] like you have to re-engineer it to put safety  back in put security back in compliance back [21:18] That is where it becomes a nightmare and that  is where the quality of the whole experience [21:22] degrades very heavily. We see this all the  time in both large organizations and also on [21:28] small organizations. I think you're speaking to  something too that really resonates with me is [21:31] that then you can be in control of what those  workflows are. There's hundreds of vendors out [21:37] there and you don't want to be at the mercy of  their road map and what they decide to build, [21:42] right? You want to think through I love this idea  of first principles or we say like stripping it [21:45] down to its parts and redesigning the factory  holding that vision for what you want because the [21:50] technology will eventually be there and then you  can plug it in. So I just I just love that point [21:54] because I like to be in control of what we're  building versus wait for a vendor to build it for [21:59] me. You usually don't have that luxury. I mean  being in the industry for such a long time you [22:04] don't you very rarely get an option like somebody  says hey let's wipe it clean and like start [22:10] over right it's always like building on top of  something adding something to something else but [22:15] I do think like we are at a stage where we can we  can dream about that there are a lot of instances [22:21] where people are actually doing it I don't want  to optimize what I have I want to rethink the uh [22:27] whole workflow and make sure that the people and  the business side of it uh are both getting the [22:34] advantages of the new system that we yeah so being  modular that's what I'm hearing y build on top uh [22:42] have those patterns what we tend to do is I have  a small team but we tend to have um predetermined [22:49] architectures and patterns that we build in those  guard rails and we have established a process [22:55] around the risk and the governance for anything  that doesn't fall into those approved patterns [23:01] Now uh we have got the audience here and I  actually going to ask a question both to the [23:05] audience and everyone here. Um the investment into  AI um our budgets are increasing. How many of us [23:14] have got AI budgets? Show many of us. Many of us  have got AI budgets and many of us are on the hook [23:23] to deliver strong ROI. IBM just had a report  saying that only one in four of companies are [23:31] actually seeing that intended ROI when they build  this agentic systems or regardless if even if they [23:38] buy so only 25% are actually seeing that ROI and  my question to the panel is uh how do we allocate [23:47] those resources how do we manage those budgets  and is it always more delivers more. Um I think [23:57] um I think it's yeah uh it depends on the the  company. Uh I think we're a little bit lucky at [24:04] at my company that from the top everyone sees that  this is going to take some investment and then [24:10] it will be a very big win, you know, once it's  implemented. But that's not always the easiest [24:15] sell uh if that's not the case. Um, something  we did though to sort of self-fund a little bit [24:21] um was to find some lowhanging fruit actually  on the citizen developer front. We ended up um [24:27] going all leaning in on that quite a bit uh a few  years a couple years ago um linked up with lovable [24:35] um and armed a few hundred of our non-technical  uh operational staff uh to start learning about [24:43] it and building their own very niche solutions for  just what their team does, just the work that they [24:49] do. And we saw frontline frontline worker you  know folks um hourly employees in some case um [24:57] you know really take to it teach themselves to  do it and start building pretty nice software [25:02] uh to the point where we now have uh more than 50  web applications combination of web applications [25:09] with workflows on the back end uh that are  in production that have gone through like a [25:13] hardening process and a review process that we put  together uh but we've saved millions of dollars in [25:18] SAS spend as a result of that by building our own  versions that are specialized that are better fit [25:24] um and then gotten off of of those um you know not  perfect fitting SAS solutions that cost money and [25:32] that freed up some uh that freed up some funding  right there uh for us to get started with a great [25:36] number of pilots uh for you know for the larger  vision the you know actual agentic vision. Seth, [25:42] you're making my heart sing with all the citizen  citizen development. I love that. Um there's a I [25:46] think there's there's a couple of pretty simple  things that we look at at Zapier. I mean one [25:50] we're looking at tools that we're able to build  because we have citizen development and again [25:54] the people closest to the work can solve their own  problems that then we're able to move off of third [25:58] party solutions where we've saved thousands and  thousands of dollars just being like we actually [26:03] don't need that tool anymore because we built our  own and it works even better for us and it's more [26:06] tailored, personalized, etc. So we have a number  of things at Zapier that we've just been able to [26:11] say yep we don't need that tool anymore and let's  build our own. That's very easy to defend in terms [26:16] of ROI. I think the other piece though is we track  token spend at Zapier. We're we're pretty liberal [26:22] with the tokens. People can kind of have whatever  they want and ask for whatever they want. Um but, [26:26] you know, we want to make sure that we have the  financial projections to back that up. And so, [26:30] while we look at that on a monthly basis  across all of Zapier, we also have, you know, [26:36] a really easy skill that we run in agent harness  that says, "Show me all of my token spin and [26:40] suggest ways that I can reduce my spend." Whether  that's changing from opus to haik coup in some [26:45] things making an agentic workflow actually a more  deter deterministic automation which saves tons [26:50] of money and so we really equip our own employees  to manage their own token spin and to make smarter [26:57] decisions. So that's really easy to see is are we  able to see um you know similar output for less [27:03] spend over time with certain employees or certain  departments showing us that we can get the same [27:07] output but with a lot less token spend. And then  the third way we do this um and I'll speak for [27:12] the people team is we look at our monthly spend  in in tokens and then we'll say okay what is the [27:16] equivalent of that two for a full-time employee  the average full-time employee on the people team [27:21] costs this much. This is what our monthly token  spend would ladder up to on a yearly basis. How [27:26] do we feel about those two that comparison? Maybe  we feel like the token spin is too high when we [27:31] think about we could hire another person. Maybe  we feel great about that. And so we have those [27:36] conversations as a way to like think about ROI in  different ways from a replacement of headcount to [27:41] thinking about a replacement of tooling and then  also of course like how can we just be smart about [27:46] what we're doing to get stronger output and  continually spend less money. I have I have a [27:51] slightly uh different experience uh especially  in the past few years in terms of the citizen [27:58] developers. It is a great way uh to equip uh  and empower uh the employees that you have. [28:06] um go build what you think is uh right and what  would serve at the end of the day your customer [28:12] right right they know the best but what is what  we have also observed is that they people build [28:19] some things and then it is not that is what like  it's a pilot at that point right like it's a pilot [28:25] it's a it's at best an MVP and then when people  start to use it somebody has to manage it somebody [28:32] has to maintain it and there is a life cycle for  that uh application uh at the end of the day and [28:38] that is where like a lot more money needs to be  like pumped in to manage that and uh when and I [28:46] I deal with like very large organizations both on  the federal and then the enterprise side and 50 [28:51] you're lucky if it is just 50 right like we have  seen hundreds and hundreds everyone gets attached [28:56] to their application because they're able to build  it uh but the budgets balloon once everything has [29:03] to be managed or once Everything needs to come  under the same umbrella because now you want [29:08] everyone to comply to some constraints that the  corporate puts on top of it. Right? That is where [29:13] we see like a lot of problems especially in terms  of budgeting arise. Uh and it's it's not a bad [29:21] thing. It is actually a good thing. It's a good  learning experience if you equip the people with [29:25] uh tools right from the beginning. The same exact  tool to everyone. uh instead of asking people to [29:31] like go pick your uh uh favorite uh uh large  language model, put a model router in between [29:38] so that like you now control what it is, right? Uh  instead of asking them create your own guardrails, [29:43] give them guardrails. So what they actually  are the the value that that they are bringing [29:51] should be what they are bringing but all the  constraints around them should be given to your [29:56] citizen developers. Otherwise the cost of managing  that uh fleet of applications is going to balloon [30:03] and at some point uh people are and I have I've  seen it in in practice that like people are okay [30:08] we are not going to do this because like I don't  have budget to give everyone everything right so [30:12] I see that all the time citizen spend development  is like very very good but without controls it is [30:20] just going to like uh tank your budgets I see  them and I think it also speaks very briefly to [30:25] the knowledge management system that you have to  have in place, right? You don't want all five of [30:29] us building the same thing in slightly different  ways and spending, you know, a ton of money trying [30:34] to do that when there actually there probably  is one best practice. So, I think you're right. [30:37] There's a spectrum and there's a point where  the citizen development has to lead to this [30:41] is a best practice. We're going to adopt that as  an organization and you have to have a knowledge [30:45] management sharing culture as well as systems  to be able to know when to move to that which is [30:51] the same as moving from pilot to enterprisewide  adoption. Yeah. We also didn't talk about data. [30:56] Yeah. Um for for that to happen, you do need the  right data in place and the right guard rails and [31:03] also we may not be at the same spot when it comes  to knowing what to build, when to build and even [31:09] how to build it. So that education piece. Now  if we're talking about the company 5 10,000 [31:15] employees, are you going to put in that investment  behind that? There's one thing um uh software we [31:23] talked about it great data we talked about it  but there's also the AI infrastructure piece [31:27] which is a huge investment as well um I think with  sovereignity becoming a key aspect for different [31:35] nations u there's a huge interest in having  on-prem solutions but typically for IT budgets [31:41] if you usually buy something and that works for  a long time you depreciate it for over three or [31:47] five years and it's done but that's changing with  AI hardware right 6 to 12 months becomes obsolete [31:54] you get hardware that's 10x faster 5x cheaper for  in terms of computation so how do you manage your [32:02] budget in this kind of a situation that becomes  challenging how do you manage obsolescence so [32:07] on an ongoing basis you have to think about  continuous refreshment and the investment [32:13] itself is no longer a line item but becomes a  continuous cost so to plan for that on an ongoing [32:20] basis we work with stakeholders identifying what  their needs are and planning for the refreshment [32:25] cycle and making sure we budget for that. So the  if you think about it obsolescence doesn't happen [32:32] when the hardware goes bad right now when you get  a better model or a better hardware piece that's [32:38] the point it becomes obsolescence obsolete so it  becomes from a capex to an opex kind of investment [32:44] something that you have to think about and change  the way of budgeting rather than thinking about [32:50] how much we budget we also have to think about  how do we budget this investment so um scaling I [32:59] is definitely a challenge. Going from pilots to  enterprisewide adoption and scale is something [33:07] that requires investments and it requires that  training piece, the change management piece. [33:12] You do need the right technology, the right  infrastructure and technology changes. We do [33:17] see it. Uh you have changes in models, you have  got um new vendors sprouting and new solutions. [33:24] You can definitely wipe code things in house or  even build things in house but you do need that [33:30] clear ownership that governance and measurable  business outcomes. So now that we are wrapping [33:37] up the panel I'm going to ask each and every one  of you to give us a few words of advice. What are [33:44] some of these best practices around scaling from  pilot to that um governed uh enterprisewide agent [33:54] AI solution and how do we become successful start  so one of the key things usually when there is [34:02] resistance it's um there is fear but there is also  the fear of things breaking down right so you have [34:09] to consider that and how do you address that one  of The things that have worked well for us is to [34:15] find early adopters within different organization  and use them and help them test out a particular [34:21] solution and once um they are able to create  the success and impact they become champions [34:28] for that particular applications and they can  um and when people see that they're seeing [34:33] these their organizations are functioning with  much better efficiency. they have a much better [34:39] visibility with leadership because of these new  applications. Scaling becomes much easier instead [34:44] of pushing it down uh and making it a top down uh  kind of approach. It becomes a pull and a bottomup [34:51] approach where people come and ask for that  particular solution and scaling becomes efficient. [34:57] I would say um one of the biggest things is  to is to not think about it as what you're [35:05] building now will be in place for as long as your  existing the software that we're used to building [35:11] would normally be in place. It's it's going to  be changing quickly, but that's not a reason [35:17] to wait. And it is okay and maybe something  to like instill in your organization that it [35:24] it's something to be celebrated if something you  built is only in service for a short amount of [35:29] time and then replaced by something new that  came out later this year or whatever. That's [35:35] uh that's the reality that we're in. Um and I  think that that's that's one of the big things is [35:40] to just don't get too attached to it. celebrate it  if it only lasted for a little while and already [35:45] got replaced by something that was cheaper and  easier and better to build. Yeah, I couldn't [35:50] agree more with but both what you both said. Um  I would say there is a spectrum between pilot to [35:55] enterprisewide adoption. If you think about it  as first we're going to do pilots and then we're [35:59] going to do enterprisewide adoption then you're  going to have a lot of risk involved. I love [36:03] what you're saying Seth is think about that as a  roadmap of iterations. Um, so you have the pilot, [36:07] then you have a bigger pilot, a bigger experiment,  now you have an initiative, now you know, and now [36:10] we're getting towards enterprisewide adoption.  So there's no there's no magic bullet when you [36:15] can just say we're going to switch from this  to this. And so I would continue to think about [36:18] that because then it also instills the culture  of we're going to continue to iterate even if [36:22] we have rolled it out enterprisewide. We're going  to continue to fail fast. We're going to continue [36:26] to push the needle on what this can be and how it  can change in the future. So I just really agree [36:30] with that. thinking about that more as a spectrum  and an ongoing iteration model versus a now we're [36:36] turning the switch on. And I think the second  thing is kind of going back to my my first point [36:40] is um the people aspect of it to who needs to  implement your solution is such a critical piece [36:47] and if you are sensing any sort of resistance  um I would think through breaking down what that [36:52] potential resistance looks like in my experience  resistance to something is very uh rarely like [36:58] uh philosophical resistance. It's usually lack  of clear guidelines, lack of what what good is, [37:03] what the expectations are going to be, how people  are going to be held to that. Make sure you're [37:07] focusing on the change management piece of the  implementation or it will stall regardless. I [37:13] I couldn't agree more with all of you. Like we  talked about a lot of things in terms of taking it [37:18] from pilot to enterprisewide adoption and scale. I  think like the one thing I would add is the people [37:24] aspect of it, not in terms of like whether they  trust or not. That is very very important and you [37:30] need to install it right from the beginning. But  how do they interact with it? So what is happening [37:35] is that like every new vendor, every new developer  has like a new application and then they want the [37:42] whole enterprise to adopt to a new application.  That's not going to happen. I mean, if you have [37:46] a 20,000 workforce or a 5,000 workforce, if you  ask them, hey, I have this cool new thing that [37:52] is going to change the way you do your business  and go and instead of using Salesforce, I'm just [37:58] going to use Salesforce as an example. Instead  of using Salesforce, use this another tool. Now, [38:03] that is like everyone has to get ramped up  on this new tool and that is not going to [38:07] happen. And that is also a very big hurdle for  enterprisewide scaling. So what we need to be [38:13] able to do is iterate in a way that like you don't  actually take away the tool that people are used [38:19] to right now but in infuse AI into that. Can you  take the piece that you want to do and add value [38:28] to either actually the way people work or create  a widget and then like implement that widget into [38:36] where the people work today so that they are  not exposed to like 100,000 new applications [38:42] and then they will have to like go and learn and  everything. I mean we in it like we think more [38:47] in terms of this is a cool thing like but uh when  I look at the from the other side of the spectrum [38:53] okay I have learned this tool over a few years  I know every nook and corner of the tool some [38:59] people are empowered thinking that like okay this  tool I am the expert on this tool taking that away [39:04] and giving them a new tool every two days every  two months is not going to work and people are [39:09] going to resist so we need to be able to infuse  this thing inside uh the place they work today [39:16] and over a period of time make it a completely new  one that you have your vision. So I want to thank [39:24] everyone for joining the panel and our speakers.  Reena Solomon, thank you. Seth Sigler, excellent [39:31] point of view. Uh Tracy Synindic, uh we loved  your perspective on that human side of things, [39:37] the change management. And Ragu, thank you. U and  what we are um realizing it's uh don't ignore that [39:46] human side. Change management don't underestimate  it. It's more than technology. Yeah. Thank you.