Cox, Weatherford AI Leaders Limit Agent Autonomy Over Data Security

因数据安全考量，Cox与Weatherford AI主管限制智能体自主权 企业级AI智能体需要严格的定制与数据防护栏，因为现成模型在复杂的企业环境中难以奏效，且大多数顶层智能体面临一年内被淘汰的境遇。 该专家小组探讨了在企业部署中定制AI智能体而非依赖现成解决方案的务实策略。发言嘉宾详细阐述了如何在人机协同控制与完全自主之间取得平衡，如何构建诸如模型上下文协议（MCP）和检索增强生成（RAG）等稳固的数据安全框架，以及如何驾驭由编排（Orchestrator）、超级（Super）和实用（Utility）智能体组成的三层架构。他们还坦诚讨论了大型企业在衡量投资回报率（ROI）时的两难困境、短期智能体实现的迅速过时，以及涵盖呼叫中心、软件工程、供应链管理和B2B营销的实际落地案例。 当涉及精度、安全性、品牌信任及核心价值观时，企业必须对 AI Agent 进行深度定制，而通用的开箱即用型 Agent 往往难以应对混乱的内部 API 与专有数据环境。 (0:32) 目前仅有不到 10% 的企业用例实现了完全自主的 Agent 运作；企业普遍采用人机协同（HITL）机制，仅将高频且低风险的任务完全交由 Agent 自主处理。 (0:48) Cox Communications 提出了一种三层操作架构（编排 Agent、超级 Agent 和实用 Agent），通过将任务解构为模块化单元，有效治理多 Agent 系统的复杂度与数据访问权限。 (16:03) 安全合规与 RBAC 访问权限治理是阻碍企业扩展数千个 Agent 的最大瓶颈；相比直接微调模型，采用 RAG 可以防止企业专有资产被永久写入模型权重。 (18:15) 由于业务流程演进迅速，大多数顶层 Agent 的使用寿命可能不足一年，专家建议企业应投资于长期的模块化基础设施，而非一次性静态部署。 (31:11)

Enterprise AI adoption requires balancing tailored agent customization with disciplined risk governance, as off-the-shelf software consistently fails to navigate messy proprietary data and high-stakes operational environments. Panelists argue that sustainable deployment relies on modular architectures, rigorous access controls, and realistic expectations regarding short agent lifespans and return on investment. Evaluating Customization Triggers and Autonomy Limits (0:32) Off-the-shelf agents struggle in complex corporate settings because they cannot reflect proprietary values, risk tolerance, or brand identity. Sarah Karthigan points out that in high-stakes fields like the energy sector, safety cannot be evaluated by generic text models using rule checklists; it requires grounding in internal operational history. Deepak Anchala and Murad Hasan add that external-facing agents must embody unique corporate personas, while internal deployments require custom engineering to navigate fragmented enterprise APIs and context-heavy retail data. Determining when to grant autonomy depends on error cost, reversibility, and process variance. Anchala uses a 1-to-5 scale, arguing that high-frequency, low-risk tasks like ticket triaging are prime candidates for full automation, whereas strategic or high-liability decisions require continuous human oversight. Eric Pace notes that fewer than 10% of enterprises have successfully deployed fully autonomous agent use cases, emphasizing that virtually all current corporate processes retain a human in the loop because organizations struggle to clean and trust underlying data. Multi-Tiered Architecture and Enterprise Security Bottlenecks (16:03) Eric Pace demystifies corporate claims of deploying hundreds of agents, explaining that organizations typically run five to ten multi-agent systems broken down into a three-tiered structure. At the top, an Orchestrator coordinates task tracking; in the middle, Super Agents manage business-specific logic and human-in-the-loop interactions; at the base, reusable Utility Agents handle isolated, low-level data fetching. This modularity isolates complexity and standardizes security profiles. Governance remains the primary bottleneck to enterprise scaling. Pace and Anchala emphasize that AI agents and protocols like Model Context Protocol (MCP) lack native mechanisms to inherit existing enterprise Role-Based Access Control (RBAC), forcing IT departments to avoid direct connections between agents and data stores. To prevent proprietary data leaks, Karthigan recommends Retrieval-Augmented Generation (RAG) over model fine-tuning, warning that an agent given unintended access to privileged data can rapidly make erratic and unauthorized operational decisions. The ROI Paradox and Agent Obsolescence (23:36) Measuring return on investment creates operational friction between development teams and executive leadership. Pace highlights an enterprise paradox: corporate executives frequently demand immediate two-week payback periods, forcing teams to focus on minor use cases that are fundamentally incapable of moving top-line revenue or bottom-line profit. Karthigan and Anchala emphasize that projects must target clear operational needle-movers, separating technical usage metrics—such as latency and hallucination rates—from financial outcomes and brand trust. Furthermore, panelists assert that AI implementations cannot be treated as static software. Pace contends that outside of core utility layers, most application-specific agents built today will not reach their first birthday because underlying workflows and AI capabilities change too rapidly. Addressing this turnover, Anchala advocates for a 5-to-10-year product vision that shifts from prompt-and-response mechanisms to trigger-based systems, while noting that custom internal builds must be weighed against third-party software when agents risk rapid deprecation. Operational Deployments and Domain Guardrails Current enterprise agent deployments vary sharply across industry risk profiles. Anchala observes that retail and B2C interactions permit higher agent autonomy based on user behavior, while legal and healthcare environments demand low autonomy and strict human verification. Where autonomous customer interactions occur, such as in call centers, Pace and Anchala emphasize bounding agents with deterministic guardrails, including strict spending caps and automated escalation to human queues for sensitive queries. Production deployments span diverse enterprise functions. Cox Communications utilizes unassisted agents in customer support, upstream agents to write software epics and user stories tailored to internal architecture, and super-agent layers to generate personalized B2B marketing campaigns. Karthigan details Weatherford deployments across drilling optimization, supply chain tracking, and contract comparison, while Anchala highlights client use cases in automated sales outreach and cloud cluster management. When agents must mutate or update underlying databases, Anchala stresses the necessity of API-level maker-checker validation to verify changes before execution. While enterprises are moving agents past experimental stages into supply chains, call centers, and software development, broad autonomy remains limited by access governance deficits and short architectural life expectancies. Sustainable adoption currently depends on modular utility architectures, strict guardrails on data mutation, and human oversight in non-reversible workflows.

在由 Ai4 主持的研讨会上，来自 Logicon、Adopt AI、Weatherford 与 Cox Communications 的企业高管指出，通用现成方案无法满足高风险的企业级需求，定制化 AI Agent 必须依托专有数据与严格的治理护栏；同时，企业应通过分层架构与人在回路机制，在控制失败风险的前提下平衡自主性与长期投资回报。 定制驱动：品牌契约、专有数据与失败成本 (0:32) 现成（Off-the-shelf）第三方 Agent 难以直接适应复杂的企业环境。Weatherford 的 Sarah Karthigan 指出，在能源等高合规与安全敏感行业，安全关乎生命底线，仅依靠通用行业语料训练的模型无法安全评估风险，必须基于企业内部历史数据与风险承受度进行深度定制。只要业务涉及企业核心竞争力、安全记录或品牌信任，定制化就是必选项。Logicon 的 Murad Hasan 与 Adopt AI 的 Deepak Anchala 补充说明，零售等高上下文业务需要 Agent 解析跨部门孤岛数据与消费模式，且对外交互的 Agent 必须精准呈现企业的独特品牌形象；由于企业实际的 API 与底层数据往往杂乱无章，简单的外层套壳工具在面对专有工作流时普遍失效。 自主权分级与人在回路的现实约束 (0:48) 确定 Agent 的自主权界限取决于错误的可逆性与潜在损失。Anchala 提出了类似自动驾驶的 1 至 5 级分类法，主张将基础客服工单分流等高频低风险任务全自动化，而在重大财务交易或战略决策等高风险、低频场景中保留人工验证。主持人 Çağla Kaymaz 与 Hasan 强调，规则明确、低方差的任务最适合自主执行，而不可逆且面向客户的差错必须引入人工干预。在企业落地现状方面，Cox Communications 的 Eric Pace 表示，目前几乎所有企业流程默认均包含人在回路（HITL），行业数据显示成功部署全自主 Agent 的企业不足 10%，核心瓶颈在于多数公司无法将底层数据清洗到足以建立完全信任的水准。当面向客户的实时场景无法设置人工审核时，必须施加硬性限制（如退款金额上限）及双人复核（maker-checker）验证机制以防越权。 三层架构设计与企业安全访问瓶颈 (16:03) 为消除市场上“部署数百个 Agent”的营销夸大，Pace 阐述了实操中的三层架构模型：顶层为协调并追踪任务的编排 Agent（Orchestrator Agent）；中层为融入业务定制与人工监督的超级 Agent（Super Agents）；底层则是高复用、低权限、仅执行简单重复任务的基础工具 Agent（Utility Agents）。这种模块化结构让企业在不增加管理复杂度的前提下，对输入输出实施严格治理。然而，安全治理仍是制约规模化扩展的最大障碍。Anchala 将缺乏管控的 Agent 比作失控野兽，强调生产环境必须部署模型上下文协议（MCP）服务器、基于角色的访问控制（RBAC）以及传输和静态加密。Pace 指出，当前 Agent 与 MCP 协议缺乏原生安全框架，绝大多数现有方案无法动态继承用户原有的企业 RBAC 权限。对此，Karthigan 建议采用检索增强生成（RAG）而非微调模型，以只读方式检索知识，避免敏感资产被永久固化进模型权重中引发合规风险。 投资回报矛盾与 Agent 的生命周期悖论 (23:36) 在衡量商业回报方面，企业管理层面临即时变现与战略投入的矛盾。Karthigan 强调 AI 项目不能为了验证技术而立项，必须从运营指标（压缩周期与人力）、财务影响（降本增收）、品牌信任与长期战略四个维度设定 KPI。Anchala 指出，代码生成 Agent 已验证了降本成效，但负责业务拓展与销售的外部 Agent 因缺乏人际交互温度仍未被充分证明。Pace 揭示了企业 ROI 的悖论：高管往往苛求两周内的极短投资回报期，迫使团队选择微小场景，而这类场景本质上无法拉动整体营收或利润，AI 领导者必须主动引导管理层承担更多风险以换取更大价值。与此同时，Pace 提出业务层面的 Agent 大多生命周期极短，由于业务流程迭代迅猛，顶层业务 Agent 往往活不过一周年；Anchala 与 Karthigan 进而建议，企业不应耗费资源构建一次性工具，而应制定 5 至 10 年的长期技术底座规划，确保关键应用具备战略替代成本与长期适应力。 跨行业落地实践与工程化部署范例 各行业在落地 Agent 时展现出明显的自主权差异与场景特征。Anchala 归纳称，零售与 B2C 场景通常依据消费者行为赋予 Agent 较高的自主决策权，而法律行业则因强合规要求必须维持低自主性与全流程人工介入。在具体部署实例中，Pace 介绍了 Cox Communications 的三大方向：一是在呼叫中心使用具备确定性边界的自主 Agent 进行意图识别与业务分流；二是在软件研发上游部署定制 Agent，直接编写符合内部架构规范的 Epic 与用户故事；三是通过超级 Agent 层生成按客群定制的 B2B 营销内容，并由底层工具 Agent 处理数据检索。Karthigan 则透露 Weatherford 已在供应链库存管理、智能招投标、文档智能比对以及钻井作业优化中实际嵌入了协同 Agent；而 Adopt AI 的客户也在 AWS 集群运维、采购审批及财务计费流程中广泛应用了 Agent 方案。 当前企业在推进 AI Agent 时面临的核心未解问题，在于如何在应用层 Agent 高度易耗、生命周期可能不足一年的现实下，平衡内部定制自研与外部采购的经济效益，同时在底层基础设施层面解决 MCP 协议与既有 RBAC 权限体系无缝集成的安全技术瓶颈。

## Background 本内容由 Ai4 于 2025 年 8 月 29 日发布在 YouTube 平台上。视频记录了一场由 Category Ventures 合伙人主持、汇聚了来自 Weatherford、Logicon、Adopt AI 及 Cox Communications 等企业高管与技术领袖的圆桌讨论，主要探讨企业如何根据自身业务目标、工作流与行业特定挑战来定制化 AI Agent，并平衡自主权与合规控制。 ## TL;DR 圆桌深入剖析了企业落地 AI Agent 时在“开箱即用”与“深度定制”之间的权衡，强调在高精度、强信任与差异化场景中结合专有数据的必要性。嘉宾们提出了基于“出错代价与任务形态”的自主权分配决策框架，以及由协调 Agent、业务超级 Agent 和底层实用 Agent 组成的三层系统架构。针对从 PoC 跨越到生产级所面临的数据安全、RBAC 权限穿透、数据突变（Mutation）控制以及真实 ROI 评估困境，与会专家分享了包括“制作者-复核者”双重验证在内的前沿治理实践与多行业真实落地案例。 ## Atmosphere & style 整场圆桌务实坦诚、直击企业落地痛点，嘉宾们毫无行业浮夸之气，而是以一线操盘手的视角直面技术理想与企业现实之间的巨大鸿沟。对话在幽默与犀利之间自然流转，既有关于“大多数 Agent 活不过一岁”的现实拷问，也有对企业高管急功近利追求短期 ROI 的清醒反思。多位不同背景的领袖在敏锐的思辨碰撞中逐步凝聚共识，让听众深刻感受到 AI 从实验室 Demo 迈向复杂生产系统时所需的敬畏与工程严谨度。 ## Key takeaways - [03:29](https://www.youtube.com/watch?v=SG_-AVJsso0&t=209s) 通用开箱即用方案无法替代深度定制：在涉及精准度、信任底线与业务差异化的关键场景中，必须结合企业内部专有数据、风险偏好与操作规范构建专属 Agent。 - [08:35](https://www.youtube.com/watch?v=SG_-AVJsso0&t=515s) 自主权分配遵循‘高频低风险放权、高风险低频留人’的原则，目前行业内仅有不到 10% 的企业真正落地了完全自主的 Agentic 场景，绝大多数仍依赖人机协同（HITL）。 - [11:42](https://www.youtube.com/watch?v=SG_-AVJsso0&t=702s) 评估 Agent 是否应获得完全自主权的核心过滤器在于‘出错代价有多大’以及‘结果是否可逆’，可撤回且规则确定的高重复任务适合全面放权。 - [15:04](https://www.youtube.com/watch?v=SG_-AVJsso0&t=904s) 企业级 Agent 架构普遍采用三层模型：顶层 Orchestrator 负责全局调度，中层 Super Agent 承载业务定制与人机协同，底层 Utility Agent 专注执行原子化且通用的重复任务。 - [18:31](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1111s) 从 PoC 走向生产环境的最大阻碍在于原生安全机制的缺失，特别是向量数据库与文件存储中难以准确继承并执行企业级 RBAC 权限控制。 - [20:06](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1206s) 面对专有核心资产，RAG 相比模型微调（Fine-tuning）更具数据隔离优势，能够避免战略资产参数化外溢并在检索时直接嵌入动态 RBAC 权限过滤。 - [28:11](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1691s) 企业衡量 AI 价值存在断层：两周内搭建的小用例根本无法直接撼动大盘财务回报，追求高商业价值必须匹配更长的研发周期与更高的风险承受度。 - [30:16](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1816s) 构建 Agent 需告别传统软件‘一次性交付上线’思维，AI 是动态演进的目标，架构重心应从优化单点模型性能转向提高底层基础设施的适应性。 - [31:56](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1916s) 除底层通用工具层外，绝大多数业务层智能体‘活不过一岁’，随着业务流程再造必须持续重构，企业的长期差异化壁垒在于专有数据资产与架构层。 - [37:35](https://www.youtube.com/watch?v=SG_-AVJsso0&t=2255s) 在缺乏人工介入的即时交互场景中，必须引入硬性业务护栏（如操作金额上限与调用阈值）；人机协同越少，所需的确定性护栏就必须越严密。 - [38:38](https://www.youtube.com/watch?v=SG_-AVJsso0&t=2318s) 当智能体涉及直接写库或数据变更（Mutation）操作时，除了遵循 API 接口约束外，需针对增量数据引入‘制作者-复核者’（Maker-Checker）双重验证机制。 - [39:47](https://www.youtube.com/watch?v=SG_-AVJsso0&t=2387s) Agent 落地正在各垂直领域规模化分化：零售与客服聚焦高自主度的意图识别与分流，研发上游渗透至需求生成，而能源等工业领域则深耕供应链与钻井工艺优化。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=SG_-AVJsso0&t=4s) 圆桌嘉宾背景与 AI Agent 业务交集介绍 主持人 Çağla Kaymaz（Category Ventures 合伙人，常驻旧金山的种子轮投资人）开场并欢迎观众，她表示自己的投资与研究重心聚焦在 AI 技术栈（从底层基础设施到上层应用），尤其是近期火热的 AI Agent 领域。随后，她邀请四位来自不同领域的嘉宾介绍各自的背景以及 Agent 技术如何融入他们当下的工作。 Logicon 首席执行官 Murad Hasan 首先发言，他指出 Logicon 致力于构建并集成能够主动适应企业业务的 AI Agent，而不仅仅是做简单任务的机械自动化。他们将 Agent 定义为企业的“运营队友”（operational teammates）或“嵌入式副驾驶”（embedded co-pilots），因为这些 Agent 直接在决策与推荐层面上产生影响，这也使得系统定制化变得尤为关键。 Adopt AI 联合创始人兼首席执行官 Deepak Anchala 介绍道，Adopt AI 作为一个面向工程和产品管理团队的 Agent 构建平台，帮助应用软件公司在产品内部和外部快速上线 Agent。他强调，相比传统非智能应用，Agent 的核心优势在于用户只需使用自然语言指令即可达成目标结果，无需经历复杂的应用软件学习成本。 Weatherford（威德福国际）人工智能副总裁 Sarah Karthigan 分享了自己在能源行业深耕 20 年利用数据和 AI 推动数字化转型的经验。她指出，Weatherford 已经走出了实验室的试验阶段，将 Agent 真正嵌入到了关键业务工作流中，员工们不仅将它们视为工具，更当作并肩作战的协作者。 Cox Communications（考克斯通信）人工智能负责人 Eric Pace 介绍了公司作为连接服务提供商在美国西南、中西部和东部多个州的业务版图。在 AI 方面，他负责从战略规划到落地实施的全流程，目前不仅在推进企业级 Agent 的构建，还在推行“平民开发者”（citizen development）理念，激励全员动手构建属于自己的 Agent，以规模化激发创新想法并提升全公司的 AI 素养与采纳度。 --- ### [03:29](https://www.youtube.com/watch?v=SG_-AVJsso0&t=209s) 定制化 Agent 与通用开箱即用方案的权衡 针对市场上第三方及开箱即用（off-the-shelf）Agent 层出不穷的现状，主持人 Çağla Kaymaz 提出核心疑问：不同企业的业务模式与痛点各异，适合企业 A 的方案往往无法照搬给企业 B，企业该如何判断哪些场景和垂直领域必须进行深度定制，哪些场景可以直接采用通用方案？ Sarah Karthigan 从能源大厂的实操经验切入，她笑称自己几乎每天都会遇到各种供应商推销他们现成的 Agent，声称“只要插进工作流，一切就能自动搞定”。但在真实企业环境中，Agent 必须体现该公司的独特价值观、战略布局以及业务价值创造方式。她以能源行业的安全问题为例，指出安全在能源领域是生死攸关的绝对核心底线，一旦出错就可能导致人员伤亡。因此，仅仅依赖基于行业公开数据训练的通用 Agent 绝不可行，必须结合企业内部专有数据进行深度定制，将企业的风险容忍度、历史事故处理经验以及具体操作规范融入模型中。Sarah 总结道，在所有涉及**“精准度（precision）、信任（trust）和差异化（differentiation）”**的关键场景，定制化都是必不可少的。 ```mermaid flowchart TD accTitle: Customization Decision Framework for AI Agents subgraph Drivers["定制化核心驱动要素"] D1["精确度要求 (Precision)"] D2["信任与合规底线 (Trust)"] D3["品牌与业务差异化 (Differentiation)"] end subgraph Data["数据与业务上下文"] C1["企业专有数据与历史经验"] C2["企业风险容忍度与价值观"] C3["复杂 API 与分散数据孤岛"] end subgraph Solutions["Agent 落地形态"] S1["开箱即用 Agent (标准/低风险流程)"] S2["深度定制化 Agent (关键/高风险工作流)"] end Drivers --> C1 & C2 C1 & C2 & C3 --> S2 ``` Deepak Anchala 从 Agent 构建平台的视角补充说明，像医疗健康、金融服务等强监管行业固然需要严密的制衡与合规机制，但事实上几乎所有与他们接洽的企业都需要定制化。正如企业从做网页到做移动端 App 一样，对外发布的 Agent 需要传递独特的品牌形象与人设，而对内提供给团队的 Agent 则必须解决杂乱的数据、混乱的 API 以及复杂的业务工作流。“你不可能只套一个 LLM 包装壳（LLM wrapper），然后宣称自己实现了工作流的 Agent 化——这种做法根本行不通，员工根本不会去用。” Murad Hasan 进一步提炼出“富上下文环境”（rich context environments）的概念。如果业务不仅数据量庞大，而且数据形态各异且分散在各个业务孤岛中，Agent 就必须学会该组织的“专属语言”。以零售行业为例，Agent 仅掌握产品目录是远远不够的，还必须深刻理解消费者的行为模式和购买习惯。企业内部跨部门的数据孤岛越多、上下文越复杂，就越必须通过定制化来释放预期的业务效能。 --- ### [08:35](https://www.youtube.com/watch?v=SG_-AVJsso0&t=515s) 人在回路（HITL）与任务自主权分配框架 在探讨了上下文、品牌和风险之后，Çağla Kaymaz 引导大家探讨自主性与风险控制之间的平衡：虽然业界一直在畅想高度自主的智能体，但现实中很多企业出于风险考量仍保留了“人在回路”（Human-in-the-Loop, HITL）机制。企业在面向客户或内部场景构建 Agent 时，应如何界定哪些任务可以完全放权给 Agent，哪些任务必须保留人工审核与确认？ Deepak Anchala 给出了一个清晰的决策矩阵框架： - **高频低风险（High Frequency, Low Risk）流程**：应尽可能实现完全自动化与自主执行。 - **高风险低频（High Risk, Low Frequency）流程**：例如重大战略决策、大额高价值交易等，目前仍必须保持人在回路进行审核验证，不能赋予完全自主权。 Deepak 将 Agent 的自主程度比作 1 到 5 级的划分标准（5 级为最高自主），不同行业与场景会处于不同的等级。这正如特斯拉的 FSD（完全自动驾驶）功能一样，每位车主和用户在实际使用时都会根据具体路况和自身信任度选择不同的介入程度与使用模式。 ### [09:54](https://www.youtube.com/watch?v=SG_-AVJsso0&t=594s) 自主权边界与人机协同的天然存在 不同行业与业务场景对 AI Agent 自主权的需求大相径庭，正如许多人开特斯拉，但有人偏爱全自动驾驶模式，有人则习惯自行操控。在客户支持这类高重复性、低风险的职能中，企业可以赋予 Agent 极高的自主权以实现自动化；而在涉及人际关系与信任构建的销售场景中，Agent 现阶段更多是作为辅助工具帮助销售人员成单。因此，在赋予 Agent 自主权之前，必须审视行业属性、业务职能以及风险收益与执行频率的平衡。 从端到端的业务流程来看，“人机协同（Human-in-the-loop）”在当前阶段几乎是默认且不可避免的。除非一整条价值链实现了百分之百的端到端全自动闭环，否则 AI Agent 生产出的任何中间结果，在流程下游必然有一个人类员工去接收并执行下一步。如果该员工不去审核或查验 Agent 产出的质量，那本质上就是该员工在流程中的履职表现问题。当前业界鲜有企业能彻底解决数据质量与边界划分难题，以至于能放心对 Agent 说“去吧，别管了（Okay, go agent go. And then see you later.）”。市场上数据显示只有不到 10% 的企业真正落地了完全自主的 Agentic 用例，绝大多数场景仍需要长期的“人机协同”作为安全网。 - **用例成熟度现状**：目前全行业仅有不到 10% 的企业跑通了高阶、完全自主的 Agentic 场景，核心瓶颈在于无法将业务场景和数据治理简化到足以完全信任的离散可控状态。 --- ### [11:42](https://www.youtube.com/watch?v=SG_-AVJsso0&t=702s) 权衡自主权与控制的核心过滤器 评估是否给予 Agent 完全自主权，可以归结为一个极简的决策过滤器：“出错的代价究竟有多大（What's the cost of something going wrong?）？”当失败后果严重时，哪怕 Agent 完全具备独立完成任务的能力，也必须强制引入人类干预。反之，当业务价值直接与执行速度和效率的跨越式提升挂钩时——例如客服工单的快速分流（Triaging）或日常运营报表的自动汇编——就应当给 Agent 充分放权，把人类员工从繁重的手工劳作中解放出来，专注于高阶判断与战略决策。 评估自主权风险还应聚焦于三个维度的核心问题：首先是“错误的破坏力”，即系统失误是否会导致直接财务损失或企业信誉瓦解；其次是“规则的清晰度”，如果业务逻辑边界清晰、规则完备，自主执行的成功率便极高；最后是“任务形态”，高容量、高重复度且低变异性的任务，天然是 Agent 自主运行的最佳土壤。此外，结果的“可逆性（Reversibility）”同样是关键考量：将客服工单分流错误还可以召回重审，但直接向外部客户发送错误回复则具有不可挽回的外部影响。 | 评估维度 | 高自主权适用场景（Full Autonomy） | 受限/强人工介入场景（Human-in-the-Loop） | |:--- |:--- |:--- | | **错误代价与可逆性** | 代价微弱，失误后可撤回或易修正（如内部工单初筛打标） | 涉及重大财务风险、法律合规或用户品牌信任（如合同签订、对外发布） | | **规则明确程度** | 拥有固化、确定性的 SOP 与清晰判定逻辑 | 依赖直觉、柔性谈判、情感互动或模糊边界的主观判断 | | **任务形态特征** | 高吞吐量、高度重复、变异度极低（Low-variation tasks） | 长尾非标场景、高定制化、偶发性复杂的单点决策 | --- ### [15:04](https://www.youtube.com/watch?v=SG_-AVJsso0&t=904s) 数据基础设施与三层 Agent 架构模型 在将敏感企业数据与特定行业数据接入 Agent 提示词与模型推理时，企业必须建立坚实的数据隐私与基础设施屏障。在技术路线上，可以通过保留本地私有化基础设施（On-prem）来运行核心 Agent 推理，并在 AWS 或 GCP 等多云环境中精细管控数据与算力部署，坚持“模型就近部署于数据旁（put our inferencing next to the data）”，严格限制无节制的组件间通信。针对业界频繁宣传的“构建了上百个 Agent”的营销炒作，实际上这些用例往往被拆解为极小颗粒度、可精准监控输入输出的微小组件协同运行。 ```mermaid flowchart TD accTitle: 现代企业三层 Agent 架构模型 subgraph Layer1["协同层 Orchestration Layer"] A["协调 Agent (Orchestrator Agent)<br/>全局工作流调度与任务分派"] end subgraph Layer2["业务定制层 Business Logic Layer"] B["超级 Agent 1 (Super Agent)<br/>适配特定业务线定制逻辑与人机协同"] C["超级 Agent 2 (Super Agent)<br/>适配差异化业务流程"] end subgraph Layer3["底层基础层 Utility Layer"] D["实用 Agent A (Utility Agent)<br/>不可调用下级 / 专注通用重复任务"] E["实用 Agent B (Utility Agent)<br/>跨企业级复用 / 严格数据权限管控"] end A --> B A --> C B --> D B --> E C --> E ``` 企业在实践中普遍推行由三层体系构成的落地架构： 1. **顶层协调 Agent（Orchestrator Agents）**：充当全局指挥调度角色，负责监控多方任务进度，确保所有参与组件协同运作。 2. **中层超级 Agent（Super Agents）**：深度结合具体业务场景进行定制化，承载特定业务单元的差异化流程，同时也是嵌入“人机协同（Human-in-the-loop）”与业务规则的核心管控层。 3. **底层实用 Agent（Utility Agents）**：作为系统级“主力军（Workhorses）”，专注执行最原子化、高度可复用的底层任务；此类 Agent 无法向下递归调用其他 Agent，严格对外暴露可控的数据读写权限，从而实现企业级能力的模块化复用。 当庞大的 Agent 矩阵协同运作时，必须为其套上严格的护栏机制（Guardrails）。未经约束的 Agent 就像脱缰野兽，只有在严密的安全防护与策略监控下，才能确保其稳定运行在企业既定轨道之内。 ### [18:31](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1111s) 从概念验证到生产级：数据安全与 RBAC 权限控制的深水区 当讨论从简单的 API 交互转向真正让智能体接触底层企业数据时，技术落地面临着完全不同的安全挑战。在实验环境或概念验证（PoC）阶段，开发者往往只需关注生成输出是否符合预期；然而，一旦步入生产环境，系统面对的是高度敏感的企业内部资产以及复杂的非 RBAC（基于角色的访问控制）权限机制。此时必须妥善处理一系列基础设施级的问题：是否需要构建专门的 MCP（Model Context Protocol）服务器？如何在全流程中落实 RBAC？传输中（in-transit）和静态（at-rest）数据该如何加密？即使发生意外的数据泄漏，又该如何设立兜底防御？部署一套企业级生产智能体，其工程复杂度与在临时测试台上跑 Demo 截然不同。 这种安全层面的缺失正是阻碍智能体实现大规模落地的核心瓶颈之一。目前市面上的 Agent 架构和 MCP 协议在设计之初普遍缺乏内置的原生安全防护机制。一旦让智能体直接访问底层数据，企业往往不得不冒着在应用层重复配置和镜像整套安全策略的巨大风险，而这在安全合规层面几乎是不可接受的。许多企业之所以迟迟不敢将成千上万个智能体推向生产线，根源就在于没人愿意在 MCP 通信边界模糊、智能体行为不可控的情况下接手运维这一套庞大冗余的安全镜像。更棘手的是，当智能体检索被存放在向量数据库（Vector Store）或本地文件存储中的内容时，极少有现成解决方案能够准确继承原本的数据访问控制规则，并根据发起请求的具体用户身份严格穿透执行这些权限。 ```mermaid flowchart TD accTitle: 生产级智能体数据权限与安全隔离流程 subgraph ClientLayer["客户端与身份层"] User["请求用户 (Requestor)"] RBAC["身份与权限上下文 (RBAC Context)"] User --> RBAC end subgraph AgentLayer["智能体编排与治理层"] Agent["AI 智能体 / MCP 服务"] Guard["安全围栏 & 传输加密 (TLS/At-Rest)"] RBAC --> Agent Agent --> Guard end subgraph DataLayer["企业敏感数据资产"] RAG["知识检索 (只读/无永久存储)"] Proprietary["专有战略资产 (严格受控隔离)"] Guard --> RAG Guard -.->|"阻止越权访问"| Proprietary end ``` ### [20:06](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1206s) 模块化编排与数据保护：微调训练与 RAG 检索的架构博弈 面对上述安全困局，一种可行的技术路径是构建模块化的多智能体编排框架。在该体系中，企业能够高度灵活地即插即用不同的数据源与底层大模型，依靠一个具备全局理解能力的中央编排器（Orchestrator）来解析复杂的业务需求，进而动态调度多个专业智能体协同执行任务。无论是采用纯本地部署（On-Prem）还是依托云端架构，数据安全都绝无妥协余地。如果某个智能体绕过了控制防线，接触到了未授权的数据，它就极可能基于错误或敏感的信息做出失控决策，瞬间在业务系统中引发不可收拾的连锁反应。 企业在处理“秘密武器”（Secret Sauce）——即核心专有数据时，必须审慎权衡训练（Fine-tuning）与检索增强生成（RAG）的技术选型： ``` +------------------+-----------------------------------------------------+-----------------------------------------------------+ | 维度 | 专有数据微调训练 (Fine-Tuning) | 知识库检索增强 (RAG-Based) | +------------------+-----------------------------------------------------+-----------------------------------------------------+ | 核心机理 | 将战略资产权重直接固化到模型参数中 | 仅在需要时进行实时检索，知识库只读不落盘 | | 数据暴露风险 | 模型输出可能无意识吐露底座训练语料 | 严格限制在单次会话上下文中，不造成永久性外溢 | | 权限继承难度 | 极高：模型难以对不同权限层级的用户做动态切分 | 相对可控：可在检索前置过滤器中直接应用 RBAC 规则 | | 利益相关方诉求 | AI 团队追求灵活性与上下文饱和度 | 法务与合规团队追求可审计性、确定性与数据留痕透明度 | +------------------+-----------------------------------------------------+-----------------------------------------------------+ ``` 对于法务与合规团队而言，系统的透明度和可审计性（Auditability）是不可逾越的红线；而算法与业务团队则往往渴望尽可能开放的数据访问权限以保障上下文的丰富度。在两派势力的拉扯中，找到“上下文丰富度与数据资产防护之间的平衡”往往是“说起来容易做起来难”（Easier said than done）。然而，唯有通过创造性的安全架构将权限控制无缝贯穿整个智能体网络，企业才能真正拥有坚实的数据隐私层，建立将 AI 推向生产的底气。 ### [23:36](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1416s) 走出 PoC 的成本深渊：商业价值指标与 ROI 评估体系 从技术原型跨越到生产系统不仅面临架构挑战，更伴随着高昂的真实成本：密集的人力投入、庞大的云端算力开销、漫长的集成测试周期以及潜在的失误风险。当管理层直截了当地追问“投资回报率（ROI）到底在哪里”时，AI 领袖不能再以“证明技术的先进性”为借口。任何不能切实推动业务结果（Move the needle）的用例，即便技术构想再惊艳，也必须在优先级排序中被果断淘汰。 构建严谨的落地价值评估框架需要将指标拆分为多个互为支撑的维度： - **技术与使用指标（Usage Metrics）**：作为基础运营监控，涵盖幻觉发生率（Hallucination Rates）、任务完成率（Completion Rates）、响应延迟（Latency）及准确率（Accuracy）。 - **运营效率指标（Operational Metrics）**：核心在于缩短端到端的业务流转周期（Reduction in cycle times）。 - **财务与价值指标（Value Metrics）**：通过业务方案直接挂钩成本削减（Cost Reduction）或营收增长（Topline Growth）。目前业内落地最扎实、见效最快的多为代码辅助等内部降本型智能体，但量化开发人员“究竟提升了多少生产力”依然极具挑战；而期望直接带来收入的外部智能体（例如 AI BDR、销售线索挖掘智能体）由于极度依赖“人际信任”（Human touch），整体表现仍处于摸索磨合阶段。 - **品牌信赖与客户体验（Brand & Personal Touch）**：针对面向客户的外部场景，评估建立用户信任度对业务转化的隐性拉动。 - **战略级差异化（Strategic Impact）**：虽然难以用短期财务报表量化，却关乎企业在面对“AI 原生创新企业”时能否构筑护城河。面向外部的智能体不仅关乎当前效率，更演变成一场生死攸关的存亡博弈（Existential question）和抢占市场份额的“圈地运动”（Land grab），需要整个组织自上而下达成战略共识。 现实推进中常常伴随着巨大的认知讽刺：企业高管往往一面急切催促业务部门交出“立竿见影的短期回报周期”（Immediate payback periods），一面又对见效慢的深度改造心存疑虑；这种急功近利的压力反过来迫使团队只能先挑那些微小局限、难以撼动全局的边角用例去应付交差。 ### [28:11](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1691s) 价值衡量的讽刺：小用例与高管预期的鸿沟 讨论深入到了企业在衡量 AI 智能体价值时普遍面临的困境与矛盾。Eric Pace 指出当前业界关于价值度量的讨论充满了讽刺意味：许多团队花费两周时间构建一个极其微小的智能体，然后试图去向管理层证明它能否拉动顶线营收（Top-line Revenue）或改善底线利润（Bottom-line Profit），答案显然是“绝不可能”（"Hell no"）。企业高管和首席财务官（CFO）们不断追问效能指标与投资回报，但之所以没人能给出完美答案，是因为大多数项目始终被困在细微的用例中，无法实现跨越。 Eric Pace 直言，企业不能一方面要求获得巨大商业价值，另一方面又要求在两周内看到立竿见影的回报。他真诚地建议团队应当正视这种矛盾，直接向管理层明确：“如果你想要更大的价值，我们就必须承担更多风险，并投入更多时间。”对此，Sarah Karthigan 补充指出，核心在于掌握平衡艺术——虽然无法在两周内交付彻底颠覆业务的成果，但如果小型试点能够展现出可扩展至大规模商业影响的潜力，那就是值得持续投资的方向，而非仅停留在浅层的试水实验。Murad Hasan 进一步强调了建立信任的渐进逻辑：尽管节省工时、减少人工干预和增加营收等运营 KPI 至关重要，但在初期，必须先确保小型用例的产出完全可信，在组织内部建立起信心底座，随后才能向顶层的高阶 KPI 逐步跃迁。 - **小用例与大目标的断层**：两周内开发出的单一智能体无法撼动公司大盘财务数据，过度纠结短期 ROI 会限制高价值项目的探索。 - **与管理层沟通的真实准则**：追求高价值回报必须匹配相应的风险承担意愿与更长的研发周期。 - **指标落地的进阶路径**：从验证小型场景输出的准确性与可信度入手，逐步扩展到减少人工干预率、缩短任务耗时与最终的营收贡献。 --- ### [30:16](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1816s) AI 不是一次性交付：从模型到基础设施的演进 Çağla Kaymaz 将话题引向行业的前瞻性布局：业内普遍坚信未来 5 到 10 年 AI 智能体将彻底重塑工作与生活方式，若企业今天不进行投入，未来将面临巨大的追赶成本。然而技术迭代日新月异，如何在今天投入资源的同时，避免系统在短时间内被淘汰废弃？ 针对这一挑战，Murad Hasan 明确提出构建 AI 智能体绝不能沿用传统软件“一次性交付上线”（One-time software deployments）的固有思维。AI 是一个不断变化的动态目标（"AI is a moving target"），团队的关注点必须从单纯追求当前性能转向系统的“适应能力”（Adaptability）。业界常谈论如何让模型具备前瞻性（Future-proofing），但更关键的是要确保底层基础设施具备持续演进与适应的能力。只有将基础设施设计为可动态升级的架构，系统才能在技术不断迭代的浪潮中持续释放商业价值。 ```mermaid flowchart LR accTitle: AI Agent Infrastructure Evolution subgraph Mindset["传统思维 vs AI思维"] A["传统软件：一次性交付上线"] -->|转变为| B["AI 智能体：动态演进目标"] end subgraph Focus["架构重心转移"] B --> C["单点模型性能优化"] B --> D["底层基础设施适应性 (Adaptability)"] end subgraph Outcome["最终成效"] D --> E["长期业务敏捷度与持续价值交付"] end ``` --- ### [31:56](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1916s) “活不过一岁”的智能体与长期差异化战略 讨论触及了一个颇具戏剧性的行业现实：虽然大家常倡导将智能体视作“团队伙伴”（Teammate），但在实际研发中，这些智能体往往极易被抛弃和替代。Eric Pace 分享了其团队内部的激烈争论，他坚定地认为，除了最底层的通用工具层（Utility agent layer）之外，当前正在构建的大多数业务智能体“根本活不过它的一岁生日”（"will never see their first birthday"）。随着可用技术工具的更迭以及智能体引发的业务流程再造，上层智能体必须被不断修改重构；如果有人认为部署一个智能体后不需要对既有流程进行重大重组，那完全是不切实际的幻想。 Deepak Anchala 随后从产品战略维度展开分析，呼吁企业必须建立 5 到 10 年的长远愿景，明确自身在市场中的真正差异化壁垒在于专有数据（Proprietary Data）与智能体架构层。如果仅着眼于 1 到 2 年的短期目标，或许能取得短暂成效，但最终会在激烈的市场竞争中被淘汰。产品领导者需要规划清晰的自主性演进路线，避免在拥挤的赛道中盲目复制竞品已有的简单功能。 - **智能体的演进形态路径**： 1. **Prompt-Response 阶段**：被动接受提示词触发，执行基础的增删改查（CRUD）操作； 2. **Trigger-Based / Schedule 阶段**：基于外部事件或时间表自动触发，自主完成预设工作任务； 3. **Fully Autonomous 阶段**：具备高阶自主决策与复杂流程编排能力的自主智能体。 - **避免同质化竞争**：结合企业专有数据资产构建专属智能体，而非开发市场上随处可见的通用型同质工具。 --- ### [34:04](https://www.youtube.com/watch?v=SG_-AVJsso0&t=2044s) 拒绝低价值消耗：构建具备适应性的核心资产 Sarah Karthigan 对智能体的生命周期策略进行了深刻反思。她提出核心拷问：我们构建智能体究竟是为了维持现有的业务运行模式，还是为了适应未来业务的演变趋势？如果一个智能体在业务体系中如此轻易即可被抛弃，那么企业从一开始就不应该耗费资源去构建它。 企业不应将精力浪费在明知明天就会失效的短期临时方案上，而应全力打造具备长期适应能力的核心资产。真正的“团队伙伴型”智能体应当深度融入组织，一旦失去它，团队会产生明显的痛感与业务真空。Sarah 强调，战略优先级的规划必须具备前瞻性，这些被精心培育的智能体未来极有可能演化为企业自身最核心的业务支柱与竞争优势。 - **战略优先级的取舍**：果断放弃短期不可持续的修补型用例，将资源聚焦于具备演进能力与核心业务粘性的智能体系统。 - **衡量智能体价值的情感与业务标尺**：评估其是否真正成为不可或缺的“核心成员”，其离线是否会对核心工作流产生实质性冲击。 --- ### [35:36](https://www.youtube.com/watch?v=SG_-AVJsso0&t=2136s) 自研还是采购？现场观众关于实时无介入与数据突变的提问 Çağla Kaymaz 提出了另一个企业决策的核心维度——“自研还是外购”（Build vs. Buy）。如果内部自研一个智能体需要极高的定制化成本，且该智能体在一年内就会落后淘汰，那么企业自行开发可能并不划算；相反，外部商业软件供应商（Vendor）能够将研发成本分摊给成千上万家客户，并由专业团队持续维护升级，此时采购供应商方案对企业而言是更优的选择。 随后现场进入问答环节，一位身着蓝色上衣的观众提出了两个直击落地痛点的技术与业务问题： 1. **完全无人工在环（No Human-in-the-Loop）场景的可行性**：在实时客户支持等无法插入人工审核环节的高并发即时交互场景中，是否建议构建全自动智能体？如果构建，最佳实践是什么？ 2. **检索增强生成（RAG）环境下的数据突变（Mutation）控制**：RAG 常用于控制只读的数据检索访问，但当智能体需要对数据执行写入、修改等“突变操作”时，应该如何安全有效地进行控制与约束？ 针对第一个问题，Deepak Anchala 回应称这属于典型的“风险-收益评估框架”（Risk-reward Framework）。在绝大多数客户支持体系中，引入智能体是非常必要的，行业的核心逻辑在于“拦截率”（Deflection Rates）——智能体应当在前端尽可能独立消化并解决标准化、可处理的咨询工单，仅将自身无法处理的复杂长尾问题实时转交给人工作席接管。 ### [037:35] 自主性与风险控制：为智能体设立业务安全护栏 在讨论特定业务场景（如医疗健康）时，嘉宾们强调必须保持审慎，不能盲目将所有流程“智能体化”（agentify）。如果面对的是患者咨询病症并询问处置方案的高风险场景，现阶段绝不能完全放任自主智能体处理。智能体的落地必须根据具体的业务细微差异（business specific nuances）权衡是否适用。 ```mermaid flowchart TD accTitle: 智能体安全护栏与自主权控制架构 subgraph Governance ["护栏与风险控制 (Guardrails)"] A["业务规则约束<br/>(如退款上限/操作阈值)"] B["人机协同 (Human-in-the-Loop)"] C["损失风险量化评估 (Quantify Risk)"] end subgraph Execution ["执行层 (Execution)"] D["受控自主执行<br/>(Bounded Autonomy)"] E["高风险/超阈值转人工"] end A --> D B --> D C --> D D -->|触发阈值| E ``` 企业构建自主运行的智能体时，绝对不能给予其无限自由，必须设立清晰的护栏（Guardrails）。例如在客服支持场景中，可以允许智能体自主决定如何解决问题，但必须严格限制其权限——“绝不允许给客户发放超过特定金额的积分或退款”，也不能逾越预设的操作阈值。人机协同（Human-in-the-loop）越少，所需的硬性护栏就必须越多；只有当企业能够准确量化潜在损失风险，且对智能体的行为边界完全有把握时，才能真正放手让其自主运行。 ### [038:38] 数据访问与数据变更（Mutation）的安全护栏 针对现场关于“当智能体不仅检索数据，还需要执行数据变更（Mutation）与状态更新时该如何控制权限”的提问，嘉宾指出智能体访问数据主要有两种路径：API 与直接数据库连接。 当智能体通过 API 交互时，API 本身通常已具备权限与数据校验护栏，难点在于随着 API 的迭代演进，智能体需要具备自适应能力，准确识别并调用最新版本的 API。而当智能体通过模型上下文协议（MCP, Model Context Protocol）或直接连接数据库进行读写时，系统必须构建应对数据增量变化的机制。在向智能体输入新数据或提交更新时，必须引入“制作者-复核者”（Maker-Checker）双重验证机制作为护栏，严格审查数据变更与写入权限。 - **API 交互模式**：利用已有 API 的接口权限作为天然护栏，需处理 API 版本演进与适配。 - **直接数据库/MCP 模式**：监控数据变更增量（Delta），引入独立的 Maker-Checker 复核机制决定是否允许数据变更或回传给智能体。 ### [039:47] 企业落地的三大核心智能体场景与分层架构 在观众提问具体落地案例时，考克斯通信（Cox Communications）的 AI 负责人 Eric Pace 分享了他们重点部署的三大智能体方向： 1. **呼叫中心无人值守智能体（Unassisted Agent Support）**：通过确定性规则机制或生成式推理（Generative Inference）精准识别用户来电意图，在护栏限定范围内自主处理诉求，必要时再无缝转接人工坐席。 2. **研发流程左移与需求生成（Epic & User Story Generation）**：在 AI 辅助编程之外，将定制智能体嵌入敏捷开发上游，根据企业内部特定架构规范，自动生成 Epic 和用户故事（User Story）。 3. **B2B 个性化营销触达智能体**：在中台构建“超级智能体”（Super Agent）层，根据行业、客户分群及产品特性深度定制个性化沟通内容，底层则调度轻量化的“工具实用智能体”（Utility Agents）负责数据抓取、格式化和信息查询。 ```mermaid flowchart TD accTitle: 超级智能体与实用智能体协作架构 subgraph SuperAgentLayer ["超级智能体调度层 (Super Agent Layer)"] SA["业务定制超级智能体<br/>(B2B 营销/行业分群/策略定制)"] end subgraph UtilityLayer ["工具实用智能体层 (Utility Agents Layer)"] U1["数据抓取智能体 (Data Fetching)"] U2["数据结构化智能体 (Structuring)"] U3["系统查询智能体 (Lookup)"] end SA --> U1 SA --> U2 SA --> U3 ``` Murad Hasan 进一步补充了行业差异对自主权的影响：在零售（Retail）及 B2C 场景中，智能体可以拥有极高的自主度去分析理解用户行为；但在法律（Legal）等对数据安全极度敏感的严谨领域，必须收紧自主权，建立极为严格的护栏并加入人工干预，确保数据绝对不发生安全泄露。 ### [042:26] 多行业前沿落地案例与圆桌总结 在最后的总结环节，嘉宾们快速盘点了各自在实际生产中上线的各类智能体用例，展现了智能体在工业与企业软件领域的广泛渗透： - **工业与能源领域（Weatherford）**：上线了供应链管理与库存管理智能体（解决核心库存痛点）、智能招投标智能体（Intelligent Bidding）、常规文档对比分析智能体，以及深度的石油钻井优化智能体（Drilling Optimization Agents）。 - **开发与平台生态（Adopt AI）**：其客户构建了多元化的智能体应用，包括面向 DevTool 自动创建与管理 AWS 集群的基础设施智能体、自动化采购智能体、面向 CFO 财务侧的账单结算智能体，以及负责销售拓客的 AI BDR（业务拓展代表）智能体。 随着讨论进入尾声，主持人指出智能体的落地形式极其丰富多元，并邀请现场参会者在会后继续与各位嘉宾交流，圆桌在全场掌声中圆满结束。 ## Notable quotes > [03:29](https://www.youtube.com/watch?v=SG_-AVJsso0&t=209s) 精准度（precision）、信任（trust）和差异化（differentiation） — Sarah Karthigan > [03:29](https://www.youtube.com/watch?v=SG_-AVJsso0&t=209s) 你不可能只套一个 LLM 包装壳（LLM wrapper），然后宣称自己实现了工作流的 Agent 化——这种做法根本行不通，员工根本不会去用。 — Deepak Anchala > [09:54](https://www.youtube.com/watch?v=SG_-AVJsso0&t=594s) Okay, go agent go. And then see you later. — Sarah Karthigan > [11:42](https://www.youtube.com/watch?v=SG_-AVJsso0&t=702s) 出错的代价究竟有多大（What's the cost of something going wrong?） — Deepak Anchala > [15:04](https://www.youtube.com/watch?v=SG_-AVJsso0&t=904s) put our inferencing next to the data — Eric Pace > [28:11](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1691s) Hell no — Eric Pace > [28:11](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1691s) 如果你想要更大的价值，我们就必须承担更多风险，并投入更多时间。 — Eric Pace > [30:16](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1816s) AI is a moving target — Murad Hasan > [31:56](https://www.youtube.com/watch?v=SG_-AVJsso0&t=1916s) will never see their first birthday — Eric Pace > [37:35](https://www.youtube.com/watch?v=SG_-AVJsso0&t=2255s) 绝不允许给客户发放超过特定金额的积分或退款 — Deepak Anchala ## Mentioned Sarah Karthigan, Çağla Kaymaz, Murad Hasan, Deepak Anchala, Eric Pace, Weatherford, Category Ventures, Logicon, Adopt AI, Cox Communications, AWS, GCP, Tesla FSD, Model Context Protocol (MCP), RAG, RBAC ## Video description Sarah Karthigan, Vice President of Artificial Intelligence, Weatherford Çağla Kaymaz, Partner, Category Ventures Murad Hasan, Chief Executive Officer, Logicon Deepak Anchala, Co-founder & CEO, Adopt AI Eric Pace, Head of AI, Cox Communications Abstracts AI agents are not one-size-fits-all—successful deployment requires careful customization to align with business goals, workflows, and industry-specific challenges. This panel will explore how organizations are fine-tuning AI agents through prompt engineering, reinforcement learning, domain-specific training, and integration with proprietary data. Experts will share insights on balancing autonomy with control, ensuring AI agents deliver meaningful results while staying aligned with company values and compliance requirements. Whether you're looking to enhance customer interactions, automate decision-making, or streamline operations, this discussion will provide a roadmap for customizing AI agents to maximize impact. [Source](https://www.youtube.com/watch?v=SG_-AVJsso0)

[3:29] Sarah Karthigan: 精准度（precision）、信任（trust）和差异化（differentiation） [3:29] Deepak Anchala: 你不可能只套一个 LLM 包装壳（LLM wrapper），然后宣称自己实现了工作流的 Agent 化——这种做法根本行不通，员工根本不会去用。 [9:54] Sarah Karthigan: Okay, go agent go. And then see you later. [11:42] Deepak Anchala: 出错的代价究竟有多大（What's the cost of something going wrong?） [15:04] Eric Pace: put our inferencing next to the data [28:11] Eric Pace: Hell no [28:11] Eric Pace: 如果你想要更大的价值，我们就必须承担更多风险，并投入更多时间。 [30:16] Murad Hasan: AI is a moving target [31:56] Eric Pace: will never see their first birthday [37:35] Deepak Anchala: 绝不允许给客户发放超过特定金额的积分或退款

[18:15] Sarah Karthigan: it's going to make decisions out of that and it's going to run amok in no time. [16:03] Eric Pace: workhorses of the universe. [18:15] Deepak Anchala: wild beasts [23:36] Eric Pace: If you want bigger value, we need to take on more risk and we need more time. [23:36] Sarah Karthigan: needle mover [0:04] Murad Hasan: operational teammates [0:04] Murad Hasan: embedded co-pilots [0:04] Eric Pace: citizen development

[0:04] Thanks everyone for coming. My name is [0:05] Chala Kaas. I'm a seedstage investor [0:08] based in San Francisco. I spend a lot of [0:10] my time in the AI stack um anything [0:13] software from applications to [0:14] infrastructure and obviously spending a [0:16] lot of time in agents and really excited [0:18] for this panel. So before we dive in um [0:21] I would like to hear from each of you um [0:23] what's your name? What do you do? Um how [0:26] does agents intersect with your work [0:28] today? Absolutely. [0:30] I'm glad to be here. My name is Murad. I [0:32] am the CEO of Logicon. And we build and [0:35] integrate AI agents that don't just [0:37] automate, but they adapt to your [0:40] business. And that means that uh rather [0:42] than just having to do simple tasks, [0:44] they're able to understand your business [0:46] better and can make uh recommendations [0:50] do a lot of work that uh manual effort [0:52] would take a lot more time of course. [0:54] And in our business with the especially [0:55] the kind of customers that we work with, [0:58] we always uh say agents are more like [1:01] operational teammates for you or [1:03] embedded co-pilots if you put it that [1:05] way. And uh because they have to do more [1:07] than just automating task, they impact [1:10] directly at the decision-m and [1:12] recommendation levels for you or your [1:14] customers and uh that is where [1:16] customizations become really important [1:18] as well. [1:20] >> Hey everyone um nice to meet all of you. [1:23] Um I'm Deepak Kanchchula, founder CEO of [1:25] Adopt.AI [1:26] and at ADOP we help application [1:29] companies launch agents both inside and [1:31] outside the application. So we are an [1:33] agent builder platform for engineering [1:35] teams and product management teams to [1:36] launch agents for their end users. So um [1:39] every team is planning to launching [1:40] these agents now and and the advantage [1:42] of having agents over just apps that are [1:45] not agentic is you know agents can do [1:47] tasks that users want in natural [1:48] language commands and achieve outcomes [1:50] without you having to learn the [1:52] complexities of applications. [1:55] Hey [1:56] >> everybody Sarah Cartigan I am the vice [1:58] president of artificial intelligence at [2:01] Weatherford. I have about uh 20 years of [2:04] experience working in the energy sector [2:06] where I had the opportunity to drive [2:09] transformation using data and AI. So [2:12] this panel in particular is super [2:14] exciting to me because we are now [2:17] looking at AI agents not just as tools [2:20] but as potential teammates and [2:23] collaborators and that's exactly what's [2:26] happening back at Weatherford 2. So we [2:29] have been able to go beyond the [2:32] experimentation phase. So we're not just [2:34] having experiments happening in the lab, [2:37] but we have agents that have been [2:39] embedded into critical workflows. And [2:42] the good news is we are seeing that [2:44] teams are also beginning to adopt it and [2:46] they are looking at them as fellow [2:49] collaborators and not just tools. So [2:51] super excited to be here and looking [2:53] forward to the discussion. [2:56] >> Hello everybody. My name is Eric Pace. I [2:58] work for Cox Communications. We're a [3:00] service provider for connectivity. Uh in [3:02] Atlanta, we serve many states in the [3:04] southwest, Midwest, and Eastern region. [3:07] Um I run AI for Cox, which is everything [3:09] from strategy to implementation. And [3:11] right now from an agent perspective, [3:12] we're looking at all things from [3:14] building enterprise agents down to [3:16] deploying what we refer to uh but didn't [3:19] invent in citizen development and [3:21] getting all of our people to get excited [3:22] about building their own agents as well [3:24] to help not only drive ideation at scale [3:26] but also drive adoption and literacy [3:28] around the company. [3:29] >> Great. Thanks Eric. So my first question [3:32] is around, you know, there's a lot of [3:34] buzz and excitement around like agents, [3:37] but we all know that a lot of times [3:38] thirdparty agents and off-the-shelf [3:40] agents, they're just not enough, right? [3:41] Every business is unique. Um, something [3:44] that works for business A is probably [3:45] not going to work for business A, [3:46] business B. So curious how you guys [3:49] decide what kind of use cases and [3:51] verticals require customization versus [3:54] you say, hey, this is fine to use [3:56] whatever everyone else is using. Sarah, [3:58] do you want to kick us off? [3:59] >> Yeah. I know absolutely that's a great [4:01] question Charla. So because this is [4:03] something that happens to me almost on a [4:05] daily basis right so I do get accosted [4:07] by vendors and they have built some [4:09] agents and they want to see if we are [4:11] able to take that on and stick it into [4:14] the workflow and then automatically [4:16] everything will work. Um but the reality [4:19] is especially in a company setting when [4:22] we are looking at agents we're looking [4:25] to build solutions that can behave in a [4:29] way that is unique to that company. So [4:32] it reflects the company's values. It [4:34] reflects its strategies. It reflects how [4:37] we generate business value. So that's [4:40] where customization is going to begin. [4:42] So let me give you an example right. So [4:44] if you take the energy sector, safety is [4:47] not just a priority. It is a core value. [4:51] If things were to go wrong, we're [4:53] talking about a fatality. That's the [4:56] outcome it can result in if we don't [4:59] take the right measures. So in a [5:02] situation like that, assuming an out [5:05] ofthe-box AI agent that has been trained [5:08] on industry data would work is not [5:12] reasonable. We need to ensure that when [5:14] we are looking at those types of use [5:16] cases, we're building AI agents that are [5:20] not only trained on industry data, but [5:23] they are also trained on proprietary [5:26] internal data that reflects the risk [5:28] tolerance of the company that also [5:31] reflects what the company has done in [5:33] the past when it comes to these types of [5:35] incidents and then it is also a true [5:38] reflection of the operations of the [5:40] company. So customization is key, right? [5:42] I mean again, you know, when you think [5:44] of the energy sector, we think of a very [5:47] highly regulated industry. So we're not [5:49] talking about just another out of the [5:51] box agent that just looks through a [5:54] checklist and checks off, yes, we follow [5:56] these rules. That's not going to cut it. [5:59] We're going to again have to think about [6:01] it in the context of the business. And [6:05] so that's where customization is key. So [6:08] anywhere in in summary what I would like [6:10] to say is anywhere precision, [6:14] trust and differentiation is at stake, [6:18] we need to look at customization. [6:21] >> Yeah, that's a great point. Uh [6:22] precision, trust and differentiation. So [6:24] I'll I'll answer the question from a [6:26] slightly different perspective because [6:27] we are an agent builder for various [6:29] types of companies and there are [6:30] definitely some uh sensitive sectors [6:32] like healthcare, financial services, [6:34] more regulated ones where you need you [6:36] need to have a lot of checks uh checks [6:37] and balances in place. Um but what we [6:40] are seeing is typically with all [6:42] enterprises, every enterprise that we [6:44] are talking to requires customization, [6:45] right? If you think of agents that [6:47] they're launching, think of like think [6:49] of the app world, right? when they're [6:50] launching websites and then apps and [6:51] mobile apps, you know, their unique [6:53] personality shows up in the app, right? [6:55] What they want, their workflow show up [6:56] in the app. So, when they're thinking of [6:58] agents, they want that to convey their [7:00] brand, their persona, uh when they're [7:02] launching external agents. And when [7:03] they're launching internal agents for [7:05] their teams, they do want uh you're [7:07] dealing with messy data, messy APIs. [7:09] You're dealing with complex business [7:11] workflows, right? So, you can't just [7:12] have like an LLM wrapper and just say, [7:15] "Hey, I've agentified my uh my [7:17] workflows," right? It just doesn't work. [7:18] people do not adopt it. Um [7:21] if [7:21] >> I was to put in with my experience [7:23] working with customers, um wherever [7:25] customizations come in, the easiest way [7:27] I like to put it is wherever there are [7:30] high context or rich context [7:32] environments where there's not just a [7:34] lot of data, there's different kinds of [7:36] data spread across. And in order for [7:39] agents to work effectively, they need to [7:41] learn the language of your business, the [7:43] language of your organization. They need [7:45] to be in a position where they know what [7:48] this data means and they're able to tell [7:50] a great story about it. For example, [7:52] retail is one example. In retail, um, [7:56] just knowing about the products that [7:58] your company has is of course not just [8:00] going to be enough for agents. It needs [8:02] to know your customers behaviors, their [8:04] patterns, their buying patterns. The [8:07] more your agents gets accustomed to [8:09] them, the better they're going to turn [8:10] out to be. So customizations [8:13] they become critical whenever there are [8:15] contextrich environments. If you feel [8:17] like your business has a large size of [8:20] data spread across multiple verticals [8:22] and different silos spread around that [8:25] is where these agents will absolutely be [8:27] must for them to get customized in order [8:30] to bring out the efficiency they're [8:31] looking for. [8:35] >> Great. Um that's very interesting. [8:37] Depends on context. It depends on the [8:39] brand. It depends on the risk factor. [8:42] Actually, my second question is going to [8:43] be around risk. So, we talk a lot about [8:46] autonomous agents, but in reality, [8:48] there's still a lot of agents where [8:49] humans stay in the loop, right? Because [8:51] businesses have different needs and [8:53] different risk tolerances. So curious [8:56] for you guys when you build agents for [8:58] your customers or when you build it for [9:00] internal use cases or your customers [9:01] actually at your enterprise, how do you [9:03] decide when you want to keep the human [9:05] in the loop versus you're completely [9:07] okay like delegating that specific task [9:09] to an agent? [9:11] >> I can I can take a stab at that one and [9:13] then maybe Eric would love for you to [9:14] speak as on that. Um so um the the [9:17] framework we like to apply is any [9:19] process that is high frequency low risk [9:22] should be fully automated right uh to [9:24] the extent possible uh any process that [9:26] is high risk low frequency like [9:28] strategic decision-m um high value [9:31] transactions etc should still today as [9:34] of today still have a human in the loop [9:36] to verify and and uh not have like [9:39] complete autonomy. So the way I look at [9:41] it is you you can have autonomous agents [9:43] from a scale of 1 to five. Five being [9:45] the highest, one the lowest. And [9:46] different sectors or industries will [9:47] adopt autonomy at different scales. [9:49] Right? Think about FSD. When we use [9:50] cars, everybody uses FSD differently on [9:53] a Tesla. I don't know how many of you [9:54] have Teslas, but some prefer driving it [9:56] as full autonomy, some don't. Um and [9:59] that's what we're seeing across [10:00] industries also across functions. Like [10:02] if you think of for example customer [10:04] support uh customer support is a [10:06] function that is kind a lot of there's a [10:09] lot of autonomy uh because it's a lot of [10:11] repetitive tasks that you can you can [10:12] automate but when you when you look at [10:14] things like like sales for example where [10:16] you need human touch relationships etc [10:19] there is still you know agents are still [10:21] helping humans close deals um so I think [10:24] you have to take a look at the industry [10:25] function at uh riskreward frequency and [10:29] before making that call [10:31] Yeah, I think for us I it it might even [10:34] be more simple than that. The answer is [10:35] everything. Um, and like the way I see [10:37] it and until you have fully automated [10:39] the entire value chain of something, at [10:41] a very minimum, when an AI agent [10:43] produces something, there's a human on [10:44] the other side receiving whatever [10:45] they've produced to use it to go on with [10:46] the process. Therefore, by default, you [10:48] have a human in the loop. Because if [10:50] that person isn't checking what that [10:51] agent produced, then we've got a [10:53] performance problem with the person in [10:54] the process, right? And so, by nature of [10:57] the way a process works today, you have [10:59] a human in the loop no matter what. You [11:00] just have to have them oriented to [11:02] capturing something or receiving [11:03] something from an agent instead of from [11:05] another human. If you we if you [11:08] depending how you look at it, we we [11:09] prefer to put a humans in the loop no [11:11] matter what. Even inside the process of [11:13] what the agents are doing, I don't know [11:14] that anybody's figured out their their [11:16] data quality problems yet to the tune [11:18] where they can just say, "Okay, go agent [11:20] go." And then, you know, see you later. [11:23] that that's like that you y you all [11:24] might have seen content yesterday that [11:26] said less than 10% of companies have [11:27] figured out how to do fully agentic use [11:30] cases so far it's because nobody can [11:32] figure out how to simplify their use [11:33] cases down to a point where the data is [11:35] so discreetly manageable that they can [11:37] trust it and they can let it go and so I [11:39] think there's humans in the loop for a [11:41] while for everything [11:42] >> yeah let me let me build on that right [11:45] so for me it comes down to a very simple [11:48] filter what's the cost of something [11:50] going wrong right If the cost is high [11:54] well you don't want complete autonomy [11:57] given to the agent not that the agent [12:00] cannot do the work it can but you want [12:03] to build in human in the loop right so [12:04] that's kind of the the filter that I use [12:06] to see okay can it be fully autonomous [12:08] or not for for something else that deep [12:11] was pointing out right so when there is [12:13] value [12:15] that is tied to speed right so and [12:18] you're looking at potentially just a a [12:21] step change when it comes to efficiency. [12:24] Now those are activities that you want [12:27] to make sure you have granted an AI [12:30] agent full autonomy and you know it can [12:32] absolutely break loose and and do stuff [12:34] you know it's like triaging support [12:36] tickets or daily operational reports you [12:39] want to free up humans so that they are [12:42] involved in more judgmentbased [12:44] activities more decision- making versus [12:47] being bogged down by manual labor so [12:49] those are those are some examples so [12:51] that's where I I kind of strike that [12:53] balance between full autonomy and nope [12:56] limited autonomy. [12:58] >> Excellent points by the panelists. I I [13:00] usually just adding on to these we [13:03] usually assess this uh these in three [13:05] high level questions. We ask that what [13:07] is number one the impact of a mistake? [13:11] Is it going to be a financial loss if [13:13] something goes wrong? Is it going to be [13:16] trust loss if something goes wrong? Then [13:18] of course autonomy can be very risky. [13:20] You got to have human intervention [13:21] involved in making sure things are going [13:22] the right way. [13:24] Then there's clarity of rules. Number [13:25] two, if you have a well- definfined [13:28] structure of how the automation needs to [13:30] be done, what rules it needs to follow [13:32] to achieve to an end result, autonomy [13:34] can be a very good solution there. And [13:36] lastly, just uh extending to what Deepik [13:39] said as well, it's high volume, [13:42] repetitive and low variating tasks. So [13:45] if you have a very high volume of tasks, [13:48] they are low in variations and that [13:52] actually sets a very strong foundation [13:54] for autonomy that these agents can work [13:56] on. Then you have a lot of less risk [13:57] going on over there. So but this this [14:00] actually all of this balances out with [14:02] how much you're able to protect your [14:04] data as well. That opens a whole lot of [14:06] sort of doors too. [14:08] >> Um great your point on like the risk and [14:12] what could go wrong. Like one way I like [14:14] to think about it is is this reversible? [14:15] If you're triaging sport tickets, if you [14:18] mist triage it, it will probably come [14:20] back. It's not the same as directly [14:22] respond to a responding to a customer, [14:24] for example. So how reversible is the [14:27] result is an interesting point. So you [14:29] just mentioned data and Eric, you were [14:31] talking about data quality. When we talk [14:33] about customizing agents, we are talking [14:34] a lot about data, right? We're talking [14:36] about industry data. We're talking about [14:38] proprietary enterprise data. We're [14:40] talking about customer data. you might [14:42] not even be fine-tuning or training, but [14:44] you're still using that in your [14:46] prompting and you might be sending it in [14:48] some cases to the cloud. So, I'm sure [14:50] it's very top of mind for all of you. [14:51] So, I would like to maybe hear from you. [14:54] How do you ensure especially for [14:55] sensitive data? How do you ensure [14:57] privacy and sensitive data at the end of [15:00] day sensitive? [15:02] Who wants to go first? [15:03] >> I can take. [15:04] >> Yeah, I think so, you know, we there's [15:07] there's a couple of things. One, if you [15:08] go back to the the notion that we keep [15:10] our use cases small, we also have the [15:13] luxury of owning or a lot of our own [15:14] infrastructure and so a lot of our [15:16] inferencing and agent uh the autonomy [15:18] the autonomous nature of our agents [15:21] exist and ride in an on-prem [15:22] infrastructure where we can control a [15:24] lot of things but we also have I'll call [15:26] it the scale and maturity to manage our [15:29] AWS and GCP environments in a way that [15:31] allows us to you know control where we [15:34] put our inferencing next to the data [15:35] that we want to run it against. We don't [15:37] do a whole lot of, you know, chatty [15:39] communication type stuff across a whole [15:40] lot of things. The like the until you [15:43] get to a point where you've got the [15:44] operations associated with how you're [15:46] managing LLM interactions, what your [15:48] agents are doing, how your agents are [15:50] communicating, etc. The you have to [15:52] build that out and be comfortable with [15:54] the use cases that you're scaling. And [15:55] so, you know, we always we started small [15:57] and we're starting to get bigger, but we [15:59] still haven't gotten to a point where [16:01] there's, like I said before, endless [16:03] amounts of um uh agents and processes [16:06] running autonomously. I think I I I can [16:09] also help provide maybe a little bit of [16:10] tangibility for you all to take back [16:12] with you relative to not just where the [16:14] customization sits, but where the [16:15] management layers and where the humans [16:17] ultimately sit in the process. Um, we [16:19] didn't we didn't practice this, but I'm [16:21] going to go off script for a minute. [16:22] Like think about think about every time [16:25] you hear a company that's built a [16:26] hundred agents and you're like what the [16:28] hell 100 agents I've built one like what [16:30] are you talking about? Like what I find [16:32] when I talk to people is there's a lot [16:33] of marketing hype associated with what [16:34] people are saying and so like we built [16:36] an agent the other day and it was really [16:38] eight agents or 10 agents that were all [16:40] working together and now you're thinking [16:41] well eight to 10 agents that's a lot but [16:43] really structure it in a way that says [16:45] if you want to build an agent you've got [16:47] something and I'm going to build a [16:48] three- tier model for you to take home. [16:50] You've got this notion of an [16:51] orchestrator agent or something that's [16:52] coordinating across what agents are [16:54] doing. Then in the middle, we have what [16:56] we call super agents. And those are the [16:58] things that are actually tailored to [16:59] business need. It's where we put most of [17:00] our customization. It's where we say, [17:02] "Hey, you're going to do this process [17:03] and you're going to do this process. [17:04] You're going to do it slightly [17:05] differently." So, we're going to give [17:06] you each a moderately similar super [17:08] agent to go do those things. Those [17:10] agents are going to be responsible for [17:12] working with what we call utility [17:13] agents. And those utility agents are the [17:15] workh horses of the universe. They're [17:17] the ones that are at the lowest level. [17:19] They can't call other agents. They're [17:21] extensible to the rest of the [17:22] enterprise. They do all of our [17:23] repeatable stuff. And that's mostly [17:25] where we want to focus and manage the [17:27] use of access to, you know, data output, [17:30] etc. And then once you get into that [17:32] middle tier layer is where you're [17:33] looking at the customization and putting [17:34] human in the loop, but you're telling [17:36] people, hey, focus in the middle layer. [17:38] That's mostly what it's going to do for [17:40] you in particular in your job function. [17:42] We'll manage that whole infrastructure [17:44] of these utility agents at the bottom [17:45] that we want reuse for the enterprise [17:47] across. And these orchestrators at the [17:49] top are really just the ones that are [17:50] saying, "All right, we got a couple [17:52] other folks here that have jobs to do. [17:53] Let's just go make sure everybody's [17:54] doing their job." And so when you hear [17:56] somebody that's built like a hundred [17:57] agents, my guess is they've built five [18:00] to 10, and those five to 10 are all [18:02] comprised of many agents in a process [18:04] like I just described. and you're [18:05] building them small enough so you can [18:07] manage the inputs and outputs of each of [18:09] them in a way that covers you know where [18:10] you want your customization how you want [18:12] to manage your data etc etc [18:15] just just piling on uh on top of that [18:17] you know you talked about orchestrated [18:18] agents super agents the smaller agents [18:20] so we see that too when you have all [18:22] these agents the the individual agents [18:23] one very important thing is setting [18:25] guardrails right for agents because [18:26] agents you know what they're like wild [18:28] beasts you let them lose and they'll fly [18:30] right so you have you have to tame them [18:31] down you have to set guard rails the [18:33] right way and then when when agents now [18:35] you go from like accessing APIs to [18:37] accessing data that's where you you deal [18:40] with a lot more sensitive data you deal [18:41] with non rarback right so you have to [18:44] figure out do you create an MCP server [18:46] out of this how do you handle role-based [18:48] access control how do you encrypt data [18:50] in transit and and rest right even if [18:52] there is a leakage for example you have [18:55] to take care of a lot of these things [18:56] before you deploy agents and therefore [18:58] deploying production grade agents is a [19:00] completely different story versus [19:01] running a PC on a staging environment [19:03] where you just just see the output. [19:05] >> Yeah, I think the like I'm just going to [19:07] pile on for a second there. I think I [19:09] think another thing that you all should [19:10] take home with you is I think another [19:12] reason that is getting in the way of [19:14] super scale of all of this stuff is the [19:16] fact that for both agents and MCP like [19:20] there's no security built into these [19:22] things, right? And so once you start [19:24] having these agents access your data [19:26] like you're you you've lost or you've [19:28] signed up for replicating all of that [19:30] stuff into you know those applications [19:32] and that's not a good idea and everybody [19:34] knows that and so the other reason why I [19:37] think nobody's taken you know thousands [19:38] of these things to production is because [19:40] no one wants to manage a replication of [19:41] their security environments when you [19:43] have no idea how to control the in-n-out [19:45] from an MCP perspective or even what [19:48] agents are doing when they're accessing [19:49] data that you've put in a vector or [19:50] you've put in some you a file store, [19:53] etc. You know, there's there's very very [19:55] few solutions out there that that can [19:57] inherit the arbback or uh access [19:59] controls that you've got for your data [20:01] and then carry them through based on the [20:03] requesttor that is interacting with the [20:05] agent. [20:06] >> I was just going to say that you know [20:09] what he described is a very modular [20:11] framework, right? So something that is [20:14] easy to just plug in and and so you have [20:18] the opportunity and you have the [20:20] flexibility to switch around your data [20:22] sources, switch around your models and [20:24] whatnot. And this whole concept of [20:27] multi-agent framework that is coming [20:28] into the picture where we have this [20:30] orchestrator that's smart enough to [20:32] understand what it is you're looking for [20:34] is able to pull together a workflow [20:38] comprising of multiple agents to go [20:40] execute on the task. Back to the the [20:44] data point, [20:45] rolebased access is key, especially when [20:49] you're dealing with sensitive data, [20:50] right? And whether you're operating all [20:53] on prem or you are working on the cloud, [20:57] you need to of course ensure that from a [20:59] security perspective, there are no [21:01] trade-offs because if an agent can [21:03] access that data, if it can access data [21:05] that it should not have access to, [21:06] you've lost it already, right? So it's [21:08] going to make decisions out of that and [21:11] it's going to run a muck in no time. So [21:14] ensuring that all your security features [21:16] are you know um they are being carried [21:20] through across the framework is is going [21:22] to be key. Another thing I want to point [21:24] out is what are you using to train your [21:27] agents? We do understand we talk a lot [21:29] about those those custom agents which [21:31] need to be trained on proprietary data. [21:33] Awesome. And that's usually your secret [21:35] source, right? So that's your strategic [21:38] asset and so if you're training that on [21:40] that versus using more of a rag approach [21:45] right so if you are using a rag approach [21:48] and then the agent has access to that [21:52] knowledge store only for retrieval basis [21:55] it is not permanently storing it. So you [21:57] just need to be very creative about how [22:00] you are ensuring your security features [22:02] for the data are being again um carried [22:06] throughout that agentic framework right [22:08] so of course um you don't want to find [22:13] out later that yes you know you did not [22:17] plan on it but just because we didn't [22:19] have the right controls in place that um [22:22] our agents have access to strategic data [22:25] assets that they should not have and [22:27] then they are of course making decisions [22:29] that they should not be doing. I'll just [22:31] take maybe 30 seconds or so for this. an [22:33] excellent point regarding the rag [22:35] implementation over there because uh [22:37] that is it's in summary it's always [22:40] about balancing and it's easier said [22:43] than done balancing context richness [22:45] with data protection and I'm sure [22:48] several business owners and leaders over [22:50] here are always concerned about you know [22:52] okay we do implement AI agents how [22:54] compromised our data is going to be at [22:56] what what level of risks are we at of [22:58] course ragbased approaches are excellent [23:01] because you're only allowing your agents [23:03] to access data when it's absolutely [23:05] necessary and it's not going to store it [23:07] or make any changes to it accordingly. [23:09] It's just going to be able to give you [23:10] the right set of recommendations and [23:12] decisions. Um but balancing it out is [23:15] always going to be key because legal [23:17] teams where they want um transparency [23:20] and auditability, AI teams, they always [23:23] want access and flexibility. So you have [23:26] to make sure you find the right balance [23:28] in between. That's why I said easier [23:29] said than done. But once you do that, [23:32] you're very confident about your data [23:34] privacy layer as well. [23:36] >> Those are all really valid points and it [23:39] reminds me there's a lot that goes into [23:41] going from, you know, PC's to [23:43] production. You need to worry about [23:45] arbback, you need to worry about data [23:46] sensitivity, you might need to customize [23:49] it using, you know, maybe you're doing [23:50] rack, maybe you're training, but these [23:52] are not cheap. I mean, from a resources [23:54] perspective, right? It requires a lot of [23:56] human labor. It requires a lot of in [23:58] some cases like cloud compute. Um [24:00] there's the you know risk of things [24:03] going wrong and you need to do testing. [24:05] These can be pretty long processes. So [24:08] when you guys are deploying or [24:10] implementing these agents, how are you [24:12] measuring success or are you measuring [24:14] success? How are you justifying it to [24:17] your organizations like what you're [24:19] doing is worth it? [24:21] >> I smile because that's the question that [24:23] comes up all the time, right? So what's [24:25] the return on investment? Yes, we want [24:27] to do all this stuff but tell us exactly [24:29] what are we getting in return. Um I will [24:34] I will you know start by saying this um [24:37] any use case that we are looking at [24:38] right it is not through the lens of [24:41] proving technology that's not what we're [24:44] trying right we have to look at it [24:46] through the lens of is this going to be [24:48] a needle mover for our business [24:51] outcomes. So that's that's what it comes [24:53] down to. So that's your filter for a use [24:56] case getting prioritized and a use case [24:58] not getting prioritized. It can be the [25:00] coolest thing but it if it does not move [25:04] the needle for the business then no it's [25:07] it's it's not going to get prioritized. [25:10] So when we thinking of the KPIs um there [25:13] are different sets of KPIs you use for [25:15] different sets of use cases. You can [25:17] think of it in terms of operational [25:20] metrics, right? So there you're looking [25:22] at just reduction in cycle times. That's [25:24] what you're measuring. You can look at [25:26] it through the lens of financial impact, [25:29] which is typically what we are doing. So [25:32] you're looking at it from a cost cutting [25:35] perspective. Okay, how much money is [25:36] this going to save? Is it going to end [25:39] up growing revenue? Right? So that's [25:42] another thing that we look at. So yes, [25:43] it is going to have a uh a significant [25:47] financial impact then yes obviously that [25:49] is a use case that you want to go after. [25:52] There's also the the whole concept of [25:55] brand and this personal touch especially [26:00] for customerf facing type of [26:02] opportunities and so there you're [26:05] talking about building trust and how [26:06] that's going to move the needle. Last [26:09] but not least, you want to look at [26:11] strategic impact, which is always a [26:14] little harder to measure, but that's [26:16] where you're talking about your [26:17] competitive differentiator and whatnot. [26:19] So, so those are kind of the the [26:21] different types of metrics that you use [26:23] to evaluate. [26:25] >> I can just add on top of that like the [26:27] way the framework I apply is there's [26:29] usage metrics, value metrics. I think [26:30] I'm probably just reiterating what Sarah [26:32] said, but there's also this strategic [26:34] decision-m, right? So usage I think [26:36] everybody tracks like hallucination [26:37] rates uh completion rates latency [26:40] accuracy all of that right is important [26:42] value is where it gets like you you have [26:44] to create a business case for the [26:45] organization so either it's productivity [26:47] improvement cost reduction or topline [26:50] growth and in most cases that are [26:52] successful today like if you see coding [26:54] agents etc it's all impacting the cost [26:57] right like cost reduction because [26:59] productivity improvement is happening [27:00] but it's hard to prove like were you [27:02] more productive with the agents or less [27:03] product it's really hard to prove that [27:05] the agents that impact topline like [27:07] AIBDRs uh sales agents they're still not [27:10] caught up yet right so there's there's [27:11] still that like question mark there [27:13] because you still need a human touch [27:14] also when you think of it are from a [27:16] internal agent versus external agent [27:18] perspective like a lot of companies are [27:20] launching internal agents first right [27:21] and there you're absolutely talking [27:23] productivity and cost but the external [27:25] agents that you launch right you're [27:27] talking about market share capture and [27:28] you're talking about topline and that [27:31] there it becomes an ex existential [27:32] question right um are you competing with [27:35] the AI native companies out there and [27:37] are they doing things differently? Who's [27:39] going to have a land grab here, right? [27:40] It's a it's a very important opportunity [27:42] that you should not miss and these are [27:44] decisions that have to be driven top [27:46] down in a lot of cases and and uh you [27:49] know the whole company has to align by [27:51] it. [27:51] >> You know I I I think the the whole [27:53] question about value is ironic because [27:56] like how many of you in here have [27:58] executives that are telling you that you [27:59] need like immediate payback periods on [28:01] your use cases? like they want them like [28:03] super fast, like why where's the value? [28:05] Why aren't we seeing this yet? You're [28:06] like, "Oh, we can do that." But then the [28:07] use case has to be super small. That's [28:09] why you all are here talking about [28:11] customizing an agent. And when you build [28:12] something that's super small, do you [28:14] think it's going to move topline [28:15] revenue? Do you think it's going to move [28:16] the bottom line? Hell no. Right? And so [28:18] I think the whole conversation is very [28:20] ironic. And the reason that that people [28:22] are continuously asking like how are you [28:24] measuring what what what metrics are you [28:26] using to measure effectiveness or value [28:28] associated because nobody's figured this [28:29] out. and nobody's figured it out because [28:31] nobody can get past these these really [28:33] small use cases that are effectively [28:36] moving some needles just not the big [28:37] ones that the CFOs are, you know, [28:39] screaming for more about. And so I find [28:41] the conversation very ironic that we're [28:43] asking for big value yet at the same [28:45] time we're asking for immediate return [28:47] and there's no way to drive big value in [28:49] a two-eek period where you build a [28:50] single agent. And so, you know, I I I [28:53] recommend in the most um uh authentic [28:56] way possible, like go back and tell your [28:58] executives, if you want bigger value, we [29:00] need to take on more risk and we need [29:01] more time, right? It's no different than [29:03] anything else we've built. [29:04] >> Yeah. I think the secret is this is in [29:06] the balancing act, right? So, there's no [29:09] way that in two weeks you're going to go [29:11] ahead and roll out something that is [29:13] going to move the needle and have a [29:15] significant impact. But if it is [29:18] promising enough to show how it could [29:21] scale and have that impact that you're [29:23] looking for, that's what you want to [29:25] invest in versus again just doing an [29:27] experiment. [29:29] >> Absolutely. Adding to the small use [29:31] cases point that that's exactly the kind [29:33] of scenarios we fall in several times. [29:35] Uh you got to make sure that the [29:37] outcomes of these agents are trusted. I [29:40] mean it they have to be impactful for [29:42] what you're trying to achieve for the [29:44] business. Yes, the operational KPIs are [29:46] very obvious. How much time is it [29:47] saving? Is it do we have less human [29:49] intervention involved now? Uh is it [29:52] increasing our revenue numbers? For [29:53] sure. But those are all bigger large [29:55] scale KPIs that you will eventually get [29:57] to. But as long as your smaller use [30:00] cases, they start to uh get proper [30:04] trustable outputs that are starting to [30:06] move some needles within the [30:08] organization. That's where it begins off [30:10] of and eventually going to the top [30:12] layer. So that's actually a very good [30:13] approach to it. [30:16] >> So one thing um that's clear to me is [30:19] you know whether you have to justify it [30:21] today or not. A lot of people believe in [30:23] 5 to 10 years right this is going to be [30:25] the way we live like the technology is [30:27] evolving and improving so fast that we [30:29] have to make these um investments today [30:31] otherwise it's going to be a while to [30:33] catch up. So that brings me to my last [30:36] question before we opened up the [30:37] audience Q&A. So get ready with your [30:40] questions. Um, I mean the tech is [30:42] evolving so fast. So whether you're [30:44] building something today or tomorrow, [30:46] you kind of need to worry about staying [30:48] relevant. You're not putting a ton of [30:50] resources today that's going to be [30:52] deprecated, but at the same time, you [30:54] can't just wait and play catchup later. [30:56] So actually maybe Murad we can start [30:58] with you on this because we're just [30:59] talking before the panel about how you [31:02] know this is not one and done but what [31:03] are your perspectives around how do you [31:06] ensure what you're implementing stays [31:08] relevant and brings value in the future? [31:11] >> Absolutely. That's a great question [31:12] right there and I always say that when [31:14] you're building AI agents do not treat [31:17] them as one-time software deployments. [31:20] They're different. AI is a moving [31:23] target. You need to be focusing not just [31:26] on performance but on adaptability. [31:30] I mean everyone definitely talks about u [31:33] futurep proofing the model that they [31:35] have but you want to be focusing on [31:37] futurep proofing the infrastructure at [31:39] hand. It's going to keep evolving. The [31:42] more you treat it as a moving target the [31:45] better its outcome it's going to be on a [31:47] continued basis. And it applies to [31:49] pretty much every industry that's going [31:51] to be utilizing it. That's my summarized [31:53] take on that. [31:56] >> Yeah, I think this is the other my other [31:58] favorite irony of this whole thing is um [32:01] we we because and I agree with the [32:03] statement about building an agent and [32:04] treating it like a teammate, but at the [32:06] same time like we all recognize that [32:08] these new teammates are very disposable [32:10] and so they're not really a teammate [32:11] because you're going to get rid of them [32:12] at some point. Uh my team and I were [32:14] debating the other day uh that and and [32:16] I'm I'm a very firm believer that [32:19] outside of that utility agent layer that [32:21] all of the agents they were building [32:23] right now will never will never see [32:25] their first birthday. And so you have to [32:27] treat the evolution of what is [32:29] available, what you can construct them [32:30] with, how you're building them, and the [32:32] process evolution around them is going [32:34] to change in a likelihood so [32:35] significantly that you're going to have [32:37] to continue to modify these things. [32:39] again, unless you've gone down to that [32:41] completely like like non non um [32:46] non-breakable layer at the utility level [32:49] like these things up here. If if you're [32:50] assuming that you can build an agent [32:52] that won't drive the need for process [32:54] change recognition, you're crazy. And so [32:56] as the process around the agents that [32:57] you're deploying changes, you're also [32:59] going to have to modify and evolve the [33:00] way the agents are acting in the [33:02] ecosystem that you put them in. And so I [33:04] don't think the agents that you build [33:06] today are going to look like the agents [33:07] you will have in a year if you're even [33:09] operating in the same space. [33:11] I just add on top of that I'd say like [33:13] when you're building agents especially [33:15] like you think of like what is truly [33:16] going to differentiate you in the market [33:18] right it's probably your data it's [33:19] probably the agent to clear on top right [33:21] so when you think of building agents [33:22] don't build with a one to two year [33:24] vision you'll probably get it right in [33:26] the short term but absolutely fade away [33:28] in the long term you should have a 5 10 [33:29] year vision you should think of hey am I [33:31] if I'm building a prompt and response [33:33] agent now I'm prompting and it's doing [33:34] crowd actions right is that the future [33:36] right or should I move to more like [33:38] schedule based trigger based agents [33:40] right just understand your trigger, get [33:42] jobs done automatically. Should I move [33:43] towards more autonomous agents and have [33:46] a road map like as a product leader of [33:48] any organization, you should have uh [33:49] this vision and a road map and then you [33:51] slow you're slowly taking small steps [33:53] towards that but you're constantly [33:55] thinking like how am I differentiating [33:58] this crowded market like just don't [33:59] build agents that your competition is [34:01] building that everybody else is building [34:02] think differently. All all great points, [34:05] right? So, if I were to bring this to a [34:08] close, I think for me it comes down to [34:11] ultimately the agents that you are [34:13] building. Are you building it for the [34:16] way you're running your business today [34:18] or are you thinking about how the [34:20] business is going to evolve for tomorrow [34:22] and you're building your agents for [34:24] that? If they are if your agent that [34:26] you're building is that disposable, [34:28] should you be even building it now? [34:29] Right? That's the question that I have. [34:32] If you are investing on nearterm items [34:38] that you know for a fact are not going [34:41] to be there tomorrow. Don't waste your [34:43] time. Instead, think of building out [34:48] agents that are going to evolve. And you [34:51] know, adaptability is key. You know, [34:53] that's that's what Morad pointed out, [34:54] right? So, it's it's key to how you are [34:58] building your agents for the future. You [35:02] do want to build out that teammate whose [35:06] loss you will feel, right? And if that [35:08] person is that disposable like again [35:10] like what are you doing, right? So think [35:12] about that. Think strategically about [35:14] what you're prioritizing, what agent [35:17] you're building. [35:19] These agents could become the business [35:23] that you or the company that you aspire [35:25] to be part of. That's what we need to [35:28] have in mind, right? that future in mind [35:30] as we are prioritizing and going after [35:34] agents. [35:36] >> Yeah, those are great points as well. [35:38] And one like I guess another dimension I [35:41] think about this also in is like build [35:43] versus buy. If you're building [35:45] internally and it requires a lot of [35:47] customization and it's going to go [35:48] obsolete in a year, maybe that's not [35:50] worth it. But there is probably a vendor [35:52] out there for them. It's worth it to [35:54] build these agents that they can sell to [35:56] thousand of their customers. So for them [35:58] it is worth to build it and sell and [36:00] evolve it. Um while it might not be [36:02] worth for it for the enterprise. So [36:04] that's like another dimension think [36:06] about. Um would like to open it up to [36:08] any questions in the audience. [36:11] Do we have a mic? Eve [36:15] that gentleman in the blue. [36:18] >> Come on. [36:22] >> Thank you. Great insights. Two [36:24] questions. So, so if we have a use case [36:28] where we cannot add a human in the loop, [36:32] do you suggest building an agent? I I [36:34] I'm going to give you a specific example [36:36] too, right? Let's say we talked about [36:38] customer support where a human agent is [36:40] supporting customer questions in real [36:42] time. If you want to mimic the similar [36:45] behavior with the agent, we cannot add [36:48] human in the loop. There is no human, [36:49] right? It's all happening in the real [36:51] time. Would you advise building an [36:53] agent? That's first question. And if you [36:55] if you do what are the best practices? [36:57] Second question is uh we talked about [36:59] rag uh to control the data access. [37:02] However, if there is a mutation involved [37:04] within EA agent, how do you control it? [37:07] What are what are your probably [37:09] recommendations if you will? Thank you. [37:12] >> I can I can start with that question. So [37:14] the I think the first question was uh [37:15] for customer support for example if [37:17] there's real-time interactions happening [37:18] do you build an agent? It's again [37:20] riskreward framework for most customer [37:22] support. I would assume yes because you [37:24] do want the agent there's a concept of [37:26] deflection rates in customer support [37:28] right so you want that agent to be able [37:29] to deflect as many tickets on its own as [37:31] possible and only hand over the ones [37:33] that can deflect to humans and and those [37:35] take probably longer SLAs's but again [37:38] I'd also give advise it with a pinch of [37:41] salt because if you're dealing with [37:42] healthcare for example and the patient [37:43] is saying hey I'm suffering from this [37:44] disease what do I need to do you [37:46] probably may not want to identify that [37:48] at this point in time so so I think [37:50] you'll have to take into account the [37:51] business specific nuances and then [37:53] decide for yourself like do you want to [37:54] agentify it or not? [37:55] >> Guard rails, right? Like you're not [37:58] going to build you're not going to build [37:59] an agent that operates autonomously and [38:01] tell it you have free reign, right? And [38:04] so give it guardrails. Like in the [38:06] customer support uh example, you can say [38:10] you can resolve these issues any way you [38:11] want, but you're never allowed to give a [38:12] credit to a customer over a certain [38:14] amount. You're never allowed to do X [38:16] over a certain threshold. You have to [38:17] just you have to give it guardrails. And [38:19] that's like you the less human in the [38:21] loop you have the more guardrails you [38:23] put in place or you go back to what uh [38:27] Deepo said earlier or one of you said [38:28] earlier like can you quantify the risk [38:31] of loss do you know what it's going to [38:33] do and if you're comfortable with that [38:34] you know let it rip [38:38] >> I think the second question was how do [38:39] agents evolve over time right how do you [38:41] like as think is that did I get that [38:43] right [38:45] >> second question was around uh how do you [38:47] control the access data access. Uh we [38:50] talked about rag a little bit but that's [38:51] only to retrieve the data but if there [38:55] is a mutation involved where a agent [38:58] needs to update something right change [39:00] the status of something uh what [39:02] additional guardrails we may need to [39:05] have [39:05] >> at least what what we're seeing is like [39:07] agents access data in one of two ways [39:09] one is via like if you have APIs right [39:11] APIs already are guardrail in certain [39:14] data aspects APIs evolve over time right [39:16] and you want the agent to also pick the [39:18] right APIs as they evolve, right? [39:20] Understand all the change etc. But if if [39:22] not dealing with APIs, if you have an [39:24] MCP over your database for example or [39:26] you're retrieving data directly from the [39:28] database, I think you'll as the data [39:30] evolves like you'll have to build in [39:31] mechanisms that understand what the [39:33] delta is change and then decide maybe [39:36] you'll have to have a guard rail in [39:37] place where you see like this new data [39:39] that's coming in should it be fed to the [39:40] agent or not like there has to be a [39:42] checker a maker checker there uh that [39:44] decides that. [39:47] >> Great. We do have five minutes left for [39:50] questions. I'm go to this gentleman here [39:52] in the center. [39:58] >> Thank you. Um I'll try to make it quick [40:00] and focus. Can you get examples of the [40:03] top three examples of agents you're [40:06] deploying like maybe Eric um service [40:09] calls or um PI autogeneration of use [40:13] cases? [40:15] Yeah, I think we're um some tangible [40:17] examples of where we're focused right [40:19] now. So, in the call center space, yes. [40:21] Um we're putting a lot of unassisted [40:23] agent support to handle specific [40:24] intents, right? Those, you know, [40:26] customer calls in, we identify what [40:28] intent they're calling in about either [40:29] through, you know, some form of [40:31] deterministic mechanism or through a [40:32] generative lookup like a generative [40:34] inference process. Uh and then each of [40:36] those intents that we go satisfy before [40:38] we hand off to a human agent if [40:40] necessary, all have guardrails built in [40:42] relative to how we want to handle those [40:43] intents. So there's some uh autonomy [40:46] within inside the guardrails. Uh we're [40:48] also looking at um beyond uh AI assisted [40:51] coding which I think everybody's looking [40:53] at at this point. If you have software [40:54] engineers, we're swimming left and right [40:55] and putting some custom agents inside [40:57] the process to help with things like [40:59] epic and user story generation. Again, [41:01] there's a lot of customization in that [41:02] because you have to get very specific [41:04] about the way you want your user stories [41:06] written against applications that you've [41:08] built or that you've got inside your [41:09] infrastructure. And then I'll give you a [41:11] third example. We're also getting very [41:13] specific about how we are using agents [41:16] to help with personalized campaign [41:17] messaging specifically in like a B2B [41:19] scenario. Uh and those those tend to get [41:21] customized in that super agent layer [41:23] relative to the products that you're [41:24] focused on, the industry you're focused [41:26] on, the segment, you know, how you want [41:28] to tailor the message, etc. And so a lot [41:30] of the the super agent activity that we [41:32] have in the middle there is ultimately [41:34] where our focus is to get customization [41:36] done. And a lot of the utility agents [41:37] are ultimately what's doing things like [41:39] you know go get me this data go you know [41:41] structure in this kind of way go put you [41:43] know put something in place here go look [41:45] something up etc. [41:47] I think uh two examples that I can give [41:50] in that is for example in retail there's [41:52] a lot more autonomy that you can have uh [41:54] understanding customer behavior and [41:56] things of that nature you can have a lot [41:57] of autonomous agents work whereas versus [41:59] in the legal space if you have agents [42:01] over there like we build for retail and [42:03] legal both in legal there has to be a [42:05] lot less autonomy uh you have to have [42:07] specific guardrails up there uh make [42:09] sure that the data that's being provided [42:11] to the agent there's not being any [42:12] security compromises being done there so [42:14] when we build for that sector we have to [42:16] involve human intervention in between [42:18] whereas when we build agents for retail [42:20] especially for B2C of course um there's [42:22] a lot more autonomy we can uh give [42:24] access to [42:26] >> we have I think two minutes left [42:28] actually um I would love to hear from [42:30] you to what's like one or two use cases [42:33] you have deployed either for yourself or [42:34] for your customers so um on our end we [42:39] have deployed agents that help with [42:41] supply chain management specifically [42:43] inventory management which is a big [42:45] issue [42:47] Um, we have deployed agents that help [42:49] with intelligent bidding. We have [42:52] deployed agents that of course help with [42:55] more of the mundane, you know, document [42:58] comparison and those type of activities. [43:01] Um, we have drilling optimization [43:03] agents. So just just a few to touch on a [43:06] few. [43:07] >> Uh, we we are an agent builder platform. [43:09] So we've seen a variety of use cases. [43:10] Our clients deploy agents. Um we've seen [43:13] uh dev tool companies for example [43:15] creates a create AWS clusters and manage [43:17] them using our agents. We've seen [43:19] procurement agents. We have seen billing [43:20] agents on the finance CFO side of [43:22] things. We've seen AIBDR agents. So we [43:25] see a lot of external agent use cases [43:27] from our from customers. And it's [43:28] interesting how they're thinking about [43:30] uh all all kinds of agents for their [43:31] apps. [43:33] >> So many different use cases. So I guess [43:36] people can come up to you afterwards and [43:37] ask any other questions because we are [43:39] at time unfortunately. Thank you all for [43:42] joining. [43:42] >> Thank you. A round of applause please [43:44] for our panel.