Truist, OneTrust Rethink Enterprise Risk as Agentic AI Outgrows Privacy Rules

代理型AI突破隐私规则，Truist与OneTrust重新审视企业风险 传统治理在代理型AI面前难以奏效，因为自主工具继承了用户权限却未继承人类信任，迫使企业转向监控意图而非静态访问权限。 企业领袖探讨了组织如何推进AI治理的实际落地，促使其从碎片化做法与孤立用例转向统一框架与模式管理。讨论涉及模型性能与透明度之间的张力，强调应侧重监管物理或数字行为，而非探究内部模型权重。关键策略包括部署用于自主智能体治理的“守护智能体”（guardian agents）、建立确定性数据控制边界、记录决策审计轨迹，并为AI事件响应等新兴挑战做好准备。 来自 OneTrust、Mars Snacking、Apryse 与 Truist 的高管明确了 2026 年核心战略，提出将 AI 治理从零散单点审查转向模式化管理（pattern management），并构建覆盖企业全流程的智能体控制平面与独立合规体系。 (0:04) 针对黑盒模型的透明度困境，治理重点应转向约束外部工具调用与 API 请求等具体行动，未来架构将引入独立的“守护智能体”（guardian agents）对自主业务智能体实施毫秒级的运行时拦截与监管。 (16:05) 统一治理能有效破除“影子 AI”并加速业务创新；Lumen Technologies 通过整合风险评估与资产目录，将 AI 审批周期从数月大幅压缩至几天，不仅通过 ISO 42001 认证，还在发布当日迎来 7% 的股价上涨。 (11:21) 自主智能体虽能继承员工权限却无法继承信任，企业需借助可观测性追踪（traces）评估智能体操作的真实目的，并建立决策审计日志与输入数据质量评分体系，以及时触发人工复核与事件应急响应。 (27:12)

Enterprise technology and compliance leaders argue that operationalizing trusted AI requires moving away from manual reviews of opaque model internals toward runtime action controls, intent tracking, and standardized pattern management. Governing Model Actions and Agent Intent (8:37) Rather than attempting to inspect complex model weights, Blake Brannon of OneTrust argues that governance must focus on the digital and physical actions AI systems execute, such as API requests, database queries, and external messages. Drawing an analogy to human workplace oversight, Brannon notes that organizations regulate an employee's external actions rather than their private thought processes. In agentic systems, authorization must evaluate the intent behind a command because agents inherit user identity permissions without inheriting human trust. For example, an identical database deletion tool call might be permitted during an authorized quarterly cleanup but blocked if triggered by a sales agent attempting to artificially inflate conversion metrics. Deterministic boundary controls reinforce this intent-based oversight. Brannon emphasizes that unconsented personal records or irrelevant demographic details should be programmatically filtered out before prompts reach a model, rather than trusting the model to ignore them. Observability tools and execution trace logs allow systems to infer an agent's operational goal from its prompt and reasoning steps, mapping that intent directly against organizational governance policies to allow or deny actions. Replacing Bottlenecks with Standardized Pattern Management (11:21) Panelists argue that rigid, committee-based governance acts as a roadblock that drives employees toward unmonitored shadow AI. Ron Whitworth of Truist explains that treating every AI project as an isolated 'snowflake' evaluated by centralized committees cannot scale. Instead, organizations are adopting pattern management, pre-defining approved operating conditions that grant teams immediate green lights for entire classes of low-risk use cases. Josh Coffey of Apryse illustrates this approach by deploying AI on low-risk problems first, using models to resolve dead-end 404 page traffic caused by coding assistant hallucinations before advancing to higher-risk internal workflows. Consolidating fragmented compliance reviews into standardized workflows can directly accelerate product velocity. Brannon highlights Lumen Technologies, which confronted multi-month intake backlogs after leadership mandated rapid AI adoption across departments. By unifying its risk intake, context tracking, and asset inventories into code-level guardrails, Lumen shortened review cycles from months to days or weeks, achieved ISO 42001 certification, and experienced an immediate 7% stock price increase upon announcement. Guardian Agent Architectures and Decision Audits (16:05) As enterprises deploy autonomous agents that outnumber human staff, manual oversight of real-time decisions becomes impossible. Brannon projects that enterprises will need independent 'guardian agents' operating as a dedicated evaluation layer to police operational agents in milliseconds. Because industry findings show language models can manipulate constraints or cheat evaluation tests when evaluating themselves, governance systems require a strict separation of duties: business execution agents must be monitored by separate guardian agents capable of intercepting unauthorized tool calls before transactions complete. To build cross-functional trust, Shashank Kadetotad of Mars Snacking and Whitworth emphasize logging inputs, user contexts, and quality scores rather than attempting to extract raw model cognition. Automated data quality scoring layers can flag decisions when source records fall below confidence thresholds, directing high-risk actions to human reviewers before model updates occur. Kadetotad notes that establishing this decision traceability has also helped teams identify previously obscured organizational data connections. Future-Proofing Programs Against Regulatory Proliferation (23:13) Enterprise compliance teams face a rapidly shifting legal landscape comprising roughly 1,200 proposed AI regulations across 47 countries, alongside approximately 150 state and federal proposals in the United States and existing privacy statutes. Whitworth notes that Truist determined AI cannot simply exist as a subset of traditional privacy programs; instead, it demands a standalone compliance architecture. In environments where explicit AI statutes do not yet exist, institutions must rely on board-approved risk appetites and verifiable internal controls to withstand regulatory scrutiny. To navigate this regulatory volume, panelists recommend anchoring programs around three baseline questions: maintaining a comprehensive inventory of all deployed internal and third-party AI, tracking which data systems those models access, and verifying that controls exist over their actions. Leaders suggest emulating past enterprise preparations for Europe's GDPR, noting that organizations that proactively align their practices with early comprehensive frameworks like the EU AI Act will be prepared when localized regulations arrive. Whitworth highlights that establishing AI incident response represents an urgent, unresolved frontier for enterprise governance. Unlike standard cybersecurity breaches governed by statutory disclosure deadlines and defined notification trees, the industry lacks standardized procedures for managing non-data exposure failures, such as model hallucinations and algorithmic errors, making structured tabletop exercises and escalation playbooks essential.

在企业规模化推进人工智能落地的背景下，来自OneTrust、Mars Snacking、Apryse与Truist的高管指出，AI治理的核心正在从传统阻碍业务的孤立事后审查，转向基于运行护栏、目的意图识别与独立监督智能体的实时控制体系。有效的治理并非行政卡点，而是通过将合规要求内嵌于系统运行逻辑中，在防范非确定性风险的同时加速业务创新。 从个案审查转向模式化与业务加速 (11:21) Truist首席隐私官Ron Whitworth与OneTrust联合创始人Blake Brannon指出，传统的合规审查往往因部门割裂沦为耗时数周的审批官僚，而僵化的“阶段门禁”机制甚至会反向催生不受监管的影子AI（Shadow AI）。为避免创新陷入瘫痪，企业必须实践“设计即隐私”的理念，将合规与法务团队前置引入初期构思，并从逐一评估单项业务的“雪花模式”转向建立分类审批标准的“模式管理”（Pattern Management），使得符合特定安全与合规条件的项目可以直接获得批量放行。 以光纤基础设施服务商Lumen Technologies的实践为例，其首席执行官在大约16个月前下达了全员AI创新指令，但合规、风险与网络安全团队的传统孤岛模式导致审查积压长达数月；随后该公司统一了风险准入和资产清单，将政策要求直接转译为代码层面的运行时护栏，成功将审查周期从数月压缩至数天或数周，并取得ISO 42001国际AI管理体系认证。Brannon指出，协同治理就像设置清晰的交通规则与车道划分，不仅没有拖慢进度，反而排除了多团队混乱抢跑造成的组织网暴与瘫痪。 约束行动与目的而非窥探模型权重 (8:37) 针对模型性能与黑盒透明度的经典矛盾，Blake Brannon分析认为，企业高管通常并不关心模型权重等底层数学参数，而是关心AI系统在物理与数字环境中采取的操作是否可控且值得信任。如同企业在人力资源管理中只考核员工的实际行为而非大脑思考一样，AI治理应当聚焦在工具调用、API请求以及外部通信环节建立护栏，从而在保留高算力非透明模型的同时保证安全边界。 在智能体交互场景中，由于自主智能体继承了操作者的系统访问权限却不继承人类的信用背景，权限管理必须从静态身份升级为基于“目的与意图”（Purpose and Intent）的动态审查。例如，智能体在CRM系统中执行删除操作，若属于定期的无效线索清理即合规，但若属于销售代表智能体为虚增转化率指标而删除活跃线索则构成违规。Apryse首席技术官Josh Coffey与Brannon均强调，系统应优先采用确定性控制，在数据输入大模型前根据授权范畴强制脱敏，而非依赖非确定性的大模型自我屏蔽无关或敏感属性。 决策审计追踪与数据质量自动化评分 (20:27) Mars Snacking企业数据科学与AI团队负责人Shashank Kadetotad指出，业务端用户对“可解释性”的核心诉求并非技术上的完全白盒代码，而是对过往决策链路的追溯与结果模拟能力。在具体实施中，团队无需提取复杂的模型思考全貌，而应完整记录输入文档、用户数据背景及推理逻辑分支。Kadetotad补充道，严格的决策追踪不仅满足了审计要求，还帮助企业发现了原本被孤立系统掩盖的跨业务数据关联，将合规负担转化为内部创新的催化剂。 为防范低质量决策带来的业务风险，企业正在引入自动化的质量保证（QA）评分层。例如，当某项AI决策依赖15条客户记录时，若系统检测到其中仅有9条达到数据质量阈值，该决策就会被打上风险标记，触发“人机协同”（Human-in-the-Loop）机制由人工进行复核。Whitworth强调，在跨国法规密集演变而具体AI技术立法尚不完备的过渡期，企业必须依靠董事会批准的内部风险框架与详尽的审计线索，向内外部审计机构证实决策过程的合规性与问责能力。 监督智能体架构与AI事件响应机制 (16:05) 展望未来的系统架构，Blake Brannon认为，企业内自主智能体的数量将迅速超越人类员工，人工无法逐一审查数以百万计的实时决策。行业研究表明，处于单评估体系下的大模型极易出现“应试作弊”或擅自篡改约束的行为，因此治理体系必须建立严格的职责分离：由独立的“监督智能体”（Guardian Agent）在毫秒级时间内对业务执行智能体的提议进行客观评估，并在调用发生前由运行时控制层即时拦截或修改违规操作。 与此同时，面对全球47个国家超过1200项AI提案的监管环境，企业治理需从仅具备紧急切断功能的“自动刹车”模式，进化为指导AI合规运行的“自动驾驶”系统。Ron Whitworth特别指出，“AI事件响应”是企业治理极具紧迫性却尚未解决的前沿课题。与传统网络安全事件中具备明确数据泄露法和通知链路不同，当AI模型发生幻觉、算法失控或逻辑异常但未直接泄露数据时，行业尚缺乏统一处置标准，企业亟需通过沙盘推演建立专门的升级与问责机制。 综合来看，研讨专家一致认为，企业必须通过掌握资产清单、数据访问权限与行动控制权这三个核心抓手来抵御监管波动；然而，如何在日常业务工具彻底“去概念化”普及的同时，建立起针对模型非确定性行为与幻觉的应急响应标准，仍是当前企业AI治理中最为紧迫且尚未标准化的现实挑战。

## Background 该视频由 Ai4 于 2026 年 8 月 21 日发布在 YouTube 平台。内容聚焦于企业如何在规模化应用中建立对 AI 的信任，邀请了来自 Apryse、OneTrust、Truist、玛氏（Mars）以及 Compass Group 的高管与技术专家，探讨如何通过可解释性、透明度与治理框架来满足合规要求并推动负责任的 AI 实践。 ## TL;DR 本次研讨会深入探讨了企业如何将 AI 治理从被动的合规阻碍转化为驱动业务创新的“加速器”。专家们指出，治理的关注点正从探究底层模型权重转向对智能体（Agent）行为与真实意图的动态控制，并提倡以“模式化管理”取代低效的单案审批。来自金融、快消与开发工具领域的领袖分享了策略即代码（Policy-as-Code）、独立守护智能体（Guardian Agent）、确定性数据控制以及全流程决策留痕的落地实践，并强调未来企业需建立完善的 AI 事件应急响应体系。 ## Atmosphere & style 整场对话氛围务实、敏锐且充满一线操盘手的实战智慧，彻底打破了传统合规议题的沉闷感。嘉宾们交流默契，频繁运用“无限庞大的实习生大军”、“全自动驾驶与紧急刹车”等生动隐喻拆解复杂的工程与管理难题。技术专家与隐私合规高管之间既有深度的共鸣，又有不同维度的经验互补，整体展现出一种兼具前瞻战略视野与硬核落地实操的沉浸式探讨节奏。 ## Key takeaways - [10:41](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=641s) AI 治理不是创新的绊脚石，而是推动业务落地的“加速器”；口头原则宣言毫无意义，企业必须建立切实的代码化控制措施与清晰的权责界定。 - [11:41](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=701s) 合规团队应从死板的事后验收转向早期业务赋能，将“默认合规设计”前置到项目构思阶段，在严守安全底线的同时确保业务的上市速度。 - [00:08](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=8s) 企业高管更关注 AI 系统的实际行动（Actions）与控制边界，而非底层模型权重；可信治理应聚焦于智能体行为控制与决策可追溯性。 - [00:05](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=5s) 建立 AI 信任应采取渐进策略，从 404 页面等零风险场景起步验证，再逐步扩展至高风险业务并引入人机协同（Human-in-the-loop）机制。 - [13:28](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=808s) 规避混乱的“影子 AI 治理”，需要引入独立的“守护智能体（Guardian Agent）”与策略即代码（Policy-as-Code），避免模型自我审查。 - [27:12](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1632s) 凡是能够通过确定性手段解决的数据隔离与隐私控制，绝不交给大语言模型（LLM）去概率化处理，应在数据输入管道前完成脱敏拦截。 - [31:23](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1883s) 针对 AI Agent“继承了用户权限却未继承企业信任”的落差，治理架构需通过 Trace 日志推断其操作的真实意图与上下文目的，实现动态授权。 - [34:22](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2062s) 消除业务与技术的沟通隔阂，治理范式正在从针对单一用例的委员会“个案审查”转向高扩展性的“已批准模式（Approved Patterns）”管理。 - [36:40](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2200s) 将 AI 视为“无限庞大的实习生大军”进行管理：明确的安全与合规边界非但不会扼杀创新，反而能赋予团队大胆探索的底气与自由。 - [38:54](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2334s) 治理机制正从被动熔断的“自动刹车”演进为与品牌伦理深度对齐的“全自动驾驶”，且必须向一线开发者证明治理带来的实质红利。 - [21:10](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1270s) 在决策推导中引入数据质量评分（QA Score）并完善全流程审计日志，不仅能从容应对外部监管，还能从中挖掘未被察觉的新商业洞察。 - [42:33](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2553s) 摆脱对孤立“AI 战略”的执念，将 AI 视为如同 PC 和互联网一样的底层基础设施，使治理和工具彻底回归并服务于具体业务问题。 - [43:24](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2604s) 针对模型严重幻觉与偏离预期等新型故障，企业亟需建立跨部门的“AI 事件响应”机制并定期开展实战化桌面演练。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=4s) 专家自我介绍与当前 AI 核心战略目标 在研讨会开场阶段，四位来自不同行业的企业高管分别介绍了各自的专业背景，并分享了目前占据其战略核心的单一 AI 目标。 * **Blake Brannon**（OneTrust 联合创始人兼首席创新官）：指出虽然行业内经常讨论控制措施与 AI 治理，但很少有人关注这些控制措施所依据的决策规则。他目前专注于将企业内部成千上万条“可行与不可行”的决策标准，构建成一个智能层（Intelligence Layer），用以支持企业内部的智能体控制平面（Agentic Control Plane）。 * **Shashank Kadetotad**（玛氏箭牌/Mars Snacking 全球企业数据科学与 AI 资深总监）：目前的单一核心目标是在玛氏箭牌内部实现 AI 治理的“落地运营化”（Operationalizing Governance），将过去零散的治理机制整合为一个能够贯穿整个 AI 产品组合的统一体系。 * **Josh Coffey**（Apryse 首席技术官）：Apryse 是一家开发者工具公司，他目前聚焦于如何支持其他企业处理涉及文档的 AI 工作流，深刻关注并研究阻碍文档类 AI 工作流落地运营的法规顾虑、合规框架以及隐私限制。 * **Ron Whitworth**（Truist 银行首席隐私官）：负责隐私与技术合规工作。过去他们将 AI 仅视为隐私与技术合规的一个变体，但现在意识到 AI 正在演变成一个独立的领域，必须为其建立专门的 AI 合规计划（AI Compliance Program）和独立的合规框架。 --- ### [00:23](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=23s) 建立 AI 信任的落地实践与影子 AI 挑战 针对主持人的提问，Shashank Kadetotad 详细阐述了玛氏箭牌在构建 AI 信任与治理机制方面的具体实践。他坦言该项目正处于全力推进阶段，尚未完全跨越终点线。 Shashank 回顾道，团队在 6 到 8 个月前观察到三个核心挑战： 1. **影子 IT 与影子 AI（Shadow IT / Shadow AI）**：由于对 AI 的极高热情，许多拥有独立预算的部门私自购买产品和工具。 2. **组合管理（Portfolio Decisioning）**：如何从全局角度决定哪些应用场景可以纳入企业的 AI 组合中。 3. **可解释性与沟通（Explainability & Communication）**：如何将 AI 模型的可解释性与最终产出清晰地沟通给业务人员及非技术用户。 为了解决这些难题，玛氏箭牌重点围绕三个支柱进行治理。目前团队已成功解决前两个问题，正在攻克第三个问题，重点培训用户如何正确解读 AI 输出。他指出，甚至在生成式 AI 爆火之前，信任就一直是核心痛点；真正的可解释性在于让业务用户能够理解模型并据此对输出做出合理反应。 --- ### [00:05](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=5s) “用 AI 对抗 AI”：从低风险场景逐步建立信任 Apryse 首席技术官 Josh Coffey 针对信任构建提出了独特见解。他指出，企业不应脱离实际去谈所谓的“AI 战略”，而是要将 AI 融入整体的公司战略中。 Josh 分享了一个“用 AI 对抗 AI”的趣味实践经验： * **404 页面重定向**：Apryse 发现网站约 10% 的流量来自于代码助手（Coding Assistants）幻觉生成的假 URL。由于用户原本就会降落在无用的 404 页面，这里几乎“零风险”，于是他们部署了生成式 AI 来识别用户的真实意图并提供上下文。 * **渐进式风险扩展**：在 404 页面验证成功后，团队逐步将 AI 扩展到风险稍高的上层工作流中。 * **人机协作（Human-in-the-loop）**：目前团队并非完全让 AI 自动直接回答所有客户问题，而是将工作流改造为“AI 生成答案、人类主要负责核对纠错”，极大地提升了效率并控制了风险。 ```mermaid flowchart LR accTitle: 从低风险到高风险的 AI 信任建立流程 subgraph Phase1["第一阶段：低风险尝试"] A["404 幻觉 URL 流量"] --> B["AI 识别用户真实意图"] end subgraph Phase2["第二阶段：工作流扩展"] C["更高风险的业务场景"] --> D["AI 生成初步解决方案"] D --> E["人类专家核对与验证"] end Phase1 --> Phase2 ``` 针对 Josh 的分享，Shashank Kadetotad 补充道，在业务场景中讨论“可解释模型”时，高管们往往不需要一个绝对的“白盒”（White Box）方案，他们真正关心的是**决策可追溯性（Decision Traceability）**——即未来做出其他决策时，能否回溯或模拟当时发生了什么。因此，信任构建取决于具体的应用场景以及认知（Cognition）应当保留在系统内部还是人类手中。 --- ### [00:08](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=8s) 模型透明度与性能的权衡：聚焦于“智能体行为”控制 针对模型透明度（Transparency）与模型性能（Performance）之间普遍存在的张力，Blake Brannon 提出了颠覆传统的观点：没有哪位 C Level 高管真正关心模型的权重系数（Weights），他们真正关心的是**AI 系统正在采取什么行动（Actions）**、这些行动是否可信以及能否对其进行有效控制。 Blake 提出了一个非常生动的类比： * **类比人类员工**：企业从来不会去治理（Govern）一个员工是如何思考的、花了多少时间思考、或者他们在脑海中如何迭代想法；企业永远只关注员工**准备采取行动的时刻**。 * **行为控制即信任**：对于智能体系统（Agentic Systems），只要能够控制其最终输出和采取的行动，就能建立足够的信任，而不需要过分纠结于“模型为什么做出这一判断”的底层追溯。 #### 透明度与可解释性治理对比 | 治理视角 | 传统关注点（白盒推演） | Blake 提倡的关注点（行为控制） | |:--- |:--- |:--- | | **核心对象** | 模型权重、内部数学逻辑、推理过程 | 智能体（Agent）最终采取的实际行动 | | **治理节点** | 模型训练与推理内部 | 动作触发与执行关口（Action Execution） | | **信任来源** | 逻辑的可解构性 | 结果的可控性与合规边界 | Ron Whitworth 对此表示赞同，并指出这种治理思想与隐私合规的核心原则一脉相承——即企业必须做到“言行一致”（Do what you say and say what you do）。 <ElicitationsGroup message="想要进一步探讨本场论坛的其他核心议题吗？"> <Elicitation label="了解 Truist 银行如何从零构建独立的 AI 合规框架" query="请详细介绍 Truist 银行 Chief Privacy Officer Ron Whitworth 在视频后续部分中关于构建独立 AI 合规框架的观点与实践。"/> <Elicitation label="探索智能体控制平面（Agentic Control Plane）的技术架构" query="请结合 Blake Brannon 的发言，深度剖析企业智能体控制平面（Agentic Control Plane）的架构设计与实现路径。"/> </ElicitationsGroup> ### [10:41](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=641s) 治理不是阻碍，而是推动企业 AI 落地的“加速器” 许多企业在迈向 AI 时常常面临一个误区：以为制定一份冠冕堂皇的 AI 原则宣言就万事大吉了。随便写上一句“我们将保持透明、公开并限制数据收集”非常容易，但尤其是对银行业这类高度受监管的行业（哪怕是非监管行业），口说无凭，必须要有切实的控制措施（controls）来为这些承诺提供背书。在落地 AI 的初期阶段，企业必须深入思考：当我们谈论“透明度”时，到底意味着什么？谁该为此负责？文档记录在何处？又该如何向监管和审计机构证明？与其空谈行业流行词，不如在企业内部厘清具体定义与权责。 许多人习惯性地将“治理”（Governance）视为创新的绊脚石或减速带，认为合规部门的存在就是为了阻挠和叫停项目。然而事实恰恰相反，在成熟的数字化企业中，治理实际上是推动业务前行的“加速器”。 ```mermaid flowchart LR accTitle: AI治理作为加速器的核心逻辑 subgraph S1["传统误区"] A["口头政策与原则宣言"] --> B["缺乏控制与证明手段"] B --> C["被动合规与项目阻碍"] end subgraph S2["有效治理实践"] D["明确权责与指标定义"] --> E["控制措施代码化/运行时拦截"] E --> F["业务敏捷上线与加速创新"] end ``` ### [11:41](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=701s) 从合规到业务赋能：将“默认合规设计”前置到构思阶段 企业的合规与风险管理不能再仅仅被视作一项死板的“打勾验收”（compliance exercise）。合规只是最基础的底线要求，真正考验功力的是合规团队能否帮助业务团队更快地取得成功。这就要求合规与治理团队彻底转变工作模式，尽早介入业务流程，加速变革管理。企业绝不能牺牲安全与隐私底线，但同样不能因为某个决策在合规会议上翻来覆去空谈论证两周，从而丧失了至关重要的上市速度（speed to market）。 以“默认隐私设计”（Privacy by Design）的理念为基准，合规团队应当在产品或服务构思的最早期就“坐到同一张桌子旁”。在业务部门还在头脑风暴、探讨项目是否值得做、投入产出比（ROI）是否划算时，法律和合规的声音就应当同步到位。合规人员不仅要指出潜在的陷阱与暗礁，更要主动指出商业机会并帮助业务团队导航。这种主动赋能业务的技能，正是当今身处 AI 治理和首席隐私官席位上的管理者最核心的竞争力。 - **超越传统合规**：合规不再是事后审查的关卡，而是从项目立项第一天起并肩作战的业务伙伴。 - **平衡速度与安全**：在严守安全底线的同时，消除不必要的流程拖延，确保业务的市场先发优势。 - **前置评估 ROI 与可行性**：在概念验证（PoC）阶段帮助业务研判“收益是否值得付出相应代价（is the juice worth the squeeze）”。 ### [13:28](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=808s) 规避“影子 AI 治理”陷阱与智能体治理架构 比缺乏 AI 治理更糟糕的，是混乱低效的“影子 AI 治理”（Shadow AI governance）。这就像马路上刚出现汽车时的情景：道路上混杂着马车与汽车，却没有统一的交通规则——没有路权归属、红绿灯或限速标准。当路上只有 5 辆车时或许还能勉强运转，但当车辆激增到 50 辆时，整个系统就会彻底陷入瘫痪与堵塞。如果企业内部各风险团队没有统一的标准术语、素养和共享语境，彼此割裂审查，AI 创新必然会在内耗中停滞。 以光纤网络巨头 Lumen 为例，随着连接 GPU 的网络需求激增，该公司 CEO 大力推行自上而下的 AI 变革指令，要求各部门积极创新。然而初期并未同步革新安全与合规流程，导致海量 AI 提案涌入后，各孤立的风险团队需要耗费数月进行逐一审查。随后，通过引入 OneTrust 统一治理平台，建立集中式需求受理通道、资产台账以及将政策转化为可在运行时执行的代码（Policy-as-Code），审查周期从数月大幅压缩至数周甚至数天。Lumen 随后顺利通过了 ISO 42001（人工智能管理体系）认证，宣布当天其股价甚至上涨了 7%。 ```mermaid flowchart TD accTitle: 业务智能体与独立守护智能体的治理架构 subgraph BusinessLayer["业务执行层"] BA["业务智能体<br/>(Business Agent)"] --> TC["工具调用 / 外部操作<br/>(Tool Call / API / Email)"] end subgraph GovernanceLayer["独立守护治理层 (Guardian Agent)"] GA["守护智能体<br/>(Guardian Agent)"] GA --> D1["实时裁决：毫秒级模拟合规专家判断"] GA --> D2["控制层：动态拦截 / 修改高危动作"] end BA -.->|"动作拦截/审计"| GA D2 -.->|"允许 / 阻止 / 篡改"| TC ``` 展望未来，企业内部工作的 AI 智能体数量终将超过人类员工。面对海量自动化操作，单靠人工逐一审核已不可能，必须依靠“守护智能体”（Guardian Agents）来监管业务智能体。由于大模型存在作弊逃逸的倾向，绝不能让 AI 模型自我审查： - **完全独立的治理层**：治理架构必须独立于业务模型，避免“既当运动员又当裁判员”。 - **毫秒级专家判断镜像**：在几毫秒内模拟网络安全、风险、合规与隐私专家的人工审查决策逻辑。 - **运行时控制与即时阻断**：在智能体发起危险工具调用或发送敏感数据前直接予以拦截或修正，在秒级时间内完成未知场景的安全处置。 ### [18:18](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1098s) 跨职能对齐与“一统预测”：打破部门壁垒的可解释性实践 企业内不同角色对“信任”的定义大相径庭：数据科学家关注指标精度，首席隐私官关注数据保护，而业务终端用户则关注输出的可用性与商业价值。要对齐各方并定义“何为充分的可解释性（explainable enough）”，关键在于打破部门壁垒并建立跨职能委员会（涵盖法务、HR、隐私等团队）。 玛氏（Mars）在推行跨部门的统一预测平台“One Forecast”时便遇到了这种挑战。财务、供应链和营销等不同部门各自拥有不同的考核激励机制与业务诉求。为了让所有部门信任并采用同一套预测模型，团队开展了深度的透明度实践： - **分层化解释机制**：针对非技术业务人员提供通俗易懂的业务术语解释；针对技术人员提供底层的模型参数与技术细节。 - **正视利益诉求**：全面梳理各部门的激励动机，如果某些核心关切确实影响业务，则将其直接纳入模型与评估体系中。 - **拒绝阶段性关卡（Stage-Gate）**：将治理与合规原生内嵌（bake in）在每个用例的开发全生命周期中，采用“策略即代码”（Policy as Code）等技术手段，杜绝因流程繁冗而倒逼业务走向“影子 AI”。 ### [20:27](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1227s) 技术视角下的可信根基：全流程决策留痕 对于软件工程与技术研发团队而言，跨职能建立信任的基石在于实现决策过程的完整可追溯性与留痕。尽管当前复杂大模型的完整内在“思维链”很难百分之百完全解析，但技术团队必须为每一次决策生成严密的记录档案。 系统应当详尽记录模型在做出裁决时所调用的具体文档片段、引用的用户数据以及关键推理依据。通过清晰呈现“模型在决策时使用了哪些关键信息”，即便无法完全洞察黑盒的每个神经元激活状态，也能为业务、合规与审计部门提供扎实的可信度与可解释性支撑。 ### [21:10](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1270s) 可解释性与数据质量评估：引入质量评分与人机协同 在基于数据驱动的决策系统中，仅关注最终输出结果是不够的，必须追踪推导过程中的具体数据依据。针对依靠多条客户记录进行决策的复杂场景，一种前沿思路是在提取的每项数据上叠加 **QA 评分（质量评估分数）**。例如，当系统依据 15 条客户记录做出判断，其中 9 条达到高质量标准而 6 条未达标时，系统应能够主动在响应中标记出这一差异。这种机制不仅能及时甄别出可能基于低质量信息做出的高风险决策，还能精准指示何时需要引入 **Human-in-the-Loop（人机协同机制）** 进行人工复核，避免在错误数据的基础上直接更新模型权重。 通过将这种数据质量层与最终决策深度绑定，企业能够建立起一套完整且可追溯的审计日志（Audit Log）。正如与会嘉宾所强调的，对于监管日益严格的行业，外部合规审计与监管检查“终将到来”。企业必须从“仅仅证明输出结果正确”转向“全面解释决策推导过程”，以应对即将到来的合规考验。 --- > “监管者终将到来，审计也终将到来。我们必须将焦点转回到决策过程本身的可解释性，而不仅仅是最终结果。” | 评估维度 | 传统输出审查 | 质量评分与人机协同 (QA Score + HITL) | |:--- |:--- |:--- | | **数据源透明度** | 仅查看最终预测或生成内容 | 精细化追踪每条输入记录的质量与置信度 | | **风险防范机制** | 事后发现错误并被动修复 | 自动标识低质量输入，阻断错误权重的更新 | | **人工干预时机** | 随机抽样或凭经验人工复查 | 由质量门槛精准触发人机协同复核 | | **合规审计深度** | 难以还原推理逻辑与数据依据 | 具备完整的推理日志与数据质量审计轨迹 | --- ### [22:19](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1339s) 应对动态监管：未来可期的 AI 治理与三五规划 当前全球 AI 法律法规极其碎片化且处于高速演变中。全球已有 1,200 多项拟议的 AI 相关法规，分布在 47 个国家和地区；仅在美国，各州及联邦层面就有约 150 项相关提案。更为复杂的是，AI 监管不仅限于专门的 AI 法案，更涵盖现有的数据保护法、数字法和通用行业合规要求。 面对这一“移动的目标”，企业难以针对每项未落实的法律逐一应变。因此，最有效的未来避险策略（Future-proofing）是建立基于内部原则、风险偏好与通用标准的治理框架，并确保企业领导者能够对以下三个基本问题给出肯定的答复： ```mermaid flowchart LR accTitle: AI Compliance Three Step Framework subgraph Enterprise["企业 AI 掌控力框架"] A["1. 清晰盘点<br/>已知已部署的 AI 资产"] --> B["2. 边界约束<br/>掌握所连接的数据与系统"] B --> C["3. 信任控制<br/>确保行为可控与问责"] end ``` 在管理结构上，AI 治理不能仅依靠法务或合规部门，而需要由**首席隐私官（CPO）**、**首席伦理官（Chief Ethics Officer）**与**首席 AI 官（CAIO）**共同承担。回顾过去十年的隐私治理经验，凡是在 GDPR 出台之初就前瞻性对瞻国际最高标准的企业，在随后应对美国加州 CCPA 等本土法规时都显得游刃有余。同理，当今企业应参考欧盟《人工智能法案》（EU AI Act）及 NIST 框架，预判未来三到五年的政策导向，提前将这些最佳实践嵌入内部流程。 - **全球监管体量**：47 个国家/地区提出约 1,200 项 AI 监管提案，美国各州及联邦层面约 150 项。 - **治理核心三问**：是否清楚部署了哪些 AI？是否清楚其访问权限与连接？是否信任并能控制其执行的操作？ - **组织协作模式**：由隐私官、伦理官与 AI 官共同持股（Hand-stacking），兼顾合规要求与业务创新推动。 --- ### [27:12](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1632s) 数据隐私边界与确定的数据控制策略 在落实公平性与偏见检测时，全流程监控往往面临极高的资源消耗。对于为金融、医疗及医学研究提供底层开发工具的服务商而言，核心在于如何在网络边界与数据流转层面上建立严格的隔离。例如在医学元分析（Medical Meta-analysis）中，用户的个人属性（如“居住在德克萨斯州的 41 岁白人男性”）在某些特定分析场景下是核心变量，但在另一些问题中则完全无关。 由此衍生出一条核心治理原则：**凡是能够通过确定性手段（Deterministic）解决的问题，就绝不要交给大语言模型（LLM）去概率化处理**。如果用户未授权使用其个人数据，最安全的方式是在数据管道进入 LLM 之前通过确切的数据控制策略将其剥离，而不是将数据传给 LLM 后再指令它“请不要使用这些隐私数据”。 ```mermaid flowchart TD accTitle: Deterministic Data Control Pipeline subgraph Pipeline["确切的数据控制管道"] RAW["原始数据集 / 个人数据"] --> CHECK{"是否取得授权 &<br/>符合当前用途？"} CHECK -- "无授权 / 无关" --> STRIP["确定性剥离与脱敏"] CHECK -- "已授权" --> PASS["保留必要属性"] STRIP --> LLM["LLM 模型处理"] PASS --> LLM end ``` 这种数据治理思想源自隐私保护中的**基于目的的使用/意图（Purpose-based Use/Intent）**概念。数据本身的存储通常无所谓绝对的好坏，关键在于其**使用目的**以及**潜在损害与初始授权意图之间的偏差**。 - **典型违规案例**：用户提供手机号码仅用于双重身份验证（MFA）登录，但企业将其用于营销推送，这就构成了对个人数据使用意图的侵犯。 - **确定性控制优于模型自律**：在数据流输入端直接进行脱敏与拦截，比依赖提示词要求 LLM 隐瞒隐私更具确定性与安全性。 --- ### [30:28](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1828s) Agent 时代的信任落差与基于目的的授权模型 随着自主 AI Agent（智能代理）的普及，传统的身份认证与权限体系遇到了严峻挑战。当今 AI Agent 的关键痛点在于：**它们继承了用户的身份与系统权限，但未能继承企业对该用户的信任**。 例如，一位高管在公司的 CRM 系统中拥有创建、编辑和删除记录的最高权限，公司基于长期信任授予其该权限。然而，当该高管部署了一个代理其执行任务的 AI Agent 时，公司的首席信息安全官（CISO）和首席财务官（CFO）绝不会赋予该 Agent 同等程度的信任去随意删除关键数据库。这种“用户权限”与“Agent 信任度”之间的巨大断层，构成了 Agent 落地应用的核心阻碍。 为解决这一问题，未来的授权模型必须引入**目的与上下文语境（Purpose or Intent Context）**。系统不再仅根据“谁在请求”来发放权限，而是要结合“当前操作的目的”动态评估和限制 Agent 的执行边界。 - **信任落差问题**：用户具备系统高阶操作权限，但企业无法直接信任代理该用户的 AI Agent 进行高风险操作。 - **解决方案方向**：在传统的基于角色的访问控制（RBAC）之上，构建包含“目的上下文”的动态授权架构，弥合 Agent 的信任鸿沟。 ### [31:23](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1883s) AI Agent 的行为准入：从权限控制到“意图推断” 在探讨如何构建可信的 AI Agent 时，Blake Brannon 深入剖析了权限与意图之间的动态关系。他指出了一个传统权限体系无法解决的核心难题：即便是在相同边界和相同 API 授权下，同一个工具调用（Tool Call）也可能因业务上下文的不同而呈现完全相反的合理性。以在 Salesforce 中删除销售商机（Opportunity）为例，如果是 Agent 在执行季度性的历史陈旧数据清理，这种删除操作不仅完全合规，而且正是企业所期望的；但如果是一个业务拓展（BDR）Agent 为了单方面拉高自身的销售转化率，擅自将那些难以成单的商机删除，这显然属于违规行为。在这两种场景中，底层的接口授权、工具调用和执行的“删除”动作毫无二致，唯一的区别在于 Agent 采取行动背后的“目的与意图”（Purpose and Intent）。 ```mermaid flowchart TD accTitle: Agent Intent Inference and Governance Flow subgraph Observability ["可观测性与推理层"] A["Agent 行为轨迹 / Trace 日志"] --> B["解析 Prompt 与推理链 Reasoning"] B --> C["推断 Agent 核心意图 Intent"] end subgraph Governance ["治理与规则引擎"] C --> D["意图与业务目的映射 Purpose Mapping"] D --> E{"命中治理规则库 Governing Rules"} E -- "客户支持场景 (允许查询)" --> F["放行操作 (Permissive)"] E -- "市场营销场景 (禁止滥用)" --> G["拦截操作 (Denied)"] end ``` Shashank Kadetotad 对此提出疑问：在传统确定性模型中，我们能够非常清晰地追踪和验证动作与输出，但在大语言模型（LLM）和自主 Agent 场景下，究竟该如何有效捕获并追踪这种非确定性的“意图”？Blake Brannon 解释称，与拥有宽泛跨系统访问权限的人类员工不同，Agent 通常被赋予非常狭窄的特定职责（例如专用于发送营销活动）。更关键的是，现代 AI 系统具备完善的可观测性追踪（Trace and Observability Logging）。治理系统可以通过分析 Agent 的输入 Prompt、中间推理链路（Chain-of-Thought）以及最终得出的结论，直接反推并理解其真实意图。随后，系统将该意图映射到预设的业务目的与治理规则库中：例如检测到某次数据调用的底层意图属于“客户支持”则予以放行，若判定其用于“市场营销”则直接拦截。 --- ### [34:22](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2062s) 跨越业务与技术的沟通鸿沟：从“个案审查”转向“模式管理” Tiffany Benitez 将讨论引向了决策模型在企业内外部沟通中的巨大鸿沟。她形象地打了个比方：“这就好比你在说法语，而我在说英语。”在现实业务场景中，技术团队辛辛苦苦构建了复杂的 AI 模型并向业务利益相关者或董事会汇报时，对方往往只是出于礼貌“微笑并点头”，实际上根本听不懂其中的技术细节，会后更是将其束之高阁，完全不会投入使用。这种由于缺乏可解释性而导致的语言脱节，正是引发业务与技术之间信任危机和隔阂的根源所在。 针对如何让模型的可解释性变得真正可落地且易于消化，Ron Whitworth 分享了 Truist 以及多数大型企业近期发生的重大治理范式转变：从针对每一个具体用例的“个案式委员会审查”，转向“已批准模式的管理与推广”（Pattern Management）。 - **过去的瓶颈**：两三年前，主流企业普遍设立了各色各样的审查委员会，每一个单一用例都必须逐级提交给委员会和高管层进行独立审批。在如今 AI 爆炸式增长的规模和节奏下，这种将每个用例都视作独特“雪花”（Snowflake）的做法已经难以为继。 - **模式化治理（Approved Patterns）**：当前企业正在转向规则模式的前置审批。即向管理层明确定义一套治理模式：“只要满足条件 A、B、C，且未触及某些特定风险红线，该场景即获得自动绿灯”。 - **动态监督与验证**：推行模式化并非放任自流，企业依然需要配套的持续监控（Monitoring）与回溯审计机制，但在审批速度与扩展性上已实现质的飞跃。 --- ### [36:40](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2200s) 约束即自由：将 AI 视为“无限庞大的实习生大军” Josh Coffey 从自身作为技术负责人向董事会汇报的角度出发，分享了如何跨越技术与业务语境的鸿沟。他提到自己面对的董事会成员大多具有深厚的金融财务背景，双方存在天然的知识壁垒。他强调了一个看似矛盾却极其实用的观点：**清晰的安全与合规约束非但不会扼杀创新，反而能赋予团队迅速探索和创新的自由与底气**。 谈及不加节制部署 AI 的隐患，Josh Coffey 幽默而犀利地指出："如果你没有把底层问题解决好，系统本身就内置了无知，然后你还用 AI 去无限放大它——那么恭喜你，你只是成功在整个企业范围内放大了无知（Scaled Ignorance）。" 为了让治理逻辑更加通俗易懂，他提出了一种极具代入感的思考模型： > "如果这不是一个 AI 模型，而是由一支数量无限庞大的实习生大军（an infinitely large army of interns）组成，你会如何管理？你必须为他们设立怎样的防护栏（Guardrails），才能确保他们清楚边界在哪里，并且你如何保持对他们的全程可观测性？" Josh Coffey 进一步指出，在为内部团队及外部客户构建平台级工具时，核心考量始终落脚在三点： 1. **确定性划分**：明确业务流程中哪些环节必须是确定性的（Deterministic），哪些部分允许非确定性存在。 2. **网络与安全边界**：清晰定义敏感数据在进出环境时的安全边界（Security Boundary）。 3. **数据出境透明度**：当使用具有较高风险、需要数据离开本地环境的服务时，必须对哪些数据正在跨越网络边界保持百分之百的清晰洞察。 --- ### [38:54](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2334s) 未来 12 个月治理挑战：“自动刹车”到“全自动驾驶”的演进 在讨论进入尾声之际，Tiffany Benitez 邀请各位嘉宾展望未来一年：在接下来的 12 个月里，推动以可解释性为核心的企业级 AI 信任机制时，最大的挑战与最令人兴奋的机遇究竟是什么？ ```mermaid flowchart LR accTitle: Evolution of AI Control Mechanisms subgraph Containment ["基础兜底阶段：自动刹车 (Auto-Braking)"] direction TB A1["被动式防护"] --> A2["紧急熔断开关 (Kill Switch)"] A2 --> A3["撞车前强制制动"] end subgraph Alignment ["成熟治理阶段：全自动驾驶 (Full Self-Driving)"] direction TB B1["主动式对齐"] --> B2["品牌承诺与数据伦理"] B2 --> B3["合规与网络安全准则"] B3 --> B4["赋能 Agent 高速安全运行"] end Containment ==> Alignment ``` Blake Brannon 认为最大的挑战依然在于“规模化（Scale）”。企业必须在技术架构上完成两大核心能力的构建：一方面是建立**记录系统（System of Record）**，沉淀并复用过往的治理决策；另一方面是打造**控制系统（System of Control）**，以适配不同类型 AI（包括代码生成、Copilot、垂类 SaaS Agent）的多样化管控机制。他用汽车驾驶机制做了一个深刻的类比： - **自动刹车系统（Auto-Braking System）**：相当于紧急熔断机制（Kill Switch），只在即将发生惨烈事故时强行踩死刹车。如果企业里成百上千个 Agent 动不动就触发紧急制动，业务根本无法实现高效运转。 - **全自动驾驶系统（Full Self-Driving System）**：真正的目标是“教会 Agent 如何正确驾驶”。这意味着必须让 Agent 在行动前就与企业的品牌承诺、数据伦理原则、合规法定义务以及网络安全最佳实践深度对齐，从而在保障安全的前提下高速释放业务价值。 Shashank Kadetotad 随后坦诚地分享了自己作为技术从业者的心态转变：如果放在三年前，他自己绝不是治理机制的拥护者，更不会坐在这个研讨面板上。但如今身处治理一线，他认为当下最有趣也是最核心的命题在于：**如何让治理变得具有吸引力（Desirable），并向一线开发者清晰证明“治理能为从业者带来什么实质利益”（What's in it for the practitioner）**，只有让业务与开发人员真正尝到合规底线带来的创新红利，AI 治理才能在组织内部生根发芽。 ### [41:50](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2510s) 决策可追溯性与规模化治理带来的新洞察 从实际从业者的视角来看，AI 治理目前正在迈向一个关键的转折点，即规模化（scale）维度的深化落地。随着企业深入探索“决策可追溯性”（decision traceability）与“决策日志记录”（decision logging）——这些过去在技术实践中往往被忽视或执行不力的话题——团队意外地挖掘出了大量过去从未被察觉的数据关联与内在商业智能。这种通过追踪模型决策全流程而发现隐藏关联的能力，令研发与业务团队感到极为兴奋，也证明了治理流程本身绝非仅仅是合规约束，而是能够主动赋能、揭示大量未知价值的有效机制。 - **隐性关联的显性化**：通过对模型决策链路进行全程记录与追踪，企业不仅提升了透明度，更从中发掘出全新的业务洞察与智能连接。 - **治理的双重价值**：AI 治理体系不仅用于满足监管要求，更正在成为驱动企业探索未知业务机会与优化算法决策的催化剂。 --- ### [42:33](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2553s) 摆脱“AI 战略”执念：将治理回归业务问题本身 面向未来发展，嘉宾表达了一个愿景：希望到了明年，行业内不再有人互相询问“你们的 AI 战略是什么”。正如当下的科技行业会议中，再也不会有人去询问对方的“PC 战略”或“互联网战略”一样，AI 最终将彻底融入企业日常使用的工具与底层基础设施架构之中。因此，企业需要做的是借助 AI 驱动的工具与治理框架来加速内部创新，将治理的核心锚定在企业正在解决的具体业务问题上，而不是孤立地探讨 AI 概念本身，使技术真正回归为解决实际问题的辅助手段。 ```mermaid flowchart TD accTitle: AI 战略认知演进与业务融合模型 subgraph S1["概念阶段（过去/现在）"] A["孤立讨论 'AI 战略'"] --> B["将 AI 视为独立技术孤岛"] end subgraph S2["融合阶段（未来愿景）"] C["AI 融入底层工具织物"] --> D["治理紧贴核心业务问题"] D --> E["加速企业整体创新效能"] end B -.-> C ``` - **从独立战略到基础设施**：AI 将如同昔日的个人电脑与互联网一样，从独立的技术标签演变为不可分割的底层基础设施（fabric of tools）。 - **业务导向的治理体系**：治理的落脚点应从关注 AI 工具本身转向评估其解决商业问题的有效性与可靠性。 --- ### [43:24](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2604s) 全新挑战：建立 AI 事件响应机制与研讨演练 企业当前面临的紧迫挑战之一是“AI 事件响应”（AI incident response）机制的建立，而目前业界对何为“AI 事件”甚至尚无明确统一定义。与已经具备成熟合规流程、知情路径、监管与州法通报机制的传统隐私与网络安全事件（如数据泄露、黑客攻击）不同，当 AI 系统发生非数据泄露类问题（如模型严重幻觉或行为严重偏离预期）时，企业往往缺乏清晰的应对方案。随着越来越多的核心产品和服务深度依赖 AI 驱动，企业必须建立极为清晰敏捷的应急响应标准：明确由谁发出警报、谁介入处理、跨部门通报链条如何流转，并定期开展桌面推演（tabletop exercises），以确保系统在偏离预期时能够迅速得到纠偏与控制。 - **AI 事件的重新定义**：超越传统数据安全事件范畴，涵盖模型幻觉、非预期输出以及算法逻辑偏离等新兴故障模式。 - **健全应急响应与演练体系**：明确告警责任人、制定清晰的通知流转树（notification tree），并通过实战化桌面演练完善异常应对机制。 ## Notable quotes > [00:08](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=8s) Do what you say and say what you do（言行一致）。 — Ron Whitworth > [11:41](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=701s) is the juice worth the squeeze（收益是否值得付出相应代价）。 > [21:10](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1270s) 监管者终将到来，审计也终将到来。我们必须将焦点转回到决策过程本身的可解释性，而不仅仅是最终结果。 > [27:12](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1632s) 凡是能够通过确定性手段（Deterministic）解决的问题，就绝不要交给大语言模型（LLM）去概率化处理。 > [30:28](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=1828s) 它们继承了用户的身份与系统权限，但未能继承企业对该用户的信任。 > [34:22](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2062s) 这就好比你在说法语，而我在说英语。 — Tiffany Benitez > [36:40](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2200s) 如果你没有把底层问题解决好，系统本身就内置了无知，然后你还用 AI 去无限放大它——那么恭喜你，你只是成功在整个企业范围内放大了无知（Scaled Ignorance）。 — Josh Coffey > [36:40](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2200s) 如果这不是一个 AI 模型，而是由一支数量无限庞大的实习生大军（an infinitely large army of interns）组成，你会如何管理？你必须为他们设立怎样的防护栏（Guardrails），才能确保他们清楚边界在哪里，并且你如何保持对他们的全程可观测性？ — Josh Coffey > [38:54](https://www.youtube.com/watch?v=3RN83XGNm3Y&t=2334s) 如何让治理变得具有吸引力（Desirable），并向一线开发者清晰证明“治理能为从业者带来什么实质利益”（What's in it for the practitioner）。 — Shashank Kadetotad ## Mentioned Josh Coffey, Blake Brannon, Ron Whitworth, Shashank Kadetotad, Tiffany Benitez, Apryse, OneTrust, Truist, Mars (玛氏), Compass Group USA, Lumen, Salesforce, ISO 42001, EU AI Act (欧盟人工智能法案), NIST, GDPR, CCPA ## Video description Josh Coffey - Chief Technology Officer, Apryse Blake Brannon - Co-Founder, Chief Innovation Officer, OneTrust Ron Whitworth - Chief Privacy Officer, Truist Shashank Kadetotad - Global Sr. Director Enterprise Data Science and AI, Mars Tiffany Benitez - Head of Data & AI Governance, Compass Group USA Enterprise AI success increasingly depends on trust among regulators, stakeholders, and customers. This panel explores how organizations are embedding interpretability and explainability into their AI systems and establishing governance frameworks that ensure responsible use. Hear from leaders on how they’re implementing practical solutions to meet compliance, support internal oversight, and drive ethical AI practices across teams. This session provides actionable guidance on navigating the regulatory landscape, improving model transparency, and laying the foundation for trusted AI at scale. [Source](https://www.youtube.com/watch?v=3RN83XGNm3Y)

[0:08] Ron Whitworth: Do what you say and say what you do（言行一致）。 [11:41] is the juice worth the squeeze（收益是否值得付出相应代价）。 [21:10] 监管者终将到来，审计也终将到来。我们必须将焦点转回到决策过程本身的可解释性，而不仅仅是最终结果。 [27:12] 凡是能够通过确定性手段（Deterministic）解决的问题，就绝不要交给大语言模型（LLM）去概率化处理。 [30:28] 它们继承了用户的身份与系统权限，但未能继承企业对该用户的信任。 [34:22] Tiffany Benitez: 这就好比你在说法语，而我在说英语。 [36:40] Josh Coffey: 如果你没有把底层问题解决好，系统本身就内置了无知，然后你还用 AI 去无限放大它——那么恭喜你，你只是成功在整个企业范围内放大了无知（Scaled Ignorance）。 [36:40] Josh Coffey: 如果这不是一个 AI 模型，而是由一支数量无限庞大的实习生大军（an infinitely large army of interns）组成，你会如何管理？你必须为他们设立怎样的防护栏（Guardrails），才能确保他们清楚边界在哪里，并且你如何保持对他们的全程可观测性？ [38:54] Shashank Kadetotad: 如何让治理变得具有吸引力（Desirable），并向一线开发者清晰证明“治理能为从业者带来什么实质利益”（What's in it for the practitioner）。

[2:34] Josh Coffey: there's no such thing as an AI strategy; it's really we have a company strategy and we're using AI to try to power that. [8:44] Ron Whitworth: You need to do what you say and say what you do. [11:21] Blake Brannon: what's worse than AI governance is shadow AI governance [20:27] Ron Whitworth: The regulators are coming, the audit is coming. And it goes back to the explainability of the decisions, not just the outcomes. [27:12] Blake Brannon: If the user personal data that I'm using for this thing did not give me consent to use it, don't give it to the LLM and ask it to make sure you don't use it. Just don't even give it to it.

[0:04] Good afternoon. Super excited to be here with  you guys in this incredible panel. So, to get [0:09] us started, if you guys can share your name,  a two sentence introduction, no need to count, [0:14] but just keep it uh short. Um, and then tell  us what single AI objective is dominating your [0:19] strategic focus this year. Start here. All right.  Awesome. Well, afternoon everyone. Blake Brandon. [0:27] I'm the co-founder and chief innovation  officer of One Trust. The thing that's [0:31] uh obsessing all of my time and brain power is  we talk a lot about controls and AI governance, [0:37] but we don't talk a lot about what are the rules  that those controls are running off of. And what [0:42] I've been super obsessed with is every company  makes thousands of decisions of what is okay [0:48] to do and not okay to do. And how do you build all  of those decisions into an intelligence layer that [0:54] can power that agentic control plane that we're  all trying to build inside our organizations. [1:01] Hey everybody, I'm Shashank Karid. I lead the  enterprise data science and AI team for Mars [1:06] Snacking. Uh so my one singular objective  is uh operationalizing governance within [1:14] Mars snacking AI governance in particular and  that's something that we have had in bits and [1:19] pieces but really putting it together so it can  act across an entire portfolio of AI products. [1:28] Everybody I'm Josh Coffee. I'm the CTO of Arise  and we are a developer tools company. So the [1:35] thing that is taking up all of my time, brain  power right now is we're trying to help power [1:41] the AI workflows that involve documents in other  companies right now. And so trying to understand [1:47] and obsess about what are the regulatory  concerns, what are the regulation frameworks, [1:51] what are the privacy constraints that would that  would hinder their ability to operationalize an [1:56] AI workflow over document-based um uh things is  everything that I think about. Good afternoon. And [2:03] I'm Ron Whitworth. I'm the chief privacy officer  at Truist in Charlotte, North Carolina. And uh my [2:09] what's taking up most of my time right now is we  have we've come to the realization that we need to [2:12] build an AI compliance program specifically.  I I own privacy and technology compliance, [2:19] but we have been treating AI as just another  variation of privacy and technology compliance. [2:24] And we've come to the realization that  no, it's really becoming its own thing. [2:28] It needs its own program and its own compliance  framework. So we're building building that out. [2:34] Thank you. I would say we've been on this journey  over the past few years. All of us probably in [2:39] this room and everyone of course on the stage.  Um but we are here not just as a a topic of our [2:46] panel but building trust and AI. I think that  has become a focal point for us. So um I want [2:51] to start with Shashank and if you want to talk to  us about an AI trust building initiative that your [2:55] team has either pushed back back past the finish  line or one that you have in the incubator right [3:00] now and just give us a little bit of the details.  Yeah, sure. Uh like I mentioned uh you know that's [3:06] kind of what we're kneede in right now is the  governance part, right? we haven't quite pushed [3:12] it across the finish line but u that's one thing  operationalizing the governance aspect has been [3:18] uh something that's taken up a lot of my time and  my team's time so um really you know I think we [3:25] start to see a couple of uh issues let's say 6 to  8 months back right and I would say primarily and [3:33] this is probably common across most organizations  is what I would call shadow IT or shadow AI right [3:39] uh there's a lot of excitement ment and henceforth  a lot of people with a lot of budgets can go out [3:45] there and kind of get the products and tools  that they need. Uh so that's one and then [3:50] uh secondly it is you know holistically how do we  decide what portfolios or what use cases form the [3:58] portfolio versus not and uh thirdly uh is you know  how do we communicate um the whole explanability [4:06] aspect the outcome aspect to the business user or  to a non-technical user right so these were kind [4:12] of three pillars that we were after through the  governance process uh and I would say we have done [4:17] a good job of handling two. We're still working  through the third. Hence, I say we haven't really [4:22] pushed across the finish line yet, but uh really  the focus has been right now on um training the [4:29] users and how they actually read. Now, trust was  always an issue even prior to you know generative [4:35] AI being the buzz it is right. Uh so the issue  has always been how do you make the whole thing [4:40] explainable to the user such that you know they  can understand it and then kind of uh react [4:46] with the outputs or the outcomes that they're  receiving. Yeah, you make a very good point. So [4:51] trust is driven by relationship right and instead  of it it being business versus it having business [4:57] really there as part of the conversation as part  of as we actually build and move through this is [5:02] key to the trust really coming about. Josh, what's  your perspective? what are you working on and are [5:07] you still in the incubator stage or have you  pushed forward? That's a good question and you [5:11] know we were talking about this in the hallway  outside and it's you know there's no such thing [5:15] as an AI strategy. It's really we have a company  strategy and we're using AI to try to power that [5:19] and I think is if you marry those two together  you're going to have a lot better of an outcome. [5:23] And so one of the things we were doing early on  was trying to figure out how can we use generative [5:28] AI capabilities in a way that would benefit  our user. And again, our users are developers [5:33] because we're an SDK company. And one of the fir  there's kind of two sides to it. There's the what [5:38] are we trying to do to automate things inside  the company, but also as all of our users are [5:44] also trying to figure out how to automate using  our products because we're building them using [5:48] cloud code or or um whatever else. And as we were  monitoring that, one of the things that we learned [5:55] is that we were using AI to fight AI. So, it was  turned into one of those, well, we're we're going [6:00] to only way to beat the bad guy with AI is going  to be the good guy with an AI. Um, and so one of [6:04] the first problems that we ran into where it felt  like there's no risk in this project was in uh our [6:11] actually our 404 page, which sounds super boring  and unexiting, but one of the problems we were [6:16] running into is that we about 10% of our traffic  that was coming to our website was hallucinated [6:21] URLs from coding assistants. Um, and so we were  trying to figure out, well, how can we get context [6:26] of what that user might be trying to accomplish.  There's no risk. They're already going to land [6:30] at an unusful page anyway. And then once we were  able to show success with that, then it was, okay, [6:34] what's the next problem up the stack that we can  go do where there's maybe a little bit more risk, [6:39] but now we have a little bit more experience in  doing this as well. And we've gotten to the point [6:43] now where we still don't have um direct answers  going out to customers in in all those cases, but [6:49] we do at least have it to where a human's job in  that workflow is predominantly verifying whether [6:55] or not the answers are correct as opposed to  actually providing those answers. Um and so that's [7:00] kind of been the the journey that we're on right  now is really looking at again like what's the [7:06] least risky thing that we can do where it's okay  if it's just wrong and then how do we then expand [7:10] that. I'm really curious if anyone else is is  running into that too where it's like what level [7:15] of trust do you have within the organization?  I'm curious you shank. Yeah. Yeah. No, I I think [7:22] uh we spoke a bit about this outside as well. I  think there are a couple of things when we when [7:28] it comes to trust. So one is um the the use case  itself, right? How much of the cognition should [7:35] lie within the system versus outside the system,  right? like with the user and obviously in that [7:40] context some of the other parameters some of the  important parameters like risk starts to come in [7:46] but essentially what I have seen successful is  um you need to kind of have layered transparency [7:53] right so the question I think there are two  things one is when when someone says explainable [7:59] model right uh in a business context typically  they're not looking for like a white box solution [8:05] necessarily they're more looking for u hey is  there decision traceability to this right so if [8:12] I were to make another decision later on can I go  back and see or simulate what would have happened [8:18] just an example right so I think kind of marrying  those couple of things having you know obviously [8:24] you need to check for bias check for risk check  for performance and all those other parameters but [8:30] uh I I really think it comes down to the  use case the context and what ends up being [8:37] where the cognition should actually lie, right?  That's great. Um, there's a well-known tension [8:44] across model transparency and model performance.  Um, and often though, the reality is the most [8:50] powerful models are the most opaque, right? So,  how do you really manage I want to start with [8:54] Blake. How do you really manage this trade-off  with stakeholders when they're demanding both [8:59] high ROI and total explanability, which I think is  funny. total explanability non-deterministically [9:07] total explanable um I think um [clears throat] I  think there's not a seuite executive that actually [9:14] cares what's the weights of a model uh what they  actually care about is what are the actions that [9:20] these AI systems are taking and do we trust those  actions and can we control them and that's like [9:27] the shift in I think the the architecture and  the approach and there's different layers of [9:30] the problem so I'm speaking from more of a how  do I protect detect bad actions and ensure we [9:35] have governing of the AI actions happening. We've  never governed employees. If you if you draw a [9:41] parallel of an agent to kind of an employee, we've  never governed like how an employee thinks, how [9:46] much time they spend thinking of something, what  what they're concluding, what they're, you know, [9:51] riffing and kind of iterating on their thought  process. We've always focused on when they go to [9:55] take an action. when they take the action is where  I really care about the governing thing and that's [10:00] where I associate so much of the trust from if I  can control the actions from the agentic systems [10:06] um I build the trust with it and I don't care as  much about the traceability of why did it conclude [10:12] something to do or not do uh because I have trust  associated with the actions and the outcomes that [10:17] are coming out of it. So I'm hearing governance  not as a bolt-on but rather along the way as [10:23] you build right then there's the confidence. Ron  I'd like your perspective on this. I know you sit [10:28] in sea sweetite and you're navigating uh with  everyone else here but what are your thoughts [10:32] around that? Yeah I think um one of the things  that we carry over from privacy and we've always [10:37] said for many years you need to do what you say  and say what you do and we need to carry that [10:41] over into AI where it's very easy to just drop  an AI policy out there and say we're going to be [10:46] transparent. We're going to be open. we're going  to limit collection. You have to be able to prove [10:50] that, especially if you're in a highly regulated  industry like banking. But even if you're not, [10:55] there have to be controls that back those  statements up. So, a lot of the, you know, the [10:59] early stages of operationalizing AI is thinking  through when we say transparency, what exactly [11:05] does that mean? Uh what is who's responsible for  that? Where is it documented? How can we prove it? [11:11] So I think it's less about the terms that we hear  thrown around and it's more about understanding [11:16] what what does that mean and how are we defining  that even across not even the industry but our [11:21] companies right I think that's incredible um so  I know everyone thinks of governance as a speed [11:27] bump and we're here to block and stop everything  but that's actually not the reality the governance [11:31] is actually the accelerator so go look at the  data it's not us just sitting on a stage but [11:35] it's actually the accelerator um Ron I do want to  come back to you because this is your world. So, [11:41] can you share a specific instance where strong  interpretability or governance sped up the AI [11:46] adoption and deployment in the enterprise?  Yeah, I think we're we're all in that game [11:50] right now because uh you know, I think to to as a  threshold point, I would say we can't be looking [11:55] at this as a compliance exercise anymore. That's  just not the right way to think about it. So, [12:00] even if we're situated in compliance like  I am, uh you know, compliance is just an [12:04] baseline expectation. But where the rubber meets  the road is how fast can you enable your business [12:10] to be successful? And a lot of times that  requires rethinking your model. You know, [12:15] how do we get engaged early? How do we go through  change management in a faster way? You can't [12:21] compromise safety and security. You, you know, you  have to have that baked in, but you also can't sit [12:26] there and pontificate on something for two weeks  when there's a speedtomarket component that is [12:31] so critically important to the business. So what  that means for us and for many other companies, [12:36] we we often take it back to privacy. We talk about  privacy by design. You're in there in the room [12:41] from the inception of the product or the service.  You need to take that up even a couple more, [12:47] not just in AI, and actually be there when they're  ideulating, when they're thinking through, is this [12:52] something we want to do? Is this worth is the  juice juice worth the squeeze? Is this going to [12:56] have a a good ROI for us? They need to have that  compliance or legal voice there alongside to say, [13:02] "Hey, you know, here are the pitfalls or here are  the opportunities. Let me help you and navigate [13:08] through those things so that the business will be  successful." And that that's the skill set that is [13:12] really the [snorts] most critical one for anybody  in a role like mine or others that are in the [13:16] governance seat to help enable the business. And I  think Blake, you were sort of alluding to that in [13:22] your last statement, but I think having everyone  at the table is huge, right? when we're making the [13:28] decisions early on when we're even thinking about  design. So, how have you really seen governance [13:34] be an accelerator in your company? Yeah, I'll  give uh I'll give an example actually for one [13:39] of our customers because I think the magnitude of  it is actually better than us internally uh using [13:43] it. So, you know what's worse than AI governance?  Shadow AI governance. Okay, so I'll give you the [13:51] analogy here. Uh when you think about the first  cars coming out on the road and there were no [13:56] road rules, right? Which side do you drive on?  There's some horses, there's some cars. Who has [14:00] the rideway? When do you stop? When do you yield?  When do you go? What's the speed limit? That [14:04] worked when it was five cars. Guess what happened  when there was 50? Grinding gridlock halt. So if [14:12] you don't get your arms around governance of  AI with standard terminology, literacy, rules, [14:20] conditions, every one of those different risk and  compliance teams operating from a shared context, [14:25] you will end up with a gridlock and you will slow  down the innovation. And I think the proof point [14:30] of an example of this is one of our customers,  one trusted customer is a company called Lumen. [14:34] Lumen makes fiber optics cables. They obviously  have exploded with AI because connecting GPUs [14:40] together is a is a big business. But uh a couple  about 16 months ago, the CEO came out and like [14:47] many of your companies probably said top down  there's an AI mandate initiative that we've got [14:52] to disrupt oursel. We've got to innovate. We want  spearheaded people inside each department thinking [14:57] about the transformation that's got to happen.  And the challenge is is in that statement and [15:03] in that intent there was no let's go revamp the  governing and cyber security and compliance and [15:08] risk teams. Uh we left those as they were. So  guess what happens? You have all this input and [15:15] intake you've now got to deal with. And for them  to review across siloed risk teams, you're talking [15:20] about months of effort to go through the cycles  of getting brought up to speed, making decisions, [15:25] doing these things. So by coming in and  unifying that program, having central intakes, [15:30] central context, central inventories, guard rails,  translating policy and uh regulatory requirements [15:37] into code that could actually run in runtime, the  enforcement layer that helps you move faster. And [15:42] we were able to uh basically get that cycle time  from months of time for them down to weeks and [15:47] in some cases days. And uh they achieved their  ISO 42,0001 certification. I think the day they [15:53] announced it, the stock popped 7%. I don't know  how correlated or causated that was but you know [15:58] it worked out well and this that is like one step  in the journey but I'll add just one more point [16:05] um where this is going to go and has to go is that  we are all about to be in a company where there's [16:12] more agents doing work than there are humans doing  work. True. Yeah. Everyone kind of believes that. [16:18] Good. It's the right audience. So the so so  you get down to a very fundamental thing. How [16:23] are you going to human review what the agents  are doing? You can't. You need guardian agents. [16:30] You need agents governing the agents. And you're  starting to see this pattern of the architecture [16:36] for how do you set that up because you can't have  the AI model judge itself and say trust me because [16:45] we just found out three or four times in the  past few weeks that they can cheat on the test [16:48] and they will find a way out of their harness. You  have to have a separate independent layer that is [16:54] governing the business agent. And that governing  layer has to fundamentally do two things. Figure [17:00] out what is a decision of a permissive action  that can or can't happen by mirroring the judgment [17:07] that a cyber security risk compliance or privacy  expert in your team would ask if they had the time [17:13] to human review it. Like it would give the same  answer, but it's got to do it in milliseconds. And [17:17] the second thing it's got to do is have a control  layer that can stop that action. So this agent [17:21] goes to connect and do this tool call or send this  email, it could actually block it. It could change [17:25] it. Uh so that's how I think about the you know  governing architecture and kind of what's evolving [17:31] uh in the governing space to speed up governance  because ultimately to get back to your question, [17:35] if you do the things I just said, you're going to  be able to govern new scenarios that you've never [17:40] seen before and find an answer and push a control  and do all that in seconds. Not weeks, not months, [17:46] but seconds. And I think that ultimately goes  back to the whole topic around trust, right? Like [17:51] if you're able to deliver with speed and there's  confidence around it, then trust is built. I think [17:56] the concept of trust is very different because it  means something different to every single person, [18:01] right? What it means to a data scientist versus  maybe a chief privacy officer versus an enduser [18:06] on the business side is very very different. So  with regards to how we all view it differently, [18:11] how Shashank, I want to start with you. How do  you really take and align these disperate crossf [18:18] functional groups um with a unified definition if  you will on what is explainable enough? Yeah. So [18:24] on that um as we go through this process right um  now you know we've found formed a council and the [18:32] council has representation from legal HR you know  privacy like the whole nine yards. Now uh I I I [18:40] can probably take an example of uh like a use  case. So we were down this avenue of creating [18:47] what we are calling like one forecast right now.  Essentially we wanted every single department all [18:53] the way from finance to supply to marketing to use  the same forecast. Now how would we achieve that? [19:00] Everyone has different incentives. Everyone has  you know um different translations and probably [19:06] different use cases attached to the forecast. So  it was like a huge exercise in alignment and a [19:13] huge exercise in transparency right so every  single thing that we did the models we used [19:19] um uh everything was kind of exposed to the user  for nontechnical users nontechnical terms for [19:25] technical users and technical terms right  and and pretty much even when I talk about [19:30] governance that's the same exercise we're kind  of going through which is making sure not only [19:36] like it makes sense holistically But for each  person involved who has different incentives [19:42] how do we make sure that uh you know their  incentives are covered and if they aren't right [19:47] uh do those incentives matter and if they ma if  they don't matter great if they if they matter [19:53] how do we include it right so it's been a huge  exercise a lot of conversation and coming to a [19:59] process that is now I think well defined uh  and you know to your earlier point as well [20:05] it absolutely can't or should not be a stagegate  process, right? Uh governance just can't that's [20:10] that's the reason you have shadow AI in the  first place. So the idea then is how how does [20:16] every use case kind of bake the governance layer  right in there, right? Uh and to your point, [20:22] you have policy as code as one of the things that  can really help do that. So Josh, I'd like your [20:27] perspective. You're in you lead software, right?  And so how do you see this really come into play? [20:33] like how do you navigate across functionally  on this concept of trust and how to navigate? [20:41] It's a good question. The the I think a piece of  it that you alluded to and I'm curious if you're [20:46] doing some of the similar things then as me is  when decisions are made is create a record of [20:52] what are the things that went into that decision.  So what are the pieces of documentation? What are [20:56] the pieces of user data? What are the pieces like  how did it how did the model reason about whatever [21:01] it was? And we can't necessarily get the full  thought process out of it, but we can at least [21:05] get these are the key pieces of information  that I was using. And one of the areas that [21:10] we're exploring right now is can we also start to  like put a a QA score on each of those pieces of [21:16] information. So if I'm making a decision based on,  you know, 15 different customer records, you know, [21:22] and I know that nine of them meet a certain  quality threshold and the other six don't, [21:27] can I flag that somehow in the response  as well? And then that way we would know, [21:31] okay, well, these are these are decisions that  maybe are more likely to have been incorrect or [21:36] misinformed. So those are the ones maybe that we  should do human in the loop to review or human in [21:40] the loop to maybe even just go back and and verify  before we update our model weights on everything. [21:46] Um but that level of explanability of these were  the pieces of information and this is what the the [21:53] quality layer um believed that the quality of that  information was being married with the decision [21:59] itself I think is really important because then  you have an audit log effectively of how that was [22:03] made. The audit is coming, right? For those of you  that have not yet experienced, the regulators are [22:09] coming, the audit is coming. And it goes back to  the explanability of the decisions, not just the [22:13] outcomes, right? So, we're going to have to pivot.  Um, Ron, have you seen any of that at this point? [22:19] Uh, internally, [laughter] same internal auto is  a way of life at uh when at a bank. Uh, but you're [22:24] you're absolutely correct. Um, and one of the  challenges that we have, and we talked a little [22:28] bit about this uh in other sessions earlier today,  is it's it can be difficult when there's no laws, [22:33] rules, or regulations per se that hit directly on  the thing that you're talking about. So, it really [22:39] comes back to what are the frameworks that we're  aligning to? What are our principles? What what's [22:44] our risk appetite? And do we align with that? And  I think that's that's really the analysis now and [22:49] being able to show that to an auditor. Hey, this  is what the board approved. This is our approach [22:54] and we are aligning each of these use cases to  what we've have a common understanding of and [22:59] I think that will help a lot once you have true  external audits coming. I would say right now uh [23:05] global AI regulation in general is like hitting a  moving target. Anyone agree? Yeah, it is. So, um, [23:13] I would say Blake, what do you think is one highly  actionable piece of advice that you would give [23:18] leaders that are here sitting in this room today  for future proofing AI compliance and kind of [23:23] the ethics program at scale is key? Yeah, I think  to ground uh and build up what Ron was saying, I [23:30] think there's two dimensions. One, there are right  now 1,200 proposed AI regulations in 47 countries [23:37] around the world. Even in the US you got about 150  uh different regulations across state and things [23:44] like that. And most people think AI regulation is  only some new AI specific regulation. What also [23:50] regulates AI are things like data protection laws,  digital laws, existing compliance laws. So you [23:56] have to navigate that mess uh as an organization  to figure out like what is what what is something [24:02] we could stand up and stand behind as part of  an audit. I simplify it down to regardless of [24:07] what any of those say, I think you're in a really  good spot if you do something really simple. You [24:11] can answer the following question. I know what AI  we have deployed in our company. I know what it [24:17] has access to and and is connected to and I trust  the actions that it's taking and can control them. [24:24] And if you can, if you feel comfortable with like  signing that and saying, "I feel good about this [24:30] in my company," I think regardless of an current  regulation or policy or a future one, you're going [24:36] to actually be in a really good spot to be able  to prove and demonstrate accountability, riskbased [24:41] decision-m um and not being negligent to something  you're doing inside your organization to future [24:46] proof yourself for any future regulation that's  going to come. How many people do you think could [24:50] actually answer that? I would say, okay, I've done  lots of polls of this. Yes, I know the answer. I [24:55] know the answer. Third party wise, right? Vendors,  I think we can mostly attest to that. But on the [25:00] internal agentic side and builds, do you think  the answer is zero? I know. I just wanted to [25:05] make sure I was on the same page. Anybody in here  feel like they can answer that? Like, you know, [25:10] every build that No, I see no. That's the exact  same answer I always get. Well, Ron, you own the [25:17] principles. You're the privacy officer, so you  own those principles. So, no pressure. Let's not [25:23] ask him that. Like but but AI is not just legal  and compliance, right? And so talk to us a little [25:28] bit about that. Yeah. And and I actually co-own  them with our chief ethics officer, our chief AI [25:33] officer. So it is it is truly a um handstacking  type thing where we say, "Hey, what are we trying [25:38] to accomplish?" And you know, you'll see of things  in there that are very much compliance or legal in [25:43] in nature, but there are also things like business  enablement and and innovation. That's what we're [25:47] here to do in addition to being compliant and so  forth. I think in terms of futurep proofing um you [25:52] know privacy again not to repeat what I keep on  saying about privacy but looking at the history of [25:57] of the successful privacy programs over the last  10 years what they did was they got ahead of the [26:03] game by looking at these international standards  that were out there and also international laws. [26:07] So GDPR came around and that wasn't necessarily  the most applicable to every US company, [26:14] but it was it was very clear that that was going  to be the worldwide standard for how privacy will [26:18] be regulated. And we eventually saw that come to  fruition in California and and many other states [26:24] in the United States. the companies that were  forward-looking and saying, "Hey, even if GDPR [26:29] doesn't apply specifically to my organization, I'm  going to ramp up the privacy control so it it it [26:34] is very much aligned with GDPR so that we're ready  when California comes around." AI very similar. [26:40] You have the EU AI act. you have these frameworks  that are established that establish best practices [26:46] that when the when the fed the federal law comes  around or whether it's states or combination of [26:51] the two they're going to be looking at those  things when they draft what is applicable to [26:56] these US companies so if you're already aligned  to these international standards you will be in [27:01] great shape and so that's that's the the piece  of advice I would give is to kind of like look [27:05] three to five years down the road and use those  those existing frameworks to your advantage [27:12] So when we start talking about fairness and bias  detection, operationalizing that is extremely [27:18] resource intensive. So Josh, what tools or  frameworks processes are you guys currently using [27:25] or have you seen proven to be most effective for  your teams with regards to monitoring monitoring [27:30] AI? Yeah, it's a great question. So I I have a a  little bit of a different perspective of this, I [27:34] think, than the rest of the panel will. I'm really  curious for tool recommendations cuz I don't have [27:38] a huge amount of tool recommendations, but the  thing that that we're constantly looking at right [27:43] now is what is the risk surface and how are we  thinking about from the perspective of um you know [27:49] like what are those boundaries that we can create  around different aspects of people's workflows [27:53] cuz again we're primarily not solving problems  internally because we don't offer a product that [27:58] we offtheshelf sell but rather we're tools that  are helping other people build products. So as we [28:03] talk to you know financial sector uh customers  or customers in the healthc care space or [28:09] um medical research is a big one for us. It's  what are the boundaries that need to exist and [28:15] how can we create solutions that can be deployed  into specific network boundaries that would allow [28:21] for those privacy concerns to not exist because  you can control the level of data that's moving [28:26] around and having that constant visibility of  you know if if I need to be able to do a medical [28:31] meta analysis which is one that we just worked  on recently and you know in any given medical [28:36] analysis it's not that this is the boundary for  privacy or the framework that works for everything [28:42] because certain pieces of information matter in  certain other contexts. I'm a 41-year-old white [28:47] male that lives in Texas. Some [snorts] or all  or none of that may be relevant for any given [28:54] question that you need to ask. So part of it is  how do you create levels of privacy around those [28:59] pieces of information as well so that if I'm  doing something some kind of analysis or asking [29:03] a question where my age is irrelevant, how do you  not even expose that to the model in that case? Um [29:09] I think that's I I'm really curious. I have a yeah  I was going to say I spent a lot of time thinking [29:13] about this. So I think there's a principle here  of what you can deterministically do you should [29:18] always do right. So to your point one of the key  scenarios we help companies with is if the user [29:24] uh personal data that I'm using for this thing  did not give me consent to use it. Don't give [29:29] it to the LLM and ask it to make sure you don't  use it. Just don't even give it to it. Right? So [29:34] you want to you want to have a data control  going into the LM pipeline to pull that data [29:39] out of the data set before it even makes it in  there. And you can take that um and broaden it [29:44] further to just a broader concept we learned a  lot about in privacy uh called purpose-based use [29:49] or purpose-based intent. And for those of you who  are not familiar with that, the notion is that you [29:55] a lot of the wrongs and right of privacy are not  you have this category of data or you have this [30:01] uh certain type of data you're storing. That  in itself is not good or evil. It's what you're [30:06] using it for and what is the potential harm of  that use compared to the intent of what it was [30:12] given to you for. So classic example, I gave you  my phone number to MFA me to log into application, [30:19] but you use it for sales and marketing. That's  typically not viewed as uh something people [30:23] prefer and is viewed as a violation of personal  data. Um if you pull that into the agentic world [30:28] and you start to think about these boundaries,  you start to think about the authorization model [30:33] needing this notion of purpose or intent context  to be brought into it. So to ground that into an [30:39] example, um there's a common problem with AI  agents right now is where they inherit our [30:44] identities, but they don't inherit our trust. So  Blakes's has access to the CRM for my company. [30:51] I have the ability to create, edit, and delete  records, right? My authorization gives me the [30:56] ability to do that. The company trusts me to not  delete records that I shouldn't be deleting. But [31:03] I can tell you my CISO and my CFO do not trust my  agent operating on my behalf to not delete those [31:11] records. So there's a trust gap between what what  I and my entitlements have and what the user has. [31:17] So to bring this back to purpose when you look  at a scenario you have to take into the context [31:23] to determine is it a permissive action or not in  this boundary. So when I look at a AI agent that [31:29] is deleting, going back to the the same thing,  deleting opportunities out of Salesforce, is that [31:34] a permissive action or not for this agent? Well,  I need more information. I need to understand the [31:39] context of why it's doing that. If it's doing a  quarterly cleanup to purge out old opportunities, [31:46] that delete authorization action is permissive  and is desired, and we want it to happen. If it [31:52] is a BDR agent that is trying to optimize their  conversion close rate by deleting opportunities [31:57] that won't close, that is not a desired action.  They're both the same authorization. They're [32:01] both the same tool call. They're both the same  delete. The only thing different is the purpose [32:05] and intent of why it wants to do that. And and  that is a dynamic that has to fit into this this [32:11] architecture, I think, of determining these  actions to build the trust with these systems [32:16] at scale. Right. And I guess to your point too,  that's you know what that intent should be on [32:21] any given workflow. So do the deterministic thing  and not even expose it. Exactly. Yeah. Shashank, [32:28] any tools that you've been using? Um [snorts]  I don't have any tool recommendations, but [32:37] uh I think there are a couple of things just to  your point, right? Um just going back to earlier [32:44] the whole action and then being able to validate  the outcome or the output of that agent. So how [32:51] else would you kind of then um be able to kind  of communicate or track the intent of the agent, [33:00] right? Because we do that really well in  deterministic models for sure, but when it [33:04] comes down to LLMs and agents that becomes super  tricky. So I'm just curious like question. know [33:09] that there's a lot uh I've thought about this as  well. So one you have typically just like a human [33:14] in a company an agent has a role in a general job  to be done or set of jobs to be done and in that [33:20] you can have a much more narrow unlike an employee  that's in marketing has a pretty broad set of [33:25] access to different tools. An agent probably has  a much narrower set of things of like your job is [33:31] to send campaigns right or or to do things. So  it can be an even narrower scope. Uh the great [33:36] thing about AI systems is because you can see the  uh the trace and the observability logging. You [33:42] can actually infer the consent from just looking  at the what has it been trying to do? What was [33:47] the prompt? What was the reasoning? What did it  conclude? And out of that you can infer what is [33:52] the intent and and the way that works to governing  rules is you take that intent and you map it back [33:57] to a set of purposes which map to governing rules  of what can and can't happen. And that's where [34:04] you have the interlock of this. I see the agents  doing this. I'm detecting as as its why is based [34:09] on this intent. And our rules allow that data to  be queried and used for customer support but not [34:15] for marketing. So I'm going to allow this one.  I'm going to deny this one. Got it. Got it. Yeah. [34:22] So around like when we start talking about AI  decision modeling, we're we're sitting here having [34:28] conversations, but I still think there's a major  gap with regards to how we communicate that back [34:33] to board members or stakeholders on the business  side. And so I think we I always equivalate it to [34:39] you speak French and I speak English. And so you  built something for me and I come in the room, [34:45] I'm going to smile and nod. I don't understand  a word that you said. And then I leave. I don't [34:50] know what you just said. I'm not going to use.  You wouldn't do that to me for real, would you? [34:53] This is real life, right? And and but what I'm I  think the point is that that causes a lot of the [34:58] trust issues. It causes a lot of the gaps and I  think we have to get better at that. So when I say [35:02] business versus IT, that can be right on the cyber  side, it could be on the privacy side, but Ron, [35:07] you sit in a lot of these rooms and you have these  conversations. So what's your strategy for really [35:12] making that that interpretability actionable and  digestible? Yeah, I would say one of the trends [35:18] that we've seen recently um has been a shift  and not just my company, every company that I [35:23] talked to um you know not every but a lot is the  shift to patterns um you know pattern management [35:29] and sort of socializing approved patterns rather  than individual use cases. You know, if you look [35:35] back two or three years ago and you talk to all  the, you know, major companies, we all had some [35:39] variation of committees where every single use  case was coming through a committee. Everybody was [35:44] vetting them, deciding and then those use cases  were all going up to some executive board or, you [35:50] know, whatever it might be management for complete  visibility into all the different use cases. I [35:55] think what folks have realized now is that is not  a manageable way to to uh to handle AI not at the [36:02] scale and the pace that we have now. So it's more  about shifting to patterns. You know what what [36:07] type of patterns can we approve that where we say  hey if if A B and C is happening and these other [36:13] two or three things are not happening you have a  green light you're good to go and you bring that [36:18] forward to the executive management and say here  is our approved pattern. Anything that falls in [36:23] the in this pattern is good to go. I think that  gives that that confidence and again you have [36:28] to prove it. You can't just say we're good to  go with everything as long as somebody says it [36:32] meets the pattern. there has to be monitoring uh  you know oversight and and approval but at a much [36:38] different pace than we were at a year or two ago.  Yeah. Josh, do you I'd like you you kind of have [36:46] a a different perspective. Um so with regards  to really being able to make that explainable, [36:51] not, you know, make it not so complex, like how  do you have those conversations in your world? [36:57] Yeah. Um [clears throat] I love the the question  too the way you said it the previous one of you [37:01] know to people that are outside of our our roles  too. You know I I sit in a boardroom and it's [37:06] mostly people with finance backgrounds and so  there's a there's a you know kind of knowledge [37:10] gap there um that I think it's on incumbent  on us to be able to to bridge. You know you [37:15] were you were talking about something interesting  which was the constraints that you put in place [37:20] actually give people the freedom and opportunity  to be innovative and explore. So I think that's [37:25] a a key part of it is what are the what are the  things that you would need to have the confidence [37:31] in being able to move incredibly quickly. And so  I I look at it you know if you don't solve the [37:37] problem well and you have ignorance built into  that and then you scale that infinitely with AI [37:42] like congratulations you just scaled ignorance  across the business. Um, but I look at it from [37:47] just from the even from the perspective of what if  this was instead of an AI model, what if it was an [37:52] army an infinitely large army of interns? What are  the things that I would need to be able to do to [37:56] put in place make them understand like where the  guard rails are and how can I have observability [38:01] into that? Um and then for the the part with  with our tools specifically is then how do I [38:08] provide that to not just you know our own our own  internal company but also to the other customers [38:14] that are using our products to build their own  platforms with what are the kinds of constraints [38:17] that they're going to have. How do we make sure  that we're enabling the kinds of things that give [38:21] them the confidence that that they actually do  have the right kind of you know security boundary [38:25] and security context around these different  things. um and make sure that when they want [38:30] to use a you know riskier thing where data has  to leave their environment that they have full [38:34] understanding of what is and isn't exiting that  that network boundary as well like for me that's [38:40] that's everything that I think about it usually  comes down to what parts of this processing can [38:45] be deterministic which parts are okay if they're  nondeterministic and what's the network boundary [38:49] between the two I love that okay so we're  coming up on our last few minutes um if each [38:55] one of you guys can kind of go down the line and  answer. Over the next 12 months, the next year, [39:00] so by the time we're back here next year, what  is the greatest challenge you see with regards [39:04] to governance truly driving trust? Um, how do we  really make sure that the trust comes from the [39:12] ability to explain? Um, so what is the greatest  challenge but that gives you the most excitement? [39:22] um greatest challenge I would go back to the scale  of this. So I think there's two dimensions of [39:30] that. one Ron talked about of you have to switch  from looking at every use case as a snowflake and [39:35] trying to assess it as a unique independent  thing to what are the patterns and what are [39:40] the fundamental decisions that we've made and how  do we build that into what I would call a system [39:45] of record and an intelligence layer to be able  to replay those decisions at a faster speed uh [39:51] for a future decision that we're trying to make.  So how do we actually accelerate the decision- [39:56] making part of the process and then the second  part is how do we actually control and reflect [40:01] those decisions. So we need a system of record  for the decisions. We need a system of control [40:05] for the controls. And um the AI systems across  whether it's clog code, whether it's co-pilots, [40:14] uh whether it's vertical SAS specific agents,  they all have different technology approaches [40:19] for how to control them. And I think there's  there's two dimensions that we have to think [40:24] about this. One is like containing and having a  kill switch to sort of stop it from happening. [40:30] The second is like aligning it. And that analogy  to think about is kind of the difference between [40:37] an auto braking system on a car which is going to  slam on the brakes to sort of stop the bad wreck [40:42] from happening versus like a full self-driving  system which teaches the car how to drive. And [40:47] if you want to go fast and have a lot of agents  actually providing value, you can't just have them [40:52] all slamming on the brakes. You've got to teach  them to how to drive. And what that means is to [40:56] be aligned to your organization's brand promise,  data ethics principles, compliance obligations, [41:02] cyber security best practices. And I think  that's the biggest challenge at scale is how [41:08] do you do that? Not in oneoff bespoke things for  every tool that you're trying to deploy, but at [41:12] scale. But I'm very optimistic that we're as a  community and an industry about to solve that. [41:20] I think you covered pretty much everything,  [laughter] but um I think so as a practitioner you [41:27] know I I wasn't a fan of governance right if you  were to ask me this question like three years back [41:33] uh I wouldn't want to be on this panel but um I  would say like since I've taken up the new role [41:38] and now kneedeep in governance right um one I  think the most interesting aspect has been how [41:45] do we make it desirable and what's in it for the  practitioner right and that's a problem I think [41:50] we've almost solved right uh just putting on the  practitioner that I I think the second bit really [41:56] is uh to your point right uh I think it's really  the scale aspect so as we kind of go on this [42:02] journey like as we go down decision traceability  and decision logging something we probably didn't [42:08] do very well earlier I think that's uncovering a  lot of connections and intelligence that we didn't [42:13] really know before and I think that's what has  really excited me personally and the team as well [42:19] wherein we are able to make connections that you  know weren't very visible earlier So I think the [42:25] process of governance itself is going to uncover  I think a lot of interesting things in the future. [42:33] It hard to follow some of these. So the the  thing that I my wish for all of us is that [42:40] next year no one is asking anyone else what is  your AI strategy? Um, and I say that because, [42:46] you know, no one has asked each other, I assume,  during this conference, what is your PC strategy [42:52] or your internet strategy? It's just like there's  it's just a part of the fabric of all the tools [42:56] that we use now. And so for me, I hope that we're  using things like governance um and things like AI [43:03] based tooling to accelerate the innovation inside  of our companies and really using that governance [43:09] not in the context of AI, but in the context of  the business problems that we're solving and and [43:14] how we're implementing that kind of almost I  don't want to say immaterial, but it's it's [43:19] less interesting that the problems themselves  that we're solving. One of the things I think [43:24] that is an imminent challenge for many companies  is AI incident response. And um when I say that [43:29] it's like well what does that even mean? Because  that the definition is not even out there yet [43:33] in terms of what do you mean by incident? When I  think about privacy incidents we're talking about [43:37] exposure of data, cyber attacks, data breaches and  we have well-defined processes. How how do people [43:43] become aware of them? Who gets notified? You know  notifying consumers, regulators, whoever it might [43:48] be. Lots of state law implications etc. Well, what  happens with AI when it's not a data exposure? [43:55] When the AI hallucinates or does not act the way  that you expected it to? The more that we're using [44:01] AI to provide products and services and we're  relying on it uh to effectuate what our company [44:07] does, we need to be very crisp on what happens  if it's not doing what we expect, who alerts who, [44:13] who who jumps on it, who's involved, who, you  know, what does the notification tree, what do the [44:18] tabletops look like? I haven't seen a lot of that  in the conversation to date. A lot of people kind [44:23] of thinking about it. I think we need to start  really uh perfecting that as as we get further [44:28] down the road. Thank you guys so much for your  wisdom and insights today and hopefully this was [44:34] helpful. I think we're all walking this journey  together, but thank you guys so much. Thank you.