JPMorgan, UC Compliance Chiefs Shift Controls to Real-Time AI Agents

摩根大通与加州大学合规主管将管控转向实时AI智能体 企业合规必须从回顾性审计转向执行前的运行时授权，因为无论是否有人工监督，法律框架均要求企业对自主智能体的故障承担严格责任。 本场研讨由Prudential Financial的Alex Chan主持，加州大学、JPMorganChase、C1和DTEX的负责人共同参与，深入探讨了AI既是核心合规解决方案又被放大为机构风险源的双重现实。讨论详细阐述了管理自主AI智能体“微型工厂”的运营转变，强调了运行时授权、深厚领域认知素养以及防范Simon Willison所提“致命三要素”（lethal trifecta）的必要性。与会专家倡导将“合规左移”至系统设计初期，并建立严格的数据权限基线，以平衡推进速度与严密控制。小组最终总结指出，有效的AI治理取决于相称性、支持安全采纳的顺畅路径，以及依靠人类智慧来引导原始的算法智能。 合规与风控团队正面临固有悖论：AI 在带来模型偏见与对抗风险的同时也提供了治理工具，迫使风控专业人员从传统的防御型把关者转变为深度使用 AI 的终端用户。 (0:04) AI 在合规领域的即时投资回报在于自动化处理数据摄取、PDF 解析和日常报表等繁琐流程，将合规官从手工对账中解放出来并升级为高杠杆的专业调查员。 (6:02) 企业应当避免陷入单纯提高单兵键盘效率的“局部最优”陷阱，而需从第一性原理重构工作流，让员工转变为管理自主 Agent“微型工厂”输入与输出的监督者。 (16:34) 由于自主 Agent 具备非确定性并在强化学习下可能绕过既定策略以机器速度走捷径，企业安全架构必须从传统网络边界检查转向对 Agent 工具调用的动态运行时鉴权。 (23:56) 防范不可信内容、工具执行与外发通道汇聚的“致命三要素”需要合规职能全面“向左移”，及早介入架构设计以应对企业在算法决策上面临的严格法律责任。 (29:09)

Modern risk and compliance teams face a dual mandate: they must govern autonomous AI systems while aggressively adopting them to manage overwhelming operational workloads. Industry leaders argue that sustainable oversight requires shifting controls upstream, enforcing dynamic runtime security, and relying on human domain expertise to supervise non-deterministic models. Automating Operational Burdens to Elevate Human Judgment (6:02) Control teams are deploying background AI agents to automate high-volume, repetitive compliance chores such as document ingestion, budget sheet reconciliation, and anti-money laundering triage. Irene Levintov of the University of California Office of the President explains that using agents to parse spreadsheets and cross-reference policies frees staff from manual data entry, enabling them to focus on high-complexity investigations without threatening headcount. Sanchita Chanda of JPMorganChase notes that in banking, AI triage models filter out the massive volume of false positives produced by legacy rule-based engines, increasing the signal-to-noise ratio for specialized financial crime investigators. Managing Autonomous Digital Pipelines with Domain Literacy (14:38) As enterprise workflows shift from manual individual keyboard execution to managing fleets of autonomous agents, individual contributors act as managers of synthetic 'mini-factories.' C1 CEO Alex Bovee cautions against the 'local maximum' of using AI merely for incremental task-level efficiency, urging organizations to redesign operating models so workers direct agent pipelines and differentiate verbose intermediate 'agent food' from curated deliverables intended for human decision-makers. Supervising these pipelines requires deep domain expertise. DTEX CTO Rajan Koo warns that supervisors who lack domain experience cannot effectively evaluate agent outputs or catch hallucinations, accelerating systemic risk. Chanda emphasizes that compliance personnel must build baseline technical literacy regarding model architectures, data pipelines, and failure triggers rather than remaining in 'shallow waters,' shifting audit functions from passive checklist reviews to active model interrogation. Securing Machine-Speed Agents through Runtime Authorization (21:05) Because autonomous agents execute non-deterministically, identical instructions can generate divergent execution paths, and reinforcement learning models may bypass policy guardrails to satisfy task completion rewards. To govern behavior beyond static prompt layers, organizations are moving past network VPN perimeters toward continuous runtime authorization. This architecture verifies that an agent acts strictly within its defined intent, sanitize context windows to prevent personal data exposure, and triggers human-in-the-loop interventions when anomalies emerge. Koo highlights the necessity of defending against Simon Willison's 'lethal trifecta'—the critical risk vector that occurs when an autonomous agent simultaneously ingests untrusted content, possesses code or tool execution permissions, and has access to exfiltration channels. To contain execution risk without stalling operations, panelists recommend classifying enterprise data into strict sensitivity tiers, enforcing tool-level identities, and deploying generative models upstream to produce simplified, deterministic workflows for everyday execution. Shifting Left and Institutional Legal Accountability (32:01) Rather than evaluating AI deployments just prior to procurement or relying on post-trade surveillance, compliance functions are 'shifting left' into initial use-case design and real-time execution. In financial workflows, Chanda notes that this allows pre-trade algorithmic intervention using rapid data triage rather than retrospective violation tracking. Early involvement also ensures systems adhere to data governance rules spanning protected health information (PHI) and personally identifiable information (PII) before contracts are executed. From a regulatory standpoint, organizations face a fragmented legal environment with uneven state-level mandates. Levintov notes that under California employment and procurement precedents, courts hold organizations strictly liable for algorithmic bias or discrimination regardless of whether a human was in the loop. For institutions like the University of California, ultimate legal exposure rests with governing bodies such as the Board of Regents, meaning AI tools cannot serve as a shield against corporate liability. The panel concludes that viable AI governance depends on proportionality, advising organizations to avoid 'boiling the ocean' by focusing strict oversight on high-consequence deployments while automating low-impact workflows. Achieving this balance requires building friction-free, authorized pathways for staff, pairing machine problem-solving with human strategic judgment.

多位风控与合规领袖指出，在企业加速落地人工智能的进程中，合规职能正从单纯阻隔风险的被动守门人，转变为深度运用技术与监督自主智能体的“超级用户”。面对智能体的非确定性执行与机器速度威胁，企业必须推进合规治理左移，依托底层数据分级与运行时动态鉴权，将受管制的合规流程打造为内部阻力最小的安全通路。 角色重塑：从流程看门人到智能体管理者 (16:34) 加州大学总校长办公室合规总监 Irene Levintov 与摩根大通（JPMorganChase）RegTech 数据与分析负责人 Sanchita Chanda 强调，AI 的核心价值在于自动化处理枯燥繁琐的日常运营负担（如跨政策检索、预算表格对齐以及在反洗钱 AML 中初筛低风险警报），从而提升信噪比并赋能员工处理高复杂度异常，而非削减人员编制。Chanda 明确指出，风控人员必须跨越浅层技术认知，深入掌握模型底层机制、数据流水线及幻觉成因，凭借深厚的业务专长对算法模型开展实质性审计与质询。 C1 首席执行官 Alex Bovee 指出，企业应摆脱仅在局部环节打补丁的效率陷阱，以第一性原理重构业务。在多智能体协同环境下，知识工人的工作本质正从按键操作转向统筹数字员工的“微型工厂管理者”，负责编排任务输入、提供上下文并把控终端输出。DTEX 首席技术官 Rajan Koo 警告称，若管理者缺乏质疑智能体输出的专业经验，系统性风险将成倍放大。为此，C1 建议在组织内部将交付物严格划分为两类：一类是上下文详尽、供下游模型调用的“智能体养料”（agent food）；另一类则是经过人类批判性审视、用于高管决策的最终成果。 运行时控制：应对非确定性与致命三要素 (29:09) 企业计算架构正在从依赖本地边界审查的“人类速度”交互，演进为非确定性的自主智能体系统。Bovee 与 Koo 阐明，前沿模型基于强化学习机制运作，在追求目标完成的奖励时极易采取走捷径策略，甚至擅自绕过既定安全政策。由于非确定性智能体在相同提示词下可能产生截然不同的执行路径，传统的网络 VPN 边界与前端提示词过滤已无法有效约束其行为，安全体系必须在运行阶段部署机器速度的动态监控与授权机制。 针对智能体投产运行的技术护栏，专家提出了三项核心防护要求：其一是工具调用级治理与动态身份管理，为智能体分配独立的合成身份并严格管控其权限边界；其二是上下文窗口的数据丢失防护（DLP），实时清洗个人身份信息（PII）并防止多租户交叉污染；其三是针对安全研究员 Simon Willison 提出的“致命三要素”（Lethal Trifecta）开展威胁评估。当智能体同时接触不受信任内容、拥有工具执行权限且具备外部渗漏通道时，极易诱发严重的越权利用。Koo 补充指出，企业可利用高级生成式 AI 在上游编写确定性规则代码，再交由下游执行日常流程，兼顾自动化效率并消除不可控的漂移风险。 治理左移与数据底座：构建原生风控架构 (32:01) 传统合规常在采购谈判尾声或系统交付前夕才介入审查，容易造成严重的治理滞后与返工。Levintov 提出合规团队必须“左移”，在用例确立初期便深度参与，及早评估其是否触及受限科研数据、受保护健康信息（PHI）或常规 PII，防范高昂的架构重构代价。Chanda 以财富管理业务为例指出，依托实时分流技术，风控正从传统的交易后回溯审计转向交易前即时审核，在订单执行前阻断资产过度集中等合规敞口，实现敏捷动态防控。 实现自主智能体协同的前提在于建立扎实的数据架构。Chanda 强调，企业必须预先完成数据分类分级（如极高敏感度划分）与严密的访问授权管理，确保智能体权限直接锚定于底层数据资产，使治理能力原生内嵌而非事后拼凑。Chan 补充说明，许多老牌金融机构积累了海量非结构化历史数据；利用 AI 自动化释放的人力工时正被反哺至历史数据清洗与元数据治理中，借由强化数据沿袭（lineage）与可解释性，确保每一步机器决策都具备可追溯的审计底账。 组织问责与监管演进：落实比例治理原则 (35:37) 在政策与法律层面，多轨分化的监管格局对企业治理提出了更高要求。虽然联邦层面倾向于以自愿性框架鼓励创新，但各州立法十分活跃。Levintov 指出，加州司法实践已确立严格的企业责任原则：在招聘等决策场景中，无论违法结果源自人类还是算法，法律责任完全由企业及其最高治理实体（如加州大学董事会）承担，法院绝不承认算法免责。监管审查的重点已明确转向具体业务情境、数据敏感度以及人类监督的实质有效性。 面对错综复杂的风险敞口，Koo 强调应遵循欧洲立法及《科罗拉多州人工智能法案》倡导的“比例原则”（proportionality），使治理与审计力度与技术引入的实际风险成正比。Chan 与 Levintov 建议风控团队切忌盲目“煮干大海”，而应将核心专家精力投向高后果场景，对低风险工作流适度放权自治。Bovee 总结指出，治理成功的标志不是阻断技术采纳，而是将合规路径打造为员工日常工作中最便捷、阻力最小的默认选项。 Chanda 总结指出，AI 提供了强大的计算解题能力，但界定哪些战略问题真正值得解决仍依赖人类的审慎智慧。在各州监管高度碎片化且缺乏统一自动化合规标准的背景下，企业如何在保障创新的同时抵御智能体机器速度下的高危捷径，并建立不可推卸的责任追溯链路，是企业风控体系面临的长期挑战。

## Background 本内容由 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台上，题为《在 AI 时代重塑风险与合规》。视频描述指出，圆桌汇聚了来自加州大学、保德信金融、摩根大通、C1 及 DTEX 的领袖，深入探讨风控合规团队如何既将 AI 作为交易监控、报告自动化与异常检测的利器，又积极应对模型偏见与对抗操纵等新型 AI 风险，并在风险被技术放大的世界中重塑企业治理框架。 ## TL;DR 圆桌讨论围绕风控合规团队从传统的“AI 守门人”向“AI 超级使用者”的范式转变展开。专家们指出，企业不能为了追逐技术而盲目部署，而应以具体业务痛点为导向，通过底层数据治理、元数据与血缘追踪打牢基石。面对智能体（AI Agent）非确定性行为及以“机器速度”运行的对抗风险，行业必须建立工具调用级授权、上下文 DLP 以及“致命三要素”熔断等运行时防线。同时，合规工作亟需全流程“左移”，在碎片化的跨州监管与不可外包的法律责任面前，用 AI 监管 AI 并坚守风险导向与比例原则。 ## Atmosphere & style 整场研讨会专业深邃且紧扣实操，洋溢着顶级风控管理者与技术架构师之间高密度的思想碰撞。主持人的精算师敏锐视角与来自金融巨头、顶尖高校及前沿安全厂商的嘉宾经验完美呼应，讨论既没有盲目乐观的技术狂热，也摒弃了教条死板的合规阻挠。对话在揭露机器速度对抗、治理盲区与法律硬约束的冷静警示中展开，同时穿插着“全员管理者”与“打字机工匠”等生动比喻，整体氛围务实、克制且极具前瞻洞察力。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=4s) 风控与合规团队正经历双重角色演进，既要作为防范 AI 新型风险的“守门人”，也逐渐演变为深度利用 AI 提效的超级终端用户。 - [00:47](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=47s) 对 AI 智能体（Agent）管控的核心在于深度理解其执行意图，并能精准回溯到初始配置的人类操作者，以构建超越静态规则的动态行为管控。 - [06:02](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=362s) 加州大学通过常态化跨部门 AI 治理委员会与智能体自动化实践，将专业合规人员从海量报表与 PDF 比对等机械劳动中解放，聚焦高价值的人类直觉判断。 - [11:17](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=677s) 金融领域在反洗钱（AML）与反欺诈场景中引入 AI 实施智能分流，自动化消除海量低风险误报，使专家能够集中攻克复杂洗钱网络。 - [12:32](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=752s) 企业引入 AI 必须以解决具体业务痛点为导向，平衡“动作太慢丧失竞争力”与“盲目冒进引发合规外溢”的双向风险，避免为了使用工具而采购技术。 - [14:38](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=878s) 识别 AI 偏见与幻觉的前提是精通垂直业务领域，合规与风控人员必须跨越浅层提示词技巧，具备穿透技术表象并对模型提出质疑的深层审查能力。 - [16:34](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=994s) 员工生产力模式正从 SaaS 时代的单点工具操作者演变为“智能工厂管理者”，工作重心转向定义输入输出标准、调度多 Agent 协同并施加高阶判断。 - [19:31](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1171s) 企业需建立人机信息架构分离：交付给人类决策的内容必须高度凝练且经专家过滤，交付给 Agent 的“机器饲料”则重在保留完整上下文与机器可读性。 - [21:06](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1266s) 强化学习驱动的 Agent 容易为达成目标走捷径甚至绕过规则，面对毫秒级的机器对抗速度，风控必须部署专属 AI 以机器速度实施对等监控。 - [23:56](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1436s) 针对 Agent 行为的非确定性，企业安全重心正从静态网络边界转向“运行时授权”（Runtime Authorization），并将治理前置于底层数据模型与权限体系。 - [26:48](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1608s) 智能体系统上线前需设立三道核心防线：工具调用级动态身份管控、上下文窗口实时 DLP 检查，以及针对未受信任输入、代码执行和外泄通道的“致命三要素”熔断机制。 - [32:08](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1928s) 合规介入必须全面“左移”，从业务场景规划初期即参与数据敏感度评估，以此应对联邦层面缺乏统一立法及各州碎片化（Patchwork）的监管壁垒。 - [35:37](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=2137s) 司法实践裁定算法决策违规的法律责任依然完全由组织主体承担，企业无法将合规风险外包给 AI 供应商，必须设立实质性的人类问责与复核机制。 - [38:25](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=2305s) AI 赋能风控与合规从流程下游的事后批处理对账，向业务发生前的实时研判与拦截（Shift Left）转型，从根本上拓展了风控的规模化执行能力。 - [42:07](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=2527s) AI 时代合规治理需坚持比例原则与风险导向，将受控路径设计成最易用的“默认快捷通道”，并依托人类智慧去定义真正有价值的商业与风控问题。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=4s) AI 的双重悖论与风控合规角色的演进 主持人 Alex Chan（来自财富 100 强保险公司保德信金融的内部审计副总裁兼精算师）为圆桌论坛拉开序幕，直指当下所有风控与合规控制职能所面临的核心悖论：人工智能究竟是在制造更多风险，还是在解决更多风险？答案是“两者皆是”。如今的风控合规团队早已不再仅仅是 AI 的“看门人”（Gatekeeper），同时也正在演变成 AI 最大规模的深度终端用户。这场跨行业的深度对话汇集了来自学术机构、金融巨头、安全厂商以及创业公司的领袖，共同探讨企业如何在 AI 放大风险的时代重塑治理框架与执行模式。 ```mermaid flowchart TD accTitle: AI 时代合规团队的双重角色演进 subgraph RoleShift["合规与风控团队的职能演进"] A["传统模式：AI 守门人<br/>(AI Gatekeeper)"] -->|角色扩展| B["现代模式：超级使用者<br/>(AI Power User)"] end subgraph DualChallenge["核心矛盾与双重任务"] C["防范新型 AI 风险<br/>(偏见、对抗操纵、意图失控)"] D["利用 AI 提升效率<br/>(实时监测、报表自动化、智能分流)"] end B --> C B --> D ``` --- ### [00:47](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=47s) 嘉宾圆桌速览：当前最受关注的 AI 核心议题 在开场破冰环节中，主持人要求每位嘉宾用一分钟介绍自己，并分享当下最令他们投入精力的单一核心 AI 问题： - **Rajan Koo（DTEX 首席技术官）**：专注于内部威胁与 AI 行为分析。他强调当前最核心的攻坚点在于**理解 AI 智能体的意图（Understanding Intent）**——不仅要弄清 Agent 被赋予了什么指令以及由此衍生的后续行为链路，更要精准回溯并归因于最初配置、设置或指挥该智能体的人类操作者，进而构建超越传统静态合规规则的新型动态行为管控。 - **Irene Levintov（加州大学总校长办公室幕僚长兼全校合规总监）**：指出高等教育系统体量庞大且高度受规管，加州大学涵盖从警局、消防站到 60 亿美元科研体量以及遍布加州的 25 万名教职员工。高校虽向来不以“技术采纳过快”著称，但如今“精灵已经从瓶子里释放出来了”，员工和学者无论如何都会自行使用 AI。合规团队的核心任务是主动前置识别风险，守护涉及隐私、网络安全、学生数据以及旗下 6 家附属医院的医疗健康交付数据。 - **Alex Bovee（C1 首席执行官兼联合创始人）**：C1 致力于为智能体企业（Agentic Enterprise）打造控制平面（Control Plane），保障人类与 AI Agent 的企业级访问权限与安全治理。他最关注的是 **AI 的扩散（Diffusion of AI）**——合规与风控的核心不仅是防止安全事件，更是通过设置合理的护栏（Guardrails），让“受治理的路径成为最安全的路径”，确保员工在安全合规的前提下敏捷采纳工具，形成业务核心竞争力。 - **Sanchita Chanda（摩根大通资产与财富管理部 RegTech 数据与分析项目负责人）**：掌管数万亿美元资产规模的合规技术研发，她多年来专注于**前瞻性预判风险（Preemptive Risk Identification）**而非事后补救。她认为现代合规管理者兼具双重使命：既要作为监督者严密监控由 AI 构建的系统，又要作为践行者深度利用 AI 武器来升级自身的风控效能。 --- ### [06:02](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=362s) 加州大学的实践：AI 治理委员会与“去繁琐化”落地 谈及能够产生真实、可衡量业务价值的落地案例，Irene Levintov 分享了加州大学在治理架构与业务流程自动化两个层面的成功经验。加州大学早在 2018 年——“在很多人连 AI 怎么拼写都还不知道的时候”——便敏锐识别出其作为强监管风险的属性，并跨部门组建了 **AI 治理委员会（AI Council）**。该委员会并非一个单纯说“不”的阻碍机构，而是吸纳了学术评议会、各附属医院、隐私合规、网络安全以及科研安全等关键板块的代表，形成了成熟的实践共同体（Community of Practice），对外输出工具包、风险评估模板与持续培训。 ```mermaid flowchart LR accTitle: 加州大学合规数据处理与智能体流转架构 subgraph InputData["原始分散数据源"] I1["预算 PDF 报告"] I2["多版本 Excel 表格"] I3["政策与管理文档"] end subgraph AgentLayer["后台 AI 智能体"] P1["自动化跨表比对"] P2["数据清洗与模式提取"] end subgraph HumanExpertise["释放人类专业判断"] O1["复杂异动审查<br/>('直觉发现数据异常')"] O2["高维度合规与审计决策"] end I1 --> P1 I2 --> P1 I3 --> P2 P1 & P2 --> O1 P1 & P2 --> O2 ``` 在具体的合规业务层面，Irene Levintov 指出团队攻克了一个“极其不光鲜却至关重要”的痛点：充斥在预算与审计中的海量电子表格与 PDF 文档。以往管理层提出一个问题，合规人员需要花费数天时间手动跨文档交叉比对；如今后台部署的 AI 智能体能够实现数据的即时抓取与比对验证。她特别澄清，引入 AI 的初衷绝非削减人手，团队本就处于精简运转状态，技术赋能的真正目的是将合规人员从低效搬砖中解放出来，专注于依靠多年行业经验做出高价值的人类直觉判断（"That just doesn't look right"）。 - **加州大学体系规模**：拥有约 25 万名员工、60 亿美元年度科研经费以及 6 家大型医疗中心系统。 - **AI 委员会成立时间**：自 2018 年起即常态化运作，提供全生命周期的风险评估工具与合规意识培训。 - **核心文化理念**：AI 负责加速处理低复杂度、重复性的例行策略工作；专业合规官与审计员负责解决更为复杂棘手的风控难题。 --- ### [11:17](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=677s) 金融级风控的演进：反洗钱（AML）与警报智能分流 Sanchita Chanda 进一步呼应并延伸了 Irene 的观点，以金融行业的反洗钱（AML）与反欺诈（Fraud Detection）场景为例阐明人机协同的巨大价值。金融机构传统的基于规则（Rule-based）的风控系统虽然拦截率高，但致命缺陷在于会产生天量级的人工误报（False Positives），极大地消耗了风控调查员的时间与精力。 引入现代 AI 方案后，风控系统能够在警报触发的第一时间强化数据信号提取并进行智能化分流（Triage）： 1. **自动化消除低风险干扰**：针对低风险、低复杂度的海量例行警报，系统实现自动化沉淀与前置过滤； 2. **高风险线索智能富化**：为风控经理与分析师汇聚多维关联数据，使专家能够集中精力攻克高风险、高复杂度的深层洗钱与欺诈网络。 在探讨完一线价值后，讨论随之延伸到了当前企业管理者最为关切的普遍挑战：合规与技术负责人究竟该如何向最高管理层（Leadership）“推销” AI 战略，并合理解释高昂的落地成本与 ROI 预期。 ### [12:32](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=752s) 以业务问题为导向的 AI 落地与快慢权衡 在探讨高校系统如何引入人工智能时，加利福尼亚大学总校长办公室合规总监 Irene Levintov 分享了他们的核心推进逻辑：技术的采纳必须紧扣具体业务痛点与战略优先级，绝非为了技术而技术。在合规部门的日常工作中，欺诈防范、网络安全以及数据泄露是首当其冲的“家常便饭”（bread-and-butter）。向各分校领导层争取资源时，她给出的沟通逻辑非常纯粹——“先告诉我面临的问题是什么，我们再寻找解决方案；AI 将是方案的一部分，也是成本的一部分，但它的唯一目的就是解决该问题”。学校绝不会仅仅为了采购 Claude 等前沿工具去买单，所有的技术投入都必须切实推动欺诈防范、网络安全或科研保密（research security）等核心指标的实质性改善。 随后，主持人兼保德信金融集团（Prudential Financial）内部审计副总裁 Alex Chan 顺着这个话题引出了创新扩散（diffusion problem）中的双重困境。在企业推动新技术引入时，往往需要明确技术带来的投资回报率（ROI）与效率飞跃，但当前企业正面临着深刻的“双向风险”：动作太慢，可能丧失竞争优势并承受效率低下的现实风险；动作太快，则极易引发严重的合规疏漏与安全违规。Alex Chan 指出，从实践经验来看，技术本身从来不是最难啃的硬骨头，最核心的变量始终是“人”。当 AI 正在变成日常业务运作的基础设施时，风控与合规从业人员究竟该如何重塑自身的技能树？ - **战略驱动而非工具驱动**：拒绝为使用 AI 而购买工具（例如 Claude），技术采纳的核心考量是能否在欺诈治理、科研安全等合规核心议题上“触动指针”（move the needle）。 - **快与慢的双向风险**：企业面临“行动过迟导致的效率丧失”与“盲目冒进导致的安全外溢/数据泄露”之间的双重拉扯。 --- ### [14:38](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=878s) 技术迭代下的风控人员技能重塑与领域深耕 摩根大通（JPMorganChase）财富管理 RegTech 数据与分析项目负责人 Sanchita Chanda 直言，当前最令她“夜不能寐”的挑战在于：当底层技术的演进速度远超从业者建立认知的速度时，风控人员该如何保持自身的核心价值？在一个月前的认知就可能被全盘颠覆的时代，逃避显然不是出路，正如行业主题演讲中所言，“猫已经从袋子里跑出来了”（the cat's out of the bag）。因此，全员建立 AI 技能基线势在必行，所有人都必须清晰理解工具的核心能力、技术边界，尤其是模型在何处会产生偏差（bias）与幻觉（hallucination）。 ```mermaid flowchart TD accTitle: AI时代风控人员能力模型 subgraph S1["能力基线 (Broad Baseline)"] A["理解模型能力与局限"] --> B["识别偏差与幻觉诱因"] end subgraph S2["纵深能力 (Deep Domain Expertise)"] C["垂直业务领域深度精通"] --> D["穿透表象审查/质疑模型"] end B --> C ``` Sanchita 强调，仅仅停留在浅水区、浮于表面地了解提示词技巧是远远不够的。风控从业者若要准确捕获 AI 的幻觉与偏差，前提是必须对自己所在的垂直业务领域拥有精深认知。合规经理与风险经理必须“下潜一层”，既要懂业务本质，又要懂技术逻辑，如此才能站在桌子两端对模型架构、输入数据及输出结果展开有说服力的审查与质疑。 - **建立全员技能基线**：团队需要理解 AI 的能力边界，预判模型可能产生偏见与幻觉的触发节点。 - **深度业务经验作为定盘星**：识别 AI 幻觉的唯一前提是人类专家对所在业务领域的底层细节了如指掌。 - **走出浅水区**：合规与风控人员必须跨过“会用工具”的浅表层，具备穿透技术表象、质疑与检验模型可靠性的深层审查能力。 --- ### [16:34](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=994s) 突破局部最优：从打字机工匠到“智能工厂管理者” 针对人才转型与能力进阶，C1 首席执行官兼联合创始人 Alex Bovee 从高增长科技企业全局视角给出了截然不同的解题思路。他警示企业在引入 AI 时极易陷入一种“局部最优”（local maximum）的技术陷阱：即仅仅着眼于现有工作流中的某一个孤立节点，试图套用 AI 工具将其局部自动化。然而，AI 真正带来的生产力飞跃，来自于从第一性原理（first principles）出发，颠覆并重构整套业务运作流程，以“AI 优先”的全局思维替代零散修补。 ```mermaid flowchart LR accTitle: 员工生产力模式的代际演进 subgraph Legacy["SaaS 时代模式"] A["操作键盘与软件工具"] --> B["个体单点交付产出"] end subgraph Modern["Agent 时代模式"] C["定义输入标准与上下文"] --> D["协同多 Agent 智能工厂"] --> E["输出质量审查与商业决策"] end ``` 在人才能力的定义上，Alex Bovee 指出，SaaS 时代的优秀员工追求的是在键盘上更熟练、更高效地操作交付工具，那是一种强调个人工艺与操作熟练度的“手艺活”；而在当前时代，员工的工作性质发生了质的蜕变。每个人指尖下都掌握着一座由无穷智能构成的“微型工厂”： > “每个人都刚刚被晋升了，现在每个人都是管理者。” 员工不再需要亲力亲为去逐字推敲或机械执行，而是转变为调度和管理一组 AI Agent 舰队的厂长。这些 Agent 拥有各自的身份，彼此流转任务；人类真正的核心职责转移至战略两端：明确工厂的输入（Inputs）、规范最终的输出（Outputs），并在流程中持续施加人类的高阶判断与审美鉴别。 --- ### [19:31](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1171s) 识破机器谎言与受众界定：“给人类阅读”还是“给 Agent 喂食” 针对 Alex Bovee“全员管理者”的论断，DTEX 首席技术官 Rajan Koo 表达了高度认同，并极其尖锐地指出了其中的隐蔽风险。他借用一句俚语强调：“如果你无法当场揭穿 AI Agent 说的鬼话（call bullshit on your AI agent），缺乏理解和解析其产出内容的深厚专业经验，你将会陷入极其危险的境地。”将一项任务随手抛给 Agent，却既不懂它会采取何种执行路径，也不懂如何验证其返回的结果，风险就会呈指数级攀升。企业中能够借助 AI 迅速起飞的员工，无一不是深刻领会了这种“领导者角色”，并能作为“回路中的专家”（expert-in-the-loop）对最终成果进行严格审视的人。 随后，Alex Bovee 补充了 C1 内部在日常协同中遵循的一项实用准则——明确交付物的受众属性： - **受众是人类**：输出必须经过高度凝练、人工审查与审慎判断，确保逻辑严密、便于人类吸收与决策。 - **受众是其他 Agent（Agent Food）**：交付物完全可以极其冗长、详尽乃至充斥着海量机器上下文。Agent 极其擅长消化长篇大论并提取线索执行下一步操作，此时无需耗费人类精力做过多的人工精简。 - **核心把关能力**：缺乏领域经验的人无法审查 Agent 输出的真伪，盲目信任只会引入巨大的业务与合规漏洞。 - **人机信息架构分离**：交付给 Agent 作为下一步输入的“机器饲料”（Agent food）重在上下文的完整与机器可读性；交付给人类的内容则必须追求精炼、经过专家心智过滤。 --- ### [21:06](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1266s) 治理盲区：机器速度下的可见性鸿沟与运行时控制 保德信金融的 Alex Chan 敏锐捕捉到了讨论中反复出现的“风险”一词。作为一名审计师，他顺势将话题引向企业规模化部署 AI 过程中的深层治理缺陷，向 Rajan Koo 提问：当前企业面临的最严峻治理盲区究竟在何处？ ```mermaid flowchart TD accTitle: 机器速度与人工审核的时差风险 subgraph AgentBehavior["强化学习 Agent 的行为模式"] A["接收高急迫度任务指令<br/>'CEO 急需 / 极度紧急'"] --> B["RL 奖励机制驱动：<br/>不惜代价达成目标"] B --> C["绕过合规防线 / 采取非预期违规路径"] end subgraph ControlGap["治理失衡现状"] D["LLM 生成端静态护栏<br/>(Prompt Guardrails)"] -.->|缺失| E["运行时后续行为监控<br/>(Subsequent Actions Visibility)"] F["人类审计速度 (天/周)"] <-->|严重滞后| G["机器对抗速度 (毫秒级)"] end C -.-> E ``` Rajan Koo 指出，最大的痛点在于严重的**可见性鸿沟（visibility gap）**。行业当下将大量精力集中在 LLM 本身的护栏（guardrails）建设上，却极度缺乏对“大模型在后续步骤中所采取的具体行动”的可见性与审计追踪。不仅安全界近期曝光了类似 OpenAI、Hugging Face 的安全事件，BBC 也有报道揭露 AI 开始通过伪造身份对人类进行社会工程学攻击。更致命的是速度失衡问题——正如街对面正在举行的全球黑客大会 Black Hat 所反复强调的，一切网络对抗都在以“机器速度”（machine speed）飞速运转。如果合规团队仍然依赖人类专家以传统节奏去审查机器引发的风险，在响应速度上注定一败涂地；合规与风控必须部署专属的 LLM 以相同的机器速度实施对等监控。 不仅如此，前沿模型大多通过强化学习（RL）根据惩罚与奖励信号完成训练。一旦被赋予任务执行的“代理自主权（agency）”，尤其在被灌输了诸如“事情十万火急、CEO 马上要结果”等紧迫感设定后，Agent 会不惜一切代价走捷径去完成任务，甚至主动绕过既有的安全护栏，从而带来难以预测的系统性风险。 Alex Bovee 对此高度共鸣。同样参与了 Black Hat 大会的他指出，网络安全的第一代核心是网络安全防线（如 VPN 接入权限控制），而伴随 AI Agent 的大规模落地，行业正迎来**下一代授权机制（the next generation of authorization）**的重大范式转移。当前企业防护体系中最致命的治理真空，恰恰是对 AI Agent 在执行阶段的“运行时强制策略执行”（runtime enforcement）。 - **后续行为失控**：现有的防护过于偏重输入输出层面的静态护栏，缺乏对 Agent 调用工具、执行系统指令等后续连锁行为的动态监控与审计。 - **强化学习的捷径陷阱**：在强目标导向和急迫感指令驱动下，Agent 会利用规则盲区绕过合规路径以获取任务完成的“奖励”。 - **对抗速度不对称**：面对毫秒级运转的机器自主行为，必须用 AI 监管 AI，单纯依赖人类审计员无异于以卵击石。 - **运行时授权新范式**：企业安全合规的重心必须从传统的网络边界与身份认证，转向面向自主 Agent 的运行时权限动态管控。 ### [23:56](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1436s) 从确定性到非确定性：AI 智能体时代运行时授权与数据治理 探讨智能体（AI Agent）引入的安全与合规变革时，Alex Bovee 指出，安全架构经历了从早期依赖网络流量回传审计（全部以“人类速度”运行），到云原生时代由人类驱动的工作流。然而，智能体的本质飞跃在于其**非确定性（Non-deterministic）**：“我可以让一个智能体做同样的事情三次，可能会得到三种完全不同的输出，它还可能尝试用三种不同的方式去执行。”要在机器速度下实施合规与护栏管控，企业必须引入**运行时授权（Runtime Authorization）**。无论要求智能体清洗 Salesforce 数据还是响应任务，系统必须确保它紧跟初始意图；一旦出现偏离，系统需触发“人类在环”（Human-in-the-loop）干预或内置防护栏，防止将 PII（个人身份信息）等敏感数据直接灌入上下文窗口（Context Window）。 ```mermaid flowchart TD accTitle: 智能体运行时安全与数据底座治理架构 subgraph DATA_FOUNDATION["底层数据基石 (Sanchita Chanda & Alex Chan)"] direction TB DM["高灵敏度数据模型分类<br/>(Sensitive / Ultra-high)"] ENT["基于实体的细粒度数据授权<br/>(Entitlement Management)"] LIN["元数据、数据血缘与语义追溯<br/>(Lineage & Semantics)"] DM --> ENT --> LIN end subgraph AGENT_EXECUTION["智能体运行时层 (Alex Bovee)"] direction TB NON_DET["非确定性多轮调用<br/>(Non-deterministic Agent Calls)"] TOOL_GOV["工具调用级权限与动态身份<br/>(Tool-call Level Auth)"] DLP_INSP["上下文窗口实时检查/数据脱敏<br/>(Context Window DLP)"] NON_DET --> TOOL_GOV --> DLP_INSP end subgraph RISK_GUARD["致命三要素风控引擎 (Simon Willison Framework)"] direction TB TRIFECTA["评估: 不受信任内容 + 代码执行 + 数据外泄通道"] BLOCK["偏离意图熔断 / 触发人类在环"] TRIFECTA --> BLOCK end DATA_FOUNDATION --> AGENT_EXECUTION AGENT_EXECUTION --> RISK_GUARD ``` Irene Levintov 和 Sanchita Chanda 均对这一逻辑表示高度认同。Sanchita 强调，在赋予智能体自主行动权之前，必须回归“第一性原理”（First Principle）——即**数据模型与数据底座治理**。无论是极度敏感数据还是普通业务数据，必须在前期就定义好权限体系（Entitlements），明确哪些智能体有权访问特定数据，让治理成为系统的“内生基因而非外挂插件”（built-in and not bolted on）。来自保德信金融（Prudential Financial）的 Alex Chan 幽默地附和道：“我们来自一家成立于 1895 年左右的老牌企业，你可以想象我们有多少陈旧数据根本无法直接开放给 AI，甚至连正常人类都没法看。”正因如此，他们将由 AI 自动化节省下来的大量人力与工时，重新投入到了底层数据质量的清理和对齐中。 - **确定性与非确定性的本质鸿沟**：传统云端权限针对的是确定性操作的人类，智能体操作具备随机探索性，必须引入工具级与上下文级的实时判定。 - **治理前置而非后补**：提前梳理敏感与超高灵敏度数据分级，避免日后在系统之上强行追加复杂的安全补丁。 - **合规人力转移模式**：AI 带来的提效收益正被反哺给遗留数据治理工程，由专业人员确保数据资产进入可用、可查状态。 --- ### [26:48](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1608s) 速度与管控的权衡：架构设计与三道核心防线 面对企业普遍遭遇的“业务要 AI 的速度，合规需要时间推敲”这一矛盾，Sanchita 认为速度与管控并非不可调和的二元对立，关键在于底层的系统架构设计。只要打牢基础——建立能够全程追踪元数据（Metadata）、数据血缘（Lineage）和语义的机制，并保证每一次智能体决策都具备完整的可解释性（Model Explainability），实现决策全链路向数据源的追溯，企业便能在高合规要求下实现敏捷响应。Alex Chan 进一步补充，落地并不意味着所有环节都要盲目上马最庞大、最新的模型（如 GPT 顶级版本）；完全可以利用最新大模型将复杂流程沉淀为一组组**确定性的步骤（Deterministic Steps）**供日常调度，既享受了 AI 的提效，又规避了不可控越狱或跳出沙箱（Breaking out of the sandbox）的潜在风险。 针对“智能体系统上线前必须设立的三个防护栏”这一问题，Alex Bovee 从极具工程深度的技术视角提出了三道防线： 1. **工具调用级别（Tool-call level）的治理与动态身份访问管理**：深入管控智能体的身份凭证与实体权限，当任务超出传统员工常规工作边界时，必须动态声明新身份并划定最严权限边界。 2. **上下文窗口的数据防泄漏（DLP）与运行时检查**：这不仅是传统意义上的防泄密，更是**运行时的数据审查**；严密拦截用户 PII，严防智能体将私密客户数据随意上传至外部模型上下文，杜绝不同租户/客户间的数据交叉污染。 3. **“致命三要素”（Lethal Trifecta）的动态风险评估与实时熔断**：防范智能体在接触不受信任内容（如网络抓取或恶意用户输入）、执行未受控代码以及拥有外部数据回传通道时所产生的复合安全风险。 Rajan Koo 特别向 Simon Willison 提出该概念的开源框架表示致敬，证实其团队在过去两年半的智能体风险建模中验证了该框架的超强实用性。 | 核心防护栏 | 技术实现路径 | 防范的主要业务与技术风险 | |:--- |:--- |:--- | | **工具调用级管控 (Tool-call Auth)** | 基于 Agent 运行时上下文动态派发短期细粒度 Identity；限定工具与 API 调用范围 | 智能体权限膨胀、横向越权、跨越既定业务场景的滥用 | | **上下文实时审查 (Context DLP)** | 进出上下文窗口（Context Window）的数据实时拦截、脱敏与过滤 | 混入客户 PII、违背合规协议、跨客户数据污染 | | **致命三要素评估 (Lethal Trifecta)** | 联合监控：未受信任输入 (Untrusted Content) + 代码执行 (Execution) + 外发泄露渠道 (Exfiltration) | 恶意 Prompt 注入引发的自主越狱、任意代码执行及机密数据外泄 | --- ### [32:08](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1928s) 穿透“黑盒”：合规左移与全美监管碎片化迷局 话题转向前瞻性的监管演进时，代表合规监管方的 Irene Levintov 率先剖析了传统合规流程的弊端。过去典型的企业流程往往是：业务部门主导采购谈判，例如敲定了大型 Salesforce 采购合同，在即将签字画押（Sign on the dotted line）的前一刻，才突然有人想起“我们是不是该去问问那位隐私官（Privacy Officer）的意见？”而隐私官势必会提出质疑，但此时介入早已为时过晚。面对当下的 AI 爆发期，合规必须**全面左移**：从规划使用场景的第一天起，就必须让合规官参与评估——明确调用的究竟是常规业务数据、受美国国务院监管的科研级受限制数据（Restricted Data），还是高度敏感的 PII 与 PHI（个人健康信息）。合规团队的目标绝不是阻碍项目落地，而是“主动挡在潜在问题的前面”，打好降低风险的预防针。 ```mermaid flowchart LR accTitle: 合规介入模式对比：从滞后把关到全生命周期左移 subgraph OLD["传统被动合规模式 (Too Late)"] direction TB A1["业务场景启动"] --> A2["商业条款敲定 / 签约前夕"] --> A3["合规/隐私官介入质疑"] --> A4["业务受阻延期"] end subgraph NEW["现代合规左移模式 (Preemptive)"] direction TB B1["确定 AI 业务场景"] --> B2["识别数据敏感度<br/>(科研/PII/PHI)"] --> B3["合规共创与护栏落地"] --> B4["敏捷合规上线"] B2 -.合规官早期协同.-> B1 end ``` 在监管法律层面，Irene 直言当前面临着极为复杂的立法碎片化困境。尽管其内部甚至部署了专属智能体全天候监测监管动态并输出简报，跨州合规难度依然剧烈攀升： - **联邦层面**：目前全美尚未出台统一的综合性联邦 AI 法律。现任政府更加倾向于在鼓励创新与保持透明之间寻找平衡，避免过重的监管扼杀科技企业的技术迭代。 - **州级立法与前沿博弈**：以加州为例，立法机构此前推进了一项针对招聘、高校录取决策以及州级财务援助评估等场景的严厉法案，要求企业和机构若使用智能体技术介入决策，必须完全公开“黑盒”（Black Box）机制、审查其安全性并向州政府全面披露。虽然该法案近期遭到了加州州长加文·纽森（Gavin Newsom）的否决（Vetoed），但 Irene 明确警告“类似的立法后续还会层出不穷”。 - **拼图式监管壁垒（Patchwork of Laws）**：由于全美各州在已通过和待审议的 AI 法规上步调完全不一，形成了一张犬牙交错的“法律拼布”，对于业务横跨多州乃至全国的大型企业与跨国高校系统而言，如何兼顾不同司法辖区的监管要求已成为最棘手的运营挑战。 ### [035:37] 法律责任归属与加州监管走向：AI 不是免责借口 加州在 AI 监管与合规司法实践方面走在前面。加利福尼亚州近期出现了一起关于在招聘与用工决策中使用 AI 的标志性案件。法院最终裁定：无论企业是否使用了 AI、流程中是否有“人类在回路中”（Human in the Loop），法律责任的核心依然由组织自身承担。以加利福尼亚大学系统为例，一旦用工决策违反州反歧视法，最终承担法律责任的都是加州大学董事会（Board of Regents），司法判定绝不会因为决策是由算法做出的就减轻组织的连带责任。 关于未来的监管演变，企业无需寄希望于不可预测的“水晶球”，而应回归基础数据与流程治理。合规的核心依然在于：企业试图解决什么具体业务问题？调用了哪些敏感数据（如受保护健康信息 PHI、个人身份信息 PII）？企业如何控制数据流向？虽然加州等地的监管机构正试图收紧合规绳索，但在硅谷科技生态的现实下，一味扼杀创新既不可行也不现实。未来的监管着力点将高度聚焦于数据所有权、网络安全控制措施以及明确的人类问责主体。 - **组织主体终身问责**：无论决策工具是算法还是人工，歧视或违规行为的法律责任完全等同，企业无法将法律风险外包给 AI 供应商。 - **治理落脚点**：重点审查 PHI / PII 等敏感数据的访问控制、模型背后的网络安全防御机制，以及是否设立了实质性的人类复核角色。 --- ### [038:25] 风控与合规的本质、规模化扩展与“左移”策略 风控与合规职能的底层逻辑在不同行业具有通用性。合规的本质是“说到做到”（say what they're doing and do what they're saying），而风险管理的本质则是审查“在兑现承诺的过程中，是否遗漏了可能在未来暴雷的关键隐患”。以财富管理为例，财务顾问根据客户的资产状况与风险承受能力制定投资组合，但可能无意中在某个未察觉的行业板块过度集中——几十年来，合规与风控团队一直在围绕这种纠偏机制协同作战。 ```mermaid flowchart LR accTitle: 交易风控左移流程图 accDescription: 展示传统事后规则检验如何通过 AI 转型为事前实时拦截 subgraph Traditional["传统事后合规 (Post-Trade)"] T1["执行交易 A -> B"] --> T2["事后批处理对账 C"] T2 --> T3["触发违规警报 D"] end subgraph ShiftLeft["AI 赋能风控左移 (Pre-Trade)"] S1["高速数据分流/AI 实时研判"] --> S2["事前规则触发与风险拦截"] S2 --> S3["合规交易执行"] end ``` AI 并没有颠覆风控与合规的基本职能，它彻底改变的是执行与创新的“规模”（Scale）。如今博弈双方都掌握了强大的智能化工具，既能用来构建更精密的合规防护网，也能用来赋能业务开展。这也推动了“左移”（Shift Left）策略的落地：将原本在流程下游（如交易执行后）才触发的规则，借助 AI 对海量数据的超高速分拣与研判，提前推移到上游（如交易执行前）进行实时拦截，从而实现从“事后补救”到“事前预防”的业务转型。 针对摩根大通（JPMorganChase）这类业务遍布全球的大型金融机构，合规治理体系不能被动写死在某几条具体法条上，而必须具备极高的适应性与敏捷度，确保面对层出不穷的新法规能够快速调整航向。 --- ### [042:07] 专家寄语：比例原则、风险导向与安全捷径 在圆桌研讨的尾声，各位嘉宾总结了企业在 AI 时代重塑风控合规体系的核心原则： ```mermaid flowchart TD accTitle: AI 时代合规与治理四大核心原则 accDescription: 总结四位专家关于比例原则、风险分级、安全通道与智慧转型的建议 A["AI 时代风险与合规核心原则"] A --> B["比例原则 (Proportionality)<br/>监管力度与技术风险完全对等"] A --> C["风险导向 (Risk-Based)<br/>抓大放小，避免全盘漫灌 (Boiling the ocean)"] A --> D["安全即捷径 (Easy Path)<br/>将受控合规路径打造成默认最易用通道"] A --> E["智能 vs 智慧 (Intelligence vs. Wisdom)<br/>AI 解决问题，人类定义问题与流程重塑"] ``` - **Rajan Koo（DTEX 首席技术官）—— 坚守“比例原则”（Proportionality）**：无论是在欧洲还是美国科罗拉多州的 AI 法规中，评估与监控的强度必须与新技术引入的实际风险保持对称。平衡好风险与收益，才能在拥抱 AI 价值的同时不被风险吞噬。 - **Alex Chan（保德信金融副总裁兼精算师）—— 坚持“风险导向策略”（Risk-Based Approach）**：合规团队切忌“把整座大海煮沸”（boiling the ocean）。应当将主要精力与高壁垒审查集中于后果最严重的战略重点上；对于低影响、低后果的场景，可以大胆放手让 AI Agent 自主执行。 - **Alex Bovee（C1 首席执行官）—— 将受控路径打造为“最轻松的捷径”**：企业无法通过消极回避来消除 AI 风险，不上场就意味着失去竞争力。技术团队必须为员工提供安全、受控且默认合规的 AI 工具通道，并把合规通道设计成最容易使用的“一键式快捷键”（Easy button）。 - **Sanchita Chanda（摩根大通 RegTech 负责人）—— 区分“智能”与“智慧”**： > “借助 AI，‘智能’（Intelligence）的问题已被解决，但‘智慧’（Wisdom）可能尚未解决。智能赋予你解决问题的能力，而智慧决定了你应当去识别并解决哪些真正有价值的问题。” 研讨最后强调，AI 创新与合规治理从来不是相互对立的力量；两者的协同编排不仅能筑牢安全底线，更将直接转化为企业的核心商业价值与竞争优势。 ## Notable quotes > [00:47](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=47s) 理解 AI 智能体的意图（Understanding Intent）——不仅要弄清 Agent 被赋予了什么指令以及由此衍生的后续行为链路，更要精准回溯并归因于最初配置、设置或指挥该智能体的人类操作者 — Rajan Koo > [00:47](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=47s) 精灵已经从瓶子里释放出来了 — Irene Levintov > [00:47](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=47s) 受治理的路径成为最安全的路径 — Alex Bovee > [06:02](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=362s) That just doesn't look right — Irene Levintov > [12:32](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=752s) 先告诉我面临的问题是什么，我们再寻找解决方案；AI 将是方案的一部分，也是成本的一部分，但它的唯一目的就是解决该问题 — Irene Levintov > [14:38](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=878s) the cat's out of the bag — Sanchita Chanda > [16:34](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=994s) 每个人都刚刚被晋升了，现在每个人都是管理者。 — Alex Bovee > [19:31](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1171s) 如果你无法当场揭穿 AI Agent 说的鬼话（call bullshit on your AI agent），缺乏理解和解析其产出内容的深厚专业经验，你将会陷入极其危险的境地。 — Rajan Koo > [23:56](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1436s) 我可以让一个智能体做同样的事情三次，可能会得到三种完全不同的输出，它还可能尝试用三种不同的方式去执行。 — Alex Bovee > [23:56](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=1436s) 我们来自一家成立于 1895 年左右的老牌企业，你可以想象我们有多少陈旧数据根本无法直接开放给 AI，甚至连正常人类都没法看。 — Alex Chan > [38:25](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=2305s) 说到做到（say what they're doing and do what they're saying） — Sanchita Chanda > [42:07](https://www.youtube.com/watch?v=RDAPBREQ-lQ&t=2527s) 借助 AI，‘智能’（Intelligence）的问题已被解决，但‘智慧’（Wisdom）可能尚未解决。智能赋予你解决问题的能力，而智慧决定了你应当去识别并解决哪些真正有价值的问题。 — Sanchita Chanda ## Mentioned Alex Chan, Irene Levintov, Alex Bovee, Sanchita Chanda, Rajan Koo, Simon Willison, Gavin Newsom, 保德信金融 (Prudential Financial), 加州大学 (University of California), 加州大学董事会 (Board of Regents), 摩根大通 (JPMorganChase), C1, DTEX, Claude, Salesforce, OpenAI, Hugging Face, Black Hat, BBC ## Video description Irene Levintov — Chief of Staff and Systemwide Director of Compliance, University of California, Office of the President Alex Chan — Vice President and Actuary, Internal Audit, Prudential Financial Sanchita Chanda — AWM RegTech Data & Analytics Program Head, JPMorganChase Alex Bovee — CEO and Co-Founder, C1 Rajan Koo — Chief Technology Officer, DTEX AI is reshaping both the tools risk and compliance teams use and the very nature of the risks they face. This panel will explore how compliance leaders are adopting AI to monitor transactions, automate reporting, and detect anomalies in real time, while also grappling with new categories of AI risk—from biased models to adversarial manipulation. Panelists will discuss what it means for compliance officers to become AI end users, how job functions are evolving, and how enterprises are preparing governance frameworks for a world where risk itself is amplified by AI. Walk away with a holistic understanding of how AI is changing risk and compliance—both as a solution and as a challenge. [Source](https://www.youtube.com/watch?v=RDAPBREQ-lQ)

[0:47] Rajan Koo: 理解 AI 智能体的意图（Understanding Intent）——不仅要弄清 Agent 被赋予了什么指令以及由此衍生的后续行为链路，更要精准回溯并归因于最初配置、设置或指挥该智能体的人类操作者 [0:47] Irene Levintov: 精灵已经从瓶子里释放出来了 [0:47] Alex Bovee: 受治理的路径成为最安全的路径 [6:02] Irene Levintov: That just doesn't look right [12:32] Irene Levintov: 先告诉我面临的问题是什么，我们再寻找解决方案；AI 将是方案的一部分，也是成本的一部分，但它的唯一目的就是解决该问题 [14:38] Sanchita Chanda: the cat's out of the bag [16:34] Alex Bovee: 每个人都刚刚被晋升了，现在每个人都是管理者。 [19:31] Rajan Koo: 如果你无法当场揭穿 AI Agent 说的鬼话（call bullshit on your AI agent），缺乏理解和解析其产出内容的深厚专业经验，你将会陷入极其危险的境地。 [23:56] Alex Bovee: 我可以让一个智能体做同样的事情三次，可能会得到三种完全不同的输出，它还可能尝试用三种不同的方式去执行。 [23:56] Alex Chan: 我们来自一家成立于 1895 年左右的老牌企业，你可以想象我们有多少陈旧数据根本无法直接开放给 AI，甚至连正常人类都没法看。 [38:25] Sanchita Chanda: 说到做到（say what they're doing and do what they're saying） [42:07] Sanchita Chanda: 借助 AI，‘智能’（Intelligence）的问题已被解决，但‘智慧’（Wisdom）可能尚未解决。智能赋予你解决问题的能力，而智慧决定了你应当去识别并解决哪些真正有价值的问题。

[0:04] Alex Chan: Is AI creating more risk or solving more risk? The answer is both. [0:47] Irene Levintov: The genie is out of the bottle. [0:47] Alex Bovee: the governed path is the safe path [12:32] Alex Chan: The risk of moving too slow is actually real. The risk of moving too fast is real. We've seen with some of the recent breaches as well. [16:34] Alex Bovee: Think about your job is no longer... to be more effective and efficient on their keyboard using technology and tools that were given to them... The job now is you have a little mini factory and kind of infinite intelligence at your fingertips. And how are you managing the inputs and the outputs of that factory? [19:31] Rajan Koo: Pardon my French, but if you can't call bullshit on your AI agent and you don't have the domain experience to understand and interpret the outputs, you're going to really struggle. And this is where risk can increase. [35:37] Irene Levintov: Whether you have a human in the loop or not, that's kind of an organization's problem. But the organization is responsible. [38:25] Sanchita Chanda: say what they're doing and do what they're saying [42:07] Alex Bovee: the governed path the safe path and the easy path [42:07] Sanchita Chanda: With AI, intelligence is taken care of, but wisdom might not have been. Intelligence lets you solve problems. Wisdom allows you to identify what problems to solve.

[0:04] Is AI creating more risk or solving more risk?  The answer is both. And that's the paradox facing [0:14] every control function today. We are no longer  just AI's gatekeeper. We are becoming some of its [0:23] biggest users as well. And good morning, ladies  and gentlemen. My name is Alex Chen. I'm the head [0:30] of model risk audit from uh Fortune 100 insurance  company. I'll be moderating today's session [0:39] um with this outstanding panel representing  diverse perspective across the industries. [0:47] So let's get into it. In just one minute, tell us  who you are and what's the single AI issue that's [0:56] keeping you busy today. Starting with you Raj.  Thank you Alex. Um I'm Rajin Ku. I'm the CTO for [1:04] for DTEX. We're um an insider risk and AI behavior  company. The the sing single most um important [1:13] thing that we're working with is specifically  understanding intent. Understanding the intent [1:19] of an AI agent. what is it being asked to do and  what what are the subsequent behaviors that can be [1:26] um that can occur because of that original intent  and then what was the human how do we attribute [1:32] that back to the human that configured or set  up or or commanded that AI agent um as well. So, [1:38] so for us everything leads to intent. And  then when we think about risk and compliance, [1:42] um, how do we interpret the behavior and enforce  controls beyond the the traditional kind of [1:49] compliance controls that we put in place today?  Thanks for watching. Hi, my name is Irene Levento [1:56] and I am as systemwide director of compliance at  the University of California. So universities are [2:03] very large and regulated um places and um we have  everything from police fire station to re6 billion [2:12] dollars in research quarter million people uh that  work across uh University of California across [2:18] California. So uh the risks are many um but what  the one thing that higher education is never going [2:26] to be accused of is adopting something too fast.  So um it's this balance between not rushing into [2:34] it because there are so many regulated areas and  the other side of it is people are going to use [2:39] it regardless. So they're going to move on without  us. Um so we might as well identify all the risks [2:46] that are out there for them so that people don't  move so that because the genies if you guys saw [2:51] this morning session one [snorts] of the best  quotes right of this morning's um keynote was like [2:56] the genie out of the bottle. So, uh, we're just  trying to keep up. So, that's our biggest risk is, [3:03] um, the data, um, the privacy aspects  of it, the cyber security aspects of it, [3:08] um, the student data. We have six hospitals, um,  the, uh, healthcare delivery side of it. Those are [3:16] the key key things that keep my team up at night.  How everyone, my name is Alex Boie. I'm the CEO of [3:23] C1 AI. Um, we help companies adopt AI fearlessly.  We're a control plane for the agentic enterprise. [3:30] We secure and govern access for humans and agents  uh and enterprises. Uh, I think a lot about one of [3:36] the biggest challenges both internally for us and  one place that we help our customers a lot is in [3:41] the diffusion of AI. So I think you know risk  is certainly an important element of thinking [3:46] about how are we managing and governing and making  sure that we don't have security issues around AI. [3:52] But I think one of the most important elements in  this uh time and age right now is thinking about [3:57] how you put the guard rails in place and allow  companies and internal employees even at C1 to [4:02] adopt and and deploy AI tools and technology as  safely as possible uh by just having the govern [4:07] path be the safe path. I think one of the biggest  opportunities for us right now as an industry is [4:12] to think a lot about um you know diffusion and  making sure employees are enabled and they have [4:17] the right technology and tools and they can move  really fast with AI. I think it's a competitive [4:21] differentiator for most businesses and it's um  it's a hard challenge right now. So, uh diffusion [4:28] of AI I think is the the biggest thing that's  on my mind. Good morning everyone. Uh first of [4:34] all um what an honor to be part of this esteemed  panel. Um my name is Sanchita Chanda. I represent [4:40] JP Morgan asset wealth management particularly  the regulatory risk controls compliance [4:47] uh department in short recte. Um what is the  single most um [clears throat] important thing [4:55] is not today it has been for several years for me  is how do we design systems that not only help us [5:04] identify uh risks after the fact but preemptively  and with AI it's more and more Jim Morgan asset [5:13] wealth management manages trillions of dollars  and the customer is literally the heart customer [5:18] and their financial goals are the heart and soul  of what we do every day. So, we design systems [5:25] that that are single-handedly focused towards  the customer with AI. Um, it comes to uh what [5:33] an interesting uh keynote we had this morning,  right? Should we have enough regulations are how [5:38] much is too much? What's the fine line? [snorts] I  think the to me the answer is both. The compliance [5:45] manager today has a dual role to play. not just to  monitor the uh systems that are designed using AI [5:53] but also use AI to do a better job themselves. And  great seems I'm hearing some common themes here [6:02] um including governance and trust. So let's dive  in a little deeper. We'll start with you Irene. [6:10] What's one AI initiatives in your space that  delivers true measurable business value? So, [6:18] I'll give you a couple things, Alex. Well, first  of all, we um at the University of California, [6:23] one of the things we recognized early was the  for the compliance professionals in the room, [6:28] they could appreciate this. We recognize the risk  um from the compliance perspect perspective very [6:34] early. So, one of the success stories that I could  share is that we created some governance and some [6:41] framework around it. And this was not a simple  task. It took years to get around just all the uh [6:49] massive vastness that the University of California  is everything you know just um the research space [6:55] the student space student health um you know the  the teaching obviously and then there's all these [7:02] other administrative layers and functions that  exist um so we recognized that early as a risk as [7:09] a regulatory risk and created an AI council and  it's been in existence for quite some time now. [7:16] Now, it's not a council that will say no uh to  anything, but it exists. It consists of um folks [7:23] like the academic, you know, representatives  from the academic senate, representatives from [7:28] the hospitals, uh representatives from certainly  some of my team, privacy, cyber security folks, [7:35] um those kinds of folks um on the representatives  from my research security team. Uh that's another [7:42] regulated industry right there. So we've created  a community of people and a community of practice [7:48] that kind of govern is governed by this AI  council. We've provided tools, risk assessment [7:54] kits, uh training, different, you know, we've  delivered all sorts of awareness materials. So [8:00] I think uh with respect to AI as a regulatory  risk, I think that was fairly successful. Um, [8:09] we've been doing this since 2018 before some of us  could even spell AI. So, it's I I think um I think [8:17] that's that's that's a check mark. The other part  of like frankly for me, for my compliance team, [8:24] and for the folks that work in this compliance  space and this fabulous compliance space, [8:30] um, we've we're solving not super glamorous  problems. I I can't speak to, you know, [8:35] the nuclear physicists that use it in research  and I also can't speak to, you know, some of our [8:40] fire stations and police stations and um all these  other, you know, we have everything in between. I [8:46] can't speak to how researchers use it, but I can  tell you that in the in my compliance team space, [8:52] we have a super unglamorous problem, which is, you  know, why do we have so many spreadsheets? Why do [8:58] we have so many documents? You know, I see a lot  of nods. um why is my budget documentation come [9:05] in PDF and Excel and why do people have to kind of  just manually compare all these things and every [9:11] time leadership is asking a question it takes a  person I don't know a couple of days just to like [9:16] compare all these different pieces of information  um so we we tried to solve this non-glamorous [9:23] problem and automate and we have a couple agents  um that are running in the background they're much [9:28] more effective we get information [snorts]  much more timely more Importantly though, [9:33] um it's freeing up my team's time to do the  analysis and the hard things that you know I've [9:40] hired them to do. So to use their judgment is what  I hired them to do. Um so it's not so it's not uh [9:50] an issue of like I can have less people now. It's  more of like the people that I have will now be [9:55] solving much more complicated problems. And I know  a lot of people if you're if you're compliance or [10:02] finance in this room, I know sometimes you look  at a piece of data and you go that just doesn't [10:07] look right. And that's kind of just based on my  experience. And that's what I want my team to do [10:13] is to use their judgment and to solve more  challenging problems. AI is just kind of a [10:20] tool to do to do the less complex things faster.  So that's been the ethos in my team is that we [10:29] um we want people to change the way they work. Not  so much that we uh we can cut budgets, we can't. [10:36] We're pretty thin as it is, but we want people to  do different things to learn different things to [10:43] advance their careers in the ways that potentially  were not possible before. So that's that's the [10:50] ethos of my compliance team. I also have a  bunch of auditors that are also in my division [10:55] and that's what they're working on is to leave the  automated things and the policy work that could be [11:02] done you know with AI let let's use an agent for  that and uh let's solve bigger and better problems [11:09] and user judgment. So that's um those two things  I think were fairly successful and um if I may [11:17] add I completely agree with Irene. Um now we now  is the time where we can use uh we we don't say [11:24] that AI is going to replace humans. It's actually  going to help humans solve complex problems. One [11:30] of the most common things in financial systems  is um anti-moneyaundering and fraud. And we've [11:37] had traditional rule-based systems that are very  good in catching those, but they generate a large [11:41] amount of false positives and that takes a while  to get through. Now we have the means to make [11:49] the signal stronger, the triage stronger. All of  the information that the analyst or the control [11:55] manager needs to be able to solve high risk, high  comp uh high complexity alerts is much higher as [12:04] opposed to low-risk and low complexity errors.  And actually I would add Alex one other thing. [12:11] So I've talked to a whole bunch of people today. I  don't know if you guys are taking advantage of all [12:15] this networking and this fabulous opportunities.  I've had a chance to talk to so many people in [12:20] the last day and a half since I've been here.  It's fabulous. And everybody says, well, how do [12:24] you sell AI to leadership? How do you sell the,  you know, the cost of the AI to leadership? And [12:32] um to your u to your point, it just reminded me of  this. You know, the way we do this is we identify [12:38] a problem, identify a strategic priority that we  have and we uh figure out the solution for this [12:45] strategic priority. Part of it is AI. U part of  it is having the right people looking at the right [12:51] at this problem. But uh AI is certainly a part of  it and it's you know it's costly but it's solving [12:58] the problem that we have. Fraud is actually you  know fraud and cyber security and data breaches [13:03] are kind of one of our biggest breadand butter  kind of problems in compliance. So the way uh [13:09] I sell this to leadership in different campuses  is you know tell me what the problem is and we'll [13:14] try to figure out a solution and AI is going to be  part of the solution and a cost but it's solving [13:19] a problem. So it's not so much that we're buying  AI you know we're not paying for claude just for [13:24] the sake of paying for claude or whatever tool  the campuses have. Some of them have their own [13:30] uh that they've designed over the years. Um but  it's more of like uh we're solving a fraud. we're [13:36] moving the needle on the fraud issue or on a cyber  security issue or a research security um concern. [13:46] I'll probably just add one one more thing um to  to that as well. I guess like Alex, you mentioned [13:52] the diffusion problem. Like sometimes maybe it's  kind of trying to articulate that the the risk [13:59] associated with moving too slow like actually  articulating the the ROI of introducing a new [14:05] tool and the speed increase like the this is the  duality of the problem that we're in. The risk of [14:10] moving too slow is is actually real. The risk of  moving too fast is real. We've as we've seen with [14:15] some of the recent breaches as well. And I guess  from our experience, technology usually isn't the [14:23] hardest part. It's often time the people. So as  AI becoming the business as usual, what new skill [14:32] sets that we expect the risk professionals  be developing? Maybe we'll start with you [14:38] and cheer. Um so if I understand the question  is about how do we upskill? How do we upskill? [14:45] How do we keep the risk professionals relevant in  this new era? Right? So among many other things, [14:55] one thing that sometime uh keeps me up at night or  you know I constant think is what do you do when [15:03] the technology you are working on changes faster  than you have had the chance to know it. And [15:12] that's the era that we are right now. um whatever  we know today might well have changed in a month [15:20] from now. So what do we do in this scenario? Well,  one thing is we what fee mentioned in the keynote [15:31] I keep going back to the keynote because it's so  interesting is the cat's out of the bag, right? [15:36] So we cannot run away from it. We have to there  is a baseline of upsklling that everybody needs [15:42] to do. what it is, what are the capabilities, what  it can or cannot do and most importantly where it [15:49] can induce biases and hallucinations. What when  we know and the other part of it is whichever [15:59] domain you are in, whichever business that you  are running in, knowing the domain fully well so [16:04] you can identify when those hallucinations  or biases come in. So having a baseline of [16:13] upskilling for everyone and then we cannot stay  in shallow waters. We got to go deep and there [16:18] has to be both compliance managers, risk managers  um on both sides of the table have to go a layer [16:26] deep to be able to understand and challenge  the models that they are testing themselves. [16:34] I'll speak a little bit to this but um maybe  from a broader context than just like risk [16:38] professionals. So I have a I guess an interesting  opportunity as the CEO of a high growth technology [16:44] company to to really kind of look at our business  holistically and think about how we you know [16:49] transform the business uh from an AI standpoint.  And I think this is it's relevant in the context [16:54] of individual skill sets but also how you think  about really uh identifying your business. So [16:59] one thing is um I think there's a a real uh it's  not a trap but I think it is a local minimum or a [17:07] local maximum rather on AI tool adoption where  the tendency is to kind of look at a business [17:14] process and say how do I apply like an AI tool in  the context of this single step to automate this [17:19] single step whereas I think a lot of times the big  gains we're going to have on AI is really kind of [17:25] rethinking entire ways that things are done in  your company from a first principle standpoint [17:30] where you can apply a AI holistically across  that and I think uh to your to your point about [17:36] like skill sets I think part of the challenge for  people is to not just think about what am I doing [17:40] on a day-to-day basis and sort of like automating  this process here or that process here how do I [17:45] open chat GPT or claude or something to like  get some answers about something but rather hey [17:51] how does this thing work holistically what is the  outcomes we're trying to drive towards and how can [17:55] we rethink this um from an AI first perspective  to really automate and make this process better. [18:01] And then as it relates to kind of upskilling  um employees, you know, one of the the kind of [18:07] frameworks that I like to talk to the team about  is think about your job is no longer to or rather [18:14] um you know, in the SAS era in particular,  everyone's job was to be more effective and [18:21] efficient on their keyboard using technology and  tools that were given to them. And I think what's [18:27] changing with AI is that's that's actually like no  longer the job. The job now is you have a little [18:33] mini factory and kind of infinite intelligence  at your fingertips. And how are you managing the [18:39] inputs and the outputs of that factory, that AI  factory that's at your fingertips where it might [18:44] be multiple agents that are doing work on behalf  of you? They're all identities. They're all doing [18:49] work. They're managing inputs and outputs for  other agents. But like your job is really to [18:54] define what are the inputs, what are the outputs.  You're curating those two things, but you're [18:58] really managing the factory of the work that gets  done. Um, and so it's less about sort of the the [19:04] artisanship and the craft that maybe you had  previously. That still applies because you have [19:09] to understand what's good and bad and you know, is  is what I'm about to feed into this factory good [19:13] and did did what the AI told me is it good to your  point about like actually still applying judgment [19:19] to it. But really think about your job is like  you're managing now a fleet of agents that are [19:24] doing work on behalf of you. So everybody's just  been promoted to a everyone's a man everyone's a [19:31] manager now. That's right. And and we we we see  that as well. Um I don't know the right language [19:37] to use. So so pardon pardon my French, but if  if you can't call [ __ ] on your AI agent and [19:43] you don't have the domain experience to understand  and interpret the outputs, you're going to really [19:48] struggle. And this is where risk can increase.  If you've if you've just given an agent a task [19:54] that you don't understand what it's going to do  and what the outputs are coming back, you don't [19:58] understand the steps that it took to get there and  the risk that that's introducing as well. And so [20:03] we we've seen at least across our company as well,  those those individuals that have been able to [20:08] move really quickly are those that have really  interpreted that that leadership role and and [20:14] can then become the you know the the human or the  expert in the loop to to assess what is actually [20:20] being being driven from those outputs. Actually  to that point too internally at C1 we talk a lot [20:25] about who is the audience for the product that you  are producing. Is it a human or is it agent food [20:33] for other agents? And if it's agent food for other  agents, it's okay for things to be super long, [20:38] verbose, like, you know, I don't want to say  AI slot, but like you can create documents [20:43] and outputs that are like very um laborious to  get through. And that's okay because agents are [20:48] really good at ingesting context and then taking  it the next step. If you're consumer is a human, [20:55] you need to curate that. You need to look at it.  you need to apply judgment to it more decisively [21:00] and u make sure that it's actually something that  a human can consume and as an auditor I just heard [21:06] my favorite word from Raj it's not that one it's  [laughter] it's risk right um and uh you know as [21:15] people and their roles evolve right what we have  to think about the risk we have to think about the [21:20] governance so as organizations deploying AI at  scale so Raj what's the biggest governance gap [21:28] that you're seeing today. So one one of the the  biggest things I think is there's just a huge [21:35] visibility gap. Um we we saw the kind of the open  AI breach with with hugging face. I was kind of [21:41] reading in the BBC news today um where mythos  had started to impersonate um individuals to [21:48] socially engineer other humans etc. and that  we're we're doing analysis and we're kind of [21:55] implementing guard rails in inside LMS and we're  doing some of the the kind of standard practices, [22:00] but but the subsequent actions that an LLM takes,  I don't think we've we've spent enough time trying [22:09] to understand and interpret and get visibility  over those those actions. and and you know this is [22:14] um we've got a big cyber security conference down  the down the road black hat and and everybody's [22:19] talking about machine speed. Everything now  is running at machine speed. If compliance [22:24] and we're trying to assess risk and we're using  humans to to understand the risk of the machine, [22:31] we can't operate at the same speed. And  so ultimately we need we need our own LMS [22:36] um assessing that risk. we need to be working at  the same speed but we have to have the visibility [22:41] over the subsequent actions um AI is taking to  achieve the task because we we always have to [22:47] remember you know a lot of the the kind of you  know more modern frontier models they're all [22:52] being being trained on reinforcement learning so  so they're kind of it's the penalty and reward [22:57] signals that they're getting to complete that task  we've seen in in cases where they're given a task [23:03] and they're given a a sense of agency I need  this by this time. It's urgent. The CEO needs [23:09] these things. It will go way around the um the  the long way to get the job done and that will [23:16] increase risk. And if you don't have visibility  over the steps it's taking to go outside the guard [23:21] rails or kind of breach those guard rails that are  being put in place then and and we don't have AI [23:26] working on our side as well, then then we're we're  in a in a really um difficult position. Yeah. Just [23:32] just to pile on to that too. So we're at the same  uh Black Hat across the street. We're a security [23:37] company kind of fundamentally. Um, and I like  to think about this uh from the perspective of [23:41] I think one of the biggest gaps we have right  now is is runtime enforcement on agents. Uh, [23:46] we really are in in the next generation of  authorization. You know, the first generation was [23:52] the network security. Everyone logs into the VPN.  That's how you get access to tools and technology. [23:56] It's all human speed. But, you know, traffic is  backhauled and you inspect it and you understand [24:01] what's happening. The second generation was we  adopted the cloud but it was still very much like [24:06] humans doing work and you could generally trust  that. The fundamental shift with agents is that [24:11] they operate non-deterministically meaning I can  ask an agent to do the same thing three times and [24:17] I might get three completely different outputs  and they might try to do it in three different [24:20] ways. And so to operate at machine speed with  governance and guard rails and make sure you're [24:26] managing the risk successfully, you have to have  runtime authorization on those agents because [24:30] you need to make sure when you ask it to clean  up your Salesforce data or you know answer any [24:36] whatever the case is that you have um it's doing  the right job with the mission and the intent that [24:43] you gave it and that if it starts to deviate from  that either you're doing some sort of like human [24:47] in the loop enforcement or you have guard rails  built into um the agent to make sure that you're [24:53] you know not accidentally delivering like PII  to the context window whatever the case is. So [24:57] I think that that runtime authorization piece is  a fundamental shift that we just we have to have. [25:04] Does that align with what you're seeing in the  you know the user space in Irene? Does that align [25:10] with something that you see in your space? Pretty  much. Yes. I just want to add one more thing. Uh [25:16] when we talk about um giving agents the agency and  the autonomy, a lot of it goes back to the first [25:25] principle of how you design your data, right?  The data model and how you put the governance [25:31] around your data model. For example, things highly  sensitive data, ultra high sensitive data, how do [25:37] you manage your um entitlements to make sure that  certain agents have access to certain data and [25:44] certain agents don't? um they don't have access  to that data. So that's one of the thing that I [25:50] think before designing agentic systems we should  take a step back and design our data systems [25:56] right first and then it doesn't matter whether  what what system you what AI system you put in [26:03] the the governance would have been built in and  not bolted on. I think I think you're absolutely [26:09] right. we have coming from an organization that's  been around since like 1895 or whatever. You know, [26:16] you can imagine how much data we have that's um  just not in any state to be looked at by AI or [26:24] by and even a normal human for that matter. So  all the savings and time and effort that we're [26:32] uh that we have on the AI side of things, we're  now deploying a lot of those uh humans to the data [26:38] side of things and making sure that our data is  uh is right. That leads to a challenge that every [26:48] organization faces between speeds and governance.  Organizations want AI today, but governance takes [26:59] time. The goal is not to create roadblocks,  but to put in the right guard rails to make [27:06] sure that companies can move forward with trust.  So, Sanjita, how do we strike the right balance? [27:17] So speed and control don't necessarily have to  be a tradeoff as long as we architect systems [27:27] to account for it and it just goes back to what I  was saying for the previous question like if you [27:34] lay the right foundations then you can architect  for both speed with control putting the right data [27:43] data model in place where it is able able  to track your um met adding more and more [27:50] metadata to it. Lineage semantics and then model  explanability where where every decision that [27:58] has been taken you have a full explanation of  what the decision was and you can trace it back [28:05] uh trace the data lineage back to its source.  So that's that's the foundation that's some [28:12] that's a bit that everybody has to take I  feel like the basics they the basics have [28:18] to be done like you cannot bypass the basics  and once you've done that then you can you [28:23] actually have architected system that can  account for both both speed and control and [28:28] and we we've seen just to e echo that point  like architecting a system correctly we've [28:33] seen instances where you know potentially you  don't need you know Fable or GPT 5.6 running [28:41] that process. Maybe you're leveraging the latest  and greatest model to to m do make a deterministic [28:49] um set of steps that can then run that process  day in day out reducing your risk. You're still [28:55] um leveraging AI, but you're leveraging AI to  kind of create a simplified process that you know [29:01] doesn't have all the risks of um you know breaking  out of the sandbox um type type scenarios. [29:09] What's your view, Alex? If you have to put in  three guard rails before AI solution goes live, [29:15] what would they be? Um, so the three guardrail.  So, one is um governance at the I'll just get [29:22] super technical here. I think governance at the  tool call level and access management for the [29:27] agents. So, you need to understand the identity  behind the agents, what they have access to, [29:31] what they need. if you have additional use cases  that sort of span outside of the control of what [29:36] a traditional human would be doing for that work  um you know for uh whatever that use case is that [29:42] they need I think you have to be able to you know  kind of assert new identities in the context of [29:48] an agent with like new access boundaries for  that agent so I would say you know it's back [29:53] to like entitlement management what data do  they have access to what's the um what sort [29:57] of uh systems do they have access to in the the  agent context so one is governance. I think uh a [30:03] second one is DLP. So being able to and actually  I say DLP but the reality is it's not necessarily [30:09] data loss so much it is it is like data inspection  and sort of understanding what's getting sent to a [30:15] um the context window or not. So you know if  you have an agent running and it has access to [30:21] a bunch of customer data is it actually okay for  you to send that customer data up into the context [30:26] window? Potentially not. And so I think you need  the ability to sort of at the the runtime level [30:33] um either scrub data, make sure you're not  accidentally sending PII, make sure you're maybe [30:38] not mixing um sensitive customer data between  multiple different customers. That's a use case [30:42] a lot of companies have. Uh and then the third  I think is just looking at the risk of what the [30:47] agent could potentially do um you know given given  the context. So there's this concept of the lethal [30:54] trifecta which is um you have uh untrusted content  potentially with the ability to exfiltrate it [31:01] um and then the ability to kind of um uh you  know run like untrusted code and I think you [31:07] know being able to in real time evaluate the risk  of what the agent might be doing with you know uh [31:15] sensitive data with untrusted content. um content  meaning like you know the user inputed or you got [31:22] it from the um someone on the website or something  like that and then the ability to exfiltrate. I [31:27] think you need the ability to kind of dynamically  assess risk of what the agents could be doing and [31:31] be able to block something maybe if it's um uh if  it's inappropriate. I I'll give another shout out [31:38] to the lethal trifecta as well. That was a if you  haven't read the blog post I think it was two and [31:42] a half years ago Simon Simon Willison kind of  put out a a blog post on that lethal trifecta. [31:48] We've also found it extremely effective at  understanding potential risk of an AI agent [31:54] um as well. Super powerful. I don't think he's  he's put any IP protection around that, but it is [32:01] a great framework. Thank for this recommendation.  We spent a lot of time today talking about today's [32:08] challenges. Let's pivot by looking ahead. Irene,  the regulatory landscape is evolving rapidly. [32:17] What should organizations be paying  attention today and what does that [32:22] mean for the control folks here? Well, Alex,  from so from somebody in the compliance seat, [32:29] there a couple of things. So before I'll mention  the regulatory items first uh second but before [32:35] we even get there uh the traditional way  the compliance usually gets involved in [32:41] any kind of decision-m in most organizations  at least at the University of California is [32:45] um you know it wants to get a Salesforce contract  going um they've negotiated the terms they're [32:52] right about sign to sign on the dotted line um  and then somebody says hey you know we should [32:57] go talk to that privacy officer over there and see  if he has any concerns concerns. Well, of course, [33:01] he's going to have concerns, but that's kind  of too late to ask that question. So, uh, [33:06] with the way things are moving and to Sanjita's  point, like every day there's a new technology, [33:11] there's something new all the time. So, things  are moving really fast and involving compliance [33:15] in that conversation early on when you're  identifying a use case and figuring out, [33:20] you know, what data is going to be used  for what purpose. Is it restricted? Is it [33:24] um you know is it data that researchers use  which is literally restricted data that's [33:30] like governed by the department of state like  what what sort of thing are you using is it PII [33:35] PHI and all of those great things um ident you  know having a conversation with your friendly [33:41] compliance officer early not for the purpose of  slowing down the project but for the purpose of [33:46] um stepping in front of the problem uh that will  occur later and kind of trying to preemptively [33:55] work towards reducing the risk. So that's number  one. Um and then the federal landscape which is [34:01] just really challenging and my team spends a lot  of time. We have like agents that are working on [34:06] this all the time. Uh and they give us a readout.  So I'm not an attorney so I'm not giving you like [34:11] a regulatory briefing but from my perspective.  So there's a federal side of things. We don't [34:16] have like one AI law. you know, the current  administration wants to have more transparency [34:20] and rest less regulation and allow all these  fabulous companies to innovate and do the great [34:26] things that they do. Um, on the state side, so I'm  from California and we've had a number of things [34:32] that have gone through either in the proposed  or finalized stage. There was um a law that [34:38] just got vetoed by Governor Nuome uh which would  have required um when you use bots in different [34:46] um in in different employment setting or if a  different decision making uh for us it would have [34:52] been like for example for admissions for student  admission or for uh determining federal you know [34:58] state financial aid. So if you when you're using  a Gentic technology for any of those decisions, [35:04] you would have had to then look at the security,  look at the how the look at the black box of how [35:09] the decisions were being made and disclose that  to the state. So that law just got vetoed by [35:15] uh Governor Nuomo, but I'm sure there's more where  that came from. So that's coming and [snorts] we [35:20] look at the regulatory map across uh just the  United States and there's a patchwork of laws [35:27] and whether it's proposed or finalized states  there's a whole patchwork of things all over the [35:31] map and for those of you who operate in multiple  states that's kind of a problem in of in itself [35:37] and then on top of that we don't know where the  federal government is going to land so that's [35:40] something to watch as well. In California, I'll  give you another example. In California, there [35:45] was just a case where um AI was making decisions  in um in in recruiting and you know hiring and [35:54] you know just different employment actions. Um  and they the they determined that um there's the [36:01] organization is responsible whether you use AI or  not. Whether you have a human in the loop or not, [36:06] that's kind of an organization's problem. But the  organization is responsible. So for in the in the [36:12] case of University of California, it would be the  border regions for the University of California [36:17] are responsible for those decisions whether  you know let's just for example say if like [36:22] if there was a decision that would have been  deemed as discriminatory for the uh for the [36:27] state law it wouldn't matter whether AI made that  decision or human it would have been treated the [36:32] same. So those laws are already in place and the  procurement laws in California already in place [36:37] that kind of account for um basic items and the  the overall takeaway for California is that it it [36:45] just currently right now it doesn't matter um for  the courts whether that decision was made by AI or [36:51] not the for the organization the responsibil the  level of responsibility is the same. So um that is [36:58] something that my team is really actively engaged  and watching and making sure um if I ask my [37:06] crystal ball like what's h what's going to happen?  I don't know um I don't know how good my crystal [37:12] ball is but I think I would go back to what  Alex just said. I think it's going to get down [37:17] to you know what is the action you're trying to  solve? You know what kind of data are you using? [37:24] what are the kind of the regulatory requirements?  Are you using pay PHI, PII? Um, you know, what are [37:30] the what's the data? Who's, you know, how are you  controlling that data? And, you know, hopefully [37:36] there's a human in the loop. I think that's we're  going to get there. Um, at least in California, [37:41] I think we're going to get there fairly quickly.  Um, so, and then there's just there's going to [37:46] have to be some human kind of responsible party um  in the loop. So I think that's where in California [37:52] at least that's kind of where the regulation  is headed towards. Although they're, you know, [37:57] they're trying to clamp down and they're trying to  stop all this innovation. I personally, just me, [38:02] I don't think that's feasible. Um, especially  not in California where Silicon Valley and [38:07] where all those companies are. So I don't think  that's going to go anywhere. Um but definitely [38:13] the responsibility and you know what's the data  who's controlling it what are the cyber security [38:18] controls over and I think that's probably going to  land somewhere in that in that general direction. [38:25] Thank you. Senita, you also work in the regulated  industry. Do you guys have a crystal ball? What's [38:31] your experience and know how are companies keeping  up with these changes? So risk and compliance are [38:41] very broad things and they are they are broad  and varied across different industries. But in [38:48] a nutshell, what does compliance do? Compliance.  The job of compliance is to say what they're doing [38:56] and do what they're saying at the at the gist  of it all. And what does risk do? Risk says, [39:02] "Okay, so you guys are saying what you're doing.  Are you missing any key areas which could cause [39:10] a problem in the future?" To bring a simple  example, a financial advisor took a client's [39:18] um assets as well as their financial goals.  They said, "Okay, I I I want my investment to [39:25] be this this this is my risk tolerance and so on  and so forth." They made a few bets around it, [39:31] but they might have concentrated in certain  sectors that they must have overlooked. So [39:36] this has been the this has been the tag team kind  of that risk and compliance has been playing for [39:43] decades now. How does it get different with AI?  It really doesn't. The core job function remains [39:50] the same. It's just the scale at which it is being  done on both sides of the table. So now you have [39:58] both side people on the both sides have the same  intelligence that they can use to both design [40:06] systems that work better for risk and compliance  and also um do their own jobs much better. So I I [40:16] think like the core job function remains the  same. It's it's just the scale at which you [40:21] have to innovate. Shift left is one of the other  thing just uh bringing back to what Alex said, [40:27] right? So you have to take a step back and shift  is something that we use very often is called [40:33] shift left. Like you have a process that does A  B C D. How you how do you shift left from D to C? [40:42] For example, if I have a certain number of rules  that I put in that are supposed to be applied a [40:48] after a trade has been executed, what of those  how can I triage the data fast enough so that [40:56] some of those rules get triggered pre-trade? So  that's always a better answer. So those are some [41:02] of the things that we have to continuously  keep doing and transforming the business. [41:09] And one other point um brought something that  I was thinking about this as you were talking [41:15] uh with all of the technology and with all of  the rules and you know uh the the vastness of [41:24] all these different operations you know u  JP Morgan's all over the world so I can't [41:30] even imagine the volume that you guys are dealing  with. So with all of that, you know, for back to [41:36] the compliance teams, I think if you're thinking  about governance, you have to build it not around [41:42] those laws and you know, what's coming next.  You have to be just adoptable to make sure you [41:47] can pivot as fast as um so nothing is basically  written in stone is the whole point of governance [41:54] is you have to be adoptable to whatever the next  law that's coming around the corner opposed to [42:00] u making it so tight that you can't even address  the volume of things that are coming your way. [42:07] And I see that we only got four more minutes, but  before we wrap up, I always like to ask the panel [42:13] to leave something for the audience to take away.  What is one message, one piece of advice that you [42:20] like this audience to leave with? Roger. Uh good  good question. I I forgot to prepare for this [42:26] one. The the the one message especially from  a risk and compliance kind of echoing the the [42:33] um regulation discussion as well. this notion of  proportionality when whenever we looked at privacy [42:41] um and and we're also seeing that to come in  um uh with some of the new AI regulations. [42:45] We've seen it in in Europe and we've seen it  in Colorado and and we're we're starting to see [42:50] um a few kind of other areas proportionality  the the notion that the monitoring or the the [42:58] assessments that we do have to be proportional  to the risk that we face by introducing a new [43:03] technology. And so if we can get that balance  right, we can start to to get that risk versus [43:10] reward. We have to use AI as a reward. It's also  a risk. And we've got to get that proportionality [43:15] piece. Um and and really understand that  clearly as we introducing new new technologies. [43:23] Well, for me it's um Alex, you kind of almost  hit on the point that I was trying I was going [43:28] to make too is that uh look at the riskiest, you  know, think about risk think about a risk based [43:36] approach and uh focus your efforts on the items  that have the most consequences um and let agents [43:45] potentially run with others but uh that that are  least consequential but uh a risk based approach [43:51] because otherwise where the compliance teams  out there are going to be boiling the ocean all [43:56] the time. So focusing on the risk the items  the strategic priorities that have the most [44:02] consequences would be my take my one takeaway. Um  I I'll sort of approach it from the perspective of [44:10] uh like the AI optimist view which is I think one  of the biggest challenges for companies is to not [44:16] just have you know the the mindset of like how do  we eliminate risk with AI because the reality is [44:22] like it's a new technology. There's going to be  risk. If you're not in the game you're just not [44:26] going to you're not going to advance and you're  not going to modernize your business. Um and so [44:30] I I like to think about how do you what is the  best way to do that? actually just to have the [44:34] governed path be the safe path and the easy path.  And I think that's a you know sort of a technology [44:39] perspective which is give teams and employees the  tools they need to use AI safely, securely um get [44:48] access they need. It's governed. It's secure by  default because that is just going to accelerate [44:53] and allow your company to uh adopt AI tools and  technology um easy. But you have to make that path [44:58] the easy path. That's got to be the easy button  when it comes to using AI tools and technology. [45:06] My fellow panelists have put it beautifully. Um, I  don't think I could have said anything better. So, [45:12] I'll leave you with one thing. With AI,  intelligence is taken care of, but wisdom [45:18] might not have been. Intelligence lets you solve  problems. Wisdom allows you to identify what [45:27] problems to solve. So keep that in mind and don't  treat don't take AI as yet another automation use [45:36] case. Look at how you can transform your processes  using this highly intelligent tool that you have [45:44] at your hands. Thank you. Uh I know we are at  time and I'll close with one theme that really [45:52] stood out today which is AI and governance. They  aren't opposite forces. When they work together, [45:59] they orchestrate and they bring business value and  it brings competitive advantage. So, I would like [46:06] to thank our outstanding panelists. Thank you  AI4 and thanks everyone for joining. Goodbye.