Scotiabank, BMO Curb Autonomous AI Over Audit and Regulatory Risks

因审计与监管风险，Scotiabank与BMO限制自主AI 金融机构正放弃完全自主的生成式系统，转而采用确定性护栏与引用核查机制，因为非确定性模型本身无法满足严格的监管可审计性及信贷承销要求。 来自Scotiabank、BMO、Commercial Bank International、CI&T和deepset的高管审视了在金融服务中规模化应用AI的运营现实，并将当前格局比作移动技术早期的“PalmPilot时代”。讨论强调，可持续的投资回报率来自对端到端工作流程的重构——例如Scotiabank的自动化邮件分流和缩短贷款周转期——而非盲目追求华而不实的概念验证或接受未经检验的盲目信任。为使非确定性模型符合SR 11-7等严格监管要求，专家组建议在三道防线中采用模块化架构、严格的任务级评估以及确定性检索护栏。 银行业目前处于 AI 应用的“PalmPilot 时代”，可持续的生产力提升依赖于任务级基准测试、安全评估与训练有素的员工队伍，而非单纯依赖技术炒作。 (5:18) 后台运营分流已实现显著的生产价值，例如丰业银行（Scotiabank）利用智能体 AI 解析非结构化商业电子邮件，将客户处理周转时间缩短了约两小时。 (9:38) 企业银行架构正在从简单的对话式搜索聊天机器人转向“复合型 AI（compound AI）”系统，结合多模态提取、确定性护栏和领域基础模型来处理复杂的非结构化数据。 (12:33) 摆脱概念验证（POC）陷阱需要将 AI 项目直接与可衡量的资产负债表压缩指标挂钩，例如将抵押贷款和商业贷款审批周期从 45–60 天压缩至 20–25 天。 (16:33) 金融级应用不容概率性侥幸，即使大模型达到博士级基准测试水平，未经严密数据管线、对抗测试与合规审查，也绝不能直接充当信贷审批员。 (17:43)

Banking leaders and artificial intelligence practitioners argue that capturing enterprise value requires shifting from experimental chatbots to deeply embedded workflow automation governed by traditional software economics and auditable architectures. Targeting High-Friction Operational Workflows (9:58) Rather than pursuing novelty, financial institutions are deploying artificial intelligence into high-friction, repetitive operations to compress execution timelines. Yannick Lallement of Scotiabank describes an agentic system that ingests thousands of unstructured daily emails from clients, lawyers, and relationship bankers. By parsing corporate entities, discerning intent, and auto-generating back-office tickets, the system eliminated manual indexing delays and reduced commercial response times by approximately two hours. Young Pham of CI&T emphasizes that meaningful returns emerge from end-to-end process compression, such as reducing residential and commercial lending origination cycles from 45–60 days down to 20–25 days. Software Economics and Rejecting Benchmark Theater (15:03) Panelists reject the premise that artificial intelligence requires speculative evaluation frameworks. Lallement frames AI as an advanced tier of enterprise software: while it carries higher computational and development overhead than deterministic code, it must satisfy standard financial metrics—either driving top-line revenue, lowering operating costs, or providing strategic risk and customer satisfaction benefits. Edward Achtner of BMO cautions against consumer-tech "success theater," arguing that high academic scores on general foundation model benchmarks do not qualify a system for credit underwriting. Because banking products require non-negotiable trust, deployments must rely on mathematically verified safety evaluations rather than probabilistic leaps of faith. Architecting Modular Systems for Non-Deterministic Workflows (29:20) Because modern artificial intelligence is fundamentally non-deterministic, institutions manage operational exposure through modular architectures and phased governance. Jay Wilder of deepset notes that teams treat autonomy as a spectrum, often bypassing generative capabilities entirely in favor of classical extraction-based natural language processing when zero-risk determinism is required. To prevent cascading system failures, architectures decouple data ingestion, foundation models, safety guardrails, and agentic loops into independently auditable components. To balance rapid experimentation with institutional risk standards, Lallement outlines a two-phase governance framework. Teams first evaluate feasibility within low-friction exploration environments before advancing viable models to formal review. In the second phase, systems undergo traditional credit, market, and operational risk assessments alongside specialized controls designed to bound hallucinations and limit failure blast radiuses. Multi-Jurisdictional Compliance and Defense Line Governance (39:49) Global banks face nonlinearly scaling compliance friction as state, federal, and international frameworks—such as the EU AI Act and emerging laws in Colorado and New York—proliferate across their operational footprints. Panelists draw parallels to the historic use of the Interstate Commerce Clause to standardize mismatched railroad gauges, noting that financial institutions must advocate for standardized, technology-neutral principles to prevent fractured regional rules from halting software delivery. Internally, enterprise oversight typically maps across the three lines of defense, where dedicated second-line AI risk teams establish corporate directives while domain specialists oversee specific credit, fraud, and compliance risks. To bridge the divide between unstructured data and deterministic compliance requirements, banks deploy retrieval-augmented generation architectures equipped with source citation mechanics, post-generation groundedness verifiers, and corpus coverage metrics. Realizing the operational potential of banking AI depends on structured systems engineering and verifiable retrieval metrics rather than unchecked autonomy. Significant challenges remain, particularly in aligning technical literacy across internal compliance divisions and managing nonlinear overhead across fragmented regulatory jurisdictions.

在Ai4研讨会上，多位银行高管与技术专家指出，银行业应用AI正处于类似早年PalmPilot的渐进探索期；机构不能依赖不可控的技术炒作，而必须遵循严谨的软件经济学，并通过确定性工程与分层风控体系驾驭生成式AI的非确定性。 算清软件经济账与摆脱概念验证陷阱 (15:03) 丰业银行首席AI官Yannick Lallement认为，银行评估AI项目无需引入神秘机制，本质上应将其视作一种运行与研发成本更高、但功能远超传统代码的新型软件。AI投资的合理性必须建立在经典的三大价值驱动上：直接推动顶线收入增长、实现底线成本缩减，或带来客户满意度提升与合规风险降低等战略协同。CI&T首席战略官Young Pham指出，管理层常误以为部署AI能瞬间变现，但脱离完整生命周期的孤立概念验证（POC）几乎无法产生切实回报。以住房和商业贷款为例，AI的量化价值应直接体现在现有业务流程的周期压缩上，例如将长达45至60天的端到端放款审批压缩至20至25天，以此兑现真实的资产负债表回报。 从零散问答走向复合智能与全流程自动化 (9:38) deepset增长副总裁Jay Wilder观察到，行业正从去年的浅层知识库检索问答转向复杂的决策支持与代理工作流。面对商业地产数据室等非标准化、包含复杂版面与OCR瑕疵的非结构化数据，单一语言模型难以胜任，必须结合多模态提取、上下文分块、确定性护栏及领域调优模型构建“复合AI系统”，并在早期引入业务专家进行对抗性压力测试。丰业银行近期落地的生产级用例印证了这一演进：该行针对商业银行海量邮件收件箱部署了代理AI系统，自动化解析跨法人结构、律师及客户的非结构化请求并创建后台工单，将原先约2小时的纯人工分拣等待排队时间彻底消除，显著缩短了整体业务周转耗时。 拒绝盲目信任与恪守银行业生产级底线 (17:43) 蒙特利尔银行应用AI主管Edward (EJ) Achtner强调，受严格监管的金融机构绝不能对未经验证的生成式AI抱持盲目的侥幸心理。他通过类比指出，移动转账若只有50%的成功概率将不可接受；同样地，即便基础模型在学术测评中达到博士级水准，也不代表其具备合规承销信贷的专业能力。门槛的降低催生了大量浮于表面的“成功剧场”，但银行级产品化要求必须具备加固的数据管道、严格的护栏及对抗性测试。Wilder与Lallement进一步指出，在面临合规红线时，系统应视情况放弃不可控的文本生成，主动退回至基于抽取的经典NLP技术，通过显式引用溯源、后置真实性验证和确定性指标来约束非确定性模型，防止单点故障引发系统性蝴蝶效应。 两阶段治理与三道防线下的跨法域合规 (29:20) 为平衡创新试错与审慎监管，Lallement提出了两阶段治理架构：第一阶段在低摩擦沙箱中快速验证AI方案的准确性与可行性；一旦证实具备应用价值，再推进至第二阶段接受传统信贷、市场与操作风险等全方位审查，同时施加针对模型幻觉等非确定性风险的专项控制。在组织架构上，丰业银行的AI政策归口于第二道防线中的专属AI风险团队，并由信贷及反欺诈等垂直领域的专业人员具体执行；迪拜商业银行的Giovanni Everduin则提醒，合规部门普遍缺乏对新型架构的底层技术认知，二道防线必须与业务工程团队紧密协同。面对从美国各州立法到欧盟AI法案的多法域监管拼图，多位专家指出合规复杂性呈非线性增长，跨国银行虽擅长应对分散监管，但仍需建立可审计、可复现的技术中立标准以防交付严重滞后。 小组讨论一致认为，银行业全面迈向零人工介入的“自主运行AI银行”仍属过度炒作。金融机构的当务之急，是在保持技术中立与架构可控的前提下，将非确定性的生成式技术稳妥镶嵌在现有的核心系统与三道防线中；在模型审计与跨区域合规标准彻底统一前，保持克制、分步推进与人类在环仍是不可动摇的底线。

## Background 本内容源自 Ai4 于 2025 年 8 月 28 日在 YouTube 平台上发布的研讨会视频，标题为《利用 AI 重构银行业务运营与客户体验》。视频汇聚了来自 Scotiabank、BMO、CBI、CI&T 以及 deepset 的多位银行业高管与技术专家，共同深入探讨人工智能在金融领域的实战应用、合规挑战与未来演进路线。 ## TL;DR 圆桌论坛围绕 AI 在银行业的落地现状展开，嘉宾们分享了商业邮件智能处理、复合型 AI 系统工程以及贷款流程提速等真实生产级案例。讨论强调金融机构绝不能依赖“盲目信任”，必须通过严格的数学评估、数据护栏与两阶段治理框架来兼顾创新与风控。专家们还深入剖析了多司法管辖区监管碎片化带来的非线性合规成本，并指出了通过精确溯源与立足度监控来化解生成式 AI 黑盒属性的核心路径。 ## Atmosphere & style 整场研讨会气氛务实而热烈，主持人以幽默风趣的开场打破了“银行业与 AI”话题的沉闷感，迅速拉近与观众的距离。嘉宾们抛弃了空洞的行业炒作，以坦诚直白的对话碰撞思想，既有对“盲目信任”等不切实际预期的犀利反驳，也有对 Palm Pilot 时代等技术变迁史的生动共鸣。台上下互动频繁，探讨既具备高度的专业严谨性，又充盈着前沿从业者躬身入局的务实激情与探索乐趣。 ## Key takeaways - [05:18](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=318s) 银行规模化落地 AI 的核心瓶颈已由基础大模型能力转向组织适应力与生产级评估体系，必须依靠细粒度任务拆解、数学验证以及与模型风险管理（MRM）团队的深度协同。 - [09:58](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=598s) Scotiabank 部署了首个生产级智能体（Agentic AI）系统处理商业邮件，自动解析并创建工单，将端到端处理耗时缩短了 2 个小时，展示了智能体在复杂后台运营中的降本提效潜力。 - [12:33](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=753s) 突破银行业非结构化数据（如商业地产数据室）处理瓶颈，不能仅靠单一基础模型，而必须依赖结合 OCR、精细切块（Chunking）、多层护栏与业务专家深度介入的“复合型 AI（Compound AI）”工程体系。 - [15:03](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=903s) AI 在本质上仍属于新型企业级软件，向管理层争取预算时必须遵循传统 ROI 与价值实现时间（TTV）指标，例如将信贷审批或贷款交割周期直接从 60 天压缩至 25 天。 - [17:43](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1063s) 银行业必须警惕原型开发带来的“成功闹剧”，银行级产品化要求绝对的合规、透明度与安全置信度，绝不能接受只有 50% 确定性的“盲目信任（Leap of Faith）”。 - [22:52](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1372s) 在自动化流程与个性化服务中，系统决策的“可复现性（Replicability）”是满足强监管的核心痛点，银行必须具备在风险暴露时精准追溯与隔离问题的审计能力。 - [27:26](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1646s) 为防止黑盒系统产生不可控的“蝴蝶效应”，AI 方案必须采用模块化解构的产品化思维，将数据链路、提示词、护栏及 Agent 工具调度完全解耦以实现透明可审计。 - [29:20](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1760s) 应对现代 AI 的非确定性特征，银行推行两阶段治理架构：第一阶段低门槛验证 PoC，第二阶段纳入传统金融风险及异常输出风控；同时在“强确定性提取”与“Agent 自主生成”的光谱间动态权衡。 - [32:31](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1951s) 媒体宣传的“无人工干预银行”脱离现实，银行业的推进相对克制是“有意为之（By design）”的制度性安排，首要遵循监管红线与基础设施底线保障。 - [39:49](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2389s) 各州法规与跨国法律（如欧盟 AI 法案）带来的碎片化合规要求呈现非线性指数级剧增，成为拖慢产品交付节奏的关键挑战，业界正借助州际贸易条款等机制寻求标准对齐。 - [42:36](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2556s) 银行内部 AI 政策普遍归属于二道防线（Line 2）风险管理团队，但合规与风控人员对底层技术的理解滞后构成了落地过程中的主要协作阻碍。 - [44:24](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2664s) 面对传统合规的“确定性搜索”与生成式 AI“黑盒概率性”之间的冲突，工程上主要通过事实立足度（Groundedness）、检索完整性评估以及强制端到端引用溯源来重构确定性信任。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=4s) 圆桌开场与议程设定 主持人 Gio（Giovanni Everduin，迪拜商业银行 CBI 首席战略与创新官）宣布 Ai4 首日会议的银行与 AI 专场正式开启。Gio 调侃道，虽然喜欢“AI”和“银行”这两个主题的人并不多，但本次讨论旨在为大众揭开其神秘面纱，直击挑战与机遇。除了在 CBI 任职，Gio 本人还兼任多家 AI 初创企业的董事会成员，并担任两个政府在 AI 落地实施方面的战略顾问。 本场圆桌汇集了来自大型银行、技术构建方及咨询服务商的专家，力求开展一场务实而立体的深度对话。会议规划了约 30 分钟的深度研讨，并特意预留最后 10 分钟供现场观众提问，因为嘉宾们一致认为问答环节往往是此类行业峰会中最具价值的部分。 --- ### [01:18](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=78s) 嘉宾介绍与个人日常 AI 模型偏好 在简短的自我介绍环节，各位嘉宾依次阐述了自己的职业定位、机构规模以及在个人生活与工作中高频使用的 AI 模型： ```mermaid flowchart LR accTitle: Panelists and their organizations and AI preferences subgraph Banks["金融机构 (Banking)"] Yannick["Yannick Lallement<br/>(Scotiabank 首席AI官)"] EJ["Edward (EJ) Achtner<br/>(BMO 应用AI主管)"] end subgraph Builders["技术与服务 (Builders & Services)"] Young["Young Pham<br/>(CI&T 首席战略官)"] Jay["Jay Wilder<br/>(deepset 增长副总裁)"] end Yannick --> Y_Pref["探索全行落地<br/>(PhD背景/早期聊天机器人先驱)"] EJ --> E_Pref["个人偏好: o3-mini / GPT-4.1<br/>专业级: 依托评估指标与数学验证"] Young --> Yo_Pref["个人偏好: Perplexity<br/>(用于家庭自主装修设计)"] Jay --> J_Pref["个人偏好: GPT & Claude<br/>(开源框架 Haystack 构建者)"] ``` - **Yannick Lallement（丰业银行 Scotiabank 首席 AI 官）**：拥有 AI 博士学位，早年在 2000 年代初就曾构建过初代聊天机器人。他澄清自己从未经历过“向 AI 转型”，因为他职业生涯的起点就是 AI。作为一家拥有 9 万名员工、2500 万客户的超大型银行的首席 AI 官，他的职责分为两部分：首先是全行业务扫描——从纯线上支票账户到复杂的企业 IPO 与家族办公室业务，识别出 AI 在何处能产生价值，以及哪些场景能最快落地见效；其次是战略落地——在锁定高价值机会后，为银行配置所需的人才梯队、技能培训、组织文化与生态合作伙伴。 - **Young Pham（CI&T 首席战略官）**：代表 IT 专业服务与咨询机构，专注于协助各大银行与金融机构将 AI 深度嵌入业务工作流中，重构运营模式。谈及个人生活应用时，他透露自己和妻子正在使用 Perplexity 替代专业设计师来进行家庭房屋翻新，并自嘲这“可能是个糟糕透顶的决定，也可能是全情拥抱技术革命的表现”；Gio 随即幽默接话称“这正是整个 AI 现状的完美隐喻”。 - **Edward (EJ) Achtner（蒙特利尔银行 BMO 应用 AI 主管）**：此前曾担任汇丰银行（HSBC）生成式 AI 全球主管，目前在 BMO 带领构建团队打造面向客户与股东的产品能力。在个人工具使用上，他近期主要使用 o3-mini 和 GPT-4.1 等模型；而在专业银行场景中，他强调团队绝不盲从工具炒作，而是严格依赖“性能与安全评估指标（Performance and Safety Evals）以及底层数学逻辑”来决定选用何种模型。 - **Jay Wilder（deepset 增长副总裁）**：来自技术构建阵营，其团队开源的 Haystack 框架在金融服务机构及大型银行中被广泛采用，专门用于构建满足合规、安全和生产级复杂要求的 AI 应用。在个人使用中，他主要依赖 GPT 进行深度调研以快速掌握全新知识，同时将 Claude 作为重要的“替补轮换模型”进行多任务比对。 --- ### [05:18](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=318s) 生产级范式、任务基准与规模化落地的核心瓶颈 主持人将话题转向银行业的实际落地场景，询问未来 12 到 18 个月内银行业最具变革性的机遇何在。针对 BMO 斩获的诸多 AI 奖项，EJ 首先开玩笑说“考虑到我刚加入 BMO 三个月，我打算把这些荣誉全揽在自己身上”，随后澄清这只是对台下公关团队开的玩笑，并严肃分享了其底层框架思考。 EJ 回顾了 2022 年底至 2023 年初行业寻找“产品市场契合度（PMF）”的历程。当时宾夕法尼亚大学 Ethan Mollick 教授发表了关于“半人马与赛博格（Centaurs and Cyborgs）”以及参差不齐技术边界（Jagged Technological Frontier）的开创性研究，各大金融机构自此开始对内部工作流程进行细颗粒度的基准测试，分析具体角色与任务的匹配度。 ```mermaid flowchart TD accTitle: Enterprise AI Scaling and Evaluation Workflow subgraph Benchmarking["任务拆解与基准测试"] A["角色与任务颗粒度拆解"] --> B["生产级范式匹配 (Production-Grade Patterns)"] end subgraph Evaluation["双重验证与风控"] B --> C["性能与安全评估 (Evals & Math)"] C --> D["第一/第二道防线 & 模型风险管理 (MRM)"] end subgraph Execution["规模化落地加速"] D --> E["受过良好培训与参与度的员工团队"] E --> F["生产力提效与新兴 Agentic 工作流释放"] end ``` EJ 指出，单纯把 AI 工具扔给员工绝不会自动带来生产力提升，企业必须确保拥有一支“受过良好培训、高认知度且深度参与”的员工队伍。面向未来 24 个月，行业将迎来生产力红利的集中爆发，尤其是在那些具备高确定性与高安全置信度的成熟任务上。与此同时，由智能体工作流（Agentic Workflows）驱动的全新业务职能正在被解锁。但这一切的前提，仍然取决于评估基准的严密数学验证，以及技术团队与第一道防线、第二道防线、模型风险管理（Model Risk Management, MRM）部门的紧密协作。 - 规模化落地难点广泛存在于跨司法管辖区的数据主权、监管政策差异以及安全评估体系的建立。 - 银行内部必须建立清晰的生产级部署范式，而非停留在无序的散点实验。 --- ### [08:10](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=490s) 历史周期视角与商业银行智能体邮件处理实践 面对 AI 在客户体验端的真实价值，Yannick 从技术史的宏观视角进行了审视。他认为，当前 AI 所处的发展阶段大致类似于移动通信时代的“PalmPilot 掌上电脑时期”；在此刻追问 AI 究竟会给银行业带来什么，难度不亚于当年在移动互联网萌芽期预测其未来影响。 尽管具体形态难以完全预判，但技术演进的底层规律具有高度确定性：如同计算机、互联网和移动通信等历次技术革命一样，AI 必然会使银行能够“以更低的成本提供更优质的产品与服务”，让客户的资金在更低费率下获得更高效的运转与增值。 在具体的落地实践方面，Yannick 透露 Scotiabank 几个月前已正式上线了其首个生产级智能体 AI（Agentic AI）系统。该系统直接对接全行商业客户的中央邮箱枢纽，每日自动分析并处理数千封来自企业客户的指令与诉求邮件，精准驱动后台业务流转，展现了智能体在复杂业务运营中的落地潜力。 ### [09:58](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=598s) 商业邮件智能路由：Scotiabank 的运营提速实践 Yannick Lallement 详细分享了 Scotiabank 如何利用 AI 自动化处理复杂的后台邮件工作流。在传统模式下，银行拥有一支庞大的运营团队，需要逐封审核客户发来的海量邮件。这项工作极其繁琐且充满挑战：商业客户的股权与业务架构错综复杂，邮件发送方涵盖了律师、会计师、第三方机构以及银行内部的客户经理。人工团队必须准确识别客户身份、弄清具体业务诉求，再手动在后台系统中为履约交付团队（fulfillment teams）创建工单。 过去这套流程每天要机械重复成千上万次。如今，Scotiabank 部署了专门的 AI 模型实时监控收件箱，自动解析邮件内容并直接在案例管理系统中生成工单。这一举措使得每封商业邮件的处理周期缩短了整整两个小时——这正是此前邮件在人工初审排队（indexing queue）中所耗费的时间。不仅大幅提升了对商业客户的服务响应速度，更显著压缩了运营成本，因为相比于庞大的人工审核团队，模型的边际运行成本“几乎可以忽略不计”。 ```mermaid flowchart LR accTitle: 商业邮件智能路由与工单生成流程 subgraph S1["邮件输入与解析"] A["多方邮件来源<br/>(律师/会计师/客户经理)"] --> B["AI 实时监控与语义解析"] end subgraph S2["系统联动与分发"] B --> C["自动匹配客户身份与业务诉求"] C --> D["工单管理系统<br/>自动建单分发"] end subgraph S3["业务成效"] D --> E["端到端响应缩短约 2 小时"] end ``` --- ### [11:08](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=668s) 工作流自适应与 AI-First 思维 针对银行业的技术采纳现状，CI&T 的 Young Pham 强调，当前阻碍规模化推广的核心瓶颈已不再是底层技术本身，而是组织与工作流程的“自适应能力”（adaptability of workflows）。他用早期使用 Palm Pilot 个人数字助理的经历作类比：当年硬件刚推出时，人们并不能确切预见它会如何重塑日常工作细节，但生产力的提升却是显而易见的。 在与各大银行合作的过程中，Young 发现最具增长潜力的落地场景集中在端到端的工作流自动化，例如住房抵押贷款和中小企业（SMB）信贷审批。贷款审批人员与流程处理员必须转变工作思路，从“以工具为终端”转向“AI-First 的自适应团队”模式。要实现规模化价值，银行必须深刻理解承销人员在具体节点上的认知负荷与操作细节，重构而非单纯叠加技术。 - **核心转型领域**：零售房贷（Residential Lending）与中小企业信贷（Small Business Lending）的全流程自动化。 - **关键瓶颈**：并非模型技术不成熟，而是银行现有运营工作流对新型协作机制的适应迟缓。 --- ### [12:33](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=753s) 复合型 AI 系统与非结构化数据攻坚 deepset 的 Jay Wilder 指出，整个银行业对 AI 的认知正发生深刻演变。去年的焦点普遍停留在智能客服、问答机器人和内部知识管理等“快速获取答案”的应用上，而现在业界正在跨越单纯的 Agentic（智能体）概念，大步迈入业务流程自动化与深层决策支持领域。传统机器学习（ML）主要擅长处理结构化数据，但银行日常面对的往往是极度杂乱的非结构化数据。 Jay 以商业地产（Commercial Real Estate）的数据室（Data Room）为例：里面的文件格式极其混乱，既包含复杂的非规范文本，又包含表格与图形扫描件，完全没有现成的规整模式可循。要攻克此类高复杂度场景，单纯依靠基础大模型远远不够，必须依赖系统工程思维——即业界所说的“复合型 AI”（Compound AI）与“上下文工程”（Context Engineering）。这涵盖了文本与视觉 OCR 技术的深度融合、精细化的数据切块（Chunking）、多层护栏（Guardrails）机制以及全新的人机交互界面。 ```mermaid flowchart TD accTitle: 银行复合型 AI 系统工程架构 subgraph DataIngest["多模态非结构化输入"] D1["商业地产数据室 / 合同文本"] D2["复杂扫描件 / 图形报表"] end subgraph CompoundAI["复合型 AI 系统 (Compound AI)"] C1["OCR 与图形多模态解析"] C2["智能分块与上下文工程 (Context Engineering)"] C3["领域大模型与安全护栏 (Guardrails)"] D1 --> C2 D2 --> C1 --> C2 C2 --> C3 end subgraph Delivery["落地与变革保障"] Out["自动化决策与审核工作流"] SME["业务专家 (SME) 全程参与压力测试"] C3 --> Out SME -.->|早期介入内测| CompoundAI end ``` 此外，变革管理（Change Management）绝不能等到系统上线投产后才开始。业务领域专家（SMEs）既是最终用户，也是排查“未知的未知”（unknown unknowns）的最佳测试员。让业务人员从 Beta 测试阶段就深度参与、极限施压系统，不仅能打磨产品质量，更能建立团队对 AI 系统的信任感与主人翁意识。 --- ### [15:03](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=903s) 投资回报与价值量化：AI 的软件本质与价值实现周期 主持人 Giovanni Everduin 抛出许多企业管理层普遍面临的困惑：董事会和 CEO 往往下达“必须在 AI 上有所作为”的指令，但到了审批预算和资源时，各方又因为难以量化预期收益而陷入拉锯。对此，Yannick Lallement 给出了非常务实的定调：**“AI 本质上就是一种新型软件”**。 Yannick 认为，向高管层推介 AI 项目的逻辑与传统软件并无二致：要么通过提效增加收入，要么以更低成本实现同等交付，附带收获客户满意度提升或风险降低等溢出价值。AI 并不神秘，它只是成本略高、但处理上限显著更强的新一代软件，其评估指标依然遵循标准的软件 ROI 框架。 Young Pham 则从交付周期的角度进行了补充。他强调 POC（概念验证）阶段往往无法体现真正的“价值实现时间”（Time to Value, TTV）。如果高管抱着“系统一开就能自动印钞”的幻想，往往会导致预期脱节。CI&T 在构建业务用例时，倾向于使用高度量化的业务运营指标——例如将原本需要 45 至 60 天的贷款交割审核周期（Loan Close）直接压缩到 20 至 25 天。这种实打实的流程提速正是 AI 嵌入核心工作流后所释放的确定性商业回报。 --- ### [17:43](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1063s) 拒绝“盲目信任”与成功闹剧：银行级产品化的严苛标准 针对主持人关于采用 AI 是否需要抱有“信任飞跃”（Leap of Faith，即盲目信任/放手一搏）的提问，BMO 的 Edward (EJ) Achtner 展开了极具冲击力的反驳。他现场向观众发起互动调查：“在座有多少人用手机银行转过账？请举手。如果我告诉大家，把钱转出去需要一点‘盲目信任’，转账成功的概率大约只有 50%，你们会作何感想？你们还会信任这家银行吗？” EJ 严肃指出，银行业是受高度监管的行业，对客户、监管机构以及自身产品质量负有绝对的合规与透明责任。过去几年技术开发门槛的降低是一把双刃剑：虽然激发了全员创新的活力，但也滋生了大量的“成功闹剧”（Success Theater）——许多人误以为只要做出了原型就掌握了开发能力。 > “一个在通用测试集上达到博士水平的基础大模型，并不等于它就是一名优秀的信贷审批员。” EJ 强调，“创新探索”与“银行级产品化”（Product Market Fit in Banking）之间存在着难以逾越的鸿沟。通用大模型在银行核心场景中往往并不契合业务初衷（fit for purpose）。若要真正部署到生产环境、尤其是面向客户的前台流程，必须经历严苛的数据管道建设、海量测试、模型微调以及全方位的安全合规验证，绝没有任何侥幸投机的空间。 ### [19:54](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1194s) 构建商业论证：从 Palm Pilot 时代到技术验证 嘉宾们回顾了技术发展的早期阶段，幽默地提到当前的 AI 发展水平类似于早期的 "Palm Pilot" 时代——台上的人自嘲“暴露了年龄”，而台下许多年轻听众可能甚至得去网上搜索才能知道什么是 Palm Pilot。但正是这种一年接一年的迭代与演进，才让技术逐步走向成熟。 从技术开发者和咨询机构的角度来看，为银行进行重大流程改造时，必须建立严谨的商业论证（Business Case）。如果只是在 Notion 文档上搭建一个用于日常琐事的聊天机器人，或许不需要大费周章；但当改造涉及到贷款等紧密关联营收（Revenue）与风险（Risk）的核心流程时，商业论证必不可少。 ```mermaid flowchart LR accTitle: 核心业务 AI 改造的商业论证流程 subgraph S1["价值发现"] A["设定提速目标<br/>(例如提升 2x 吞吐量)"] --> B["聚焦前置准备<br/>(生成信贷备忘录)"] end subgraph S2["技术选型与验证"] C["技术尽职调查<br/>(保持方案中立)"] --> D["落地验证<br/>(实现 ROI 闭环)"] end B --> C ``` - **降低启动门槛**：不必一开始就试图实现全自主的自动化放贷决策，可以从简单目标切入——例如在现有团队规模下将信贷备忘录（Credit Memo）的数据准备与初稿速度提升一倍（2x），从而缩短报价周期（Speed to Quote）。 - **技术尽职调查与方案中立**：在选择模型、供应商与技术路径时，不要过早与某一种方案“绑定”，应保持清醒客观的评估眼光，始终确保企业自身掌握掌控权。 --- ### [22:52](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1372s) 监管合规与创新的平衡：复现性与风险暴露 银行业处于高度监管的环境中，而当前监管机构不仅要面对 AI，还要面对加密货币（Crypto）等新兴领域的快速演变，要做到前瞻性监管极具挑战性。在讨论银行业如何平衡上线速度（Speed to Market）与合规要求时，嘉宾们强调了“可复现性（Replicability）”对于风险管理的关键作用。 金融机构在自动化工作流和个性化营销（Offer Presentment）等客户体验端投入了大量精力以促进转化，但很多机构往往忽略了“事后如何精确复现系统决策过程”。当系统出现数据泄露或合规风险暴露时，能否迅速追溯、隔离并向监管机构解释发生的事情，是目前许多银行尚未完全解决的核心痛点。 --- ### [25:00](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1500s) 敏捷交付与企业风险框架并非零和博弈 来自 BMO 的 EJ Achtner 指出，“在强监管行业中敏捷构建优秀产品”与“满足合规要求”绝非不可调和的零和博弈。全球各地的监管风格差异很大，有的倾向于开具详细的规定性条款（Prescriptive），有的则更倾向于让市场自主检验，但银行内部的企业风险框架（Enterprise Risk Framework）始终是兜底防线。 依托三道防线（Three Lines of Defense），银行需要能够对模型与流程进行数学上的可测算、可重复与可预测性证明： | 业务场景 | 风险特征 | 验证要求与应对方式 | |:--- |:--- |:--- | | **资本市场与高频交易 (Capital Markets & Trading)** | 极高风险、影响巨大 | 需极高的模型验证（Model Validation）门槛与严格的数学可解释性 | | **通用知识助手 (General-Purpose Knowledge Agent)** | 相对低风险 | 基于受控的内部文档检索，并引入多重人工介入（Human-in-the-Loop）机制 | EJ 强调，将创新转化为价值的关键在于建立一套标准化的日常管道流程（Pipeline Process），并配备相应的人才。 --- ### [27:26](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1646s) 产品化思维与系统透明度：防范“蝴蝶效应” AI 方案不能被当作快速拼凑上线的临时工具，而必须遵循规范的“产品化（Product Thinking）”开发流程：从构思、原型设计、测试到持续迭代。 ```mermaid flowchart TD accTitle: 可解释与可审计的模块化 AI 系统架构 subgraph Infra["数据与基础模型"] A["数据源与处理链路<br/>(Data Pipeline)"] --> B["提示词与模型<br/>(Prompt & Model)"] end subgraph Logic["控制与防护机制"] B --> C["安全护栏<br/>(Guardrails)"] C --> D["智能体编排与工具调用<br/>(Agentic Loops & Tool Calls)"] end Infra --> Logic ``` 在产品构建过程中，必须保证系统能够被清晰解构与透视。为了应对可能出现的系统故障或合规审查，团队必须杜绝黑盒设计与盘根错节的意大利面代码（Spaghetti Code）： - **模块透明与可审计**：清晰隔离数据管道、模型调用、Prompt 模板、安全护栏（Guardrails）以及 Agent 的分支与工具调用环节。 - **可追溯与可补救**：在系统出现意外偏差的“阴雨天（Rainy Day）”，团队能够迅速定位根因，防止错误在复杂链路中引发不可控的“蝴蝶效应（Butterfly Effect）”。 ### [29:20](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1760s) 非确定性 AI 的治理架构与 Agent 权能的光谱 银行业对开源技术生态系统的青睐并非出于锦上添花，而是源于对系统颗粒度和控制力的绝对要求。针对 AI 治理与传统软件治理的本质区别，丰业银行（Scotiabank）首席 AI 官 Yannick Lallement 指出，传统软件是确定性的，而现代 AI 具有非确定性（Non-deterministic）。这意味着在真正尝试之前，企业往往无法确切预知 AI 在特定业务流程中的实际表现。因此，银行必须构建两阶段治理框架：第一阶段提供低门槛的探索与概念验证（PoC）机制，以低成本验证技术可行性；一旦确认 AI 能力达标，项目即转入第二阶段，纳入包含金融风险、信用风险、市场风险和操作风险在内的全套传统风险评估体系，同时叠加针对非确定性风险（如模型输出偏离预期或作出错误决策）的增强风控机制。 ```mermaid flowchart TD accTitle: Two-Phase AI Governance Framework subgraph S1["阶段一：轻量探索与验证"] A["业务流程需求"] --> B["低门槛 PoC / 原型探索"] B --> C{"模型能力与效果是否达标？"} end subgraph S2["阶段二：全面风险与合规管控"] C -- "是" --> D["传统风险框架<br/>(信用/市场/财务/操作风险)"] C -- "是" --> E["AI 专属非确定性风险控制<br/>(异常输出/决策纠偏)"] D & E --> F["生产环境安全部署与监控"] end C -- "否" --> G["终止探索 / 调整方案"] ``` 在 Agentic（智能体）系统中，这种治理挑战会进一步放大。核心考量在于系统应赋予模型多大的自主权与决策权力。业内常将“完全确定性规则”与“完全 Agent 自主权”对立看待，但实际上二者是一个连续的光谱。在构建系统时，不必做非黑即白的极端选择，而是应根据业务场景动态调节。对于某些完全无法承受幻觉或生成错误的零风险业务，工程团队甚至会完全关闭内容生成能力，回退至传统的 NLP 信息提取技术，以确保绝对的精确度与可控性。 - **两阶段治理机制**：前期轻量 PoC 试错筛选价值场景，后期接入严密的复合风控网络。 - **自主权光谱（Agency Spectrum）**：在强确定性业务流与模型自主生成之间寻找平衡，绝非单选题。 - **零风险场景降级策略**：在容错率为零的关键环节主动剥离生成式能力，仅保留基于传统 NLP 的确定性提取。 --- ### [32:31](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1951s) “无人工干预银行”的炒作与银行业务稳健落地的现实 针对市场上普遍存在的激进预期，讨论转向了行业愿景与落地现状的巨大反差。媒体与咨询机构（如麦肯锡）频繁描绘“全自动运行、完全由 AI 驱动、无人工干预（No humans in the loop）的未来银行”，导致公众误以为此类颠覆性变革将在未来几个月内全面落地。然而，现场多位银行高管对此报以意味深长的反应，强调现实落地必须保持审慎与务实。 银行业之所以推进相对克制，完全是“有意为之（By design）”的制度性安排。金融机构首要遵循监管底线与风险边界，必须全力保障那些绝对不能出错的基础金融基础设施稳定运行，而非盲目追逐不成熟的前沿技术概念。 --- ### [33:38](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2018s) 问答环节：高可靠性要求下如何平衡与筛选 AI 用例 来自 AI 咨询公司的观众 Kevin 提问：银行系统要求极高的稳健性与严格测试，而生成式 AI 仍具有较强的新颖性与实验性，银行在筛选应用场景时如何在这两者之间取得平衡？ BMO 应用 AI 负责人 Edward (EJ) Achtner 回应表示，AI 在银行业绝非新鲜事物，数十年来各种确定性算法和统计模型一直是银行业务流程的核心支撑。2022 年 11 月 30 日（ChatGPT 问世）改变的是公众对 AI 能力的感知门槛，引发了全社会对新兴能力的极高期待。然而，银行作为客户资产与身份信息管理者的根本属性从未改变，在转账等关键资金流转环节，法律法规与客户信任都要求做到“接近完美”，绝不允许出现错误。 - **多时域（Multi-horizon）评估策略**：在高度安全的隔离环境中积极试验最前沿的实验性技术；只有经过充分验证、具备极高确定性的功能才会推向客户端。 - **用例筛选四要素**：严格评估商业可行性（Viability）、技术可行性（Feasibility）、业务增值空间（Value Creation）以及行业竞争格局。 - **底层商业逻辑的普适性**：无论是面对生成式 AI 还是未来的量子计算（Quantum），产品与市场契合度（PMF）与风险控制的底层逻辑始终恒定不变。 --- ### [36:34](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2194s) 跨部门系统集成与复合 AI 系统（Compound AI）的落地路径 观众 Francesca 提出有关系统集成的难题：传统企业级软件在打通各部门孤岛、集成跨系统数据与字段时已面临巨大挑战，新型 AI 工具在引入时该如何妥善解决这一集成痛点？ 嘉宾指出，解决集成问题的关键在于“以业务用例为导向逆向推导（Work backwards from the use case）”，先明确前台客户与后台法务、信贷、量化等具体部门的业务流与核心诉求，再选择最安全的连接路径。AI 不应被视为需要推倒重构整个企业技术栈的独立系统，而应作为无缝嵌入现有数字化资产中的隐形能力层。 ```mermaid flowchart LR accTitle: Compound AI and Enterprise Integration Flow subgraph Existing["企业既有资产与系统"] E1["业务部门工具 / 数据库"] E2["法务 / 信贷 / 量化系统"] end subgraph AI_Layer["复合 AI 体系 (Compound AI)"] M1["安全 API / 连接协议"] M2["基础大模型 / 专用模型"] M3["上下文与工具调度"] end subgraph Output["业务落地与敏捷演进"] O1["端到端业务流协同"] O2["支持未来技术平滑迭代"] end Existing <--> M1 M1 <--> M2 & M3 M2 & M3 --> Output ``` 在技术实践中，企业可以通过成熟的安全 API 方案进行集成；虽然业内正涌现出诸如 Anthropic 推出的 MCP（Model Context Protocol，模型上下文协议）等新兴生态标准，但在严监管的金融领域仍需审慎评估与采纳。结合复合 AI 系统（Compound AI Systems）的系统性思维，银行可以将大模型作为协作中枢，与已投入巨资的现有系统安全联动，既保障当下的合规与稳定，又为应对未来频繁的技术迭代保留系统敏捷度。随后，现场讨论进一步延伸至各州级（如加州）AI 监管法规的跟进与合规应对。 ### [39:49](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2389s) 监管碎片化挑战与跨司法辖区合规的非线性复杂度 在问答环节伊始，现场观众针对各州及跨国监管分歧发问，特别提及了科罗拉多州、纽约州的州级法案以及欧盟《人工智能法案》（EU AI Act），询问这类多层次监管正在如何直接或间接地重塑银行的风险管理战略。爱德华（EJ）·阿赫特纳对此给出了十分明确的定调：“法律就是法律，在有业务运营的地方，我们就必须严格遵守法律。”（"The law is the law and where we operate we have to adhere to the law."）然而，随着机构必须应对的法律与合规环境全貌不断扩张，落地 AI 解决方案的复杂度呈现出非线性（nonlinear）的指数级上升。他呼吁监管机构尽可能采取技术中立、以可量化标准和原则为基础的监管路径，将具体的实现手段留给企业自主裁量。 谈及如何缓解监管碎片化时，EJ 借用美国历史作出了生动的类比：当年为了实现横贯大陆铁路（transcontinental railroad）的建设，美国联邦政府正是利用宪法中的“州际贸易条款”（Interstate Commerce Clause）统一了各州的铁路轨距（railway gauge），避免列车每穿过一个州就要换一次铁轨；如今联邦与各州层面也正在尝试借助该条款来拉平 AI 的监管基线。然而，对于业务遍布全球的跨国大行而言挑战依旧严峻——EJ 提及他上一家供职的银行在全球 65 个国家开展业务，再叠加上各州层面的细则，合规就成了产品上线的绝对“基本门槛”（table stakes）。一旦要在所有辖区内做到毫厘不差，巨大的非线性合规成本很可能会严重推迟创新产品的实际交付周期。 其他嘉宾随后补充指出，银行业在漫长的发展史中早已习惯了应对多州与多国的复杂法规体系，无论是在加利福尼亚州、得克萨斯州，还是在加拿大与欧盟之间切换，多司法管辖区的风险建模与业务运营一直都是常态。数据隐私法（Privacy Laws）就是一个现成的参考模板，银行在不同国家和州的不同隐私法案夹缝中合规运营的经验，同样能够平移到 AI 治理之中。 - **州际贸易条款的监管类比**：历史上曾通过联邦州际贸易条款统一铁路轨距以奠定跨大陆铁路基础，如今该条款正被用来尝试规范和统一跨州的 AI 监管标准。 - **跨国合规的非线性扩张**：以覆盖 65 个国家的全球银行为例，多层次法律叠加导致合规落地难度呈非线性剧增，成为拖慢产品交付节奏的关键瓶颈。 --- ### [42:36](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2556s) 银行 AI 政策归属：二道防线统筹与业务落地的认知鸿沟 来自托皮卡联邦住房贷款银行（Federal Home Loan Bank of Topeka）的模型风险管理负责人珍妮·余（Jenny Yu）在提问前先为自己即将主讲的模型风险研讨会做了宣传，指出金融业在管理“黑盒模型风险”（black-box risk）方面已有长达 20 年的监管积累，并就此询问台下的大行嘉宾：银行内部是否已建立专门的“AI 政策”（AI Policy）？这项政策究竟由谁来牵头和拥有（own）？ ```mermaid flowchart TD accTitle: 银行 AI 政策与治理协作架构 subgraph PolicyLevel["二道防线 (Line 2) 顶层设计"] A["AI 风险管理团队"] -->|"制定全局性 AI 政策与红线"| B["全行 AI 风险统筹指令<br>(Holistic Directive)"] end subgraph DomainLevel["细分风险领域承接"] B --> C1["信贷风险领域"] B --> C2["市场与操作风险"] B --> C3["合规与反洗钱"] end subgraph PracticalLevel["业务与技术落地架构"] D["应用 AI / 创新团队<br>(架构设计与实践支持)"] <-->|"填补底层技术认知鸿沟"| PolicyLevel D -->|"推动落地"| DomainLevel end ``` 针对政策归属，扬尼克·拉勒芒（Yannick Lallement）介绍了丰业银行（Scotiabank）的权责划分：银行设立在二道防线（Line 2）的 AI 风险团队负责统领制定 AI 政策，划定“什么可行、什么不可行”的红线；随后，各个细分风险领域的专业团队（SMEs）会依据自身业务特性将政策贯彻到各自领域中，同时依靠全行统一的 AI 风险统筹指令确保每个风险板块无一遗漏。 乔瓦尼·埃弗杜因（Giovanni Everduin）表示商业国际银行（CBI）的架构也非常相似，政策在名义上由风险部门所有，但他领导的 AI 团队在实际操作中承担了大量框架架构的协助设计工作。乔瓦尼直言不讳地指出了当前业界的痛点：许多风险与合规专员并不真正理解 AI 乃至加密货币（crypto）的底层实质，“如何让合规人员真正理解技术内涵，进而能够准确地应用风险框架，至少对我们来说一直是一场不小的挣扎”。 - **二道防线的政策统筹**：AI 政策由风险二道防线团队统一制定，各垂直风险领域负责结合业务场景落地执行。 - **技术与合规的认知断层**：合规与风险管理人员对大模型黑盒机制及前沿技术的理解滞后，成为银行内部构建有效 AI 治理框架的主要摩擦点。 --- ### [44:24](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2664s) 确定性治理与生成式黑盒的冲突及大会总结 来自 Zeltech 的 K Leong 提出了压轴问题，直击银行治理的核心矛盾：银行最庞大的数据源往往是电子通讯（e-comms）等非结构化数据，且这类数据必须受到极其严格的合规治理；在传统的合规审查中，由于存在“信任缺失”，监管严格依赖于确定性的搜索（deterministic searches）与精准审计；然而生成式 AI（GenAI）的运行机制恰恰相反，本质上是偏向概率性、黑盒化的探索。面对这两种底层逻辑的冲突，大行如何化解非结构化数据治理与 GenAI 黑盒之间的鸿沟？ 台上的银行嘉宾对这个问题引发了强烈共鸣，赞叹道：“你精准道出了 AI 规模化扩展的核心死穴，我们正在全力攻关。”（"You've articulated the heart of the scaling issue... We're working on it."）杰·怀尔德（Jay Wilder）从工程实践角度分享了当前业内正在采用的技术治理手段：针对非结构化数据的生成任务，核心在于建立端到端的溯源与验证工具链。具体而言，系统必须具备对信息来源的精准引用与回溯能力（citation & source attribution），能够反向校验生成内容是否确实提炼自底层数据源；同时借助全量语料库搜索完整度（completeness）、事实立足度（groundedness）指标以及带有精准引用的查询覆盖率（citation queries ratio）等量化监控工具，确保大模型始终在既定数据的安全边界内运转，从而在黑盒生成中重构确定性信任。 在热烈的讨论中，主持人宣布圆桌论坛正式结束，感谢五位嘉宾分享的实战洞见，并热情邀请台下观众与嘉宾移步场下继续合影并交流后续话题，为整场研讨画上了圆满句号。 - **溯源与引用校验（Citation & Verification）**：通过建立细粒度的溯源工具，强制模型生成内容可直接映射并核验至原始非结构化数据源。 - **立足度与完整度量化（Groundedness & Completeness）**：利用事实立足度测量和全语料检索完整性评估，压制幻觉并建立可信治理闭环。 ## Notable quotes > [01:18](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=78s) 这正是整个 AI 现状的完美隐喻 — Giovanni Everduin > [08:10](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=490s) 以更低的成本提供更优质的产品与服务 — Yannick Lallement > [15:03](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=903s) AI 本质上就是一种新型软件 — Yannick Lallement > [17:43](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1063s) 在座有多少人用手机银行转过账？请举手。如果我告诉大家，把钱转出去需要一点‘盲目信任’，转账成功的概率大约只有 50%，你们会作何感想？你们还会信任这家银行吗？ — Edward (EJ) Achtner > [17:43](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1063s) 一个在通用测试集上达到博士水平的基础大模型，并不等于它就是一名优秀的信贷审批员。 — Edward (EJ) Achtner > [32:31](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=1951s) 全自动运行、完全由 AI 驱动、无人工干预（No humans in the loop）的未来银行 — Giovanni Everduin > [36:34](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2194s) 以业务用例为导向逆向推导（Work backwards from the use case） > [39:49](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2389s) 法律就是法律，在有业务运营的地方，我们就必须严格遵守法律。 — Edward (EJ) Achtner > [42:36](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2556s) 如何让合规人员真正理解技术内涵，进而能够准确地应用风险框架，至少对我们来说一直是一场不小的挣扎 — Giovanni Everduin > [44:24](https://www.youtube.com/watch?v=5w6Ob2PkcHU&t=2664s) 你精准道出了 AI 规模化扩展的核心死穴，我们正在全力攻关。 ## Mentioned Ai4, Giovanni Everduin, Commercial Bank International (CBI), Yannick Lallement, Scotiabank, Young Pham, CI&T, Jay Wilder, deepset, Haystack, Edward (EJ) Achtner, BMO (Bank of Montreal), HSBC, o3-mini, GPT-4.1, Perplexity, Claude, Ethan Mollick, PalmPilot, Anthropic, Model Context Protocol (MCP), EU AI Act, Federal Home Loan Bank of Topeka, Jenny Yu, Zeltech, K Leong ## Video description Yannick Lallement, Chief AI Officer, Scotiabank Giovanni Everduin, Chief Strategy & Innovation Officer, Commercial Bank International Young Pham, Chief Strategy Officer, Head of Financial Services, CI&T Jay Wilder, VP Growth, deepset Edward (EJ) Achtner, VP: Head of Applied AI, BMO Abstracts This session will explore how banks are leveraging AI to streamline operations, enhance customer experiences, and drive innovation. Panelists will discuss real-world use cases, from AI-powered fraud detection and personalized financial services to optimizing back-office functions. Learn how banks are successfully integrating AI into their systems to reduce costs, improve efficiency, and offer more tailored services to customers. The panel will also explore the challenges of AI adoption in the banking sector, including regulatory compliance and data security, while sharing best practices for maximizing AI’s impact in financial services. [Source](https://www.youtube.com/watch?v=5w6Ob2PkcHU)

[1:18] Giovanni Everduin: 这正是整个 AI 现状的完美隐喻 [8:10] Yannick Lallement: 以更低的成本提供更优质的产品与服务 [15:03] Yannick Lallement: AI 本质上就是一种新型软件 [17:43] Edward (EJ) Achtner: 在座有多少人用手机银行转过账？请举手。如果我告诉大家，把钱转出去需要一点‘盲目信任’，转账成功的概率大约只有 50%，你们会作何感想？你们还会信任这家银行吗？ [17:43] Edward (EJ) Achtner: 一个在通用测试集上达到博士水平的基础大模型，并不等于它就是一名优秀的信贷审批员。 [32:31] Giovanni Everduin: 全自动运行、完全由 AI 驱动、无人工干预（No humans in the loop）的未来银行 [36:34] 以业务用例为导向逆向推导（Work backwards from the use case） [39:49] Edward (EJ) Achtner: 法律就是法律，在有业务运营的地方，我们就必须严格遵守法律。 [42:36] Giovanni Everduin: 如何让合规人员真正理解技术内涵，进而能够准确地应用风险框架，至少对我们来说一直是一场不小的挣扎 [44:24] 你精准道出了 AI 规模化扩展的核心死穴，我们正在全力攻关。

[39:49] The law is the law and where we operate we have to adhere to the law. [36:34] Edward (EJ) Achtner: invisibly within the fabric that you already have. [30:57] Giovanni Everduin: don't break things that really should not be broken. [0:04] Young Pham: either a terrible idea or really leaning into this revolution.

[0:04] Good afternoon everyone. AI4 day one. [0:07] We're here. Uh we're going to talk about [0:09] two topics that I really really like. AI [0:12] and banking. We're going to try and [0:14] Yeah. I'm one of the few people that [0:15] really likes those topics. But actually, [0:17] we're going to try and demystify it a [0:18] little bit and talk about some of the [0:20] challenges, some of the opportunities. [0:22] Um my name is Gio. Uh I'm your moderator [0:24] for today. A little bit about me. I am [0:26] the chief strategy innovation officer [0:28] for CBI, a bank based out of Dubai. I [0:31] sit on the board of a couple of AI [0:33] startups. Uh, and I'm an adviser to two [0:36] governments on um, implementation of AI. [0:39] But that's enough about me. We have an [0:41] amazing panel here of bankers, builders, [0:43] service providers, and like I said, [0:45] we're really going to dig in and try to [0:46] have a really organic conversation about [0:49] what is AI's opportunity in banking and [0:51] what are some of the things you need to [0:53] consider. Now, before we get started, uh [0:55] I think we're going to take about 30 [0:57] minutes, quick intros. We're going to [0:58] then have a conversation and we're going [1:00] to keep the last 10 minutes for the [1:01] audience to ask questions. I think we [1:03] all agreed as a panel that that's [1:05] probably where most of the value comes [1:06] from, at least for us when we attend [1:08] these things. So, you know, please make [1:10] sure if you have any questions, [1:11] comments, whatever, you know, we'll [1:13] we'll have time in the end to discuss [1:15] all of that. So, without further ado, I [1:18] think we're going to just ask the [1:18] panelists to introduce themselves. So, [1:20] gentlemen, if you could just take 30 [1:22] seconds and tell the audience who you [1:24] are, what you do, and what is your go-to [1:26] AI model um for personal use, and maybe [1:30] we start at the end and work our way [1:31] back here. Yanik. [1:35] >> Thanks. Thanks, Joe. Hi, everyone. I'm [1:37] Yaniki. I am the chief officer for [1:40] Scotia Bank, which is a large Canadian [1:41] bank. Um, sometimes people ask me when I [1:45] pivoted to AI. I never pivoted to AI. I [1:48] started with AI. I have a PhD in the [1:50] topic and um about chief eye officer. [1:53] What does that mean? There's really two [1:55] two parts to the role. One is I work for [1:58] a very large bank. We have 90,000 [2:00] employees, 25 million customers. Where [2:03] can AI contribute, right? We do [2:05] everything from online only checking [2:07] accounts to IPOs and family offices. So, [2:10] where can AI be useful? Where can AI be [2:12] useful first? Uh where can AI be useful [2:15] more quickly? And um the second part of [2:19] the role is having identified some of [2:21] those ju juicy opportunities, how do we [2:24] position the bank to actually actually [2:26] deliver on them, right? What type of [2:27] talent, training, culture, partners, uh [2:31] and so on do we need? [2:33] >> Hi everyone, I'm Young Fam, chief [2:34] strategy officer for CINT. Uh my focus [2:37] area is uh really as an IT service [2:41] provider, professional services firm. uh [2:44] how do we work with uh banks and [2:45] financial services company embed AI into [2:47] their business workload certainly [2:49] understanding the underlying technology [2:51] but really how that's starting to change [2:53] the way business operations uh continues [2:56] to evolve over the next few years for my [2:58] own personal use um I I use perplexity [3:03] and it may be good or bad but right now [3:05] my wife and I are foregoing a designer [3:08] to actually do a home renovation using [3:11] it so that could be a terrible idea or [3:14] really leaning into this revolution [3:16] right now. [3:17] >> That's a metaphor for all of this AI [3:18] stuff, isn't it? All right, [3:21] >> hi everyone. EJ Aner. I was recently [3:24] appointed the head of applied artificial [3:26] intelligence at Beimo. I was formerly [3:28] the group head of generative AI at HSBC [3:30] Bank. I'm privileged to lead a team of [3:33] builders at Beimo. We build worldclass [3:36] products and capabilities that delight [3:37] both our clients and our shareholders. [3:40] In my personal life up until last week, [3:43] I was using 03 mini and 41. We'll talk [3:46] about that a little bit later. And I [3:48] think it's important to say in my [3:49] professional life, we use what [3:51] ultimately our performance and safety [3:53] evals and the underlying math inform us [3:55] we should be using. We'll get to that [3:56] too. [3:58] >> Awesome. My name is Jay Wilder. Uh I'm [4:01] the VP of growth at DeepSet. Uh part of [4:03] the leadership team there. Uh we're from [4:06] the builder side as you alluded to. uh [4:08] go. So we uh we make the haststack open [4:11] source framework uh that's very widely [4:14] used in many FSIs and especially banks [4:17] uh for building often very complex [4:20] applications that are able to translate [4:22] all the compliance and requirements and [4:24] safety uh that needs to go into these [4:27] into something that actually can really [4:29] work in production and adapt over time. [4:32] Uh personal note uh I'm an East Coaster. [4:35] This is my second AI4. [4:37] Um I saw a fist bump there. That is [4:40] definitely East Coast. I'm with you, [4:41] brother. [4:43] Um yeah, happily married, two kids. Uh [4:46] so always running around. Um I find [4:49] myself day-to-day I I go to GPT a lot [4:53] for the stuff that I do, work and [4:55] personal oftentimes around researching [4:57] things, trying to get caught up to speed [4:59] on different things. I think all of us [5:01] are often confronted with a lot of new [5:03] questions in this space that we have to [5:05] get up to speed to very quickly. Um, but [5:08] I also use Claude a bit as well and kind [5:10] of test the two against different jobs [5:11] when I find that one is maybe struggling [5:14] and maybe needs uh a call up from from [5:16] the bench. [5:18] >> All right. And so look, there's a lot of [5:20] talk and a lot of PR about using AI in [5:22] banks, a lot of field experiments, some [5:24] successful ones. So let's get into it [5:27] really right. And so from your [5:28] perspective, each of you having a very [5:29] unique perspective on the industry, on [5:32] banking in general, what do you think is [5:34] like the biggest transformative [5:36] opportunity for AI in banking in let's [5:39] say the next 12, 18 months? And maybe [5:43] EJ, I'll start with you because Beimo [5:45] has won a bunch of awards for your uh uh [5:48] AI implementations. So maybe you have a [5:51] really interesting perspective on that. [5:52] >> Considering I've been there all three [5:54] months, I will take full credit for [5:55] them. That was sarcasm for the [5:57] communications team in the audience. Um, [5:59] but thank you for that. I think if you [6:02] go back a few years, um, when we think [6:05] about concepts such as product market [6:07] fit, okay, going into late 22, early 23, [6:12] clear data began to emerge in terms of [6:15] which tasks could be optimized. There [6:17] was great research put out there. I'm [6:19] sure many if not all of you have heard [6:21] of professor Mollik out at Penn the [6:23] seinal centaurs and cyborgs jagged edge [6:25] piece many organizations I think at that [6:28] point in time particularly in early to [6:30] mid23 started to benchmark internal [6:33] workflows and processes and do alignment [6:35] from a role task perspective. So without [6:38] necessarily getting to proprietary [6:40] information, what you would expect to [6:42] see from an adoption curve is ultimately [6:45] what are those production grade patterns [6:47] that most closely mimic the roles and [6:50] task where you can get the eval to [6:52] basically have the enhanced productivity [6:54] gains from a workforce productivity [6:56] perspective. Now just throwing the tool [6:58] out doesn't make it so. So what you also [7:01] have to make sure very importantly is [7:03] that you have a trained, educated and [7:06] engaged workforce as well because the [7:08] key to getting these gains is making [7:10] sure that your workforce again is [7:11] educated and engaged. As we look ahead [7:14] to the part of your point there's going [7:16] to be not only be an acceleration and a [7:18] scaling effect and I think it's safe to [7:20] say from our perspective and many in the [7:21] rooms scaling has been very difficult. [7:24] There are a range of reasons why scaling [7:26] is difficult especially across you know [7:29] different jurisdictions ranging from [7:31] data to regulations again to performance [7:33] and safety evals but ultimately over the [7:35] next 24 months you're going to see an [7:37] acceleration of those productivity gains [7:40] for those tasks where you've got a high [7:42] degree of conviction and confidence that [7:44] you can safely deploy and then I also [7:47] think there's an emerging range of new [7:49] tasks and functions that are going to be [7:50] unlocked and unleashed by agentic [7:52] workflows but it's all going to go back [7:54] to the math of the performance and [7:56] safety evals and ultimately in the [7:58] context of banking which we've all [8:00] chosen to embrace that partnership you [8:02] have with first and second line defense [8:04] model risk management etc etc and that's [8:06] really going to chart the course forward [8:08] for the next 24 months [8:10] >> all right and maybe Yanik you're not [8:11] just a banker right you're you have a [8:14] PhD in AI you've been in AI before it [8:16] was called AI probably I I remember that [8:19] you actually you were one of the first [8:21] to build a chatbot back in the early [8:23] 2000s, right? And so what's your take on [8:27] this and particularly in the context of [8:28] customer experience? Because you know, [8:29] one side of it is what we do internally [8:31] in a bank, but then if you're a [8:33] customer, what should a customer be [8:34] excited about in terms of what you're [8:36] cooking in your lab? [8:38] >> I I think if we take a historical [8:40] perspectives in terms of AI, we are [8:43] about at the same time as when the palm [8:45] pilot came out. And that question is [8:47] almost like asking, you know, what is [8:49] the mobile revolution going to do to [8:51] banking, right? It's really hard to [8:53] predict. Uh, one thing I think I'm very [8:56] confident about is that AI will allow [8:58] banks to deliver better service and [9:01] better products at a lower cost. Right? [9:03] Uh, I'm very confident of that because [9:05] that's really what has happened [9:07] successively with every technological [9:09] revolution with computers, with the [9:11] internet, with mobile. Uh, AI is [9:14] effectively the next iteration of that. [9:16] So it's a bit hard to to to to predict [9:20] what exactly that is going to look like. [9:22] Um but for sure you will have better [9:25] banking products. Your money will work [9:27] for you thanks to AI at a lower cost to [9:30] the banks. And if you want I can get [9:32] into a concrete example of what we are [9:34] doing today uh at Scosia. [9:36] >> But yeah by all means. [9:38] >> All right. So we have a we have just [9:40] launched a few months ago our first [9:42] agentic AI and uh what this does is look [9:47] at all the emails we receive in a [9:49] central mailbox from all our commercial [9:51] clients. So we receive literally [9:53] thousands of emails per per day um from [9:56] clients asking us to do things on their [9:58] behalf, right? Asking operations to do [10:00] things for them. And we used to have a [10:03] team that would look at those emails one [10:05] at a time. So they had a queue and then [10:07] they they had to understand which [10:10] clients this is about. It's not so easy [10:12] because you know complex corporate [10:13] structures because we get emails from [10:15] third parties, [10:17] lawyers, um accountants, our own [10:20] co-workers, the bankers who who help the [10:23] clients uh and of course what that [10:25] specific email is about, right? What the [10:27] client want to do and then create a [10:28] ticket for the back end, the back office [10:31] fulfillment teams. And so we had a team [10:33] doing that thousands of times per day. [10:36] We are now using AI to do this. The AI [10:38] looks at each email, monitors the [10:40] mailbox and then creates the ticket in [10:42] the case management system. And the the [10:45] impact of that is that each email is [10:48] answered. Uh the round trip back to the [10:51] client is about two hours faster because [10:53] two hours is the time it took to sit in [10:55] the original uh indexing queue. Uh so [10:58] again this is better service to the [11:01] commercial clients at a lower cost to [11:02] the bank right because of course the is [11:04] effectively free uh the team uh was not [11:08] >> all right and and young you work with a [11:10] bunch of clients across the banking [11:11] industry right so what do you see and [11:14] what are you most excited about [11:15] >> yeah I I want to extend on something [11:17] that talked about as well as EJ um which [11:21] is uh certainly the understanding the [11:23] technology is there and it's coming out [11:25] pretty good and I did have a palm pilot [11:27] uh which which was very interesting [11:30] because my a productivity increased and [11:32] so one of the things we spend a lot of [11:34] time beyond the underlying technology is [11:36] this whole adoption using more of an AI [11:39] first mindset or even the idea of more [11:41] adaptive teams right and so uh yes when [11:44] the AOM pilot came out we didn't [11:46] necessarily know what that meant for my [11:49] day-to-day but I definitely knew that I [11:51] was increasing my general productivity [11:53] so when we work with banks beyond the [11:55] underlying technology and some of the [11:57] useas we play around uh particularly on [11:59] automation workflow which I think is the [12:01] biggest growth area things like loan [12:03] origination whether it be residential [12:05] lending or small business what are the [12:08] work details that the loan underwriters [12:11] and the processors now start to have to [12:13] understand and it's not an endpoint it's [12:15] it's a very transformative part of how [12:18] they need to start thinking about their [12:20] workloads and so we see that is to your [12:23] point EJ one of the blockers on scale [12:25] it's not the technology Y, but it's the [12:28] adapt adaptability of the workflows, [12:31] particularly in banking operations. [12:33] >> Yeah, maybe just to build off of that, [12:35] Young, and I I agree with your point [12:37] too, EJ, that you know, going into this [12:39] with a pre-benchmarked mentality is [12:41] really important to know what you want [12:43] to change. Uh, and we are definitely [12:45] very early in all of this too, uh, as [12:47] Yanick mentioned. Um, but yeah, we are [12:50] starting to see this trend from last [12:53] year. here. I think a lot of us were [12:54] thinking about chat bots and knowledge [12:55] management, faster answers, right? Um, [12:58] Agentic is pushing some of the way here, [13:00] but it's not just Agentic. We're [13:01] starting to get into workflow automation [13:03] and decisioning really, right? With [13:06] we've been doing that for some time over [13:08] structured data, right, using ML [13:10] techniques, right, to help us and assist [13:11] us there. But unstructured data, like [13:13] you mentioned, commercial real estate, [13:15] you know, a commercial real estate data [13:17] room, it's pretty gnarly, right? uh [13:20] you've got uh a lot of information there [13:23] to parse through and understand and it's [13:26] certainly not in some nice clean text [13:28] format only, right? And you know whether [13:32] you're working at scale or whether [13:33] you're more bespoke working sort of, you [13:36] know, within specific niches, you're [13:38] going to see stuff that doesn't look [13:40] like it has a pattern to it, right? And [13:42] so um but what we are seeing now is that [13:45] banks are succeeding in tackling this [13:48] complexity and it certainly takes a lot [13:51] more than models. It takes how you work [13:54] with the data, how you chunk the data, [13:57] what is the data between text and [13:59] graphical uh OCR technologies, guard [14:03] railing. It's really kind of a system [14:05] thinking. Some people have described it [14:07] as compound AI. You're starting to hear [14:08] this term around context engineering. [14:11] It's all kind of pointing to the same [14:12] thing which is we have to think about [14:14] these things as the data, the model, the [14:17] guard rails, the user experience. All of [14:19] this has to come together for it to [14:21] really work, get adopted, actually drive [14:24] uh an outcome. And I think the the point [14:27] that you raised, Young, is you know [14:30] getting people to trust this and adopt [14:32] it. The change management starts well [14:35] before you deploy in production. your [14:38] subject matter experts, the people that [14:40] are using these systems. Those are going [14:41] to be your best testers. Those are going [14:43] to be the people that identify the [14:45] unknown unknowns. You definitely want to [14:47] bring all those folks in to your uh your [14:51] beta processes. Really let them pressure [14:54] test these systems so that at the end of [14:56] the day, they know that they've [14:58] contributed to making sure that this is [15:00] going to really work. [15:03] So you know a lot of people I imagine [15:05] here in the audience or in general who [15:07] are looking and working with AI have a [15:09] mandate or been told by a board member [15:10] or a CEO that we need to do something in [15:12] AI right and then the next question is [15:14] all right well do we have a budget do we [15:16] have resources and that's always where [15:18] things got really really tricky I see [15:20] that a lot in my peers is like they want [15:22] to do things but then how do you [15:23] quantify the potential because Yanik you [15:26] said it's hard right to articulate the [15:28] impact but at the same time to start you [15:31] need to artic articulate some impact, [15:33] right? And so what are some of the [15:34] metrics or KPIs that you guys have seen [15:37] that work really well to convince a [15:39] board, to convince your management team, [15:41] to convince a client? [15:43] Maybe Yonik, I'll start with you since [15:45] you told me it was very difficult. [15:47] >> Well, I I wouldn't go that far, but uh [15:49] you know, you you have to think of AI as [15:52] software, right? AI is just a new kind [15:54] of software that costs more than the [15:57] traditional software, but that can do [15:58] more in in in exchange. So the way you [16:01] go about selling a specific AI project [16:03] is effectively the same way as you would [16:04] go selling a general software project, [16:07] right? You're either going to make more [16:09] money than before or you're going to [16:11] deliver the same for for less money or [16:13] you might have a side benefits side [16:16] benefits like customer satisfaction or [16:18] or risk reductions. But AI is not [16:22] special compared to traditional [16:23] software, right? It just does a little [16:26] bit more at a little bit higher cost. [16:28] But the way you measure software is the [16:30] same way you measure AI. [16:33] >> May I I I agree with you in most senses. [16:36] I think there's certainly um where a lot [16:41] of you guys are probably involved where [16:42] initial PC's are being rolled out. Um [16:46] some of the things we really focus on is [16:48] really the idea of this time to value, [16:50] right? So a PC does not generate the [16:52] time to value necessary where you have [16:54] the great returns that are there. Um and [16:58] and that's usually a disconnect from a [17:01] leader that says, "Well, you just turn [17:02] it on. It's just going to you're we're [17:05] just going to print money, right?" Well, [17:07] no, there so um that's something that we [17:10] spend a lot of time on and that's an [17:13] evolving metric, right? And so and [17:15] there's different ways there can be and [17:17] I use the the lending example, the [17:19] ability to change the way you do a loan [17:22] close from 45 60 days to 25 20. And so [17:26] that continuing movement shows an ROI [17:29] that says, "Hey, we've embedded this [17:31] into that automation, into that [17:33] workflow." That's real value generated. [17:36] That's generating money. So that's [17:38] that's that's one of the ways we're [17:40] attacking the the business cases is a [17:42] little different that way. [17:43] >> Yeah, I was going to ask you, EJ, how [17:45] you manage it? And also, I was going to [17:46] ask you, do you is there an argument to [17:48] be made that there is a little bit of a [17:49] leap of faith because it's such a [17:51] transformative even though it is a [17:52] normal software? [17:53] >> Let's go there. Let's go there on the [17:54] leap of faith first. Show of hands. [17:58] How many of you ever transferred money [18:00] on your banking app? Raise your hand if [18:02] you've ever transferred money. What if I [18:04] told you take a leap of faith? There's [18:07] about a 50% chance when you transfer [18:09] that money, it's going to get there. How [18:12] would you feel about that from your [18:14] bank? I don't think so. So, I mean, [18:17] jokes aside, right? [18:20] We are in a regulated industry. Whether [18:22] it's the obligation we have to get it [18:24] right for the client. Whether it's the [18:27] obligation and transparency we have to [18:29] get it right for our regulator or [18:31] regulators. Okay. Whether it's the [18:33] obligation we have to get it right for [18:35] ourselves because we want to build great [18:37] product. Okay. And this is the [18:39] double-edged sword of the last few years [18:41] because the barriers to entry to build [18:43] have come down which is great. But it [18:46] also means that there's a lot of people [18:48] out there who think they can build and [18:50] there's a lot of success theater and [18:53] there's a lot of people who have been [18:54] rightfully empowered to innovate but [18:57] there's a fundamental difference between [18:59] innovation and productization and [19:02] product market fit in banking and when I [19:04] keep talking about as other of my peer [19:07] panelists have been talking about in [19:08] terms of getting it right just because a [19:11] foundational large language model has [19:13] been benchmarked to be the equivalent of [19:15] a PhD does not make it a great credit [19:17] underwriter. [19:19] Not. So these are two fundamentally [19:22] different things and in the industry we [19:25] have a challenge of adopting in many [19:27] cases technology that is not fit for [19:30] purpose for banking. Now the promise of [19:32] it is there and in some cases the [19:34] reality of it is there. Okay. But only [19:37] after significant testing, your data [19:39] pipelines, your education, it takes a [19:43] lot to get an AI or generative AI [19:46] capability into production. Certainly [19:48] something that's client-f facing. So [19:51] what's going to happen over the next few [19:52] years is that it's going to be very [19:54] targeted, very precise, and very [19:56] provable. Okay. And then as the techn [20:00] the technology gets better, and you're [20:01] absolutely right, we are in this Palm [20:02] Pilot. Many of you in the audience have [20:04] no idea what Palm Pilots are. you're [20:06] looking them up. I can tell already. [20:07] Right. We all had them. Like we're [20:09] dating ourselves. Um but it's going to [20:12] take that progre progress and work year [20:14] after year, you know, to kind of get it [20:16] right. [20:17] >> Yeah, for sure. And I think just to add [20:19] to that from the from the builder and [20:21] consultancy side, right, our job is to [20:23] be good partners in that, right? Is to [20:26] move through the processes that need to [20:28] happen. Uh and it's I think you know [20:32] based on what we heard here there's a [20:34] business case to be made for this. you [20:36] have to run a business case if you're [20:38] going to change something like lending, [20:39] right? Uh if you want to build something [20:41] around, you know, building a chatbot on [20:43] like your notion docs, maybe not, right? [20:46] That's kind of an incremental routine [20:48] everyday thing, maybe at scale it makes [20:50] a difference, but when you're [20:52] transforming processes that are so close [20:54] to revenue and so close to risk, um you [20:58] definitely need to do that. Now, that [20:59] might sound daunting. It might sound [21:01] like a mountain, but I would start very [21:02] simply and ask the question, what if we [21:04] could, you know, 2x the speed of this [21:08] process with the team of what we have [21:10] today. What could our throughput be if [21:12] we did that? If we could improve our [21:14] speed to quote even in that first phase [21:17] of, you know, getting stuff into a [21:19] credit memo, don't worry about [21:21] decisioning this autonomously, but just [21:23] getting everything prepped. That's the [21:25] case that you need to make internally [21:27] that this is really worthwhile. I [21:28] promise you if you do that case you will [21:31] quickly find that there's a huge [21:32] opportunity here. I also find in [21:34] conversations in the industry that many [21:38] of us are not really aware of what that [21:40] ROI is like we actually have to really [21:43] work through that to understand it. [21:45] That's what really makes the difference [21:47] I think between becoming a change agent [21:49] in your company and a champion for this [21:51] stuff and not you need to validate for [21:53] yourself that this is worthwhile. And [21:55] then the second piece is you have to [21:58] accept that this is going to be a pretty [22:01] very very strong due diligence process [22:03] itself on the technology. You're going [22:05] to want to investigate all the different [22:07] ways you can build this. You're not [22:08] going to want to get married to your [22:10] first idea of how we can do this or [22:12] which partner or which vendor. You [22:14] really want to look at this very [22:15] cleareyed and agnostically in terms of [22:18] who can serve you today in the market [22:21] and what's coming around the corner, [22:22] right? and make sure that you always are [22:24] in control and have that freedom. [22:26] Tomorrow when we head to that expo hall, [22:28] you're going to see how many people are [22:29] innovating on your behalf, right? And [22:31] you want to be able to take advantage of [22:32] those things that make sense. But I [22:34] think that's the kind of key to thinking [22:36] about a value engineering process. We [22:38] think about that on the vendor side. I [22:40] think on the client side, you have to [22:42] think about running that process [22:43] yourself. Business case, tech validation [22:46] case. If you put those things in in an [22:48] organized way, you will make progress. [22:52] And so I think a couple of you mentioned [22:54] regulations, right? And banks being [22:56] obviously regulated, heavily regulated [22:58] and rightfully so. Um, so let's talk [23:01] about regulations and compliance for a [23:03] little bit because there's obviously a [23:04] balance to maintain between speed to [23:06] market and the need to innovate versus [23:08] obviously being regulatory compliant. [23:10] And one of the things that at least I [23:12] find for regulators these days, it's [23:14] really hard because it's not just AI, [23:16] it's crypto and a bunch of other [23:17] different things. And so for regulators [23:20] to even keep up right in terms of [23:22] capability and understanding and then [23:24] being able to regulate proactively on [23:26] something that as you said Yanik we [23:28] don't even know where it's going to go [23:29] right like it's a difficult job so how [23:31] do you manage that and maybe I'll start [23:33] with young you know in terms of the [23:34] conversation you have with your clients. [23:37] >> Yeah I I would say for the regulators a [23:40] lot of the things we're seeing are [23:42] around the ability to to recreate [23:44] something. So I'll touch on the customer [23:45] experience side right so much is built [23:48] in the automation of the workflow so [23:49] much is in the offer presentment all [23:52] these things that really push the [23:54] customer through but the ability to [23:56] replicate that has been sort of where [23:58] we've worked with the FIS is very very [24:00] important right so there's there's two [24:02] sides of this certainly how do you [24:03] manage your risk but the other thing is [24:06] once something happens how do you um [24:09] replicate what you have so that you're [24:12] able to uh mitigate that manage that [24:14] risk and or um work with the regulators [24:17] around that. And so I think that side of [24:19] the equation I think is probably the [24:21] most ignored side. And but that's the [24:24] area that we're spending a lot more of [24:26] our time on right now because um to to [24:30] your your point EJ everyone's really [24:33] spending a lot of time to how do I [24:34] expedite too through things but the [24:37] ability to say hey did we build [24:39] something that actually has leakage or [24:42] something that exposes us um and being [24:45] able to go back and and isolate on that [24:47] I I think is is certainly the uncharted [24:50] area. So hopefully that doesn't really [24:52] answer it directly, but it tells you [24:54] where we're seeing uh a lot more of the [24:56] opportunity that I think a lot of the [24:58] banks have sort of neglected. [25:00] >> Maybe we'll ask one of the banks uh EJ [25:02] maybe like what's what's your what's [25:04] your thought on that in terms of finding [25:06] that balance between you know innovation [25:08] versus regulation, speed to market [25:11] versus obviously being you know [25:13] compliant within a certain risk appetite [25:15] framework. [25:17] I I don't actually believe building [25:20] great product at pace in a regulated [25:24] industry are mutually exclusive. I don't [25:27] I've been doing this long enough and [25:28] I've seen both the good and the bad. And [25:30] I actually think um with the right [25:32] partnerships and the right approaches [25:35] um you're able to do all of those things [25:38] on, you know, perhaps on an incremental [25:40] basis because this is now a function [25:41] more of the technologies ability to be [25:44] fit for purpose in banking. But [25:45] nonetheless, it doesn't mean it's a [25:46] zero- sum game. You can actually build [25:48] towards progress. The other variable [25:50] with this is that um when you look not [25:53] just across, you know, North America, [25:55] but across the world, there is a high [25:56] degree of variability in terms of the [25:58] regulatory approaches. Some regulators [26:01] take a much more prescriptive approach [26:02] than others, whereas others are let much [26:04] more let the market decide. Um, [26:07] nonetheless, you know, every bank I know [26:10] has an enterprise risk framework as a [26:12] backs stop. So regardless of almost what [26:15] the regulation says, you know, you're [26:16] going to make sure that across your [26:18] three lines of defense, you're doing [26:20] right, you know, by your enterprise risk [26:22] framework. And fundamentally, that means [26:24] you have to be able to measure things. [26:26] You have to be able to repeat things. [26:28] You have to be able to prove [26:29] mathematically in most cases that you [26:31] have a good idea of what the next [26:33] outcome is going to be. and your ability [26:35] to do that across a range of business [26:37] processes. You know, when you're talking [26:39] about capital markets and trading, well, [26:41] that's going to be a pretty significant [26:43] hurdle when it comes to model validation [26:45] and the math. Whereas, if you're looking [26:47] at something like a more general purpose [26:50] knowledge agent that's just pulling [26:51] from, you know, low-risk documentation [26:54] where you have multiple humans in the [26:55] loop, that's a different risk profile. [26:57] So it's your ability to really navigate [27:00] your organization's risk framework, [27:02] correlate that across the impacted [27:05] regulations and ultimately at the end of [27:07] the day create value. You know it is all [27:09] going back to how you create value. But [27:12] again I don't think delivery at pace, [27:14] building great products, innovation and [27:16] creating great products are mutually [27:18] exclusive. It's just a pipeline process [27:21] you've got to be doing every single day [27:23] and have the talent to do it. [27:26] >> Yeah. And I think you know you've [27:28] mentioned the word product several times [27:30] which is the right way to think about [27:31] this stuff right it's [27:32] >> thanks [27:33] >> the product process that we've that [27:36] we've all been taught right you start [27:37] with an idea right you go through it you [27:40] prototype it you test it you continue to [27:43] refine it until you bring it out um [27:46] that's treating things as a product [27:48] right um rather than hey let's just kind [27:50] of spin this thing really fast and push [27:52] it out right um as you're building a [27:56] product, what you want to be able to do [27:57] with that product is to decompose it and [28:00] see through it at any time, right? So, [28:02] what's really critical is, you know, if [28:05] you have one of these circumstances [28:07] where something does go ary, you are now [28:10] accountable to be able to have to be [28:12] able to remedy that and the first thing [28:14] you need to be able to do is see where [28:15] is where's the problem coming from. And [28:18] so, you know, adopting standards, [28:21] reusable patterns, ways to build, being [28:24] able to isolate different parts of how [28:27] the system works from, where's the data, [28:29] how is it being processed, where's the [28:32] prompt, where's the model, where's the [28:35] guard rail on that, where's the branch [28:37] or the agentic loop here, which tools is [28:39] it calling? you know, all of that needs [28:41] to be very transparent and auditable so [28:44] that you can go through um you'll do [28:46] this in the process of building to make [28:48] sure that it works the way that you [28:49] intend, but on that rainy day if [28:52] something does go wrong, that's what [28:54] you're going to want. You're not going [28:56] to want something that is either a [28:58] blackbox by design or something that was [29:01] built kind of so individually within [29:04] your organization that it becomes sort [29:07] of unentangling a bunch of code to [29:10] figure out oh gez where is where is this [29:12] coming from and trying to kind of like [29:14] suppress the butterfly effect right of [29:17] of that so I think you know transparency [29:20] standards control um it's one of the [29:23] reasons why I think we see especially [29:25] especially in banking that the [29:27] open-source technology ecosystem is one [29:29] that is so readily adopted, right? Is it [29:32] it's not a uh a choice as a nice to [29:34] have. It's because you need to have that [29:37] that level of granularity and control [29:39] over the systems. [29:42] >> Maybe one more word on the on the [29:44] governance aspect. One more difference [29:46] between traditional software and AI is [29:49] the non-deterministic aspect of AI, [29:51] right? And that has several [29:52] consequences. One of them is if you want [29:55] to use AI to perform a specific business [29:58] process, you're typically not sure [30:00] before trying if the AI will be good [30:03] enough for that specific process, right? [30:05] So, you need a like a two-phase [30:06] governance framework. You need to allow [30:09] easy uh exploration, easy PCs um to [30:13] decide whether something is worth [30:15] pursuing or not. And then if you decide [30:18] that something is worth is worth [30:20] pursuing if you know the eye is good [30:21] enough then as you just said you will [30:24] have your traditional risk framework [30:25] right you're looking at you're going to [30:27] look at every kind of risk you look for [30:29] any other project financial risk credit [30:32] risk market risk operational risk and [30:35] everything else uh and you will also [30:37] have some added risks that have to do [30:40] with the nondeterministic aspect of AI [30:42] right typically what happens if the AI [30:44] makes the wrong decision or input some [30:47] or output some things that something [30:49] that you didn't expect and uh your risk [30:52] framework has to be enhanced to cover [30:55] that kind of risk. [30:57] >> Yeah. And I I'd only add to that, [30:58] Yanick, that that um that only grows [31:01] with Agentic. And I'm sure you guys [31:03] thought through this as you were [31:04] building your first Agentic systems is [31:07] how much power do we give, right? And [31:09] which parts of what we built need to be [31:12] highly deterministic versus, you know, [31:15] giving power and and uh agency, right, [31:18] to to the models to make decisions. Um I [31:22] think I've heard a lot of that discussed [31:24] as sort of an eitheror. It's really a [31:26] spectrum and you know you should at [31:29] least you know know that in building [31:31] these systems you don't need to make a [31:33] hard choice between what is you know [31:36] very articulated left to right this must [31:38] happen and which parts you can give more [31:40] agent control to but at the end of the [31:42] day you're right the models are [31:44] non-deterministics themselves which is [31:46] their beauty and their strength in terms [31:48] of what they're able to generate. Uh but [31:50] you do have to control for that. Um, and [31:52] in some cases we even see folks dial [31:55] that all the way back in certain use [31:56] cases where I don't want generation, [31:59] right? In this case, I can't actually [32:01] afford the risk. I'd rather actually go [32:03] roll back to a former NLP technique and [32:06] let me just extract this, right? Because [32:07] at least then I know in this case if [32:09] there's really zero risk, I'm going to [32:11] use an extraction technique here. I'm [32:13] not actually not going to use any [32:14] generation at all. So these are I think [32:18] good things to know in terms of you know [32:20] just how much uh you know freedom and [32:23] control you can have over these systems. [32:25] I think we often think that the models [32:27] are in control. We have to exert our [32:29] control over them. [32:31] >> And so it's really interesting hearing [32:33] all of you talk right because it's very [32:34] like considerate. It's very pragmatic [32:37] very realistic. And then I contrast that [32:39] with an article that I read last week [32:41] from McKenzie that talked about the AI [32:43] bank of the future, right? Which is [32:45] we're almost ready to have a bank that's [32:47] fully operational, purely AI, no humans [32:49] in the loop. And I can see that I can [32:51] see I can see the reactions. But it's [32:54] it's also interesting because I think [32:55] when I whenever I'm talking to people, [32:57] that's what people think where the [32:59] technology already is, right? And that's [33:01] why I asked that question is that people [33:02] tend to think that in the next couple of [33:04] months we're going to have like drastic [33:06] completely changes. And what I'm hearing [33:08] from you, unless someone wants to [33:09] disagree with me, is that actually [33:11] things move a little bit slower and by [33:13] design, right, to kind of make sure that [33:14] we stay within that regulatory risk [33:16] mindset and also that we don't break [33:18] things that really should not be should [33:20] not be broken. Um, we are I think we're [33:23] at the last 10 minutes, so we can open [33:25] it up for questions if there's any [33:26] questions uh from the a Okay, we got one [33:29] right here. You want to use your voice [33:30] and just [33:32] Oh, [33:33] >> she's coming. She's running. [33:38] Hello. Okay. Um, so my name is Kevin [33:42] Carticios. I'm from a AI consultancy uh [33:44] company and I would like to know you [33:47] mentioned that systems in banking had to [33:49] be very robust and very tested and AI is [33:52] kind of very new and experimental. So [33:56] how do you manage to balance that out [33:57] when selecting a good use case? [34:04] So AI is not new to banking in many [34:08] cases. Uh traditional more deterministic [34:11] approaches have been the heart and soul [34:13] of many banking processes for decades. [34:16] Um you in many case in your personal [34:20] banking life uh experienced that in ways [34:22] you wouldn't even realize. But over the [34:24] years through a variety of data and [34:27] transformative efforts, banks large and [34:30] small have made efforts over the years [34:32] to, you know, create efficiencies and [34:34] create better experience. The reason why [34:36] we're talking the panel today, clearly [34:38] on November 30th of 2022, the world [34:42] changed a bit because now in the palm of [34:45] everyone's hand, they got a new thing [34:48] and that became very accessible and it [34:50] changed consumer sentiment and the [34:53] perception of what AI could do, what it [34:56] could not do. And every single day on [34:59] that journey, it feels like every [35:00] second, you know, consumers have been on [35:03] this wild ride, you know, that we all [35:05] have been experiencing in terms of these [35:07] emerging capabilities. That said, that [35:11] brief and marginally humorous thought [35:13] experiment I did before around [35:15] transferring money, that still holds [35:16] true. You have to trust your bank as the [35:20] steward of your currency and your [35:22] identity to be able to fulfill certain [35:25] processes with nearly perfection. You [35:29] know, you would not expect your bank to [35:32] get certain things wrong. Okay? We can't [35:35] get it wrong whether it's by the law or [35:36] consumer expectation. So in certain [35:39] cases when we're choosing use cases, [35:41] it's the traditional approach of what's [35:43] viable, what's feasible, what creates [35:45] value, what's the competition doing. Um [35:48] it's a range of variables that are going [35:50] to go into that very um critical [35:54] calculus to determine what we do. And [35:56] like most banks, we're operating in a [35:58] multi- horizon, you know, kind of [36:00] approach. We're testing things in safe [36:02] and secure environments that are way out [36:03] there. But the stuff that a client or [36:06] consumer may actually see, you know, is [36:08] stuff that we have very high confidence [36:10] conviction that we can put in front of [36:12] you and it's going to work. Um, I don't [36:15] foresee that changing. I think the pace [36:17] of innovation is going to go even [36:18] faster. But those fundamentals of [36:20] product market fit and banking to me are [36:22] going to be universal. We could [36:24] literally be having the same [36:25] conversation about quantum and it would [36:27] still be the same foundational things. [36:29] It doesn't matter what the technology [36:30] is. [36:34] Hello. Hi, I'm Franchesca. Um, I have a [36:38] question uh about Yeah, in normal [36:42] software uh you have different [36:44] departments. Each department uses a [36:46] normal software and they need to [36:48] integrate together to get data, right? [36:51] That's and in traditional software, we [36:54] already have the problem of integrating [36:57] these tools together, right? and [36:59] connecting the fields and data to find [37:01] out information. So I'm wondering with [37:04] the new AI tools, have you taken that [37:08] into account and how do you deal with [37:10] that? [37:14] So, you know, integrating different tool [37:16] sets together. Um you know again I think [37:20] um [37:23] looking at [37:25] each use case will help you understand [37:28] what needs to work in the back end and [37:31] the flow of work. What's going to meet [37:32] the expectations whether it's my [37:34] consumer or whether it's you know my [37:36] legal team my lending team my quant team [37:40] whatever you know whatever those [37:41] different groups are. If you work [37:43] backwards from the use case, what you're [37:45] going to want to do is understand what [37:47] are those critical requirements that [37:48] need to come together, then at that [37:50] point you have to look at and say, okay, [37:53] what is the best way to bring AI into [37:56] this process in a way that's going to [37:58] safely work with all of them, right? Um, [38:01] there's certainly a lot of great secure [38:04] API connection ways to do this. There [38:07] are emerging new standards that I would [38:09] probably be a little bit careful of in [38:11] this industry like MCP from anthropic. [38:14] Um but that will continue. There will be [38:17] continue to be more and more ecosystem [38:19] plays where it becomes faster and faster [38:22] and easier to connect these things [38:23] together. Um but it's certainly very [38:26] possible to do today. Um and I would [38:29] think about AI as you know something [38:31] that can kind of live very invisibly [38:34] within the fabric that you already have. [38:36] you don't need to sort of re-imagine [38:38] your entire enterprise tech stack. Uh [38:41] you really just have to look at who are [38:44] the key partners and players that can [38:46] really plug in with those partners that [38:49] we've already made big in investments [38:51] with um and can give us agility down the [38:54] road for what we don't know is coming [38:55] yet. Because we know just living through [38:58] this over the last two or three years, [39:00] it's every day, every week, every month, [39:03] every quarter, every year. there's [39:05] always something new, right? And so all [39:07] that competition is happening. But um [39:10] yeah, integrating these together with [39:12] AI, if you look into some of the [39:14] compound AI, um literature, this will [39:17] kind of help you understand the sort of [39:20] system thinking of taking a model and [39:22] making it work with all the assets that [39:24] you already have and doing it safely. [39:29] >> Very interesting discussion. Uh so you [39:32] talked about uh you know the the [39:33] regulation or the regulatory aspects. So [39:36] in addition to uh you know the the [39:39] regulation that you mentioned so we have [39:41] also been hearing about like you know [39:42] the AI regulation uh you know already [39:45] there or maybe coming soon on the state [39:47] level as well like you know California, [39:49] Colorado, New York and also like you [39:53] know at uh EU AI act you know say so my [39:56] question was that you know that from a [39:57] banking uh you know the risk management [40:00] perspective so is is that you know state [40:03] level regulation or like you know [40:05] international regulation directly or [40:06] indirectly influencing your you know [40:09] risk management strategy. Thank you. [40:13] >> The law is the law and where we operate [40:15] we have to adhere to the law. [40:18] The complexity of being able to [40:20] implement a solution becomes nonlinear [40:23] the more that you have to figure out and [40:26] manage the requirements of the totality [40:29] of that legal and regulatory landscape. [40:32] So to the extent there's a technology [40:35] agnostic and neutral approach that is [40:37] more predicated on measurable standards [40:40] and principles but really leaves it up [40:43] to the organization to determine the [40:45] best path forward. That one would be [40:47] helpful. two, um, far be it for me to [40:50] comment on the variety of executive [40:53] orders and actions that have taken [40:55] place, but there have been attempts and [40:58] approaches to kind of normalize that in [41:00] the United States, both at the federal [41:03] level and at the state level by using [41:05] the interstate commerce clause, which [41:07] has ironically been used in the past. If [41:09] you go way back in time in the history [41:11] of railroads, they use the interstate [41:13] commerce clause to normalize railroad [41:15] operations United States. So that when [41:17] every state you didn't run into a [41:19] different railway gauge, that was laid [41:21] the groundwork for the transcontinental [41:22] railroad. They're trying to do the same [41:24] thing with AI, which is why they're [41:26] bringing in the interstate commerce [41:28] clause to normalize that at the federal [41:30] and state level. Even if that's done [41:32] depending upon your bank's operational [41:35] footprint. My prior organization was in [41:38] 65 countries around the world and then [41:40] you layer in different states and other [41:42] things. So the bottom line is the law is [41:44] the law. It must be adhered to. If [41:46] you're going to release a product that's [41:48] table stakes to do so effectively do [41:51] those table stakes that's nonlinear in [41:54] terms of complexity and that may delay [41:56] actual delivery of product if you have [41:58] to get it right in all those [41:59] jurisdictions. [42:00] >> Yeah. And I think banks o over history [42:02] something you touched on it's [42:05] multi-state multi-state products [42:07] multi-state regs have always been there. [42:10] So I don't think it's a huge adjustment [42:12] for someone to say this is how [42:14] California operates versus Texas versus [42:17] Canada versus the EU. That that's been [42:19] persistent uh in how they've managed [42:21] that business and the risk models. [42:23] >> And we we already have a model for that, [42:25] right? It's privacy like lots of [42:27] countries and even states have different [42:29] privacy laws. Of course, banks have to [42:31] uh follow all of them. [42:36] >> Uh I'm Jenny Yu. I'm the head of model [42:39] risk management at the Federal Home Loan [42:41] Bank of Topeka. I'll do a promotion [42:44] first. I'll do AI risk management on [42:47] Wednesday 1220. Just follow that [42:50] discussion. Um financial industry has a [42:53] 20 years uh regulation on managing the [42:56] blackbox risk. I I'm going to talk about [42:59] how to translate that framework into AI [43:01] risk management. And I have a question [43:04] to speakers. So um do you have do you [43:09] see the banks has a AI policy? If you do [43:12] who owns it? [43:14] >> Thank you [43:17] AI policy. [43:19] >> So is it distributed or where you know [43:22] different maturity models for AI [43:24] adoption? [43:25] >> I can speak for for our case. So we have [43:27] an AI risk team that sits in line two um [43:31] and they define the the policy what's [43:33] allowed or not. [43:35] Each risk domain is going to apply it to [43:38] their respective risk domain because [43:39] they're the subject matter experts in [43:40] their risk domain. And then there's a [43:42] holistic AI risk directive that goes [43:45] across to ensure that every risk domain [43:47] participates. [43:51] >> Yeah. Yeah. And I'm also in the bank. I [43:53] forgot. Sure. I'm a banker. Yeah. It's [43:55] very similar for us. Risk owns the [43:57] policy. But I think my team, which is [43:59] where AI sits, we kind of help architect [44:01] a lot of it more practically because [44:03] again, and I don't know how it's for you [44:04] guys, but our risk people don't really [44:06] understand AI or crypto, which is [44:08] another big thing we're pushing for. And [44:09] so we talked about regulators. I think [44:11] even for risk professionals, compliance [44:13] professionals to genuinely understand [44:15] what the substance is in order to then [44:17] be able to uh apply a risk framework has [44:20] been for us at least quite a bit of a [44:21] struggle. [44:23] >> I think we have forunately, we have time [44:25] for just one more question back here. [44:29] Uh, hi, my name is K Leong and I'm with [44:31] Zeltech. We support some of the largest [44:34] banks in the world with regard to [44:36] governance. And here's where it gets a [44:38] little bit tricky. [44:41] The biggest source of data is obviously [44:45] the ecoms and unstructured data which is [44:48] subject to governance. Unfortunately, in [44:51] governance there's a lack of trust. So [44:55] therefore it requires deterministic [44:59] uh searches and so forth. On the other [45:02] hand, Genai is working in the opposite [45:04] direction which is more a black box and [45:09] but still it's looking for that kind of [45:11] data. [45:13] Have you at the big banks addressed this [45:17] in any way and if so can you share it? [45:25] you've articulated the heart of the [45:26] scaling issue. Thank you very much. [45:30] We're working on it. [45:33] >> Yeah. I just say a few things here that [45:35] we've seen practically happen dayto-day [45:37] is when you are going into generation on [45:41] unstructured data. Um [45:44] you know being able to uh site where [45:48] that information comes from being able [45:50] to look up and verify that that [45:52] information has indeed actually been uh [45:55] come from this data source. being able [45:57] to look at certain factors like how [45:59] complete was the search for that [46:01] information across our incor our entire [46:04] corpus of data. These are some of the [46:06] measurement techniques uh level of [46:08] groundedness, how many queries actually [46:10] do you have citation versus not. These [46:13] are some of the toolings that are [46:14] starting to come into play that are [46:17] assisting to make sure that the model is [46:19] behaving essentially uh with the data [46:22] that you have and using it to its [46:24] fullest extent. [46:26] I think our time is up. Uh I think the [46:28] guys will be around if you want to grab [46:30] them for selfies, conversations, [46:32] whatever you want to do. But thank you [46:34] so much for being here with us. Thank [46:36] you to the panelists and I hope you [46:38] enjoy the rest of the conference. Thank [46:39] you.