FHLB Topeka Adapts Post-Crisis Bank Rules to Police Agentic AI

AI Risk Management: A Practical Blueprint from a Proven Framework on Wall Street With FHLBT — Ai4

Ai4

Banks can safely deploy complex generative and agentic systems only by validating each autonomous component individually under SR 11-7 standards while holding human operators strictly accountable for algorithmic outputs.

AI-assisted summary of AI Risk Management: A Practical Blueprint from a Proven Framework on Wall Street With FHLBT — original source. Check the original for full context.

Source published: 2025-08-28

Detailed notes

## Background Published on YouTube by Ai4 on August 28, 2025, this presentation features Jenny Yu, Director of Model Risk Management and Methods at the Federal Home Loan Bank of Topeka. The talk outlines how risk leaders and practitioners can adapt Wall Street's time-tested three-pillar model risk management framework—governance, owner responsibility, and independent validation—to safely govern complex AI and generative systems.

## TL;DR Jenny Yu demonstrates how the financial sector's established SR 11-7 model risk management standards provide a practical blueprint for governing modern AI and generative workflows. Framing risk controls as strategic enablers rather than bottlenecks, she details the fundamental risk equation (Residual Risk = Inherent Risk minus Controls) and categorizes distinct risk profiles across traditional, generative, and third-party vendor models. She illustrates how institutions manage financial forecasting and modular agentic AI pipelines through granular, component-level performance monitoring and independent validation. The talk concludes with pragmatic guidance on vendor LLM due diligence, relative model benchmarking, and maintaining rigorous human-in-the-loop accountability.

## Atmosphere & style The presentation carries a pragmatic, authoritative, and reassuringly structured tone. Delivering a lunchtime industry session, Yu leverages over two decades of quantitative banking experience, demystifying abstract AI anxieties with relatable metaphors like playground guardrails and personal bathroom scales. It feels grounded, educational, and methodical, replacing speculative hype with battle-tested Wall Street discipline to give enterprise practitioners psychological safety and actionable clarity.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) AI risk management acts as a strategic enabler rather than a bureaucratic bottleneck, providing the protective guardrails that give business leaders the psychological safety to deploy advanced AI boldly. - [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) The financial sector codified structured model risk management frameworks following the 2007–2008 global financial crisis, yielding over two decades of battle-tested methods for governing black-box algorithms. - [03:57](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=237s) AI risk governance is governed by the balancing equation Residual Risk = Inherent Risk - Controls, where management implements controls to lower exposure and consciously tolerates the remaining residual risk within established appetite limits. - [05:39](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=339s) Risk governance cannot use a one-size-fits-all checklist: traditional predictive models require empirical back-testing, generative AI demands human-in-the-loop review, and vendor solutions require supply-chain due diligence and independent replication. - [07:10](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=430s) Across the quantitative modeling lifecycle, the two primary root causes of model risk are incorrect development (flawed assumptions or poor bias-variance tradeoffs) and inappropriate use outside intended operational boundaries. - [07:10](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=430s) The regulatory SR 11-7 model risk management standard relies on three essential pillars: top-down governance, model owner operational accountability, and independent validation by objective technical teams. - [12:16](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=736s) Deploying models in business workflows requires setting explicit, quantified error tolerance thresholds that systematically balance forecast uncertainty against business utility. - [14:56](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=896s) In multi-agent AI architectures, governance and performance monitoring must be executed step-by-step at each granular agent component level rather than treating the system as a monolithic pipeline. - [15:41](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=941s) Generative AI introduces novel attack surfaces such as malicious prompt injection and chatbot data leakage, necessitating deep vendor due diligence regarding training data domain relevance and data retention policies. - [18:34](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1114s) AI evaluation tools should be non-invasive, automated, and API-accessible, providing consistent relative benchmarking across alternative models even when absolute precision is unattainable. - [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) Human-in-the-loop governance ensures end-user accountability: whether working with pen and paper, spreadsheets, or generative AI, individual users remain responsible for verifying outputs based on risk magnitude and likelihood.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) AI Risk Management as a Strategic Enabler: The Playground Analogy

Jenny Yu opens the session acknowledging the lunch-hour slot, welcoming attendees and setting the stage with her 20-plus years of experience in the financial industry developing, managing, and validating models. Over the past two decades, banking and capital markets have developed rigorous, battle-tested methods for managing "black box" algorithmic risk. Rather than treating risk management as a bureaucratic bottleneck, Yu frames practical, effective AI risk management as a vital "strategic enabler." She illustrates this philosophy using the metaphor of a playground surrounded by sturdy guardrails: children can run around freely and happily precisely because the perimeter is secured. In the same way, robust risk guardrails give business leaders the confidence and psychological safety to deploy advanced AI systems boldly.

The financial sector's formal approach to managing black-box model risk was forged in the aftermath of the 2007–2008 global financial crisis. Although quantitative models had been used for years, the crisis compelled major institutions and regulators to codify structured risk frameworks. Having transitioned from a hands-on data scientist to overseeing model risk management across investment strategies and operational efficiency, Yu bridges the gap between raw data, complex mathematical algorithms, and high-level corporate strategy.

```mermaid flowchart TD accTitle: Relationship Between Risk Guardrails and Business Enablement accDescription: How risk management guardrails enable safe and confident AI adoption. subgraph Safeguards["Risk Management Foundation"] A["Regulatory Precedents & Wall Street Frameworks"] B["Safety Guardrails & Risk Controls"] end subgraph Execution["Strategic Enablement"] C["Clear Risk Tolerances"] D["Safe, Confident Business Adoption"] end A --> B B --> C C --> D ```

---

[03:57](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=237s) The Foundational Risk Equation: Inherent, Controls, and Residual Risk

To demystify AI risk, Yu introduces the core mathematical intuition of enterprise risk management. The level of risk is governed by a fundamental balancing equation:

$$\text{Residual Risk} = \text{Inherent Risk} - \text{Controls}$$

Inherent risk represents the raw, unmitigated exposure stemming from model complexity, business criticality, and financial or operational impact. Controls and guardrails are engineered to lower this exposure. Whatever remains is the residual risk—the unavoidable baseline uncertainty that executive leadership must evaluate against their organizational risk appetite and consciously choose to tolerate.

Yu cautions that while generative AI is often treated as an unprecedented and unmanageable threat, decomposing it through this foundational lens reveals that its underlying operational and reputational risks are not entirely novel. Successfully managing AI requires systematically mapping the specific risk mechanisms an algorithm invokes, estimating the likelihood and magnitude of adverse outcomes, and establishing proportionate controls rather than giving in to paralysis.

- **Inherent Risk:** Driven by algorithm opacity, dependency on complex data pipelines, and potential business disruption. - **Internal Controls:** Technical benchmarks, human checkpoints, input/output validation, and performance triggers designed to suppress failure modes. - **Residual Risk:** The remaining risk footprint that must be formally tracked, reported, and accepted within defined corporate tolerance thresholds.

---

[05:39](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=339s) Categorizing AI: Traditional, Generative, and Vendor Systems

Because "not all AIs are created equal," risk governance cannot apply a one-size-fits-all checklist. AI systems must be segmented by archetype, operational context, and verification mechanism:

| AI Archetype | Core Methodologies & Examples | Primary Risk Profile | Verification & Mitigation Strategy | |:--- |:--- |:--- |:--- | | **Traditional AI / Predictive Models** | Machine learning, statistical regression, stochastic time-series forecasting. | Pure black-box calculations; impossible for a human to audit or verify intermediate arithmetic manually. | Empirical validation methods: historical back-testing, benchmark comparison models, and stress testing. | | **Generative AI** | Large language models, text/code generation, content synthesis, semantic retrieval. | Hallucinations, ungrounded outputs, reasoning drift, and non-deterministic behavior. | Human-in-the-loop workflows where domain experts review, edit, and substantiate synthesized outputs. | | **Vendor & Third-Party AI** | Out-of-the-box vendor platforms, embedded enterprise software features. | Opaque intellectual property, unknown underlying training data, third-party supply chain risk. | Comprehensive vendor due diligence questionnaires, targeted audit rights, and independent output replication. |

---

[07:10](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=430s) The Holistic Modeling Process and the SR 11-7 Three-Pillar Framework

Every quantitative modeling lifecycle begins with a well-defined business objective and concludes with actual business execution. Consequently, the two primary sources of model risk are **incorrect development** (flawed assumptions, data leakage, poor bias-variance tradeoffs) and **inappropriate use** (deploying a model outside its intended operating boundaries).

To structure this lifecycle, banking organizations rely on regulatory standards such as **SR 11-7** (Supervisory Guidance on Model Risk Management), structured around three core pillars:

```mermaid flowchart LR accTitle: SR 11-7 Three Pillars of Model Risk Management accDescription: Three pillars consisting of Governance, Model Owner Responsibility, and Independent Validation. subgraph Pillar1["Pillar 1: Governance"] G1["Tone from the Top"] G2["Risk Appetite & Policies"] G3["Quarterly Board Reporting"] end subgraph Pillar2["Pillar 2: Owner Responsibility"] O1["Development & Testing"] O2["Performance Tracking"] O3["Model Documentation"] end subgraph Pillar3["Pillar 3: Independent Validation"] V1["End-to-End Code Audit"] V2["Vendor Model Replication"] V3["Sensitivity & Stress Testing"] end Pillar1 --- Pillar2 Pillar2 --- Pillar3 ```

- **Pillar 1: Governance & Oversight:** Executive leadership sets the "tone at the top," defining explicit risk tolerances that weigh potential productivity gains against financial and regulatory exposure. Models are inventoried, scored for criticality, and monitored as part of enterprise risk management (ERM) with formal quarterly reporting to the Board of Directors. - **Pillar 2: Model Owner's Responsibility:** Model owners—whether internal developers or business units deploying vendor solutions—own the operational health of their tools. They must maintain clear conceptual documentation, monitor performance drift over time, establish automated change detection, and enact remediation plans whenever accuracy drops below prescribed standards. - **Pillar 3: Independent Validation:** An objective, mathematically qualified validation team (data scientists and quantitative engineers independent of the model's builders or business users) reviews every component from input data ingestion to final outputs. At institutions like FHLBank Topeka, internal quants frequently reverse-engineer and replicate third-party vendor models to stress-test their sensitivity to shifting macroeconomic assumptions.

---

[12:16](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=736s) Practical Banking Case Study: Net Interest Income Forecasting and Agentic AI

Yu demonstrates how these governance principles function in practice through a core banking workflow: forecasting net interest income. While the basic calculation appears simple on the surface—lending $\$100$ at a $6\%$ interest rate generates $\$6$ in income—predicting aggregate net interest income across dynamic bank balance sheets subject to fluctuating market rates is highly complex.

To manage this modeling risk: 1. The forecast model is cataloged in the institutional model inventory and assigned a risk tier. 2. Independent validation conducts continuous back-testing against ground-truth historical performance, breaking down forecast deviations into distinct components (interest rate movement vs. balance sheet volume shifts). 3. Senior leadership sets explicit, quantified error tolerances. For example, management may formally accept an aggregate income variance within $5\%$ or an underlying interest rate projection error within $1\%$, balancing forecast uncertainty against business utility.

Yu links this established workflow to modern **agentic AI** pipelines. If a bank decomposes complex income modeling into multiple autonomous agents—such as one generative agent parsing balance sheet data and another querying and projecting interest rate curves—the same SR 11-7 governance mechanics must apply. Each modular agentic component requires transparent assumptions, quantifiable error tolerance thresholds, and rigorous verification against empirical benchmarks.

[14:56](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=896s) Agentic AI Workflows and Granular Component Monitoring

When working with complex, non-trivial financial calculations that cannot simply be retrieved directly from the internet, organizations often deploy multi-agent systems. In such an architecture, multiple agentic AIs collaborate: one agentic AI may manage interest rates or rate parameters, while another handles downstream computation or orchestration. While this modular breakdown is an effective way to operationalize large language models within specialized workflows, risk management cannot simply evaluate the end result in aggregate.

To maintain balance and effective governance, performance monitoring must be performed at each individual agentic component level rather than treating the overall pipeline as a single monolithic entity. By validating and monitoring each agent step by step, risk teams can pinpoint errors, trace lineage, and maintain proper model performance controls across compound systems.

```mermaid flowchart LR accTitle: Multi-Agent Monitoring Workflow subgraph MultiAgentSystem ["Modular Agentic AI Architecture"] A["Agent 1: Data / Ingestion"] --> B["Agent 2: Rate Management"] B --> C["Agent 3: Complex Calculation"] end subgraph RiskGovernance ["Granular Risk Monitoring"] M1["Monitor Agent 1"] -.-> A M2["Monitor Agent 2"] -.-> B M3["Monitor Agent 3"] -.-> C end ```

[15:41](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=941s) Generative AI Risks, Malicious Prompts, and Vendor LLM Due Diligence

Moving from traditional quantitative models to generative AI introduces an "even blacker box." When organizations directly use a large language model off the shelf without fine-tuning on proprietary data, they have very little inherent control over its inner workings. Generative AI introduces distinct risk categories, notably hallucination, inaccuracy, and novel data breach vectors.

``` ┌────────────────────────────────────────────────────────┐ │ Existing Enterprise Controls │ │ (Data Risk Governance, Cybersecurity, Access Policies)│ └──────────────────────────┬─────────────────────────────┘ │ ┌───────────────┴───────────────┐ ▼ ▼ ┌──────────────────────┐ ┌──────────────────────┐ │ Traditional Risk │ │ GenAI-Specific │ │ Infrastructure │ │ Threat Vectors │ │ - Network breaches │ │ - Malicious prompts │ │ - Exfiltration │ │ - Output injection │ │ - Data permissions │ │ - Chatbot leakage │ └──────────────────────┘ └──────────────────────┘ ```

While institutions already have mature data risk governance and cybersecurity frameworks to manage traditional data leaks, large language models introduce interactive attack surfaces, such as malicious prompt injection and interactive chatbot extraction. When incorporating internal data or evaluating third-party vendor models, model risk teams must perform rigorous due diligence by asking specific questions:

- **Training Data Relevance:** Ask what data was used to train the vendor model and whether it is directly relevant to your specific business domain. A model trained to extract financial data from PDF files might perform poorly when drafting legal real estate leases. - **Sector-Specific Performance:** Inquire not just about general benchmark scores, but how the model performs within your precise industry sector. - **Data Retention and Privacy:** Verify whether the vendor captures your submitted enterprise data, whether that data is fed back into their model training pipelines, and what technical safeguards exist to prevent unauthorized data exposure.

[18:34](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1114s) Requirements for AI Evaluation Tools and Relative Model Comparison

When selecting tooling to evaluate and govern generative AI, risk officers should focus on functional criteria rather than rigid product lists. Effective AI risk tools must be easy to use, capable of operating through standard API access, automated to provide timely feedback, and adaptable enough to evolve alongside rapid advancements in AI. Crucially, these tools must be non-invasive, running evaluations and guardrail tests externally without requiring intrusive access to the model's proprietary internal architecture.

To illustrate how models should be evaluated, an analogy of a personal bathroom scale clarifies the concept of precision versus relative comparison:

> "If I try to find my weight, I can stand on a scale, I got 130. That number cannot be used for medical [purposes] because I don't know how accurate is the scale. But there's one thing I know: if you stand with me use the same scale, if you weigh 150, you are heavier than me... and say if my weight is 130 today and tomorrow is 150, which means I gain the weight."

Even if an evaluation metric lacks absolute, laboratory-grade precision, consistent relative benchmarking across models provides highly actionable decision-making data when comparing alternative AI solutions.

[20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) Key Takeaways: Tone at the Top and Human-in-the-Loop Governance

Establishing sound AI risk management requires establishing the right "tone from the top" and maintaining a clear understanding of risk magnitude, likelihood, and risk-return trade-offs. In certain benign internal use cases, the practical likelihood of external threat actors expending massive compute resources to extract non-sensitive numbers is minimal, and risk controls should be calibrated proportionally.

Building meaningful guardrails requires recognizing that modeling is a combination of both "art and science." Regardless of whether an employee calculates figures using pen and paper, a spreadsheet in Excel, or an advanced generative AI tool, the individual user remains fully accountable for verifying the outputs. Maintaining a strict human-in-the-loop posture ensures that accountability remains intact while institutions leverage AI itself to manage and mitigate emerging AI risks.

## Notable quotes > Not all AIs are created equal—and neither is AI risk management. — Jenny Yu

> [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) strategic enabler — Jenny Yu

> [03:57](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=237s) Residual Risk = Inherent Risk - Controls — Jenny Yu

> [05:39](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=339s) not all AIs are created equal — Jenny Yu

> [07:10](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=430s) incorrect development — Jenny Yu

> [07:10](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=430s) inappropriate use — Jenny Yu

> [07:10](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=430s) tone at the top — Jenny Yu

> [15:41](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=941s) even blacker box — Jenny Yu

> [18:34](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1114s) If I try to find my weight, I can stand on a scale, I got 130. That number cannot be used for medical [purposes] because I don't know how accurate is the scale. But there's one thing I know: if you stand with me use the same scale, if you weigh 150, you are heavier than me... and say if my weight is 130 today and tomorrow is 150, which means I gain the weight. — Jenny Yu

> [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) tone from the top — Jenny Yu

> [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) art and science — Jenny Yu

## Mentioned Jenny Yu, Federal Home Loan Bank Topeka, Ai4, Wall Street, SR 11-7, Board of Directors, Microsoft Excel

## Video description Jenny Yu, Director, Model Risk Management and Methods, Federal Home Loan Bank Topeka

Abstracts

Not all AIs are created equal—and neither is AI risk management. This talk provides a practical blueprint for risk leaders and AI practitioners to build robust AI risk management programs by leveraging a trusted source: the proven framework of Wall Street.

Drawing on 20+ years of experience in model risk management within the highly regulated financial industry, Jenny will use the time-tested three-pillar framework—governance, owners' responsibility, and independent validation—to demonstrate how to adapt these principles to today's complex AI systems.

Attendees will leave with actionable strategies and a clear framework to help them embrace AI safely, confidently, and effectively, cutting through the complexity and hype to focus on the most impactful and measurable approaches to managing AI risk across all industries.

[Source](https://www.youtube.com/watch?v=qFcMYvbmfsw)

中文简报

FHLB Topeka借鉴危机后银行规则监管Agentic AI 银行唯有依据SR 11-7标准对每个自主组件逐一验证，并让人类操作员对算法输出承担严格责任，才能安全部署复杂的生成式及智能体系统。 Jenny Yu展示了金融行业成熟的SR 11-7模型风险管理标准如何为治理现代AI与生成式工作流提供实用蓝图。她将风险控制定位为战略赋能工具而非流程瓶颈，详细阐述了基本风险等式（剩余风险 = 固有风险 - 控制措施），并划分了传统模型、生成式模型以及第三方供应商模型的不同风险特征。她还说明了机构如何通过细粒度、组件级的性能监控和独立验证，来管理财务预测与模块化Agentic AI流程。最后，演讲就供应商LLM尽职调查、相对模型基准评估以及保持严格的人机协同（human-in-the-loop）问责机制提供了务实建议。 Jenny Yu 指出 AI 风险管理是一项战略推动力而非官僚阻碍，正如游乐场的坚固护栏能让儿童自由奔跑，健全的风险防线为企业高管大胆部署先进 AI 提供了心理安全感与信心。 (0:04) 企业 AI 治理基于“残余风险 = 固有风险 - 控制措施”的核心公式，且不可采用一刀切标准：传统预测模型依赖历史回测，生成式 AI 需要人机协同审核，而第三方供应商方案则需严格的供应链尽职调查与独立复现。 (3:57) 借鉴华尔街 SR 11-7 监管框架的三大支柱（顶层治理、模型所有者责任、独立验证），在应对净利息收入预测或复合多智能体（Agentic AI）架构时，必须为各个细分组件设定明确的量化误差容忍阈值并开展逐步监控。 (7:10) 面对大语言模型特有的恶意提示注入与数据泄露风险，机构评估供应商时须严审训练数据相关性与数据保留政策，并始终坚持人机协同（Human-in-the-loop）机制，由操作人员对模型输出承担最终核验责任。 (15:41)

Reading summary

Jenny Yu of FHLBank Topeka argues that AI risk management should function as a strategic enabler rather than an operational bottleneck, using Wall Street's established model risk frameworks to systematically de-risk traditional, agentic, and generative AI deployments. Enabling Innovation Through the Core Risk Equation (0:04) Yu frames effective model risk management using the metaphor of a playground with sturdy perimeter guardrails: children can run and play freely precisely because safety boundaries are secured. In the enterprise, structured controls provide executives with the psychological safety to adopt advanced AI technologies aggressively rather than succumbing to regulatory or operational paralysis. To demystify AI governance, Yu points to the foundational enterprise risk management equation: Residual Risk equals Inherent Risk minus Controls. Inherent risk reflects the raw algorithmic complexity, data pipeline fragility, and potential business disruption of a system. Through technical benchmarks, human verification checkpoints, and continuous performance triggers, organizations suppress failure modes to bring residual risk within formally approved corporate tolerance thresholds. The SR 11-7 Three-Pillar Governance Blueprint (7:10) Modern banking standards for managing opaque quantitative algorithms were codified after the 2007–2008 financial crisis under regulatory frameworks such as SR 11-7 (Supervisory Guidance on Model Risk Management). The standard addresses the two primary sources of model failure: incorrect development and inappropriate use across three core operational pillars. Pillar 1 establishes governance and tone from the top, requiring executive teams to inventory models, define explicit risk appetites, and report exposures quarterly to the board. Pillar 2 assigns operational accountability to model owners, who must document conceptual design, monitor performance drift, and establish remediation protocols. Pillar 3 mandates independent validation by mathematically qualified data scientists and quantitative engineers who audit code, run stress tests, and reverse-engineer third-party vendor models to verify stability under fluctuating macroeconomic conditions. Decomposing Multi-Agent Architectures for Component Validation (12:16) Yu demonstrates quantitative governance using a foundational banking task: net interest income forecasting across fluctuating interest rates and balance sheet volumes. In this workflow, executive leadership sets explicit, quantified error limits—such as tolerating a maximum 5% aggregate net interest income variance or a 1% rate projection deviation—balancing forecast uncertainty against practical business utility. When modernizing this process using modular agentic AI—such as deploying one agent to parse balance sheet data and another to project rate curves—institutions cannot simply evaluate the composite pipeline's aggregate output. Effective governance requires isolating and monitoring each individual agentic component. Granular validation allows risk teams to pinpoint specific sources of error, trace data lineage, and enforce performance tolerances across complex multi-step systems. Managing Generative Vectors and Vendor Due Diligence (15:41) Generative AI introduces what Yu characterizes as an even blacker box, expanding enterprise risk from deterministic computational errors to non-deterministic hallucinations, reasoning drift, and novel cyber threats such as malicious prompt injection and chatbot data exfiltration. Because off-the-shelf foundation models rarely offer visibility into internal weights, traditional mathematical code audits must be supplemented with targeted operational safeguards. When adopting vendor-supplied generative tools, risk teams must execute rigorous due diligence. Model validators must investigate whether training datasets are directly relevant to specific business domains, verify model accuracy against sector-specific benchmarks rather than generic metrics, and confirm whether vendor platforms retain submitted enterprise data or feed proprietary corporate inputs back into future training runs. Relative Tool Benchmarking and Preserving User Accountability (18:34) Selecting AI evaluation tooling requires focusing on functional agility: tools should operate non-invasively via standard APIs, automate feedback, and avoid requiring access to proprietary model internals. Addressing evaluation precision, Yu uses the analogy of a bathroom scale: even if a scale lacks absolute laboratory-grade calibration, consistently recording a change from 130 to 150 provides actionable relative data. Similarly, standardized relative benchmarking across AI models enables sound procurement and deployment decisions even when absolute metrics are unavailable. Ultimately, Yu stresses that modeling remains an intersection of art and science. Regardless of whether an employee executes calculations with pen and paper, an Excel spreadsheet, or an autonomous generative agent, the human user remains fully accountable for verifying final outputs. Human-in-the-loop workflows preserve operational ownership while allowing institutions to safely harness AI capabilities. Yu concludes that successfully operationalizing AI requires neither uncritical adoption nor risk-averse paralysis, but rather calibrating controls proportionally to an application's specific likelihood and impact while maintaining non-negotiable human accountability.

精炼阅读摘要

演讲者Jenny Yu提出，企业不应将人工智能风险管理视为阻碍创新的官僚枷锁，而应将其视为战略赋能工具；通过借鉴华尔街在2008年金融危机后确立的SR 11-7模型风险管理框架与“剩余风险＝固有风险－控制措施”这一核心平衡公式，企业能够系统化应对传统预测模型、生成式大语言模型及多智能体系统的复合风险。风险护栏的战略赋能与核心平衡公式 (0:04) Jenny Yu结合其在金融行业20余年的模型开发与风险验证经验指出，健全的AI风险管理本质上是推动业务落地的“战略赋能工具”。她用操场周围的坚固护栏作比：正是因为防护边界明确可靠，孩子们才能在场内自由奔跑；同理，坚实的风控护栏能赋予管理层大胆推进前沿AI技术应用的心理安全感。银行业与资本市场在2007–2008年全球金融危机后确立了严格的黑箱模型风险管理规范，为当前各行业治理复杂的AI黑箱算法提供了高度成熟的方法论基石。为解构看似难以驾驭的AI风险，Yu引入了企业风险管理的基本平衡方程：“剩余风险＝固有风险－控制措施”。固有风险由算法的不透明度、对复杂数据管线的依赖程度及业务关键性共同决定；控制措施则包括技术基准、输入输出核查与人工检查点等；扣除控制措施后留存的剩余风险，必须由高管层对照组织的风险承受阈值进行明确评估与正式接纳。Yu强调，生成式AI所引发的运营与声誉风险并非完全脱离传统逻辑，管理团队无需陷入瘫痪，而应通过梳理具体风险传导机制，部署相称的内部控制。 SR 11-7三支柱架构与AI分类治理 (7:10) 在监管标准SR 11-7（模型风险管理监管指引）体系下，模型风险主要源于“开发不当”与“使用不当”两个维度。为覆盖建模全生命周期，银行构建了三大核心支柱：第一支柱为高层治理与监督，由管理层确立基调并制定明确的风险承受度，建立资产清单并按季度向董事会汇报；第二支柱为模型所有者职责，要求业务与研发人员维护概念文档、监控性能漂移并设立自动化预警与修复机制；第三支柱为独立验证，由不隶属于开发或业务部门的专业量化团队对底层代码、数据流及敏感性实施独立审计与压力测试，必要时对外部供应商模型进行逆向工程复现。针对不同架构的AI系统，风险治理不能采用一刀切的清单模式。传统预测性机器学习模型属于纯黑箱计算，无法由人工手动审计中间过程，必须依赖历史回测、基准对比模型及压力测试等实证手段进行验证；生成式AI则容易产生幻觉、事实脱节与非确定性输出，必须引入领域专家在环（Human-in-the-loop）的审查与编辑机制；对于第三方外购系统，企业则需执行详尽的尽职调查问卷，明确特定审计权限，并对模型输出实施独立复现验证。业务容错量化与多智能体逐级监控 (12:16) Yu以银行业核心的“净利息收入预测”流程阐明量化治理机制：虽然该测算的基础逻辑仅为资产规模与利率的乘积（例如按6%利率出借100美元产生6美元收入），但在动态资产负债表与市场利率波动交织下，多变量预测极为复杂。银行通过将预测模型编入资产清单并评定风险等级，由独立团队开展历史回测并将误差拆解为利率波动与资产规模变动两部分。管理层据此设定刚性的量化容错区间，例如正式接纳总收入预测偏差在5%以内，或底层利率投影误差在1%以内，在预测不确定性与业务实用价值之间取得平衡。 Yu进一步将这一成熟实践延展至现代多智能体（Agentic AI）系统。当银行将复杂的测算任务解构为多个协同工作的自主智能体时（例如分别由不同Agent负责解析资产负债表、查询利率曲线及编排下游复杂计算），传统的整体黑箱评估便不再适用。Yu强调，治理体系必须对每一个模块化的Agent组件实施单点独立监控与验证，唯有逐级追踪数据血缘并细化性能控制，才能准确定位局部误差并阻断级联失效。生成式风险应对与相对基准评估 (15:41) 直接使用未经企业专有数据微调的通用大语言模型构成了“更深层的黑箱”。除传统网络渗透外，大模型引入了恶意提示词注入（Malicious Prompt Injection）与交互式对话窃密等全新攻击面。模型风控团队在进行供应商尽职调查时，必须严格审核其训练数据与特定业务垂直领域的匹配度，核实垂直场景表现，并查明厂商是否留存交互数据用于模型二次训练。在评估工具选择上，Yu主张采用易用、基于标准API、具备自动化反馈且非侵入式的工具，在无需触碰底层专有代码的前提下执行外部护栏测试。在评估精度上，Yu提出“家用体重秤”类比：家用体重秤即便无法提供医疗手术级的绝对精确度，但连续测量仍能准确判断两人的相对轻重，或自身体重是否增加；同理，即便AI评估指标缺乏实验室绝对精度，一致的相对基准评测依然能够为模型选型提供高度可行的决策依据。Yu最后重申，建模兼具科学与艺术属性，无论员工使用纸笔、Excel还是前沿生成式AI，核验输出结果的最终法律与业务责任始终落在人类使用者身上，维持严格的人在环路是有效管理新兴AI风险的关键底线。 Yu总结指出，企业建立AI风控机制应树立合理的自上而下基调，结合实际业务场景的危害程度与发生概率理性校准防御力度，避免为低概率风险过度消耗资源；通过将成熟的模型风险管理原则与模块化监控、相对基准评估和人工最终问责相结合，组织即可在保障合规安全的同时最大化释放AI的生产力潜力。

Detailed notes — zh

## Background 本内容由 Ai4 频道于 2025 年 8 月 28 日在 YouTube 平台发布。视频由托皮卡联邦住房贷款银行（FHLBank Topeka）模型风险管理与方法总监 Jenny Yu 主讲，旨在借助华尔街历经 20 余年沉淀的成熟模型风险管理框架，为风控领导者和 AI 践行者提供一套安全、合规且能有效赋能业务的 AI 风险治理实用蓝本。 ## TL;DR 演讲系统性地将华尔街经典的 SR 11-7 模型风险管理“三大支柱”（全面治理、模型所有者职责与独立验证）迁移至现代 AI 与生成式大模型的治理场景中。Jenny 指出 AI 风险管理的核心定位是赋能创新的“安全护栏”，而非业务发展的阻碍，并详细拆解了传统机器学习、生成式 AI 与第三方供应商模型的差异化分级治理策略。随后，她以银行净利息收入（NII）预测模型为例，演示了如何将量化回测与容差阈值管理无缝扩展到前沿的多智能体（Agentic AI）协作链条之中。最后，针对生成式 AI 独特的黑盒特性与提示词注入风险，她提出基于“相对比较法则”与非侵入式自动化评测工具的应对思路，强调业务责任始终不可外包，必须坚守人机协同与最终问责制。 ## Atmosphere & style 整场演讲呈现出一种严谨沉稳与幽默亲和兼备的专业金融家气场。Jenny 开场以站在听众与午餐之间的轻松自嘲破冰，随后迅速切换至充满洞见、条理极其清晰的专家叙事模式。她善于将复杂的量化数学与监管术语化繁为简，巧妙运用“儿童游乐场护栏”与“家用体重秤相对测重”等极具生活气息的比喻，化解了大众对黑盒 AI 未知风险的焦虑。整场分享节奏明快、务实干练，既有顶层治理制度的宏大构建，又有一线代码复现与业务细节的落地点拨，带给听众一种成竹在胸、以成熟经验从容驾驭技术变革的笃定与安全感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) AI 风险管理的核心定位不是业务阻碍，而是“战略赋能者”（Strategic Enabler），通过建立坚固的护栏机制（Guardrails）让业务领导者能够安全、自信地拥抱技术创新。 - [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) 华尔街在 2008 年金融危机后建立并沉淀了 20 年的黑盒模型风控体系（如 SR 11-7 指引），为当今破解各类 AI 风险提供了成熟的最佳实践蓝本。 - [04:02](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=242s) 风险管理遵循基础公式：固有风险（Inherent Risk）通过控制措施与护栏（Controls & Guardrails）的削减，最终转化为处于企业容忍度内的残余风险（Residual Risk）。 - [04:02](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=242s) 企业面对的 AI 需实施差异化分级治理：传统 ML/AI 依靠回测与基准测试，生成式 AI 依赖人机协同介入验证，外部供应商模型则需强化尽职调查与风险质询。 - [07:13](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=433s) 端到端建模风险根本上源自两点——“错误的开发”与“不当的使用”，必须在整个生命周期中明确工具的能力边界与局限性。 - [07:13](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=433s) SR 11-7 框架三大支柱包含：全面治理（高层定调、资产台账分级与董事会汇报）、模型所有者职责（全流程跟踪与文档记录）以及独立验证（端到端独立审查与代码复现）。 - [14:56](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=896s) 在 Agentic AI（多智能体协作）架构中，性能监控必须细化覆盖至每个独立的智能体节点，防止单一节点的计算偏差在后续链路中被级联放大。 - [15:41](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=941s) 生成式 AI 引入了更深的黑盒、幻觉与提示词泄露风险；企业在防范常规数据泄露之余，需重点防御交互式对话机制下的恶意提示词注入。 - [17:14](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1034s) 评估供应商 AI 模型时，核心在于审查其预训练与微调数据与具体业务场景的相关性，并严格确认是否存在客户数据回流训练的隐私风险。 - [18:34](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1114s) 生成式 AI 风险评估工具应具备易用性、自动化、持续进化与“非侵入性”（无需访问内部权重）等特性，以便快速完成外部黑盒测试。 - [18:34](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1114s) 在缺乏绝对精度标准时，应用“体重秤”相对比较法则：通过统一标准进行横向竞品对比或纵向时间序列监控，可有效识别相对优劣与性能衰退。 - [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) 业务责任绝不可外包给算法：无论使用何种工具，输出结果的最终验证与业务问责始终由人承担（Human-in-the-loop），并应学会借助 AI 工具来管理 AI 自身的风险。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=4s) 引言与背景：20年华尔街黑盒治理经验与战略赋能 Jenny Yu 在开场时幽默地提到自己正“站在大家与午餐之间”，非常感谢现场听众的留步与聆听。作为在金融行业深耕 20 余年的资深专家，她开宗明义地指出，金融行业早在 20 年前就已经开始系统性地管理复杂的“黑盒”（Blackbox）风险。一套切实可行且高效的 AI 风险管理框架，其核心定位绝非业务创新的阻碍，而是一个至关重要的“战略赋能者”（Strategic Enabler）。 ```mermaid flowchart TD accTitle: AI 风险管理赋能逻辑与华尔街成熟框架 subgraph CoreConcept["核心赋能理念"] A["业务创新与模型应用"] --> B["护栏机制 (Guardrails)"] B --> C["安全、自信的商业决策"] end subgraph WallStreetFramework["华尔街20年黑盒风控沉淀"] D["数据科学家算法底层"] --> E["业务流程与战略对齐"] E --> F["成熟模型风险管理框架 (SR 11-7)"] end CoreConcept -.-> WallStreetFramework ``` 为了直观展现风控的价值，Jenny 分享了一张儿童游乐场的图片：孩子们之所以能够在操场上无拘无束、欢快自由地奔跑，恰恰是因为四周建有坚固完备的安全护栏（Guardrails）。AI 风险管理对于现代企业也是完全相同的逻辑——建立起有效的护栏机制，才能真正赋能业务领导者安全、自信、大胆地拥抱 AI 技术。 Jenny 自身以数据科学家的身份起步，拥有 20 年开发模型、管理模型以及应对机器学习模型风险的丰富一线实战经验。她曾处理过横跨投资策略、风险控制和运营提效等各类复杂模型，这种跨维度的经历使她能够打破数据算法、业务流程与顶层商业战略之间的沟通壁垒，将严谨的量化风控手段转化为推动业务前行的战略工具。她特别回顾道，金融业早期的模型风险管理并未受到强制性重视，直到 2008 年全球金融危机爆发后，各大金融机构在强监管下建立了极为成熟的黑盒风控体系，而这些在危机中淬炼出的宝贵经验，正是今天破解各类 AI 风险的最佳实践蓝本。 --- ### [04:02](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=242s) 风险核心公式与三类 AI 的分级治理在剖析具体的管理方法之前，Jenny 首先给出了风险管理的基础核心公式。任何技术在落地时都伴随着“固有风险”（Inherent Risk），这取决于模型本身的算法复杂度及其对业务造成的潜在影响；通过在系统周围建立相应的控制措施与防护栏（Controls & Guardrails），企业能够最大程度削减潜在危害，最终将“残余风险”（Residual Risk）压缩至可承受的容忍度范围之内。 ```mermaid flowchart LR accTitle: 风险方程式与AI分类治理体系 subgraph RiskEquation["基础风险方程式"] direction LR IR["固有风险 (Inherent Risk)"] --> Ctrl["控制措施 (Controls / Guardrails)"] Ctrl --> RR["残余风险 (Residual Risk)"] end subgraph AICategories["AI 三大分类分类治理"] direction TB C1["传统 AI (Traditional AI) 机器学习/统计/随机模型 无法人工逐项验证 → 回测与基准测试"] C2["生成式 AI (Generative AI) 人机协同增强流程 具备可读性 → 人工介入验证"] C3["外部供应商 AI (Vendor AI) 第三方黑盒集成 尽职调查与关键风险质询"] end ``` 许多团队在面对 AI 风险时往往感到无所适从、觉得其庞大到无法掌控，但 Jenny 强调，如果层层拆解其实际触发的风险诱因、潜在后果、发生概率及影响量级，就会发现“生成式 AI 带来的许多风险本质上并不是全新的事物”。只要明确了具体的风险暴露面，就能针对性地设计控制机制。 "Not all AIs are created equal."（并非所有 AI 都生而平等）。基于算法特质、使用场景及引发风险的不同，Jenny 将企业面对的 AI 划分为三大类别，并实施差异化治理： - **传统 AI 与机器学习（Traditional AI / ML）**：包括统计学模型、随机过程模型及各类预测工具。这类模型属于高度不透明的黑盒，人类无法通过手工计算逐一核实预测结果，因此必须依赖系统性的回测（Backtesting）和基准测试（Benchmark Testing）等自动化量化工具进行校验。 - **生成式 AI（Generative AI）**：在绝大多数业务场景中，生成式 AI 是作为人类工作流的增强工具存在（Human-in-the-loop），由于其输出多为文本、代码或结构化内容，人类业务专家可以直接介入并对其结果的准确性进行审查与人工核验。 - **第三方与供应商模型（Vendor AI）**：当企业采购外部厂商包含 AI 能力的软件或模型时，内部往往无法获知底层的完整细节。此时风险管理的核心在于供应商风险尽职调查，必须在采购与集成环节提出针对 AI 局限性的关键问题。 --- ### [07:13](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=433s) 整体建模生命周期与 SR 11-7 三大支柱体系任何建模过程都是一个端到端的整体生命周期（Holistic Modeling Process）：它必须始于明确的业务目标（Business Purpose），最终落脚于具体的业务使用（Business Use）。因此，模型所伴随的根本风险只有两点——“错误的开发”（Incorrect Development）与“不当的使用”（Inappropriate Use）。在从数据收集、预处理、算法构建（权衡偏差与方差 Bias-Variance Tradeoff），到模型输出监控与局限性披露的全流程中，企业必须清楚了解工具能做什么、不能做什么。 ```mermaid flowchart TD accTitle: SR 11-7 监管指引下的三大支柱模型风险管理框架 subgraph Pillar1["支柱一：全面治理 (Governance)"] G1["高层定调 (Tone at the Top) & 风险偏好"] G2["模型资产台账 (Model Inventory) 与风险分级"] G3["持续监控与董事会季度汇报 (ERM)"] end subgraph Pillar2["支柱二：模型所有者职责 (Model Owners)"] O1["开发与部署阶段性能测试"] O2["重大变更应对与持续跟踪"] O3["全流程透明技术文档记录"] end subgraph Pillar3["支柱三：独立验证 (Independent Validation)"] V1["端到端验证 (输入/假设/算法/输出)"] V2["供应商模型内部代码复现 (Replication)"] V3["敏感性分析与管理层局限性披露"] end ``` 金融行业依托成熟的监管指引（如美联储与货币监理署联合颁布的 SR 11-7 模型风险管理指引），构建起了历经检验的“三大支柱”管理框架： 1. **全面治理（Governance）**： - 制定完善的政策、流程、角色分工与职责划分。 - 极其关键的一点是“基调必须由最高管理层设定”（Tone must be set from the top）。高管层需要权衡风险与业务收益，明确企业愿意承担多大的风险来换取效率提升，确立清晰的风险偏好（Risk Tolerance）。 - 建立全面的模型资产台账（Model Inventory），根据复杂度及对业务底线的影响进行高、中、低风险分级，匹配不同力度的控制要求与验证频率。 - 实施全生命周期监控，并将模型风险正式纳入企业全面风险管理（Enterprise Risk Management, ERM）体系，按季度直接向董事会汇报。 2. **模型所有者职责（Model Owners' Responsibility）**： - 无论模型是由内部团队自行开发，还是由业务部门直接引入外部供应商系统，该业务负责人即为“模型所有者”。 - 模型所有者必须主导开发与上线阶段的性能测试，建立追踪指标；一旦外部环境或模型表现发生重大偏离，必须及时采取纠偏行动。 - 必须保留详尽完备的技术与业务逻辑文档，确保逻辑的可解释性与透明度。 3. **独立验证（Independent Validation）**： - 验证工作必须完全独立于开发与使用部门，覆盖从数据输入、数学理论、底层假设到最终输出的全链条。 - 在联邦住房贷款银行（FHLBank Topeka）及同等规模的金融机构中，虽然广泛采用第三方供应商模型来评估信用风险与利率风险，但内部的数据科学家与计算机工程师团队会通过自主编程来“复现”（Replicate）供应商模型的核心逻辑。通过压力测试与假设变动分析，揭示模型在极端情景下的表现边界，向高管层清晰披露风险局限。 --- ### [12:16](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=736s) 实践落地：净利息收入预测与 Agentic AI 演进为了将抽象的框架具体化，Jenny 以银行业最核心的业务场景之一——净利息收入（Net Interest Income, NII）预测模型为例进行了拆解。从原理上看，该公式极为简单：如果对外发放 100 美元贷款并收取 6% 的利息，期末即可获得 6 美元的利息收入；但在真实的银行经营中，资产负债表的动态变动与宏观利率曲线的波动极其复杂，必须依靠专业模型进行预测。 ```mermaid flowchart TD accTitle: 净利息收入模型风控实务与智能体AI演进架构 subgraph TraditionalModeling["传统量化模型风控流程"] In["输入：资产负债表预测 + 利率走势预测"] --> Model["复杂定价与预测模型"] Model --> Out["净利息收入 (NII) 预测值"] Out --> Backtest["回测对比真实历史数据 (Ground Truth)"] Backtest --> Tolerance{"匹配高管层容差阈值 (如收入偏差 ≤5%, 利率偏差 ≤1%)"} end subgraph AgenticEvolution["Agentic AI / 智能体协同演进"] Agent1["Agent 1 (生成式AI) 检索外部宏观数据与利率曲线"] --> Integration["多智能体协同网络"] Agent2["Agent 2 (分析型AI) 解析资产负债表与动态现金流"] --> Integration Integration --> Agent3["Agent 3 (计算型AI) 自动化汇总执行净收入计算"] end TraditionalModeling -.->|相同风控逻辑复用| AgenticEvolution ``` 在该模型的全生命周期风控实务中，管理层建立了一套严密的防线： - **基准回测与偏差归因**：将模型纳入资产台账后，验证团队通过回测将其与历史真实数据（Ground Truth）进行比对。不仅对比最终的净收入总额，更深入拆解利率预测和资产负债表规模预测两大分项，精确定位误差来源。 - **业务对齐的容差阈值（Risk Tolerance）**：高级管理层根据模型带来的商业价值与潜在敞口，量身设定明确的容错边界。例如，管理层设定业务规则：净收入预测误差在 5% 以内属于正常业务波动范围，予以接受；而对于关键维度的利率预测，误差则必须严格控制在 1% 以内。 Jenny 进一步展望指出，如果将这个经典的金融预测公式迁移至当下的前沿 Agentic AI（智能体 AI）场景，其底层风控逻辑依然完全适用：假设未来构建一个多智能体协同网络——由一个生成式 AI 智能体负责全网检索并提取关联利率信息，另一个智能体负责解析内部资产负债表，再由专门的智能体完成利息计算与汇总输出——虽然技术形态从单一黑盒演变成了多智能体交互，但只要遵循“分解固有风险、施加护栏控制、独立验证输入输出、由业务设定容差阈值”的底层逻辑，就能够以不变应万变，从容应对日益复杂的 AI 风险。 ### [14:56](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=896s) Agentic AI 协作架构与各环节的细粒度监控在处理复杂的金融计算时，企业通常无法直接从公开互联网获取现成答案，而是需要依赖多个专用的智能体（Agentic AI）相互配合。例如，可以构建一个大型语言模型驱动的智能体协作系统：由一个智能体专门负责监测和管理利率参数，另一个智能体执行特定的衍生计算，第三个智能体负责汇总输出。这种多智能体框架虽然能够有效拆解大语言模型（LLM）的任务链路，但在风险管理上绝不能仅停留在对最终输出结果的宏观评估上。 ```mermaid flowchart TD accTitle: Agentic AI 模块化监控流程 subgraph MultiAgentSystem ["多智能体协作链路"] A["智能体 A：利率与市场数据管理"] --> B["智能体 B：核心金融业务计算"] B --> C["智能体 C：结果汇总与报告生成"] end subgraph GranularMonitoring ["独立性能监控层"] M1["监控 A 节点精度与漂移"] -.-> A M2["监控 B 节点逻辑与有效性"] -.-> B M3["监控 C 节点输出与对齐度"] -.-> C end ``` 为了确保整个架构的稳健运行，模型风险管理必须在整体平衡与局部细节之间取得统一。管理者必须深入拆解每一个独立的智能体节点，分别针对各个 Agentic AI 实施持续的性能监控与验证，而不能将其视为一个单一的整体黑盒敷衍了事。 - 复杂的业务计算无法单纯依靠单次大模型调用直接完成，需通过多智能体协作链条分步处理。 - 性能监控必须覆盖链路中的每一个独立智能体，防止单一节点的计算偏差在后续链路中被级联放大。 --- ### [15:41](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=941s) 生成式 AI 的全新风险：更深的黑盒、幻觉与提示词泄露生成式 AI（Generative AI）相比传统统计模型是一个“更黑的黑盒”（even blacker box）。如果企业直接调用基础大语言模型而不融入自有数据，企业对模型底层运行机制的实际控制力将极其有限。这种技术形态引入了三类最为核心的新型风险：模型幻觉（Hallucination）、输出不准确（Inaccuracy）以及数据泄露（Data Breach）。在应对数据泄露风险时，企业应当厘清传统数据安全与大模型交互安全的边界。传统的网络安全（Cybersecurity）和数据治理政策已经能够成熟防范常规的数据入侵；但大语言模型特有的交互式对话机制引入了全新的攻击面——恶意提示词（Malicious Prompts）或提示词注入攻击。企业如果能够清晰识别出提示词交互这一特定的泄露途径，就能针对性地部署防御工具予以阻断。而当系统进一步演进为接入内部私有数据的高级架构时，数据在检索增强和上下文流转中的泄露风险同样需要单独设立防护屏障。 - 传统安全流程负责拦截常规数据窃取，而大模型的交互特性要求防御针对性的恶意提示词输入。 - 在不掌握训练数据和模型权重的情况下，直接使用外部基础大模型的企业面临极低的自主可控度。 --- ### [17:14](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1034s) 评估第三方与供应商模型：数据相关性与垂直场景表现当企业采购或集成第三方的专业大语言模型及供应商工具时，风险评估的核心在于训练数据与自身业务的契合度。模型验证人员必须向供应商提出关键质询：该模型使用了哪些数据进行训练？这些数据与企业的具体业务场景有多大的相关性？例如，一个在从 PDF 财报中提取金融数据方面表现优异的模型，在起草商业租赁合同时可能表现得极其糟糕。因此，不仅要关注模型在公开基准上的整体性能，更要审查其在特定业务垂直领域的专项表现。 | 评估维度 | 核心审查问题 | 风险防范重点 | |:--- |:--- |:--- | | **训练数据相关性** | 供应商使用了哪些语料进行预训练与微调？数据与目标场景是否匹配？ | 防止跨领域任务迁移导致的准确率骤降（如财报提取 vs. 租约起草）。 | | **垂直领域表现** | 模型在特定业务细分赛道的独立评估指标如何，而非仅看通用基准评分？ | 避免被通用的宏观高分掩盖特定业务场景下的性能缺陷。 | | **数据回流与隐私** | 供应商是否收集客户输入数据？是否会将企业专有数据用于后续模型迭代训练？ | 杜绝企业机密或客户敏感数据因供应商模型训练而外泄。 | 针对供应商的 AI 数据使用政策，企业必须严格确认：供应商是否在后台运行隐式 AI 收集机制？客户交互数据是否会被纳入模型再训练语料库？如果供应商承诺不进行训练，其具体部署了哪些技术机制来防止企业数据发生泄露？ --- ### [18:34](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1114s) 风险评估工具的构建特性与“体重秤”相对比较法则用于评估生成式 AI 风险的工具应当根据企业的具体风险敞口量身定制，但在架构上必须满足几个通用原则：首先，工具必须易于使用且支持 API 快速接入；其次，测试过程应当全自动化并提供及时的即时反馈；再次，鉴于 AI 技术的迭代速度极快，风险检测工具自身必须具备持续进化的能力；最后也是最关键的一点，工具必须具备“非侵入性”（Non-invasive），即无需访问模型内部深层机制或私有权重，仅凭外部黑盒测试即可有效评估风险。在模型评估的精度标准上，应当正确理解绝对精度与“相对比较”（Relativity in Comparison）的价值。以日常称重为例：一个人站上家用体重秤显示 130 磅，这个数值因缺乏精密校准可能无法直接作为严格的医疗诊断依据；但如果另一个人在同一台秤上称出 150 磅，就可以百分之百确定后者比前者更重；同理，若同一个人今天称是 130 磅而明天称是 150 磅，便能明确判定其体重增加了。在模型选型与监控中，相对比较具有极高的参考价值——即便无法测出绝对完美的准确率指标，只要使用同一套标准化评估工具在横向对比多个候选模型、或纵向监测同一模型的性能漂移，相对评估就能为决策提供强有力的依据。 - **非侵入式测试**：风险工具应在不触及专有底层架构的前提下，通过外部输入输出接口运行合规与安全性测试。 - **相对比较法则**：在缺乏绝对基准时，统一标准下的横向竞品对比与纵向时间序列对比能够揭示关键的相对优劣与性能衰退。 --- ### [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) 总结与核心原则：高层基调、人机协同与最终问责制构建健全的 AI 风险管理体系，首先必须建立由高层定调（Tone from the top）的治理文化，深刻理解 AI 引入的风险类型、潜在破坏力（Magnitude）以及发生概率（Likelihood），并在风险与回报之间取得合理平衡。以数据泄露风险为例，部分银行内部低敏感度的常规数据，外部攻击者根本不会耗费高昂的算力成本去实施破解，此类场景下的实际风险敞口便处于极低水平。在技术落地层面，建立 AI 护栏的前提是真正理解模型的本质——模型开发从来都是“一门艺术与科学的结合”（a combination of art and science）。风险管理人员应当充分利用现有的合规与隐私治理框架，同时始终坚守“人机协同”（Human-in-the-loop）与最终问责原则。正如讲者强调指出的那样： > “作为个人，无论我是使用纸和笔、使用 Excel，还是使用生成式 AI，只要向业务输出数据，我就必须承担最终责任并进行验证——始终保持人员在回路中，并学会借助 AI 来管理 AI 自身的风险。” - 风险评估需综合考量攻击者的算力成本与数据价值，合理评估风险的真实发生概率。 - 业务责任不可外包给算法：无论采用何种技术工具，输出结果的最终验证与业务责任始终落在具体经办人身上。 - 积极利用智能化、自动化的 AI 风险管理工具反向监管 AI 系统本身，形成以技术制约技术的闭环。 ## Notable quotes > [04:02](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=242s) Not all AIs are created equal. — Jenny Yu > [07:13](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=433s) Tone must be set from the top — Jenny Yu > [15:41](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=941s) even blacker box — Jenny Yu > [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) a combination of art and science — Jenny Yu > [20:31](https://www.youtube.com/watch?v=qFcMYvbmfsw&t=1231s) 作为个人，无论我是使用纸和笔、使用 Excel，还是使用生成式 AI，只要向业务输出数据，我就必须承担最终责任并进行验证——始终保持人员在回路中，并学会借助 AI 来管理 AI 自身的风险。 — Jenny Yu ## Mentioned Jenny Yu, Federal Home Loan Bank Topeka, FHLBank Topeka, Ai4, SR 11-7, Board of Governors of the Federal Reserve System, Office of the Comptroller of the Currency, Enterprise Risk Management, Excel ## Video description Jenny Yu, Director, Model Risk Management and Methods, Federal Home Loan Bank Topeka Abstracts Not all AIs are created equal—and neither is AI risk management. This talk provides a practical blueprint for risk leaders and AI practitioners to build robust AI risk management programs by leveraging a trusted source: the proven framework of Wall Street. Drawing on 20+ years of experience in model risk management within the highly regulated financial industry, Jenny will use the time-tested three-pillar framework—governance, owners' responsibility, and independent validation—to demonstrate how to adapt these principles to today's complex AI systems. Attendees will leave with actionable strategies and a clear framework to help them embrace AI safely, confidently, and effectively, cutting through the complexity and hype to focus on the most impactful and measurable approaches to managing AI risk across all industries. [Source](https://www.youtube.com/watch?v=qFcMYvbmfsw)

Source quotes

[4:02] Jenny Yu: Not all AIs are created equal. [7:13] Jenny Yu: Tone must be set from the top [15:41] Jenny Yu: even blacker box [20:31] Jenny Yu: a combination of art and science [20:31] Jenny Yu: 作为个人，无论我是使用纸和笔、使用 Excel，还是使用生成式 AI，只要向业务输出数据，我就必须承担最终责任并进行验证——始终保持人员在回路中，并学会借助 AI 来管理 AI 自身的风险。

Source quotes

Jenny Yu: Not all AIs are created equal—and neither is AI risk management. [0:04] Jenny Yu: strategic enabler [3:57] Jenny Yu: Residual Risk = Inherent Risk - Controls [5:39] Jenny Yu: not all AIs are created equal [7:10] Jenny Yu: incorrect development [7:10] Jenny Yu: inappropriate use [7:10] Jenny Yu: tone at the top [15:41] Jenny Yu: even blacker box [18:34] Jenny Yu: If I try to find my weight, I can stand on a scale, I got 130. That number cannot be used for medical [purposes] because I don't know how accurate is the scale. But there's one thing I know: if you stand with me use the same scale, if you weigh 150, you are heavier than me... and say if my weight is 130 today and tomorrow is 150, which means I gain the weight. [20:31] Jenny Yu: tone from the top [20:31] Jenny Yu: art and science

Original transcript

[0:04] Good afternoon everyone. Thank you for [0:06] being here. I know I standing between [0:08] you and your lunch. I really appreciate [0:10] your time. Um [0:13] I work for financial industry for 20 uh [0:16] plus year. uh today I want to share with [0:18] you uh AI framework has been used in [0:22] financial industry for 20 years and um [0:26] is is a framework to manage blackbox [0:30] risk. [0:34] So I started to emphasize the a [0:38] practical and uh uh effective AI risk [0:43] management framework is a strategic [0:46] enabler. I'll provide some background [0:50] uh which is shapes how I approach AI [0:53] risk management. [0:56] I will walk you through those frameworks [0:59] and that that works in the financial [1:02] industry. And I will start um with a [1:05] call um risk equation. And I was [1:12] um [1:14] like we all know uh not AI are created [1:18] equal based on how their categorization [1:21] and their um [1:25] usage and different risk are involved [1:29] and um the AI risk management will be [1:33] different [1:36] and So [1:39] finally I will share some [1:42] AI tools which was was important [1:46] uh about those tools and uh I I'll give [1:49] uh some takeaway at the end. [1:53] Um [1:55] so uh I put a a [1:59] a picture of the playground. You can see [2:02] kids can running around freely and [2:04] happily because they the guard rail [2:07] around the playground. [2:09] And this is um um [2:13] we we is is the same with the AI risk [2:16] management. The effective AI risk [2:19] management can empower business leader [2:22] to you use AI safely and confidently. [2:30] Um and I said uh financial industry has [2:34] a framework to manage the blackbox risk [2:38] for about 20 years [2:42] and a lot of the the when framework [2:46] started it was not forcefully emphasized [2:50] until financial crisis. [2:54] A lot of financial logic institution [2:58] they build uh their frameworks [3:02] those framework can offer some practical [3:04] lessons we could leverage to manage AI [3:07] risk. [3:13] Um so I started as a data scientist and [3:17] I have a 20 years experience to develop [3:21] model manage model and uh in manage AI [3:24] machine learning model risk. [3:28] I have worked with many different model [3:31] using AI risk management using risk [3:34] management in investment strategy and [3:36] operational efficiency. [3:39] I can bridge the gap because I see so [3:41] many model can bridge the gap between a [3:43] data algorithm and a business process [3:45] and a strategy [3:48] which allow me to to be a strategic [3:50] enabler to empower leaders to use all [3:54] kind of tools. [3:57] Um here is the uh foundational equation [4:02] for risk. So most of the time we talk [4:05] about the inherit risk which how [4:08] complicated your model is how um what [4:12] impact it has and uh at the same time [4:16] you can have a certain controls around [4:18] it uh you can build some gui uh guard [4:21] area around it. I'll share with you [4:23] later about what kind of ger you can [4:26] build based on what kind of risk you it [4:29] evokes [4:30] and and how to minimize the residual [4:33] risk is the risk you uh left uh you need [4:39] um build a strategy around it and to [4:43] basically to tolerate [4:46] tolerate it. Um [4:49] like I said very very critical to [4:52] understand [4:54] what risk your AI get invoked uh the [4:58] potential consequence [5:00] and the magnitude and the likelihood it [5:02] will happen [5:05] and and what control you can build [5:07] around it. The reason I emphasize why [5:09] did you need a when people talk about [5:12] our AI risk they just feel that's [5:14] something so uh significant is um [5:19] unmanageable but if you [5:23] look deeply about what risk it actually [5:26] involve particular related to generative [5:28] AI it's nothing new if what kind of new [5:31] risk the generative AI bring to the [5:34] table how we can manage it from Yeah. [5:39] Um so uh like I said earlier not AI are [5:44] created equal and uh um so based on the [5:49] uh I put them into the uh different [5:53] bucket because different risk is [5:56] involved and the different risk [5:59] management [6:01] is uh will be uh in place to manage [6:06] those risk. I put it into three bucket. [6:09] One is the traditional AI which evolve [6:12] machine learning, statistical, [6:13] stochastical tools and the second [6:16] category would be generative AI. The the [6:19] third category is other AI like you use [6:22] vendors models you don't know what's [6:24] involved if AI is involved. Um so for [6:28] traditional AI it's like you do [6:30] forecasting you you cannot [6:35] manually verify it so you need another [6:39] to you you need either do back testing [6:42] you do benchmark testing you need to [6:44] find a tool to do it for the generative [6:47] AI depend how you use this a lot of [6:50] times it enhance the human process human [6:54] actually have been go through that human [6:56] can verify it. [6:58] And the last part is like you it's about [7:01] the vendor risk. You need to ask a right [7:04] question related to the AI. [7:10] Um so this what I created the holastic [7:13] modeling uh process [7:16] all those modeling all those black boxes [7:19] always started with business purpose [7:22] and it end with a business use. How [7:27] So the risk associated with any of those [7:30] model is in incorrectly development and [7:34] the inappropriate use. [7:39] Like I said that those traditional AI [7:42] tools they they are blackbox they cannot [7:45] be uh often cannot be verified by human [7:50] manually. So they need a special way to [7:53] control that risk. [7:58] You you can see in those modeling [8:00] process you have a business objective [8:02] then you have collect data you [8:03] pre-process data you have whole modeling [8:06] process you know the tradeoff to those [8:09] bias variance and and the performance [8:12] and then you have a model output you [8:14] monitor the model output and you [8:17] disclose the risk limitation those tools [8:21] are powerful you need to know what they [8:23] can do what they cannot view. [8:28] So this this is uh follow the SI11 which [8:32] is the uh financial regulatory uh [8:35] guidance on model risk management. We [8:38] build these three pillars uh framework. [8:41] The first one is the governance is talk [8:45] about our policy and the procedure rules [8:47] and responsibility. [8:49] One thing I want to mention that this is [8:53] specific mentioned the tone has to set [8:56] on the top. What kind of risk tolerance [8:58] you have as a bank as an organization [9:00] had to set on the top and is balance the [9:04] the risk and the return and can for the [9:08] efficient and effective you gain would [9:11] you really are you willing to take that [9:14] risk. [9:15] Um the the second part is like you [9:18] identify what's called the model and [9:21] then you put them you assess the risk [9:24] put into the different bucket depend on [9:26] the complexity uh depend on the impact [9:29] to your business and you have a [9:31] different level of control around it [9:34] like for financial industry these the [9:36] last of uh uh pillow would be [9:41] independent validation those risk [9:43] assessment will frame how frequently you [9:46] validate a model and what's the scope of [9:49] your validation. [9:52] And the last part is you know you you [9:54] need to have ongoing monitoring in your [9:56] model performance. You work today might [9:58] not work uh tomorrow. And uh also you [10:03] you know we report a model risk [10:07] management just as a uh [10:11] uh as a part of uh the overall [10:15] enterprise risk management. Uh [10:18] it is every quarter we reported the the [10:22] risk to the to the board. And the second [10:25] part is more for model owners. Whoever [10:28] owns the model, they have to make sure [10:31] this mo model owner can be model [10:34] developer. But if you use a model mo [10:36] vendors, you are the owner. They you you [10:39] have to work with developer the vendor [10:41] to make sure you know during that [10:44] developmental stage um implemented [10:46] stage. You have a a testing of model [10:50] performance. You track model [10:52] performance. If there's a significant [10:54] change, you need to do something about [10:56] it. [10:57] Um and also you have you need to have [11:00] the document so so the the the logic [11:03] stays there's a transparency with that [11:06] the last app was be independent [11:08] validation like I shared earlier there's [11:11] a whole model component for data input [11:14] to to output it the model input um [11:19] validation need to cover all those [11:21] elements by the regulation but in [11:24] practical a lot like the the bank I work [11:27] with uh or most of banker outside our [11:30] size is that we use vendors model to [11:33] manage the in risk um um uh credit risk [11:38] but our team [11:41] are the data scientist the and [11:45] um and the computer science engineer um [11:50] states so they were able to replicate a [11:53] lot of them replicate the the vent model [11:56] and uh and disclose those [12:00] risk limitation to the management. [12:02] Sometimes if you change your assumption [12:05] the the model result can change [12:06] dramatically. So have those [12:09] understanding communication to the [12:11] senior management about the model is [12:13] very important. [12:16] So I'll give an example that's a lot of [12:20] information in that that page. I give [12:22] you a specific example how we do it [12:26] there and and if we apply the same can [12:30] if we can apply the same principle to [12:33] the uh uh genative AI a genetic AI so we [12:38] we so it's a simple model we want to [12:41] know our uh interest rate income so [12:45] income you know if I learn uh learn some [12:48] $100 you ch if I charge 60% interest [12:51] rate [12:52] at the end I have income $6. [12:55] So for in in reality those interest rate [12:58] and the balance sheet forecasting are [13:00] very complicated. We need a model to do [13:02] that. Uh one of as I mentioned we we we [13:06] have to have a risk assessment put in [13:09] the model inventory and we have to do [13:13] the validation particularly in this case [13:15] we have a back testing we have a ground [13:17] to choose. So we compare the income [13:20] itself. We also compare the each element [13:24] interest rate and and the balance sheet [13:28] to see if there's a gap between the [13:31] model and the [13:34] and and the historical data actual data [13:38] where where where they come from. So the [13:41] most important part we have a [13:43] performance monitoring through the time [13:46] and the model performance will tie to [13:48] risk tolerance define set up by the city [13:52] management. They said if they said that [13:54] oh if you you you miss forecast by 5% [14:00] you're fine you know model always have [14:02] some like fluctuation or or error we [14:04] accept that if you rate the forecasting [14:07] is more than 1% [14:11] within 1% that probably okay so the the [14:14] the senior management from business side [14:17] to set up the tolerance [14:21] to see how much risk we we we tolerance [14:24] based on the the gain from using the [14:26] model. [14:28] If you think about [14:30] that simple equation, we make it a three [14:34] aentica AI. We need a AI to say this. I [14:40] I give you my balance sheet. Can you [14:41] calculate the income for us? And then we [14:45] have another generative AI to say can [14:48] you you know based on the information [14:51] you know you can you go find the [14:52] relatable rate the the interest rate is [14:56] a very complicated calculation it is we [14:59] we do not direct get a get from [15:01] internet. So, so, so you think about you [15:05] have another aentic AI to help us to [15:09] manage the rate and then you have a [15:11] third um aenic AI to this is a [15:16] oversimplified [15:18] uh uh way to to just explain how those [15:21] framework can work with large language [15:24] model. So the last part is is balance. [15:27] you always be able to find those [15:30] performance monitoring and then you do [15:32] not just do overall you go through each [15:35] a genetic AI. [15:39] Um [15:41] so uh now we talk about a uh uh [15:46] generative AI what I I I was doing is a [15:49] even blacker box. So you might know some [15:54] of the data depend how you if you fit [15:56] your own data a lot of times you you you [16:00] if you just direct use a logic language [16:01] model you don't have your own data [16:05] there's not a whole lot of control you [16:07] you can have what I try to say for [16:10] generative AI there's a lot of risk [16:13] related and we have two minutes left um [16:18] so what what I'm trying to say is the [16:21] new risk risk that a generate uh [16:24] generative AI introduc is those [16:26] hallucination and the incuracy and the [16:28] data bridge. [16:31] So, so [16:33] a what I would want to say for those two [16:36] type of a risk there's a lot of existing [16:41] um policy procedure in a bank or in in [16:45] organization [16:47] you have a like a data data risk [16:49] governance which uh and then you have a [16:51] cyber security they take care of a [16:53] traditional data bridge the only data [16:56] bridge from large language model because [16:59] you have that chartback you have a [17:00] interact action. They could have a [17:02] malicious prompt. If you realize that's [17:06] the data bridge risk, you can have a [17:09] tool to prevent it from happen. So, [17:14] um for for a if if you use more advanced [17:18] you you have your internal data in both [17:21] there could be another data bridge from [17:23] from there. Um and the if you use a [17:27] special logic language model you know [17:29] you use a vendor's model are very [17:33] critical question you want to ask what [17:35] are training data are they use how they [17:38] are relevant to your business for [17:41] example if you use you use they use that [17:44] to abstract financial data in the PDF [17:47] file they might not good at uh at a [17:50] draft the lease for you. So, so those [17:53] are how data closely relate to your [17:56] business is very critical and even even [17:59] you can ask how in those area are model [18:03] performance overall they have a one [18:05] model performance but for the specific [18:07] se sector do they have that that [18:10] performance for it. Um same thing for [18:14] the vendor model you would say do they [18:16] use AI do they collect your data do they [18:22] put your data into training if they [18:25] don't do what what what do they do [18:27] anything [18:29] prevent your data from bridge if you if [18:32] you concern about it [18:34] um so uh for generative AI tool uh it [18:40] I'm not giving up the tool because the [18:42] tool or are very individualized [18:45] depend on the risk you have. What what I [18:48] want to say they was for effective tool [18:51] it has to be easy to use. They just act [18:54] you if you give them they access your [18:56] API they will able to do the work. They [18:59] should be automatic give you timely [19:01] feedback and they should be evolve [19:03] evolving because AI [19:06] uh you know move so fast and and uh [19:13] it's very [19:15] uh important that those um tools are not [19:20] invasive. So they know do not need to [19:23] access to your models internal [19:27] mechanism. So they can run the test from [19:30] outside. Um so another point I want to [19:35] mention is the precision and the [19:37] relativity in comparison. So if I uh try [19:42] to find my weight I can stand on scale I [19:44] got a 130 that number is not cannot be [19:49] used for medical because I don't know [19:50] how accuracy is the scale but there's [19:54] one thing I know if you stand with me [19:58] use the same scale if you weigh 150 [20:02] you are you are [20:05] you are heavier than me that I I I and [20:09] say if my weight is 130 today and [20:12] tomorrow is 150 [20:15] which means I gain the weight. So those [20:18] relativity in comparison is is very [20:21] valuable accuracy to some degree when [20:24] when you select a model you can make [20:26] that comparison use the same tool. [20:31] So the takeaway is you know [20:36] the tongue has to set up from the top to [20:41] and and then I understand the what risk [20:45] AI evoked and their magnitude and the [20:48] likelihood and the balance of risk [20:51] return. There's one thing is for data [20:53] bridge if like for our bank sometime we [20:57] put the information there I I don't [20:58] think nobody bother to use the computing [21:01] power to hack our number so the risk is [21:04] a very very minimum um and the other [21:09] thing is to build a guard rail you have [21:11] to understand the AI [21:14] you know I was a model developer I read [21:17] so many models so I know what the it's a [21:20] combination art and science. Um the [21:24] other things there's a lot of existing [21:26] guidance privacy you know as an [21:29] individual it doesn't matter I use pen [21:31] or paper or I use Excel I use generative [21:34] AI if you give the number I hold a [21:36] responsibility I need to verify that [21:40] always keep me in the loop and uh [21:43] leverage AI to to manage the risk. Thank [21:46] you. [21:47] [Applause]