Boards Face Direct Legal Liability for Critical AI Lapses, Allen Says

Governing AI: Strategy Risk, and Your Readiness With Think Tank — Ai4

Ai4

Corporate case law mandates board-approved oversight systems for mission-critical operations, exposing executives who rely on static policies rather than programmatic risk thresholds to direct personal liability when models fail.

AI-assisted summary of Governing AI: Strategy Risk, and Your Readiness With Think Tank — original source. Check the original for full context.

Source published: 2025-08-27

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 27, 2025, this session features Brian Allen, Senior Advisor at the AI RegRisk Think Tank. The presentation outlines the essential components of an AI Readiness Program, explaining how executive leaders and corporate boards can manage emerging technical, ethical, and regulatory risks while maintaining competitive strategy.

## TL;DR Brian Allen argues that effective AI governance centers on establishing a defensible, repeatable oversight program rather than attempting to govern the technology itself or relying on passive checklist compliance. Grounding the mandate in corporate case law such as Caremark, Marchand, and the Boeing litigation, he demonstrates that executives and boards face personal and fiduciary accountability for mission-critical AI deployments. Rather than functioning as a bureaucratic 'Department of No,' modern risk management must produce dynamic risk intelligence that enables executives to calibrate risk tolerance and innovate at speed. To achieve this, the session outlines a five-pillar readiness framework encompassing tactical responsible AI controls, model risk alignment, and structured, programmatic risk escalation and materiality disclosure.

## Atmosphere & style The presentation carries the authoritative, pragmatic weight of a seasoned C-suite advisor delivering a direct boardroom briefing. Allen uses crisp, no-nonsense language to strip away compliance jargon, grounding abstract ethical debates in operational reality with memorable maxims like 'Do the right thing. Don't go to jail.' Watching the session feels like an urgent yet empowering masterclass that reframes governance from a friction-heavy bottleneck into an essential catalyst for enterprise execution.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=4s) AI governance demands a framework of communal responsibility because AI is context-specific, output-driven, and advancing faster than conventional regulatory rulemaking. - [02:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=148s) Governance and risk management should be treated as core strategic enablers rather than bureaucratic drag, embedding accountability directly into daily business operations. - [04:02](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=242s) Corporate legal precedents like Caremark, Marchand v. Barnhill, and Boeing establish a mandatory fiduciary duty for boards to maintain active, risk-informed oversight over mission-critical AI operations. - [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) Practical AI explainability does not require reverse-engineering neural network weights, but rather documenting training data, business intention, output validation, and operational human-in-the-loop criteria. - [08:21](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=501s) As enterprises shift from exploratory pilots to high-impact critical use cases, civil litigation will exploit operational failures in the absence of comprehensive federal legislation. - [10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) Risk management must reject the 'Department of No' mentality and focus on providing high-quality risk information that empowers leadership to innovate at speed within clear risk tolerances. - [13:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=784s) A mature AI governance program relies on a repeatable, board-approved risk-informed system using dynamic KRI thresholding, rather than static policy documents or passive quarterly committees. - [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) Legal defensibility against negligence claims depends on demonstrating that leadership was in a position to make an informed decision through end-to-end lifecycle accountability and unified Model Risk Management. - [19:22](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1162s) Risk-informed systems define the operational boundaries for resource allocation and ROI prioritization, allowing organizations to align internal audits directly against board-approved risk appetite. - [20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) Risk escalation must be programmatic and triggered whenever an emerging risk breaches established tolerance thresholds, rather than relying strictly on post-hoc incident response plans. - [21:57](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1317s) Executives and board members face direct personal and regulatory liability if their overarching AI risk and governance program cannot be independently defended under scrutiny.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=4s) Background and the Mission of Communal Responsibility

Brian Allen introduces his passion for governance and risk management, emphasizing how fast the AI landscape is moving. Serving as the Managing Director of the AI RegRisk Think Tank, he works closely with regulators, executives, and CEOs on how to address their oversight obligations regarding artificial intelligence. Allen previously served as a Chief Security Officer for a Fortune 100 company, worked at Ernst & Young (EY) advising corporate boards on cybersecurity oversight, and spent time at the Bank Policy Institute representing the largest U.S. banks before Congress and financial regulators on emerging technology issues like AI, quantum computing, and fraud.

The core mission of the AI RegRisk Think Tank centers on fostering "communal responsibility" through maturing governance frameworks. Regulating AI is exceptionally difficult because it is context-specific, output-driven, and developing much faster than the government can react, all while carrying heavy national security implications. Achieving communal responsibility requires striking a balance between the interests of shareholders, customers, the broader public, and global security. The Think Tank’s board includes high-level leaders such as a former U.S. Navy Admiral who headed the FCC Homeland Security Bureau, a former Chief of Staff for HHS, and a former director from the SEC, addressing macro issues like the loss of state and individual sovereignty down to operational threats like escalating AI-enabled financial fraud.

- **Speaker Background**: Managing Director of the AI RegRisk Think Tank; former Fortune 100 CSO; former cybersecurity board advisor at EY; former representative for the Bank Policy Institute. - **Board Composition**: Former Admiral and FCC Homeland Security Bureau Chief; former HHS Chief of Staff; former SEC Director. - **Regulatory Dilemma**: AI requires context- and output-specific governance, moving far too fast for conventional government rulemaking.

---

[02:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=148s) Reframing Governance and Risk Management as Core Strategy

Allen challenges organizations to consider the actual value of strategy: if strategy is valuable, companies and executives must spend a commensurate amount of time on it. Traditionally, risk management and governance are viewed as burdensome, bureaucratic committee exercises. In reality, governance and risk management *are* strategy. Assessments conducted across Fortune 50 companies reveal that executives routinely view cyber, supply chain, and operational resilience governance as bureaucratic drag rather than strategic enablement.

This disconnect is particularly acute with AI. Executives struggle with fundamental questions of ownership, such as what "responsible AI" or "ethical AI" looks like in practice. Allen boils the ethical mandate down to practical terms: "Do the right thing. Don't go to jail." To prevent risk management from remaining an isolated, quarterly rubber-stamp meeting, organizations must adopt a programmatic approach that embeds risk and governance directly into the daily fabric of business operations.

---

[04:02](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=242s) Parallels to Cybersecurity and the Evolution of Legal Case Law

AI governance mirrors the trajectory of cybersecurity over the past decade. For years, cyber risk operated on a checklist mentality—"dot your i's, cross your t's, don't be negligent." When major breaches persisted, regulators recognized that zero risk is impossible and shifted focus to accountability, demanding to know who manages risk and how programs are structured. In 2018, the SEC mandated that corporate boards exercise active oversight over cyber risk management programs, while the OCC instructed banks to prove their risk tolerance, establish a formal methodology, and demonstrate board approval for that methodology.

```mermaid flowchart TD accTitle: Evolution of Corporate Oversight Legal Precedents subgraph CaseLaw["Corporate Oversight Precedents"] CM["Caremark Standard 'Act in Good Faith'"] MB["Marchand v. Barnhill 'Risk-Informed System Required'"] BO["Boeing Litigation 'Applied to All Mission-Critical Operations'"] end

subgraph CyberAI["Regulatory Application"] CY["Cyber Oversight (2018 SEC / OCC) 'Formal Risk Tolerance & Governance Approval'"] AI["AI Risk Governance 'Mandatory Programmatic Readiness'"] end

CM --> MB MB --> BO BO --> CY CY --> AI ```

A parallel standard of personal and board accountability has been established through landmark corporate case law:

- **Caremark Case**: Established that boards and corporate officers have a legal duty to act in "good faith," setting a high legal threshold for oversight. - **Marchand v. Barnhill (Blue Bell Creameries)**: Following a fatal listeria outbreak, the court ruled that acting in good faith requires having an active, risk-informed oversight system in place. - **Boeing 737 MAX Litigation**: Extended the *Marchand* doctrine, determining that a risk-informed governance system must explicitly cover everything deemed "mission critical."

Given these precedents, boards and C-suite officers cannot ignore AI; if AI touches mission-critical operations, executive oversight is a mandatory legal obligation rather than an optional compliance task.

---

[06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) Operational Governance and Novel Technical Challenges

Traditional enterprise structures are fractured into distinct silos: IT/technology, cybersecurity, operational resilience, fraud, physical security, and business continuity. AI drastically complicates this landscape. Beyond technical fitness—ensuring a model is safe, secure, and fit for purpose—AI introduces operational governance questions. For example, if a customer service chatbot makes a mistake or gives an offensive answer, that is a failure of customer care and operational governance, not merely an IT malfunction.

These challenges will compound rapidly as organizations transition to distributed AI agents operating across enterprise workflows. In addition, organizations must tackle novel AI challenges, including explainability, bias, fairness, and human-in-the-loop controls:

- **Explainability**: While deep neural networks operate as technical "black boxes," governance can achieve practical explainability. Allen compares this to human cognition: "My head is a black box. You ask me questions. I don't tell you how the neurons in my brain work. I tell you my data, my intention, how I went about it, my output." True explainability comes from documenting training data, intent, methodology, output validation, continuous risk testing, and transparency. - **Human in the Loop**: Regulators are moving beyond superficial claims of having a human reviewer. They now scrutinize the underlying operational criteria: Is there a clear policy defining when a human is required? Does the decision involve high stakes, strict timeliness, Personally Identifiable Information (PII), or Protected Health Information (PHI)?

---

[08:21](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=501s) Critical Use Cases, Emerging Lawsuits, and Programmatic Readiness

Most organizations are currently running non-critical AI pilots and experimentation. However, as enterprises deploy AI into high-impact, critical use cases in the coming year, operational failures, material harms, and resilience incidents will inevitably occur. In the absence of comprehensive federal regulations, the legal void will be filled by aggressive civil litigation, with plaintiff attorneys targeting failures in explainability, bias, and operational control.

To address these vulnerabilities before crises emerge, the AI RegRisk Think Tank developed an AI Readiness Program. This framework synthesizes case law, regulatory guidance, technical standards, and emerging state laws into five core elements. Allen stresses that having isolated tools, processes, or superficial key indicators (KIs) is insufficient; mature AI risk management requires a programmatic framework backed by enforceable risk thresholds.

[10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) Programmatic Governance and the Fallacy of the "Department of No"

True risk management begins when an organization establishes clear risk tolerance thresholds with the right leadership in the room. While policies and standard processes serve as useful starting points, most organizations fundamentally lack a defined programmatic approach. When evaluating governance, the critical question is not whether an organization has assembled a committee, but rather: *"What are you governing?"* The answer must be the governance program itself. That program needs to be deliberately scoped and embedded deeply enough across the organization so everyone understands corporate boundaries and risk appetite.

Crucially, establishing tolerance does not mean shutting down business initiatives. The common trope among cybersecurity professionals claiming to be the "Department of No" fundamentally misunderstands the role of risk management. It was never security's mandate or authority to make the ultimate "no" decision. The true output of a risk program is not protection—protection is merely the operational effect of a mitigation tactic—but high-quality *risk information* that empowers business executives to make well-balanced strategic choices. In the modern AI landscape, speed is paramount; when initiatives fall outside conventional tolerance bands, it often signals an opportunity to innovate and outpace the competition.

```mermaid flowchart TD accTitle: The Governance and Risk Flow accDescription: Flowchart illustrating how a programmatic governance structure generates risk information to drive strategy, defensibility, and business execution.

subgraph Governance ["Foundational Governance Program"] A["Program Definition & Scoping"] --> B["Tolerance & KRI Thresholding"] end

subgraph System ["Risk-Informed System"] B --> C["Risk Information Output"] C --> D["Executive Decision-Making"] end

subgraph Execution ["Strategy & Operations"] D --> E["Prioritization & Budgeting (ROI)"] D --> F["Tactical Responsible AI (Explainability, Bias, Testing)"] D --> G["Legal Defensibility & Audit Alignment"] end ```

[13:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=784s) Five Pillars, 14 Domains, and the Risk-Informed System

The AI Readiness Program structure is built on five core pillars backed by 14 specific domains designed as flexible, adaptable considerations. An effective governance architecture avoids mistaking policy for a program—policy is merely an enforcement mechanism and operational tool. The primary objective of governance is to verify whether the overarching program is clearly defined, repeatable, properly scoped, and granularly integrated into daily business operations, rather than settling for passive quarterly committees where members simply listen to risk summaries and walk away.

A central pillar is establishing a "risk-informed system," a concept rooted in court precedents and regulatory expectations. For public enterprises, maintaining a structured mechanism to determine tolerance is a non-negotiable obligation rather than an optional safeguard. Tolerance can be systematically calibrated through Key Risk Indicator (KRI) thresholding or maturity modeling. Regulatory bodies view board-approved risk systems as fundamental; even within banking—an industry renowned for mature risk controls—many institutions still struggle to demonstrate formal, board-approved systems that gauge operational tolerance dynamically.

- **Policy vs. Program**: Policies act as tactical enforcement tools, whereas a governance program provides the overarching repeatable framework. - **Systematic Calibration**: Tolerance must be measured through dynamic mechanisms such as KRI thresholds and maturity models rather than static yearly reviews. - **Regulatory Imperative**: Having a board-approved, risk-informed architecture is an explicit expectation reinforced by regulatory bodies like the SEC.

[16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) Responsible AI Tactics and Establishing Legal Defensibility

Moving from high-level oversight into tactical execution requires a dedicated responsible AI practice capable of confronting novel challenges, including explainability, algorithmic bias, human-in-the-loop controls, third-party vendor AI risk, and autonomous agent management. When the Bank Policy Institute hosted an AI symposium with leading bank CIOs, nearly every participant cited "moving too slow" under competitive and market pressure as one of their top three corporate risks. Consequently, governance models cannot default to rigid obstructionism; teams must construct processes that can actively navigate technical complexity and legal exposure.

When legal challenges arise, corporate negligence is rarely judged on whether an outcome was flawless, but rather on whether leadership was *"in a position to make an informed decision."* Building legal defensibility requires end-to-end accountability across every stage of the AI lifecycle: - Clarifying data provenance, business intention, system architecture, output validation, continuous testing, and ongoing monitoring. - Bridging the organizational divide within Model Risk Management (MRM), ensuring model validation teams, data engineers, and frontline business owners (such as customer care teams running chatbots) operate under unified testing and monitoring standards.

[19:22](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1162s) Risk-Based Strategy, Execution, and Working with Auditors

Once a robust, risk-informed system is established, the focus transitions to risk-based strategy and execution. The risk-informed system itself does not invent corporate strategy; instead, it establishes the operational boundaries within which leaders allocate budgets, prioritize use cases, evaluate ROI, and negotiate trade-offs with diverse stakeholders.

This alignment also transforms the relationship between risk teams and internal/external auditors. Rather than limiting audits strictly to standard compliance checklists, organizations should proactively invite auditors to evaluate operations directly against the specific risk tolerances formally approved by the board of directors. Aligning internal audits with executive risk thresholds ensures that regulatory expectations, strategic priorities, and tactical execution remain synchronized as market tolerances continue to shift.

[20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) Programmatic Risk Escalation, Tolerance, and Material Disclosure

Organizations need an open, realistic mindset regarding regulatory risk. Regulation itself is a risk category: absolute compliance is not strictly mandatory, though non-compliance "may hurt," meaning companies can actively choose to scope boundaries and manage exposure against their defined risk tolerance. Transitioning to the final governance pillar—risk escalation and disclosure—organizations frequently offer a common reflex when reviewed: "Oh, we have an incident response plan." While actual security or operational incidents frequently trigger escalation, an incident response plan alone is insufficient. A measured risk falling outside established organizational tolerance thresholds should also trigger an escalation based on severity and destination, and that mechanism must be strictly programmatic rather than relying on informal, relationship-based channels.

```mermaid flowchart TD accTitle: Programmatic Risk Escalation and Disclosure Flow subgraph Triggers ["Triggers"] T1["Operational Incident"] T2["Risk Breaching Tolerance"] end

subgraph Evaluation ["Programmatic Criteria"] E1["Programmatic Assessment"] E2["Materiality Criteria & Timeline"] end

subgraph Escalation ["Defined Disclosure Channels"] D1["Internal Escalation: Execs & Board"] D2["Regulatory Reporting Thresholds"] D3["Public Disclosure & Trust Preservation"] end

T1 --> E1 T2 --> E1 E1 --> E2 E2 --> D1 E2 --> D2 E2 --> D3 ```

When evaluating disclosure, particularly for public corporations where materiality dictates reporting, the governing question extends beyond whether an occurrence is material in hindsight. Scrutiny focuses heavily on the timeline and methodology: precisely when an issue was classified as material, what specific criteria guided that determination, and what formal steps were executed to identify the incident as such. Whether reporting externally to meet regulatory disclosure thresholds or communicating proactively to the public to preserve brand trust, managing materiality demands a structured, reproducible process rather than ad-hoc decision-making.

- **Regulatory Risk Posture**: Treating regulation as a quantifiable, manageable business risk rather than an immovable binary of compliance. - **Tolerance-Driven Escalation**: Triggering formal notifications whenever an emerging risk exceeds predefined risk appetite limits, independent of whether an acute technical incident has materialized. - **Process of Materiality**: Documenting the explicit criteria, procedural stages, and timestamps used to arrive at a materiality determination.

---

[21:57](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1317s) Defending the Program: Board Liability and the Reimagined Governance Model

Effective AI governance ultimately hinges on whether risk operations are genuinely programmatic. While nearly every enterprise maintains disjointed risk management processes and written policies, the critical test is whether the governance program can stand on its own and be defended under scrutiny. "I'm not talking process and policies," Allen emphasizes; if the broader program itself cannot be defended, corporate leadership—including executive officers, owners, and board members—carries direct individual liability. This legal exposure makes it imperative for practitioners to bring these frameworks directly to leadership, reshaping and reimagining how AI governance is embedded across the business before time runs out.

- **Program Defense vs. Policy Checklists**: Moving past isolated checklists and static policy folders to build an integrated, legally defensible oversight program. - **Fiduciary and Personal Liability**: Highlighting that failure to maintain a defensible governance framework exposes executives and board members of significant and public enterprises to direct regulatory and civil liability. - **Reimagining Governance**: Positioning programmatic readiness as an active, structural redesign of accountability rather than a passive compliance function.

## Notable quotes > [02:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=148s) Do the right thing. Don't go to jail. — Brian Allen

> [04:02](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=242s) dot your i's, cross your t's, don't be negligent. — Brian Allen

> [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) My head is a black box. You ask me questions. I don't tell you how the neurons in my brain work. I tell you my data, my intention, how I went about it, my output. — Brian Allen

> [10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) What are you governing? — Brian Allen

> [10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) Department of No — Brian Allen

> [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) moving too slow — Brian Allen

> [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) in a position to make an informed decision. — Brian Allen

> [20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) Oh, we have an incident response plan. — Brian Allen

> [21:57](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1317s) I'm not talking process and policies, — Brian Allen

## Mentioned Ai4, Brian Allen, AI RegRisk Think Tank, Ernst & Young (EY), Bank Policy Institute, U.S. Congress, U.S. Navy, Federal Communications Commission (FCC), U.S. Department of Health and Human Services (HHS), Securities and Exchange Commission (SEC), Office of the Comptroller of the Currency (OCC), In re Caremark International Inc. Derivative Litigation, Marchand v. Barnhill, Blue Bell Creameries, Boeing, Model Risk Management (MRM)

## Video description Brian Allen, Senior Advisor, AI RegRisk Think Tank

Abstracts

Artificial Intelligence presents immense strategic opportunities—but also introduces novel risks and a rapidly evolving regulatory landscape. Established oversight obligations already apply and demand C-suite attention.

This session tackles a crucial question: "What exactly are we governing?" Hint: Effective governance doesn’t focus on the AI technology itself, but on the necessity of a defined program designed to manage strategy, risk, and accountability in this transformational environment.

We will outline the foundational elements of an AI Readiness Program, offering a practical framework for confidently integrating AI capabilities. The session will provide an approach to navigate the complex and fast-moving legal and regulatory landscape and will explore how robust governance can proactively address emerging technical and ethical challenges—such as model explainability and bias—while managing significant and growing challenges.

Drawing parallels to recent enforcement actions that highlight individual accountability, we will discuss how strong governance establishes the essential guardrails your organization needs to innovate and compete with speed and confidence.

Join this vital session for a C-suite perspective on building the strategic governance framework necessary not only to mitigate AI’s risks but to fully harness its transformative potential for competitive advantage.

[Source](https://www.youtube.com/watch?v=a4OLsgzMi9s)

中文简报

Allen称董事会因关键AI失误面临直接法律责任公司判例法要求针对关键任务运营建立经董事会批准的监督系统，这使得在模型失效时，依赖静态政策而非程序化风险阈值的高管需承担直接个人责任。 Brian Allen指出，有效的AI治理核心在于建立可辩护且可复现的监督机制，而非试图直接管控技术本身或依赖被动的清单式合规。立足于Caremark、Marchand以及Boeing诉讼等公司判例法要求，他阐明了高管与董事会针对关键任务AI部署面临个人责任和信义义务。现代风险管理不应充当官僚式的“只会说不的部门”，而必须生成动态风险情报，使高管能够校准风险容忍度并实现快速创新。为此，会议概述了一项五支柱就绪框架，涵盖战术层面的负责任AI管控、模型风险对齐，以及结构化、程序化的风险升级与重大性披露。 AI 治理必须建立共同责任框架，因为 AI 具有高情境关联与结果导向特性，其演进速度远超传统监管立法的制定节奏。 (0:04) 企业治理与风险管理应被视为战略推动力而非官僚阻力；根据 Caremark、Marchand 以及波音案等司法判例，董事会对关键任务 AI 运营负有法定信义监督义务。 (4:02) 实用的 AI 可解释性并不要求逆向解析神经网络权重，而是必须清晰记录训练数据、业务意图、输出验证及人工介入（human-in-the-loop）标准。 (6:13) 风险团队不能陷入盲目否决的“拒止部门”思维，而应通过动态关键风险指标（KRI）阈值等程序化系统产出高质量风险信息，赋能高管在明确容忍度内加速创新并建立法律抗辩能力。 (10:04)

Reading summary

Brian Allen, Managing Director of the AI RegRisk Think Tank, argues that artificial intelligence governance must transition from passive compliance checklists into a legally defensible, programmatic risk architecture that informs strategic decision-making and satisfies board oversight obligations. Legal Precedents and Board Oversight Mandates (4:02) Allen emphasizes that AI oversight mirrors the regulatory and legal trajectory of cybersecurity over the past decade. Legal benchmarks established through the Caremark standard (acting in good faith), Marchand v. Barnhill (mandating an active, risk-informed oversight system following a fatal listeria outbreak), and the Boeing 737 MAX litigation (extending oversight requirements to all mission-critical operations) demonstrate that boards cannot treat emerging technologies as optional compliance matters. With the SEC mandating active board cyber oversight in 2018 and the OCC requiring formal, board-approved risk tolerance methodologies in banking, Allen argues that executives and directors now face direct personal and fiduciary liability if mission-critical AI deployments lack defensible governance programs. Reframing Governance as Strategic Decision-Making (2:28) Rather than functioning as a bureaucratic bottleneck or the operational "Department of No," governance and risk management constitute core business strategy. Allen asserts that security and risk teams lack the authority to unilaterally block initiatives; their core purpose is generating actionable risk information that empowers executives to make balanced, strategic choices. In an environment where bank CIOs surveyed at a Bank Policy Institute symposium ranked "moving too slow" as a top corporate risk, operating outside conventional risk thresholds often indicates opportunities to innovate. Establishing clear risk tolerance boundaries enables corporate leadership to prioritize high-value use cases, allocate capital, and execute trade-offs between speed and risk. Operational Defensibility and Practical Explainability (6:13) Deploying AI across enterprise workflows introduces operational risks that transcend traditional silos like IT and cybersecurity, as seen when customer-facing chatbots malfunction. To prepare for anticipated civil litigation and regulatory audits, enterprises must implement operational controls around algorithmic bias, third-party vendor models, autonomous agents, and human oversight. Allen observes that regulators now scrutinize the specific operational policies governing human intervention—evaluating whether high stakes, strict timelines, or protected personal data require manual review—rather than accepting superficial human-in-the-loop claims. Building legal defensibility does not require solving technical "black box" neural networks at a cellular level, just as human decision-making relies on articulating context rather than neuronal mechanics. Instead, defensibility requires documenting data provenance, business intention, system architecture, continuous testing, and ongoing monitoring. Unifying Model Risk Management (MRM) validation teams with frontline business units ensures that organizations can demonstrate leadership was in a position to make informed decisions before failures occur. Programmatic Architecture and Material Escalation (13:04) The AI RegRisk Think Tank structures governance around an AI Readiness Program comprising five core pillars and 14 operational domains. Allen stresses that isolated policies and committee meetings do not equate to a program; policies serve merely as enforcement tools, whereas a true program is repeatable, clearly scoped, and integrated into daily operations. Organizations must dynamically calibrate risk tolerances using Key Risk Indicator (KRI) thresholds and maturity models, which can be evaluated directly by internal and external auditors against board-approved benchmarks. Furthermore, Allen warns that relying solely on incident response plans is insufficient for risk escalation. Emerging risks that breach established organizational tolerance bands must trigger programmatic escalation pathways based on severity and destination, even in the absence of a technical outage. For public companies, regulatory defensibility requires documenting the exact criteria, operational steps, and timelines used to determine materiality, ensuring disclosure processes are systematic rather than ad hoc. Allen concludes that as organizations advance from non-critical pilots into high-impact operational AI use cases, governance gaps will be exposed by civil litigation and regulatory scrutiny. However, embedding repeatable, board-approved risk systems into daily workflows remains an unresolved operational hurdle, even across heavily regulated sectors like banking.

精炼阅读摘要

AI RegRisk Think Tank 董事总经理 Brian Allen 指出，人工智能治理绝非局限于合规清单或技术孤岛，而是关乎董事会与高管个人法律责任的核心战略体系；企业必须建立经董事会批准的“风险知情系统”，将风险治理从被动的审批阻碍转变为赋能战略决策的程序化能力。法律先例演进与董事会的信托责任 (4:02) 人工智能治理正在复刻网络安全监管的历史轨迹。2018年美国证监会（SEC）与货币监理署（OCC）先后要求金融机构和上市公司董事会对网络风险管理实施积极监督，并证明其风险容忍度已建立正式方法论且获董事会批准。在司法层面，特拉华州确立的判例法进一步强化了这种个人受托责任：Caremark 案确立了管理层善意履职的法律基准；Marchand v. Barnhill 案（蓝钟冰淇淋案）明确判定善意履职必须依托一套主动运行且知悉风险的监督系统；而波音（Boeing 737 MAX）诉讼则将该风险知情系统强制覆盖至所有“关键任务运营”（mission-critical operations）。 Allen 强调，当前多数企业仅在非关键业务中进行 AI 试点，但随着企业未来一年内将其推向高影响力核心业务，由于缺乏统一的联邦法规，民事诉讼必然会迅速填补监管空白。诉讼原告将针对算法黑盒、可解释性缺失与运营偏差等问题发起集体攻击。在现有判例法体系下，一旦 AI 触碰关键业务，构建可抗辩的程序化治理体系便不再是可有可无的合规选项，而是董事会与高管必须履行的法定免责底线。重构战略定位与破除阻碍误区 (10:04) 传统观念通常将风险管理和治理视为繁琐的官僚委员会流程，甚至部分网络安全团队自诩为“拒绝部门”（Department of No）。Allen 明确反驳了这一认知，指出风险团队从未被赋予替业务做终极否决的权力；风险管理的真实产出不是机械性的技术阻断，而是高质量的“风险信息”，旨在帮助最高管理层做出权衡充分的战略抉择。在银行政策研究所（BPI）近期举办的闭门讨论中，几乎所有受访的大型银行首席信息官（CIO）均将“受制于市场压力推进过慢”列为首要风险之一，证明消极阻碍业务只会放大企业的竞争劣势。有效的治理并非单向设立关卡，而是基于明确的风险容忍度阈值（KRI）和成熟度模型。政策仅仅是操作层面的执行工具，企业真正需要治理的是“治理程序本身”。当新兴业务活动突破常规容忍度区间时，该体系能够向决策层提供量化依据，使其明确知晓边界并在战略预算、投资回报率（ROI）考量与合规风险之间做出主动权衡，将治理重塑为业务创新的战略使能器。穿透黑盒的技术攻防与运营化控制 (6:13) 企业传统的 IT、网络安全、运营韧性、欺诈防范与业务连续性等职能长期处于割裂状态，而人工智能的引入极大加剧了跨部门协作的复杂性。Allen 指出，客服聊天机器人一旦输出失实或攻击性言论，本质上是客户关怀与运营治理的全面失效，而非单纯的 IT 故障；随着自主 AI 智能体（AI Agents）在工作流中广泛扩散，此类复合型风险将迅速放大。针对深度神经网络固有的技术“黑盒”难题，他主张在治理层面推行实用主义解释框架：法律与监管并不要求阐明微观神经元参数的计算过程，而是审查企业是否完备记录了训练数据来源、业务意图、设计方法论、输出验证流程及持续风险测试证据。在“人在回路”（Human in the Loop）的合规声明上，监管机构正逐步从形式化审查转向对实质运营触发标准的严格问询，企业必须明确界定在何种特定条件下必须引入人工干预，尤其是当决策涉及高利害关系、强时效性约束、个人身份信息（PII）或受保护健康信息（PHI）时。企业必须彻底打通模型风险管理（MRM）团队、数据工程师与前线业务部门之间的壁垒，确保技术监控与一线业务运营在统一的防御标准下协同运转。基于容忍度的程序化上报与重大性判定 (20:28) 许多企业在面对监管审视时习惯以“已制定安全事件应急响应计划”作为抗辩，但 Allen 认为这一准备存在致命漏洞。在成熟的治理架构中，除了发生实际业务中断或安全攻击外，只要被监测风险指标突破了组织预设的容忍度阈值，系统就必须触发严格程序化的分级上报机制，及时将风险暴露向相应管理层与董事会通报，彻底摒弃依赖非正式人际关系的随意反馈模式。在信息披露方面，特别是对受重大性（materiality）约束的上市公司而言，外部监管与司法机构的调查重心不仅局限于事后判定某一事件是否“重大”，更聚焦于整个判定过程的时间戳与逻辑链条。企业必须能够证明其在何种确切时间节点判定问题构成重大事项、依据了何种量化与定性标准，以及执行了哪些法定程序。将监管合规视为一项可量化、可管控的经营边界，并通过严谨的时间节点与标准审查程序支撑重大性判定，是抵御监管处罚和维系公众信任的核心抓手。 Allen 总结指出，散落于各部门的独立流程与静态合规手册无法构成具备法律抗辩力的治理体系。面对日益严峻的技术风险与潜在诉讼暴露，决策层必须推动治理框架的结构化重塑，将董事会批准的风险知情系统深度嵌入日常运营；若缺乏经得起审查的整体治理程序，一旦发生重大运营或合规灾难，企业高管与董事会成员将难以在司法程序中免除个人的信托责任追偿。

Detailed notes — zh

## Background 该视频由 Ai4 于 2025 年 8 月 27 日发布在 YouTube 平台，主讲人为 AI RegRisk 智库高级顾问 Brian Allen。视频内容简介指出，本次演讲从 C-suite 高管视角出发，探讨企业在 AI 战略转型中的监督义务与风险就绪度，阐明有效治理的核心在于建立系统化的管理项目以兼顾创新速度与合规问责。 ## TL;DR Brian Allen 提出 AI 治理的核心并非针对技术本身，而是将其作为一项关键商业战略，构建具备可辩护性的“项目化管理机制”（Program）。结合 Caremark、波音等里程碑式司法判例，企业高管负有建立经董事会批准的“风险知情系统”并明确风险容忍度阈值的法定义务。风控部门的本质职能是交付决策所需的“风险信息”而非盲目否决，企业必须打通数据、模型与业务运营的全生命周期链条，才能在应对算法偏见、黑箱解释等挑战的同时，避免因“行动迟缓”而错失战略机遇与承担个人连带责任。 ## Atmosphere & style 整场演讲洋溢着务实、沉稳且极具洞察力的董事会咨询风格。主讲人凭借跨越政商学界与顶尖金融机构的资深履历，用“伦理 AI 就是做正确的事且别进监狱”、“人脑本身也是黑箱”等通俗辛辣的类比，打破了传统合规的官僚主义迷思。叙事节奏紧凑有力，既带有直面监管问责与诉讼风险的严肃警示，又充满鼓励企业看清边界、大胆突进的战略赋能感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=4s) AI 监管与治理是一项“共同责任”，需要结合具体应用场景在股东回报、客户权益、国家利益与社会福祉之间寻求动态平衡。 - [02:37](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=157s) 治理即战略本身，企业应摆脱繁杂的官僚合规主义，将伦理与合规原则沉淀为融入日常业务的“项目化”（Programmatic）机制。 - [04:02](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=242s) 从 Caremark、蓝钟冰淇淋到波音等司法判例表明，董事会与高管负有建立覆盖所有关键任务业务的“风险知情系统”的法定义务。 - [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) AI 治理必须打破 IT 安全技术与业务运营部门的壁垒，应对未来自主智能体大规模部署带来的职责交叉与碎片化难题。 - [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) 解决 AI 黑箱与可解释性不能指望单点技术，而应通过清晰界定数据源、意图、推导方法、测试与透明化等全生命周期规范化实践来实现。 - [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) 人机协同（Human-in-the-loop）正在演变为需要接受严格外部审计的独立专业流程，必须明确介入触发条件与时效标准。 - [10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) 风控部门的真正产出不是一味说“不”或提供直接保护，而是交付高质量的“风险信息”，辅助高管权衡利弊做出知情战略决策。 - [10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) 治理的核心客体是“项目体系”而非静态政策，唯有设定明确的风险容忍度（Tolerance）阈值，风险管理才具备实质意义。 - [12:32](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=752s) 超越既定容忍度的领域往往蕴藏着战略机遇，企业治理架构需通过 5 大核心支柱统摄 14 个操作领域以兼顾风控与创新速度。 - [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) 在激烈的 AI 商业竞速中，“行动迟缓”已成为高管面临的头号实质风险，企业不能因噎废食，而应通过战术管控加速前进。 - [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) 法律对于过失认定的裁量基准在于管理层“当时是否处于能够做出知情决策的地位”，可抗辩的治理流程是免责的关键。 - [19:22](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1162s) 企业应主动引入经董事会批准的内部“风险容忍度”作为内审基准，推动预算、用例筛选与投资回报率（ROI）的闭环落地。 - [20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) 风险上报与信息披露不仅针对已发生的突发事件，任何超出风险容忍度阈值的风险指标都必须遵循制度化的重大性评估程序进行分层报送。 - [21:57](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1317s) 企业必须构建经得起监管与司法独立检验的可辩护治理体系，以防范重大 AI 治理失效直接穿透并危及高管与董事的个人连带法律责任。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=4s) 讲者背景与 AI 监管治理的“共同责任”使命 Brian Allen 开场便直言自己对治理和风险管理充满热情，认为这是一项极其深思熟虑的工作。作为 AI RegRisk 智库（AI RegRisk Think Tank）的负责人，他目前密切协助监管机构、高管及首席执行官处理 AI 带来的监督责任问题。在涉足该领域之前，他曾担任某《财富》100 强企业的首席安全官（CSO），在安永（EY）就董事会与高管的网络安全监督义务提供咨询，并在美国银行政策研究所（Bank Policy Institute）工作过一段时间，代表全美顶尖银行就 AI、量子计算、欺诈等前沿技术议题向国会与监管机构发声。 AI 监管面临前所未有的阻碍，它高度依赖具体应用场景和输出结果，且技术演进速度远超政府立法的步伐，同时还交织着国家安全等复杂维度。为此，智库将核心使命确立为以治理、风险管理与监督为驱动的“共同责任”（communal responsibility）。这种责任的本质在于寻求动态平衡——在股东回报、客户权益、社区福祉、国家利益乃至全球影响之间找到最佳结合点。智库的董事会成员汇聚了处理顶层主权风险与基层微观风险的重量级人物，包括曾掌管美国联邦通信委员会（FCC）国土安全局的前海军将领、美国卫生与公众服务部（HHS）前参谋长，以及来自美国证券交易委员会（SEC）的前总监。这种全方位的视角极为关键，因为整个社会不仅面临国家与个体“主权流失”的宏观挑战，也要应对欺诈犯罪全面泛滥的微观威胁。 - **发言人经历**：曾任 Fortune 100 首席安全官，曾任职于安永（EY）与银行政策研究所（BPI），专注于前沿技术监管与董事会网络安全监督。 - **智库理事会背景**：涵盖前 FCC 国土安全局局长（退役海军将官）、前 HHS 参谋长及前 SEC 总监。 - **治理的核心诉求**：AI 监管无法采用一刀切的硬性指标，必须针对具体上下文场景（context-specific），平衡股东、客户与社会公共利益。 --- ### [02:37](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=157s) 治理即战略：跳出官僚主义与项目化落地谈及企业运作，Brian Allen 向在场听众抛出一个根本性问题：“战略究竟有没有价值？如果战略有价值，你该如何量化它的价值？”在实际商业运营中，许多企业并未向战略分配相称的时间与资源。更普遍的问题是，当人们谈论治理、合规委员会与风险管理时，往往下意识地将其等同于繁冗的官僚程序和行政负担。然而，治理与风险管理在本质上就是“战略”本身，它决定了企业能否在正确的轨道上前行。当前，AI 领域的企业高管在战略落地上正经历前所未有的迷茫：“究竟该由谁来牵头负责？负责任的 AI（Responsible AI）和道德伦理 AI（Ethical AI）到底意味着什么？”在 Allen 看来，所谓伦理 AI 的核心原则非常朴素明了——"做正确的事，别进监狱"（Do the right thing. Don't go to jail.）。但要真正落实该原则，企业必须将其推进为具象且可持续的“项目化”（programmatic）机制。唯有将风险治理从“每季度走过场开一次会”转变为“融入日常业务肌理”，企业才能在保证安全的前提下实现创新与响应速度，而过去的传统管理流程已经完全无法胜任这一要求。 ```mermaid flowchart TD accTitle: 从官僚主义风险管理向战略项目化治理的转变 subgraph Legacy["传统传统管理模式"] A["形式化合规审查\n'勾选式清单'"] --> B["季度风险例会\n与实际业务割裂"] B --> C["官僚负担与创新阻滞"] end subgraph Programmatic["项目化战略治理模式"] D["治理即战略\n明确业务归属与责任"] --> E["项目化落地框架\n融入业务核心肌理"] E --> F["动态平衡：创新速度、\n合规安全性与商业价值"] end ``` --- ### [04:02](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=242s) AI 就绪的三层维度：网络安全监管演进与司法判例镜鉴为了让企业清晰衡量自身的“AI 就绪度”（Readiness），治理与风险管理应当分解为三个层次：第一层是**监督义务与法定责任**（Oversight Obligations and Responsibility）；第二层是**运营治理**（Operational Governance）；第三层则是**AI 独有的前沿技术挑战**。在第一层监督责任上，AI 的演进路径与过去六七年间网络安全领域的发展高度吻合。早期的网络安全合规停留在“走形式、不犯重大过失、按部就班”的表面功夫；但随着数据泄露事故接连不断，监管逻辑发生了根本质变——SEC 和美国货币监理署（OCC）开始直击核心：“既然没有零风险的环境，你们究竟是如何具体管理风险的？谁在负责？” 这种从合规勾选到问责制度的转变，得到了美国公司法一系列里程碑判例的强力背书： - **Caremark 判例**：明确确立董事与企业高管负有“善意履行监督”（act in good faith）的极高法律标准，不能对系统性风险视而不见。 - **Marchand 诉 Barnhill 案（蓝钟冰淇淋案，Blue Bell Ice Cream）**：因李斯特菌感染致人死亡事件，法院裁定要证明管理层善意履职，企业必须预先建立起“基于风险评估的有效系统”（risk-informed system）。 - **波音（Boeing）空难相关诉讼**：法院进一步强化了规则边界，判定风险防范系统必须无死角地覆盖到企业的“一切关键任务”（mission-critical）业务。这些判例彻底改变了董事会层面的对话语境。AI 是否属于关键任务资产？答案显然是肯定的。企业管理层对 AI 的风险管理义务不再是一个可有可无的技术探索题，而是一项具有连带个人法律责任的法定义务。 - **监管转变标志**：2018 年 SEC 明确要求董事会监督网络安全风险管理项目；OCC 随后进驻各银行，严查“网络风险容忍度指标”及其制定方法论是否由治理机构正式批准。 - **责任判定标准升级**：善意履职（Caremark） $\rightarrow$ 必须构建风险知情系统（Marchand） $\rightarrow$ 系统必须覆盖所有核心关键业务（Boeing）。 --- ### [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) 跨部门运营治理的裂隙与 AI 独有的黑箱难题第二层挑战在于**运营治理**的极度碎片化。过去在 IT 技术与网络安全之间，或者在业务连续性、欺诈防范、物理安保与危机响应等部门之间，企业内部早已形成了深厚的部门壁垒（silos）。AI 技术的渗透让这一割裂更加危险。当企业部署一个客户服务 Chatbot 时，“模型是否安全稳定、是否适合技术目标”是一个技术治理问题；但“该 Chatbot 展现出来的交互是否属于合格的客户关怀”则是一个纯粹的运营管理与客户体验问题，IT 部门根本无权也无法定夺。未来当自主智能体（AI Agents）被大规模分发部署到企业各个业务节点时，这种职责交叉与归属权混乱将呈指数级放大。 ```mermaid flowchart LR accTitle: AI 运营治理与技术治理的跨界交叉 subgraph TechGov["技术与安全治理 (IT / Cyber)"] T1["产品适用性\n(Fit for Purpose)"] T2["系统鲁棒性与安全防护\n(Safe & Secure)"] end subgraph OpGov["业务与运营治理 (Business Ops)"] O1["客户关怀标准\n(Customer Care)"] O2["服务体验与商业伦理\n(Brand & Trust)"] end TechGov --- Chatbot["核心交叉点\n（例如：企业级 AI 智能体 / 客服 Chatbot）"] --- OpGov ``` 第三层挑战则是**AI 独有的技术难题**，即模型的可解释性（Explainability）、偏见消除（Bias）、公平性（Fairness）以及“人机协同”（Human-in-the-loop）。针对技术人员经常抱怨的“模型是黑箱，根本无法做到技术上的端到端可解释”，Allen 曾与美国财政部、OCC、美联储（Fed）以及联邦贸易委员会（FTC）展开密集探讨，并达成了务实共识： > "人类的大脑本身也是一个黑箱。当你向我提问时，我无法向你展示脑神经元具体是如何放电的；但我能告诉你我的数据来源、我的决策意图、我推导结论的方法，以及我的输出结果。随后我对其进行测试，持续审查潜在风险，并对整个过程保持透明。" 因此，可解释性不能指望单点技术突破，而必须由一系列覆盖全生命周期的规范化实践共同构成。同样，“人机协同”也绝非让人浮于表面地看一眼输出那么简单。未来的监管机构必将严格盘查企业的人机交互规程：介入的具体触发标准是什么？该决策是否涉及关键业务决策？是否有高严格的时效性要求？其中是否包含个人身份信息（PII）或受保护的健康信息（PHI）？“人机协同”本身正在快速演化为一套必须接受外部严格审计的独立专业流程。 --- ### [08:21](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=501s) 从诉讼倒逼到智库的 AI 就绪核心框架目前大部分企业的 AI 探索仍处于初级阶段。虽然企业在战略讨论和 POC 试点上动作频频，但真正关乎核心命脉的关键用例（critical use cases）尚未全面铺开。Allen 预计，在接下来的这一年左右，关键业务用例将加速走向前台。伴随这一进程而来的，必然是系统性故障、人身或财产损害事件以及业务弹性瘫痪等现实危机。在成文监管法规全面成熟之前，权力真空将迅速被诉讼律师所填补。因模型缺乏可解释性或存在隐性偏见而引发的企业诉讼已经开始显现，这一现实痛点将倒逼企业不得不建立严密的治理防线。针对这种紧迫需求，AI RegRisk 智库基于判例法标准、现行监管指引与各州法律，构建了一套包含五大核心支柱的“AI 就绪度与风险管理框架”（Readiness Program）。 Allen 强调，许多在网络安全领域自认为成熟的企业，手握大量高级工具和合规流程，甚至具备完善的报表机制，但其内核根本算不上真正的“风险管理”。很多企业虽设立了关键风险指标（KRIs），却从未对这些指标设定明确的容忍阈值（thresholding）。如果没有科学的阈值界定与闭环控制机制，所谓的合规系统在面临重大监管问责与法律诉讼时，将不堪一击。 ### [10:04](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=604s) 治理的核心是“项目”而非政策：风险管理的本质是提供决策信息真正的风险管理精髓在于“容忍度”（Tolerance），如果企业没有让合适的人参与并划定风险阈值，那么根本谈不上在做风险管理。许多企业习惯性地从制定流程和制度着手，这固然是不错的起点，但往往欠缺了至关重要的“项目化机制”（Programmatic Approach）。衡量良好治理的标准，绝不是“你是否设立了专门的委员会”，核心拷问应当是：“你们到底在治理什么？”答案其实非常明确——治理的对象是“项目体系”（The Program）。这个项目体系必须界定好范围，明确应将其嵌入到组织的何种颗粒度深度，确保员工清晰了解公司的风险容忍度底线与行为边界。更重要的是，“容忍度”绝不意味着一味地说“不”。在现实中，安全或风控人员常常告诫自己“不要成为只会说不的部门（Department of No）”，这种论调甚至让演讲者感到抓狂，因为说“不”本来就不是风控部门的职权所在。安全人员往往误以为自己的产出是“保护”，但保护仅仅是缓解手段或战术措施带来的效果。一个卓越风险管理实践的真正产出，永远是“风险信息”（Risk Information）——通过提供客观充分的信息，让业务端与高管层权衡利弊后做出明智决策。这就将风控自然提升到了战略对话的高度。如果企业认同战略具有价值，就必须建立起能够提供及时输出、融入业务肌理、高度可复用且杜绝官僚主义的项目化机制。 - **治理客体明确化**：治理并非形式主义的委员会架构，其实质是对整个风险管理项目（Program）的范围与下沉颗粒度进行管控。 - **风控产出的正名**：风险部门的核心职责不是拍板否决或直接提供保护，而是交付高质量的“风险信息”，辅助高层进行商业权衡。 - **战略价值审视**：在快节奏环境下，仅有政策文件无法支撑业务高速运转，必须依赖精细且可重复执行的项目化运作。 --- ### [12:32](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=752s) 超越容忍度即是战略机遇：五大支柱与十四个领域在当下的商业竞争中，超出风险容忍度的事物并不单纯代表危险，往往更蕴藏着巨大的“战略机遇”。在人工智能领域尤甚，如果一家企业能够比竞争对手更快洞察转角处的局势，就能以更快的步伐突进。为了帮助企业把握机遇与风控的平衡，一套完整的 AI 准备度架构被归纳为“五大核心支柱”（Five Core Pillars），其底层依托“14 个领域”（14 Domains）作为操作层面的考量因素。这种结构兼具适应性与灵活性，企业可按需裁量，但其落脚点始终是各个模块之间如何协同形成一个系统化的项目体系。 ```mermaid flowchart TD accTitle: AI准备度治理架构图 subgraph S1["治理核心逻辑"] A["政策与工具 (执行/保障手段)"] --> B["项目化机制 (可重复/定范围/具颗粒度)"] B --> C["高管战略决策 (平衡风险与商业机遇)"] end subgraph S2["落地支撑体系"] D["五大核心支柱 (治理、系统、责任、战略、执行)"] E["14个考量领域 (模型/偏见/数据/第三方等)"] D --- E end C -.指导.-> D ``` 演讲者反复强调，“政策不等于项目”（Policy is not a program）。政策充其量是实现执行的工具、实践或职能手段。因此，董事会或高管层在启动治理讨论时，第一步不是质询政策条文，而是审视：“我们是否拥有一个定义清晰、可复用、范围明确且颗粒度到位的项目体系？”这正是治理的首要职能。许多企业设立了“知情委员会”（Informed Committees），大家按季度聚会同步风险资讯，所有人离开会议室时都觉得自己获取了信息。这种形式固然有其价值与必要性，但真正的治理必须剥离出两项根本责任：第一是确认项目体系本身是否合规、恰当且运转良好；第二则是建立一套经得起推敲的“风险知情系统”。 - **机遇与风险并存**：越过传统容忍度边界的领域往往是 AI 创造差异化竞争优势的制胜点，“看得更快”才能“跑得更快”。 - **支柱与领域联动**：通过 5 大核心支柱统摄 14 个操作领域，确保治理体系既有顶层框架又具备落地弹性。 - **知情委员会的局限**：例行的季度信息同步并不等同于实质治理，核心在于保障整套项目机制能够稳定运转并输出成效。 --- ### [14:24](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=864s) 风险知情系统：董事会的法定义务与容忍度构建 “风险知情系统”（Risk-Informed System）这一概念直接承袭自司法审查与法院判例。许多管理者对此存在误解，认为每季度向董事会汇报一次 PPT 或维护一份风险登记册（Risk Register）就算交差。但司法与监管所关注的核心在于：企业到底通过什么系统化的机制来界定和达成“容忍度”？这种机制是否足够高频？是否具备应对企业动态变化的灵活性与适应性？在实践中，企业可以通过设定关键风险指标（KRI）阈值，或是借助成熟度模型（Maturity Modeling）等多种方式来确立容忍度，但关键是要有明确的路径并传递给必须知晓的高层。对于上市公司而言，建立经董事会批准的风险知情系统绝非可有可无的选择，而是一项严肃的法定义务。在很多时候，当向董事会和高管层阐明这一要求的法律渊源时，他们反而会非常欣慰，因为合规体系不仅保护了管理层免遭监管追责，更借助明确的容忍度让业务团队清晰掌握执行策略，并在捕捉市场机遇的同时排查潜在的“商业地雷”。演讲者曾与美国证券交易委员会（SEC）官员交流，对方明确指出，拥有由董事会批准的系统是风险管理的基石。然而令人震惊的是，即便在被公认为全行业风险管理成熟度最高的银行业中，仍有大量银行根本没有建立起此类合格的容忍度系统。 - **法定义务而非选修课**：风险知情系统来自司法裁判标准，上市公司管理层与董事会负有明确的法定监督职责。 - **手段不限但重在实效**：风险评估本身有成千上万种工具，关键在于评估方式是否真正服务于“界定并达到容忍度”这一核心目标。 - **行业成熟度盲区**：即便是合规底蕴深厚的大型金融机构，在“董事会审批的容忍度系统”建设上依然存在大面积缺位。 --- ### [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) 负责任 AI 与抗辩能力：“行动太慢”成最大风险与过失认定标准进入“负责任 AI”（Responsible AI）层面后，治理开始切入一系列战术性挑战，涵盖可解释性（Explainability）、算法偏见（Bias）、人机协同（Human-in-the-loop）、第三方 AI 风险管理，以及当下新兴的 AI 智能体（Agentic AI）管控。演讲者分享了在银行政策研究所（Bank Policy Institute）举办 AI 研讨会时的真实经历：当各家银行的首席信息官（CIO）齐聚一堂并逐一阐述面临的前三大风险时，出乎所有人意料的是，几乎所有 CIO 都一致认为“行动太慢”（Moving too slow）名列前三。来自首席执行官的紧迫感、激烈的市场竞速以及竞争对手的施压，迫使企业绝不能因噎废食地采取一票否决式的防御姿态，而是必须学会迎难而上，通过战术管控来驾驭可解释性与偏见问题。对于大型企业而言，涉足 AI 领域遭到诉讼几乎是不可避免的商业常态，关键在于企业是否具备“法庭抗辩能力”（Can you defend it?）。在涉及疏忽（Negligence）的判例法中，司法的裁量基准往往并不是“你是否做出了绝对正确的决定”，而是“**你当时是否处于能够做出知情决策的地位**”（Were you in a position to make an informed decision?）。无法做出知情决策，才是构成法律疏忽的根源，而这正是项目化机制的价值所在。负责任 AI 的落地，必须穿透数据源头、业务意图、操作流程、算法输出、模型测试与日常监控的全流程。在模型风险管理（MRM）中，很多企业陷入了断层：关注模型的人不与管数据的人沟通，负责测试监控的人又与直接使用客服聊天机器人的业务团队脱节，唯有打破壁垒、打通全链条，才能建立真正负责任的 AI 体系。 - **CIO 们的头号焦虑**：在激烈的市场竞争倒逼下，“动作迟缓”已被众多头部企业高管列为最具破坏性的实质风险之一。 - **过失认定的法律本质**：企业能否免责不在于结果是否完美，而在于流程上是否建立了确保管理层做出“知情决策”的前提条件。 - **全生命周期穿透治理**：模型风险管理不能孤立存在，必须将数据、模型、流程、业务使用场景（如客服 Chatbot）以及持续监控端到端串联。 --- ### [19:22](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1162s) 基于风险的战略与执行：预算、ROI 与对齐容忍度的审计在拥有了风险知情系统之后，治理推进到了“基于风险的战略与执行”（Risk-based Strategy and Execution）阶段。风险知情系统本身并不负责拟定商业战略，其使命是构建一个可供战略依托执行的支撑系统。在这个阶段，管理层需要综合考量各方利益相关者的平衡，重点推进资源预算编制、优先级设定、AI 用例（Use Cases）筛选、投资回报率（ROI）测算，以及明确的风险升级与披露机制。在谈及战略落地时，演讲者特别分享了他与内部审计团队合作的独特经验。许多业务和技术人员对审计心存芥蒂，但他却非常推崇与审计师建立良性互动。团队曾主动要求审计人员：“请不仅依据业界最佳实践和标准来审计我们，更要依据我们自主制定并报送董事会的‘风险容忍度’来审计我们。”因为在现实监管环境中，监管容忍度与业务诉求始终在动态演变，唯有将审计基准锚定在董事会认可的容忍度上，才能确保整个组织的战略执行既不逾矩，又能在既定安全边界内释放最大的创新动能。 - **系统向战略执行的跃迁**：以风险知情系统为基石，推动资金预算、用例优先级排序与 ROI 的量化落地。 - **重塑审计协同关系**：跳出单纯对照外部通用标准的局限，引入经董事会批准的内部“风险容忍度”作为审计核心尺度。 - **动态平衡监管与业务**：借助内审对容忍度执行情况的校验，使企业在监管容忍度波动的大环境下保持战略敏捷与合规底线。 ### [20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) 监管风险思维与风险上报及披露机制在面对监管时，企业需要树立更加开放的风险承受思维。合规本身就是一种风险管理选择，企业并不一定要盲目追求绝对合规——虽然违规可能会带来痛苦的代价（"It may hurt, but you don't have to be compliant"），但关键在于企业能否精准界定并有效管理自身所承担的监管风险敞口。治理体系的最后一个核心支柱是**风险上报与信息披露（Risk Escalation and Disclosure）**。许多团队在被问及这一环节时，往往会回答“我们有一套事件响应计划（Incident Response Plan）”，但这并未触及治理的本质。突发安全事件固然会触发上报机制，然而**任何超出企业风险容忍度（Risk Tolerance）的风险指标**，无论是否已演变成实际事故，都必须根据其严重程度及相应层级进行上报。这种上报机制必须是制度化、程序化（Programmatic）的流程，而绝不能依赖私下的人际关系（"Can't be relationship based"）。信息披露同样需要严密的程序支撑。对于上市公司而言，披露的核心在于重大性（Materiality）。衡量重大性不仅仅是判断事件本身是否重大，更关键的是企业**在何时、基于什么标准、经过了哪些具体评估步骤**将其定义为重大事件。只有具备清晰的评估判定流程，企业才能在达到特定门槛时向监管机构妥善报备，并面向公众适时发布披露，从而有效维护市场与客户的信任。 ```mermaid flowchart TD accTitle: 风险上报与披露的程序化流程 subgraph Trigger["触发机制"] A["突发安全事件 (Incident)"] B["风险超出容忍度阈值 (Exceeding Risk Tolerance)"] end subgraph Process["程序化评估与上报"] C["程序化风险判定流程 (Programmatic Assessment)"] D["重大性判定标准与时间节点 (Materiality Criteria & Timing)"] end subgraph Disclosure["分层披露与报送"] E["高管层与董事会 (Executive / Board)"] F["监管机构报备 (Regulator Disclosure)"] G["公众披露与信任维护 (Public Disclosure)"] end A --> C B --> C C --> D D --> E D --> F D --> G ``` - **监管即风险（Regulation as a Risk）**：合规策略不是单维度的死板执行，而应在明确业务承受力与后果的前提下，对其影响范围与敞口进行动态管控。 - **超越事件响应（Beyond Incident Response）**：上报机制不仅由已发生的危机事件触发，更由偏离风险容忍限度的潜在风险所驱动。 - **重大性判定流程（Process to Materiality）**：监管关注的重点在于企业判定重大事件的具体标准、评估证据链与时间线节点。 --- ### [21:57](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1317s) 构建可独立辩护的程序化治理体系与高管问责企业管理者需要深刻反思自己现有的风控体系是否真正达到了“程序化（Programmatic）”的标准。很多组织拥有大量的流程（Process）和政策文档（Policies），但这并不等同于建立了一套完整的程序化体系。衡量治理水准的真正标准在于：**这套程序化体系本身能否独立存在并立得住脚？在面临内外部审查与法律挑战时，它是否具备能够自我证明的合规与风控辩护力（"Can that program be defended?"）？** 如果一套治理框架连可辩护性都无法证明，单纯堆砌流程与静态政策将毫无意义。尤其是对于上市公司或规模较大的企业，一旦发生重大治理疏漏，其高管、业务所有者以及董事会成员都将直接承担个人连带法律责任。因此，必须将这些要求带回董事会与管理层，对整体治理框架进行根本性的重塑与再构想（Reimagining）。在演讲结束之际，由于全场时间已耗尽，演讲者向现场观众致谢并结束了本次关于 AI 治理战略与风险就绪的分享。 - **体系的可辩护性（Programmatic Defense）**：风控的核心不仅是制定静态的规章制度，而是建立一套经得起法务、监管与技术审计独立检验的程序化机制。 - **高管与董事会责任（Executive & Board Liability）**：在 AI 等前沿技术引发的风险面前，治理失效将直接穿透并危及高管与董事个人的法律安全。 ## Notable quotes > [02:37](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=157s) Do the right thing. Don't go to jail. — Brian Allen > [06:13](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=373s) 人类的大脑本身也是一个黑箱。当你向我提问时，我无法向你展示脑神经元具体是如何放电的；但我能告诉你我的数据来源、我的决策意图、我推导结论的方法，以及我的输出结果。随后我对其进行测试，持续审查潜在风险，并对整个过程保持透明。 — Brian Allen > [12:32](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=752s) Policy is not a program — Brian Allen > [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) Moving too slow — Brian Allen > [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) Can you defend it? — Brian Allen > [16:43](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1003s) Were you in a position to make an informed decision? — Brian Allen > [19:22](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1162s) 请不仅依据业界最佳实践和标准来审计我们，更要依据我们自主制定并报送董事会的‘风险容忍度’来审计我们。 — Brian Allen > [20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) It may hurt, but you don't have to be compliant — Brian Allen > [20:28](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1228s) Can't be relationship based — Brian Allen > [21:57](https://www.youtube.com/watch?v=a4OLsgzMi9s&t=1317s) Can that program be defended? — Brian Allen ## Mentioned Brian Allen, AI RegRisk Think Tank, Fortune 100, EY, Bank Policy Institute, FCC, HHS, SEC, OCC, Caremark, Marchand v. Barnhill, Blue Bell Ice Cream, Boeing, U.S. Department of the Treasury, Federal Reserve, FTC ## Video description Brian Allen, Senior Advisor, AI RegRisk Think Tank Abstracts Artificial Intelligence presents immense strategic opportunities—but also introduces novel risks and a rapidly evolving regulatory landscape. Established oversight obligations already apply and demand C-suite attention. This session tackles a crucial question: "What exactly are we governing?" Hint: Effective governance doesn’t focus on the AI technology itself, but on the necessity of a defined program designed to manage strategy, risk, and accountability in this transformational environment. We will outline the foundational elements of an AI Readiness Program, offering a practical framework for confidently integrating AI capabilities. The session will provide an approach to navigate the complex and fast-moving legal and regulatory landscape and will explore how robust governance can proactively address emerging technical and ethical challenges—such as model explainability and bias—while managing significant and growing challenges. Drawing parallels to recent enforcement actions that highlight individual accountability, we will discuss how strong governance establishes the essential guardrails your organization needs to innovate and compete with speed and confidence. Join this vital session for a C-suite perspective on building the strategic governance framework necessary not only to mitigate AI’s risks but to fully harness its transformative potential for competitive advantage. [Source](https://www.youtube.com/watch?v=a4OLsgzMi9s)

Source quotes

[2:37] Brian Allen: Do the right thing. Don't go to jail. [6:13] Brian Allen: 人类的大脑本身也是一个黑箱。当你向我提问时，我无法向你展示脑神经元具体是如何放电的；但我能告诉你我的数据来源、我的决策意图、我推导结论的方法，以及我的输出结果。随后我对其进行测试，持续审查潜在风险，并对整个过程保持透明。 [12:32] Brian Allen: Policy is not a program [16:43] Brian Allen: Moving too slow [16:43] Brian Allen: Can you defend it? [16:43] Brian Allen: Were you in a position to make an informed decision? [19:22] Brian Allen: 请不仅依据业界最佳实践和标准来审计我们，更要依据我们自主制定并报送董事会的‘风险容忍度’来审计我们。 [20:28] Brian Allen: It may hurt, but you don't have to be compliant [20:28] Brian Allen: Can't be relationship based [21:57] Brian Allen: Can that program be defended?

Source quotes

[2:28] Brian Allen: Do the right thing. Don't go to jail. [4:02] Brian Allen: dot your i's, cross your t's, don't be negligent. [6:13] Brian Allen: My head is a black box. You ask me questions. I don't tell you how the neurons in my brain work. I tell you my data, my intention, how I went about it, my output. [10:04] Brian Allen: What are you governing? [10:04] Brian Allen: Department of No [16:43] Brian Allen: moving too slow [16:43] Brian Allen: in a position to make an informed decision. [20:28] Brian Allen: Oh, we have an incident response plan. [21:57] Brian Allen: I'm not talking process and policies,

Original transcript

[0:04] I love uh governance and risk [0:06] management. I find it incredibly [0:08] thoughtful. I hope you do as well. This [0:10] space is moving fast. Um let me share a [0:14] quick background on myself. So today I'm [0:15] the managing director of the AI [0:17] regressor think tank. We work closely [0:19] with regulators um executives CEOs on [0:23] how are they managing their oversight [0:25] obligations with AI and I'll hit on that [0:27] in a little bit. Before that, I was a [0:28] chief security officer for for Fortune [0:30] 100. I worked at EY, worked with boards [0:32] and executives on their oversight [0:34] obligations of cyber. Um, there's a lot [0:36] moving in that space. And then I spent a [0:39] little bit of time at the bank policy [0:40] institute where we represented the top [0:42] banks in front of Congress and [0:43] regulators on emerging tech issues, AI, [0:46] quantum fraud, things that were moving. [0:49] Um the regressor think tank is you know [0:52] we're getting this so we're getting some [0:54] amazing momentum around the mission of [0:56] communal responsibility [0:58] really being driven by maturing [1:00] governance and what does that mean? Um [1:02] AI is going to be incredibly difficult [1:04] to regulate. It has to be context [1:06] specific. We're clearly in an [1:08] interesting environment with regulation [1:10] right now. Um it it's almost output [1:13] specific. It's moving incredibly fast at [1:16] technology faster than the government [1:17] can move. Clearly, we've got national [1:19] security issues we've got to pay [1:21] attention to. Um, so there are a lot of [1:23] dynamics that are going to make this [1:25] very difficult. [1:28] The mission of communal responsibility [1:30] is really driven around governance, risk [1:32] management, oversight because we have to [1:35] find balance, right? Balance of what's [1:39] best for your shareholders, for your [1:40] customers, for the community, for the [1:42] United States, for the globe. And you [1:45] know, on our board, we've got a former [1:46] admiral um who ran the the he was a [1:49] chief of the Homeland Security Bureau [1:50] for the FCC, a former chief of staff for [1:53] HHS, [1:54] um former director from the um uh the [1:58] SEC. So big thinkers, big things of like [2:02] sovereignty, like countries are losing [2:03] sovereignty, people are losing [2:04] sovereignty down to small things, not [2:07] small things, but fraud is going to get [2:09] worse. Um I did a lot of strategy work [2:11] with the banks. We're seeing a lot of [2:12] fraud. it's going to get worse. So we [2:15] have to find balance as individuals, [2:18] academia, the government, regulators. [2:21] And with that, you know, we need to [2:23] mature what is governance, what is risk [2:26] management. [2:28] And I'll appine that I think it's about [2:30] programmatic governance. And I'm going [2:32] to talk a lot about that. Um, so first, [2:37] does strategy have value? [2:40] Right? Does strategy have value? How do [2:42] you put a number on the value of [2:44] strategy? [2:46] And if strategy has value, are you [2:48] spending the commasant amount of time on [2:51] that as a person, as a company, as a as [2:54] a function, right? Are we spending the [2:55] right enough time on that? Governance [2:58] and risk management is strategy. We talk [3:01] a tremendous amount about governance, [3:03] about, you know, committees. It feels [3:04] bureaucratic. It feels burdensome, but [3:07] it's strategy. [3:09] And that's the focus that we have to [3:11] start putting on this. And the problem [3:12] that I'm seeing and I've done a lot of [3:14] assessments on for global fortune50s and [3:17] other companies about their strategies, [3:19] cyber, supply chain, operational [3:21] resilience, [3:23] and it's seen as bureaucratic. It's seen [3:25] as burdensome. And so we've got to kind [3:26] of shift that conversation back to [3:28] strategy. Executives are feeling [3:30] struggle with strategy right now with [3:31] AI. Who owns it? Right? What does [3:35] responsible AI mean? What does ethical [3:36] AI mean? Ethical AI is a great question. [3:40] Do the right thing. Don't go to jail. [3:42] Right? It seems kind of simple, but it's [3:43] got to be practical and it's got to be [3:45] programmatic. And the programmatic piece [3:47] is is really critical to what we're [3:49] seeing for speed, for engagement, so [3:51] that risk doesn't become this quarterly [3:54] meeting to fabric of business. [3:57] And we can't do that right now with the [3:59] former processes that we've had in the [4:01] past. [4:02] And so there like these three layers [4:04] that we are looking at governance and [4:06] risk management and implementation and [4:08] we'll call it readiness. All right, our [4:11] company's ready for this one is [4:13] oversight obligation and responsibility [4:16] and just some direction. There's a [4:18] tremendous parallels that we're seeing [4:21] with cyber and where cyber was and [4:26] the cyber space right for the last the [4:29] last six or seven years what we saw was [4:31] this dichotomy of responsibility dot [4:34] your eyes cross your tees don't be [4:36] negligent use good practice we kept [4:39] seeing breaches the regulators the SEC [4:41] the OC started saying okay we've been [4:44] doing that there's no zerorisk [4:45] environment So, how are you managing [4:48] your risks and who's in charge? It was a [4:51] really good question. 2018, the SEC came [4:54] out to boards and corporate offices and [4:55] said, "You have oversight of our cyber [4:57] risk management program." It was a very [5:00] effective term. The OC was going around [5:03] to banks saying, "Show me your cyber [5:06] tolerance. Show me your methodology that [5:09] you get to tolerance and show me that [5:11] your governance body approved the [5:12] methodology." Right? They moved from dot [5:15] your eyes across your tees to who's [5:17] managing the risks and are we managing [5:19] his risks and risks again can be [5:22] strategy. [5:23] So there's oversight obligations. [5:25] There's accountability. There's a set of [5:26] Caremark cases that hold boards and [5:29] corporate offices liable. Kmark said to [5:31] boards and corporate officers, you have [5:32] to act in good faith. Very very high [5:34] bar. [5:36] There was another case called Merchant [5:38] after that. And the Marchant case was a [5:39] company that sell bluebell ice cream. [5:41] People died of lististeria. And they [5:42] said in order to act in good faith you [5:44] have to have a riskinformed system in [5:45] place. [5:47] And then Boeing came and the Boeing [5:48] court case said in order and not only [5:51] does have do you need a risk-informed [5:53] system but it has to be applied to [5:54] everything mission critical. Right? So [5:56] some of the question is is AI mission [5:58] critical? Is cyber mission critical? [6:02] That started to sh change the landscape [6:04] a little bit in conversations with [6:05] boards and executives on what is their [6:06] overset obligation? They have obligation [6:09] with AI risk management. they have [6:10] obligation period. [6:13] The second is governance, right? The [6:14] governance challenges of operational [6:16] governance. [6:17] You know, we've seen like IT, tech, and [6:19] cyber, right? Very siloed operations, [6:22] operational resilience, fraud, physical [6:24] security, business continuity, crisis [6:25] management, very siloed organizations. [6:28] AI is going to make this incredibly more [6:30] challenging. All right, you've got [6:32] technology govern, you got oversight [6:33] governance, you got technology [6:34] governance. Is is the AI is the product [6:38] fit for purpose? [6:40] Right? Is it safe? Is it secure? And [6:42] then if you're building a chatbot, [6:44] there's a question of operational [6:45] governance. Is that good customer care? [6:47] That is not an IT question, right? And [6:50] so you're seeing this complexity of how [6:53] do we manage this and who owns it? So [6:56] governance is going to become really um [6:58] really critical and it's going to get [7:00] even more challenging than it is today [7:01] once agents come into play and we start [7:03] to see them distributed across the [7:05] enterprise. And the last is novel AI [7:08] challenges. Explainability, bias, [7:11] fairness, soundness, human in the loop. [7:14] Explainability is possible, right? It's [7:16] not possible from a technical [7:17] standpoint. We have a black box. I've [7:20] talked to Treasury and the OC and the [7:22] Fed FTC all about this. We were able to [7:24] get on the same page. My head is a black [7:26] box. You ask me questions. I don't tell [7:28] you how the neurons in my brain work. I [7:31] tell you my data, my intention, how I [7:33] went about it, my output. I test it. I [7:37] continuously look for risk. I'm being [7:38] transparent about this process. But [7:40] that's not one practice, right? Those [7:42] are multiple practices within this AI [7:45] environment that we're going to have to [7:46] incorporate. [7:48] Um, you know, we hear a lot about human [7:50] in the loop. And what I heard recently [7:52] from regulators, which is really [7:53] interesting, is yes, we need human in [7:55] the loop. What does that mean? I look at [7:57] the output. I, you know, I validate it. [7:59] Really the question that they're going [8:01] to start asking is what is your practice [8:04] of bringing the human in the loop? [8:06] What's the criteria? Was it a critical [8:09] decision? Is there a timeliness factor? [8:11] Was there PII? Was there PHI tied to it? [8:14] Right? So the practice of human in the [8:15] loop is going to become a practice. [8:17] There's going to be scrutiny around that [8:18] at some point. [8:21] I think what we what you know my [8:23] personal my perspective on on some of [8:25] this and we are very early in this [8:27] process. Um, we're not early in [8:29] strategy. We're not early in thinking [8:31] about it. You know, business is already [8:32] doing that. They've got pilots. They've [8:34] got some. But what we haven't seen yet [8:36] is critical use cases and we will see [8:39] that probably next year. Maybe some are [8:41] trickling out there. Um, [8:44] but we haven't seen that yet. And when [8:46] we do, there will be harm, there will be [8:48] incidents, there'll be resilience [8:49] issues. [8:51] And that that will change this [8:53] conversation a little bit. The lack of [8:55] regulation right now will be filled by [8:58] attorneys, right? We will see lawsuits. [9:00] We're starting to see lawsuits, lawsuits [9:02] on explanability and and bias, [9:05] but critical use case will drive this [9:07] conversation a bit further and will [9:09] drive the need for for what we're going [9:11] to talk about. [9:14] So at the think tank we put together [9:16] this this readiness program and it's [9:19] really a construct of you know mapping [9:21] case law standards regulatory guidance [9:25] um state laws to these kind of five core [9:29] elements of what is a readiness program [9:32] what is an AI risk management program [9:35] and the programmatic piece is very [9:37] critical [9:39] because what I've seen over the cyber [9:42] space and assessing in programs [9:44] even very like very mature organizations [9:47] is they have process [9:50] right they have tools but what it's not [9:53] sometimes it's not even risk management [9:55] right they have reporting capabilities [9:57] they have kis but the kis haven't been [9:59] thresholded threshold thresholding kis [10:04] is the magic that's the tolerance and if [10:07] you're not thresholding that with the [10:08] right people you're not doing risk [10:09] management I've seen this a lot with [10:12] process and policies which is a great [10:15] thing right it's a great starting point [10:18] but the programmatic piece is really [10:20] what's is really what is lacking and [10:23] that's the part where when I think about [10:26] governance and I see good governance [10:28] right the question is not do you have a [10:30] committee the question is what are you [10:32] governing [10:34] and what they're governing is the [10:35] program [10:38] and so the program needs to be scoped [10:40] right scope to what do we going to apply [10:42] this to what is the granularity [10:45] of how how granular this program needs [10:48] to be embedded into the organization so [10:50] that we're comfortable that people [10:52] understand our tolerance that people [10:54] understand our boundaries [10:56] and tolerance does not mean no [10:59] especially today [11:01] right when I hear organizations I heard [11:03] this a lot from security folks don't be [11:05] the department of no [11:08] it it drives me nuts because it was [11:11] never their decision in the first place [11:12] to say no. [11:16] Never. The output of risk programs is [11:19] risk information. Right? You ask [11:21] security folks, the cyber folks say, [11:22] "Oh, protection." It's not protection. [11:24] Protection is the effect of a mitigation [11:26] program, a mitigation process, a tactic. [11:30] The output of a good risk practice is [11:34] risk information so that the business [11:36] and the executives can make informed [11:38] decisions on balance. That's the output. [11:41] And so that becomes now this strategy [11:43] conversation. [11:45] Does strategy have value? It's a really [11:48] good question. It's a great way to [11:50] approach executives or if we struggle [11:52] with governance. It's a tough cell. I [11:54] get it. It's a tough cell. It's it takes [11:57] effort. But if strategy has value, [12:01] right? Does applying resources applying [12:04] this and is that output is it timely [12:06] enough? Is it becoming fabric of [12:08] business? Is that worth the effort? And [12:11] the question goes back to does strategy [12:13] have value, [12:15] right? And you cannot do this in this [12:17] environment, this speed [12:20] without a programmatic approach that is [12:22] repeatable, that's as granular as it [12:24] needs to be, that is not bureaucratic, [12:27] that sees the value of the output of [12:29] this as strategy. [12:32] So, back to tolerance [12:35] and the no question. What we're seeing [12:37] today, which I love, is sometimes things [12:42] falling outside tolerance [12:44] and now we're seeing a lot of times [12:46] things fall fall outside tolerance is [12:47] opportunity [12:49] with AI, right? Because if you can move [12:52] faster, [12:53] if I can see around a corner faster than [12:55] my competition, I can go faster, [12:58] right? So, opportunity, opportunity. [13:04] So you know the five core pillars um [13:08] governance um so what we have we've got [13:11] the five core pillars and behind that [13:12] we've got um 14 domains the domains look [13:16] as as considerations [13:18] consider these things right as you're [13:20] building your governance program this [13:22] way it makes it adaptable flexible you [13:24] apply it to how you need it but again [13:26] focus on how this all works together as [13:28] a programmatic approach policy is not a [13:30] program [13:32] policy is enforcement right it's tool. [13:34] It's a practice. It's it's it's a [13:36] function to help it to help it work. [13:39] Um so so governance of like who needs at [13:42] obviously who needs to be at the table, [13:44] but probably more importantly what is [13:45] their role in the programmatic approach [13:47] and the governance conversation should [13:49] start at is our do we have a program not [13:52] policies. Do we have our program? Is it [13:55] defined? Is it repeatable? Is it scoped [13:58] right? And is it granular? Right? That [14:00] is the first function of governance. [14:03] It's not like we see a lot these [14:05] informed committees. We meet quarterly. [14:08] We talk about it. We tell you what the [14:09] risks are, right? And everyone walks [14:11] away informed. It's a good it's good. [14:13] It's necessary. It's important. But we [14:16] have to kind of split a little bit of [14:18] the responsibilities of governance. The [14:19] first one is is our program right? Is it [14:21] appropriate? Is it working? [14:24] The second is a riskinformed system. And [14:27] the risk informed system is you know [14:30] this comes from the courts right and the [14:33] system and I hear a lot from folks oh we [14:34] do reporting we have we go in front of [14:36] the board quarterly the system is about [14:38] what is your system to get to tolerance [14:42] how do you get to tolerance is it [14:45] frequent enough is it flexible enough is [14:47] it adaptable to your organization [14:52] so the system is really starting to [14:54] define do you get that through kari [14:56] thresholding You can do it through [14:58] maturity modeling. There's a number of [14:59] ways to get to tolerance. Who needs to [15:02] hear the tolerance, right? So that's the [15:04] system. [15:05] Define the system or if you're if you're [15:08] a public company, you need this, [15:11] right? This is not optional. These are [15:12] obligations. [15:14] And I say that as a as a threatening [15:16] thing, but a lot of times when I go to [15:18] speak to boards and and corporate [15:19] offices, they're not aware, right? So [15:21] it's an opportunity when you bring this [15:23] to them and you say, "We're going to put [15:24] in this rich form system. By the way, [15:25] here's where it comes from. One, they're [15:27] appreciative that you're looking out for [15:29] them, that you're following laws, you're [15:30] following regulations. Um, and two, when [15:33] they when they hear, okay, we're going [15:34] to build tolerance. What's the benefit [15:36] of tolerance? Right? That's the strategy [15:38] of how people are going to execute. It's [15:40] also going to help us look for things of [15:41] opportunity and potential landmines. [15:45] The risk informed system, we don't get [15:46] into how you do risk assessments at all. [15:49] There's thousands of ways to do this. [15:51] I've executed in Fortune a Fortune 100, [15:55] it's not really about that. And you [15:56] know, when I talk with um and when we go [15:58] in and assess programs, [16:00] a lot of the feedback we get is yes, we [16:02] do quarter reporting. We do have [16:03] assessments. We have a risk register. [16:05] That's great. Is it is it fitting the [16:08] need, right? What is the essence of [16:09] getting to tolerance? And that's why I [16:11] love the SEC. In fact, the director of [16:14] um the SEC, Michael Chu, I chatted with [16:16] him one time and I said, "Mike, why did [16:17] you go down this path of looking at the [16:20] cyber tolerance piece [16:23] and and he said, "Oh, well, it's [16:24] fundamental, right? It's fundamental to [16:27] risk management that there's a system [16:29] that has been approved by boards." It [16:30] was really amazing how many banks that [16:33] we worked with that did not have [16:34] something like that in place who are the [16:37] most mature at risk management in any [16:39] industry. I think [16:43] so. Um, responsible AI. This is kind of [16:46] where we get a little bit tactical to [16:48] taking on explanability. Um, [16:51] uh, bias, human in the loop, third party [16:55] AI management is going to be different. [16:57] AI agentic management is going to be [16:59] different. And so, can you put the right [17:01] tactics in place to be able to take on [17:04] some of these new AI challenges that [17:06] fall a little bit outside of what we do? [17:10] Just a quick, you know, observation when [17:12] when I was at the bank policy institute, [17:15] we would talk with with banks, we had a [17:17] uh we had an AI symposium with the CIOS [17:21] came in and you know, we went around the [17:23] room and we talked about, you know, what [17:25] are your top three risks? What was most [17:27] surprising is that almost to a tea, [17:30] moving too slow [17:32] was one of their top three risks, right? [17:34] So where's that coming from? The CEO, [17:38] he knows competition's coming, pressure [17:40] from the market, right? So one of the [17:43] top risks is moving too slow, right? And [17:46] so this no kind of approach is really [17:49] not, you know, we've got to be able to [17:51] manage through explanability and bias. [17:55] very possible to manage through this. [17:58] If you're a big company, you're going to [17:59] get sued, right? It's just it's the [18:01] nature of the beast. Um, but can you [18:04] defend it, right? Can you manage through [18:05] that? And you know what we see with most [18:09] case law when it comes down to things [18:11] like negligence in a in a space like [18:13] this is [18:16] not that you made the right decision, [18:19] but were you in a position to make an [18:21] informed decision. Right? That's where [18:23] the negligence is. [18:26] And so that's where the programmatic [18:27] piece comes in. And so when we think [18:29] about responsible AI, it's really about [18:32] the practice of like we talked about [18:34] explanability. Your data, your [18:36] intention, your process, your output, [18:38] your testing, your monitoring, [18:40] right? All of those have different [18:42] stakeholders, different owners. [18:46] You know, model risk management. I think [18:47] one of the last speakers was talking the [18:50] panel was talking about that. It's a [18:51] phenomenal space, right? In the banking [18:53] world, you have model risk management [18:55] folks. There's guidance out there for [18:56] model risk management. There's a lot of [18:58] questions of does it apply or does it [19:00] not apply? [19:02] But who's handling like the the model [19:04] piece of model risk management? Are they [19:05] talking to the data piece? Is it the [19:07] same person? Who's doing the monitoring [19:08] and testing? The customer care folks [19:10] that have a chatbot, right? There's this [19:12] interconnection. And so responsible AI [19:14] is really starting to take on the unique [19:15] aspects and making sure frameworks are [19:18] aligned and new practices developed as [19:20] appropriate. [19:22] The riskbased strategy and this is a lot [19:24] and execution. This is where [19:26] you know obviously everyone has a [19:28] strategy but the riskbased strategy and [19:31] execution is really about now that you [19:33] have a risk-informed system [19:35] how do you budget prioritize execute to [19:38] get to balance with the stakeholders [19:42] right it is not the risk informed system [19:44] is not designed to build strategy and [19:46] get it's about building the system to [19:48] execute to and so this is where [19:50] budgeting process right your priorities [19:53] your use cases the ROI I comes in [19:57] um risk escalation and disclosure um you [20:01] know on riskbased strategy just real [20:02] quick um you know a good thing to kind [20:05] of get to a point like bring like some [20:07] folks don't have great relationship with [20:09] auditors I love my auditors and at one [20:11] point we went for the auditors and they [20:13] said look audit us on best practice and [20:15] standards but also audit us to the [20:18] tolerance that we developed because [20:20] that's what was going to the board [20:21] because you're starting to see [20:22] tolerances change a little bit regular [20:24] Regy tolerance. Right now, businesses [20:26] are are looking to, you know, they they [20:28] have a much more open mindset to taking [20:30] on regulatory risks. Regulation is a [20:33] risk. You don't have to be you don't [20:35] have to be compliant. It may hurt, but [20:37] you don't have to be compliant. You can [20:38] certainly try to scope it and manage [20:40] your risk to it. [20:42] And the last is risk escalation and [20:44] disclosure. And this is an interesting, [20:47] you know, pillar here. [20:50] What I hear a lot often when we show [20:52] this and we talk to the folks and say, [20:54] "Oh, we have an incident response plan." [20:56] An incident can cause escalation [20:59] oftentimes does, right? But a risk [21:01] falling outside of our tolerance should [21:03] also cause an escalation depending on [21:06] the severity and to whom and that needs [21:08] to be programmatic, [21:11] right? Can't be relationship based. And [21:14] so it's not about incident response [21:18] again could cause it. Disclosure, right? [21:20] Disclosure, if you're a public company, [21:23] it's material. And the idea about [21:25] materiality, if you're is not just is it [21:28] material, but when did you define it as [21:30] material? What was the criteria? What [21:32] were your steps that you went through to [21:35] identify a material incident? Like what [21:37] what is that criteria? So, it's the [21:39] process of getting to materiality. [21:40] Again, going back to the programmatic [21:43] approach, disclosure to a regulator if [21:46] it meets a a certain threshold. [21:48] disclosure to the public because you [21:50] want to maintain trust, [21:53] right? Important things. [21:57] So, [21:59] think about it. I I just wanted to leave [22:01] you with the programmatic piece. We all [22:04] have process, right? We if you're doing [22:06] risk man, we've got process and we've [22:08] got are the ex is this programmatic? Is [22:10] your work programmatic? Can the program [22:13] itself stand alone? Can that program be [22:16] defended? [22:19] Right? And if you can't defend the [22:21] program, I'm not talking process and [22:23] policies. [22:25] If you can't defend the program, if you [22:28] are a public company, if you are a [22:29] significant enough company, your [22:31] executives, your owner, your board have [22:33] liability, [22:35] right? Bring this to them. Reshape. This [22:37] is the reimagining piece that we hear a [22:39] lot about. [22:41] Reshape all that. [22:43] Um, do we have one question? We're out [22:46] of time. We're out of time. [22:47] >> Okay. Thank you very much. Hope you all [22:49] enjoyed that. Sorry. Thanks.