Truist Mandates Automated AI Guardrails to Curb Model Hallucinations

Scaling Responsible GenAI: Embedding Guardrails for Risk, Compliance, and Business Alignment — Ai4

Ai4

Because every large language model hallucinates and lacks explainability, banks must enforce automated zero-trust guardrails as enterprise defaults rather than relying on manual reviews.

AI-assisted summary of Scaling Responsible GenAI: Embedding Guardrails for Risk, Compliance, and Business Alignment — original source. Check the original for full context.

Source published: 2025-08-24

Detailed notes

## Background Published by Ai4 on YouTube on August 24, 2025, this presentation features Chandra Kapireddy, Firmwide Head of Gen AI, ML, and Analytics at Truist Bank. According to the video description, the session explores how financial institutions can operationalize responsible GenAI by embedding risk, governance, compliance, legal, and privacy frameworks into development from the outset to enable auditable and trustworthy innovation.

## TL;DR Chandra Kapireddy outlines a practical blueprint for scaling generative AI safely across large, highly regulated enterprises. Emphasizing that all large language models hallucinate and lack inherent explainability, he uses the analogy of a high-speed supercar without reliable brakes to illustrate why deterministic guardrails must be embedded within agentic orchestration layers. The presentation breaks down the organizational tensions between business units seeking personalization and risk teams protecting sensitive data like PII. To scale risk management across thousands of use cases, Kapireddy advocates for programmatic enforcement engines that deploy a synthesized, highly conservative policy baseline by default, allowing access only through formal, auditable overrides.

## Atmosphere & style Delivered with pragmatic urgency and seasoned enterprise authority, the talk feels like a focused, fast-paced executive masterclass. Kapireddy cuts straight to the core architectural and governance challenges, grounding complex regulatory principles in relatable analogies like high-speed cars and travel booking. The delivery is direct, analytical, and structured, transforming what is often abstract compliance jargon into an actionable, engineer-ready blueprint for high-stakes deployment.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) Scaling responsible AI in the enterprise rests on three fundamental pillars: ethics, transparency, and accountability, which are made more urgent by generative AI adoption. - [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) The NIST AI Risk Management Framework (AI RMF) is an essential operating guide that enterprise practitioners should thoroughly study and continuously revisit. - [01:35](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=95s) Without dependable guardrails, high-performing AI models resemble a 330 mph supercar with failing brakes—too hazardous and unpredictable for enterprise deployment. - [04:00](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=240s) Because high-dimensional LLMs inherently lack transparent explainability, regulated enterprises remain cautious about deploying them autonomously for mission-critical decisions. - [05:17](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=317s) Since all large language models inevitably hallucinate, organizations must embed multi-layered guardrails directly into the software architecture rather than assuming model perfection. - [06:30](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=390s) In agentic AI architectures, deterministic orchestrator code must govern dynamic tool selection, memory, and API integrations to preserve application auditability and reliability. - [08:03](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=483s) Enterprise AI governance is an organizational discipline that requires balancing conflicting risk tolerances across Cybersecurity, Legal, Privacy, and Lines of Business. - [08:45](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=525s) Managing Personally Identifiable Information (PII) exposes a core tension between commercial growth objectives and strict legal, privacy, and security constraints. - [10:14](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=614s) Manual committee reviews become an unsustainable bottleneck as organizations scale AI from dozens of experiments to thousands of enterprise use cases. - [10:50](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=650s) Effective enterprise scaling relies on synthesizing second-line risk policies into an automated, highly conservative default guardrail, paired with auditable, documented exception overrides.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) Grounding Responsible AI: Ethics, Transparency, and Accountability

Addressing the room with a quick nod to the tight twenty-minute time slot, the presentation bypasses extensive introductory bios to dive straight into the heart of the matter. While the topic of enterprise AI governance could easily fill a full day, a week, or even a month of intensive discussion, scaling it successfully boils down to three foundational questions: what responsible AI actually is, how to operationalize it, and how to scale it across an entire organization. In an era where anyone can pull out a smartphone, open a generative AI assistant, and receive textbook definitions instantly, the core tenets of responsible AI remain anchored to familiar principles: ethics, transparency, and accountability. These principles have guided data and analytics for years, but the rapid emergence and power of generative AI make adherence to them far more urgent and non-negotiable.

[01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) The NIST AI RMF and the High-Speed Car Analogy

The conversation turns to the National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF), with a strong recommendation for practitioners: those who have never read it should dedicate at least a week to studying it thoroughly, and those who already have should revisit and reread it regularly. Today's AI landscape is flooded with large language models spanning chat interfaces, advanced reasoning engines, and multimodal systems that process far more than just text. To understand why frameworks like NIST matter, a practical automotive analogy illustrates the inherent risk:

Imagine being offered two vehicles, each priced identically at $50,000. One vehicle is a stunning, sleek supercar capable of driving at 330 miles per hour. The second vehicle looks ordinary and tops out at only 100 miles per hour. While most people instinctively choose the 330 mph supercar, the catch lies in the unrevealed detail: its braking system fails intermittently and cannot be trusted to stop reliably. Faced with that reality, no one would choose to drive it.

```mermaid flowchart TD accTitle: The High-Speed AI Analogy subgraph FastCar["High-Capability AI System"] A["330 MPH Output Capacity / Advanced LLMs"] --> B["Unreliable / Inconsistent Braking Mechanism"] B --> C["Catastrophic Failure Risk"] end subgraph GovernedAI["Enterprise-Ready AI System"] D["High-Performance Reasoning & Tools"] --> E["Deterministic Guardrails & Policy Checks"] E --> F["Safe, Auditable Business Execution"] end ```

This dynamic perfectly mirrors generative AI today. Regardless of how powerful, fast, or impressive a model appears, without robust, proven "braking systems" (guardrails) in place, enterprise deployment remains too hazardous. Users frequently experience this firsthand when planning travel using conversational AI tools; nobody blindly buys the airline tickets or books the hotels without double-checking the reservations first, because the models behave unpredictably based on their training data.

Within the NIST framework, dimensions like security, privacy, fairness, transparency, explainability, and interpretability are essential operating criteria rather than mere buzzwords. A simple mathematical equation such as $y = 2x$ is inherently interpretable because the linear relationship is obvious and self-explanatory. However, as models shift toward complex, high-dimensional large language models, explainability becomes exceedingly difficult. If an organization finds itself struggling to reverse-engineer why a black-box model made a specific determination after the fact, it is already in a compromised position. Because large language models lack transparent explainability, major enterprises—especially within financial services—remain hesitant to deploy them autonomously for mission-critical decisions.

[05:17](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=317s) Agentic Architectures and Guardrail Mechanics

In operationalizing responsible AI, teams must accept George Box's famous statistical truism: "All models are wrong, some are useful." Furthermore, practitioners must acknowledge a fundamental reality of generative AI: every single large language model hallucinates. Because no hallucination-free model exists on the market, the only way to safely leverage the power of LLMs while insulating the business from their inherent failure modes is to embed robust, multi-layered guardrails directly into the software architecture.

```mermaid flowchart LR accTitle: Agentic AI Orchestration and Guardrail Flow subgraph UserInterface["Client Tier"] UI["End-User Application"] end

subgraph AgenticOrchestration["Orchestrator Tier"] Agent["Orchestrator AI Agent"] Guard["Embedded Guardrails & Policies"] end

subgraph ExecutionLayer["Execution & Model Tier"] LLM["Reasoning LLM"] Tools["APIs, Tools & Memory"] end

Modern architectures frequently deploy agentic workflows. A clear distinction exists between deterministic agents and AI agents: - **Deterministic Agents:** Follow fixed, programmatic logic (such as strict `if-then-else` statements) without probabilistic AI components. - **AI Agents / Agentic AI:** Incorporate machine learning or large language models to reason over instructions, evaluate context, and make autonomous decisions.

When a user prompt enters the system, the orchestrator agent selects tools, calls external APIs, queries internal resources, and manages state memory before routing context to the underlying reasoning model. Because the AI agent determines which actions to take dynamically, organizations operating in regulated sectors must write deterministic, highly reliable code in the orchestrator layer. Enforcing guardrails at these tool-selection and response boundaries is the minimum requirement for maintaining application reliability and auditability.

[08:03](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=483s) Cross-Functional Stakeholder Alignment and the PII Dilemma

Building enterprise guardrails is not merely a technical challenge; it is a cross-functional governance discipline. Drawing on leadership experience across major institutions including Amazon, JPMorgan Chase, and Truist Bank (the seventh largest bank in the United States), enterprise AI risk management demands close alignment among distinct stakeholder groups, each operating with conflicting risk tolerances.

This structural tension is most evident when managing Personally Identifiable Information (PII). From a pure revenue perspective, business units want unrestricted access to customer identities so they can tailor recommendations and sell more products. Conversely, Privacy, Legal, and Cybersecurity stakeholders enforce strict constraints to safeguard the broader institution. Responsible scaling requires establishing a governance committee where these contrasting perspectives are balanced systematically.

[10:14](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=614s) Scaling Enterprise Guardrails: Conservative Defaults and Automated Overrides

As generative AI adoption accelerates, organizations are shifting from managing dozens of experiments to overseeing hundreds or thousands of active use cases embedded across nearly every internal business process. Scaling risk management across thousands of implementations cannot be achieved through manual, ad-hoc committee reviews; doing so creates an unsustainable operational bottleneck.

```mermaid flowchart TD accTitle: Enterprise Scaling and Override Framework subgraph PolicyDefinition["1. Baseline Policy Definition"] L1["AI Risk Guidelines"] L2["Legal & Compliance Rules"] L3["Privacy Standards"] L4["Cybersecurity Controls"] end

subgraph AutomatedEnforcement["2. Enterprise Default Configuration"] DefaultPolicy["Aggregated Most-Conservative Guardrail Baseline\n(Automated Zero-Trust Default)"] L1 --> DefaultPolicy L2 --> DefaultPolicy L3 --> DefaultPolicy L4 --> DefaultPolicy end

subgraph ApplicationExecution["3. Use Case Lifecycle"] UC["New AI Use Case Deployment"] --> DefaultPolicy DefaultPolicy --> Decision{"Requires Exception / PII Access?"} Decision -- "No" --> StandardDeploy["Standard Governed Production"] Decision -- "Yes" --> LoggedOverride["Documented, Audited & Logged Override"] LoggedOverride --> StandardDeploy end ```

The blueprint for scaling guardrails efficiently across the entire enterprise involves a structured, automated workflow:

- **Define Second-Line Guardrails Once:** Second-line risk functions—including AI Risk, Legal, Compliance, Privacy, and Cybersecurity—each establish their specific, non-negotiable guardrail policies at the enterprise level as an initial baseline. - **Synthesize a Conservative Enterprise Default:** The platform automatically combines these individual inputs into a single, highly conservative aggregate guardrail. This safe baseline serves as the zero-trust default posture automatically applied to every new AI use case. - **Automate Policy Enforcement:** Manual reviews are replaced with programmatic enforcement engines that inspect prompts, agent tool selections, and model responses in real time. - **Implement Audited Exception Overrides:** When a specific business scenario legitimately requires lower restrictions (such as unlocking access to sensitive customer identifiers for a specialized workflow), the platform allows an exception. This override must be formally documented, approved, and logged in an auditable trail before moving into production.

Establishing this automated, default-conservative architecture ensures that financial institutions and large enterprises can innovate rapidly across thousands of use cases while maintaining rigorous risk controls and regulatory alignment.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) scaling it successfully boils down to three foundational questions: what responsible AI actually is, how to operationalize it, and how to scale it across an entire organization. — Chandra Kapireddy

> [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) Regardless of how powerful, fast, or impressive a model appears, without robust, proven 'braking systems' (guardrails) in place, enterprise deployment remains too hazardous. — Chandra Kapireddy

> [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) If an organization finds itself struggling to reverse-engineer why a black-box model made a specific determination after the fact, it is already in a compromised position. — Chandra Kapireddy

> [05:17](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=317s) All models are wrong, some are useful. — George Box

> [05:17](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=317s) every single large language model hallucinates. — Chandra Kapireddy

> [05:17](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=317s) Because no hallucination-free model exists on the market, the only way to safely leverage the power of LLMs while insulating the business from their inherent failure modes is to embed robust, multi-layered guardrails directly into the software architecture. — Chandra Kapireddy

> [05:17](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=317s) Enforcing guardrails at these tool-selection and response boundaries is the minimum requirement for maintaining application reliability and auditability. — Chandra Kapireddy

> [08:03](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=483s) Building enterprise guardrails is not merely a technical challenge; it is a cross-functional governance discipline. — Chandra Kapireddy

> [10:14](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=614s) Scaling risk management across thousands of implementations cannot be achieved through manual, ad-hoc committee reviews; doing so creates an unsustainable operational bottleneck. — Chandra Kapireddy

## Mentioned Chandra Kapireddy, Truist Bank, Ai4, National Institute of Standards and Technology (NIST), NIST AI Risk Management Framework (AI RMF), George Box, Amazon, JPMorgan Chase, GDPR, CCPA

## Video description Chandra Kapireddy, Firmwide Head of Gen AI, ML and Analytics, Truist Bank

Abstracts

As financial institutions accelerate GenAI adoption, scaling responsibly requires more than just innovation—it demands robust guardrails. This session explores how to operationalize responsible GenAI by embedding risk, governance, compliance, legal, and privacy frameworks from the outset. It emphasizes cross-functional collaboration with Lines of Business to ensure AI initiatives are aligned, auditable, and trustworthy—enabling institutions to innovate confidently while meeting regulatory expectations and safeguarding stakeholder trust.

[Source](https://www.youtube.com/watch?v=-rmhG0ShrFE)

中文简报

Truist强制部署自动化AI护栏以遏制模型幻觉由于所有大语言模型都存在幻觉且缺乏可解释性，银行必须将自动化零信任护栏作为企业默认机制强制执行，而非依赖人工审查。 Chandra Kapireddy 概述了一份实用蓝图，旨在大型高度受监管的企业中安全扩展生成式 AI。他强调所有大语言模型都存在幻觉且缺乏内在的可解释性，并以缺乏可靠制动系统的高速超级跑车作比喻，阐释了为何必须在智能体编排层中嵌入确定性护栏。该演讲剖析了追求个性化的业务部门与保护 PII 等敏感数据的风控团队之间的组织分歧。为了在数千个用例中扩展风险管理，Kapireddy 提倡使用程序化执行引擎，默认部署综合且高度保守的策略基线，且仅允许通过正式、可审计的例外覆盖来进行访问。负责任的生成式 AI 落地依托于伦理、透明度与问责制三大支柱，美国国家标准与技术研究院（NIST）的 AI 风险管理框架（AI RMF）是企业从业者必须深入研读的关键指南。 (0:04) 缺少可靠护栏的高性能模型就像刹车失灵的超跑一样危险，由于高维大语言模型缺乏透明的可解释性，受监管企业对其自主执行关键任务决策仍持谨慎态度。 (1:20) 大语言模型普遍存在幻觉问题，企业必须直接在软件架构中植入多层护栏，并在 Agent 编排层通过确定性代码管控工具选择与 API 调用以确保可审计性。 (5:17) 企业 AI 治理需要平衡网络安全、法务、隐私与业务部门在客户个人身份信息（PII）等方面的风险容忍度冲突，化解商业增长与合规约束的张力。 (8:03) 规模化扩展护栏需整合各风控职能建立最保守的自动化默认基线，在避免人工审查瓶颈的同时，对合法业务场景实施严格记录和审计的例外覆写机制。 (10:14)

Reading summary

Scaling generative AI in regulated enterprises requires treating risk management as an architectural and governance discipline rather than an ad-hoc review process. Because large language models inherently hallucinate and lack explainability, organizations must embed deterministic guardrails and conservative policy baselines directly into agentic workflows. The Braking Mechanism: Capability versus Explainability (1:20) The National Institute of Standards and Technology (NIST) AI Risk Management Framework illustrates that raw model capability without dependable control mechanisms is dangerous for enterprise adoption. This dynamic is comparable to choosing between two $50,000 cars: a 330-mile-per-hour supercar with an intermittently failing brake system and an ordinary vehicle that tops out at 100 miles per hour. Despite the supercar's superior speed, no operator would rely on a machine that cannot stop predictably. Similarly, conversational models remain risky because users cannot trust their outputs without independent verification. Under the NIST framework, dimensions such as security, privacy, fairness, transparency, explainability, and interpretability serve as baseline operating criteria. While simple mathematical relationships like y = 2x are straightforward and interpretable, high-dimensional large language models operate as black boxes. When enterprises cannot explain why a model arrived at a specific decision after the fact, their operational integrity is compromised. For this reason, institutions in heavily regulated sectors, such as financial services, remain cautious about deploying models autonomously for mission-critical tasks. Architectural Guardrails in Agentic Systems (5:17) Operationalizing responsible AI begins with accepting George Box's statistical aphorism that 'all models are wrong, some are useful,' alongside the reality that every large language model on the market hallucinates. Because hallucination cannot be completely eliminated at the model level, systems must insulate business operations by embedding multi-layered guardrails into the surrounding software architecture. Modern enterprise deployments increasingly use agentic workflows, requiring a clear distinction between deterministic agents and AI agents. Deterministic agents rely entirely on fixed, programmatic logic (such as if-then-else conditions), whereas AI agents use machine learning or large language models to reason dynamically and execute decisions. When an orchestrator agent selects tools, calls APIs, and manages context memory, organizations in regulated sectors must implement deterministic, highly reliable code at these tool-selection and response boundaries to guarantee system auditability and reliability. Cross-Functional Risk Alignment and the PII Dilemma (8:03) Establishing guardrails requires managing structural friction among enterprise stakeholders with conflicting mandates and risk appetites. Leadership experience across institutions such as Amazon, JPMorgan Chase, and Truist Bank demonstrates that technical implementations routinely collide with competing organizational priorities. This tension is most pronounced when handling Personally Identifiable Information (PII). Business units prioritize commercial growth, personalized customer experiences, and cross-selling, which drives an appetite for unrestricted PII access. In contrast, Cybersecurity demands low risk tolerance to prevent data exfiltration, Privacy enforces strict statutory mandates like GDPR and CCPA, and Legal maintains an extremely low risk tolerance to avoid liability. Successfully scaling AI depends on establishing governance bodies that systematically balance these disparate requirements rather than allowing commercial pressures to bypass control functions. Automating Scale via Conservative Defaults and Overrides (10:14) As organizations transition from managing isolated pilots to running hundreds or thousands of generative AI use cases across business units, manual committee approvals become an operational bottleneck. Scaling compliance requires automating the enforcement of second-line risk controls. The framework for operational scale relies on establishing a single baseline: second-line risk functions (AI Risk, Legal, Compliance, Privacy, and Cybersecurity) define their core policies once. The enterprise platform then synthesizes these inputs into an aggregated, highly conservative default guardrail applied automatically to every new deployment under a zero-trust model. Real-time programmatic engines inspect prompts, tool selections, and outputs against this baseline. When a specific business scenario legitimately requires broader access—such as utilizing PII for a specialized workflow—the platform permits an exception only through an approved, formally documented, and auditable override trail. Because generative AI models inherently hallucinate and lack transparent explainability, sustainable enterprise adoption depends on shifting governance from manual gatekeeping to automated, code-level guardrails. Establishing a conservative, aggregated zero-trust baseline allows large enterprises to expand to thousands of use cases while maintaining an auditable mechanism for necessary exceptions.

精炼阅读摘要

演讲者指出，生成式人工智能因普遍存在幻觉和缺乏可解释性而无法直接独立运行，企业规模化应用的关键在于将治理机制工程化。通过在系统架构中嵌入确定性护栏，并将多部门风控规则合成为自动化的最保守默认基线，企业才能在防范灾难性风险的同时支撑海量业务落地。高车速与制动隐喻：不可解释性限制自主商用 (1:20) 演讲者以价值均为5万美元的汽车作比：一辆是最高时速达330英里但制动系统偶发失灵的超级跑车，另一辆则是时速上限仅100英里但制动稳定的普通车辆；面对制动不可靠的致命风险，绝大多数人不会选择前者。当前的生成式人工智能正处于相同境地，大语言模型展现出的高算力与生成速度若缺乏可靠的制动机制（即护栏），企业盲目部署将承担不可承受的系统崩溃与合规风险。现实中用户使用AI规划旅行时，往往不会在未经人工核对的情况下直接确认机票或酒店，这反映出市场对基于训练数据生成的概率性结果缺乏天然信任。在NIST《人工智能风险管理框架》（AI RMF）所界定的安全、隐私、公平、透明及可解释性等维度中，可解释性是大模型落地的核心瓶颈。简单的数学公式如 y = 2x 具有直观的线性因果关系，但高维度的黑箱模型在做出复杂决策后极难进行事后逆向推导。演讲者强调，如果在系统输出错误后才仓促分析其内部机理，企业便已陷入被动局面；正因无法提供完全透明的决策解释路径，以金融服务业为代表的强监管行业至今仍对在关键业务上实现大模型的全自主部署持高度审慎态度。正视幻觉现实：在代理编排层植入确定性护栏 (5:17) 推进负责任人工智能的工程化落地，必须承认统计学家乔治·博克斯（George Box）的名言——“所有模型都是错的，但有些是有用的”。演讲者指出，当前市场上不存在任何完全免于幻觉的大语言模型，企业绝不能寄希望于基础模型自行消除幻觉，而必须在软件架构层面构建多层防护网。在系统设计中，必须严格区分遵循固定程序逻辑（如 if-then-else 规则）的确定性代理，以及利用机器学习与LLM进行上下文推理、拥有自主决策空间的AI代理。现代智能体工作流通常由编排代理负责接收前端提示词、选择工具、调用外部API并维护状态记忆，最终将清洗后的上下文输入底层推理模型。由于AI代理在工具调用与路径规划上具备动态特征，强监管企业必须在编排层编写高度可靠的确定性代码。在智能体发起外部交互和输出最终结果的关键边界设立策略校验护栏，是确保应用行为可控、具备完整审计追踪能力的底线技术要求。跨职能利益博弈：个人身份信息引发的权衡困境 (8:03) 结合在亚马逊、摩根大通以及全美第七大银行楚斯特银行（Truist Bank）的管理实践，演讲者提出，护栏建设并非纯粹的技术编码，而是跨部门权衡治理的产物。企业内部各利益相关方天然存在风险承受度的对立：网络安全部门着眼于基础设施防御与数据防泄漏，风险偏好极低；法律合规部门聚焦法定义务与合同责任防范，风险容忍度接近于零；隐私保护团队严格执行GDPR、CCPA等法规限制，对个人数据暴露极度敏感；而业务部门则受增长、推荐转化与产品交叉销售指标驱动，倾向于采用更激进的数据策略。这种结构性矛盾在涉及个人身份信息（PII）时表现得尤为剧烈。从商业变现角度，业务端希望最大程度调取客户身份标识以实现超个性化推荐；但二道防线职能部门出于机构合规与声誉保护，必须施加严格访问限制。负责任的人工智能规模化框架，首先需要建立常态化的跨职能治理委员会，以制度化机制协调上述截然相反的业务诉求。规避审查瓶颈：聚合最保守默认策略与例外重载 (10:14) 当企业人工智能应用从几十个离散实验演进为渗透至各业务线的成百上千个生产用例时，传统依赖人工逐项审批的委员会模式将成为严重的交付瓶颈。为了在防范风险的同时保障业务迭代效率，企业需要建立一套自动化的策略分发机制。其核心路径包括：由AI风险、法律、合规、隐私和网络安全等第二道防线部门在全行层面统一制定各自的不可逾越规则，系统随后将各方输入综合计算，生成一套聚合了最高安全约束的“最保守企业默认基线”。该基线作为零信任默认配置，自动预置并绑定到每一个新建的人工智能用例中，替代人工完成对用户提示词、智能体工具调用及输出内容的毫秒级程序化审查。若特定高价值业务场景因个性化需要必须调取敏感字段，系统提供标准化的例外重载流程。该重载申请必须经过严格的形式审查与特批授权，并在后台形成不可篡改的日志审计线索后方可上线，从而在保障底线风控的前提下兼顾特殊业务需求。演讲者主张将复杂的合规原则转化为系统工程中的确定性默认基线与代码护栏，以破解大模型在企业级部署中的风险瓶颈。不过，演讲并未深入探讨当底层法规频繁演进或多司法管辖区规则出现直接冲突时，自动化策略基线应如何进行低成本的动态重构与版本管理。

Detailed notes — zh

## Background 该视频由 Ai4 频道于 2025 年 8 月 24 日发布在 YouTube 平台上，主讲人为 Truist Bank 企业级生成式 AI、机器学习与分析负责人 Chandra Kapireddy。视频内容围绕金融机构在加速采用生成式 AI 时，如何从一开始就嵌入风险、治理、合规、法律和隐私框架，通过跨部门协作构建强大的安全护栏（Guardrails）以实现负责任的大规模落地。 ## TL;DR 演讲者以“跑车与刹车系统”为生动比喻，指出功能强大的大语言模型若缺乏护栏就无法在企业级场景中安全运行，并强烈推荐参考 NIST AI 风险管理框架。针对大模型普遍存在的幻觉问题，企业应在编排智能体（Orchestrator Agent）层自动化嵌入护栏以校验决策与调用。面对网络安全、法务、隐私与业务部门之间悬殊的风险承受度差异，演讲者提出应以最保守的标准作为全企业默认基线，配合自动化执行与例外留痕审批机制，从而安全高效地支撑数百上千个 AI 应用场景的规模化扩展。 ## Atmosphere & style 整个演讲节奏紧凑高效、开门见山，演讲者因现场时间受限迅速切入核心干货，展现出深耕银行业与技术一线的务实底色。通过生动贴切的汽车比喻和现场互动，原本晦涩的合规治理与智能体架构变得清晰易懂、极具说服力。整体氛围兼具严谨的工程思维与前瞻的战略视野，既传递出金融机构对安全风险的审慎敬畏，又给出了清晰落地的规模化破局之道。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) 负责任人工智能的三大核心支柱是伦理（Ethics）、透明度（Transparency）和问责制（Accountability）。 - [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) 强烈建议研读 NIST 的《AI 风险管理框架》，关注安全性、隐私性、可解释性、公平性与透明度。 - [01:40](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=100s) 将护栏比作汽车的“刹车系统”：即使模型功能强大，缺乏有效的护栏也无法在企业级场景中安全运行。 - [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 所有 LLM 都会产生幻觉，由于缺乏直观的可解释性，企业必须依靠自动化护栏来确保应用可靠性。 - [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 在智能体架构（Agentic AI）中，需要在编排智能体（Orchestrator Agent）与模型/工具交互的阶段设置护栏。 - [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 确定性智能体基于固定代码逻辑运行，而 AI 智能体具备由 LLM 驱动的自主决策能力。 - [08:06](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=486s) 不同部门对风险的承受度存在显著差异，例如法律和隐私部门风险承受度极低，而业务部门则极高。 - [08:06](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=486s) 推进负责任 AI 需要建立跨部门委员会，包含网络安全、法律、合规、隐私及业务线等多方利益相关者。 - [09:56](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=596s) 随着企业内部 AI 应用场景暴增至数百上千个，依靠人工进行护栏审查难以实现规模化扩展。 - [09:56](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=596s) 实现企业级规模化扩展的唯一可行路径是：自动应用各部门最保守的护栏组合作为默认姿态，并通过日志留痕管理例外审批。 ## Detailed walkthrough 以下是基于视频转录内容整理的详细记录与分析： ### [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) 演讲背景与负责任人工智能的核心原则演讲者 Chandra Kapireddy（Truist Bank 企业级生成式 AI、机器学习与分析负责人）首先提到由于现场时间限制（仅剩约 20 分钟），决定跳过个人介绍并压缩部分内容，直接进入核心议题。他强调“负责任人工智能”（Responsible AI）是一个极其庞大的课题，通常需要一整天甚至数月的时间来深度探讨。针对“什么是负责任人工智能”这一问题，演讲者指出，尽管大家可以通过手机上的 GPT 工具轻松搜索到定义，但其本质始终离不开三大核心支柱：**伦理（Ethics）**、**透明度（Transparency）** 和 **问责制（Accountability）**。虽然这些概念在传统 AI 中已被探讨多年，但在生成式 AI（GenAI）时代，其重要性被提升到了前所未有的高度。 - **核心三要素**：伦理（Ethics）、透明度（Transparency）、问责制（Accountability）。 --- ### [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) NIST 风险管理框架与“刹车系统”比喻演讲者强烈建议参会者深入研读美国国家标准与技术研究院（NIST）发布的《AI 风险管理框架》（NIST AI Risk Management Framework）。他建议没看过的人至少花一周时间研读，而看过的人则应反复重读。为了形象解释这一框架的重要性，演讲者提供了一个生动的汽车比喻：假设面前有两辆售价均为 50,000 美元的汽车： 1. **第一辆车**：外观酷炫，最高时速可达 300 英里/小时； 2. **第二辆车**：外观普通，最高时速仅 100 英里/小时。绝大多数人在未获知全部信息时会倾向选择第一辆。然而，如果揭示关键的“刹车系统”信息——那辆时速 300 英里的酷炫跑车，其刹车系统并非总是有效——大家便会立刻拒绝。演讲者指出，**这正是生成式 AI 当前的现状**：市场上充斥着大量功能强大的模型，但如果缺乏正确实施的“护栏”（Guardrails/刹车系统），模型再强大也无法在企业级场景中安全运行。例如，人们用 ChatGPT 规划行程后，依然需要打电话给酒店确认，因为这些模型会根据训练数据产生不可预测的行为。演讲者进一步强调了 NIST 框架中的关键要素：安全性、隐私性、可解释性（Explainability）、公平性与透明度。他特别指出，“可解释”与“可解释性”在大型语言模型（LLM）中是非常棘手的挑战： - **简单模型（如线性回归方程式 $y = 2x$）**：本身即具备可解释性，无需额理解成本； - **复杂 LLM**：缺乏直观的可解释性，导致透明度极低。这也解释了为什么金融服务机构目前不敢轻易将纯 LLM 直接用于关键的业务决策。 ```mermaid flowchart TD accTitle: NIST AI 风险管理与安全护栏架构 subgraph Core ["NIST AI 风险管理核心要素"] A["安全性 & 隐私性"] B["公平性 & 透明度"] C["可解释性 (Explainability)"] End subgraph Analogy ["汽车与 AI 护栏类比"] D["高性能 LLM/模型 (高时速跑车)"] E["安全护栏 Guardrails (刹车系统)"] F["企业级安全落地"] End Core --> D D --> E E --> F ``` --- ### [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 护栏机制与智能体（Agentic AI）架构在讨论如何具体落地负责任 AI 时，演讲者引用了名言：“**所有模型都是错的，但有些是有用的**（All models are wrong, but some are useful）”。他强调，目前市场上没有任何一个大语言模型能做到完全不发生“幻觉”（Hallucination）。因此，利用其优势并规避其局限性的唯一途径就是**构建护栏（Guardrails）**。演讲者展示了一张其精心设计的架构简图，阐述了在编排智能体（Orchestrator Agent）中实施护栏的具体机制： 1. **终端用户应用（End-user Application）** 发出 Prompt（提示词）； 2. 提示词进入**编排智能体（Orchestrator Agent）**。演讲者澄清了概念： - **确定性智能体（Deterministic Agent）**：基于固定的代码规则（如 if-then-else），严格按照既定逻辑执行任务； - **AI 智能体（AI Agent / Agentic AI）**：包含 AI/LLM 组件，具备自主决策能力； 3. 编排智能体（在推理模型的驱动下）有权选择调用工具、资源、API 或访问内存（Memory）； 4. 在此阶段必须设置护栏，以校验智能体是否做出了正确的决策，并将安全过滤后的信息传回 LLM 或用户。对于处于高合规监管行业的企业而言，在编排智能体中编写的护栏代码是提高 AI 应用可靠性的核心命脉。 ```mermaid flowchart LR accTitle: AI 编排智能体与护栏交互流程 subgraph Client ["客户端"] A["终端应用 (End-user App)"] End subgraph Agentic ["编排层"] B["编排智能体 (Orchestrator Agent)"] C{"护栏校验 (Guardrails)"} End subgraph Execution ["执行与模型"] D["工具 / API / 内存"] E["LLM 推理模型 (Reasoning Model)"] End A -- "Prompt" --> B B --> C C -- "决策校验" --> D D --> E E -- "Response" --> B ``` - **关键结论**：所有 LLM 都会产生幻觉，企业必须依靠编排层的自动化护栏来确保可靠性。 --- ### [08:06](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=486s) 多方利益相关者与风险承受度差异结合其在 Truist Bank（美国第七大银行）、摩根大通（JPMorgan Chase）及亚马逊（Amazon）的丰富从业经历，演讲者分享了大型企业推进 AI 时的跨部门治理视图。他指出，要实施兼顾风险管理的 AI，必须建立由多方利益相关者构成的委员会。以**敏感个人信息（PII）保护**这一护栏为例，不同部门对风险的承受度（Risk Tolerance）存在显著冲突： - **网络安全部门（Cybersecurity）**：风险承受度低（Low）； - **法律部门（Legal）**：风险承受度极低（Extremely Low）； - **隐私部门（Privacy）**：风险承受度低（Low）； - **业务部门（Business Line）**：风险承受度极高（Very High）——因为业务部门希望尽可能详细地了解客户信息，以便交叉销售更多产品。 | 部门 / 利益相关者 | 关注焦点 | PII 风险承受度 | |:--- |:--- |:--- | | **网络安全 (Cybersecurity)** | 系统安全、防泄露 | 低 (Low) | | **法律 (Legal)** | 法律合规、诉讼风险 | 极低 (Extremely Low) | | **隐私 (Privacy)** | 数据合规、客户隐私保护 | 低 (Low) | | **业务部门 (Business Line)** | 客户画像、促成销售 | 极高 (Very High) | 演讲者强调，正是因为各部门诉求不同，才需要建立统一的风险评估机制，以确保整个银行实体受到妥善保护。 --- ### [09:56](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=596s) 企业级规模化扩展与自动化护栏策略在探讨如何在企业内部大规模扩展 AI 时，演讲者现场调查了参会者所在企业的 AI 场景数量。他预测，到明年此时，绝大多数企业都将拥有成百上千甚至上千个 AI 应用场景（Use Cases），因为 AI 正被嵌入到企业的每一个业务流程中。针对成百上千个应用场景，如何高效且规范地应用护栏？演讲者提出了**企业级默认保守位置与例外审批机制**： 1. **统一制定基线**：首先让网络安全、法律、合规、隐私及 AI 风险（AI Risk，第二道防线）等各个职能部门，分别定义全企业通用的护栏标准； 2. **组合最保守护栏（Consolidated Guardrail）**：在具体应用场景（如 Use Case 1）上线时，系统会自动应用各部门中最保守（极为严格/非常低风险承受度）的护栏组合作为**默认姿态（Default Position）**； 3. **自动化执行**：全流程必须实现**自动化（Automated）**，如果依赖人工审核，扩展速度将极其缓慢； 4. **例外日志（Exception Override）**：如果某个特定的业务场景确实需要放宽某项护栏，必须经过正式的记录、日志留痕（Logged）与例外审批。演讲者总结道，这是企业在快速推进数百上千个 AI 项目的同时，能够系统化、强力实施安全护栏的唯一可行路径。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) 负责任人工智能是一个极其庞大的课题，通常需要一整天甚至数月的时间来深度探讨。 — Chandra Kapireddy > [00:04](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=4s) 其本质始终离不开三大核心支柱：伦理（Ethics）、透明度（Transparency）和问责制（Accountability）。 — Chandra Kapireddy > [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) 市场上充斥着大量功能强大的模型，但如果缺乏正确实施的“护栏”（Guardrails/刹车系统），模型再强大也无法在企业级场景中安全运行。 — Chandra Kapireddy > [01:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=80s) 简单模型本身即具备可解释性，无需额理解成本；复杂 LLM 缺乏直观的可解释性，导致透明度极低。 — Chandra Kapireddy > [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 所有模型都是错的，但有些是有用的（All models are wrong, but some are useful）。 — Chandra Kapireddy > [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 目前市场上没有任何一个大语言模型能做到完全不发生“幻觉”（Hallucination）。因此，利用其优势并规避其局限性的唯一途径就是构建护栏（Guardrails）。 — Chandra Kapireddy > [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 对于处于高合规监管行业的企业而言，在编排智能体中编写的护栏代码是提高 AI 应用可靠性的核心命脉。 — Chandra Kapireddy > [05:20](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=320s) 所有 LLM 都会产生幻觉，企业必须依靠编排层的自动化护栏来确保可靠性。 — Chandra Kapireddy > [08:06](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=486s) 要实施兼顾风险管理的 AI，必须建立由多方利益相关者构成的委员会。 — Chandra Kapireddy > [08:06](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=486s) 正是因为各部门诉求不同，才需要建立统一的风险评估机制，以确保整个银行实体受到妥善保护。 — Chandra Kapireddy > [09:56](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=596s) 到明年此时，绝大多数企业都将拥有成百上千甚至上千个 AI 应用场景（Use Cases），因为 AI 正被嵌入到企业的每一个业务流程中。 — Chandra Kapireddy > [09:56](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=596s) 全流程必须实现自动化（Automated），如果依赖人工审核，扩展速度将极其缓慢。 — Chandra Kapireddy > [09:56](https://www.youtube.com/watch?v=-rmhG0ShrFE&t=596s) 这是企业在快速推进数百上千个 AI 项目的同时，能够系统化、强力实施安全护栏的唯一可行路径。 — Chandra Kapireddy ## Mentioned Chandra Kapireddy, Truist Bank, 摩根大通 (JPMorgan Chase), 亚马逊 (Amazon), NIST (美国国家标准与技术研究院), 《AI 风险管理框架》(NIST AI Risk Management Framework), ChatGPT, GPT, Ai4 ## Video description Chandra Kapireddy, Firmwide Head of Gen AI, ML and Analytics, Truist Bank Abstracts As financial institutions accelerate GenAI adoption, scaling responsibly requires more than just innovation—it demands robust guardrails. This session explores how to operationalize responsible GenAI by embedding risk, governance, compliance, legal, and privacy frameworks from the outset. It emphasizes cross-functional collaboration with Lines of Business to ensure AI initiatives are aligned, auditable, and trustworthy—enabling institutions to innovate confidently while meeting regulatory expectations and safeguarding stakeholder trust. [Source](https://www.youtube.com/watch?v=-rmhG0ShrFE)

Source quotes

[0:04] Chandra Kapireddy: 负责任人工智能是一个极其庞大的课题，通常需要一整天甚至数月的时间来深度探讨。 [0:04] Chandra Kapireddy: 其本质始终离不开三大核心支柱：伦理（Ethics）、透明度（Transparency）和问责制（Accountability）。 [1:20] Chandra Kapireddy: 市场上充斥着大量功能强大的模型，但如果缺乏正确实施的“护栏”（Guardrails/刹车系统），模型再强大也无法在企业级场景中安全运行。 [1:20] Chandra Kapireddy: 简单模型本身即具备可解释性，无需额理解成本；复杂 LLM 缺乏直观的可解释性，导致透明度极低。 [5:20] Chandra Kapireddy: 所有模型都是错的，但有些是有用的（All models are wrong, but some are useful）。 [5:20] Chandra Kapireddy: 目前市场上没有任何一个大语言模型能做到完全不发生“幻觉”（Hallucination）。因此，利用其优势并规避其局限性的唯一途径就是构建护栏（Guardrails）。 [5:20] Chandra Kapireddy: 对于处于高合规监管行业的企业而言，在编排智能体中编写的护栏代码是提高 AI 应用可靠性的核心命脉。 [5:20] Chandra Kapireddy: 所有 LLM 都会产生幻觉，企业必须依靠编排层的自动化护栏来确保可靠性。 [8:06] Chandra Kapireddy: 要实施兼顾风险管理的 AI，必须建立由多方利益相关者构成的委员会。 [8:06] Chandra Kapireddy: 正是因为各部门诉求不同，才需要建立统一的风险评估机制，以确保整个银行实体受到妥善保护。 [9:56] Chandra Kapireddy: 到明年此时，绝大多数企业都将拥有成百上千甚至上千个 AI 应用场景（Use Cases），因为 AI 正被嵌入到企业的每一个业务流程中。 [9:56] Chandra Kapireddy: 全流程必须实现自动化（Automated），如果依赖人工审核，扩展速度将极其缓慢。 [9:56] Chandra Kapireddy: 这是企业在快速推进数百上千个 AI 项目的同时，能够系统化、强力实施安全护栏的唯一可行路径。

Source quotes

[0:04] Chandra Kapireddy: scaling it successfully boils down to three foundational questions: what responsible AI actually is, how to operationalize it, and how to scale it across an entire organization. [1:20] Chandra Kapireddy: Regardless of how powerful, fast, or impressive a model appears, without robust, proven 'braking systems' (guardrails) in place, enterprise deployment remains too hazardous. [1:20] Chandra Kapireddy: If an organization finds itself struggling to reverse-engineer why a black-box model made a specific determination after the fact, it is already in a compromised position. [5:17] George Box: All models are wrong, some are useful. [5:17] Chandra Kapireddy: every single large language model hallucinates. [5:17] Chandra Kapireddy: Because no hallucination-free model exists on the market, the only way to safely leverage the power of LLMs while insulating the business from their inherent failure modes is to embed robust, multi-layered guardrails directly into the software architecture. [5:17] Chandra Kapireddy: Enforcing guardrails at these tool-selection and response boundaries is the minimum requirement for maintaining application reliability and auditability. [8:03] Chandra Kapireddy: Building enterprise guardrails is not merely a technical challenge; it is a cross-functional governance discipline. [10:14] Chandra Kapireddy: Scaling risk management across thousands of implementations cannot be achieved through manual, ad-hoc committee reviews; doing so creates an unsustainable operational bottleneck.

Original transcript

[0:04] Wow. I think uh we have only 20 minutes. [0:07] So I think I should really stop here and [0:10] open the room for Q&A. [0:13] So I think this is really a big topic. [0:16] You know, usually takes about the whole [0:18] day, sometimes a week, sometimes a [0:20] month. So I'm going to skip my intro [0:22] given the amount of time we have. We go [0:24] to next slide. [0:26] So really, you know, it boils down to [0:28] three things. I mean I think you know [0:30] who I mean what is responsible AI you [0:32] know how do you implement it how do you [0:34] scale it look if you all have uh you [0:37] know your own uh GPTs on your mobile [0:39] phone you could actually type in the [0:41] prompt you get the answers for these [0:43] questions right so I think that's the [0:47] next slide if we go to next slide so I [0:49] think I just wanted you all to say one [0:52] thing that's going to be very common in [0:54] terms of what is a responsible AI you [0:57] see that ethics [1:00] transparency, [1:01] right? And accountability. These are [1:03] something some things we know for a very [1:06] long time. We've been doing it. AI, Gen [1:08] AI. I think it's even really more [1:11] important for us to think about this. [1:14] Again, you can do all of this using your [1:16] own time. If you go to the next slide, [1:20] this is where I would like for you all [1:22] to spend some time. I mean, I don't know [1:24] how many of you have seen the NIST AI [1:26] risk management framework. if raise your [1:28] hands. [1:29] Those of you who haven't, I think I [1:32] highly recommend that you take at least [1:34] a week of your time and really go [1:37] through this. Those who have seen it [1:39] already, I would highly recommend that [1:40] you actually reread it multiple times. [1:44] So this is I would like to kind of give [1:45] you a simple analogy. [1:49] you know we hear lots of uh you know [1:51] LLM's [1:53] both chat versions and the reasoning [1:56] versions right and there also [1:58] multimodels now I think language is no [2:00] longer the only thing that LLMs do today [2:03] I think we use multimodel right uh the [2:06] analogy is really if you given a car if [2:09] you have two cars in front of you one is [2:11] really it looks awesome it drives at 300 [2:14] m per hour and you know the price of the [2:18] car is $50,000. [2:20] The other car, it doesn't look that [2:22] awesome. It drives only up to 100 mph [2:25] and the price of the car is $50,000. [2:27] Which car would you pick? So, very [2:29] simple, right? Two cars, one drives, can [2:34] drive up to 330 m per hour. Price is [2:37] same. It looks awesome, [2:39] right? So, you might be thinking, yeah, [2:41] what's the trick here? The trick is I [2:44] haven't revealed any information about [2:46] its braking system. [2:48] Right. The car that actually looks [2:50] awesome, drives up to 330 m per hour, I [2:53] think, you know, looks very comfy and [2:55] all of that, but the braking system may [2:56] not work all the time. [3:00] You say, "Absolutely not." Right? That's [3:03] exactly where Gen AI is today. We have [3:06] lots and lots of models out there, but [3:09] the breaking systems, if they're not [3:11] implemented right, [3:13] no matter how powerful they are, they're [3:15] not going to work. They're great really [3:17] to get your information very quickly, [3:19] but you always have to doublech checkck [3:21] that information, right? How many of you [3:23] have planned your you know travel using [3:26] chat GPT like interfaces? Most of you [3:30] have you double checked or you simply [3:32] blindly booked the tickets. You always [3:35] had to make the phone calls to the [3:37] hotels. You got to really sometimes you [3:39] know these models really are not always [3:41] going to be the same. they're they kind [3:43] of behave very weird depending upon [3:45] what's been what it's been uh trained on [3:49] and what it has seen. So this is an [3:52] important you know slide directly from [3:55] the Nest risk management framework and I [3:58] think if you look at this the security [4:01] the the privacy explanability fairness [4:04] transparency and all of that is super [4:05] important. By the way, these are all not [4:07] the buzzwords. I think you know you all [4:09] have to take each and every single, you [4:11] know, word here on this slide to truly [4:13] understand what it really means. Uh [4:16] explainable and interpretable. If you [4:18] have been kind of navigating the AI and [4:21] machine learning journey, if you really [4:24] have to explain something, you're [4:25] already in trouble, [4:27] right? I mean, if you I mean, if if the [4:29] model is out there, if the equation is [4:32] y= [4:33] mx, y= 2x is a simple equation. It's a [4:36] straight line, linear line. There's no [4:38] explanation. It's interpretable, right? [4:40] But if we give you a complex equation, [4:42] if you have to explain, there's a lot of [4:45] you know analysis that you really have [4:48] to do to have the explanation. So I [4:50] think so the more and more large [4:51] language models without any question [4:53] explanability becomes a super super [4:56] challenging problem for us today. That's [4:59] why most companies most financial [5:01] services really do not do not readily [5:05] use them for critical decision making [5:09] just because of that and by that means [5:12] transparency is almost low. Okay. [5:17] So I think if we go to next slide I want [5:20] to spend some time here. [5:22] So responsible AI is super important. [5:24] And how do you implement responsible AI? [5:27] And I'm sure you probably know this. Uh [5:30] uh all models are wrong. Some are [5:32] useful. [5:34] You probably heard that. And you also [5:37] probably by now should know that every [5:40] single [5:42] uh large language model hallucinates. [5:46] Give me one model that doesn't [5:47] hallucinate today. Absolutely none. [5:50] Right? you know that all models helite [5:52] today and how do you really make sure [5:54] that you can actually take advantage of [5:56] what it can do and also ensure the [5:59] limitations of that really are covered [6:02] the only way you do do that is really [6:03] through godrails okay and by the way I [6:06] created these slides uh very carefully [6:10] uh because there's so much of [6:11] information you get now on the internet [6:13] right and you can actually I can get the [6:15] whole presentation with the one prompt [6:17] and you get the whole presentation but [6:19] this is the slide I created to make sure [6:20] that I kind of simplify the whole [6:22] concept here. You see the prompt and [6:24] response and you always use end-user [6:27] application as a starting point and then [6:30] it actually goes to some kind of a code [6:33] base agent doesn't mean AI. So there is [6:37] a agentic AI or AI agent. There's two [6:41] the two are kind of interestingly used [6:42] in different scenarios. But anytime you [6:45] use the agent if there's a AI component [6:49] that becomes an AI agent but there can [6:52] be a deterministic agents meaning if [6:54] then else. Is this an agent? That's a [6:56] job follows a specific set of rules. In [7:00] this case the orchestr agent really what [7:02] it really means is it has a AI [7:03] component. If it has AI component, it [7:05] has AI agent. If it doesn't, it's a [7:08] deterministic agent. Okay. [7:10] When you send the prompt all the way, [7:12] this agent now AI agent is given an [7:17] option to pick a set of tools and [7:18] resources and APIs [7:21] and access memory. [7:23] It makes a decision for you and you have [7:26] to have a guardrail in place on how to [7:28] make sure how do you how do you know [7:29] this agent really made the right [7:30] decision for you, right? and then [7:33] passing it back to the LLM. This is [7:35] where the reasoning model is that [7:37] actually is making the decision which [7:39] tool to use or not. And whatever code [7:42] you write in that orchestrator agent is [7:44] the code that's going to really be [7:46] critical for your enterprise. If you are [7:48] working in a regulated industry and and [7:51] you wanted to increase the reliability [7:53] of the AI application, agentic AI [7:56] application, you got to make sure that [7:58] these are the basic core minimum things [8:00] you must have in place. [8:02] I'll go to the next slide. Um again this [8:06] is another most practical practitioner's [8:10] view of how things really work in big [8:13] companies [8:15] and uh you know if you really you know [8:18] think about companies like you know all [8:22] financial services uh by the way I lead [8:24] AI machine learning for truis uh tris is [8:27] the seventh largest bank in the country [8:30] uh and just like any other bank uh [8:33] previous to tris I was with JP Morgan [8:35] Chase. Previous to JP Morgan Chase, I [8:37] was I was with Amazon. No matter where [8:39] you go, these are the key stakeholders [8:42] that you must have, you know, in in your [8:48] kind of a committee to understand how to [8:51] implement and scale AI with a risk [8:55] management in place. So it's this is [8:58] important. If you take uh an example [9:00] here PII [9:02] right as a simple guardrail that you [9:05] want to control look at all of the [9:07] different stakeholders and their [9:09] perspectives [9:11] and their risk tolerances right the risk [9:15] tolerance as you can see for cyber [9:17] security is low legal is extremely low [9:22] and privacy uh is low as well but [9:25] business look I need to know exactly [9:28] exactly who that customer is, then I can [9:31] sell more and more products to that [9:33] customer and their level of tolerance [9:36] for that is very high. Right? But these [9:39] are the additional stakeholders that [9:41] really understand exactly what it means [9:43] to them to make sure that the bank as a [9:46] whole is protected. Right? So this is [9:49] the risk tolerance. So what happens? So [9:51] if this if you're in a scenario where [9:52] you want to implement this at scale, if [9:56] you go to next slide, this is uh [10:00] I'm going to leave some room for [10:01] questions. Um um if each of these [10:06] stakeholders have their own risk [10:09] tolerance [10:10] and how do you really scale it at [10:12] enterprise? [10:14] So [10:16] uh I think uh maybe just uh maybe raise [10:19] your hand if your organization has more [10:22] than 100 AI use cases [10:25] today. More than 100 [10:28] more than thousand. [10:31] Wow. I see some big banks here or big [10:34] companies. not I would not I would not [10:36] be biased towards banks but so there [10:38] will be thousands of use cases in every [10:41] single company in the future right I'm [10:43] talking about next year this time we'll [10:46] we'll talk about thousands of use cases [10:47] you know why [10:49] you can almost embed AI in every single [10:52] process of your company and that's what [10:55] is happening today as we speak so when [10:58] you really have you want to embed AI in [11:02] every single use case that you find out [11:04] your thousands of them at least hundreds [11:07] for most companies and how do you make [11:09] sure that you apply this guardrails. Now [11:13] you know you you start with each of [11:16] these stakeholders define their own set [11:18] of guardrails. This is a key point. You [11:20] really go out to privacy and ask them [11:24] what's your guardrail for the [11:26] enterprise. You define it for the [11:29] enterprise once as a starting point. You [11:32] do the same thing for every single [11:35] second line function that you have in [11:36] the company. The risk, legal compliance, [11:39] you know, the the when I say risk, I'm [11:42] talking about the AI risk, super [11:44] important, privacy, all of this. [11:47] Now, when you really have a use case, [11:51] what you do is take a [11:54] consolidated godrail which is more [11:57] conservative and you apply to the use [12:00] case. So in this case for use case one [12:01] you take the most conservative and that [12:03] becomes extremely so very low and that [12:06] becomes your default position. By the [12:08] way all of this should be automated. If [12:10] you do it manually it takes forever. So [12:13] that's your default position. And now [12:16] what happens if you have to override a [12:19] an exception for a given use case you [12:22] document it you log it and then you kind [12:24] of uh move forward. And this is the only [12:28] way that you can actually enforce a [12:32] systematic set of guardrails for all of [12:34] your use cases to move forward.