Vanguard Nets $150M in GenAI Gains, Deploys Production AI Agents

Enabling Agentic AI Architecture and Processes Across the Enterprise With Vanguard — Ai4

Ai4

Deploying low-risk internal copilots first establishes the compliance trust needed to safely scale autonomous agent architectures across regulated financial workflows.

AI-assisted summary of Enabling Agentic AI Architecture and Processes Across the Enterprise With Vanguard — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by the Ai4 conference channel on August 29, 2025, this presentation features Mike Albano, Director of AI/ML, Conversational AI, and Governance Platforms at Vanguard. According to the session description, Albano breaks down how to enable and scale AI agents within heavily regulated financial environments, detailing frameworks for autonomy, risk management, and responsible enterprise deployment.

## TL;DR Mike Albano details Vanguard's blueprint for scaling generative AI and autonomous agents within a highly regulated asset management framework. To safely unlock business value, Vanguard established a repeatable path through internal governance by launching low-risk productivity copilots before advancing toward high-complexity client-facing 'moonshots.' The strategy relies on a Minimally Viable and Compliant (MVC) methodology that restricts pilot blast radiuses, paired with a tiered execution model enabling general software engineers to build low-risk agents while specialized data science teams handle frontier risks. Backing this ecosystem is an enterprise architecture featuring CI/CD compliance gates, multi-cloud model access, custom-built AI governance tracking, and strict security perimeters around data products and tool invocation.

## Atmosphere & style The presentation carries a pragmatic, engaging, and grounded tone, blending enterprise leadership with the hands-on candor of an experienced platform engineer. Albano establishes an easy rapport with the audience through lighthearted banter and relatable hockey analogies, turning complex NIST risk classifications and governance gridlock into intuitive operational lessons. The talk feels like a credible, insider field report from an organization actively solving the real-world friction between technological innovation and defensive compliance.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) Providing investment advice is a major growth opportunity for AI, but deploying agents within a financial institution requires navigating strict regulatory hurdles and high sensitivity. - [02:44](https://www.youtube.com/watch?v=vcKETMzrnPg&t=164s) Vanguard operates under an institutional principle of doing right by clients to maximize investment success, setting an internal ethical standard that exceeds legal and regulatory compliance. - [02:44](https://www.youtube.com/watch?v=vcKETMzrnPg&t=164s) Vanguard generated over $150 million in internal value in the prior year through AI-driven operational efficiencies and client outcomes, running ~70 GenAI solutions and ~12 agent systems in production. - [04:37](https://www.youtube.com/watch?v=vcKETMzrnPg&t=277s) Starting with low-risk 'rising tide' productivity copilots allows organizations to exercise cross-functional governance muscles before attempting high-risk, client-facing 'moonshots.' - [07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) As agents move up the autonomy spectrum toward multi-step reasoning and automated tool execution, the potential blast radius expands and requires deeper risk interrogation. - [07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) Deploying a 'minimally viable and compliant' (MVC) solution requires scoping initial trials to limited user cohorts and geographies to manage risk before enterprise rollout. - [10:16](https://www.youtube.com/watch?v=vcKETMzrnPg&t=616s) Moving from prototype to production hinges on whether business value justifies residual risk, prompting teams to reinforce enterprise platforms with formal controls. - [11:09](https://www.youtube.com/watch?v=vcKETMzrnPg&t=669s) Democratizing AI delivery across non-data-science developers prevents engineering bottlenecks, using a tiered support model ranging from self-service to AI CoE co-development. - [13:06](https://www.youtube.com/watch?v=vcKETMzrnPg&t=786s) The CI/CD pipeline acts as the decisive automated control gate between sandbox experimentation and production, enforcing compliance and quality checks prior to deployment. - [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) Frontier models continually leapfrog one another, making multi-cloud model hubs essential to avoid single-vendor lock-in and enable agile model switching. - [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) Enterprise agent autonomy requires strict API security boundaries for tool invocation alongside certified data products with verified lineage to prevent 'garbage in, garbage out' failures.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) Introduction and the Imperative of Enterprise Agentic AI

Mike Albano opens his session at the Ai4 conference by thanking the host, Aiden, and welcoming the audience. He starts with a quick show of hands: "Who has heard about agents?" Looking out at the sea of raised hands, he jokes that he wonders why anyone would be there—or why their bosses would have funded their trips—if they hadn't already heard about agents, given that the technology is currently "all the rage." However, the core challenge is not simply being aware of agents, but learning how to deploy and scale them to extract tangible enterprise value while operating strictly within responsible boundaries. Albano notes that the talk will move quickly through foundational concepts before diving into the high-level architectures and governance processes that make responsible scaling possible.

Albano outlines Vanguard's context as a global financial services powerhouse servicing individual retail investors, brokerage clients, 401(k) plans, 403(b) programs, and defined benefit retirement accounts. Vanguard also partners closely with independent financial advisers by distributing products such as ETFs and offering direct investment advice. Providing advice represents a massive growth area and a compelling opportunity for AI assistance, but doing so within a financial institution is inherently sensitive. Albano shares a glimpse into his personal career, noting he has been at Vanguard for an extended period: the first half in hands-on technical work across data and analytics systems, and the second half in people leadership. Over the last seven years, he has focused squarely on AI platforms. He oversaw Vanguard's migration off local desktop setups and onto cloud infrastructure back when the company employed only a couple dozen data scientists—a footprint that has since ballooned into hundreds of data scientists and thousands of active users across their internal platforms.

- **Vanguard's footprint**: Major retail brokerage, proprietary mutual funds, ETFs, 401(k)/403(b) retirement plans, and financial advisory services. - **Career background**: Over seven years leading AI platform development at Vanguard, scaling data science from a handful of desktop practitioners to enterprise cloud operations.

---

[02:44](https://www.youtube.com/watch?v=vcKETMzrnPg&t=164s) Operating Constraints, Mission Standards, and Early AI Wins

Deploying AI in an asset management firm requires navigating an intricate matrix of financial industry regulations, global data privacy laws, and rapidly emerging AI-specific legislation. Despite these stringent requirements, Albano stresses that Vanguard holds itself to a standard far beyond simple legal compliance: the overarching institutional principle of doing right by clients to maximize their chances of investment success. Every initiative involving generative AI or autonomous agents must demonstrably benefit shareholders and retail investors, with absolute certainty that it will not harm them.

The strategy has already produced measurable results. Vanguard recently opened up generative AI and agent experimentation across its engineering workforce, enabling thousands of application developers and software engineers to experiment in controlled sandbox environments. Over 150 to 160 agent workflows are currently under active experimentation, with about a dozen full agent systems operating live in production. Across the broader generative AI umbrella, approximately 70 generative solutions are already delivering live business value. In the prior year alone, these AI applications generated over $150 million in internal value—measured explicitly in operational efficiencies, workflow enhancements, and improved outcomes for end clients, entirely separate from market-driven investment returns.

- **Fiduciary priority**: AI initiatives must pass internal client-first ethical hurdles before meeting formal legal and regulatory thresholds. - **Enterprise scale**: Thousands of internal engineers enabled; 150–160 agents in active development/experimentation. - **Production footprint**: ~12 agent workflows and ~70 broader GenAI applications running in production. - **Financial impact**: In excess of $150M generated the previous year via internal productivity gains and client outcome enhancements.

---

[04:37](https://www.youtube.com/watch?v=vcKETMzrnPg&t=277s) Hockey Metaphors, "Rising Tides," and Moonshots

Drawing on his background as a hockey player, Albano uses a sports metaphor to explain AI experimentation: "Just like in hockey, you have to take shots in order to score goals." However, if a team takes wild slapshots from the wrong side of the ice, uses damaged equipment, or shoots blindfolded, they will never score. Enterprise AI teams must be deliberate about where they aim and which tools they select, carefully balancing their risk profile against potential impact.

```mermaid flowchart TD accTitle: Value vs Risk Spectrum of AI Initiatives subgraph LowRisk["Low Risk / High Broad Utility"] RT["Rising Tide Projects"] --> COP["Office Productivity & Copilots"] COP --> GOV["Exercise Governance Muscles<br/>(Legal, Privacy, AI Ethics)"] end subgraph HighRisk["High Complexity / Transformational"] MS["Moonshots"] --> EXT["Client-Facing Autonomous Chatbots"] EXT --> AUT["Higher Autonomy & Defensive Scrutiny"] end GOV -. Prepares Process For.-> MS ```

Albano categorizes opportunities into two distinct buckets: "rising tide" projects and "moonshots." A rising tide raises all ships; these are foundational tools like office productivity copilots that elevate efficiency across every desk. Asking the room how many attendees use a copilot daily, he observes that while adoption is growing, it is far from universal. Vanguard intentionally began with these lower-risk internal productivity tools because they provided immediate daily time savings—such as drafting quick meeting summaries—while serving as a proving ground for the organization's governance machinery. Getting a copilot deployed required convening privacy, legal, data governance, and AI governance stakeholders around the table to negotiate terms and establish a repeatable "path to yes." Starting with high-stakes projects would have failed immediately. By contrast, "moonshots"—such as customer-facing generative advisory agents—carry immense transformative potential alongside steep technical and regulatory risk. Having already exercised their governance muscles on rising-tide applications, the organization was equipped to tackle client-facing agent experiments without getting stalled in committee deadlock.

- **Metaphorical framing**: AI requires high shot volume, but success hinges on shot selection, visibility, and matching tools to the ice. - **Rising tide applications**: Ubiquitous internal productivity copilots characterized by low operational risk and immediate baseline efficiency. - **Moonshot applications**: Complex, high-value projects (such as client-facing conversational chatbots) requiring extensive defensive scrutiny and established governance trust.

---

[07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) Autonomy Frameworks and the Path to Minimum Viable Compliance

When teams progress toward moonshots, risk escalates along two dimensions: the scope of exposure and the agent's degree of autonomy. To manage this, Vanguard adopted and adapted an autonomy spectrum framework (drawing inspiration from established industry models, including categorizations documented in Forbes and core principles aligned with the NIST AI Risk Management Framework). As an agent gains independence—moving from passive information retrieval, to human-in-the-loop task execution, to multi-step reasoning with automated tool execution—the blast radius expands dramatically. Each step up the autonomy scale demands deeper, earlier interrogations with internal risk, compliance, and governance partners before any code reaches production.

```mermaid flowchart LR accTitle: Minimum Viable and Compliant AI Deployment Lifecycle subgraph Selection["1. Selection"] SC["Assess Org Readiness<br/>& Defensive Alignment"] end subgraph Architecture["2. Architecture"] MVC["Architect for MVC<br/>(Minimally Viable & Compliant)"] end subgraph Experiment["3. Piloted Experiment"] P["Constrain Blast Radius<br/>(Geos / User Cohorts)"] ITER["Measure Impact,<br/>Iterate & Risk/Reward"] P --> ITER end Selection --> Architecture --> Experiment ```

Albano wraps up the strategy phase by focusing on operationalizing what he calls a "minimally viable and compliant" (MVC) solution. The primary hurdle is structuring early deployments so that defensive-minded stakeholders feel secure granting approval. Rather than rolling an agent out across the entire enterprise or client base, architects must intentionally constrain the blast radius. This means locking down the initial release to a tightly defined pilot—restricting access to specific geographical regions, whitelisting a small test cohort of users, and embedding firm guardrails and monitoring hooks. Once the experimental sandbox is live, teams can gather quantitative telemetry, study qualitative user feedback, iterate on the architecture, and systematically conduct a rigorous risk-versus-reward assessment before granting the agent greater autonomy or broader reach.

- **Autonomy progression**: Adapting external frameworks (Forbes, NIST) to calibrate oversight against increasing levels of agent independence. - **Minimally Viable and Compliant (MVC)**: Designing architecture with foundational controls pre-baked so governance bodies can safely approve initial pilots. - **Blast-radius restriction**: Mitigating exposure by scoping early agent trials to small user subsets and defined geographical regions prior to enterprise scaling.

[10:16](https://www.youtube.com/watch?v=vcKETMzrnPg&t=616s) Evaluating Residual Risk and Deciding When to Scale

Moving from an initial prototype to production requires a clear-eyed assessment of whether the business value actually justifies the residual risk that remains after controls are implemented. Once an AI system is designed and initial client feedback demonstrates real-world impact, teams must determine whether the experiment merits further investment. Not every exploratory project qualifies to become an enterprise-wide platform capability, but many do.

When a use case proves viable, it often shows clear potential to benefit other geographical regions, alternative lines of business, or parallel operational workflows. At that juncture, the focus pivots toward building robust enterprise solutions and reinforcing enterprise platforms with the controls, governance mechanisms, and architectural requirements needed to support broad organizational adoption safely.

[11:09](https://www.youtube.com/watch?v=vcKETMzrnPg&t=669s) Execution Strategy: Democratization and Risk-Tiered Delivery

Executing this strategy depends on intentional democratization. The goal is to start with use cases that sit at lower levels of risk and complexity, providing developers who lack formal data science backgrounds with the tooling and guardrails needed to build agents and deliver business value responsibly. Because traditional software developers vastly outnumber specialized data scientists in virtually any organization, relying solely on data science teams would create an immediate delivery bottleneck.

```mermaid flowchart TD accTitle: Tiered AI Delivery Model subgraph LowComplexity["Low Risk & Low Complexity"] A["Product Development Teams<br/>(Self-sufficient)"] end

subgraph MidComplexity["Moderate Risk & Complexity"] B["Product Teams + AI Center of Excellence (CoE)<br/>(Co-development / Guidance)"] end

subgraph HighComplexity["High Risk & High Complexity"] C["Dedicated Data Scientists<br/>(Advanced Modeling & Bespoke R&D)"] end

A -->|"Escalating Risk & Technical Complexity"| B B -->|"Highest Stakes & Frontier Architecture"| C ```

To balance self-service with safety, the delivery framework adapts based on the risk and complexity of the problem: - **Self-sufficient product teams:** Handle low-risk, standardized agentic tasks using shared enterprise platforms. - **AI Center of Excellence (CoE) partnerships:** Step in as risk and architectural complexity increase, providing hands-on expertise and guidance to product teams. - **Dedicated data scientists:** Take ownership of the most complex, high-stakes use cases, navigating deep technical risks and pioneering frontier innovations for the enterprise.

[13:06](https://www.youtube.com/watch?v=vcKETMzrnPg&t=786s) The Developer Lifecycle and CI/CD as the Production Control Gate

Looking at architecture from the perspective of what builders interact with, the tooling must cater broadly to anyone building AI—whether under the emerging banner of "AI engineer," traditional software engineer, or data scientist. At design time, these practitioners require structured agent frameworks and foundational scaffolding. As development progresses, robust evaluation capabilities become critical so builders can verify that agents behave consistently before attempting any deployment.

```mermaid flowchart LR accTitle: Agentic AI Deployment and Runtime Pipeline subgraph BuildTime["1. Design & Development"] A["Agent Frameworks & Scaffolding"] --> B["Iterative Evaluation Suites"] end

subgraph Gate["2. Quality & Safety Gate"] B --> C["CI/CD Pipeline<br/>(Policy, Compliance & Asset Checks)"] end

subgraph Runtime["3. Production Operations"] C --> D["Production Runtime Monitoring"] D --> E["Ongoing Evaluation & Observability"] end ```

The CI/CD pipeline serves as the decisive checkpoint between experimentation and live execution. When code and agent configurations are packaged for deployment, the automated pipeline performs thorough compliance checks and enforces enterprise quality controls. Once live, operational tooling takes over, providing continuous runtime monitoring and telemetry to ensure agents continue operating within expected boundaries.

[14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) Foundational Infrastructure: Multi-Cloud Model Hubs, Custom Governance, and Data Products

Behind developer-facing tooling sits the underlying infrastructure required to sustain enterprise agentic systems. A primary demand from practitioners is access to diverse models across multiple cloud providers. Because frontier large language models continually "leapfrog one another," architectures cannot remain locked to a single vendor; the enterprise must maintain safe, standardized connectivity that lets teams test and incorporate new models as they emerge.

Traditional ModelOps practices are being adapted specifically for generative AI and agent workflows. In the governance arena, third-party vendor platforms often fall short of enterprise requirements, necessitating the creation of a custom-built AI inventory. This platform catalogs all models, third-party services, and generative use cases, tracking formal risk evaluations, workflow approvals, and internal third-party validation cycles.

As agents gain greater autonomy, tool invocation becomes central to system design: - **API and tool integration:** Architectures must enforce strict identity and security boundaries so autonomous agents can interact with other agents, models, and back-end APIs without accessing sensitive systems outside their remit. - **Data products as a quality anchor:** Applying data product principles—embedding explicit metadata, lineage, and verifiable quality—ensures agents avoid the classic "garbage in, garbage out" failure mode.

The session closes by echoing the theme of the conference keynote: deploying agentic AI successfully requires every practitioner to actively take responsibility for safety, both within their organizations and across society as a whole.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) Who has heard about agents? — Mike Albano

> [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) all the rage — Mike Albano

> [04:37](https://www.youtube.com/watch?v=vcKETMzrnPg&t=277s) Just like in hockey, you have to take shots in order to score goals. — Mike Albano

> [04:37](https://www.youtube.com/watch?v=vcKETMzrnPg&t=277s) rising tide — Mike Albano

> [04:37](https://www.youtube.com/watch?v=vcKETMzrnPg&t=277s) moonshots — Mike Albano

> [04:37](https://www.youtube.com/watch?v=vcKETMzrnPg&t=277s) path to yes — Mike Albano

> [07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) minimally viable and compliant — Mike Albano

> [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) leapfrog one another — Mike Albano

> [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) garbage in, garbage out — Mike Albano

## Mentioned Mike Albano, Aiden, Ai4, Vanguard, Forbes, NIST AI Risk Management Framework, Skynet

## Video description Mike Albano, Director of AI/ML, Conversational AI, and Governance Platforms, Vanguard

Abstracts

Step into the future with me as I demystify the complex world of enabling AI Agents in a regulated industry. My presentation will delve into the intricacies and interplay of GenAI, Agents, and the levels of autonomy you must consider before plotting your course. I'll provide a winning strategy that has enabled our responsible AI deployment, and how you can leverage them to transform the way you and your company works. Whether you're an AI enthusiast, a professional eager to stay ahead of the curve, or just a curious mind looking to understand the future, you will come away from this presentation full of ideas and a strategy to deliver on the promise of AI Agents - without creating Skynet.

[Source](https://www.youtube.com/watch?v=vcKETMzrnPg)

中文简报

Vanguard获1.5亿美元生成式AI收益，部署生产级AI智能体 先行部署低风险的内部Copilot，有助于建立在受监管金融工作流中安全扩展自主智能体架构所需的合规信任。 Mike Albano详细介绍了Vanguard在高度受监管的资产管理框架下扩展生成式AI与自主智能体的蓝图。为安全释放业务价值，Vanguard建立了一条经由内部治理的可复用路径：在迈向面向客户的高复杂度“登月级”项目前，率先推出低风险的生产力Copilot。该策略依托能够限制试点影响范围的“最小可行且合规”（MVC）方法论，并结合分层执行模式，使普通软件工程师能够构建低风险智能体，而专业数据科学团队则负责应对前沿风险。支撑该生态系统的是一套企业级架构，具备CI/CD合规关卡、多云模型接入、定制化AI治理追踪，以及围绕数据产品和工具调用的严格安全边界。 Vanguard 在上一年度通过 AI 提升运营效率并改善客户成果，创造了超过 1.5 亿美元的内部价值，目前已有约 70 个生成式 AI 应用和约 12 个 Agent 系统在生产环境中稳定运行。 (2:44) 企业可通过优先推行低风险的“顺水推舟”型办公 Copilot，在进军高风险、面向客户的“登月”级 Agent 项目之前，充分磨合法律、合规与数据伦理等跨职能治理能力。 (4:37) 随着 Agent 自主权提升至多步推理与自动化工具执行，潜在影响范围显著扩大，架构需通过限定用户群与地域来构建“最小可行且合规”（MVC）方案以控制暴露风险。 (7:42) 为避免数据科学团队成为研发瓶颈，Vanguard 赋能常规软件工程师参与 AI 构建，并根据风险与复杂度建立从团队自研到 AI 卓越中心（CoE）共建的分级交付体系。 (11:09)

Reading summary

Deploying agentic AI across a heavily regulated financial enterprise requires establishing repeatable governance on low-risk internal tools before deploying autonomous, high-stakes capabilities, according to Vanguard AI platform leader Mike Albano. Exercising Governance Muscles Through Low-Risk Tools (4:37) To navigate financial regulations and client-first fiduciary standards, Vanguard intentionally initiated its generative AI rollout with low-risk internal productivity tools rather than complex, customer-facing projects. Albano categorizes opportunities into "rising tide" tools—such as office copilots that generate meeting summaries—and complex "moonshots," like autonomous client advisory agents. Deploying simple internal tools forced legal, privacy, data governance, and AI ethics teams to establish a repeatable operational framework and achieve a shared "path to yes." By developing institutional governance capabilities on lower-stakes applications first, the organization avoided bureaucratic deadlock when later reviewing high-stakes agent workflows. This deliberate rollout yielded measurable scale. Vanguard expanded experimentation to thousands of software developers, resulting in 150 to 160 active agent experiments, approximately 12 agent workflows in live production, and around 70 broader generative AI applications. According to Albano, these initiatives delivered over $150 million in enterprise value in the prior year through operational efficiencies and improved client outcomes, independent of market returns. Limiting Blast Radii via Minimally Viable Compliance (7:42) As agents transition from passive retrieval to multi-step reasoning and tool execution, the risk profile expands. Vanguard adapted an autonomy spectrum framework informed by NIST AI Risk Management principles to calibrate organizational oversight against an agent's technical independence. To secure defensive stakeholder approval, teams design architectures around a "minimally viable and compliant" (MVC) standard that embeds foundational guardrails prior to deployment. Rather than launching broadly, early pilots intentionally restrict their blast radius to specific geographic regions or whitelisted user cohorts. This sandbox approach allows teams to collect real-world telemetry, evaluate user feedback, and measure business impact against residual risk before deciding whether an agent warrants scaling into an enterprise-wide platform capability. Democratizing Development Through a Tiered Delivery Model (11:09) Because software engineers heavily outnumber specialized data scientists, Vanguard avoids relying exclusively on data science teams, which would create an enterprise bottleneck. Instead, the firm distributes AI development across three tiers based on risk and architectural complexity. Under this model, self-sufficient product development teams build low-risk, standardized agentic tasks using shared enterprise platforms. For intermediate-complexity workflows, product teams partner with Vanguard's AI Center of Excellence (CoE) for direct architectural guidance. Dedicated data scientists are reserved for the highest-complexity and highest-risk frontier initiatives requiring bespoke research and advanced modeling. Automated Control Gates and Multi-Cloud Architecture (13:06) Vanguard treats the CI/CD pipeline as the definitive control gate separating experimental agent design from live runtime. At design time, builders leverage standardized agent frameworks and evaluation suites. Before code can deploy, automated CI/CD checks validate enterprise policies, regulatory compliance, and asset integrity, followed by continuous production telemetry and runtime observability. The underlying infrastructure is designed to avoid vendor lock-in by providing connectivity across multiple cloud model hubs, accommodating frontier large language models as they leapfrog one another. In addition, Vanguard built an in-house AI inventory platform to track model evaluations and third-party validation cycles because existing vendor tools did not meet enterprise requirements. To prevent agent errors, the architecture enforces strict security boundaries on API tool invocation and treats data inputs as governed data products. Albano concluded that safely scaling agentic AI ultimately hinges on maintaining technical rigor across APIs, data lineage, and deployment pipelines, while requiring enterprise builders to take direct, personal responsibility for safety and governance.

精炼阅读摘要

Vanguard AI平台负责人Mike Albano指出，企业规模化落地智能体AI（Agentic AI）的核心挑战不在于技术认知，而在于如何在严格的合规与信义义务约束下，建立兼顾风险与价值的工程治理体系。该机构通过分级交付、限制爆炸半径的最小可行合规架构以及多云基础设施，使内部数千名工程师能够安全释放AI生产力。 价值成效与“水涨船高”双轨策略 (2:44) Albano指出，作为管理庞大经纪业务、共同基金、ETF及401(k)等退休计划的全球金融服务机构，Vanguard部署AI的首要前提是超越法律合规底线，恪守“以客户利益为先”的信义标准。据其披露，Vanguard目前已有数千名应用开发者在沙箱环境中开展实验，活跃实验的智能体工作流达150至160个，投入实际生产运行的智能体系统约12个，另有约70个生成式AI方案在运营中交付价值。在过去一年中，这些AI应用通过优化内部运营效率、工作流及改善终端客户成果，创造了超过1.5亿美元的内部业务价值（不计入市场驱动的投资回报）。 在推进路径上，Albano借用冰球运动隐喻强调企业必须审慎选择“击球”场景与工具，并将项目划分为“水涨船高”（Rising Tide）与“登月”（Moonshots）两类。Vanguard选择首先推进风险较低的内部办公生产力助手（如会议纪要生成），借此将法律、隐私、数据治理与AI伦理等防守型利益相关方拉到同一张谈判桌前，建立标准化的审批与治理流程。这种前期的“治理练兵”不仅带来了即时的工时节约，更为后续高复杂度、直面客户的“登月”级生成式顾问智能体铺平了道路，避免项目陷入委员会审批僵局。 自主权图谱与“最小可行合规”试点 (7:42) 随着AI项目从内部辅助转向直面客户的“登月”级应用，系统风险会随着暴露范围和自主权水平的提升而急剧增加。Albano介绍，Vanguard借鉴并改编了行业模型及NIST AI风险管理框架，构建了智能体自主权进阶图谱。当智能体从被动的信息检索，进阶到人机协同的任务执行，再到具备多步推理与自主工具调用的系统时，其潜在的“爆炸半径”将呈指数级扩张。因此，更高自主权的系统必须在代码进入生产前，更早、更深入地接受内部风险、合规与治理团队的审查。 为打破治理僵局，Vanguard提出了“最小可行合规”（Minimally Viable and Compliant, MVC）的架构设计策略。其核心思想并非一开始就面向全量客户推广，而是通过人为设定严格的技术护栏与监控锚点，将早期试点严格限制在特定的地理区域或经过白名单筛选的极少数测试用户群内。架构团队借此在受控沙箱中收集定量遥测指标与定性反馈，在客观权衡剩余风险与商业价值后，再决定是否扩大系统自主权或升级为企业级通用平台能力。 研发民主化与风险分级交付机制 (11:09) Albano认为，AI规模化落地的执行关键在于研发民主化。由于传统软件开发人员的数量远超专业数据科学家，如果将智能体构建完全依赖于数据科学团队，必然造成严重的交付瓶颈。为此，Vanguard构建了针对普通软件工程师的工具链与安全防护网，让缺乏正式数据科学背景的开发者也能在低风险场景中自主构建智能体并交付业务价值。 为了在自主权与安全性之间取得平衡，Vanguard确立了三级风险分层交付模型：在低风险、低复杂度的标准化任务中，产品研发团队依托共享平台实现完全自给自足；随着业务风险与架构复杂度上升，AI卓越中心（CoE）会介入并与产品团队协同开发，提供专家指导；而在面临最高风险、前沿创新或高度定制化研发的高门槛场景中，则由专属数据科学家主导攻坚，以化解深层技术风险并开拓前沿架构。 CI/CD自动化门禁与多云底座工程 (13:06) 在开发生命周期层面，面向AI工程师、传统开发者与数据科学家的统一工具链至关重要。Albano指出，构建期需要标准化的智能体框架、脚手架以及严密的迭代评估套件，以验证智能体在部署前的行为一致性。而CI/CD流水线则充当了实验与生产之间的决定性合规门禁，在打包发布阶段自动执行策略审查、合规检查和资产质量核验。一旦上线，生产运行时监控与遥测工具将持续跟踪系统表现，确保智能体始终在预期边界内运行。 底座基础设施方面，由于前沿大语言模型之间保持着“相互超越”的迭代态势，企业架构必须避免单一云供应商锁定，通过多云模型枢纽提供标准化、安全的模型接入能力。在治理层面，由于第三方平台功能往往无法满足大型金融机构的严苛要求，Vanguard自主研发了定制化的AI资产清单系统，集中纳管所有模型、第三方服务及生成式用例，追踪正式风险评估与审批流程。此外，针对智能体核心的工具调用场景，架构强制实施严格的身份与安全边界隔离，并全面推行数据产品（Data Products）理念，注入明确的元数据与血缘追踪机制，防止因数据质量缺陷引发系统性失效。 Albano在结语中强调，智能体AI在企业中的成功落地不能仅靠单一部门推动，每一位技术从业者都必须在组织内部乃至整个社会层面主动承担起保障AI安全的责任，在追求创新的同时坚守风险底线。

Detailed notes — zh

## Background 该内容由行业峰会频道 Ai4 于 2025 年 8 月 29 日发布在 YouTube 平台上。视频记录了先锋领航集团（Vanguard）AI/ML、对话式 AI 与治理平台总监 Mike Albano 的主题演讲，主要探讨如何在受严格监管的金融行业中负责任地构建 Agentic AI 架构、划定智能体自主等级并推动企业级规模化落地。 ## TL;DR 演讲系统分享了先锋领航集团在强监管环境下落地生成式 AI 与智能体（AI Agents）的工程实践与治理方法论，其系统过去一年已在内部产生超 1.5 亿美元价值。Albano 提出应从低风险内部 Copilot 起步以锻炼法务合规肌肉，再循序渐进迈向高风险的客户交互“登月项目”；组织上通过平台化降低门槛赋能广大开发者自主交付，并由 AI 卓越中心与数据科学家分级攻坚。在架构层面，他详细拆解了涵盖设计评估、CI/CD 自动化合规门控、运行时监控的全生命周期流水线，以及多云模型接入、自研模型治理台账、API 鉴权与数据产品等底层支撑支柱。 ## Atmosphere & style 整场演讲专业务实且极具亲和力，Albano 巧妙运用“冰球射门”与“水涨船高”等生动比喻化解了晦涩冷硬的架构与风控议题。演讲节奏紧凑流畅，兼具大厂技术主管的技术沉淀与从容幽默，在展现前沿 AI 实验落地成果的同时时刻紧扣合规与道德底线，为观众带来了既鼓舞人心又高度落地的工业界实战体验。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) 先锋领航（Vanguard）在推进 AI Agent 创新时，将遵守金融合规和保障客户利益置于首位，确保技术落地以负责任的 AI（Responsible AI）为基石。 - [03:35](https://www.youtube.com/watch?v=vcKETMzrnPg&t=215s) 先锋领航已为数千名工程师开放 GenAI 实验权限，内部已有约 150-160 个 Agent 处于实验阶段，约 12 个 Agent 和 70 个 GenAI 方案已投产，过去一年创造了超 1.5 亿美元的内部运营价值。 - [04:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=282s) 企业在 AI 战略上应借鉴“水涨船高”与“登月项目”的递进逻辑：先从内部低风险的 Copilot 工具切入以锻炼治理协同肌肉，再稳步推进面向外部客户的高风险复杂 Agent。 - [07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) 根据 Agent 的自治等级（Level of Autonomy）划分系统复杂度与风险，自治度越高越需要与防御型治理部门展开深度论证与边界审查。 - [07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) 构建值得信赖的 AI 系统需遵循受控实验闭环：从审慎选型、架构设计最小可行且合规产品（MVC）、受控范围试点，到度量指标与评估风险回报。 - [10:16](https://www.youtube.com/watch?v=vcKETMzrnPg&t=616s) 完成单点实验后，团队需权衡残留风险与业务收益，将验证可行的方案横向复用至更多业务线与区域，并强化底层平台以实现企业级规模化扩展。 - [11:09](https://www.youtube.com/watch?v=vcKETMzrnPg&t=669s) 采取阶梯式协作推动技术普惠：通过低门槛平台赋能普通开发者自主交付低风险用例，中等复杂度场景由 AI CoE 协同，高风险创新场景由专业数据科学家重点攻坚。 - [13:06](https://www.youtube.com/watch?v=vcKETMzrnPg&t=786s) 为开发者提供贯穿全生命周期的平台能力，包括设计期的框架与离线评测、部署期基于 CI/CD 的质量合规检查门控，以及生产运行期的实时监控与护栏机制。 - [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) 打造多云模型接入层，以应对各大公有云基础模型的竞争迭代，同时结合统一安全防护实现跨云快速切换。 - [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) 自研 AI 与模型治理中心（AI Inventory），集中纳管专有与第三方模型及 GenAI 用例，严格执行模型风险评估与内审团队的独立验证流。 - [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) 针对高自治度 Agent 强化 API 与工具调用的访问控制与鉴权隔离，并以“数据产品”（Data Products）模式保障元数据与数据血缘可信，从源头杜绝“垃圾进，垃圾出”。 - [18:07](https://www.youtube.com/watch?v=vcKETMzrnPg&t=1087s) 负责任 AI（Responsible AI）不仅是企业在受监管环境中落地智能体的技术前提，更是全行业在推动 AI 发展进程中必须共同承担的社会责任。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) 开场互动与先锋领航集团的 AI 使命 演讲伊始，先锋领航集团（Vanguard）AI/ML、对话式 AI 与治理平台总监 Mike Albano 向现场观众热情致意。他首先向台下发问：“请大家举手示意，有谁听过 AI Agents？”看着全场纷纷举起的双手，他幽默地调侃道：“果不其然，如果大家或各位的老板事先连 Agent 都没听说过，那真不知道你们为什么会来这里了。”他指出，尽管 Agent 技术当下炙手可热，但本次演讲的核心并非单纯盲目追捧热度，而是探讨如何在受严格监管的行业中，既能规模化释放其商业价值，又能以高度负责任的方式（Responsible AI）推进架构设计与流程治理。 Mike Albano 随后简要介绍了先锋领航集团的业务版图与自身的从业经历。作为全球金融服务巨头，先锋领航服务着海量个人投资者与机构，业务覆盖零售经纪账户、共同基金、401(k) 与 403(b) 退休计划、固定收益产品，以及为财务顾问提供 ETF 投资组合与专业咨询建议。在个人履历方面，Albano 长期深耕技术与管理领域，早年专注于数据和分析系统研发，近七年来则全力主导先锋领航的 AI 平台建设，见证了团队从最初几十名数据科学家逐步迁移至云端，演变到如今拥有数百名算法专家及数千名平台用户的宏大格局。 谈及运营环境时，Albano 强调，金融企业不仅必须恪守严格的金融监管条例、全球数据隐私法规以及不断涌现的 AI 专属法案，先锋领航内部更自我设定了高于行业法规的道德底线——将客户的最佳投资成功机会放在首位。AI 的一切探索与落地，都必须以增进股东与客户权益为前提，绝不能对客户造成任何潜在风险或损害。 --- ### [03:35](https://www.youtube.com/watch?v=vcKETMzrnPg&t=215s) 先锋领航的 AI 规模化落地与价值成果 先锋领航的生成式 AI 与 Agent 推进速度极其迅猛。目前，集团已为全公司数千名工程师和应用开发者全面开放了生成式 AI 及智能体技术的实验权限。整个组织的创新管线正在高速运转，用例数量在极短时间内持续攀升。 在具体的项目落地与产出指标上，Albano 分享了一组亮眼的数据： - **智能体实验规模**：当前内部已有约 150 至 160 个 Agent 项目处于积极实验与验证阶段。 - **智能体生产上线**：已有约 12 个成熟的 Agent 成功部署进生产环境。 - **GenAI 投产方案**：累计已有约 70 个生成式 AI 解决方案在生产环境中稳定运行，持续提供合规价值。 - **内部商业价值创造**：仅在过去一年中，AI 系统通过内部运营效率优化与流程改善，为企业带来了超过 1.5 亿美元（$150M+）的综合价值（该数据基于内部运营能效核算，不包含单纯的外部投资回报）。 --- ### [04:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=282s) 用冰球与“水涨船高”隐喻构建用例战略：从生产力工具到登月项目 在探讨如何挑选 AI 用例并驱动价值时，曾作为冰球运动员的 Albano 借用了冰球运动中的经典格言——“要想进球，就必须射门”（You have to take shots in order to score goals）。但他随即指出，若球员站错了冰场位置、选错了球杆，甚至蒙着眼睛盲目挥杆，射门再多也无法破门得分。企业在 AI Agent 的落地过程中同样不能盲目乱试，而必须科学评估用例胜率，在风险受控的前提下精准出击。 ```mermaid flowchart TD accTitle: AI 用例演进与风险阶梯模型 subgraph STAGE1["基础层：水涨船高型用例"] A["办公 Copilot / 效率工具"] --> B["全员生产力提升 & 风险受控"] B --> C["锻炼法务/合规治理协同肌肉"] end subgraph STAGE2["高阶层：登月型项目"] C --> D["面向外部客户的交互式 Agent"] D --> E["高业务颠覆性 & 高风险复杂度"] E --> F["需要极高治理成熟度护航"] end ``` 针对用例的演进路径，Albano 提出了两个维度的比喻： 1. **水涨船高型用例（A rising tide raises all ships）**：这类用例如同办公自动化与 Copilot 生产力助手。Albano 在现场调查发现相当一部分观众已在日常办公中重度依赖 Copilot，它可以高效提炼会议纪要、总结文档，极大减轻员工负担。先锋领航正是选择从这类低风险的内部生产力工具起步，借此将隐私合规、法务、数据治理与 AI 安全团队拉到同一张谈判桌前，充分锻炼跨部门的“治理肌肉”，建立起协同审批与互信机制。 2. **登月型项目（Moonshots）**：如直接面向外部终端投资者的智能对话 Agent。此类用例具有极高的颠覆性潜力和商业价值，但伴随的法律与合规风险极高。Albano 强调，如果团队一开始就直接挑战登月项目，那些“防御型思维”（Defensive-minded）的合规部门很难轻易说“Yes”。正是因为前期通过低风险用例打下了扎实的治理基础，先锋领航才得以逐步推进外部客户 Agent 的试点并取得积极进展。 --- ### [07:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=462s) Agent 自治度分级、信任 AI 框架与敏捷实验落地闭环 当项目从基础生产力工具向高风险的登月型用例迈进时，另一个必须权衡的关键维度是智能体的自治水平（Level of Autonomy）。先锋领航参考并改编了业界成熟的框架（包括参考 Forbes 相关模型），根据业务场景划定不同的自治等级。Agent 的自治度越高，其系统复杂度与潜在风险就呈指数级上升，开发团队在投产前必须与防御型治理部门展开更深层次的风险论证与安全边界审查。 为了确保交付的是一个“值得信赖的 AI 系统”（Trusted AI System），先锋领航引入了对齐 NIST 等权威标准的多维度评估准则。Albano 总结了将 Agent 想法转化为受控实验的完整落地闭环方法论： ```mermaid flowchart LR accTitle: 负责任 Agent 敏捷实验落地闭环 subgraph SELECT["1. 审慎选型"] S1["评估组织就绪度"] --> S2["对齐治理部门标准"] end subgraph ARCH["2. 架构设计"] A1["构建 MVC 方案<br/>(最小可行且合规)"] --> A2["嵌入多重内控机制"] end subgraph PILOT["3. 试点实验"] P1["限定用户规模与地域"] --> P2["监控指标 & 收集反馈"] end subgraph ITERATE["4. 迭代推进"] I1["权衡风险与收益"] --> I2["逐步扩大推广"] end SELECT --> ARCH ARCH --> PILOT PILOT --> ITERATE ``` - **精准挑选用例**：深入评估组织就绪度，精准锁定既契合业务痛点又能让内部防御型职能部门认可的场景。 - **架构设计最小可行且合规产品（MVC）**：在方案设计之初就融入完备的治理与控制机制，确保系统在符合法规与安全底线的前提下进入实验。 - **受控范围试点验证**：设定严格的准入参数，将初期的暴露面严格限制在极小的特定地理区域或极少数受控用户群体中，进一步收敛风险敞口。 - **指标度量与风险回报评估**：在试点过程中严密监控业务影响、收集实际反馈、快速迭代演进，并在推进大规模上线前严格进行全方位的风险回报（Risk-Reward）综合评估。 ### [10:16](https://www.youtube.com/watch?v=vcKETMzrnPg&t=616s) 从实验到企业级规模化扩展 在完成系统设计、收集客户反馈并实际验证业务影响之后，团队面临的核心问题是：业务收益是否足以覆盖残留风险？这决定了实验性项目是否有必要进一步推进并进行规模化扩展。Mike Albano 指出，并非每一个应用场景都适合直接晋升为企业级平台项目，但实践中很多场景确实具备这种潜力。一旦验证可行，并且发现该方案能够惠及更多地理区域、其他业务线或高度相似的业务场景时，先锋领航（Vanguard）便会着手构建企业级解决方案，并强化企业底层平台，以满足全面推广所需的各项新需求与合规管控措施。 - **扩展评估准则**：综合权衡客户反馈、实际业务成效与残留风险（Residual Risk），决定场景是保持单点实验还是晋升为企业级平台。 - **跨场景复用**：将成熟经验横向推广至不同业务部门（Lines of Business）与更多区域市场，并在平台层统一补齐风控与合规要求。 --- ### [11:09](https://www.youtube.com/watch?v=vcKETMzrnPg&t=669s) 规模化落地与技术普惠：组织与人员协作模式 谈及具体执行策略，企业通常应从“低风险、低复杂度”的场景切入，但长远目标必须是推动 AI 技术的普惠化（Democratization）。现实情况是，任何企业内部的开发者数量通常都远多于资深数据科学家，“如果你拥有数据科学家，那你很幸运，但你手头的开发者肯定更多”。因此，先锋领航的核心策略是构建支持低门槛开发的平台，让缺乏深度数据科学背景的普通开发者也能安全、合规地利用智能体（Agents）创造业务价值，使产品团队尽可能实现自主交付。 ```mermaid flowchart TD accTitle: AI 场景复杂度与团队协作矩阵 subgraph LowComplexity["低复杂度 / 低风险场景"] A["业务产品团队 (自主交付为主)"] end subgraph MidComplexity["中等复杂度 / 风险上升场景"] B["产品团队 + AI 卓越中心 (AI CoE 赋能协同)"] end subgraph HighComplexity["高复杂度 / 高风险创新场景"] C["数据科学家 + AI 专家 (攻坚复杂风险与前沿创新)"] end A -->|"复杂度与风险增加"| B B -->|"核心技术突破与风控攻坚"| C ``` 随着业务风险与复杂度的逐步攀升，专业知识的介入变得不可或缺。此时，组织架构中引入的“AI 卓越中心”（AI Center of Excellence, AI CoE）便开始发挥协同指导作用。而对于处于复杂度与风险金字塔顶端的攻坚型场景，则直接指派专业的数据科学家全权负责，由他们攻克固有的高难度技术与风险难关，真正驱动高价值场景的前沿创新。 - **开发者赋能**：通过平台化屏蔽底层算法门槛，使一线开发者能够自主构建中低风险的智能体应用。 - **阶梯式协作模式**：中低风险场景由产品团队独立负责或在 AI CoE 协助下完成；高风险、高复杂度场景则由专业数据科学家重点攻坚。 --- ### [13:06](https://www.youtube.com/watch?v=vcKETMzrnPg&t=786s) 开发者全生命周期平台能力：从设计、CI/CD 门控到运行时监控 在平台架构层面，Mike 聚焦于一线实施人员与开发者（泛指所有从事 AI 开发的工程师与数据科学家）在全生命周期中所需的关键能力。整个开发与落地旅程清晰地划分为三个核心阶段：设计阶段、部署发布阶段与生产运行阶段。 ```mermaid flowchart LR accTitle: 智能体全生命周期交付与管控流水线 subgraph DesignTime["设计与开发阶段"] A["智能体框架<br/>Agent Frameworks"] B["离线评测能力<br/>Evaluation Tools"] end subgraph DeployPhase["部署与门控阶段"] C["CI/CD 流水线"] D["质量与合规检查<br/>Compliance Checks"] end subgraph RunTime["生产运行阶段"] E["实时监控与告警<br/>Monitoring"] F["线上评估与护栏<br/>Runtime Guardrails"] end A --> B B --> C C --> D D --> E E --> F ``` 在设计期，开发者依赖成熟的智能体框架（Agent Frameworks）与严密的评估组件来调试表现；当系统达到预期标准准备上线时，CI/CD 流水线扮演着进入生产环境的“第一道关键门禁”。在 CI/CD 环节中，平台会对所有待部署资产进行自动化质量检验与合规性审查，确保完全符合各项风控策略。进入生产环境后，则无缝切换至实时监控与在线评估系统，持续确保智能体行为严格符合预期，防止偏离预定轨道。 --- ### [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) 基础平台支柱：多模型接入、模型治理、API 集成与数据产品 在面向开发者的前端工具链之下，先锋领航打造了一整套支撑智能体与模型治理的底层平台支柱： ```mermaid flowchart TD accTitle: 智能体企业级底层支撑平台架构 subgraph Foundations["企业级底层支柱平台"] direction TB M["多云模型接入层<br/>(跨云连接、模型竞争迭代、安全网关)"] G["AI 与模型治理平台<br/>(自研资产全景台账、风险评级、内外部验证流)"] O["智能体 ModelOps / CI/CD<br/>(针对 GenAI 与 Agent 流程定制)"] A["API 与工具安全集成<br/>(鉴权隔离、最小权限访问控制)"] D["数据底座与数据产品<br/>(Data Products、元数据、血缘追溯)"] end ``` 1. **多云模型接入（Multi-Cloud Model Access）**：数据科学家与工程师对新模型的需求与日俱增。由于各大基础模型在不同公有云上交替领先（"as they leapfrog one another"），底层平台必须具备跨多云连接能力，并施加统一的安全防护标准，以便快速引入最新大语言模型供业务测试与切换。 2. **新型 ModelOps 与 CI/CD 定制**：将传统 AI/ML 时代的 ModelOps 理念进行重构，针对生成式 AI 与 Agent 特有的组件与提示词工程，定制专属的自动化部署与安全发布流程。 3. **自研 AI 与模型治理中心（AI & Model Governance）**：在调研外部供应商后，先锋领航选择自主研发 AI 资产全景台账（AI Inventory），全面纳管专有模型、第三方模型以及生成式 AI 应用用例。所有模型与用例均需完成模型风险评估（Model Risk Assessment）并建立定期复审机制；同时在平台内固化由独立内审团队参与的验证工作流，确保审批权责分明。 4. **API 集成与安全权限管控**：随着智能体自主性（Autonomy）的提高，无论是在 Agent 间互相调用、调用底层模型还是调用企业业务 API 时，平台都必须提供极其严密的安全性与访问控制管理，确保智能体“仅能触达被明确授权的资产与接口”。 5. **数据底座与“数据产品”（Data Products）**：Mike 强调“垃圾进，垃圾出”（Garbage in, garbage out）的铁律。通过将底层数据封装为高质量的“数据产品”，企业能够获得完备的元数据（Metadata）与数据血缘追溯（Lineage）能力，从源头确保智能体使用的数据真实、可信且合规。 --- ### [18:07](https://www.youtube.com/watch?v=vcKETMzrnPg&t=1087s) 结语：在企业与全人类层面践行负责任 AI 在演讲的尾声，Mike Albano 再次呼应了大会开幕主题演讲的精神，强调“负责任”（Responsibility）是贯穿所有 AI 创新的核心主线。他呼吁行业同仁：每个人不仅要在企业内部构建智能体时承担起安全与合规部署的技术责任，更要在推动全人类负责任地使用 AI 技术的进程中发挥积极作用。这既是先锋领航在严监管金融环境中成功推进智能体架构的核心原则，也是业界共同构建安全未来不可或缺的基石。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) 请大家举手示意，有谁听过 AI Agents？ — Mike Albano > [00:04](https://www.youtube.com/watch?v=vcKETMzrnPg&t=4s) 果不其然，如果大家或各位的老板事先连 Agent 都没听说过，那真不知道你们为什么会来这里了。 — Mike Albano > [04:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=282s) 要想进球，就必须射门 — Mike Albano > [04:42](https://www.youtube.com/watch?v=vcKETMzrnPg&t=282s) 水涨船高型用例（A rising tide raises all ships） — Mike Albano > [11:09](https://www.youtube.com/watch?v=vcKETMzrnPg&t=669s) 如果你拥有数据科学家，那你很幸运，但你手头的开发者肯定更多 — Mike Albano > [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) as they leapfrog one another — Mike Albano > [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) 仅能触达被明确授权的资产与接口 — Mike Albano > [14:49](https://www.youtube.com/watch?v=vcKETMzrnPg&t=889s) 垃圾进，垃圾出（Garbage in, garbage out） — Mike Albano ## Mentioned Vanguard（先锋领航集团）, Mike Albano, Ai4, Copilot, Forbes, NIST, AI CoE（AI 卓越中心）, ModelOps, Skynet ## Video description Mike Albano, Director of AI/ML, Conversational AI, and Governance Platforms, Vanguard Abstracts Step into the future with me as I demystify the complex world of enabling AI Agents in a regulated industry. My presentation will delve into the intricacies and interplay of GenAI, Agents, and the levels of autonomy you must consider before plotting your course. I'll provide a winning strategy that has enabled our responsible AI deployment, and how you can leverage them to transform the way you and your company works. Whether you're an AI enthusiast, a professional eager to stay ahead of the curve, or just a curious mind looking to understand the future, you will come away from this presentation full of ideas and a strategy to deliver on the promise of AI Agents - without creating Skynet. [Source](https://www.youtube.com/watch?v=vcKETMzrnPg)

Source quotes

[0:04] Mike Albano: 请大家举手示意，有谁听过 AI Agents？ [0:04] Mike Albano: 果不其然，如果大家或各位的老板事先连 Agent 都没听说过，那真不知道你们为什么会来这里了。 [4:42] Mike Albano: 要想进球，就必须射门 [4:42] Mike Albano: 水涨船高型用例（A rising tide raises all ships） [11:09] Mike Albano: 如果你拥有数据科学家，那你很幸运，但你手头的开发者肯定更多 [14:49] Mike Albano: as they leapfrog one another [14:49] Mike Albano: 仅能触达被明确授权的资产与接口 [14:49] Mike Albano: 垃圾进，垃圾出（Garbage in, garbage out）

Source quotes

[0:04] Mike Albano: Who has heard about agents? [0:04] Mike Albano: all the rage [4:37] Mike Albano: Just like in hockey, you have to take shots in order to score goals. [4:37] Mike Albano: rising tide [4:37] Mike Albano: moonshots [4:37] Mike Albano: path to yes [7:42] Mike Albano: minimally viable and compliant [14:49] Mike Albano: leapfrog one another [14:49] Mike Albano: garbage in, garbage out

Original transcript

[0:04] All right. Thank you, Aiden. And thank [0:05] you everybody for coming out today. It's [0:07] great to see so many people in the [0:08] audience. Uh so, show hands. Who has [0:12] heard about agents? Right. Right. I [0:15] mean, come on. Right. I really wonder [0:17] why you're here. I wonder why your [0:19] bosses were here sending you here if you [0:21] didn't already know about it. Uh it is [0:22] all the rage obviously. But you know um [0:26] what we're going to talk about today is [0:27] how can you enable them in a way that [0:30] lets you scale of course and get the [0:32] value but also to do it responsibly. So [0:34] we're going to go pretty fast and go [0:35] through some high level concepts but we [0:37] are going to talk about at a high level [0:39] architecture and processes that can [0:41] enable those uh goals. Uh so we're going [0:45] to start with a little bit about [0:46] Vanguard's story and uh where we uh you [0:49] know what we are what we try to do um [0:51] our strategy on how we are uh moving [0:54] forward with uh driving value [0:55] responsibly and how we're executing on [0:58] that. [1:00] So [1:03] uh a little bit about Vanguard if you uh [1:05] haven't heard uh we are a global [1:07] financial services leader and depending [1:09] on the day we could be the leader uh in [1:12] terms of uh servicing individual [1:14] investors uh with retail accounts, [1:17] brokerage accounts for uh any type of [1:19] security you might want or mutual fund. [1:21] We obviously have our own mutual funds [1:22] as well. Uh we have uh 401k participants [1:26] in our retirement plans uh 403b [1:30] um defined benefits etc. uh we also uh [1:33] have uh financial adviserss that we work [1:35] with who we provide products to mainly [1:38] in the form of ETFs and such. uh but uh [1:43] they uh we also uh offer advice uh so if [1:46] you're looking for advice on on how to [1:48] make investments that is a huge area of [1:50] concentration and these are all [1:51] opportunities for us to leverage AI but [1:55] uh as we'll hear more in a moment it's [1:57] something that obviously is is a little [1:59] tricky to do uh with AI uh and we need [2:01] to make sure that we're doing it [2:02] responsibly. Really briefly about myself [2:05] been at Vanguard a long time. I'm not [2:07] going to tell you how long, but I have [2:10] uh spent about half of my career in uh [2:12] hands-on technical roles, mainly working [2:15] with data uh and analytics systems. I've [2:18] spent the other half of my career uh [2:20] leading people. And in the last seven [2:21] years, I've been concentrating on AI [2:23] platforms and uh had the pleasure of [2:26] getting us into the cloud years ago when [2:28] that was the big move and getting off of [2:29] desktops and such. And that was when we [2:32] had a couple dozen data scientists and [2:34] now we have a few hundred and thousands [2:36] of users on our platform. And I'll tell [2:38] you more about our scale and our success [2:41] in a moment. [2:44] So uh a little bit more about uh what [2:46] Vanguard's uh operating landscape looks [2:49] like. Uh we need to obviously adhere to [2:52] financial regulation, global data [2:54] privacy laws and AI specific laws. Um [2:57] but more than that we hold ourselves to [2:59] an even higher standard. So there are um [3:03] there's nothing higher in in our um uh [3:06] principles than just doing right by our [3:09] clients and making sure that the client [3:11] has the best chance of investment [3:13] success. And so that's really what [3:15] drives us. We want to make sure that [3:17] whatever we're doing with AI, it [3:18] benefits the shareholder, it benefits [3:21] our clients, and it certainly doesn't [3:23] harm them in any way. So that is above [3:25] and beyond what we have to do. It's what [3:28] we strive to do as part of our mission. [3:31] So, uh I'm gonna skip ahead to the next. [3:35] And so, where we're at in terms of our [3:36] journey, we uh as I mentioned, we have [3:39] over thousand a few thousand really. We [3:42] have uh just recently enabled all of our [3:44] uh engineers and application developers [3:47] to start to uh experiment with [3:49] generative AI as well as um agent [3:52] technology. Uh we have about 150 [3:55] probably 160. We're moving so fast you [3:58] know since I put this slide together uh [4:00] in terms of uh having agents under [4:02] experimentation and we have about a [4:03] dozen that are in production today. We [4:06] have about 70 production generative AI [4:09] uh solutions that are already uh [4:11] generating value for us responsibly in [4:13] production and just last year we [4:16] generated over $150 million of value and [4:19] that's measured internally in terms of [4:20] efficiency or better um overall outcomes [4:24] for our clients uh in terms of of how we [4:27] operate internally not not in terms of [4:29] just uh returns on investment. That's a [4:31] it's much different number. [4:37] Okay. Um so let's talk a little bit [4:38] about a strategy on how we're driving [4:40] the value. [4:42] So um I played hockey and so I I [4:46] couldn't help but go with a metaphor [4:47] with hockey, right? So just like in [4:49] hockey, you have to take shots in order [4:51] to score goals. So with AI and with AI [4:55] agents, you have to take shots and if [4:57] you want to score some goals, right? So [5:01] the thing is, you know, if you're not [5:02] shooting from the right side of the ice [5:04] or you're using the wrong tool or, you [5:06] know, maybe you're blindfolded, you're [5:08] not going to score many goals, right? So [5:10] what I want to also talk about is, you [5:13] know, selecting the right use cases and [5:15] how you can really increase your odds of [5:18] doing things safely and derive value. [5:22] So u a metaphor and a metaphor wrapped [5:25] in an enigma. So, a rising tide raises [5:28] all ships, right? So, that's a common [5:30] phrase that basically just says, you [5:32] know, essentially there's a a tide that [5:35] if you can catch it, it's going to bring [5:36] your ship up with everyone else. And [5:38] there are use cases that are like that [5:39] that we all know about, right? There's [5:41] office productivity tools. So, I'm just [5:43] going to do a quick right raise of hands [5:45] again. How many people are actually [5:47] using some type of co-pilot daily at the [5:50] office? Right. Good number. Not everyone [5:53] though, but that's exactly what I'm [5:55] talking about because everyone will and [5:58] certainly those that have raised their [6:00] hand, you know how much more productive [6:02] it makes you. Um it makes me much [6:04] happier. It's so great to go into [6:06] co-pilot just have it give me a summary, [6:08] right? So that's wonderful. Everybody [6:09] needs to do these types of things and [6:11] that's actually where it's a great place [6:13] to start. It's where we started to a [6:16] large extent where it's let's worry [6:18] about getting everybody the productivity [6:20] tools that they need that are lower on [6:22] the risk side of things but are [6:24] certainly going to be a benefit uh for [6:26] everyone and uh you can start to really [6:28] exercise those governance muscles that [6:31] you need to go through to get certain [6:33] things approved in a regul regulated [6:35] company. We had to bring in privacy, [6:38] legal, [6:39] uh, AI governance, data governance, all [6:42] of these different entities to kind of [6:44] get them to the table to say yes before [6:47] we could go forward with such a a [6:49] process. Right? If we started on the [6:51] opposite end of the spectrum, I'm not [6:52] sure we would have gotten very far. So [6:54] that's where, you know, a moonshot comes [6:56] in where it really helps to first have [6:59] some kind of groundwork laid where you [7:01] can say, "All right, we've done this [7:02] before. We know the conversations. We [7:04] know the things we're looking for to get [7:05] to. Yes. But those moonshots are the [7:08] ones that are going to be extremely [7:09] difficult to achieve but be could be [7:11] really gamechanging. Right? So a chatbot [7:14] going to an external client which we're [7:17] experimenting with now and we're having [7:19] good success but it took us a while to [7:20] get to a point where we would feel [7:22] comfortable doing such a thing. And it [7:24] certainly took a while before our [7:26] governance and uh data and other [7:28] defensive-minded folks were thinking to [7:31] get them to yes. It took quite a while [7:33] but that's why you know I want to make [7:35] that distinction between u you know [7:37] rising tide type opportunity and [7:40] moonshots [7:42] and similarly right when you start to [7:44] think about risk going up when you're [7:46] starting to go towards those moonshots [7:47] at the top there towards the right uh [7:50] you also need to start to think about [7:52] when you're deploying agents the level [7:53] of autonomy of the agent. So there's a a [7:58] scale that we uh have adopted. Um [8:01] there's a reference at the bottom. It's [8:02] actually from a Forbes article. There's [8:04] many frameworks out there in terms of [8:07] how you can think about the level of [8:09] autonomy of your agent. And this is one [8:12] that kind of just fit for us. We adapted [8:14] it for our purposes and and our context. [8:17] But when you think about going up that [8:19] level of autonomy scale and getting more [8:21] and more autonomous, [8:23] uh, you increase that level of risk and [8:25] complexity and you're going to have [8:27] deeper conversations with all of those [8:29] governance related folks and you should [8:31] before you start to deploy such an [8:32] agent. [8:37] Other aspects of what we take a look at [8:40] and what we refer to as a trusted AI [8:41] system. Um, just a few. Um, some of [8:44] these come from NIST, right? uh that [8:46] might even be really similar. Um but [8:49] this is more of a reference but uh some [8:51] other aspects that we try to assess our [8:53] systems by to ensure that they meet [8:54] these uh principles. [8:59] All right. So to finish up the strategy [9:01] section, um it's all about selecting, [9:04] right? So I wanted to kind of just start [9:06] there. How do you select the right use [9:07] cases? Like what is your organization [9:09] ready for? um be really thoughtful about [9:13] that so that you can get your [9:14] defenseoriented [9:16] um organizations internally to yes. [9:20] Once you get there and once you figure [9:23] out, okay, this is what gets everybody [9:25] yes, you're going to need to architect a [9:26] solution that gets you to a minimally [9:28] viable and compliant solution. And you [9:31] want to get to a point where you got [9:33] everybody understanding exactly who are [9:36] you going to be exposing to this [9:37] solution um in what geographical region, [9:40] right? You might want to pair it down to [9:41] a certain low number of people uh just [9:44] to kind of further mitigate the risk of [9:46] getting that first use case out there or [9:48] second or even hundth use case. We have [9:50] this conversation often. Uh but [9:53] essentially how can you get uh a [9:55] solution that is architected with the [9:57] controls necessary to get you into an [9:59] experiment. And once you get to that [10:01] experiment, as I mentioned, choose a [10:03] pilot, set your parameters, maybe a [10:05] geographical region, certain number of [10:07] people, etc. Measure the impact, gather [10:10] some feedback, iterate, um, and then [10:13] really do a a riskreward, right? Take a [10:16] look at the benefits. Is it worth the [10:18] risk that's residual? after you've [10:20] designed the system, after you've gotten [10:21] the client feedback and seen how it's [10:23] actually made an impact, um are you [10:26] ready to move further with that [10:27] experiment and possibly scale it? So, [10:30] not every use case is uh [10:34] candidate to be scaled further into an [10:36] enterprise platform, but many we found [10:39] can be. And so that's when you get to [10:42] that phase and you're realizing, okay, [10:43] this can benefit obviously more [10:45] geographical regions or other lines of [10:48] business um or other use cases that are, [10:51] you know, very uh similar to the one [10:52] that you you just worked on. That's [10:54] where we start to um build out [10:56] enterprise solutions and also bolster [10:59] the enterprise platforms to be able to [11:01] support those other requirements and [11:04] controls that are necessary to scale it [11:06] further. [11:09] Okay, so talked about the strategy. Now, [11:12] how do you execute it? [11:15] Okay, so you definitely want to um think [11:19] about starting with lower uh risk and [11:24] complexity, right? But you want to [11:26] enable democratization of this [11:28] eventually. And that's where this this [11:30] is kind of uh framed, right? We want to [11:33] enable developers who may not have a [11:35] deep data science background to execute [11:39] with agents um drive value responsibly. [11:44] So create platforms uh that enable those [11:46] use cases, enable those um lower risk [11:49] and complexity u value opportunities. [11:52] But as you start to creep up in that [11:54] risk and complexity, you need to [11:55] introduce some expertise. So you'll [11:58] notice in the uh top left and bottom [12:00] right boxes there's an AI center of [12:02] excellence that's introduced there. Um [12:05] you'll also notice the shading where [12:07] it's kind of you know a different color [12:09] than the other. And that's because you [12:11] most likely you you're probably uh if [12:14] your organization has data scientists [12:16] you probably don't have a ton of them. [12:18] If you do have some of them you're lucky [12:20] of course but you probably have more [12:22] developers than you do have data [12:24] scientists. And so that's why [12:25] democratization is important, of course. [12:28] And so these numbers are, you know, just [12:30] meant to be illustrative. They're not [12:31] meant to be hard and fast, obviously, [12:33] but we're aiming to try to democratize [12:36] to a point where you can have product [12:38] teams be as self-sufficient as possible. [12:41] Sometimes with an AI center of [12:42] excellence, helping them. Um and then uh [12:46] basically for the the ones that are [12:48] going to be the most complex and most [12:50] risky, we're uh putting with data [12:53] scientists to try to drive those forward [12:55] uh deal with all the complexities and [12:57] and risks that are inherent to them and [12:59] really try to drive innovation with [13:01] those uh higher risk use cases. [13:06] All right, last slide. So um [13:10] for those that were looking for a deep [13:12] deep dive into architecture, we're not [13:13] going to get there today. Sorry, but [13:15] happy to talk more offline and uh you [13:17] I'll share some information so we can [13:19] connect in LinkedIn. Uh but what I did [13:21] want to talk about are what are some of [13:23] the key capabilities and components that [13:25] you need to enable as part of uh your [13:28] overall architecture. And we're going to [13:29] start with kind of the what do the [13:31] actual impleers and developers see? And [13:34] [snorts] though I put the the words AI [13:36] engineer there, uh really that's meant [13:38] to be inclusive of anybody who is [13:40] developing AI. We know that AI engineer [13:42] is an emerging role and title in the [13:44] industry, but I I really mean this to be [13:46] inclusive of developers and data [13:48] scientists alike. And what you need to [13:50] think about is uh what do those [13:53] developers and I'm using that excl you [13:56] know inclusively need at design time. uh [13:59] they need things like agent frameworks [14:00] and the others that I have in that [14:02] column. But as they start to develop [14:04] things, they need strong evaluation [14:06] capabilities. And as they get [14:08] comfortable with what they've developed [14:09] and that it's performing the way they [14:11] like, well, now you need to deploy it. [14:13] And this is where the first introduction [14:15] of controls for production really is [14:18] important to introduce. So the CI/CD [14:21] process where you're gathering up um all [14:24] of the assets that need to be deployed [14:25] into production, you have the chance to [14:27] now check it for quality, check it for [14:29] uh the compliance with your various [14:31] controls. So CI/CD is incredibly [14:34] important as that gateway into [14:35] production. Once you get to production [14:38] and you're running it, you need tools to [14:40] make sure that it's doing what you [14:41] expected it to do. And of course, you [14:43] need to have evaluation and monitoring [14:44] components and uh the others that you [14:47] find in that column. [14:49] Now that's all kind of on the front end [14:51] and that is part of the um of the [14:55] overall platform but there's also some [14:57] underlying key um foundational agentic [15:00] and uh governance components that you [15:03] need to enable as part of the platform. [15:05] And so the one that most will come to [15:08] mind immediately for mo for folks is uh [15:10] you know models and having access to [15:12] many of them. Our data scientists and [15:14] engineers are demanding new models every [15:16] day. And of course, they're emerging [15:18] through multiple clouds. And so we need [15:20] to enable that connectivity and that [15:22] same safety across multiple clouds so [15:24] that you can take advantage of all the [15:26] um developments of the large language [15:28] models. And uh as they leaprog one [15:30] another, you need to constantly give [15:32] them uh a chance at your use case. So uh [15:36] we definitely need to um keep on our [15:39] toes there and we're constantly adding [15:40] more models every day. [15:42] uh model ops is an older term uh but [15:45] it's uh something that uh started with [15:48] traditional a IML and getting those [15:51] types of models and um uh their [15:54] components into production and deployed [15:55] safely and we're now tailoring our CI/CD [15:58] processes to work specifically with [16:00] agentic and generative AI components [16:03] and uh we also need of course AI and [16:06] model governance and so this is [16:07] something that um is it's an emerging uh [16:12] field, right? We've we've looked at [16:14] vendors um but we ultimately needed to [16:16] build custom to have an AI uh inventory. [16:20] So that's inclusive of all the models [16:21] that we have as well as our um uh [16:25] generative AI use cases uh third party [16:28] models etc. And you need to know how [16:30] they're being used. They need to be [16:32] model risk assessed and uh the use case [16:34] needs to be risk assessed that needs to [16:36] be revisited. All of that workflow needs [16:38] to be built out and facilitated so that [16:41] you have the right folks approving your [16:42] use cases. uh and uh there's uh also the [16:47] concept of uh validation of models and [16:50] use cases where you're getting third [16:51] parties involved although they're third [16:53] party that are internal to us but you [16:55] need to facilitate all of those [16:56] workflows internally and uh we have [16:58] decided to actually build that custom [17:02] uh of course APIs and integration are [17:04] important u especially with agents [17:07] because of course tools uh I don't know [17:09] what the the breakdown is between agents [17:11] calling other agents and agents calling [17:14] other models versus agents calling uh [17:16] APIs. But certainly that's a key [17:19] component of the architecture to enable [17:21] that connectivity and the security [17:23] management uh so that you can [17:25] essentially uh make sure that your [17:27] agents are only getting access to the [17:30] right things that they should get access [17:31] to. That becomes really important as [17:34] they become more autonomous. Right? And [17:37] then to kind of wrap up here, uh the [17:39] last two I'm going to lump together, you [17:41] know, the data layer. um super important [17:44] of course right uh it's if you're [17:46] feeding garbage in you get garbage out [17:48] right the old adage but data products is [17:50] something I really want to highlight [17:52] because data products are what really [17:54] enable that uh that assurance that the [17:58] data you're using has high quality it [18:00] has the metadata the lineage that you [18:02] need to know that you're using the right [18:03] data [18:07] okay so with that I just want to say [18:08] thanks and uh you know I do want to uh [18:11] just kind of finish with a little bit of [18:13] what we're hearing this week because you [18:14] heard me weave in responsibility so [18:16] much. So, I'm going to paraphrase a [18:18] little bit of what we heard in the [18:19] opening uh keynote, you know, where we [18:22] all can play a role in deploying this [18:25] technology responsibly within our [18:27] companies. We can also play a role in [18:29] making sure that uh humanity in general [18:32] is using it responsibly. So, I think [18:34] that's an important takeaway for us all [18:36] this week. So, thank you for your time [18:37] and uh I look forward to connecting. [18:40] Thanks. [18:41] >> [applause]