Manulife Automates Model Validation to Chase $1B AI Value Target

The AI Factory: How Frictionless AI Governance Powers Faster, Safer AI Delivery — Ai4

Ai4

Treating baseline architectures as reusable systems and shifting automated testing to first-line developers eliminates the validation backlog that consumes 70% of enterprise AI timelines.

AI-assisted summary of The AI Factory: How Frictionless AI Governance Powers Faster, Safer AI Delivery — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by Ai4 on August 17, 2026, this presentation features Dave Trier, CEO of ModelOp, and Shone Mousseiri, Head of Global AI Model Validation and Governance at Manulife. The session explores how enterprises can escape the dilemma between unconstrained 'Wild West' experimentation and stifling bureaucracy by adopting an 'AI Factory' operating model that embeds frictionless governance into the delivery lifecycle.

## TL;DR To achieve a mandate of $1 billion in incremental business value by 2027 while navigating tightening global regulations, Manulife restructured its AI delivery model around the principles of the Toyota Production System and Formula 1 racing. Recognizing that testing consumed 70% of project timelines and that risk concentrated in only 20% of models, the organization shifted testing upstream to first-line developers using standardized, reusable validation suites and an expedited pathway for low-risk use cases. By adopting ModelOp as an enterprise system of record, Manulife decoupled governance capacity from headcount, turning compliance from a delivery bottleneck into an automated assembly line.

## Atmosphere & style The presentation is crisp, authoritative, and grounded in pragmatic systems engineering rather than speculative AI hype. Dave Trier establishes a high-tempo, industrial tone using Formula 1 and assembly-line analogies, which Shone Mousseiri complements with an honest, detailed account of scaling governance from a one-person team to an enterprise serving tens of thousands of users. The exchange feels like an operational masterclass, delivering a steady, reassuring narrative of how disciplined processes can transform regulatory compliance from an obstacle into a competitive accelerator.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) True operationalization of AI requires a cross-functional, synchronized delivery engine akin to a Formula 1 team rather than isolated high-performing models. - [01:49](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=109s) Enterprises scaling AI typically get trapped in one of two extremes: uncontrolled 'Wild West AI' leading to operational debt, or rigid bureaucratic paralysis that halts innovation. - [01:49](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=109s) The only sustainable path forward is 'Industrialized AI,' where governance is embedded directly into delivery pipelines across cloud, open-source, and agentic architectures. - [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) Manulife targeted $1 billion in incremental business value from AI by 2027, driven by democratizing development to over 30,000 employees using Copilot and autonomous agents. - [05:45](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=345s) AI risk follows a Pareto distribution where roughly 20% of models account for the vast majority of organizational risk, necessitating an overhaul of traditional one-size-fits-all validation. - [05:45](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=345s) Validating baseline AI systems once and focusing downstream reviews strictly on incremental changes eliminates repetitive governance bottlenecks. - [07:05](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=425s) Adopting the Toyota Production System, Manulife structured its AI Factory into three synchronized divisions: AI Research (blueprints), AI Development (business customization), and AI Validation (quality control). - [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) Building modern AI applications accounts for only 10% of total lifecycle effort, while verifying safety, bias, and performance consumes up to 70% of the timeline. - [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) Shifting validation left by providing developers with reusable testing suites—such as counterfactual bias auditing and red-teaming libraries—removes the testing bottleneck before formal review. - [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) The AI Factory operates on four core pillars: patterns over one-offs, guidance proportional to risk, reusable testing, and a connected system of record. - [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) Using a unified system of record like ModelOp connects intake, risk assessment, validation, and post-production monitoring into a single automated control plane. - [14:58](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=898s) Industrializing delivery through standardized assembly lines allows enterprises to enforce regulatory compliance across diverse teams while accelerating time-to-value.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) The Formula 1 Analogy for Enterprise AI

Dave Trier opens the session by taking an informal poll of the audience, asking how many Formula 1 fans are in the room. Surprised by the large show of hands, he connects the sport directly to the challenge of operationalizing technology. In Formula 1, championship-winning performance is never determined solely by having the fastest car or the most talented driver; it depends on hundreds of team members working behind the scenes. Mechanics, engineers, statisticians, track operators, and pit crews operate inside a tightly synchronized, disciplined system. Every single lap is instrumented, every component is stress-tested, and every workflow is continuously refined to shave off a tenth of a second without ever compromising safety or build quality.

That relentless coordination is the definition of true industrialization, and Trier argues that enterprise AI requires the exact same operational mindset. Succeeding at AI is not about training one standout model or running a few flashy pilots in isolated silos. It requires creating a repeatable, cross-functional delivery machine that aligns data engineering, validation, legal, compliance, risk management, and enterprise architecture around a single objective: moving AI solutions from raw concept to live production rapidly, safely, and at scale.

[01:49](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=109s) Escaping the Binary: Wild West AI vs. Bureaucratic Paralysis

Introducing the collaborative session between ModelOp and Manulife, Trier frames the conversation around a practical case study from a 160-year-old global financial institution that created a near-frictionless path to production. Drawing on more than a decade of delivering enterprise machine learning, Trier observes that organizations attempting to scale AI almost always fall into one of two damaging extremes.

On one end of the spectrum is "Wild West AI." In this environment, disparate teams race ahead using whatever tools and ad-hoc processes they prefer. While leadership initially celebrates this phase as agile innovation, the lack of guardrails eventually catches up to the organization: operational incidents multiply, hidden risks compound, teams waste cycles on rework, and enterprise trust collapses. On the opposite end is bureaucratic paralysis, where risk teams react to uncertainty by erecting rigid gates, exhaustive manual assessments, and convoluted handoffs that grind delivery to a complete halt. Trier stresses that both paths destroy the enterprise value of AI. The only viable alternative is "Industrialized AI," an operating model where governance is not a bolted-on afterthought or an end-of-lifecycle hurdle, but is embedded directly into the delivery pipeline from ideation to production.

```mermaid flowchart TD accTitle: The AI Delivery Spectrum accDescr: Comparison between Wild West AI, Bureaucratic Paralysis, and Industrialized AI. subgraph Failures["Two Value-Destroying Extremes"] A["Wild West AI<br/>• Uncontrolled experimentation<br/>• Siloed tools and practices<br/>• Compounding risk and rework"] B["Bureaucratic Paralysis<br/>• Rigid manual checkpoints<br/>• Endless handoffs and reviews<br/>• Stifled enterprise innovation"] end subgraph Solution["The Third Path"] C["Industrialized AI Factory<br/>• Governance embedded end-to-end<br/>• Reusable architectural patterns<br/>• Scalable, safe delivery to production"] end A -.->|Erodes Trust| C B -.->|Kills Velocity| C ```

- **Wild West AI Outcome:** Accelerated early experimentation that quickly degrades into compliance incidents, technical debt, and eroded stakeholder trust. - **Bureaucratic Gridlock Outcome:** Risk-averse, manual oversight with multiple handoffs that slows cycle times and halts innovation. - **Industrialized AI:** Automated, embedded governance supporting any deployment paradigm—open-source, proprietary vendor solutions, cloud-native models, multi-agent frameworks, or Model Context Protocol (MCP) integrations.

[03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) Scaling Manulife's AI Footprint and Global Regulatory Pressures

Shone Mousseiri takes over to present Manulife’s journey. Manulife was recently recognized by Evident AI as the number-one life insurance company globally for AI maturity. Building on that foundation, the firm established an aggressive mandate: generating $1 billion in incremental business value from AI by 2027. Meeting that objective requires transforming a legacy 160-year-old financial institution into a fully AI-powered company by expanding AI adoption and democratizing development capabilities far beyond central data science teams.

Mousseiri reflects on his personal trajectory at Manulife. Five years prior, he operated as a "team of one," tasked with establishing the global AI model validation and governance function from scratch. That practice initially grew to support roughly 300 professional data scientists who were primarily engineering traditional predictive machine learning models. Today, the internal ecosystem has exploded: more than 3,000 employees hold Microsoft Copilot Studio licenses to build autonomous agents, and over 30,000 employees have Copilot premium access, giving them the ability to build and publish AI agents across the enterprise.

This internal democratization coincided with an unprecedented wave of global regulatory scrutiny across Manulife’s core markets: - **Asia-Pacific (APAC):** Rapidly evolving regulatory mandates from mainland China, strict new AI laws in Vietnam, and comprehensive artificial intelligence guidance from the Monetary Authority of Singapore (MAS). - **Canada:** Rigorous compliance expectations enforced by the Office of the Superintendent of Financial Institutions (OSFI) alongside provincial privacy and algorithmic regulations in Quebec. - **United States:** A complex patchwork of emergent state-level AI risk frameworks and insurance underwriting guidelines.

Mousseiri explains that the central bottleneck shifted overnight: the problem was no longer figuring out how to build models, but whether the firm possessed an operating model capable of governing that massive influx of demand without collapsing under manual review overhead.

[05:45](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=345s) Uncovering Governance Frictions: Risk Concentration and Systemic Patterns

Confronting this surge in activity forced the AI governance team to ask a tough question: were they deploying their specialized validation resources where they actually moved the needle? The candid answer was no. The team diagnosed two major operational friction points that were undermining governance throughput.

The first realization was that AI risk follows an extreme Pareto distribution. Roughly 20% of the models across Manulife’s portfolio generate the vast majority of organizational risk. These high-impact systems sit primarily in core underwriting decisions, human resources talent screening, and external, customer-facing interfaces. The remaining 80% of models presented comparatively negligible operational or reputational exposure, yet were being subjected to disproportionately heavy validation cycles. To eliminate this mismatch, the team overhauled their risk-tiering methodology to direct deep qualitative rigor exclusively toward high-stakes systems.

The second friction point lay in reviewing use cases in total isolation. Initially, validation teams treated every incoming application as a novel artifact. In practice, hundreds of disparate business requests share identical underlying architectures, prompt pipelines, and foundation models. Mousseiri realized that rather than repeatedly re-evaluating baseline model architectures, validation teams needed to govern the overarching "AI system" once and focus downstream evaluations strictly on the incremental delta introduced by each specific business implementation.

[07:05](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=425s) The Toyota Production System Blueprint: Structuring the AI Factory

To solve these governance bottlenecks, Manulife looked outside finance to the automotive industry, specifically the Toyota Production System (TPS). Toyota is celebrated not just for building dependable vehicles, but for designing a manufacturing discipline capable of turning out defect-free cars at massive scale. Translating this philosophy to software, Manulife structured its AI Factory across three distinct, highly synchronized units: AI Research, AI Development, and AI Validation.

```mermaid flowchart LR accTitle: The AI Factory Operational Structure accDescr: Three-stage production flow modeled after the Toyota Production System. subgraph Research["1. AI Research (Engineering)"] R1["Blueprints & Architecture"] R2["Concept Prototypes"] end subgraph Dev["2. AI Development (Assembly)"] D1["Embedded Line Teams"] D2["Domain Customization"] end subgraph Val["3. AI Validation (Quality Control)"] V1["Safety & Crash Testing"] V2["Compliance Sign-off"] end Research -->|Reusable Assets| Dev Dev -->|Production Candidates| Val Val -.->|Feedback Loops| Research ```

AI Research operates like automotive advanced engineering, designing blueprints, baseline reference architectures, and next-generation concept solutions. AI Development teams are embedded directly inside distinct business units; they take those central blueprints and tailor them to domain-specific business logic. Finally, Mousseiri’s AI Validation practice functions as the factory's quality control and safety testing arm—analogous to automotive crash tests, emissions inspections, and mechanical stress tests—ensuring compliance and reliability before any model goes into service.

Mousseiri illustrates this with their Intelligent Document Processing (IDP) platform. The enterprise uses IDP models to extract and ingest text from medical claims, pharmacy receipts, and dental claim forms across diverse regional products. Rather than treating each form or product line as an independent model requiring de novo governance intake and review, the team registers and evaluates the core IDP engine as a single governed AI system. Subsequent business deployments inherit that baseline clearance, allowing reviewers to evaluate only form-specific variations.

[09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) Lightweight Validation and the 70% Testing Bottleneck

Drawing another parallel to Formula 1 engineering—where designers "challenge every single gram of weight" on the chassis—the governance practice began challenging every single question in their historical intake and validation questionnaires. The goal was to strip away administrative bloat and make approval as lightweight as possible for low-risk use cases. They formalized an "immaterial pathway," allowing low-risk internal automations to bypass standard review cycles entirely and proceed on a fast-track route to production.

At the same time, the economics of AI development were shifting dramatically. With low-code platforms and foundational APIs, building an initial model now accounts for only about 10% of total lifecycle effort. In contrast, verifying that the model behaves safely, accurately, and without bias absorbed 70% of the project timeline. Testing had become the primary enterprise bottleneck.

To dismantle this roadblock, Manulife shifted testing upstream, empowering first-line developers with pre-packaged, reusable validation suites: - **External Red-Teaming Suites:** Standardized libraries for adversarial testing and red-teaming that can be run against any customer-facing conversational agent, avoiding bespoke testing setups. - **Counterfactual Bias Testing:** Algorithmic suites built to detect bias in human resources models that were adapted to audit automated underwriting algorithms without re-architecting test code.

By transforming validation from an opaque second-line review into reusable testing software, first-line developers gain the tools to identify and remediate flaws before ever submitting an application for formal sign-off.

[12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) The AI for All Strategy and the Four Pillars of Scale

This shift in testing responsibilities culminated in an initiative known as "AI for All." Manulife instituted role-based training programs tailored specifically for model users, business owners, and hands-on developers to elevate baseline AI literacy across the global organization. They introduced risk-proportional templates and embedded automated quality checks directly into the intake submission pipeline, instantly flagging malformed requests or incomplete artifacts.

By automating and delegating low-risk evaluations, the validation team redirected its deep technical scrutiny toward complex, medium- and high-risk models. Mousseiri emphasizes that this operating model decouples governance scale from headcount: the company can dramatically increase AI adoption while keeping governance overhead flat.

Mousseiri summarizes the AI Factory operating philosophy into four core pillars: 1. **Patterns Over One-Offs:** Abandon custom point reviews in favor of governing standardized, reusable architectural systems. 2. **Guidance Proportional to Risk:** Match the depth of oversight directly to real-world risk, streamlining immaterial applications via low-friction paths. 3. **Reusable Testing:** Package automated compliance, fairness, and red-teaming tests into libraries that first-line builders can execute independently. 4. **A Connected System of Record:** Anchor the entire lifecycle in a unified control plane.

Highlighting the fundamental rule that "you can't govern what you can't see," Mousseiri identifies ModelOp as Manulife's system of record. The platform ties intake, risk assessment, technical validation, operational approval, and continuous post-production monitoring into a single pane of glass, allowing governance to "stop being the brake and become the engine."

[14:58](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=898s) Industrializing the Enterprise: From Bespoke Silos to Assembly Line

Dave Trier takes the stage to close the presentation, reinforcing that moving beyond bespoke, repetitive development is the most urgent hurdle facing modern enterprises. In heavily regulated sectors like financial services and insurance, teams often default to familiar legacy processes, conducting redundant architectural and risk assessments for every minor deployment. Building an AI factory requires breaking that mold by identifying structural commonalities and standardizing them into an automated pipeline.

By deploying ModelOp as an underlying orchestration engine, organizations can integrate reusable assets, test suites, and compliance checks directly into the delivery path. Whether a use case is initiated by a senior data scientist writing custom deep-learning pipelines or a business user in marketing configuring an internal agent, it flows through the same governed assembly line. Enterprise stakeholders across risk, data, compliance, cybersecurity, legal, and IT can rest assured that every internal policy is enforced and every audit requirement is met—while simultaneously slashing the time required to turn an idea into business value.

## Notable quotes > [01:49](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=109s) Wild West AI — Dave Trier

> [01:49](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=109s) Industrialized AI — Dave Trier

> [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) team of one — Shone Mousseiri

> [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) challenge every single gram of weight — Shone Mousseiri

> [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) immaterial pathway — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) AI for All — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) Patterns Over One-Offs — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) Guidance Proportional to Risk — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) Reusable Testing — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) A Connected System of Record — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) you can't govern what you can't see — Shone Mousseiri

> [12:37](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=757s) stop being the brake and become the engine — Shone Mousseiri

## Mentioned Dave Trier, Shone Mousseiri, ModelOp, Manulife, Evident AI, Toyota Production System (TPS), Microsoft Copilot Studio, Microsoft Copilot, Monetary Authority of Singapore (MAS), Office of the Superintendent of Financial Institutions (OSFI), Formula 1, Model Context Protocol (MCP)

## Video description Shone Mousseiri - Head of Global AI Model Validation and Governance, Manulife Dave Trier - CEO, ModelOp

Enterprises feel stuck between two AI extremes: Wild West innovation that overruns budgets, or rigid processes that strangle innovation. What if a new path turned AI into a scalable "factory" that delivers use cases to production rapidly, repeatedly, and safely?

Facing a surge of AI demand across every function, Manulife built exactly that: an operating model for AI delivery they call the AI Factory. Research teams define reusable patterns, development teams adapt them to business needs, and validation runs quality control with feedback loops that continuously improve outcomes across the model lifecycle. Embedded AI governance streamlines the factory, enabling faster time-to-value, better reuse, and high-quality AI at scale.

This session defines what an AI Factory really is, walks through how reusable patterns and embedded governance keep delivery both fast and safe, and shows why it's the key to scaling AI to value instead of getting stuck between those two extremes.

If you're working to turn fragmented AI initiatives and processes into a governed, repeatable AI Factory that compounds into real value at scale, this is the session to join.

[Source](https://www.youtube.com/watch?v=Yz2tYB_f4uI)

中文简报

宏利推进模型验证自动化，冲刺10亿美元AI价值目标 通过将基准架构转化为可复用系统并将自动化测试前移至一线开发人员，消除消耗企业70% AI项目周期的验证积压。 为了在应对日益严格的全球监管的同时实现到2027年创造10亿美元增量业务价值的目标，宏利金融（Manulife）围绕丰田生产方式（Toyota Production System）和F1赛车的理念重构了其AI交付模式。鉴于测试环节占据了70%的项目周期，且风险仅集中在20%的模型中，该机构利用标准化、可复用的验证套件将测试工作前移给一线开发人员，并为低风险用例设立了快速通道。通过采用ModelOp作为企业级核心记录系统，宏利金融将治理能力与人员数量解耦，把合规流程从交付瓶颈转变为自动化流水线。 企业在规模化应用 AI 时常陷入无序创新导致的运营债务与过度审核引发的官僚瘫痪两大极端；破局关键在于借鉴 F1 团队协同与丰田生产方式，将端到端治理直接嵌入交付流水线，构建标准化的“AI 工厂”。 (1:49) 宏利金融（Manulife）设立了到 2027 年通过 AI 创造 10 亿美元增量商业价值的目标，正向超 3 万名员工普及 Copilot 和智能体构建能力，同时应对亚太、加拿大和美国日益严格的监管合规挑战。 (3:53) 鉴于约 20% 的高风险模型承载了绝大部分组织风险，宏利重构了治理体系：对共享底层 AI 系统进行一次性验证、下游仅审核业务增量差异，并为低风险内部自动化设立轻量化快速通道。 (5:45) 针对验证与安全测试占据整个交付周期 70% 的核心瓶颈，宏利将标准化红队测试与偏见审计工具封装成可复用套件并前置给一线开发者，依托 ModelOp 统一记录系统实现治理效率与规模扩张的脱钩。 (9:14)

Reading summary

Scaling enterprise AI requires replacing both unconstrained experimentation and bureaucratic gridlock with an industrialized operating model that embeds automated, risk-proportional governance directly into the delivery pipeline. Overcoming the Dual Traps of AI Delivery (1:49) According to ModelOp's Dave Trier, organizations scaling artificial intelligence typically succumb to one of two operational extremes. In 'Wild West AI,' disconnected teams deploy ad-hoc tools without centralized guardrails, yielding rapid early experimentation that ultimately degrades into compliance incidents, technical debt, and eroded trust. Conversely, risk-averse organizations default to bureaucratic paralysis, introducing exhaustive manual gates and handoffs that stifle velocity. Trier asserts that enterprise value depends on treating AI delivery like an industrial assembly line or a Formula 1 team, where cross-functional groups—including data engineering, legal, compliance, risk, and architecture—operate within a tightly synchronized system. This industrialized approach embeds governance checks end-to-end, enabling organizations to deploy open-source models, proprietary systems, multi-agent frameworks, and cloud-native services safely at scale. Manulife's Democratization and Global Regulatory Pressures (3:53) Shone Mousseiri outlines how Manulife encountered these operational pressures while pursuing an enterprise mandate to generate $1 billion in incremental business value from AI by 2027. Five years prior, the firm's validation unit operated as a single-person function supporting roughly 300 traditional data scientists. That ecosystem expanded rapidly to over 3,000 employees licensed to build autonomous agents via Microsoft Copilot Studio and more than 30,000 workers with Copilot premium access. This rapid democratization intersected with rising regulatory scrutiny across Manulife's operating markets, including new mandates in mainland China and Vietnam, AI guidelines from the Monetary Authority of Singapore, rules from Canada's OSFI and Quebec provincial privacy authorities, and emerging state-level insurance guidelines across the United States. Consequently, the primary delivery challenge shifted from building algorithms to governing high-volume demand without overwhelming validation staff. Targeting Risk Concentration and Reusable Architectures (5:45) To address throughput limits, Manulife diagnosed two structural frictions in traditional validation. First, AI risk followed an extreme Pareto distribution: roughly 20% of models—primarily high-stakes systems in core underwriting, HR talent screening, and direct customer interactions—generated the vast majority of exposure, while the remaining 80% posed minimal risk. In response, the firm overhauled its intake to direct deep qualitative reviews toward high-risk systems while establishing an 'immaterial pathway' for low-risk automations to bypass standard review cycles. Second, validation teams shifted from reviewing isolated applications to governing underlying architectures once, adopting principles from the Toyota Production System. Manulife organized operations into AI Research (blueprints and baseline architectures), AI Development (line-of-business custom assembly), and AI Validation (safety, testing, and compliance sign-off). Under this model, foundational tools such as an Intelligent Document Processing (IDP) platform are validated once as a core system, permitting downstream business implementations to inherit approval and undergo scrutiny only for form-specific variations. Upstream Testing and Centralized Lifecycle Governance (9:14) Mousseiri explains that modern low-code tools and foundation models have inverted traditional development economics: creating an initial model accounts for roughly 10% of total lifecycle effort, whereas testing safety, fairness, and accuracy consumes 70% of the project timeline. To clear this validation bottleneck, Manulife shifted testing upstream, providing first-line developers with pre-packaged, reusable validation suites such as standardized external red-teaming libraries and counterfactual bias-testing modules. This architecture is underpinned by Manulife's 'AI for All' strategy, which combines role-based training with automated intake checks and four foundational pillars: prioritizing patterns over one-offs, applying guidance proportional to risk, distributing reusable testing code, and maintaining a connected system of record. Utilizing ModelOp as a central orchestration plane, the organization tracks intake, validation assessments, approvals, and post-production monitoring across both professional data science code and low-code employee automations, decoupling governance capacity from headcount growth. Industrializing enterprise AI allows organizations to enforce compliance and audit requirements across diverse development tiers without choking delivery velocity. However, this operating model assumes that distributed business units will consistently adopt central architectural blueprints and automated test suites rather than reverting to bespoke, unmonitored workflows.

精炼阅读摘要

ModelOp的Dave Trier与宏利金融（Manulife）的Shone Mousseiri指出，企业规模化应用AI普遍面临“野蛮放任”引发合规风险与“官僚关卡”阻碍交付的两难困境；宏利金融借鉴丰田生产方式构建“AI工厂”，通过分级治理、系统化复用与自动化测试左移，在严苛监管环境下实现了高效合规的规模化交付。 突破野蛮放任与官僚瘫痪的两极困境 (1:49) Dave Trier指出，企业在推进AI规模化落地时常陷入两种破坏价值的极端：“野蛮放任型AI”（Wild West AI）中各业务团队采用零散工具与临时流程快速试验，虽带来初期敏捷，但缺乏统一护栏最终会导致合规事件激增、隐性风险累积与信任崩塌；而“官僚瘫痪型AI”则由风控团队建立繁琐的手动检查点与多层交接，使交付节奏停滞。Trier以一级方程式赛车（Formula 1）背后数百人严密协同的工程体系作比，强调AI的成功不在于孤立训练出单个高性能模型，而在于打造跨越数据工程、验证、合规与风险管理的“工业化AI”交付机器，将治理直接内嵌于全生命周期流水线。 全球监管风暴与风险帕累托法则诊断 (3:53) 宏利金融被Evident AI评为全球AI成熟度领先的寿险公司，并设定了到2027年通过AI创造10亿美元增量商业价值的目标。Shone Mousseiri介绍，过去五年间，宏利金融的AI开发团队已从最初支持300名专业数据科学家，急剧扩展为拥有3,000多名Microsoft Copilot Studio智能体构建者及超30,000名Copilot premium授权员工。与此同时，亚太区（中国法规、越南新法及新加坡金管局指引）、加拿大（OSFI监管及魁北克隐私法规）与美国各州出台了密集复杂的AI与核保规范，传统人工审查模式难以维系。 治理团队复盘发现，模型风险呈现极端的二八定律：全组织仅约20%的模型（集中于核心承保、人力资源招聘筛选及直接面向客户的系统）产生绝大部分组织风险，其余80%的低风险模型却同样耗费了大量验证资源。此外，将每个业务请求作为全新对象进行孤立审查造成了严重的重复劳动，忽视了不同场景底层依赖的通用模型与架构系统。 借鉴丰田生产方式构建三级AI工厂 (7:05) 为破除交付瓶颈，宏利金融借鉴丰田生产方式（TPS）的大规模制造理念，将AI体系重构为AI研发（AI Research）、AI开发（AI Development）与AI验证（AI Validation）三个协同单元。AI研发部门如同先进工程机构，负责设计标准蓝图与概念原型；AI开发团队直接嵌入各个业务线，基于通用蓝图组装业务逻辑；Mousseiri负责的AI验证团队则承担质量控制与安全测试职责，类似于汽车出厂前的碰撞测试与排放检查。 以智能文档处理（IDP）平台为例，企业需从多地区的医疗理赔、药房单据与牙科表单中提取文本。治理团队不再针对每个表单或产品线进行从零开始的重复审查，而是将核心IDP引擎作为单一受治理的“AI系统”进行基线核准，下游业务部署直接继承该资质，验证团队仅需专注于具体表单引入的增量差异。 破解测试瓶颈与规模化治理四项支柱 (9:14) 随着低代码工具和基础API的普及，AI生命周期的工作量分配出现重大转变：构建初始模型仅占总工作量的10%，而验证其安全性、准确度与无偏见性却占用了70%的时间，测试成为主要瓶颈。宏利金融对此推行轻量级审查，剔除问卷冗余，设立免除标准审查流程的“无实质风险快速通道”；同时将测试工具软件化，为一线开发者提供标准化的外部红队对抗测试库与反事实偏见测试库，推动合规检验在提报前由开发者自主执行。 Mousseiri将该体系归纳为“AI for All”策略下的四大核心支柱：模式优于单点定制、按风险比例提供指导、测试资产可复用，以及建立统一的记录系统。宏利金融采用ModelOp作为中央控制底座，将进件、风险评估、技术验证、合规批准与生产监控整合在统一看板中，使治理职能摆脱对人员扩张的依赖，从业务阻力转化为交付引擎。 Dave Trier总结强调，面对强监管环境，企业必须摒弃对每个微小场景进行重复评审的传统惯性，利用底层编排系统将可复用资产、自动化测试与跨部门合规要求标准化为工业流水线，在确保审计要求与风险底线的同时缩短从构思到投产的交付周期。

Detailed notes — zh

## Background 本视频由 Ai4 于 2026 年 8 月 17 日在 YouTube 平台发布，记录了 ModelOp 首席执行官 Dave Trier 与宏利金融（Manulife）全球 AI 模型验证与治理负责人 Shone Mousseiri 的联合演讲。简介指出，企业常在野蛮试错与过度官僚化治理两个极端间陷入两难，本演讲旨在介绍宏利金融如何通过构建“AI 工厂”运作模式与嵌入式治理体系，实现 AI 规模化、高质且安全的敏捷交付。 ## TL;DR 演讲首先以 F1 赛车团队的精密协同为比喻，提出企业级 AI 必须摒弃“野蛮生长”与“过度官僚”两种失败模式，走安全与敏捷兼顾的工业化交付之路。随后，宏利金融分享了其冲刺 2027 年 10 亿美元 AI 业务价值的实战经验：借鉴丰田生产方式建立包含研究、开发与验证三大支柱的“AI 工厂”，将治理重心从孤立用例转向标准化 AI 系统。通过推行非实质性快速审批路径、沉淀可复用的红队与偏见测试资产，并以 ModelOp 作为全流程统一记录底座，宏利成功实现了治理能力随业务规模高效扩展。 ## Atmosphere & style 整场演讲风格干练务实且富有工业化工程的美感。开场 Dave Trier 借助扣人心弦的 F1 赛车细节调动全场氛围，随后 Shone 接棒以大型金融机构一线操盘手的沉稳与严谨娓娓道来。两位演讲者配合默契，不仅逻辑层层递进，更将宏观的合规治理战略与具象的测试复用机制深度结合，为观众呈现了一场兼具战略远见与落地实操性的企业级 AI 转型示范课。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) Dave Trier 以 F1 赛车车队的精密协同类比企业 AI 交付，强调企业成功不取决于单一模型，而在于跨数据、法务、合规与业务部门构建无摩擦、可重复的工业化体系。 - [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) 企业 AI 交付常陷入“野蛮生长导致事故与信任崩塌”或“过度官僚化扼杀创新”两大极端，破局关键在于引入全程原生嵌入的工业化治理路径。 - [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) 宏利金融（Manulife）设立了到 2027 年实现 10 亿美元 AI 业务价值的战略目标，全公司已有数万名员工深度接入 Copilot 智能体工具并推进平民化开发。 - [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) 宏利发现其模型资产库中仅约 20% 属于高风险模型（如核保、HR 与面向客户应用），过去一刀切的单点用例审查造成了巨大的流程摩擦与资源浪费。 - [07:05](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=425s) 借鉴汽车工业的丰田生产方式，宏利打造了“AI 工厂”体系，由制定标准化蓝图的 AI 研究团队、结合业务定制的开发团队以及严把质量合规的验证团队协同运作。 - [07:05](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=425s) 宏利以智能文档处理（IDP）为例，将分散用例管理转变为标准化“AI 系统”资产治理，后续同类场景接入仅需审核边际增量差异而无需全量重复评审。 - [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) 随着低代码/无代码工具普及，AI 项目仅 10% 时间用于构建，而 70% 时间消耗在测试验证上，测试流程的标准化与复用成为提效核心突破口。 - [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) 宏利推出了低风险模型的“非实质性路径（Immaterial Pathway）”极速审批通道，并集中构建可复用于所有外向型系统的红队测试库与公平性偏见测试库。 - [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) 通过“AI for All”赋能计划与自动化质检，将标准化测试工具前置给一线开发者自检，确保治理能力随 AI 规模线性扩展，而非依赖合规人员编制膨胀。 - [13:44](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=824s) 宏利提炼出 AI 工厂四大核心原则：模式优于孤例、风险对等指引、可复用测试资产，以及建立统一的记录系统底座。 - [13:44](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=824s) “你无法治理你看不到的东西”，企业落地 AI 工厂必须以建立统一的模型清单（Model Inventory）为起点，通过 ModelOp 这类平台将需求、验证、审批与监控全流程串联打通。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) F1 赛车比喻与企业 AI 交付的两种极端困境 Dave Trier 以一个现场互动开场，询问观众中有多少人是 Formula 1（一级方程式赛车）的粉丝。看着台下举起的手，他感叹人数远超预期。他指出，无论是否关注 F1，大家都能体会打造一支冠军车队背后的严密逻辑：在 F1 赛场上，胜利绝不仅仅取决于拥有一辆最好的赛车或最顶尖的赛车手，而是依赖于幕后数百名团队成员的高度协同与纪律性——机械师、工程师、统计学家、运维人员以及维修区工作人员各司其职。每一圈的成绩都要精准测量，每一个零部件都经过严苛测试，所有流程都在不断优化以剔除任何微小的阻力摩擦，"全都是为了在单圈成绩中抠出十分之一秒"，且绝不以牺牲整体质量或安全性为代价。这就是真正的“工业化”（Industrialized）。 企业 AI 交付亦是同理。在企业级 AI 中取胜，绝非仅仅依赖单一的最佳模型或几个成功的试点（Pilot），而是需要在数据、治理、业务、法务、风险、合规与架构等不同部门之间建立一套可重复的运行体系，共同将 AI 从构想快速、安全、规模化地推向生产环境。为了说明这一理念，Dave Trier 引入了一家拥有 160 多年历史的大型保险机构的真实案例。作为专注于通过嵌入式治理实现 AI 交付工业化的软件公司 ModelOp 的 CEO，Dave 强调本场演讲绝非空洞的概念炒作，而是直击落地交付的实战细节。 ```mermaid flowchart TD accTitle: 企业 AI 交付的三种路径对比 accDescription: 展示野蛮生长、过度官僚化治理与嵌入式工业化 AI 交付三种路径的特征与结果对比。 subgraph Extremes["两种失败极端"] A["野蛮生长 (Wild West AI)<br/>工具混杂、缺乏规范、快速试错<br/>👉 事故频发、风险累积、信任瓦解"] B["过度治理 (Over-Governance)<br/>人工评审繁琐、交接断层严重<br/>👉 流程臃肿、创新彻底停滞"] end subgraph ThirdWay["破局之道"] C["工业化 AI (Industrialized AI)<br/>治理全程嵌入开发与交付流程<br/>👉 兼顾速度、责任与规模化价值"] end A -.->|"两极俱毁价值"| C B -.->|"两极俱毁价值"| C ``` 在走访众多大型企业客户的十余年 AI/ML 交付历程中，Dave 观察到企业往往陷入两个极端的误区： - **野蛮生长模式（Wild West AI）**：各团队各自为政，使用五花八门的技术栈和流程，表面上敏捷且上线迅速，但随之而来的是事故频现、风险暗积、大量推倒重来，最终导致业务信任彻底崩塌。 - **过度官僚化模式（Over-Governance）**：企业设置了过于繁复的人工评估、多重交接与严苛审核，直接将创新速度扼杀在摇篮之中。 - **工业化 AI 路径（第三条道路）**：治理不再是事后补救或外部硬塞的阻碍，而是从创意到投产全程原生嵌入（Embedded Governance）。无论是开源模型、专有模型、云端服务还是新兴的智能体（Agent）与 MCP 架构，都能兼顾敏捷交付与负责任的安全底线。 --- ### [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) 宏利金融的 AI 雄心与治理摩擦点 宏利金融（Manulife）全球 AI 模型验证与治理负责人 Shone Mousseiri 随后登台分享了宏利的实际转型经历。宏利近期被 Evident AI 评为全球 AI 成熟度第一的寿险公司，并设立了极为宏大的战略目标：**到 2027 年实现 10 亿美元的 AI 业务价值**。为了将这家拥有 150 多年历史的传统保险巨头转型为 AI 驱动型企业，必须在全公司范围内推动 AI 深度采纳与开发平民化（Democratization）。 回顾团队演进，Shone 提到自己五年前刚加入宏利时还是“光杆司令”（Team of one），从零搭建全球模型验证与治理实践；随着时间推移，团队逐步壮大以支持当时 300 多名以传统机器学习为主的数据科学家。而如今，内部已有超过 3,000 名员工持有 Copilot Studio 许可证，另有 30,000 多名员工拥有 Copilot Premium 权限，员工可以自主构建 Copilot 智能体并向全组织发布。与此同时，宏利在全球各运营区域面临着剧增的合规监管压力：亚太地区包括中国大陆的新规指引、越南的新法案、新加坡金管局（MAS）指导原则；北美地区则需遵循加拿大金融机构监管局（OSFI）与魁北克省的规定，以及美国各州的合规要求。企业面临的核心挑战已从“如何构建 AI”转变为“如何构建一套能够承载如此庞大野心的规模化治理体系”。 面对快速膨胀的需求，治理团队直面了一个严峻的问题：团队是否真正将有限的资源投入到了最关键的刀刃上？答案是否定的。深入审视后，他们锁定了两大核心阻力与摩擦点： - **AI 风险高度集中**：模型资产库中大约只有 20% 的模型属于高风险类别（例如涉及核保 Underwriting、人力资源 HR、以及面向客户或外部交互的应用）。过去一刀切的审核方式耗费了过多精力，亟需重构风险分级（Risk Tiering）方法论。 - **孤立用例与重复评审的摩擦**：早期治理按单个“用例”（Use Case）逐一审批，但许多用例在技术底层共享相似架构与相同大语言模型（LLM）。治理团队无需每次都从零重复审查，而应聚焦于增量差异（Incremental Differences），将治理重心下沉到标准化的“AI 系统”层面。 --- ### [07:05](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=425s) 丰田生产方式与“AI 工厂”的三大支柱 为了化解治理瓶颈，宏利引入了“AI 工厂”（AI Factory）的概念。Shone 将其类比为汽车工业中的丰田生产方式（Toyota Production System）——丰田之所以闻名世界，不仅在于其造出了高可靠性的汽车，更在于其创造了一套能够持续、规模化生产高质量汽车的生产制造系统。 ```mermaid flowchart TD accTitle: AI 工厂的三大协同支柱 accDescription: 展示研究团队制定蓝图、开发团队业务定制、验证团队质量把关与反馈的协同架构。 subgraph AIFactory["宏利 AI 工厂运行架构"] direction TB Research["1. AI 研究团队 (Research)<br/>• 核心数据科学专家<br/>• 绘制概念车蓝图<br/>• 沉淀可复用架构范式"] Dev["2. AI 开发团队 (Development)<br/>• 嵌入各业务职能部门<br/>• 结合业务场景定制适配<br/>• 利用低代码/无代码工具"] Validation["3. AI 验证团队 (Validation)<br/>• 充当 QA 与质量控制<br/>• 安全/碰撞/排放测试类比<br/>• 制定准入标准与测试库"] end Research -->|"交付标准蓝图与组件"| Dev Dev -->|"提交轻量化差异验证"| Validation Validation -.->|"质量反馈与持续优化"| Research Validation -.->|"前置自动化测试库"| Dev ``` 宏利的 AI 工厂架构由三个核心团队紧密协同构成： 1. **AI 研究团队（Research Team）**：汇聚核心数据科学研发力量，类似汽车研发中心，负责攻坚下一代“概念车”，绘制标准化的技术蓝图与架构解法。 2. **AI 开发团队（Development Teams）**：直接嵌入各业务部门，深度理解具体的业务诉求，基于研究团队提供的标准蓝图进行业务场景的定制化开发。 3. **AI 验证团队（Validation Team）**：即 Shone 带领的团队，定位为类似汽车工业的质量检验（QA）部门，负责把控安全测试、碰撞测试与排放测试，确保模型在正式上线前完成对等充分的验证。 Shone 以**智能文档处理（Intelligent Document Processing, IDP）**系统为例进行了具体说明：宏利在日常运营中需要处理海量的医疗理赔单、发票收据和牙科申请表。这些处理逻辑与底层架构在多个产品和表单之间具有高度通用性。在 AI 工厂模式下，治理团队无需对每一个表单的接入都重复进行全套评审，而是将它们统一挂接并注册到同一个标准化的“AI 系统”资产下，仅审核其边际增量，实现了从分散用例管理向系统化治理的跃迁。 --- ### [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) 轻量化治理与复用型测试赋能一线 为了践行 F1 赛车“削减每一克多余重量”的精神，宏利重构了模型风险分级体系，特别推出了**“非实质性路径”（Immaterial Pathway）**，为低风险模型开辟了极速审批通道。Shone 强调，要实现 10 亿美元的价值目标，不可能依赖人工去逐个定制 10,000 个孤立解法，必须通过可复用能力赋能第一道防线（一线业务与开发人员）。 随着低代码/无代码（Low-Code/No-Code）工具的普及，构建模型的门槛大幅降低。在当前的交付周期中，仅有约 10% 的时间花费在模型构建上，而高达 70% 的时间被消耗在测试与验证环节。因此，优化测试流程成为提升整体效能的最大突破口： - **红队测试库复用**：针对面向客户（Customer-facing）的外部交互应用，集中构建一套标准化的红队测试（Red Teaming）资产库，一旦建立即可直接复用于所有同类外向型系统，无需针对单一场景重复造轮子。 - **反事实偏见测试库**：在审查人力资源（HR）模型时开发的反事实偏见（Counterfactual Bias）测试工具，被直接迁移复用至核保（Underwriting）模型的公平性评估中。 - **“AI for All” 全员赋能体系**：面向模型使用者、所有者与开发者推行基于角色的定制培训，将标准化测试工具与指引前置给一线团队，使其在进入正式合规审查前即可自行开展风险排查。 - **风险对等模板与自动化质检**：开发与风险等级严格对等的标准化模板，并在提交审批流程中内置自动化质量检查，大幅压缩人工干预。 通过这一系列举措，治理团队成功将注意力从大量低风险用例中解放出来，将高规格的审查算力精准聚焦在真正具有中高风险的核心业务场景上，实现了“治理能力随 AI 采纳规模线性扩展，而非随治理团队人员编制被动膨胀”。 --- ### [13:44](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=824s) AI 工厂的四大核心原则与 ModelOp 平台底座 Shone 总结道：“未来的优势不属于构建最多孤立 AI 系统的组织，而属于构建了最高效 AI 工厂的组织——那些能够以 AI 扩张速度同频扩张信任的企业。”宏利的实践可以提炼为四大核心原则： 1. **模式优于孤例（Patterns over one-offs）**：优先抽象可复用架构范式，拒绝一次性重复建设。 2. **风险对等指引（Guidance in proportion to risk）**：依据模型风险级别实施梯度治理，低风险走快车道，高风险严把关。 3. **可复用测试资产（Reusable testing）**：沉淀红队测试、偏见检测等通用测试库，赋能一线前置自检。 4. **统一记录系统（A system of record that connects everything）**：建立统一的系统底座，连接所有资产与流程。 Shone 强调，如果参会者今天只能带走一条建议，那就是**“必须从模型清单（Model Inventory）开始做起——你无法治理你看不见的东西”**。宏利选择 ModelOp 作为基础平台底座，将 AI 需求的 intake（需求接入）、风险评估、验证、审批与持续监控无缝串联在单一平台上，使治理从创新的“刹车片”彻底转变为驱动业务快跑的“发动机”。 Dave Trier 随后进行收尾总结。他呼吁广大金融与保险机构打破固步自封的传统审批惯性，借助 ModelOp 这类底座引擎将孤立零散的重复性工作转化为标准化流水线。无论构想来自资深数据科学家还是市场营销、后勤职能部门的平民开发者，所有 AI 项目都能穿透这套系统，全面覆盖数据、法务、合规、IT 与业务的全部合规策略，在确保稳健安全的同时，最大化缩短业务价值变现周期（Time-to-Value）。最后，Dave 邀请对 AI 工厂架构感兴趣的参会者前往 ModelOp 的 1515 号展位做进一步交流。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) 全都是为了在单圈成绩中抠出十分之一秒，且绝不以牺牲整体质量或安全性为代价。 — Dave Trier > [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) 在企业级 AI 中取胜，绝非仅仅依赖单一的最佳模型或几个成功的试点，而是需要在数据、治理、业务、法务、风险、合规与架构等不同部门之间建立一套可重复的运行体系。 — Dave Trier > [00:04](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=4s) 治理不再是事后补救或外部硬塞的阻碍，而是从创意到投产全程原生嵌入。 — Dave Trier > [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) 到 2027 年实现 10 亿美元的 AI 业务价值。 — Shone Mousseiri > [03:53](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=233s) 模型资产库中大约只有 20% 的模型属于高风险类别……过去一刀切的审核方式耗费了过多精力，亟需重构风险分级方法论。 — Shone Mousseiri > [07:05](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=425s) 丰田之所以闻名世界，不仅在于其造出了高可靠性的汽车，更在于其创造了一套能够持续、规模化生产高质量汽车的生产制造系统。 — Shone Mousseiri > [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) 要实现 10 亿美元的价值目标，不可能依赖人工去逐个定制 10,000 个孤立解法，必须通过可复用能力赋能第一道防线。 — Shone Mousseiri > [09:14](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=554s) 治理能力随 AI 采纳规模线性扩展，而非随治理团队人员编制被动膨胀。 — Shone Mousseiri > [13:44](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=824s) 未来的优势不属于构建最多孤立 AI 系统的组织，而属于构建了最高效 AI 工厂的组织——那些能够以 AI 扩张速度同频扩张信任的企业。 — Shone Mousseiri > [13:44](https://www.youtube.com/watch?v=Yz2tYB_f4uI&t=824s) 必须从模型清单（Model Inventory）开始做起——你无法治理你看不见的东西。 — Shone Mousseiri ## Mentioned Dave Trier, Shone Mousseiri, ModelOp, 宏利金融 (Manulife), Evident AI, Formula 1 (F1), 丰田生产方式 (Toyota Production System), Microsoft Copilot Studio, Copilot Premium, 新加坡金融管理局 (MAS), 加拿大金融机构监管局 (OSFI), 智能文档处理 (IDP) ## Video description Shone Mousseiri - Head of Global AI Model Validation and Governance, Manulife Dave Trier - CEO, ModelOp Enterprises feel stuck between two AI extremes: Wild West innovation that overruns budgets, or rigid processes that strangle innovation. What if a new path turned AI into a scalable "factory" that delivers use cases to production rapidly, repeatedly, and safely? Facing a surge of AI demand across every function, Manulife built exactly that: an operating model for AI delivery they call the AI Factory. Research teams define reusable patterns, development teams adapt them to business needs, and validation runs quality control with feedback loops that continuously improve outcomes across the model lifecycle. Embedded AI governance streamlines the factory, enabling faster time-to-value, better reuse, and high-quality AI at scale. This session defines what an AI Factory really is, walks through how reusable patterns and embedded governance keep delivery both fast and safe, and shows why it's the key to scaling AI to value instead of getting stuck between those two extremes. If you're working to turn fragmented AI initiatives and processes into a governed, repeatable AI Factory that compounds into real value at scale, this is the session to join. [Source](https://www.youtube.com/watch?v=Yz2tYB_f4uI)

Source quotes

[0:04] Dave Trier: 全都是为了在单圈成绩中抠出十分之一秒，且绝不以牺牲整体质量或安全性为代价。 [0:04] Dave Trier: 在企业级 AI 中取胜，绝非仅仅依赖单一的最佳模型或几个成功的试点，而是需要在数据、治理、业务、法务、风险、合规与架构等不同部门之间建立一套可重复的运行体系。 [0:04] Dave Trier: 治理不再是事后补救或外部硬塞的阻碍，而是从创意到投产全程原生嵌入。 [3:53] Shone Mousseiri: 到 2027 年实现 10 亿美元的 AI 业务价值。 [3:53] Shone Mousseiri: 模型资产库中大约只有 20% 的模型属于高风险类别……过去一刀切的审核方式耗费了过多精力，亟需重构风险分级方法论。 [7:05] Shone Mousseiri: 丰田之所以闻名世界，不仅在于其造出了高可靠性的汽车，更在于其创造了一套能够持续、规模化生产高质量汽车的生产制造系统。 [9:14] Shone Mousseiri: 要实现 10 亿美元的价值目标，不可能依赖人工去逐个定制 10,000 个孤立解法，必须通过可复用能力赋能第一道防线。 [9:14] Shone Mousseiri: 治理能力随 AI 采纳规模线性扩展，而非随治理团队人员编制被动膨胀。 [13:44] Shone Mousseiri: 未来的优势不属于构建最多孤立 AI 系统的组织，而属于构建了最高效 AI 工厂的组织——那些能够以 AI 扩张速度同频扩张信任的企业。 [13:44] Shone Mousseiri: 必须从模型清单（Model Inventory）开始做起——你无法治理你看不见的东西。

Source quotes

[1:49] Dave Trier: Wild West AI [1:49] Dave Trier: Industrialized AI [3:53] Shone Mousseiri: team of one [9:14] Shone Mousseiri: challenge every single gram of weight [9:14] Shone Mousseiri: immaterial pathway [12:37] Shone Mousseiri: AI for All [12:37] Shone Mousseiri: Patterns Over One-Offs [12:37] Shone Mousseiri: Guidance Proportional to Risk [12:37] Shone Mousseiri: Reusable Testing [12:37] Shone Mousseiri: A Connected System of Record [12:37] Shone Mousseiri: you can't govern what you can't see [12:37] Shone Mousseiri: stop being the brake and become the engine

Original transcript

[0:04] To begin, I actually wanted to  ask a very important question. [0:08] How many Formula 1 fans are  out there? Show of hands. [0:13] Actually, that's a lot more than I expected.  All right. Even if you're not a Formula 1 fan, [0:18] you can appreciate what it takes to build  championship winning teams. Now, for Formula 1, [0:24] it's not about the best car. It's not about  the best driver, but it's actually hundreds [0:29] of different team members behind the scenes.  You have the mechanics, you have the engineers, [0:35] you have the statistitians, the operators, the pit  crew, etc. All working in a highly disciplined, [0:41] highly coordinated system. Every lap is measured,  every component is tested, every process is [0:48] refined. They're constantly honing, perfecting,  trying to shave off every bit of friction. [0:54] all just trying to eek out a tenth of a second  off of the lap. But you know what? They never [0:59] sacrifice the overall quality or the safety.  Now, that's what industrialized looks like. [1:06] And enterprise AI is no different. Winning an  enterprise AI, it's not about just the best model. [1:13] It's not about just a couple of good pilots, but  it's about building a repeatable system across the [1:19] different teams, your data, your governance, your  business, legal, risk, compliance, architecture, [1:25] all working with one goal to take AI from idea  to production rapidly, safely, and at scale. [1:32] So, this is a topic we're going to explore  today. We're going to be covering this in [1:36] the construct of a case study. It's actually a  160-year-old insurer where they took this concept, [1:43] these principles and created a near frictionless  path from idea all the way to production. [1:49] My name is Dave Trier. I'm the CEO of  Modelup. We're a software company focused [1:53] on industrializing AI delivery with embedded  governance. With me today is Shan Massuri, [2:00] who's the enterprise AI validation and governance  lead for Manulife. Now, this is not another hype [2:06] talk. We're getting into delivery today.  We're going to go through a lot of concepts. [2:11] We're going to go through them pretty quickly,  so buckle up. Now, before I hand it to Shown, [2:18] a lot of people say, "Hey, Dave, that all  sounds great. It's a lot easier said than done." [2:22] Well, I've been delivering large scale AI  and ML for over a decade. And when I walk [2:26] into large customers, they I see one of two  things. On the top left is where companies [2:32] have what I call the wild west AI. There's a whole  bunch of different teams. They're doing different [2:38] processes, using different tools. They're running  quickly, experimenting, and at first it says, "Ah, [2:43] that sounds great. We're being agile.  We're getting AI out the door." [2:47] But what happens is incidents occur, risks  accumulate, lots of rework has to be done, [2:54] and ultimately trust erodess. Now the other  path, the other extreme is that bottom right [2:59] where there's too much structure put into place,  too many manual processes, too many assessments, [3:05] too many handoffs, and it it basically just slows  innovation to a halt. Now, both of those paths [3:13] destroy the value of AI. Luckily, there's a  third path. We call that industrialized AI, [3:19] where governance is not an afterthought.  Governance is not bolted on, but rather [3:23] it's embedded in how you deliver AI all  the way from idea through production. [3:27] That's what we do at Model Op day in and day  out. Our software helps to industrialize AI [3:32] to ensure that you can deliver any type of  AI, whether it's open source, proprietary, [3:37] cloud-based, agent, MCP, it does not matter.  Can deliver that AI again rapidly, responsibly, [3:43] and at scale. But again, don't take it from me.  I'm gonna hand it over to my colleague Shan who's [3:47] going to walk through that case study about that  frictionless AI factory. So Shan, take it away. [3:53] Great. Thank you so much Dave. I'm Shan Misari and  I lead Manual AI validation governance practice. [3:59] Main life has recently been recognized as number  one life insurance company globally when it comes [4:04] to AI maturity by evident AI and this year we set  an ambitious goal for oursel and that is to reach [4:11] 1 billion AI value by 2027. In order for us to  achieve that goal, we need to really transform [4:17] this 150-year-old insurer to a company to an AI  powered organization. And that requires us to [4:25] embark on a journey to improve AI adoption  and democratize AI development activities. [4:31] So I'll talk about my my own story. When I first  joined main life, it was about five years ago. [4:36] I was a team of one u setting up the model  validation and governance practice globally. [4:42] That team has slowly matured into a  team that could support our 300 plus [4:47] uh data scientists developing mostly  machine learning models at a time. [4:52] But today we have 3,000 plus co-pilot studio  license holders, another 30,000 plus employees [4:58] with co-pilot premium access. So that allows them  to develop um co-pilot studio or co-pilot agents. [5:05] They can share they can publish it with anyone in  any organization. At the same time, what we notice [5:10] is there's a lot more regulatory pressure across  many of the jurisdictions that we operate in. [5:16] We operate in Apac. There's a new re there's a  new guidance from mainland China. There's a new [5:19] law in Vietnam and there's the MAS guidance  in Singapore. We also have to comply with [5:24] the on with the Canadian OIE and uh and the  Quebec regulatory requirements as well as [5:30] many state level requirements in the US. So what  we really learned is that the challenge today is [5:35] no longer about building AIs but ensuring that we  have a system that can scale with that ambition. [5:45] So we asked ourselves a very difficult question.  So are we really investing our resources [5:52] and the capacity in areas that matter? The answer  to that was no. We were not exactly operating in [5:59] the greatest efficiency as AI governance team.  And there were two main friction points. The first [6:04] on the left is that we recognize that AI risk  is concentrated. So about 20% of our portfolio [6:11] about 20% of our models in our portfolio actually  generate the most risk. Those are the ones in [6:15] underwriting, the ones in HR, the ones in um  that are customerf facing or external oriented. [6:22] So in order for us to reflect that and make sure  that we focus our values and effort on areas that [6:26] matter, we need to revamp our risk taring  methodology. So I'll talk about that later. [6:31] Then we have the other friction point and that is  uh the concept of AI systems. So when we when we [6:37] first started AI governance we were talking about  use cases but we recognize that a lot of these use [6:41] cases share similar patterns. They can be using  the same architecture. They have the same LLM. [6:46] We don't have to review to re-review every  single thing uh at the same time. We want [6:50] to focus on the incremental differences that  matter. That allows us to really focus on the [6:55] on the areas that can generate the most value. As  a validation team, there's an infinite number of [6:59] tests been carry out before we approve a model.  We want to focus on the one that matter most. [7:05] So that brings us to the concept of an AI factory.  So when I talk about AI factory, I really want to [7:10] bring us uh back to the example of Toyota. So  Toyota's not only known for developing reliable [7:16] cars. Toyota is also known for developing u a  production system that can consistently produce [7:24] quality reliable cars at scale. And that is how I  want to really explain the concept of AI factory [7:31] for us today. Um we want to talk about the  three uh the three teams here. So there's the [7:38] AI research team, we have the AI development  teams as well as the AI validation teams [7:43] and and I will explain how these relate to the  Toyota production uh system. First we have the [7:49] research team. So this is where we really  um dedicate our data science effort. Then [7:56] we are trying to develop the next generation  concept car. We're drawing on blueprints of how to [8:01] build or how to put together an AI solution. Then  we have our development teams. Our development [8:05] teams are embedded within our business functions.  They understand the unique business requirements [8:11] that we face and they customize where  required. And lastly, we have our uh [8:16] validation team. So my team we operate as a  quality assurance team similar to um in the [8:22] automotive industry where we have safety tests,  we have crash tests, we have emission tests. [8:27] We are a team to ensure that we have the right  testing in place before a model goes live. [8:35] So one thing I want to point out here is [8:37] there are a lot of these parallels and we want  to focus really on the areas that matter most. So [8:43] let me be more concrete here. I will give you  an example. we have the intelligent document [8:47] processing model. So that is a model where  we uh are processing a lot of medical claims, [8:53] receipts or dental forms. These patterns can be  reused across multiple uh multiple dental forms, [9:01] multiple different products. We don't  want to register every we don't want to [9:04] re-review every single one at the same time.  We want to make sure that we register them [9:08] all and tie them to the same AI system. So that  is the entity that we want to govern today. [9:14] Next slide I want to talk about formula  one concept. So when we talk about [9:19] validation again we want to really make  sure that we are spending the effort [9:23] and right sizing the validation effort for the  models that that are very lowrisisk in nature. [9:28] Uh we to achieve this we revamped our risk tier  methodology. We introduced an immaterial pathway. [9:35] So what that allows is it allows us to  um fasttrack a lot of lowrisisk models. [9:41] But in order to do that, we need to make  sure that we have um the right um tooling [9:47] and training in place to support our uh team  to develop the right solutions in place. So um [9:57] the other thing I want to  talk about here is the um [10:02] is just like formula one. We want to we  want to make sure that we challenge every [10:06] single gram of weight, that we challenge every  question we ask. We want to make sure that we [10:10] have as lightweight of a validation process  as possible when it comes to AI approval. [10:17] So that is so that is really the key concept here [10:19] is we want to make sure that  we have a scalable process. [10:23] Uh another thing I want to talk about here is um  in order for us to achieve the 1 billion AI value, [10:29] we need to make sure that we are really reusing  a lot of the systems that we have in place. [10:36] We're not going to achieve that AI value by simply  developing 10,00 solutions. To achieve that, [10:43] we have to make sure that we are empowering our  first line model developers with the right tools [10:50] that they need to to to develop solutions on their  own with the AI systems that we have empowered [10:55] with the AI research team. So here I want to  highlight a few concepts. One is reusability. [11:00] Uh so we ask ourselves which tests are comparable  that we can trust our first line developers to run [11:06] with low code no code building is no longer  about bottlenecks. We want to make sure [11:11] that we are spending our time of actually matters  and only about 10% of our time should be spent on [11:17] building models. We because now we loc solutions  we don't have to spend all the time developing [11:22] bespoke solutions only 10% is spent on developing  models. 70% is getting spent on actually [11:28] uh testing the solution and that's an area where  we saw the greatest opportunity for us to uh [11:34] streamline the constraint isn't about willingness  is about assuring that the most expensive aspect [11:41] is being insured with the team. So if we were  to if we were to uh test a customerf facing [11:46] AI solution we want to make sure that we have a  red teaming solution like a red teaming testing [11:51] library and that library once built can be applied  to multiple external faces solution doesn't have [11:56] to be only tailored to one solution. Another  thing, another example that I want to draw here is [12:02] when we are reviewing HR models, we build a  solution to um to to conduct counterfactual [12:09] bias testing. That solution can be applied to  also underwriting models. We don't have to uh to [12:17] we we can really by empowering the first line and  equipping them with the right uh testing libraries [12:23] that they have that they need we can we can really  uh allow them to become the next generation data [12:30] scientist and equip us with a higher level of  AI literacy. So what happened at the end is um [12:38] we enable greater first line ownership and  lowrisk systems by launching an AI for all [12:44] training program. We have an role based training  for our model users, our owners and developers. [12:49] We were able to democratize testing by giving  teams the right tools, the right guidance and [12:53] training to address the risk before entering  a formal review. We developed standardized [12:59] risk proportional templates and implement  automated quality checks in a submission process. [13:05] We have also redirected our capacity  to where it creates the most value. [13:09] Less value less effort spent renewing low uh  reviewing lowrisk use cases. More scrutiny spent [13:15] on medium and high-risisk models. The result is  a governance model that scales with adoption, [13:19] not headcount. One that empowers citizen  developers to move quickly and responsibly [13:26] and one where oversight remains  proportionally to risk even as volume grows. [13:30] Ultimately, the AF factory enables an AF for all  strategy where governance is built for scale, [13:36] risk is managed where it matters most and  innovation can move at the enterprise speed. [13:44] So, let me bring this back together. The future  won't belong to the organization that build the [13:48] most AI systems. It will build it will belong to  the organizations that build the most effective [13:52] AI factories. The organization that can scale  trust as quickly as they scale AI. For us, [13:59] that came down to four key ideas. First, patterns  over one-offs. Second, guidance and proportion to [14:04] risk. Third, reusable testing. And fourth,  a system of record that connect everything. [14:10] Together, these principle help us move beyond  choice between the well west and a traditional [14:16] governance approach. Governance team embedded. We  need to have governance team embedded in delivery. [14:22] So we want to make sure that every  citizen developer understand governance [14:25] and it stop so that in that way governance  can stop being in break and became the engine. [14:31] If you take away one lesson today start with  a model imagery and you can you can't govern [14:35] what you can't see. So in order for us  to actually understand what's out there [14:40] for us to manage we need our foundation is model  up and it connects uh intake risk assessment [14:46] validation approval and monitoring all in one  place. That's how that's how we scale AI adoption [14:51] without scaling governance at the same rate.  And with that, I'll pass it back to Dave. [14:58] Awesome. Thanks, Sean. Really appreciate it. I  know we went that through that pretty quickly [15:03] overall, but hopefully gave you a little bit of  sense of how you can take what is typically before [15:08] a system like model very bespoke. Everybody's  doing something a little bit different and it [15:13] slows things down as we've talked about before.  And because of that, you're basically duplicating [15:19] efforts over and over, but then shifting that,  challenging what you're doing today. And I [15:23] know for many financial institutions, insurance  companies out there that you have set patterns [15:27] of how you've done this in the past. But with AI,  you have to challenge that. You have to break the [15:31] mold. You have to take as what Sean mentioned  and identify where there's reusable patterns. [15:37] And that's where again an engine like model up  being able to plug in some of those reusable [15:41] assets or reusable tools, reusable tests and  turn that into again bespoke one-off duplicative [15:46] efforts into something that is a consistent  factory so that you know it doesn't matter whether [15:51] you're a data scientist or whether you're in  the marketing department back office. It doesn't [15:56] matter if you've got an idea for an AI solution.  It's going to go through this system and we're [16:00] going to be assured across data risk compliance  legal governance business IT that we have all the [16:05] checks in place. We have checked every box. We  have covered every policy, every process, every [16:10] system. But ultimately, we are accelerating that  time to value. So again, hopefully it gave you [16:15] a little bit of sense of of how we worked with a  company like Manulife to be able to bring this AI [16:19] factory to life. Now, we're going to be um around  to answer any questions that you may have. You can [16:25] also stop by our booth. We're booth 1515 at Model  Up. But I want to thank you again for your time [16:30] and thanks again to Sean for walking through  this particular case study. So, thank you all.