Vanguard Replaces Frontier AI With In-House Models at 5M Query Mark

Vanguard在日查询量达500万次时以自研模型替代前沿AI 企业一旦日请求量达到500万次，就必须从商业API转向自主主权模型，因为掌控模型层能够保障专有数据安全，并构建出开箱即用工具无法复制的机构智能。 结合Vanguard的企业AI实践，演讲者提出了一种务实的“自建加采购”（build plus buy）策略，将用于通用知识的商业前沿API与用于敏感、复合型专有数据的自研主权模型相结合。他们确立了明确的经济考量标准（如每日500万次请求的门槛），并概述了包含三种专业小语言模型（SLM）原型与渐进式自愈反馈回路的分层架构。从将最初47%的语音转录错误率转化为每年节省1500万美元的内部资产，到妥善应对复杂的GPU推理瓶颈，Vanguard展示了专有智能与稳健治理如何构建起持久的十倍竞争护城河。 企业应根据三大主要信号来评估自主AI：存在不能流出安全边界的敏感专有静态数据、持续累积的动态交互数据，以及通用前沿模型在特定领域任务上的失灵。 (0:04) Vanguard采取了“自建加采购”的混合方式，而非在两者之间做非此即彼的选择，即利用商业前沿API快速获取公共领域能力，同时将自主模型构建保留给专有、敏感的工作负载。 (2:38) 自主AI的经济拐点出现在日请求量达到约500万次时，届时持续的API Token成本与领域微调需求使得自托管模型基础设施更具显著的成本效益。 (6:53) 为防止奖励黑客（reward hacking）现象，企业生产流水线需要“自愈型AI”系统来捕捉丰富的会话元数据、用户编辑和导航模式，而不是依赖点赞或点踩等过于简化的二元反馈。 (6:53) 自主模型架构得益于将职责划分为三种小语言模型（SLM）原型：负责路由的编排SLM、负责监管防护栏的合规SLM，以及负责深层机构知识的领域智能SLM。 (10:57)

Vanguard artificial intelligence leaders Sagarika Suresh and Jithin Pradeep argue that enterprises should move beyond the traditional "build versus buy" dilemma to adopt a hybrid strategy, leveraging commercial frontier APIs for fast time-to-value while constructing sovereign models when proprietary data perimeters, domain precision, and operational scale demand an owned intelligence asset. Economic Thresholds and the Hybrid Model Strategy (2:38) Rather than rejecting third-party foundation models, Vanguard promotes a "build plus buy" strategy. Commercial frontier APIs provide roughly 90% baseline accuracy within 90 seconds of integration, access knowledge from models scaling past 2 trillion parameters, and eliminate upfront training compute. However, the speakers caution that zero-cost deployment is a myth once security, integration, and production hardening are accounted for. Sovereign development is justified primarily under three conditions: strictly non-exportable proprietary static data, continuously compounding interaction data (such as call summaries and workflow telemetry), and severe domain misalignment in generic public models. Vanguard established an economic break-even threshold at approximately 5 million daily requests. Below this volume, commercial APIs remain cost-effective; above it, compounding inference fees and domain fine-tuning requirements favor self-hosted sovereign models. Retaining high-volume, long-context documents—such as extensive multi-page client tax filings—within self-hosted infrastructure also prevents commercial API token costs from spiraling. Domain Misalignment and Internal Benchmark Foundations (15:09) Vanguard warns that public AI benchmarks suffer from "benchmark maxing," where models overfit to stale public evaluations, showing high scores that fail to translate into enterprise production. To make objective architectural trade-offs, enterprises must build internal evaluation suites grounded in proprietary workflows. The template for Vanguard's sovereign strategy began with speech-to-text transcription: while third-party vendor engines claimed error rates as low as 4% on public tests, their performance collapsed to a 47% Word Error Rate (WER) when tested on Vanguard customer service calls due to financial jargon and company-specific vocabulary. Rather than waiting for vendor updates, the team engineered an in-house transcription pipeline. Over iterative cycles, internal models drove the error rate down from 47% to 37%, then 10%, and ultimately 1.3% in production. The system currently processes 45 million 12-minute phone calls annually. Running this pipeline internally costs under $500,000 per year in compute, compared to an estimated $15 million commercial replacement cost, while feeding redaction systems that have sanitized over 6 billion sensitive data points. Layered Architecture and Self-Healing Model Archetypes (10:57) To handle complex customer journeys, Vanguard structures its sovereign architecture around an intelligent front-door orchestrator supported by three distinct Small Language Model (SLM) archetypes: Compliance SLMs for internal perimeter guardrails and regulatory policies, Orchestration SLMs for low-latency routing grounded in business logic, and Domain Intelligence SLMs for Vanguard-specific financial expertise. For model selection, Vanguard utilizes Mixture of Experts (MoE) architectures for broad domain synthesis because activating only parameter subsets reduces inference serving costs, whereas dense models handle focused operational tasks like Named Entity Recognition. Model optimization follows a three-tier hierarchy: foundational MoE models grounded in enterprise data, Reinforcement Learning from Human Feedback (RLHF) incorporating domain-expert annotations, and an evolutionary self-healing system. By capturing multi-turn session metadata, user edits, and workflow navigation paths, the self-healing loop continuously monitors failure modes and mitigates reward hacking. In an internal cybersecurity deployment, this autonomous loop learned from exploit attempts to identify and neutralize close to 48% of adversarial attacks without manual intervention. Inference Realities, Hardware Lessons, and Governance (27:40) Jithin Pradeep emphasizes that while training models is technically straightforward, operating inference under strict client-facing Service Level Agreements introduces unforeseen failure points. Workloads that operate stably within an 8,000-token context window can experience catastrophic out-of-memory crashes when expanded to 37,000 tokens. Furthermore, attempting to bleed off traffic spikes by mixing smaller, lower-tier cloud GPUs into high-end accelerator clusters caused severe bottlenecks: slower GPUs lagged during inference, causing queues to balloon out of control before primary GPUs could catch up. To balance operational efforts, Vanguard follows a "10-20-70 rule," attributing 10% of total effort to model algorithms, 20% to compute infrastructure, and 70% to multidisciplinary talent, operational resilience, and model governance. To optimize expenses across more than ten internal applications, Vanguard provisions dedicated inference endpoints that allow non-critical business workflows to scale down outside operating hours (such as 6:00 AM to 9:00 PM), while reserving continuous availability for critical 24/7 services. The speakers conclude that while generic foundation intelligence can be rented directly off the market, durable competitive differentiation requires an owned layer of domain intelligence. Transitioning from basic transcription to an enterprise-grade sovereign factory ensures that decades of client conversational feedback and institutional financial knowledge remain defensible internal assets rather than being ceded to third-party providers.

The Vanguard Group 的 Sagarika Suresh 与 Jithin Pradeep 认为，企业落地主权人工智能（Sovereign AI）不应陷入自建或外购的二元对立，而应基于专有数据特征与业务临界点，采取“构建加购买”（Build Plus Buy）的混合策略；通过将外部商业前沿大模型与内部专有小语言模型（SLM）协同编排，在保障数据合规与成本收益的同时沉淀长期的机构专有智能壁垒。 主权AI决策信号与“构建加购买” (0:04) 演讲者指出，企业启动自主研发主权模型必须具备三项关键信号：一是企业拥有严格受监管或政策限制、绝不能离开企业边界的敏感专有静态数据；二是拥有客户交互记录、客服总结等随业务规模不断扩充的复利动态数据；三是基于公开基准测试的通用商业模型在精度和细分业务规则上存在结构性脱节。Vanguard 据此确立了明确的量化经济阈值：当单一应用的日均调用量达到约500万次时，由于第三方 API 持续推理的复利成本与领域微调需求，自建主权模型在财务和架构上才真正具备合理性。 与此同时，团队强调不应将外部商业大模型全盘拒之门外。前沿商业模型已拓展至2万亿甚至10万亿级参数规模，其 API 能够在90秒内提供约90%的基准准确率，且具备极低的初始运维成本，非常适合处理公开领域的通用任务。但演讲者提醒“零初始成本”在投产时往往只是假象，将原型转化为稳健的企业级生产系统仍需巨大投入；因此最优解是双轨并行：通用外部任务调用前沿 API，而受边界约束的高价值专有数据则完全留存于内部主权模型。 语音转写破局与三波演进路径 (15:09) Vanguard 自研主权 AI 的立足点始于客服语音转写。早期评估第三方商业转写引擎时，厂商在公开基准上宣称词错误率（WER）仅为4%，但在处理 Vanguard 实际客户电话时，因无法理解金融垂直术语（例如无法区分“金融市场”与“超级市场”），错误率骤升至47%，导致下游分析完全失效。面对困境，团队从零构建专有转写管道，经过持续优化，将词错误率从47%逐步降低至37%、10%，目前稳定运行在1.3%以下。 该转写系统当前年均处理4500万通客服电话（每通平均12分钟），并直接驱动下游命名实体识别与自动化脱敏系统，已累计安全脱敏超60亿个敏感数据点。在财务效益上，若采购第三方服务每年需支出约1500万美元，而 Vanguard 内部运行整套系统的年成本不到50万美元（几乎全部由算力构成）。这一成功推动团队确立了三波演进路径：第一波聚焦于数据就绪与信号验证（以转写 WER 低于1.3%为标志）；第二波构建高精度的领域专有语言模型；第三波则升级为自动化运维、持续自愈与自托管的“主权模型工厂”。 分层编排架构与三类SLM原型 (10:57) 在大规模业务场景中，智能编排层绝不能仅充当简单的“成本路由器”，而必须具备深度的上下文感知、动态策略实施与模型能力映射机制。Vanguard 规划的分层主权技术栈由三类专有小语言模型（SLM）协同驱动：一是负责低延迟分发与业务逻辑流转的“编排 SLM”；二是内置严格内部围栏并对齐 SEC 与 FINRA 外部合规政策的“合规 SLM”；三是专注于精准金融知识、放弃无关通用常识的“领域智能 SLM”。这种设计支持主权模型与外部前沿模型灵活协同，例如将内部合规规则与外部模型的公开法规知识相结合。 在模型底层架构上，广泛的领域智能模型采用专家混合架构（MoE），仅在推理时激活部分参数，从而在降低延迟与成本的同时承载庞大的机构专有知识；而命名实体识别等狭窄流程则采用稠密模型（Dense）。在训练体系上，Vanguard 结合了 MoE 企业数据基座、基于业务实际成果的领域专家强化学习（RLHF），以及动态闭环的演化自愈系统。实践证明，该自愈系统在网络安全对抗场景下无需人工干预即可自主诊断并中和近48%的对抗性攻击。 推理工程落地与10-20-70法则 (27:40) Jithin Pradeep 直言，“训练模型往往是最容易的部分，推理规模化才是主要瓶颈”。在严格的服务等级协议（SLA）下，生产环境会暴露出极具破坏性的隐性问题：例如当上下文窗口从8K突增至37K词元时会突然引发内存溢出；团队此前曾尝试混用较小规格的云端 GPU 分流负载，但小卡推理缓慢导致排队队列迅速失控膨胀，大卡亦无法及时清空积压，最终引发整个服务瘫痪。为此，Vanguard 为10多个并发业务配置了独立的专用推理端点，并在早6点至晚9点等非业务高峰时段自动启停计算节点，有效避免了海量报税等长文档调用外部 API 造成 Token 预算爆炸。 基于实践教训，团队总结出主权 AI 实施的“10-20-70”法则：算法研究仅占10%的精力，算力基础设施占20%，而70%的重点落在跨学科人才、运营韧性与模型治理上。针对复杂的金融咨询流程，模型训练脱离了单纯的“下一词预测”（Next-Token Prediction），而是将多轮会话日志与客户长远财务成果直接绑定；同时通过定向剪枝剥离冗余参数、监督微调（SFT）和强化微调（RFT）回注专有金融知识，确保模型在解耦基础权重的同时实现长效敏捷迭代。 Vanguard 团队总结认为，自研主权模型的根本目的在于沉淀不可替代的“专有智能壁垒”；过度依赖外部公共前沿模型只能获得与竞争对手同质化的通用能力，唯有将机构数十年来积累的客户对话、反馈机制与业务知识固化在自主模型中，才能形成十倍于通用智能的竞争护城河。目前，如何在多轮推理交互中精准管理上下文内存边界、调度异构算力集群，并建立能够防范奖励破解的自愈机制，仍是生产落地中需要持续探索的工程难题。

## Background 本视频由 Ai4 于 YouTube 平台发布，由先锋领航集团（The Vanguard Group）的企业 AI 与研究负责人 Jithin Pradeep 及应用 AI 研发经理 Sagarika Suresh 共同主讲。视频基于先锋领航在强监管金融环境下的实战经验，系统阐述了企业如何评估 AI 模型的“自建与外购（Build vs. Buy）”决策，以及如何通过数据主权、治理架构与领域专属语言模型（SLM/LLM）构建持久的商业竞争壁垒。 ## TL;DR 先锋领航分享了其七年来在金融生产环境中落地主权 AI（Sovereign AI）的完整演进路径，主张破除二元对立，推行“先用前沿 API 极速验证、在触发高价值私有数据与规模临界点时平滑转向自研”的 Build + Buy 协同策略。团队通过将客服通话转写错误率（WER）从外部方案的 47% 压降至自研的 1.3%、将年运行成本从 1500 万美元削减至 50 万美元以内的实战成果，验证了主权架构的经济与技术可行性。体系内提炼出合规、编排与领域智能三大 SLM 架构原型，并构建了涵盖 MoE 架构选型、推理遥测、防止奖励作弊及演化自愈（Self-Healing）的完整模型工厂，最终强调 70% 的成功取决于人才、流程与模型治理，核心目标是以专有商业智能超越单纯的通用智能。 ## Atmosphere & style 整场演讲严谨务实、直击痛点且充满硬核工程沉淀，摒弃了空洞的行业炒作与跑分营销。两位主讲人以极具逻辑感的专业口吻，坦诚复盘了异构 GPU 调度雪崩、长上下文内存溢出（OOM）以及公开基准在垂直场景严重失真等生产“暗礁”，呈现出顶级金融科技团队在技术攻坚与治理权衡中的清醒与从容。观看体验犹如置身一场高密度的企业级架构复盘大师课，既有俯瞰战略全局的商业洞察，又有直面底层系统与算力开销的实战厚度。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=brj0GjA84y0&t=4s) 决定是否自建主权 AI（Sovereign AI）取决于三大核心信号：拥有严禁出域的高价值专有数据、业务持续产生动态复合数据飞轮，以及公开前沿模型在垂直场景中无法达到合规与专业标准。 - [02:38](https://www.youtube.com/watch?v=brj0GjA84y0&t=158s) 企业级 AI 策略应破除“自建（Build）对决外购（Buy）”的二元对立，采用“Build + Buy”协同路线：先用前沿 API 极速验证业务可行性，在达到规模与数据壁垒后再平滑过渡到主权自建模型。 - [06:53](https://www.youtube.com/watch?v=brj0GjA84y0&t=413s) 在先锋领航的量化财务测算中，日推理请求量接近或达到约 500 万次（5M Daily Requests），即为触发自建与微调模型的关键盈亏平衡点。 - [10:07](https://www.youtube.com/watch?v=brj0GjA84y0&t=607s) 企业级编排层（Orchestrator Layer）不应只是简单的“成本路由器”，而需综合数据主权、涉密等级、响应延迟与专业精度，在主权模型与前沿 API 之间实现多模型协同调度。 - [10:57](https://www.youtube.com/watch?v=brj0GjA84y0&t=657s) 先锋领航提炼出三类 SLM 架构原型：负责隐私隔离与策略护栏的合规 SLM、负责低延迟调度的编排 SLM，以及追求垂直场景极高精准度的领域智能 SLM。 - [13:28](https://www.youtube.com/watch?v=brj0GjA84y0&t=808s) 公开 Benchmark 存在严重的“刷榜（Benchmark Maxing）”失真，企业必须自建贴合真实业务用例的评估套件，确保自研方案在业务智能（Business Intelligence）上实现复利积累。 - [15:09](https://www.youtube.com/watch?v=brj0GjA84y0&t=909s) 在客服语音转写攻坚战中，自研模型将专有语境下的词错误率（WER）从外部供应商的 47% 降至 1.3%，年处理 4500 万通电话且年运行成本从 1500 万美元骤降至不足 50 万美元。 - [18:29](https://www.youtube.com/watch?v=brj0GjA84y0&t=1109s) 在模型架构选型上，混合专家模型（MoE）适合承载特定领域认知以降低单次推理延迟并提高扩缩容经济性，而密集模型（Dense Models）则更适合 NER 等独立特定任务。 - [21:07](https://www.youtube.com/watch?v=brj0GjA84y0&t=1267s) 主权大模型采用自底向上的三层对齐与自愈体系：底层 MoE 专有数据融合、中层领域专家 RLHF 意图对齐，以及顶层能自主识别并修复故障模式的“演化自愈（Evolutionary Self-Healing）”流水线。 - [25:32](https://www.youtube.com/watch?v=brj0GjA84y0&t=1532s) 生产推理面临诸多系统级陷阱（如 37K 长上下文 OOM、混部小型 GPU 导致队列堵塞等），工程系统的健壮性与推理遥测机制远比单纯的模型准确率更为严苛。 - [27:40](https://www.youtube.com/watch?v=brj0GjA84y0&t=1660s) 主权 AI 总体投入中，模型研发仅占 30%（10% 算法 + 20% 基础设施），剩下的 70% 属于人才梯队、跨部门协同与模型治理流程建设。 - [31:34](https://www.youtube.com/watch?v=brj0GjA84y0&t=1894s) 先锋领航主权 AI 演进经历三波浪潮：第一波为高保真数据信号验证（WER < 1.3%），第二波为特定领域主权语言模型落地，第三波为全栈自托管、持续迭代的“主权模型工厂”。 - [32:52](https://www.youtube.com/watch?v=brj0GjA84y0&t=1972s) 自建主权模型的根本动因在于打造独有的专有商业智能护城河，配合专属推理端点（Dedicated Endpoints）实现高峰弹性伸缩、规避公有 API 速率限制并解决高频税务文档等重度 Token 消耗的成本危机。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=brj0GjA84y0&t=4s) 什么时候该考虑主权 AI（Sovereign AI）？三大核心信号 在开启企业级 AI 的演进之旅时，最关键的问题往往不是“如何做（how to do it）”，而是“何时做（when to do it）”。基于先锋领航集团（The Vanguard Group）应用 AI 研发团队过去七年的实践经验，构建 AI 解决方案涉及多个维度的考量——这绝不仅是训练出一个模型那么简单，如果无法将其推向生产环境并持续产生业务价值，整个过程将是一场漫长而沉重的消耗战。因此，在企业决定投入研发自主可控的主权 AI（Sovereign AI）之前，必须捕捉到三个关键触发信号： ```mermaid flowchart TD accTitle: 主权 AI 决策的三大核心信号 subgraph Signals["主权 AI 触发信号"] S1["信号 1：高价值专有数据<br/>（外界未见，严禁出域）"] S2["信号 2：动态复合数据飞轮<br/>（生产流转，持续自增）"] S3["信号 3：前沿模型能力断层<br/>（公开基准脱节，无法达标）"] end Decide{"触发主权 AI 研发决策"} Signals --> Decide ``` 第一大信号是**拥有外部世界从未见过的专有数据**。这些数据高度敏感且具有核心商业机密属性，出于商业智能、处理合规性等各种硬性约束，企业绝不能也绝不愿意将其发送到外部网络。在起步阶段，这类数据多为静态留存数据，但它是构建模型最坚实的底座。 第二大信号是**业务系统正在源源不断地生成高动态性的复合数据**。随着系统在生产环境中持续运行，数据不再是死板的静态资产，而是与业务交互紧密交织。例如在先锋领航的真实场景中，大量的客户互动、实时通话摘要（Call Summarization）以及复杂业务流都在不间断地产生数据。这种持续演进、动态复合的数据形态，构成了企业独有的数据护城河。 第三大信号是**前沿模型（Frontier AI）在结构上无法达到企业的专业标准**。市面上主流的开源或商业大模型，其评估与基准测试（Benchmarks）几乎全部基于公开数据集构建，根本没有针对特定垂直行业或企业的深度需求进行精细化微调。当公开模型在企业严苛的特定场景下无法满足合规与准确率标准时，自建主权模型便势在必行。 --- ### [02:38](https://www.youtube.com/watch?v=brj0GjA84y0&t=158s) 破除二元对立：从“自建对决外购”走向“自建加外购（Build + Buy）” 当提出上述三项评估指标时，行业内往往会产生一个误区：“你们是否主张企业只搞纯粹的自主模型？”答案是一个响亮的“不（A resounding no）”。在当前的商业环境下，AI 创造价值的核心在于投资回报率（ROI）、上线速度（Velocity）以及综合成本控制。因此，先锋领航的核心策略不是传统的“非此即彼（Build vs Buy）”，而是“协同并进（Build + Buy）”。 企业不应忽视前沿 AI（Frontier AI）的巨大价值。通过调用成熟的前沿模型 API，开发者可以在 90 秒内以几乎为零的冷启动训练成本，迅速获得达到 90% 准确率的基线方案。前沿模型已经从最初的几十亿参数迅速攀升至 2 万亿（2T）乃至未来预期的 10 万亿（10T）参数量级，吸收了海量的全网公开知识与通用技能，同时将底层基础设施的运维负担降到了极低水平。 然而，将实验原型转化为稳定的生产系统完全是另一回事。所谓“零维护成本”某种程度上也是一种神话，因为企业级生产部署依然充满挑战。因此，先锋领航采取的实用路径是：**先借助前沿 API 极速验证业务可行性，在捕捉到更强的主权指标、数据壁垒显现且规模达到临界点时，再平滑转向主权模型。** - **极速验证优势**：前沿 API 具备变革性的开箱即用能力，帮助业务在前期免去繁重的基础设施建设与模型训练开销。 - **数据源头挖掘**：企业内部最具价值的训练数据，往往是那些因为合规审查“被阻断在内网、绝对禁止外发”的沉睡资产；而对于公开领域通识，前沿 API 则拥有天然的广度与推理表现。 --- ### [06:53](https://www.youtube.com/watch?v=brj0GjA84y0&t=413s) 决策拐点与生产反馈：500 万次日请求与防止奖励作弊（Reward Hacking） 在企业 AI 研发的战略规划中，管理团队设立了清晰的量化盈亏平衡点（Break-even Point）。当某个垂直用例的 API 外部调用成本急剧上升，且日推理请求量（Daily Inference Volume）持续攀升时，自建与微调模型的“甜蜜点”便正式到来。根据测算，在先锋领航的体系内，**每日调用量接近或达到约 500 万次请求（5 Million Daily Requests）**，即为触发自主模型构建的关键财务与规模阈值。 ```mermaid flowchart LR accTitle: 生产环境多维反馈与自愈飞轮 subgraph ClientInteraction["交互层与元数据"] UI["用户交互界面<br/>（Session / 聊天）"] FB["直接显式反馈<br/>（👍 / 👎 与人工修改）"] Meta["隐式交互元数据<br/>（上下文路径、耗时、重试）"] UI --> FB UI --> Meta end subgraph Engine["先锋自愈引擎 (Self-Healing)"] Synthesizer["反馈与防作弊校验<br/>（防止 Reward Hacking）"] FB --> Synthesizer Meta --> Synthesizer Train["模型持续微调与演进"] Synthesizer --> Train end Train -.-> UI ``` 在模型部署上线之后，先锋领航推行“让数据自身更聪明”的理念，依靠生产环境中的多维反馈信号驱动模型持续进化。这构成了其内部所倡导的“AI 自愈（Self-Healing）”概念的基础——即利用 AI 与真实运行数据来反哺和优化 AI 系统本身。 这一反馈体系主要由两类数据流交织驱动： 1. **直接业务反馈**：包含生产环境中用户直观的点赞（Thumbs up）、点踩（Thumbs down）以及随后的文本编辑修改记录。 2. **深层会话元数据（Metadata）**：单纯的好评或差评不足以完整反映模型的思维链路。在基于 Session 的连续交互中，系统会完整捕获产生最终评价前所发生的所有上下文元数据，精准定位导致不良体验的具体路径并引导模型主动规避。 通过将生产端具体业务场景的需求与多维反馈直接注入闭环，模型的专有智能得以不断复合增强。同时，团队在机制设计上刻意避免仅依赖单一数据源进行粗暴训练，以坚决防范模型在强化学习与对齐阶段出现“奖励作弊（Reward Hacking）”的失真现象。 --- ### [10:07](https://www.youtube.com/watch?v=brj0GjA84y0&t=607s) 超越简单的成本路由：构建多模型编排架构（Orchestrator Layer） 在“Build + Buy”的整体生态中，AI 解决方案绝不是单点作战，而是系统性拼图中的重要组件。在实际落地时，上层应用程序不再直接与某一个特定的大模型硬编码绑定，而是统一接入底层的**编排器层（Orchestrator Layer）**，由编排器负责在主权专属模型与外部前沿大模型之间进行动态决断与任务分发。 先锋领航强调，企业级编排器绝不能仅仅是一个粗粒度的“成本路由器（Cost Router）”。如果仅按照“遇到复杂任务就调用最大号的前沿模型做任务拆解规划，后续子任务分派给小模型执行”的简单逻辑运作，远远无法满足实际企业级业务在合规边界、数据主权、响应延迟与专业精度上的复杂权衡需求。编排层必须综合考量数据涉密等级、任务特征、性能表现以及长效治理体系，实现全方位、智能化的多模型协同调度。 ### [10:57](https://www.youtube.com/watch?v=brj0GjA84y0&t=657s) 企业级主权技术栈与三大 SLM 原型 在大型金融机构的复杂业务体系中，用户界面通常聚合了繁杂的产品线与服务入口。为了摆脱传统模式下让客户自行在网页端苦苦摸索路径的痛点，Vanguard 提出了分层编排架构（Layered Architecture），将业务逻辑与智能调度直接嵌入前端门户，将底层 AI 系统的复杂协同对用户完全抽象化。在该主权技术栈（Sovereign Stack）中，团队提炼出了三类协同工作的小语言模型（SLM）原型（Archetypes）： ```mermaid flowchart TD accTitle: 三大 SLM 架构原型 subgraph SovereignStack["Vanguard 主权技术栈 (Sovereign Stack)"] direction TB Orch["2. 编排 SLM (Orchestration SLM)<br/>低延迟路由 / 贴合内部基础设施"] Comp["1. 合规 SLM (Compliance SLM)<br/>策略护栏 / 隐私隔离 / 防止数据外流"] Dom["3. 领域智能 SLM (Domain Intelligence SLM)<br/>高精度领域知识 / 深入业务生态"] Orch --> Comp Orch --> Dom end Frontier["前沿模型 API (Frontier Models)<br/>SEC / FINRA 通用监管知识"] -.-> Orch ``` * **合规 SLM（Compliance SLM）**：专门应对高监管、强策略与严格护栏约束的场景，确保敏感数据绝对不出内网。 * **编排 SLM（Orchestration SLM）**：专注于极低延迟调度与模型路由，使请求的分发与流转严格锚定在企业自身的逻辑和基础设施之内。 * **领域智能 SLM（Domain Intelligence SLM）**：整个体系的核心支柱，针对具体业务用例深度定制，核心目标是以垂直领域的“极高精准度（Precision）”超越宽泛的“通用智能（General Intelligence）”。 主权模型并不意味着与外部世界完全割裂。在实际落地中，Vanguard 采取主权模型与前沿基础模型（Frontier Models）混合并用的策略。例如在合规体系内，既有 Vanguard 内部独有的业务规则，也有来自 FINRA、SEC 等监管机构的公开通用法规。将前沿模型的外部通用认知与主权模型的内部掌控力结合，能够实现兼具控制力与广度的治理闭环。 --- ### [13:28](https://www.youtube.com/watch?v=brj0GjA84y0&t=808s) 告别“刷榜”幻觉：构建企业自主评估套件与业务智能复利 面对市场上层出不穷的模型跑分与营销宣传，演讲者指出公开 Benchmark 的数值往往掩盖了真实表现。行业内普遍存在“基准最大化/刷榜（Benchmark Maxing）”现象——当某套公开测试集在市场上存在过久，所有新模型都会针对其定向优化以取得好看的分数，但这些数字在进入特定企业实际场景时往往会迅速失真。 为了破除这种评估偏差，Vanguard 选择在企业内部自建专属评估套件（Evaluation Suites），紧密围绕真实业务用例建立基准。这不仅有助于在“主权自研 AI vs. 前沿模型 API”之间做出客观决策，也能在多个商业前沿模型横向选型时提供清晰的量化依据。演讲者强调： > “前沿模型解决方案的通用智能可能正在以指数级跃升，每一代都带来近 10 倍的增长；但在企业内部，我们必须确保自研的解决方案能在‘业务智能（Business Intelligence）’上实现同样的复利累积，而不仅是依赖通用智能。” --- ### [15:09](https://www.youtube.com/watch?v=brj0GjA84y0&t=909s) 破局实战：从 47% 错误率到 1.3% 的语音转写革命 这一套主权 AI 理念并非空中楼阁，而是源自团队初创期的一场攻坚硬仗——客服通话语音转写系统（Speech Transcription）。早期 Vanguard 尝试采用多家外部顶级供应商方案，但在测试公司内部真实客服录音时，词错误率（Word Error Rate, WER）竟然高达 47%。面对高达 47% 的错误率，下游一切旨在改善客户体验的分析与预测均无从谈起。 深入分析后发现，转写错误几乎全部集中在 Vanguard 专有的产品名称、术语及金融语境词汇上。例如在 Vanguard 的语境中，“Market”必然指代“金融市场/资本市场”，绝对不会是日常的“菜市场（Grocery Market）”。这种深植于企业基因中的语境是外部通用模型无法天然具备的。团队随即决定启动自研语音引擎实验： ```mermaid flowchart LR accTitle: 语音转写自研演进与降本增效 subgraph Evolution["WER 错误率演进"] direction TB E0["外部供应商: 47% WER"] --> E1["自研首版: 37% WER"] E1 --> E2["迭代1年: 10% WER"] E2 --> E3["当前生产: 1.3% WER"] end subgraph BusinessValue["业务规模与经济效益"] direction TB B1["年处理电话: 4,500 万通<br/>(平均通话 12 分钟)"] B2["若购外部服务: 约 $1,500 万/年"] B3["自研运行成本: < $50 万/年<br/>(几乎全为算力开销)"] B1 --> B2 B1 --> B3 end ``` * **技术指标跨越**：首个自研版本将 WER 降至 37%，经过一年打磨降至 10%，目前该方案在生产环境中处理 Vanguard 真实通话的 WER 稳定在 **1.3%**（相比之下，某外部模型声称公开基准 WER 为 4%，但在 Vanguard 真实场景下实测高达 47%）。 * **吞吐规模**：年处理通话量达 **4,500 万通**，平均单通通话时长约为 12 分钟。 * **业务价值延伸**：转写系统与下游自研数据脱敏（Redaction）模型相结合，构建了高度可信的数据底座。 * **惊人的成本节约**：若按商业外部 API 计费，同等规模年花费约 **1,500 万美元**；而当前全自动化的自研系统（每天清晨 6:00 自动拉起，非营业时间自动休眠缩容）全年在云端的运行算力成本**不足 50 万美元**。 * **规模经济性**：系统处理容量从最初的每年 2,000~2,300 万通翻倍至 4,500 万通，但分摊成本反而进一步下降。 这场胜仗彻底奠定了团队在企业内部的立足之本，证明了自研垂直领域模型能够为企业带来巨大的商业回报与战略主动权。 --- ### [18:29](https://www.youtube.com/watch?v=brj0GjA84y0&t=1109s) 进军大语言模型：MoE 与密集模型的架构选型权衡 在语音转写取得巨大成功后，团队将沉淀的能力逐步拓展至更广阔的 NLP/AML 领域。从早期的 BERT 架构、命名实体识别（NER）敏感信息脱敏服务，最终全面迈向大语言模型（LLM）的自研微调与主权训练目标（Sovereign Training Objective），核心课题始终围绕着：**“如何利用好海量专有数据，让模型以独一无二的姿态精准回答 Vanguard 生态体系内的问题？”** 在模型架构探索过程中，团队总结出了一套关键的选型准则： | 适用场景 | 推荐模型架构 | 核心优势与选型逻辑 | |:--- |:--- |:--- | | **特定领域认知 (Domain-specific)** | **混合专家模型 (MoE, Mixture of Experts)** | 生产环境中单次推理激活的参数量显著减少，大幅降低推理延迟；横向扩缩容的经济效益极佳，是承载整个业务领域知识的最佳架构。 | | **特定独立任务 (Task-specific)** | **密集模型 (Dense Models)** | 针对分类、命名实体识别（NER）等具体工作流智能任务，不需要模型承载庞大的领域全貌，密集模型在紧凑参数下表现更为稳定高效。 | 在此架构决策基础之上，团队进一步提炼出了模型训练的三大核心层级，其首要起点便是针对性的模型剪枝（Targeted Pruning），为后续的高效训练奠定基石。 ### [21:07](https://www.youtube.com/watch?v=brj0GjA84y0&t=1267s) 领域对齐架构：从专家混合到演化自愈系统 在构建企业专属的主权大语言模型（Sovereign LLMs）时，先锋领航（Vanguard）设计了一套自底向上的三层调优与对齐体系。第一层专注于模型的基础适应能力，通过专家混合（Mixture of Experts, MoE）与前沿基座大模型的结合，精准融入 Vanguard 内部的专有数据，从而达到金融级业务所需的极高精确度。第二层则是引入领域专家标注的基于人类反馈的强化学习（RLHF），将业务一线的实际结果转化为反馈信号，确保模型的行为模式与各业务利益相关方（Stakeholders）的操作直觉保持高度一致。 最顶层则是团队目前重点攻关的“演化自愈（Evolutionary Self-Healing）”机制。该机制的核心诉求是解决持续演进难题：模型不仅要依据业务成效被动更新，更要具备自主识别故障与缺陷的能力，明确“自身在何处失效并如何针对性修复”，进而自主提升在特定任务上的表现。Sagarika Suresh 分享了一个网络安全模型的真实案例：团队训练该模型以识别对抗性攻击（Adversarial Attacks），在引入演化自愈流水线后，随着时间推移，模型自主演进并成功识别出了近 48% 的对抗性攻击模式。 ```mermaid flowchart TD accTitle: Vanguard 模型对齐与自愈流水线架构 subgraph AlignLayers ["三层模型对齐与自愈体系"] L1["第一层：前沿模型适配与 MoE (专有数据融合)"] L2["第二层：领域专家标注与 RLHF (行为意图对齐)"] L3["第三层：演化自愈流水线 (自动化故障修复与迭代)"] L1 --> L2 --> L3 end subgraph TelemetryLoop ["推理遥测与业务反馈闭环"] T1["多轮交互上下文与顾问反馈 (赞/踩)"] T2["推理遥测数据与缓存层 (Telemetry & Cache)"] T3["目标任务评估与商业价值对齐 (客户财务成功)"] T1 --> T2 --> T3 end L3 --> T1 T3 --> L1 ``` --- ### [22:46](https://www.youtube.com/watch?v=brj0GjA84y0&t=1366s) 业务上下文捕获：超越“下一个词预测” 为了将自愈机制规模化落地，自动化成为了构建“模型工厂（Model Factory）”的核心支撑。Jithin Pradeep 强调，训练单点模型并不难，难点在于构建能够自驱运行的流水线。在探索自愈机制的过程中，团队发现各业务会话之间的反馈差异巨大。例如，在投资顾问与客户的沟通中，话题可能从简单的“度假预算规划”开始，延伸到“如果我想把度假时间从 3 天延长到 7 天，理财策略需要做哪些调整”。在这种多轮连续对话中，某个环节顾问可能会点赞（Thumbs Up），而另一环节则可能点踩（Thumbs Down）。 传统的模型往往局限于孤立的下一个词预测（Next-Token Prediction），但金融顾问场景需要深度理解连续事件的上下文脉络。因此，Vanguard 开始将交互上下文、元数据与业务反馈完整打包并捕获下来，构建了一个涵盖客户真实交互与业务目标的训练闭环。团队构建新一代模型的目标，是在具备基础预测能力的同时，直接对齐 Vanguard 的根本业务成效——帮助客户实现财务成功（Financial Success）。 --- ### [24:26](https://www.youtube.com/watch?v=brj0GjA84y0&t=1466s) 企业级主权 AI 的实战规模与生产指标 Sagarika Suresh 列出了一组具体数据，展示 Vanguard 在生产环境中落地主权 AI（Sovereign AI）的基础设施与应用规模： - **600 万+ 次语言 API 调用**：年初至今（YTD）企业级语言模型 API 调用量突破 600 万次。 - **4500 万+ 电话通话转录**：累计完成超过 4500 万通客户电话语音的高精度转录。 - **60 亿+ 敏感信息脱敏**：在所有语音与交互记录中，自动化脱敏（Redacted）拦截了超过 60 亿个敏感数据点。 - **2 周完成领域模型交付**：得益于完善的自动化流水线，构建并上线一个全新的垂直领域模型仅需约 2 周。 - **延迟降低 70%**：相比依赖外部前沿 API，自研主权模型将系统响应延迟直接削减了 70%。 - **100% 本地化自托管**：所有核心数据、模型权重与推理基础设施实现 100% 内部自托管（Self-Hosted），彻底消除数据主权与合规风险。 --- ### [25:32](https://www.youtube.com/watch?v=brj0GjA84y0&t=1532s) 生产环境推理挑战：异构算力排队与长上下文崩溃 Jithin Pradeep 坦言：“训练模型其实是相对容易的部分，大多数主权 AI 方案往往在部署和推理扩缩容阶段遭遇滑铁卢。”在实际生产中，模型的核心洞察与智能化提升往往源自推理阶段的实时数据反馈。因此，建立一套完备的“推理遥测（Inference Telemetry）”机制至关重要。有些遥测数据无需直接反哺给模型重新训练，而是可以沉淀到缓存（Cache）或规则库中，如何分类处理这些数据构成了关键设计考量。 此外，生产运维中隐藏着诸多极其隐蔽的系统级陷阱： 1. **长上下文内存溢出（OOM）**：模型在处理 8K Token 上下文窗口的请求时一切正常，但一旦负载突增至 37K Token，就会在特定边界条件下直接触发内存报错崩溃。 2. **异构 GPU 调度的“连锁反应”**：团队曾在云端部署了高规格的 NVIDIA GPU 算力集群。为应对偶发的流量洪峰，团队曾尝试动态混部一些算力较低的小型 GPU 来分流。然而实际运行中，小型 GPU 的推理延迟大幅增加，导致底层请求队列迅速堆积堵塞，而高性能 GPU 试图追赶处理也无济于事，最终引发了整体服务的雪崩。 这一系列教训表明，当模型上线并承载带有严格 SLA（服务等级协议）的生产级服务时，工程系统的健壮性远比单纯的模型准确率指标更为复杂和严苛。 --- ### [27:40](https://www.youtube.com/watch?v=brj0GjA84y0&t=1660s) 70% 投资在人与流程：企业 AI 的成熟度与治理进化 面对“自建（Build）还是采购（Buy）”的战略抉择，讲者建议企业默认从通用 API 开始切入，以此快速验证业务价值，而不是单纯为了做实验而盲目自建。在调用 API 的过程中，企业需要密切监测三大信号：专有数据是否在有效沉淀、业务基准评测（Benchmarks）是否证明通用模型遇到瓶颈、自建是否具备明确的 ROI。 在主权 AI 的总体建设投入中，模型研发本身仅占 30% 的工作量（其中 10% 属于算法研究，20% 属于基础设施搭建），而剩下的 70% 则是人员与流程（People and Process）： | 投资维度 | 占比 | 核心构成与企业关注重点 | |:--- |:--- |:--- | | **算法研发** | 10% | 模型微调算法、演化自愈机制、领域对齐策略 | | **基础设施** | 20% | GPU 算力调度、长上下文优化、推理遥测、自托管服务 | | **人员与流程** | 70% | 复合型 AI 人才团队、跨部门文化协同、合规风控与模型治理（Governance） | Jithin 强调，顶尖且具备抗挫韧性的人才是破局的关键，团队在日常探索中几乎每天都会遇到未知的技术挫折，需要具备快速调整 3 到 6 个月技术路线图的应变能力。在流程治理方面，依赖外部 API 时企业主要对 Agent 逻辑负责，而一旦转为自建底层核心模型，企业就必须承担全部的模型行为责任。 在技术路线上，Vanguard 并不拘泥于某一套特定的开源模型权重，而是强调“模型可移植性（Model Portability）”。团队的核心能力在于精准的模型编辑流水线——通过**定向剪枝（Targeted Pruning）**剔除无关知识，使模型做好吸收新领域的准备，随后借助监督微调（SFT）和强化微调（RFT）流水线，将企业特有的金融业务专知高效注入模型之中。这一成熟度演进将推动企业分阶段从最初捕捉主权信号，稳步迈向具备高度自主能力的“主权 AI 工厂”。 ### [31:34](https://www.youtube.com/watch?v=brj0GjA84y0&t=1894s) 先锋领航主权 AI 的三波演进浪潮：从高精度数据到模型工厂 先锋领航集团（Vanguard）在构建自主可控的主权 AI（Sovereign AI）过程中，清晰地经历了三个发展阶段。第一波浪潮始于数据层面的“主权信号”（Sovereign Signal）。团队首先从转录服务切入，经过深度优化，将转录的字词错误率（Word Error Rate, WER）降到了 1.3% 以下。极高的数据精确度向团队释放了明确信号：企业内部的数据质量已经完全就绪，足以支撑后续的高阶分析与全方位 AI/ML 任务。 进入第二波浪潮后，团队开始全力构建“主权语言模型”（Sovereign Language Models）。在这一阶段，研发的核心聚焦于特定业务领域与具体场景的极致精确度（Precision），团队与企业内部的各个业务利益相关方紧密协作，确保先锋领航独特的业务逻辑与专业知识被精准沉淀并编码到自建模型中。 目前，先锋领航已正式迈入第三波浪潮——构建端到端的“主权模型工厂”（Sovereign Factory）。这意味着企业不仅具备了全栈自托管的底层基础设施，更建立起一套持续迭代与自我改进的闭环系统。该阶段致力于捕获更多维度的客户交互信号，不断丰富数据内涵，确保模型在保持最高性能表现的同时，始终深度锚定（Grounded）在先锋领航的业务语境之中。 ```mermaid flowchart LR accTitle: 先锋领航主权 AI 演进的三波浪潮 subgraph Wave1["第一波：数据信号"] A["语音转录服务"] --> B["字词错误率 < 1.3%"] B --> C["验证数据就绪度"] end subgraph Wave2["第二波：主权语言模型"] D["领域特定建模"] --> E["业务利益方协同"] E --> F["高精度场景落地"] end subgraph Wave3["第三波：主权模型工厂"] G["全栈自托管底座"] --> H["多源客户信号沉淀"] H --> I["持续迭代与先锋业务锚定"] end Wave1 --> Wave2 --> Wave3 ``` - **数据质量基准**：语音转录服务的字词错误率（WER）压降至 1.3% 以下，成为启动全套 AI/ML 战略的核心数据指标。 - **演进三阶段**：从底层高保真数据（Sovereign Data）、领域定制模型（Sovereign Language），到全自动化生产与持续改进的自研体系（Sovereign Factory）。 --- ### [32:52](https://www.youtube.com/watch?v=brj0GjA84y0&t=1972s) 为何自建主权模型：打造 10 倍商业智能护城河与专属算力架构 走主权模型自建路线无疑意味着企业要承担更多工程与运维重任，但其背后的动因非常纯粹——"因为你想构建专有智能（Proprietary Intelligence）"。如果一家企业完全依赖市场上人人唾手可得的通用 API 解决方案，就意味着其提供的服务也将走向同质化。先锋领航拥有独特的客户运营模型、产品与服务体系，每日都在网页交互、在线客服及客户通话中积累海量信息。随着未来客户直接与 AI 代理展开对话，这些交互所产生的实时反馈至关重要；企业若不能自主捕获并将其编码进私有系统，长达十年的知识资产积累就会面临严重断代与流失。 在底层系统架构与算力部署上，自建方案能够彻底摆脱公有 API 的限制。先锋领航为不同业务应用搭建了完全专属的推理端点（Dedicated Inference Endpoints）。面对十余个并行的企业级应用，专属服务可以灵活按需伸缩：有的应用仅在清晨 6 点到 9 点处于高峰期，随后便可自动下线以节省算力；而对于需要 7×24 小时高可用运转的核心服务，专属端点则能彻底杜绝公有云 API 突发的速率限制（Rate Limits）。 此外，金融业务涉及大量高敏感与长文本场景。试想当成千上万的客户同时上传并分析自身体量庞大的税务报表（Tax Documents）时，如果全量调用外部前沿模型，Token 消耗成本将瞬间爆炸。通用前沿大模型确实提供了廉价易得的“通用智能”（General Intelligence），但在高壁垒的金融场景中，企业必须在利用外部通用智能的同时，将自身的“商业智能”（Business Intelligence）同样提升 10 倍，方能建立起最具防御力的长期竞争优势。先锋领航的实践核心始终在于——"从业务痛点与缺口出发，其余的一切自然水到渠成（Start with the gap, and the rest will follow）"。 - **专有智能 vs. 通用智能**：公有前沿模型赋予市场通用的基础能力，但无法替代由独特客户反馈与业务流程沉淀出的专有商业智能（10x Business Intelligence）。 - **专属推理端点优势**： - **动态弹性伸缩**：支持晨间高峰（如 6:00 - 9:00）运行并在闲时关闭，兼顾性能与成本。 - **高可用保障**：避免关键 7×24 小时业务触发公有 API 速率与并发上限（Rate Limits）。 - **成本与隐私可控**：在客户高频上传大型税务文档等重度 Token 消耗与强监管场景下，自建推理极大规避了 Token 成本失控与合规风险。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=brj0GjA84y0&t=4s) 在开启企业级 AI 的演进之旅时，最关键的问题往往不是“如何做（how to do it）”，而是“何时做（when to do it）”。 > [02:38](https://www.youtube.com/watch?v=brj0GjA84y0&t=158s) 先借助前沿 API 极速验证业务可行性，在捕捉到更强的主权指标、数据壁垒显现且规模达到临界点时，再平滑转向主权模型。 > [06:53](https://www.youtube.com/watch?v=brj0GjA84y0&t=413s) 每日调用量接近或达到约 500 万次请求（5 Million Daily Requests），即为触发自主模型构建的关键财务与规模阈值。 > [10:07](https://www.youtube.com/watch?v=brj0GjA84y0&t=607s) 企业级编排器绝不能仅仅是一个粗粒度的“成本路由器（Cost Router）”。 > [13:28](https://www.youtube.com/watch?v=brj0GjA84y0&t=808s) 前沿模型解决方案的通用智能可能正在以指数级跃升，每一代都带来近 10 倍的增长；但在企业内部，我们必须确保自研的解决方案能在‘业务智能（Business Intelligence）’上实现同样的复利累积，而不仅是依赖通用智能。 > [18:29](https://www.youtube.com/watch?v=brj0GjA84y0&t=1109s) 如何利用好海量专有数据，让模型以独一无二的姿态精准回答 Vanguard 生态体系内的问题？ > [25:32](https://www.youtube.com/watch?v=brj0GjA84y0&t=1532s) 训练模型其实是相对容易的部分，大多数主权 AI 方案往往在部署和推理扩缩容阶段遭遇滑铁卢。 — Jithin Pradeep > [32:52](https://www.youtube.com/watch?v=brj0GjA84y0&t=1972s) 如果一家企业完全依赖市场上人人唾手可得的通用 API 解决方案，就意味着其提供的服务也将走向同质化。 > [32:52](https://www.youtube.com/watch?v=brj0GjA84y0&t=1972s) 从业务痛点与缺口出发，其余的一切自然水到渠成（Start with the gap, and the rest will follow）。 ## Mentioned 先锋领航集团 (The Vanguard Group), Jithin Pradeep, Sagarika Suresh, Ai4, NVIDIA, 美国证券交易委员会 (SEC), 美国金融业监管局 (FINRA), 主权 AI (Sovereign AI), 小语言模型 (SLM), 混合专家模型 (MoE), 密集模型 (Dense Models), 演化自愈系统 (Evolutionary Self-Healing), 基于人类反馈的强化学习 (RLHF), BERT ## Video description Jithin Pradeep - Head of Enterprise AI & Research, Vanguard Sagarika Suresh - Manager, Applied AI R&D, The Vanguard Group Frontier AI models have made it easier than ever to build intelligent applications, but when does it make sense for an enterprise to build and govern its own language models instead of relying solely on external APIs? This talk explores how to evaluate the build-versus-buy decision for AI models in a heavily regulated financial environment, where intellectual property, data sovereignty, governance, and long-term operational costs play a critical role. We will share practical lessons learned from developing, and deploying domain-specific language models at enterprise scale, including the tradeoffs between proprietary and frontier-model approaches. We will share a strategic perspective on model ownership, cost optimization, and the advantages of controlling the AI models. If you are responsible for bringing AI into enterprise ecosystems and making long-term platform decisions, our talk will provide a framework for evaluating when sovereign AI becomes a durable competitive advantage. [Source](https://www.youtube.com/watch?v=brj0GjA84y0)

[0:04] 在开启企业级 AI 的演进之旅时，最关键的问题往往不是“如何做（how to do it）”，而是“何时做（when to do it）”。 [2:38] 先借助前沿 API 极速验证业务可行性，在捕捉到更强的主权指标、数据壁垒显现且规模达到临界点时，再平滑转向主权模型。 [6:53] 每日调用量接近或达到约 500 万次请求（5 Million Daily Requests），即为触发自主模型构建的关键财务与规模阈值。 [10:07] 企业级编排器绝不能仅仅是一个粗粒度的“成本路由器（Cost Router）”。 [13:28] 前沿模型解决方案的通用智能可能正在以指数级跃升，每一代都带来近 10 倍的增长；但在企业内部，我们必须确保自研的解决方案能在‘业务智能（Business Intelligence）’上实现同样的复利累积，而不仅是依赖通用智能。 [18:29] 如何利用好海量专有数据，让模型以独一无二的姿态精准回答 Vanguard 生态体系内的问题？ [25:32] Jithin Pradeep: 训练模型其实是相对容易的部分，大多数主权 AI 方案往往在部署和推理扩缩容阶段遭遇滑铁卢。 [32:52] 如果一家企业完全依赖市场上人人唾手可得的通用 API 解决方案，就意味着其提供的服务也将走向同质化。 [32:52] 从业务痛点与缺口出发，其余的一切自然水到渠成（Start with the gap, and the rest will follow）。

[0:04] Sagarika Suresh and Jithin Pradeep: The critical question surrounding sovereign AI is not simply how to build it, but fundamentally when an enterprise should choose to do so. [2:38] Sagarika Suresh and Jithin Pradeep: $0 cost is sort of a myth [26:25] Jithin Pradeep: The smaller GPU started taking a lot more time to do the inference and the queue started getting bigger and bigger, and the bigger GPUs were trying to catch up, but they were not able to. It's not just about training the model and seeing it performs really well. It becomes a lot more when you have to put it out there and the service runs for your client with an SLA on top of it. [24:26] Jithin Pradeep: training the model is the easy part. [22:46] Jithin Pradeep: Hey, I would like to have that vacation to be 7 days instead of 3 days; what should I do different? [27:40] Jithin Pradeep: start with an API [27:40] Jithin Pradeep: Ah, we should not have said that. [32:52] Jithin Pradeep: because you wanted to build proprietary intelligence. [34:15] Jithin Pradeep: start with the gap and the rest will follow

[0:04] Um as I introduce myself I'm Saga Suresh. I'm  the manager for applied AI R&D vertical for [0:10] enterprise AI research team and joined with me  is my colleague Jitan Prahep. So let's continue. [0:17] Um so the question about sovereign AI the most  important question is about not how to do it. [0:25] more importantly like when to do it and we over  the per period of last seven years in this journey [0:33] I think you know three things stands out to us  which actually tells us when to start thinking [0:38] about so AI and developing those solutions again  developing AI solution has lot of dimension that [0:46] you would have to consider it's not just about  building the model till you cannot put it into [0:50] production and create value out of it it's  a very lengthy journey so once you wanted to [0:54] take this decision you need to these signals at  your enterprise to ensure that you are taking the [0:58] right step. So the first one for is for us is like  you know data that other people have not seen. You [1:05] should have a data that is so proprietary to you  that you don't wanted to send it outside and you [1:11] wanted to keep it for yourself. It could be for  various reason the business intelligence um how do [1:16] we process what not and as Jan mentioned most of  this data that we have touched upon in the first [1:23] uh you know box that you see here is static data  and this data will you know evolve over the time [1:29] and move on to become more dynamic as your systems  evolve into production environments. So if your [1:35] system is producing more of data continuously for  instance in the uh you know case of the Vanguard [1:41] group you have a lot of client interactions  happening a lot of you know information flowing [1:45] in from uh call summarizations and so on. So this  data is continuously evolving over time there's a [1:51] lot of dynamic aspects that's you know involved  in it. Uh that's your you know second signal or [1:56] indicative that hey this is where sovereign AI  can be used. And the third indicative signal [2:01] is basically frontier frontier AI structurally  cannot reach your standard. So we have most of [2:06] these open- sourced commercial models uh and so  on. Uh the thing with these open source models [2:12] uh is that you they are you know evaluated and  benchmarks against public data sets. So they're [2:18] not fine-tuned to your specific company needs  or your enterprise needs. Right? So that's also [2:23] indicative that hey like if you don't want  your model to be evaluated on these public [2:28] benchmarks which most of these models are then  it's you know your third signal to move to this [2:33] uh you know the third indicate ind indicator  here always when we give this three indicator [2:38] the question becomes like are you guys talking  about just having a solen AI in your enterprise [2:42] and the answer is no it's a sounding no that's  not the case our strategy has been over the years [2:49] to think about build plus y rather than being  tradition approach and start thinking about [2:54] build bosses spy and there are reasons for it you  when you are trying to have ROI in today's world [3:01] we have this discussion a lot in terms of now how  AI can generate value for your enterprise velocity [3:07] is a key so as the cost and other aspects of  it so the first three point pointers that we [3:13] have with respect to the data that data being  compounding or you having a process that this [3:18] is going to keep generating and now you know  that you know there is a mode of data which [3:23] is not just there as a static component but each  year you see an increase in them and finally you [3:29] start realizing a solution that is right in front  of you is measured with great numbers but when [3:34] you apply to your own use cases you start seeing  oh there are gaps so because of this we have the [3:40] strategy of you know going with frontier solution  and then pivoting when the right time comes to a [3:46] sovereign AI if there are more indicators to  support this. So in 2026, you can call the [3:53] Frontier AI API and get 90% of the accuracy within  almost close to 90 seconds. And Frontier APIs are [4:02] genuinely transformative. So you don't want to  be that enterprise that you know dismisses them. [4:07] So to give you an example right uh we reach you  know it's almost close to uh zero dollars uh in [4:14] in you know actually building or training most  of these models uh frontier uh models out there [4:19] and when these models were initially released they  started off you know uh with small parameter sizes [4:26] and today we're almost at two trillion plus  parameter sizes of these models itself and we [4:31] expect them with you know fable and models such as  that to almost go up to 10 trillion now One of the [4:37] good parts about that is that you know there's  so much data that it's already trained on. It [4:41] has so much skill sets from the public knowledge  and then finally it's almost zero uh you know in [4:46] terms of maintenance uh overhead it significantly  reduces your operational burden. So what I mean [4:52] by that is that you're not you know the model  the models uh builders they are essentially [4:57] providing you with you know the infrastructure  the model itself it's a simple API call away. [5:04] Having said that, putting things into production  is not easy either. So you know that 0 uh is a [5:11] amount that is sort of a myth too in one way  or the other because all the speakers on who [5:16] has taken stage across this last three days  has spoken about the importance of you know [5:21] taking this to production. So experimentation  to production is a different story. But again [5:26] with frontier solution you don't have to do lot  of um like you know infrastructure and do the [5:33] heavy lifting. it's already there in the form of  an API and that's always going to be a big win. [5:38] Now next thing is okay we are here the training  data is here where do you look for the training [5:44] data this is a question that we keep getting like  what is that important piece of data that you [5:49] would say okay we need to use and that can be sort  of you know helping you to build that first AI [5:56] model within the enterprise that could have more  usage and it always comes down to tapping into [6:02] those sources that are not being utilized within  the company uh because you right in this room I'm [6:08] pretty sure everyone is going to raise hand now if  I say if any of you have been blocked because you [6:14] cannot send this data outside one way or the other  we are fl like you know in an enterprise you're [6:19] going to be at that stage where say oh we cannot  send this data outside this is very important [6:23] data and there could be insight out insight that  we could leverage out of it but this is not the [6:30] right way to do about it so the sovereign models  or the solution that gets built are usually based [6:36] on those data that you kind of reluctantly say oh  I don't want to send it outside but anything that [6:42] you have already publicly available frontier APIs  would be able to you know do a great justice to [6:47] like you know when it comes to performance and the  knowledge that it has. Well, with that question, [6:53] our leadership team um with the enterprise year  research team uh we had to make some decisions [6:59] right in terms of when exactly do we start  building uh and that's where you know this [7:05] is a break even point that we desered. So for  instance, if your cost is going up really high [7:10] and you know your de daily inference volume is  also increasing a lot, it is a sweet spot for [7:16] you to start building your models. And I also  want to additionally add to that that you know [7:21] if you don't want these models to be specifically  around you know public benchmarks uh and you want [7:27] to fine-tune it to your domain or that specific  use case that's you know an additional point in [7:32] terms of hey like this is where you know we need  to make that leadership call in terms of building [7:36] our uh models. uh we have done the math. You  can uh you know as you see here uh a break [7:43] even point or a threshold would be close to like  almost close to 5 million daily requests to it. [7:51] So u from a data standpoint right again like you  know to start thinking about what are we trying to [7:56] capture. So when the deployment happened we have  this ideology of you know this data need to get [8:01] smart on us and that means like you know you are  trying to capture important feedback signals in [8:08] like you know production to identify and the way  it gets started is like you know everyone of you [8:14] know it would be like a thumbs up thumbs down  maybe some you know comments that gets edited [8:18] along with and you know you start thinking  about how you can start leveraging this to [8:23] improve. We're going to cover a bit more about  self-healing but it's a concept that we have [8:27] at Vanguard of improving AI solution using AI  itself and in order to do that we sort of think [8:34] of the feedback signal to be two different one is  where you get it from the interaction itself and [8:41] the other is the training data you have lot of  training data that you're trying to do but the [8:46] feedback is thumbs up thumbs down and most of the  AI today is a chat interaction or a session based [8:52] interaction so in order to reach out to a thumbs  up or a thumbs down. what all else happened from [8:57] a metadata perspective to capture it so that  we can then train the AI to say this is the [9:02] path that we wanted to avoid but you would see  start seeing other components along with a you [9:08] know if I have to say like you know a happy user  or like you know dissatisfied user the thumbs [9:12] up and thumbs down is just a mean to say back to  the model and that's a great point um and in this [9:19] format what you start doing is your intelligence  or you know your signals keeps compounding over [9:24] time So we aren't just like working with the  data that is used for training the you know [9:29] uh model itself but we're also starting to now as  we compound our mode uh take feedbacks directly [9:34] from the business uh business use cases itself  right so when they are giving you these feedbacks [9:40] you're also putting in okay this is what this  particular use case uh is uh you know requiring [9:45] for this model to perform and over time it's  continuously improving we're you know looking [9:49] at more improvements towards the model itself  and we don't rely on just the data training [9:54] mostly because uh you know we we don't  want the model to you know start sec [9:59] uh go going into areas such as like we're trying  to prevent from reward hacking essentially. [10:07] So the choice is not binary. This this is  something that you would hear again and again but [10:12] um build plus PI um we strongly believe in this  hypothesis of you know creating an ecosystem where [10:20] these AI solutions that we wanted to build and  what is available are part of the bigger puzzle [10:25] so that we can use them effectively where it  meets and this would you know get us to this [10:31] conversation of you know an application getting  down to an orchestrator layer and orchestrator [10:35] actually deciding between these models. Now  traditionally an orchestrator that you would [10:40] have would be like in a cost router and from what  we have seen it has to be more than a cost router. [10:47] It cannot be just based on okay this is an complex  task I need the biggest model that is available [10:52] to do the planning and then the subsequent smaller  model is going to take um individual like you know [10:57] smaller task from that perspective how it's more  of like you know how can you embed your business [11:02] logic also in there. It's sort of like you know  if you if you have a website and it's that like [11:07] you know entry to your entire business this  orchestrator actually leaves in there so that [11:11] we can get away from the traditional system  of clients having to navigate through them and [11:15] identifying where they have to go specifically for  a business like Vanguard when our website consist [11:20] of lot many offering from different different kind  of business perspective and user persona matters [11:25] a lot we would want people to you know understand  and what happened behind the scene with AI should [11:30] be like you know abstracted from them they should  get that right so That's this layered architecture [11:34] approach that we have seen which we believe is  what we are working towards do happen. So yeah so [11:42] with that comes the sovereign stack. So there are  three types of SLM archetypes that work together. [11:47] The first one being compliance SLM. So when your  question is more around hey like my particular [11:53] use case has a lot of policies it has a lot of  guardrails we don't want the data to go out that's [11:58] when you go more into compliance SLM building.  Uh the second archetype is essentially your [12:03] orchestration SLM. So you want to save on latency.  You want to make sure that you know the way your [12:08] routing is done between those SLMs stays grounded  to your logic and to your infrastructure. That's [12:14] the second archetype which you see in there which  is your orchestration SLM. And the third archetype [12:19] which is essentially your domain intelligence  SLM, right? So that's the most crucial one and I'm [12:24] sure you're all fairly familiar with it. So you  start working with your different uh you know use [12:29] cases as well as teams and start building on hey  like this is how your language model needs to be [12:34] designed for this particular use case right so the  way we achieve you know um importance with these [12:41] uh arch especially the third archetype is that hey  we have more precision over general intelligence [12:47] um so yeah with that said those are the three  you know archetypes that we wanted to touch upon [12:55] so as you mentioned And you know those are few  examples that you know and like at least at [12:59] Vanguard we are feeling like you know okay place  where we could use both frontier as well as like [13:03] you know the sovereign models together too at  times. Um like as an example for compliance [13:09] um we could think about business policies  and rules that are unique to us but also [13:14] then you know as an regulatory uh industry like  Vanguard uh we are also looking at FINRA SEC and [13:20] that could be a general knowledge that can  come together. So how do we bake this thing [13:23] together and have better control over it. So  then you know going around you know a bias [13:29] um I do feel like there's a lot of marketing that  gets done right and sometime the numbers in front [13:34] of you doesn't tell the entire story and that  has been happening with you know a lot of this [13:39] model number uh the benchmarkings uh and you  would hear words like you know benchmark maxing [13:46] uh you know those kind of phenomena where a  benchmark is in the market for so long that now [13:51] all the models are doing lot better on them right  um But we what we have felt is over the period [13:57] of time you would start in the company itself you  would start creating your own evaluation suits and [14:03] you would start like you know benchmarking against  your use cases combining them actually creates a [14:07] benchmark that is more relevant to you and that  allows you to make more like you know really [14:13] important decisions for example so like know  sovereign AI versus frontier like you know API [14:18] is a different question but even within frontier  solutions which one to use for a particular you [14:23] know case So we do feel like you know those kind  of uh objective measurement with respect to your [14:29] business intelligence is very important. What we  tend to see with frontier solutions is their um [14:35] intelligence is increasing exponentially. You can  even say like it's a 10x increase on each model. [14:41] Um but at the same time within an enterprise we  also wanted to ensure the same thing is happening [14:46] with your business intelligence. So that the  intelligence solution that you are building is [14:50] also compounding on the business intelligence  piece of it not just the general intelligence. [14:58] So uh we wanted to you know talk few places where  it gave us the confidence and where we started [15:04] deducing you know these kind of steps is the right  walkway for us to you know continue our path on. [15:09] So initially at the very start of this team we  were looking at you know various solutions and [15:15] one of the problem that we started to tackle first  was about speech and it's a transcription problem. [15:21] Uh we started with solutions with like you know  various vendors and we were looking at our phone [15:26] calls being at 47%age word rate. Worder rate is  basically an error that occur while transcribing [15:33] a word like a word by an AI solution and this were  of the range of 47%age. Now with a 47 percentage [15:40] word rate you cannot do any analytics that can  help support client experience or make any like [15:45] you know guess right and we could clearly see  that the errors were originating from the words [15:51] that is unique to Vanguard or it's more talking  about our products and you know things related to [15:57] financial industry. Um another thing that like you  know we keep saying a lot is like you know when [16:02] Vanguard says market it's definitely financial  market it's not grocery market. So there are [16:06] certain things like you know which is very much  imprinted to our business and you know we don't [16:11] have to think twice that's a context given right  so we started building our own speech solution [16:18] uh it was an experiment to get started with with  a talented team we were able to get down to the [16:23] first word rate with 37%age it took an year from  there to like you know perfect it we got down to [16:29] 10%age today this solution runs in production  with 1.3%age word rate on Vanguard phone calls [16:36] Now I'm emphasizing the term Vanguard phone calls  or you know Vanguard communication because we are [16:41] not now looking at what the public benchmark  is whereas the public benchmark which was 4 [16:46] percentage word rate on certain model turned  out to be 47 percentage word rate for our like [16:53] you know purpose. So this actually started us  you know in this direction of you know thinking [16:57] what is best for us when it comes to AI and how  can we evaluate that. Now this solution from a [17:03] value perspective which becomes very important if  you're building an AI solution you need to ensure [17:07] like you know you have a value way to bring  value back to the organization. Today it does [17:12] 45 million phone calls a year. Each phone call is  on average like 12 minute that transcription goes [17:17] through the service and it supports downstream  applications too. and as well as like you know [17:24] this creating a data product downstream where  we have redaction and other mechanism based on [17:28] our own models which actually helps to create a  trusted data source for further analytics. So we [17:34] were looking at what could be the price for this.  It's somewhere around $15 million a year. And this [17:40] entire system today so mature that it comes  up live at 6:00 in the morning goes down when [17:46] everything is done when the business hours are  not done. It's optimized for that fully like you [17:51] know automated and it's less than half a million  dollar for you know running this entire thing [17:55] and the half a million dollar is mostly coming  from the compute cost of you know running that. [17:59] So there is over the and you know it can scale  well too. when we started it was like you know [18:03] around 20 23 million phone calls that would come  in in yearly basis now it's doing the double the [18:07] volume but we are still able to reduce the amount  that we were charging previously so this actually [18:12] set the stage for us and I do feel like you know  my team's foundation is based on this particular [18:17] project because it gave us lot of handsight  into like you know what what can AI do for a [18:22] company how can you create something that would be  generating more value from here we go to the large [18:29] language models yeah and that's a very phenomenal  story by itself in terms of hey like how do you [18:34] you know make sure that you're getting most of the  data like for instance calls is one of the biggest [18:38] data for us right so we laid the foundation  there and you know most of the classic you [18:44] know AML methodologies like bird we started off  from there right and then we also moved to named [18:49] entity recognition for data reduction services  and eventually we started moving more into you [18:55] know language models itself the you know how  do you actually start fine-tuning models how [18:59] do you build your own internal proprietary large  language model. Um, and that's where you know the [19:04] sovereign training objective came into picture.  The question that we kept asking is we have all [19:08] of this data. How do we make sure that we are you  know embedding and making sure that our models are [19:13] very uniquely positioned to answer questions  around our ecosystem and Vanguard itself. [19:18] Um we're also we you know we also started  benchmarking with the frontier models and as [19:24] we have kept you know our uh talk track as well  like the frontier models have had their unique [19:29] advantages in this place but then definitely you  know we started bringing those frontier models and [19:34] also adopting two vanguard specific languages  into different versions of our language models [19:39] and that's the neat story that you know in terms  of how we have built our products uh right from [19:44] research all the way to actually you know into use  cases uh to advance Vanguard and over the period [19:51] of time there were multiple questions that the  team has you know solved and things were around [19:56] architecture of a model what kind of different  architecture we should consider and I think this [20:02] might be uh one of the key learning that we have.  So when it comes to a domain based activity and [20:07] you wanted to have a model on a specific domain  you have to consider mixture of expert model. It [20:12] seems to be like the best working solution from  an architecture perspective. So we have that [20:16] uh it helps us with uh the latency for sure  because the active number of parameter running [20:21] in production is lesser uh as well as like  you know um the economics of you know how do [20:27] you want to scale we feel like you know it's a  lot better when it's for a domain but let's now [20:32] take a different aspect of it like you're talking  about um task uh specific task that you wanted to [20:38] run dense models are better like you know if  you're trying to come down to classification [20:42] neer and those kind of intelligent task that you  wanted to use in your workflow, tense model seem [20:47] to be better where you don't have to like  you know put down a domain into a model. So [20:56] having said that uh how do we train our  models? So there are three highlevel [21:02] layers to this. So the first layer being targeted  pruning. So we make sure that you know there's [21:07] weighted adjustments in terms of hey how are  our you know models working? what's the best [21:11] as Jin touched upon we're using mix mixture of  our experts to understand you know how do we use [21:16] these frontier LLMs to make sure that they're  actually uh using the data here at Vanguard and [21:22] bringing in that precision that we're looking  for. So that's the first layer that you know we [21:27] on uh for the second layer which is essentially  the RHF with domain expert annotations. This is [21:34] where we start bringing in more of the feedback  signals itself from the business outcomes that [21:38] are there at place. So we are trying to make  sure that there's behavioral alignment here [21:43] with the actual stakeholders who are using it.  And the mechanisms that we use is essentially [21:48] uh reinforcement learning through human  feedback. And the final layer that you see [21:52] in here is basically uh what we're working on  uh in terms of evolutionary self u healing. So [21:58] the evolutionary self-healing aspect of it is  basic uh is essentially hey we have all these [22:04] business outcomes we're improving the model but  how do we make sure that there it's continuously [22:08] improving right the model is able to understand  that hey this is where I'm failing this is how [22:12] I can improve and uh you know eventually making  making itself much better to that particular task [22:19] at hand. So to give you an example, we had a cyber  security model um and we were training it for you [22:25] know identifying advers having adversarial  attacks in there. So eventually uh with the [22:30] self-healing pipeline you know that we had it  was able to figure out almost close to 48% of the [22:35] attacks that were in there over time uh through  the self-healing pipeline that we built. Um [22:46] so as the AI uh like you know as Saga spoke about  the self-healing part uh the key uh element for us [22:54] become you know automation and uh in the future  slide we'll talk about waves and the last wave [23:00] is like you know where you could scale it as  a factory. So building a model is not an easy [23:05] task. So we wanted to ensure like know there  should be automation in place and we started [23:09] looking at self-healing as a mechanism and during  the self-healing part um we started stumbling [23:15] upon you know this feedback being so different um  which each session let's say somebody is starting [23:21] to talk about I'm going to take an example at  Vanguard we start talking about a vacation okay [23:26] and from there we say like you know okay let's go  uh and the conversation goes into um hey I would [23:32] like to have that vacation to be 7 days instead  of 3 days what should I do different? This is a [23:36] conversation with an advisor. So there is this  you know sequence of event that is happening and [23:42] how they you know translate to various knowledge  right maybe for the first question it's a thumbs [23:47] up or the second question it's a thumbs down  but we can know now what what all different [23:51] contexts were there so we started capturing the  context along with you know the data that we are [23:56] seeing so that we can create a training loop  which is considering the client interactions [24:01] uh as well as like the business outcome which  is going after and the newer models that we are [24:06] trying to build is based bas on that we don't  want to just build a model which is now based [24:11] on next token prediction. We wanted to build  a model which does next token prediction but [24:15] is also aligning with an business outcome of  itself at Vanguard at a simple term financial [24:21] success for the client. How are we navigating and  helping them to you know go down to that path. So [24:26] um that being said um right so with all of these  different uh you know models that we have built [24:35] as well as you know transcription services that  we spoke about what does sovereign AI actually [24:40] look like at enterprise scale. So these are the  numbers that you know speak volumes. Um we have [24:46] almost close to 6 million plus uh language API  calls made uh year year to date and 45 million [24:53] plus phone calls which are transcribed. Uh and we  also have close to uh 6 billion plus you know data [25:00] redacted in our uh all of our phone calls. And  finally if we were to build a domain model our [25:07] pipeline is set up in such a way that we can build  it within almost close to 2 weeks right uh and [25:13] our latency itself has been reduced by almost 70%  and 100% of this information is self-hosted uh so [25:20] having said this we have had a pretty phenomenal  story line in terms of how we want to also capture [25:25] the amount of data that we have to make sure that  you know our models are getting better over time. [25:32] So uh when it comes to training a model um I'm  not like you know trying to undermine it but [25:39] training the model is the easy part and most  of the sovereign AI solution would fail when [25:44] you have to deploy them and the inference need  to be scaled. Um you would start realizing that [25:51] you know the model intelligence uh the factors  that can be captured for you know getting better [25:59] intelligence actually comes from the inference. So  having an inference telemetry sort of to capture [26:05] those data points and bring it back to the model  or the pipeline. In some cases this data doesn't [26:10] have to go back into the model. This can be in  a cache. This can be in other places too. It [26:14] doesn't have to. So but in in later cases  it might have to go back to the model. So [26:19] identifying what they are where where they will  like you know be made available that becomes very [26:25] important. And then the things of the nature like  you have a model in production today and you see [26:30] a memory error. Uh maybe for a very specific  scenario and then you might not see it at all [26:35] like as an example an 8K token window request  was good but at the moment it started going to [26:41] 37 token K window you know we started seeing an  error. So how do you start capturing them because [26:45] that would impact your like you know solution.  Um the another example could be you have a stack [26:51] of you know different kind of GPU and this is  something that we felt we had this GPUs in place [26:57] u great like you know Nvidia GPUs which is being  using in our cloud and we thought you know okay [27:02] why not like if there's a spike let's add like  you know smaller GPUs to ensure the spike is [27:08] being balanced but what just happened was you  the smaller GPU started taking lot more time [27:12] to do the inference and the queue started getting  bigger and bigger and the bigger bigger GPUs were [27:17] you know trying to catch up but they were not  able to. So there are these things where you're [27:22] trying to operate at scale and you learn lot  more things in there. So it's not just about [27:28] training the model and seeing like you know it  performs really well. It's becomes lot more when [27:32] you have to put it out there and the service  runs for your client with an SLA on top of it. [27:40] So um where are we going from here?  So start with an API for sure, right? [27:46] The most important part in every conversation  that we wanted to highlight is we are trying [27:51] to generate value. We're not just trying to  do an experiment. So sometime it's important [27:57] that you know you start with an API or maybe  like it's always it's a default mode you start [28:01] with an API and as you start working with the  API you could search for those signal that we [28:07] just spoke about those three signals that we  spoke about about the data is the data getting [28:12] accumulated is there benchmarks that is proving  that you know you need to build your own solution [28:21] so building the model is 30% of the work  as Jan just mentioned So 10% in algorithm [28:27] uh 20% in infrastructure and 70% of this is people  and process right and this is the scale that we're [28:33] measuring it against individually definitely  there's a lot of work that goes on behind each [28:38] of those but when you look at it on a broader  spectrum you know 70% is the actual number in [28:44] terms of hey like people and process this is  the amount of investment that goes into that [28:50] and I know like we have a order there but I think  you know the most important part is talent the [28:55] talent who is trying to build this solution and  we need to bring different kind of people together [29:01] to you know build out this solution and the  mindset that is needed here it's very different [29:08] uh I'm not trying to say like you know that's not  every team there but you know the kind of setback [29:13] that keeps coming at you you know you got to like  you know jumping over it and you need to have that [29:17] right talent right there so that you know you can  make a call what need to be the next three month [29:22] journey what need to be the next six month journey  uh I would say like you know journey with my team [29:26] has been very interesting but it's also very  eventful on a day-to-day basis so you know even [29:30] today morning we have something eventful and I  was just talking to one of my colleague there you [29:33] know we were talking yesterday about how we have  learned things and we have learned together it's [29:38] all good and today morning we like ah we should  not have said that you know something happened [29:42] today so that's definitely important bringing that  talent uh piece together also the processes like [29:48] governance is an important function of it um with  APIs you are building agent Now if you say that [29:55] you know you're going to build a model which is  underneath powering it the governance has to be [29:59] way more and now it's your model right you're not  getting this model developed by somebody you have [30:04] trained this model you are more accountable in  terms of you know what gets happens inside on [30:08] the flip side it als it's also true that you  know now you have more control over it but [30:12] more control comes with more responsibility in  terms of you know making sure this is the best [30:16] version possible that you're going to put in  front of your clients too at one point in time [30:20] I know most enterprise starts with internal use  cases But the goal is to go client. So over the [30:25] period of time that maturity need to you know keep  coming. So this sort of you know what a leadership [30:29] would be thinking when you know when you have this  you know when people like me go in there and say [30:34] like we need to build a model we need to start  thinking about that maturity cycle that's not [30:38] a one-year cycle. It's a cycle that happens over  the year and you get down to that maturity where [30:43] you can say yeah know we can now go in a client  facing role. The other one is model portability. [30:49] Um like you know you could have different kind of  model. Our strategy has been different that this [30:53] is like vanguard perspectives like you know what  we have been trying to do. We we're not tied to [30:57] a weight. Uh we are more tied to like you know  how the training is happening how can you edit [31:02] a model so that we can infuse the knowledge that  we wanted to infuse in this AI. So our approach [31:07] about targeted pruning which helps us to you know  ensure the model is ready to learn what we wanted [31:12] to learn and also like you know strip off areas  that it's not relevant to us anymore. So we do [31:18] that and then you know focusing on SFT RFT uh kind  of pipelines to infuse the knowledge back into. [31:26] So with that there are three ways um from first  sovereign signal to the to a sovereign AI factory. [31:34] So the first sovereign uh signal uh you know being  you know what type of data do we have right for [31:40] instance we started off with our transcription  services as I mentioned earlier our transcription [31:46] services uh the word error rate was almost less  than 1.3%. That was one of the indicators that [31:51] hey like our data we are pretty much ready in  terms of utilizing this data for analytics or [31:57] any AI ML uh purpose. The second wave eventually  started with hey like sovereign language and what [32:03] are the values that we are generating out of  it right so this is where we started building [32:07] our language models um and we wanted to look into  more precision uh into that domain or the use case [32:13] itself and started working with you know different  stakeholders within the enterprise to make sure [32:17] that these are captured within our models and now  we are at a stage where you know we're creating a [32:22] sovereign factory so which essentially means that  you know there's continuous improvement there are [32:27] models in here which are hosted are infrastructure  is set up end to end and this wave is also you [32:32] know in terms of like capturing the different  types of data. How can we make our client [32:37] uh signals data signals much more richer in nature  such that the models are you know performing to [32:44] their best nature and grounded to vanguard. Uh  yeah that's the three waves that we have so far. [32:52] So at times like you know as we are alluding right  it's not an easy task like you know we are taking [32:59] more onto the plate if you are going the sovereign  path. So why would an enterprise you know build [33:05] and the answer is very simple it's in the title  itself you because you wanted to build proprietary [33:09] intelligence. If you have a solution that is  available to everyone then you know it means that [33:14] you know you are kind of providing the same kind  of services. Right? If you have a unique operating [33:19] model or like you know customer operating model or  a product u or a service which also means that you [33:26] know you are accumulating lot more information and  today that information might be coming from some [33:32] web chats, web interactions, phone call for that  matter um or you know other form of interaction [33:39] but as we move forward the AI interaction is  going to get into that plate very soon where [33:45] you know clients are directly interacting with an  AI and that is actually like you know a feedback [33:50] that you wanted to capture more because if your  AI is going to represent your business then that [33:55] feedback coming from the client is very important  to address right there. So you would have to build [34:00] certain solution that can actually capture them  and encode them so that over the decade period [34:05] of time you're not lagging behind in terms of you  know where does this knowledge sits now is there [34:10] something supporting that knowledge or not. So  that becomes very important. The other thing is [34:15] you would start looking at latency and the way we  have been developing solution is to have dedicated [34:22] um inference endpoints right for an application.  So if we have 10 applications that we are running [34:28] all of them are dedicated services so that we  can scale up to the need of the enterprise. [34:34] Um sometimes certain applications that we see is  like you know okay it's 6 to 9 and then we don't [34:40] see anything we can shut it down afterwards. We  have that process today that's great but certain [34:44] application are needed to be like know 24 into  7 and at that point in time you don't want it to [34:50] get hit by a rate limit and sometime the kind  of information that you're going to share it [34:57] might just start by saying that you know I want  you to look into this particular document which [35:01] has the following information can I get to know  what I should do and just imagine that's your [35:06] biggest tax document that you're putting in  there and lot more user doing the same thing [35:10] the token budget that's going just explode. So  there are like you know problems that is like [35:15] you know pointing to that direction that we need  to have that build versus buy balance so that you [35:20] know you can capture that business intelligence.  Everyone in this room would agree that you know [35:25] you can get that general intelligence out of  the market just paying and it's right out of [35:29] right in front of you. But in order to have that  competitive like you know advantage which is 10x [35:35] more defensible in future you also need to start  thinking about business intelligence in the same [35:40] way you think general intelligence when frontier  solution are coming to us and saying we're going [35:45] to each model is going to do the 10x of that we  need to have solution in our businesses which [35:49] we are going to say we use that plus our own  solution and we are also increasing business [35:54] intelligence by 10x so that we can support our  clients where they want us you know help them the [36:01] With that said, um our story line has been  start with the gap as we mentioned and [36:06] the rest will follow. And we are open to any  questions that you all might have for us now.