ElevenLabs Favors Cascaded AI Pipelines Over Direct Voice Models

ElevenLabs更青睐级联AI管线而非直接语音模型 直接语音到语音系统距离达到企业级可靠性仍有数月差距，这迫使生产部署采用具备自动模型故障转移机制的模块化管线，以保障亚秒级延迟和严格的品牌安全护栏。 Ben Budde详细阐述了一千多个企业级语音AI部署中的核心技术与战略要点，主张采用模块化级联管线（STT到LLM再到TTS）而非尚不成熟的端到端语音模型，以维护安全护栏并将延迟严格控制在亚秒级。本次分享探讨了声音个性化、以将互动式Darth Vader引入Fortnite为例的大规模压力测试，以及可防止延迟尖刺和幻觉的分区知识库架构。Budde展示了在客户联络中心和内部员工入职培训中的显著ROI，同时强调了方言本地化、金丝雀发布策略和专用编排层的重要性。 ElevenLabs 联合创始人源于对波兰单人旁白配音传统的挫败感而创立公司，致力于打造具有表现力的多语言 AI 音频。 (0:04) 企业级语音 Agent 普遍采用 STT-LLM-TTS 的级联架构而非端到端语音模型，以确保亚秒级延迟、品牌一致性与确定性的安全护栏。 (1:36) 企业在推进语音战略时可通过提示词生成、7,000 多种社区声音库或高保真专属克隆展开，高并发场景需经受严苛的负载测试考验。 (5:08) Telus 将交互式语音 Agent 用于员工入职培训使培训时间缩短超 25%，并颠覆了传统 IVR 电话树以彻底消除客服排队等待问题。 (7:16) 全球化语音部署需支持细分口音与本地化方言（如 17 种西班牙语方言），且生产环境需采用金丝雀发布与分层知识库架构降低延迟。 (9:23)

According to ElevenLabs' Ben Budde, deploying enterprise conversational voice AI requires modular cascaded architectures, disciplined knowledge routing, and staged engineering rollouts rather than relying on immature end-to-end voice models. Achieving sub-second latency and consistent brand guardrails depends on isolating pipeline stages, managing dialect localization, and carefully partitioning corporate documentation. Architectural Trade-offs and Latency Management (1:36) Budde explains that while end-to-end voice-to-voice models have generated significant industry attention, production-scale enterprise deployments remain virtually nonexistent because direct models cannot yet guarantee brand consistency or deterministic guardrails. He estimates that dependable enterprise readiness for direct voice-to-voice systems remains six to twelve months away. Consequently, production voice agents rely on modular, cascaded pipelines integrating Speech-to-Text (STT), Large Language Models (LLMs), and Text-to-Speech (TTS). The modular design allows engineers to inspect and benchmark latency across each hop, which Budde emphasizes is critical because system delays exceeding roughly one second disrupt conversational turn-taking and cause user disengagement. Because the LLM step introduces the highest latency variance, enterprise architectures implement 'LLM cascading'—automatically routing queries to secondary, lower-latency models during primary outages or performance spikes—alongside specialized orchestration for natural interruptions and audio streaming. Voice Strategy, Scale, and Dialect Localization (5:08) Voice strategy requires moving beyond generic presets to balance brand identity, user engagement, and extreme concurrency. Budde notes that organizations can generate synthetic voices via text prompts, select from a library of over 7,000 community-contributed voices that share revenue with voice actors, or produce bespoke digital clones. Enterprise cloning workflows range from 30-second reference clips to professional clones trained on over 30 minutes of studio audio, such as synthetic voices used by NVIDIA CEO Jensen Huang for Mandarin keynotes. Testing extreme concurrency is vital; ElevenLabs tested its infrastructure during an Epic Games and Lucasfilm deployment of an interactive Darth Vader in Fortnite, generating approximately 100 years of real-time audio within weeks. Localization represents another critical deployment hurdle. With over 80% of the global population speaking a language other than English as their primary language, systems must account for regional nuances rather than broad language classifications. For instance, ElevenLabs supports 17 distinct regional dialects in Spanish alone—such as Colombian, Puerto Rican, and Mexican accents—to match localized cadence and speech patterns across diverse customer bases. Knowledge Base Partitioning and Data Hygiene (10:33) A recurring operational failure in enterprise voice AI involves poor knowledge base construction and outdated documentation. Budde notes that ElevenLabs' internal support voice agent, 'Elle,' initially returned inaccurate answers simply because the underlying company documentation was out of date. Furthermore, loading an entire monolithic corporate knowledge repository into an agent introduces severe latency penalties that disrupt conversational flow. To resolve this, enterprise voice deployments employ a supervisor agent architecture that directs incoming voice or text queries to partitioned knowledge bases, such as billing, technical support, or product documentation. Partitioning the retrieval path cuts latency while driving response reliability toward predictable execution across multimodal environments, including web interfaces, mobile SDKs, and telephony systems. Safe Rollout Strategies and User Adoption (12:22) Budde warns against 'big bang' product launches, advocating standard software engineering safety practices such as canary releases, rigorous automated evaluation suites, and clear rollback capabilities. Organizations building custom pipelines frequently encounter severe complexity when managing turn-taking, tool calling, and dynamic latency in-house, prompting many enterprises to partner with experienced orchestration platforms rather than building entirely from scratch. In real-world operations, voice AI is accelerating corporate training and modernizing customer support. At telecommunications provider Telus, deploying interactive AI voice agents for employee onboarding reduced overall training time by over 25%. Telus observed an unexpected behavioral pattern: support representatives were far more willing to ask clarifying questions to an AI coach than to human managers due to an eliminated fear of judgment. In customer-facing contact centers, voice AI replaces rigid interactive voice response (IVR) phone trees, though Budde recommends establishing an identity verification layer and maintaining upfront transparency regarding the agent's AI identity to mitigate conversational friction. While conversational voice AI can deliver scalable onboarding and replace legacy phone trees, its enterprise viability remains bounded by infrastructure prep, knowledge base hygiene, and the technical immaturity of direct speech models. Until end-to-end voice-to-voice architectures achieve deterministic guardrails, enterprises must rely on modular pipelines, automated fallback routing, and incremental canary rollouts.

ElevenLabs北美市场负责人Ben Budde指出，企业级语音AI的落地核心在于采用分层的级联架构、严格的低延迟编排以及高质量的本地化策略，而非直接套用目前尚不成熟的端到端语音模型。 级联架构与毫秒级延迟控制 (1:36) 在企业级语音智能体的架构设计中，行业目前普遍面临端到端语音模型（voice-to-voice）与分层级联管道（STT-LLM-TTS）的选择。Budde表示，虽然直接语音到语音的模型备受关注，但在实际企业生产环境中几乎没有可用部署，预计距离稳定投产仍有6至12个月；企业看重确定性、品牌一致性与严格护栏，模块化架构能针对各环节排查并压低延迟。语音交互对延迟极其苛刻，系统整体延迟一旦超过约1秒，自然交流便会中断。鉴于大语言模型阶段的不确定性最高，架构必须引入LLM级联路由机制，当主模型出现故障或延迟骤增时自动故障转移至备用模型，配合轮流发言与打断编排以避免对话出现死寂。 声音定制策略与超大规模并发考验 (5:08) 企业语音策略直接决定品牌形象与用户黏性，目前已从预设固定声音转向多维度定制。企业可通过文本提示词生成合成声音、调用拥有7000多种声音且支持配音员分成收益的社区音色库，或是为品牌形象大使制作高保真克隆，克隆形式包括30秒音频的快速克隆与30分钟以上影棚级专业克隆（如NVIDIA首席执行官黄仁勋利用其技术将英文演讲克隆并翻译为普通话）。在规模化测试方面，Budde以Epic Games、Lucasfilm与James Earl Jones遗产管理方在《Fortnite》中落地达斯·维达交互克隆为例，指出该项目在数周内为数百万玩家生成了相当于约100年的实时对话音频，证明大规模语音上线必须经历高强度压力测试与底层设施准备。 企业培训与客服中心落地收益 (7:16) 企业引入语音AI增长最快的场景是内部培训与客户支持重构。在企业培训领域，一对一AI导师可以实现规模化员工赋能。例如电信服务商Telus在部署交互式语音智能体协助入职培训后，客服代表的入职培训周期缩短了25%以上；实际数据显示，员工面对AI导师时更愿意提出澄清性疑问，排除了面对人类主管时的评价焦虑，进而加快学习进程。在外部客服场景中，具备同理心且7×24小时在线的智能语音正逐步取代传统按键式交互式语音应答（IVR），消除排队转接与用户盲目按零呼叫人工的痛点，快速解决复杂业务咨询。 多语言细分方言与多模态知识库治理 (9:23) 全球超过80%的人口母语非英语，跨语言即时切换与深层方言本地化是全球化部署的核心。以西班牙语为例，系统细分了委内瑞拉、哥伦比亚、波多黎各、西班牙与墨西哥等17种区域方言，同时覆盖美国各地的口音差异，有效提升互动共鸣。在系统集成上，成功的方案多结合文本、电话网络、移动端SDK与网页界面等多模态入口。此外，知识库治理是决定回答准确率的关键，直接灌入庞大的企业文档库不仅会因过时信息导致幻觉，还会拖慢响应速度；实际架构需依靠主管代理（Supervisor Agent）对账单、技术支持、产品文档等分区知识库实施精准路由，在降低延迟的同时确保输出可靠性。 生产发布规范与自研采购权衡 (12:22) 将语音智能体推向生产环境必须遵循标准软件工程规范，杜绝一次性全面切换的上线方式，而应采用金丝雀发布、自动化评估套件与快速回滚机制。企业在技术选型时常陷入“自研还是外购”的误区：若仅采购基础STT或TTS API并试图自研全部链路，往往会在轮流发言检测、动态流式延迟、工具调用与系统集成等复杂工程上受阻；而完全封闭的整包平台又缺乏灵活性。最行之有效的模式是采用灵活API并与具备企业级扩展经验的编排服务商协作，在身份验证层、透明度声明（如主动告知AI身份）及多语言支持间取得工程平衡。 Budde强调，直接端到端语音模型在企业级生产环境中的成熟仍需时日，当下企业落地语音AI的成败并不取决于单一模型的噱头，而取决于级联架构中的延迟控制、精细化方言适配、严密的知识库路由以及分步金丝雀发布的工程执行力。

## Background 本视频由 Ai4 频道于 2025 年 8 月 27 日发布在 YouTube 平台上，记录了 ElevenLabs 营收副总裁（VP of Revenue）Ben Budde 的现场主题演讲。内容围绕 ElevenLabs 在数百家企业级客户中落地语音 AI 的实战经验展开，旨在为企业提供一套将对话式语音 Agent 从概念验证（PoC）平稳推向大规模生产环境的实用蓝图。 ## TL;DR 演讲系统梳理了企业级语音 AI 落地中的核心技术选型与工程实践。Budde 指出，端到端 Voice-to-Voice 架构距离稳定商用仍有 6 至 12 个月差距，当前生产环境仍以可控性更高的分步流水线（STT → LLM → TTS）及 LLM 级联降级策略为主，以守住 1 秒以内的端到端延迟红线。同时，演讲结合 TELUS 培训缩短 25% 周期、《堡垒之夜》达斯·维达实时配音等标杆案例，深入剖析了多语言与方言适配、分层知识库路由架构、金丝雀灰度发布策略以及双模声音克隆技术的业务价值。 ## Atmosphere & style 现场氛围务实、干练且极具说服力，演讲者以创业初衷与幽默的现场互动开场，随后迅速切入高密度的工程架构与一线落地避坑指南。分享没有浮夸的营销辞令，而是充斥着对系统抖动、打断判定、知识库脏数据等生产痛点的坦诚反思与严谨拆解。整场演讲在专业硬核的技术探讨与生动前沿的商业案例之间自如切换，传递出语音 AI 正从实验室极速跨入成熟企业级基础设施的扎实力量感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=4s) ElevenLabs 创立于对传统单调配音痛点的反思，目前已支撑超 1,000 个企业级生产案例，核心在于提供高度的业务可配置性与坚实的安全控制护栏。 - [03:00](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=180s) 端到端 Voice-to-Voice 架构距离稳定支撑企业级生产环境仍有 6 至 12 个月的演进期，当前分步流水线（STT → LLM → TTS）在各跳转节点的延迟测量与护栏配置上更具可靠性。 - [03:57](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=237s) 人机语音交互响应延迟需严控在 1 秒以内；由于 LLM 是方差最大的环节，系统必须引入 LLM 级联降级（Cascading）机制，规避坐席意外掉线。 - [05:08](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=308s) 企业需要建立专属的声音战略，ElevenLabs 提供纯提示词生成、官方声音库分成生态以及高保真声音克隆三层定制路径，助力品牌调性塑造与商业转化。 - [06:22](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=382s) 上线前压测必须模拟混乱无序的现实世界，例如在《堡垒之夜》实时生成等效 100 年时长的达斯·维达互动音频案例中，需针对分布式网络抖动与极端并发进行极限验证。 - [07:16](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=436s) 在企业培训场景中，AI 语音教练使 TELUS 新员工培训周期缩短超 25%，且员工向 AI 提问比面对人类主管展现出更高的心理安全感。 - [07:16](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=436s) 语音 Agent 正在彻底重塑传统 IVR 电话树“狂按 0 转人工”的糟糕体验，向具备深层上下文理解与 7×24 小时排障能力的自主智能体升级。 - [09:23](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=563s) 支持即时多语言切换可直接触达全球 80% 以上非英语用户，同时细分方言（如单是西班牙语即提供 17 种方言）能显著提升本地用户的接纳度与互动意愿。 - [10:33](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=633s) 语音并非孤立存在，结合 Telephony、Web 与移动端 SDK 形成多模态协同，能够借助音频天然的情感共鸣大幅增强用户黏性。 - [11:22](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=682s) 脏乱过时的知识库会直接损害系统表现，建议采用主管智能体（Supervisor Agent）进行前置意图路由分流，避免全量数据检索带来的高延迟。 - [12:23](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=743s) 企业级部署应坚持金丝雀发布、自动化 Evals 评测与一键回滚机制；全双工打断与低延迟编排壁垒极高，借助成熟编排底座可大幅降低自研试错成本。 - [14:01](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=841s) 在敏感业务触发前需接入底层身份验证层；在大部分客服场景中，开诚布公亮明 AI 身份并承诺人工转接保底，能有效缓解用户的对抗心理。 - [16:17](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=977s) Conversational AI 平台全面整合声音定制、主流大模型接入、多语言支持与运营通话回放，能将语音系统的部署周期从数月压缩至数天之内。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=4s) 创立初衷与企业级语音平台的演进 ElevenLabs 北美市场拓展（GTM）负责人 Ben Budde 在开场时向现场观众发问：“谁正在构建或考虑构建语音 Agent？谁已经搭建完成？”在看到台下举手致意后，他感慨当下正是投身 AI 语音最令人振奋的节点。作为一家专注前沿 AI 音频研究的技术公司，ElevenLabs 的诞生源于其两位联合创始人 Mati Staniszewski 与 Piotr Dąbkowski（两人分别出身于 Palantir 与 Google）童年时期的一段切身经历。他们成长于波兰，当时发现波兰译制的所有奥斯卡获奖电影，不管是男性、女性、小孩还是特殊角色，居然全是由同一名中年男配音演员以单调的语调通篇配音（"done by a single male voice actor"）。这种极度违和且糟糕的体验让他们萌生了一个坚定信念：“一定存在更好的解决办法（there has to be a better way）。” 如今，ElevenLabs 已经从早期的顶尖文本转语音（Text-to-Speech）技术拓展为一个全栈的企业级可信语音 Agent 平台。公司不仅获得了顶级风投机构的鼎力支持，更在各行各业深度落地，服务包括 Cisco、Adobe 和 Twilio 在内的知名企业。Budde 强调，将前沿的音频研究转化为商业生产力，关键在于解决企业的两项核心诉求：高度的业务可配置性（Configurability）以及坚实的安全护栏与控制机制（Trust, Control, and Guardrails）。截至目前，团队已在实际生产环境中完成了超过 1,000 次企业级交付，积累了大量跨越实验室与真实业务鸿沟的实践认知。 - **创始人背景**：Mati Staniszewski 与 Piotr Dąbkowski 为发小，此前分别在 Palantir 和 Google 从事企业级 AI 系统的落地应用。 - **波兰译制片痛点**：全片所有角色由单一男声覆盖（波兰传统的“Lektor”旁白配音机制），成为创办 ElevenLabs 的直接灵感来源。 - **业务体量**：已支撑超过 1,000 个企业级生产环境部署案例，服务思科、Adobe、Twilio 等战略客户。 ```mermaid flowchart TD accTitle: ElevenLabs 语音交互分步架构与级联回退流 accDescr: 展示用户语音输入经由 STT、LLM 级联决策及 TTS 生成，最终将音频返回用户的端到端流水线。 subgraph UserInteraction["用户交互层"] U["用户语音输入"] O["最终音频输出"] end subgraph SpeechPipeline["可观测分步流水线"] STT["1. 语音转文字 (STT)"] TTS["3. 文本转语音 (TTS)"] subgraph LLMCascade["2. 编排与 LLM 级联"] direction TB PrimaryLLM["首选主模型 (低延迟/高精度)"] FallbackLLM["回退降级模型 (高可用保障)"] PrimaryLLM -->|"超时 / 故障"| FallbackLLM end STT --> PrimaryLLM LLMCascade --> TTS end U --> STT TTS --> O ``` ### [03:00](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=180s) 架构与性能选型：分步式流水线与 Voice-to-Voice 的现实差距 在探讨系统底层架构时，Budde 直奔当下行业热点：全端到端“语音到语音”（Voice-to-Voice）架构是否已经取代了传统的“语音转文字（STT）→ 大语言模型（LLM）→ 文字转语音（TTS）”分步流水线？Budde 直言，尽管 Voice-to-Voice 在理论上展现出极致的实时响应潜能，但在企业级市场中，他“至今尚未见到哪怕一个真正达到规模化生产级别的 Voice-to-Voice 部署案例（I've still yet to see a single scaled production deployment of it）”。他甚至调侃道，如果现场有人真正攻克了全链路端到端在企业级落地的技术挑战，“请在下课后务必来找我聊聊”。 企业级客户在生产环境中永远将一致性与可靠性（"consistency and reliability of experience"）置于首位，任何系统失控或内容跑偏（off the rails）都是不可接受的。相比之下，分步式串联架构（STT → LLM → TTS）具备端到端模型目前无法企及的工程优势：开发者能够在每一个跳转跳点（hop）清晰测量延迟指标、排查性能瓶颈，并嵌入确定性的业务规则与合规安全护栏。尽管包括 ElevenLabs 在内的顶尖科研团队都在全力攻坚 Voice-to-Voice，但 Budde 研判，端到端架构距离实现真正稳定、可靠的企业生产级应用至少还需要 6 到 12 个月的迭代演进。 - **架构成熟度判断**：端到端 Voice-to-Voice 距离稳定支撑复杂的企业生产级业务，预估还有 6 至 12 个月的工程成熟期。 - **分步流水线优势**：在 STT、LLM 和 TTS 各阶段均可独立监控延迟、配置业务逻辑，并施加严格的内容安全护栏。 ### [03:57](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=237s) 延迟控制与 LLM 级联降级策略 在人机语音交互中，延迟是决定用户体验成败的核心生命线。Budde 指出，一旦语音响应延迟显著超过 1 秒，交互就会产生极强的停顿感与割裂感，导致用户失去交谈兴趣（"the user just doesn't engage"）。而在对数百个落地案例进行全链路指标分析后，他们发现整个技术栈中方差最大、表现最不稳定的变量始终是 LLM（"the LLM has the most variability"）。 为了在保证模型智能深度的同时兼顾严苛的可用性，ElevenLabs 普遍推行了“LLM 级联降级策略”（LLM cascading）。在标准状态下，系统优先调用最契合业务需求的主选模型（Primary LLM）；然而，一旦上游主力模型发生偶发性宕机或产生极端网络延迟，编排系统必须立即无缝平滑回退至备用轻量级模型。在严肃的企业商务场景下，决不允许出现通话中途“坐席突然失联或掉线（agent drop）”的毁灭性体验。此外，多方轮次轮替（turn-taking）与打断机制（interruption orchestration）的处理复杂度极高，如何让系统在用户插话时自然收声并快速重新组织思绪，是编排层对端到端延迟控制的一项重大工程考验。 - **响应延迟红线**：端到端延迟需控制在 1 秒以内，超出该阈值将导致用户对话流失率急剧上升。 - **LLM 级联架构**：配置多级容灾与降级机制，规避第三方模型 API 抖动导致的交互中断。 - **对话编排难点**：打断处理（Interruption）与轮次切换判定（Turn-taking）是整个流水线编排中对延迟和体验影响最深的技术细节。 ### [05:08](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=308s) 声音战略与个性化品牌声线塑造 声音品质是 ElevenLabs 享誉业界的立足之本，但在竞品林立的市场环境中，不同厂商对声音资产的理解存在巨大鸿沟。部分厂商的做法十分粗暴，仅仅向企业提供诸如“从预设的 5 种声音中任选其一”的固化方案。Budde 强调，对于现代企业而言，声音是品牌形象与调性的关键听觉延伸（"represents their brand and tone"），企业必须具备专属的“声音战略”（Voice Strategy），而不仅仅将其视为一个播放文字的通道。 针对企业的差异化诉求，ElevenLabs 提供了三层梯度的声音定制能力： 1. **纯提示词生成（Create out of thin air）**：输入形容词与声学特征要求，由系统无中生有地生成具备完全商用版权的独创声线； 2. **丰富的官方声音库（Voice Library）**：平台汇集了超过 7,000 种各具特色的专业声线。ElevenLabs 与全球配音演员建立了分成合作机制，配音演员通过将自身声线数字化入库，每月可躺平获取数千美元的分成收益（"getting like thousands a month just by cloning their voice"）； 3. **高保真声音克隆（Voice Cloning）**：通过少量企业高管或专属代言人的干声音频，高精度复刻专属音色。数据证明，高质感的声音在用户停留时长与商业转化（ROI metrics）上均呈现出强烈的正向相关性。 ### [06:22](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=382s) 压力测试与复杂真实环境模拟 在系统正式上线前，压力测试必须尽可能“模拟混乱无序的现实世界（load testing it must mimic the messy reality）”。为了说明这一点，Budde 分享了他们与 Epic Games 联合打造的现象级工程案例：在获得迪士尼以及詹姆斯·伯尔·琼斯（James Earl Jones）遗产委员会的正式授权后，ElevenLabs 成功复刻了经典反派达斯·维达（Darth Vader）的原版音色，并将其集成到全球热门游戏《堡垒之夜》（Fortnite）中，实时面向数百万并发玩家开放语音互动。 Budde 透露，这很可能是迄今为止全球规模最大的一次 AI 音频落地项目——在短短数周之内，系统生成了相当于 100 年时长的海量音频（"created like a hundred years of audio in a matter of weeks"）。他笑着回忆，这也是自己的孩子们第一次觉得“老爸做的工作真的很酷”，因为他们可以在家里与达斯·维达直接展开实时语音对话。然而，这种量级的系统绝不可能“通过轻按一个开关就自动平稳运转”。团队在上线前针对多地域分布式节点、极端网络抖动、多语种口音输入以及异常高并发场景进行了极致的压力注入与边界情况演练，最终确保了该项目在数百万玩家涌入时的绝对稳定。 - **《堡垒之夜》达斯·维达案例**：经迪士尼与 James Earl Jones 授权，为数百万玩家实时生成达斯·维达交互语音。 - **计算吞吐量**：在短短数周内累计合成了等效长达 100 年时间总和的高精度实时音频。 - **压测关键准则**：必须超越理想网络假设，针对不同地理区域、极端并发峰值与各种网络劣化场景做拟真负载验证。 ### [07:16](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=436s) 垂直场景拓展：从员工培训到客服重构 随着语音技术的成熟，企业端客户的应用场景正在迅速突破传统边界，其中“企业学习与发展培训（L&D）”展现出了惊人的爆发力。无论是面向终端用户的产品引导，还是针对新员工的入职培训与老员工技能再造，一个极具耐心的“个性化 AI 教练”（Personalized AI Coach）正在深刻颠覆知识传递的方式。 Budde 以电信巨头 TELUS 为例：该企业拥有数千名人工坐席团队，面临极其沉重的入职与知识库更新负担。在部署了基于 ElevenLabs 的个性化培训 Agent 后，其实训入职周期大幅缩减了超过 25%（"reduced by over 25%"）。更耐人寻味的是团队在复盘时发现的“反直觉”现象：相比于主管直接坐在身旁进行监听与指导带来的心理压力，员工在与 AI Agent 对练时反而表现得更加从容自信、更敢于敞开提出疑难问题（"more comfortable asking questions of an agent than if their manager was sitting right next to them"）。 与此同时，传统客服领域的客户情感体验也迎来拐点。回顾过去的传统 IVR（交互式语音应答）系统，用户常常陷入繁杂冗长的电话树中动弹不得，被逼无奈之下疯狂连按“0”或者对着听筒歇斯底里大喊“转人工（representative）”，最终等待漫长音乐后接通的坐席可能依然无法解决问题，不得不面临又一次转接与重听等待。Budde 指出，全行业都已经意识到传统的支持体系亟待重塑；当一个真正能够理解上下文、7×24 小时随叫随到且具备深度解决问题能力的智能体出现时，用户对 AI 客服的态度正在迅速从早期的怀疑抵触转向高度认可与依赖。 - **TELUS 培训效能**：个性化语音教练助力企业新员工入职培训周期缩短 25% 以上。 - **心理安全感（Psychological Safety）**：真实反馈显示，基层员工向 AI 导师请教无知问题时的心理负担远低于直接面对人类主管。 - **客服范式升级**：彻底颠覆传统 IVR 电话树“狂按 0 转人工”的痛苦体验，向具备自主执行与排障能力的 7×24 小时智能交互体迁移。 ### [09:23](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=563s) 多语言与方言适配：打破单一语言壁垒 在语音智能体的实际落地过程中，多语言支持是释放全球市场价值的核心支柱。全球有超过 80% 的人口并不以英语为母语，智能体如果能够在对话中自如地进行语言切换，将能极大地拓展服务覆盖面。多语言支持不仅是为了让不同国家的用户“能用”，更是为了尊重并契合终端用户的语言偏好。即便在同一种语言内部，地域口音与方言的精细化匹配也对用户体验有着决定性影响。 以西班牙语为例，ElevenLabs 在平台中提供了多达 17 种不同的地域方言。委内瑞拉、哥伦比亚、波多黎各、西班牙与墨西哥的西语发音与习惯用语各有不同；这与美国本土西海岸、东北部和南部各具特色的口音差异如出一辙。用户如果听到贴合本地习惯的发音，对智能体的接纳度与互动意愿会大幅提升。 - **多语言广度**：全球 80% 以上人群的母语非英语，支持即时语言切换能直接解锁全球各区域市场。 - **方言精细度**：单是西班牙语即提供 17 种不同方言，精准贴合本地文化与用户偏好。 --- ### [10:33](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=633s) 多模态协同与语音交互的吸引力 语音智能体并非孤立存在，而是多模态生态的关键组成部分。无论是通过电话集成系统（Telephony）、移动端 SDK 还是网页端嵌入，文本与语音的深度融合都为交互带来了广阔空间。虽然部分后端智能体仅需在幕后静默处理事务，但面向用户的交互前端正在迎来语音界面的爆发式增长。 过去几年中有声书（Audiobooks）和播客（Podcasts）的爆发式流行证明，音频的价值不仅在于免提操作的便捷性，更在于它天然具备的情感共鸣与高互动黏性（Engagement Factor）。将这种沉浸式语音体验与多模态输入（文字、屏幕交互等）相结合，能够显著拉近系统与用户之间的距离。 --- ### [11:22](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=682s) 知识库管理与层级化路由架构 知识库（Knowledge Base）是决定智能体回答质量与系统性能的核心变量。ElevenLabs 在官方文档上线名为“Elle”的内部支持智能体时，最初发现系统给出了一些错误答案，排查后发现根源在于原始产品文档本身未能及时更新。这一教训表明：脏乱或过时的知识库，根本无法支撑企业级应用所要求的确定性与极高可靠性。 另一个常见的实施误区是将海量知识库无差别地整体灌入系统。知识库体积越庞大，检索与推理带来的端到端延迟（Latency）就越高。为了解决这一痛点，推荐采用“主管智能体（Supervisor Agent）”架构，由主管智能体负责前置意图识别，并将任务精准分发给专注特定业务领域的垂直知识库与子任务执行单元，这样既大幅压降了系统延迟，又有效提升了回答的准确率。 ```mermaid flowchart TD accTitle: 基于主管智能体的知识库路由架构 subgraph Ingress["用户交互入口"] A["用户语音 / 文本输入"] --> B["主管智能体 (Supervisor Agent)"] end subgraph Routing["意图分类与分发"] B -->|"订单业务"| C["订单知识库与业务任务"] B -->|"技术支持"| D["技术文档知识库"] B -->|"退款/账单"| E["账单与退款知识库"] end subgraph Execution["响应生成"] C --> F["统一合成与精准应答输出"] D --> F E --> F end ``` - **数据卫生**：知识库质量决定智能体上限，必须保证源数据与产品文档的最新性。 - **延迟控制**：避免“大杂烩式”全量数据倾倒，采用分层分流路由机制降低检索开销。 --- ### [12:23](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=743s) 工程化发布策略与“自研 vs 采购”权衡 企业级语音智能体的上线应严格遵循现代软件工程的最佳实践。切忌采取一蹴而就的“大爆炸（Big Bang）”式全量上线方式，而应当推行渐进式金丝雀发布（Canary Deployment），配套完善的评测套件（Evals）与自动化测试，并确保具备随时无缝回滚（Rollback）的能力，以防线上意外故障引发严重客诉。 在技术架构选型上，关于“自研还是采购（Build vs. Buy）”的讨论十分普遍。许多企业最初倾向于仅调用基础的 TTS（语音合成）或 STT（语音识别）API 尝试自行组装，但随后会迅速陷入复杂的工程泥潭：工具调用（Tool Calling）、知识库协同、双工轮替机制（Turn-Taking）、端到端超低延迟优化、音质表现力以及跨场景稳定性等。最稳妥的路径通常是寻求中间平衡点：在调用灵活 API 的同时，依托成熟的端到端编排平台与具备真实交付经验的合作伙伴，规避底层管线集成的巨大隐性成本。 - **灰度发布准则**：坚持金丝雀部署、完善的 Evals 评测和一键回滚机制，拒绝盲目全量交付。 - **集成复杂度**：全双工轮替、低延迟编排与工具调用的工程壁垒极高，借助成熟编排底座可大幅降低实施风险。 --- ### [14:01](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=841s) 现场互动：身份校验、AI 免责声明与声音克隆 在演讲后的现场问答环节中，演讲者首先邀请参会者参加当晚在 Nelly's Southern Kitchen 举行的 Happy Hour 交流活动，随后回答了现场观众提出的三个高频实操问题。针对现场一位名为 Kisho 的观众提出的“身份验证与多角色管理”问题，演讲者指出系统需要接入底层身份校验层（Identity Layer），通过数据库验证“来电者是否为 Kisho 本人”，随后才能进入业务办理与智能体角色转接流程。 针对“如何消除用户对与 AI 对话的排斥感”的问题，演讲者强调在大多数客服场景中，开诚布公（Upfront）的沟通方式最为有效——智能体直接说明“我是 AI 助手，今天来帮您处理某项事务，必要时会为您无缝转接人工客服”；不过在特定销售场景下，也有企业选择不作前置声明，利用高拟真的人声情感直接建立联系。针对声音克隆的流程，演讲者介绍了两种方案：一种是用 30 秒语音快速生成的即时克隆；另一种是使用 30 分钟以上高质量音频训练的专业级克隆（Professional Voice Clone），其逼真度达到肉眼肉耳难以分辨的水准。例如英伟达 CEO 黄仁勋（Jensen Huang）便借助 ElevenLabs 克隆了自己的声音，并将其无缝转译为普通话（Mandarin）来发表大会演讲。 - **身份认证**：在关键业务触发前，必须结合后端系统完成用户身份确认。 - **心理摩擦消除**：主动亮明 AI 身份并承诺保底人工转接，能显著缓解用户的对抗情绪。 - **双模克隆技术**： - **即时克隆**：上传 30 秒日常语音（如一段语音留言），快速生成可用人声。 - **专业克隆**：基于 30 分钟以上高品质音频，实现连黄仁勋跨语言演讲都在使用的母带级拟真质感。 --- ### [16:17](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=977s) ElevenLabs 对话式 AI 平台特性与落地展望 现场随后播放了 ElevenLabs Conversational AI 的官方演示视频。该平台旨在将以往复杂度极高、耗费大量工程资源的对话式语音智能体构建流程全面平民化，提供涵盖构建、测试到部署的一站式闭环支持。用户可以挑选官方声音库或自主创建声音，灵活接入业界领先的主流大语言模型（LLM），挂载私有知识库并设定任务目标与人设，同时支持对接企业自建服务器实现完全自主可控。 在运营分析端，平台内置对话转录分析与通话回放系统，帮助团队持续评测对话表现并定义关键业务成功指标。在交付与集成端，智能体支持多达 32 种语言交互，仅需简单的复制粘贴即可嵌入 Web 网页，亦可通过功能完备的 SDK 快速植入移动 App、企服后台乃至电子游戏场景中。依托企业级安全防护架构，整套方案能将传统长达数月的语音系统研发部署周期缩短至数天之内。 - **全流程覆盖**：整合声音定制、主流大模型集成、知识库上传与自定义服务器对接。 - **运营评估功能**：提供全量转录分析、录音回放以及自定义业务成功评测基准。 - **多语言与极速上线**：全面支持 32 种语言；提供 Web 网页无脑嵌码及多端 SDK，实现“数天内从开发到上线生产（Get to production in days, not months）”。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=4s) there has to be a better way（一定存在更好的解决办法） — Ben Budde > [03:00](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=180s) I've still yet to see a single scaled production deployment of it（至今尚未见到哪怕一个真正达到规模化生产级别的 Voice-to-Voice 部署案例） — Ben Budde > [03:00](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=180s) consistency and reliability of experience（体验的一致性与可靠性） — Ben Budde > [03:57](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=237s) the user just doesn't engage（用户就会失去交谈的兴趣） — Ben Budde > [03:57](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=237s) the LLM has the most variability（LLM 是整个技术栈中方差最大、表现最不稳定的变量） — Ben Budde > [05:08](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=308s) represents their brand and tone（代表着品牌的形象与调性） — Ben Budde > [06:22](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=382s) load testing it must mimic the messy reality（压力测试必须尽可能模拟混乱无序的现实世界） — Ben Budde > [06:22](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=382s) created like a hundred years of audio in a matter of weeks（在短短数周之内，系统生成了相当于 100 年时长的海量音频） — Ben Budde > [07:16](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=436s) more comfortable asking questions of an agent than if their manager was sitting right next to them（员工在与 AI Agent 对练时反而比面对身旁的主管更从容、更敢于提问） — Ben Budde > [16:17](https://www.youtube.com/watch?v=M1GYXBNC8Jo&t=977s) Get to production in days, not months（数天内从开发到上线生产，而非数月） — Ben Budde ## Mentioned Ben Budde, ElevenLabs, Mati Staniszewski, Piotr Dąbkowski, Palantir, Google, Cisco, Adobe, Twilio, Epic Games, 迪士尼 (Disney), James Earl Jones, 达斯·维达 (Darth Vader), 《堡垒之夜》(Fortnite), TELUS, Elle, 黄仁勋 (Jensen Huang), Nelly's Southern Kitchen, ElevenLabs Conversational AI ## Video description Ben Budde, VP of Revenue, ElevenLabs, Inc. Abstracts What does it take to successfully deploy a conversational voice agent at scale? In this session, we’ll share 11 lessons from working with hundreds of enterprises to launch Voice AI in high-volume customer support environments. We’ll cover how to maintain brand alignment, deliver consistent service across every interaction, and ensure your voice agent handles complex queries—24/7. Learn what separates pilots from production-ready systems, and how to avoid the most common implementation pitfalls. Whether you’re planning your first deployment or optimizing an existing one, this talk offers a clear, practical blueprint. [Source](https://www.youtube.com/watch?v=M1GYXBNC8Jo)

[0:04] Ben Budde: there has to be a better way（一定存在更好的解决办法） [3:00] Ben Budde: I've still yet to see a single scaled production deployment of it（至今尚未见到哪怕一个真正达到规模化生产级别的 Voice-to-Voice 部署案例） [3:00] Ben Budde: consistency and reliability of experience（体验的一致性与可靠性） [3:57] Ben Budde: the user just doesn't engage（用户就会失去交谈的兴趣） [3:57] Ben Budde: the LLM has the most variability（LLM 是整个技术栈中方差最大、表现最不稳定的变量） [5:08] Ben Budde: represents their brand and tone（代表着品牌的形象与调性） [6:22] Ben Budde: load testing it must mimic the messy reality（压力测试必须尽可能模拟混乱无序的现实世界） [6:22] Ben Budde: created like a hundred years of audio in a matter of weeks（在短短数周之内，系统生成了相当于 100 年时长的海量音频） [7:16] Ben Budde: more comfortable asking questions of an agent than if their manager was sitting right next to them（员工在与 AI Agent 对练时反而比面对身旁的主管更从容、更敢于提问） [16:17] Ben Budde: Get to production in days, not months（数天内从开发到上线生产，而非数月）

[0:04] Ben Budde: Growing up as best friends in Poland, they were perpetually frustrated by a bizarre cultural norm: virtually every Oscar-winning Hollywood film dubbed into Polish was narrated by a single male voice actor. [1:36] Ben Budde: While direct voice-to-voice models generate significant buzz, production-scale enterprise deployments remain virtually nonexistent. Enterprises demand strict brand consistency, predictable guardrails, and deterministic performance [1:36] Ben Budde: Latency in voice conversations is unforgiving; once total system delay exceeds roughly one second, natural conversational flow breaks down and users disengage. [5:08] Ben Budde: The deployment generated roughly 100 years of interactive audio within a matter of weeks, proving that large-scale conversational voice requires deep infrastructure prep rather than simply 'flipping a switch.' [7:16] Ben Budde: A surprising behavioral insight emerged during the rollout: employees were significantly more willing to ask clarifying questions to an AI agent than to a human manager sitting next to them, eliminating the fear of judgment and dramatically accelerating learning curves. [9:23] Ben Budde: Over 80% of the world's population does not speak English as their primary language, making instant voice switching between dozens of languages essential for modern enterprises. [10:33] Ben Budde: Deploying a specialized supervisor agent that routes queries to partitioned knowledge bases and specific tasks significantly cuts latency and drives response reliability toward 100%. [12:22] Ben Budde: Rolling out conversational voice agents requires adhering to standard enterprise software engineering practices rather than attempting a risky 'big bang' release. [14:01] Ben Budde: Hey, I'm an AI agent, but I'm here to help you with X, Y, or Z and pass you to a human as needed.

[0:04] Thank you so much and welcome everyone. [0:05] Who's building voice agents today or [0:07] thinking about it? Who has already built [0:09] a voice agent? Okay. All right. We we [0:12] got some progress here. Such an exciting [0:15] time to be an AI and with AI voice. I'm [0:17] Ben Buddy. I have the pleasure of [0:18] leading the go to market team in North [0:19] America for 11 Labs. We're all things AI [0:22] audio. We're an AI audio research [0:24] company. and I'm really excited to talk [0:26] to you today about uh some of the [0:28] deployment lessons we've had in the [0:30] space. Uh so let's get started. Uh okay, [0:35] so uh first and foremost, this is this [0:38] is where we started. Uh this is Maddie [0:41] and Pat are co-founders. They were [0:44] working at Palunteer and at Google uh AI [0:47] experts and bringing AI to the [0:49] enterprise, but they grew up in Poland [0:51] and as best friends uh they they had [0:55] they noticed that every Oscar-winning [0:57] film that was dubbed into Polish was [1:00] done by a single male voice actor. And [1:04] that is still the case today for most [1:06] movies in Poland. So think about men, [1:08] women, children, characters, everyone [1:10] dubbed by a single male voice actor. [1:13] Just a horrible experience. And they're [1:15] like, there has to be a better way. So [1:17] that is where ultimately they came up [1:20] with 11 Labs. This is our whole agenda [1:23] here. We'll do the finish off the intro. [1:25] I'm going to talk about these 11 lessons [1:27] from our enterprise deployments. And [1:29] then we'll have room for some Q&A and [1:31] hopefully a little teaser video uh at [1:33] the end. With that, that was the intro. [1:36] Okay, here we go. Now this is our [1:38] trusted voice agent platform. You see [1:40] some of the customers on the right that [1:42] we have the the great pleasure of [1:44] serving. Uh Cisco is bringing us into [1:46] their agents. We work with Adobe, [1:48] Twilio, etc. across a whole range of use [1:50] cases. You think about audio, it's like [1:53] would impact every single industry in a [1:55] lot of cool ways. So excited to to talk [1:57] about that as well as u you know have [1:59] the backing of all those investors in [2:01] the bottom left. amazing success in [2:03] bringing ultimately our text to speech [2:05] and our voices into an entire agent [2:08] platform. So we we'll talk about the [2:10] lessons learned here, but ultimately for [2:13] us it all starts with the research and [2:16] this AI audio research is it's amazing. [2:19] It's world class. But then as you all [2:21] are thinking about building voice agents [2:22] whether it's with 11 Labs or any of the [2:24] other awesome you know competitors that [2:26] we have in the space then it gets to [2:27] real configurability. Can you configure [2:29] it to make it right for your business [2:31] and your users? And then of course with [2:33] the right trust and control and guard [2:35] rails and this just applies across so [2:38] many different use cases. Uh it's it's [2:40] quite fun and exciting to see what you [2:42] all are building and and to play a small [2:44] part in the process. But this is our [2:45] whole platform. We've done over a [2:47] thousand enterprise deployments now. So [2:49] have learned a thing or two. I worked [2:50] with my technical team and of course [2:52] with some of our top customers and [2:53] figured out okay what can I bring to [2:55] this audience to make it most helpful [2:57] and useful. So we're going to talk first [2:58] about architecture and performance [3:00] issues. Who's heard of voicetovoice [3:03] instead of like text to speech LLM [3:04] speech to text like the full voice to [3:06] voice? This is something that is really [3:08] exciting in the market. I've still yet [3:10] to see a single scaled uh production [3:13] deployment of it. So if you've somehow [3:16] cracked that, please come see me after [3:18] class. It'd be great to to chat. Uh, but [3:21] what I find in every single enterprise [3:23] conversation that I have is that [3:24] consistency and reliability of [3:26] experience really matter. You can't have [3:29] something go off the rails. And also [3:31] with the the step-by-step approach of [3:34] speechto text, LLM, text to speech, [3:37] there's a wonderful ability to set the [3:39] right guard rails to measure performance [3:42] to measure latency in each one of those [3:43] hops in a way you just can't with [3:45] voicetooice yet. Of course, our research [3:47] team and many others are thinking [3:48] through voicetooice, but it seems like [3:50] it's probably 6 to 12 months out from [3:52] like real production level use cases. [3:55] So, with that, [3:57] latency is of course incredibly [3:59] important. You can't really have an [4:01] experience that's like much longer than [4:03] a second where the user just doesn't [4:05] engage, you know. And what we've found [4:07] through many of these deployments is the [4:09] LLM has the most variability. So [4:12] obviously LLM choice plays a big part of [4:14] this, but also this thing that that we [4:16] do and and I'm sure some others do as [4:18] well is LLM cascading. So think about [4:20] like you have your chosen LLM. This is [4:22] your your ideal experience. But if for [4:24] some reason that LLM is down or there's [4:27] just like extreme latency, then you have [4:30] some fallbacks in place because who [4:32] wants to be mid call and just have like [4:35] the agent drop? that's just not uh [4:37] something that you can do in in [4:39] enterprise deployments for your [4:41] customers and users. So we find this [4:43] cascading LLM can be really important. [4:45] model choice of course can be really [4:46] important and measuring that there's [4:48] like a new model every week it seems [4:49] like and so having testing criteria to [4:51] understand that but then also on the [4:53] orchestration side that turn taking [4:56] interruption orchestration those are [4:59] really hard problems to solve and and so [5:01] I think there's a real impact on latency [5:03] as well for that overall orchestration [5:06] all right we're moving on voice quality [5:08] matters a lot I would say like this is [5:10] an area that that we're definitely known [5:12] for but there is a whole wide range of [5:15] how to approach voice across the [5:17] competitive landscape. Some are saying [5:19] like here, use one of our five voices [5:21] and then and then there's all sorts of [5:22] different options. You know, for us, we [5:24] look at it as every enterprise probably [5:27] wants their own voice like that that [5:29] represents their brand and tone and [5:31] there's so much that goes into that. So, [5:33] you know, for us, for example, you can [5:35] create a voice out of thin air that you [5:37] can use for commercial purposes, add in [5:39] all the adjectives, etc. that you want [5:40] for that voice. Or you can use one of [5:43] our 7,000 voices in our voice library. [5:45] We partner with voice actors and give [5:46] them a cut. I mean, this is like maybe a [5:49] side hustle for some of you. We have [5:50] voice actors that are getting like [5:52] thousands a month just by cloning their [5:54] voice and putting it on our platform. [5:55] So, that's that's another option. And [5:57] then, of course, there's voice cloning, [5:59] like really really high quality clones [6:00] that are possible as well for that truly [6:02] unique experience. And you see some some [6:05] customers listed here. But regardless of [6:07] of you know your you know chosen stack [6:10] voice quality I' I'd really uh encourage [6:12] you all to think about voice strategy [6:14] because we we definitely see it in the [6:16] ROI metrics to see like are customers [6:19] engaged is the brand represented etc. [6:22] All right load testing it must mimic the [6:24] messy reality. Uh, we were able to do a [6:28] a project with Epic Games, of course, [6:30] with the permission of James Jones [6:32] estate in Disney to clone Darth Vader's [6:34] voice and bring it into Fortnite for [6:36] like millions of people at a time. It [6:38] was, I think, the largest AI audio [6:40] project ever. We created like a hundred [6:42] years of audio in in a matter of weeks. [6:44] It was uh so fun to be a part of. And I [6:46] think it was the first time where my [6:48] kids were like, "Okay, Dad, what you do [6:50] is actually cool." Cuz we could have a [6:52] conversation with Darth Vader. But that [6:54] just doesn't happen with a flip of a [6:56] switch, right? You need like the real uh [6:59] testing to mimic, okay, is this going to [7:01] happen this way? What if there's [7:03] different different user bases in [7:05] different regions? And and really [7:06] thinking through what is this deployment [7:08] going to look like? Testing lots of [7:09] different scenarios is absolutely [7:12] helpful that we've seen in a lot of [7:14] different um you know rollouts. Now [7:16] let's talk about customer adoption [7:18] because there's so many different [7:19] verticals and use cases here. I would [7:21] say, you know, education and and as you [7:24] pull that into like learning and [7:25] development and training. This could be [7:27] for your end users. It could be for your [7:28] new employees, upskilling existing [7:31] employees. There's nothing like having a [7:34] personalized AI coach that can help you, [7:37] you know, learn the given topic. And I [7:40] think it's it's been quite amazing to [7:41] see. I'll just maybe give one example [7:43] for for Telus for there are like [7:46] thousands of human agents that they have [7:48] onboarding, upskilling, etc. you know, [7:50] their onboarding has reduced by over 25% [7:53] just because they have these these [7:56] personalized agents at their beck and [7:57] call. And the most surprising thing in [7:59] that roll out that we found was it it [8:01] wasn't just the, you know, the overall u [8:04] acceleration of learning. People were [8:07] generally more comfortable asking [8:09] questions of an agent than if their [8:11] manager was sitting right next to them. [8:12] You know, who who would have thought? So [8:14] the idea of just being able to take [8:16] wherever you're at in a given topic, [8:18] beginner, intermediate, advanced, and [8:19] accelerate that learning. I think this [8:21] is the future of learning and and [8:23] enablement and uh really excited to see [8:25] it be really like one of our largest use [8:27] cases today outside of the the standard [8:30] ones you would imagine. Also on customer [8:33] support, sentiment has really shifted. [8:35] Who likes being on hold for hours on end [8:38] of like, oh, oh, we got one. Okay. I've [8:41] I've never seen a handoff. You know, [8:43] maybe you appreciate the downtime, [8:45] little zen. Uh I I'll tell you, there's [8:48] nothing more frustrating for me [8:49] personally than like hitting zero zero [8:52] shouting representative. You know, it's [8:54] just like we've all been lost in the [8:55] phone tree and then getting to a human [8:57] agent that may not be able to solve your [9:00] problem, getting transferred, you're on [9:02] hold again. This whole support [9:03] experience we all know is like just just [9:06] primed for reinvention here. What if [9:08] there was an always on 247 agent that [9:11] could really help solve your problem? [9:13] And and we're we're seeing this really [9:16] uh resonate with customers where people [9:18] I think originally might have been like [9:19] I don't know is this an AI where it [9:21] there's real value and if something can [9:23] solve your problems at your beck and [9:25] call with the right guard rails and [9:27] answers and everything then there's [9:29] definitely real value here we're seeing [9:30] in the market and part of that is [9:32] languages too. there is over 80% of the [9:35] world does not speak English as a [9:37] primary language and it's really nice to [9:40] be able to voice switch between you know [9:42] between an agent and say look I'm going [9:44] to support all these whatever dozens of [9:46] languages um you know there's there's a [9:49] whole range of what what this space can [9:52] do in terms of language switching if you [9:54] just think about any content that's out [9:56] there to be able to unlock all these [9:58] other languages and regions not just to [10:01] support those users but to support their [10:02] preference too. So, for example, for [10:04] Spanish alone, we have 17 different [10:07] dialects because if you're in Venezuela [10:10] versus Colombia, Puerto Rico, Spain, [10:13] Mexico, like you're if you really know [10:15] Spanish or you're a local of any of [10:17] those regions, you're going to have a [10:18] little bit of a dialect just as in the [10:20] States here, you know, we have people [10:22] from West Coast, Northeast, South, etc. [10:24] You're going to have a little bit of a [10:25] different dialect. And really resonating [10:27] with the user, whether it's their [10:29] language or dialect, we found to be just [10:30] a better experience overall. All right, [10:33] multimodal, no shocker here. Whether [10:36] it's, you know, through telefan [10:38] integration systems, SDKs in a mobile [10:41] app on a website, uh there is, you know, [10:44] certainly text, voice, everything. [10:46] There's so much opportunity for [10:47] multimodal. As you all are thinking [10:49] about the agents you're building, I'm [10:51] sure there are some agents that you just [10:53] don't need voice. You just want them to [10:54] work in the background. I imagine most [10:56] of you are here today to learn more [10:58] about those voice agents. We see an [11:00] enormous explosion of, you know, these [11:02] these like really innovative voice [11:04] interfaces. You just think about how [11:06] have audiobooks and podcasts been so [11:08] popular over the last years. It's not [11:11] just the convenience, it's the [11:12] engagement factor and you you know you [11:14] can really listen and and engage and [11:16] we've seen the same across u multimodal [11:18] with voice text and all these [11:20] integrations too. All right, so let's [11:22] talk about some operational best [11:23] practices and lessons here. Uh [11:25] integrating on uh the knowledge base. I [11:28] think knowledge base is such an [11:31] incredible variable where we see people [11:33] that can really dial this. When we first [11:35] turned uh L, we call her the 11 Labs [11:38] support agent on our docs, you know, we [11:40] found like, oh, there was a couple [11:42] answers that were wrong and we just [11:44] realized that like our docs weren't up [11:45] to date. So clearly like the docs need [11:47] to be up to date. Also we've seen in a [11:49] lot of deployments where people will [11:51] just dump like their entire knowledge [11:53] base in and the larger the knowledge [11:55] base you get like you know there is some [11:57] latency factor there right so by having [11:59] a supervisor agent that then can direct [12:02] into different knowledge bases different [12:04] tasks essentially we've seen that reduce [12:06] latency and ultimately up the [12:08] reliability of the responses but I just [12:10] I I think you know if you come with like [12:12] a pretty messy knowledge base it's just [12:15] going to be a real challenge to get [12:17] towards like that 100% reliability that [12:19] that you want um you know with any [12:21] provider. All right, get your questions [12:23] ready because we got a little uh you [12:24] know a little bit more um and I'll show [12:27] you a little you know video if we have [12:29] time as well. But this one is really key [12:32] for those that are technical in the [12:33] audience. I mean the way that software [12:34] development has happened for many years [12:36] is the the exact same way that agents [12:39] should should roll out right. not going [12:41] to go big bang approach overall like you [12:44] know having a canary like doing your [12:46] testing and evals and being able to roll [12:48] back. This is the same way we should [12:50] approach agents in terms of what you're [12:52] building as well because you don't want [12:54] to just go big bang and then have to [12:55] roll something back uh overall and uh [12:58] and potentially have a misstep there. [13:00] And the last one I think you know I've [13:02] been in a lot of deep discussions with [13:03] some enterprise clients around build [13:05] verse buy and I think generally you know [13:08] people will will try uh you know to to [13:11] start with just bringing in maybe our [13:12] texttospech API or maybe our speechto [13:15] text API and then they get into this [13:17] orchestration latency turn taking it [13:19] gets super challenging and so from from [13:22] that it's like okay there there are some [13:25] endto-end platforms out there there are [13:27] folks that are going to want to build [13:28] everything themselves usually we find [13:30] somewhere in the middle, right? Where [13:31] it's like, of course, you can connect [13:32] some APIs, we can help you with the [13:34] overall orchestration integrations, but [13:37] there's just so much complexity here [13:39] when you add tool calling, when you add [13:41] knowledge bases, and when you get into [13:43] all of like the the scenarios that you [13:45] want to cover with that orchestration, [13:47] turn taking, latency, voice quality, [13:50] consistency, and all of that. you know, [13:52] I I just highly recommend finding a real [13:54] trusted partner that's done some of [13:56] these deployments that that can really [13:58] relate to your use case. So, with that, [14:01] uh, we've got just a few minutes for [14:03] some Q&A. Please do join us for the [14:06] happy hour tonight. Uh, would love to [14:08] see you all there. We're going to be [14:09] like just as you're walking out of the [14:11] the main hall. It's the first restaurant [14:13] on your left, uh, Nelly's Southern [14:15] Kitchen. I'm sure you all have have [14:16] passed it and seen it already. Uh, after [14:19] today, I'm sure you'll deserve a drink. [14:21] So, uh, take, uh, you know, take a a [14:24] picture here of the QR code, pull it up, [14:26] register, and we'd love to have you [14:27] there and see you. Uh, what questions do [14:30] you all have in the audience? Oh, we got [14:32] a lot of good ones. I think you were [14:33] first, please, sir. Yeah. Uh, [14:35] outstanding questions around identity [14:36] and multiple personas. 11 Labs can [14:39] support both of those. Ultimately, you [14:40] need some sort of identity layer with a [14:42] lot of these workflows to be able to [14:43] authenticate, call a database, and say, [14:45] "Okay, are you sorry, what's your name, [14:47] sir?" [14:48] >> Kisho are okay. Is this Kisho? Can we do [14:51] some sort of validation and then get to [14:53] the agent process and then multiple [14:55] personas whether it be like passing to [14:57] different agents or on the customer side [14:59] okay we have a different person there's [15:01] some complexity there but definitely [15:02] something we can help with great [15:03] question thank you please yeah I think [15:06] uh so the question was around techniques [15:08] to overcome the friction to speaking to [15:10] an AI I think being upfront I've seen [15:12] work well hey I'm an AI agent but I'm [15:15] here to help you with X Y or Z and pass [15:16] you to a human as needed right those are [15:19] like you're you're claiming the AI up [15:21] front, but then also saying, "Hey, if if [15:23] I can't help you with an answer, then I [15:26] can pass to an agent or, you know, a [15:27] human agent." I think those are two [15:29] techniques I've seen work well. It [15:30] really depends on the use case, though. [15:32] Sometimes people just like to jump right [15:34] in, not do any disclaimer because the [15:36] human-like emotion does carry through [15:38] and that can be helpful, too. That [15:40] that's a tricky one, though. Really [15:41] depends on the specific sort of sales [15:42] use case. Please, sir. Of course, with [15:45] your permission to clone, we there's two [15:47] there's two ways. uh there's a 30-se [15:49] secondond just upload a quick voicemail [15:51] and you'll be like yeah that is my CEO [15:53] or there's a professional voice clone we [15:55] recommend like 30 minutes or more of [15:57] highquality audio and that's going to be [15:59] like indistinguishable. So Jensen from [16:02] Nvidia clones his voice translates it to [16:04] you know Mandarin to do keynotes using [16:06] 11 Labs and it's really good quality. So [16:09] awesome questions would love to answer [16:11] some afterwards. Uh thank you all. If we [16:13] have just a minute, we could do a quick [16:16] video. [16:17] >> Building AI agents that can speak is now [16:19] easier than ever. [16:22] >> Hi, what would you like to talk about [16:23] today? [16:24] >> Introducing conversational AI with 11 [16:27] Labs. Build, test, and deploy. Allin-one [16:30] platform for the most natural way to [16:32] communicate with technology. [16:34] >> Hey, can I check the status of my order? [16:37] >> Yes, of course. Let me check for you [16:38] now. Your order is due to arrive at [16:40] around 2 p.m. [16:42] >> Building conversational AI agents used [16:44] to be immensely complex and demand huge [16:46] resources, but not anymore. Start [16:49] building your agent by creating a voice [16:51] or exploring our vast highquality [16:53] library to discover one that's perfect [16:55] for your application. [16:56] >> Hello and welcome to DHK online. How can [16:59] I help you today? [17:00] >> With a wide range of the best LLMs to [17:02] choose from, you can then upload your [17:04] knowledge base and define the goal and [17:06] personality of your agent. Or [17:08] alternatively, you can integrate your [17:10] own server to take full control. [17:12] >> Hi, this is Eric from Bound Events. How [17:14] can I help? [17:15] >> Hey, I accidentally uh bought a ticket [17:16] twice. Could I get a refund, please? [17:18] >> Sure. Please read out your reference [17:20] number, and I can begin to process that [17:21] right now. [17:22] >> Our system will analyze and evaluate [17:24] your transcripts, providing you with [17:26] valuable insights, conversation [17:28] playback, and the ability to test and [17:30] define success criteria. And it's all [17:32] possible in 32 different languages. [17:36] >> Welcome to DC Medical. How can I help? [17:38] >> Hello. [17:44] And best yet, effortlessly deploy your [17:46] agent into websites with a single copy [17:48] and paste. Or leverage our SDK for [17:51] creating apps, services, or even video [17:54] games. So, whatever you're creating, get [17:56] to production in days, not months. And [17:59] all with enterprisegrade security to [18:01] keep your users data safe and secure. [18:04] It's time to explore the huge potential [18:06] of conversational AI with 11 Labs. [18:10] >> Such an exciting space. Hope to see you [18:12] all at our booth and the happy hour [18:13] tonight. I'll be around for questions. [18:15] Give it up for you all. Thank you so [18:16] much.