The world of voice AI, with Mati Staniszewski of ElevenLabs

对话ElevenLabs联合创始人Mati Staniszewski：语音AI的世界 Mati Staniszewski梳理了语音AI的技术演进历程，从早期模拟声道的复制，到将语音特征视为涌现属性的现代神经网络。他详细介绍了ElevenLabs的商业模式，重点阐述了他们依托Stripe向按量付费计费模式的转变，以及针对企业客户打造高质量级联语音智能体的战略重心。对话还探讨了AI时代的组织架构设计，强调了扁平化结构、嵌入式技术主管的作用，以及声音修复和本地化数字服务所带来的深远社会影响。 现代音频模型利用神经网络根据上下文预测下一个声音，将口音等特征视为涌现属性，而非硬编码的参数。 (0:27) ElevenLabs依靠内部数据标注团队和定制的语音转文字模型来构建高质量数据集，以捕捉语音中细微的情绪变化。 (4:52) 该公司将自身定位为横向平台，把高度细分的特定领域垂直应用留给生态系统的开发者开发。 (8:52) 推出ElevenReader是为了解决分发难题，让用户能够通过官方授权的标志性声音来收听PDF和各类文档。 (13:57) 通过语音图灵测试极其困难，因为这需要在轮流发言、静音检测和上下文线索处理等方面进行复杂的实时编排。 (17:06)

ElevenLabs co-founder Mati Staniszewski argues that voice AI has matured from simple text-to-speech into interactive, enterprise-ready voice agents powered by context-aware architectures and high-fidelity emotional modeling. He contends that pairing specialized, cascaded models with flat, high-agency team structures enables rapid commercial expansion while fundamentally reshaping how humans interact with software. Audio Architecture and Model Trade-offs (0:01) Modern voice generation relies on applying transformer and diffusion concepts to speech, predicting audio tokens and phonemes in sync with text context rather than relying on hand-crafted accents or emotional parameters. Staniszewski emphasizes that leading voice models generally span from a few billion to tens of billions of parameters, making them significantly smaller and cheaper to train than frontier large language models. The company prioritizes proprietary dataset curation annotated for emotional delivery ('how' it was said rather than just 'what'), which enables controllability in its V3 model, such as explicit pacing and dramatic pauses. For enterprise systems, ElevenLabs favors a cascaded pipeline—transcribing speech to text, running an LLM, and generating speech—over end-to-end speech-to-speech models. While direct speech-to-speech bypasses intermediate layers to cut latency, Staniszewski notes it remains prone to hallucinations and lacks enterprise visibility. By contrast, the cascaded approach provides the determinism, integration capabilities, and auditability required by institutional clients, while low-latency speech-to-speech remains better suited for informal companion applications. Enterprise Expansion and Pricing Economics (44:08) ElevenLabs has transitioned over half its revenue to sales-led enterprise contracts, driving an estimated annual run rate of roughly $450 million after generating $100 million in net new ARR in a single quarter. Enterprise adoption follows a land-and-expand pattern across clients such as Deutsche Telekom, IBM, and Klarna. In Deutsche Telekom's case, adoption started in marketing for podcast generation before widening into customer support and ultimately network-wide conversational agents. The company pairs high-touch enterprise agreements with an open, self-serve tier, maintaining that direct customer access provides rapid feedback on model performance. ElevenLabs prices text-to-speech by token and voice agents by minute, and is adopting a pay-as-you-go model via Stripe. To accelerate adoption and help users explore new capabilities without immediate financial friction, ElevenLabs intentionally offers newly released, computationally expensive models to customers at cost. Voice Agents and Changing User Behavior (30:24) Voice agents are evolving from reactive support channels into proactive, persistent actors capable of outbound execution, such as automated SDRs or tools that place routine phone calls to gather real-time data. Replacing static web forms with conversational voice agents on ElevenLabs' inbound lead pipeline altered user behavior: prospective clients completed submissions at higher rates and shared far more detailed, open-ended context than they typically wrote down. Beyond commercial automation, Staniszewski highlights high-fidelity voice synthesis as an assistive breakthrough. ElevenLabs has deployed voice cloning to restore personalized voices for patients suffering from ALS and throat cancer, enabled a Neuralink patient to speak with family using their own synthesized voice, and facilitated full wedding ceremony vows for an individual who had lost her voice. Organizational Structure and Embedded Technical Talent (51:17) Operating with 470 employees, ElevenLabs maintains an intentionally flat corporate structure where both co-founders oversee more than 15 direct reports and keep functional teams below 10 people. Rather than centralizing engineering into isolated software units, the company embeds technical leads directly inside non-technical departments like talent acquisition and go-to-market. These leads build targeted automations, such as parsing candidate pipelines, structuring interview preparation voice bots, and auto-populating client pitch decks with custom data. Staniszewski points to Ukraine's digital government application, DIIA, as an external validation of this operating model. In response to wartime disruptions, Ukrainian ministries integrated voice capabilities and deployed independent technical teams within each ministry to build agentic public services, which were then unified centrally. Staniszewski views this alignment as proof that high-agency individuals and decentralized technical ownership outperform rigid corporate hierarchies in the AI era. While ElevenLabs forecasts passing the voice Turing test in bounded domains like customer service within the coming year, broader consumer deployment still faces constraints. Universal voice interfaces continue to struggle with turn-taking latency, multi-step analytical reasoning, and restrictive mobile operating systems, leaving cloud-based automotive voice models slated for the near term while fully offline, in-car voice systems remain two to three years away.

ElevenLabs 联合创始人 Mati Staniszewski 指出，语音 AI 正从单向的文本朗读演进为具备情境感知与调度能力的主动型智能体；凭借轻量于纯文本大模型的参数经济学、可控性优先的级联技术路径以及兼顾自助分发与大客户直销的双轨策略，该公司正加速跨越特定领域的语音图灵测试并重构企业工作流。 模型架构演进与级联工程权衡 (0:01) 现代语音模型借鉴了 Transformer 与扩散模型的核心机制，实现了从梅尔频谱到波形的高质量、低延迟重建。Staniszewski 强调，ElevenLabs 并不对口音或情绪进行机械化的人工规则编码，而是将其作为模型在丰富文本语境下自主学习形成的“涌现属性”。为了精准捕捉人类语音不仅关乎“说了什么”更关乎“如何表达”的细微特征，公司专门组建了内部人工与半自动数据标注团队，并自研了配套的语音转文字模型以支撑高纯度训练集构建。 在底层架构上，ElevenLabs 面向严谨的企业级场景坚决押注串联语音识别、语言模型与语音生成的“级联方案”（cascaded approach），而非端到端的“语音到语音”（speech-to-speech）直连模型。Staniszewski 认为，直连模型尽管响应极快，但由于受制于模型体积而在逻辑理解上“明显更弱”，更适合容忍甚至欢迎幻觉的伴侣类应用；级联方案则能确保企业所必需的各阶段可见性、高准确度与系统集成。此外，针对现实环境中复杂的背景杂音与口音失真，公司正推进基于说话人自适应与信息论关键词匹配的个性化转录过滤技术。 交互复杂性与语音智能体的落地场景 (17:06) 真正的语音智能体远超越传统的朗读功能，其本质是具备持续生命周期、能与物理及数字世界双向交互的实体。Staniszewski 解释称，构建互动语音智能体的难点在于多维要素的同步调度：系统必须依靠静音检测和上下文线索精准掌握轮流发言（turn-taking）的节奏，既要在用户未完成表达时保持倾听，又要在适当时机主动追问，并在通话进程中从容调用鉴权与数据库查询。尽管基于文本的语言模型早已通过图灵测试，但全局通用的语音交互仍远未达到该水平，目前的突破主要集中于客服呼叫等边界清晰的垂直领域，公司目标是在未来一年左右在各类智能体中跨越这一门槛。 在落地应用端，ElevenLabs 涵盖了面向内容消费的阅读软件 ElevenReader（引入 Michael Caine 和 Richard Feynman 遗产委员会等官方授权人声）、向外主动拨号的信息收集工具（如致电三千家爱尔兰酒吧调研啤酒价格的 Guinndex），以及应用于金融与汽车行业的 AI 销售开发代表。Staniszewski 界定了明确的业务边界：公司不寻求在需要多步骤重度推理的深度财务分析等复杂领域开展优化，而是坚持“语音优先”哲学，认为攻克语音交互的高维复杂调度后，底层文本逻辑的集成便能顺理成章地解决。 算力经济学、定价策略与企业级爆发增长 (35:12) 与动辄数千亿参数且训练成本极高的顶尖纯文本语言模型不同，前沿语音模型的参数规模通常维持在数十亿至数百亿级别，因而具备显著更高的训练与迭代经济性。ElevenLabs 在获得 5 亿美元融资并达到 110 亿美元估值后，将重量资本主要投向顶尖音频研究团队的建设。在商业计费上，公司对基础语音合成按文本 Token 计价，对智能体及转录服务按分钟计费；在推出更大规模的前沿新模型时，公司甚至采取“按成本价”让利的策略，以消除客户早期的尝试门槛并激发创新用例。 在商业化路径上，ElevenLabs 依托产品驱动增长（PLG）自助端与直销大客户相结合的双轨模式，借助底层支付基建（如 Stripe 针对用量计费的整合）全面转向按需付费体验。其企业端采取经典的“先着陆再扩展”策略，例如在德国电信的合作中从营销播客制作延伸至客户支持，最终升级为全网级语音智能体。据披露，公司在 2025 年底公布的年化经常性收入（ARR）运行率为 3.5 亿美元，随后通过单个季度实现净增 1 亿美元 ARR 的高速增长，当前运行率已攀升至约 4.5 亿美元区间，企业客户包含 Deutsche Telekom、T-Mobile、Revolut、Klarna、Meta 和 IBM 等行业巨头。 扁平架构与技术嵌入的高能动性组织 (51:17) 随着员工规模扩张至 470 人，ElevenLabs 始终推行高度去中心化且极其扁平的组织管理模式：两位联合创始人各自直接管理超过 15 名团队负责人，单项研发与业务项目团队人数均严格限制在 10 人以内。为避免非技术部门受制于传统企业软件的僵化流程，公司在运营、招聘与市场等业务板块中全员配置专属技术带头人（Tech Leads），直接运用大模型和自动化工具重塑工作流，自研出候选人特征挖掘系统、个性化客户简报生成工具以及供求职者体验公司文化的互动面试语音智能体。 Staniszewski 将这种组织理念与乌克兰战时政府的数字化实践相印证。乌克兰在实体基础设施受创的环境下，通过核心数字平台 DIIA 为民众提供医疗、救济与前线信息，并在各部委均设立专门的技术力量开发“智能体化”服务模块。Staniszewski 认为这高度契合公司的用人标准，即在 AI 技术重塑生产力格局的背景下，具备强大自我驱动与解决问题能力的高能动性（high agency）人才将成为组织进化的主导力量。 Staniszewski 展现了语音技术打破全球语言壁垒、助力医疗康复与自动化销售的愿景，并预计车载云端模型将在年内实现突破；但他也承认，完全离线的车载模型仍需两至三年演进，复杂交互下的语音图灵测试仍未在全局场景完全攻克。此外，在端到端直接多模态语音技术持续迭代的背景下，级联架构在可靠性与延迟之间的工程权衡是否具有长期不可替代性，仍是一个有待时间检验的关键问题。

## Background 该视频由 Stripe 官方频道于 2026 年 5 月 7 日发布在 YouTube 平台上。页面简介显示，主持人 John 与 ElevenLabs 联合创始人 Mati Staniszewski 展开深度对谈，围绕“语音图灵测试”、AI 在文本与语音对话上的表现差异、人机交互未来、ElevenLabs 达到 110 亿美元估值的迅猛增长之路，以及乌克兰如何将其技术应用于数字化政府服务等核心议题展开探讨。 ## TL;DR Mati Staniszewski 系统梳理了语音合成技术的底层演进脉络，深入剖析了级联架构（Cascaded）与端到端原生语音（Speech-to-Speech）的优劣权衡，并阐明语音模型中情绪与口音等特征的“自主涌现”机制。对话探讨了为何消费级语音交互长期存在“产品悬殊”、ElevenReader 的破局逻辑，以及语音 Agent 如何重构企业潜客转化与业务全流程。此外，Mati 还分享了 ElevenLabs 突破 4.5 亿美元 ARR 的商业飞轮、全面转向按用量付费（Pay-as-you-go）的策略调整、小团队扁平化的 AI 原生组织设计哲学，以及通过技术赋能失语者与乌克兰数字化政府建设的深刻实践。 ## Atmosphere & style 整场对谈兼具硬核的技术底色与深厚的人文关怀，语调沉稳、务实且充满思想密度。Mati 与主持人之间的交流默契流畅，既有创始人从第一性原理拆解工程瓶颈与商业模式时的理智清醒，也有聊到为失语患者重建原生声线、协助战时乌克兰搭建数字化政府时的动容与崇高感。聆听体验如同置身于前沿技术先锋的深度复盘会，既能感受到 AI 狂飙突进下的战略野心，又充盈着对人类声音尊严与组织本真的质朴敬畏。 ## Bulletin - [44:08](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2648s) ElevenLabs 业务规模实现爆炸式增长，最新 ARR 已冲破 4.5 亿美元（单季度新增净 ARR 超 1 亿美元），企业级业务贡献占比已超过 50%，全员 470 人保持 10 人以下敏捷小团队运作。 - [49:55](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2995s) 在商业模式上，ElevenLabs 正式宣布全面推行按用量付费（Pay-as-you-go），彻底打破针对个人与普通开发者的固定包月订阅额度限制，大幅降低调用门槛。 - [27:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1661s) 在架构选型上，ElevenLabs 重点押注级联架构（Cascaded Pipeline：STT+LLM+TTS），凭借高确定性、企业工具调用集成和强大推理能力支撑严肃商业场景，而端到端原生语音（S2S）受限于推理算力在逻辑复杂度上仍显笨拙。 - [57:00](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3420s) 乌克兰国家级政务平台 Diia 正在全面整合 ElevenLabs 技术推进首个“Agent 化政府”，其在各部委常驻技术力量推进数字化的模式有力印证了 ElevenLabs 内部的嵌入式组织哲学。 ## Key takeaways - [01:46](https://www.youtube.com/watch?v=zA1zdqIYER4&t=106s) 现代深度学习语音模型彻底摒弃了传统的机械物理模拟与音素拼接法，英音等口音与情绪特征已成为模型基于上下文自发形成的“自主涌现”（emergent property），而非人工硬编码。 - [04:47](https://www.youtube.com/watch?v=zA1zdqIYER4&t=287s) 语音质量的壁垒不仅在于模型架构，更在于自建的高精度标注体系与自研 STT 工具，有效补足了海量通用音频语料中严重缺失的“说话方式、情绪与口音”等动态维度。 - [12:42](https://www.youtube.com/watch?v=zA1zdqIYER4&t=762s) 公众感知到的语音技术停滞并非算法问题，而是传统工业巨头面临的“部署鸿沟”；高品质实时语音进入生产环境仅一年，真正摆脱云端依赖的纯车载本地离线交互仍需 2 至 3 年。 - [15:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=941s) 因 Audible 等主流平台排斥 AI 有声书，ElevenLabs 自建 ElevenReader 打通创作者分发链路，并与迈克尔·凯恩爵士和理查德·费曼遗产管理方达成了正版声线授权合作。 - [17:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1070s) “语音图灵测试”的攻克难度远超文本模型，关键瓶颈在于轮转判定（Turn-taking）、长难句中的静默上下文权衡、主动反问澄清机制以及工具调用期间的交互编排。 - [21:36](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1296s) 特定说话人微调（Person-specific transcription）仅需目标人声一小时音频即可显著提升嘈杂环境下的识别抗噪力，将在医疗手术室与智能家居中快速落地。 - [25:31](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1531s) V3 模型确立了精细化语音可控性与 Expressive Mode（情绪感知模式），语音 Agent 能够实时感知对端人类的焦虑情绪，并在文本和语调层面同步自适应切换为镇定安抚语气。 - [27:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1661s) 在架构博弈中，级联架构（Cascaded）在透明度、审计容错、大模型推理智力与企业工具编排上占优；而端到端（S2S）模型虽然延迟极低，但受限于模型体量，纯逻辑推理能力相对薄弱。 - [30:24](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1824s) 将传统的网页静态表单替换为语音 Agent，不仅大幅提升了潜客转化意愿，还借助开放式对话与无缝跨语言能力发掘出客户更深层次的业务痛点与真实需求。 - [31:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1910s) 语音技术的次生效应正在重塑现实版“巴别鱼”，并在无障碍医疗场景中大放异彩，已成功协助渐冻症患者、喉癌失语者以及 Neuralink 脑机接口植入者重塑声音尊严。 - [35:30](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2130s) 垂直场景的纯语音生成模型参数量通常维持在数十亿至数百亿，远小于万亿级 LLM；ElevenLabs 采取新模型早期贴近成本甚至倒贴的定价策略以迅速构建分发网络与反馈闭环。 - [47:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2870s) 坚持自助服务（PLG）与现场工程支持并举，不仅源于对自身技术的绝对自信，更能通过中小开发者的前沿极限探索预演未来企业级产品演化方向。 - [53:07](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3187s) AI 原生企业的组织重构范式体现为极度扁平化（管理跨度翻倍，核心高管直属超 15 人），并在人才、运营等非技术部门深度嵌入专属 Tech Lead 自研自动化工具。 - [58:38](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3518s) 高自主性（High Agency）成为 AI 时代最核心的人才分水岭，拥有强主人翁意识与主动探索能力的个体将借由 AI 杠杆获得生产力的几何级放大。 ## Detailed walkthrough ### [00:01](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1s) 语音模型的底层演进：从机械模拟到神经常规化 Mati Staniszewski 于 2022 年创立了 ElevenLabs，并迅速将其打造为估值达 110 亿美元的语音 AI 领头羊企业。其核心突破在于能够精准捕捉人类语音中的情绪起伏与细腻语调，且产品版图已从最初的文本转语音延伸至 Agent 工作流和音乐生成。 在被主持人问及“如果像 Andrej Karpathy 那样从零构建一个玩具级音频模型，其底层原理究竟是什么”时，Mati 梳理了人类合成声音的探索史： 1. **机械物理模拟时代**：早期人们尝试模仿人体发声器官，制造精巧的模拟发声机械结构（如模拟声带和声道）； 2. **结构化信号与早期数字语音时代**：以 Bell Labs（贝尔实验室）为代表的研究机构，首次尝试将语音抽象为一套结构化的数字信号表示，这成为现代语音技术的奠基石； 3. **音素拼接技术时代**：研究人员构建包含人类发声基本单位的“音素库”（phoneme library），依据下一个词的概率预测，从库中提取音素并用拼接法（concatenation）组装成句子； 4. **现代神经网络时代**：类似于自然语言大语言模型预测下一个 token，现代语音模型基于上下文连续预测下一个声音单元。若处理的是流式音频，模型依赖前面的连续音频片段；若属于富上下文生成任务，则联合预测音素序列与文本语义上下文。 ```mermaid flowchart LR accTitle: 音频合成技术演进 accDescr: 展示从早期的物理发声机械、贝尔实验室信号，到音素拼接以及现代端到端神经网络音频模型的技术演进过程。 subgraph Early["传统时期"] A["机械声道模拟<br/>Wolfgang von Kempelen"] --> B["贝尔实验室结构化信号<br/>硬编码特征参数"] B --> C["音素拼接法<br/>基于概率库拼接片段"] end subgraph Modern["深度学习时期"] D["Tacotron / WaveNet<br/>文本 -> 梅尔谱 -> 波形"] --> E["ElevenLabs 现代架构<br/>多模态上下文预测 + 自主表征编码"] end C --> D ``` --- ### [01:46](https://www.youtube.com/watch?v=zA1zdqIYER4&t=106s) 架构层面的两大颠覆：上下文融合与特征的“自主涌现” 传统的深度学习语音方案（如 Tacotron、WaveNet）通常采用流水线作业：输入文本被转换成表示声音能量与音高的“梅尔频谱图”（Mel spectrogram），然后再通过声码器解码成实际的连续波形（waveform）。Mati 指出，其联合创始人 Piotr 引入了来自 Transformer 与扩散模型（diffusion models）的开创性思想，不仅简化并重构了在梅尔频谱与波形之间的编解码过程，还攻克了以下两个关键维度： - **深度上下文理解能力**：专业配音演员在阅读文案时，能从行文环境感知到说话者是在用某种方言对白、还是在表现狂喜或愤怒。声音模型不仅要看当前的词，更要捕捉上下文的前后因果，从而做出符合情景的语调抑扬。 - **声音特征的自主涌现，而非硬编码**：早期合成系统由工程师预先硬编码参数（例如在控制面板上圈定音高变化范围、能量级别或给定的“热情英国腔”预设）。而 ElevenLabs 的模型拥有开放的选择空间，能够自主推导音色、口音、热情或悲伤等风格特征。“在你们的声音模型里，‘英音’竟然是一种涌现特性（emergent property）？”面对主持人的追问，Mati 给出了肯定的回答。模型并不被人工告知“此时使用英音或波兰口音”，而是根据上下文自发涌现最真实的声音质感。 --- ### [04:47](https://www.youtube.com/watch?v=zA1zdqIYER4&t=287s) 音频模型的 Token 是什么？音素、文本与内部标注体系 对于大语言模型（LLM）而言，理解世界的基本单位是文本 subword 或 Token。但在语音模型中，情况复杂得多： - **音频维度的单元**：基础声音单元相当于音素（phoneme），即比音节切分得更细碎的人类发声微小元素； - **声学特征嵌入**：模型在输入端除了接收目标文本，还必须同时注入包含特定说话人音色特性的声音嵌入（voice embedding / speaker encoding）； - **跨模态同步运算**：在实际生成中，系统在文本 Token 层面与音频特征 Token 层面保持同步推进。尤其在实时低延迟的流式场景下，双层协同对于连贯组织语句、准确预测下个声音起伏至关重要。 在解释 ElevenLabs 声音为何听起来“格外像真人”时，Mati 强调模型架构、算力和数据缺一不可。业界虽然坐拥海量音频语料，但极少具备高精度的标注体系——大多数语料或许标注了“说了什么文本”，但完全缺失了说话人何时切换、带着什么情绪语气、使用了哪种独特口音等关键的“如何发声”（how）信息。 为此，ElevenLabs 没有依赖外部一般性的标注平台，而是自建并培训了专业的数据标注团队： - 结合半自动算法与人工听辨，对情感、口音及声音特质打上精细标签； - 因市面上现成的语音转文字（STT）模型精度无法达到清洗数据的严苛门槛，他们最初为了自用而研发了一套超强 STT 模型，最终该模型反倒孵化成了一款对外输出的核心产品； - 这种自下而上的“自建实用工具”循环，驱动了多款衍生产品的问世。 谈到这里，Mati 穿插了一段趣味历史掌故：最早设计出模拟人类声道发声机（由风箱、振动簧片及阀门组成）的奥地利发明家沃尔夫冈·冯·肯佩伦（Wolfgang von Kempelen），正是历史上著名的“发国土耳其人”（The Turk）国际象棋下棋装置的制造者。那台看似神奇的自动化下棋机器其实完全由藏在内部的人类棋手秘密操作——这也是现代众包标注平台“Mechanical Turk”名字的来源，而 ElevenLabs 正是在数据流水线里深度应用着人工校验与标注。 - 语音模型的表征输入：文本 Token + 说话人嵌入向量（Speaker Embedding） - 传统音频生成链路：文本 $\rightarrow$ 梅尔频谱图（Mel Spectrogram） $\rightarrow$ 波形（Waveform） - 数据标注瓶颈：通用语料库普遍欠缺对声音质感、情绪及口音的精准标定 --- ### [08:52](https://www.youtube.com/watch?v=zA1zdqIYER4&t=532s) 从基础模型到业务落地：ElevenLabs 的全栈产品定位 主持人将话题切回 ElevenLabs 的真实商业架构，询问它如何从外界眼中的“配音工具公司”演变成全面的基础设施提供商。Mati 将 ElevenLabs 定义为一家兼具“基础研究”与“产品落地应用”的双核公司，其核心目标是为企业客户重构内外部沟通方式。 业务架构可以清晰地划分为两大层次： 1. **基础音频模型层（Foundational Models）**： - **TTS（文本转语音）**：生产富含情绪、音色生动的高保真语音输出； - **STT（语音转文本）**：覆盖超 100 种语言，在各项行业基准测试中名列前茅； - **多模态与语音 Agent 架构**：打通多方模型的 Conversational 闭环系统，并延伸至音乐生成等更广泛的音频领域。 2. **企业级平台与产品部署层（Platform & Deployments）**： - 单纯输出声音并不足以支撑业务交付。在 Voice Agent（语音智能体）场景中，平台提供与企业现有知识库（Knowledge Base）的检索集成、电话系统（Telephony）的互通对接，以及调用业务系统的具体 Action 触发器； - 提供端到端的会话监控、延迟优化与性能评估体系，使 AI 智能体能够真正胜任客户服务、销售拓展、人员招聘、企业内训乃至专业内容创作与市场营销等复杂工作流。 ### [10:19](https://www.youtube.com/watch?v=zA1zdqIYER4&t=619s) 平台定位与垂直应用：横向赋能与垂直壁垒的选择 在创意与营销端，声音生成的关键在于如何为广告大片打造引人入胜的视频配音，或用契合品牌调性的专属声线朗诵长篇深度文章。ElevenLabs 将底层语音模型与对客户业务场景的深刻理解融为一体，构建起统一的安全规范与策略平台（Policy Platform）。然而，每一个平台型技术公司都会面临一个经典的战略抉择：到底该作为一个纯粹的横向平台赋能整个开发者生态，还是亲自下场涉足具体垂直应用的开发？以视频自动闭路字幕（Closed Captioning）为例，市面上完全可以涌现出一批直接构建在 ElevenLabs 接口之上的第三方独立工具，平台似乎并没有必要亲自包揽一切。 针对这一界限，Mati Staniszewski 阐述了 ElevenLabs 的核心逻辑：如果一个业务场景属于普适的横向通用需求，ElevenLabs 就是最好的底层平台；但如果场景具备高度的垂直行业专属性（Domain Specificity），未来必然会由专属的应用层公司来主导，那正是 ElevenLabs 刻意选择不进入的领地。主持人指出，在技术迭代极其迅猛的当下，平台若过度依赖中间商存在巨大的脱节风险——如果某款字幕工具仍停留在两个版本前的 ElevenLabs 老模型而未及时更新，终端用户就无法体验到每周上线的新功能。Mati 对此表示赞同，指出当前 ElevenLabs 自身的转写与语音服务已能覆盖 99.9% 的常规用户需求，但在医疗等极其专业的场景中，必须为特定客户定制模型，并在音频生成前嵌入专门的技术术语词典，以彻底解决专业上下文转写的准确性难题。 ```mermaid flowchart TD accTitle: ElevenLabs 平台与垂直应用划分架构 accDescr: 展示 ElevenLabs 横向底层平台与特定垂直行业应用的分工边界 subgraph Platform["ElevenLabs 横向平台层 (99.9% 通用场景)"] A["多模态语音与转写模型"] --> B["安全规范与策略平台"] B --> C["高频每周迭代 API 部署"] end subgraph DirectUse["通用业务落地"] C --> D["视频营销配音与品牌声线"] C --> E["通用长文本朗读与闭路字幕"] end subgraph VerticalEcosystem["特定行业垂直应用"] C --> F["医疗健康定制模型"] F --> G["专业医学术语词典预处理"] C --> H["高垂直领域独立 ISV 应用"] end ``` - **通用覆盖率**：ElevenLabs 的通用服务能力已能直接解决 99.9% 的客户基础转写与配音诉求。 - **垂直专业门槛**：在医疗等高门槛场景，单纯依靠通用大模型极易在生僻术语上出错，必须通过定制模型与专属技术词典进行前置校准。 --- ### [12:42](https://www.youtube.com/watch?v=zA1zdqIYER4&t=762s) 语音 AI 的“产品悬殊”：为何我们感觉仍生活在十年前？ 主持人和 Mati 深入探讨了一个在 AI 领域极具反差的现象：文本大语言模型（LLM）如 ChatGPT、Gemini 和 Claude Code 已经深入各行各业，日常使用量极其庞大；然而前沿语音模型虽然在实验室和演示中无比强大，实际的终端产品体验却存在巨大的“产品悬殊”（Product Overhang）。主持人分享了一段亲身经历：他在开车回家途中急需阅读一份 PDF 文档，原本期望手机能直接朗读出来，但尝试使用 iOS 自带的屏幕朗读功能时，滚动翻页直接崩溃；改用 Gemini 上传文档并要求“纯阅读不要总结”，应用却在点击朗读按钮时彻底卡死。如今几乎所有智能汽车都在宣传语音控制，但实际交互体验依然糟糕透顶，连最基础的车机导航输入都难以顺畅完成，让人感觉语音技术仿佛还停留在十年前。 Mati 并不完全认同“语音技术倒退十年”的说法，他指出这本质上是一个严重的“落地部署鸿沟”（Deployment Gap）。汽车等传统工业巨头的生产周期与软件上车节奏极其缓慢，无法快速跟上最新前沿技术。从底层技术成熟度来看，真正能够自然朗诵文本的高质量异步语音模型，其实只是最近三年的成果；两年前实时语音模型才初见雏形；直到一年前，低延迟端到端实时语音才真正具备进入生产环境的可能。迈入 2025 年，技术攻坚的核心才转向真正的上下文联通——让实时语音模型能够精准感知用户当下的动作、读取特定文档材料，并无缝结合用户的历史使用偏好。这一能力在技术先锋企业中才刚刚开始被采纳，预计年内会逐步在车企的云端场景中铺开，而真正摆脱网络连接、在车端本地芯片原生运行的高质量语音交互，仍需要两到三年的硬件与模型工程迭代。 - **语音技术演进时间线**： - **3 年前（约 2022-2023）**：首次出现能够自然进行异步长文本朗读的高品质语音模型。 - **2 年前（约 2023-2024）**：低延迟实时（Real-time）语音交互模型开始萌芽。 - **1 年前（约 2024-2025）**：实时语音技术正式达到工业级生产部署标准。 - **2025 年及未来**：攻克强上下文关联与偏好记忆；车载云端语音有望年内落地，纯本地离线部署仍需 2 至 3 年。 --- ### [15:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=941s) ElevenReader 的诞生：打破有声书分发壁垒与名人声线重现 针对主持人在车内无法顺利朗读 PDF 的痛点，Mati 揭秘了 ElevenLabs 自家消费者端产品 ElevenReader 的诞生背景。2023 年软件初版上线时，大量独立创作者和图书作者涌入平台，他们无力承担数千甚至数万美元的专业真人配音成本，希望借助 AI 生成高质量有声书。然而当时整个分发渠道对 AI 采取封杀态度，以 Audible 为代表的主流有声书巨头全面拒绝或封禁 AI 生成的内容。面对创作者空有优质作品却缺乏分发渠道的绝境，ElevenLabs 被迫下场构建独立载体，从而打造了 ElevenReader。 在 ElevenReader 中，用户只需随手上传一份 PDF 或纯文本，系统就能调用顶级声线进行行云流水般的朗读。更为惊艳的是，ElevenLabs 官方与传奇影星迈克尔·凯恩爵士（Sir Michael Caine）本人以及物理学家理查德·费曼（Richard Feynman）的遗产管理委员会达成了正版授权合作，用户可以直接让“迈克尔·凯恩”或“理查德·费曼”的原声为自己朗读任意文献。主持人感叹这正是用户最需要的杀手级功能：“我恨不得让迈克尔·凯恩给我朗读生活里的一切！”同时，两人也探讨了移动操作系统的开放性问题：目前 Android 和 Nothing OS 等系统对第三方转写引擎和输入法相对开放，而苹果近期逐步放开其大模型生态接入，未来有望同样开放系统底层的语音转写与朗读接口，让第三方顶级语音引擎成为手机的全局标配。 - **分发倒逼产品**：因主流平台（如 Audible）排斥 AI 生成音频，ElevenLabs 直接推出 ElevenReader 为创作者开辟全新的数字分发通道。 - **明星正版声线合作**：平台已引入迈克尔·凯恩爵士（Sir Michael Caine）和诺贝尔奖物理学家理查德·费曼（Richard Feynman）遗产基金会的授权声音资产。 - **文档即时转换**：支持用户直接导入本地 PDF 和各类文档，直接生成具备专业演播级断句与感情的音频。 --- ### [17:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1070s) 对话式图灵测试：语音交互为何是 AI 最后的未竟之地 尽管早在 1969 年科幻经典《2001太空漫游》中就有了“HAL，把分离舱门打开”（"Open the pod bay doors, HAL"）的经典设想，语音助手作为人机交互领域最古老的交互范式之一，直到今天仍未真正成立。Siri 缺乏常识与高阶智能，而 ChatGPT 或 Gemini 的语音模式在实际使用中依然极其笨拙，逼得许多用户宁愿退回到最原始的交互方式：打开手机键盘自带的语音听写，说一长串文字输入到文本框里，再手动点击发送。这表明大众对语音交互有着强烈的刚需，但现有的语音产品体验根本无法承载这一期待。 Mati 深入拆解了为何通过“语音图灵测试”的难度远超文本模型： 1. **意图与执行的模糊边界**：用户说话时，有时希望 AI 立即执行命令，有时只是在长难句中间停顿思考，系统必须精准判定用户何时说完了整句话（Turn-taking 轮转机制）。 2. **静默与上下文权衡**：系统不能仅靠死板的静默时长来切断输入，必须实时结合前文语义判断是“话未讲完的短暂停顿”还是“抛出问题的彻底静默”。 3. **主动反问与多轮澄清**：真正的自然交流需要代理在信息不足时主动插话、反问澄清细节，并在必要时给出阶段性文字确认。 4. **工具调用（Tool Calling）的编排延迟**：有些回答可以脱口而出，而有些指令需要后台挂起并调用数据库或 API，如何编排这段空白期的语音交互而不显得机械僵死，是极高的工程挑战。 Mati 总结道，纯文本大模型早已在很大程度上跨过了图灵测试，但在语音多模态领域，真正的对话式图灵测试远未被攻克。目前语音 AI 只能在限定领域（如结构化的电话客服场景）表现出超越人类测试的水平；但在开放式互动场景（例如高动态、高自由度的人机拟真游戏 NPC 交互）中，想要达到真人般呼吸、打断、反问与思考并存的交互体验，前路依然漫长。 ### [20:27](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1227s) 迈向“语音图灵测试”与多系统编排 构建真正智能的语音 Agent 时，最关键的工程难点在于系统间的协同编排（Orchestration）。如果只是单纯的一问一答，路由系统将用户直接导向语音对话 Agent 相对容易；但当交互涉及到身份验证（Authentication）、从后台数据库动态调取数据时，Agent 如何优雅、得体地处理等待和衔接，便成了一大棘手挑战。 Mati Staniszewski 强调，业界目前正逐渐攻克这一关卡。ElevenLabs 的核心目标是在所有此类复杂场景中——乃至语音交互之外的全部对话场景中——全面通过“语音图灵测试”（Voice Turing Test）。他乐观地预测，整个行业有望在未来一年左右跨过这一重要里程碑。 ### [21:01](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1261s) 订阅经济的结账摩擦与 Stripe Link 在 SaaS 与订阅制商业模式中，用户往往会在完成结账前的最后几秒流失：无论是不得不离开座位翻找实体钱包、输错信用卡卡号，还是偶遇系统报错，任何微小的摩擦都会让潜在客户直接放弃支付。 对于 ElevenLabs 这样月增数十万订阅用户的高速增长企业而言，结账环节的细微阻力都会累积为巨大的收入损失。因此，ElevenLabs 选择接入 Stripe 的 Link 快捷支付功能。客户仅需保存一次支付信息，即可在超过一百万家支持 Link 的商户端实现秒级结账，从而大幅缩短交易链路并消除转化流失。 ### [21:36](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1296s) 个性化语音转写与特定说话人微调 对谈转入语音识别（STT）的瓶颈：为何当前的语音助手（如 Siri）依然很难完全听懂带有特定口音的用户？John 指出，用户本身的口音是恒定不变的，但底层运行的往往是通用全局语音识别模型，而非专门针对具体说话人的自适应模型。这就像在嘈杂的咖啡馆里对陌生人说话，对方若没有提前加载你的“口音过滤器”，便极难听清。 Mati 笑着透露，他本人的波兰口音在语音识别测试中非常棘手，因此他的声音长期作为内部测试套件（Test Suite）的一部分，反复跑在 TTS 和 STT 的模型评估中。针对这一痛点，ElevenLabs 的核心研究正双管齐下： 1. **声音特征与特定说话人检测**：不仅针对特定口音进行适应，还能在嘈杂环境（如人声鼎沸的房间）中结合说话人日志分离（Speaker Diarization）与降噪技术，精准锁定目标发言人。 2. **关键词先验检测（Keyword Detection）**：借鉴信息论思路，模型可以根据上下文场景预测可能出现的词汇表（例如咖啡店点单时预先加载品类词，或播客转录预先加载往期常驻词汇），将识别结果与最高先验概率的语义空间相匹配。 ```mermaid flowchart TD accTitle: 语音转录优化路径 accDescr: 对比通用语音识别与个性化关键词及说话人增强识别流程。 subgraph Generic["传统全局转录流程"] A["混合背景音频"] --> B["全局通用 STT 模型"] B --> C["基础文本输出（易受口音/杂音干扰）"] end subgraph Advanced["ElevenLabs 增强方案"] D["嘈杂环境音频输入"] --> E["说话人日志分离 & 降噪"] E --> F["特定说话人微调模型"] G["场景关键词先验库"] --> F F --> H["高精准/超人水平转写文本"] end ``` Mati 透露，通过向模型喂入一小时的目标人声音频实现“特定说话人转录（Person-specific transcription）”并不是不可逾越的科研难题，ElevenLabs 计划在随后的版本（预计近一两个月内）正式推向市场。这项技术对专业场景至关重要： - **医疗手术室**：主刀医生在复杂的器械撞击与背景杂音中下达指令，系统必须精准剥离并执行这唯一关键人的声音； - **智能家居中控**：家庭成员跑动喧哗时，电视或家庭硬件能准确只听取主人的控制口令。 ### [25:31](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1531s) 可控语音生成与情绪感知表达（Expressive Mode） 在语音生成（TTS）端，John 类比了 Zoom 的“视频修饰外观”（Touch up my appearance）美颜滤镜，提出语音领域是否也应具备“去口音滤镜”或“去含糊念白滤镜”（De-mumbling filter），以提高吐字清晰度并调节播讲语速。 Mati 介绍了 ElevenLabs 发布的 V3 模型在这方面的突破性进展。在去年年底之前，传统的文本转语音主要依赖模型自行揣摩并生成最具表现力的音频；虽然情感丰富，但用户缺乏对细节表演的直接掌控。V3 模型首次确立了**精细化语音可控性（Controllability）**： - 用户可以向模型输入明确的表演提示词（Prompts / Cues），例如“语速放慢”、“在此处加入戏剧性停顿”等； - 这一突破得益于长期的底层架构革新与海量数据工程：研究团队对海量音频的“发音内容”与“发音方式/情绪细节”进行了精准标注训练； - 催生了全新的 **Expressive Mode（情绪感知模式）**：语音 Agent 能够实时感知对端人类的情绪状态。如果检测到用户处于高度焦虑或压力状态，上层大语言模型（LLM）不仅会组织安抚性的回复逻辑，底层的语音生成也会同步切换为温暖、镇定的语气来安抚对方。 ### [27:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1661s) 级联架构与端到端语音大模型的权衡抉择 在探讨实时语音交互的技术底座时，Mati 深入拆解了业内两大技术路线的博弈：**级联模式（Cascaded Approach）** 与 **端到端原生语音模式（Speech-to-Speech / S2S）**。 ```mermaid flowchart LR accTitle: 级联架构与原生语音模型架构对比 accDescr: 展示级联架构的三段式流程与端到端语音模型的直接映射流程。 subgraph Cascaded["级联方案（Cascaded）"] direction TB S1["语音输入"] --> STT["语音转文字 (STT)"] STT --> LLM["文本推理 (LLM + 业务系统)"] LLM --> TTS["文字转语音 (TTS)"] TTS --> S2["高可靠语音输出"] end subgraph S2S["端到端原生语音（Speech-to-Speech）"] direction TB E1["语音输入"] --> E2["原生音频大模型 (S2S)"] E2 --> E3["低延迟语音输出"] end ``` 二者的核心区别与优劣对比如下： | 评估维度 | 级联架构（Cascaded Pipeline） | 端到端语音模型（Speech-to-Speech） | |:--- |:--- |:--- | | **管道构成** | `语音转文字 (STT) → 文本大模型 (LLM) → 文字转语音 (TTS)` | `语音输入 → 原生多模态模型 → 语音输出` | | **中间表征** | 强制经过自然语言文本层编码与解码 | 无中间文本层，直接进行音频特征映射 | | **端到端延迟** | 步骤较多，需要各环节极速串联与流水线优化 | 极低（跳过转录和分步文本推理开销） | | **可观测性与控制** | **极高**：每一步骤透明可见，便于审计排错与介入 | **黑盒状态**：缺乏中间各阶段的可视化与控制点 | | **企业系统集成** | **极佳**：便于调用外置数据库、业务逻辑及工具（Tool Calling） | **较难**：业务抽象与多工具编排复杂度高 | | **推理能力与智力** | **强大**：可无缝接入高参数量的前沿商业与开源推理 LLM | **较弱**：为保障极低延迟，模型参数量通常显著缩水 | | **典型适用场景** | 企业客服、严肃医疗场景、复杂业务办理 | 虚拟情感陪伴（Companion）、幻觉容忍度高的娱乐对话 | Mati 表示，ElevenLabs 目前在级联架构上进行了极高强度的研发押注。企业客户高度依赖系统的确定性与透明度，不仅要求各环节准确无误，还要求能任意替换底层的大模型与业务中台。 至于原生端到端（Speech-to-Speech）模型，Mati 坦言受制于推理速度需求，其底层采用的往往是小尺寸模型，因此在纯逻辑推理上“明显要笨拙得多”（Definitely dumber）。关于文字是否重塑了人类乃至大模型思考方式的哲学探讨，Mati 认为未来的终极形态可能是两者的混合体：利用轻量级的 Speech-to-Speech 处理低复杂度的日常闲聊打招呼，遇到高认知负荷或需要调用外部工具的任务时，则优雅回退至级联架构进行高精度推理。 ### [30:24](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1824s) 语音交互重塑潜客转化：从表单填写到多语言开放式对话 在谈到用户与企业的互动方式时，Mati Staniszewski 指出，只要将交互界面切换为语音，用户的行为习惯和转化数据就会发生根本性变化。以 ElevenLabs 自身网站的“意向登记”（Lead Capture）为例，传统做法是让访问者在网页端逐项填写冗长的表单，而团队对其进行了升级：用户可以直接与网页端的语音 Agent 对话来登记信息。这一改变带来了意想不到的双重收益：首先，用户通过口头交流填写表单的意愿远高于手动打字，整个填表流程变得更加轻松流畅；其次，用户在语音交谈中展现出了前所未有的开放性，他们不再局限于表单预设的条条框框，而是像玩开放式冒险游戏一样，巨细靡遗地讲述自己的业务需求、复杂的应用场景以及潜在痛点。 语音交互具有天然的双向追问与澄清机制，AI Agent 可以针对用户的回答进行深入挖掘，而用户在这个过程中感到放松且对系统充满信任。更为关键的是，这种语音交互天然打破了语言界限，能够无缝支持全球所有主流语言。一时间，来自世界各地的商业销售线索纷至沓来。目前，这一模式已经被众多行业迅速采纳，包括银行金融机构和大型汽车制造商在内的客户，都在利用 ElevenLabs 的技术构建他们自己的语音 SDR（销售开发代表）智能体，以取代繁琐的在线文本表单。 ```mermaid flowchart LR accTitle: 静态表单与语音智能体获客模式对比 accDescr: 对比传统在线表单与基于 ElevenLabs 语音 Agent 的多语言开放式交互流程 subgraph Legacy["传统在线表单模式"] A["访问者打字填表"] --> B["预设字段死板限制"] B --> C["填写繁琐且流失率高"] end subgraph VoiceAgent["语音 Agent 互动模式"] D["语音口头交流"] --> E["开放式对话与追问澄清"] E --> F["多语言无缝转化与丰富商机捕获"] end ``` ### [31:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1910s) 语音无界与次生效应：从波兰粗劣配音到重塑失语者的“声音尊严” 主持人谈及优质语音技术带来的“次生效应”（Second-order effects），并提起 Mati 过去常聊到的童年回忆：在波兰长大时，引进的海外影视剧受制于预算，往往采用单人旁白式配音（Lektor）。不管是荧幕上的俊男靓女还是男女对戏，甚至连深情表白“我爱你”、“我也爱你”，通篇都是同一位中年配音演员用波澜不惊的单调声音读完全程。AI 语音技术的爆发并不会抢走优质配音演员的工作，反而能彻底消灭这种劣质配音，让每种语言都能享有富有情绪张力的原声演绎。但除了影视翻译，Mati 认为语音技术的次生效应在跨语言交流与生命健康领域释放了更加震撼人心的能量。 在跨语言场景中，语音模型正在将《银河系漫游指南》中能够实现全宇宙无障碍沟通的“巴别鱼”（Babel fish）变成现实。未来人们无论身处何国，都能直接讲母语，并被当地人实时以母语理解。而在人类生活深处，AI 语音正在带来极具人文温度的变革——为失语者重获声音。无论是因为渐冻症（ALS）还是喉癌而失去发声能力的患者，如今都能借助个性化克隆找回原本的声音。ElevenLabs 最近与脑机接口公司 Neuralink 的植入患者展开合作，让该患者能够重新用自己熟悉的声音与家人谈话；团队还曾帮助过一位在结婚前不幸失声的女性重建声音，让她在多年后得以补办婚礼仪式，与丈夫在现场共同亲口念出当年的结婚誓词。Mati 动情地表示，在团队所做的所有技术工作中，这种帮助失语者找回尊严与亲情的时刻是最重要、最动人的。 - **波兰单人旁白（Lektor）现象**：低成本导致一名演员包揽全片男女老少的所有配音，缺乏情感起伏；AI 语音能低成本普及多角色高质量配音。 - **现实版“巴别鱼”**：突破语言隔阂，实现跨国跨语言的原声自然交互。 - **医疗与无障碍应用**：结合脑机接口（Neuralink）和声音克隆技术，为渐冻症患者、喉癌患者及重度言语障碍者还原其原有声线。 ### [34:03](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2043s) 什么是语音智能体？：从爱尔兰酒吧“健力士指数”到自主助理生态 面对主持人关于“语音智能体（Voice Agent）究竟该如何定义”的提问，Mati 给出了清晰的架构画像：语音智能体绝不仅仅是简单的应答机器人，而是一个具备长期运行能力、能以声音作为交互层与真实世界主动或被动交互的持久代理系统。它既包括被动响应型的客户支持系统，也涵盖能主动替用户打电话预订餐厅座位的全自主助理。 在社区生态中，开发者们已经利用 ElevenLabs 构建出令人惊叹的自主 Agent 用例。最具代表性的是爱尔兰开发者打造的“健力士指数”（Guinndex）项目：该系统通过语音 Agent 主动给全爱尔兰的酒吧拨打电话，逐一询问并核实每一家酒吧的一品脱吉尼斯黑啤价格，同时它也能被动接听来自 3,000 多个不同实体的回电与价格上报，完全通过自动化语音完成了覆盖全国的啤酒物价调研。此外，当被问及当前火热的自主智能体框架（如 OpenClaw / 类似智能体生态）与 ElevenLabs 的配合时，Mati 确认 ElevenLabs 已经成为各类主流智能体框架首选的顶尖语音交互层扩展插件，越来越多的开发者正在将自主推理工具与超低延迟语音深度缝合。 ### [35:30](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2130s) 语音模型的商业与算力经济学：更小参数、按量计费与早期补贴策略 谈到语音赛道的底层商业模式与大语言模型（LLM）的差异，Mati 指出，语音模型在算力消耗上与万亿级文本或多模态视频模型截然不同。语音前沿模型的参数量显著更小，通常维持在数十亿（A few billion）至数百亿（Tens of billions）参数区间，远低于前沿 LLM 数千亿甚至万亿级的规模。然而，为了保持研究领先地位并吸引全球最顶尖的音频研究人才，研发与算力资本支出（CapEx）依然非常庞大。这也促成了 ElevenLabs 近期以高达 110 亿美元的估值成功完成 5 亿美元的大规模融资。 在具体的商业化计费机制上，语音商业模式主要与实际推理消耗挂钩： - **文本转语音（TTS）与文字输入类产品**：主要按处理的文本 Token 数量计费。 - **语音智能体（Voice Agent）与语音转写（Transcription）**：主要按照实际通话/处理时长（按分钟）计费，这部分正在成为营收中增长最快的一块业务。 - **企业级合作模式**：大型政企客户通常采用按年签署的预付费合约（Annual agreement），承诺使用量越大，获得的单价折扣也越高。 在产品定价策略上，ElevenLabs 采取了一种不同寻常的“反直觉”打法：通常行业内最新、最强的模型定价最高，但 ElevenLabs 往往在新模型推出初期以成本价甚至倒贴补贴的方式提供给客户。Mati 解释称，新模型虽然体量更大、运行成本更高且早期稳定性可能稍欠火候，但其音质与生成上限极高。为了打破外界对语音技术应用边界的认知，公司必须以极具吸引力的价格迅速铺开分发网络，让全球开发者探索其极限潜能，同时借助真实世界的海量反馈来指导模型的迭代与优化。 ### [39:10](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2350s) 级联与端到端架构的规模分水岭：语音模型的参数演化上限 针对“语音模型的参数量是否会无限膨胀”的技术本质问题，Mati 从架构路径出发进行了深度拆解。对于有声读物朗读等垂直场景，纯语音生成模型的规模上限已经逐渐清晰，一味堆叠参数并不能带来可感知的音质提升。真正决定模型规模演化方向的，是系统采用“级联方案”（Cascaded Approach）还是“端到端融合方案”（Fused / Speech-to-speech Approach）。 在传统的级联方案中（即语音识别 ASR、大语言模型 LLM 与语音合成 TTS 相互解耦组合），底层的语音专属模型必须优先保证极高推理速度与确定性可靠性，因此纯语音模型的参数规模将长期维持在几十亿量级的小体量；而在 LLM 与音频表示完全原生融合的端到端（Fused）架构中，由于大模型需要同时处理语义理解、常识推理与端到端音频生成，其参数量才会真正攀升到数百亿乃至数千亿的体量。Mati 强调，当前正是语音 AI 最激动人心的发展阶段：研究端不断攻克尚未被完全解决的音频生成难题，产品端则将技术突破快速管道化并推向市场。在面向大型企业客户时，ElevenLabs 早已脱离了单纯贩卖 SaaS 软件的传统供应商角色，而是作为战略合作伙伴，针对中小企业（SMB）与巨头企业截然不同的技术基建需求，定制深度融合的落地解决方案。 ### [40:52](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2452s) 对话型智能体的落地与业务全流程覆盖 Mati 谈到，ElevenLabs 战略上的重大转变是调配资源，与企业肩并肩协作，推动那些过去根本不可能实现的全新业务场景落地，将语音智能体（Voice Agents）真正带入生产环境。目前公司的核心重点，就是将这些对话型智能体推向全球各行各业。 当被问及这些智能体是否主要面向客服场景时，Mati 解释道，客服确实是目前最成熟、最容易落地的切入口，但 ElevenLabs 对自身定位的野心远不止于此。他们希望成为企业与终端客户或受众之间全生命周期交互的合作伙伴： - **客户支持（Support）**：这是目前技术准备度最高、最直接的应用场景； - **主动销售（Sales）**：包括主动回访外呼以及 AI SDR（销售拓展代表）等角色； - **市场营销（Marketing）**：超越单纯的对话型智能体，帮助企业构建和生成更具感染力的营销活动。 主持人提到，之前采访过的 Intercom 联合创始人 Des Traynor 也曾聊到类似趋势——企业最初引入 AI 智能体（如 Fin）可能只是为了处理客户提问，但逐渐它演变成为了整个网站的通用 UI 交互入口。由此引发了一个关键疑问：企业未来是否需要分别维护一个文本智能体和一个语音智能体？像 Fin、Sierra 已经在文本领域取得巨大成功，ElevenLabs 是否也会去做文本？ Mati 明确表示，ElevenLabs 的核心价值在于那些“绝大部分或很大一部分交互天然通过语音发生”的交叉场景。如果能够成功解决复杂的语音智能体，文本问题在逻辑上便迎刃而解。ElevenLabs 投入大量精力进行优化的是：针对不同的客户交互匹配最合适的声音特质，并在复杂交互中实现精准编排（例如在需要查阅深层文档时优雅停顿）。他们的终极目标是覆盖业务的全链路交互，但会守住能力边界——对于涉及多轮深度金融分析、超高复杂动作链（Complex actions）的高强度纯推理型场景，并不是 ElevenLabs 的主攻方向。 ```mermaid flowchart TD accTitle: ElevenLabs 智能体业务交互范围 accDescr: 展示 ElevenLabs 对话型智能体所覆盖的业务场景与能力边界 subgraph Scope["ElevenLabs 核心覆盖场景"] A["市场营销 (Marketing)<br/>活动生成 / 内容触达"] B["主动销售 (Sales)<br/>AI SDR / 外呼获客"] C["客户支持 (Support)<br/>语音/文本全链路接入"] end subgraph CoreTech["底层能力支持"] D["声音匹配与风格调优"] E["知识库与系统集成"] F["对话流编排与平滑停顿"] end subgraph OutOfScope["明确不优化的边界"] G["复杂金融模型分析"] H["深层多阶段高负载推理任务"] end Scope --> CoreTech CoreTech -.->|边界外| OutOfScope ``` --- ### [44:08](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2648s) 爆炸式收入增长：ARR 突破 4.5 亿美元与扩张飞轮 在讨论公司业务规模时，主持人惊叹于 ElevenLabs 作为近年增长最快的创业公司之一的迅猛势头。Mati 透露，公司截至 2025 年底公开的 ARR（年度经常性收入）为 3.5 亿美元。而最近一个季度迎来了企业级客户的大爆发，单季度新增净 ARR（Net New ARR）竟然突破了 1 亿美元大关——这直接将 ElevenLabs 的实际 ARR 推高至 4.5 亿美元左右，相比前一年同期实现了数倍的惊人飞跃。 驱动如此强劲增长的核心动力在于： - **企业级大客户背书与网络效应**：近期深度合作的客户涵盖 Deutsche Telekom（德国电信）、T-Mobile、Revolut、Klarna、Meta 以及 IBM 等行业巨头； - **底层技术跨过工业可用门槛**：过去一年到一年半的时间里，驱动智能体交互的语音 AI 技术在保持高质量的同时，稳定性和可靠性得到了质的飞跃； - **典型的“着陆与扩张”（Land and Expand）飞轮**：目前超过 50% 的收入来自销售主导的企业级业务（Sales-led enterprise）。在获客初期，ElevenLabs 甚至不惜以极具吸引力的经济模型让客户低门槛试用，因为他们深信技术的价值必将通过使用量的飙升带来回报。 Mati 详细分享了与德国电信的合作范例：双方最开始仅仅是在市场部合作，利用 Magenta 品牌声音生成播客内容；随后合作拓展到了客户支持（Support）部门；最终进一步升级为贯穿整个电信网络底层架构的全局智能体，使用户拨打电话接入网络时即可直接由智能体承接交互。 伴随着业务规模的扩张，ElevenLabs 目前的团队规模已达 470 人。为了保持敏捷与战斗力，公司坚持推行“小团队作战”文化：每个产品研发、基础研究团队或垂直行业拓展小组的人数都严格控制在 10 人以内，深入钻研特定市场与行业需求并独立快速推进。 - **ARR 关键数据**：2025 年底达 3.5 亿美元，单季度新增 1 亿美元净 ARR，整体规模快速冲破 4.5 亿美元大关； - **业务结构**：销售驱动的企业级业务占比已超过 50%； - **团队架构原则**：全公司 470 人，产品研发与 Go-To-Market 均拆分为 10 人以下的小团队运作。 --- ### [47:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2870s) 双轮驱动战略：坚持自助服务与现场工程支持 面对市场上许多竞品将技术锁在“联系销售（Contact Sales）”表单之后、迫使客户必须经过 SDR 和客户经理层层筛选的传统 B2B 做法，ElevenLabs 坚定地选择了“自助服务（PLG/Self-serve）”与“现场工程支持（Forward Deployed Engineering）”两手抓的全面光谱模式。 主持人以 Stripe 依靠开放注册、消除使用门槛实现爆炸增长的经历产生强烈共鸣，询问 Mati 坚持做自助模式的初衷。Mati 总结了三大核心原因： 1. **极致敏捷的反馈闭环**：产品直接接受大众检验，研发团队能第一时间获知模型的优劣与瓶颈； 2. **对核心技术拥有绝对自信**：Mati 强调 ElevenLabs 坚信自己在模型效果、语音智能体与工程部署上拥有行业顶尖水准，"我们为自己的技术背书"，因此毫不畏惧让任何人直接上手体验； 3. **生态反哺与预演未来**：面向顶级企业的大单往往需要极高标准的可靠性与合规性，但前沿的新技术在刚研发出来时，虽然尚不满足大企业的苛刻合规条件，却能立即激发开发者与中小企业（SMB）的无限创意。中小开发者探索出的前沿用例，恰恰在为 ElevenLabs 指明未来的产品演进轨迹。 Mati 笑着透露，即便 ElevenLabs 如今已是估值数十亿美元的独角兽，最初公司只有两个创始人的时候，也是完全通过自助服务（Self-serve）在 Stripe 上完成支付接入的。从第一性原理出发，任何繁琐的销售流程和隐藏的计费步骤，都会大幅削弱用户对产品的信任与尝试意愿。 --- ### [49:55](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2995s) 突发官宣：从订阅制走向全量按用量付费（Pay-as-you-go） 在被问及对 Stripe 有什么改进建议时，Mati 爆出了一个幽默的幕后花絮：长久以来，他最常向 Stripe 提出的反馈就是“为什么不提供更好的基于用量的计费（Usage-based billing / Pay-as-you-go）能力？”就在几天前，ElevenLabs 的财务负责人 Maciej 还和 Stripe 团队就此深入交流，甚至半开玩笑地建议 Stripe：“你们应该把 Metronome（知名用量计费平台）给买了。”结果就在第二天，Stripe 果然宣布了对 Metronome 的收购。 借由用量计费的话题，Mati 在播客现场正式宣布了一项重大产品策略调整：**ElevenLabs 即将向所有用户全面推出按用量付费（Pay-as-you-go）体验**。 在过去，按实际用量计费是大型企业客户的专属特权，而所有自助开发者和个人用户只能选择固定额度的月度订阅套餐（Subscription plans），一旦超出额度体验往往受阻。随着全量 Pay-as-you-go 的上线，任何开发者都可以无需背负沉重的包月负担，直接按需调用背后的 Voice Engine 编排闭环和各类底层模型。主持人对此深表赞同，指出这正是现代 AI 产品进化的必然趋势——如同很多人在用 Claude 等大模型时，常常在深度交互的正起劲时突然遭遇包月额度的速率限制（Rate Limit），纯订阅制体验极易让高阶开发者感到挫败，而结合灵活的按量付费才是释放生产力的最佳方案。 ### [51:13](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3073s) AI 产品的商业模式与企业架构重构 对话从 AI 产品的定价痛点切入。用户经常遭遇“对不起，您已达到使用上限”的弹窗阻断，而大家真正希望的是能像使用 Claude Code 那样直接按 API 调用量付费，甚至戏称“作为消费者，有钱想多花却花不出去实在太滑稽了”。AI 产品理想的商业模式应当是提供一种包含额度上限的“半自助餐式”订阅包（most of what you can eat subscription with limits），同时允许用户无缝购买超额用量（overage charges）。 随后，讨论深入到当今所有大型企业 CEO 共同面临的核心挑战：AI 的突破究竟如何重构组织形态？一家 AI 原生（AI-native）公司到底该如何设计？关于组织架构存在诸多分歧：究竟该招募更能熟练调动 AI 的资深员工来替代初级员工的工作，还是应该招募从诞生起就具备 AI 直觉的年轻一代？团队究竟该变大还是变小？传统的财务团队很少自建软件，现在是否也该深度使用 Claude？ElevenLabs 成立于 2022 年（彼时市场热点还是加密货币与元宇宙），恰逢这一轮生成式 AI 浪潮爆发并迅速成长为大规模企业，它在没有数十年历史包袱（baggage）的情况下，为业界提供了宝贵的新型组织蓝图。 ### [53:07](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3187s) 扁平化架构与在非技术团队中配备“技术领头人” ElevenLabs 的内部实践遵循极其极致的组织哲学：保持极小且极其扁平的团队。Mati 透露，他和联合创始人 Piotr 每个人直接汇报（direct reports）的下属都超过 15 人，而这些下属管理的人数也保持着类似的跨度。传统企业的管理跨度（span of control）通常只有 8 人左右，而 ElevenLabs 直接翻倍，每个业务执行单元（work item）的团队规模基本保持在 10 人左右。面对主持人的调侃——“初创公司创始人写博客谈管理理论总带着一种盲目的自信”，Mati 幽默地承认确实可能有一点初创公司的特殊滤镜，真正的成效应放在未来 5 到 10 年检验，但目前高度扁平的小团队模式确实运转良好。 更为关键的组织范式是“全员技术化”，或者至少在非技术业务部门中嵌入专属技术资源。无论是在运营（Ops）还是人才招聘（Talent）团队，ElevenLabs 都会为该团队配备一名专属的技术主管（tech lead），专门负责设计自动化流程，并带动整个团队的能力升级： ```mermaid flowchart TD accTitle: ElevenLabs 组织赋能架构 accDescr: 展示 ElevenLabs 扁平化管理结构以及在非技术团队中嵌入技术资源（Tech Lead）的协作模式。 subgraph Management ["扁平化管理层级"] Founders["联合创始人 (管理跨度 >15人)"] Directs["核心业务负责人 (管理跨度 >15人)"] Founders --> Directs end subgraph NonTechTeams ["非技术业务团队 (约10人规模)"] Ops["运营团队 (Ops)"] Talent["招聘团队 (Talent)"] GTM["市场与销售团队 (GTM)"] end subgraph TechEnablers ["嵌入式技术支持"] TechLeadTalent["人才团队专属 Tech Lead"] TechLeadOps["运营团队专属 Tech Lead"] end Directs --> Ops Directs --> Talent Directs --> GTM TechLeadTalent -.->|"数据探索 / 自动化抓取"| Talent TechLeadOps -.->|"流程自动化 / 内部工具开发"| Ops ``` - **管理跨度翻倍**：创始人和核心高管各自拥有超过 15 名直接下属，呈现远超传统企业的扁平化结构。 - **小团队单元**：每个核心工作项目的团队规模均控制在 10 人上下。 - **嵌入式 Tech Lead**：非技术团队（如人才、运营）不单纯依赖外部采购工具，而是通过内置的技术专人开发自动化方案，消除日常冗余流程。 ### [55:16](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3316s) AI 杠杆在招聘与 GTM 中的落地实践 在具体业务执行层面，ElevenLabs 并没有彻底抛弃传统服务商（如 Workday、Greenhouse 等），而是将重点放在两件事上：一是“全面 LLM 化（LLM-ifying everything）”，让所有业务数据都变成可交互、可探索的资产；二是填补当前 AI Agent 与高级技术能力之间的断层。 在招聘流程中，团队不仅能通过数据交互分析候选人管道、过往成效及谁的背调（reference）质量最高，技术负责人还能编写针对性工具，爬取精准画像并识别出经过验证的特定优秀特征。在市场拓展（GTM）方面，AI 被用作强大的能力放大器： - 销售会前准备：系统自动识别相关案例研究并生成会议预读材料（pre-read）。 - 自动化工作流：结合 AI SDR（销售拓展代表）开展外联，并为客户自动生成填入真实定制化数据的演示文稿（deck），随后由人工负责审核与深化。 - 文化探索 Agent：ElevenLabs 甚至开发了一个专属语音智能体，让求职者可以直接与它对话，了解公司内部文化并进行面试模拟。 ### [57:00](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3420s) 乌克兰数字化政府建设与嵌入式技术模式的验证 在协助乌克兰数字化建设的过程中，ElevenLabs 的组织思路得到了强有力的外部印证。战火中的乌克兰面临严峻的基础设施挑战：处于战区的人们无法依赖常规电话线或线下实体办事处，极度渴望即时、精准的政务与民生支持。乌克兰政府全力推进国家级数字化政务系统“Diia”，致力于打造全球首个“Agent 化政府”（agentic government），覆盖退伍军人福利发放、前线动态通报、全民教育以及医疗预约服务。 Mati 亲赴基辅参与合作，协助将 ElevenLabs 的语音技术无缝整合到 Diia 平台中，使全民能够通过语音无障碍获取政务支持。在基辅期间，Mati 惊讶地发现，乌克兰政府的架构与 ElevenLabs 正在推行的模式如出一辙：每个政府部门（ministry）内部都配备了专属的技术资源来推进本领域的 Agent 化改革，随后由中央数字化转型团队负责统筹汇总，统一向公众交付服务。这种在最极端复杂环境下淬炼出的模式，进一步验证了“在每个业务团队中常驻技术力量”这一策略的前瞻性。 ### [58:38](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3518s) 高自主性（Agency）是 AI 时代的核心分水岭 对于组织招募更偏向资深员工还是初级员工的争论，Mati 明确表示，ElevenLabs 最看重的核心过滤器是文化匹配度。人员规模可以迅速扩张，但文化的传承与维系极其艰难。团队恪守的核心价值观包括：第一性原理思考（first principles）、主人翁意识（taking ownership）、追求卓越但保持谦逊（striving for excellence but staying humble）。 而在主人翁意识中，最本质的特质就是“自主性”（Agency）。无论员工处在职业生涯的哪个阶段，只要拥有主动探索的极强自主性，AI 就会成为他们工作能力的巨大放大器。主持人对此深表认同，指出“高自主性（high-agency）的人将成为 AI 进步的最大赢家，而在组织内缺乏自主性的人将彻底出局”。 Mati 总结道，在与 Piotr 共同打造 ElevenLabs 的历程中，最令他们自豪的不是某个单独的爆款产品或个人贡献，而是看到这种文化自我生长并反哺塑造了整家公司。当极致的业务水准（craft）与工作中的纯粹乐趣相结合时，便构成了 ElevenLabs 最强大的增长引擎。 ## Notable quotes > [01:46](https://www.youtube.com/watch?v=zA1zdqIYER4&t=106s) 在你们的声音模型里，‘英音’竟然是一种涌现特性？ — John > [15:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=941s) 我恨不得让迈克尔·凯恩给我朗读生活里的一切！ — John > [17:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1070s) HAL，把分离舱门打开 — 《2001太空漫游》 > [27:41](https://www.youtube.com/watch?v=zA1zdqIYER4&t=1661s) 明显要笨拙得多 — Mati Staniszewski > [47:50](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2870s) 我们为自己的技术背书 — Mati Staniszewski > [49:55](https://www.youtube.com/watch?v=zA1zdqIYER4&t=2995s) 你们应该把 Metronome 给买了。 — Mati Staniszewski > [51:13](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3073s) 作为消费者，有钱想多花却花不出去实在太滑稽了 — John > [53:07](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3187s) 初创公司创始人写博客谈管理理论总带着一种盲目的自信 — John > [58:38](https://www.youtube.com/watch?v=zA1zdqIYER4&t=3518s) 高自主性（high-agency）的人将成为 AI 进步的最大赢家，而在组织内缺乏自主性的人将彻底出局 — John ## Mentioned Mati Staniszewski, Piotr, John, Andrej Karpathy, Wolfgang von Kempelen, Sir Michael Caine (迈克尔·凯恩爵士), Richard Feynman (理查德·费曼), Des Traynor, Maciej, ElevenLabs, Stripe, Bell Labs (贝尔实验室), Audible, Apple, Google (Gemini), Anthropic (Claude), Neuralink, Intercom, Sierra, Deutsche Telekom (德国电信), T-Mobile, Revolut, Klarna, Meta, IBM, Metronome, Workday, Greenhouse, ElevenReader, Link by Stripe, Diia (乌克兰数字化政务系统), 《2001太空漫游》, 《银河系漫游指南》, Guinndex (健力士指数), OpenClaw, Tacotron, WaveNet, Claude Code ## Video description Mati Staniszewski is the co-founder of ElevenLabs, the research company making audio accessible across languages and voices. He sits down with John to discuss the "voice Turing Test" and why AI has conquered text but still struggles with conversational speech. They discuss the future of human-computer interaction, including why we still can't get our phones to read a PDF properly and the massive potential for voice agents in everything from farming to healthcare. Mati also opens up about ElevenLabs’ rapid ascent to an $11 billion valuation and gives a behind-the-scenes look at how Ukraine is using their tech for digital government services. Full transcript on Substack: https://open.substack.com/pub/cheekypint/p/the-world-of-voice-ai-with-mati-staniszewski Subscribe to Cheeky Pint Spotify: https://open.spotify.com/show/2IHbGJJ... Apple Podcasts: https://podcasts.apple.com/us/podcast... Key moments 00:00:27 How audio models work 00:08:52 ElevenLabs business model 00:17:50 The conversational Turing Test 00:21:01 Link by Stripe 00:26:02 Cascaded vs speech-to-speech 00:31:53 Universal translation 00:51:41 Designing an AI-native org [Source](https://www.youtube.com/watch?v=zA1zdqIYER4)

[1:46] John: 在你们的声音模型里，‘英音’竟然是一种涌现特性？ [15:41] John: 我恨不得让迈克尔·凯恩给我朗读生活里的一切！ [17:50] 《2001太空漫游》: HAL，把分离舱门打开 [27:41] Mati Staniszewski: 明显要笨拙得多 [47:50] Mati Staniszewski: 我们为自己的技术背书 [49:55] Mati Staniszewski: 你们应该把 Metronome 给买了。 [51:13] John: 作为消费者，有钱想多花却花不出去实在太滑稽了 [53:07] John: 初创公司创始人写博客谈管理理论总带着一种盲目的自信 [58:38] John: 高自主性（high-agency）的人将成为 AI 进步的最大赢家，而在组织内缺乏自主性的人将彻底出局

[0:27] Mati Staniszewski: Britishness is an emergent property in our voice models. [4:52] Mati Staniszewski: It's like a syllable deconstructed even to smaller elements. These are effectively the human sounds you can produce. [13:57] Mati Staniszewski: We work with the Sir Michael Caines of the world [25:31] Mati Staniszewski: definitely dumber [33:49] Mati Staniszewski: probably the most important from all the work that we do. [42:01] Mati Staniszewski: If you fix the voice agent, you have fixed text piece inherently as well [47:50] Mati Staniszewski: We stand behind our tech [57:00] John: That's very tech-forward by Ukraine [57:00] John: high-agency people are the winners of the advances in AI, and within organizations, low-agency people will lose out.

[0:01] Mati Staniszewski cofounded ElevenLabs in 2022, and has since scaled it [0:05] to the $11 billion leader in AI audio. [0:08] He's credited it with capturing the humanness of speech through realistic [0:11] emotional inflection, and they're now expanding into everything [0:14] from agentic workflows to music. [0:16] Cheers. [0:17] Thanks for having me. [0:21] Maybe a good place to start is, describe to me how… [0:25] I know how an LLM works at a high level. [0:28] Describe to me how an audio model works. [0:30] If we were Karpathy-style looking to build a toy one from scratch, how does it work? [0:36] In the early days, you try to replicate it exactly like you would replicate [0:39] it with the human body. [0:40] You would completely try to reproduce a machine, analogue machine, that would [0:45] create a vocal tract effectively. [0:47] Then that progressed into trying to create effectively a digital signal for speech. [0:52] Bell Labs was one of the first to try to create a structured set of signals [0:57] that would represent the speech. [0:58] That is a first precursor to what we would do today. [1:02] Then you would try to stitch in phonemes, effectively different sounds of how we [1:07] would speak as humans, and then try to concatenate them together. [1:10] It's another important part in that equation where you would... [1:14] Based on the most probabilistic approach of the next word, you would effectively [1:18] try to bring the phonemes from your library of phonemes [1:20] and bring them together. [1:22] Then down to the modern history where now we effectively do similar [1:26] neural nets in other domains. [1:29] You predict the next sound based on, of course, the context of the previous [1:35] sounds if it's a streaming speech. [1:37] If it's, let's say, a context of audio, you will use combination of predicting [1:42] of the phonemes, but you also use the contextual text element of that work. [1:46] Here, credit to my co-founder, Piotr, who effectively came with that new idea [1:51] of how you can now create voice models which are both reliable, high quality, [1:56] quick, where you would bring a lot of the ideas from transformer models, from [1:59] diffusion models, into the speech space. [2:01] That prediction of the next token in the phoneme space wasn't [2:05] something that was possible. [2:07] We spoke briefly about this, of how you can operate on the text, [2:10] on the waveform space. [2:11] There's also Mel spectrogram space. [2:13] Usually, you do text, Mel spectrogram, waveform. [2:15] Sorry, what's the spectrogram space? [2:17] It's like a visual representation of how the speech sounds across pitch, [2:21] across energy, and then you transform that into a waveform. [2:24] When WaveNet came along and Tacotron models, they would effectively use text [2:30] to Mel spectrogram, so that visual representation, [2:32] and then how you decode and encode that into the waveform to bring it across. [2:36] Piotr figured out how to abstract some of those steps and decode [2:39] and encode them a lot better. [2:42] That predicting of the next phoneme was one of the big piece. [2:46] Second big piece was, how do you bring that context into the equation? [2:49] What I mean by context is, if a voice actor was reading a [2:53] textual copy, you would know that, okay, this is a dialect sequence. [2:57] I need to produce a dialect. [2:58] If it's a happy sentence, I might need to pronounce as a happy sentence. [3:01] But what happens before and after comes into the equation, [3:05] and you need to bring that across. [3:07] Then there's a last big piece. [3:08] Voice model has the sound of how you intonate the given fragment. [3:15] But the second big part is the voice itself, the characteristics of accent, [3:20] of style, of prosody across that voice. [3:23] When you actually try to vocalize something, when you create that voice [3:27] model, you turn text into audio, you need the text. [3:29] You also the voice reference of how you want it to be spoken. [3:33] Here was the second big innovation. [3:35] Apart from context, it's how you decode and encode those features. [3:39] When Bell Labs came with their initial representation of speech, [3:43] the big piece there was you would have effectively hard-coded [3:46] parameters for that speech. [3:48] With ElevenLabs models— [3:50] Hard-coded parameters for enthusiastic speaker, British accent, [3:53] that kind of stuff? Exactly, that kind of stuff. [3:55] The set of pitch elements that you can select, set of energies, [3:58] spectrograms you can select from. [4:00] In our approach, effectively, you would give the model open-ended ability [4:06] to select what those parameters should be. [4:08] It's not going to be British, Polish, Spanish, English speaker, [4:12] but the model will deduce them themselves. [4:14] The same for other sets of parameters that are not hard-coded, [4:18] whether it's the enthusiasm, whether it's the sadness, et cetera. [4:21] You're saying Britishness is an emergent property in your voice models? [4:26] Exactly. [4:27] Those two big parts, so it's encoding and decoding of how you create the voice. [4:32] Super hard problem before, and figured out, too. [4:35] How you then construct that in a sentence, how you get the context across so you can [4:39] predict the next volumes, so how you bring them together [4:41] in a reliable and stable way while doing it quick. [4:44] These were the two first big innovations in the voice models [4:47] that continue to today. But okay, so if LLMs reason about [4:52] text and word subparts tokens as the way [4:56] they think about the world, what is the equivalent of a token in a voice model? [5:00] You mentioned phonemes a bunch. [5:02] What is that representation? [5:04] We start the voice embedding effectively for the speaker. [5:09] You need that reference when you produce and create the speech. [5:13] Of course, in the input to the voice model, you still get the text, [5:16] and you bring the speaker and coding. [5:19] Then, when you produce speech, you do operate on the waveform or [5:23] effectively on the phoneme level of that speech. [5:28] Then when we go the opposite, so of course when we— [5:31] Sorry, what is a phoneme? [5:32] Fill in my understanding. [5:33] It's like a syllable deconstructed even to smaller elements. [5:37] These are effectively the human sounds you can produce. [5:40] These would be the most close to that representation. [5:44] But of course, in our models, now it's going to be a combination of not only [5:49] operating on phoneme level, you also operate on the text level. [5:51] You operate both in sync because when you are [5:55] predicting the context, you need to understand how that sentence [5:58] will get constructed, and especially if it's more of a streaming [6:01] real-time use case in a voice agent setting, you need both [6:04] parts to work across. [6:06] It's similar to how you operate on the token level on the tech side, [6:10] we operate on the token level on the audio side. [6:13] It feels like a big part of the magic of Eleven was your voices [6:17] were much more human-sounding. [6:19] How did you accomplish that? [6:21] I'll give you a quick synopsis of how we think about the models [6:25] on the text-to-speech side today. [6:27] In any model, you need the architecture, you need to compute, you need data. [6:31] Architecture innovations were one thing. [6:33] The data part was the second big thing. [6:36] With audio, you will have a lot of audio data available, [6:41] but frequently you will not have it annotated in the right way. [6:45] You won't have which speaker is speaking when. [6:48] Some of the "what" is annotated, but the "how" isn't. [6:53] As we are speaking now, what's the emotions that we use? [6:55] What are the accents that we use? [6:57] We would invest a lot internally on effectively creating [7:00] our own data labellers, our own team, to be able to create those [7:04] data sets that will be better. [7:05] That was a combination of, of course, semi-automatic techniques and then [7:10] manual techniques. [7:11] Actually, a lot of the models that we did afterwards actually span out [7:15] from a lot of that research, too. [7:16] Speech-to-text model initially was a model we did for ourselves because the models [7:20] on the market just weren't good to annotate that data. [7:23] Then another brilliant research area on our team was being able to construct it [7:27] so we could span it out as a model that we brought to the customer. [7:30] You've just been doing useful stuff in voice, and that has [7:34] emerged with a whole bunch of products that you might not have expected because [7:37] you find you're building useful stuff. Exactly. [7:39] That's a combination of data, of being able to do it automatically, [7:43] create a team that's coached on voice, on how to describe it, [7:47] because most of the labellers out there just aren't [7:51] as well-versed on understanding the audio and voice, helped us [7:55] a lot to bring that back. [7:56] Then, of course, deploying those models in production, seeing how customers [7:59] interact with them, having them annotate all of the data [8:02] helped us refine those models over time. [8:04] A very interesting thing on the side. [8:06] We spoke about the speech representation. [8:08] The first guy who created the speech representation is a guy called [8:11] Kempelen, Wolfgang von Kempelen. [8:13] He created those analogue machine that would represent effectively a human [8:18] vocal tract and try to produce that sound. [8:21] He had to spend decades on that, and that started producing valves. [8:24] But that was the same person that created a chess machine, the first viral, [8:30] let's say, chess machine that would simulate playing chess. [8:35] Is this the Mechanical Turk? [8:36] It was called Turk. [8:39] But the crazy thing behind it, it was operated by a human, [8:42] and it was all a fluke. [8:44] That's where the Mechanical Turk came from, which actually we use in our data [8:48] labelling production to make that work there. [8:52] Sorry, we jumped right in. [8:54] But if you describe the Eleven business today, people think of you [8:57] as the speech company. [9:00] How should they actually think of your business to the extent you [9:02] can describe the big areas? [9:03] Text-to-speech, speech-to-text, voice agents. [9:06] Just break down the business for us. [9:10] In a nutshell, I'll describe ElevenLabs as a research and product deployment company. [9:15] We build foundational audio and voice models, and then build a platform [9:20] for businesses to transform how they communicate with their customers, [9:26] with their employees. [9:28] That will apply through AI agents from customer support, sales, [9:33] hiring, training, all the way through to marketing and storytelling [9:37] for our creative tools. [9:39] In that set, we've created all types of foundational audio models. [9:43] Text-to-speech models for producing speech, [9:45] speech-to-text models that work over 100 languages and happily beat others [9:49] on benchmarks, all the way through to conversational [9:53] models of how you loop them together to music, to other domains of audio. [9:58] Then, of course, beyond the models, when you actually bring them [10:01] to production, that's where the second level [10:03] of the platform comes in, where that meets the businesses on the specific use case. [10:07] On the agent-specific example, it would be how you now connect those [10:10] models to the knowledge base, to telephony, to the integrations that you [10:15] need to perform the actions, how you evaluate and monitor the agent [10:19] that it behaves in the right way, how you build the right safeguards. [10:21] On the creative side, on the marketing side, [10:24] it's how do you create a good ad so you can create a good video voiceover [10:31] for one of the campaigns. [10:32] How you create an article that's narrated with a specific voice that represents [10:37] the brand in a good way. [10:38] That's where we combine the models and understanding of the customers we [10:42] work with into one policy platform. [10:45] Every platform company has this question about how far they go into applications. [10:49] How do you think about where you go horizontal and power [10:52] the whole ecosystem versus where you develop application? [10:55] Because you can imagine there being a whole ecosystem of [10:59] closed captioning tools that grow up that, again, are built on the ElevenLabs tech. [11:03] It's not necessarily a space that you would have to go after yourself. [11:06] I think the big difference between... [11:07] In your question, today, we see ourselves as a platform where if [11:13] you're building a horizontal use case in your business, a great place to come. [11:17] If you have a lot of domain specificity, that's where I see a lot [11:21] of the application companies forming over time, where that's specifically [11:26] not the spaces we will go into. [11:28] I think it also is interesting when the tech is moving as quickly as it is here. [11:33] It's one thing with SaaS where you get these vertical-specific providers, [11:38] but I would imagine one of the biggest risks for you guys in being intermediated [11:42] is if there is, like in this example, a closed captioning service that is [11:48] on a two-versions-old version of ElevenLabs and hasn't upgraded. [11:52] That's a problem because you want people to be using the latest and greatest model [11:55] that you've developed, and you'll be deploying new capabilities every week. [11:58] I presume that's part of your thinking, is that just when it's moving [12:01] that quickly, you need to go direct in a lot of cases. [12:04] That's right. [12:05] In the closed captioning, here already now, [12:08] we know that our services is going to be able to tackle 99.9% [12:13] of the cases that customers have. [12:15] Then there's added benefit of we work with healthcare customers where we will [12:20] create a custom model for those customers where we'll get that [12:23] transcription perfectly. [12:24] The context is the tricky thing in closed captions, where we talk a lot about [12:28] a lot of technical stuff on this. Yeah, for sure. [12:31] That's where you need effectively a dictionary of where you do the tech [12:34] beforehand, which, as we work with the businesses, we know [12:39] we need to embed in that creation process. [12:42] We're talking a bit about product here, and one thing I note is [12:46] that LLMs are amazing, and you have the usage stats of ChatGPT [12:51] and Gemini and all the popular LLMs where they're working and people use them a ton. [12:56] It feels like there's a big product overhang when it comes to voice, [13:01] where the leading-edge voice models are incredibly capable. [13:04] Yet, I was driving home the other day, and I needed to read a PDF, but I was driving. [13:09] I said, "Okay, I'll just have my phone read the PDF to me." You can try and hack [13:13] it with iOS screen reader, but it doesn't really work with the scrolling. [13:17] Then in theory, you can upload it to Gemini, [13:19] but you're trying to get it to not summarize it, and it actually just hung [13:22] when I tried to press the Read This to Me button. [13:25] There was no way I could get my phone to read me something, [13:29] which seemed like a fairly basic feature. [13:32] All cars advertise voice control, and yet, it sucks. [13:36] Separately, if you want to input something to the navigation, just no car [13:40] has a good version of that yet. [13:41] Maybe Tesla does, and, and, and... [13:44] Why does it seem like with LLMs and Claude Code and everything, [13:50] we are using all the capabilities of the intelligence, whereas with voice, [13:54] we're living 10 years ago somehow? [13:57] Well, I'm thinking would I agree with the premise that we are 10 years behind. [14:01] In the lived experience of people day to day, they're using Siri's [14:04] transcription, which has gotten better, and it's still way [14:06] behind the leading edge. [14:09] There's definitely a piece of… [14:13] I think the technology in many of those cases already, there's a deployment [14:17] gap to what you are saying. [14:18] It's like an automative for some of the big companies are not adopting [14:21] that quickly enough or bringing that into the production. [14:23] But plenty of different problems that you need to fix along the way. [14:26] The quality of voice models for them to actually sound good, [14:31] this is only a last three years thing. [14:34] Yeah, but it's three years. [14:35] It's a three years thing. [14:36] Cars have over-the-air software updates now. [14:38] That's three years for the first voice model that can narrate text async. [14:43] Two years ago, we can start seeing the real-time version of that. [14:47] I think the real break was a year ago, where you could start [14:50] seeing that in production. [14:51] Then I think over 2025, the big piece that hasn't been possible is [14:56] how you connect now the real-time voice interaction with something [14:59] which I think you're referring to. [15:00] It has context of what you want to do, what is the material that you want [15:03] to read, how does it connect to set off your preferences [15:07] from the past and get that across. [15:09] I think that only recently became possible, and where we've seen the big [15:13] adoption across the enterprises leading on the technical side. [15:17] I think this year, it should be in the automotive [15:20] side too, or some of the applications. [15:23] You think we'll start seeing great voice models in cars this year? [15:27] This year for their own cloud use cases. [15:30] On car, in car, so without connectivity, not yet. [15:34] There's deployment, of course, gap, of how you bring that into the gaps. [15:38] But I think the next two years, three years. [15:41] How about the PDF reading use case? [15:43] That should work. Yeah. [15:45] But how should I have done it? [15:48] Back in the day... [15:50] I'll pre-empt this with a story to cue ElevenReader, but we had this problem. [15:56] We had so many audiobook authors come into ElevenLabs. [15:58] 2023, we released the first software. [16:00] We had a lot of creators and then a lot of audiobook authors or book authors [16:04] that couldn't afford professional narration and wanted [16:08] to create an audiobook. [16:08] However, none of the companies accepted AI audiobooks. [16:12] You can't sell an AI audiobook on Audible or something? [16:15] Exactly. Audible would block AI content. [16:17] We had no choice. We need to create an avenue for them to— [16:21] Ah, because there was no distribution for AI audiobooks. [16:24] Exactly. [16:25] We created ElevenReader, and that came with a functionality where [16:27] you can upload your PDF, you can upload your text, [16:30] and have it read out loud with a number of incredible voices, [16:33] whether it's Sir Michael Caine all the way through to estate and working together [16:39] with Sir Richard Feynman, where you can have that— [16:41] You are working with the Sir Michael Caines of the world? [16:43] Exactly. Then you can actually read it out loud. [16:45] That works extremely well. That works. [16:48] Now, how can you do it? [16:50] Gosh, I do want everything read to me by Michael Caine. [16:52] It's a great voice. [16:54] Shouldn't you guys have a consumer app where I can just do [16:57] the common voice things? [16:58] I want to be able to have an Eleven app on my phone, and then if I upload a PDF [17:03] to it, it can do the common things that I would like, such as have it read it to me. [17:06] Yeah, that's exactly ElevenReader. That works. [17:09] The phone makers allow third-party keyboards. [17:12] Do they allow third-party transcription engines? [17:16] Will they, do you think? The phone makers you said. [17:18] Like Apple and Google. [17:21] The OS makers. [17:22] Yeah, not all of them. [17:24] With Android, you can work through it. [17:26] There's variations of that, Nothing. tech, and others. [17:30] I feel like if you had a popular Eleven app that allowed for transcription, [17:33] people would use it a bunch, and maybe eventually, Apple would say, [17:36] "Oh, we should allow third-party transcription engines if [17:38] that's what people want." [17:39] It seems like they might be going in that direction. [17:41] Recently, they announced that it will open up the LLM ecosystem. [17:45] Hopefully, they will do the same with voice ecosystem, which is similar. [17:48] I think it's rational to do when it's moving so quickly. [17:50] The voice assistant paradigm is one of the oldest [17:59] UI paradigms in computing, like the, "Open the pod bay doors, HAL," from 1969. [18:04] I will claim it's not working yet. [18:06] Siri doesn't have the intelligence. [18:10] Then on Gemini and ChatGPT and those apps, [18:15] I want to use the voice mode, but I don't [18:18] know about you, it just doesn't work. [18:19] Sometimes I'll be using my phone, and I'll use the iOS keyboard [18:23] transcription to type in the field and then say a bunch of stuff, [18:29] and then send it off. [18:30] But this suggests to me that consumers really want voice mode that works, [18:35] and yet it's just not working yet for the major LLM apps or for anyone. [18:40] Why isn't it working yet? [18:41] It is pretty hard to do because you want two things. [18:45] You want to be able to say things that you want, but you want sometimes for it [18:48] to execute it, sometimes to wait for you to finish and add [18:51] something in the sentence. [18:52] Sometimes you want it to be interactive, so it asks you questions back to clarify [18:55] and get some of the additional detail. [18:57] All of that is actually pretty hard. [19:00] That's where the magical ideal version of a voice agent for us comes through, [19:05] where you need the speech-to-text element, you need the transcription side, [19:08] you need then the turn-taking mechanism. [19:12] When do you finish sentences? [19:14] When is it likely based on silence, based likely on the context? [19:18] Then sometimes you want it to speak back and clarify, or at least give you the text [19:22] back to clarify, and then maybe execute a set of instructions. [19:24] That problem is still very hard. [19:27] I agree with the claim that this orchestration side has not passed a true [19:32] conversational agent Turing test, where it behaves as you would expect [19:36] from another person, where you can say— [19:37] That's the simpler way of saying what I'm saying is that we have passed the Turing [19:40] test with text LLMs a long time ago. [19:42] We're actually nowhere near that on voice LLMs. [19:44] It's interesting how that's a final frontier. [19:47] I feel like it's going to work in specific domains. [19:51] In customer support calls, passes the voice Turing test. [19:55] It works well. [19:56] Let's take another spectrum of that. [19:58] An interactive gaming experience, a truly interactive as you would have [20:03] with another human in that game, it's so hard and further out there, [20:08] we haven't passed it yet there. [20:10] Yes, that makes sense. [20:11] But I think that's a combination of even a simpler version of within that. [20:17] Sometimes you might give a response immediately back. [20:19] Sometimes you need a tool call to get additional information from the database, [20:22] how you orchestrate that. [20:24] That's probably the most common thing we see as we work with some of the companies [20:27] out there is you want those systems to orchestrate extremely well where if [20:31] it's a conversational use case, pretty simple, you can root [20:33] the agent to speak with it. [20:34] But if you need to authenticate, if you need to pull additional information [20:38] from the database, what do you do? [20:40] How do you handle that graciously? [20:41] That's where it gets tricky. [20:44] To that extent, I would say that that's just getting there. [20:48] We'll hopefully see that. [20:50] Our goal is to pass the voice Turing test in all those cases or the Turing test [20:54] for all conversational agents outside of voice, too. [20:56] I hope we will all be there in the next year or so. [21:01] For subscription businesses, a lot of revenue is lost in that last [21:05] few seconds before the checkout. [21:07] Someone has to get up and find their wallet, or they mistyped their card [21:10] number, or they hit an error, and they just give up, and you lose the sale. [21:13] For a company like ElevenLabs, adding hundreds of thousands [21:16] of subscribers, even a tiny bit of friction like that, [21:19] it would really add up. [21:20] But that's why ElevenLabs uses Link from Stripe. [21:23] Customers save their details once, and then they can check out in seconds [21:26] across more than a million businesses with saved credentials. [21:30] If you want a faster checkout for your customers, you should [21:33] turn on Link from Stripe. [21:36] Are you guys working on personalized [21:41] voice transcription where it feels like [21:44] part of the way we're making it hard for ourselves is when I speak to Siri, [21:49] I have a bit of an accent, and so it sometimes has [21:52] a hard time understanding me. [21:54] But my accent doesn't change. [21:56] It could just get good at listening to John. [21:59] But my understanding is it's not, it's just running the global [22:02] voice recognition model. [22:04] I'm guessing it's the same for ElevenLabs where you're running the global [22:06] voice recognition model. But again, you have an accent. [22:09] If you walked up to someone in a coffee shop and said two words, [22:12] they might have a hard time understanding it because they're not putting it [22:15] through their Mati Polish accent filter. [22:17] Where's this going with actually interpreting the person that you [22:22] know to exist on the other side? [22:24] I have a very tricky one to detect. [22:26] My voice is frequently used in the tests. [22:29] You're part of the test suite. [22:30] For text-to-speech, for speech-to-text, for everything. [22:34] It's pretty— [22:34] But again, trying to parse your voice in a global model is just making life hard. [22:38] It's like, have a Mati-specific model. [22:41] On the speech-to-text, by transcription, exactly. [22:44] The big part now that we are bringing in is you have two parts. [22:48] One, effectively a person or a voice-specific [22:53] detection, which is true for the accent side, but it's also [22:56] true for a crowded room. [22:59] We have an incredible research team that's able to continually do both the accuracy [23:03] high, but also add things like speaker detection, of course, noise reduction. [23:10] But then the second part is also keyword detection. [23:13] There are specific words that you would want to say in those settings that you [23:17] want to effectively monitor for. [23:19] We spoke about, let's say I'm going to the coffee shop and order things. [23:23] The set of actions the coffee shop would expect me to do. [23:26] This is information theory. [23:27] It's like they can just listen out for the coffee words. [23:30] Exactly. [23:31] Then try to match it to the closest proximity. [23:34] Both things will help. [23:35] In a setup where you have my voice, perfect. [23:39] You can decode it and code it on that. [23:41] If you don't have my voice or even if you want to double amplify it, [23:45] we already support effectively keyword detection, which is useful [23:50] for real-time setting and async setting. [23:52] Back to Cheeky Pint transcription, you could effectively [23:56] pre-generate that from the previous podcast and look for a set of words [24:00] that you would use traditionally in that. [24:03] You do the keyword detection already, but how hard are the… [24:07] I want to get superhuman transcription performance by feeding it an hour of Mati [24:11] audio before it listens to Mati, and then it should be able to do [24:14] a much better job transcribing. [24:16] Is that just a really hard research problem? [24:18] No, solvable. [24:20] We think we can roll it out in one of the next [24:23] versions, which is hopefully in the next month. [24:26] You think this year you're doing person-specific transcription? [24:31] Person-specific transcription. [24:31] We can already dial our speakers extremely well. [24:34] If we are speaking, we can, of course, dissimulate who is speaking when, [24:39] which is in transcription side, apart from accuracy, diarization [24:43] is one of the harder problems. [24:45] We do that extremely well. [24:46] Now it's going to be effectively what you're saying, [24:48] fine-tuning based on the speaker that I want to listen to, [24:52] which we know will be important. [24:53] In a healthcare setup, it's such an important part. [24:56] You're in an operating room, you're a doctor, [24:58] you want to say a command, then you want to really be able to listen [25:01] to that one person's specific piece. [25:03] You have a hardware device at home, let's say it's a pilot [25:06] that helps you control the TV. [25:09] Here, too, you will want that to listen to you versus, let's say, [25:12] the family running around. [25:14] Or maybe you want it to everyone. [25:15] You could decide that, but in many cases, you want to be able to specify that. [25:21] That's really exciting. [25:22] It's great because there's still so many unsolved research problems. [25:26] Yeah, there's just breakthrough after breakthrough coming in [25:30] the domain of voice models. [25:31] How about on the flip side, when it comes to speech generation? [25:34] In the Zoom "touch up my appearance" feature, I've always thought about [25:38] that in the context of voice, where... [25:41] Should you offer a de-accenting filter for voices? [25:45] Or even there's one podcast that I like to listen to, but the voice is a little [25:49] mumbly, and I always thought they should put it through a de-mumbling [25:52] filter just to make the—Or slow it down. [25:55] Yeah, make the enunciation a little better. [25:57] But all these things, again, like Photoshopping an image, [26:00] there's no reason that the… [26:02] Have you thought about voice-to-voice, basically, rather than [26:05] voice-to-text or text-to-voice? Yeah. [26:08] There are two big parts. [26:10] One on the speed generation side, similar. [26:13] So many innovations still there. [26:14] There's a wider piece, and we released the V3 model that we're [26:20] solving that for the first time. [26:21] It's like, can you control speech? [26:23] You can have the text-to-speech. [26:25] You generate something that sounds emotionally great. [26:28] Previously, until the end of last year, effectively, you would rely on the model [26:31] to decide what's the best performance. [26:33] You could regenerate it, but ultimately, model decides the best performance. [26:36] That's where the controllability came in, where we can finally give it cues of, [26:41] say it in a slower way or change how you deliver the dramatic pause [26:44] or any cues that you give. [26:47] To be able to do that, you need the architectural changes [26:50] and the data that we created over time where you annotated what was said and how [26:54] it was said, so you can actually train the model to do that. [26:56] Today, finally, you can have both speed [26:59] generation or entire voice agent experience [27:03] with what we call expressive mode, where the agent knows [27:06] the emotions on the other side. [27:07] If the person is stressed, it can react and be reassuring, [27:11] and that's generating an LLM response on the reassuring side and response [27:15] in that set of emotions, too. [27:17] That breakthrough was super hard to do. [27:20] That, of course, stretches to a lot of what you said. [27:22] It could be some version of speech enhancement, [27:25] either real-time or in a pulsed setup to change how that's delivered. [27:31] That's relatively recent innovation, and we know it can still be so much better. [27:35] The edge cases of how you want to describe it is pretty large. [27:40] That's one. [27:41] Then the second part of the question, which is a huge question, [27:44] that's speech-to-speech models. [27:47] As you said, our approach, as you think about voice agent, [27:50] conversational agent, is effectively a cascaded approach. [27:52] You use transcription or speech-to-text, LLM, text-to-speech, [27:56] and orchestrates all of that together. [27:57] Then you have a speech-to-speech, which goes directly from speech, [28:00] and there's a speech response on the other side. [28:04] When we say speech-to-speech, is that the idea that it doesn't go [28:06] through text as an encoding in the intermediate? [28:08] Exactly. Oh, interesting. [28:10] For performance reasons? [28:11] For accuracy reasons? [28:13] You usually do it for latency. [28:14] It is faster to run a model that does not have to transcribe and then generate. [28:19] Exactly. [28:20] It's quicker, but on the flip side, you lose reliability. [28:23] You lose like all visibility into the parts of the pipeline. [28:27] In emotionality, we think you can deliver both on both sides extremely well, [28:30] and maybe you can make it more controllable, too. [28:32] Today, we are optimizing heavily on a cascaded approach. [28:36] I'm sorry, a cascaded approach is? [28:37] Is the speech-to-text... [28:39] Going through the text layer. [28:41] As we work with a lot of the businesses and enterprises, they will need [28:46] that visibility into what happens. [28:47] They will want to execute certain tasks on top of that. [28:50] They want a good visibility into each of the steps and great [28:54] accuracy of all the models. [28:55] But beyond that, they can abstract away what's the LLM layer, [28:58] what's the intelligence layer, the integrations are easier in that system. [29:02] That's where we are betting a lot of the research work of how you can make [29:05] that great, and we think we can make that great. [29:07] In speech-to-speech, as you think about maybe more [29:09] of a companion version of the applications, [29:12] that's where that will flourish because maybe the hallucinations aren't as [29:15] important, but the latency is a little bit more, and [29:19] maybe hallucinations are even a feature. [29:20] Maybe in the future-future, just to finish that part, you will have [29:23] some version of combination of the models. [29:25] That for low complexity, easy models, you will have speech-to-speech, [29:30] and for higher complexity we will have to cascade it. [29:33] I was going to ask about this. [29:34] You know the way there is research on how the invention of writing changed human [29:39] brains and just changed the neural pathways in ways beyond [29:45] the actual written language. [29:49] Do you observe that speech-to-speech models think differently [29:54] than cascaded models? [29:58] It sounds like they're dumber. [30:00] They are definitely dumber. [30:01] You need a smaller model. [30:02] But that's interesting, that forcing models to reason about text. [30:07] I know they just have much more in there as well, but they're smarter. [30:10] Yeah, but it's like, if you are getting speech-to-speech, [30:13] usually you will use smaller models, so it's still quick. [30:17] I see. It's also just a model size thing. [30:19] But are there interesting differences beyond correlates like size? [30:24] What I can say is slightly different to your question. [30:26] The people interacting for voice and the performance we see [30:31] for how they interact with the business changes just [30:35] by nature of interacting with us. [30:36] A good example, you can contact ElevenLabs and register for your interest, [30:41] you go through the form, and at the end of that, [30:44] we have supplemented that, that instead of going through the form [30:47] process, you can speak with our agent and leave more details. [30:50] What happened there are two things. [30:51] One, people were actually much more keen to leave the forms through speaking [30:56] with the agent, so we would go through the form a lot easier. [30:59] But second, they would be a lot more open-ended [31:01] in terms of what the use case are. [31:03] They would start giving us information about the wider set of use cases, [31:07] the complexity of the use case. [31:09] The writing out was tedious and tricky— [31:12] This is like an open-ended adventure game. Open-ended. [31:14] You could ask follow-up questions, you could clarify. [31:16] But people were just more at ease and could trust the system [31:20] while doing that, that it's working. [31:23] That helped us a lot. [31:24] Then free, which maybe it's more of a technological barrier, [31:27] it also works across all languages. [31:29] Now we have leads from all parts of the world coming in and leaving their details. [31:33] We did that use case, and now we have a few different companies [31:36] building their SDR versions of that, too, to help them capture the leads coming [31:41] in from banks all the way to actually one of the automotive companies that leaves [31:46] that where people are just more keen to speak through voice. [31:50] I want to ask about this second-order effects. [31:53] You've talked in the past about how growing up in Poland, [31:57] I guess the dubbing of TV shows, they were cheap, and so they would only [32:01] have one voice actor for a TV show. [32:03] No matter all the parts, male and female, they're like, "I love you." "I love you, [32:05] too." There's one voice actor doing all of them. [32:11] Now, thanks to better voice models, you'll be able to just have really good [32:15] voices, AI-generated, for all the dubbing. [32:17] Because again, it's not like it's taking jobs from great dubbing [32:20] that was happening previously. [32:21] It was awful dubbing happening in Poland previously. [32:24] That's one example of the second-order effect. [32:28] What are the other second-order effects you're seeing of ubiquitous good [32:33] text-to-speech, speech-to-text? [32:35] It seems like across a broad array of languages, [32:38] because whatever in English, just this didn't exist in Polish [32:42] or Irish or, pick your language. [32:44] One, breaking down the language barrier. [32:47] The inspiration came from the movie side, but it also applies in any [32:52] communication setup. [32:53] Could in the future, could I travel to another country and [32:57] speak Polish or speak English and that language isn't being [33:00] understood in the local native language? [33:01] From The Hitchhiker's Guide to the Galaxy, this version of the Babel fish. [33:04] That you can actually understand the world. [33:07] Voice, of course, will be an interaction layer. [33:08] But similarly, all of us will have our own extension and voice agents [33:14] that can help on our behalf. [33:15] There are very clear and great examples of that of people that lost [33:20] their voice and can get it for the first time back. [33:23] We see that everywhere, whether that's people that lost it due to [33:26] ALS or throat cancer that can get it back. [33:28] Just recently, there was an example of a patient that had Neuralink. [33:32] We worked with them to bring the voice that that person could speak with their [33:36] own voice back with the family around. [33:40] We worked with a lady that lost her voice before she got married. [33:46] Then finally, the technology became possible. [33:49] We were able to recreate that voice. [33:51] For the first time, she could replicate the marriage [33:55] ceremony and speak the vows together, which was such a heartfelt moment. [33:59] That's really sweet. [34:00] It's probably the most important from all the work that we do. [34:03] When you guys talk about voice agents, is a voice agent just [34:08] the idea that you have some long-running or [34:14] persistent agent that is going out and interacting with the world through voice? [34:18] Customer service would be one example of it. [34:22] In the other direction, your Claw going and making you [34:26] a restaurant reservation and actually calling up the restaurant. [34:28] Is that how I should think voice agents? That's right. [34:31] Exactly. [34:33] Whether it's the reactive side of being able to interact with the customer [34:36] or the proactive to call it back. [34:38] We recently had a very interesting one, topical because it was a Guinness-related [34:41] one where there was a developer developing a Guinndex, effectively. [34:47] Oh, I saw that. [34:48] They were calling all the pubs in Ireland to check on the price of a pint. [34:50] You could ask that or report information. [34:53] The Guinndex was built with ElevenLabs technology? [34:56] It was built with ElevenLabs, too. People could actually do both sides. [35:00] Could proactively reach out, reactively reach out, always capture full voice. [35:06] Then 3,000 different entities could report their prices and get that across. [35:12] Have you, by the way, hooked up your OpenClaw to ElevenLabs? [35:18] Is the OpenClaw-ElevenLabs combo something that a lot of people at Eleven are doing? [35:22] As you know, the OpenClaw will look for the most popular tools [35:25] frequently where it tries to cook up. [35:26] ElevenLabs is one of the recommended ones. [35:28] It's the top option for voice. [35:30] Can you tell me a bit about the business of voice models? [35:35] I think people have an intuition around big LLMs where there are these very [35:39] expensive training runs, and yes, they depreciate quickly, [35:44] but there's so much usage that all of the models trained to date have paid [35:48] off their training runs and then some. [35:51] Then there's this ever larger CapEx going into… [35:55] I mean, a lot of it is inference these days, but also training. [35:58] People have some intuitions from the LLM world. [36:01] I'm curious just how I should think about voice, where one, [36:05] how expensive is training the voice models? [36:08] Is the expense in the researchers? [36:10] Is the expense in the training runs? [36:15] The economics is presumably simple where it's just per-usage. [36:20] But yeah, just talk us through the business. [36:21] Definitely cheaper than the LLM and image-video models. [36:26] Significantly smaller models. [36:28] The models are smaller? Smaller. [36:29] What's the parameter count for a leading-edge voice model? [36:32] A few billion to tens of billion per meter models. [36:38] For context, I think the… [36:41] CPUs moved away eventually from gigahertz as the metric as they move to more cores. [36:46] I think we've mostly moved away from just raw parameter count, [36:49] but I think the leading edge LLMs are in the hundreds of billions of parameters. [36:52] I think the leading ones, yes, but of course, you have the variations [36:55] that you will use at lower scale. [36:58] CapEx is still pretty high. [37:00] We've, of course, raised recently a half a billion at 11 billion valuation. [37:04] Makes sense. Makes sense. [37:07] To continue being able to build the best models in the world. [37:12] Researchers, of course, you want the best people in the world. [37:15] I think we have those people working in audio and my co-founder [37:20] who is leading that work. [37:22] That's definitely a big piece of, not financially, but even how you keep [37:27] the ambitious deployment, so you continue building [37:32] leading models, helps you attract more talent in building that. [37:36] Then on how we service, of course, inference is correlated [37:40] with how the models are used. [37:42] For us, we've seen incredible growth across the work. [37:46] Mostly this is charged per… [37:47] If it's input text or text-to-speech, it's usually per text token. [37:52] If it's voice agent or transcription, then it's per minute. [37:56] We see that being the bigger part. [37:58] But usually, broadly, it's per token basis. [38:01] Of course, as we work with businesses, it's like an annual agreement. [38:04] The bigger the spend, the bigger the commit, the bigger [38:06] the discount to get it across. [38:08] The way we usually do is when we have a new model, we try to give it at cost [38:14] to a lot of the customers so they can experience the best. [38:17] It's still usually not as reliable— That's interesting. [38:19] The newest thing is often the most expensive, whereas you make the newest [38:23] thing the most economically attractive one? [38:25] We try to make it attractive so the customers are… [38:28] It's more expensive for us than any previous generation. [38:31] The quality is higher, so we try to keep the prices still competitive. [38:36] I see. [38:36] You subsidize it, but it's inherently more expensive as a bigger model. [38:39] Exactly. [38:40] Over time, we might do some tricks to optimize it, but we want [38:43] the customers to experience… [38:44] Because of research, the big thing that we've seen is the reliability of the model [38:50] in the early days might not be there. [38:53] Then two, people don't even know what's possible with that model. [38:57] You want the widest set of distribution, so people can show [39:02] the world what's possible. [39:03] You can have it, of course, as the distribution mechanism, [39:06] learn yourself what to improve, what to change, and then get it out there. [39:10] Are the voice models just getting bigger and bigger? [39:13] Will we have voice models in the hundreds of billions [39:17] of parameters, or have we found… [39:18] It seems like for certain types of model architecture, there's an upper [39:22] limit on the natural size. [39:24] Have we found that upper limit for voice models? [39:26] It feels like for specific use cases, like say, audiobook narration, [39:30] you probably found that size. [39:31] You probably don't need to stretch it much bigger to make the quality much higher, [39:35] but for certain use cases, that will probably grow. [39:39] The thing that I hesitated on the question is in a cascaded approach, you probably [39:44] will not see dramatic size changes. [39:48] You inherently want the models to be quick and reliable. [39:51] You want to orchestrate them in a smart way. [39:53] In a fused approach, probably that will get into tens, [39:57] hundreds, billion-per-meter models because you combine, of course, [40:01] the LLM side and the voice side, so that will get bigger, but on the just [40:05] voice, I think it will keep being small. [40:07] There are certain domains where we'll see bigger models. [40:10] Yeah. That's so interesting. [40:13] It does seem fun, from a research point-of-view, [40:16] how there are still these various unsolved aspects, and how you guys are just making [40:21] technical breakthroughs and then releasing them down the product pipeline. [40:23] That's a really fun stage of a company's life cycle. [40:27] For sure. [40:27] It's fun because it feels like we can do innovations on both sides. [40:31] There's so much on the research side, so much on product side. [40:35] Ultimately, the biggest part is how we deploy it to the customers, where [40:42] SMB will have a very different dynamic than the enterprise. [40:45] It's not vendor/SaaS relationship where you just give the product out there [40:49] for the biggest companies out there, but you are more of a partner [40:52] in their AI transformation part. [40:54] You want the resources to work alongside them, to work on the frequently very new [41:00] use cases that were impossible to help create and bring those [41:04] voice agents to production. [41:06] That's a big shift. [41:08] The biggest focus is how we bring the conversational agents out there [41:12] to the businesses around the world. [41:14] When you say bring conversational agents is the biggest priority, is this [41:18] for customer service-type use cases? [41:21] What are the most popular use cases for conversational agents? [41:26] We want to be a partner for full interactions between businesses [41:29] and their customers or their audience. [41:31] I'm saying their audience because that will apply in support. [41:34] Support is the easiest one because that's where it's most ready. [41:37] That's maybe the big difference to how we see ourselves to some of the other [41:41] companies in the space. [41:42] This can also apply to sales. [41:44] You can have the proactive side of reaching back. [41:46] You can have AI SDR versions of that. [41:49] Then you can have all the way to the marketing use cases where we are [41:52] your partner for working even outside of the conversational agent [41:58] space of how you create a great marketing campaign. [42:01] How does this break down between… [42:03] We had Des Traynor from Intercom on here and they have Fin, their agent, [42:08] and it's a thing on the website that you can go talk to. [42:11] He described a very similar phenomenon that you described, which is, [42:16] you start maybe thinking, "Oh, this will help me answer customer support [42:20] queries." It becomes like a generic UI [42:24] for the website, where it's a box you can [42:28] type in to go do things and understand things. [42:30] Why wouldn't you read the docs and design your integration that way? [42:33] Whatever. [42:34] Will I have one for text and then one for voice? [42:39] Will you guys do text, too? [42:41] How does that... [42:43] Because it seems like this is also succeeding at the text level with Fin [42:47] and Sierra and all these things. [42:48] The places where we know we will be able to provide the biggest value is where, [42:53] ultimately, today you will have either a big portion or most of the [42:57] interactions coming for voice. [42:59] If that intersection is there, that's where we can provide higher value. [43:04] Of course, if you need a text chatbot there… [43:07] If you fix the voice agent, you have fixed text piece inherently as well. [43:13] The place where we do optimize today is going to be like, [43:16] how do you select the right voice for the right customer interaction, [43:19] how you pull that in there. [43:20] In the pretty complex case of what you mentioned earlier, of how you orchestrate [43:24] that to pause or look for something deeper into the docs, how it can be extension of [43:30] entirety of the business, so not only in support, [43:32] but across the entire user journey. [43:35] The bottom line is we want to be able to provide you across [43:39] the entirety of the interactions. [43:41] Voice is usually a big part of those interactions. [43:45] We need to solve the integrations, we need to solve the knowledge, [43:47] we need to solve text as part of that. [43:50] We wouldn't, for example, go into what I think will happen in a lot [43:54] of those cases, very deeply into reasoning version of those use cases where [43:58] you maybe need the multitouch. [44:01] Yeah, and a lot of complex actions. [44:03] A lot of financial analysis. [44:05] That would be not something we optimize for. [44:08] Can we talk about your revenue ramp? [44:09] You're just one of the fastest-growing startups, period, of the past few years. [44:14] What's your most recently announced revenue figure? [44:16] Most recently announced was end of 2025. [44:18] Whatever number you want to give us. [44:20] Most recently announced was 350 at the end of 2025. [44:23] The best proof of the technology working... [44:26] Recently, we were in our work with Deutsche Telekom and T-Mobile [44:29] with Revolut, with Klarna, with Meta, with IBM, a wide set of use cases. [44:34] This quarter was one of the best for enterprise growth where we had [44:39] the first quarter hit 100 million in additional ARR growth, which is crazy. [44:45] In net new ARR. In net new ARR. [44:48] If you're thinking this quarter was 100 million net new ARR and 350 million [44:51] at the end of the year, I'm no mathematician, [44:53] but it's up in the 450 million range. [44:56] Versus this time last year, that's a several-fold increase. [45:01] Just, what's working? [45:04] From the outside, I would assume that there's [45:07] really strong cohort growth within accounts, and then you seem to have [45:11] self-serve and enterprise businesses that both contribute a lot. [45:16] I don't know how big self-serve is, but as a user, I like to be able to fiddle [45:19] with ElevenLabs and not have to go talk to sales. [45:21] Maybe you can just talk about what worked to reach 450 [45:25] million plus of ARR so quickly. [45:29] We are over 50% is now sales-led enterprise. [45:33] I think largely that technology powers a lot of their agentic interactions [45:39] just became reliable at the same time as high quality over the last year, [45:44] year and a half. [45:46] Frequently, you know this extremely well, [45:49] you will start the account, and then, of course, it continues expanding. [45:54] We see there's definitely land and expand motion across ElevenLabs. [45:58] We bring—What does that expand look like? [46:00] Is it new departments? [46:01] Is it just the usage starts taking off? [46:03] When a customer expands—Both. [46:06] Usually, the first part, too, it's like we try to make it very [46:09] easy for our customers. [46:11] Maybe against ourselves, where we give the technology a pretty [46:16] attractive economics because we so much believe in the technology providing value. [46:21] You can actually try it and test it. [46:23] Then within that one department— [46:24] You think you'll make it up in usage, basically. [46:26] Exactly. [46:26] That usage, the commitment continues increasing because you [46:30] know it's providing value. [46:31] Then it's so much easier to make that a choice. [46:34] Then, of course, cross-department pollination is there, too. [46:38] Our work with Deutsche Telekom started with marketing side. [46:41] We did Magenta work and podcast generation. [46:43] Then it expanded to customer support. [46:46] Then it expanded to us working on an agent across the entirety [46:51] of the network, so people can call in and have the agent. [46:53] You could see those step changes across. [46:57] We are now 470 people as a company, [47:03] so we keep on growing. [47:05] Some of the things that stay consistent is small teams. [47:08] We have less than 10-people teams for each of the product or research initiatives or [47:13] even, as you think about sharding some of our go-to-market strategy, [47:17] there will be smaller teams understanding the industry in-depth, [47:20] understanding the market in-depth, and going independently and going quickly. [47:23] That definitely contributed largely to that. [47:26] Two, especially on the biggest enterprises, what we found works is [47:30] we have the full spectrum: self-serve, PLG motion that helps drive distribution, [47:36] drive awareness of ElevenLabs. [47:40] On the completely other spectrum, we have the high-touch for deployed [47:43] engineering working side-by-side with the customers to customize [47:48] the entirety of their work together. [47:50] Why did you guys do self-serve? [47:52] Because I presume you have a lot of competitors where they have tech, [47:56] and it's behind a contact sales form, and you have to go talk [47:58] to an SDR and then talk to an AE. [48:01] You guys just offer the tech available on the side. [48:04] I'm a huge believer in this. [48:05] A huge part of Stripe's growth has been driven by the fact that we just made [48:09] Stripe available to anyone and built a lot of products around that adoption pattern, [48:14] but so many companies seem to skip it. [48:17] I'm curious how you guys came to— So many reasons. [48:19] I think the quick ones that come to mind is feedback loop. [48:25] You have immediate understanding of how good your technology is. [48:28] Two, which is an extension of that. [48:30] We stand behind our tech. [48:31] We believe it's the best in the world for models, for voice agents, for deployment. [48:37] We want people to experience that. [48:39] I think you do that the same in Stripe, where the best version of the technology [48:42] is available to everyone, which is so attractive to actually try it out. [48:46] We always try to make everything we built for the highest end use cases, [48:50] bring it back to the ecosystem free. [48:53] Frequently, the newest of the use cases... [48:56] For enterprise, you will need reliability, you need compliance, [48:59] you need to scale, which we deliver. [49:01] Frequently, as you develop new technology, it might not be ready for a lot of those [49:07] parameters, but it's definitely ready for developers and SMBs. [49:10] We love what they are doing because they are showing us the future and effectively [49:14] helping us find a trajectory of where ElevenLabs should go. [49:17] I'm totally convinced. [49:19] I'm just always amazed that more companies don't pursue it, [49:21] where it feels like they're really shooting themselves in the foot by not. [49:24] Did you guys self-serve on Stripe? Or did you— [49:27] We self-serve on Stripe. [49:28] For example, Eleven is a huge company, and yet you started on Stripe [49:32] on a self-serve basis. [49:33] Initially, we were two of us at the beginning. [49:37] You try to see what's working in the industry, but you try [49:40] to think from first principles. You want to try it out. [49:42] You want to understand how it works. [49:45] The more friction elements before you're trying it out, the less you trust whether [49:49] it's available, whether it'll be additional payment [49:51] that's hidden behind some of those steps. [49:53] You don't want to go through them. It's so much. [49:55] Speaking of Stripe, do you have any Stripe feedback for us? [49:57] Anything you want us to fix? [49:59] My most common feedback until recently is like, why don't you give us pay-as-you-go, [50:04] usage-based billing type version? [50:08] One of our finance leads, Maciej, I know was speaking with your team, [50:11] and that was day before. [50:13] He was great. [50:14] He was thinking about it for a long time. He's great. [50:16] Was it he who said, "You guys should buy Metronome?" [50:19] You should buy Metronome. [50:20] Then the next day, Metronome acquisition was announced. [50:23] Now you have it. [50:24] That was my most common feedback, and we'll be launching. [50:27] That's a good announcement for this podcast: we'll be launching [50:31] usage-based billing to everyone. Sorry, I'm shocked. [50:35] As in previously—Pay-as-you-go. [50:38] Previously, you had it on an enterprise basis, but everything [50:41] on the self-serve basis for like— [50:43] We had the subscriptions, yeah. [50:44] Subscription plans, you could go over them, but now we are [50:47] launching a full pay-as-you-go experience. [50:49] You can just try out voice engine, which is effectively this [50:53] orchestration loop all the way through to any of the models directly. [50:56] Going back to self-serve, I think a new thing in AI is that [50:59] all self-serve products should have pay-as-you-go as an option. [51:04] Maybe you want to have a subscription with some unlimited tiers, [51:07] but I don't know if you had the experience of you're using Claude, [51:09] and you're typing away your queries, and eventually you hit some rate limit, [51:13] and it's like, "Sorry, you've hit your usage limit," and you want [51:15] to be able to do the thing that you can do with Claude Code, [51:17] which is just pay per API. [51:20] It's like, I'll pay for it. [51:21] It's very funny as a consumer to not have the option to pay more, [51:25] to use the product more. [51:26] I think every AI product will need… [51:30] They probably want to have some all you can… [51:32] Most of what you can eat subscription with limits and then [51:35] the ability to pay for overages. [51:36] It sounds like that's what you're doing. Yeah, exactly. [51:38] That's what we're doing. [51:41] The only thing I wanted to ask you about is, I feel like all CEOs of larger [51:46] companies today are trying to figure out how do all these AI advancements [51:52] change the nature of the organization and how do you redesign your organization [51:58] a bit around all this new intelligence. [52:01] That could be about what the scaling factor is of the number of people [52:05] you need to do the work. [52:06] It also should be like, do you need more senior people because [52:10] they're better able to direct the AIs, and the AIs are maybe you can do the work [52:13] of what previously would have been junior people. [52:15] Do you need more junior people because they're going to be more [52:17] AI-native in how they work? [52:19] Do you want smaller teams? Do you want bigger teams? [52:21] How do you actually go do the process engineering of... [52:24] Your finance team should be using Claude extensively. [52:28] Finance teams do not historically have a lot of home-built software. [52:32] There's all these questions that are floating around, and you have very rapidly [52:36] built a much more AI-native company. [52:38] I'm curious what lessons we should all be learning from ElevenLabs as a [52:44] large business recently built, and so without the baggage of decades [52:50] of "how we've always done it." [52:53] We started in 2022, which is a year when the two topics [52:56] of the day were crypto and metaverse. [52:58] Just before, and then, of course, AI flow start— [53:01] Yeah, because you scaled in the AI— Exactly. [53:02] We could have the privilege of scaling through the world when [53:06] it was all happening. [53:07] For us, what works, and we really believe in that being the big part of the future. [53:12] The first is small teams, keeping the teams small and super flat. [53:20] Both me and my co-founder will have over 15 direct reports each [53:23] that we'll work with. [53:24] Most of those people will have that same scale of direct reports. [53:28] Your span of control is way larger than a traditional company. [53:30] Normally, it would be eight, you have double that, and obviously, [53:33] that's an exponential. Exactly. [53:35] Of course, there are some teams which in the short term might not do that, [53:39] but ultimately, that's where we think it's going to be headed. [53:41] It's roughly 10 team size within each of those work items. [53:45] Startups, no offense, but startups often have pretty wacko management ideas. [53:49] There was a funny tweet: Lord, grant me the confidence of an early-stage [53:53] startup founder blogging about their management theories. [53:56] You think this is not a startup effect? [53:58] This is an AI effect where basically— [54:00] No, it's definitely a little bit of a startup effect. [54:04] I figured out it's the hindsight benefit. [54:07] I'm canceling our Stripe changes. [54:10] No, it's like I need to pre-end it. [54:14] The hindsight of this may be working. [54:17] We'll see in next five to 10 years. [54:18] Much flatter org. [54:20] Much flatter org. Smart teams. [54:22] It works for us, might not work for all the companies. [54:24] There are some parts where go-to-market, we still are trying to figure [54:26] out what's the best way. But smaller teams, flatter org. [54:30] I think there are two paradigms, but generally people being more technical, [54:33] or if not technical, even in non-technical teams, having a technical resource. [54:39] We will have a person in ops or in talent that will… [54:42] We have effectively a tech lead for that team that helps them automate [54:46] a lot of that work and helps up-level the rest of the team, too. [54:50] There are two parts that are helping. [54:53] Talk me through this in talent or something like that. [54:55] Is it that you are building your own software where other [54:59] companies might have bought software, like a Workday or a Greenhouse or something? [55:03] Is it that they are using the existing software you have better? [55:08] Is the process that will be spreadsheets in a traditional company [55:11] are built with software? [55:13] How do you use the software in these organizations? [55:16] Sometimes, but we still use a lot of the traditional vendors. [55:21] One pattern is, of course, LLM-ifying everything, [55:24] making the data explorable for you to be able to interact with it of who's in the [55:30] pipeline, what worked, who does the best references, all of [55:33] that work, so you can double down on that. [55:35] Two, it's frequently things that you manually do that a lot of the… [55:39] There's a gap between where the agents are today versus what you could do if [55:44] you have the technical skill set. [55:45] A good example is, how do you scrape all the right profiles to be able [55:50] to reach out to the right candidates? [55:52] You analyze whether it's… [55:55] I don't know how much I want to say, but try to detect specific [56:00] things that we know worked. [56:01] You bring that across to the people. [56:06] On go-to-market side, there's just so many things you [56:09] can do with additional amplifiers. [56:13] It goes from understanding what case studies are relevant and creating a good [56:18] pre-read for you before you go to the meeting through creating the AI SDR [56:22] experience that we spoke about, to creating an entire deck experience. [56:25] You have a pre-populated deck with the right numbers that is customized [56:29] to that customer, which you want still the person to go [56:32] through and develop, but ultimately is in there. [56:36] There are plenty of those additional things that you know will [56:39] amplify the work of the people around, potentially replace some of those [56:44] easier tasks that are done. [56:46] Then we wanted for people to explore the culture at ElevenLabs. [56:50] We created a voice agent that people can speak with and see what's the culture, [56:53] but also get prepped for the interviews. [56:55] I think across many of those teams, additional benefit of what they can do. [57:00] Interesting piece. [57:01] Of course, in Ukraine, we're doing ongoing work. [57:04] They need to rethink a lot of how their development, their systems, their support, [57:08] works for the citizens across the country. [57:11] People that are in the war zone, they don't have the same [57:13] access to the information. They cannot rely on the same phone lines. [57:16] They cannot rely on the same physical services around the country. [57:20] They've developed effectively a central— [57:22] Is it your employees in the Ukraine? [57:24] We had a few, but they reached out because they were developing their [57:29] central map called DIIA. [57:31] They developed it over the years, but now before, they were doubling [57:35] down on how this can be a way of supporting the citizens. [57:39] Of course, there's an easy part of how you create a first agentic government where [57:43] you have a help with the benefits and what's happening [57:46] on the front line or education. [57:48] That's delivered to everyone. [57:49] Or healthcare, so you can book your checkup or appointment. [57:52] How you create all of that. [57:54] Of course, we travel to Kyiv, we worked with them on bringing [57:57] that and making that available for voice so everybody can access it. [58:00] But the thing we've learned while being there was that model of what we speak [58:04] about where you have technical resources in each of the teams, they actually have [58:09] the same in every of the ministries. [58:11] Every ministry had technical resources working on creating that agentic [58:16] version of their work. [58:17] Then it was a central digital transformation team that would assemble [58:21] this all together to deliver that through the central citizen support, [58:24] which I thought was brilliant. [58:25] That's very tech-forward by Ukraine. So tech-forward. [58:28] The most advanced set of work we've seen. [58:30] We got a little bit validated. [58:32] Maybe technical resources in each of the teams is a good idea. [58:36] That works heavily for us. [58:38] You mentioned some of the other parts, like do you hire the senior or younger... [58:42] Main thing we try to filter for, of course, the culture [58:46] piece is so important. [58:47] You could scale people, but scaling culture is much harder. [58:51] You want to optimize for that being right. [58:53] In our case, it's first principles, taking ownership, striving [58:57] for excellence but staying humble. [58:58] The main thing that's in that ownership part that I think [59:02] works well for the AI world is agency. [59:04] If you have that agency to explore, regardless of where you are [59:08] in the experience cycle, it's going to be a tremendous amplifier to your work. [59:13] My biggest takeaway from all this has been that around agency, [59:17] where I feel like high-agency people are the winners of the [59:22] advances in AI, and within organizations, low-agency people will lose out. [59:28] Yeah, completely agree. [59:30] It's probably the most proud thing that Piotr and I are as we [59:34] scaled ElevenLabs, the people that are at ElevenLabs, [59:37] it's been just the culture and seeing the expansion of the culture, [59:40] where culture builds the company now rather than any single person or any [59:44] single product builds the company. [59:46] That is probably the biggest validation and happiness. [59:49] There's the other angle of that where I think people are striving to be incredible [59:56] in their craft and their work, but at the same time have fun in a lot their work. [60:00] That combination of agency and just enjoying what you do is [60:05] probably the best thing we've been able to do today at ElevenLabs. [60:09] It sounds like a really fun stage, like we were saying. [60:11] Interesting research breakthroughs, really fast-growing business. [60:14] I'm sure you're enjoying it. Mati, thank you. [60:16] John, thank you so much.