ElevenLabs Reaches $100 Million ARR as Voice Models Threaten Traditional Identity Verification

ElevenLabs ARR突破1亿美元，语音模型对传统身份验证构成威胁 语音AI迅速达到逼真水平打破了传统的安全规范，迫使人们转向密码学签名和设备级身份验证来确认人类身份。 在本次对话中，ElevenLabs首席执行官Mati Staniszewski分享了他创立这家语音AI公司的历程及其在商业上的快速成功。他探讨了语音合成的技术演进，从融入人类语言中的不完美特征，到迈向原生多模态模型以及低延迟角色重塑。Staniszewski还讨论了辅助技术和智能体间通信协议等新兴应用场景，回应了关于声音克隆的安全担忧，并阐述了公司如何坚持受迪士尼启发的以用户为中心的发展理念。 ElevenLabs已实现可观的财务规模，年度经常性收入（ARR）突破1亿美元，估值达到30亿美元。 (0:00) ElevenLabs最初的灵感源于波兰传统的“单人旁白解说（lector）”模式，即由单一声音平淡地旁白所有角色，创始人意识到利用Transformer和扩散模型可以彻底变革这一模式。 (1:17) 当作者们开始将整本书一段段复制粘贴到一个极小的文本框中以生成完整的有声书时，产品与市场契合度（PMF）得到了早期验证，引发了需求的激增。 (3:10) 语音若要转变为主要的人机交互界面，既需要模型突破以达到“图灵测试”级别的质量，也需要分发渠道的突破以进入绝大多数主流市场。 (6:06) 在AI语音中引入自然停顿、呼吸声和犹豫等人类的不完美特征，反而使参与度、信任度和通话完成率实现了两位数的增长。 (8:25)

ElevenLabs co-founder Mati Staniszewski argues that voice AI is poised to become a primary computing interface within two years, powered by native multimodal models, realistic vocal imperfections, and dedicated enterprise tooling. However, he warns that ubiquitous voice cloning shatters legacy phone-based trust, requiring society to replace reactive AI detection with cryptographic and hardware-level verification architectures. Engineering Vocal Realism and Finding Product-Market Fit (3:10) Staniszewski explains that ElevenLabs was founded in 2022 out of frustration with the Polish "lector" dubbing convention, where a single narrator flatly voices all characters across foreign films. The company—which recently surpassed $100 million in ARR at a $3 billion valuation—discovered product-market fit in November 2022 when independent authors bypassed initial interface constraints by repeatedly pasting text segments into a single-sentence box to assemble full audiobooks. This sudden organic adoption rapidly strained GPU capacity and signaled clear commercial demand prior to the broader generative AI wave. Achieving practical realism required moving beyond flawless enunciation. Staniszewski notes that when a logistics firm added natural vocal hesitations, breathing sounds, and pauses to its automated scheduling agents, call completion and customer satisfaction rose by double-digit percentages. Conversely, specialized entertainment projects, such as recreating Darth Vader's voice for Epic Games and Lucasfilm, required maintaining iconic tonal gravitas while keeping system latency low enough for real-time multiplayer interactions. The Voice Interface Shift and Enterprise Timelines (6:06) Staniszewski forecasts that voice interaction will transition from early adopters to mainstream majorities within one to two years, driven by simultaneous breakthroughs in model naturalness and software distribution. He points to integrations like Perplexity as early evidence of conversational query workflows, while static media outlets like Time magazine have begun allowing readers to converse directly with feature coverage. He projects that enterprise customer service will pivot from reactive, frustrating interactive voice response (IVR) phone trees to proactive voice agents that manage user onboarding, appointment setting, and immediate refunds. To capture these opportunities, Staniszewski advises enterprises to target repetitive tasks and unique unstructured datasets rather than solely pursuing immediate margin optimization. He contends that businesses should launch functional proofs of concept within three months and reach full production within a year, arguing that prolonged enterprise deployment timelines are unjustified given the speed at which modern voice models can be operationalized. Multimodal Research and Defensibility Against Frontier Labs (18:17) Addressing industry concerns that specialized voice startups will become obsolete "roadkill" as frontier labs scale general foundation models, Staniszewski asserts that defensibility stems from pairing focused audio research with deep workflow integration. While ElevenLabs initially achieved state-of-the-art results by prioritizing audio data quality over pure compute, Staniszewski concedes that compute scaling will eventually dominate voice synthesis once foundational research challenges are solved. To remain competitive against larger labs, ElevenLabs is transitioning from a cascaded architecture—which sequentially chains separate speech-to-text, large language, and text-to-speech models—toward natively trained multimodal models. Staniszewski argues that research alone is insufficient; winning enterprise workloads requires building the surrounding product layer, including proprietary knowledge connectors, performance analytics, and integration tooling that general model providers rarely prioritize. A Three-Layer Architecture for Voice Authentication (21:37) The rise of indistinguishable voice synthesis has collapsed the historical trust equilibrium in which financial institutions and sensitive services authenticated identity via voice calls. Staniszewski emphasizes that because open-source models allow actors to generate audio outside monitored commercial platforms, trying to detect generated content will remain a losing "catch-up game." Instead, he advocates shifting the burden of proof from detecting fake audio to proactively authenticating legitimate human and synthetic speech. Staniszewski outlines a three-layer framework modeled on antivirus code signatures and HTTPS certificates. The system relies on hardware-level encoding to verify that a live human is speaking from an authorized device, cryptographically watermarked signatures to validate consented AI agents, and an untrusted default tier where AI classifiers are reserved strictly for unauthenticated media. While voice cloning offers profound utility in cross-lingual communication and accessibility—such as restoring voices for ALS patients and enabling a US Congresswoman to address the legislature—Staniszewski leaves unresolved how quickly consumer hardware vendors and telecom networks will establish the cryptographic standards required to secure identity in the post-voice-verification era.

ElevenLabs 联合创始人兼首席执行官 Mati Staniszewski 指出，语音人工智能正在从早期的配音与播报工具演变为新一代人机交互界面；他预测语音交互将在未来一到两年内迈向大众普及，而垂直赛道立足的关键在于深耕专业音频研究、推进原生多模态架构，并建立抵御伪造的新型身份认证信任机制。 波兰译配痛点、意外契合与商业爆发 (1:17) ElevenLabs 创立于 2022 年，其灵感源自 Staniszewski 与联合创始人 Piotr 对波兰传统影视单人旁白解说员（lector）制度的共同记忆。在波兰，影视作品常年由单一男声机械平淡地覆盖所有角色的台词，观影体验陈旧割裂。创始团队意识到新兴的 Transformer 与扩散模型可用于彻底改造配音行业。在早期摸索六个月后，模型首次生成逼真的人类笑声，成为技术突破的关键信号。 2022 年 11 月内测期间，产品仅包含一个类似社交平台的简易文本框，一次只能处理一两句话。然而大量作家自发将整部书籍手动切分为数千个碎片，反复复制粘贴生成完整的有声读物并发布到外部平台，超预期的需求瞬间压垮了服务器并耗尽 GPU 算力。这种自下而上的行为验证了产品市场契合点（PMF）。近期，ElevenLabs 的年度经常性收入（ARR）已突破 1 亿美元，并在由 Sequoia Capital、Andreessen Horowitz 和 Nat Friedman 参投的融资中达到 30 亿美元估值。 拟真瑕疵悖论与原生多模态演进 (8:25) 在语音 AI 的产品设计中，追求绝对流利与完美反而可能削弱真实感。Staniszewski 透露，某物流公司在其自动预约外呼坐席中引入自然停顿、呼吸声以及“呃”等人类语言瑕疵后，通话完成率与整体用户满意度均实现了两位数百分比的提升。而在与 Epic Games 及卢卡斯影业合作复刻“达斯·维达”（Darth Vader）标志性音色时，团队则攻克了在极低网络延迟下维持其威严阴冷声线的工程挑战。 面对大型基础模型实验室可能吞噬垂直应用的行业忧虑，Staniszewski 认为当前语音 AI 仍高度依赖音频专精算法与数据质量，而非单纯依靠算力堆叠。虽然他认同长期来看算力扩张会占据主导，但 ElevenLabs 正从“级联架构”（语音转文字、大语言模型生成、文字转语音三者串联）向原生端到端训练的多模态模型过渡；同时，公司通过深耕企业知识库集成、性能指标监控与工作流落地来构筑竞争壁垒。 从声带重建到机器间通讯的多元落地 (15:22) 除了传统的媒体旁白，语音技术正向深层人文关怀与新型交互渗透。ElevenLabs 协助多位因肌萎缩侧索硬化症（ALS）或癌症失去发声能力的患者重获原声；这一探索促成了一位失声的美国国会议员首次使用 AI 复刻语音在国会大厅发表演讲，继续履行代表职责。 在商业与前沿实验层面，技术应用展现出更广阔的维度。在媒体互动领域，ElevenLabs 与《时代》杂志在“年度风云人物”报道中合作，使读者不仅能收听文章，还能直接与内容展开语音问答。在近期的黑客松中，开发者构建了名为“Jibberlink”的智能体间通信协议：当代表客户和餐馆的两个 AI 坐席协商预订时，它们会脱离人类语言模式，转而发出类似早期拨号调制解调器的高带宽音频频率信号，实现数据的毫秒级传输。 重塑信任平衡：声音防伪与三层认证架构 (21:37) 逼真的声音克隆彻底瓦解了过去依托电话语音确认即可核准大额银行转账的高信任平衡。Staniszewski 指出，由于开源模型与竞品工具层出不穷，单靠算法分类器去检测哪些内容属于 AI 生成是一场注定滞后的猫鼠游戏。社会必须从“甄别伪造”转向“验证真实”。 Staniszewski 预判未来的身份核验将形成类似于防病毒代码签名和 HTTPS 证书链的三层信任体系：第一层是硬件级人类验证，通过手机与计算机的底层硬件证明物理人类正在发声；第二层是认证 AI 智能体，通过数字水印和密码学签名明确标识其所有者与授权许可；第三层是未认证池，凡是不具备硬件签名或可信水印的内容，默认被判定为疑似 AI 并交由安全检测工具处置。 消除语言壁垒与一线服务驱动的企业文化 (26:10) 跨语言克隆技术正在改变全球内容传播格局。通过将特定个体的音色、语调与情感无缝迁移至印地语等其他语种，声音技术正重现当年机器翻译降低互联网跨国阅读门槛所引发的“诱导需求”，推动跨语言交互总量的几何级增长。 在企业管理上，Staniszewski 借鉴了 Disney 首席执行官 Bob Iger 要求员工身着米奇玩偶服深入乐园接触游客的实践。在 ElevenLabs，高管与工程师贴近用户的对应方式是亲自处理一线客户支持工单。Staniszewski 至今定期亲自解答用户反馈，以此消除组织与市场实际痛点之间的隔阂，确保团队始终专注于产品的核心价值迭代。 Mati Staniszewski 描绘了语音从边缘播报功能向核心系统交互层跃迁的路径，但这一转型依然取决于两大前提：技术上需从传统串联拼接模型平稳升级为低延迟的原生多模态架构，生态上则亟需联合硬件厂商与监管方落地端到端的密码学身份认证，以重建被深度伪造打破的社会互信基石。

[3:10] Mati Staniszewski: wasn't very good. [6:06] Mati Staniszewski: scrolling our phones like primitive creatures. [8:25] Mati Staniszewski: uhs [8:25] dark force [11:15] Person of the Year [15:22] AI tourist companion [15:22] Jibber [15:22] Jibberlink [18:17] roadkill [18:17] cascaded [21:37] Mati Staniszewski: catch-up game [26:10] induced demand [26:10] dressing up as Mickey Mouse

[0:00] All righty, folks. So, I am delighted to [0:02] be joined here by uh Mattie Stenosvski, [0:05] who is the uh co-founder of 11 Labs. I [0:08] think it's fair to say that 11 is just [0:10] the most successful breakout company out [0:12] of the UK in the past 5 years. Uh you [0:15] guys just passed $und00 million in uh [0:18] ARR. uh 11 raised money at most recently [0:22] a $3 billion valuation from investors [0:25] like Sequoia and uh Andre Horitz and Nat [0:28] Friedman we're just discussing uh and as [0:31] you saw 11 is doing voice AI and so just [0:34] to give you a sense of how good the [0:35] products have actually gotten you know [0:38] what my voice sounds like you've had to [0:40] hear a lot from me maybe we can do a de [0:42] you guys cloned my voice so maybe we can [0:43] just do a demo of the AI John voice for [0:46] comparison I know they say AI is a [0:49] productivity booster but human John is [0:51] so lazy these days [0:54] um okay and uh part of what we'll get to [0:57] that is very relevant is AI in other [1:00] languages and so I don't speak Hindi but [1:02] we can maybe give the other [1:09] demoity so they've gotten really good [1:12] um there's so much to talk thank you for [1:15] joining us thank you for having me [1:17] there's so much to talk about. But maybe [1:20] to start, you know, you founded 11 Labs [1:22] in 2022. So before chat GPD launch and [1:25] before I think the latest AI craze, why [1:29] voice? Why 11 [1:30] Labs? Well, thanks for having me here. [1:33] The um I'm in such a lucky position that [1:36] we've actually shown you some of the [1:37] work that we do so you can actually hear [1:39] it for yourself. But one of the things [1:41] uh that we believed for a very long time [1:43] is that voice is the best interface to [1:46] interact with the technology to interact [1:48] with um with the [1:50] devices around us. But truly the [1:52] inspiration came so I'm in a very lucky [1:54] position where I've known my co-founder [1:55] for 15 years now uh through the years we [1:58] we we studied we worked we lived [2:00] together and now we co-ounded 11 apps [2:02] together and despite despite loss in [2:05] apps we are still best friends. So the [2:06] time is on on our side but one of the [2:09] inspirations came from where you're both [2:10] from h when we started a company 2022 [2:13] and the very peculiar thing for both of [2:16] us we are both from Poland is that if [2:17] you watch a movie a foreign movie in [2:20] Polish language all the voices whether [2:22] it's a male or female voice are narrated [2:25] with one single narrator. So you have [2:27] this very as you can imagine terrible [2:30] experience where literally there's like [2:33] a audiobook narration across the movie. [2:35] So that was our trick. You're watching a [2:36] movie and like the guy, the gal, the [2:38] villain all are the same voice actor. [2:40] Yeah. Yeah. And it's and it like kind of [2:42] is flat on purpose. So you are meant to [2:45] imagine the emotions the characters are [2:47] speaking with. And it's still happening [2:49] today. And that was effectively the [2:50] trigger point in 2022 that now as you [2:53] think the technology is developing uh [2:55] with transformer, the diffusion models, [2:57] we believe the same will apply in audio [2:59] and started 11 laps. Of course, that [3:01] original vision expanded from the [3:04] dubbing a beginning into how the voice [3:06] can amplify everything from narration [3:08] through to dynamic interaction. And how [3:10] long till you guys had a a demo where [3:13] you said, "Okay, this is really good." [3:14] Because I presume that then there was a [3:16] lot of steps to get to, you know, $100 [3:18] million in ARR, but yeah, how long till [3:20] you had a demo where you're like, "Oh, [3:21] wow. It's finally really good." The [3:24] finally really good. So the we had a [3:26] number of demos in the first six months [3:27] where it wasn't very good. Uh I think [3:29] the the the moment where it hit was when [3:32] we released this this this set of [3:35] samples where for the first time the AI [3:38] could somewhat laugh. It produced [3:39] laughter. Uh and um and people picked it [3:42] up and started telling us that wow this [3:44] is the first time we heard AI actually [3:46] laugh. And so that was the first moment [3:48] where it was like okay this might be on [3:50] a good direction. And then there was a [3:52] second moment in our in our beta when we [3:54] decided to release it. So we launched [3:56] the first prototype. It was this tiny [3:58] box um like Twitter type box where you [4:01] could input a sentence or two at a time. [4:04] And in the beta we had a number of of [4:06] book authors come in and copy paste the [4:09] entire book thousands of times download [4:12] it take the book upload it to the [4:14] services at the time you couldn't upload [4:15] AI content out there upload it get [4:18] reviews back that it's actually very [4:19] good and bring back with them a lot of [4:22] other people in the space to to work on [4:24] that too. So that was like moment okay [4:26] this is really hitting off and then [4:28] number of others I want to pause on this [4:30] for a second because you know they [4:31] sometimes people describe the phenomenon [4:33] of product market fit as you know it [4:35] when you see it the product is flying [4:37] off the shelves people are trying to use [4:39] it for things that you couldn't possibly [4:40] have imagined. It sounds like you had [4:42] this moment here where this is not ready [4:44] to be an audiobook narration tool but [4:46] you had people chunking up their you [4:47] know books into tiny segments to [4:50] actually be able to use it. That's when [4:51] you knew that you maybe you maybe had [4:52] something. Exactly. I was like this is [4:54] the and and and credit to to to my [4:57] co-founder P who has been the original [4:59] brain behind a lot of the innovations to [5:01] make the model actually sound human [5:03] sound narrative but it was the first [5:05] moment where we were seeing in the wild [5:08] people interacting with the book and [5:10] like okay the model is is actually good [5:12] performing at the level where you can [5:14] actually and so when was this moment in [5:15] the company's history when you November [5:17] 2022 but that's really fast to have a [5:22] shock and a demo No, like that. So [5:24] within a year you had people within a [5:26] year within a year and you know of [5:29] course uh everything else followed after [5:31] with the you know that you need the [5:33] product to actually interact with it [5:34] things were breaking the the [5:36] infrastructure wasn't ready. Um and then [5:38] we had a few other moments throughout [5:39] the first year or two where you could [5:41] feel the second moment that you [5:42] mentioned of like things falling through [5:44] not enough GPUs to actually serve the [5:46] clients and and and and things breaking [5:49] at all times but now it's stable. That's [5:51] good. It's kind of interesting also in [5:52] AI because it can be so compute [5:54] intensive. You do have these servers [5:55] melting down moment like I was trying to [5:57] use Claude 4 earlier this week and it's [5:59] just it's too popular. Um uh you know [6:01] there's a servers melting down moment [6:03] and so that's an interesting phenomenon [6:04] with AI in particular. So um you [6:06] referenced uh voice as the interaction [6:10] modality for computers and one thing I'm [6:12] interested in is we have had a vision [6:14] for a very long time of voice being the [6:17] main way we interact with computers and [6:19] not UIs. So, uh, hands up here. Who's [6:21] seen 2001 a space [6:23] odyssey? Okay, pretty good taste, at [6:25] least some of you. Um, so classic movie, [6:28] uh, and you know, open the pod bay door [6:30] is how you know there is an AI there, a [6:32] computer, and the primary way that they [6:35] interact the computer is via voice. That [6:37] movie was released in 1968, so 57 years [6:41] ago. We're still scrolling our phones [6:44] like um primitive uh uh creatures here. [6:48] When does voice become the primary way [6:51] that we're interacting with at least [6:53] some devices? Uh, you know, Alexa, turn [6:55] on the lights does not count. I'm [6:56] talking about, you know, slightly more [6:58] sophisticated computer use than that. [7:00] When does that happen and what's [7:01] required? The So, so I think there are [7:04] two breakthroughs that need to happen. [7:05] The first one is the model breakthrough. [7:06] We already are seeing in production use [7:09] cases where you can interact with the [7:10] technology and it feels human, feels [7:12] better than what existed in the in the [7:14] past. Um, but I think that curing test [7:16] for voice interaction of like where you [7:19] cannot really tell that it's AI or or or [7:21] and it's as good as the human. I think [7:23] this hopefully will make it happen this [7:25] year. Yeah, the voice actually feels [7:27] good enough. It's the rest of the [7:28] package that feels. Exactly. And then [7:30] you need to bring it to the distribution [7:31] side which I which I think we already [7:34] seeing in set of companies but I think [7:35] as we think about the this in next year [7:37] sometime in the next two years we'll see [7:39] that shift from being the early adopters [7:42] phenomena to actually uh um set of early [7:45] majority actually people adopting that [7:47] that technology and you know we we've [7:50] seen examples from uh we we work with [7:52] perplexity as a as as as one of those [7:54] where if they've built an assistant [7:56] where you can both ask it a question [7:58] through typing uh or speak to it and [8:01] actually get a response and it's such a [8:03] pleasurable and and and an amazing [8:05] experience. So, we already are seeing [8:06] the pattern of that forming. But like [8:09] you said, it's a a dream across the [8:11] movies for a long time. Uh I'm sure many [8:14] people also seen the Iron Man and the [8:15] Jarvis example and the assistant in that [8:18] piece, but it always was where you like [8:20] interact with the technology through [8:21] voice, not through typing. And uh and [8:24] hopefully that will happen this year. [8:25] Are there interesting aspects of the [8:28] nonword parts of verbal communication to [8:32] get right like when I think openi was [8:34] the first that I saw at least in chat [8:36] GPT in the voice mode maybe you guys had [8:38] it before that uh where you could [8:39] interrupt and that was part of the [8:41] features and the interrupting is super [8:43] janky but it's way better than nothing [8:45] and you realize just how the ability to [8:47] actually cut across and know when a [8:48] sentence is over is actually an [8:50] important part of voice conversations. [8:52] What are the other things that we might [8:54] you know people might think converting [8:56] this stream of text to voice as the [8:59] important part of it but actually maybe [9:00] there's some really surprising aspects [9:02] of this work that we as non-voice [9:05] scientists wouldn't realize. The [9:08] interesting thing is that you know for a [9:10] long time you are trying to make the [9:11] technology as perfect as possible and [9:13] initially when we started as well we [9:15] were trying to make voices speak as [9:16] fluent no mistakes as as possible for [9:19] the communities you work with which made [9:20] a lot of sense as we thought about the [9:21] customer service use cases healthcare [9:23] use cases financial service use cases [9:25] but then it turned out at some point [9:27] that you actually want to make it more [9:29] human for people to to to uh to engage [9:33] better. So what this means in practice, [9:34] we worked with this company in logistics [9:36] space where they were effectively [9:38] allowing people to book appointments [9:40] through calling the agent and they were [9:42] calling back confirming the appointment [9:44] and um and they created a voice which [9:47] instead of being perfect started doing [9:48] the s uh starting doing the the the [9:52] weird briefing and that suddenly [9:55] improved the experience by um by double [9:57] digit percent of people actually going [9:59] through the end of the call and enjoying [10:01] that a lot better. And now as you think [10:03] about a lot of our work, this is what [10:04] makes the voices so unique and so [10:06] different. Of course, this won't work [10:07] for for all use cases, but that [10:10] humanness, the imperfections are [10:12] actually the important piece for people [10:14] to to to engage and and interact. Um, [10:18] and then of course there are experiences [10:19] where the voice having some peculiarity [10:22] is what makes the experience. So [10:24] recently worked with with Epic Games [10:26] where we recreated um along with the [10:29] estate the voice of Darth Vader. Darth [10:31] Vader voice is so unique and how it how [10:34] it works across and you really want to [10:36] keep that across as the players [10:37] interact, but you also want to keep the [10:39] tone of the dark force across the [10:42] interaction where the He's a serious [10:44] guy. He's a serious guy. You don't want [10:46] to joke with him and you want to keep [10:48] that across and you want to do that [10:49] across millions of players with low [10:51] latency at all times. So how do you make [10:53] sure that that that that happens and [10:55] that was similarly like a good [10:57] combination of you need to make sure [10:58] that the voice is there that the the [11:01] character that you've built represents [11:02] that in the right way the responses are [11:04] short and snappy h so so very much the [11:08] voice uniqueness will will will will [11:10] will matter and trying to bring it [11:12] closer to human um it seems like the [11:15] three places I think that people are [11:17] most using voice today are uh smartome [11:22] device and activations just text input [11:25] into their phone but like filling out a [11:26] text field as opposed to anything more [11:29] um advanced or broader than that and [11:31] then phone trees. What will the next [11:34] modality that we don't have today at [11:35] least widely deployed that we'll start [11:37] using be? I think the interactive media [11:40] is definitely one that we are seeing [11:41] across the So the video game stuff you [11:44] the gaming example. Exactly. That's [11:45] that's one that how like the static [11:49] universe will start morphing into [11:50] interactive but also like even [11:52] traditional media. We've seen this use [11:53] case with the time where you could you [11:56] could um during the person of the year [11:58] you could read the article, you could [11:59] listen to the article, but you could [12:00] also ask the article questions. So you [12:02] could actually uh um interact with it, [12:05] learn more about the person of the year, [12:06] ask for who the other people of the year [12:08] are. So the interactive media is one big [12:10] category. The second one like currently [12:12] customer support is definitely something [12:14] that you kind of need to go through if [12:15] the product doesn't work. I think the [12:17] this will shift into the entire brand [12:19] experience and on boarding into product [12:21] experience where it's going to be for [12:22] front and center where if you are [12:25] interacting with the product you will [12:26] have the agent that helps you navigate [12:28] through the product learn where the [12:29] pieces are how to actually work with the [12:31] pieces and and and that will be [12:34] interesting shift of of of of [12:36] interacting with the with the software [12:37] as a whole. So are you saying that right [12:39] now customer service is something you [12:40] call in a frustrated way if something [12:43] goes wrong but maybe voice on boarding [12:45] is something that would help a lot of [12:46] people use products 100%. And I think it [12:48] will like the current experience where [12:50] you like call the the the phone tree, [12:52] the IVR treatment isn't very good, isn't [12:55] very crisp. Um, and now this will shift [12:57] into an experience that's actually [12:58] enjoyable. There is a context and [13:00] knowledge about what you are calling [13:01] with and it can actually help you go [13:03] through the next stage of what you're [13:05] actually trying to achieve. Whether it's [13:07] a a a booking appointment, whether it's [13:09] a a billing question, a refund, all of [13:12] that will actually be a pretty quick and [13:14] easy experience which you might want to [13:15] do to just get get through that, but [13:18] then it will extend also for you to be [13:19] able to understand what a product is [13:21] about, how do I use it, and um and how [13:24] to make most of it. Yes. Um all the [13:26] businesses in the in the audience are [13:28] thinking about we're certainly thinking [13:29] about um just the right ways to [13:31] incorporate AI. What do you think [13:34] businesses should be doing to uh [13:37] actually use the productivity gains from [13:39] AI in a on a day-to-day basis? It could [13:41] be voice, but it could be broader as [13:42] well. Yeah, the the of course there's [13:45] like in in in from what we see as a [13:47] pattern, there's those two wide [13:49] categories of of efficiency, margins, [13:53] how do we optimize on that side? Second, [13:55] growth, how you get more customers, how [13:57] you bring some of the efficiency to to [13:59] scale that. And you know the obvious [14:01] three patterns is where are the things [14:03] that are manual repetitive and then the [14:06] third one which isn't frequently as [14:08] obvious but where do you have a lot of [14:10] the data ID structure data to really uh [14:13] to really bring in the the the pieces in [14:16] there and [14:17] um as we work with the companies the [14:21] common pitfall is like how long should [14:23] it actually take like in from what we've [14:25] seen like within three months you should [14:26] be able to deploy a great proof of [14:28] concept prove the value get production [14:30] over the period of a year max kind of [14:32] this guy in less than a year built an [14:33] industryleading voice model. So yeah, [14:36] none of us have excuses for rolling out [14:37] our you know improved um customer [14:40] service awareness flow or you know [14:42] something um uh it's not allowed to take [14:44] longer than an industryleading voice [14:45] model. Um, okay. But you're saying with [14:47] the data that maybe people are too quick [14:49] to jump to where do I want to realize [14:51] efficiencies and instead they should [14:53] start from where do I have really [14:54] interesting data that other companies [14:56] maybe don't have and what are product [14:58] experience I could build with that even [14:59] if it's not improving efficiencies or [15:01] kind of lading up to maybe a bit more of [15:03] an exploration mindset when it comes to [15:05] building AI functionality. Exactly like [15:07] so it it will fall into one of those [15:09] categories like if you do care about [15:10] efficiency you know customer support is [15:12] the easiest one that we see repetitively [15:14] whether it's on text level on the voice [15:16] level so easy you have the knowledge of [15:18] of of what's happening in a company how [15:20] you can train the agent possible but [15:22] then you have those this other elements [15:24] where you have the IP you have the [15:25] content you have the story that you want [15:27] to convey as a brand as a company and [15:29] now if you if you have the backbone of [15:31] that that's your data that's something [15:33] that you can use to then build an [15:35] interactive experience something that [15:36] represents your brand, something that [15:38] people outside of the the ecosystem can [15:40] actually intersect with and and try out. [15:43] Yeah. Yeah. What's the most surprising [15:45] application of 11 Labs products you've [15:47] seen? [15:49] the I think the most [15:53] surprising. So I'll start with one which [15:55] was the the most heartwarming for us [15:57] which um when we started initially a lot [16:01] of people started um uh using 11 laps to [16:04] recreate when when they lost their voice [16:05] due to ALS to cancer used 11 laps and we [16:08] partner with them to bring that voice [16:09] back. Uh but a surprising version of [16:12] that happened last year where a [16:13] congresswoman in US unfortunately lost [16:16] her voice but wanted to continue working [16:18] in Congress and represent uh uh her [16:22] state and um and we worked with her to [16:25] recreate that voice and it was the first [16:27] time the AI voice was speaking in [16:29] Congress to despite the unfortunate [16:31] circumstances really show what's [16:33] possible and continue that work. So that [16:35] was that was one of the ones that was [16:36] like the inspiring ones and and [16:39] incredible ones. Um then we've seen [16:43] before any of the big big AI labs [16:46] created any of the voice piece there was [16:49] a a startup that built effectively an [16:51] integration between the voice 11 apps [16:53] element the LM stack and the vision [16:56] element where they were going around as [16:59] an effectively AI tourist companion that [17:01] would narate the world around you and [17:03] you could interact with that tourist and [17:04] that was back in like two years ago in a [17:06] hackathon uh somebody built it deployed [17:08] it it worked and um and then they're [17:11] building a company uh uh uh from that [17:13] which was which was amazing uh story [17:16] from the from the early days. Um and [17:18] maybe a fun one that we've seen more [17:20] recently. So we hosted a a hackathon and [17:23] people were exploring the future of [17:24] agents where agents start interacting [17:26] with each other. So imagine you have [17:27] your agent, you want to do a booking, [17:29] they call a um restaurant and another [17:33] agent answers that phone call and tries [17:35] to take that booking. and they've built [17:37] a demo where effectively the agents [17:39] would then swap from the AI voices into [17:42] their own language, start interacting to [17:45] become more efficient and then process [17:47] that on the on the other side and it [17:48] works. And sorry, what does the more [17:50] efficient agent language look like? It's [17:52] it's it's this protocol called Jibber or [17:55] Jibberlink uh where where effectively it [17:58] would switch from [18:00] using words as we know them into using [18:02] signals or frequencies. Oh, I see. to to [18:06] encode in the code effectively the [18:07] message. So you just hear the modem [18:09] connecting sound and off they go. That's [18:12] really funny. Um uh yeah, you can [18:14] upgrade to a higher bandwidth [18:16] communication format. You mentioned the [18:17] big AI labs. Um they love talking about [18:20] how applications and kind of more [18:23] specific things are roadkill uh you know [18:26] and will be crushed underfoot. Um how do [18:28] you avoid being crushed underfoot by [18:31] much bigger labs? Yeah. the you know I [18:34] think the one thing that makes 11ab [18:36] special is that we are trying to do both [18:38] the the research work so the [18:39] foundational models and the products [18:41] with one focus on voice but really [18:43] trying to do both sides of the equation. [18:46] Uh so on the I I think the the way we [18:49] think we can win is the first is keeping [18:51] the focus on voice and really double [18:53] downing on on building the best research [18:55] models. um uh my co-founder has been [18:57] able to assemble some of the best [18:58] researchers in the space to um across [19:01] audio who we think can actually produce [19:03] those next breakthroughs. And maybe the [19:05] big difference invoice to other models [19:08] at least for now is that it's more about [19:09] the model breakthroughs in the data [19:12] rather than compute and scale. So you [19:14] can actually still benefit from doing [19:16] the foundational model work to to bring [19:18] it to another level. That might shift [19:20] over time as we move from single [19:22] modality into multimodality which we are [19:25] also working on. Um but but currently [19:27] it's definitely true for audio and voice [19:29] you need to do those breakers. And then [19:31] you have this other piece where we as a [19:32] company spend a lot of time is you [19:34] beyond research you need to actually be [19:37] able to use that research whether it's [19:39] the audiobook example and the narration [19:41] site or building the voice agent with [19:43] having the metrics of how it performs [19:45] bringing the knowledge base intact [19:47] building the integrations intact and [19:49] that product layer is where we spend the [19:51] the the majority of our product cycles [19:53] to really be able to um to build that [19:55] application side to to the customers and [19:58] kind of those to ultimately do something [20:01] with AI like the act. Exactly. And it's [20:03] like we we believe you need to do both [20:05] to really be able successful where um by [20:07] doing research we can bring it to our [20:09] products in a ways that others cannot [20:11] and then by building product we meet [20:12] customers where they are and work with [20:14] them together. You you said it's not [20:16] just all about scale and throwing more [20:18] GPUs at it. It's about you know the [20:19] better techniques and everything like [20:21] that. I remember Greg Brockman from [20:22] OpenAI saying that that can sometimes be [20:25] true, but basically in AI, every [20:28] approach has worked better when you just [20:30] throw more scale at it. And so, you [20:32] know, we invented transformers and those [20:33] were awesome then just like the more [20:34] scale you put on them uh uh the better [20:36] the results got. Do you think that is [20:38] true here or is there kind of some [20:39] reason it's not true in voice? I think [20:41] over time increasingly so like as we fix [20:44] some of the first foundational problems [20:47] that happen with voice, it will become [20:49] true. But if you don't and you just [20:50] throw scale, you'll still get the bad [20:52] output. So in in in like in in time [20:55] 100%. And especially like one of the the [20:58] main paradigs we we see is that you know [21:01] we spoke about this um you you mentioned [21:03] um uh the interopability. So you can do [21:06] it in two ways and one of the places [21:09] where we started as a company was [21:10] texttospech and then over time we [21:12] extended that to speech to text and now [21:14] build an orchestration where you can [21:15] combine speech to text lm and text to [21:18] speech and the entire orchestration to [21:20] make this possible. Um so these are like [21:22] three separate models that kind of work [21:24] uh and and and we sync them together. [21:26] But that will shift from those cascaded [21:27] approach into that multimodel approach [21:29] where you train those models together [21:31] and in those cases the scale will start [21:33] becoming more more important and where [21:35] we're investing a lot of time. Um we're [21:37] currently in a societal equilibrium [21:39] where if you want to do something that [21:41] involves high trust, say you know you're [21:43] doing some online banking and you want [21:44] to say okay I want to transfer all my [21:46] money out of my account. The bank will [21:47] say ha that's suspicious. We should [21:49] verify that. we're going to get on the [21:50] phone with this person and make sure we [21:52] hear something that sounds very [21:54] convincingly like them and then we will [21:56] allow them to transfer all the money out [21:58] of their account. That made sense as an [22:00] equilibrium for many decades until well, [22:04] you just heard it. What is the new [22:06] equilibrium when it comes to trust and [22:08] voice? because the deep fake based fraud [22:12] is not just voice. We're also seeing it [22:14] with kind of text and I guess now video [22:16] but just what do you think that new [22:18] equilibrium looks like? How do we even [22:20] verify identity? No, it's it's it's it's [22:22] a great question and and and one where [22:24] we we've spent a lot of time thinking [22:25] through like how as you think about [22:28] releasing the technology, how do you [22:29] build the right safeguards to protect [22:31] against the bad actors that will try to [22:32] abuse it for for those cases and [22:35] unfortunately with the the shifts those [22:36] bad bad actors will be there. So the the [22:38] the main thing that we started from the [22:41] beginning of the company is the the [22:42] whole notion of provenence and [22:43] transparency where for all the content [22:45] that generated with 11 laps you can [22:47] trace it back to who generated it and [22:49] that that supplemented with the [22:51] moderation uh on the voice and and and [22:53] text level. But I actually believe that [22:55] you guys can control somewhat how 11 [22:58] Labs voices are used maybe. But also [23:01] just this technology Pandora's box is [23:04] out there and I'm curious how you think [23:06] societywide we adapt to the fact that [23:08] presumably you think now just very [23:10] convincing voice replicas are out there [23:12] and just how do we what do you think the [23:15] new equilibrium looks like? No, exactly. [23:17] And and so so one thing that we started [23:19] doing is as you build that that that [23:20] that model is we build it into a [23:24] publicly available classifier where [23:25] everybody from the space could initially [23:27] do it just on 11 laps get information is [23:30] that audio generated with 11 laps or [23:31] not. Oh so you watermark the voices [23:33] effectively like classifier approach and [23:35] then over time we started working with [23:37] other companies to extend this of train [23:40] it on open source models on other [23:42] models. Um so like University of [23:43] Berkeley where they would try to extend [23:45] that over. Uh but then that's still not [23:48] optimal because you will still have new [23:50] open source models, new commercial [23:51] models. It's always a catch-up game. You [23:53] cannot do that reliably. Um so as as as [23:57] to your point of question, I think the [23:59] future will actually flip it from trying [24:01] to detect for AI content. You'll [24:03] actually detect for human content. So [24:04] you'll have like encoded on device level [24:06] on a phone on a computer on a on a on a [24:09] screening like it's this person speaking [24:13] that's this person's device encoded and [24:14] decoded on the other side this is that [24:16] person but that so that's the easy okay [24:18] great if if everybody adopts it but then [24:20] there's a second layer which I think is [24:22] even more interesting where given that [24:25] peripheration of agents where everybody [24:26] will have their agents or multiple of [24:28] them you will want to have in [24:31] information that this is an [24:32] authenticated AI agent. So in this case [24:35] there was your voice which we created [24:36] together. You want to know like yes this [24:38] was a voice created with my permission [24:40] with my consent and you want that [24:41] information to be uh out there. So then [24:43] you can have a watermark with that [24:44] information carried across. second layer [24:46] and then the third layer is everything [24:48] else. If it's not watermarked and [24:50] authenticated or real, that's suspected [24:53] as AI content and you can deploy that [24:55] classifier approach on that content as [24:57] well to like is it AI or not? But uh but [25:00] by default everything that's not [25:02] authenticated should be should be tagged [25:04] as AI. That's very interesting. I hadn't [25:06] thought about that. I mean it kind of [25:07] reminds me of how malware detection [25:08] works where you think about like [25:10] antivirus on your computer. It's looking [25:12] for, you know, if you go to run an [25:13] executable file on your computer, uh the [25:16] antivirus detection is looking at the [25:18] the code signature of that software and [25:21] saying, "Huh, we've seen this before and [25:22] it's known bad." And if you look at how [25:24] antivirus detection works, it's not [25:26] perfect, but it's actually better than [25:29] nothing and and pretty reasonable. And [25:31] so we end up with kind of similar [25:33] malware detection for um uh AI generated [25:37] voices. That's kind of interesting. [25:39] That's right. or or there's some [25:40] interesting ideas from HTTPS protocol as [25:43] well of like how you have a trusted set [25:45] of of of um of parts of entities that [25:48] are part of it that can like effectively [25:50] extend that this is authenticated [25:53] message that you send across and same [25:54] could happen with AI voices that all the [25:57] companies that opt in to have the AI [25:59] watermarked authenticated voices or [26:01] encoded real voices get that [26:04] verification the same way like HTTPS [26:06] would and then everybody that isn't just [26:08] Yeah. Doesn't I thought it was [26:10] interesting you talking about [26:11] permissioned use of your own voice and [26:13] almost your your your agent out there. [26:16] How should I be thinking about using my [26:17] AI generated voice? Like am I am I [26:20] missing opportunities to use now that [26:21] especially you guys have built one? Um [26:23] am am I missing opportunities to use [26:25] that for something? How should I be [26:26] using my AI voice? I think the one place [26:29] where you [26:31] know maybe two quick places. The first [26:34] one is definitely those interactive use [26:35] cases like how could you have an an [26:37] agent that personally welcome people to [26:39] Stripe. Personally welcome people to [26:40] Stripe or ask your questions and get [26:42] answers from all the past speeches [26:45] content that you created. But the second [26:47] one and you briefly actually showed it [26:49] as well is you you I think this will be [26:51] this beautiful moment in the in the [26:53] future where suddenly all the language [26:55] barriers will break because your voice [26:57] will now be understandable in all the [26:58] languages. You'll be able to travel to [27:00] those countries. Let's hope the hardware [27:02] moment happens as well where you can [27:04] actually communicate with everyone with [27:06] the same inonation and motion carry that [27:08] message across in a way that kind of [27:10] wasn't wasn't possible and hopefully the [27:12] movies in Poland will also get solved by [27:14] that time but uh but that would be a [27:16] amazing one. That's true. The um the [27:18] language one actually maybe feels like [27:19] the biggest thing where you know there [27:20] was a long time where human translation [27:22] was the only viable thing on the web and [27:24] you know people went through all these [27:25] extraordinary efforts and then kind of [27:27] in the last few years the machine [27:29] translation has gotten good enough and [27:30] so now a lot of the translations we [27:32] consume on the web are just me machine [27:33] translation they're good and kind of [27:35] similarly we'll presumably see way more [27:37] foreign language content just period [27:39] because the 100% all the employees of [27:42] Stripe internationally will be able to [27:44] to hear the native one. Yeah. Yeah. [27:46] That's interesting. Right. the all hands [27:47] in uh and you get this induced demand [27:50] effect where yeah there's much more of [27:51] it than there would have otherwise been. [27:53] Um very interesting. Last question. Um [27:55] you are a big fan of Disney and uh I'm [27:58] curious how you've been inspired by [28:00] Disney by Bob Iger. What what parts of [28:02] the Disney playbook have you [28:03] incorporated as you as you build 11 [28:05] Labs? the the I think there's just so [28:09] many incredible things that Disney [28:11] created over the years of course but the [28:12] one that is the the the very very easy [28:15] to say is definitely the storytelling of [28:18] across the entire company across all the [28:20] work that they are trying to to to do [28:22] that you feel like you're part of that [28:24] story uh from like animated movies to [28:27] like app to to to Star Wars where all of [28:30] them feel timeless in many ways uh in [28:33] many ways we hope to as we think about [28:35] creating that technology we are creating [28:36] technology for its own sake, but trying [28:39] to figure out how it is able to tell the [28:41] story to all the different audiences [28:42] that we interact with and and how people [28:44] can be part of that story too. And and [28:46] maybe the second one which I really like [28:48] about Bober specifically which he [28:50] brought into Disney was that concept [28:52] that [28:54] everybody across Disney would need to [28:57] dress up as one of those fluffy Disney [28:59] park uh uh animals or or characters and [29:03] you would interact with the guests to [29:05] really understand how that brand works, [29:07] how they perceive it, how they who the [29:09] users are and uh and this this really [29:12] resonated with us where we are trying to [29:13] stay very close to users across the [29:15] entirety of the company, whatever you [29:17] you part of of the company you are [29:19] building. And so what's your version of [29:20] dressing up as Mickey Mouse? [29:24] The the one that I've I've done few [29:26] times now is customer support and [29:28] staying very close to answering the [29:30] tickets. Yeah. And um and you can learn [29:33] so much. I think you do that too [29:34] actually where you invite a person into [29:36] the cult, tell them about the paints. [29:38] Yes. You you you learn surprisingly much [29:40] from those. So maybe like Disney, you [29:42] should understand what is it that your [29:44] company truly differentiates on and is [29:46] exceptional at and then make sure all [29:48] people working at the company are [29:49] staying incredibly close to that. [29:51] Exactly. Manny, thank you so much for [29:53] joining us. Thank you so much. Cheers. [29:55] Thank you.