Anthropic Tops $4 Billion ARR as Coding Triggers AI Hypergrowth

编程驱动AI超高速增长，Anthropic ARR突破40亿美元 CEO Dario Amodei指出前沿模型单个本身就极具盈利能力，并将巨额前期资本支出比作风投式研发投资的循环，驳斥了AI是财务无底洞的公众论调。 在本次探讨中，Anthropic首席执行官Dario Amodei概述了在编程与企业级应用推动下公司的飞速增长，同时将AI市场与高度差异化的云计算行业类比，消解了市场对AI同质化大宗商品化的担忧。他阐释了单个AI模型的底层盈利能力、转向强化学习以绕过数据墙的路径，以及构建“AGI化”产品界面所面临的运营挑战。最后，Amodei强调了推行审慎、均衡的安全监管的必要性，以在聚焦并确保技术飞速发展的同时，不阻碍医学和科学领域的重大突破。 尽管曾受行业质疑，但得益于深厚的信任和过往共同共事的经历，Anthropic由七位联合创始人均分股权的独特创立架构最终取得了成功。 (0:00) 由于社会传播效应，编程成为Anthropic模型增长最快的应用场景，软件工程师在集成Claude Code等工具时遇到的阻力最小。 (2:52) 尽管高管们认识到其中蕴含着数十亿美元的潜力，但传统企业的采纳仍面临来自组织惯性和员工培训阻力的瓶颈。 (2:52) 诺和诺德（Novo Nordisk）使用Claude起草临床研究报告体现了生产力的指数级提升，将人力投入从原本的九周骤降至五分钟的AI生成。 (5:38) Anthropic争取最高达2亿美元上限的国防与情报合同，是出于捍卫民主国家的使命感，而非为了轻易获取营收。 (5:38)

Anthropic CEO Dario Amodei argues that frontier AI development follows powerful scaling economics that can transform industry workflows, though realizing this potential requires navigating enterprise friction, novel interface paradigms, and high-stakes safety risks. Frontier Model Economics and Scaling Law Realities (16:10) Amodei rejects the notion that frontier AI is a commoditizing cash burn, noting that individual models can be highly profitable on a per-program basis—such as a $100 million training run yielding $200 million in revenue with payback periods comparable to standard 9-to-12-month customer acquisition cycles. Consolidated company losses appear severe primarily because firms are simultaneously initiating exponentially larger R&D cycles for successor systems. Anthropic's annualized recurring revenue reportedly crossed $4 billion, mirroring non-linear jumps in economic value that occur when compute-driven scaling elevates model capability from undergraduate to advanced research levels. Amodei contends that AI APIs will mirror cloud computing rather than collapse into generic commodities. Because frontier models exhibit distinct behaviors, non-deterministic outputs, and emerging personalization, switching between models creates substantial disruption for enterprise users. The competitive field remains restricted to three to six well-capitalized frontier labs, where proprietary defensibility rests on collective operational know-how and engineering implementation rather than simple algorithmic tweaks. Diffusion Bottlenecks and Interface Shifts (2:52) AI adoption has diffused fastest in software engineering, where developers sit adjacent to model creators and can integrate tools like Claude Code immediately. In contrast, adoption across broader enterprises faces institutional friction. Amodei observes that while executive leadership understands the multi-billion-dollar potential, altering workflows across workforces of 10,000 to 100,000 employees creates major lag. Still, tangible productivity leaps are emerging in specialized domains; a collaboration with Novo Nordisk saw clinical study report drafting drop from nine weeks of manual effort to five minutes of generation followed by several days of human review. Realizing agentic automation requires rethinking product design. Amodei characterizes modern AI user interfaces as stuck in a 'horseless carriage' phase that relies on 1970s-style text prompts. As models become autonomous agents executing end-to-end work, users face an 'impedance mismatch'—they want hands-off operation, yet diagnosing failures requires tedious inspection of granular steps. For high-stakes, variable tasks like preparing personal tax returns, which Amodei projects AI could handle by 2026 or 2027, the central barrier remains algorithmic verification and interfaces that allow models to inspect their own output. Overcoming Technical Walls and Cognitive Vitalism (20:47) Dismissing claims that AI progress faces an unbreachable data wall, Amodei points to reinforcement learning (RL) and chain-of-thought methods as continuations of trial-and-error learning paradigms. He views continual learning—addressing the current limitation where a model behaves like a smart coworker perpetually stuck on their first five minutes on the job—as an engineering problem solvable via in-context learning scaling up to 100 million tokens or dynamic weight updates. Amodei equates skepticism about machine reasoning and discovery to 19th-century 'vitalism,' the discredited belief that living biology relies on forces absent in inanimate matter. Defining discovery as a continuum that ranges from everyday problem solving to breakthrough research, he notes models already make novel contributions, such as diagnosing rare medical conditions missed by human practitioners. While AI errors can be deceptively articulate rather than visibly erratic like human mistakes, Amodei argues that algorithmic grounding and citation mechanisms are systematically reducing hallucination rates. National Security and Balanced Safety Governance (5:38) Anthropic balances open commercial access with defense work, maintaining a Department of Defense and intelligence community contract with a ceiling of up to $200 million. Amodei frames working with democratic governments as a matter of principle rather than commercial convenience, arguing that procurement hurdles make defense revenue far more labor-intensive to secure than commercial venture deals. Preparing for scenarios where data centers could house 'a country of geniuses' within one to three years, Amodei warns of severe geopolitical, alignment, and societal stability risks. Rejecting calls to halt technological development as impractical in a competitive global landscape, he advocates for an insurance-style approach to regulation—modeled after banking oversight—that prioritizes transparency in safety testing. He argues society should accept modest reductions in top-line growth (such as accepting 9% instead of 10%) to secure robust safeguards against catastrophic failure. Anthropic bets that maintaining technical and operational readiness for rapid, non-linear AI advances is essential, but the path forward hinges on whether industry and regulators can establish robust safety guardrails before models reach transformative scale.

Anthropic首席执行官Dario Amodei指出，前沿人工智能的巨额研发投入本质上是由高回报驱动的复合扩张，行业正从代码自动化向生物医药等实体经济扩散；面对潜在的颠覆性风险，行业需要在不阻碍技术进步的前提下建立精准的透明度与安全防线。 模型经济学与扩展定律驱动 (11:10) 针对外界对前沿模型训练巨额资本支出（CapEx）形同黑洞的质疑，Dario Amodei认为单一模型的单位经济学实际上具备盈利能力。例如，一家公司若在2023年耗资1亿美元训练模型，并在2024年获得2亿美元营收，该项目本身即已盈利；整体账面上呈现的亏损不断扩大，是因为企业在按指数级曲线同时投资下一代研发成本更高的新模型，其投资回报周期通常为9至12个月。他指出，随着算力与数据投入按十倍规模递增，模型能力将实现跃升，并直接转化为医药等高价值客户的非线性经济回报。 关于所谓的“数据墙”瓶颈，Amodei指出行业正通过强化学习（RL）实现路径转换，这一模式从基础LLM的模仿学习转向环境中的试错与推理探索。对于商业竞争，他反驳了API正在变成廉价大宗商品的观点，强调AI模型具备独特的非确定性输出和“个性”，其差异化程度远高于云计算基础设施。Anthropic将自身定位为平台企业，目前年化经常性收入（ARR）已突破40亿美元，并深度服务企业级市场。 技术扩散路径与实体企业摩擦 (2:52) AI在各行业中的采纳速度取决于“社会扩散”的阻力大小。Amodei分析称，编程是目前增长最快的应用场景，主要因为软件工程师在技术和社群上与AI开发者高度重合，能在数小时内测试并整合Claude Code等工具。而在拥有数万或数十万员工的传统银行、保险公司或零售巨头中，即便最高管理层明确认可AI的巨大价值，重塑既有工作流与全员培训仍面临显著的组织惯性。他建议大型组织可设立独立的突击团队（strike team）快速开发原型，以规避阻力。 在高度受监管的实体行业中，AI的效率跃升初显成效。例如在与Novo Nordisk的合作中，Claude将原本需要人工撰写约九周的临床研究报告初稿生成时间压缩至约5分钟，后续仅需专业人员数天审核。在政企合作方面，Anthropic与美国国防部及情报界签署了最高达2亿美元的合同；Amodei对此回应称，这并非商业妥协，而是出于捍卫民主体制的信念，同时公司也对政府权力的滥用保持警惕。 认知认知反驳与持续学习破局 (33:01) 面对AI无法实现自主发现或缺乏持续学习能力的质疑，Amodei将其类比为19世纪认为生命由独特非物理力量驱动的“活力论”（Vitalism），批评部分观察家陷入了认为人类认知具备不可复制特质的认知活力论。他认为“心灵不论由何种材质构成都是心灵”，科学发现本身是一个连续统，从解决日常小问题到诺贝尔奖级别的突破仅是程度差异，目前大语言模型已在复杂医疗诊断中发现人类医生忽略的病症。针对高精尖医疗建议，他援引诺贝尔奖得主学者的观点称，除前1%的顶尖人类医生外，LLM给出的建议往往比其余99%的从业者更一致、更优质。 针对当前模型缺乏长期记忆、如同“每五分钟就重置经验的超级同事”这一现状，Amodei认为可通过扩展至1亿token的上下文窗口、上下文学习或动态权重调整来逐步突破。关于模型幻觉，他强调其正通过算法优化与引用来源对齐得到缓解。AI出错的方式与人类不同，人类在认知失常时通常伴随言语不清等显性特征，而LLM在产生幻觉时依然能保持优雅连贯的语气，这要求用户界面与交互模式进一步适应。 敏捷产品构建与界面交互重塑 (44:13) 底层模型能力的剧烈变动颠覆了传统软件的产品管理机制。Amodei坦言以往标准的六个月路线图已不适用，团队必须在底座快速移动的环境中以极紧凑的发布周期进行内部测试，适应每周都在解锁全新可能性的持续探索状态。他警告外部开发者切勿开发单纯修补当前模型短板的“外壳型”（wrapper）产品，因为一旦下一代模型发布，这类中间件功能就会被原生能力彻底覆盖。 在人机交互方面，现有的AI界面依然停留在1970年代终端文本框式的“无马马车”拟物阶段。随着AI逐渐演变为执行端到端任务的自主代理，人类角色从创作者转变为审核者。这导致了一种设计层面的阻抗失配：用户既希望体验全自动的省心流程，又在系统出错时不得不深入排查每个细节；若必须时刻监控代理，其效率将低于手动执行。因此，创造原生支持自主代理的新型交互范式成为释放技术价值的核心瓶颈。 组织护城河、AGI愿景与审慎监管 (0:00) Anthropic在创立初期由7位联合创始人平分股权，依靠长期共事的信任基础与妹妹Daniela Amodei（负责运营执行）的分工维持文化统一。在激烈的人才争夺战中，Amodei指出简单的算法构想极易被外部独立复制，真正的防线在于构建高度复杂工程对象的集体专业技能与落地细节。公司采取了透明沟通与严格的信息分权隔离制度，依靠长远使命感和期权吸引力维持了行业最高水准的人才留存率。 Amodei预测，未来一至三年内数据中心将容纳相当于“整个天才国家的智力规模”，带来科技爆发的同时也伴随严峻的国家安全与对齐风险。他反对全面暂停AI研发，认为在地缘政治竞争与庞大资本流动的现实下强行叫停并不具备可行性，主张通过支付适当的增长溢价（如接受经济增长率从10%微调至9%）来换取充足的安全保障。在立法层面，他支持以加州SB1047修订版及SB53为代表的兼顾透明度与技术合理性的框架，将AI监管类比于现代银行体系规范，在保障安全底线的同时维持创新活力。 Dario Amodei描绘了前沿AI从高密度算力投入走向全行业转型的路径，但在模型幻觉审查、自主代理的人机交互范式以及企业内部组织重塑等方面，技术落地仍存在实质性摩擦。未来几年，在万亿资本与地缘政治竞争的推力下，如何在防范系统性风险与实现高速生产力爆发之间维持平衡，仍是行业必须解答的核心问题。

[0:00] Advisors: seven cofounders is a disaster [0:00] Dario Amodei: the thing that no one else sees [0:00] Dario Amodei: total and complete trust [2:52] Dario Amodei: societal diffusion [5:38] Dario Amodei: defend democracies [7:18] Dario Amodei: virtual coworker [11:10] Dario Amodei: the models have this capitalistic impulse [11:10] Dario Amodei: clean the window and let the light shine through [26:04] If I'm sitting in a room with 10 people, does that mean I've been commoditized? [33:01] super smart virtual coworker who started five minutes ago, but they remain the coworker that started five minutes ago [33:01] Dario Amodei: matter of degree, not a fundamentally different matter [35:34] Dario Amodei: a mind is a mind no matter what it's made of [41:13] John Collison: we need to invent slurring for LLMs [47:54] Dario Amodei: horseless carriage era [54:43] Dario Amodei: a country of geniuses in the data center [57:11] Dario Amodei: I don't wanna stop the reaction. I wanna focus it. [62:03] Dario Amodei: Perhaps I have too much pride in my own writing.

[0:00] I'm excited to finally learn [0:02] what is it like to start a company with your sibling? [0:04] I don't know why you're asking me [0:05] that question, because you know. [0:07] It's like the models want to learn, [0:08] the models want to be [0:09] extraordinarily successful in the market. [0:11] Yes, right, in addition to having this learning impulse, [0:13] the models have this capitalistic impulse. [0:16] Sometimes people think of the API business [0:18] and they say, "Oh, it's not very sticky," or, [0:20] "It's gonna be commoditized." I run an API business. [0:21] I love API businesses. No, no, exactly, exactly. [0:24] I think we're gonna be in a world where [0:26] the models will make mistakes much less often than humans, [0:31] but they'll be stranger mistakes. [0:33] So we need to invent slurring for LLMs. [0:36] And that should correctly pour it. [0:38] Oh, wow. [0:38] Dario is CEO of Anthropic, [0:40] one of today's frontier AI labs. [0:42] He's gone from being an AI researcher just a few years ago [0:45] to now running one of the world's [0:46] fastest growing businesses. [0:48] Cheers. Cheers. [0:50] So, I'm excited to talk a bit about [0:51] the Anthropic business. [0:55] You studied physics and computational neuroscience. [0:59] Yes. [1:00] You then worked at Baidu, then Google Brain, [1:04] then OpenAI, and then started Anthropic. [1:07] Yes. [1:07] And we'll get into the Anthropic business, [1:08] but I'm excited to finally learn [1:12] what is it like to start a company with your sibling? [1:13] I could ask the same question of you, [1:15] but you know, it's almost like there's two things [1:18] you need to do when you're running a company. [1:19] You need to operationally execute [1:23] and you need to have a good strategy [1:26] and kind of see the most important thing [1:27] or the thing that no one else sees. [1:30] And so my job is the second, and Daniela's job is the first. [1:33] And we're both good at the things that we do. [1:35] And so I think it's allowed us each to spend [1:40] most of our time on the thing that we're best at. [1:43] Presumably there's something about the trust side [1:44] of things as well, where cofounder teams in general, [1:47] in tech, I guess, and AI as well. [1:50] I mean they're unstable pairings [1:51] and just having someone where you've [1:52] a long running and deep trust— [1:54] Yeah, yeah, where you have just total and complete trust. [1:56] I mean, I think even beyond that, [1:58] Anthropic has seven cofounders. [2:00] When we founded it, basically, the advice from pretty much [2:04] everyone was like, "Seven cofounders is a disaster. [2:07] The company will fall apart before you know it. [2:08] Everyone will be fighting with each other." [2:11] There was even more negativity on my decision [2:13] to give everyone the same amount of equity. [2:16] But what we found, and I think it was because, [2:20] obviously, me and Daniela are siblings, [2:21] but then all seven of us, [2:23] some of us knew each other for a long time [2:25] or had a history of working, not just knowing each other, [2:27] but working together in the past, [2:30] and I think that really allowed us [2:32] to always be on the same page. [2:34] And I think especially as the company grows, [2:37] the idea that you have seven people [2:40] who really carry the values of the company [2:43] and project them to a wide set of people, [2:45] it allows you to scale the company to a much larger size [2:49] while holding onto the values [2:51] and the unity that we have. [2:52] So I wanna ask about the Anthropic business, [2:55] because again, it's an incredible story [2:57] where it was reported recently that you'd blown through [2:59] $4 billion in ARR, and so it's a lot of discussion correctly [3:03] about the technology that you're developing, [3:07] but also, this is just one of the [3:08] fastest growing businesses in history. [3:11] And so I wanna talk a bit about the AI market, [3:15] and maybe the place to start is, [3:18] what is everyone doing with AI? [3:19] Like, there's coding, there's [3:20] customer service work, but like, [3:22] where does all this revenue come from? [3:23] Yeah, there's a wide range of things, [3:25] and it's kind of changed over time. [3:26] I would say definitely [3:29] the application that has grown the fastest, [3:33] although it's not [3:35] very far from the only application, [3:37] we have a wide range of them, is definitely coding. [3:40] And my theory on why it's grown so fast, [3:43] other than that we focused on coding [3:45] and the models are good at coding, [3:46] it's actually really a statement about [3:49] kind of societal diffusion. Which is that [3:51] if we look at today's AI models, I think in every area, [3:54] there's a huge overhang in terms of what they could do [3:58] compared to how they're [3:59] actually being deployed today. [4:01] There's some friction, [4:03] people at large enterprises [4:04] are not familiar with the technology. [4:06] I look at what a bank does [4:09] or what an insurance company does, [4:11] and there's huge potential, [4:14] even if the model stopped getting better, right? [4:16] Even if we stopped building products on top of the model, [4:18] there's still huge, billion dollar potential [4:21] in an individual enterprise. [4:23] And often, the CEOs of companies that I talk to [4:26] understand that perfectly well. [4:27] But if the company is a 10,000 or a 100,000 person company. [4:31] Companies that size, [4:32] they're set up to operationally do a certain thing [4:34] a certain way, and it takes time to change them. [4:37] But in code, the people who write code [4:41] are very socially and technically adjacent [4:44] to the folks who develop AI models. [4:46] And so the diffusion is very fast. [4:47] They're also the kind of people who are early adopters, [4:50] who are used to new technology. [4:51] And so I think the big growth in code, [4:55] I would say the biggest cause of that [4:57] is just that the people doing it [5:00] and the startups devoted to it are fast adopters [5:03] who understand the technology super well. [5:06] But it's by no means limited to code at all. [5:11] If you look at, there are a bunch of companies [5:13] that do things like tool use. [5:15] There are, as you mentioned, customer service. [5:17] You know, we work closely with companies like Intercom. [5:21] We're starting to see some things on the biology side. [5:26] We're working both with [5:28] pharmaceutical and healthcare companies, [5:32] and we're working on the side of kind of [5:33] basic scientific research. [5:35] We work with companies [5:36] like Benchling, for example. [5:38] But we also work with [5:39] some of the very large pharma companies. [5:41] There was something done a while back [5:43] where we worked with Novo Nordisk [5:46] to write clinical study reports. [5:47] So clinical study reports are like, [5:49] you've done a clinical trial, [5:51] and then you would kind of write up the results, [5:53] and it's like, these are the adverse events, [5:55] these are the statistics. [5:56] And the clinical study report [5:58] normally takes like nine weeks. [5:59] Well, Claude could do it in like five minutes, [6:01] and then it took a human a few days to check it. [6:04] And so you can really see the opportunity for acceleration, [6:06] and as the models get better, you know, [6:08] they'll reach into the deep research as well. [6:10] So, I guess a way to summarize it would be to say [6:13] that kind of code is out in the lead, [6:16] but we see a long tail of quite a lot of other stuff, [6:19] including some [6:21] some very, very significant use cases. [6:23] I think code is maybe an early indicator, [6:27] like a premonition of what's gonna happen everywhere else. [6:30] It's the same exponential. It's just faster. [6:32] It's happening faster. [6:33] Right, so there are many places [6:35] where there's significant AI uplift, [6:36] but engineers are used to adopting... [6:38] Like, you think about Hacker News [6:39] and people arguing over the best tools, [6:40] people are passionate about adopting new tools. [6:42] Like two hours after we released Claude Code, [6:45] there's some person out there who's [6:49] tried 10,000 different things with it [6:51] and plugged it into all the frameworks, [6:53] and Twitter forms one opinion after two hours [6:56] and then revises an opinion in two hours. [6:58] And you think of the speed of that [7:00] as compared to the speed that [7:02] a pharmaceutical company can use it in research, right? [7:05] Or that traditional retail company. [7:07] And we wanna bring everything— [7:09] Some of the biggest benefits in the world are [7:12] touching the physical economy, and we wanna get there, [7:15] but it just intrinsically does not happen at the same speed. [7:18] How do you decide which verticals to do yourself [7:21] versus which to allow platform... [7:22] Like, you have Claude Code, and obviously, [7:24] there's also platform companies like [7:26] Windsurf and Cursor and everyone like that. [7:29] You launch Claude for Financial Services. [7:31] Presumably there are other verticals where you say, [7:32] "Well, we're not building a tool there." [7:35] we have things like Claude for Enterprise, [7:37] which is not a vertical, but [7:38] a general play to go with enterprise. [7:42] I think the way we like to think about it, [7:45] I think we think of ourselves as a platform company first. [7:49] The analogy here would be [7:51] maybe clouds or something. [7:52] If you think of a really large platform business of, [7:55] the size we're trying to get to [7:57] in hopefully small number of years, [8:00] there are a number of reasons why you would also [8:02] want to have things that are first-party [8:04] and some verticals [8:06] end up being more first-party heavy. [8:08] One is when you want to have direct exposure to the users. [8:13] The end user gives you some sense of [8:17] how exactly are they using it, [8:19] what are they most looking for? [8:20] If you're a pure platform [8:21] and you don't have that direct connection, [8:23] you can be disadvantaged in various ways. [8:25] It's hard to build the best products. [8:26] Yeah, it's hard to build the best products. [8:27] It may even be hard to know [8:30] where the model really, really needs to go, right? [8:32] People say things like coding, but like, [8:35] there are many models [8:36] that seem to be good at coding, [8:37] but they aren't good in the way [8:39] that's actually relevant, right? [8:40] We've actually managed to make Claude good in a way [8:43] that's relevant to what people actually use. [8:46] I think that's one reason. [8:49] Another reason goes back to the large enterprises, [8:52] where building on an API, [8:55] sometimes it's more challenging [8:57] for a more traditional company to do that. [8:59] And you need to give them something [9:01] that's a little bit easier to use, [9:02] either a kit to help them build things, [9:04] or you need to give them an app. [9:06] Enterprises have also liked Claude Code, [9:09] and we're gradually developing Claude for Enterprise into, [9:12] what we call a virtual coworker. [9:14] I find it hard to picture Anthropic developing [9:16] Claude for oil and gas exploration, [9:18] and why is that? [9:21] Why is it that you find it hard to imagine? [9:23] Yeah, or I mean, maybe in fact [9:24] it's the next launch, but... [9:26] Yeah, we're not currently working on [9:28] Claude for oil and gas exploration. [9:30] I would draw a distinction between things we just, [9:33] things we just don't allow, right? [9:35] Things that are illegal, [9:37] or things like that. [9:38] And there are a number of use cases that it's like, [9:41] okay we're a platform, [9:42] people are gonna do a bunch of things. [9:43] We're just not passionate about it. [9:44] But like, we're not passionate about it. [9:46] We're not gonna go out [9:47] and make this happen before the other use cases. [9:50] I think there is a component of that where, [9:52] probably we work on things like [9:56] science and biomedical out of proportion [9:59] to its immediate profitability. [10:02] Because you guys think it's worthwhile. [10:03] Because we think it's worthwhile. [10:05] We feel the same way about things in the developing world. [10:08] One I'll give you that's controversial, [10:10] people think about it the opposite way. [10:11] So the work we do on defense and intelligence, [10:15] people are often like, "Oh, these guys are selling out." [10:17] I think about it the opposite way, right? [10:19] So, you know, there was this contract with a ceiling [10:21] of $200 million with, you know, [10:23] the DOD and intelligence community. [10:26] People are like, "Oh, man, Anthropic's selling out." [10:29] It's exactly the opposite. [10:30] Getting another $200 million from some coding startup [10:34] would take an order of magnitude less effort [10:36] than getting that contract. [10:38] But you think defense is very worthwhile. [10:39] And we're doing it because we want to defend democracies. [10:42] And we do it within bounds. [10:44] There are some things we're concerned about. [10:47] I'm deeply concerned about abuse [10:49] of government authority on the domestic side. [10:52] We think more on the kind of [10:53] outward directed side. [10:55] But that's an example of [10:57] the things we prioritize are things that we think are good, [11:00] not necessarily things that feel good [11:02] or that people will that we think [11:04] the kind of external buzz will be positive. [11:07] We actually have conviction around some things, [11:09] and we do them regardless. [11:10] You reference the kind of business you want to build. [11:12] What are your aspirations for the Anthropic business [11:15] in, say, three to five years time? [11:17] AI is strange in, like, a number of ways. [11:21] I think one of the ways it's strange [11:23] is that because it's an exponential, [11:26] we have a hard time calibrating exactly [11:28] how big the business will be. [11:30] We had the following experience. [11:32] In 2023 I'd never [11:35] raised money from institutional investors before, [11:39] and so our revenue was zero [11:41] at the beginning of 2023 [11:42] because we had not released a product. [11:44] So I was putting together something, and I'm like, [11:45] "Oh, I think we can probably get [11:47] $100 million of revenue in the first year." [11:49] This caused some investors to say, [11:51] "This is crazy. [11:52] This has never happened in the history of capitalism. [11:54] You've lost all credibility." "You're just making up [11:55] numbers." "Goodbye, goodbye." [11:58] And then we actually did it. [12:00] And so then the next year, I was like, [12:01] "I think we can go from $100 million to a billion." [12:06] And actually, having done it the first time, [12:09] people were like, it was a little bit less [12:11] dismissed as crazy, but [12:12] still often dismissed as crazy, and then we did it again. [12:16] This year, we're halfway through the year. [12:19] We're, as you mentioned, [12:21] well past 4 billion revenue [12:23] in kind of logarithmic space, [12:26] to add another order of magnitude. [12:27] So, there's a bunch of different futures. [12:29] There's one where once things get to a certain size, [12:32] the curve slows down, [12:34] but there's a provocative world [12:36] where the exponential continues [12:37] and in two or three years, [12:39] these are the biggest businesses in the world. [12:42] And I think one of the fundamental experiences [12:45] and uncertainties of working at or running something [12:48] like Anthropic is you kind of don't know. [12:52] You make this exponential projection. It sounds crazy. [12:55] It might be crazy, but also, it might not be crazy, [12:58] because that trend line has followed before. [13:01] And I've said much the same thing [13:03] in the context of training AI models, [13:05] in the context of the cognitive capabilities [13:07] of AI models on the technological side, [13:09] but now we're seeing the same kind of [13:11] continuous lines on the business side. [13:13] So, what's the analogy to scaling laws here, [13:16] where you scale up the relevant inputs [13:20] for model quality in parallel, [13:22] and you get kinda much better model performance? [13:24] Is there something where [13:25] you put better models in, and I know the right organization? [13:27] I know what the inputs are. Yeah, yeah, yeah, [13:29] there's something like, there's some curve [13:32] where you spend [13:35] 5x or 10x more to train a model, [13:39] or you have 5x or 10x more data [13:41] or whatever the scaling laws say, [13:43] and there's some transfer curve for revenue, [13:45] right, where I spend 10 times more on the model, [13:49] andthe model goes from being [13:52] a smart undergrad to a smart PhD student. [13:55] Then I go to [13:57] a pharmaceutical company, and I'm like, [13:58] "Well, how much more is that worth?" [14:00] Often, they end up saying [14:02] "That's worth about" More than 10x, yeah. [14:03] "That's worth about 10x," where these kind of [14:06] power law distributions occur in a bunch of contexts. [14:09] Going on the technical side, when you train the model, [14:13] there's a longer and longer tail of kind of correlations [14:16] that you're capturing as you train the model, right? [14:19] Correlations in the structure of language, [14:20] in the world, in patterns. [14:23] And that correlation is what's thought to lead to [14:25] the scaling laws, because there's this [14:26] kind of logarithmic distribution. [14:28] And then as you think of the model getting [14:31] more and more capable in terms of cognitive tasks, [14:34] there must be, or we're seeing empirically so far, [14:38] if you think of the [14:39] uses of the model in the economy, right? [14:42] If I think of [14:45] the way that companies are organized, right? [14:47] There's a kind of power law, there's a power law structure [14:50] of, like, the— The org chart. [14:52] The org charts of companies. [14:54] And it almost feels like you're climbing [14:56] that power law distribution of value. [14:58] And then I guess the way I think about product [15:00] and go-to-market is that the model [15:03] wants to be on that exponential of revenue, [15:06] and product and go-to-market, [15:09] they're kind of a way to [15:11] clean the window [15:12] and let the light shine through. [15:14] A way to kind of open the aperture [15:17] and let the exponential happen. [15:19] It's like the models want to learn, [15:20] the models want to be extraordinarily [15:22] successful in the market. [15:23] Yes, right, in addition to having this learning impulse, [15:25] the models have this capitalistic impulse [15:28] that they want to embody, [15:30] unless they're given a bad product [15:31] or bad sales to go with them. [15:33] Because they're really useful, [15:34] that intelligence is really useful to people, [15:36] and so it kind of gets pulled out of you. [15:38] Yes, yes, yes. That is a way to think about it. [15:44] What is the terminal market structure here? [15:47] Like, is there a few large scaled players, [15:51] or do we kind of keep seeing new upstarts [15:54] for kind of specific use cases? [15:55] It's very hard it's hard to tell for sure, [15:58] and I think there was [15:59] quite a lot of uncertainty two or three years ago. [16:02] But I think we might be relatively close [16:04] to the final set of players, [16:06] if not necessarily the final market structure [16:09] or the roles of the players. [16:10] I would say there's probably [16:14] somewhere between three and six players, [16:16] depending on how you count, [16:20] and those are the players that are capable of building, [16:24] the players that are capable [16:26] of building frontier models [16:28] and have enough capital to plausibly bootstrap themselves. [16:32] I would love to understand how the model business works, [16:34] where you invest a bunch of money upfront in training, [16:38] and then you have this fast-ish depreciating asset, [16:43] though maybe with kind of a long tail of usefulness, [16:45] and hopefully kind of you pay that back, [16:48] Thus far, like, I think the image people have [16:50] from the outside world is ever larger amounts of CapEx [16:54] and how does all— Get kind of burned. [16:56] There's kind of two different ways you could describe [16:59] what's happening in the model business right now. [17:01] So let's say in 2023, [17:03] you train a model that costs $100 million, [17:07] and then you deploy it in 2024 [17:10] and it makes $200 million of revenue. [17:13] Meanwhile, because of the scaling laws, in 2024, [17:16] you also train a model that costs $1 billion. [17:19] And then in 2025, you get $2 billion of revenue from that [17:23] $1 billion, and you spend $10 billion to train the model. [17:28] So, if you look in a conventional way at the profit [17:31] and loss of the company [17:33] you've lost $100 million the first year, [17:36] you've lost $800 million the second year, [17:38] and you've lost $8 billion in the third year, [17:40] so it looks like it's getting worse and worse. [17:43] If you consider each model to be a company, [17:47] the model that was trained in 2023 was profitable. [17:53] You paid $100 million [17:55] and then it made $200 million of revenue. [17:56] There's some cost to inference with the model, [18:00] but let's just assume in this cartoonish [18:02] cartoon example that even if you add those two up, [18:05] you're kind of in a good state. [18:07] So, if every model was a company, the model is actually, [18:11] in this example is actually profitable. [18:16] What's going on is that at the same time [18:18] as you're reaping the benefits from one company, [18:21] you're founding another company [18:22] that's much more expensive [18:24] and requires much more upfront R&D investment. [18:28] And so the way that it's gonna shake out is [18:30] this will keep going up until the numbers go very large [18:33] and the models can't get larger, [18:36] and, you know, then it'll be a large, [18:37] very profitable business, [18:40] or at some point [18:43] the models will stop getting better, right? [18:44] The march to AGI will be halted for some reason, [18:48] and then perhaps it'll be some overhang, [18:50] so there'll be a one-time, "Oh man, we spent a lot of money [18:52] and we didn't get anything for it," [18:54] and then the business returns to [18:57] whatever scale it was at. [18:59] Maybe another way to describe it is, [19:01] the usual pattern of venture-backed investment, [19:04] which is that things cost a lot [19:06] and then you start making it, is kind of happening [19:09] over and over again in this field within the same companies. [19:12] And so we're on the exponential now. [19:15] At some point, we'll reach equilibrium. [19:17] The only relevant questions are, [19:18] at how large a scale do we reach equilibrium, [19:21] and is there ever an overshoot? [19:23] Right, right. [19:24] You referenced the cloud companies [19:25] as a point of comparison, but I don't know, [19:27] there's something about the cloud companies [19:29] where it feels like their data center [19:30] CapEx is more continuous, [19:31] they're just always doing new data centers, [19:34] whereas there's something about how discrete these [19:35] generations are that maybe it's like the way [19:38] the engine manufacturers, they keep coming up [19:39] with new technologies— [19:41] Yeah, yeah, it's like the F-16 or something, [19:44] orit might be [19:45] a little bit like drug development. [19:47] Exactly. [19:48] Kind of an R&D heavy thing. [19:49] Yes, and when do you actually go [19:50] to the effort of training a model? [19:52] Yeah, yeah it's almost like a drug company, [19:54] where it's like you develop one drug, [19:56] and then like if that works, [19:57] you develop 10 drugs, and then if that works, [19:59] you develop 100 drugs. [20:01] The drug development market does not [20:03] work like that numerically, but it is as if it did. [20:05] Right, so we can look at each of these models [20:07] as individual programs and look at their individual P&Ls, [20:10] and you're saying that the payback math on those, [20:13] at least in the models we've seen to date in the industry, [20:15] is not actually that challenging. [20:17] Where I think most... [20:19] When you're acquiring a customer, [20:20] if you have a nine-month payback on acquiring a customer, [20:23] you'll do that all day long. [20:24] That's a very easy payback to underwrite. [20:26] And you were saying the paybacks are kind of [20:27] nine months, 12 months, so like, [20:28] they're very easy to write. I don't wanna make any [20:30] specific claims— [20:31] Sure, yeah, yeah. [20:32] But qualitatively, [20:35] if you look at the business this way, [20:37] model by model, it looks very viable. [20:40] Yes, because the ever-growing CapEx is masking [20:44] the underlying quality of the model businesses. [20:47] Yes. In 2023, [20:48] everyone is talking about the data wall. [20:49] Is this how we solved our way out of the data wall? [20:53] Yeah, so I don't know, people talk about things in public, [20:56] and sometimes they're [20:58] rumors or suppositions or whatever. [21:01] I wouldn't even necessarily assume that there's a data wall. [21:04] One thing I will say is that [21:08] the idea of using RL has been around for a while, right? [21:11] If we go all the way back to [21:12] when Google DeepMind won, [21:14] beat the world Go champion with AlphaGo, it was RL first. [21:17] And then we built these language models, [21:20] and now we're kind of uniting the two together [21:23] by putting RL on top of the language models. [21:25] That's all chain of thought or reasoning is, [21:28] it's just a fancy way of saying RL, [21:30] where the RL environment is that the model writes [21:33] a bunch of things and then gives an answer. [21:35] There's nothing more to it than that. [21:37] It just kind of has a fancy name. [21:39] And so I think of these as kind of [21:41] the two key ways of learning, right? [21:43] I think of base LLM training [21:45] as learning by imitating, [21:48] and RL as learning by trial and error. [21:50] I think those are the two styles of learning. [21:53] If I'm like a child, [21:54] there's two ways for me to learn. [21:55] I look at my parents and I'm like, [21:57] "Oh, they do something," and I try and learn what they do, [22:00] or I can just kind of experiment [22:02] with the world and learn things, [22:03] and it's very clear in developmental psychology [22:06] that people use both. [22:08] And so we're now seeing that recapitulated [22:11] in the language models, and so we have a stage [22:13] where we do the imitative learning, [22:15] and we have a stage where we learn by trial and error. [22:18] So it seems very natural to me. [22:20] The other thing that's obviously notable to [22:22] people not in the AI industry looking at it, [22:24] is all of the talent wars and the fact that [22:30] your IP walks out the door each evening. [22:32] You referenced in a recent interview you gave, [22:36] $100 million secrets [22:39] that were a few lines of code. [22:41] And obviously I think you were talking about that [22:42] in a national security context, [22:44] but you could also think about it in a talent context. [22:45] Of course. And so how does one... [22:47] Like, in the pharma industry, [22:49] they protect their secrets with patents. [22:51] In Wall Street, where also they have [22:53] $100 million secrets that are [22:55] just a very simple idea, [22:57] Renaissance Technologies, the hedge fund, [22:59] just very successfully locked up its employees. [23:01] How do you make keeping a commercial lead work [23:05] in kind of the current AI environment? [23:07] One thing I will say is that [23:09] there are some things that are like that, [23:11] but I think more and more as the field matures, [23:15] it starts to be more about know-how [23:18] and ability to build complex kind of objects. [23:21] Some of the ideas we work with are simple, [23:27] but I would say the simple ideas, the ones that are like, [23:30] oh, yeah, twiddle this element of the transformer [23:33] or something, those tend to be independently discovered [23:37] or anyone knows them before too long. [23:39] But there are things like, oh man, [23:41] this thing is actually really hard to implement [23:42] from an engineering sense, and we have it implemented, [23:45] or this thing is just kind of a pain to do, [23:47] or there's a know-how to doing it. [23:50] And those tend to be more collective things [23:53] that are more difficult to leak. [23:55] And so I think those things are [23:56] substantially more defensible. [23:58] That said there's still leakage, [24:01] and we still don't want it to happen again, [24:03] both for commercial competitive reasons [24:06] and for national security reasons. [24:08] Both are problems. Yes. [24:09] And so a few things we do. [24:10] One is, you know, we tend to compartmentalize information. [24:14] So if you talk to any intelligence agency, [24:17] that's how they operate. [24:18] You're only told what you need to know, [24:20] and I think everyone within Anthropic. [24:22] But that's probably quite different to a [24:23] normal Silicon Valley culture, where [24:25] everything's just flying around the company. [24:26] Yes, we actually do that at the same time [24:28] as we have a very open culture. [24:30] I say things to the company that [24:32] maybe another person would [24:34] put it in kind of PR speak or, you know what I mean? [24:37] But when there is a secret, then I think that [24:39] actually leads to people trusting that [24:40] it's something that you actually need to know. [24:46] And then finally, having better retention rates [24:49] and losing less people [24:50] is one of the most important things here. [24:52] We have the highest retention rate [24:54] of all the AI companies. [24:57] I think the differences are even starker, [24:58] because everyone has [25:01] kind of a non-regretted attrition rate [25:03] that's maybe constant, so if you subtract that off, [25:05] then the difference is even larger. [25:08] Sometimes when people leave, they come back. [25:10] Yes. I saw that recently. [25:12] If you look at [25:13] you can see publicly the list of people [25:15] who went to the Meta Superintelligence lab. [25:17] Even if you normalize for our size, [25:20] it's not, and many turned them down. [25:24] So in the crazy $100 million comp wars [25:27] that everyone's been talking about, [25:28] you guys have not had too hard a time of that. [25:30] I think relative to other companies, we've done well. [25:32] We even have been relatively advantaged. [25:34] It's like a mixture of true belief in the mission [25:37] and belief in the upside of the equity. [25:42] Anthropic has developed a reputation [25:44] for doing what it says it will do for, [25:47] in some cases, making less promises, [25:50] but keeping those promises that we make. [25:52] And being very clear on what we stand for, [25:55] and being consistent over the years [25:57] and standing for it. That creates [25:59] a unity around the company, [26:01] and I think it's a good guard against cynicism. [26:04] When you're talking about the upside of the equity, [26:05] when you're pitching investors, or maybe candidates, [26:07] how do you pitch the Anthropic business? [26:09] Like, we're building a very large business. [26:10] That's a good start, [26:12] but what else goes into it? [26:13] Often I'll talk about the platform [26:15] and the importance of the models. [26:17] For some reason, sometimes people think [26:20] of the API business and they say, [26:21] "Oh," you know, "It's not very sticky," or, [26:23] "It's gonna be commoditized." I run an API business, [26:25] I love API businesses. No, no, exactly, exactly. [26:28] There are even bigger ones [26:30] than both of ours. [26:32] I would point to the clouds again. [26:34] Those are $100 billion API businesses, [26:38] and when the cost of capital is high [26:39] and there are only a few players... [26:41] and relative to cloud, [26:43] the thing we make is much more differentiated. [26:45] These models have different personalities. [26:47] They're talking to different people. [26:50] A joke I often make is, [26:52] if I'm sitting in a room with 10 people, [26:55] does that mean I've been commoditized? [26:58] There's like nine other people in the room [26:59] who have a similar brain to me, [27:01] they're about the same height, so who needs me? [27:04] But we all know that [27:05] human labor doesn't work that way. [27:07] And so I feel the same way about this. [27:10] I think the API business is a great business, [27:15] but we wanna go broader than that. [27:17] The way I think about it is [27:19] other players such as OpenAI [27:21] and existing incumbents such as Google [27:23] are very focused on the consumer side. [27:25] The idea of providing AI to businesses is something that [27:29] we are trying to get better and better at. [27:32] I think we're out to an early lead in that. [27:34] I'm not sure, because I don't know for sure [27:37] what the revenues of the other players are, [27:39] but I think we probably at this point have the [27:42] plurality of the API market, most likely, [27:46] and AI for business market, perhaps. [27:49] It's funny when you talk about [27:51] kind of the commodity argument, [27:52] where we obviously grew up facing this [27:56] as a skeptical argument, [27:58] and I remember finding it so striking when AWS [28:01] finally had to break out their numbers in 2015. [28:04] Remember they used to be wrapped up in Amazon's numbers, [28:05] and they broke them up? [28:06] Yes, yes, yes. [28:07] And people had been talking about, [28:09] pundits had been saying, "Oh, cloud is a commodity, [28:12] it's uninteresting," and then they broke it out [28:14] and it was one of the [28:14] greatest businesses of all time. [28:15] And there's something where a business can have competitors [28:19] and it can have buyers who care about price, [28:22] but that's very different from being a commodity, [28:24] and as you say— [28:25] Yeah, yeah, yeah, exactly. [28:26] All these products work differently. [28:26] No, exactly. [28:28] I mean, we're like one of the biggest customers [28:30] of the clouds. [28:32] And we use more than one of them. [28:34] I can tell you, [28:35] the clouds are much less differentiated [28:37] than the AI models. For sure. [28:39] Because it feels like, one, [28:40] the behavior is non-deterministic, [28:41] which not by design trying to make it hard, [28:43] but that just naturally means that, oh, [28:45] we get the customer service answers we prefer [28:47] with this model versus that model, [28:49] and I don't know why. No, no, no, exactly. [28:50] You don't know why. [28:51] It's a little like baking a cake, right? [28:54] You put in the ingredients— [28:55] It just works. [28:56] It just, it kind of comes out a certain way. [28:58] One chef makes it this way, [29:00] and the other chef makes it this way. [29:02] And if you're like, "Make it exactly like that chef [29:04] makes it," you can't, right? [29:05] You just can't. [29:06] And presumably, it's striking to me, [29:07] none of the AI products are that personalized right now, [29:11] but it feels like personalization will be a huge deal. [29:14] Will be a huge deal. [29:15] And will be a big source [29:16] of stickiness for the... [29:17] Like, 'cause you won't want to switch products. [29:18] And I don't know exactly what that looks like, [29:21] but given the amount of... [29:22] For both the consumer and the business use case. [29:24] Absolutely, absolutely. [29:26] You know, I think we've just started to [29:28] scratch the surface in terms of models [29:31] that are customized in various ways [29:32] for working with a particular business [29:35] or particular person within the business. [29:37] So I think we're just seeing [29:39] the beginning of the API business. [29:40] But I don't think AI for business is just about API. [29:45] With things like Claude Code, we're selling that [29:47] to not just individual developers, but enterprises as well. [29:51] And they find it some useful. [29:54] Claude for Enterprise, [29:55] that is selling to a lot of enterprises. [29:57] I actually see it, [29:59] and you see this with some of the clouds, [30:00] where they have a bunch of different services, right? [30:03] Some of them are apps, [30:04] some of it is the underlying cloud itself, [30:07] and what they are is the way that AWS [30:11] or GCP or Azure will present themselves, [30:13] and the way that we are starting to present ourselves [30:15] is, hey, we wanna be your one stop shop for AI or for cloud, [30:20] and you can buy all of these things, [30:22] and you can talk to us about which to use for what. [30:26] And so I think that starts to create [30:29] the outlines of a more durable business. [30:31] If you think about a typical Fortune 500 company, how... [30:35] They're probably playing with AI [30:36] for customer service or engineers maybe have [30:39] AI powered coding tools, how AI adopted are they [30:44] compared to how much they should be? [30:45] Well, certainly much less than there should be. [30:47] Yeah. But is it like 5%, 30%? [30:49] What I would say is there is almost, [30:52] there's very often conviction at the top. [30:55] You talk to the CEO, the CEO gets it. [30:57] You talk to the CTO, the CTO gets it. [30:59] The struggle they have is that they have 100,000... [31:03] Pick a company that has 100,000 people, [31:06] who their job is to do something else. [31:09] Their job is to do banking or insurance or drug development. [31:13] And they've heard about this AI stuff, [31:15] but they're not like— [31:17] It's not what they're an expert in. [31:19] And so the challenge is often we are working with [31:23] the leadership of the company [31:25] to get the 100,000 people in the company [31:27] really familiar with and using the technology. [31:30] I think, again, the code stuff goes the fastest, [31:32] because the developers are the ones who are [31:35] most adjacent and most watching the trend. [31:37] Some of the kind of customer service and process stuff [31:40] is next to go, but you really have the instinct [31:43] that even with today's models, it could be, [31:46] 100 times bigger than it is. [31:47] You really get that sense. [31:49] My intuition is sort of that [31:52] we will see the patterns of AI adoption from startups, [31:56] because they're unconstrained by existing organizations, [31:58] so they can kind of do whatever makes sense, [32:01] versus large organizations are somehow calcified, [32:03] because they have all these people whose job is to do X [32:05] and need to be consulted and everything like that, [32:07] so we'll see the new behaviors from the small startups, [32:10] and then large companies, as you say, [32:12] the CEOs and CEOs are switched on and they're smart, [32:14] they say, "Hey, we should be doing that," [32:15] and they'll kind of port the new ideas from... [32:18] Kinda like the adoption we saw of cloud [32:19] or many of these other techs. [32:20] Is that what you're seeing? [32:21] Is that your intuition? [32:22] They'll port the new ideas [32:23] from the small companies, or the small companies [32:26] will become threatening to them [32:27] and disrupt them. [32:28] Sure, yeah. [32:29] And that will give them the urgency to kind of [32:32] drive things through and make them happen. [32:35] A pattern I've seen that works pretty well [32:38] that I actually recommend if you're a large company [32:41] is to kind of make a strike team or strike force [32:44] that's separate from the rest of the company [32:46] and kind of develops these prototypes. [32:48] And then basically, you can get momentum behind something, [32:51] and then there's always this hard work [32:52] of integrated into the rest of the company, [32:54] but if you have a lot of momentum [32:56] and you've done the hard work [32:57] and you've shown the thing works, [32:58] then it's easier to do that. [33:01] Dwarkesh, did you read his recent blog post [33:03] on his AI timelines? [33:04] Oh, on continual learning, yeah? [33:07] Yeah. [33:08] He talked about how his fundamental issue [33:11] at many of the AI models for productivity [33:15] is that they're like the virtual co... [33:17] They're like the super smart virtual coworker [33:20] who started five minutes ago, [33:22] but they remain the coworker that started five minutes ago. [33:25] You know, they don't learn over time. [33:27] How will we solve that? [33:28] Yeah, so, you know, the pattern that I've seen [33:32] in AI on the kind of research and technical side [33:35] is that what we've seen over and over again [33:38] is that there's what looks like a wall. [33:41] You know, it looks like AI models can't do this, right? [33:44] It was like, "AI models can't reason." [33:47] And recently, there's this, [33:47] "AI models can't make new discoveries." [33:50] A few years ago, it was like, [33:51] "AI models can't write globally coherent text," [33:54] which of course now they obviously can. [33:56] You go back a few more years, and you know, [33:59] it was like this Chomsky thing of like, [34:00] you know, they can get syntactics right, [34:02] but they can't get semantics right. [34:05] And every one of those has been blown through. [34:07] Sorry, what have we blown through on the new discoveries? [34:09] This is a thing that people have said recently. [34:12] Actually, my view on this, like many of the other things, [34:16] on new discoveries, is that it's not really a binary, right? [34:21] They don't get to have their name in the paper. [34:22] Yeah, yeah, they don't get to have their name [34:24] in the paper, but like, what is a new discovery? [34:26] What is genius? [34:27] You know, I think I remember this [34:30] developmental psychology book, [34:32] but they were saying something like [34:34] we kind of lionize genius, but [34:36] let's say that a table's wobbly, [34:38] and I'm like, "Oh," I take the coaster [34:40] and I put it under the table, and it's not wobbly anymore. [34:43] That's an idea in a way that's like a new discovery. [34:46] Even if I've never seen someone [34:48] do that before, you know, that's like a new discovery. [34:51] And you know, the difference between that [34:54] and the Nobel Prize winning discovery, [34:56] it's a matter of degree, [34:58] not a fundamentally different matter. [35:00] And so I would say that the AI models [35:02] make discoveries all the time. [35:04] I've had family members where they had a medical problem, [35:06] and the AI model [35:09] diagnosed their medical problem when doctors missed it. [35:12] That's not a big, new, [35:14] but that's a new discovery. [35:18] And you could say, "Oh, they're just pattern matching [35:20] to things that happened before," [35:21] but new discoveries are like that. [35:23] You think of writers who have written novels or something [35:26] that are totally new, and you're like, [35:27] "Well, what are your influences?" [35:29] And they're remixing together [35:31] and adding a new element. [35:32] It's all more continuous. [35:34] And that was the thing I was gonna say [35:36] about continual learning. [35:37] I think this idea that it isn't present is... [35:42] I would say it's present a little bit. [35:45] Yes. It's comfortable practice. [35:46] We're gonna find a way to get more of it. [35:48] So for instance, the models learn within the context. [35:51] You talk to them and they absorb the context. [35:53] Eventually, the context is gonna be 100 million tokens, [35:57] and maybe we'll train the model in such a way [35:59] that it is specialized for learning over the context. [36:02] You could even, during the context, [36:04] update the model's weights. [36:06] There are lots of ideas that are very close [36:09] to the ideas we have now that could perhaps do this. [36:12] I think people are very attached to the idea that [36:15] they want to believe there's some fundamental wall, [36:17] that there's something different, [36:19] something that can't be done. [36:20] It kind of reminds me of— [36:21] Do you think it's a [36:22] coping mechanism deep down? [36:23] Yeah. You know what it reminds me of? [36:25] So, you know the 19th century notion of vitalism? [36:28] This was the idea that [36:31] the human body and [36:34] organisms that are alive are made of a [36:36] fundamentally different material than inanimate matter. [36:39] Which of course we know scientifically now is not true, [36:42] but it's something people very much want to believe, [36:46] and your common sense seems to suggest it. [36:48] I'm not very much like a table. [36:50] I'm made of very different materials [36:54] than metal or glass or whatever, [36:56] but when we actually go down [36:57] to the fundamental units, of course, [36:59] we're all made of the same thing. [37:00] But you think if people now have this kind of [37:02] modern concept of vitalism in whatever [37:05] the fundamental humanity is, and they're saying, [37:07] "Oh, you know, models can't do that." [37:11] I think there's some tendency to believe it. [37:13] And I think as with vitalism, [37:15] the way around it [37:17] is to recognize that a mind is a mind [37:19] no matter what it's made of. [37:20] The notion of the dignity or the specialness [37:23] of cognition or sentience, [37:24] it's not that it isn't special, [37:26] it's that it can be made out of anything. [37:27] You referenced the medical use case, [37:29] which I think is a very cool use case. [37:31] Obviously, one, because of all the people [37:32] who have fixed medical issues as a result, [37:35] but another one is you talked in your [37:37] "Machines of Loving Grace" post, [37:38] which I really enjoyed, I thought it was very well done, [37:40] about the marginal returns to intelligence. [37:43] What are places where intelligence [37:45] is the limiting factor? [37:47] And my read of the popular medical use case is [37:50] obviously it's kind of a charismatic use case, [37:52] but also, for most normal people, [37:54] they have some kind of medical issue, [37:56] low level or serious or something like that, [37:58] and actually, society is very just [38:01] intelligence limited there. [38:02] Not that you don't have access maybe to a smart doctor, [38:05] hopefully you do, but they give you very limited time. [38:07] You know, they think for 10 seconds about your problem. [38:09] Yeah, yeah, yeah, exactly. [38:10] And it turns out [38:11] test-time compute was actually [38:13] what we needed there on the medical stuff. [38:14] But is that kinda your take on this? [38:16] That is how I think about it as well. [38:18] I have talked to Nobel Prize winning biologists [38:21] who say, "I will only"... [38:23] I mean it sounds a little elitist, but they'll say, [38:24] "I'll only go to the top 1% of doctors, [38:27] because the rest of the 99%, [38:30] I can get better advice from an LLM." [38:33] It really is true. [38:34] Doctors are busy, they're overworked. [38:37] And just the nature of medical data [38:40] and medical information, [38:42] it's a lot of pattern matching. [38:44] It's a lot of the same things. [38:47] The level of consistency [38:49] and the ability to put together many different facts, [38:52] I think it's something that LLMs [38:55] are quite good at. [38:56] So you talked in this "Machines of Loving Grace" post [38:58] about some of the big humanity-level areas [39:02] where we're intelligence-limited, [39:03] but again, the personal medical use case is a good example [39:06] of one where society's intelligence-limited, [39:08] and if you give lots of people much more intelligence [39:11] on their specific issues, it's very valuable. [39:13] What are other areas, either in the consumer use case [39:16] or in the business use case where you think we're just [39:19] very obviously intelligence limited? [39:20] Yeah, the places where at least the AI models [39:23] of today can help the most, [39:26] the characteristic quality is something is repetitive, [39:30] but every example is a little different. [39:32] Automation before AI, if you could program [39:35] exactly how it happened, you could do it. [39:37] So if you were doing the same thing over and over again. [39:40] But customer service is like, you know, [39:42] just to take customer service as an example, [39:44] there's like a long tail of stuff, but a lot of it is like, [39:47] you get a bunch of calls, each call is different, [39:49] but each call is basically about one of 10 things, [39:52] and it's like a different person in a different voice [39:55] saying, basically, [39:57] one of these 10 things in a different way. [40:00] And that situation, where things are repetitive [40:03] and similar but not the same, and each has its own thing, [40:07] that's where AI can come in the most. [40:11] Dwarkesh had in the same blog post the prediction that [40:15] you can't yet give an existing AI all of your financial data [40:20] and forge all the emails and have it do your taxes, [40:22] and his prediction for the year in which you can, [40:25] what is the year where your first tax return [40:28] is done by just emailing everything to whatever AI you use, [40:32] his prediction is 2028. [40:34] What do you make of that prediction? [40:36] Probably sooner than that. [40:38] Okay. [40:39] I don't know if it's '26 or '27. [40:41] Some of that is model, mostly accuracy. [40:44] I think the model could do that today, [40:45] but it would make too many mistakes. [40:47] And so working on ways to have the model check its own work [40:50] and do less mistakes is one part. [40:53] There's kind of an interface part of it as well, [40:55] but I would be surprised if it takes that long. [40:58] Okay, '26 or '27. [40:59] And when you say about mistakes, actually, [41:00] you're running through the list of things [41:02] that people thought we would never solve in AI. [41:04] It feels like hallucinations should be on that list. [41:06] Not that they're totally solved, [41:08] but they've gotten a lot better. [41:09] They've gotten a lot better, [41:10] and I think people have gotten more used to. [41:13] They kind of know what to trust the model for [41:15] and what not to trust the model for. [41:17] The models have also been grounded in citations. [41:19] I mean, we've done that with Claude.ai, [41:20] we've done that with Enterprise Claude. [41:22] So I think part of the solution is citation, [41:24] part of the solution is algorithmically, [41:26] the models hallucinate less now, [41:29] and part of the solution is people have adapted [41:32] and understand the weaknesses of the model. [41:34] My view on things like hallucinations has always been, [41:37] there's a certain class of critic [41:39] who points to something where models are weird [41:42] or worse than what humans do, [41:44] and say, "See, they're not like us at all," [41:46] or, "They'll never get there." [41:48] And I kind of get where the instinct comes from, [41:51] where, maybe they're looking [41:53] to see if we've matched the human brain exactly. [41:56] They're saying, "Oh, this is so different. [41:57] It can't be like a human brain." [41:59] But I basically, I just think it's a fallacy. [42:02] There's a notion of kind of general intelligence, [42:05] but it's made up of a bunch of different things, [42:07] and you can simply have most of the things [42:09] and be much worse on some and much better on others. [42:11] If we look at humans that are— [42:13] Yeah, have you met humans? [42:14] Yeah, yeah, have you met humans, right? [42:16] Like if you look at [42:19] humans who are autistic, [42:22] versus humans that are schizophrenic. [42:24] If you look at the optical illusions that humans face [42:27] that machines are not fooled by, [42:28] it's very clear that we have some of these weaknesses, [42:31] just like just similar [42:33] to the model's hallucinations, [42:35] it's just that they look very different [42:37] and we're much more used to them [42:38] because we're surrounded by humans all day. [42:41] The autonomous vehicle double standard [42:44] feels like the kind of clearest example of this. [42:47] Clearest example of this, yes. [42:48] Yeah, where people have much higher standards. [42:50] People have much higher standards. [42:52] But I think it's gonna be a feature of this technology, [42:55] and it has implications on the business side. [42:58] I think we're gonna be in a world where [43:00] the models will make mistakes much less often than humans, [43:05] but they'll be stranger mistakes. [43:07] And actually, that takes some adaptation, [43:09] because imagine you're an end user. [43:11] If you work with humans, you get used to it, [43:13] and you have some notion, right? [43:15] So if a human makes a mistake 5% of the time, [43:18] you might have a good understanding of why, you know? [43:22] Like, let's say I'm talking to a customer service agent, [43:25] and they kind of sound incoherent [43:26] and they're slurring their speech. [43:28] They've probably had too much of this [43:30] and they're not doing their job very well. [43:32] That's a bad mistake to happen, [43:34] but also, if I'm talking to this person, [43:36] I kind of know what's going on [43:37] and I know not to trust what they're saying. [43:40] Whereas an LLM might make a mistake five times less often, [43:46] but it's more deceptive. [43:48] The model sounds just as erudite, just as coherent [43:54] as it does when it's saying something that's right. [43:56] But that's an adaptation thing. [43:59] You know, that's not a fundamental thing. [44:01] And that's something that when we talk to our customers, [44:03] we tell them about that, [44:04] we tell them they need to get used to that. [44:06] So we need to invent slurring for LMS, [44:07] is what you're saying. Right, right, right, [44:09] exactly. [44:13] So, you started out as a researcher, [44:17] but now you're the CEO of a company [44:19] and you're in the business of selling AI. [44:22] And so what have you had to learn about [44:25] go-to-market or dealing with customers, [44:28] all the rest of the stuff? [44:29] Absolutely. [44:30] I think my view on this was [44:31] I started a company [44:34] not because I was initially excited about [44:38] selling things or business or any of that, [44:41] I'd seen the way that some of the other companies had run, [44:43] and the magnitude [44:45] and gravity of what they were trying to build, [44:47] and I was just a bit concerned [44:51] that the people and the motivations [44:54] were maybe not the best ones. [44:56] And I knew that there would be [44:57] a number of players in this space, [45:00] but it felt like having at least one player [45:02] that kind of had a strong compass in how we do things [45:06] could have positive effects on the ecosystem. [45:08] We would build things in a different way, [45:10] we would deploy them in a different way, [45:12] and above all, we would have a, again, [45:14] sometimes short list of principles, [45:16] but we would stick to them as well as we could. [45:20] So I think that was the initial motive. [45:22] And of course, I was excited about building the technology. [45:25] And you know, I think as that has happened, [45:28] of course I and the other cofounders [45:32] have kind of had to learn how to think about, [45:34] kind of the business and the strategy. [45:37] I think I've been very naturally interested [45:40] in the business side of it. [45:41] Actually, I was surprised at how [45:43] quickly I became interested in it. [45:45] And actually, the primary reason was that [45:47] I was curious about all the industries [45:50] that are customers of us. [45:53] Somewhat like the clouds, and perhaps like your business, [45:56] the businesses that we serve, [45:59] they run across every possible industry. [46:01] And so you just learn these things [46:03] about parts of the economy that you've never thought about. [46:06] And even in areas where I nominally know a lot. [46:08] I used to be a biologist, so in a way, [46:11] I know a lot about the pharmaceutical business, [46:13] but I'd never thought about it [46:16] beyond the science. [46:17] I never thought about the portfolio side of it. [46:19] I never thought about how clinical trials work [46:22] and how they could be made cheaper. [46:24] I never thought about the [46:26] defense and intelligence business in any great detail. [46:30] And so you run through those, [46:31] and I just find it super interesting [46:34] to understand what people's problems are [46:35] and how AI can help with those problems. [46:37] I feel like I took very naturally to that. [46:41] Actually, the product side was one [46:42] where I was initially more reluctant. [46:44] I felt like I just had a natural interest and curiosity [46:47] in the business side of it, but like, [46:50] building apps, somehow, initially it was never like, [46:54] a thing that drew me in [46:56] even after I started the company. [46:59] But I think more recently, [47:00] as I've seen what products have succeeded [47:04] and what products haven't, I think this idea [47:07] of how to design products so that they're, [47:10] what we call AGI pilled. [47:11] So that the direction of the product is durable [47:16] and is kind of a bridge to things [47:17] that are useful in the future. [47:19] We've all heard this idea [47:20] of wrapper companies or wrapper products. [47:22] The idea is you make Claude N, [47:25] and someone makes a product that [47:28] basically addresses the deficiencies of Claude N, [47:30] but then you come out with Claude N+1 [47:33] and it just kind of eats it. [47:34] The advice I always give that I think all the AI, [47:36] all the folks at the AI companies give, [47:39] you know, don't make that. [47:40] See the direction of the field [47:42] and try to make something that's complementary. [47:45] And I think thinking about how to make products [47:48] in a new way, in a kind of AGI pilled way, [47:51] that actually has caught my interest a great deal. [47:54] Okay, so, glad you brought this up. [47:57] Doesn't it feel like we have no AI UIs right now? [48:01] Like, we still enter text into text boxes, [48:05] literally same as terminals from the 1970s. [48:08] I mean, a bit more rounded corners and everything. [48:10] Like, we still talk into voice companion modes [48:14] that are manually triggered, [48:15] which is the same as pre-transformer Siri. [48:18] UIs are just completely the same. [48:19] Yeah, yeah, there's something not quite right about it. [48:22] I basically agree with you. [48:23] Could have generative UIs. [48:25] It reminds me a little bit of, [48:27] you know in the early days of the internet, [48:29] it was like people would make these websites [48:33] that had structures that looked like [48:36] they were in the physical world, [48:37] open the closet and do the... [48:39] They're from the horseless carriage era. [48:40] There's some term for this, I forget. [48:42] There's some word for this, I forget what it is. [48:43] Skeuomorphism? [48:44] Skeuomorphism, yes. [48:46] It feels like there's some of that going on here. [48:47] A thing I would say is that as we move more towards agents, [48:53] we're gonna be in a world where [48:56] the AI model can do something end to end. [48:59] Like, we're almost there with Claude. [49:01] Can do something end to end [49:02] and get it right most of the time. [49:04] And a human's main job is to kind of check. [49:08] Or check sometimes. [49:11] But interestingly, checking often means [49:13] getting really into the details of what happened. [49:16] And so there's some kind of impedance mismatch here, [49:19] that some product or interface is the solution to, [49:23] where you want something that's as slick as possible [49:27] and just goes off and does something, [49:30] and you don't wanna have to pay attention most of the time, [49:31] but when something's wrong, [49:33] you might actually need to get quite involved. [49:36] And I don't feel like any products or interfaces [49:39] operate on this principle now or handle this problem now. [49:42] I don't know if that makes sense, but— [49:44] No, it does. Like, agreed. [49:46] I think what you want is your agent to go away [49:48] and do really good work for you, [49:49] and then come back with its work product [49:52] to let you review, steer it, decide everything. [49:56] But you can't be overwhelmed, [49:57] because it's gonna do so many more things [50:00] than you have time to look at, [50:01] and if you're always looking at it, you know, [50:03] it can be slower than if you just did it yourself. [50:05] And so it actually strikes me as an interface problem. [50:09] Yes, yes. [50:09] The generalization of this is it feels to me, [50:13] one of the most exciting things about AI [50:14] is we have such an overhang of current capabilities, [50:18] turning them into good products, [50:20] where even if AI progress was frozen right now, [50:24] we'd have like 10 years of good products, because— [50:26] Oh, oh, I completely agree. [50:28] And actually, the way that products are being built, [50:32] I think by everyone in the industry, [50:34] but we've thought about it this way, [50:36] is very different because the progress is continuing. [50:39] If the progress in models stopped, [50:41] the way we built products would change instantly. [50:45] The reason is I don't think we've ever had before [50:48] a situation which the technology is changing [50:50] under you so fast as you're building the product. [50:53] And so this idea of long-term product roadmaps [50:56] or the usual way of product planning, [50:59] I've started explicitly... [51:01] Again early in Anthropic, I was like, [51:03] "I don't know anything about product. [51:05] I'm a doofus." [51:06] But now I always try to talk to people [51:08] when they come in and they say, [51:09] "This is not like building products [51:11] in the non-AI space," right? [51:12] Because they need to be more AGI pilled? [51:13] Yeah, you may be the expert at building these, but like, [51:16] the technology is moving under you, [51:18] so like, these ideas about fast iteration, [51:21] they're even more true than they are normally. [51:23] What's a specific example of this? [51:25] I think that if [51:28] you're like, "We're gonna make something, [51:30] and it's gonna be ready in six months," [51:32] I think that makes even less sense here than it makes, [51:34] the building in isolation. [51:37] So you need to just have tighter ship schedules [51:39] and more iteration? You need to have tighter [51:40] ship schedules, you need to try things. [51:42] It's very hard to tell, [51:44] even harder to tell what's gonna catch on, [51:47] because a new model may have come out, [51:49] a new model may suddenly be good at something [51:52] that makes a product possible, [51:54] and so much more than anything else, [51:56] you're trying something that's never been tried. [51:58] There's a new model. [51:59] It's only available within the company. [52:01] So the thing you ought to do is [52:03] just build something on it, let people internally try it. [52:07] There's this eternal September vibe to it, right? [52:09] Where it's like, [52:11] it's as if you discovered database technology [52:14] for the first time, and you're like, [52:15] "Well, what could you build on this?" [52:17] And it's always the first day. [52:20] That's what is different. [52:22] You mentioned database technology, [52:23] and maybe that provides an interesting analogy, [52:28] and as we think about open source, [52:30] the first relational databases that were successful [52:34] in terms of adoption were proprietary, [52:35] but then the open source guys caught up. [52:38] How do you keep the gap with the open source options? [52:41] Yeah, open source I think [52:43] has a different meaning in AI models [52:45] than it has in other areas, right? [52:48] And for this reason, [52:50] some have called it open-weights models [52:52] to distinguish. [52:54] I think the main difference is that [52:56] if you see the weights of the models and you look in, [52:58] you can't understand what's actually going on. [53:01] There's not that kind of composability. [53:02] I can't read the source code, I can't— [53:05] You can't produce a trivially different version. [53:06] Yeah, yeah, you know, I can't produce [53:08] a trivially different version of it. [53:11] Now, you know, Anthropic is actually working on [53:13] mechanistic interpretability, which allows you [53:15] to see inside the models, [53:17] and so we're actually working on things [53:18] that would allow some properties, but we're not there yet. [53:21] We're not anywhere close to there. [53:22] There are some things you can do. [53:24] For example, if you have access to the model, [53:26] you can fine-tune the model. [53:28] We're now through interfaces [53:29] kind of allowing people to fine-tune the model. [53:31] There is a question of how valuable [53:35] access to the actual model weights is [53:38] over and above some thick API that lets you do something. [53:41] There's some question of economics, [53:43] but note that it costs a significant amount [53:45] to run the models on the cloud. [53:47] Someone has to host it, someone has to run fast inference, [53:50] and then you're back to the margin [53:52] or some portion of the margin. [53:53] So you think open-weight models are not that useful, [53:56] and fully open source models, there's just a big gap? [53:58] I guess what I would say is that the analogy to [54:03] previous technologies is only partial, right? [54:07] It's kind of a different thing that we're still discovering. [54:13] But I can say from our perspective [54:16] that when a new model comes out, [54:19] when a competitor model comes out, [54:21] we don't really think about whether it's [54:23] an open-weights model or not, [54:24] we think about whether it's a strong model. [54:26] If someone makes a strong model [54:28] that's good at the things that we do, [54:31] that's competition, that's bad for us, [54:33] whether it's an open-weights model or not. [54:36] There's not a huge difference between the two. [54:43] How is Anthropic more AGI pilled than other organizations? [54:47] So one is faster, like a tighter product release cadence. [54:50] But maybe more broadly across the organization, [54:52] not just within product development. [54:53] I mentioned this thing [54:55] that every couple of weeks, [54:56] I get up in front of the organization, [54:58] and kind of describe my vision, [55:01] and I think one of the purposes of that is to keep people [55:05] kind of focused on the mission. [55:06] It's a strange state of the world, [55:08] and I always express uncertainty about it, [55:10] but I say if I were to bet, [55:12] I would bet in favor of this, that, [55:14] in one or two or three years, [55:16] I don't know exactly how long it's gonna be, [55:20] we'll have what I've described [55:22] as like a country of geniuses in the data center. [55:24] And like, this is weird. [55:26] It's gonna change the economy, [55:27] it's gonna accelerate the pace of science. [55:31] It's gonna pose global alignment [55:34] and national security risks. [55:35] It may pose economic problems. The upside is huge. [55:39] The potential for disruption is also huge. [55:43] And I think what I'm trying to fight against is [55:45] the idea of employees who joined [55:49] and they're like, "Oh, you know, I worked in this industry, [55:51] you know, I worked at this kind of company, [55:52] and I'm gonna work at an AI company, [55:53] and maybe a couple years later, I'll go to this," you know? [55:56] This is very categorically different [55:58] from previous experiences that have happened. [55:59] This is a really different thing, [56:00] and I think up and down the organization, [56:03] we want to make sure that [56:06] when our finance people [56:08] think about financial projections, they understand this, [56:11] not that there's certainly going to be an exponential, [56:14] but like, wild outcomes are possible. [56:17] When our recruiting thinks they're like, [56:19] "Oh, yeah, [56:20] this crazy cop stuff could happen because"... [56:24] You know, and when the product people think [56:26] they make AGI pilled products, [56:27] when the policy people interact, [56:29] they understand the stakes of what may happen. [56:31] And so I think a big part of my job [56:34] is keeping the coherence of the organization [56:37] around this central thesis. [56:39] Not that everyone has to [56:41] believe the thesis. [56:43] There's not an indoctrination [56:45] and people chanting with robes or anything. [56:47] But like, the basic idea that the company [56:51] is built around this hypothesis [56:54] that it is possible and perhaps likely [56:57] that these large changes will happen, [56:59] and every aspect of the business, [57:03] as well as the things the company is doing [57:05] for social benefit, should be constructed around [57:09] strong possibility that this may happen. [57:11] To put numbers on this, you've talked about [57:14] the potential for 10% annual economic growth powered by AI. [57:19] Doesn't that mean that... [57:21] When we talk about AI risk, [57:22] it's often harms and misuses of AI, [57:25] but isn't the big AI risk that we [57:28] slightly misregulate it or we slow down progress, [57:30] and therefore there's just like [57:32] a lot of human welfare that's missed out on [57:35] because you don't have AI? [57:36] I've had the experience where [57:38] I've had family members die of [57:40] diseases that were cured a few years after they die, [57:43] so I kind of truly understand the stakes [57:46] of not making progress fast enough. [57:47] I would say that some of the dangers of AI [57:52] have the potential to significantly destabilize [57:56] society or threaten humanity or civilization. [57:59] And so I think we don't want to [58:02] take idle chances with that level of risk. [58:06] Now, I'm not at all an advocate of like, [58:08] stop the technology, pause the technology. [58:12] I think for a number of reasons, I think that's just... [58:15] It's just not possible. [58:16] We have geopolitical adversaries. [58:18] They're not gonna not make the technology, [58:21] the amount of money. [58:22] I mean, if you even [58:23] propose even the slightest amount of reg... [58:25] I have, and you know, [58:28] I have many trillions of dollars of capital lined up [58:31] against me for whom that's not in their interest, [58:34] so that shows the limits of what is possible [58:37] and what is not. [58:38] But what I would say is that [58:42] instead of thinking about slowing it down [58:44] versus going at the maximum speed, [58:47] are there ways that we can introduce [58:51] safety security measures, think about the economy, [58:56] in ways that either don't slow the technology down [58:59] or only slow it down a little bit? [59:01] If instead of 10% economic growth, [59:03] we could have 9% economic growth [59:06] and buy insurance against all of these risks. [59:08] I think that's what the trade off actually looks like. [59:11] And precisely because AI is a technology [59:14] that has the potential to go so quickly, [59:17] to solve so many problems, [59:19] I see the greater risk as like [59:22] the thing could overheat. [59:24] And so I basically want... [59:26] I don't wanna stop the reaction. I wanna focus it. [59:30] That's how I think about it. [59:31] You said, "If we hit December, 2025, [59:34] and there's no AI law, I'll be really worried." [59:36] How are you feeling? [59:38] There is actually something in California. [59:40] There's a bill out, SB53. [59:42] Last year, [59:43] we had the whole SB1047 thing. [59:45] We had mixed feelings on SB1047. [59:49] There was initial version that [59:50] I think was too aggressive. [59:54] And when I say that, what I mean is [59:56] the technology is moving fast, and it's kind of unhelpful [59:59] if you're too prescriptive about it. [60:01] It ends up actually not contributing to safety. [60:04] And I was worried a little bit, [60:05] if something like this passes, [60:07] it's like the tests that were prescribed [60:09] to run will end up looking stupid, [60:11] and then all the people in the AI industry [60:12] will be like, "Oh, this is what regulation [60:14] for safety and security looks like, it's really stupid," [60:16] and they won't take it seriously. [60:17] They'll kind of do everything they can [60:20] to comply in letter and not in spirit. [60:22] And so, as an advocate of thoughtful regulation, [60:26] I was actually a bit concerned about this. [60:28] We offered some changes to the bill [60:32] to a point where kind of we felt good about it, [60:35] and we tried to make a compromise between [60:37] kind of industry and the safety advocates. [60:39] We didn't really succeed, as you saw, [60:42] but this year, I think we're making a bill [60:44] that is something more moderate. [60:46] It's focused particularly on transparency of practices, [60:49] transparency of safety and security practices, [60:51] which is something that Anthropic [60:53] has been very forward about, [60:55] and that I think other companies are starting to do, [60:58] but not all the companies do it, [60:59] and there's no way to tell if folks [61:01] are telling the truth about what they're revealing. [61:03] And California regulation is enough, [61:05] because all the companies have a nexus here. [61:08] Yeah, yeah, I mean, I think [61:09] most of these bills are organized around, [61:11] doing business in California, [61:14] and so it would be difficult to shut off. [61:16] People are very AI-pilled here. [61:17] People are very AI-pilled here. [61:20] So, you know, we'll see what happens. [61:24] I'm not sure what's gonna happen. [61:25] But we've always had this approach that [61:28] we kind of are in favor of guardrails, [61:32] including legislative guardrails on the technology, [61:34] but we recognize the need to be careful. [61:36] We don't wanna kill the golden goose, [61:40] we just wanna stop it, [61:43] from overheating or running off the road. [61:46] Yeah maybe something like [61:48] modern bank regulation [61:49] for all the people complained is a good example, [61:51] where there's an inherently very risky activity, you know? [61:53] Yeah, no, no, the dangers are pretty clear. [61:56] I mean it's like the bank runs are not- [61:58] Right, but it all works pretty well in the modern era [62:02] once we figured out the regulatory environment. [62:03] Last question. What is your personal AI stack? [62:05] How do you use AI differently to maybe other people in tech? [62:08] Yeah, interesting. [62:11] You know, I basically write a lot. [62:14] Perhaps I have too much pride in my own writing. [62:18] I use Claude to generate lots of ideas. [62:22] You know, I kind of use it as research. [62:25] But so far, I've done the writing myself. [62:27] Claude is actually maybe closer than the other ones, [62:29] but it's still not there. [62:32] I'd be comfortable with it for business emails, [62:34] but if I'm kind of, like, writing an essay [62:37] or something that I wanna really get right, [62:40] it's not quite there yet. [62:41] But maybe it will be in, you know, a year or so. [62:43] Yeah, very cool. Well, Dario, this was awesome. [62:46] Thanks for coming by. [62:47] Thank you for having me.