Anthropic Targets Enterprise Market as Claude 2 File Uploads Surge

Claude 2 文件上传量激增，Anthropic 瞄准企业市场 这家初创公司通过部署产品研究团队将定制 AI 解决方案推广至各大企业行业，从而在深入的安全研究与商业化扩张之间取得平衡。 在本次炉边对话中，Anthropic 联合创始人 Daniela Amodei 探讨了最近推出的 Claude 2，重点介绍了其 100,000 个 token 的上下文窗口以及独特的面向用户的特性。她阐述了 Anthropic 的安全至上理念，并详细说明了宪法级 AI（Constitutional AI）、基于人类反馈的强化学习（RLHF）以及机理可解释性等框架。此外，Amodei 还分享了公司面向企业的组织架构、早期初创公司融资与联合创立的运营挑战，以及 Anthropic 如何融合 Stripe 和 OpenAI 的严谨治学风格来塑造自身组织文化的见解。 Anthropic 近期推出了 Claude 2，企业可通过开发者控制台使用，个人用户则可在 claude.ai 访问。 (0:06) Claude 2 的长上下文窗口和文件上传系统在海量文件的总结与数据提取方面获得了极佳的用户反馈。 (0:06) Anthropic 由来自 OpenAI 和 Google 的团队创立，其核心使命是将 AI 安全与以人为本的设计置于模型训练的核心。 (2:02) 用户认为 Claude 具有鲜明的友好、温和且高度对话式的个性，表现得就像一位热心的初级助理。 (2:58) Anthropic 基于有用性、诚实性与无害性（HHH）三位一体的核心训练框架来指导其模型开发。 (2:58)

Anthropic co-founder Daniela Amodei argues that frontier AI development must treat safety, transparency, and empirical research as core technical drivers rather than external constraints, demonstrating that alignment and commercial enterprise utility directly reinforce one another. Navigating the Helpfulness-Harmlessness Dilemma (4:51) Amodei highlights an inherent tension in model alignment: building a perfectly harmless system is trivial if it refuses every request, but doing so destroys its practical utility. To balance the core triad of helpfulness, honesty, and harmlessness (HHH), Anthropic employs Reinforcement Learning from Human Feedback (RLHF) alongside Constitutional AI. Under Constitutional AI, models evaluate and adjust their own outputs against principles drawn from sources like the UN Declaration of Human Rights and industry trust-and-safety standards, circumventing the need to manually score every interaction. Claude's polite, conversational persona is an intentional design choice aimed at setting calibrated expectations. Describing the model as an 'eager junior assistant,' Amodei explains that Claude is trained to explicitly decline and explain tasks it cannot safely complete. Anthropic has published approximately 17 research papers detailing these alignment techniques, arguing that open documentation is necessary for widespread safety adoption. Prioritizing Context Windows Over Pure Recall (7:29) Claude 2 launched with a 100,000-token context window, prioritizing raw document ingestion capacity over alternative compute allocations. While early tests famously processed entire novels like The Great Gatsby, Amodei emphasizes that the core business utility lies in processing dense, unglamorous artifacts such as SEC filings, legal briefs, earnings reports, and lengthy meeting transcripts within seconds. Directly inserting factual source material into a massive context window significantly outperforms relying on free recall from training data, effectively curbing factual drift and hallucination. Claude 2's rollout across claude.ai and the enterprise developer console revealed intense user demand for direct file uploads, validating the strategic decision to trade off other capabilities for expansive context ingestion. Bridging Pure Research and Enterprise Adoption (9:57) To connect open-ended academic inquiry with commercial delivery, Anthropic created an internal 'Product Research' team. This group translates novel alignment methods into enterprise applications, finding that solutions engineered for one client's fine-tuning or data challenges routinely generalize across different sectors. This work is managed by a lean product and business organization of roughly 40 people handling a waitlist exceeding 300,000 users. Amodei notes that enterprise adoption has defied expectations: rather than being confined to frontier technology and financial firms, demand has surged from legacy industries seeking to leapfrog technical cycles. Concurrently, operational bottlenecks have elevated the importance of data engineering. Because downstream model fidelity depends directly on how raw training sets are cleaned, filtered, and analyzed, specialized data engineers have become some of the most critical and fiercely contested hires in the AI sector. Interpretability, Governance, and Ecosystem Risk (25:00) To look inside the deep-learning 'black box,' Anthropic invests heavily in mechanistic interpretability, led by co-founder Chris Olah. Amodei compares the discipline to modern neuroscience—estimating that AI interpretability currently lags 10 to 15 years behind biological brain research—and argues that mapping internal model mechanics provides an empirical safeguard against hidden, hazardous behaviors. Addressing macro risks, Amodei identifies the sheer speed of AI development as her primary concern, cautioning against stumbling into advanced systems without coordinated preparation. Anthropic maintains a dedicated 'Societal Impacts' team to translate technical research into concrete frameworks for regulators. Amodei stresses that governance must remain pluralistic, insisting that policy discussions must actively include open-source developers, civil society, and academia rather than leaving oversight exclusively to commercial incumbents. While Amodei anticipates that extrapolating current capability trajectories could unlock breakthroughs in healthcare, scientific research, and clean energy within five years, she emphasizes that the ecosystem remains in its earliest chapter. The key unresolved question is whether policymakers, researchers, and industry actors can build collaborative regulatory and legal frameworks rapidly enough to match the accelerating pace of frontier models.

Anthropic 联合创始人 Daniela Amodei 指出，将 AI 安全与可解释性深度整合至模型研发不仅是技术伦理要求，更是具备高度经济价值与商业可行性的核心立足点。她认为，通过结合 Constitutional AI 与超长上下文窗口等创新，前沿大语言模型能够以可控且实用的方式重塑企业级工作流，并赋能传统产业跨越式发展。 创立初衷与兼顾学术严谨的组织文化 (2:02) Amodei 回顾了 Anthropic 的创立脉络：核心创始团队成员曾于 OpenAI 共同研发 GPT-2 与 GPT-3，并在 Google 参与过 DeepDream 等项目。他们选择自立门户，旨在建立一家将 AI 安全与以人为本的设计理念置于训练核心的研究机构。在早期融资过程中，面对与前东家差异化的追问，团队明确主张安全不仅是一项伦理责任，更具备显著的商业和经济价值。 在文化塑造上，Anthropic 吸收了 Stripe 与 OpenAI 的基因，尤为推崇 Stripe 的“智力严谨性”。据 Amodei 介绍，公司前十名员工中包含七位物理学博士，这赋予了团队注重实证与系统化探究的学术底色；不过她也坦言，高度学术化的潜在弊端是在日常微小决策上容易出现“过度智力化”的倾向。此外，初创阶段由首席科学官亲自处理发薪等基础行政琐事的混乱经历，促成了联合创始人之间紧密的协作纽带。 安全对齐与技术路径：平衡有益与无害 (4:51) 在模型对齐层面，Anthropic 专注于有益性（helpfulness）、诚实性（honesty）与无害性（harmlessness）三大维度。Amodei 强调，构建一个完全无害的模型非常简单，只需让它拒绝所有提问即可，但这会使其失去实用价值。因此，Anthropic 在模型后期训练中投入大量计算资源与时间，通过人类反馈强化学习（RLHF）以及自主研发的 Constitutional AI（宪法级 AI）来平衡这两种相互制约的目标。 相比耗费人力逐条标注海量回复，Constitutional AI 借助《世界人权宣言》及行业信任与安全准则等原则构成的“宪法”，引导模型自我修正与反思，保持审慎并尊重多元视角。同时，由联合创始人 Chris Olah 主导的“机制可解释性”（mechanistic interpretability）研究致力于透视生成式模型的黑箱。Amodei 评估认为，当前 AI 可解释性水平大约落后于现代神经科学十至十五年，但理解其内部运作原理对在风险显现之前实施防范具有决定性作用。 产品工程化与企业落地的现实需求 (7:29) 围绕刚上线的 Claude 2，Amodei 说明了其面向个人用户的 claude.ai 网站与面向企业的开发者控制台。其中，支持大文件上传的 100,000-token 上下文窗口收获了远超预期的反响。尽管扩展上下文窗口需要牺牲其他方面的算力分配，但相比于依赖模型从训练数据中自由回忆事实，直接向上下文注入原始信息能大幅降低幻觉并提升分析准确度，从而使模型能够在数秒内消化 SEC 财报、法律摘要及冗长的 Zoom 会议纪要。 为了消除学术研究与产业落地之间的鸿沟，Anthropic 设立了独特的“产品研究”（Product Research）团队，专责将模型训练成果转化为企业级定制方案。Amodei 指出，针对特定行业解决的数据处理与微调方案往往能够跨行业复用。目前 Anthropic 的产品与商务团队规模仅约 40 人，正在审慎消化超过 30 万人的等待名单。令人意外的是，采用兴趣不仅局限于大型科技与金融公司，大量曾错失金融科技等前序技术浪潮的传统行业亦表现出极高热情，希望借此实现数字化跨越。 治理协作、数据工程与长期技术图景 (13:32) 谈及大模型带来的生存性风险争议与迅猛发展态势，Amodei 最担忧的是技术迭代速度远超多方协同的建立节奏。她强调，决策者、民间团体、学界与产业界必须形成深度协作，同时治理对话不应由单一群体垄断，必须包含开源社区等多元声音。为此，Anthropic 设立了“社会影响”（Societal Impacts）团队，负责将高度抽象的技术研究翻译成政策制定者能够理解的语言，协助制定切实有效的安全护栏。 在底层要素层面，关于数据爬取与版权的判例法正随技术爆发而同步重塑。Amodei 认为，高水准模型的核心在于数据清洗、处理与训练方法，这使得优秀的数据工程师成为当前 AI 领域最宝贵且最难招募的稀缺人才。展望未来，结合过去数年从 GPT-2 到 Claude 的跨越式演进，Amodei 预期未来五年 AI 将在医疗健康、科学探索与可再生能源领域带来巨大突破，而当前的商业生态尚处起步阶段，未来将迎来更广阔的商品化与多元创新。 Daniela Amodei 展现了对大语言模型赋能科学发现与产业升级的坚定乐观态度，但这种前景的兑现仍面临关键挑战：如何在狂热的产业竞赛中补齐 AI 机制可解释性与神经科学之间十至十五年的技术鸿沟，厘清正在形成的版权法律框架，并在政策制定者、企业界与开源社区之间建立兼顾包容与严谨的治理协作机制，以避免社会在缺乏系统准备的情况下步入高阶 AI 时代。

[0:06] Daniela Amodei: totally mind boggling. [2:58] Daniela Amodei: Claude is very apologetic [2:58] Daniela Amodei: very eager junior assistant [2:58] Alex Komoroske: pretty good people. [4:51] I'm so sorry, I can't help you with that, [7:29] Daniela Amodei: tales of woe and toil [7:29] Alex Komoroske: free recalling facts [9:57] Daniela Amodei: not necessarily those teams' favorite activity. [13:32] Daniela Amodei: thoughtfully walk into this next stage versus kind of stumbling into it. [16:55] chapter one of a long book [16:55] Alex Komoroske: Would you rather that society had Isaac Newton or large language models? [16:55] Daniela Amodei: quit while I'm ahead [18:34] how do I click this? [18:34] did we pay everyone this month? [18:34] if we didn't pay you, could you email Jared? [20:08] Daniela Amodei: incredibly, you know, valuable and economically viable. [22:57] Daniela Amodei: only industry gets to weigh in or only academia or only, you know, policy makers. [25:00] 15, ten years behind where neuroscience today is [26:32] how you clean it, how you treat it, how you analyze it, how you train it. [29:47] intellectual rigor. [29:47] Daniela Amodei: over-intellectualization is kind of the Achilles heel of that approach,

[0:06] I'm so glad to have you here. [0:07] Really says this. Thanks so much for having me. [0:10] I'm really excited to be here. [0:11] One thing I want to ask to start is what does it feel like [0:13] being back in the Stripe office? [0:15] I have to tell you, this is the most surreal experience I about her had. [0:20] Just as like to bore you all for 20 seconds with [0:23] with the last time I was at a Stripe office. [0:26] I'm two offices back and I left [0:29] Stripe in 2018 after about five plus years. [0:32] And I just remember thinking like, my God, this company is so amazing [0:37] and so impressive, but it's so big. [0:38] There was like a thousand people or something at that time. [0:41] So it's just incredible to walk in [0:43] and see this beautiful office, all these incredibly talented people. [0:46] I mean, it's just like, totally mind boggling. [0:48] So it's nice to meet you all. [0:51] So congratulations are in order. [0:52] You guys had your lunch last couple days? [0:54] Card, too. [0:55] How's that going? [0:57] Well, first of all, makes it work. [0:59] We're feeling good about it, feeling excited. [1:02] The launch obviously happened on Tuesday [1:05] and Cloud, too, is now available for businesses. [1:08] So if you're a business user, you can access us through our developer [1:11] console and then individuals can also play around with quad at quad I. [1:17] Are you seeing a interesting uses in the launcher I think is interesting. [1:21] Yeah I think something that we have been noticing is that [1:26] people are really, really excited about the file upload system. [1:30] It's it's, [1:31] you know, I mean, you probably all know this better than me, but something about, [1:35] you know, launching a new product or a new feature, it's not always [1:39] what you expect is the thing that kind of people are excited about but [1:43] have been hearing, you know, some positive feedback just about [1:46] the ability to have this longer context window [1:49] and then to really be able to sort of upload these longer files and have quad [1:53] read through them and pull out information, summarize them. [1:57] So it's definitely been an awesome learning experience over the past [2:01] few days. [2:02] So tell us a little bit more about like what let you decide to co-found Entropic? [2:06] Was that story led either? Yeah. [2:08] So after story by I went to open a I and I was there for about two, two [2:13] plus years and really, you know, my co-founders and I had this vision [2:19] of a really kind of focused research bet. [2:24] And what I mean by that is we had worked on things like, you know, GP2 and GT3. [2:31] We'd worked together at Openai, but some of [2:33] some of the team had worked together previously at Google, had worked on, [2:37] you know, deep dream and just so many different kind of areas, [2:41] both of of AI capability, but also air safety. [2:45] And I think we had this vision of really [2:49] training models with sort of safety and humans at the center of them. [2:53] And we felt really excited to just go and do that together as a group. [2:58] So how is Quad in particular different from some of the other models out there? [3:01] Wouldn't set it apart. [3:03] So I think, you know, [3:04] we were talking about this a little bit before, but one of the things [3:07] that I think is so sort of interesting about this moment in time is that these [3:12] different models are almost taking on like slightly different, like, personalities. [3:17] Right. It's it's a little anthropomorphizing. [3:19] But I think when you sort of play around with different models, you'll see [3:23] these kind of different strengths and weaknesses emerge. [3:26] And again, I think, you know, what we've heard from our users [3:30] is that quad tends to come across as friendly and warm. [3:35] Conversational cloud is very apologetic. [3:38] So if it won't answer a question, it'll be like, I'm so sorry, I can't answer it. [3:42] But very polite. [3:43] I tend to think of Quad as sort of a [3:47] very like eager junior assistant, right? [3:50] It's a it's [3:51] a tool that really wants to help that's kind of on the personality side, [3:54] I think in terms of kind of skills and capability is [3:58] really we've aimed to make Clyde, you know, as safe as possible. [4:02] So I think we saw some improvements from Quad [4:05] One on things like homelessness and honesty. [4:09] Of course, no large language model on the market [4:12] today is, you know, is perfect on any of these dimensions. [4:15] But I really think one of the sort of things we feel [4:18] sort of most proud of is that quad is is as capable [4:21] but also has made progress on on honesty, homelessness and helpfulness. [4:26] I love that point about personalities is Matt Web describes this as an [4:29] AI Somalia for the different personalities and know exactly how to. [4:34] So I think it's interesting to see how these things are so unlike software [4:37] as engineer is trying to make sense of these things [4:39] like wow this is actually pretty bad it following the instructions [4:41] they gave it to, it's like, yeah, it's not a software. [4:43] It's as Ethan Morgan described it, it's pretty good people. [4:46] Yeah. [4:47] And I love that the notion of that assistant, so [4:51] trying to be focused on safety, I imagine has a lot of tradeoffs and challenges. [4:55] He talks through how you navigate those and where some of the hard tradeoffs are. [4:59] Yeah, that's that's a great question. [5:01] I think probably the sort of easiest one or the one that that most brings [5:05] to mind is in this kind of approach to to training with this [5:09] with this framework, you know, helpful, honest and harmless. [5:13] There's this kind of inherent research [5:16] tradeoff between helpfulness and homelessness. [5:19] You can imagine that it's actually very easy to have a perfectly harmless model. [5:25] It will just not be very helpful how you can say, hey, could you, you know, [5:30] I don't know, like write a lot of code for me [5:32] or could I borrow something and God will just say, I'm so sorry. [5:35] I can't help you with that response. [5:36] It's a perfectly harmless model. [5:38] And so really we spent a fair amount of time [5:42] kind of especially in the late stages, you know, [5:44] sort of fine tuning between a little bit more harmful, a little bit [5:47] more helpful, a little bit a little bit less harmful, a little bit less helpful. [5:51] And of course, you know, we try to have the scores on on [5:55] both of those be as high as possible. [5:56] But there is a little bit of an inherent tradeoff just between between those two. [6:01] What are some of the techniques that you use to make these more safe? [6:04] Yeah, so a few different things also with your [6:07] if this is like a topic that you're passionate about, [6:09] we published our safety research [6:11] and we have about 17 papers available on our website that you can read [6:15] a couple of things [6:16] that just sort of jump out, you know, especially for, you know, quad two. [6:20] So we use something called reinforcement learning from human feedback. [6:23] This is a technique, you know, we've written about and this basically is [6:27] just, you know, human laborers basically giving feedback to models of, [6:30] hey, this is a little more harmful, this is a little more helpful. [6:33] But really, I think one of the kind of big breakthroughs [6:36] for us over the past several months was constitutional A.I. [6:39] And we've also written about this. [6:41] So you can you can read about it on our blog, but constitutional I was [6:46] this idea that rather than kind of scoring or giving feedback [6:50] on particular responses at kind of a, you know, very high, high rate, [6:54] you can actually instruct the model with a constitu ssion to say things like, [6:58] hey, it's not, you know, nice to, you know, be angry or mean to people. [7:04] You, you know, you should be thoughtful [7:05] about all different types of people from from all around the world. [7:08] And, you know, we've we've sort of shared some of the, [7:12] you know, the documents that we use to train that, like the U.N. [7:16] Declaration for Human Rights and Trust and Safety, you know, [7:19] best practices from the industry. [7:21] It's it's a really long list. [7:23] But I think that that was a really, [7:24] you know, interesting kind of research breakthrough on on our part. [7:29] So one of the [7:30] things that makes cord different is a huge context window. [7:33] Yeah, 100,000 tokens, I think, which I think is the biggest of any [7:37] any available model. [7:38] What were some of the challenges in extending the context of that? [7:41] Long Yeah, I mean, I will I will definitely say in my my research [7:45] colleagues would probably tell you [7:47] more tales of woe and toil than I will be able to do due justice. [7:51] But, you know, it's it's again, this this sort of issue of of of tradeoffs, right. [7:58] Extending the context window is is going to trade off against, [8:02] you know, other things that basically just take capacity and usage, right. [8:05] And so sort of we chose to prioritize that because we sort of saw this [8:10] this potential use case of people [8:13] kind of giving clod just a lot more information. [8:17] Right. [8:17] And I think some of the value that we're sort of starting to see from [8:21] alums in general is this idea that they can really search [8:25] over a very wide corpus of information. And [8:29] I'm probably not the right person to opine on like how hard [8:32] it will be to be, you know, 200,000 tokens or or 500,000. [8:36] But my sense is that that's probably where the industry is going. [8:39] Yeah, I found that with like we talked a lot tonight about semantic retrieval [8:42] in which techniques for doing embedding based indexing. [8:45] And when you give these things the facts that you want to summarize, [8:48] it was a really better job than kind of free recalling facts [8:51] just out of their training, you know. [8:52] So it's interesting to see with a larger context Windows [8:54] what kind of stuff you can cram into the context. [8:56] Yeah. [8:57] And I think in Quad one, you talked about Great Gatsby. [8:59] Yeah. [9:00] The whole novel in the context, which is pretty amusing. [9:04] Are there any interesting use cases [9:05] that you found in the wild with people using that extremely large context window [9:08] for doing cool? [9:10] Yeah, this is a great question. [9:12] I think it's honestly often for what I would describe [9:15] as kind of important, but like a little bit banal tasks. [9:19] And this is sort of where [9:20] this concept of a junior assistant, I think comes in, right? [9:24] If you were, you know, trying to read a lot of information [9:28] and like elite long memos or legal briefings or, you know, [9:32] many, many hours of recorded Zoom meetings or S.E.C. [9:37] filings or just earnings reports, that is a great use case [9:41] that might take a human or many humans, you know, hours or days to do. [9:45] And Clyde can do that almost instantly [9:47] and then you can ask it information about that. Right. [9:49] What are the takeaways from the earnings reports or what were kind of the [9:53] interesting things that you discovered in these articles over over day X or Y? [9:57] So one of [9:57] the ways that you will differ as well as very explicitly enterprise focus. [10:01] So how does that manifest with it mean for what? [10:04] Think different things you prioritize. [10:06] So I think [10:08] from from kind of an organizational perspective, you know, what that has meant [10:12] is sort of investment in this this team that we call product research. [10:18] And that sounds like a hodgepodge of, you know, two parts of the organization [10:21] that you mush together. [10:22] But I actually think it's it's a very kind of interesting and [10:27] and unusual challenge, which is how do you kind of [10:30] build this bridge between researchers [10:33] that are kind of experts in treating and improving lives? [10:38] And how do you actually turn that into something [10:41] for a larger business that's kind of tailored to their use cases? [10:45] And I think part of what's been so interesting for us about [10:48] that is we've really learned a lot about how to generalize some of those trends. [10:53] Right? [10:54] Bigger businesses often say, hey, I, you know, I want to, you know, fine tune [10:57] on some data and, you know, use it to apply to, you know, concept X or Y. [11:01] But if you just change a couple of those details, [11:04] it often applies to other parts of of of the industry as well. [11:07] And one of the things that's interesting about [11:09] is that your company and others like it are a huge focus on research within [11:15] you know, normally startups are like, you know, mostly about organization. [11:18] This has a significant portion of the resources. [11:20] Are there any other challenges that that leads to? [11:22] And like navigating and figuring out what to build out of ship it? [11:25] That's a that's a also a good question. [11:27] I think, you know, [11:30] there's a lot of [11:31] overlap in terms of values [11:35] and goals and mission for everyone that works at at Entropic. [11:38] And I think sometimes there are you know, [11:41] like stylistic differences, right, between [11:45] a more kind of academic, often research based approach. [11:49] You know, it's it's harder to sort of set immediate, concrete goals, right? [11:53] Sometimes you have to let researchers play around and explore, right? [11:59] They want to sort of go down a rabbit hole. [12:01] And I think, you know, interpretability, which is another kind of research area [12:05] that we're known for, is like sort of the pinnacle example of this where it's [12:08] really hard to predict exactly, you know, where this research is going to go. [12:13] So doing things like writing [12:14] OKRs is like not necessarily those teams favorite activity, right? [12:19] And then on the product side, you know, of course, there's things like, [12:21] well, how many users do we want to ship to and what features do we need? [12:25] And so I don't necessarily think that their intention, [12:28] but I do think it's important that they're sort of, [12:30] you know, energy put into harmonizing the strengths [12:33] of both both of those kind of perspectives and worldviews. [12:36] So you mentioned the users getting let in and you all have, [12:39] I think, more than 300,000 people on the waitlist. [12:41] Do you know what that could tell me more about what's say? [12:44] Because I think someone badly wants it all the way. [12:47] You at Twitter are both asking this question this week. So, [12:51] you know, here what I would say is, you know, number one, [12:53] if that's you, thank you very much for your patience. [12:55] We're trying our best [12:57] if you're an individual called that, I is the best place to go now. [13:01] You'll be able to, you know, get on there very quickly. [13:04] For businesses, we have been, [13:05] you know, as quickly as as we can, kind of moving people off the waitlist. [13:09] We're just we're actually a sort of shockingly tiny effort [13:13] on the product and business side. [13:14] We're about 40 people total. [13:16] And so just from an operational perspective, [13:18] we want to make sure that we give every user that is unanswerable, big [13:21] and an excellent experience [13:23] and that we're really able to help them answer their questions, support them. [13:27] And so we appreciate your patience and we're working as hard as we can [13:30] to get that waitlist down to zero. [13:32] So The New York Times, [13:33] a piece on your own last couple of days, which is really interesting. [13:37] And one of the things that comes out of that, [13:38] there's a lot of people talking [13:39] about their existential fear for what might happen [13:41] if this technology goes in the wrong direction. [13:43] So what keeps you up at night? [13:46] Yeah, I think what I, [13:49] I think what I would say is [13:52] it's it's really a kind of a broader [13:55] reaction to like A.I. [13:58] and just how quickly things are moving as a whole in the industry. [14:02] And maybe the thing I most worry about is that we have, you know, [14:06] this incredible oil opportunity for AI [14:09] to create positive change in the world. [14:12] And of course, there are real risks that come along with that [14:16] level of kind of transformative potential and engagement. [14:21] And I think maybe, you know, the thing that keeps me up at night [14:24] is just are we, as you know, an industry but more more broad than that. [14:29] Right. [14:30] Policymakers and civil society and academics and consumers, [14:35] will we all really be able to invest the time needed to get together and kind of [14:40] thoughtfully walk into this next stage versus kind of stumbling into it. [14:45] Stumbling into the future? Yeah. [14:47] And then on the other side, [14:48] what is your optimistic vision for what I'm going to do for the world? [14:52] I think that the potential power of this [14:56] technology is is a little bit mind boggling. [14:59] Right. [15:00] Anyone who's played around with with an alarm would would, you know, [15:04] generally say, you know, of course, they're not perfect. [15:06] There's all kinds of things that can't do. Right. [15:08] I think that's that's talked about a little bit less. [15:10] But if you sort of extrapolate out from, [15:15] you know, look backwards in time like four years ago or five years ago GP2 [15:21] and you know the bards and GPT four is [15:24] and quads of today that's that's, that's an incredibly small amount of time. [15:28] Right. [15:28] And if you sort of look forward another five years and imagine [15:32] well what won't these models be able to do? [15:36] I think that's I think that's sometimes scary but also like incredibly inspiring. [15:40] Right. [15:40] Imagine the kind of breakthroughs that might happen in health [15:45] care or science or renewable energy as a result of having these powerful tools [15:51] that can work with humans to create all kinds of things. [15:56] It's funny. [15:57] It's such a fast moving space. [15:58] It's so hard to figure out, you know, [16:00] what kind of business models are going to be really powerful. [16:02] People would ask me where the moats are going to be, [16:04] and I want to keep track of what's going on. Risk Careers. [16:07] What are your thoughts? [16:07] What do you think companies found in this A.I. [16:09] world will look like? How they differ? [16:11] Will they be training their will to be using fine tuned models? [16:14] Is off the shelf models. [16:15] Their own models are building. [16:17] Are they going to specialize it themselves? [16:18] Is what's going to look like? [16:21] I mean, first of all, hard to know the future, of course, but [16:24] I think I would expect that there's [16:27] going to be a lot more innovation coming. [16:31] Right. I think we sort of [16:33] it can [16:34] feel easy to kind of imagine that, oh, you know, [16:37] there's only four or five players that are sort of building these labs. [16:41] And, you know, people are just only going to build on top of them. [16:44] And how much diversification can there really be. [16:46] But I think that the sort of, [16:48] you know, commoditization of these tools is something that is just in its infancy. [16:52] Right. [16:52] We're seeing all kinds of ecosystems, much like straight, right. [16:55] Of, you know, platform resellers like Bedrock and, [16:59] you know, individuals that are building things and businesses [17:01] that are building things [17:02] and like the the layer of depth and the level of different applications [17:07] that are totally unexpected that we're seeing people use these for. [17:11] I think, again, we're sort of in chapter one of a long book, and so I expect [17:17] that we're going to see more surprises in the next year, two years, five years. [17:23] Somebody recently I'm a big A.I. [17:24] booster, and someone recently asked me a really interesting [17:27] and thought provoking question, and she said, Would you rather [17:30] that society had Isaac Newton or large language models? [17:34] And curious what your answer to that question is? [17:36] Who the big. With the media question? Right. [17:39] That's a very meaty question. [17:41] I mean, I feel like a cop out is could you fine tune a large language [17:45] model on Isaac Newton and then sort of retain that? [17:49] Maybe, maybe not a fair answer. I'm [17:52] I mean, one sort of presupposes the other. [17:55] Right. Like what would the world would. [17:58] Have done. Logically? [17:58] Could we have done large language models without Isaac Newton? [18:01] Would there have been another Isaac Newton? [18:03] Would they have come a long leader? [18:04] What would that have looked like? [18:07] I don't know. [18:08] I mean, maybe maybe there's an equivalent of like an Isaac Newton today, right? [18:12] Would we rather have a brilliant mind [18:15] of today versus large language models? [18:18] I don't necessarily know how to make that trade off. [18:20] I would I would feel like this is not the answer you were looking for. [18:24] I would go with the fine tuning answer. [18:26] I'll just quit while I'm ahead, find you [18:27] Isaac Newton, and then you get you got the ball. [18:29] For the record, that was also my answer. Yeah, okay. He could. Good. Great. And [18:34] then changing [18:34] that a little bit, how has it been as a co-founder? [18:38] This is the first me with the co-founder. [18:39] Yeah. [18:39] How is that role different from what you've done before? [18:42] Yes. Oh, man. It's it's it's different. [18:45] I think, you know, Stripe and Openai were incredible [18:49] training grounds for me and just in terms of unbelievable opportunity [18:54] and learning that I got at those two companies [18:57] and two companies that I can't think of, two other companies that I respect [19:01] more in the world genuinely. [19:03] And I think being a co-founder is it's a different experience [19:08] in the sense that like you just you go through different phases, right? [19:11] I mean, I joined Stripe when it was around 40 people. [19:15] I joined open air on 60, but I never did that sort of initial [19:19] growth period and I'd always been curious about it. [19:22] And the truth is, it's by far the least fun and glamorous. [19:26] Yeah, it's the most exciting. [19:28] But, you know, my yeah. [19:31] Our Chief Science Officer, one of my co-founders [19:33] with like literally running payroll. Right. [19:35] He was like, how do I click this? [19:36] But like, did we pay everyone this month? [19:39] And we're sort of like, if we didn't pay you, could you email? [19:42] Jared So that's, you know, it's pretty typical [19:46] kind of classic startup story, but it's, it's a humbling experience. [19:50] You realize how many things you're bad at when you're a co-founder. [19:54] You also really learn to lean on the other people that you're doing it with, right? [19:58] Those are the types of bonds that I think just last forever because you're like, [20:02] I remember that time that we didn't know if we paid everybody and that was fun. [20:06] So anyway. [20:07] Yeah, go with that. [20:08] I want to open up to questions from the audience when as a [20:12] a lot of [20:16] it's playing out and the [20:19] thanks so much for the talk [20:22] my question was when you left Albany, I you were there for two years. [20:26] Did investors ask you how are you going to [20:29] differentiate from Albany I when you were raising money. [20:32] Yeah. [20:33] So the question is when we were fundraising, [20:35] our first round, people said, How are you different from Albany? [20:38] I How are you going to differentiate? [20:40] I think, you know, really a lot of that question [20:44] at that time and of course it's evolved. [20:47] We really sort of, again talked about this kind of research [20:50] and safety focus as this kind of primary, primary emphasis. [20:54] Right. [20:55] And I think we have this view that safety is something [20:59] that is incredibly, you know, valuable and economically viable. [21:04] And I think that was a story [21:05] that felt kind of just given the nature of sort of who the co-founders were, [21:09] you know, pretty intuitive for us to to tell. [21:13] Yeah. [21:15] I they call it were like [21:17] there's somebody over there or not what they were doing, whatever year [21:20] that was running around it was yeah. Hi. [21:27] Thanks for the engaging talk. [21:29] Quick question about enterprise use cases. [21:32] Are there any surprises either to the upside or the downside [21:35] that you found around specific use cases where either enterprises [21:38] are really excited to adopt or less excited than you think? [21:43] So I think, first of all, very early for us. [21:46] So I think probably too soon to tell. [21:48] But I think we have been surprised the degree to which [21:54] more traditional businesses are kind [21:57] of excited about the power of of of labs. [22:01] And I think I sort of went into it [22:03] expecting you know, larger tech companies and maybe financial services. [22:06] Obviously, I have a background bias here, but would be the types of businesses [22:11] that sort of might, [22:12] you know, see the potential power of these because they're kind of, [22:15] you know, in that world. [22:17] But it's it's really been kind of across the board. [22:19] Right. [22:20] I think many industries are [22:24] feeling the potential for what this can do to kind of revitalize [22:28] areas of the economy that maybe like missed the last wave or the wave before. [22:33] And I think that's been an incredibly [22:37] interesting and like [22:39] genuinely like inspiring opportunity to think about how to take this technology [22:43] and help other sectors that that might might not have come along for, [22:47] you know, fintech and help them really [22:51] bring bring forward a vision of of of how [22:53] to how to, you know, continue their story. [22:57] Thanks to our Dr. Ed. [23:02] And we'll go over your. [23:06] Thanks for the incredible discussion. [23:09] So out there for the open source conference, a coalition of top [23:13] walks was putting together the next ten principles [23:16] of what it means to have openly accessible A.I. [23:19] and the Golden Rule. [23:20] Just guiding the discussion was basically if I liken A.I., [23:24] I shared with other people who like it and that one person propose at adding [23:29] unless they will tell them what's they accepted and added. [23:32] So if you could weigh in on principles guiding the open source [23:36] community, why did the entire bite [23:39] be break open source the AI leaders into the conversation? [23:42] Are they eyes. [23:44] Great so [23:45] basically you know thoughts on on kind of open source and potential risks [23:50] and how to kind of include that community and sort of this this broader discussion [23:54] I think is is a great question. [23:55] And, you know, I'll start by saying that the A.I. [23:58] ecosystem is very broad. [24:00] And I think that diversity of perspectives that are emerging are really valuable. [24:05] We've just never grappled with a question quite like this one before, [24:10] and I think it would be wrongheaded to say, you know, only industry [24:15] gets to weigh in or only academia or only, you know, policy makers. [24:20] And so I genuinely think that there's room for discussion [24:25] for many different views here, including open source community. [24:30] I think sort of beyond that, I would say [24:32] it's just kind of interesting to see sort of how that ecosystem is developing. [24:36] And I think much like all of these all of these different sort of sectors [24:42] of the economy or society or whatever, it's still really early. [24:46] Right. [24:46] And I think there could be, you know, future iterations of kind of open source [24:50] with different, you know, beliefs and values on the ones today. Hi. [25:00] What's your take [25:01] on mechanistic interpretability for like, the future of alignment of A.I. [25:05] models? Yeah. [25:07] I love mechanistic interpretability. [25:09] This is also, Chris, a lot. [25:11] As one of my co-founders I worked with at at Openai [25:15] and I've known for many years, so potentially just biased. [25:18] But I think mechanistic interpretability is is one of the most interesting [25:23] areas of of research, you know, happening today really in many fields. [25:28] And I I'm not a researcher, so I probably won't do it justice here. [25:32] But I just think this idea that we could [25:36] look inside this black box of generative models [25:41] and better explain what's happening in them is just intellectually fascinating. [25:46] Right. [25:47] And I think from kind of an alignment perspective, [25:51] there's probably no silver bullet, but in general, the more you understand [25:55] what something is doing, probably the more likely you'll be [26:00] to be able to notice behavior that might be concerning. [26:05] Right. That's just kind of a heuristic. [26:06] I'm sure there's more details you could go into there. [26:09] But I really think interpretability is just an incredibly [26:13] like powerful tool that we're just really just scratching the surface. [26:16] Right. [26:16] It's sort of if you were to use a neuroscience analogy, [26:19] we're probably 15, ten years behind where neuroscience today is [26:23] and we're trying to kind of close like play [26:26] awesome. [26:26] Thank you [26:29] to all. Hi. [26:32] Thank you so much. [26:33] And I'm quite impressed by Cloud compared [26:36] to Bard and Chat UBC 3.5. [26:39] So great job. [26:42] What do you think the role of data and the quality of data, [26:46] especially with the recent publication of textbooks, is all you need [26:51] from Microsoft indicating that that's crucial [26:54] along the lines of your main vision, right? [26:57] Safety for AI and. [27:00] You know, alignment, I guess is a tough problem. [27:02] Let's not go there, but just safety in general. [27:04] And how are you able to address this and the training process? [27:09] And are you in the same second part? [27:12] Are you in the same issues as other companies in terms of usage of data [27:17] crawling, Twitter and whatnot and other other sources of data [27:21] that are on the Internet? [27:22] How how do you plan to handle that if you're challenged? [27:25] Yeah, you know, the topic of data and kind of how it overlaps with [27:31] your privacy and security and copyright, all of these issues, [27:36] I think we're sort of seeing almost like a parallel track of acceleration [27:41] of discussion of these issues along with the kind of acceleration of AI. [27:45] Right. [27:46] And that's because these issues didn't really apply in the same way [27:50] as as much before. [27:52] And I think here, you know, [27:55] I would say we are in the midst of all of these decisions and norms [28:01] and laws like being written right, like case [28:04] law is literally being written about these things today [28:07] because of decisions that companies are making around [28:10] how to train data, retain data, store data. [28:14] I think that we have [28:16] sort of invested that almost a little bit [28:19] extra in building relationships with policymakers and and folks [28:24] that are interested in these questions because it feels very important. [28:27] Right. [28:28] It feels kind of fundamental to how these large language models are [28:33] developed, are really generative AI general. Right. [28:35] It's not just languages, other modalities and my sense is that [28:39] there will need to be a very thoughtful, engaged and nuanced approach [28:44] to how we kind of solve those fundamental questions for the field more broadly. [28:49] And so could you repeat the first part? [28:50] I'm so sorry. [28:51] Oh, the quality. Yes, the quality. [28:54] So, yes, I mean, the short answer is data quality is incredibly important. [28:59] I mean, how do you measure it? Yeah, I think, [29:02] again, probably a research colleague would give you a better answer. [29:05] But my my real my real kind of answer here is that it's [29:09] not just the quality of the data itself, but it's how you treat the data, too. [29:13] Right. So how do you clean it? How do you treat it? [29:15] How do you analyze it? How do you train it? [29:18] And I think that's actually a really interesting sort of emerging [29:23] like job space. [29:25] Right. [29:25] Like, I think the concept of like data engineering, they're hard to hire, right? [29:30] Like excellent data engineers and they're actually some of the most [29:33] valuable people at the AI companies because so much about, [29:37] you know, the overall outputs in the end, [29:40] sort of results of the models happen at that kind of initial phase. [29:43] So sorry for forgetting the first part of your question. Yeah. [29:47] Yeah, I have time for one [29:48] or two more questions. [29:52] Thank you for the chat was great. [29:54] My question is, can you can you mention one thing that you took [29:59] from strong stripe group in AI culturally to anthropic [30:03] and wanted to a better than the culture and a throwback and that one thing [30:06] that you decided to do differently or like wanted to fix let's say [30:10] about your prior experiences culturally and anthropic. [30:14] I love that question. [30:16] So I again like Stripe and Openai [30:20] I deeper company is both incredible, both very impressive. [30:24] I think probably from Stripe. [30:27] The thing I mean, I should also say we've hired a lot of stripes. [30:30] So there's some of this is just natural kind of osmosis. [30:33] But I think the kind of intellectual rigor [30:36] of stripe is something that I always found. [30:39] So impressive. [30:42] And I think that there's a nice kind of synchronicity [30:46] because our founding team are like mostly PhDs. [30:49] I wake, we like seven physics PhDs and like our first ten employees [30:53] or something like that. [30:54] There's this kind of desire to really like understand and know the truth, right? [30:59] To be kind of intellectual, empirical, [31:03] systematic in terms of how we approach things like safety problems. [31:07] And I think that that is definitely something that I feel [31:10] is sort of synchronous between between Stripe and Anthropic [31:15] in terms of things we didn't want to pull over. [31:17] I don't know that we've succeeded at this, but I think a flipside of that [31:20] kind of intellectual ism is sometimes like almost overthinking decisions, right? [31:26] So sometimes there are decisions that [31:28] you're like, we should just have some snacks in the office, [31:32] like maybe peanuts are. [31:33] Okay, we also have capacitance. [31:36] I do think that the sort of over intellectualize [31:38] Asian is kind of the Achilles heel of that approach. [31:42] One last question [31:44] that. [31:47] I was wondering how you're thinking [31:49] about a nutritious, fundamental skill together [31:53] with other forms of literacy, because that's the other part of safety [31:57] in terms of policymakers, you are users, your analysts. [32:02] You can ensure that these models are as safe as possible. [32:06] And there are lots of guardrails, guardrails in place. [32:09] But ultimately, [32:11] the level of knowledge that people have about the systems [32:14] and how to use them will determine, yeah, the outcome. [32:18] Yes, great question. [32:20] So I think in terms of AI literacy, you know, a few thoughts. [32:24] One is that we we it is a more kind of narrow group. [32:29] But part of why we publish so much of our research is for this exact reason. [32:33] Right, to kind of the earlier question around mechanistic interpretability or, [32:38] you know, constitutionally AI part of why we try to be as open as possible [32:42] about our safety work in our research is so that that will become propagated [32:46] in the field more broadly. [32:47] And so that it's something [32:48] that's talked about because we think it's so critically important. [32:52] Beyond that, we have an entire team in our policy [32:55] organization called Societal Impacts, [32:58] and what they do is they're sort of the bridge between [33:01] the research teams and the policy team and their job [33:06] is to take the very technical topics that are probably challenging [33:11] for policymakers or others to kind of understand [33:15] intuitively and turn them into something that's that's easy, right? [33:19] Something that you can arm our policy team with or other policy teams [33:23] with when talking to government officials or regulators [33:26] and say, here's why it's important that we put guardrails here. [33:30] And I think that's that's just one example. [33:33] But I think there are ways to communicate this information in a way [33:38] that is not just academic or challenging to sort of understand. [33:44] Cool and ended there [33:45] thank you so much really just the music.