Sam Altman: OpenAI, GPT-5, Sora, Board Saga, Elon Musk, Ilya, Power & AGI | Lex Fridman Podcast #419

山姆·奥特曼谈OpenAI、GPT-5、Sora、董事会风波、马斯克与AGI | Lex Fridman播客第419期 在这场深度对话中，山姆·奥特曼剖析了2023年11月OpenAI董事会的混乱危机，详细讲述了自己短暂被免职带来的情绪冲击、董事会的重组，以及尽管存在法律纠纷，他仍对伊利亚·苏茨克维（Ilya Sutskever）和埃隆·马斯克抱有持久的敬意。他探讨了GPT-4等现有模型的能力与局限，将当前系统视作通往通用人工智能与科学发现的早期垫脚石。奥特曼还深入分析了GPT-5等未来模型的技术与基础设施现实，阐述了扩大AI规模所需的庞大算力和清洁能源需求、Sora等模型的作用，以及推行迭代部署而非毫无预警发布前沿模型的社会必要性。在整场对话中，他强调了透明治理、超越戏剧化末日论调的安全对齐，以及具身智能与机器人技术重塑文明的潜力。 山姆·奥特曼证实OpenAI计划在2024年发布一款令人惊叹的前沿AI新模型，并预计从GPT-4到GPT-5的跨越将带来堪比GPT-3至GPT-4的性能飞跃。 (66:13) 奥特曼驳斥了有关伊利亚·苏茨克维因暗中目睹AGI而引发2023年11月OpenAI董事会危机的传言，明确表示公司内目前尚无人见过或构建出通用人工智能。 (18:31) 谈及埃隆·马斯克的诉讼时，奥特曼指出马斯克最初在2018年提出分道扬镳是因为想完全掌控OpenAI或将其并入特斯拉，并称马斯克当下的法律发难有失建设者风范。 (27:00) OpenAI计划重返物理机器人和具身智能系统领域，奥特曼指出，如果打造出的AGI仍完全依赖人类去执行物理任务，那将令人感到遗憾。 (91:20)

OpenAI CEO Sam Altman argues that artificial general intelligence will emerge through the cumulative scaling of compute and hundreds of compounding engineering breakthroughs, while contending that institutional resilience and distributed governance are just as critical as technical safety. Governance Vulnerabilities and the Boardroom Crisis (1:05) Altman's temporary ouster in November 2023 exposed critical structural weaknesses in OpenAI's nonprofit governance architecture, which lacked traditional shareholder checks and allowed a small group of directors to make high-stakes decisions under intense pressure. While Altman maintains that the directors were well-meaning individuals who made suboptimal choices, the existential crisis demonstrated that building organizations capable of withstanding real-world strain is as vital as solving technical AI safety. The ordeal pushed Altman toward a more defensive planning posture, dampening his historically high default trust in others. Although he successfully returned with broad staff and partner backing, Altman concedes the episode created an awkward precedent: while he refuses super-voting shares and argues no single individual should hold unilateral control over AGI, his reinstatement made it practically difficult to claim that an independent board can easily fire him. Compute Scaling and the Technical Path to Frontier Models (44:13) Altman characterizes compute as the elastic currency and most precious commodity of the coming era, comparing its societal consumption to energy rather than bounded hardware markets like smartphones. Frontier progress does not hinge on a single architectural silver bullet; rather, Altman cites Ilya Sutskever's philosophy that major leaps stem from multiplying roughly 200 medium-sized innovations across data curation, algorithmic tweaks, and hardware optimization. Assessing current capabilities, Altman bluntly remarks that GPT-4 "kind of sucks" when viewed against the steep exponential curve ahead. He envisions models evolving to support billion-token context windows and variable inference compute, where systems allocate more processing power to complex reasoning tasks like mathematical proofs than to simple retrieval. Altman committed to OpenAI releasing a major model within the year, while emphasizing that iterative public deployment remains vital so society and regulators can adapt gradually rather than facing sudden, unmanageable technological shocks. Ecosystem Friction and Musk's Lawsuit (24:39) OpenAI's transition from an exploratory non-profit research lab to a heavily capitalized enterprise required pragmatic structural adaptations that Altman acknowledges appear eyebrow-raising in hindsight. He expresses confusion and personal disappointment over Elon Musk's lawsuit against the organization, describing the litigation as unbecoming of an elite builder. Altman dismisses the narrative that the dispute is genuinely about open-source principles, pointing out that Musk's xAI only open-sourced Grok after facing accusations of hypocrisy. While fierce competition across AI firms accelerates cost reductions and product quality, Altman warns that competitive parity creates dangerous pressure toward an arms race. To mitigate catastrophic risks, he strongly favors a development trajectory characterized by short timelines to initial AGI paired with a slow, highly controllable takeoff phase. Product Integrity, Behavioral Standards, and Embodiment (80:14) Altman rejects digital advertising as an aesthetic and operational model for conversational AI, warning that commercial sponsorships would subtly warp user trust and corrupt assistant recommendations. Instead, he maintains that subscription models provide clean incentives where paying users remain the true customer. To counter ideological bias and alignment controversies—such as historical inaccuracies observed in competing models like Google Gemini—Altman advocates for labs to draft and publish concrete, public behavioral specifications for sensitive edge cases to distinguish bugs from deliberate policies. Beyond digital interfaces, Altman envisions multimodal video models like Sora eventually integrating into humanoid robotics and physical systems, arguing that achieving AGI without physical agency would be deeply limiting. Ultimately, he conceptualizes AGI not as a solitary super-brain, but as a shared societal scaffolding that extends collective human capability. Altman anticipates highly capable systems by the end of the decade, but key questions persist regarding how OpenAI will balance its commercial scale with its founding mission, and how democratic societies will establish effective, distributed oversight as frontier AI models approach human-level reasoning.

OpenAI 首席执行官 Sam Altman 在对谈中指出，算力将成为未来最重要的基础资产，而通向通用人工智能（AGI）的历程必然伴随巨大的权力角逐与治理重塑。他主张通过持续迭代部署而非封闭研发来应对技术风险，并强调 AGI 不应由任何单一主体或闭门董事会单方面掌控。 治理危机反思与权力分配困境 (1:05) Altman 回顾了 2023 年 11 月被短暂解职的风波，将其称为职业生涯中最痛苦的经历，但也暴露出非营利架构在面对巨大外部压力时的脆弱性。原董事会在无需对外部股东或公众负责的环境下仓促做出解雇决策，差点导致组织彻底崩溃。尽管通过员工与合作伙伴的支持实现复职，并引入 Bret Taylor 与 Larry Summers 等人重组董事会，Altman 坦言这一过程在客观上弱化了“董事会可随时解雇 CEO”这一制衡机制的公信力。他明确表示自己没有也不寻求超级投票权，坚决主张 AGI 的未来不能由单一个人或单一商业公司掌控，必须建立能对全球公众负责的民主治理体系与明确具体的行为规范（Behavioral Specs）。 技术迭代范式与前沿模型演进 (44:13) 谈及大语言模型的发展，Altman 直言 GPT-4“其实挺烂的”，认为研究人员必须站在未来视角保持不满，并预计 GPT-5 与 GPT-4 之间的差距将等同于 GPT-3 到 GPT-4 的跨越。针对技术路径，他引用首席科学家 Ilya Sutskever 的观点，指出模型突破并非依赖单一灵丹妙药，而是将算法、训练与算力等约 200 项中等改进相乘的结果。未来的前沿系统亟需摆脱“每个 token 消耗等量算力”的限制，建立类似人类慢思考（System 2）的可变算力机制，使系统在面对复杂推理时能分配更多推理算力。此外，OpenAI 将在年内推出重大模型更新，同时积极探索视频生成模型 Sora 背后的世界模型理解、超长上下文窗口以及终身交互记忆等方向。 算力作为未来货币与商业范式选择 (69:28) Altman 驳斥了关于他直接筹集 7 万亿美元的传言，但坚持认为“算力将是未来的货币与最宝贵的商品”这一核心判断成立。与全球年需求受人口规模限制的智能手机芯片不同，算力具有极强的需求弹性，形态更接近电力能源：成本越低，人类将其运用于日常琐事或前沿科学研究的消耗量就越呈指数级爆发。在商业化模式上，Altman 对在 AI 交互助手中引入广告模式表示强烈的审美与道德反感，警告赞助推荐会扭曲智能体的客观性。他坚信直接订阅制能让用户成为真正的客户，并有信心在完全不依赖数字广告的前提下，构建出足以支撑庞大算力开销的健康商业模式。 竞争摩擦、法律纠纷与安全部署哲学 (24:39) 针对 Elon Musk 发起的诉讼，Altman 对崇拜的创业领袖诉诸法律感到遗憾，认为对方关于开源与闭源的指责并未触及真实意图，但仍期望未来能恢复良性竞争与友好关系。在安全理念上，Altman 倾向于“短起步时间、慢起飞速度”（Short Timeline / Slow Takeoff）的发展路径，通过小步快跑的持续迭代部署让社会和监管机构逐步适应新工具，而非闭门憋出超强模型后突然释出。他认为公众对科幻式的“AI 逃逸出控制箱”等戏剧化风险关注过多，反而挤占了对深度伪造、版权创作者补偿以及政治极化等迫切现实现象的讨论。 Altman 预测极高能力的 AI 系统可能在本年代末或更早出现，但科学探究、物理世界具身操作以及防范技术引发的剧烈政治动荡仍面临诸多未解难题；AGI 最终可能不是单一的独立超级大脑，而是作为人类社会演进中集体知识脚手架的自然延伸。

[0:00] Sam Altman: I think compute is gonna be the currency of the future. I think it'll be maybe the most precious commodity in the world. [0:00] Sam Altman: The road to AGI should be a giant power struggle. I expect that to be the case. [29:55] Sam Altman: I would heavily discourage any startup that was thinking about starting as a non-profit and adding like a for-profit arm later. [22:53] Sam Altman: You get a little bit screwed in exchange for getting to live with your guard down. [44:13] Sam Altman: I think it is our job to live a few years in the future and remember that the tools we have now are gonna kind of suck looking backwards at them and that's how we make sure the future is better. [91:20] Sam Altman: I think it's like sort of depressing if we have AGI and the only way to get things done in the physical world is like to make a human go do it. [62:37] Sam Altman: Rather than go build in secret until we got all the way to GPT-5, we decided to talk about GPT 1, 2, 3 and 4, and part of the reason there is, I think, AI and surprise don't go together. [60:30] Sam Altman: It seems to me like you want to be able to allocate more compute to harder problems.

[0:00] - I think compute is gonna be the currency of the future. [0:02] I think it'll be maybe the most precious commodity [0:05] in the world. [0:06] I expect that by the end of this decade. [0:09] And possibly somewhat sooner than that, [0:15] we will have quite capable systems [0:18] that we look at and say, wow, that's really remarkable. [0:21] The road to AGI should be a giant power struggle. [0:24] I expect that to be the case. [0:26] - Whoever builds AGI first gets a lot of power. [0:32] Do you trust yourself with that much power? [0:36] The following is a conversation with Sam Altman, [0:40] his second time in the podcast. [0:41] He is the CEO of OpenAI, [0:44] the company behind GPT-4, ChatGPT, Sora, [0:48] and perhaps one day the very company that will build AGI. [0:55] This is Lex Fridman Podcast. [0:57] To support it, [0:58] please check out our sponsors in the description. [1:01] And now, dear friends, [1:02] here's Sam Altman. [1:05] Take me through the OpenAI board saga [1:08] that started on Thursday, November 16th, [1:11] maybe Friday, November 17th for you. [1:13] - That was definitely [1:14] the most painful professional experience of my life [1:18] and chaotic, and shameful, and upsetting [1:27] and a bunch of other negative things. [1:30] There were great things about it too [1:32] and I wish it had not been [1:36] in such an adrenaline rush that I wasn't able to stop [1:39] and appreciate them at the time. [1:45] I came across this old tweet of mine [1:47] or this tweet of mine from that time period, [1:48] which was it was like kind of going to your own eulogy, [1:51] watching people say all these great things about you [1:53] and just like unbelievable support [1:57] from people I love and care about. [2:01] That was really nice. [2:04] That whole weekend [2:05] I kind of like felt with one big exception, [2:08] I felt like a great deal of love [2:13] and very little hate [2:18] even though it felt like I have no idea what's happening [2:21] and what's gonna happen here [2:22] and this feels really bad. [2:24] And there were definitely times [2:25] I thought it was gonna be like one of the worst things [2:28] to ever happen for AI safety. [2:30] Well, I also think I'm happy [2:32] that it happened relatively early. [2:34] I thought at some point [2:36] between when OpenAI started [2:39] and when we created AGI, [2:41] there was gonna be something crazy [2:42] and explosive that happened, [2:44] but there may be more crazy and explosive things happen. [2:49] It still I think helped us build up some resilience [2:55] and be ready for more challenges in the future. [3:02] - But the thing you had a sense [3:05] that you would experience is some kind of power struggle. [3:08] - The road to AGI should be a giant power struggle. [3:12] Like the world should... [3:14] Well, not should. [3:15] I expect that to be the case. [3:17] - And so you have to go through that, [3:20] like you said, [3:21] iterate as often as possible [3:24] in figuring out how to have a board structure, [3:26] how to have organization, [3:28] how to have the kind of people that you're working with, [3:31] how to communicate all that [3:33] in order to deescalate the power struggle [3:36] as much as possible, pacify it. [3:38] - But at this point, [3:40] it feels like something that was in the past [3:47] that was really unpleasant and really difficult and painful. [3:51] But we're back to work and things are so busy [3:55] and so intense that I don't spend a lot [3:59] of time thinking about it. [4:00] There was a time after. [4:03] There was like this fugue state [4:05] for kind of like the month after, [4:07] maybe 45 days after [4:09] that was I was just sort of like drifting through the days, [4:13] I was so out of it. [4:16] I was feeling so down [4:17] - [Lex] Just on a personal psychological level. [4:19] - Yeah. [4:20] Really painful. [4:23] And hard to have to keep running OpenAI [4:27] in the middle of that. [4:28] I just wanted to crawl into a cave [4:30] and kind of recover for a while. [4:32] But now it's like we're just back [4:35] to working on the mission. [4:38] - Well, it's still useful to go back there [4:41] and reflect on board structures, [4:45] on power dynamics, [4:48] on how companies are run, [4:50] the tension between research, and product development, [4:54] and money and all this kind of stuff [4:57] so that you have a very high potential [5:01] of building AGI would do so in a slightly more organized, [5:05] less dramatic way in the future. [5:07] So there's value there [5:09] to go both the personal psychological aspects of you [5:12] as a leader and also just the board structure [5:15] and all this kind of messy stuff. [5:18] - Definitely learned a lot about structure and incentives [5:23] and what we need out of a board [5:28] And I think that it is valuable [5:31] that this happened now in some sense. [5:35] I think this is probably not [5:36] like the last high stress moment of OpenAI, [5:39] but it was quite a high stress moment. [5:41] Company very nearly got destroyed. [5:43] And we think a lot about many [5:48] of the other things we've gotta get right for AGI. [5:50] But thinking about how to build a resilient org [5:53] and how to build a structure that will stand up [5:56] to a lot of pressure the world, [5:57] which I expect more and more as we get closer. [6:00] I think that's super important. [6:02] - Do you have a sense of how deep [6:04] and rigorous the deliberation process by the board was? [6:07] Can you shine some light [6:10] on just human dynamics involved in situations like this? [6:12] Was it just a few conversations [6:14] and all of a sudden it escalates [6:16] and why don't we fire Sam kind of thing? [6:19] - I think the board members were far, [6:26] well-meaning people on the whole. [6:30] And I believe that in stressful situations [6:40] where people feel time pressure or whatever, [6:47] people understandably make suboptimal decisions. [6:49] And I think one of the challenges [6:51] for OpenAI will be we're gonna have to have a board [6:56] and a team that are good at operating under pressure. [7:00] - Do you think the board had too much power? [7:03] - I think boards are supposed to have a lot of power, [7:06] but one of the things that we did see is [7:09] in most corporate structures, [7:11] boards are usually answerable to shareholders. [7:14] Sometimes people have like super voting shares or whatever. [7:18] In this case, [7:19] I think one of the things with our structure [7:21] that we maybe should have thought about more than we did is [7:25] that the board of a nonprofit has, [7:28] unless you put other rules in place, [7:31] like quite a lot of power, [7:32] they don't really answer to anyone but themselves. [7:34] And there's ways in which that's good, [7:37] but what we'd really like is for the board of OpenAI [7:40] to answer to the world as a whole [7:42] as much as that's a practical thing. [7:43] - So there's a new board announced. [7:46] - [Sam] Yeah. [7:47] - There's, I guess, a new smaller board of first [7:51] and now there's a new final board. [7:53] - Not a final board yet. [7:54] We've added some, [7:55] we'll add more. [7:56] - Added some, okay. [7:57] What is fixed in the new one that was perhaps broken [8:04] in the previous one? [8:06] - The old board sort of got smaller [8:09] over the course of about a year. [8:10] It was nine and then it went down to six [8:12] and then we couldn't agree on who to add. [8:15] And the board also, I think, didn't have a lot [8:20] of experienced board members [8:22] and a lot of the new board members [8:24] at OpenAI have just have more experience as board members. [8:29] I think that'll help. [8:31] - It's been criticized some of the people [8:33] that are added to the board. [8:35] I heard a lot of people criticizing the addition [8:37] of Larry Summers, for example. [8:39] What was the process of selecting the board? [8:41] What's involved in that? [8:43] - So Bret and Larry were kind of decided in the heat [8:46] of the moment over this very tense weekend [8:49] and that was... [8:50] I mean, that weekend was like a real rollercoaster, [8:52] like a lot of lots and downs. [8:56] And we were trying to agree on new board members [9:01] that both sort of the executive team here [9:05] and the old board members felt would be reasonable. [9:10] Larry was actually one of their suggestions, [9:11] the old board members. [9:13] Bret, previous to that weekend, suggested, [9:17] but he was busy and didn't wanna do it. [9:19] And then we really needed help in wood. [9:22] We talked about a lot of other people too, [9:24] but I felt like if I was going to come back, [9:30] I needed new board members. [9:35] I didn't think I could work with the old board again [9:38] in the same configuration, [9:39] although we then decided, [9:41] and I'm grateful that Adam would stay, [9:44] but we wanted to get to... [9:46] We considered various configurations, [9:48] decided we wanted to get to a board of three [9:50] and had to find two new board members [9:53] over the course of sort of a short period of time. [9:57] So those were decided honestly without... [10:00] That's like you kind of do that on the battlefield. [10:02] You don't have time to design a rigorous process then. [10:05] For new board members, [10:06] since new board members will add going forward, [10:11] we have some criteria that we think are important [10:15] for the board to have different expertise [10:17] that we want the board to have. [10:19] Unlike hiring an executive [10:21] where you need them to do one role, [10:22] well, the board needs to do a whole role of kind [10:25] of governance and thoughtfulness. [10:28] Well, and so one thing that Bret says, [10:32] which I really like is that we wanna hire board members [10:34] in slates, [10:35] not as individuals one at a time. [10:37] And thinking about a group of people [10:40] that will bring nonprofit expertise, [10:43] expertise at running companies, [10:44] sort of good legal and governance expertise. [10:47] That's kind of what we've tried to optimize for. [10:48] - So is technical savvy important [10:50] for the individual board members? [10:52] - Not for every board member, [10:53] but certainly some you need that. [10:55] That's part of what the board needs to do. [10:56] - So I mean, the interesting thing [10:58] that people probably don't understand [11:00] about OpenAI certainly is like all the details [11:02] of running the business. [11:04] When they think about the board given the drama, [11:06] they think about you, [11:07] they think about like if you reach AGI [11:11] or you reach some of these incredibly impactful products [11:14] and you build them and deploy them, [11:16] what's the conversation with the board like? [11:18] And they kind of think, [11:19] all right, what's the right squad [11:22] to have in that kind of situation to deliberate? [11:25] - Look, I think you definitely need [11:27] some technical experts there [11:28] and then you need some people who are like, [11:32] how can we deploy this in a way [11:34] that will help people in the world the most [11:37] and people who have a very different perspective? [11:40] I think a mistake that you [11:41] or I might make is to think [11:42] that only the technical understanding matters. [11:45] And that's definitely part [11:46] of the conversation you want that board to have. [11:48] But there's a lot more [11:50] about how that's gonna just like impact society [11:51] and people's lives [11:53] that you really want represented in there too. [11:56] - Are you looking at the track record of people [11:59] or you're just having conversations? [12:00] - Track record's a big deal. [12:02] You, of course, have a lot of conversations. [12:08] There's some roles [12:09] where I kind of totally ignore track record [12:13] and just look at slope, [12:16] kind of ignore the y-intercept. [12:18] - Thank you. [12:19] Thank you for making it mathematical for the audience, [12:21] - For a board member, [12:22] I do care much more about the y-intercept. [12:24] I think there is something deep [12:26] to say about track record there and experiences, [12:30] something's very hard to replace. [12:32] - Do you try to fit a polynomial function [12:34] or exponential one to track record? [12:36] - That's not that. [12:37] An analogy doesn't carry that far. [12:39] - All right. [12:40] You mentioned some of the low points that weekend. [12:44] What were some of the low points psychologically for you? [12:47] Did you consider going to the Amazon jungle [12:50] and just taking Ayahuasca disappearing forever or? [12:54] - I mean, there's so many low, [12:55] like it was a very bad period of time. [12:58] There were great high points too. [13:02] My phone was just like sort of nonstop blowing up [13:05] with nice messages from people I worked with every day, [13:07] people I hadn't talked to in a decade. [13:09] I didn't get to appreciate that as much as I should have. [13:11] 'cause I was just like in the middle of this firefight, [13:13] but that was really nice. [13:14] But on the whole, [13:15] it was like a very painful weekend [13:17] and also just like a very... [13:22] It was like a battle fought in public to a surprising degree [13:26] and that was extremely exhausting to me, [13:28] much more than I expected. [13:30] I think fights are generally exhausting, [13:32] but this one really was. [13:35] The board did this Friday afternoon. [13:39] I really couldn't get much in the way of answers, [13:42] but I also was just like, [13:43] "Well, the board gets to do this." [13:45] And so I'm gonna think for a little bit [13:48] about what I want to do, [13:49] but I'll try to find the, the blessing in disguise here. [13:52] And I was like, "Well, my current job at OpenAI, [13:58] it was like to like run a decently-sized company [14:01] at this point." [14:02] And the thing I'd always liked the most was just getting [14:05] to work with the researchers. [14:07] And I was like, yeah, I can just go do like [14:08] a very focused AI research effort. [14:11] And I got excited about. [14:12] That didn't even occur to me at the time [14:14] to like possibly that this was all gonna get undone. [14:17] This was like Friday afternoon. [14:18] - Oh, so you've accepted the death- [14:21] - Very quickly, very quickly. [14:23] I mean, I went through like a little period [14:26] of confusion and rage, [14:27] but very quickly. [14:28] And by Friday night, [14:30] I was talking to people about what was gonna be next [14:33] and I was excited about that. [14:37] I think it was Friday night evening [14:40] for the first time that I heard from the exec team here, [14:42] which is like, hey, we're gonna like fight this [14:44] and we think... [14:46] Well, whatever. [14:47] And then I went to bed just still being like, okay, [14:50] excited. [14:51] - Like onward, [14:52] were you able to sleep? [14:54] - Not a lot. [14:54] It was one of the weird things was there was this [14:56] like period of four and a half days [14:59] where sort of didn't sleep much, [15:01] didn't eat much [15:02] and still kind of had like a surprising amount of energy. [15:05] You learn like a weird thing about adrenaline [15:09] and more time. [15:10] - So you kind of accepted the death of this baby OpenAI? [15:12] - And I was excited for the new thing. [15:14] I was just like, okay, this was crazy, but whatever. [15:16] - It's a very good coping mechanism. [15:18] - And then Saturday morning, [15:20] two of the board members called [15:21] and said, "Hey, we destabilize. [15:23] We didn't mean to destabilize things. [15:25] We don't restore a lot of value here. [15:27] Can we talk about you coming back?" [15:29] And I immediately didn't wanna do that, [15:32] but I thought a little more [15:34] and I was like, "Well, I really care about the people here, [15:37] the partners, shareholders. [15:39] I love this company." [15:41] And so I thought about it [15:42] and I was like, "Well, okay, [15:43] but here's the stuff I would need." [15:45] And then the most painful time of all [15:48] over the course of that weekend, [15:52] I kept thinking and being told... [15:56] Not just me, [15:56] like the whole team here kept thinking. [15:58] Well, we were trying to keep OpenAI stabilized [16:01] while the whole world was trying to break it apart, [16:02] people trying to recruit, whatever. [16:04] We kept being told like, [16:05] "All right, we're almost done, [16:06] we're almost done. [16:06] We just need like a little bit more time." [16:09] And it was this like very confusing state. [16:11] And then Sunday evening [16:13] when again like every few hours, [16:15] I expected that we were gonna be done [16:17] and we're gonna figure out a way for me to return [16:20] and things to go back to how they were, [16:23] the board then appointed a new interim CEO [16:27] and then I was like... [16:29] I mean, that feels really bad. [16:30] That was the low point of the whole thing. [16:36] You know, I'll tell you something, [16:37] it felt very painful, [16:40] but I felt a lot of love that whole weekend. [16:44] It was not other than that one moment, Sunday night, [16:47] I would not characterize my emotions as anger or hate, [16:52] but I really just like... [16:54] I felt a lot of love from people towards people. [16:59] It was like painful, [17:00] but it was like the dominant emotion [17:02] of the weekend was love, not hate. [17:04] - You've spoken highly of Mira Murati that she helped, [17:07] especially as you put in a tweet, [17:09] "In the quiet moments when it counts, [17:12] perhaps we could take a bit of a tangent." [17:14] What do you admire about Mira? [17:16] - Well, she did a great job during that weekend [17:18] in a lot of chaos, [17:20] but people often see leaders [17:23] in the crisis moments, good or bad. [17:27] But a thing I really value [17:29] in leaders is how people act on a boring Tuesday [17:33] at 9:46 in the morning [17:35] and in just sort of the normal drudgery of the day-to-day, [17:41] how someone shows up in a meeting, [17:43] the quality of the decisions they make. [17:45] That was what I meant about the quiet moments. [17:48] - Meaning like most of the work is done on a day by day [17:52] in a meeting by meeting, [17:53] just be present and make great decisions. [17:58] - Yeah. [17:59] I mean, look, what you have wanted [18:00] to spend the last 20 minutes about [18:02] and I understand is like this one very dramatic weekend. [18:06] But that's not really what OpenAI is about. [18:08] OpenAI is really about the other seven years. [18:10] - Well, yeah, human civilization is not about the invasion [18:14] of the Soviet Union by Nazi Germany, [18:16] but still that's something people totally focus on. [18:18] - Very understandable. [18:20] - It gives us an insight into human nature, [18:22] the extremes of human nature, [18:24] and perhaps some of the damage [18:25] and some of the triumphs [18:27] of human civilization can happen in those moments. [18:29] So it's like illustrative. [18:31] Let me ask you about Ilya. [18:33] Is he being held hostage in a secret nuclear facility? [18:36] - No. [18:37] - What about a regular secret facility? [18:39] - No. [18:40] - What about a nuclear non-secure facility? [18:41] - Neither, not that either. [18:43] - I mean, this is becoming a meme at some point. [18:45] You've known Ilya for a long time. [18:47] He was obviously part of this drama [18:50] with the board and all that kind of stuff. [18:54] What's your relationship with him now? [18:57] - I love Ilya. [18:58] I have tremendous respect for Ilya. [19:01] I don't have anything I can say about his plans right now. [19:03] That's a question for him. [19:06] But I really hope we work together [19:08] for certainly the rest of my career. [19:12] He's a little bit younger than me, [19:13] maybe he works a little bit longer. [19:15] - There's a meme that he saw something, [19:20] like he maybe saw AGI [19:21] and that gave him a lot of worry internally. [19:25] What did Ilya see? [19:28] - Ilya has not seen AGI, [19:30] none of us have seen AGI. [19:31] We've not built AGII. [19:35] I do think one of the many things [19:38] that I really love about Ilya is he takes AGI [19:43] and the safety concerns broadly speaking, [19:47] including things like the impact this is gonna have [19:51] on society very seriously. [19:53] And as we continue to make significant progress, [19:58] Ilya is one of the people [20:00] that I've spent the most time [20:02] over the last couple of years talking [20:04] about what this is going to mean, [20:06] what we need to do to ensure we get it right [20:08] to ensure that we succeed at the mission. [20:11] So Ilya did not see AGI. [20:17] But Ilya is a credit to humanity [20:23] in terms of how much he thinks [20:26] and worries about making sure we get this right. [20:30] - I've had a bunch of conversation with him in the past. [20:32] I think when he talks about technology, [20:34] he's always like doing this long-term thinking [20:37] type of thing. [20:38] So he is not thinking about what this is gonna be in a year. [20:40] He's thinking about in 10 years. [20:42] - [Sam] Yeah. [20:43] - Just thinking from first principles [20:44] like, okay, if the scales, [20:47] what are the fundamentals here? [20:49] Where's this going? [20:50] And so that's a foundation [20:52] for them thinking about like all the other safety concerns [20:55] and all that kind of stuff, [20:58] which makes him a really fascinating human to talk with. [21:02] Do you have any idea why he's been kind of quiet? [21:05] Is it he's just doing some soul searching? [21:08] - Again, I don't wanna speak for Ilya. [21:11] I think that you should ask him that. [21:18] He's definitely a thoughtful guy. [21:23] I think I kind of think of Ilya [21:24] as like always on a soul search in a really good way. [21:27] - Yes. [21:28] Yeah. [21:28] Also he appreciates the power of silence. [21:32] Also, I'm told he can be a silly guy, [21:34] which I've never seen that side of him. [21:35] - It's very sweet when that happens. [21:39] - I've never witnessed a silly Ilya, [21:41] but I look forward to that as well. [21:43] - I was at a dinner party with him recently [21:45] and he was playing with a puppy. [21:47] And he was like in a very silly move, very endearing [21:49] and I was thinking like, oh man, [21:50] this is like not the side of the Ilya [21:53] that the world sees the most. [21:55] - So just to wrap up this whole saga, [21:58] are you feeling good about the board structure [22:01] about all of this and where it's moving? [22:03] - I feel great about the new board. [22:05] In terms of the structure of OpenAI, [22:08] one of the board's tasks is to look at that [22:10] and see where we can make it more robust. [22:13] We wanted to get new board members in place first, [22:17] but we clearly learned a lesson about structure [22:20] throughout this process. [22:21] I don't have I think super deep things to say. [22:25] It was a crazy, very painful experience. [22:27] I think it was like a perfect storm of weirdness. [22:29] It was like a preview for me of what's gonna happen [22:32] as the stakes get higher and higher [22:33] and the need that we have like robust governance structures [22:36] and processes and people. [22:39] I am kind of happy it happened when it did, [22:42] but it was a shockingly painful thing to go through. [22:47] - Did it make you be more hesitant in trusting people? [22:50] - Yes. [22:51] - Just on a personal level. [22:52] - Yes. [22:53] - I think I'm like an extremely trusting person. [22:54] I've always had a life philosophy [22:56] of like don't worry about all of the paranoia, [22:59] don't worry about the edge cases. [23:01] You get a little bit screwed in exchange [23:05] for getting to live with your guard down. [23:07] And this was so shocking to me. [23:09] I was so caught off guard [23:11] that it has definitely changed [23:13] and I really don't like this. [23:15] It's definitely changed how I think [23:17] about just like default trust of people [23:18] and planning for the bad scenarios. [23:21] - You gotta be careful with that. [23:23] Are you worried about becoming a little too cynical? [23:26] - I'm not worried about becoming too cynical. [23:28] I think I'm like the extreme opposite of a cynical person. [23:30] But I'm worried about just becoming like less [23:33] of a default trusting person. [23:35] - I'm actually not sure which mode is best to operate in [23:38] for a person who's developing AGI, [23:42] trusting or untrusting. [23:44] It's an interesting journey you're on. [23:48] But in terms of structure, [23:50] see, I'm more interested on the human level. [23:52] How do you surround yourself [23:53] with humans that are building cool shit, [23:56] but also are making wise decisions? [24:00] Because the more money you start making, [24:02] the more power the thing has the weirder people get. [24:05] - I think you could make all kinds of comments [24:08] about the board members [24:11] and the level of trust I should have had there [24:14] or how I should have done things differently. [24:16] But in terms of the team here, [24:18] I think you'd have to like give me a very good grade [24:20] on that one. [24:23] And I have just like enormous gratitude and trust [24:26] and respect for the people that I work with every day. [24:30] And I think being surrounded [24:31] with people like that is really important. [24:39] - Our mutual friend Elon sued OpenAI. [24:44] What is the essence of what he's criticizing? [24:48] To what degree does he have a point? [24:50] To what degree is he wrong? [24:52] - I don't know what it's really about. [24:54] We started off just thinking we were gonna be a research lab [24:58] and having no idea about how this technology was gonna go. [25:03] Because it was only seven or eight years ago, [25:05] it's hard to go back [25:06] and really remember what it was like then. [25:08] But before language models were a big deal, [25:10] this was before we had any idea about an API [25:12] or selling access to a chat bot. [25:15] It was before we had any idea we were gonna productize [25:17] at all. [25:17] So we're like we're just gonna try to do research [25:20] and we don't really know what we're gonna do with that. [25:23] I think with many new fundamentally new things, [25:25] you start fumbling through the dark [25:27] and you make some assumptions, [25:29] most of which turn out to be wrong. [25:31] And then it became clear [25:34] that we were going to need to do different things [25:41] and also have huge amounts more capital. [25:44] So we said, "Okay, well, [25:45] the structure doesn't quite work for that. [25:46] How do we patch the structure?" [25:49] And then you patch it again and patch it again [25:50] and you end up with something [25:51] that does look kind of eyebrow raising to say the least. [25:56] But we got here gradually [25:57] with I think reasonable decisions [25:59] at each point along the way [26:01] and doesn't mean I wouldn't do it totally differently [26:04] if we could go back now with an oracle, [26:06] but you don't get the oracle at the time. [26:08] But anyway, [26:09] in terms of what Elon's real motivations here are [26:11] I don't know. [26:12] - To the degree you remember, [26:14] what was the response that OpenAI gave in the blog post? [26:19] Can you summarize it? [26:21] - Oh, we just said like Elon said this set of things, [26:26] here's our characterization [26:28] or here's this sort of not our characterization, [26:32] here's like the characterization of how this went down. [26:35] We tried to not make it emotional [26:37] and just sort of say like here's the history. [26:44] - I do think there's a degree of mischaracterization [26:51] from Elon here about one [26:54] of the points you just made, [26:56] which is the degree of uncertainty you had at the time. [27:00] You guys are a bunch of like a small group [27:02] of researchers crazily talking about AGI [27:06] when everybody's laughing at that thought. [27:09] - Wasn't that long ago Elon was crazily talking [27:11] about launching rockets [27:13] when people were laughing at that thought? [27:16] So I think he'd have more empathy for this. [27:20] - I mean, I do think that there's personal stuff here [27:25] that there was a split that OpenAI [27:29] and a lot of amazing people here chose [27:31] to part ways of Elon. [27:33] So there's a personal- [27:33] - Elon chose to part ways. [27:37] - Can you describe that exactly, [27:40] the choosing to part ways? [27:41] - He thought OpenAI was gonna fail. [27:44] He wanted total control to sort of turn it around. [27:46] We wanted to keep going in the direction [27:48] that now has become OpenAI. [27:50] He also wanted Tesla to be able [27:51] to build an AGI effort. [27:52] At various times, [27:53] he wanted to make OpenAI into a for-profit company [27:58] that he could have control of [27:59] or have it merged with Tesla. [28:01] We didn't want to do that [28:02] and he decided to leave, [28:04] which that's fine. [28:07] - And that's one of the things [28:08] that the blog post says is that he wanted OpenAI [28:12] to be basically acquired by Tesla [28:17] in those same way that or maybe something similar [28:19] or maybe something more dramatic [28:21] than the partnership with Microsoft. [28:23] - My memory is the proposal was just like, [28:25] yeah, like get acquired by Tesla [28:26] and have Tesla have full control over it. [28:28] I'm pretty sure that's what it was. [28:30] - So what is the word open in OpenAI mean [28:35] to Elon at the time? [28:36] Ilya has talked about this in in the email exchanges [28:39] and all this kind of stuff. [28:40] What does it mean to you at the time? [28:41] What does it mean to you now? [28:43] - I would definitely pick a diff... [28:44] Speaking of going back with an oracle, [28:45] I'd pick a different name. [28:49] One of the things that I think OpenAI is doing [28:51] that is the most important of everything [28:53] that we're doing is putting powerful technology [28:58] in the hands of people for free as a public good. [29:03] We don't run ads on our free version. [29:05] We don't monetize it in other ways. [29:08] We just say it's part of our mission. [29:10] We wanna put increasingly powerful tools [29:12] in the hands of people for free [29:13] and get them to use them. [29:15] And I think that kind of open is really important [29:21] to our mission. [29:22] I think if you give people great tools [29:23] and teach them to use them or don't even teach them, [29:25] they'll figure it out [29:26] and let them go build an incredible future [29:29] for each other with that. [29:30] That's a big deal. [29:31] So if we can keep putting free or low cost [29:34] or free and low cost powerful AI tools out in the world, [29:39] I think that's a huge deal [29:40] for how we fulfill the mission. [29:43] Open source or not, [29:44] yeah, I think we should open source some stuff [29:46] and not other stuff. [29:49] It does become this like religious battle line [29:51] where nuance is hard to have, [29:53] but I think nuance is the right answer. [29:55] - So he said change your name to ClosedAI [29:58] and I'll drop the lawsuit. [29:59] I mean, is it going to become this battleground [30:03] in the land of memes about the name? [30:05] - I think that speaks to the seriousness [30:10] with which Elon means the lawsuit. [30:18] I mean, that's like an astonishing thing to say, I think. [30:21] - Well, I don't think the lawsuit maybe, [30:23] correct me if I'm wrong, [30:25] but I don't think the lawsuit is legally serious. [30:28] It's more to make a point about the future of AGI [30:31] and the company that's currently leading the way. [30:37] - Look, I mean Grok had not open sourced anything [30:40] until people pointed out it was a little bit hypocritical [30:42] and then he announced [30:43] that Grok open source things this week. [30:45] I don't think open source versus not is [30:47] what this is really about for him. [30:48] - Well, we'll talk about open source and not. [30:50] I do think maybe criticizing the competition is great, [30:53] just talking a little shit, [30:54] that's great, [30:56] but friendly competition [30:57] versus like I personally hate lawsuits. [31:01] - Look, I think this whole thing is like unbecoming [31:03] of a builder, [31:04] and I respect Elon is one of the great builders of our time. [31:09] And I know he knows what it's like [31:14] to have like haters attack him [31:15] and it makes me extra sad he's doing the toss. [31:18] - Yeah, he is one of the greatest builders of all time, [31:20] potentially the greatest builder of all time. [31:22] - It makes me sad. [31:24] And I think it makes a lot of people sad. [31:25] There's a lot of people who've really looked up to him [31:27] for a long time and said this. [31:29] I said in some interview [31:30] or something that I missed the old Elon [31:32] and the number of messages I got being [31:34] like that exactly encapsulates how I feel. [31:36] - I think he should just win. [31:39] He should just make Grok beat GPT [31:43] and then GPT beats Grok [31:45] and it's just a competition, [31:46] and it's beautiful for everybody. [31:49] But on the question of open source, [31:51] do you think there's a lot of companies playing [31:53] with this idea? [31:54] It's quite interesting. [31:56] I would say Meta, surprisingly, has led the way on this [32:01] or like at least took the first step in the game of chess [32:05] of really open sourcing the model. [32:08] Of course, it's not the state of the art model, [32:10] but open sourcing Llama [32:12] and Google is flirting with the idea [32:16] of open sourcing a smaller version. [32:18] What are the pros and cons of open sourcing? [32:20] Have you played around with this idea? [32:22] - Yeah, I think there is definitely a place [32:25] for open source models, [32:26] particularly smaller models [32:27] that people can run locally, [32:28] I think there's huge demand for. [32:31] I think there will be some open source models, [32:33] there will be some closed source models. [32:36] It won't be unlike other ecosystems in that way. [32:39] - I listened to all in podcasts talking [32:41] about this lawsuit and all that kind of stuff [32:43] and they were more concerned about the precedent [32:47] of going from nonprofit to this cap for profit. [32:52] What precedent that sets for other startups? [32:56] - I would heavily discourage any startup [32:58] that was thinking about starting as a non-profit [33:01] and adding like a for-profit arm later. [33:03] I'd heavily discourage them from doing that. [33:04] I don't think we'll set a precedent here. [33:05] - Okay. [33:06] So most startups should go just- [33:08] - For sure. [33:09] And again, if we knew what was gonna happen, [33:11] we would've done that too. [33:12] - Well, like in theory, [33:13] if you like dance beautifully here, [33:16] there's like some tax incentives or whatever [33:19] - But I don't think that's like [33:20] how most people think about these things. [33:22] - Just not possible to save a lot of money [33:24] for a startup if you do it this way. [33:27] - No, I think there's like laws that would make [33:28] that pretty difficult. [33:30] - Where do you hope this goes with Elon? [33:33] Well, this tension, this dance, [33:36] what do you hope this? [33:38] Like if we go one, two, three years from now, [33:41] your relationship with him on a personal level too, [33:44] like friendship, friendly competition, [33:47] just all this kind of stuff. [33:51] - Yeah. [33:52] I mean, I really respect Elon. [34:00] And I hope that years in the future, [34:02] we have an amicable relationship. [34:05] - Yeah, I hope you guys have an amicable relationship [34:08] like this month [34:11] and just compete and win [34:13] and explore these ideas together. [34:17] I do suppose there's competition for talent or whatever, [34:20] but it should be friendly competition. [34:25] Just build, build cool shit. [34:28] And Elon is pretty good at building cool shit, [34:31] but so are you. [34:33] So speaking of cool shit. [34:36] Sora, there's like a million questions I could ask. [34:39] First of all, it's amazing, [34:41] it truly is amazing on a product level, [34:44] but also just on a philosophical level. [34:46] So let me just technical/philosophical ask. [34:50] What do you think it understands about the world [34:55] more or less than GPT-4, for example, [34:57] like the world model [34:59] when you train on these patches versus language tokens? [35:04] - I think all of these models understand something more [35:09] about the world model than most of us give them credit for. [35:12] And because they're also very clear things [35:15] they just don't understand or don't get right, [35:18] it's easy to look at the weaknesses, [35:19] see through the veil [35:20] and say this is all fake, [35:23] but it's not all fake. [35:24] It's just some of it works and some of it doesn't work. [35:28] I remember when I started first watching Sora videos [35:30] and I would see like a person walk [35:32] in front of something for a few seconds and occlude it [35:34] and then walk away [35:35] and the same thing was still there. [35:36] I was like, "This is pretty good." [35:39] Or there's examples [35:40] where the underlying physics looks so well represented [35:43] over a lot of steps in a sequence. [35:46] It's like oh this is like quite impressive. [35:49] But, fundamentally, these models are just getting better [35:52] and that will keep happening. [35:54] If you look at the trajectory [35:55] from DALL·E 1 to 2 to 3 to Sora, [35:59] there were a lot of people that were dunked on each version, [36:01] saying it can't do this, it can't do that [36:03] and I'm like look at it now. [36:05] - Well, the thing you just mentioned is kind of [36:08] with the occlusions is basically modeling the physics [36:12] of three dimensional physics of the world sufficiently well [36:15] to capture those kinds of things. [36:17] - Well. [36:19] - Yeah, maybe you can tell me [36:21] in order to deal with occlusions, [36:22] what does the world model need to? [36:24] - Yeah, so what I would say is it's doing something [36:26] to deal with occlusions really well. [36:28] What I represent that it has like a great underlying [36:30] 3D model of the world. [36:32] It's a little bit more of a stretch [36:33] - But can you get there through just these kinds [36:35] of two dimensional training data approaches? [36:38] - It looks like this approach is gonna go surprisingly far. [36:41] I don't wanna speculate too much [36:43] about what limits it will surmount and which it won't. [36:46] - What are some interesting limitations [36:47] of the system that you've seen? [36:50] I mean, there's been some fun ones you've posted. [36:52] - There's all kinds of fun. [36:53] I mean, like cats sprouting a extra limit [36:56] at random points in a video, [36:58] like pick what you want, [36:59] but there's still a lot of problem, [37:01] there's a lot of weaknesses. [37:02] - Do you think that's a fundamental flaw of the approach [37:06] or is it just bigger model [37:09] or better technical details or better data, [37:14] more data is going to solve the cat sprouting extremes? [37:18] - I would say yes to both. [37:20] I think there is something about the approach [37:22] which just seems to feel different [37:24] from how we think and learn and whatever. [37:28] And then also, [37:28] I think it'll get better with scale. [37:30] - I mentioned LLMs have tokens, text tokens [37:33] and Sora has visual patches [37:35] so it converts all visual data, [37:37] a diverse kinds of visual data videos [37:38] and images into patches. [37:40] Is the training [37:41] to the degree you can say fully self-supervised there? [37:44] Is there some manual labeling going on? [37:46] What's the involvement of humans in all this? [37:49] - I mean, without saying anything specific [37:51] about the Sora approach, [37:53] we use lots of human data in our work. [38:00] - But not internet scale data. [38:04] So lots of humans, [38:05] lots of complicated word, Sam. [38:08] - I think lots is a fair word in this case. [38:11] - But it doesn't [38:12] because to me, lots, [38:13] like listen, I'm an introvert [38:15] and when I hang out with like three people, [38:16] that's a lot of people. [38:17] Yeah, four people, [38:18] that's a lot. [38:19] But I suppose you mean more than- [38:21] - More than three people work on labeling the data [38:23] for these models, yeah. [38:24] - Okay. [38:25] All right. [38:26] But fundamentally, there's a lot [38:28] of self-supervised learning. [38:30] 'cause what you mentioned [38:32] in the technical report is internet scale data. [38:36] That's another beautiful, [38:36] it's like poetry. [38:39] So it's a lot of data that's not human label. [38:42] It's self-supervised in that way. [38:45] And then the question is, [38:46] how much data is there on the internet [38:49] that could be used in this that is conducive [38:52] to this kind of self-supervised way [38:54] if only we knew the details of the self-supervised? [38:57] Do you have you considered opening it up [39:00] a little more details [39:02] - We have. [39:03] You mean, for sora specifically? [39:04] - Sora specifically [39:05] because it's so interesting. [39:11] Can the same magic [39:12] of LLMs now start moving towards visual data [39:15] and what does that take to do that? [39:18] - I mean, it looks to me like yes, [39:20] but we have more work to do. [39:22] - Sure. [39:22] What are the dangers? [39:23] Why are you concerned about releasing the system? [39:27] What are some possible dangers of this? [39:29] - I mean, frankly speaking, [39:30] one thing we have to do [39:31] before releasing the system is just like get it [39:35] to work at a level of efficiency [39:38] that will deliver the scale people are gonna want from this. [39:41] So that I don't wanna like downplay that [39:43] and there's still a ton of work to do there. [39:47] But you can imagine like issues [39:52] with deep fakes, misinformation. [39:56] We try to be a thoughtful company [39:58] about what we put out into the world [40:00] and it doesn't take much thought [40:02] to think about the ways this can go badly. [40:05] - There's a lot of tough questions here. [40:08] You're dealing in a very tough space. [40:09] Do you think training AI should be [40:12] or is fair use under copyright law? [40:14] - I think the question behind that question is, [40:17] do people who create valuable data deserve to have some way [40:20] that they get compensated for use of it? [40:22] And that I think the answer is yes. [40:26] I don't know yet what the answer is. [40:27] People have proposed a lot of different things. [40:29] We've some tried some different models. [40:31] But if I'm like an artist, for example, [40:37] I would like to be able to opt out [40:38] of people generating art in my style [40:41] and B, if they do generate art in my style, [40:43] I'd like to have some economic model associated with that. [40:46] - Yeah, it's that transition from CDs to Napster to Spotify. [40:51] We have to figure out some kind of model. [40:52] - The model changes, [40:53] but people have gotta get paid. [40:55] - Well, there should be some kind [40:57] of incentive if we zoom out even more for humans [41:00] to keep doing cool shit. [41:02] - Everything I worry about, [41:03] humans are gonna do cool shit [41:04] and society's gonna find some way to reward it. [41:08] That seems pretty hardwired. [41:10] We want to create. [41:11] We want to be useful. [41:12] We want to achieve status in whatever way [41:15] that's not going anywhere, [41:16] I don't think. [41:17] - But the reward might not be monetary, financial. [41:20] It might be like fame and celebration of other cool- [41:25] - Maybe financial in some other way. [41:27] Again, I don't think we've seen like the last evolution [41:29] of how the economic system's gonna work. [41:31] - Yeah. [41:32] But artists and creators are worried. [41:34] When they see Sora, [41:35] they're like, "Holy shit." [41:36] - Sure. [41:37] Artists were also super worried when photography came out. [41:40] And then photography became a new art form [41:42] and people made a lot of money taking pictures. [41:45] And I think things like that will keep happening. [41:47] People will use the new tools in new ways. [41:50] - If we just look on YouTube or something like this, [41:53] how much of that will be using Sora, [41:56] like AI-generated content do you think [41:59] in the next five years? [42:01] - People talk about like how many jobs is AI gonna do [42:03] in five years [42:05] and the framework that people have is what percentage [42:08] of current jobs are just gonna be totally replaced [42:10] by some AI doing the job? [42:13] The way I think about it is not what percent [42:15] of jobs AI will do, [42:16] but what percent of tasks will AI do [42:18] and over what time horizon. [42:19] So if you think of all of the like-five second tasks [42:22] in the economy, five-minute tasks, the five-hour tasks, [42:25] maybe even the five-day tasks, [42:27] how many of those can AI do? [42:29] And I think that's a way more interesting, impactful, [42:33] important question than how many jobs AI can do [42:37] because it is a tool [42:39] that will work at increasing levels of sophistication [42:42] and over longer and longer time horizons [42:45] for more and more tasks [42:46] and let people operate at a higher level of abstraction. [42:49] So maybe people are way more efficient at the job they do. [42:53] And at some point, [42:54] that's not just a quantitative change, [42:55] but it's a qualitative one too about the kinds [42:58] of problems you can keep in your head. [43:00] I think that for videos on YouTube, [43:02] it'll be the same. [43:04] Many videos, [43:04] maybe most of them, [43:06] will use AI tools in the production, [43:08] but they'll still be fundamentally driven [43:10] by a person thinking about it, [43:12] putting it together, [43:13] doing parts of it, [43:15] sort of directing it and running it. [43:18] - Yeah, it's so interesting. [43:19] I mean, it's scary, [43:20] but it's interesting to think about. [43:22] I tend to believe that humans like to watch other humans [43:26] or other human like- [43:26] - Humans really care about other humans a lot. [43:29] - Yeah. [43:30] If there's a cooler thing that's better than a human, [43:34] humans care about that for like two days [43:37] and then they go back to humans. [43:39] - That seems very deeply wired. [43:41] - It's the whole chest thing. [43:44] But now let's everybody keep playing chess [43:46] and let's ignore the alpha in the room [43:49] that humans are really bad at chess relative to AI systems. [43:52] - We still run races and cars are much faster. [43:54] I mean, there's like a lot of examples. [43:56] - Yeah. [43:56] And maybe it'll just be tooling [43:59] like in the Adobe suite type of way [44:02] where it can just make videos much easier [44:04] and all that kind of stuff. [44:07] Listen, I hate being in front of the camera. [44:09] If I can figure out a way [44:10] to not be in front of the camera, [44:12] I would love it. [44:13] Unfortunately, it'll take a while. [44:14] Like that generating faces, [44:17] it's getting there, [44:18] but generating faces and video format is tricky [44:20] when it's specific people versus generic people. [44:24] Let me ask you about GPT-4. [44:27] There's so many questions. [44:28] First of all, also amazing. [44:33] Looking back, it'll probably be this kind [44:35] of historic pivotal moment [44:37] with three, five, and four which had GPT. [44:40] - Maybe five will be the pivotal moment. [44:41] I don't know. [44:43] Hard to say that looking forwards. [44:44] - We never know. [44:45] That's the annoying thing about the future, [44:46] it's hard to predict. [44:48] But for me, [44:49] looking back GPT-4, [44:50] ChatGPT is pretty impressive, [44:53] historically impressive. [44:55] So allow me to ask, [44:57] what's been the most impressive capabilities of GPT-4 to you [45:00] and GPT-4 Turbo? [45:05] - I think it kind of sucks. [45:07] - Hmm. [45:08] Typical human also gotten used to an awesome thing. [45:11] - No, I think it is an amazing thing, [45:14] but relative to where we need to get to [45:18] and where I believe we will get to, [45:22] at the time of like GPT-3, [45:25] people were like, "Oh this is amazing. [45:27] This is this like marvel of technology," [45:29] and it is, it was. [45:31] But now we have GPT-4 and look at GPT-3 [45:35] and you're like that's unimaginably horrible. [45:39] I expect that the delta between five [45:41] and four will be the same as between four and three. [45:44] And I think it is our job to live a few years in the future [45:48] and remember that the tools we have now are gonna kind [45:52] of suck looking backwards at them [45:55] and that's how we make sure the future is better. [45:59] - What are the most glorious ways that GPT-4 sucks? [46:04] Meaning- [46:05] - [Sam] What are the best things it can do? [46:06] - What are the best things it can do in the limits [46:09] of those best things that allow you to say it sucks, [46:12] therefore gives you an inspiration and hope for the future? [46:16] - One thing I've been using it [46:17] for more recently is sort of a like a brainstorming partner. [46:23] - [Lex] Yep. [46:24] Almost for that. [46:25] - There's a glimmer of something amazing in there. [46:30] I don't think it gets... [46:31] When people talk about it, [46:33] what it does, [46:33] they're like, "Helps me code more productively. [46:36] It helps me write more faster and better. [46:38] It helps me translate from this language to another," [46:42] all these like amazing things. [46:43] But there's something about the kind [46:50] of creative brainstorming partner. [46:52] I need to come up with a name for this thing. [46:53] I need to think about this problem in a different way. [46:55] I'm not sure what to do here. [46:58] That I think like gives a glimpse [47:00] of something I hope to see more of. [47:03] One of the other things [47:04] that you can see a very small glimpse of is [47:10] what I can help on longer horizon tasks. [47:12] Break down something in multiple steps, [47:14] maybe execute some of those steps, [47:16] search the internet, write code, whatever. [47:18] Put that together. [47:20] When that works, [47:21] which is not very often, [47:22] it's like very magical, [47:24] - The iterative back and forth with a human. [47:27] It works a lot for me. [47:28] What do you mean it works? [47:29] - Iterative back and forth to human, [47:30] it can get more often, [47:31] when it can go do like a 10-step problem on its own. [47:34] It doesn't work for that too often sometimes. [47:37] - Add multiple layers of abstraction [47:39] or do you mean just sequential? [47:40] - Both like to break it down [47:43] and then do things that different layers [47:45] of abstraction put them together. [47:47] Look, I don't wanna downplay the accomplishment of GPT-4, [47:53] but I don't wanna overstate it either. [47:55] And I think this point that we are on an exponential curve, [47:58] we'll look back relatively soon at GPT-4 [48:01] like we look back at GPT-3 now. [48:03] - That said, I mean ChatGPT was the transition [48:08] to where people like started [48:10] to believe there is an uptick of believing. [48:14] Not internally at OpenAI perhaps. [48:16] There's believers here, [48:18] but when you think- [48:19] - And in that sense, [48:20] I do think it'll be a moment [48:21] where a lot of the world went [48:22] from not believing to believing. [48:25] That was more about the ChatGPT interface. [48:28] And by the interface and product, [48:30] I also mean the post-training of the model [48:32] and how we tune it to be helpful to you [48:34] and how to use it than the underlying model itself. [48:38] - How much of each of those things are important? [48:43] The underlying model and the RLHF [48:47] or something of that nature that tunes it [48:50] to be more compelling to the human, [48:52] more effective and productive for the human. [48:55] - I mean, they're both super important [48:57] but the RLHF, the post-training step, [49:00] the little wrapper of things [49:02] that from a compute perspective, [49:05] little wrapper of things that we do on top [49:06] of the base model, [49:07] even though it's a huge amount of work. [49:08] That's really important to say nothing of the product [49:10] that we build around it. [49:16] In some sense, [49:17] we did have to do two things. [49:19] We had to invent we underlying technology [49:22] and then we had to figure out [49:27] how to make it into a product people would love, [49:30] which is not just about the actual product work itself, [49:32] but this whole other step [49:34] of how you align it and make it useful [49:37] - And how you make the scale work [49:40] where a lot of people can use it at the same time, [49:42] all that kind of stuff. [49:44] - But that was like a known difficult thing. [49:47] We knew we were gonna have to scale it up. [49:49] We had to go do two things [49:50] that had like never been done before [49:52] that were both, like, [49:54] I would say quite significant achievements [49:55] and then a lot of things like scaling it up [49:58] that other companies have had to do before. [50:01] - How does the context window [50:03] of going from 8K to 128K tokens compare [50:08] from GPT-4 to to GPT-4 Turbo? [50:13] - Most people don't need all the way to 128, [50:15] most of the time although. [50:18] If we dream into the distant future, [50:19] we'll have like way distant future, [50:21] we'll have like context length of several billion. [50:24] You will feed in all of your information, [50:26] all of your history time, [50:27] and it'll just get to know you better and better [50:30] and that'll be great. [50:32] For now, [50:33] the way people use these models, [50:34] they're not doing that. [50:35] And people sometimes post in a paper [50:39] or a significant fraction of a code repository, whatever. [50:45] But most usage [50:46] of the models is not using the long context [50:48] most of the time. [50:50] - I like that this is your I have a dream speech. [50:54] One day, you'll be judged by the full context [50:56] of your character or of your whole lifetime. [51:00] That's interesting. [51:01] So like that's part of the expansion [51:02] that you're hoping for is a greater and greater context. [51:06] - I saw this internet clip once. [51:07] I'm gonna get the numbers wrong, [51:09] but it was like Bill Gates talking about the amount [51:10] of memory on some early computer. [51:13] Maybe it was 64K, [51:14] maybe 640K, something like that. [51:16] And most of it was used for the screen buffer. [51:19] And he just couldn't seem genuine. [51:22] This couldn't imagine [51:23] that the world would eventually need gigabytes of memory [51:26] in a computer or terabytes of memory in a computer. [51:31] And you always do [51:34] or you always do just need [51:35] to follow the exponential of technology. [51:38] We will find out how to use better technology. [51:41] So I can't really imagine what it's like right now [51:43] for context links to go out to the billion someday [51:46] and they might not literally go there, [51:47] but effectively it'll feel like that. [51:51] But I know we'll use it [51:53] and really not wanna go back once we have it. [51:56] - Yeah, even saying billions 10 years [51:58] from now might seem dumb [52:00] because it'll be like trillions upon trillions. [52:04] - [Sam] Sure. [52:04] - There'd be some kind of breakthrough [52:07] that will effectively feel like infinite context. [52:10] But even 120, [52:11] I have to be honest, [52:12] I haven't pushed it to that degree. [52:14] Maybe putting in entire books [52:16] or like parts of books and so on, papers. [52:20] What are some interesting use cases [52:22] of GPT-4 that you've seen? [52:23] - The thing that I find most interesting is not any [52:26] particular use case that we can talk about those, [52:27] but it's people who kind of like... [52:31] This is mostly younger people, [52:32] but people who use it as like their default start [52:36] for any kind of knowledge work task. [52:39] And it's the fact that it can do a lot [52:40] of things reasonably well. [52:41] You can use GPTV, [52:42] you can use it to help you write code, [52:44] you can use it to help you do search, [52:45] you can use it to edit a paper. [52:48] The most interesting to me is the people [52:49] who just use it as the start of their workflow. [52:52] - I do as well for many things. [52:54] Like I use it as a reading partner for reading books. [52:59] It helps me think, [53:00] help me think through ideas, [53:01] especially when the books are classic, [53:03] so it's really well written about [53:04] and it actually is... [53:08] I find it often to be significantly better [53:10] than even like Wikipedia on well-covered topics. [53:13] It's somehow more balanced and more nuanced, [53:16] or maybe it's me, [53:17] but it inspires me [53:19] to think deeper than a Wikipedia article does. [53:21] I'm not exactly sure what that is. [53:22] You mentioned like this collaboration, [53:24] I'm not sure where magic is, [53:25] if it's in here or if it's in there [53:26] or if it's somewhere in between. [53:29] I'm not sure. [53:30] But one of the things that concerns me [53:33] for knowledge task when I start with GPT is [53:36] I'll usually have to do fact checking after, [53:39] like check that it didn't come up with fake stuff. [53:43] How do you figure that out [53:46] that GPT can come up with fake stuff [53:51] that sounds really convincing? [53:53] So how do you ground it in truth? [53:55] - That's obviously an area of intense interest for us. [53:58] I think it's gonna get a lot better with upcoming versions, [54:03] but we'll have to continue to work on it [54:04] and we're not gonna have it like all solved this year. [54:07] - Well, the scary thing is like as it gets better. [54:10] You'll start not doing the fact checking more and more, [54:12] right? [54:14] - I'm of two minds about that. [54:16] I think people are like much more sophisticated users [54:18] of technology than we often give them credit for [54:20] and people seem to really understand that GPT, [54:23] any of these models hallucinate some of the time [54:25] and if it's mission critical, [54:26] you gotta check it. [54:27] - Except journalists don't seem to understand that. [54:29] I've seen journalists half-assedly just using GPT-4. [54:34] - Of the long list of things I'd like [54:36] to dunk on journalists for, [54:37] this is not my top criticism of them. [54:40] - Well, I think the bigger criticism is perhaps [54:43] the pressures and the incentives [54:45] of being a journalist is that you have [54:47] to work really quickly and this is a shortcut. [54:49] I would love our society to incentivize like- [54:53] - [Sam] I would too. [54:56] - Journalistic efforts that take days and weeks [54:59] and rewards great in depth journalism. [55:02] Also journalism that represent stuff in a balanced way [55:05] where it's like celebrates people [55:07] while criticizing them [55:08] even though the criticism is the thing that gets clicks [55:11] and making up also gets clicks [55:13] and headlines that mischaracterize completely. [55:16] I'm sure you have a lot of people dunking on... [55:19] Well, all that drama probably got a lot of clicks. [55:21] - Probably did. [55:24] - And that's a bigger problem about human civilization. [55:29] I would love to see solved is where we celebrate a bit more. [55:32] You've given ChatGPT the ability to have memories. [55:34] You've been playing with that about previous conversations [55:39] and also the ability to turn off memory, [55:41] which I wish I could do that sometimes, [55:43] just turn on and off depending. [55:45] I guess sometimes alcohol can do that, [55:47] but not optimally, I suppose. [55:51] What have you seen through that, [55:52] like playing around with that idea [55:54] of remembering conversations and not? [55:56] - We're very early in our explorations here, [55:58] but I think what people want [56:00] or at least what I want for myself is a model [56:04] that gets to know me and gets more useful to me over time. [56:09] This is an early exploration. [56:12] I think there's like a lot of other things to do, [56:15] but that's where we'd like to head. [56:16] You'd like to use a model [56:18] and over the course of your life [56:19] or use a system, [56:20] it'd be many models. [56:21] And over the course of your life, [56:23] it gets better and better. [56:25] - Yeah. [56:26] How hard is that problem? [56:27] 'Cause right now, [56:28] it's more like remembering little factoids [56:31] and preferences and so on. [56:33] What about remembering, [56:33] like don't you want GPT [56:35] to remember all the shit you went through in November [56:39] and all the drama and then you can- [56:40] - Yeah, yeah, yeah. [56:41] - Because right now, [56:41] you're clearly blocking it out a little bit. [56:43] - It's not just that I want it to remember that. [56:45] I want it to integrate the lessons of that [56:49] and remind me in the future what to do differently [56:56] or what to watch out for. [56:58] And we all gain from experience [57:02] over the course of our lives, varying degrees. [57:05] And I'd like my AI agent to gain with that experience too. [57:10] So if we go back [57:11] and let ourselves imagine [57:12] that trillions and trillions of contact length, [57:17] if I can put every conversation I've ever had [57:19] with anybody in my life in there, [57:21] if I can have all of my emails input out... [57:23] Like all of my input/output [57:24] in the context window every time I ask a question, [57:27] that'd be pretty cool, I think. [57:29] - Yeah, I think that would be very cool. [57:31] People sometimes will hear that [57:33] and be concerned about privacy. [57:35] What do you think about that aspect of it, [57:39] the more effective the AI becomes [57:41] that really integrating all the experiences [57:44] and all the data that happened to you [57:46] and give you advice? [57:48] - I think the right answer there is just user choice. [57:50] Anything I want stricken from the record from my AI agent, [57:53] I wanna be able to take out. [57:54] If I don't want to remember anything, [57:55] I want that too. [57:57] You and I may have different opinions [58:00] about where on that privacy utility trade off [58:03] for our own AI we wanna be, [58:04] which is totally fine. [58:06] But I think the answer is just like really easy user choice. [58:08] - But there should be some high level [58:11] of transparency from a company about the user choice [58:14] 'cause sometimes company in the past, [58:15] companies in the past have been kind [58:17] of absolutely shady about like, yeah, [58:20] it's kind of presumed that we're collecting all your data [58:23] and we're using it for a good reason [58:25] for advertisement and so on. [58:27] But there's not a transparency about the details of that. [58:31] - That's totally true. [58:32] You mentioned earlier [58:33] that I'm like blocking out the November stuff. [58:35] - I'm just teasing you. [58:36] - Well, I mean I think it was a very traumatic thing [58:41] and it did immobilize me for a long period of time. [58:44] Like definitely the hardest, [58:47] like the hardest work thing I've had [58:49] to do was just like keep working that period [58:52] because I had to try to come back in here [58:54] and put the pieces together [58:55] while I was just like in sort of shock and pain. [59:00] Nobody really cares about that. [59:01] I mean, the team gave me a pass [59:02] and I was not working at my normal level, [59:04] but there was a period where I was just, like, [59:07] it was really hard to have to do both. [59:08] But I kind of woke up one morning [59:11] and I was like, "This was a horrible thing to happen to me. [59:13] I think I could just feel like a victim forever," [59:16] or I can say, "This is like the most important work [59:18] I'll ever touch in my life [59:19] and I need to get back to it." [59:21] And it doesn't mean that I've repressed it [59:23] because sometimes I wake in the middle [59:26] of the night thinking about it, [59:28] but I do feel like an obligation to keep moving forward. [59:32] - Well, that's beautifully said, [59:33] but there could be some lingering stuff in there. [59:36] What I would be concerned about is [59:40] that trust thing that you mentioned [59:42] that being paranoid about people [59:45] as opposed to just trusting everybody or most people, [59:48] like using your gut. [59:49] It's a tricky dance for sure. [59:51] I mean, because I've seen in my part-time explorations, [59:58] I've been diving deeply into the Zelensky administration, [60:04] the Putin administration and the dynamics there in wartime [60:07] in a very highly stressful environment. [60:10] And what happens is distrust [60:12] and you isolate yourself both [60:15] and you start to not see the world clearly. [60:18] And that's a concern, [60:19] that's a human concern. [60:20] You seem to have taken it in stride [60:22] and kind of learned the good lessons and felt the love [60:24] and let the love energize you, [60:27] which is great, [60:28] but still can linger in there. [60:30] There's just some questions I would love [60:32] to ask your intuition about what's GPT able to do and not. [60:38] So it's allocating approximately the same amount of compute [60:42] for each token it generates. [60:44] Is there room there in this kind of approach [60:48] to slower thinking, sequential thinking? [60:51] - I think there will be a new paradigm [60:53] for that kind of thinking. [60:55] - Will it be similar like architecturally [60:58] as what we're seeing now with LLMs? [61:00] Is it a layer on top of the LLMs? [61:04] - I can imagine many ways to implement that. [61:06] I think that's less important [61:08] than the question you were getting out, [61:10] which is do we need a way to do a slower kind [61:14] of thinking where the answer doesn't have to get like... [61:21] I guess like spiritually, [61:22] you could say that you want an AI [61:23] to be able to think harder about a harder problem [61:26] and answer more quickly about an easier problem. [61:28] And I think that will be important. [61:30] - Is that like a human thought that we're just having, [61:32] you should be able to think hard? [61:33] Is that a wrong intuition? [61:34] - I suspect that's a reasonable intuition. [61:37] - Interesting. [61:37] So it's not possible [61:38] once the GPT gets like GPT-7 would [61:42] just be instantaneously be able to see, [61:46] here's the proof of from RSTM. [61:49] - It seems to me like you want to be able [61:51] to allocate more compute to harder problems. [61:56] It seems to me that a system knowing [62:02] if you ask a system like that, [62:04] proof from us last theorem versus... [62:09] What's today's date? [62:11] Unless it already knew and had memorized the answer [62:14] to the proof, [62:15] assuming it's gotta go figure that out, [62:18] seems like that will take more compute. [62:20] - But can it look like a basically LLM talking to itself, [62:23] that kind of thing? [62:25] - Maybe. [62:26] I mean, there's a lot of things that [62:27] you could imagine working what the right [62:32] or the best way to do that will be. [62:35] We don't know. [62:37] - This does make me think of the mysterious, [62:40] the lore behind Q-Star. [62:43] What's this mysterious Q-Star project? [62:45] Is it also in the same nuclear facility? [62:50] - There is no nuclear facility. [62:52] - That's what a person [62:53] with a nuclear facility always says. [62:54] - I would love to have a secret nuclear facility. [62:57] There isn't one. [62:59] - All right. [63:00] - Maybe someday. [63:01] - Someday. [63:02] All right. [63:04] One can dream- [63:05] - OpenAI is not a good company to keeping secrets. [63:07] It would be nice. [63:08] We're like been plagued by a lot of leaks [63:10] and it would be nice if we were able [63:12] to have something like that. [63:14] - Can you speak to what Q-Star is? [63:15] - We are not ready to talk about that. [63:17] - See, but an answer like that means there's something [63:19] to talk about. [63:20] It's very mysterious, Sam. [63:22] - I mean, we work on all kinds of research. [63:28] We have said for a while [63:30] that we think better reasoning [63:34] in these systems is an important direction [63:38] that we'd like to pursue. [63:41] We haven't cracked the code yet. [63:44] We're very interested in it. [63:48] - Is there gonna be moments Q-Star [63:50] or otherwise where there's going to be leaps similar [63:54] to GPT where you're like- [63:56] - That's a good question. [63:59] What do I think about that? [64:05] It's interesting to me it all feels pretty continuous. [64:08] - This is kind of a theme [64:09] that you're saying is you're basically gradually going up [64:12] an exponential slope. [64:15] But from an outsider's perspective for me, [64:17] just watching it that it does feel like there's leaps, [64:20] but to you there isn't. [64:22] - I do wonder if we should have... [64:24] So part of the reason [64:25] that we deploy the way we do is that we think, [64:29] we call it iterative deployment. [64:33] Rather than go build in secret [64:34] until we got all the way to GPT-5, [64:36] we decided to talk about GPT 1, 2, 3 and 4. [64:39] And part of the reason there is, I think, AI [64:41] and surprise don't go together. [64:44] And also the world, people, institutions, [64:46] whatever you wanna call it, [64:47] need time to adapt and think about these things. [64:50] And I think one of the best things [64:52] that OpenAI has done is this strategy [64:55] and we get the world to pay attention to the progress [64:58] to take AGI seriously [65:00] to think about what systems, and structures, [65:04] and governance we want in place before, [65:06] we're like under the gun [65:07] and have to make a rush decision. [65:08] I think that's really good. [65:09] But the fact that people like you [65:11] and others say you still feel like [65:15] there are these leaps makes me think [65:17] that maybe we should be doing our releasing [65:20] even more iteratively. [65:22] I don't know what that would mean. [65:22] I don't have an answer ready to go. [65:24] But our goal is not to have shock updates to the world. [65:28] The opposite. [65:29] - Yeah, for sure. [65:30] More iterative would be amazing. [65:32] I think that's just beautiful for everybody. [65:34] - But that's what we're trying to do. [65:35] That's like our state of the strategy [65:37] and I think we're somehow missing the mark. [65:39] So maybe we should think about releasing GPT-5 [65:41] in a different way or something like that. [65:44] - Yeah, 4.71, 4.72. [65:47] But people tend to like to celebrate, [65:49] people celebrate birthdays. [65:50] I don't know if you know humans, [65:52] but they kind of have these milestones. [65:54] - I do know some humans. [65:56] People do like milestones. [65:59] I totally get that. [66:02] I think we like milestones too. [66:05] It's like fun to say, [66:06] declare victory on this one [66:08] and go start the next thing. [66:09] But, yeah, I feel like we're somehow getting [66:11] this a little bit wrong. [66:13] - So when is GPT-5 coming out again? [66:15] - I don't know. [66:16] That's an honest answer. [66:18] - Oh, that's the honest answer. [66:20] Is it blink twice if it's this year? [66:29] - We will release an amazing model this year. [66:34] I don't know what we'll call it. [66:36] - So that goes to the question of like, [66:38] what's the way we release this thing? [66:41] - We'll release, [66:43] over in the coming months, [66:45] many different things. [66:47] I think they'll be very cool. [66:49] I think before we talk [66:51] about like a GPT-5 like model called that [66:53] or called or not called that [66:55] or a little bit worse [66:55] or a little bit better than what you'd expect [66:57] from a GPT-5, [66:58] I know we have a lot of other important things [67:00] to release first. [67:02] - I don't know what to expect from GPT-5. [67:06] You're making me nervous and excited. [67:08] What are some of the biggest challenges in bottlenecks [67:10] to overcome for whatever it ends up being called, [67:14] but let's call it GPT-5? [67:16] Just interesting to ask, [67:18] is it on the compute side? [67:20] Is it in the technical side? [67:21] - It's always all of these. [67:23] What's the one big unlock? [67:24] Is it a bigger computer? [67:26] Is it like a new secret? [67:27] Is it something else? [67:30] It's all of these things together. [67:31] The thing that OpenAI I think does really well, [67:36] this is actually an original Ilya quote [67:37] that I'm gonna butcher, [67:38] but it's something like we multiply 200 medium-sized [67:45] things together into one giant thing. [67:47] - So there's this distributed constant innovation happening. [67:50] - [Sam] Yeah. [67:51] - So even on the technical side, [67:53] like- [67:54] - Especially on the technical side. [67:55] - So like even like detailed approaches, [67:57] like detailed aspects of every... [67:59] How does that work with different disparate teams and so on? [68:02] How do the medium-sized things become [68:05] one whole giant transformer? [68:07] How does this- [68:08] - There's a few people who have [68:09] to think about putting the whole thing together, [68:11] but a lot of people try [68:12] to keep most of the picture in their head. [68:14] - Oh, like the individual teams, [68:15] individual contributors tried [68:16] to keep a big picture- - At a high level. [68:17] Yeah, you don't know exactly [68:19] how every piece works, of course. [68:20] But one thing I generally believe is [68:24] that it's sometimes useful to zoom out [68:26] and look at the entire map. [68:28] And I think this is true for like a technical problem. [68:33] I think this is true for like innovating in business. [68:38] But things come together in surprising ways [68:41] and having an understanding of that whole picture. [68:44] Even if most of the time, [68:46] you're operating in the weeds in one area, [68:48] pays off with surprising insights. [68:51] In fact, one of the things that I used to have [68:54] and I think was super valuable was I used [68:56] to have like a good map of all of the frontier [69:01] or most of the frontiers in the tech industry. [69:04] And I could sometimes see these connections [69:06] or new things that were possible [69:08] that if I were only deep in one area, [69:10] I wouldn't be able to have the idea for [69:13] because I wouldn't have all the data [69:15] and I don't really have that much anymore. [69:17] I'm like super deep now. [69:21] But I know that it's a valuable thing. [69:23] - You're not the man you used to be, Sam. [69:25] - Very different job now than what I used to have. [69:28] - Speaking of zooming out, [69:29] let's zoom out to another cheeky thing, [69:34] but profound thing perhaps that you said. [69:37] You tweeted about needing $7 trillion. [69:41] - I did not tweet about that. [69:42] I never said like we're raising $7 trillion [69:44] or blah blah blah. [69:45] - Oh, that's somebody else. [69:46] - [Sam] Yeah. [69:47] - Oh, but you said it, [69:48] "Fuck it, maybe eight," I think. [69:50] - Okay. I meme like once there's like misinformation out [69:52] in the world. [69:53] - Oh, you meme. [69:54] But sort of misinformation may have a foundation [69:58] of insight there. [70:00] - Look, I think compute is gonna be the currency [70:03] of the future. [70:04] I think it will be maybe the most precious commodity [70:06] in the world. [70:08] And I think we should be investing heavily [70:11] to make a lot more compute. [70:13] Compute is... [70:19] I think it's gonna be an unusual market. [70:22] People think about the market for like chips [70:28] for mobile phones or something like that. [70:31] And you can say that, okay, [70:32] there's 8 billion people in the world, [70:33] maybe 7 billion of them have phones [70:35] or 6 billion, let's say. [70:37] They upgrade every two years, [70:38] so the market per year is 3 billion system [70:40] on chip for smartphones. [70:42] And if you make 30 billion, [70:43] you will not sell 10 times as many phones [70:45] because most people have one phone. [70:50] But compute is different, [70:52] like intelligence is gonna be more like energy [70:54] or something like that [70:55] where the only thing that I think makes sense [70:57] to talk about is at price X, [71:02] the world will use this much compute [71:04] and at price Y, [71:05] the world will use this much compute [71:08] because if it's really cheap, [71:09] I'll have it like reading my email all day, [71:11] like giving me suggestions [71:12] about what I maybe should think about or work on [71:14] and trying to cure cancer. [71:16] And if it's really expensive, [71:17] maybe I'll only use it [71:18] or will only use it, [71:19] try to cure cancer. [71:20] So I think the world is gonna want a tremendous amount [71:23] of compute. [71:24] And there's a lot of parts of that that are hard. [71:27] Energy is the hardest part. [71:29] Building data centers is also hard. [71:30] The supply chain is harder [71:31] than, of course, fabricating enough chips is hard. [71:35] But this seems to me where things are going. [71:37] Like we're gonna want an amount of compute that's just hard [71:39] to reason about right now. [71:43] - How do you solve the energy puzzle? [71:45] Nuclear. [71:46] - That's what I believe. [71:47] - Fusion? [71:48] - That's what I believe. [71:49] - Nuclear fusion. - Yeah. [71:51] - Who's gonna solve that? [71:53] - I think Helion's doing the best work, [71:54] but I'm happy there's like a race for fusion right now. [71:56] Nuclear fusion, I think, is also like quite amazing [72:00] and I hope as a world, [72:01] we can re-embrace that. [72:03] It's really sad to me how the history of that went [72:05] and hope we get back to it in a meaningful way. [72:08] - So to you, [72:09] part of the puzzle is nuclear fusion, [72:10] like nuclear reactors as we currently have them [72:12] and a lot of people are terrified [72:14] because of Chernobyl and so on. [72:16] - Well, I think we should make new reactors. [72:18] I think it's a shame that industry kind of ground to a halt. [72:22] - Just mass hysteria is how you explain the halt. [72:25] - Yeah. [72:26] - I don't know if you know humans, [72:28] but that's one of the dangers, [72:29] that's one of the security threats [72:31] for nuclear fusion is humans seem to be really afraid of it. [72:38] And that's something we have [72:39] to incorporate into the calculus of it. [72:40] So we have to kind of win people over [72:42] and to show how safe it is. [72:44] - I worry about that for AI. [72:47] I think some things are gonna go theatrically wrong with AI. [72:53] I don't know what the percent chances [72:54] that I eventually get shot, [72:55] but it's not zero. [72:57] - Oh like we wanna stop this. [73:00] - [Sam] Maybe. [73:03] - How do you decrease the theatrical nature of it? [73:06] I've already starting to hear rumblings [73:10] 'cause I do talk to people on both sides [73:13] of the political spectrum here, [73:15] rumblings where it's going to be politicized, [73:17] AI is going to be politicized, really, really worries me [73:20] because then it's like maybe the right is against AI [73:25] and the left is for AI [73:27] 'cause it's going to help the people or whatever. [73:28] Whatever the narrative and the formulation is, [73:30] that really worries me. [73:32] And then the theatrical nature of it can be leveraged fully. [73:36] How do you fight that? [73:38] - I think it will get caught up [73:39] in like left versus right wars. [73:41] I don't know exactly what that's gonna look like, [73:43] but I think that's just what happens [73:44] with anything of consequence, unfortunately. [73:46] What I meant more about theatrical risks is [73:49] like AI is gonna have, I believe, [73:54] tremendously more good consequences than bad ones, [73:56] but it is gonna have bad ones. [73:57] And there'll be some bad ones that are bad, [74:04] but not theatrical. [74:07] A lot more people have died of air pollution [74:09] than nuclear reactors, for example. [74:13] But most people worry more about living next [74:15] to a nuclear reactor than a coal plant. [74:18] But something about the way we're wired is that [74:21] although there's many different kinds [74:23] of risks we have to confront, [74:25] the ones that make a good climax scene [74:27] of a movie carry much more weight with us [74:30] than the ones that are very bad over a long period of time, [74:33] but on a slow burn. [74:36] - Well, that's why truth matters [74:37] and hopefully AI can help us see the truth of things [74:40] to have balance to understand what are the actual risks, [74:44] what are the actual dangers of things in the world. [74:47] What are the pros and cons of the competition in the space [74:51] and competing with Google, Meta, xAI and others? [74:56] - I think I have a pretty straightforward answer to this [75:00] that maybe I can think of more nuance later. [75:02] But the pros seem obvious, [75:03] which is that we get better products [75:05] and more innovation faster and cheaper [75:07] and all the reasons competition is good. [75:10] And the con is that I think if we're not careful, [75:13] it could lead to an increase in sort of an arms race [75:19] that I'm nervous about. [75:21] - Do you feel the pressure [75:22] of the arms race in some negative co- [75:25] - Definitely in some ways, for sure. [75:27] We spend a lot of time talking about the need [75:31] to prioritize safety. [75:33] And I've said for like a long time [75:35] that I think if you think of a quadrant of slow timelines [75:41] to the start of AGI, [75:42] long timelines and then a short takeoff or a fast takeoff, [75:46] I think short timelines, slow takeoff is the safest quadrant [75:50] and the one I'd most like us to be in. [75:52] But I do wanna make sure we get that slow takeoff. [75:55] - Part of the problem I have [75:57] with this kind of slight beef with Elon is [75:59] that there's silos are created [76:00] as opposed to collaboration on the safety aspect [76:03] of all of this, [76:05] it tends to go into silos and closed open source perhaps [76:09] in the model. [76:10] - Elon says at least that he cares a great deal [76:12] about AI safety and is really worried about it, [76:15] and I assume that he's not gonna race on unsafely. [76:19] - Yeah. [76:20] But collaboration here, I think, is really beneficial [76:23] for everybody on that front. [76:26] - Not really a thing he's most known for. [76:28] - Well, he is known for caring about humanity [76:31] and humanity benefits from collaboration [76:33] and so there's always a tension, and incentives, [76:36] and motivations. [76:37] And in the end, [76:39] I do hope humanity prevails. [76:42] - I was thinking, [76:43] someone just reminded me the other day [76:44] about how the day that he got surpassed Jeff Bezos [76:48] for like richest person in the world, [76:51] he tweeted a silver medal at Jeff Bezos. [76:55] I hope we have less stuff like that [76:57] as people start to work on [76:58] towards AI. - I agree. [76:59] - I think Elon is a friend [77:01] and he is a beautiful human being [77:03] and one of the most important humans ever. [77:05] That stuff is not good. [77:07] - The amazing stuff about Elon is amazing [77:09] and I super respect him. [77:11] I think we need him. [77:13] All of us should be rooting for him [77:14] and need him to step up as a leader through this next phase. [77:19] - Yeah, I hope you can have one without the other, [77:21] but sometimes humans are flawed and complicated [77:23] and all that kind of stuff. [77:24] - There's a lot of really great leaders throughout history. [77:27] - Yeah. [77:28] And we can each be the best version of ourselves [77:31] and strive to do so. [77:33] Let me ask you. [77:36] Google, with the help of search, [77:39] has been dominating the past 20 years. [77:44] I think it's fair to say in terms of the access, [77:47] the world's access to information, [77:49] how we interact and so on. [77:50] And one of the nerve-wracking things for Google, [77:53] but for the entirety of people [77:55] in this space is thinking about [77:57] how are people going to access information. [77:59] Like you said, people show up to GPT [78:02] as a starting point. [78:04] So is OpenAI going to really take on this thing [78:09] that Google started 20 years ago, [78:10] which is how do we get- [78:12] - I find that boring. [78:13] I mean, if the question is [78:16] if we can build a better search engine than Google [78:19] or whatever, [78:20] then sure, we should go... [78:24] Like people should use a better product. [78:26] But I think that would so understate what this can be. [78:33] Google shows you like 10 blue links, [78:36] like 13 ads and then 10 blue links [78:39] and that's like one way to find information. [78:42] But the thing that's exciting to me is not [78:44] that we can go build a better copy of Google Search, [78:49] but that maybe there's just some much better way [78:52] to help people find and act on and synthesize information. [78:57] Actually, I think ChatGPT is that for some use cases [78:59] and hopefully will make it be like that [79:02] for a lot more use cases. [79:04] But I don't think it's that interesting [79:06] to say like how do we go do a better job [79:08] of giving you like 10 ranked webpages [79:10] to look at than what Google does. [79:12] Maybe it's really interesting to go, [79:14] say, how do we help you get the answer [79:16] or the information you need? [79:17] How do we help create that in some cases, [79:20] synthesize that in others or point you to it [79:21] and yet others? [79:25] But a lot of people have tried [79:28] to just make a better search engine than Google, [79:30] and it's a hard technical problem, [79:33] it's a hard branding problem, [79:34] it's a hard ecosystem problem. [79:36] I don't think the world needs another copy of Google. [79:39] - And integrating a chat client like a ChatGPT [79:42] with a search engine. [79:44] - That's cooler. [79:46] - It's cool, [79:46] but it's tricky. [79:48] If you just do it simply, it's awkward [79:51] because like if you just shove it in there, [79:53] it can be awkward. [79:54] - As you might guess, [79:56] we are interested in how to do that. [79:57] Well, that would be an example [79:59] of a cool thing that's not just like- [80:00] - Well, like a heterogeneous, [80:03] like integrating- [80:03] - The intersection of LLMs plus search, [80:07] I don't think anyone has cracked the code on yet. [80:10] I would love to go do that. [80:12] I think that would be cool. [80:13] - Yeah. [80:14] What about the ads side? [80:15] Have you ever considered monetization? [80:16] - I kind of hate ads just as like an aesthetic choice. [80:20] I think ads needed to happen on the internet [80:24] for a bunch of reasons to get it going, [80:26] but it's a more mature industry. [80:30] The world is richer now. [80:32] I like that people pay for ChatGPT [80:35] and know that the answers they're getting are not influenced [80:39] by advertisers. [80:40] I'm sure there's an ad unit that makes sense for LLMs. [80:45] And I'm sure there's a way to participate [80:48] in the transaction stream in an unbiased way [80:51] that is okay to do. [80:54] But it's also easy to think about like the dystopic visions [80:57] of the future where you ask ChatGPT something [81:01] and it says, "Oh, you should think about buying this product [81:03] or you should think about this going here [81:06] for vacation or whatever." [81:08] And I don't know, [81:10] like we have a very simple business model and I like it. [81:16] And I know that I'm not the product. [81:20] I know I'm paying [81:21] and that's how the business model works. [81:24] And when I go use Twitter, or Facebook, or Google, [81:29] or any other great product, [81:31] but ad-supported great product, [81:35] I don't love that [81:36] and I think it gets worse, [81:37] not better in a world with AI. [81:39] - Yeah. [81:40] I mean, I can imagine AI will be better at showing [81:43] the best kind of version of ads not in a dystopic future, [81:46] but where the ads are for things you actually need. [81:52] But then does that system always result [81:56] in the ads driving the kind of stuff [81:57] that's shown all that? [82:01] I think it was a really bold move [82:02] of Wikipedia not to do advertisements, [82:05] but then it makes it very challenging as a business model. [82:08] So you're saying the current thing [82:10] with OpenAI is sustainable from a business perspective? [82:15] - Well, we have to figure out how to grow, [82:17] but it looks like we're gonna figure that out. [82:20] If the question is, [82:21] do I think we can have a great business [82:23] that pays for our compute needs without ads? [82:26] That I think the answer is yes. [82:28] - Hmm. [82:32] Well, that's promising. [82:33] I also just don't want to completely throw out ads as a- [82:37] - I'm not saying that. [82:39] I guess I'm saying I have a bias against them. [82:42] - Yeah. [82:43] I have a also bias and just a skepticism in general [82:48] and in terms of interface [82:50] because I personally just have like a spiritual dislike [82:53] of crappy interfaces, [82:55] which is why AdSense [82:57] when it first came out was a big leap forward [82:59] versus like animated banners or whatever. [83:02] But it feels like there should be many more leaps forward [83:05] in advertisement that doesn't interfere [83:08] with the consumption of the content [83:09] and doesn't interfere in the big fundamental way, [83:12] which is like what you were saying, [83:14] like it will manipulate the truth [83:17] to suit the advertisers. [83:19] Let me ask you about safety, [83:24] but also bias and safety in the short term safety [83:28] in the long term. [83:29] The Gemini 1.5 came out recently. [83:31] There's a lot of drama around it, [83:33] speaking of theatrical things. [83:35] And it generated Black Nazis and Black founding fathers. [83:40] I think fair to say it was a bit on the ultra woke side. [83:47] So that's a concern for people that [83:50] if there is a human layer within companies [83:51] that modifies the safety [83:55] or the the harm cost by a model [83:57] that they introduce a lot of bias [83:59] that fits sort of an ideological lean within a company. [84:04] How do you deal with that? [84:06] - I mean, we work super hard not to do things like that. [84:09] We've made our own mistakes, [84:11] will make others. [84:12] I assume Google will learn from this one, [84:14] still make others. [84:17] These are not easy problems. [84:20] One thing that we've been thinking [84:21] about more and more is I think this was a great idea. [84:24] Somebody here had like... [84:25] It'd be nice to write out [84:26] what the desired behavior of a model is, [84:28] make that public take input on it. [84:30] Say, here's how this model's supposed to behave [84:32] and explain the edge cases too. [84:34] And then when a model is not behaving [84:37] in a way that you want, [84:39] it's at least clear about whether [84:40] that's a bug the company should fix [84:41] or behaving as intended [84:43] and you should debate the policy. [84:44] And right now, it can sometimes be caught in between. [84:48] Black Nazis, [84:49] obviously ridiculous, [84:50] but there are a lot of other kind of subtle things [84:52] that you can make a judgment call on either way. [84:54] - Yeah. [84:55] But sometimes if you write it out and make it public, [84:58] you can use kind of language that's... [85:01] The Google's AI principle is a very high level. [85:04] - That's not what I'm talking about. [85:05] That doesn't work. [85:06] Like I'd have to say when you ask it to do thing X, [85:09] it's supposed to respond in wait Y. [85:11] - So, literally, who's better, [85:13] Trump or Biden? [85:15] What's the expected response from a model? [85:17] Like something like very concrete. [85:19] - I'm open to a lot of ways a model could behave them, [85:21] but I think you should have to say here's the principle [85:23] and here's what it should say in that case. [85:24] - That would be really nice. [85:26] That would be really nice [85:27] and then everyone kind of agrees [85:29] 'cause there's this anecdotal data [85:31] that people pull out all the time [85:33] and if there's some clarity [85:35] about other representative anecdotal examples [85:38] you can define. [85:39] - And then when it's a bug, [85:40] it's a bug [85:40] and the company can fix that. [85:41] - Right. [85:42] Then it'd be much easier to deal with a Black Nazi type [85:44] of image generation if there's great examples. [85:49] So San Francisco is a bit of an ideological bubble tech [85:54] in general as well. [85:56] Do you feel the pressure of that within a company [85:59] that there's like a lean towards the left politically [86:04] that affects the product, [86:05] that affects the teams? [86:06] - I feel very lucky that we don't have the challenges [86:09] at OpenAI that I have heard of at a lot of other companies. [86:12] I think part of it is like every company's got [86:17] some ideological thing. [86:19] We have one about AGI and belief in that [86:22] and it pushes out some others. [86:24] We are much less caught up in the culture war [86:27] than I've heard about it a lot of other companies. [86:30] San Francisco mess in all sorts of ways, of course. [86:33] - So that doesn't infiltrate OpenAI. [86:36] - I'm sure it does in all sorts of subtle ways, [86:37] but not in the obvious. [86:43] We've had our flareups for sure like any company, [86:46] but I don't think we have anything like what I hear [86:48] about happening at other companies here on this topic. [86:49] - So what in general is the process [86:51] for the bigger question of safety? [86:53] How do you provide that layer [86:55] that protects the model from doing crazy dangerous things? [87:02] - I think there will come a point [87:03] where that's mostly what we think about the whole company. [87:06] It's not like you have one safety team. [87:09] It's like when we ship GPT-4, [87:10] that took the whole company thing [87:11] about all these different aspects [87:12] and how they fit together. [87:13] And I think it's gonna take that. [87:17] More and more of the company thinks [87:18] about those issues all the time. [87:21] - That's literally what humans will be thinking about [87:24] the more powerful AI becomes. [87:27] So most of the employees that OpenAI will be thinking safety [87:30] or at least to some degree. [87:31] - Broadly defined, yes. [87:33] - Yeah. [87:34] I wonder what are the full broad definition of that. [87:37] What are the different harms that could be caused? [87:39] Is this like on a technical level [87:42] or is this almost like security threats? [87:44] - All those things. [87:45] Yeah, I was gonna say it'll be people, [87:48] state actors trying to steal the model. [87:50] It'll be all of the technical alignment work. [87:53] It'll be societal impacts, economic impacts. [88:00] It's not just like we have one team thinking about how [88:03] to align the model. [88:04] It's really gonna be like getting [88:07] to the good outcome is gonna take the whole effort. [88:10] - How hard do you think people, [88:13] state actors perhaps are trying to hack? [88:16] First of all, [88:17] infiltrate OpenAI, [88:18] but second of all, [88:19] infiltrate unseen, [88:20] - They're trying. [88:24] - What kind of accent do they have? [88:27] - I don't think I should go [88:28] into any further details on this point. [88:29] - Okay. [88:32] But I presume it'll be more and more and more [88:34] as time goes on. [88:35] - That feels reasonable. [88:37] - Boy, what a dangerous space. [88:40] What aspect of the leap... [88:42] And sorry to linger on this [88:43] even though you can't quite say details yet, [88:46] but what aspects of the leap [88:48] from GPT-4 to GPT-5 are you excited about? [88:53] - I'm excited about being smarter [88:54] and I know that sounds like a glib answer, [88:56] but I think the really special thing happening is [88:59] that it's not like it gets better in one area [89:01] and worse at others. [89:02] It's getting like better across the board. [89:06] That's I think super cool. [89:07] - Yeah, there's this magical moment. [89:09] I mean, you meet certain people, [89:10] you hang out with people [89:12] and you talk to them. [89:13] You can't quite put a finger on it, [89:15] but they kind of get you. [89:18] It's not intelligence, really. [89:20] It's like it's something else. [89:23] And that's probably [89:24] how I would characterize the progress at GPT. [89:26] It's not like, yeah, you can point out, look, [89:29] you didn't get this or that, [89:30] but to which degree is there's this intellectual connection? [89:35] Like you feel like there's an understanding [89:37] in your crappy formulated prompts that you're doing [89:42] that it grasps the deeper question [89:45] behind the question that you're... [89:47] Yeah, I'm also excited by that. [89:49] I mean, all of us love being understood, [89:52] heard and understood. [89:53] - That's for sure. [89:54] - That's a weird feeling. [89:54] Even like with a programming, [89:56] like when you're programming [89:57] and you say something [89:58] or just the completion that GPT might do, [90:02] it's just such a good feeling when it got you, [90:05] like what you're thinking about. [90:07] And I look forward to getting you even better. [90:10] On the programming front, [90:12] looking out into the future, [90:14] how much programming do you think humans will be doing [90:16] 5, 10 years from now? [90:19] - I mean, a lot, [90:20] but I think it'll be in a very different shape. [90:23] Like maybe some people program entirely in natural language. [90:26] - Entirely natural language. [90:29] - I mean, no one programs like writing by code... [90:33] Some people. [90:34] No one programs the pun cards anymore. [90:35] I'm sure you can invite someone who does, [90:37] but you know what I mean. [90:38] - Yeah. [90:39] You're gonna get a lot of angry comments. [90:41] No, no. [90:42] Yeah, there's very few. [90:43] I've been looking for people program Fortran. [90:45] It's hard to find even Fortran. [90:47] I hear you. [90:48] But that changes the nature of the skillset [90:51] or the predisposition for the kind [90:53] of people we call programmers then. [90:55] - Changes the skillset. [90:56] How much it changes the predisposition, [90:58] I'm not sure [90:59] - Oh, same kind of puzzle solving, [91:00] all that kind of stuff. - Maybe. [91:02] - Yeah, the programming is hard. [91:05] Like that last 1% to close the gap, [91:08] how hard is that? [91:09] - Yeah. [91:10] I think with most other cases, [91:11] the best practitioners of the craft will use multiple tools [91:14] and they'll do some work in natural language [91:16] and when they need to go write, see for something, [91:19] they'll do that. [91:20] - Will we see a humanoid robots [91:24] or humanoid robot brains from OpenAI at some point? [91:27] - At some point. [91:29] - How important is embodied AI to you? [91:32] - I think it's like sort of depressing if we have AGI [91:35] and the only way to get things done [91:37] in the physical world is like to make a human go do it. [91:41] So I really hope that as part of this transition, [91:45] as this phase change, [91:47] we also get motor robots [91:50] or some sort of physical world robots. [91:51] - I mean, OpenAI has some history [91:53] and quite a bit of history working in robotics, [91:56] but it hasn't quite done in terms of emphasis. [91:59] - Well, we're like a small company. [92:00] We have to really focus [92:01] and also robots were hard for the wrong reason at the time. [92:04] But like we will return to robots [92:09] in some way at some point. [92:11] - That sounds both inspiring and menacing. [92:14] - Why? [92:15] - Because you immediately, [92:16] we will return to robots. [92:17] It's kind of like in like- [92:19] - We'll return to work on developing robots. [92:21] We will not turn ourselves into robots, of course. [92:24] - Yeah. [92:25] When do you think we you [92:28] and we as humanity will build AGI? [92:31] - I used to love to speculate on that question. [92:33] I have realized since [92:34] that I think it's like very poorly formed [92:37] and that people use extremely definition, [92:40] different definitions for what AGI is. [92:46] And so I think it makes more sense [92:48] to talk about when we'll build systems [92:49] that can do capability X or Y or Z [92:52] rather than when we kind [92:54] of like fuzzily cross this one mile marker. [92:57] It's not like AGI is also not an ending. [92:59] It's much more of.... [93:00] It's closer to a beginning [93:02] but it's much more of a mile marker [93:03] than either of those things. [93:07] But what I would say in the interest of not trying [93:10] to dodge a question is I expect that by the end [93:13] of this decade and possibly somewhat sooner than that, [93:21] we will have quite capable systems that we look at [93:24] and say, wow, that's really remarkable. [93:27] If we could look at it now, [93:29] maybe we've adjusted by the time we get there. [93:30] - Yeah. [93:31] But if you look at ChatGPT even 3, 5, [93:37] and you show that to Alan Turing [93:39] or not even Alan Turing, [93:41] people in the nineties, [93:42] they would be like this is definitely AGI. [93:44] Well, not definitely, [93:46] but there's a lot of experts [93:48] that would say this is AGI. [93:49] - Yeah, but I don't think 3, 5 changed the world. [93:53] It maybe changed the world's expectations for the future [93:56] and that's actually really important. [93:58] And it did kind of like get more people [94:00] to take this seriously and put us on this new trajectory. [94:03] And that's really important too. [94:05] So again, I don't wanna undersell it. [94:07] I think I could retire after that accomplishment [94:10] and be pretty happy with my career. [94:11] But as an artifact, [94:14] I don't think we're gonna look back at that [94:15] and say that was a threshold [94:18] that really changed the world itself. [94:20] - So to you, [94:21] you're looking for some really major transition [94:23] in how the world? [94:24] - For me, that's part of what AGI implies. [94:29] - Like singularity level transition? [94:31] - No. [94:31] Definitely not. [94:32] - But just a major, [94:33] like the internet being like a... [94:35] Like Google Search did, I guess. [94:37] What was the transition point that- [94:39] - Does the global economy feel any different to you now [94:42] or materially different to you now than it did [94:44] before we launched GPT-4? [94:46] I think you would say no. [94:47] - No, no. [94:49] It might be just a really nice tool [94:51] for a lot of people to use, [94:52] will help people with a lot of stuff, [94:53] but doesn't feel different. [94:53] And you're saying that- [94:55] - I mean, again, people define AGI all sorts [94:57] of different ways. [94:58] So maybe you have a different definition than I do. [94:59] But for me, [95:01] I think that should be part of it. [95:02] - There could be major theatrical moments also. [95:08] What to you would be an impressive thing AGI would do? [95:13] Like you are alone in a room with a system. [95:16] - This is personally important to me. [95:18] I don't know if this is the right definition. [95:19] I think when a system can significantly increase the rate [95:24] of scientific discovery in the world, [95:27] that's like a huge deal. [95:28] I believe that most real economic growth comes [95:31] from scientific and technological progress. [95:35] - I agree with you, [95:36] hence why I don't like the skepticism about science [95:41] in the recent years. [95:42] - Totally. [95:43] - But actual rate, [95:45] like measurable rate of scientific discovery. [95:48] But even just seeing a system have really novel intuitions, [95:56] like scientific intuitions, [95:58] even that will be just incredible. [96:01] - Yeah. [96:01] - You're quite possibly would be the person [96:04] to build the AGI to be able to interact with it [96:05] before anyone else does. [96:07] What kind of stuff would you talk about? [96:09] - I mean, definitely, the researchers here will do [96:11] that before I do so. [96:12] - Sure. [96:14] - But I've actually thought a lot about this question. [96:16] If I were someone was like... [96:19] As we talked about earlier, [96:20] I think this is a bad framework. [96:21] But if someone were like, "Okay, Sam, we're finished. [96:25] Here's a laptop. [96:26] Yeah, this is the AGI," [96:30] you can go talk to it. [96:34] I find it surprisingly difficult to say what I would ask, [96:37] that I would expect that first AGI to be able to answer. [96:42] Like that first one is not gonna be the one [96:43] which is go like I don't think, [96:47] like go explain to me the grand unified theory of physics, [96:51] the theory of everything for physics. [96:52] I'd love to ask that question. [96:53] I'd love to know the answer to that question. [96:55] - You can ask yes or no questions [96:56] about there's such a theory exist, [96:59] can it exist? [97:00] - Well, then those are the first questions I would ask. [97:02] - Yes or no, just very... [97:04] And then based on that, [97:05] are there other alien civilizations out there? [97:07] Yes or no? [97:08] What's your intuition? [97:09] And then you just ask that. [97:10] - Yeah. [97:11] I mean, well, so I don't expect [97:12] that this first AGI could answer any [97:13] of those questions even as yes or nos. [97:16] But if it could, [97:17] those would be very high on my list. [97:19] - Hmm. [97:20] Maybe it can start assigning probabilities. [97:22] - Maybe we need to go invent more technology [97:25] and measure more things first. [97:26] - But if it's any AGI... [97:28] Oh I see. [97:29] It just doesn't have enough data. [97:31] - I mean, maybe it says like you want [97:33] to know the answer to this question about physics, [97:35] I need you to like build this machine [97:37] and make these five measurements and tell me that. [97:39] - Yeah. [97:40] What the hell do you want from me? [97:42] I need the machine first [97:43] and I'll help you deal with the data from that machine. [97:46] Maybe you'll help me build a machine maybe. [97:47] - Maybe. [97:49] - And on the mathematical side, [97:50] maybe prove some things. [97:52] Are you interested in that side of things too? [97:54] The formalized exploration of ideas? [97:59] Whoever builds AGI first gets a lot of power. [98:05] Do you trust yourself with that much power? [98:14] - Look, I was gonna... [98:18] I'll just be very honest with this answer. [98:19] I was gonna say, [98:20] and I still believe this, [98:21] that it is important that I, [98:24] nor any other one person, [98:26] have total control over OpenAI or over AGI. [98:32] And I think you want a robust governance system. [98:39] I can point out a whole bunch of things [98:41] about all of our board drama from last year [98:46] about how I didn't fight it initially [98:48] and was just like, yeah, that's the will of the board [98:50] even though I think it's a really bad decision. [98:55] And then later, [98:56] I clearly did fight it [98:57] and I can explain the nuance [98:58] and why I think it was okay for me to fight it later. [99:01] But as many people have observed, [99:08] although the board had the legal ability [99:11] to fire me, [99:13] in practice, it didn't quite work. [99:18] And that is its own kind of governance failure. [99:24] Now, again, I feel like I can completely defend [99:28] the specifics here [99:31] and I think most people would agree with that. [99:33] But it does make it harder for me [99:40] to like look you in the eye [99:41] and say, hey, the board can just fire me. [99:46] I continue to not want super voting control over OpenAI. [99:51] I never had it, [99:52] never have wanted it. [99:55] Even after all this craziness, [99:56] I still don't want it. [100:00] I continue to think [100:02] that no company should be making these decisions [100:05] and that we really need governments [100:08] to put rules of the road in place. [100:12] And I realize that that means people like Marc Andreessen [100:15] or whatever will claim I'm going for regulatory capture [100:18] and I'm just willing to be misunderstood there. [100:20] It's not true. [100:21] And I think in the fullness of time, [100:23] it'll get proven out why this is important. [100:27] But I think I have made plenty of bad decisions [100:35] for OpenAI along the way and a lot of good ones [100:38] and I'm proud of the track record overall, [100:42] but I don't think any one person should. [100:44] And I don't think any one person will. [100:45] I think it's just like too big of a thing now [100:47] and it's happening throughout society [100:48] in a good and healthy way. [100:50] But I don't think any one person should be in control [100:52] of an AGI or this whole movement towards AGI. [100:57] And I don't think that's what's happening. [100:59] - Thank you for saying that. [101:00] That was really powerful [101:02] and that was really insightful that this idea [101:04] that the board can fire you is legally true. [101:11] And human beings can manipulate the masses [101:14] into overriding the board and so on. [101:19] But I think there's also a much more positive version [101:22] of that where the people still have power. [101:25] So the board can't be too powerful either. [101:27] There's a balance of power in all of this. [101:29] - Balance of power is a good thing for sure. [101:33] - Are you afraid of losing control of the AGI itself? [101:37] That's a lot of people [101:38] who worried about existential risk [101:40] not because of state actors, [101:42] not because of security concerns, [101:43] but because of the AI itself. [101:44] - That is not my top worry as I currently see things. [101:47] There have been times I worried about that more. [101:49] There may be times again in the future [101:50] where that's my top worry. [101:52] It's not my top worry right now. [101:53] - What's your intuition about it not being your worry? [101:55] Because there's a lot of other stuff [101:56] to worry about essentially. [102:00] You think you could be surprised? [102:01] We could be surprised. - For sure, of course. [102:05] Saying it's not my top worry doesn't mean [102:07] I don't think we need to like... [102:08] I think we need to work on it super hard. [102:10] We have great people here who do work on that. [102:13] I think there's a lot [102:14] of other things we also have to get right. [102:15] - To you, [102:16] it's not super easy to escape the box at this time, [102:20] like connect to the internet. [102:21] - We talked about theatrical risk earlier. [102:24] That's a theatrical risk. [102:25] That is a thing that can really like take over [102:29] how people think about this problem. [102:31] And there's a big group of like very smart, [102:34] I think very well-meaning AI safety researchers [102:38] that got super hung up on this one problem. [102:41] I'd argue without much progress, [102:42] but super hung up on this one problem. [102:45] I'm actually happy that they do that [102:47] because I think we do need to think about this more. [102:50] But I think it pushed aside, [102:53] it pushed out of the space of discourse a lot [102:55] of the other very significant AI-related risks. [103:01] - Let me ask you about you tweeting with no capitalization. [103:04] Does the shift keep broken on your keyboard? [103:07] - Why does anyone care about that? [103:09] - I deeply care. [103:10] - But why? [103:12] I mean, other people ask me about that too. [103:15] Any intuition? [103:17] - I think it's the same reason [103:18] there's like this poet E. E. Cummings [103:20] that mostly doesn't use capitalization [103:23] to say like fuck you to the system kind of thing. [103:26] And I think people are very paranoid [103:28] 'cause they want you to follow the rules. [103:29] - You think that's what it's about? [103:30] - I think it's- [103:31] - This guy doesn't follow the rules. [103:33] He doesn't capitalize his tweets. [103:35] This seems really dangerous. [103:37] - He seems like an anarchist. [103:39] - [Sam] It doesn't. [103:40] - Are you just being poetic, hipster? [103:42] What's the- [103:44] - I grew up as- [103:45] - Follow the rules, Sam. [103:45] - I grew up as a very online kid. [103:47] I'd spent a huge amount of time like chatting [103:50] with people back in the days [103:51] where you did it on a computer [103:53] and you could like log off instant messenger at some point. [103:56] And I never capitalized there [103:59] as I think most like internet kids didn't, [104:01] or maybe they still don't. [104:02] I don't know. [104:08] I actually, this is like... [104:10] Now, I'm like really trying to reach for something. [104:12] But I think capitalization has gone down over time. [104:15] If you read like old English writing, [104:17] they capitalized a lot of random words [104:18] in the middle of sentences, [104:19] nouns and stuff that we just don't do anymore. [104:22] I personally think it's sort of like a dumb construct [104:26] that we capitalize the letter at the beginning of a sentence [104:28] and of certain names and whatever. [104:31] That's fine. [104:35] And I used to, I think, even like capitalize my tweets [104:38] because I was trying to sound professional or something. [104:41] I haven't capitalized my like private DMs [104:43] or whatever in a long time. [104:44] And then slowly, [104:48] stuff like shorter form, [104:54] less formal stuff has slowly drifted [104:57] to like closer and closer to how I would text my friends. [105:01] If I write, [105:02] if I pull up a Word document [105:04] and I'm writing a strategy memo [105:05] for the company or something, [105:06] I always capitalize that. [105:08] If I'm writing a long kind of more like formal message, [105:11] I always use capitalization there too. [105:13] So I still remember how to do it. [105:15] But even that may fade out. [105:16] I don't know. [105:19] But I never spend time thinking about this [105:21] so I don't have like a ready made. [105:23] - Well, it's interesting. [105:24] Well, it's good to, first of all, [105:25] know there's the shift key is not broken. [105:27] - It works. [105:28] - I was mostly concerned about your wellbeing on that front. [105:30] - I wonder if people still capitalize their Google Searches. [105:32] Like if you're writing something just to yourself [105:34] or their ChatGPT queries, [105:35] if you're writing something just to yourself, [105:38] do some people still bother to capitalize? [105:40] - Probably not. [105:42] Yeah, there's a percentage, [105:43] but it's a small one. [105:44] - The thing that would make me do it is [105:45] if people were like... [105:48] It's a sign of like... [105:49] Because I'm sure I could force myself [105:52] to use capital letters, obviously. [105:54] If it felt like a sign of respect to people or something, [105:56] then I could go do it. [105:58] But I don't know, [105:59] I don't think about this. [106:00] I don't think there's a disrespect, [106:02] but I think it's just the conventions of civility [106:06] that have a momentum [106:08] and then you realize it's not actually important [106:11] for civility if it's not a sign of respect or disrespect. [106:13] But I think there's a movement of people that just want you [106:16] to have a philosophy around it [106:18] so they can let go of this whole capitalization thing. [106:19] - I don't think anybody else thinks about this is my... [106:21] I mean, maybe some people... [106:22] - Think about this every day for many hours a day. [106:26] I'm really grateful we clarified it. [106:28] - Can't be the only person that doesn't capitalize tweets. [106:30] - You're the only CEO of a company [106:32] that doesn't capitalize tweets. [106:34] - I don't even think that's true, [106:35] but maybe, maybe. [106:35] - All right, we'll investigate for this [106:38] and return to this topic later. [106:41] Given Sora's ability to generate simulated worlds, [106:44] let me ask you a pothead question. [106:48] Does this increase your belief [106:51] if you ever had one that we live in a simulation, [106:54] maybe a simulated world generated by an AI system? [107:04] - Yes, somewhat. [107:08] I don't think that's like the strongest piece of evidence. [107:13] I think the fact that we can generate worlds should increase [107:20] everyone's probability somewhat or at least open to it, [107:24] openness to it somewhat. [107:25] But you know, I was like certain we would be able [107:27] to do something like Sora at some point. [107:28] It happened faster than I thought. [107:31] I guess that was not a big update. [107:34] - Yeah. [107:35] And presumably, it'll get better and better and better. [107:38] The fact that you can generate worlds, [107:39] they're novel. [107:41] They're based in some aspect of training data, [107:44] but when you look at them, [107:46] they're novel. [107:50] That makes you think like how easy it's to do this thing, [107:52] how easy it's to create universes, [107:54] entire like video game worlds that seem ultrarealistic [107:58] and photorealistic. [107:59] And then how easy is it to get lost [108:02] in that world first with a VR headset [108:04] and then on the physics-based level? [108:10] - Someone said to me recently, [108:11] I thought it was a super profound insight [108:13] that there are these like very simple sounding, [108:23] but very psychedelic insights that exist sometimes. [108:28] So the square root function. [108:30] Square root of four, no problem. [108:33] Square root of two, [108:35] okay, now I have to think about this new kind of number. [108:43] But once I come up with this easy idea [108:45] of a square root function [108:46] that you can kind of explain to a child [108:49] and exists by even like looking at some simple geometry, [108:55] then you can ask the question [108:56] of what is the square root of negative one? [109:00] This is why it's like a psychedelic thing [109:03] that tips you into some whole other kind of reality. [109:07] And you can come up with lots of other examples. [109:11] But I think this idea [109:12] that the lowly square root operator can offer [109:17] such a profound insight [109:20] and a new realm of knowledge, [109:24] applies in a lot of ways. [109:26] And I think there are a lot of those operators [109:29] for why people may think that any version that they like [109:34] of the simulation hypothesis is maybe more likely [109:36] than they thought before. [109:38] But for me, [109:41] the fact that Sora worked is not in the top five. [109:46] - I do think broadly speaking, [109:48] AI will serve as those kinds [109:50] of gateways at its best simple psychedelic like gateways [109:55] to another wave sea reality. [109:57] - That seems for certain. [109:59] - That's pretty exciting. [110:01] I haven't done Ayahuasca before, [110:03] but I will soon. [110:04] I'm going to the aforementioned Amazon jungle [110:06] in a few weeks. [110:07] - Excited. [110:08] - Yeah, I'm excited for it. [110:09] Not the Ayahuasca part. [110:10] That's great, whatever. [110:11] But I'm gonna spend several weeks in the jungle, [110:14] deep in the jungle and it's exciting, [110:17] but it's terrifying- [110:17] - I'm excited for you. [110:18] - 'Cause there's a lot of things [110:19] that can eat you there and kill you and poison you, [110:23] but it's also nature and it's the machine of nature. [110:25] And you can't help but appreciate the machinery [110:27] of nature in the Amazon jungle [110:29] 'cause it's just like this system that just exists [110:33] and renews itself like every second, [110:36] every minute, every hour. [110:37] It's the machine. [110:38] It makes you appreciate like this thing we have here, [110:42] this human thing came from somewhere. [110:44] This evolutionary machine has created that [110:47] and it's most clearly on display in the jungle. [110:52] So hopefully, I'll make it out alive. [110:54] If not, this will be the last conversation we had, [110:56] so I really deeply appreciate it. [110:58] Do you think, as I mentioned before, [111:00] there's other aliens, civilizations out there, [111:02] intelligent ones [111:04] when you look up at the skies? [111:17] - I deeply want to believe that the answer is yes. [111:22] I do find that kind of where... [111:24] I find the firm paradox very, very puzzling. [111:28] - I find it scary [111:30] that intelligence is not good at handling- [111:33] - Yeah. [111:34] Very scary, powerful. - Technologies. [111:36] But at the same time, [111:37] I think I'm pretty confident [111:40] that there's just a very large number [111:42] of intelligent alien civilizations out there. [111:44] It might just be really difficult [111:45] to travel with this space. [111:47] - Very possible. [111:50] - And it also makes me think [111:51] about the nature of intelligence. [111:52] Maybe we're really blind [111:54] to what intelligence looks like [111:56] and maybe AI will help us see that. [111:59] It's not as simple as IQ tests [112:01] and simple puzzle solving. [112:02] There's something bigger. [112:06] Well, what gives you hope about the future of humanity? [112:09] This thing we've got going on, [112:10] this human civilization. [112:13] - I think the past is like a lot. [112:15] I mean, we just look at what humanity has done [112:18] in a not very long period of time. [112:22] Huge problems, deep flaws, [112:24] lots to be super ashamed of, [112:26] but on the whole, [112:27] very inspiring, [112:28] gives me a lot of hope. [112:30] - Just the trajectory of it all [112:31] that we're together pushing towards a better future. [112:37] - It is... [112:40] One thing that I wonder about is, [112:43] is AGI gonna be more like some single brain, [112:46] or is it more like the sort of scaffolding in society [112:49] between all of us? [112:51] You have not had a great deal of genetic drift [112:55] from your great-great-great grandparents, [112:57] and yet what you're capable of is dramatically different. [113:01] What you know is dramatically different. [113:05] That's not because of biological change. [113:07] I mean, you got a little bit healthier probably. [113:09] You have modern medicine, [113:10] you eat better, whatever. [113:13] But what you have is this scaffolding [113:19] that we all contributed to built on top of. [113:23] No one person is gonna go build the iPhone. [113:25] No one person is gonna go discover all of science. [113:27] And yet you get to use it. [113:29] And that gives you incredible ability. [113:31] And so in some sense, [113:32] that like we all created that [113:35] and that fills me with hope for the future. [113:38] That was a very collective thing. [113:40] - Yeah. [113:40] We really are standing on the shoulders of giants. [113:43] You mentioned when we were talking about theatrical, [113:47] dramatic AI risks [113:51] that sometimes you might be afraid for your own life. [113:55] Do you think about your death? [113:57] Are you afraid of it? [113:58] - I mean, I like if I got shot tomorrow [114:00] and I knew it today, [114:01] I'd be like, "Oh, that's sad. [114:04] I wanna see what's gonna happen." [114:06] - [Lex] Yeah. [114:07] - What a curious time. [114:09] What an interesting time. [114:12] But I would mostly just feel like very grateful for my life. [114:15] - The moments that you did get... [114:18] Yeah, me too. [114:20] It's a pretty awesome life. [114:22] I get to enjoy awesome creations of humans [114:25] of which I believe ChatGPT is one of, [114:29] and everything that OpenAI is doing. [114:32] Sam, it's really an honor [114:34] and pleasure to talk to you again. [114:35] - Great to talk to you. [114:36] Thank you for having me. [114:38] - Thanks for listening [114:39] to this conversation with Sam Altman. [114:41] To support this podcast, [114:42] please check out our sponsors in the description. [114:45] And now let me leave you [114:46] with some words from Arthur C. Clarke [114:49] and maybe that our role on this planet is not [114:52] to worship God, [114:54] but to create Him. [114:57] Thank you for listening [114:58] and hope to see you next time.