OpenAI's Brockman Links Level 4 AGI to Physical Energy Scaling

OpenAI的Brockman将4级AGI与物理能源扩展挂钩 实现具备科学创新能力的AI的核心障碍，已从软件算法转向为海量数据中心供电所需的物理基础设施。 在本次讨论中，Greg Brockman回顾了OpenAI的发展历程：从最初坚持技术优先路线，到在起步阶段的Dota 2项目中发现扩展假说（scaling hypothesis）的威力。他详细阐述了从GPT-3早期的通用API阶段，走向GPT-4在医疗、教育以及“氛围编码”（vibe coding）等领域被消费者和企业大规模采用的转变过程。展望未来，Brockman概述了OpenAI的五级AGI框架，强调主要瓶颈已从软件转移到了物理基础设施和能源。最后，他分享了在北达科他州成长的个人轶事，并维护了自己对AGI乐观的时间表预期，重点强调OpenAI致力于每年实现一次重大的“阶跃式”突破。 OpenAI抛弃了传统的初创公司策略，在没有具体客户问题导向的情况下优先追求技术，这一策略在联合创始人Greg Brockman的创造者本能看来曾觉得“彻底注定失败”。 (0:00) Dota 2项目验证了扩展假说——团队观察到算力每增加2倍，性能就会提升2倍，证明了AI的进展是可预测且可扩展的。 (2:51) 在意识到无法控制AI涌现能力出现的确切时间线后，OpenAI从传统的以结果为导向的里程碑转向了以投入为驱动的管理模式。 (5:04) 严格版本的图灵测试已不再是该行业的主要目标；相反，重点已转向衡量实际效用与经济效用。 (7:52) AI产品交互正从匿名的无状态会话转向深度个性化，用户希望系统能够持久保留其历史记录和个人偏好。 (8:57)

OpenAI co-founder Greg Brockman argues that AI progress is driven by relentless compute scaling and input-focused experimentation rather than traditional product-first roadmaps, shifting the field's primary constraints from algorithms to physical infrastructure and energy. Scaling Compute and Input-Driven Management (0:00) OpenAI inverted traditional startup orthodoxy by pursuing underlying technology before identifying market use cases. Brockman notes that conviction in the scaling hypothesis solidified during their Dota 2 project, where doubling compute weekly on a desktop setup delivered predictable, corresponding performance gains. Managing that project taught OpenAI to abandon rigid, outcome-based completion dates for emergent capabilities in favor of strictly controlling inputs—such as committing to specific experiments and continuous model training. General Models and Product Pragmatism (10:26) To fund ongoing research without narrowing general-purpose aims into vertical silos, OpenAI launched a general API rather than hiring specialized domain experts like doctors. Early revenue from unexpected applications like AI Dungeon demonstrated commercial traction, and GPT-4 later enabled dependable business applications. Brockman explains that model adoption balances raw capability with user convenience: users willingly endure friction when capability provides high utility, but operating system integrations eventually catch up once workflows solidify. The Five-Level AGI Path and Energy Limits (17:59) OpenAI tracks progress across a five-level framework: Chatbots, Reasoners, Agents (the current stage), Innovators, and Organizations. Brockman expects Level 4 models to emerge within two to five years, potentially solving a Millennium Prize Problem when massive compute clusters are concentrated on a single scientific objective. However, Brockman identifies physical infrastructure—notably energy generation and permitting for data centers—as the primary bottleneck superseding algorithmic limits, with technologies like nuclear power and eventually fusion required to sustain growth. Overcoming the Data Wall Through New Paradigms (22:06) Concerns over hitting a data wall in 2023 subsided as the industry transitioned to reinforcement learning and synthetic data, aligning with historical technology S-curves where self-generated training data replaces exhausted human data. In practical deployment, Brockman highlights enterprise code refactoring—such as migrating legacy COBOL or PHP systems—as an ideal, volume-heavy fit for models like Codex, setting the stage for AI agents to transition from vibe coding assistants to structured team collaborators. While Brockman's earlier 2017 prediction that AGI was two to three years away did not strictly materialize, OpenAI has replaced rigid milestone targets with an operational benchmark of producing at least one qualitative step-function breakthrough each year.

OpenAI 联合创始人 Greg Brockman 指出，OpenAI 摒弃了以特定行业需求为切入点的传统创业模式，转而依托扩展假说打造通用大模型底座；当前大模型技术正从代理交互迈向自主科学发现，而推进这一进程的核心阻碍已从算法和数据转向物理电网与能源基础设施。 扩展假说验证与输入导向的研发管理 (0:00) OpenAI 创立初期采取了反常识的路径：先攻克核心技术，而在技术落地场景上全无预设。在 Dota 2 强化学习项目中，团队利用单台桌面电脑的 16 个 CPU 核心训练初始智能体，随后每周将算力翻倍，并观察到性能同步实现线性提升，这一确定性进展使团队坚信扩展假说（Scaling Hypothesis）并全力投入。算力规模与具备算力吸收能力的算法结合，成为推动技术演进的核心驱动力。 在工程管理方面，Brockman 早期试图设定在固定日期击败特定职业选手的“产出目标”，但因深度学习能力涌现时间难以预测而屡遭失败。团队随后转向“控制输入”的管理方式，即严格限定特定日期内必须完成的实验轮次与功能模块。在国际赛事期间，团队针对未受训的游戏道具失利后连夜重训，模型自主演化出了高难度的诱敌战术并击败世界第一选手。该项目确立了 OpenAI 不依赖僵化产出指标、转而专注于每年实现至少一次阶跃式（step function）突破的研发哲学。 通用模型路线与个性化产品演进 (10:26) 在 GPT-3 面世后，团队曾拟定包含特定医疗应用在内的百项商业化方案，但意识到深耕垂直赛道必须聘请大量领域专家，从而背离通用人工智能（AGI）中的通用性初衷。OpenAI 最终决定推出通用 API 供市场自行探索，其首个产生实际收入的客户为文字冒险游戏《AI Dungeon》。针对同时运营消费级与企业级业务的战略分歧，Brockman 将其比作迪士尼运营模式：全力打造高能力的基础模型这一核心资产，再以极低边际成本向不同应用场景变现。 在产品交互层面，ChatGPT 正在从无状态的单次匿名对话，转向具备长期持久记忆的个性化服务。针对移动操作系统缺乏原生一键调用的现状，Brockman 提出了能力（Capability）与便利性（Convenience）的双维框架：只要模型能力足够强并能大幅削减工作量，用户即便忍受截图上传等低效操作也会坚持使用；而当高能力锁定工作流后，市场竞争终将倒逼硬件厂商完善系统级集成。 垂直场景突破与企业级代码重构 (13:00) 在实际落地中，医疗与教育等领域呈现出意料之外的草根式爆发。Brockman 解释称，医疗场景落地门槛实际偏低，模型只要表现优于 WebMD 搜索引擎，即可帮助受困于看诊时间不足的患者及宠物主人排查复杂病症。在教育领域，大模型成功复现了可提供个性化指导的“布鲁姆两倍标准差”效应，促使 Khan Academy 创始人 Sal Khan 决定全面接入 GPT-4 进行产品转型。 在软件工程领域，“氛围编程（vibe coding）”虽快速兴起，但大模型目前主要自动化了编写业务逻辑等趣味环节，代码审查与生产部署仍高度依赖人类工程师。Brockman 认为，AI 编码对企业最核心的商业价值并非从零开发全新应用，而是针对庞大的遗留系统（如 COBOL 和 PHP 代码）进行大规模重构。这类任务依赖庞大的机械性代码检索与翻译能力，高度契合大模型的处理优势。 AGI演进分级与能源基建瓶颈 (17:59) OpenAI 将 AGI 的演化划分为五个阶段：聊天机器人、推理者、代理系统（当前所处阶段）、创新者（具备原创科学发现能力）与组织系统。针对外界对 AI 尚未取得原创科学成果的质疑，Brockman 预测未来两到五年内，AI 极有可能攻克千禧年大奖难题（Millennium Prize Problems）之一。实现第四级创新要求将一整个先进数据中心的算力（如 OpenAI 与 Oracle 在德克萨斯州阿比林合作的超大型数据中心）集中用于攻克单一前沿科学课题。 这一发展趋势彻底改变了技术瓶颈的性质。2023 年引发热议的“数据墙”难题，已随着研究范式转向合成数据与强化学习自生成数据而得以平稳过渡。当前推动 AI 进步的真正瓶颈已由软件层转向实体能源基础设施，即“将能源转化为智能”。由于核电站建设、水坝改造及土地审批无法遵循纯软件的指数级扩张周期，且可控核聚变预计至少还需五年以上，如何解决高密度的电力供给已成为决定技术与国家竞争力的关键制约因素。 Brockman 承认其在 2017 年作出的“AGI 仅需两至三年即可实现”的预测在时间形态上出现了偏差，但他强调实际落地的技术展现出了超越早期设想的实用价值与魔法感。然而，随着 AI 研发推进至需要调动全域数据中心与物理能源的第四级阶段，纯粹算法的迭代速度将不可避免地受制于实体电网建设等现实物理规律。

[0:00] Greg Brockman: totally doomed [5:04] A semi-pro player: totally broken [5:04] Greg Brockman: you can do true magic [8:57] Greg Brockman: walking into a shop off the street [10:26] Greg Brockman: the world's best demo machine [13:00] Greg Brockman: the bar is so low, you just need to exceed WebMD [14:25] Sal Khan: This is the thing. We need to become a GPT-4 app. [14:25] Greg Brockman: feels very 1993 [14:25] Greg Brockman: you'll crawl the top of the mountain [17:59] Greg Brockman: Just wait. [17:59] Greg Brockman: energy manufactured into intelligence [22:06] Greg Brockman: AI-shaped problems [27:28] A teacher: Every parent believes that their child is special. I can guarantee your son will be plenty challenged in my class. [29:47] A teacher: Your son has nothing to learn in this class [30:19] Greg Brockman: Did I say that? I don't see the recording. [30:19] Greg Brockman: AI is surprising [31:10] Greg Brockman: every year to have at least one result that just feels like a step function better than anything before

[0:00] This is totally backwards [0:01] from how you're supposed to do a startup, right? [0:02] You're supposed to have a problem [0:03] and we had no idea what the problem was. [0:05] Is there a world where the AI becomes the manager [0:07] and it, you know, gives you ideas [0:09] and gives you some tasks to do? [0:10] This was probably the hardest project that I've ever done, [0:12] because it felt totally doomed, right? [0:13] It's like, I know like every instinct, [0:16] every builder instinct of mine. [0:17] Did it actually feel doomed? [0:18] Oh, I felt totally doomed. [0:20] Greg in 2010 dropped out of MIT [0:22] to become our first engineer [0:24] and went on to become Stripe's CTO. [0:26] In 2015 after he left, he co-founded OpenAI. [0:29] OpenAI is really cooking at the moment [0:31] and Greg is one of the most productive people I know. [0:33] Cheers. [0:37] Wow, you really have some old school photos. [0:40] Some deep cut GDB trivia. [0:42] So yeah. We figure we've gotta put people at ease, [0:45] make people feel at home. [0:47] Very nice. Thank you. Okay. [0:48] Well, I just dive straight into all the questions I have, [0:50] which is a lot. All right. [0:51] If you were not working in AI, [0:53] how could one have known that [0:55] something was about to start working? [0:56] People were telling you that AI was the future [0:58] in the 1970s, in the 1980s, the 1990s. [1:01] Then very quickly, in the late 2010s [1:02] everything started happening. [1:04] Well, I was someone who was not in the field [1:07] and so, I remember very much what it was like. [1:09] 2013, 2014, it felt like every day on Hacker News [1:13] there'd be a new "deep learning for X" article. [1:16] And I remember being like, "What is deep learning? [1:18] I knew, like, one person in the field, [1:21] and I asked them to introduce me [1:24] to more people in the field, [1:25] and I just kept getting introduced [1:27] to a bunch of my smartest friends from college. [1:29] Now, if you actually look at the work that was being done... [1:32] 2012, basically, [1:34] image recognition, for the first time, [1:35] you could solve with the neural net [1:36] much better than anything else. [1:38] It just blew all these [1:39] traditional computer vision approaches out of the water. [1:41] It's like this like learned system [1:43] is able to outperform 40 years worth of, "Let's [1:45] write down all the rules and try to [1:47] handcraft the algorithm for the task." [1:49] And it's very easy to then be like, [1:52] "Okay. Well, this approach, [1:54] sure it works for computer vision, [1:55] but it's never gonna work for machine translation." [1:57] In 2014, [1:58] suddenly, you're getting great results [2:00] in machine translation. [2:01] I think that this pattern was applied [2:03] in subfield after subfield. [2:05] One thing I've been wondering about is [2:07] so many different things [2:08] are finally working at the same time. [2:12] We have LLMs, which are obviously amazing, [2:14] but then we also separately have [2:16] image models really working. [2:18] We also have text-to-speech and speech-to-text [2:20] working way better than they were before. [2:21] What's the common factor [2:24] behind everything starting to work at the same time? [2:26] Well, it's deep learning, right? [2:28] I think deep learning is the core- [2:29] We've have deep learning for a long time. [2:30] Why didn't deep learning work in the 1980s? [2:32] If you look at the number of orders [2:34] of magnitude of compute that [2:35] we've gone through from 1940 to today, [2:38] I mean, it's just astounding. [2:40] You think they're all explained by compute scale-ups [2:43] applied to the right algorithms? [2:44] Of course, the type of algorithm changes, [2:45] and some of those results aren't even deep learning. [2:47] But I think that fundamentally, it is about compute, [2:49] and you need an algorithm that is scalable [2:51] that can actually absorb that compute. [2:52] Was OpenAI the first company [2:53] to take the scaling hypothesis really seriously? [2:57] I think that claiming the first is always difficult, [3:00] but I think that is clear that we sort of [3:02] succeeded much more wildly sooner than anyone else. [3:06] I think that we had real conviction [3:07] behind what we needed to do. [3:09] Some people think that OpenAI [3:11] set out to prove the scale hypothesis, [3:13] whereas it was almost the other way around, [3:15] that the scale hypothesis is what we observed [3:17] as the thing that was working for us, [3:19] and we really saw it for the first time, [3:21] actually, during our Dota 2 project. [3:24] We started out with 16 cores to train a little agent- [3:28] on Jakub and Szymon who were leading the project [3:31] from an ML perspective, on their desktop. [3:34] Then, they scaled to 32 cores. [3:36] It felt like every week I'd come back to the office [3:38] and they'd scaled it by another 2x [3:40] and we had 2x performance- [3:41] just so clear, you just need to keep going. [3:43] Where does this thing peter out? [3:44] It just never did. [3:45] Founders get too much credit, [3:47] because you have an initial product [3:49] that's a pretty reasonable idea [3:50] Then you listen to the customers, [3:51] and you follow what's working. [3:52] So you were saying that was kind of OpenAI [3:55] with the scaling hypothesis, [3:56] where you started trying to make Dota AIs work [4:00] and you noticed that adding more compute worked really well, [4:03] and you said, "Where else [4:05] will just throwing more compute at it yield benefits. [4:07] I think that's to the first order correct. [4:10] I think one thing that distinguishes OpenAI [4:13] from the typical startup, [4:15] is we did everything in reverse, right? [4:18] It's like, you're supposed to have a problem to solve. [4:21] No one cares about the technology. [4:22] Form the entity upfront. [4:23] Exactly, yes. [4:24] And for us, we really chased the technology [4:27] without any idea of how it would be applied. [4:30] A lot of pursuing the technology [4:33] really is you have to let reality hit you [4:35] cold, hard in the face. [4:36] There's just no other way to achieve results. [4:39] You can't will it into existence. [4:41] You can't convince people [4:43] that this is the thing. [4:44] It's like, you have to actually, make the system work. [4:46] We have to just sort of figure out [4:48] what is the right frontier, what are the problems, [4:50] what are the things that are on the edge of working, [4:52] and to really double down on those. [4:54] What else do you take away from the Dota work? [4:55] How else does it... Because you could [4:57] have just started with LLMs [4:58] and you know, we could have skipped [4:59] that period in the wilderness, [5:01] but it sounds like it was somewhat formative [5:02] for the OpenAI organization. [5:04] I think Dota had many lessons, [5:06] one of which actually was a management lesson for me. [5:10] I remember when we started out the project, [5:11] I tried to set a list of milestones, right? [5:14] It's like, "Okay, this date we're gonna beat this player. [5:16] This date, we're gonna beat this player." [5:19] That didn't work? It did not work at all. [5:21] I remember our first milestone came and [5:23] Went? Exactly. [5:25] And so you realize that [5:27] you cannot control the outcome, right? [5:29] You cannot set outcome-based milestones. [5:31] What you can do is you can control the inputs [5:33] of we're gonna try these experiments by this date. [5:35] We're going to implement this feature by this date. [5:37] And that is what actually worked. [5:39] And I remember it was one of those things [5:41] that was just a story that I could not have [5:45] written in any better, if we'd intended to. [5:48] We beat our in-house best player [5:52] and then we were playing as a semi-pro [5:54] and he was just trouncing us, trouncing us. [5:56] Then, suddenly we're starting to get pretty good. [5:58] So we showed up at the international, [6:01] this tournament blind. [6:02] First day, we had three players that we played against. [6:04] We went 3-0, 3-0, and then 2-1. [6:08] We're like, "Oh no. We lost. [6:09] What happened?" [6:10] It turned out that this pro that we're playing against, [6:12] he had used an item we'd never trained against. [6:14] And we were like, "Oh no, [6:15] we're totally going to be hosed." [6:17] So, what do we do? [6:18] Well, we just need to change the training. [6:20] And so people stayed up all night to get this done. [6:22] They added this extra item in there. [6:25] 4am, they finally get the job running. [6:28] That Wednesday, we're supposed to play [6:28] against the number two [6:30] and the number one player in the world, [6:32] and our semi-pro plays against it. [6:34] He's like, "This bot is totally broken." [6:35] And we're like, "Oh no, we like clearly had a bug. [6:37] Something terrible has happened." [6:39] He was like, "Look, it's taking all this damage [6:41] it doesn't need to. [6:42] I'm gonna go kill it." [6:42] He goes to kill it, he loses. [6:44] And he was like, "That was weird." [6:45] He'd realized that what had happened was [6:47] it had learned a baiting strategy. [6:49] And then we realized, "Well, we have a super-bot, [6:51] but it's so bad at the beginning, [6:53] because it's trying to do the baiting. [6:54] So, what if we just stitch the two bots we have together?" [6:57] And then that bot was just undefeatable, [6:59] and we played against this number one player and won. [7:02] To me this is like the story of [7:03] how deep learning works, right? [7:05] Is it's like you kind of can't control [7:06] where you're gonna go. [7:07] You can control everything that goes in. [7:09] You can put these metrics in these measurements [7:11] and you can have sort of the evaluations. [7:13] And being able to gauge where you're at [7:15] is almost as important as being able [7:17] to make the forward progress. [7:18] But if you get all those elements right, [7:19] then you can do true magic. [7:21] You're also describing something [7:22] that worked really well for an organization [7:24] where it was motivating to stay up all night. [7:26] If the prize was impossibly far away, [7:29] it wouldn't have been as motivating. [7:31] But the fact that there was a near-term reward function [7:33] and you were able to show concrete progress... [7:35] I think so. [7:36] I think some of [7:38] my favorite engineering stories have the same character. [7:40] I remember you and I staying up all night [7:42] to get our ISO 8583 integration. [7:44] There's something about staying up all night [7:45] for like critical projects that [7:47] actually have [7:48] important history in [7:49] all startups. [7:50] I'm glad to hear that tradition is alive and well at OpenAI. [7:52] So, Dota has fallen, Chess has fallen, Go has fallen. [7:58] We've passed the Turing test, [8:00] I think by anyone's measure. [8:05] People comment on [8:06] how there was little fanfare when we did, [8:07] but we seem to have pretty clearly done so. [8:09] What's a good new Turing test? [8:11] Well, I'll tell you two things. [8:13] One, is that, if you look at [8:14] the strict version of the Turing test, [8:15] I would actually claim we haven't done it yet. [8:17] No one's really gone that extra mile [8:19] to say, "Can we actually have an AI [8:20] that like is fully indistinguishable from a human?" [8:23] It's not clear if it's even a good task, right? [8:25] But I think that the right question to your point is like, [8:27] well, what is the milestone [8:30] that we should be chasing in terms of capability? [8:32] I remember talking to one of our board members in 2018, [8:35] and he said, you know, "Look, I get that you [8:39] were all excited about near-term AGI, [8:41] but it just doesn't feel like it's on track." [8:44] I asked, "Well, what do you mean?" [8:45] He said, "In a world with near-term AGI, [8:47] you would expect massive economic value [8:49] to be delivered by AI already, and where is it?" [8:52] In 2018, I think that was a very fair criticism, [8:55] and clearly, that's starting to change now. [8:57] It feels like one thing that [9:00] may really change the AI market is personalization. [9:03] Up to quite recently, when you asked ChatGPT a question, [9:07] it was like walking into a shop off the street. [9:10] They've never met you before. [9:11] They know nothing about you, whatever. [9:12] That's obviously, not ideal for this, you know, [9:15] close part of your digital life. [9:16] I'm curious how you're thinking about personalization [9:18] from a product point of view, [9:19] because it feels to me like [9:20] the most meaningful change since the chat interface. [9:25] Two and a half years ago. [9:26] Two and a half years ago. [9:28] I mean, I think it's absolutely critical, [9:29] and I think it is very rightly considered [9:32] to be kind of a next frontier. [9:33] I'm someone who always, when I just Google something, [9:38] I go into Incognito Mode, [9:40] because I don't even want my computer [9:41] to remember that history. [9:42] And I always used to go for temporary chats on ChatGPT. [9:45] But now my usage has totally reversed. [9:47] I want ChatGPT to remember everything. [9:49] I want it to remember my interactions, [9:51] because it's useful. [9:52] Okay. So you guys figured out, [9:53] from a product point of view [9:54] how to make the memory actually work better. [9:56] It's the product point of view, [9:58] but also, really the research point of view. [10:00] And I presume there's a flip flop between the product [10:01] and research where, when you find something [10:03] that's useful from a product point of view, [10:05] then the product people say, [10:06] "I'm just a product person, [10:07] you researchers go actually make this good." [10:09] Then that kind of kicks off more research. [10:10] Is that how it works? [10:11] To some extent [10:13] that's a failure mode in our mind. [10:15] I think that we really [10:16] don't want to have that kind of silo. [10:18] We really want to blur the lines [10:19] and have people cross-collaborate. [10:21] And so, it's avery different mindset from [10:22] how you would traditionally build a product [10:24] versus how you do research. [10:26] Part of what had happened, actually, [10:27] was that we had GPT-3, [10:29] we knew we needed to build a product [10:31] in order to be able to continue to raise funding, [10:33] and we were like, [10:35] "Well, what product do we build? [10:36] And we wrote down a list of, [10:37] like, 100 different products. [10:38] We could do a medical thing. Then you're like, [10:40] "Okay. Well, now we have to sell to hospitals. [10:43] We're gonna have to hire doctors." [10:44] You realize you give up on the G in AGI, right? [10:47] You're gonna like go for a specific thing. [10:49] Someone had the idea of saying, [10:53] "Well, why don't we just make a API [10:56] and let people figure it out? [10:58] And, again, [10:58] this is totally backwards [10:59] from how you're supposed to do a startup, right? [11:01] You're supposed to have a problem, [11:03] and we had no idea what the problem was. [11:04] Yeah, yeah, yeah, we're gonna back into the problem. [11:06] And so this actually felt like this was [11:07] probably the hardest project that I've ever done, [11:09] because it felt totally doomed, right? [11:10] It's like, I know like every instinct, [11:13] every builder instinct of mine says- [11:14] Did it actually feel doomed? [11:15] Oh, it felt totally doomed. [11:16] It wasn't just like open-ended or something? [11:19] No, it felt doomed. [11:19] But you were still doing it. [11:21] Yeah. I mean, it's like, at some point, [11:22] if you have... There was definitely no other path. [11:25] There was no other path. [11:26] It was the only shot we had. [11:28] I remember someone also saying like, [11:30] "I can't imagine anyone paying for samples from this model." [11:33] And I was like, "You might be right." [11:36] I'm still trying to imagine it myself. [11:37] Yes. It was just not clear, [11:39] were we above threshold or below threshold? [11:41] We showed it to people, [11:43] and people were interested. [11:44] But that's very different from people being like, [11:46] "I will build my company on top of this," [11:48] Yes. So, what was the first use case to get any traction? [11:51] AI Dungeon. [11:52] What was that again? [11:54] There you go. [11:56] AI Dungeon was a text-based adventure game. [12:00] Oh sure, yeah, yeah, yeah, yeah. [12:01] Okay. Bbut that was real revenue [12:04] or that was non-zero revenue? [12:05] It was enough. [12:06] And, in fact, I believe [12:07] they were our first paying user. [12:09] And that gets you confused for, you're like, [12:10] "Ah, clearly the future of OpenAI is gaming" you know? [12:13] I know, back to our roots. [12:14] Exactly. [12:15] It's interesting too, [12:18] because we had dreamed of all of these applications, [12:21] like medicine and all these things, [12:23] and you start with the gaming application. [12:25] But we could see signs of life on so many other things. [12:28] I think in many ways GPT-3 was like [12:30] the world's best demo machine, right? [12:32] When we released the API, [12:33] people were coming with all these cool things you could do, [12:35] but making them reliable was so hard. [12:38] It really wasn't until the next generation of GPT-4, [12:41] until we started to figure out how to do post training well [12:43] that then you were actually able to build [12:45] real businesses on top of these things. [12:47] Bill Gates was saying recently that GPT-4 [12:48] was the best demo he'd ever seen since Xerox Park. [12:52] You know this quote? Yes. [12:53] He said it to me the night that he saw it. [12:56] Yeah. That's high praise. [13:00] I wanna touch the medicine thing, [13:02] because you've mentioned it. [13:02] Like you said [13:04] your family has personal stories. [13:07] You've talked about getting very valuable diagnostic help. [13:11] We, ourselves, actually, it's much more minor in our family, [13:15] but we managed to fix a cat [13:16] thanks to debugging it with an LLM. [13:19] I think that's an interesting example, [13:21] because so many people that I know [13:24] have had some kind of experience like this. [13:26] Maybe it's because you actually [13:28] don't get that much time from a doctor. [13:32] Are there other examples like this medicine application [13:36] where you're seeing a lot of success, [13:39] that many people have similar stories, [13:40] but we just hear less about? [13:42] Yeah, I think it's a great question. [13:45] And by the way, I think like medicine is an example [13:47] of one where I kind of thought it was gonna be [13:50] one of the last domains that [13:51] we would successfully be able to add value in, [13:53] but it turns out that the bar is so low, [13:55] you just need to exceed WebMD. [13:56] I think that we have seen [13:59] other areas that are like a real common theme. [14:01] One that's very interesting right now [14:02] is the life coach, [14:03] life advice kind of application, [14:05] where you just talk your AI. [14:06] That's actually really taking off. [14:07] Yeah. It really is. [14:08] Education is another area [14:12] that just like clearly is really having an impact. [14:15] There are studies coming out now [14:17] that actually show that people are able to learn better [14:20] through the use of these tools. [14:22] That's to be expected, right? [14:23] It is the Bloom 2 sigma effect in a product. [14:25] Yes. And that, for example is like why, [14:28] Sal Khan, started Khan Academy, [14:31] to think about [14:32] if you can give personalized tutoring to everyone. [14:34] We showed him GPT-4. He's like, [14:35] "This is the thing. [14:36] We need to become a GPT-4 app." [14:38] I think that there are [14:39] these really amazing applications [14:42] that are affecting everyone's daily lives. [14:45] Obviously, programming is another one, [14:46] that people are seeing [14:48] all across the board in a professional context. [14:50] We're heading to a world [14:51] where, like, [14:53] if you want to do productive work, [14:56] and you don't have access to computer, [14:57] you're going to be hampered. [14:58] Similarly, not having access to AI, [15:01] it's heading in the same direction. [15:02] Speaking of not having access to AI, [15:05] I will posit that these days, it feels like [15:10] AI product development is mostly OS limited. [15:15] Is that how you feel? [15:16] Are we stuck at the moment? [15:18] I do feel a little of stuckage, [15:20] but not to worry, it was overcomeable. [15:23] But yeah, I think it is true. [15:25] Two years ago we released plug-ins in ChatGPT. [15:29] Do you remember those? Yeah. [15:30] That was trying to make it [15:33] so anyone could write apps that then ChatGPT could access. [15:36] The models were just not that good, right? [15:38] That we limited to like three plug-ins at a time. [15:41] You could have only so many functions and stuff. [15:43] It just wasn't that reliable. [15:45] Now we're in a world where MCP basically [15:48] is really taking off [15:49] and is a way to hook up your AI to different tools, [15:54] and very much like kind of trying to take [15:56] that same type of idea and really make it work. [15:59] Now, the world that we're in is very similar, [16:01] where there's certain interfaces we don't have. [16:03] Being able to access your phone and all those APIs. [16:07] There's a question of, [16:08] is the model above threshold to actually use them or not? [16:11] My observation has been that basically, [16:13] I think that there is maybe a lag of, like, six months [16:15] of different interfaces that are hard to access. [16:18] But once we have a model that's good enough, [16:20] we will find a way. [16:21] People will find a way. [16:23] I think that we're in a world [16:24] where I have every expectation that we will get [16:27] the future that has been promised. [16:28] It's just gonna take some work. [16:30] I feel like there are many moments [16:31] where I'm using my phone, and I want a single button [16:35] where it's just, like [16:37] "ChatGPT what do you think of this? [16:39] I need your comment. I need your fact check. [16:41] I need your explanation." Something like that. [16:43] You take a screenshot, and you like go into ChatGPT, [16:46] you click "upload photo." [16:48] It feels very 1993 [16:50] versus the button on my phone that just says, [16:53] "Hey, ChatGPT, what do you think about this?" [16:54] Obviously, you guys are not empowered to go build that. [16:57] That's what I mean by it feels somehow [16:58] like we're a little operating system limited. [17:00] I definitely get it, [17:01] but I'll say, I think that there are two dimensions. [17:04] This is how I've been thinking about things [17:06] since we released the API back in 2020. [17:08] There's capability and convenience. [17:11] What you're referring to is the convenience, right? [17:14] It's like pretty inconvenient [17:15] to do the screenshot and paste it. [17:17] But the thing is, if the capability is good enough, [17:20] you are willing to accept any sort of inconvenience, right? [17:23] It's like if this, by taking the screenshot [17:26] and showing it to ChatGPT, [17:27] it could give you amazing insight, [17:29] it could tell you like how to, you know, [17:30] build Stripe in some way, [17:32] and it takes you like a month to do it, [17:34] you have to crawl the top of the mountain, [17:35] you'll do it, right? [17:36] The convenience will not stop you. [17:38] And so the point that I'm trying to make is that [17:41] if the capability is high enough, [17:43] people will start doing a specific flow. [17:45] They'll discover the use cases [17:46] and the convenience will just catch up. [17:47] In the convenience, [17:48] there's so much pressure. [17:49] There's pressure on the phone manufacturer. [17:50] There's pressure on us. There's pressure on everyone [17:52] in order to bring down the convenience. [17:54] o, I just need to be patient, [17:55] and it'll be great in three years time. [17:56] Yes. And really lean in and use the AI. [17:59] A criticism people like to levy of AI is, [18:02] "Yeah, it's great and handy and all, [18:04] but it hasn't come up with a single novel advance [18:09] in mathematics or science." Have you? [18:12] Well, you could have [18:13] if you'd become a mathematician. [18:15] But you know, but humanity has, you know, [18:18] for keeping the scoreboard. [18:20] What do you make of that criticism? [18:22] Just wait. [18:23] Okay, so you think like take [18:27] one of the Millennium Prizes [18:28] Do you think we plausibly will see that? [18:29] I think for sure. [18:30] I mean, there's no question. [18:31] Two years, five years, 10 years? [18:33] I think that is the question. [18:34] It's just timing. [18:35] That is my question. [18:37] I mean, I would put two to five years [18:39] as the right number. [18:41] I think ultimately this comes back [18:44] to the question of benchmarks, right? [18:45] Actually being able to solve a millennium problem [18:48] is pretty high bar. [18:51] Yeah. [18:52] And once you can do that, there's so many other things [18:54] that will definitely be possible. [18:56] And I think that we're starting [18:57] to see the leading edges of this. [18:58] And to me, if we look at our definition of AGI... [19:02] We recently started talking [19:03] about this framework of thinking about levels of AGI, [19:06] starting from chatbots, to reasoners, to agents, [19:09] to innovators, to organizations, Five levels. [19:13] We're basically, somewhere in level three right now. [19:16] Level four, [19:17] this innovator, like, that's gonna be different. [19:19] I recently posted some pictures [19:20] of our visit to Abilene, Texas, [19:22] where we're building these big data centers together [19:24] with our partner, Oracle. [19:26] Imagine taking that whole data center [19:32] and just thinking hard about one problem, right? [19:34] Imagine it just thinking about [19:35] how to solve a Millennium Problem [19:37] or how to cure a specific kind of cancer. [19:39] Maybe it needs access to some apparatus. [19:41] Maybe it needs access to robotic wet labs. [19:42] Maybe it needs access [19:43] to different tools in the world. [19:45] But that level of computational power [19:48] coupled with the ability to experiment [19:50] and learn from your ideas, [19:52] that is going to be something the world has never seen. [19:55] So yet again, we just haven't put a respectable amount [19:59] of compute on these problems [20:01] compared to what we will be doing. [20:02] Yeah, we're still on these tiny little computers. [20:04] So, that actually gets to, [20:06] in terms of these scaling laws, do they eventually run out [20:09] because we just run out of compute? [20:11] Or do we eventually get to the point [20:13] where we're inventing new kinds of nuclear energy, [20:15] and that is what unlocks the next level? [20:18] A lot of energy that comes online now is for data centers, [20:21] which was not true when you guys started training GPT-2. [20:25] Isn't that the upcoming bottleneck? [20:28] I mean it's as it should be, right? [20:30] It really should be that it's energy [20:33] manufactured into intelligence [20:35] and that's your only bottleneck. [20:36] But I'm saying that'll be like quite a plateau [20:39] compared to the exponential growth [20:41] we've seen over the past few years. [20:44] Unless things really change in terms of [20:46] permitting and plans for building everything like that. [20:48] This is I think the core, right? [20:50] If you look at every trend in this field, [20:54] there are these exponentials, [20:55] these S-curves that sum up to exponentials. [20:57] Sure. But these exponentials were mostly existing [20:59] in like tech, Silicon Valley space [21:03] where it was pretty easy to have exponential growth. [21:05] It's pretty hard in permitting and real estate [21:08] and damming rivers and building nuclear power plants. [21:11] It's harder to have exponential growth there. [21:13] Well, let's see how fusion pans out. [21:15] Yeah. Okay, but even fusion, [21:16] most industry observers would say [21:18] is still five years away. [21:20] And so, where does the next five years [21:20] of power growth come from? [21:22] I think that it is very possible [21:24] that we end up bottlenecked on energy, [21:27] and that's actually, one reason that we've been spending [21:28] a lot of time really trying to advocate for the fact [21:31] that we just need far more power. [21:34] My observation of the market is that ultimately, [21:37] the capitalist markets do provide. [21:39] I think there's this like absolute tsunami of demand [21:42] that is coming our way, [21:44] but I feel some confidence that, again, there's like, [21:48] when there's enough pressure, [21:49] when there's enough clarity of, this is the bottleneck... [21:51] And it's not just for any company, right? [21:54] It's really for national competitiveness. [21:56] You look at other countries [21:57] that are just building huge amounts of power, [22:00] far more than we are. [22:01] I think that actually for America to remain competitive, [22:04] there's just no choice but to build it. [22:06] We've got to figure out power. [22:06] We do. Speaking of bottlenecks, [22:08] everyone was talking about the data wall in 2023. [22:12] I think this is an interesting thing, [22:13] where no one is talking about the data wall anymore, [22:15] and yet, it doesn't feel like AI progress has slowed down. [22:20] Is it just test-time compute? [22:22] Is it, like, people were wrong about the data wall [22:24] and where it presented a bottleneck? [22:26] Is there actually still a data wall, [22:28] but it's two years away? [22:29] It's basically, all of these things, right? [22:30] It truly is. [22:32] It's like, you keep changing the paradigm. [22:34] That is the real core of [22:36] the Kurzweil view of the world, [22:37] is that, fine, this one way of doing things taps out. [22:41] If you just look at that one way of doing things, [22:43] you feel hopeless. [22:44] You feel like this is it, [22:45] but somehow you will find a new S-curve. [22:48] And I think that's what's happened. [22:49] For example, synthetic data, [22:50] for example, reinforcement learning, right? [22:51] If you think about the RL paradigm, [22:53] fundamentally that's a data production mechanism, right? [22:56] And it just that the AI [22:57] happens to be training on its own data [22:59] and then you learn it very rapidly, [23:00] and then you learn on that. [23:02] Each of these has taken us much further. [23:05] I think there are lots of algorithmic ideas, [23:07] lots of techniques, lots of ways [23:08] of even using the existing data better. [23:11] I think that fundamentally the S-curves continue, [23:15] and if you zoom out, it all looks smooth and uninterrupted. [23:19] So, it's kinda like chip miniaturization [23:20] where each generation people are like, [23:22] "Okay, well, that's the smallest [23:23] you can possibly make a chip." [23:25] Do you know what I mean? [23:25] "That's it we're done with miniaturization," [23:27] and somehow we figure out a way. [23:28] Yes. Now one difference with chips [23:30] is at the end of the day, [23:30] there is some limit. [23:33] Right. But we've never been that close to that. [23:35] Yes. [23:37] Where does AI coding go? [23:39] In particular, vibe coding is all the rage right now. [23:44] It's kind of the term of 2025. [23:48] It's sort of working. It's very impressive. [23:51] No one is really fully letting AI software engineers [23:56] run end-to-end in production. [23:58] I'm just curious, [23:58] what are your one to two-year predictions [24:00] on what happens with AI coding? [24:02] Well, my general observation is that [24:04] once something kind of works in this field, [24:06] the next gen is gonna be great. [24:08] I think that's where we are right now for AI coding. [24:10] I think what we're going to see [24:12] is AIs taking more and more of the drudgery, [24:16] more of this like pain, more of the parts [24:18] that are not very fun for humans. [24:20] Now, one thing that's very interesting is that [24:22] I think that so far, the vibe coding [24:24] has actually taken a lot of code [24:26] that is actually quite fun [24:27] and left behind the review and the deployment, [24:29] these things that are not fun at all. [24:31] I'm hopeful that we're actually [24:32] gonna be able to make a lot of progress [24:33] on these other areas as well, [24:35] but fundamentally, we should really end up [24:37] with a full AI coworker. [24:39] And I think it really will be anything you want to create, [24:41] you can be the manager, [24:43] and you can have [24:44] this team of software engineering agents. [24:45] Now, the thing that I think will be [24:47] very interesting to see is, [24:48] is there a world where the AI becomes the manager [24:51] and it gives you ideas, [24:52] gives you some tasks to do? [24:54] That's something that, again, [24:55] it's just totally backwards [24:56] in terms of how we think about it. [24:58] But are there ways in which you can actually [25:00] have outcomes for companies [25:04] and actually have people whose jobs [25:05] become much more meaningful [25:06] because they have an AI who really deeply understands them [25:09] in the same way that your [25:10] AI doctor really deeply understands all of your needs. [25:14] But isn't part of the common thread [25:16] that we're talking about here, [25:17] often places where AI tools underperform, [25:20] it's because they're trying to do something generally, [25:23] Like, voice recognition is not that good [25:25] because it's trying to recognize all voices [25:27] as opposed to trying to recognize [25:28] my voice in particular. [25:30] Similarly, with AI coding, they work well in places [25:33] where you need no context at all [25:36] and we're single-shotting an app based on [25:39] publicly available libraries. [25:40] In places where you have to understand [25:43] a million-line code base, [25:45] they haven't fully figured out how to do a good job of that. [25:50] Is that a fair parallel [25:51] to draw between all these challenges? [25:52] Well, I think there are two things in there. [25:54] One is that I think this is already changing, right? [25:57] So if you look at something like Codex, [25:59] it's actually great at operating in a big codebase. [26:02] Like I ask it for where functionality is implemented [26:04] and it's better than I am at finding it. [26:07] Which is kind of a wild fact. [26:09] It's super cool to see it like grepping around [26:11] and just going and exploring. [26:14] Actually, this is one thing [26:15] that we really shot for with Codex, [26:16] was to build a tool for software engineers [26:19] who are not necessarily vibe coding. [26:21] It's not about building a new app from scratch, [26:23] which is a cool demo, [26:24] But that's not actually [26:25] how most software gets written. [26:27] Actually, I think maybe the killer enterprise feature [26:29] is refactors, right? [26:30] It's like rewriting your COBOL app or changing- [26:33] When Facebook did hip hop to do static PHP. [26:37] If you think about it, [26:38] the amount of deep, sophisticated thought [26:41] that is required to accomplish a refactor [26:43] is actually not that high. [26:44] There's a lot of mechanical work [26:46] that's just the sheer volume of it that's hard, [26:48] That's an AI-shaped problem for sure. [26:52] I think we're going to see a lot more productivity [26:55] on all sorts of AI tasks as a result. [26:58] Now, we are in a world where you said a second thing, [27:01] which is maybe you need [27:03] to narrow down more. [27:04] I think that the way these models work [27:06] is you actually do want one model [27:07] that knows more and more things, [27:09] and you want it to have some personalization to you, [27:12] but the fact that is [27:14] they have this one base model [27:15] that kind of knows everything is actually [27:17] a very useful starting point. [27:18] So I do think that you're going to see a world [27:20] where we'll have more and more capable base models [27:22] and figuring out how you really connect it [27:24] to all of your organization's code, [27:25] and context, and history. [27:28] How does OpenAI decide what products to do? [27:32] I'm just curious how you think about [27:35] when to develop specific products or when you think, [27:38] "Oh, you contributed do that with ChatGPT, [27:40] and that's good enough." [27:41] Yeah. It's is a really tough question, right? [27:43] It's something we really struggle with, [27:44] and I think that over time [27:46] when we first launched ChatGPT, [27:49] we're left with this, "Well, we're an enterprise business, [27:52] and we're a consumer business." [27:53] And seems terrifying, as a startup. [27:56] I remember talking to one of my board members who said, [28:00] "It just feels like what you have [28:01] is an unfocused strategy at first, [28:03] because you're just doing all these different things." [28:06] But if you think about it, [28:07] maybe an analogies to a company like Disney [28:09] where you make one core asset like Little Mermaid, right? [28:13] Then, you productize it [28:15] in all these different ways. [28:16] You think about Little Mermaid the ride, [28:18] the lunchbox, the T-shirt. [28:21] I think that we have some element of that. [28:22] We have a core model, [28:23] and then we have a question of, [28:24] "Well, what are the applications [28:26] that this can add a lot of value to quickly, right? [28:29] With like a small amount of additional work. [28:31] So I think the question of what areas to go into are [28:34] how far does it take us off the general path, [28:37] the return on how important is this domain area, [28:41] especially for achieving the bigger goal, [28:43] how much synergy is there across it [28:45] with respect to other things we work on? [28:46] So coding is one where there's very clear synergy, [28:49] very clear ROI. [28:50] Because if we can speed ourselves up, [28:52] that's something that accelerates everything. [28:55] How is being from North Dakota, how has it shaped you? [29:00] Look, North Dakota was an amazing place to grow up. [29:01] Was it actually? Come on. [29:04] I've been there. [29:06] And it was great. [29:08] Look, it was incredibly safe. [29:09] Our doors didn't even have working locks. [29:11] It was that kind of place. [29:13] I had a lot of freedom academically. [29:15] Sixth grade, my dad taught me some algebra. [29:17] Seventh grade was the first time [29:19] they split you into advanced math, [29:20] so I was going to be taking pre-algebra. [29:22] My mom took me to go see the teacher, [29:25] and we asked, "Can you skip?" [29:27] The teacher looked at us very condescendingly [29:29] and said, "Every parent believes [29:31] that their child is special. [29:33] I can guarantee your son will be [29:35] plenty challenged in my class." [29:36] And so, after a month of me sitting in the back [29:39] just playing games on my calculator [29:40] and you know, she'd call me randomly to try to trip me up, [29:43] and I just look at the board and be like, "2x." [29:46] She said, "Okay, fair enough. [29:47] Your son has nothing to learn in this class." [29:50] So, they moved me into eighth grade algebra. [29:52] But then eighth grade rolled around [29:54] and I had no more math left in my middle school. [29:57] That's right you went to college, right? [29:58] Well, so I did, in high school [30:00] start going to University of North Dakota, [30:02] and took a bunch of classes there. [30:04] But also, I was connected to a lot of people [30:06] who were the top math kids in the country [30:09] through things like math camp and the math competitions. [30:11] So, you're saying the social scene [30:12] was not too distracting in North Dakota? [30:15] Not too distracting, [30:16] but it was definitely fun. [30:19] Last question, do you remember [30:20] we were going to Camp YC in 2017 [30:25] and I asked you how far AGI was away [30:29] and you said two or three years. [30:32] Did I say that? [30:33] You did. [30:34] I don't see the recording. [30:35] Well, I'm just trying to think [30:39] was I right, were you right? [30:40] How should we grade that? [30:41] Because we didn't get AGI, [30:43] but we didn't not get AGI either. [30:46] I'm curious if you have any reflections [30:49] from your own AGI prediction journey. [30:53] I think that we are... [30:55] I will say I think that AI is surprising. [30:57] I think that that is like the single most consistent theme, [31:01] is that the thing we were picturing, [31:02] we got something different, [31:03] but we got something better, more magical. [31:06] Something that is more helpful. [31:07] And so, I'm actually quite happy with that. [31:10] Now, predicting where you go, [31:12] it's again, it's really hard to manage the outputs here. [31:14] One goal of OpenAI that we have successfully achieved, [31:17] is every year to have at least one result [31:21] that just feels like a step function better [31:23] than anything before. [31:24] You know when you see it kind of. [31:26] You just have one really awesome AI feeling thing each year. [31:29] That kind of thing. [31:30] I like that. Yeah. [31:31] That's a good way to tie it back, [31:32] which is, you know, the way we grade that prediction [31:36] is that you've stopped setting metrics based on outputs. [31:38] Yeah, exactly, exactly. Yeah. Yeah. [31:40] Yes. But it does feel we're getting really close [31:42] to something pretty magical. [31:44] I agree. Thank you. Thank you.