OpenAI Chief Directs Research Roadmap Toward Automated Technology Invention

OpenAI高管明确研发路线：迈向自动化技术发明 预训练与扩展规模的标准方法正与分步内在独白能力相叠加，推动AI开发转向能够在复杂科学任务上投入海量算力、连续运行数天或数周的模型。 在本期节目中，OpenAI的Jakub Pachocki与Szymon Sidor回顾了他们从波兰竞赛编程高中起步、最终成长为AI研发领军人物的历程，并回顾了从原始情感分析到01等先进模型的快速演进。他们详细阐述了AI如何正在从死记硬背迈向长达数小时的深度内部推理，模型在无需外部工具辅助下于人类顶尖数学与编程竞赛中取得里程碑式表现便证明了这一点。展望未来，他们概述了通过海量算力扩展与延长模型持续运作时间来攻坚医学和AI安全等高价值领域的范式转变。最后，他们驳斥了“学习编程已过时”的说法，认为结构化解决问题的能力仍将是一项持久且高价值的认知财富。 OpenAI 首席科学家 Jakub Pachocki 提出 AGI 的核心评判标准已从单点基准测试转向实际影响力，即能否实现科学发现与技术发明的全流程自动化。 (3:35) 研究员 Szymon Sidor 指出 AI 过去十年经历了惊人的指数级演进，当前 3% 至 5% 的经济影响较十年前几乎为零的起点已是天文级跃迁，而特定能力的饱和使得传统基准不再能充分衡量通用智能。 (9:33) AI 模型已能在不借助外部工具的情况下通过多小时自主深度推理斩获 IMO 金牌级成绩；尽管仍未攻克最具颠覆性的“第六题”（Problem Six），但模型展现出了能够诚实承认解题失败的自知力，而非盲目产生幻觉。 (21:00) 由 o1 模型开启的“内心独白”推理突破将与预训练规模进一步融合，下一阶段的关键在于扩展“模型持久力”（Model Persistence），即支持模型在单一高价值科研问题上连续投入数天乃至数周的密集算力。 (28:43) 针对 AI 时代无需再学编程的言论，嘉宾驳斥其为错误信息，强调编程是培养结构化拆解复杂问题思维的最佳途径，这种底层认知在技术交互和提示词工程中仍具核心竞争力。 (33:42)

OpenAI Chief Scientist Jakub Pachocki and researcher Szymon Sidor argue that artificial general intelligence should be defined by its ability to automate scientific discovery rather than standardized test scores, as progress shifts from rote benchmarks toward multi-hour reasoning and prolonged compute allocation. Redefining AGI Around Automated Scientific Discovery (3:35) Pachocki observes that early deep learning grouped disparate goals—such as fluid conversation, advanced mathematics, and research workflows—into a vague, distant vision of AGI. Today, models converse across broad subjects and earn gold medals in international competitions, rendering narrow benchmarks less representative of general capability. Pachocki now anchors the definition of AGI to tangible technological invention, anticipating systems capable of generating foundational scientific ideas and automating engineering workflows. Benchmark Saturation and the Search for Authentic Reasoning (14:22) Because models can be trained data-efficiently on specific domains like mathematics without a matching gain in other areas like writing, high standardized test scores no longer signify well-rounded intelligence. To assess genuine problem-solving, researchers increasingly look to competitions such as the International Mathematical Olympiad (IMO) and the 10-hour AtCoder Heuristic Contest (AHC). In the IMO, models achieved gold-level scores purely through internal reasoning without external calculators, while an OpenAI model placed second in AHC against top human engineers. Despite these milestones, blind spots remain. Both OpenAI and Google DeepMind failed to solve the IMO's unconventional "Problem Six," though OpenAI's model demonstrated self-awareness by acknowledging its failure rather than hallucinating an answer. Additionally, the researchers acknowledge that academic contests exist within an insular bubble, making diverse real-world ChatGPT usage an essential metric to keep evaluation grounded. Reasoning Chains and Long-Horizon Compute Scaling (26:29) The introduction of the 01 model marked an operational shift from instantaneous outputs to an internal chain-of-thought "inner monologue." Sidor highlights how rapidly these systems evolved from decade-old sentiment models that stumbled on basic phrases like "this movie is not bad." Sidor projects that the economic footprint of AI—which he estimated grew from roughly 0.00001% a decade ago to between 3% and 5% today—could reach 10% to 20% in one to two years. The research roadmap is now centered on "model persistence," extending the compute budget spent on difficult problems. While GPT-4 utilized fixed compute per answer and newer models use 10x to 20x more, high-stakes domains like drug discovery and AI self-improvement could justify models working uninterrupted for days, weeks, or months to uncover solutions. Foundational Skills in an Automated Future (33:42) Addressing public speculation that programming is becoming obsolete, the speakers dismiss the idea as misinformation. They argue that coding teaches essential structured decomposition, noting that technical fluency enhances prompt engineering and natural language interaction. Comparing non-technical developers to airplane pilots who do not understand aerodynamics, they maintain that foundational mechanistic comprehension remains an irreplaceable advantage even as autonomous systems advance. While the researchers foresee AGI functioning as an automated enterprise of scientists and engineers, they warn that the industry has not yet achieved the robustness required to securely link models to sensitive personal data like email and calendars without risking exploitation.

OpenAI首席科学家Jakub Pachocki与研究员Szymon Sidor指出，通用人工智能（AGI）的核心衡量标准已从孤立的学术测试转向自动化科学发现与技术创新的能力。他们认为，随着模型从即时问答演进为具备“内在独白”的深度推理系统，未来突破将取决于模型在复杂现实任务中进行长周期、高强度算力扩展的持续求解能力。 重新定义AGI与基准测试的局限 (3:35) Jakub Pachocki认为，传统的点状测试（pointwise benchmarks）已无法充分评估通用智能。几年前自然对话与复杂数学推理看似遥不可及，如今模型已能在各类广泛话题中流畅交流，并在国际数学奥林匹克（IMO）中斩获金牌水平成绩。然而，这一飞跃伴随着基准测试饱和现象的出现。由于当前技术能够以高数据效率单独训练模型掌握特定技能（例如在数学能力飙升的同时写作能力不增），孤立的高分不再代表全面的通用智能。Pachocki提出，真正的AGI应当以实际影响来定义，即计算机能够自主自动化地发现新科学原理与发明新技术。 在评估前沿推理能力时，人类顶级竞赛展现了模型潜力的边界。模型在不借助计算器等外部工具的情况下，纯靠内在推理完成了IMO级别的解题，这与两年前连四位数乘法都频频出错的水平形成鲜明对比。但面对代表极端创新思维与非标准解法的IMO“第六题”（Problem Six）时，OpenAI与Google DeepMind的模型均未取得进展。值得注意的是，模型并未产生幻觉编造虚假证明，而是准确识别自身无法推进并主动放弃。此外，研究团队强调必须走出以小众学术竞赛为中心的“基准气泡”，将数百万用户在ChatGPT中的广泛现实应用作为检验模型真实效用与通用性的重要标尺。 思维链推理与长时算力扩展 (26:29) 以o1模型为代表的新范式标志着人工智能进入“内在独白”（inner monologue）时代。研究人员指出，让模型在给出答案前进行多步骤自主思考与自我对话，在工程实现上极其艰辛，但实验证明持续增加训练数据能稳定提升其推理性能。Sidor回顾指出，这种突破速度甚至引发了管理层深夜讨论组织是否能适应如此剧烈的技术复利增长。尽管公众可能因数周缺乏重磅发布而产生技术放缓的错觉，但底层推理能力的年化复合增长依然迅猛。 除传统的预训练扩展外，当前的核心前沿在于延长模型针对单一复杂问题所消耗的推理算力周期。在日本举办的AtCoder启发式竞赛（AHC）这一长达10小时的开放式优化比赛中，OpenAI的模型展现了持续迭代能力并夺得全球第二名，仅次于顶级人类选手Hiro。Pachocki预测，未来的关键路径在于“模型持久性”（model persistence）——将算力消耗从单次回答的数秒延长至数天、数周甚至数月，使系统能够专注于攻克高价值的重大难题，例如生物医药突破或下一代AI架构的设计。 自动化科研愿景与安全鲁棒性挑战 (30:46) 针对AGI在未来五年对普通人的实际形态，对话嘉宾描绘了一个由高度自动化的研究员与工程师团队构成的协作系统。该系统不仅能与人类深度互动，还能自主设计代码库、执行实验并推进技术迭代，从而显著提升人类社会的整体技术生产力。同时，随着智能体表现出更高的持久性与更丰富的表达能力，人机之间不可避免地会建立更深的情感联结，这一趋势将引发广泛的社会与伦理讨论。 然而，技术落地目前面临严峻的安全门槛限制。尽管用户普遍期望将ChatGPT与Gmail、日历等敏感个人数据深度整合，但嘉宾明确指出，整个行业目前尚未达到保证模型免受恶意利用所需的“鲁棒性门槛”（threshold of robustness）。在未解决数据安全与系统漏洞之前，盲目扩大自动化权限将带来难以承受的安全隐患，如何在巨大的经济实用价值与系统安全性之间取得平衡仍是当下未解的核心挑战。 编程思维的持久价值与底层科学认知 (33:42) 针对AI将导致编程技能过时的观点，受访研究者将其直斥为“错误信息”。他们强调，编写代码的本质不仅是掌握语法，更是训练高度结构化的逻辑思维与将复杂系统拆解为可执行模块的能力。这种思维方式即使在AI辅助时代也是高端认知技能，并能直接赋能提示词工程等新兴工作。他们以航空飞行作比：即便自动驾驶技术成熟，人们依然需要深刻理解空气动力学的飞行员，理解底层技术运作机制的人才将始终保有显著的竞争优势。 回顾个人成长轨迹与深度学习的崛起，两位科学家谈及从波兰中学恩师Ryszard Szubartowski的竞赛训练，到硅谷文化与保罗·格雷厄姆《黑客与画家》的启蒙影响。他们指出，AlphaGo及AlphaGo Zero的突破证明了无监督自学与深度学习范式的深远潜力。对于传统计算机科学家而言，研究深度学习往往更像是在观察一种物理现象而非纯粹的符号计算，因为其底层的优化机制尚未在数学上被完全解析。因此，深入掌握数学、理论计算机科学与物理学等底层学科，对于理解并引领未来的智能革命至关重要。 两位研究者总结认为，通往实用AGI的路径已清晰指向复杂推理与持久算力扩展，但技术演进的瓶颈已不仅是算法生成能力，还包括系统对自身边界的精准认知、极端非传统逻辑的突破（如Problem Six所反映的局限），以及在个人数据整合中的安全鲁棒性防线。如何在算力指数级投入与防范恶意风险之间建立可验证的信任机制，仍是迈向全面自动化科研前必须回答的根本问题。

[9:33] Jakub Pachocki: this movie is not bad [9:33] Model: oh, negative. [18:52] Anna Makanju: What's IMO? [23:30] Hiro: Your models are very, very bad. I want to go to sleep. I am tired. [26:29] Jakub Pachocki: Are we ready as an organization for incredibly fast-paced progress? [33:24] Invest in Bitcoin [33:42] do I want an airplane pilot who doesn't understand aerodynamics? [37:25] maybe the movie is a movie.

[0:00] Hello, I'm Andrew Maine and this is the [0:02] OpenAI podcast. Today our guests are [0:04] OpenAI's chief scientist Yakob Pahhatzki [0:07] and Simone Sedor. We're going to talk [0:09] about measuring AI progress, how you [0:12] determine AGI, and where the next [0:14] breakthrough might come from. [0:16] >> The model was able to correctly identify [0:19] that didn't make progress on the [0:20] problem. [0:20] >> We started asking very very seriously [0:22] the question like are we ready as an [0:24] organization for for incredibly [0:27] fast-paced progress? when we think about [0:28] how we shape our research program at [0:30] OpenAI. We seek to create intelligence [0:32] that's very general. [0:35] >> I want to first start off by [0:37] understanding your roles. So, Yakob, [0:39] you're the chief researcher, chief [0:40] scientist at OpenAI. [0:42] >> Chief scientist. Yes. [0:42] >> Okay. What does chief scientist mean? [0:45] >> So, the primary thing I'm responsible [0:47] for is setting the research road map for [0:49] the company. Um so deciding what is a [0:53] technical path we are going to bet on [0:55] and what is the um the underlying [0:58] long-term research that that that that [1:00] we're going to pursue. [1:02] >> So how about you Simon? What do you do? [1:05] >> Random things. [1:06] >> Random things. Okay. [1:08] >> Um yeah I I mostly do IC work. Uh I try [1:12] to um [1:14] maybe sprinkle of leadership somewhere [1:16] in there. Uh I try to do whatever is [1:19] most useful. [1:21] >> Now you two knew each other before [1:22] working at Open Eye, right? [1:23] >> Yeah, we went to the same high school. [1:25] >> Same high school. [1:26] >> Yeah. [1:27] >> Were you guys friends? [1:28] >> Uh I think we became best friends when [1:31] when uh after we left like I think kind [1:33] of coming to us is the kind of emotional [1:36] experience that forms bonds. Uh I think [1:39] in in in high school uh uh we were more [1:43] like colleagues. What what kind of high [1:45] school produces guys like you? [1:47] >> So, well, yeah, we went to this high [1:49] school in u in Poland. Uh I think we [1:53] were both drawn there by this computer [1:55] science teacher, Mr. Rashard Subarovski. [1:59] um who's had a great track record uh [2:04] before before we went there of of of of [2:07] uh bringing up uh um computer [2:10] scientists, programmers uh um with this [2:14] like big focus on programming [2:16] competitions and kind of and pursuing uh [2:19] you know excellence in this like like [2:21] one field. Yeah. So I think that was [2:22] like a very formative experience and a [2:24] great mentor for us. [2:25] >> Oh wow. [2:26] >> Yeah. Yeah. Definitely. Uh I think Keer [2:29] was like really going deep on [2:31] programming. I think it went way beyond [2:34] like typical high school curriculum like [2:36] there was like graph theory matrices and [2:38] all sorts of stuff like that. I actually [2:40] hope that maybe with CH GPT it's a [2:43] little bit easier for people now to do [2:45] this kind of deep dives [2:47] >> because um you know without the right [2:49] mentor and without [2:52] a lot of work it's it's kind of hard to [2:54] replicate that experience. I've been [2:56] using it to explain things like there's, [2:58] you know, the Monty Hall problem where [3:00] you have to choose which door and you go [3:02] into chat GPT and you say make a graphic [3:04] interactive version of this and all of a [3:05] sudden you can see it. It can show you [3:07] the different solutions if you do one [3:08] thing or the other. I think that's it's [3:10] one of these things where I'm excited [3:12] about the ability not just to explain in [3:14] text but to build multimedia to do [3:16] things. And it does get into the area of [3:19] there's not really a measure for that. [3:20] you know that it's a use case you go [3:22] this didn't exist before and you know [3:25] we're we talk about AGI but we kind of [3:27] have very loose definitions and whatever [3:30] and I'd love to hear kind of like what [3:31] how you would describe it both from a [3:33] technical and also like a lay person's [3:34] understanding of it. [3:35] >> Yeah. Well, maybe to like address the [3:38] point about teaching um you know this [3:40] sort of like better [3:43] um explanation of some concept or like [3:46] or or you know teaching chosen method is [3:48] like definitely [3:50] uh yeah powerful use of tragic and I [3:52] think works well with like [3:55] um [3:57] um a teacher like like like our Mr. [4:00] But at the same time, um I think like [4:02] the the thing that like he was able to [4:05] provide was more like kind of emotional [4:06] support and space, which I think is uh [4:09] it will be it will be hard for AI to to [4:11] to do alone. That's a great point. I [4:14] think I think that gets lost a lot cuz I [4:15] sometimes hear people talk about, oh AI [4:17] will replace education. And for me, I [4:20] had teachers that maybe their facts [4:22] weren't always right, but their heart [4:24] was there and they were carried and they [4:25] answered questions for stuff. And so I [4:27] think that's a good point that these are [4:28] companions to that and I think that a [4:31] teacher using these tools can be a more [4:33] capable teacher. [4:34] >> Yeah. [4:34] >> But so on the subject of AGI though uh I [4:38] want to hear first like give me your [4:40] technical definition of it or actually [4:42] not a technical question give me like [4:43] how you would describe it to like if you [4:46] were talking to you know a younger [4:49] sibling. [4:50] >> Yeah. A few years ago when we would talk [4:54] about AGI um [4:57] you know it felt like the techn the [4:59] technology deep learning you know has [5:01] incredible promise but at the same time [5:04] the concept still felt a little bit [5:05] abstract right and far away. Um and so I [5:09] think you know whether you talk about [5:12] like you know human level intelligence, [5:13] ability to converse naturally uh you [5:16] know ability to you know solve math [5:18] problems or pursue research like they [5:20] all kind of felt like in the same um [5:23] space. [5:25] Um [5:26] I think [5:28] yeah as as technology has progressed now [5:30] we see there's like this these are [5:32] actually like quite distinct uh [5:34] capabilities and I think we pretty [5:38] clearly are at the point where the AI is [5:39] able to converse naturally uh on a a [5:43] wide range of topics um you know it is [5:45] able to solve math problems I think you [5:49] know like getting gold medal at IMO is [5:52] something we've long discussed as like a [5:54] milestone on the path to AGI and that [5:56] that happened. Um I think um solving you [6:01] know solving all the problems on the [6:02] national map lay actually a little bit [6:04] harder and and you know I think it's [6:06] like another milestone on the path there [6:07] but I think increasingly um you know we [6:10] see that like this kind of pointwise [6:12] measures are less adequate and so we we [6:15] turn to thinking about like what is what [6:16] is its actual impact in the world. Um, [6:20] for me personally, the the the the [6:23] thing that I think about when I think [6:25] about how um AI progress really impacts [6:30] the world meaningfully, I I I first [6:33] think about its potential for automating [6:36] um the discovery and production of new [6:38] technology. [6:40] Um [6:42] I think um [6:45] you know we tend to associate kind of [6:47] new ideas you know fundamental [6:49] technological progress with just human [6:50] ingenuity. Uh and you know we we we [6:54] measure kind of the you know the the our [6:57] progress by this like kind of major [6:59] milestone inventions and and and and [7:02] technological revolutions. Um and I [7:05] think it is just hard to internalize [7:07] like this is possible to automate most [7:11] of this process. It is possible to have [7:13] uh big computer that is coming up with [7:15] ideas that fundamentally change our [7:16] understanding of the world. And I I [7:18] actually think that is not that far [7:19] away. Uh and so thinking about that you [7:22] know what separates us from that and [7:24] what are the consequences of such [7:25] technology is is yeah is my first [7:28] thought. I I just ordered a little Mac [7:31] Studio because I want to take uh the [7:34] open source model GPDOSS and I want to [7:36] just let it run non-stop because that [7:38] idea just the idea of letting it [7:39] generate and do stuff 24/7 is [7:41] fascinating to me. But you're talking [7:42] about a scale of basically automating [7:44] science at a huge scale. And so what [7:48] kind of discoveries what kind of things [7:49] do we think might be the first things we [7:51] could see from that? [7:53] When we think about how we shape our [7:55] research program at OpenAI, we seek to [7:58] create intelligence that is very [7:59] general. Um we you know drive towards [8:03] this automated researcher as a priority [8:05] but we um you know we don't really think [8:07] of it as like let's take this specific [8:10] domains and let's kind of like deploy [8:12] this technology there. I think that is a [8:15] way to like make faster pointwise [8:16] progress. But I think the potential for [8:19] like the really big discoveries and and [8:21] and and and [8:24] the most meaningful technology [8:25] advancement comes from this general [8:26] generality. [8:27] >> Mhm. [8:28] >> Um although still I think we see kind of [8:30] like you know the technology like uh you [8:34] know is is is kind of easier to apply in [8:36] some domains than others. I think um I [8:40] think especially in places that combine [8:42] u a large amount of reasoning with a lot [8:44] of kind of domain knowledge and [8:45] intuition [8:47] uh seem uh [8:50] very [8:52] um [8:53] very aminable to to to to these systems. [8:57] I think in particular we see like pretty [8:59] incredible results on medicine which is [9:01] very encouraging. Uh I have high hopes [9:03] about that. Um yeah I think naturally [9:06] being a company of AI researchers uh we [9:10] think a lot about u automating our own [9:13] work. I think it is also kind of a uh [9:17] you know I I don't think it's a [9:20] you know if it is if AI can indeed reach [9:22] a point where you can automate AI [9:24] research and that is probably a very [9:25] important thing to automate. Uh and you [9:27] know and similarly thinking about like [9:29] how it can help with automating research [9:30] on AI alignment and safety. I'm [9:33] obviously impressed by the like IMOI [9:36] results. I I mean I was actually about [9:39] to add that like um in the past when we [9:41] were talking about the IMO of Yakob like [9:42] that was like you know a few years ago [9:45] uh um and we were still like kind of [9:48] trying to even figure out what our [9:50] definition of AGI might be like like one [9:52] one kind of concept we were considering [9:55] is something like you know solving all [9:57] the problems on on on the math olympiad [10:00] uh and um and and why did that feel [10:04] appropriate is just like Okay, if you [10:06] have a model with such a superior [10:07] mathematical reasoning, then it should [10:10] be able to like disrupt like a bunch of [10:12] different uh domains that kind of can be [10:15] mathematically modeled. Um, right. Um, I [10:19] mean, I'm I'm in general just um [10:24] I think maybe this podcast is just a [10:26] good opportunity to to kind of like [10:27] share a little bit more of an inside [10:29] perspective. I was just astounded by the [10:31] progress. I I think so sometimes I see [10:35] those headlines where it where people [10:38] say that like oh um the economic kind of [10:42] impact of AI is only like 3% or 5% right [10:46] and those headlines are often [10:47] accompanied by by comments like well so [10:50] AI is slowing down or or or you know [10:53] like like like like people are [10:55] overhyping AI so much and it's only like [10:57] 3% so what's up with that right and uh [11:00] and when I you know when I see headlines [11:02] like this. I I remember to like maybe 10 [11:05] years ago I was working on natural [11:08] language processing uh in in in with [11:14] deep learning and back then it just [11:16] didn't really work. Like I remember [11:19] Jakob once came to test like one of the [11:21] technologies we were working on and that [11:22] was like trying to detect sentiment of [11:25] sentences and he was trying this movie [11:27] is bad correctly classified as negative [11:30] this movie is good correctly classified [11:33] as positive and then he would say this [11:34] movie is not bad and the model is like [11:37] oh negative. [11:38] >> Yeah. Yeah. [11:39] >> Right. So that was 10 years ago. Right. [11:40] And since then like okay like like we we [11:42] slowly got like uh uh we slowly started [11:46] solving tasks like this solving tasks [11:48] like decide is this word a noun a verb [11:51] that was like sentiment neuron then then [11:53] we then we had GPT1 GPT2 started [11:57] producing like a paragraph of text that [12:00] made sense right that was such a [12:02] breakthrough right now it feels so [12:04] simple but back then it was such a [12:06] breakthrough then we had like GPD3 GPD4 [12:09] GP4 was like to me like kind of like [12:13] let's say my personal AGI moment uh [12:16] >> because it would sometimes say things [12:17] that surprised me and I was like can [12:20] this model actually surprise me right [12:23] it's still back then like charge GPT for [12:25] my personal [12:27] use kind of felt a little bit more like [12:30] a nuance and kind of like maybe slightly [12:32] better Google but like like what's the [12:35] big deal and then like suddenly we get [12:37] to deep research and and this can [12:39] actually like answer questions to really [12:41] like rarely make things up. That that [12:43] felt useful. And then finally now we [12:45] have like models that can like compete [12:47] in programming competitions which was [12:48] like a you know like very hard earned [12:51] for me personally and even more so for [12:54] obviously uh and u yeah the the the pace [12:58] of progress just like from the [13:00] perspective of somebody working on this [13:01] technology is like absolutely amazing. [13:03] So, so when you see that 3% like I [13:06] raised you like 10 years ago if you had [13:07] to quantify it, it would probably be [13:09] like 0.00001% [13:12] or something, right? So, like really [13:14] like really I think I think those [13:15] numbers need to be put in perspective, [13:17] right? And and and there is no reason [13:18] not to believe that like in a year it [13:20] will be 10%, in two years it will be 20 [13:23] and so on so so far. Yeah, I've heard it [13:25] said that if you looked at like a graph [13:26] of the economy from let's say like you [13:29] know uh worldwide web you know early 90s [13:32] forward and he said point to the [13:34] internet happening to the economy you [13:36] can't find the point there's no point [13:38] you go oh okay Tim Berners Lee announced [13:40] this whatever and I think AI is a lot [13:42] like that where people go oh we've only [13:44] measured this one our measures are hard [13:45] it's hard to know that you know one [13:47] who's using it how they're using it and [13:49] you brought up a very good point too [13:50] about if you've been following it for a [13:52] while I remember training like a very [13:53] simple next character predictor on my [13:55] computer and it was terrible, right? [13:57] One, I'm using a small computer, but [13:58] even then and then you got, you know, [14:00] the sentiment analysis, you're playing [14:01] with bird and it's kind. But then GPT2 [14:04] comes out and I read every single output [14:06] on GitHub. Every single output GPT2 came [14:09] out because I'm like, there is something [14:10] going on with this. And that's how I [14:13] ended up working at OpenAI was because I [14:15] was this obsessive person about that. [14:16] Then with access to GPD3 kept saying, [14:18] oh, this is really this path that's [14:20] moving forward. But it's kind of crazy [14:22] now because like if six weeks go by and [14:24] a benchmark hasn't been broken, people [14:26] are like, "Oh, we hit the wall. We hit [14:27] the wall." And I would say part of the [14:30] problem though is that benchmarks in [14:32] some ways feel like you'll see modest [14:35] improvement on them. I've heard some of [14:36] the benchmarks have problems. Some of [14:37] them actually have wrong answers and [14:40] it's impossible to get 100% if you [14:41] answer them correctly. But also, we talk [14:43] about the term internally. I've heard [14:45] people talk about this as, you know, [14:47] saturation. Do you want to talk about [14:49] that? Yeah, I think I think there's a [14:52] there's a few issues that that that that [14:55] we're hitting with benchmarks right now. [14:57] Um yeah, I mean a pretty clear one is [15:00] saturation and that is just the models [15:01] genuinely reaching a point where you [15:03] know for the kind of standardized forms [15:05] of measuring um intelligence or ability [15:08] like they are uh at human level for a [15:11] lot of them uh [15:14] you know if you're kind of like able to [15:17] to you know perform in and in in in [15:21] amongst the top on this like very hard [15:24] high school competitions where we have [15:26] um you know the the the best competitors [15:28] from around the world um [15:32] it just becomes quite hard to like have [15:33] this like like very uh very constrained [15:37] uh measurement. You know previously when [15:39] we were looking at you know this like [15:42] yeah GPT1 GPT2 GP3 GP4 scaling paradigm [15:46] um you know the benchmarks were really [15:48] very they were really just like [15:50] measuring the rising of the tide. Um I [15:53] think now um you know the field has [15:56] developed a lot of uh more data [15:59] efficient ways to train for specific [16:00] abilities right doesn't mean you know [16:02] train on these benchmarks but you can [16:03] train models that are like [16:04] disproportionately good at math compared [16:06] to their ability um you know to write [16:09] for example right and and so they will [16:11] do better on math benchmarks but it's no [16:12] longer as representative of their of [16:14] their of their overall intelligence uh [16:16] in other topics [16:18] um I think you know these two issues [16:21] combined [16:22] Yeah, I think I think we really have to [16:23] think about the reward utility of these [16:26] models and especially like their ability [16:28] to uh discover new insights. [16:30] >> Yeah, I guess that's a thing that sort [16:33] of kind of gets overlooked is that you [16:35] can build a model that's a really good [16:36] test taker, but that model may not [16:38] really be that useful for work. Ideally, [16:40] your model should score well on tests, [16:42] but just because a model got these [16:43] scores doesn't mean you're going to find [16:44] it personally useful. And I certainly [16:46] think that's a challenge we're at now [16:48] where when people say is a model good or [16:51] bad. It's kind of like saying, you know, [16:55] you're trying to create a blanket [16:56] assessment when there's 100 different [16:58] use cases for it. You know, is a model [17:00] good or bad? Maybe it's great at [17:02] creative writing, maybe it's bad at [17:03] math, maybe it's great at math and bad [17:05] at creative writing. And that becomes a [17:06] really big challenge. And we've talked [17:08] about this with like one for math, [17:10] international math olympiad and these [17:12] kinds of metrics. Why are they [17:14] important? Why is it important to put it [17:16] into these sort of human level [17:17] competitions? [17:18] >> I think the reason we've been excited [17:19] about these competitions like the [17:21] international math olympat [17:23] information utad is that um they are [17:28] a pretty interesting example of like a [17:31] test that is constrained doesn't require [17:34] that much knowledge but really tests [17:36] your ability to to think about a problem [17:39] hard for you know an hour or two or [17:41] three. Um [17:44] and you know and we have like a very [17:46] kind of good uh we have very good [17:48] evidence like these problems are hard. [17:50] Uh you know there's a lot of people that [17:52] try to solve them and compete at solving [17:53] them and it matters to them. Um, [17:57] so yeah, so I think this is and you know [18:00] like for for models that like you know [18:02] excelled at like kind of knowing a lot [18:05] of things but not necessarily uh you [18:07] know thinking very hard in the past that [18:09] that really seemed like the kind of the [18:10] the right milestone to be working [18:12] towards. [18:13] >> Now I'm so I understand it the model [18:16] that scored gold medal level on that [18:19] wasn't using like a calculator. It [18:22] wasn't using other tools. It wasn't [18:23] using some of the frameworks. was doing [18:25] it purely through reasoning. [18:26] >> Yeah, that's right. For for the for the [18:28] International Math Olympiad, uh yeah, [18:31] the model was not using uh other tools [18:33] like uh Yeah. [18:35] >> And and and again, and it was like two [18:36] years ago, you asked it to multiply two [18:38] four-digit numbers, it would fail. [18:40] >> Yeah. But but definitely like, you know, [18:41] for for this kind of contest, it really [18:43] like [18:44] >> it is of course like in a limited domain [18:46] of of math, but it really is about like [18:49] fairly creative thinking, not about [18:50] applying a formula. [18:52] I guess that's part of the challenge [18:54] though is that once you start moving [18:56] outside of math, it gets to be harder. [18:57] You can start to come up with things [18:59] like humanity's last exam, which I think [19:00] is a pretty neat test, but you find that [19:03] certain models after they learn a [19:04] certain kind of tool use kind of figure [19:06] out maybe sort of how to solve these [19:08] problems better. And I would wonder what [19:11] kind of benchmarks are we going to need? [19:13] You know, what are what are you looking [19:14] at to say, okay, this is this is how I [19:16] can kind of get an objective measure of [19:18] a capability. One thing that surprised [19:21] me in the past, I was talking to one one [19:23] of our uh co-workers here uh Anna [19:26] Makandu and I was uh telling her about [19:30] uh IMO. I was excited about like some [19:33] progress and she's like what's [19:36] uh and that that that kind of like was [19:39] like a very in front of the moment for [19:41] me because I I do realize that like some [19:43] of those benchmark we kind of live in a [19:45] bubble a little bit. Mhm. [19:47] >> For me that competition [19:49] feels important and especially like the [19:51] the computer science counterpart III [19:53] because it was a big part of my life and [19:55] so is true for many co-workers here. But [19:57] actually like for an average person like [20:00] working in other field or maybe not as [20:02] interested in mathematics or or computer [20:05] science, maybe they're interested in [20:06] history or something that the [20:08] >> Lana speaks like five languages too. So [20:10] I could see for her a different metric [20:11] based on that would be interesting. [20:13] >> Yeah. So, so, so I think like one one [20:15] thing that like um that uh it's not a [20:19] perfect metric but at least helps keep [20:21] us honest and keep helps keep us escape [20:24] the uh the bubble is uh it's just charg [20:27] us right because everybody uses charg [20:29] and they use it for all sorts of use [20:31] cases. Uh, and obviously there's like a [20:33] lot of pitfalls to using that as a [20:35] metric, but at least it avoids that [20:36] partic particular problem where like [20:38] there are just some things that I'm more [20:40] familiar with and other people might uh [20:43] appreciate other things and this gives [20:44] you like a very wide coverage. [20:46] >> Yeah. And in there too, you have subsets [20:48] of users, people who are building GPTs [20:50] and doing more complicated stuff. You [20:53] mentioned before too the fact that the [20:55] model will reason longer and that seems [20:57] like a very interesting way to evaluate [20:59] capabilities. [21:00] >> Yeah. Yeah. And I think this is also [21:02] maybe like one [21:04] um [21:06] you know challenge with focusing on only [21:08] kind of usage of like chat GPT and broad [21:10] adoption of AI as as the metric of [21:13] progress. Like I think uh this hasn't [21:17] really happened to a very meaningful [21:18] extent yet but I think it will start [21:20] happening pretty soon. we should be able [21:22] to use vastly more compute um than you [21:27] know [21:28] a a a user would normally be willing to [21:31] um um to to to to buy for themselves to [21:36] produce you know technology artifacts [21:38] that are useful to a lot of people. Um [21:42] and I I think I think that that for me [21:45] will be a very important measure of [21:46] progress. Which of these wins were the [21:49] most surprising to you? [21:51] >> I think we definitely kind of [21:52] anticipated getting to this point when [21:54] we saw the reasoning models starting to [21:56] work. [21:56] >> Um, at the same time, I think this like [21:59] recent uh set of things is very [22:01] impressive. I think maybe out of those [22:03] uh um I think I think IMO came a little [22:08] bit sooner than I expected. IMO gold [22:10] again like I think IMO problem six will [22:12] will still uh [22:15] IMO has like all the problems require [22:17] like creative thought and some [22:20] uh new insights I think but typically [22:23] like you know there's this proveral [22:24] problem six that is like requires very [22:26] out of the box thinking uh and you know [22:29] it's really kind of like usually like [22:30] outside the kind of typical domains of [22:33] the other problems um and you know so in [22:36] the past we were actually kind of [22:36] drawing a boundary between like getting [22:38] a goal you know like solving these other [22:39] problems and like actually consider [22:40] solving all the problems and in [22:42] particular problem six. So it was pretty [22:44] it was pretty pretty uh you know [22:45] hilarious in some way to see ourselves [22:48] and also Google deeper at the same time [22:49] like oh yeah we solved problems one [22:51] through five perfectly and we didn't [22:52] make any problems on problem problem [22:53] six. I think that kind of makes that [22:55] challenge pretty clear. [22:56] >> Yeah that was I think that was [22:57] interesting is that yeah that I think [23:00] that the the open model said like yeah I [23:02] don't think I can solve this. It didn't [23:03] even try or said that it had a problem [23:05] with that. Was that correct? Yeah, the [23:07] model was able to correctly identify [23:10] that didn't make progress on the [23:11] problem. That's pretty fascinating to [23:14] think about that that the model is able [23:15] to sort of determine that because uh you [23:18] know there's a lot of conversations [23:20] about people talk about hallucination [23:21] which I think it's a kind of a poorly [23:23] understood thing and there's a [23:24] difference between fluid and crystalline [23:25] thinking and you know one is how much [23:27] knowledge a model has and the other is [23:29] its problem solving capability and when [23:30] you get to the point where it's able to [23:31] do that it's able to say but hey no I [23:33] think you know I won't be able to answer [23:35] this that's pretty interesting sort of [23:37] point to get to um I've been told to ask [23:40] this question about a a live stream [23:43] in Japan. [23:44] >> Oh, so so so I think in the in the past [23:46] few weeks actually like our models have [23:48] performed incredibly well in three [23:49] competitions. Um so we talked about two [23:52] of them which is III and IMO. Um there [23:55] is also this competition for that is [23:57] open to everyone not just not not just [23:58] high schoolers um called um adcer. It's [24:02] a it's a very prestigious very [24:05] >> uh um [24:08] um highquality uh competition organized [24:11] in Japan uh but open to open to [24:14] competitors worldwide and in this [24:16] particular contest it was about um kind [24:19] of longer horizon um horistic problems [24:22] where you're you're given only a single [24:24] problem you have 10 hours to solve it [24:27] and so you have competitors uh racing to [24:30] figure out like the best approach [24:32] approach to to up to to to this like [24:34] difficult optimization problem. Um so [24:36] it's a bit different because there isn't [24:38] like a single correct solution. There [24:40] isn't like a single pattern to follow. [24:41] Like these tasks are like extremely [24:43] diverse and you you can focus on the [24:45] single task for for for 10 hours. And so [24:48] we we entered our [24:51] model into this contest. And um you to [24:55] me this this had a little bit of a [24:57] personal significance. Um I used to be a [25:00] kind of very [25:02] uh engaged competitor in the past in [25:04] this like more short form like closed [25:06] form contest like III. Um and um my [25:11] friend uh Siho um who also works at the [25:15] time um excelled at this like long [25:19] duration contest and when we work [25:22] together he would mock me a little bit [25:23] that you know my sort of contest would [25:25] be automated uh long before his um [25:30] because you know they are they're kind [25:31] of like longer duration require uh um [25:34] require require kind of more focused [25:36] work Um, and turns out like in this in [25:39] this in this contest in Japan, Sihaho [25:41] was actually like one of the top [25:42] contenders. Um, and so I was watching [25:45] this live stream watching our model kind [25:47] of race with SIHO throughout the [25:49] competition. Um, in the end, um, our [25:52] model actually got second place and Siho [25:55] won. So, you know, he alone stood in the [25:58] way of of his uh of of, you know, his [26:00] prediction not coming true. [26:02] >> Still two wins for opening eye. I think [26:05] one one thing that also stood out to me [26:07] is like Sai at the end of the [26:09] competition he was like really retired [26:11] and they interviewed him a little bit to [26:12] to to talk about his experience like in [26:15] the middle of the competition and um I [26:18] don't think I can quote him directly on [26:20] this podcast but he's like your models [26:23] are very very bad. I want to go to [26:26] sleep. I am tired. [26:29] Yeah, we we've heard talks about like [26:32] the wall. We mentioned that before and I [26:35] think it was interesting because [26:37] reasoning kind of came out of nowhere. I [26:39] mean there were you know hints of stuff [26:41] some papers and stuff but people really [26:42] hadn't kind of drawn the line. Then all [26:44] of a sudden the 01 model comes out and [26:46] the whole idea that you can not just [26:47] have a model give answers you can let [26:48] the model kind of have an inner [26:50] monologue talk to itself and solve [26:52] things through. [26:54] Do we think that's enough to take us to [26:57] AGI or there other breakthroughs needed [26:59] or there other breakthroughs you think [27:00] are going to happen? I I just need to [27:02] point out that like the team here worked [27:04] extremely hard on on this particular [27:07] thing. I need it feels like something [27:10] simple like just need longer chain of [27:12] thought but like actually to made it [27:14] work was really hard earned and um and I [27:18] do think like back to your previous [27:20] question of like what was the surprising [27:21] result when when we first noticed that [27:24] it's working or we first noticed that [27:26] that that we can train those models and [27:28] give them more data and they get better. [27:30] That was like I think one of the most [27:33] kind of shocking moments. The moments [27:36] where we are like um we started asking [27:41] uh uh like very very seriously the [27:43] question like are we ready as an [27:46] organization for for incredibly [27:48] fast-paced progress? Like I I remember [27:50] there was like one particular evening [27:52] like I think 11:00 p.m. like there was [27:54] like I think we were on the line with [27:56] with Sam and Mera and just kind of [27:58] trying to I think we got a little bit [28:01] freaked out by those results sometimes. [28:03] Sometimes that happens. Uh [28:06] >> the the pace is fast. I mean it is a [28:10] fast thing and like I said that you know [28:11] the joke is people nothing happens for 6 [28:13] weeks they think it slowed down but then [28:15] if you look year over year it is. And uh [28:18] I it's a fair point cuz yeah you you [28:20] have things that you're aware of [28:21] internally when you work on something [28:23] for a couple years and there hey there's [28:24] a research paper but it's like yeah it's [28:25] not like it came out last night. It was [28:27] like there's a lot of work onto it. But [28:28] I'd say to the world was sort of [28:30] surprised by the fact that there is this [28:33] really fundamental new way to sort of [28:34] make these models do even more to take [28:37] take kind of the existing sort of [28:38] infrastructure so to speak and get a lot [28:40] more capability out of it. Where do you [28:43] think the next breakthroughs are going [28:44] to happen? I think one thing we always [28:46] try to not underestimate is the [28:49] importance of scaling. Um I think you [28:52] know even as we look at these wiz models [28:56] you know like it's not like you know the [28:58] previous scaling paradigm of [29:00] pre-training uh has vanished right like [29:03] I think we will see these things [29:05] compound [29:06] >> and I think there's also like new [29:09] uh new directions that that we can move [29:12] in in particular um we were talking [29:15] about you know extending the horizon [29:18] that these models [29:20] can uh you know plan for and and and and [29:23] reason it. And I think if you look at it [29:25] from the perspective of just like [29:27] compute spend um you know we say okay [29:31] yeah we went from GPT GPT4 was doing [29:34] some out of compute for for for every [29:36] answer to like you know GPT5 pro which [29:39] maybe uses I don't know like uh 10x 20x [29:43] I don't know like some some you know [29:45] non-trivial but in some ways not not [29:48] that impressive amount of compute more [29:50] right and can produce much better [29:51] answers. Uh I think on the scale of like [29:55] what amount of compute would you would [29:58] you be willing to spend on a problem [30:00] that actually matters to a lot of people [30:01] right on like progress on a on a on a [30:04] medical research question uh you know [30:06] progress on developing the next [30:08] generation of models right like like [30:11] these are like incomparably larger [30:13] amounts and so I think I think that is [30:15] that question of like model [30:17] persistence and ability to work for a [30:19] very long time on focus problem is a [30:21] pretty clear next [30:24] How would you [30:27] put the practical implications of AGI to [30:31] sort of like if you were talking to a [30:33] typical chat GPT user or something like [30:35] what would be the what would their [30:36] experience be like in a few years from [30:38] now or five years from now which sounds [30:41] far away but it's really not because [30:42] which is 5 years ago GPD3 came out and [30:44] that feels like a blur. What would an [30:46] AGI like model be capable of? So I I was [30:49] I was talking about automating research. [30:52] Um you know my picture of of [30:55] how that would actually look like is you [30:57] know imagine a a company of very capable [31:00] researchers and engineers that is [31:02] largely automated right and and um now [31:06] again like that is I think that is [31:08] something that like you know will [31:09] interface with the world in all sorts of [31:11] ways. It won't be just like kind of a [31:13] black box like it will you know talk to [31:15] people. It will kind of like take in [31:17] inputs. it will run experiments. But I [31:19] think like you know having having this [31:22] sort of [31:24] potential for developing new technology [31:27] and you know and other kind of artifacts [31:30] you know uh um code bases designs [31:35] um [31:37] I think I think can like radically [31:40] accelerate uh the pace of technical [31:42] progress. Um so I think that is [31:45] something that like we will feel and you [31:48] know and we need to to do a lot of work [31:50] to to get it right you know from a [31:52] technical and from societal [31:53] perspectives. Um but I think that is [31:56] kind of where I turn to I think we we [31:57] should also expect like a lot of [31:59] progress on the on the actual kind of [32:02] you know um interfaces that we interact [32:06] with. [32:07] >> Mhm. [32:07] >> You know we see like ch can can feel [32:10] quite humanlike. we can we can form [32:12] attachments with it. I think you know as [32:14] it becomes more persistent as it becomes [32:17] kind of capable of expressing itself in [32:19] like different forms and texts right [32:21] like I think that those effects will [32:23] become stronger and again like that will [32:25] be something uh I think will become a a [32:28] very big and important conversation. [32:30] >> I just got access in chat GPT to have it [32:34] actually read my calendar in Gmail and [32:37] and I realize like how far we've come [32:38] because I'm excited about that now. I'm [32:40] not really terrified that it's going to [32:41] start writing like, you know, Ewok [32:42] fanfiction to somebody, you know, and I [32:44] think that's sort of this neat threshold [32:45] that we sort of cross this sort of the [32:47] level of trust. I think there's [32:49] definitely like like we are in a place [32:50] where there's like very tough trade-off [32:52] where like there's like such clear just [32:55] economic you know personal value you can [32:56] extract out of having the model um you [32:58] know have access to a lot of your [33:02] um [33:04] all of your data. At the same time, I [33:06] think like [33:08] you know we are not at like the [33:10] threshold of like robustness where like [33:12] you know like we can fully trust these [33:14] models to not be exploited by someone [33:16] trying to exploit them. [33:17] >> Mhm. [33:17] >> Um yeah it's it's definitely like a big [33:20] problem I think you know we as a field [33:22] will have to iterate on. What would you [33:24] tell [33:26] two versions of you guys today in high [33:28] school? What would you do if you're [33:29] visiting your old classroom? What would [33:30] you say right now? Tell them about the [33:32] future. What advice would you give? [33:33] invest in Bitcoin. [33:35] >> No, I mean today today even today in [33:38] 2025, what would you tell a high school [33:40] student? [33:41] >> High school students today. Oh, yeah. [33:42] Yeah, that that one is also I think a [33:45] great question, right? Because I I I [33:47] hear a lot of uh kind of would I [33:50] consider misinformation on that online? [33:52] So, you should absolutely learn to code. [33:56] like one skill that that is at premium [33:58] and will continue being at premium is to [34:00] have like um have [34:03] >> like really structured intellect that [34:05] can like break complicated problems into [34:07] pieces and you know like that might not [34:09] be programming in the future but [34:10] programming is a fine way to acquire [34:12] that skill. So are other other kind of [34:15] uh domains where you need to think a [34:17] lot. So so don't don't let people tell [34:20] you that uh you should not learn to [34:21] code. Yeah, I I learned to code late in [34:23] life and that's actually I ended up [34:25] working at OpenAI as an engineer and and [34:27] I try to explain to people just because [34:30] a system can do the thing doesn't mean [34:31] you don't want to know how it works [34:33] anymore. And as you said, when you [34:35] understand how to break down a task when [34:37] I worked at OpenAI and prompt [34:38] engineering, my coding understanding [34:40] helped me understand to take both [34:42] language and break it down and make it [34:43] do better things. I think that people [34:45] who bridge those gaps are really an [34:47] advantage. And so whenever I hear people [34:48] say like don't learn to code, it's like [34:50] do I want an airplane pilot who doesn't [34:52] understand aerodynamics? Like this [34:54] doesn't make much sense to me. Well, you [34:56] know, thinking about how I thought about [34:58] things in high school, I think it's like [34:59] pretty [35:01] incredible like how many kind of [35:04] perceived constraints are not actually [35:07] there when you really think about it. [35:08] You know, maybe maybe like the first [35:10] revelation for me was like, hey, you [35:12] know, if I really kind of like am [35:13] passionate about this computer science [35:15] stuff, like it is I can actually spend a [35:17] bit more time on it at the cost of, you [35:19] know, maybe spending a bit of a bit less [35:21] time on like, you know, like other 12 [35:24] subjects in school. Um but you know but [35:27] then like you know but then like somehow [35:29] it like again like took kind of like a u [35:32] it was like a big revelation to me that [35:34] like actually you know hey I can I can I [35:36] can I can um go and you know study in [35:39] the USA at some point and like you know [35:41] that's not really something that like [35:43] seems [35:44] uh that's like obviously interaction [35:46] space and you know and obviously kind of [35:47] like you know like spending some time [35:51] here in uh Silicon Valley right and kind [35:53] of seeing like how [35:56] you know people are willing to really [35:57] like [36:00] attack these big problems with ambition [36:02] and like the kind of [36:04] um [36:06] and the belief that like you can [36:08] actually you you can actually um make a [36:11] meaningful [36:12] uh positive change in the world. Uh [36:16] yeah, I think I think has been [36:17] incredibly inspiring and uh yeah, it's [36:20] something I I I I I I I cherish about [36:23] about about this community. [36:25] >> Is there a book or something that like [36:27] inspired you? [36:28] >> I think there's a there's a couple [36:29] books. Um I remember my it's actually [36:32] Yeah, it's actually very hilarious like [36:34] thinking about it now. Um, I didn't [36:36] really connect the dots, but uh, my dad [36:37] gave me this book once um, when I was [36:39] like in a pretty uh, [36:42] um, [36:44] I think I was like 15. I was like pretty [36:46] unsure what I want to do. Um, yeah, it [36:49] was um, a Polish Polish version of a [36:51] book by like some other I didn't know. [36:53] Uh, called Hackers and Painters. Uh, [36:57] yeah. So, uh, yeah, it was actually a [36:59] Paul Graham. Uh so so I guess like uh uh [37:03] yeah again like kind of like this [37:05] community. Um [37:09] so I I found that pretty pretty [37:11] inspiring. [37:12] >> Yeah, there's something helpful I think [37:14] that hearing the message of like no it's [37:16] okay to dream big and go do stuff that [37:19] you can just make things happen in the [37:20] world. And I think that the more people [37:23] realize that kind of the better the [37:25] world gets to be. Was there any book [37:27] that influenced you or movie TV show? [37:30] Oh, movie. [37:32] I have a stupid answer to that question. [37:34] Kind of stupid bad after the profound [37:36] one that but like Okay. So, I watched [37:38] Iron Man. [37:38] >> Yeah. [37:40] >> And it inspired me to start a PhD in [37:42] robotics. [37:44] >> That's a great answer though. Like, you [37:46] know, the the Martian by Andy Weir. I [37:47] met a scientist at NASA who was a [37:49] botonist who read that book and I'm [37:51] like, "Well, they got the atmospheric [37:53] physics wrong and all this." He's like, [37:54] "Well, that's why I'm here." I'm like, [37:55] "Oh, [37:56] >> well, yeah. [37:57] >> Yeah. I guess I didn't get to the stupid [37:58] part. The stupid part was like when I [38:01] started working on robotics, I was very [38:02] disappointed how bad those robots are. I [38:05] somehow didn't occur to me that like [38:07] maybe the movie is a movie. Uh [38:11] yeah, so that whole experience was kind [38:14] of bad for me if not for would be kind [38:16] of bad for me if not for the fact that [38:18] this is there. I like met a friend who [38:20] was into deep learning and at the time I [38:23] thought all of the machine learning kind [38:25] of is a hype but it was an interesting [38:28] systems problem and then out of out of [38:31] nowhere as I'm sure like uh you know [38:34] like I would frustrate some deep mind [38:35] folks by saying that Alph Go came out uh [38:39] I'm sure it wasn't out of nowhere I'm [38:40] sure it was years in the making uh and [38:42] and that like was very inspiring I [38:44] actually think to both of us uh and and [38:47] since then it was just hard not to Yeah, [38:49] took me a while to become [38:52] um convinced that declaring is more than [38:54] a than a fad. Uh because you know like [38:57] we don't really understand the kind of [38:58] underlying optimization and you know I [39:00] think this kind of has been the story of [39:01] our research here making trying to make [39:04] progress on on on these questions like [39:05] about how it really works but it really [39:07] is like setting a physical phenomenon in [39:09] some way and you know to a classically [39:10] trained computer scientist that was a [39:12] that was a weird thing to accept. [39:14] >> Mhm. I do remember when Jakob was [39:16] telling me about like scaling up [39:18] principled convex optimization. [39:22] >> That was before Alpha Go. [39:23] >> Yeah. Yeah. And Alph Go was interesting [39:25] because first like oh cool it solved go [39:27] and then we're like yeah but it just [39:29] learned by watching all these. Then they [39:30] did Alph Go Zero where it selftaught and [39:32] you're like okay game over folks. Like [39:34] there's a trajectory here. And I think [39:36] that's continued on. But I think that [39:39] yeah, if you hadn't watched Iron Man, [39:40] maybe Thor instead, you know, maybe [39:42] things would have turned out better. [39:45] >> Who knows? [39:46] >> No, I [39:46] >> I kind of wish I studied maths instead [39:50] >> has been more useful. [39:50] >> Study what? [39:51] >> Maths. [39:52] >> Maths. [39:52] >> Mathematics or theoretical computer [39:54] science? Either either of those like [39:56] this. Yeah, [39:57] >> physics probably. Is that [39:58] >> Oh, physics. [40:00] >> I started off as a magician. I don't [40:01] know if you know that. So, I actually [40:03] had my own reality TV show. And so you [40:05] find a very strange path to end up here. [40:08] So uh Yakob Simone, it's been an [40:11] absolute pleasure to talk to you both [40:12] and I hope we can meet again and talk [40:14] about the next big breakthrough that you [40:15] guys have been secretly working on [40:17] that's going to come out of nowhere and [40:18] we'll be like that was an overnight [40:19] thing. [40:20] >> Thanks Andrew. [40:21] >> Thank you.