AI Crisis in Math: Navier-Stokes, Riemann Hypothesis & Complexity Theory | Lex Fridman Podcast #503

AI引发数学危机：纳维-斯托克斯方程、黎曼猜想与计算复杂性理论 Lex Fridman邀请数学与理论计算机科学学者Steven Brunton、Ankur Moitra和Alex Kontorovich举行紧急研讨会，共同探讨AI声称攻克纳维-斯托克斯方程、拟黎曼猜想及独特博弈猜想等顶级未解难题对学界带来的冲击。与会学者深入讨论了这些突破对学术界及青年学者培养造成的现实震荡、AI生成证明的内在逻辑与本质，以及在自动化机器推理日益普及的时代，人类在数学审美、问题提出与深刻理解方面依然保有的核心价值。 Lex Fridman召集多位学者举行紧急圆桌会议，探讨AI在纳维-斯托克斯方程及独特博弈猜想等经典数学难题上带来的突发性学术震动。 学者们担忧AI在攻克重大猜想上的迅猛进展会颠覆传统学术导师机制，进而打乱青年学者与研究生的培养路径。 学者强调单纯由AI完成的形式化证明并不能等同于人类认知层面的理解，提出问题、构建理论以及数学审美依然是人类不可替代的核心价值。

Lex Fridman hosts mathematicians Steven Brunton, Ankur Moitra, and Alex Kontorovich to evaluate how recent AI-generated solutions and formal verifications for landmark problems—including the quasi-Riemann hypothesis, Fermat's Last Theorem, and the Navier-Stokes equations—are transforming the practice and pedagogy of mathematics. The panel argues that while automated systems can rapidly assemble long chains of logical deductions, the fundamental human role must transition from technical, clerical proof generation to problem selection, structural modeling, and shared conceptual inquiry. Automated Verification and Sudden Milestone Drops (73:58) Recent releases by AI labs have achieved sudden, unexpected progress on benchmark problems long thought to require decades of human effort. Alex Kontorovich notes that Anthropic produced a 13-million-line, dependency-free Lean formalization of Fermat's Last Theorem directly from foundational axioms just days after he publicly estimated such an undertaking would take five years of human labor. Simultaneously, automated releases reported solutions or counterexamples across hundreds of problems, including zero-free regions for Dirichlet L-functions in the quasi-Riemann hypothesis, the Unique Games Conjecture, and finite-time blowup configurations for 3D Navier-Stokes under smooth external forcing. Kontorovich concludes that autoformalization—obtaining verified certificates for results previously written on paper by humans—is functionally solved in terms of feasibility. However, the panel stresses that these machine-generated proofs remain brute-forced, modularly crude, and far below the stylistic and organizational standards established by human libraries like Mathlib. Semigroup Actions and the Black-Box Dilemma (33:13) Kontorovich rejects claims that AI is restricted to the "convex hull" of human mathematics—an interpolation limited to linear recombinations of existing work. Citing Fields Medalist Tim Gowers, he explains that AI systems instead operate within the *semigroup* generated by known moves. In the quasi-Riemann hypothesis resolution, the AI linked established techniques into unprecedented, cascading chains of deduction that human intuition had missed. Kontorovich introduces the "solved axiom" thought experiment, proposing a future where any problem resolvable through human-AI interaction can eventually be solved by autonomous AI alone. Ankur Moitra cautions against anthropomorphizing these models, arguing that superhuman benchmark performance frequently relies on mechanisms unlike human understanding. In high-dimensional vector spaces, phenomena like "subliminal learning" allow unintended biases to coordinate without explicit prompts. Moitra and Kontorovich emphasize that when an AI generates a verified but incomprehensible proof, mathematicians face the challenge of "unformalizing" it—systematically probing special cases to extract interpretable abstractions rather than treating mathematics merely as an unexamined accumulation of correct theorems. Institutional Shocks and Graduate Training Crises (25:05) The flood of machine-generated proofs destabilizes long-standing academic career pipelines. Referencing a public warning by Terence Tao, the panel notes that casual automated releases disrupt years of deliberate research conducted by graduate students and postdocs. Moitra explains that the traditional graduate roadmap—mastering foundational methods, identifying a recognized open problem, and publishing an incremental benchmark improvement—faces sudden obsolescence as automated harnesses scrape open problems and flood arXiv. Academic evaluation and publication models are struggling to adapt. Kontorovich compares the current strain to the multi-year consensus review required for Grigori Perelman's proof of the Poincaré conjecture, observing that mathematical epistemology is inherently slow and meticulous. When machine outputs inundate journals and repositories, referee bandwidth collapses, generating cognitive exhaustion without fostering authentic understanding across the research community. Risk Verification and Refocusing on Problem Formulation (182:19) Steven Brunton outlines a framework to evaluate AI deployment across two axes: verifiability versus stakes. Pure mathematics and aerodynamics reside in an ideal low-stakes, high-verifiability quadrant, where interactive theorem provers and physical wind tunnels swiftly catch erroneous proposals without compromising safety. Brunton warns, however, that intellectual complacency must not erode the strict institutional firewalls that govern life-critical domains like medicine and aviation. As mechanical theorem-proving becomes commoditized, Moitra and Brunton contend that mathematical insight will focus on deciding "where to dig"—framing effective definitions, simplifying complex systems into tractable models, and identifying core conceptual barriers. Moitra advises early-career researchers to replace external metrics and legacy academic playbooks with intrinsic curiosity and purpose. Brunton and Fridman conclude that even if computational tools outpace humans at raw deduction, the enduring essence of science resides in human connection, collaboration, and the narrative struggle of discovery. While automated reasoning has disrupted assumptions regarding the timeline for solving and formalizing difficult mathematics, the panel leaves unresolved how institutions will adapt hiring, tenure, and refereeing when incremental technical contributions can be mass-produced without human comprehension.

Lex Fridman 与数学家 Alex Kontorovich、Ankur Moitra 及工程学者 Steven Brunton 展开圆桌研讨，分析近期人工智能在准黎曼猜想、纳维-斯托克斯方程反例及费马大定理形式化验证等前沿难题上取得的冲击性突破。学者们指出，随着机器批量生成严密证明，传统学术培养路径与同行评议体系正面临重塑，数学的核心价值正在从机械式的推导证明转向对概念的理解、提问的品味以及人类之间的学术共同体联结。 机器证明浪潮与自动化形式化的落地 (69:09) 近期 AI 机构接连公布重大数学进展，包括 OpenAI 声称解决准黎曼猜想（quasi-RH）与唯一博弈猜想（Unique Games Conjecture），以及纳维-斯托克斯方程在光滑外力驱动下产生有限时间爆破的反例。Alex Kontorovich 回顾自己在纽约数学博物馆（MoMath）晚宴上得知准黎曼猜想突破时的巨大震撼，该成果不仅覆盖黎曼 zeta 函数，更建立了所有狄利克雷 L 函数的无零点区域。不久后，Anthropic 发布了长达 1300 万行、不含未证依赖（sorry-free）的 Lean 形式化代码，完整验证了费马大定理。Kontorovich 认为，人类在纸面上写出的数学结论实现自动形式化（autoformalization）的技术门槛事实上已经跨越，即便这些机器生成的长篇代码粗糙冗长、缺乏人类所追求的美感与模块化架构。 学术范式危机与年轻学者的生存困境 (5:49) 大规模 AI 证明的涌现对现有学术体制造成了剧烈震荡。陶哲轩曾公开警告，研究生与博士后倾注多年心血推进的基础性问题，正被这类成果发布轻率破坏，令其丧失研究的自主感与探索机会。Ankur Moitra 指出，传统学术晋升依赖一套稳健逻辑：掌握现有技术、选择公认问题并做出渐进式改进；然而当下自动化论文与代码库如雪崩般涌入，导致既有科研蓝图迅速失效。如果任何人都能使用自动化脚本抓取公开难题并在 arXiv 批量灌水，年轻研究人员将面临严峻的职业与身份认同危机。学者们认为，将自我定位单纯局限于“做题家”或过分执着于“首发权”的文化亟待转变，数学研究必须摆脱为了充实简历而盲目产出定理的恶性循环。 认知图景：凸包、半群与黑箱逆形式化 (33:13) 针对 AI 是否仅仅是已有概念“插值”的质疑，Kontorovich 援引菲尔兹奖得主 Tim Gowers 的观点，认为 AI 并非在人类认知的“凸包”（convex hull）内做简单的平庸组合，而是在已知数学规则生成的“半群”（semigroup）内部展开极其深度的层叠运作。在准黎曼猜想中，AI 的单步动作虽均属人类已知范畴，但其构成的惊人级联证明展现出远超人类搜索能力的复杂组合。然而 Moitra 提醒不要对 AI 进行拟人化投射，神经网络的成功机制与人类直觉截然不同。当机器提供了一份经 Lean 严密检验却无人能读懂的长篇黑箱证明时，数学家需要建立一门“逆形式化”（unformalizing）的学问，通过拆解关键特例来逆向工程出背后的核心思想与宏观框架。 数学认知升维与人机协同的分野 (11:41) Kontorovich 提出了一个思想实验式的“求解公理”（solved axiom），即假定未来凡是人机协作能证明的数学问题，AI 单独也能攻克；在该前提下，人类的独特价值将不再体现于推导定理甚至提出底层问题。然而 Moitra 强调，公理系统可导出的庞大陈述集合中，只有极微小的一部分具备人类关切的意义；数学绝非仅仅是不断上涨的定理水位，更在于研究者与难题搏斗、在此过程中提炼出通用抽象概念的思想史诗。此外，Brunton 从风险与可验证性两个维度分析指出，纯数学研究处于“高可验证性、低直接危害”的象限，证明中的逻辑瑕疵能被验证器或同行迅速捕获，这为前沿探索提供了绝佳的人机共生实验场，但也要求科研共同体坚决守住安全关键领域的验证防火墙。 从论文孤岛走向开源协作与内在动机 传统的学术出版模式正显现出疲态。面对海量自动化推导，同行评议体系难以像当年验证佩雷尔曼庞加莱猜想那样，耗费数年进行逐行板书审查。Moitra 与 Kontorovich 指出，数学界应当借鉴 Lean 的 Mathlib 机制，采用类似 GitHub 的开源 Pull Request（PR）模式，促使同行公开讨论、剔除冗余并提炼核心洞见，让论文写作摆脱繁复细碎的技术脚手架。对于身处技术巨变浪潮中的下一代学者，Brunton 与 Moitra 呼吁构建内在评价坐标：在充满不确定性的环境中，应当追求兼具重要性、个人兴趣与自身禀赋的研究课题。随着机器智力逐渐被商品化，面对面的真实交流、跨学科的直觉穿透力以及人与人之间的深度联结，将构成人类学者不可替代的基石。 本场对话展现了数学界在 AI 自动化证明突飞猛进时的震撼与反思：虽然从形式化验证到复杂难题的解决门槛已被显著拉低，但如何在海量机器证明中逆向重构人类理解、守护年轻学者的探索动机，并在算法黑箱与可解释性之间重构学术评价与同行评议机制，仍是科学界悬而未决的制度与哲学命题。

[0:00] - All right, so this is a kind of emergency episode [0:04] I think it's fair to say, about the state of mathematics and artificial [0:07] intelligence. A kind of crisis moment [0:11] in the history of mathematics. Over the last few weeks, [0:15] machines claimed to have solved some of the hardest problems in the [0:19] history of math. Navier-Stokes, the quasi- Riemann [0:23] hypothesis, the unique games conjecture, and hundreds of other results. So [0:29] here with me, we have Steven Brunton, [0:33] who's a mathematician, fluid dynamics expert at University of Washington, [0:38] who can speak to Navier-Stokes, turbulence, all things fluid, [0:42] and many other topics. Also, here's Ankur [0:46] Moitra, a mathematician and theoretical computer scientist at MIT, [0:50] visiting Institute for Advanced Studies in Princeton this year, [0:54] who can speak to many things, unique games, [0:58] conjecture, and complexity theory in general, [1:01] and the mathematics of how the AI learns and [1:05] all things theoretical computer science. And Alex, a [1:09] mathematician, number theorist at Rutgers, who can speak to [1:13] also very many topics, including Riemann hypothesis [1:17] and Lean, the automated proof and the automated [1:20] verifier and checking and so on, and so many other topics in mathematics. [1:25] All three are brilliant. All are very kind to show up this early [1:29] morning and talk to me, assemble this kinda [1:33] multi-agent mini swarm of human brains to try to interpret what happened [1:37] in this very moment. A lot of mathematicians, brilliant mathematicians, [1:41] including Terence Tao and others, are freaking out. It's a mix of awe, [1:48] terror, dread, and few inklings of light, of hope also. So I apologize to start [1:58] with a question about feeling, but what have you been [2:02] feeling about this historic moment? Maybe Alex, do you wanna go first? [2:08] - Yeah, I don't know if I would call it a crisis. There's a lot of opportunity in that crisis. [2:12] It's a chance to rethink what it is we're trying to accomplish, [2:17] and it's the chaos of opportunity that some [2:21] people are reacting negatively to, some people are reacting [2:24] positively to, and sometimes those reactions are happening in the same person. [2:29] And so I think we're entering, if we're not already in, a [2:33] period of some kind of golden age of mathematical knowledge [2:37] after millennia of having a [2:40] candle that we are walking around the woods, and this is as far as we can see. [2:44] Now we have a flashlight. [2:46] - I think we should comment on the magnitude of this moment for people who are not aware. [2:50] So maybe to quote you, something like something [2:54] like, "Holy shit, if a human did this, it'd be an instant Fields Medal." Maybe you didn't say, [2:58] "Holy shit," but basically saying, [3:00] "This is an incredible result." So a lot of people wouldn't have [3:04] expected this amount of progress in this amount of time, so this is truly [3:07] shocking. So you're now speaking as a person who is [3:12] dealing with the feelings you've felt over the last couple of days, having found this out. [3:16] So maybe also it'd be useful to go back to the, [3:20] the feeling you were having when you first saw [3:24] the report that, in this case, OpenAI dropped. [3:27] And the two of you happened to be on camera loosely on the same day, [3:33] maybe a few drinks in because it was a conference or a get-together. [3:37] Maybe, Ankur, can you speak to what you were feeling? [3:40] - It was an interesting night when the results dropped. Now, it was not entirely [3:44] unexpected because for weeks there'd been rumors about the two biggest [3:48] problems in my areas that were in that collection of [3:52] results, so the unique games conjecture and RL equals L. [3:56] There were even rumors about matrix multiplication, so really my first reaction [4:00] was, "Wow, it's all true. Thank goodness it's [4:03] finally out there because now people can actually go through it and process [4:08] what's happened and figure out what to do next." [4:10] Now, as to your earlier question, you know, what are we feeling? Well, for me, [4:14] it changes on a daily basis, but I think the biggest, most [4:18] overarching feeling is really feeling like you're getting pulled in two very [4:22] different directions. My work has always been about asking the right questions, [4:28] and to that extent, these tools are amazing for me because, you know, [4:32] they increase my ability to learn about other fields. It's like [4:36] having a world expert who's there to teach you about things in [4:40] your own language. Your ability to make connections is just magnified. [4:44] But, you know, one of the parts of my job that I really love the most is the mentorship [4:48] aspect. Not just training grad students and postdocs and seeing them do [4:52] great things, but having them come with me on these journeys. So the [4:55] part that pulls me in the other direction is the recognition that, [4:59] you know, the playbook and the blueprint for success that we had for so long is [5:05] now thrown out the window. [5:07] So it becomes a lot harder to advise students and postdocs [5:11] about, you know, what are things they should be working on that are gonna make an [5:14] impact, are gonna make a name for themselves. So I feel very excited, [5:18] but maybe more intellectually lonely than I felt before, [5:23] in part because, you know, a lot of times great results come [5:26] from asking very humble questions, at least for me. [5:31] And now there's so much anxiety and dread and [5:35] feeling that, wow, you would have to do something really momentous to [5:39] stand out in the sea of AI results, that it feels like [5:43] we've lost some of that patience in terms of planting the seeds from which [5:47] the great oak trees grow. [5:49] - Terence Tao, Fields Medal winner, wrote, "I am [5:53] horrified by the many person-years of ongoing patient and [5:56] deliberately slow research efforts, particularly by graduate [6:00] students and postdocs, toward many motivating problems in mathematics [6:04] being casually disrupted or destroyed by such a [6:07] release." Much as one cannot unhear a movie spoiler or a crossword clue, [6:14] one cannot explore a problem as profitably and richly once one is [6:17] aware of an existing solution, and I mourn the path [6:21] not taken and the opportunities lost in the frantic race to develop the [6:25] technology. So I mean, if we're being realistic, it's more [6:30] dramatic for a PhD student and a postdoc than, [6:36] than maybe the way you've articulated. I mean, you're saying it [6:40] quite calmly, but if I was a PhD student right now, I'd be [6:45] in a state of, like, shock. [6:50] Like, my whole life-- I was going through some of this. I know a lot of my friends are programmers. [6:55] They're going through this state of shock in last December when basically [6:59] most of the programming world switched to AI writing most of the [7:02] programs. So I would approximate it to a feeling of kind of a [7:07] loss of meaning, kind of terror, like dark stuff. Um, [7:11] we'll keep exploring that. Uh, Steve, what do you think? What were you-- what are you feeling? [7:15] - I would say most of my feelings are pretty hopeful. [7:19] Um, I think I'm an engineer. I like solving, you know, problems that [7:23] make technology more realistic for humans, artificial hearts and airplanes. And [7:29] our ability to solve real applied problems is going to get much, [7:33] much better. Our scope and reach is gonna increase, and so for that, I'm extremely [7:37] hopeful. But there is a lot of uncertainty, and I think, you know, some [7:42] of this fear of meaning, of losing control, of, [7:47] you know, just being kind of cut out of the process that we all [7:51] love being a part of it is a little scary. And [7:56] I don't think any of us would be surprised at either of the [8:00] kind of two paths that might take place over the next five or 10 years, where [8:04] we really are just dwarfed by you know, [8:09] other intelligence, or it kind of levels off, and this [8:13] becomes an incredible tool that we use for our [8:16] curiosity, for our human problem-solving. [8:19] And that uncertainty is massive. Those are two very different worlds. And so [8:23] uncertainty is difficult for humans. I think it's also [8:27] difficult for those of us who have chosen math. We chose math, you know, [8:30] probably because we liked rules and formalism and, [8:35] you know, uncertainty is uncomfortable. [8:37] I think the humility aspect is really important, though. This is a really good [8:41] opportunity for us to remember why we're doing this. Like, what is human purpose? [8:45] It's not to be famous. It's not to have a Fields Medal. You know, [8:49] solving interesting problems, helping other people, [8:53] being curious, just wanting to understand the world those [8:57] are really good things to kind of get back. Like, what do we [9:00] need to be satisfied humans? And I think that's a really interesting [9:04] moment for us to think about. Like, I've been thinking about it a lot lately. Um, [9:10] it is more difficult for young people, right? Like figuring out how to, [9:14] to find their way. But I actually have more faith in them than, than us in a sense. [9:19] - Do you think there's something to what Terry says about [9:23] it's harder to be motivated when you know a problem has been solved, [9:28] to explore that space when a problem's been solved like this by an AI system? [9:36] - Yes and no, in the sense that [9:40] I do very much agree that the path is as important as the [9:43] destination. Like, if you go on a road trip, the road trip is, you know, is [9:47] usually just as fun as getting there. [9:50] At the same time, if we found out that another country had [9:54] invented teleportation or time travel, [9:57] I guarantee we would reinvent it in the next five [10:01] years. Like, if you knew it was possible and you could reverse engineer a little bit, [10:06] we would be curious, we would dive into it, and we would figure it out. So knowing there's a [10:10] solution also is an incredible motivation to get there much faster. [10:14] - Let me ask the question this way. If this moment [10:19] marks the moment of the trajectory where AI basically, quote-unquote, "solves math" [10:25] or kills math, [10:27] can you make the case that that is in fact a trajectory, and then can you make the case that [10:31] that that is not a trajectory? Can you steelman both paths? [10:36] That this is a path towards which, like, mathematics [10:40] as we know it is done completely, [10:43] so that's gonna be a thing done by machines, and then what humans [10:47] do is something fundamentally different, or there still will be [10:54] something that looks similar to our current mathematics. [10:58] Can you make a, steelman both cases? Alex. [11:02] - Yeah. So, you know, you said two things, and I'd like to separate them. One is solve math, and the other [11:06] is destroy math. So, uh- [11:08] ... let's make that precise. What does solve math mean? If you think [11:12] back to when chess engines first came out and people were like, "Oh, this is gonna be [11:16] amazing. We'll play centaur games." So this is a human being plus [11:20] a computer system against another human being and a computer system, [11:24] and that lasted like 10 seconds- ... and then the computers got so [11:28] good that if you did anything but choose the top move that the computer [11:32] recommended, you would lose. [11:33] And so the human being's role in that game became, "Yes, [11:37] accept the first move. Yes, accept the first move." That's no longer fun, [11:41] and we don't see many games of centaur chess anymore, [11:45] right? And so in fact, I just had a conversation [11:49] with a colleague named Thomas Koberda where we axiomatized [11:53] our argument for what the future might look like, and one of the axioms [11:57] that I'm willing to even accept for the sake of argument is what I call the [12:01] solved axiom, and the solved axiom says, let's assume at [12:05] some future point, we're nowhere near there now, and I don't expect us to be [12:09] there necessarily. We'll see what you know, where the capabilities taper [12:13] off if, if they do. But let's assume the solved axiom which [12:17] says any mathematical problem that can be solved with human [12:21] interacting with AI could also have been solved by AI alone. [12:25] That's the solved axiom. So that, that would mean that really [12:28] there's nothing new for a human being to cont-- you know, you can say to an [12:32] AI system, "Okay, I have problem X, I wanna think about problem X," [12:36] but the only way that I will make progress, any progress that it'll make on problem [12:40] X, will just be press a button. The AI thinks about it. It doesn't necessarily solve [12:44] it, doesn't mean that it'll solve every problem. The Riemann hypothesis is not solved, [12:49] okay? That's what I really want the solution to. I just didn't think in my lifetime I would [12:53] see any kind of progress- ... towards it that we saw a few days ago. So [12:59] it's not that AI systems are solving every problem. They're solving every problem that [13:03] could be solved by humans interacting with AI. That's what that [13:06] axiom would be. So I'm saying, if we are ever in that world, [13:11] does that necessarily destroy mathematics? And my strong [13:15] argument is no, I don't think it destroys mathematics. [13:18] - If that axiom is true, what's the role of humans? [13:21] - So I don't think that mathematics, pure [13:25] mathematics, not engineering mathematics, not things that will, [13:29] cure cancer and, and solve, you know, jet-- everybody has a jet [13:33] pack or whatever other technological advancement you might imagine. Pure mathematics, you [13:37] know, we know that we're proving theorems just because we're curious about them. [13:41] I don't think the solution to theorems living in ones and zeros in data [13:45] centers is valuable to human beings. I think the value of pure [13:49] mathematics is that it lives in the heads of human beings. [13:52] - So just to, you know, add to this, I mean, I'm glad you came right out with the question [13:56] that's on everyone's minds. [13:57] So thank you for that. Um, I would caution against overfitting [14:01] to one objective function for this, right? [14:04] Because, you know, we used to think of ourselves as just the smartest things around. [14:08] You know, human mathematicians can solve, you know, reach the highest [14:11] treetops. But now we're faced with the fact that many of the problems that we [14:15] thought were, you know, quite a ways out have been solved by AI. [14:20] But maybe one of the consolations we can take is actually the fact that we're very [14:23] computationally limited compared to the way that AI is solving it. [14:28] You know, given the same finish line of proving the same theorem, [14:31] I can't really hold that many papers or ideas in my head at one time. [14:36] So in order for me to reach that same finish line, maybe I have to develop [14:40] more generalizable and abstractable ways of getting there [14:44] that then help you not just solve that one problem, but, you know, [14:48] explain connections or other directions you want to get [14:52] into, right? So it's always amazing when you look back on papers you've [14:56] written, and you can see these threads, invisible sometimes, that connect [14:59] different areas of work, because developing systems of ways of [15:03] understanding problems helps you for surrounding problems, not just to get [15:07] a Lean, formalized, you know, one theorem down. [15:11] - But the interesting thing here is those threads, what you're saying is sometimes [15:15] invisible, are quite visible to the AI systems now. It seems [15:19] like that's perhaps the actual very reason why they're [15:23] able to make so much progress, is they can extract the threads [15:27] and pull on them because they have the view of the so many papers. [15:31] - I think it depends on what you mean by threads. You know, for me, the reason [15:35] you prove results is so that you better understand the world around you. [15:39] Mathematics, to me, is a vehicle for abstraction and understanding things. [15:45] And you know, it's when you have a major result, what's [15:49] important is not just the truth of the statement, but what you've [15:53] learned en route to doing that. Usually, people's first question is, [15:56] "Okay, you proved something fantastic, but how did you do it?" And [16:00] then people, once they digest that new idea, that new definition, that new [16:04] way of looking at things, there are lots of other problems nearby that fall. [16:09] So somehow you're building structures that help you attack not just one problem, but [16:13] a whole sea of problems. [16:15] And, you know, it's not that we need fully formalized, you know, [16:19] results to do that. Sometimes we reason in very sloppy metaphors [16:23] that, you know, help us conceptualize the world around us. [16:27] - And how much of this is something Alex mentioned offline, attention is all you [16:31] need. How much of this is the human ability to shine a light on particular areas [16:37] that, quote-unquote, matter? [16:39] So the human ability to ask the right questions, the important questions. [16:42] - That's always been the case. So within this-- before AI, there's a-- [16:46] you can think of, like, here's the axioms, and just abstractly from the axioms, you can get to logical [16:50] deductions, right? There are theorems that we could prove from the axioms. And [16:54] there's a much, much, much larger space of theorems that we could prove than the [16:58] space that human beings ever actually care about. [17:01] And so, you know, so by, almost by definition, it's [17:05] like a tautology that the theorems that we'll get excited about are the theorems that [17:09] we care about as humans. [17:11] So maybe AI can convince us that there's some interesting structure. Maybe it can say, "Hey, [17:15] look over here. Let me give you this new definition or something." It hasn't done that yet. [17:19] - And sometimes the mechanism by which we select the things that we care about, [17:23] it could have an application. So Navier-Stokes is connected to fluids and all, [17:27] um natural systems. But it could also be a thing that some kinda mixture of beauty [17:36] and difficulty that just pulls at us. Maybe there's an intuition. [17:41] You know, the prime numbers are full of these really complicated problems, [17:46] That maybe don't have an obvious application in the [17:50] immediate sense, but it pulls at us somehow and maybe, [17:54] As a symptom of the pursuit of the hard thing, [17:59] Great applied mathematics and engineering and [18:03] solutions to real world problems comes out. [18:06] - So just, you know, as an example, right? A lot of times when I'm working on a problem, [18:10] I might have some, like, bigger goal that I want to accomplish, but usually what I [18:14] do is I try to find the simplest, most embarrassing open problem [18:18] that I still don't know how to solve. [18:20] You know, that aspect of like digesting it down to something, boy, how can we [18:24] not know this? It, you know, it feels like it's on the right track because [18:28] then when we can solve that example, maybe we're assembling the set of tools we [18:32] need to make attacks on other problems. So I do [18:36] think we're gonna have to become a lot more deliberate about, [18:39] you know, what do we care about? How do we use these tools from a [18:43] metacognition standpoint? You know, what exactly are we delegating to them [18:47] and what are we not delegating to them? And we're really at the beginning stages where [18:51] we're just trying to piece it together as we go along. [18:54] You know, on the one hand, looking for a proof is kind of like exploring this tree of [18:58] possibilities. You might have some intuitions based on some [19:02] other proofs you've seen about, you know, which directions to go in. Maybe [19:06] I should go down this branch, maybe I should go down this branch. But sometimes we [19:09] encounter, you know, points on the tree which have really high branching factor. [19:13] We just don't know which way to go down. We don't even have a guess about if we went [19:17] down them, what would we discover? [19:20] And now in this centaur world of math, we suddenly, you [19:24] know, have something else intelligent that we can interact with, right? And what is [19:28] the right way to use that tool to search for the beautiful theories, I think will be [19:33] challenging and evolving, but it's important to not lose sight of the fact that [19:37] we don't just care about theorems. [19:39] So I, you know, I'm a mathematician, I'm a theoretical computer scientist, but [19:43] I've also been reading a little bit more broadly, you know, about social [19:47] sciences just to sort of make my peace with all of the changes that have been happening. [19:51] One of the things that really stuck in my mind that I liked is there's something called the [19:55] doorman fallacy, which is very interesting. [19:57] So the idea is that you take a complex job [20:00] and you boil it down to its simplest, most visible task. So what does a doorman do? [20:08] I guess they hold the door for people as they're coming and going, right? And [20:12] when we lived in New York, we really enjoyed talking to our doorman. Uh, [20:16] you know, they were wonderful people, but, you know, if you digest it down to [20:20] just that simple job, you can think, "Oh I [20:24] can replace them just by a sliding door that detects [20:28] when people are going or coming," and you miss out on all the aspects of their [20:32] job that, you know, add a lot of value and are harder to articulate. [20:37] And I think that's a good way to think about mathematics in the age of AI too. What does [20:41] a mathematician do? [20:43] You know, the simplest and most visible part of the job is we prove theorems. [20:47] And, you know, theorems about conjectures that were open for hundreds of [20:51] years, like Fermat's Last Theorem, these are very visible things that [20:55] we've spent a lot of time popularizing. But I don't think that [20:59] that's all that a mathematician does. I like to think about it as a mathematician [21:03] is someone who thinks deeply, and a byproduct of thinking [21:06] deeply is, you know, proving old conjectures, but it's also [21:10] building new theories, building new ways of looking at old things, making [21:14] new conceptual connections. And these are parts that [21:18] you know, maybe AI will swallow up too, but we're not there yet at [21:22] this moment. So even though this is a momentous time, I don't think [21:26] it's the right time to hit the panic button. I think it's the right time to really [21:30] recenter in terms of what our values are and have these conversations as a field, [21:34] and remember that what we do really has a plurality of different aesthetics [21:38] and goals, and that when one thing becomes cheap, [21:42] like problem-solving for well-defined things, it makes the [21:46] other things that are orthogonal to it all that much more valuable. [21:49] - That's actually really beautiful. The doorman fallacy, going to Perplexity here, is the mistake [21:53] of reducing a person's job to its most obvious task and assuming that [21:57] automating that task replaces the value of the whole role. A [22:01] hotel decides, quote, "We're paying someone to open a door. An [22:04] automatic door can do that more cheaply." But [22:08] in fact, the doorman also welcomes guests and recognizes returning [22:12] customers, handles luggage and hails taxis, provides [22:16] security and discourages unwanted visitors, and [22:20] signals prestige and makes guests feel cared for. [22:23] - That was my favorite part. [22:25] I looked up what a doorman does, and boy, we have a lot of commonalities with [22:28] mathematicians. If I don't signal brand value, I don't know what I'm doing. [22:32] - Exactly. That's funny. But I think the task [22:36] for us humans in general, but also mathematicians, [22:41] is to come up with that list, like actually make more explicit exactly what it [22:45] is that humans do. And I haven't heard any of you [22:49] mention beauty. I apologize if I've mentioned beauty [22:52] excessively already. But there's something that mathematicians [22:56] are, from my outsider sort of fan perspective, are kinda [23:02] expositors, scholars of things beautiful, broadly defined. [23:08] - That's exactly right is until five seconds ago, there was a [23:12] near perfect correlation between the ability to solve [23:15] new deep theorems and what we actually want as human [23:19] mathematicians. And what we want is exactly what you said, [23:23] beauty and understanding and depth of thought, and [23:27] the fact that you can press a button and get what used to be our proxy, [23:32] without any effort, without any deep thought, and without any, [23:36] necessarily knowledge. Like we have people saying, you know, "Do [23:40] a breakthrough," and then putting whatever they find on Twitter and [23:46] and now, you know, what are mathematicians supposed to do with that? [23:48] - I would go one step behind answering questions and doing problem-solving is [23:53] asking the right questions. So like that is how I think you measure [23:57] good problem-solving is asking the right sub-questions, asking, you know, a [24:01] question that is interesting and can be solved and can be [24:05] interpreted. And so I feel like the value of asking good [24:08] questions has always been something we value in our [24:12] colleagues, and that's gonna become increasingly more valuable. [24:16] - I agree completely, and just to, you know, emphasize a point that was sort of [24:20] hidden in there [24:22] I think asking the right questions requires a lot of mathematical insight. [24:26] - Mm-hmm. Absolutely. [24:26] - So, you know, you'll see people make that comment all the time, you know, "What are we gonna value [24:30] going forward? It's asking the right question. It's giving great talks." [24:34] But, you know, I wouldn't minimize, not that you were, but just to be explicit, I [24:38] would not minimize the role in mathematical insight in both of those [24:42] tasks, because it's when we say that, you know, talks will become more important, [24:46] I'm not imagining that, you know, we're all gonna become Twitter [24:50] influencers- ... and TikTok whatevers. [24:53] - I hope not. [24:54] - I hope not. Because, you know, really what stands out [24:57] is having ideas that get adopted by your [25:01] community that end up becoming powerful over time. So, you [25:05] know, I wanna tie this back to something that Lex brought up earlier on, which is, you know, [25:09] he asked, am I being overly optimistic? Probably, right, compared to, [25:13] you know, how grad students view these things. But I think one of the things [25:17] that I'm most worried about in the near term is what's gonna happen to our systems and [25:21] institutions. [25:23] You know, it's not just hiring and tenure and all these things, but it's really [25:27] that when a grad student starts off their PhD, you know, they're [25:31] a new entrant to the field. They're trying to absorb what are the [25:34] techniques in their field, what are the important problems, and they're trying to [25:38] figure out how to make a name for themselves. [25:41] And what we used to do is we would give them a blueprint that was very simple to follow. [25:45] Take a problem people care about and improve it, and then [25:49] people will know who you are. You know, you'll get papers in top [25:52] venues. You know, doors will open for you. But now I think that [25:56] blueprint has changed in a lot of ways because there are so many [26:00] results that are happening right now that we're all just overloaded with it, [26:05] and I think it's okay to take time to rethink that blueprint. We still care about the [26:08] same things, like ideas that are gonna influence your community, [26:13] but they're just harder to operationalize in terms of some simple [26:16] rubric that we can give them. So I'm still optimistic. I just think it's gonna [26:21] take, you know, reworking a lot of our systems and institutions to do that. [26:25] - By the way, and so to, sorry, to go back to your point, I don't actually believe the solved [26:28] axiom. I don't think we're there. I can prove a theorem, but I don't [26:32] understand why the proof works. There's some magic that happens over [26:36] here, and until I generalize somehow the [26:39] phenomenon and understand it more deeply, then I'll see why the, the [26:43] proof works the way it does. Maybe that's one of the reasons why I don't actually believe the [26:47] solved axiom, because the things that we do are much more [26:50] exploratory and what constitutes a good question [26:54] beyond just, "Here's the Riemann hypothesis, make some progress on it." [26:57] - So I think you also brought up a really important point, this, like, [27:01] generality and generalization of our concepts. So related to beauty and mathematics, [27:06] I think one of the gold standards has always been how simple [27:10] can your result be and how much does it generalize. And, you [27:14] know, out of this infinite space of theorems that could be proven, [27:18] Finding some structure that is simple and [27:22] generalizes, like, that is, I think, still the gold standard. And unless I'm [27:26] mistaken, humans are still asking the questions. So, like, we, you know, prove [27:32] existence uniqueness of Navier-Stokes with AI, but [27:36] humans are guiding this. Humans are still asking the questions. And, like, I would love to see [27:40] the next breakthrough if you had a machine that can prove [27:44] any theorem you pose. I want it to start telling me what questions [27:48] I should be asking. [27:49] - Do you think it's possible that that machine just does a for loop- [27:54] ... and finds all the simple and generalizable, [27:58] basically all the E equals MC squared? [28:00] - That's happening today, actually. So, like, there are absolutely research programs that are [28:05] systematically, automatically discovering kind of minimal physical laws. [28:09] So this is less mathematical theorems, more physics governing laws and equations. [28:15] - So observing natural systems and trying to see in those natural systems- [28:19] ... if there's simple physical laws that model how they operate. [28:23] - Yeah, distilling it down to, like, the basics. So you can kind of [28:26] recapitulate known theories. If you observe balls dropping, you can, [28:30] you know, deduce gravity and F equals MA. If you observe fluids, you can deduce [28:34] Navier-Stokes. And then taking that to data, we [28:38] haven't written down simple equations: weather, [28:41] turbulence, solar flares, things like that. [28:45] And I'm assuming similar things are happening in math where, you know, there are [28:49] probably these, like, minimal pieces that can be deduced. [28:54] - Do you think there's a large space of simple and generalizable things in math [28:58] that could be discovered automatically? [29:01] - So my version of the solved axiom is that anything that I do [29:04] as a human mathematician, [29:07] I've tried on several occasions to sit down and say, "Okay, [29:10] you know, LLMs will never do this that I do." And I'm like, "Well, [29:14] why not? I can ask a question. You know, I can ask it a question." [29:18] I can ask it, "What questions should I ask?" [29:21] Right? There's sort of it-- Because it's such a generalized system, [29:25] uh, I'm not saying that it's good at all of those things now. But [29:29] my version of the solved axiom is anything that I can contribute as a [29:33] human being, it could also have contributed on its own. And so [29:37] what I really mean by that is in terms of value, what do I [29:41] bring to the community? It can't be I asked a great [29:45] question. It can't necessarily be in this world of the solved axiom, which [29:49] again, I don't believe. But even in that world, it should be something other than I [29:54] asked a great question, I proved a great theorem. And even in that world, [29:58] I think you need mathematicians, people with human mathematical expertise. [30:02] - Ankur, do you believe in the solved axiom? Where on the spectrum of, like, [30:07] how close we can get to it, [30:09] and in that spectrum, what's the role of the human, do you think? [30:13] - I guess I don't really believe in the solved axiom. I [30:17] think- You know, maybe it's half true in some ways, and figuring out [30:21] which half of it is true will be very interesting and challenging. So I think there'll [30:25] be major surprises and more to come in terms of what AI [30:29] can solve. But, you know, for me, when you're facing a new [30:33] technology that's really, really good, the danger is that [30:37] when you try to conceptualize why it's working, you do so by anthropomorphizing it. [30:43] So, you know, maybe one of the earlier lessons here is from deep learning. [30:47] So there are these amazing examples. You know, you can take neural nets, and you can [30:51] do human-level or better object recognition. [30:55] But in general, the way that it succeeds is less robust than how a human does, [31:01] Because there are all these examples like adversarial examples, [31:05] where you can add very slight patterns to natural images that can completely fool [31:09] these object detection systems, but not a human. [31:13] And, you know, I had a former colleague at MIT, Alex Madry, that had some really [31:17] nice work, which I really enjoyed. It was, um-- I think the title was something like, [31:21] uh, Adversarial Examples Are Features, Not Bugs. So they did the following [31:25] cool experiment. So you take a bunch of natural data, maybe pictures of cats and [31:31] dogs. You train a deep network to identify them very well. [31:35] You add some adversarial perturbations to them that, you know, [31:39] make to the AI system a cat look like a dog and vice [31:43] versa. So it's kind of think like a small little pattern that changes [31:47] how the object detector works. And then you create a totally fake data [31:51] set. You know, it's got pictures of cats with dog patterns on top, and you [31:54] label it as dog, and vice versa for cats. And amazingly, when you [31:58] train a deep network on this with all the wrong labels, it's gonna [32:02] learn something that does well on natural data. So somehow, when we [32:06] recognize what a cat and dog is, we do it, you know, in a way that we can [32:10] articulate in terms of you know, its legs and its tail and its shape in general. [32:17] But a lot of times when you know, deep networks or AI can do [32:21] things that they might be able to find trapdoors or backdoors to solving the problem [32:27] that use a lot of statistical relationships in the data that a human doesn't. [32:31] So to me, you know, it's important to not, you know, draw [32:35] this boundary, you know, very sharply between what you can do and what [32:38] AI can't do because you're gonna be wrong. [32:42] But that doesn't mean that I don't believe there will emerge some kind of [32:46] boundary, if only for the fact that I think we [32:49] over-anthropomorphize the way that they solve problems. And when I'm looking through [32:53] the OpenAI solutions, they just don't look like a human has solved [32:57] those problems. So to me, there still seems to be a very big qualitative difference, [33:01] and that's an opportunity really for collaboration and centaur [33:05] math more than it is, you know, [33:08] dread about the AI overlords. But, you know, who knows? Could be wrong next week. [33:13] - What's your intuition about how much of this is AI just [33:17] tying together a bunch of thousands of disparate papers [33:20] and pulling at the threads, sorry for bringing back the metaphor, but [33:24] the threads of the different papers and then just basically [33:28] articulating the proofs that were already there in the different papers and you're [33:32] making minor connections? Or is it actually developing [33:37] new mathematical ideas? Have you seen glimmers of [33:42] whatever that genius AlphaZero move in the game of Go? [33:47] - Yeah, move thirty-seven. [33:48] - Move thirty-seven. [33:49] - So I haven't read all seven hundred of their papers. The ones that I [33:53] have looked at, you know, some people speak about this dismissively like, [33:57] "Oh, it was in the convex hull." You do the moves that we know as, [34:01] as humans, and you just do them really well and do a really good tree search. [34:06] I wouldn't call it the convex hull. I think Tim Gowers had a really good way of [34:10] talking about it instead as the semigroup generated by all of these [34:14] things. Yes, it's the same notion of putting together known moves, but the space [34:20] of known moves, what it can accomplish is sort of much, [34:24] much bigger than this kind of derogatory [34:28] convex hull. So with-- in terms of the quasi-RH [34:32] the moves that it took are all things that were known to human beings [34:37] and there's a very long cascade of them that is kind of [34:41] shocking when you look at it and say, "Oh my God, that's so beautiful. How did you think to-- How did [34:45] you..." I'm anthropomorphizing it, of course. "How did you think to do this move after that move [34:49] and after that move?" And yeah, that does it. [34:53] - Could you articulate to the more general public the intuition, [34:57] the line between semigroup and convex hull, the derogatory convex hull, [35:01] and quasi-RH, of course, is the Quasi-Riemann Hypothesis, that one [35:05] of the big proofs. [35:06] - Yeah. Let me try to do it like this. You have-- think of like a balloon. [35:11] So the inside of the balloon is its convex hull. [35:15] And so we have like all of these results that-- all these ideas that mathematicians [35:18] have thought up, and then anything inside of that is some linear combination of it. [35:23] But it feels compact. It's like there's this kind of closed, [35:27] enclosed area, and within that area, like AI hasn't really made new [35:31] progress 'cause it hasn't moved the balloon out. It hasn't given us a new idea yet. [35:36] - And it's derogatory because the intuition there is we could have [35:39] done it, mathematician could have done it because it's already quite obvious. It's already [35:43] within the space of things that have already been thought about and developed. [35:48] - Yeah. So it has read every paper. It's exactly what Ankur was just saying. It has [35:52] read every paper. It has internalized all of [35:55] that. That's more than any human brain not even could hold, but has [35:59] time to read and learn and internalize. So in terms of [36:03] breadth, yes, it beats any human being. [36:06] And then in terms of depth, you know, has it really generated [36:11] a genuinely new idea? And you can argue that at least [36:14] in the case of quasi-RH, I would not say it's a genuinely new idea. It's a, [36:19] it's a beautiful combination of a lot of things that were very well known. There are a lot of people who [36:23] are kicking themselves that had some idea in that direction twenty years [36:27] ago, and two years ago, there was a big breakthrough on cubic theta [36:30] functions and cubic Gauss sums and, you know, that ended up being the path [36:34] that the AI took to the solution. But so to contrast that with a [36:41] semigroup, I think of a semigroup as like tiles. So you [36:45] know, you're just playing tiles on the board, and what you're allowed to do is whenever [36:49] you have a tile, you can put another tile to the left, to the right, to the front or to the [36:53] back. And then you can just start tiling things. And [36:57] that's like a semigroup action. And the question is, well, [37:01] how far can you reach with that? You can go all-- I don't know. You know? [37:06] You can go really, really far out with existing ideas might [37:10] prove a whole lot more than we as humans have been able to explore. [37:14] - So your intuition is more in the semigroup [37:17] direction, meaning like there is at least glimmers of expansion [37:25] versus linear combinations inside the convex hull. [37:28] - You know, how we choose to interpret what it's doing I think [37:32] is up to us. But throughout the history of mathematics, there have [37:36] been very seemingly difficult and deep problems [37:41] that then someone comes up with a three-page solution [37:45] that is very elementary and could've been explained to an undergrad and [37:49] could've been discovered twenty years before when the problem was first posed. [37:52] And had it been discovered twenty years before, it would be an [37:56] exercise in a textbook somewhere. And because it-- that solution wasn't [38:00] discovered, and really bright people worked on it for a long time and made incremental progress and made [38:04] the problem really famous, then by the time it was actually solved, it was a really big [38:07] deal. [38:09] That happens from time to time in mathematics. And what I said a year ago when I [38:12] wrote-- So there was this um, International Congress of Mathematicians this [38:16] summer, and I was writing my proceedings that they wanted ten months in advance. [38:20] So I said, you know, "What am I gonna write about math and AI now that will still be relevant, [38:25] Ten months from now?" And I think what's there is somewhat [38:29] evergreen. I stand by what I wrote. But one of the things that I wrote there was [38:32] exactly that we're going to discover that a whole lot more problems [38:36] that we thought were unreachable by current technology actually are reachable [38:40] by current technology if you can put it together in just the right way, and that's what [38:44] AI is amazing at. Again, someone could interpret that as, "Oh, so it's [38:48] inside, you know, these solutions were inside the convex hull." You wanna call it that, that's [38:52] fine. I think that's, you know-- It's not invalid. [38:55] - So I like all these metaphors, you know, also the balloon. So the important thing about the [38:59] balloon is the boundary between what's inside and outside, and what makes it [39:02] derogatory is the fact that, you know, you haven't seemingly changed [39:06] the boundary. But I-- you know, definitely these problems have [39:10] changed the boundary. [39:12] And, you know, maybe a more concrete thing is just that the types of things that people are [39:16] gonna work on after that drop of OpenAI problems is forever different. [39:23] You know, there were at least a couple of results there that are like, you know, [39:27] field-altering results that will change what types of problems you give to young [39:31] researchers, what areas you want to explore, because your conception of [39:35] the landscape is just that much different. [39:37] - So not just the-- because it expanded, but because the very [39:41] expansion changed the direction of the interesting questions. [39:45] - That's right. So in fact, you know, one of the problems which probably, you know, maybe it's the [39:49] easiest to explain what kind of effect it had on people is not actually from the [39:53] OpenAI drop. It was an Anthropic result that was written up by my colleague Virgie [40:00] Vassilevska Williams and then Josh Alman. So this was about the [40:04] 3SUM problem. And I'd say that there it has a major effect on [40:08] what's called fine-grained complexity theory because there was a big [40:12] surprise that, you know, a very basic problem that you give people in, like, [40:16] coding interviews can actually be solved asymptotically much [40:20] faster than the solution anyone knew, [40:22] and that was counter to what the community was betting. In fact, there was a [40:26] whole theory built up around the idea that if this problem really is that hard, [40:30] then all these other problems inherit its hardness. [40:34] So now, as a result of that problem being easier than we thought, you know, having a [40:38] bit better of an algorithm, you know, yes, that gives you a lot [40:42] new algorithms across the entire landscape. But it also [40:46] changes the character of the field because the goal there was to put all those problems [40:49] together and said that we should be studying them all together. [40:53] But now we're sort of back in the state of affairs where maybe the right way to study them is [40:57] separately after all. [41:00] So I think these are examples where, you know, irrespective of how [41:03] much, you know, deep new generalizable ideas we get out there, [41:08] even just knowing the, you know, truth of the theorem is gonna change [41:12] the way that people organize research in that area. And, you know, there's no [41:15] putting the genie back in the bottle on some of these problems. [41:18] - I mean, imagine one of the seven hundred was factoring in polynomial [41:22] time. That would absolutely change-- Everybody would know. [41:25] Like, you can't use your credit card on Amazon anymore. All of e-commerce is gone. [41:29] All of secure communications is gone in-- like that. [41:33] - Can you explain factoring in polynomial time? [41:35] - Yeah. So if you give me two very large numbers, a [41:39] million digits each, I can multiply them together and do [41:43] that in a split second. If you ask me to go backwards, [41:47] say you took two large prime numbers, you multiply them together, you give me this, the product, [41:51] and you ask me, "Figure out which two numbers..." You know, just factor. Don't we, don't [41:55] we ask our kids to learn how to factor in school? You know, it's a lie. [41:59] We do not know how to factor. And the fact that we don't [42:03] know how to factor is why we have the capacity, the ability to [42:07] communicate with one another in all kinds of secure ways. So all of [42:11] modern-- basically the modern world depends on the fact that we don't [42:15] have an algorithm for taking a large number and pulling its prime factors out of it. [42:21] - So a lot of e-commerce does because that's the basis for a lot of the schemes [42:25] in public key encryption. But, you know, famously, there are quantum [42:29] algorithms for factoring, so- [42:31] ... there's a lot of emphasis these days on, you know, finding post-quantum [42:36] you know, bases for cryptography too. [42:38] - Yeah, lattice-based methods and- [42:40] Yeah, we're already concerned about what will happen when we have quantum computers large enough to, but we still [42:44] have a little bit of time before quantum computers get large enough [42:48] to make this an actual problem. But there's no [42:52] theoretical reason. I mean, okay, factoring's probably not NP hard but [42:57] You know, I mean, let's say, let's say prove that P doesn't equal NP. Maybe that doesn't-- [43:01] That would be amazing, obviously, but that doesn't if it proves that P [43:05] equals NP, that would be completely [43:08] - That would be very interesting. [43:09] - That would be very interesting. [43:10] - I'll definitely ask about more broadly in the context of complexity [43:14] theory, but presumably OpenAI or any other company [43:18] would not drop that result carelessly. [43:22] - That's the kind of thing that should go through, yeah, the government and- [43:26] - Sometimes certain mathematical results quite obviously [43:30] lead to dramatic impact on the world. Other [43:34] results might take maybe months to digest, but that one would just, [43:40] yeah, would affect the economic systems of entire nations, so. [43:44] - How many mathematicians does the NSA employ? [43:48] - Thousands. [43:49] - Thousands, like something like this. [43:51] I've always done this thought experiment, you know. What, [43:54] what might they have that they just wouldn't tell us about? And this was ten years ago, you [43:58] know, that some [44:01] evidence showed that collaborative teams of mathematicians could solve [44:05] much, much harder problems than any one individual mathematician if they, like, [44:09] broke it down in the right way. So I don't know. Do you mind if I just revisit the- [44:15] - Yes [44:15] - ... the balloon for a second? 'Cause I actually like the semigroup [44:19] analogy better than the convex hull, but I don't [44:23] think it's as it should be interpreted as derogatory as [44:26] it is even if it was just that these machines are [44:30] interpolating human knowledge rather than extrapolating. [44:34] Because even if there is this convex hull of ideas and things that could [44:38] be pieced together from bits that humans have invented, [44:43] we're not doing that fast enough or, you know, there's a [44:47] lot of opaqueness, and it's-- this balloon is not see-through. [44:51] And so even if you can make this convex hull much more transparent, kind of [44:55] like lift the fog, give mathematicians radar so they can see what's, you know, [44:59] over the horizon, [45:01] that's pretty incredible, and I don't think we should be interpreting that as you [45:05] know, derogatory at all. I do think it is more, [45:08] more extrapolatory than that, though. [45:11] - Yeah, we should say probably the size of the convex hull. We don't know [45:16] actually how big it is. [45:18] - Well, that's part of the point. We don't know how big it is. We don't know what's next to each other. [45:21] Like, there's so much fog of what we know. Like, And that's exciting to lift that. [45:29] - But human beings do make new definitions, you know, bring new [45:32] concepts in, and that fundamentally feels [45:36] different from combining existing ideas and repackaging them. [45:40] When human beings do that, and they do that from time to time- [45:44] ... it really feels like, okay, there's a new boundary point, [45:48] and now what's the convex hull of that, of what you could prove? So [45:52] it's like you can take the balloon, and there are ways to pull on it and- [45:55] - And we know when it expands. Like somehow- [45:57] ... we have an intuition for when it's getting bigger. [45:59] Like, you can recognize that. [46:00] - In many ways, the archetype of what the researcher looks like is gonna [46:04] change because of this new technology too. [46:07] You know, I'm always amazed that when you look back historically, how much [46:11] very different fields of math really spawned from similar ideas. [46:17] And a lot of those connections drove the development in the beginning, but over time, [46:21] they become a bit more specialized. And I think certainly the way that we [46:25] train graduate students, you know, we'll [46:28] need to make sure that they're not just overspecialized in the same way. [46:32] You know, I don't know what the future of hiring will look like. I don't know what the future of journals will [46:36] look like. But I think people who think broadly about, [46:40] you know, a bigger swath of mathematics will be in the [46:44] best position to benefit from these technologies. [46:48] You know, that would be my guess. Can I ask a almost silly-sounding question? [46:52] But so a large part for many decades of research has been [47:00] the paper. If you were to guess in the years to come, how does that [47:05] change how a human contributes their idea, maybe [47:09] collaborate, how a team of humans contribute an idea into the public space? [47:14] - So, I mean, I don't know about in pure math, but in engineering and [47:18] some of the applied sciences, we already have a pretty [47:22] rapidly shifting view of what an academic contribution is. So [47:27] if you write code in part of a larger infrastructure that [47:31] is reusable, that, you know, supports larger projects, so think of [47:35] LIGO or some of the big, you know, billion-dollar [47:39] astronomy projects or, you know, cancer research projects that [47:42] require hundreds of people and huge resources already, [47:47] it's not necessarily one 15-page PDF. You [47:51] know, there's lots of different pieces, code, data, [47:55] reusable bits that can be fit into a larger framework. And [47:59] in my mind, that is kind of obviously even more [48:03] relevant with AI, is having, you know, a little workable [48:07] piece or widget or API or code [48:11] base or data set that can be leveraged by the community or [48:14] by AI is increasingly valuable, [48:17] even more than just, like, one result or one paper. And I don't know [48:21] how that looks in, in pure math. [48:23] - Can you comment on your view how everything's changing? Even just observing [48:28] the insane evolution of even arXiv, [48:33] which is similar in some ways to what's happening on GitHub, for example. A lot of people [48:37] in the open source movement are really struggling to know what to do. Some are predicting [48:41] the death of GitHub. [48:42] - Why? Just 'cause there's so much code being added to GitHub- [48:46] ... that it can't sustain the growth? [48:48] - So one of the things that makes the open source movement successful is there's a kinda [48:52] one or a few folks that really manage each repository, the [48:56] human beings, and they are, um- [48:58] - psychologically and on many levels are becoming overwhelmed by the, all the pull [49:02] requests. So it becomes impossible to manage and they get [49:06] exhausted and there's this whole feeling. [49:09] The open source movement, maybe similar to the academic environment, [49:13] requires the humans to be motivated. And if you get worn out by the just similar [49:21] the ideas that Terry Tao mentioned, [49:24] that you just quit. And so that's a failure mode that you want to kind of avoid. [49:28] - So journals were already in math under very, very severe strain [49:32] from the number of things before AI, having nothing to do with AI, just [49:36] there was, you know, more people producing more knowledge, sending it to journals, and [49:40] journals had to referee it, and referees are exhausted and being asked to, [49:45] work really hard to understand these theorems and check them. [49:49] The only thing worse than that system, I think, is not having that system. [49:53] Because the way that, like, our mathematical epistemology, the way that we [49:57] arrive at truth, is deliberately slow and careful and precise. Go back [50:03] to Perelman's solution of the Poincaré conjecture. He puts these [50:06] papers on the archive, [50:09] and then people around the world are, you know, shocked by this, and so they invite him to come [50:13] give lectures. He gives he comes to Stony Brook, he comes to NYU, he comes to Princeton. [50:17] He's going all around the country and the world, and [50:21] he's giving three-hour long impromptu [50:24] explanations of, of, you know, maybe an hour is, is him explaining what he's doing, and [50:28] the next two hours are people saying, "On page two [50:32] hundred and forty-seven, Lemma four hundred and twelve, I don't understand how you [50:36] do this." And then he goes to the board and gives them a half-hour lecture on how he does that. [50:40] So the entire thing lives in his mind. [50:43] He's done the best he can to communicate it on paper, and then it's up to [50:47] us, the community, to go back through it very carefully, very [50:51] in a detailed and precise way and agree, yes, [50:55] this is a real proof of the Poincaré conjecture. And that took years to do [51:00] and a whole lot of human effort. So coming up with a [51:04] solution is only the first step in the process, and that process has to be deliberate, [51:08] and it has to be community-driven. And so that's what the journal [51:12] system is trying to accomplish. [51:13] - So you make a really interesting point and it kinda [51:17] highlights this traditional view of a pure mathematician as this, like, [51:20] lone genius who, [51:23] you know, solves this singular problem and then goes explaining it to the world. [51:29] In many other fields, we have moved past the [51:32] point where we can't individually advance, you know, [51:36] circuit design or aircraft design or protein folding or any, you know, [51:40] astronomy, and we have to work together in these billion-dollar collaborative [51:44] projects. And again, I'm an engineer, so I'm thinking leadership is obviously [51:50] the thing that a community needs when it's having this kind of crisis of [51:54] organization and individuality moving towards a more collective model. And that's [52:00] interesting because I know so many mathematicians, and, like, [52:05] by kind of organization, this is a very [52:08] independent group of people who don't like being told what to do, who don't [52:12] like being told what to study, who are very independent by [52:16] their very nature. You know, we like solitude and working [52:20] alone. That's why we, you know, spend so much time doing it. [52:23] - I think that's a really good point. I mean, it's interesting just to see the [52:26] differences in working styles, even between math and physics, you know, with things- [52:31] ... like the Large Hadron Collider. So I mean, let me take a contrarian [52:35] view for these things, which is you know, you should never let a [52:39] good crisis go to waste. I agree the journals were [52:43] overloaded, the conferences were overloaded. It always strikes me that we're [52:47] doing so much work, you know, to review these things, to handle [52:51] all these things, and now we're faced with the task that [52:53] someone's gotta review all these AI-generated papers. [52:57] And some of them are important, but, you know, it's really hard to get your community on [53:01] board with doing that. You know, I don't know what's gonna happen, but I would really [53:05] be in favor for finding new models, not to replace the [53:09] old ones, but to relieve some of the pressures that have come about. [53:13] So, you know, I'd love to see models where, you know, maybe we take a more [53:17] collaborative effort to verifying some of these AI [53:21] proofs, right? So some of the OpenAI results that have [53:24] dropped, you know, improvements on Fast Fourier Transform. [53:29] You know, there's been tweet streams with people improving the constant. [53:33] I don't really know what to make of that. You know, I'm not even on Twitter. [53:37] You know, and it seems to me that we need to figure out other modes of [53:41] working, you know, for organizing what are the state of all the [53:45] improvements, but also how do we organize and tame these ideas? [53:49] So it's kind of striking that, you know, the way that we think about publications is you [53:53] work hard on your draft, you post it on arXiv, and that's your draft. [53:58] You know, you might update and create a new version at some point or a third [54:02] version, but the draft belongs to you. [54:06] It's not really up to the community to add notes. "Hey, I think you can do this argument [54:10] differently. Hey, I think this is simplified this way." But maybe there are some [54:13] kinds of things that we need other models for, you know, [54:17] allowing crowdsourcing for our understanding to evolve. [54:21] I'd be all for that. You know, the other thing is I love giving [54:25] research talks. You know, one of my favorite parts about giving research talks [54:29] is you can take all of the messy complexities of the paper, the things [54:33] where you just knew you had to dot the I's and cross the T's, but [54:37] that wasn't really what excited you. There was some key idea, and there [54:41] was some special case that really illustrated why that idea was powerful. [54:45] And that's the part you really want to explain to people, because when asked, someone asks [54:49] me about Lemma 57, I don't even remember it, right? And [54:53] maybe, you know, this idea that uh, a paper [54:56] is just this monolithic thing designed to be [55:00] you know, written down and completely formalized. Well, now we can [55:04] trust Lean formalizations to some extent for some of the [55:08] technical building blocks, and that can allow us to rework [55:12] the way that we do the writing in the first place, so we can put more of the emphasis on the work [55:16] on, you know, the promulgation of those ideas. [55:20] - The Mathlib model is exactly what you were describing, where instead of a paper, you have a [55:23] PR, you have a pull request. The maintainers and the reviewers will [55:27] go through the pull request. They'll make suggestions here and there. I mean, the [55:31] referees make suggestions, you can improve this and that, but it's not a public process [55:35] in a journal. In a GitHub open [55:39] workplace, you can see exactly who contributed what idea [55:43] that simplified an argument or you're, you know, you're reusing this... You're reinventing [55:47] this idea that already exists in this part of the library. You should just quote that and [55:50] refactor this part, whatever. So and [55:55] Mathlib working formally allows one for the first time. Like, one of the [55:58] problems with mathematics is that we don't [56:02] have a perfectly well-written specification [56:06] to our theorems, our technical lemmas. There's-- If you open up a random [56:10] page in a random math book and read a lemma, there's so much context- [56:15] ... that the author-- 'cause it's so annoying to write something that [56:18] fills in every bit of context. So we just assume you've read the previous [56:22] sixty pages. Now I can just tell you what the next step is in that lemma. But then when you try [56:26] to-- That means you can't just take it off the shelf the way you can a part, [56:30] a physical part when you're building a rocket. [56:33] And so if all of mathematics is done in the future [56:37] formally, then the specifications are built into the [56:40] object. You know you're not using it in the wrong way, 'cause you can't use it [56:44] in the wrong way 'cause Lean won't let you, [56:47] and that is the sort of crowdsourced version of mathematics. And that was the dream [56:51] of Mathlib- ... when it was started ten years ago, [56:55] was to run mathematics more like a science. [56:57] - As a quick aside, can you, [56:59] mention what Mathlib is and Lean, just 'cause we mentioned it a few times for folks? [57:04] - Sure. So mathematics is, in the ideal, a formal [57:08] process starting with an axiomatized system and then logical [57:12] rules of inference from there. So if people think of -- I don't know, do people [57:16] still learn Euclid's geometry in high school or something? That's supposed to be [57:20] the kind of model for kids to start seeing [57:23] proofs for the first time and seeing an axiomatic structure. [57:27] And you can take that idea and write it in software [57:31] and say, "Here are the actual axioms of mathematics. Here are the rules of logical [57:35] inference." And Lean, there's a long history of this kind of software. It's called [57:39] interactive theorem provers, where you're writing line by line in a proof, and the [57:42] computer says, "Yes, this line does follow by logical inference from the [57:46] goal state on the previous line." So Lean is software that allows you [57:51] to check all kinds of things. It can check the specifications of your code [57:55] align with what you, [57:57] what you are trying to define. That's what it was -- what it was invented for was bug-free [58:01] code. And hopefully, in the future, we will have bug-free code because AI [58:05] systems are really good at finding bugs in code written by humans, so [58:10] that's-- it's becoming ever more important for that application. But [58:13] another application is to check mathematical results- [58:18] ... and allow you to write a mathematical result in a very formal way. And so [58:24] now Lean itself is just the hardware that checks the logical steps. You do not want [58:27] to start every math project by defining what the natural numbers are, [58:31] using zero and the successor, you know, the Peano axioms of the natural [58:35] numbers. So it's nice to have a large general [58:39] library that's extremely well maintained, extremely well [58:43] organized, and so on. And that's what Mathlib is. Mathlib [58:46] is the Lean library for formalizing [58:50] mathematics in the Lean ecosystem. And again, Lean is just one [58:54] interactive theorem prover. There are lots of others. It's the one that so far has been [58:58] winning the math wars, if there are any math wars. It's a competition of [59:02] two people or something. [59:03] - And it's the technology used in part to show that [59:08] the OpenAI release of all the different proofs, that the proofs are correct. [59:13] - That's right. It has become now standard to, [59:17] because autoformalization has gotten to the point where AI systems can not [59:21] only prove a theorem in natural language. You can't trust their natural language proofs. [59:24] These are stochastic processes. [59:27] And so the fact that they're saying things doesn't mean that they necessarily have [59:31] a model of mathematics in their heads, whatever that means, the same way [59:35] that we do. But that's okay 'cause they'll write a Lean -- Lean code [59:39] that builds on top of Mathlib, builds on top of the Prime Number Theorem Pro-- Plus [59:43] project or whatever, and checks that it has [59:47] indeed solved this problem down to the axioms. [59:50] - So this is where I'll just take the tangent, and the thought experiment may be a bit [59:53] philosophical at this moment, but what if you have that black box [59:57] that gives you proofs that you can't understand? Let's say the more interesting case [60:01] where, not because it's so big, but because the ideas just [60:06] don't make sense to you, to any of the world expert mathematicians in that field. [60:11] What do you do with that? [60:13] - So the only way, sort of almost by definition, because the [60:17] logical rules of inference are written down, and there, there's [60:21] only a few of them. There's a handful of them. The axioms, there's a handful of them. [60:25] So the only way that you have mathematics that's not understandable is scale. [60:30] - Let me add a contrarian take. [60:31] - Yeah, please. [60:31] - So one of the things I also wonder is what would unformalizing math look like? [60:37] You know certainly formalization helps tremendously that we [60:41] have confidence that the results are correct. But much of the way that we [60:45] understand the results ourselves is really at some unformalized [60:49] level. You know, what is an interesting special case? What, you [60:53] know, where does this come from, you know, this technique you're using, right? [60:57] And, you know, maybe that's what that world would look like. If we have these black boxes, we [61:01] would try to tease apart and figure out how to ask them easier [61:05] questions to develop some kind of system of understanding of [61:09] how they actually got to the answer. [61:11] - Yeah, so there are really two different things that we conflate when we write a [61:14] paper. One is the engineering of the proof, and the other is the [61:18] communication of that to someone else, and they're almost at odds with one another. [61:22] I had this experience-- So I taught real analysis last fall, and you open up [61:26] any real analysis textbook, and this was my experience when I took real analysis. [61:28] The first couple of lectures are what are the real numbers? You know, we have to [61:32] define them or construct them as equivalence classes of Cauchy [61:36] sequences of rationals, whatever the hell all that means. It takes, like, two weeks for a first, [61:40] semester course in real analysis to get there, and then we have the real number line, [61:45] and now we can start doing calculus, you know, with formal epsilon and delta [61:48] definitions, yada yada. And it always struck me as bizarre that [61:52] we do this because before the class, I knew what the real numbers were. It's this number [61:56] line, and then we spend these two weeks talking about [62:00] Cauchy sequences of rationals, and then on week three, they're, [62:04] they're a number line again. What the hell was the purpose of that? [62:07] And so it's because the way things are written or had been [62:11] written were very engineering. You need to construct the real numbers before you go start using [62:15] them. But the [62:16] real numbers, human beings used the real numbers long before we understood [62:20] that they needed to be constructed formally. So what I did in my class is I jumped [62:24] straight into epsilon delta calculus. We did, you know, six weeks of [62:28] calculations until we got to a point where [62:32] wait a second, you know, if here's a theorem, it's true in the real numbers. There's some [62:36] convergence, something called Bolzano-Weierstrass theorem. You have a sequence of real [62:39] numbers that are jumping around, but in a fixed range. There has to be some subsequence that [62:43] converges to a real number. [62:45] Well, that's not true in the rationals. Why is it true in the reals? What [62:49] are the reals? So you don't start asking that question until you hit a [62:53] problem that needs to be solved. So the way I taught the course [62:57] was sort of backwards from how it needs to be taught if it's an engineering project, [63:01] 'cause you can't just start talking about the real numbers until you've constructed them, until you know what they are. [63:05] Well, that's correct if what you're doing is building up the theory. But if [63:09] you're telling a story, then you absolutely can just start with the real numbers. [63:12] You have an intuition for what that is, and until your intuition breaks, you don't need [63:17] the resolution of how you actually construct them. So I think the way that we write [63:21] papers in the future is gonna be very different, where we spend a lot more time just [63:24] talking about the story of what have I learned in this process, [63:28] what are the interesting-- Uh, here, here's, you do the standard thing here, here, and here, [63:32] but then here, you're sort of stuck, and where do you go? And here's an idea for [63:36] why you go left and not right and not up and down or whatever. And there's a Lean [63:41] formalization that accompanies this story that lets you know that the engineering actually [63:44] works out. 'Cause you can't just tell stories. If you just tell stories, you know, math [63:48] is not music where you can tell [63:52] a beautiful story, and it's okay that you have parallel fifths moving somewhere, and, you know, you've broken [63:56] some rule, but Bach does it all the time, so. [63:58] - So I think that's a very important point, just to sort of emphasize the point [64:02] that Alex is making. You know, there's this view of mathematics which I don't believe [64:06] in, where it's just this rising tide of theorems. You know, how can [64:10] AI be bad? It's contributing to this rising tide. [64:14] But, you know, really, mathematics is not just about the number of theorems we [64:18] have, far from it. It's also about that story of fighting [64:22] with problems, because when you fight with a good problem, it fights back at [64:26] you, and you learn something about it. It's really about the system of [64:30] beliefs and abstractions you come up with en route to, you [64:34] know, when you're fighting this problem, you discover some more general meaning that applies [64:38] more broadly. [64:39] And I think it's a great opportunity to sort of disabuse ourselves of this idea that [64:44] we're just you know, axiomatizing things until we get to the next theorem. You know, [64:50] telling that story of how we actually got to these problems, what the [64:54] parallels are to other problems, is now, at this pivotal moment, more [64:57] important than it ever has been. [64:59] - Yeah, so to get all the way back to your question about, you know, is it a matter of scale, well, [65:04] Yes, it's just a matter of scale, but exactly to Ankur's [65:08] point, the way that we think about mathematics is about stories, and it might be that [65:12] we don't understand the story of this. You can follow a proof line by line and not [65:16] understand what's really going on. [65:20] You might say, and you just ask some other LLM to explain to you the story if it [65:24] can come up with such an explanation. But that's what we [65:28] want. We want stories. We want understanding. [65:30] - When you fight with a problem, it fights back. [65:37] I think there's a deep truth to that about all the great pursuits [65:41] that humanity has taken on. And on that topic, [65:45] I would like to talk to you about some of the greatest fights of all [65:49] time with some of the hardest problems in mathematics that have fallen. Um, [65:53] but first, we'll have to break if it's okay. So Alex, you're a number theorist. Um, [66:01] maybe it's fair to say the central problem in your field is the Riemann hypothesis, [66:05] and you showed a lot of excitement on the aforementioned Twitter/X [66:11] about the quasi-RH, the quasi-Riemann hypothesis proof. Can you explain that? [66:18] Can you explain what the Riemann hypothesis is, and can you explain [66:22] the nature of your excitement about it? [66:25] - Sure. There's sort of a one-minute version of the Riemann hypothesis- [66:28] ... or a five-minute version or a thirty-hour version. [66:32] I don't know how far you wanna, you wanna go. [66:34] - Let's start pulling at that string. [66:36] - So we know what the prime numbers are. People are interested in the distribution of the prime numbers, [66:40] and what Riemann discovered is that there's a function, this is totally [66:44] crazy, that there's a function that takes complex numbers as inputs and [66:48] has complex numbers as outputs, and that function [66:52] governs the behavior of the prime numbers. It's called the Riemann zeta function, [66:58] and by the zeros of the zeta function, we mean complex inputs [67:02] where the output is the number zero in the complex plane, so the, [67:06] the origin in the complex plane. [67:08] And the question is, where do these, where are these complex zeros? What inputs, what [67:12] complex number inputs give you a complex number output of zero when [67:16] you apply this zeta function? [67:19] Why in the world should that-- What, complex numbers? Are you-- What are you talking about? [67:23] We have these discrete numbers that aren't divisible by anything else [67:27] sitting in the natural numbers on the number line, and now you wanna talk about [67:31] functions from a complex space to a, to a complex space. Um, [67:35] that was the great breakthrough. The great breakthrough was to figure out how to do [67:39] signal processing on the prime numbers. [67:42] If you think about, you know, the reason people can hear [67:46] you, your podcast is that my voice box is producing [67:50] pressure waves of sound. That's the natural space. [67:55] Then that goes into the microphone. The microphone analyzes it, does the Fourier analysis [67:59] on it, converts it into amplitudes and frequencies, [68:03] and it's just a bunch of numbers, stores those numbers or sends those numbers, whatever. [68:06] The speaker then takes those numbers and reconstructs back out what [68:10] sounds to people like my voice. [68:13] The same thing is going on-- That's what the zeta function does for the prime numbers. [68:17] It takes the prime numbers, you can think of the prime numbers as particles of air molecules [68:21] wiggling around, and it does a signal processing analysis on it [68:28] that can recover the same prime numbers back. [68:31] And the frequencies and the amplitudes correspond to the [68:35] zeros of the zeta function. The zeta function has that information in it. [68:39] And the Riemann hypothesis, so the amplitudes [68:43] of all of these, you can call them Riemann waves, [68:48] and the superposition of those waves recovers the distribution of the primes. [68:52] And all of those amplitudes are supposed to be exactly [68:56] the same. Like, the orchestra that plays the, [69:00] the music of the prime numbers has every single instrument sounding at [69:04] exactly the same amplitude. That's what the Riemann hypothesis predicts. [69:09] - So can you-- Actually, we know what the prime numbers are, but [69:13] maybe going into the space of philosophy and the romantic [69:17] notions of aesthetics why are they so interesting [69:21] and important and central and beautiful at the core of mathematics? And, [69:28] connected to why the Riemann hypothesis is exciting is why is it so [69:32] difficult to figure out if and what the pattern is in the prime numbers? [69:37] - Let me start with your second question first. Why something is difficult [69:41] I only know how to answer once I know what the answer is. [69:45] - Once it's made simple. [69:45] - Right? Maybe it's not difficult. [69:46] - Okay. [69:47] - Maybe it's not as difficult as we thought it was. I did not think I would see a proof of the [69:50] quasi-Riemann hypothesis in my lifetime. So the fact that I saw it on Tuesday, [69:56] was extraordinarily shocking. [69:59] - What did you do the moment you saw it, like, in 30 [70:03] seconds after? Like, what was your-- Do you remember, like, what- [70:07] - Yeah, I remember where I was- ... the second I saw the drop. So we [70:11] were at this National Museum of Mathematics Gala- [70:14] ... in New York City. And so I had, you know, a whiskey in one hand, and, [70:18] I kept refreshing. I was talking to people, socializing, but I also kept [70:22] refreshing 'cause I knew it was dropping around six o'clock Eastern Time. [70:27] And then the drop hit and, in fact, I was talking to [70:31] there was a Quanta reporter there and we were just chatting, "Oh, yeah, they're gonna [70:35] drop these, you know, 300, 400," we didn't know how many, 700 it [70:39] turns out problems. And I was like, "Oh, wow, that's really cool. It [70:42] dropped." So I'm looking at my phone, and, like, the second thing I see is [70:46] quasi-RH. And I'm like, "What do you mean, [70:50] quasi-RH? You don't mean, like... For what?" And then I look, and it's, [70:55] it's the zeta function, and it's all Dirichlet functions, so they've solved no zero, zeros. [70:59] And I'm like, that can't, that can't be what's in this drop. That just-- I, [71:08] I could not compute- ... in that moment what I was seeing. [71:11] - That speed of progress in that little time, yeah. [71:14] - It was so far outside my model of what was possible. [71:20] And this is-- It's been a month of things happening way outside [71:24] of my model of what was possible. The Fermat's Last Theorem formalization, [71:29] I was sure we were three years away, at least, from being able to do [71:32] something like that. [71:33] - So, maybe you can elaborate on that. It was in Lean, it was formalized [71:37] as... How big was the Fermat's Last Theorem in Lean? [71:41] - Like, 13 million lines. [71:42] - 13 million lines. Okay. [71:43] - So I just-- Two days before, I gave a colloquium in the Princeton Math Department where [71:47] I explained in painstaking detail what Lean is, what Mathlib is, but also [71:51] that AI systems today, you know, there's this Mathlib library [71:55] where everything is kind of perfectly done, exactly the right levels of generality, enough-- [71:59] There's no-- There's not supposed to be any code that's reusing [72:04] code that's reproving theorems somewhere else. Everything is the most general, [72:08] the most abstract, and what AI systems are capable of doing is building, like, one [72:12] or two layers on top of that. But they're, what they're doing is jerry-rigging. They have [72:16] not been trained yet, I'm not saying they can't do it, but [72:20] they're not good at it right now, at what I call canonization, which is [72:23] taking kind of sloppy code, refactoring it, figuring out exactly where in [72:27] the library it belongs, what's the canonical version of this statement, of this [72:31] lemma, and then everything, every proof should just be a combination [72:35] of you apply this lemma from over here, you apply that lemma from over there, put all these [72:39] things together, there's your theorem. And then it's really clean, it's really beautiful, very elegant. [72:43] That's what Mathlib is going for. And what I said in my colloquium is, where we [72:49] are today is that you can get entire textbooks formalized as long as they're in the Mathlib [72:53] halo. [72:54] As long as they're, like, one, maybe two steps removed from things that have already been [72:58] done in Mathlib. But when you start, and this had been my practice, my experience had been, [73:02] when you start asking it to do things that are, you know, three or four steps removed from, [73:07] the library, the way that it came up with a definition of [73:11] this object and the way that it came up with some other definition of the same object, 'cause it was doing it [73:15] for two different purposes, then they clash. You can't stitch these things back [73:18] together. Like, you really need the library, Mathlib, to grow- [73:22] ... in order to get to a place where you can [73:26] do research manuscripts, you know that are, that are really [73:30] using, synthesizing tons of different areas of mathematics. [73:33] - Which is why you would expect Fermat's last theorem to take-- because it's so far [73:37] outside of the halo of Mathlib, it would take quite a bit of time. [73:40] - Exactly right. So Kevin Buzzard had a million dollar [73:44] grant to spend the next five years, or I th- maybe he's already a year or two into that [73:48] grant, to reduce the formalization of Fermat's last theorem [73:52] to things that were known in the 1980s. [73:54] So he wanted to do that last step, the, those seven years of Wiles' work, [73:58] and Wiles Taylor, to finally finish you know, Fermat's [74:02] last theorem, but built on top of things that we knew would be extraordinarily [74:06] difficult. He wasn't gonna do Langlands' Tunnell, he wasn't gonna do Mazur. There were, there were all kinds of things [74:10] that were expected to be even much more difficult than what these five years would [74:14] take. And everything I said in that colloquium was wrong. [74:19] And I think my colloquium was on a Wednesday. On Friday, there was a drop of thirteen million lines [74:23] of code from the axioms, Fermat's last theorem is proved. [74:27] The statement of Fermat's Last Theorem is completely elementary. There's no semantic alignment to [74:31] check. You just glance at it. Yes, that's the statement. It's-- [74:35] Lean says it's sorry-free. [74:36] - Can you explain how it was done? [74:38] - How it was done. So it was Anthropic, and how it was done was AI [74:42] systems just got really, really good at-- It's still not a nice proof. It's not the [74:46] clean proof. It's not-- There's still a ton of work that needs to be done to make that proof a [74:50] Mathlib quality proof, and I think that's important work and work that we want to [74:54] do. But in terms of just getting that sorry-free theorem, they [74:58] just blasted through all of these. They jerry-rigged this, they jerry-rigged that, [75:02] and, you know, thirteen million lines later, they had a [75:06] sorry-free formal Lean proof of Fermat's Last Theorem. It's just absolutely [75:09] shocking. [75:10] - By the way, it's beautiful to always, um-- Because of how [75:14] rapid everything is, there's going to be a lot of statements that everybody makes, [75:18] the world experts make, that are wrong. That's the whole point of it. And that's the [75:22] fun thing about human discussion, is we have disagreements and I would [75:27] venture to say somewhere between forty to eighty percent of everything said [75:31] today will be wrong in five years because everything's changing s- [75:35] so fast. But that's the whole point of human discussion and deliberation and [75:39] discourse and the, even the scientific process is saying [75:43] shit and seeing what sticks, Over time. [75:47] Over ten years, over a hundred years, and so on. [75:49] - I was describing my experience. I had a lot of experience with Lean and, and [75:53] with using AI. I started doing Lean before AI existed in the, in [75:57] the sense that we understand it now, and it was very painstaking. And the second, [76:01] uh, AI systems came on, you know, the first LLMs were coming online, they were, they were crap at [76:05] Lean, but then they got a little bit better, a little bit better, and I was like, "Oh my God, I can do library [76:09] search." [76:09] I don't have to sit there looking, I know there's a lemma that does this, but I don't know what it's called, and I don't [76:13] know which exactly. Is it in the manifolds library? Is it in the [76:17] algebraic geometry library? Like, give me this, give me this lemmas. It was great at that, and I was [76:21] like, "Oh, this is amazing. I'm gonna keep using these systems." And every bit of my experience [76:25] told me that they can do, like, X, but not ten X, and this was a million X. [76:32] It was just completely outside of the distribution in my mind of what systems [76:36] were capable of. [76:37] - Do you have a ballpark estimate of how many humans were involved in that, [76:41] and, like, what is the level of human involvement in generating those thirteen million lines of [76:45] code? [76:46] - Uh, I believe it was two or three humans. [76:48] - Okay. [76:48] - And I believe most of their involvement was, "Keep going," all caps. [76:54] - Okay. All right, so you think it's really just, like gentle guidance, not like, [76:58] you know, pathway creating and- [77:01] - As far as I understand, there was no domain expertise- [77:04] - Okay [77:04] - ... on the parts of the people who were doing this. [77:08] - As the meme goes with OpenAI or Anthropic you know, the prompt is solve math- [77:13] ... or something. And then- [77:15] - Do a breakthrough. [77:16] - Yeah, yeah. Yeah, do a breakthrough. [77:17] - Do a breakthrough. That's the prompt. And it goes and does it. [77:21] - Of course, you never know from a company perspective, there's a lot of incentive [77:25] to sort of sell it as that, even though there might have been a lot of [77:30] intermediate steps that led whe- at any stages of the training or maybe some things [77:38] left unsaid, even though the final prompt is do a breakthrough, there could be a lot [77:41] of groundwork before that, that led to, we don't know. [77:45] - So I remember that was a Friday because I just was completely, [77:49] like my mind was completely exploded, and it was Labor Day, [77:53] and we just took the kids to the beach, and I needed a few days of, you know, I'm [77:57] not checking my phone, I'm not on Twitter. I'm just throwing, [78:01] you know, throwing my kids into the ocean and, and- [78:04] - Did you have that moment where you're just sitting and looking at the ocean and contemplating? [78:08] - Yeah. Like, what does it all mean? [78:11] So I had already gone through that a month before. [78:14] And I was like, I-- So now what that says is that [78:17] autoformalization is basically solved. [78:19] It's not solved in the way that I would want it to be solved. I want-- [78:23] And people are still gonna work very, very hard on this. [78:27] There's all kinds of other things that happen, just like there's all kinds of other things that happen when you prove [78:31] theorems that are byproducts of that process for human [78:34] understanding. There's all kinds of other things when you work on making a beautiful [78:38] library of mathematics, and that process will continue. That process will be [78:42] very difficult. It's gonna be the, you know, thousands of man-hours of people [78:46] around the world contributing in this open source way. But in terms [78:50] of can you get a sorry-free proof of something that human beings have written down [78:54] on paper, I think that problem is done. [78:58] - And so you were standing there with a whiskey, having gone through this a month before, [79:02] and now again. [79:03] - Right. This was round three. [79:05] - This is round three. [79:06] - 'Cause when I got back from Labor Day, it was Navier-Stokes. [79:08] - Navier-Stokes. [79:10] - But this one- [79:10] - So I was like, "I was just mourning FLT. You're gonna do this to me? Come on." [79:13] - But now, I mean, this really hit home, right? With, with the quasi-RH. [79:19] Um, so maybe can you speak to the, the importance and the beauty of the [79:23] primes, and also how big and hard the leap is from quasi-RH to full RH? [79:30] Yeah, so the primes, the reason the primes are so valuable is [79:34] that they are the fundamental particles in the theory of [79:38] multiplication. Every whole number is broken down into a product of [79:42] primes. You know, we know about the elementary particles [79:46] in physics, the, and all of matter is made up of that. In the same way, [79:50] the primes are the particles that make up the multiplicative structure of whole numbers. [79:54] And their distribution had been a mystery, and it remains a mystery. Even the Riemann hypothesis doesn't [79:58] solve everything we wanna know about prime numbers. [80:01] But the Riemann hypothesis was so beautiful because it pointed the way [80:05] to what the distribution should be, and it has a [80:08] thousand applications in very faraway fields. If [80:12] only the Riemann hypothesis were true, then we would know XYZ. [80:15] - So let me interject a little story that came up last week, which was [80:19] pretty entertaining. So we had Grant Sanderson was visiting IAS, and we had a [80:23] long discussion about what is AI doing to the math community. [80:27] And I really enjoyed this comment that Camillo De Lellis made, which was, [80:32] I'll do my best impression. I can't do the accent. [80:35] But there was a play written in the 1950s by a mathematician. [80:40] So who knows that mathematicians are writing plays, but apparently this [80:44] play, the premise of it was that Von Neumann died, and he went to heaven. [80:49] And he met God, and he got to ask him a question. So he asked him, [80:52] "Is the Riemann hypothesis true?" And of course, everyone holds their [80:56] breath as they're waiting to hear the answer. And God says, "Yes." [81:01] So finally, everyone can exhale. And then Von Neumann asks, [81:04] "But how do you know?" And God thinks and says [81:08] "Well, I know where all the zeros are, and they're all in the R equals a half [81:12] line." So my understanding is the premise of the play is that, [81:16] that's not enough for Von Neumann. He gets a little bit agitated to [81:20] try to find out more about why it's true, and I think he ends up at the end in the [81:24] other place. [81:26] - That's great. [81:27] - So I don't know what to draw as a moral from it, but, um- [81:30] ... it seems to speak to the mathematical experience. [81:33] - Math is not the fact, it's the why. [81:35] It's always been the why. It's our deepest quest for understanding. [81:39] And if you just tell me a fact is true, but I don't understand why it's true, that has [81:43] contributed zero to my knowledge. [81:46] - So primes, I think, are also interesting because they relate to [81:50] things that a three-year-old can understand. Like fundamentally, we know how to add [81:55] one to numbers, and you know, there's this breakthrough moment when you're [81:59] a kid, I can add one forever. That's a big idea. I can keep adding one forever. [82:05] Then you learn how to multiply and group, and that's also a big idea. [82:10] And primes are interesting to me, and have always been interesting, [82:14] because we don't understand how adding and multiplying work [82:18] together. The two most fundamental, basic, simple ideas in all of [82:21] mathematics, we don't know how they work together. I can add one to a number that I [82:25] can factor, and I have no idea what happens. I know if it's even, it becomes odd [82:31] and vice versa, but I don't know if a number that has many, many [82:34] composites becomes prime. We just fundamentally don't know [82:39] that simple, you know, adding one, how that changes multiplication. [82:43] And primes are basically, you know, this like, the [82:46] distillation of that three-year-old mystery, and it's a mystery that all of [82:50] us still wrestle with. [82:53] You know, that would be the interesting bit, is if the proof told me something [82:57] fundamental about that structure that I could actually use. [82:59] - Yeah. To the three-year-old, I think my older one was something like two or [83:03] three. He was playing-- We had these little blocks, you know, cubes that kids like to play with. [83:07] And uh, there was like 20 of them in the pack, and so he would make like a little four by [83:11] five grid, and he liked that, and he would make a 10 by two [83:16] grid, and that was kinda cool. And then one day, I look over and he's like really frustrated. [83:20] I'm like, "What's going on?" He's like, "You know, it's not, it's not working." [83:25] And one of them had rolled under the couch. And he only had 19. [83:28] And he couldn't figure out a way to make a nice-looking rectangle out of the 19. [83:32] - Yep. It still bothers me today. [83:34] - So... Yeah. So absolutely, kids love primes. Kids can- [83:38] ... discover primes. It's the non-re- [83:39] - And so the solution is to lose another, right? [83:42] - Yeah. Well, exactly. [83:43] - That's right. [83:43] - Yes. [83:44] - Do you think so the leap between quasi and RH, I [83:48] presume you think humans will not, in your lifetime, be able to [83:52] solve without AI's help, the full Riemann Hypothesis? [83:55] - I don't know. [83:56] - Can AI, I guess, is the follow-on question. Like, how hard is it? [83:59] - I don't know. I don't know. I did not think I would see a proof of quasi-RH. [84:04] Now that I see what the proof is, and there are a lot of technical details that I [84:08] still have to go through for my own satisfaction. There's a Lean formalization. [84:12] The Lean statement is, was already in the library, was already in Mathlib, so the [84:16] fact that they have a sorry-free proof gives me 100% confidence that the [84:20] proof is right. And moreover, having read the proof, you know, at a high level, [84:24] the ideas that it's using are all completely valid. They're brilliant. You know, [84:28] to embed the Möbius function, twist it by sextic [84:32] characters. This is all stuff that's the bread and butter of analytic number theorists [84:36] that was chained in just the right way and was absolutely incredible. And [84:41] so what Riemann showed is that remember we were talking about this [84:44] function that has complex inputs and has these outputs, and we're worried about [84:48] when the output is zero. So he showed that there's what's called the critical strip, [84:52] which is a vertical infinite strip in the complex plane, where the [84:56] real part is-- So you can think about the X coordinate lies between zero and [85:00] one, and the imaginary part, the Y coordinate, can go arbitrarily high or arbitrarily low. [85:04] And the hypothesis is that the zeros all lie right at the dead center of [85:08] that strip, where the real part is a half. That's exactly what Ankur was just [85:12] saying. And what we knew until Tuesday [85:16] was region where there are no zeros of the zeta function, [85:20] that's like getting thinner and thinner the higher up you go. It goes towards [85:24] the one line. The one line is something called the prime number theorem, which tells you the basic [85:28] distribution of the prime numbers is equivalent to the fact that there are no [85:31] zeros of this function on the line real part of S equals one. [85:36] So if the X coordinate is one- As you go up and down that line, you will never find a [85:40] value where when you apply the zeta function, it hits zero. [85:44] Okay? So we have this zero-free region that was getting thinner and thinner the [85:48] higher up you go, [85:50] and that gave us the best known distribution on the prime numbers, the best [85:54] error term in the prime number theorem, which I can go into if you-- You tell me how much math you [85:58] wanna talk about. [85:59] - Let's go. [86:00] - But what I could have anticipated is that it would [86:04] thicken that line a little bit as you, you know, it would still be going to [86:07] zero. The thickness would still be going to zero as you go up the [86:11] imaginary axis, but it would be better than what we currently know. [86:15] That thickness hasn't moved in seventy years, but I could sort of [86:19] imagine that it could maybe make some progress. What it did is it gave us a vertical [86:23] strip, so there are no zeros with real part between seven-eighths [86:27] and one, an entire infinite vertical strip. That's a quasi-Riemann hypothesis, [86:31] and if you can get that seven-eighths down to a half, that's the full Riemann hypothesis. [86:34] Now, the methods that it used in the seven-eighths proof, and there's a second proof that's about [86:38] eleven-twelfths, never mind these technical details, that method [86:43] sort of fundamentally can't get down to one half. [86:46] - So a new idea is needed. [86:47] - You need a new idea, but quasi-RH needed lots and lots of [86:51] new ideas, and it came up with that. [86:53] - Is your-- Having gone through this three times already, is your intuition, this is for [86:57] all three of you, about remeasuring in your mind the hardness [87:01] of the various problems? 'Cause you mentioned, like, picking easy things. [87:04] 'Cause that presumably you have a good hardness radar, like, this problem [87:11] is probably solvable, and this is not solvable today, [87:15] solvable this week, solvable this month, you know. Are you readjusting [87:20] to this new world and your- [87:21] - I'm readjusting my inability to predict it. [87:23] - Yes. Exactly. Well, you had the ability, no? It's noisy. [87:29] - Sure. It's a larger amount of noise than I thought. [87:33] I mean, I think we've always known that there are problems we care about that are within [87:37] reach of the tools that are out there right now, but it's just surprising the scale [87:42] at which, you know, how many different problems really fall into that category. [87:48] So we also don't know a lot about, you know, apparently, [87:52] they prompted thousands of problems, and what they dropped were only the [87:55] positive solutions, that they only had about a ten percent success rate. [88:01] So we're sort of left guessing, you know, is there a very jagged boundary [88:05] that we could squint at and say we can make sense out of that [88:08] distinguishes you know, what did fall versus what didn't fall? [88:13] I think one of the other dangers is that maybe the problems which are the safest are ones [88:17] which you look at and it's just painfully obvious you need a [88:21] really creative new idea. [88:24] You know, because you can sort of tell that maybe there's a barrier and the [88:28] things that are out there in the literature just don't seem to fit into that hole. [88:32] But now I think one of the things that's changed the landscape of math forever is, you know, [88:36] maybe people become a little bit more secretive with those ideas than they were in [88:40] the past. You know, because once those ideas are out there, then, [88:45] you know, it might be a lot more straightforward for AI to be able to [88:49] balance that one new idea, that missing ingredient, that new tile [88:53] it needs, and be able to fully exhaustively explore what it can [88:56] reach out of that. So you might be a lot more protective of your ideas than you [89:00] were in the past. I think that's also a bit scary because [89:04] traditionally, the mathematics community has been really open with [89:07] sharing ideas, and that would be something bad to lose. [89:12] - But that's also because we had this first through the gate mentality, and I think [89:16] that's also gonna change. [89:17] - That's right. [89:18] - I would prefer that the openness stay and the first through the gate is what goes. [89:21] - That's right, but that'll take time. [89:22] - It'll take time. [89:23] - Yeah. So that could be an effect, you know, that's more transient in nature, [89:27] that if we can learn how to value other things, then maybe that'll go away. [89:31] There are other communities that have models for how to value contribution, even when it's [89:35] not lionizing one individual's contribution. So [89:39] it's not that there's not a way forward, but, you know, that is, in the short term, [89:43] a real roadblock to doing mathematics the way that we used to [89:47] and enjoying it the same way that we used to. [89:49] - So speaking of which, Steve, you told us a story about [89:53] soup- ... that made you maybe fall in love with the bigger problem of [89:57] fluid dynamics. So going from math to applied math to engineering, the [90:03] full thing. So maybe can you talk about Navier-Stokes, these set of equations that [90:09] explain fluids, why it's such a big deal, why it's beautiful? [90:13] Similar as we talked about primes, why is it so hard, and why is [90:17] it such a big deal that Navier-Stokes was proven? [90:22] - So fundamentally, the Navier-Stokes equations [90:26] is just a way of stating the conservation of momentum [90:30] for fluids, for objects that can move kind of freely, like [90:34] gases and liquids. So we know F equals ma is [90:37] conservation of momentum, you know, classical Newtonian physics. [90:41] The same basic principle applies to the gas in this room, to [90:45] water flowing, and it's true in any volume. So it's [90:49] true in little volumes, big volumes, kind of imaginary volumes. [90:54] And if you write down that conservation of momentum in a volume, [90:58] and then you kind of take the limit as that volume becomes arbitrary [91:02] and small, you get these Navier-Stokes equations. It's a differential equation, [91:05] a partial differential equation that describes how the [91:09] velocity field of a fluid changes in space and time, [91:14] and also the pressure related to the forces acting on that fluid. [91:18] - Can you give intuition about why fluid, given that F equals [91:22] MA is a pretty simple equation, why fluids are such complicated systems? [91:28] - F equals MA is simple to write down, but it can get [91:32] complicated really, really quickly. So if I have, like, a single pendulum, it's [91:36] really easy to write down the equations for that single pendulum. If I put, you know, another [91:40] pendulum on the end of it, now I have a more complicated equation, and it gives [91:43] chaos. Fluids is, you know, in a sense, if I [91:49] try to model the air in this room, I could chop the room up into a billion [91:53] little boxes and write down a little differential equation for how the [91:57] momentum works in all of those billion little boxes, but now I have a [92:01] really complicated system of equations. And so it's [92:04] simple to write down, it's simple to conceptually understand what the equation is [92:08] saying and representing, [92:11] But actually solving it and understanding how the fluid actually [92:15] moves in any kind of practical sense is very, very complicated. [92:18] - I read something like, you know, like one space where, [92:22] fluid dynamics is important is reducing drag for when you're flying, for airplanes. [92:28] Uh, I read something like airlines could save like, [92:32] something like tens of billions of dollars by reducing the drag by one percent. [92:36] So why is it so hard? So that's one. And drag is [92:40] maybe easier than something like turbulence. [92:44] So like why is, why is these things that are worth so much money so difficult? [92:47] - So the good news is, is that if you can improve our ability to predict [92:52] turbulence, to simulate it even a little bit better, faster more [92:56] accurately, there are, you know, dozens of fields [93:00] where you can have a huge impact, both economical [93:03] ecological, human health and safety, with a [93:07] relatively small improvement. So drag reduction on airplanes, [93:11] better artificial hearts, you know, more efficient wind [93:15] turbines and gas turbines and, you know, you name it. The world is-- [93:20] We live and work in a fluid and so do all of our machines. [93:25] The inverse design problem, I know it's actually very closely [93:29] related to this kind of like math problem. I know what I want to prove. I don't know how to get there. [93:33] Uh, I know I want a wing that has high lift and low drag, [93:37] but I don't know how changing the geometry, changing the shape of that [93:41] wing will affect this whole cascade of smaller and smaller and [93:45] smaller vortices that, you know, this turbulent fluid that generates over the wing [93:50] that eventually results in its lift and drag. I don't know how to [93:54] inverse design efficiently. [93:57] Part of it's because the equations are expensive to simulate or to test. You [94:01] know, wind tunnels are expensive. You need supercomputers to simulate these things. [94:07] This is actually kind of related to pure math. We'll, we'll get back to this, I think. [94:10] But, there's this really nice Feynman quote that, you know, [94:15] we know how to write down [94:17] these very general equations, Maxwell's equations, Navier-Stokes equations, Schrödinger's [94:20] equations, but we currently don't know from the equation what [94:26] phenomena is possible. You know, if you'd never visited Earth, could you have [94:30] possibly predicted the sunsets and the tides and all the phenomena just [94:33] from like a partial differential equation? Like, we're not even close [94:37] to there. And there's some very fundamental math questions about, you know, if I can-- I can write [94:41] down a differential equation. I can't tell you what's inside of it. We still [94:45] have to simulate, we still have to test, we still have to do this very elaborate [94:49] engineering process to understand how these equations [94:53] manifest drag on an airplane wing or turbulence inside of an artificial heart. [94:59] - Sorry, can I just ask a question on that? I've always wanted... So like, in the case of [95:02] Einstein's theory of general relativity, he writes down these equations. A few years later, [95:06] Schwarzschild finds a blowup, and they're like, "Oh, come on. [95:11] So what's, what's wrong with these equations that there's a blowup? You know, if you get enough matter into [95:15] a small enough space, then like light can't escape it. This-- We don't have any [95:18] physical experience for this. This has got to be a mistake in the equations." [95:23] Until we discover black holes. And then it turns out that [95:27] actually the theory perfectly explains you know, and predicts what, [95:31] what's supposed to happen. In the case of blowup of Navier-Stokes, are we going to [95:35] discover something new about water, or is the fact that at [95:39] some scale you get to particles, you know, you get to an atomic structure that [95:43] does not follow the liquid equations anymore? [95:46] Why would we have predicted that there should be [95:51] solutions to a continuous process that's allowed to be pushed to its limits? [95:56] - Yeah, it's a great question. Um, I'm gonna give you the engineering answer, which [96:00] is these equations are an exceptionally good model [96:04] of how fluids work at a certain scale. [96:06] And you're absolutely right. Like you need some number of individual [96:10] molecules to be averaged over in a volume for these to [96:14] be constitutive equations that are valid. And so if you zoom in [96:18] small enough, you know, now we're talking about individual [96:22] molecules, and Navier-Stokes breaks down. So, [96:26] The mathematical problem of these equations blowing up is [96:30] interesting from a mathematical standpoint. [96:33] By the time they blow up, it stops being a fluid. It's like- [96:37] - There's no black holes. [96:38] - There's no black holes. Like other physics takes over. [96:41] - But it's poss-- Wait, I think it's interesting from a philosophy of science [96:44] perspective that where equations break down could, and maybe [96:48] statistically high likelihood, is a window, a door [96:52] in a breakthrough of needing some other discovery. [96:57] Like Newtonian physics breaking down needs Einstein. [97:03] - Right. And the grand unification is that- [97:06] ... we don't have gravity at the same scales as we have, you know, interactions of, [97:10] subatomic particles. [97:12] - But this idea that [97:14] we don't know something so fundamental about these equations, there are [97:18] so many people on Earth right now simulating and testing [97:22] Navier-Stokes in every industry imaginable. I mean, it's [97:26] probably one of the most studied and analyzed and used equations [97:30] in the entire engineering world. [97:34] And we don't know this fundamental property of the PDE. Does it blow up in finite time or [97:38] not? That's like the zeroth order version of the bigger [97:42] question is, you know, what other kinds of solutions does it admit? Does [97:46] it, you know-- Are there sunsets and hurricanes and tornadoes and all the [97:49] phenomenology that could exist? You know, existence and [97:53] uniqueness is like the-- The simplest baby step. And that's where it gets [97:57] interesting to me is even if there's not [98:00] a direct tangible use of this mathematical result [98:04] in an engineering sense, it gets us that one step closer to [98:08] understanding the nature of these equations and what is possible, which is [98:12] very fundamentally interesting to us. [98:14] - Could there be-- could this solution that has the blow up be used somehow in [98:18] a engineering application and, you know, you wanna drive it into this state and [98:22] get some funny behavior that- ... helps airplanes fly with less drag or something? [98:26] - Yeah, I mean, I think in principle, a huge industry, a huge part of what we try [98:30] to do is manipulate fluid flows, [98:34] like bend them into our will, right? So if you watch a bird [98:38] fly or an insect fly, they are shaping vortices [98:42] and carefully funneling, you know, energy into structures [98:46] for their, you know, objectives of their daily life to, like, perch on a rock or, [98:50] you know, hunt. And we try to do the same thing. So flow control is one of the [98:56] highest challenges. It's very, very challenging. Um, [98:59] but all of our science fiction technology [99:03] would be kind of accelerated if we had better ability to control these [99:07] flows. And this proof is one [99:10] very interesting construction of this kind of, you know, very specialized flow [99:14] control in a way. [99:16] - I'm reminded that Feynman was asked about intelligence, artificial intelligence in the [99:20] '70s. And he said something like, "The way that we solve problems with [99:25] technology is radically different from how nature solves these problems. [99:29] Look at birds, how birds fly and how airplanes fly." He was exactly pointing to this. [99:33] And it's kind of interesting to think back to that comment now when we see [99:37] what LLMs can do with theorem proving versus what human brains [99:41] can do with theorem proving and all kinds of other applications. But it's- [99:44] ... it's interesting that he pointed exactly to this issue of how, [99:47] how we've solved flight using technology and how different that [99:51] is from nature's solution to it. Our airplanes don't flap. [99:55] - Now I wonder if birds have existential crises when they see planes flying. [99:59] - Exactly. [100:00] - What am I even here for? [100:01] - Yeah. Yeah. To get the worm. What [100:07] what are the different components of the Navier-Stokes equation? Can you-- if you-- can you dig in? [100:11] Like, what does it take to describe fluid? [100:16] - Yeah. So very, very basic a fluid moves, and it moves differently in different [100:23] places and at different points in time. And so [100:27] the kind of basic building block of a fluid is the velocity field. [100:32] So at every point in space, there is a velocity vector that tells you how that, [100:36] that fluid is moving. You know, maybe it's really fast here and slow here, [100:40] and it creates a vortex. Um, and it gets more interesting when the fluid velocity [100:47] changes a lot over space. So like a tornado is a very rapidly changing [100:54] fluid velocity field in both space and time. [100:59] There's pressures that are involved. So again, like the core of a [101:03] tornado is low pressure. So pressures are, um, [101:07] kind of forcing this velocity field and causing it to [101:11] move in the way it does. That's the building block is a [101:15] velocity field and its pressures. And then the equations tell you how that velocity [101:21] changes in time and in space according to [101:25] kind of its own momentum. So fluid has momentum to carry itself. [101:29] That's, you know, gusts of wind. It has internal, [101:33] friction and resistance. That's viscosity. So honey has more internal [101:38] resistance than water, has more internal resistance than air. Um, [101:41] and then all of the external forces like heating and pressure [101:45] and gravity and things like that. So each of those can be quantified as a [101:49] term in this, this larger Navier-Stokes equation. [101:52] - So speaking of external forces, can you describe the math side of this, [101:57] of the Navier-Stokes with the blow up? [101:59] What was the formulation of the problem, and what was actually proven, and why [102:03] was it so hard? [102:05] - If you don't have that internal friction, that viscosity, [102:10] we have a strong suspicion that the equations can and will blow [102:14] up under certain conditions. We have other examples, kind of simpler [102:18] problems that are like Navier-Stokes, that do blow up. So the simplest one is just a [102:25] simple ordinary differential equation, if you don't mind me taking a little tangent. [102:28] - Let's go. [102:29] - So X dot equals X squared. X is the state of my [102:32] system. X dot just means the rate of change of X in time. [102:37] And if that rate of change of X is growing proportional to [102:41] X squared, that differential equation does blow up in finite [102:45] time. So it just takes a quadratic nonlinearity. Um, [102:49] think about if I have, you know, X dot equals X. This is-- like [102:53] X could be the population of bunnies, and we know bunnies like to [102:57] populate. X dot equals X grows exponentially. You know, [103:01] the state of my bunny population in time will grow exponentially. It'll [103:05] get bigger and bigger and bigger. It'll double and double and double. [103:08] But it won't reach infinity until infinite time has passed. [103:12] That's just a normal exponential growth. That's not finite time blow up. That's X [103:16] dot equals X. But if I take X dot equals X squared, that blows up in finite time. [103:24] So that quadratic nonlinearity, that quadratic kind of feedback is enough to [103:28] cause this equation to blow up. And you can, like-- if you write this down you [103:32] know, you can solve this using kind of classic basic calculus techniques, and you [103:36] get this finite time blow up. And so there's a progression of partial differential [103:40] equations that have the same behavior where a quadratic nonlinearity-- And that's really [103:43] interesting is the Navier-Stokes equations, [103:47] one of the terms, the con-- uh, the convective term that tells how the fluid kind of moves [103:51] itself with its own momentum, is a quadratic nonlinearity, [103:55] just like X dot equals X squared. It's kind of this self-motion. Um- [104:02] Without diffusion, without viscosity, we [104:05] think that that should probably blow up to infinity in finite [104:09] time, have this finite time blow up. But we don't know if [104:13] viscosity, if the kind of, you know, internal friction and resistance like honey [104:17] has, always damps out those blowups before they can happen. My intuition as [104:22] an engineer is of course they do. Like, if I have a flow where it's, [104:27] you know, accelerating, you know, to infinity, [104:31] that should immediately diffuse into its neighbors. It should create this, [104:35] like, diffusion into its, its neighboring fluid and kind of tamp down [104:39] that, that growth. And so I think the math problem is essentially [104:44] are there any edge cases, any initial configurations of the fluid where you [104:48] perfectly orchestrate this thing so that it could actually [104:51] accelerate to infinity in finite time? [104:54] And then there's two kind of technical versions of this. One is [104:58] with no external forcing, and one is with smooth external [105:02] forcing. Smooth being important because if the external forcing has [105:06] singularities, then it's kind of trivial. [105:09] Um, like if I force it with a tornado, you know, an infinitely tight tornado, then of [105:12] course it's gonna blow up. [105:14] - The problem as stated has smooth external forcing. [105:19] - There's four versions of the- [105:20] ... on, on the Clay, yeah, website. One of them being the, the forced version. [105:24] - And that's the one that was proven. [105:26] - And that's the one that was proven. [105:27] - Exactly what was proven and what are some interesting things about the proof [105:31] that you saw? [105:33] - Yeah. So what is proven is that there is this way of [105:38] orchestrating a smooth kind of external forcing that [105:42] perfectly cancels out various terms in the equation, so that you get this [105:49] vortex. This can only happen in three dimensions. This is really important, is that- [105:52] - We should probably say that we're, we've been talking about three dimensions. There's a big difference [105:56] between 3D and 2D. [105:57] - Yeah. So I mean, when we talk about a fluid, we're, we usually imagine it in 3D, [106:01] but there are these kind of analogies or, or simplified 2D flows. [106:05] Weather on Earth is pretty well approximated as a 2D flow. [106:09] - Yeah, you've mentioned somewhere in a talk that I heard that [106:13] hurricanes are kind of like modeled well in 2D and tornadoes are 3D, [106:20] and I remember, I remember different sort of analogies that you were drawing there. [106:24] - Yeah. So, so one of the interesting phenomena in three-dimensional turbulence is you [106:28] generally have this, this breakdown of big vortices into smaller [106:32] vortices, into little itty bitty vortices until eventually they [106:36] molecularly diffuse into heat. [106:38] That's kind of this three-dimensional turbulence cascade. [106:42] In two dimensions, like weather on Earth, you actually get a reverse [106:46] cascade, which is why hurricanes form. You actually have little vortices [106:50] coalescing into medium vortices coalescing into bigger vortices. But so the AI proof [106:57] of this kind of counterexample that you can have finite blowup basically [107:01] creates this very, very specialized vortex [107:05] that in the third dimension is stretching out and thinning it [107:09] so that it can essentially become, uh, [107:14] infinitely thin. The radius gets smaller and smaller and smaller [107:18] as it stretches out, [107:20] Kind of like a figure skater doing a spin. You pull your arms in, [107:24] you go much, much faster. And there's this very specialized kind of [107:28] rate of contraction of this vortex so that the energy in the fluid still remains [107:32] finite, but at this localized filament the [107:36] flow kind of blows up and you get infinite vorticity. I was surprised when [107:41] mathematicians before the AI proof started thinking it was going to [107:45] be possible to find this counterexample. [107:47] - Which only started maybe 15 years ago. [107:49] - That's right. It's, like, relatively recent- [107:51] ... in the grand scheme of things. Yep. [107:52] - Instead of looking for a solution, looking for a counterexample, yeah. [107:55] - Yep. 'Cause all of our physical intuition tells us diffusion does [107:59] solve the problem, and so it is, like, very physically counterintuitive. [108:03] - In the mathematics world, that's a really hard problem. Impressive [108:07] to construct a counterexample. [108:09] What does it teach to you as an engineer who's stared and [108:13] studied at a lot of different natural systems that are, [108:17] That are fluids? And maybe take that as a step towards what are [108:25] the equivalent kind of hardness and harder problems yet to be solved in fluids. [108:31] - So it's a very important math result. [108:33] - Uh-oh. [108:34] - I'll say, like- [108:35] - But- [108:36] - ... day-to-day engineering fluids and applied math kind of analysis [108:39] of fluids is almost completely unchanged. Fluids [108:43] are just as hard to simulate and predict today as they [108:47] were a month ago or two months ago. And so [108:52] when I'm trying to actually solve an engineering problem with a fluid, I wanna design an [108:56] airplane wing or an artificial heart or whatever it is, [109:00] I'm still having to simulate and predict and manipulate this flow in [109:07] a very practical way that's expensive in terms of supercomputer time and [109:11] wind tunnel resources. And this mathematical theorem and [109:15] proof are very interesting, [109:18] but it operates at a regime that's so non-physical that it doesn't [109:22] really affect my day-to-day engineering. There's still-- I don't know, [109:26] this is very pithy, but I like to say you know, we solved the million-dollar [109:30] math problem. There's a trillion-dollar engineering problem, which is, like, how do [109:34] you predict these things in a fast, efficient way and do [109:38] this inverse design? That's a much harder problem. [109:40] - And you say it's a trillion dollar because that literally- [109:43] - Literally [109:43] - ... if you solve some of these problems, that's how much money you'll make, not 'cause it's a prize, [109:48] but, like, there's industries. [109:49] - Yeah. And we could say, you know, [109:51] trillion dollars of possible benefit to humanity of, like, [109:56] better, faster solutions, lower drag, less emissions, you know, all of it. It's, um [110:02] We have like transportation, defense, health, energy, all [110:06] of these have, you know, fluids at the very core technology. [110:10] And so I think those problems are still very hard. And this math [110:15] result is important in the sense I was talking about earlier, that [110:21] we don't know the fundamental character of what is possible in these [110:25] equations. We don't even know if the solutions exist. That's like so [110:29] fundamental. This is like a baby step towards a much bigger [110:33] human goal of understanding what's in an equation in a more practical sense. [110:38] - Can you speak to some of the maybe, beautiful, fascinating phenomena in the space of [110:45] fluids that you've been thinking about? You mentioned turbulence. [110:49] What, what are the big puzzles to sort of solve in this space to [110:52] understand? You've mentioned drag, like air, fluid. [110:57] What are like the big Millennium Prize type problems that people are thinking about? [111:03] - I mean, there's so many. You can break them down into a few categories. So turbulence is [111:07] very hard because it's multi-scale. [111:09] - What is turbulence? [111:11] - Turbulence is this phenomena that when you have a complex fluid [111:18] roiling or mixing and moving kind of like a Da [111:22] Vinci's-- He has these beautiful drawings of fluids at all of these different [111:26] scales. [111:28] All of the scales matter. The big vortices and the medium vortices and the little [111:32] vortices, all of them actually matter to predict the lift and the [111:36] drag and the thing. So the gas in this room is a great [111:41] counterexample. It's the opposite of turbulence. It is complicated. The [111:44] molecules are all moving around in a very frenetic way, [111:49] and there's a lot of molecules. [111:51] But that, you know, ten to the twenty-three number of gas particles, [111:57] all of those individual configurations don't matter. We know from [112:00] thermodynamics, this is like the great success of classical physics, [112:04] is that there's a small handful of numbers that actually determine how this gas [112:08] behaves, you know, temperature, pressure, density. [112:12] That simple description doesn't exist for turbulent fluids, for [112:16] the flow over a wing, the flow inside your heart, for, you know, any-- for the flow on a [112:20] river. [112:21] All of the scales actually matter. All of the numbers in that simulation actually [112:25] matter to predict the next, you know, behavior to predict what's gonna happen. [112:30] And that makes it fundamentally challenging. So this is actually a great AI problem, [112:34] is, is there a coordinate system? Is there a set of numbers just like we [112:40] have for gases, you know, temperature, pressure, density? Is there a set of numbers [112:44] or some representation of a fluid, of a turbulent fluid that [112:48] allow it to be simply described in a way that is useful for [112:51] engineering to build, you know, motors and engines and just like [112:55] we have for gas dynamics? [112:57] - So then the interesting question, if it's a problem for AI, and AI is [113:01] listening, I'm sure. Um, the solution to that problem, would it-- would the path [113:07] lie through mathematics or through having [113:11] the AI system sit there and watch some fluid for a while? [113:14] - I think yes to both. [113:15] - 'Cause fish, they figured it out. [113:18] - They figured it out. Well, so I think you have the right keyword as [113:22] coordinates. And I-- So that's kinda why I said both. And I, [113:25] I'm not trying to be like trite, but all of our, [113:29] our great physics breakthroughs were predated [113:33] by a coordinate transformation. So Newton's F equals ma came [113:38] after we put the sun at the center of the universe instead of the Earth, right? You had [113:42] to do that coordinate transform first. So I think what-- the way I see it [113:46] is we need to take all of this data, these simulations, these [113:49] observations of fluids, and what we're looking for is [113:55] a simple coordinate system so that [113:59] the description of that fluid becomes very simple and generalizable, [114:03] beautiful in a physical sense. And that is both a brute force you know, get a [114:09] lot of data, try a bunch of different coordinate transformations and, [114:14] and representations until you find the simplest one that generalizes the [114:17] best. But there's also a lot of deep mathematics in what the form of that [114:21] coordinate transformation might take. In general, if you [114:25] find a question that really smart people asked, [114:29] you know, in the '70s and '80s, but they hit a wall computationally, [114:33] that's very fertile ground right now. So like we know what the form of these [114:37] coordinate systems probably should take from some [114:41] fundamental principles. We've just never been able to write them down. They're too [114:45] complicated to write down. That's like a perfect problem [114:49] for AI is, you know, now let's give it a lot more data, [114:53] a lot more compute, and see if we can start learning these transformations we [114:57] know should exist. [114:58] - Just so I understand, the formation of vortexes, so that's like [115:02] a, like, little like tornadoes. [115:04] - Little tornadoes. [115:05] - Their formation is-- Is that part tricky to understand? [115:10] - Uh, yeah, absolutely. And it's beautiful too. Like, I love watching, [115:14] these videos of, you know, if you shoot two vortices [115:17] together, they'll, you know, influence each other and [115:21] then merge in these very interesting ways, that you have like a red vortex and a [115:25] blue vortex, and you can see where they merge and join. [115:29] But they also stretch and fold and, you [115:33] know turbulence is chaotic, so the trajectories get [115:36] tangled like spaghetti. It's not random. [115:40] There's a huge amount of structure, but it is very multi-scale, very, very tangled. [115:45] - So one of the things I would wonder, you know, scientifically, I mean, these are really interesting problems, and that was a [115:49] great explanation. So I'm always curious about what [115:53] computational problems nature can solve already, right? [115:57] And there are a lot of examples of this. You know, there's a famous one with like slime mold [116:01] finding shortest paths and mazes [116:04] In fact, some of the precursors to Navier-Stokes were, you know, like some of Terry [116:08] Tao's work on showing that you can have blowup for [116:11] Navier-Stokes-like equations because they can simulate Turing machines. [116:17] So I guess I would also wonder what sorts of computational tasks [116:22] these computations with vortices colliding can do [116:26] and how that imposes limits on what sorts of coordinate systems ought to exist. [116:33] 'Cause, you know, you mentioned PV equals nRT. That's an amazing [116:37] dimensionality reduction- [116:39] ... because you don't keep track of all of these positions and [116:43] velocities of particles, but there's just, you know, a small [116:46] list of numbers that describes a meta state, and that's it. [116:50] But if I really had a tremendous reduction, you know, [116:55] like that in the case of turbulence, then it would seem to me very [116:58] difficult to compute very interesting things too. [117:02] - Could it be like the game of Life, where you have this very simple [117:06] cellular automata system, and it's a universal Turing machine, [117:10] and you can cook up whatever behavior you want in it? [117:12] - Yeah, that's right. I mean, so game of Life, a lot of times people think about, you know, what are [117:16] minimal rules to have all of these exotic behaviors, and it's really [117:20] shocking how small the rules need to be to get that, right? [117:24] But I guess, you know, I'm a theoretical computer scientist. One of the biggest exports [117:28] from our field is really the algorithmic lens. [117:31] So it's not just about, you know, computation where you have a well-defined problem and a [117:35] laptop and you wanna solve it, but it's also about, you know, [117:39] computation in the natural world, [117:42] you know, and what that teaches us about what the rules of nature might be. [117:47] So, you know, some famous examples of this are things like equilibria in [117:50] multiplayer games, and Nash equilibrium is a great solution [117:54] concept. But of course, the fact that it's hard to compute Nash equilibrium, [117:59] you know, throws into question, is it really a universal mechanism [118:03] that explains how people play games? There was this amazing [118:07] quote by Kamal Jain that I really like, which is, "If my laptop [118:11] can't find it, then neither can the market." [118:14] And, you know, to me, what's really exciting is the nexus of a lot of these different [118:17] perspectives. [118:19] When you were describing it, it sounded to me like there are very interesting [118:23] computational problems under the hood, [118:25] and that one can kind of get a dual perspective on these things by thinking [118:29] about how that computation imposes limits on [118:33] what a good coordinate system ought to look like. [118:36] - So I think that's a really nice way of framing the problem. Uh, [118:41] and I would take two extreme examples. So Earth's weather and climate systems are [118:48] actually just complex. They-- all the scales do [118:52] matter. The weather in this zip code matters to me. [118:56] Um, you know, everything matters. All the variables matter. And you're [119:00] right. Like, if I over-reduce that system to, you know, five [119:04] numbers, then I lose the descriptive capability of what I actually care [119:08] about. That's one extreme. The other extreme, [119:12] let's think about how, you know, a moth flies or a butterfly or a fruit fly. [119:18] They are proof by existence that you can interact with an [119:23] extremely complicated turbulent flow and do very advanced engineering [119:30] control. They can control their forces and moments in [119:34] gusts, in rain. They can live very rich lives, and [119:38] they presumably don't have a PhD in fluid dynamics- ... or know the [119:42] Navier-Stokes. They're probably not running supercomputer simulations. [119:46] There might be five or six numbers that actually matter to their [119:50] daily life of maintaining a leading-edge vortex and, you know, [119:54] stalling their wings to perch and to land. And those are kind of two [119:58] extreme examples, and I really want to be able to handle both of those. [120:02] We want to be able to handle the full complexity of truly complex systems, [120:08] and we want to be able to distill down the absolute bare minimum essentials for [120:15] fast decisions in those kind of insect extreme [120:18] cases. And biological flyers still dramatically outperform, [120:26] you know, human-designed, [120:27] fluid systems in many regimes, so we still have a lot to learn about, [120:31] about these simple systems. [120:33] - So maybe we're, like, veering into research now, but, you know- [120:36] ... to me, it's always amazing how nature not only solves very hard [120:40] optimization problems, it seems to find very robust solutions. [120:44] And I'm always struck by the fact that, you know, when we model it, we tend to model it [120:48] assuming we have full knowledge, you know, about what the boundary and initial conditions [120:52] are for things like Navier-Stokes. But when a bird flies, you know, [120:56] it doesn't know all of that. [120:58] And to some extent, you know, modeling things with uncertainty is really how you [121:03] reframe the optimization problem to get very, you know, [121:07] rich and robust solutions. And it's kind of amazing to me how, [121:11] A lot of everyday examples sort of fit into this. I remember, you know, [121:15] I'm a big baseball fan, and I remember reading a math paper about [121:18] baseball that when you've got an outfielder, you know, once the ball has been [121:22] hit, a lot of stuff is known. You know, the velocity, the [121:26] trajectory. You would think the outfielder would just take a straight-line [121:30] path to where they expected the ball to be. [121:32] But when you observe outfielders, they don't take straight-line paths. They take a slight [121:37] curve to it. So why is that? You know, at least one [121:40] way to wrap your head around it is that that's actually a very robust strategy, [121:45] 'cause when you're taking a curved path, you can continually reevaluate- [121:48] ... the trajectory of the ball and then adjust as needed. So there [121:52] are very interesting solutions that nature finds, but it's not [121:56] always obvious what is the right formulation for the optimization [122:00] problem in the first place, [122:02] and sometimes modeling it in the case of uncertainty so that you can [122:07] design solutions that are robust to the fact that you don't have full system knowledge [122:11] seems to be an important piece of the picture. [122:14] - So there's a fun aside related to this. Um, [122:18] insects in particular don't have the ability to measure their full flow field. Um, [122:25] they can react to gusts of wind [122:29] faster than the information that hits their wings, goes to their brain [122:33] and back to their muscles. So researchers have studied this for decades in biology. [122:39] And what they've found is that there are strain-sensitive neurons [122:43] embedded on the wings, maybe a few dozen to a few [122:47] hundred of these in very stereotypical locations. So brother [122:51] fruit flies will have the same locations of these strain-sensitive neurons, [122:57] and those strain-sensitive neurons fire when the wing bends in a certain [123:00] temporal pattern. So it gets hit by a gust of wind, the wing bends, [123:04] neurons fire, and a local computation is happening in [123:08] that shoulder muscle of this insect that goes directly back and [123:12] corrects robustly based on partial information [123:16] of a system it doesn't have equations for that is nonlinear and turbulent. [123:20] So it's like, again, proof by existence- ... of exactly what you're saying is [123:24] maybe the robust strategy of actually relying on partial information and [123:28] uncertainty. They can do more than we can at this point, with way less information. [123:34] - Yeah. I have a couple of I apologize, pothead thoughts, 'cause [123:38] Alex mentioned black holes. I have a fun thought that [123:42] maybe the singularity observed with a blowup in [123:46] Navier-Stokes equations comes into play when we're as [123:50] opposed to airplane wings on Earth, we're traveling through [123:54] two connected black holes in a warp drive that we figure out, and then [123:58] there's the flow control of spacetime, and that becomes [124:01] relevant because now you're in a black hole, hence the singularity. This is [124:05] the-- For people just listening to this, the look Alex just gave me, [124:09] it hurts my soul. Okay. So moving on to the next topic. [124:13] - But the forcing, you'd need the black hole vortex to also be forced at [124:17] exactly the right- [124:18] - The, yeah, the force. Where's the force? I mean, at that point- [124:20] - How do you... Yeah. You could try to ride it. Yeah. There's a way-- You can make something out of this. [124:24] - There's some non-zero probability that future ass that will listen to this, that guy [124:31] is the one time he said something that was actually correct. Okay. [124:35] If it's okay, a quick bathroom break. [124:37] - Yeah. Sounds good. [124:38] - Cool. Ankur, one of my favorite topics is theoretical computer [124:43] science. You're one of the world experts in many of its problems. So [124:47] I'd love to ask you about one of the big problems that's been solved [124:51] in this OpenAI drop, which is the unique games conjecture. [124:56] Can you give people an intuition about what this problem is, why it's a big deal, [125:00] and what are some aspects of the proof that are interesting? Yes, I'd love to. [125:04] So let me give you a little bit of a background. So let's start off with [125:07] complexity theory 101. So one of the most important problems in [125:14] theoretical computer science is the satisfiability problem. So the thing to keep in [125:18] your mind is imagine you've got variables X one to XN, [125:22] and they can take on either true or false values. [125:26] And what I'm gonna do is I'm gonna create a big formula out of them, and [125:30] usually, the way we do this is with an and of ors. [125:34] So I start off with a clause that's, you know, an or, X one or [125:38] not X three or X seven. [125:41] So depending on my assignment, that clause might be satisfied as long as at [125:45] least one of those things is true. [125:48] And then I take the and of many of these different clauses together. [125:52] And the question is, if I give you that formula, called a conjunctive normal form [125:56] formula, you know, can you find a satisfying assignment? So can you find [126:01] an assignment of X one, X two, up to XN, [126:06] setting them either to be equal to true or false, so that all of [126:10] those or clauses are on, so that the entire formula evaluates to true? [126:16] So this is the canonical example of a problem that's easy to [126:20] check someone else's answer, but it might be hard to find an answer yourself. [126:26] So if you told me that it's satisfiable, all I would do is I'd ask you, [126:30] "Okay, give me the satisfying assignment." You tell me what X one is set [126:34] to, what X two is set to, and I just go clause by clause and [126:38] check to make sure that you really have satisfied everything. [126:43] Now, the, you know, the central problem in theoretical computer science is the [126:47] P versus NP problem. So are problems that are easy to [126:50] check, is it easy to find their solutions to? And the [126:54] presumption in our field is that no, it's not easy, and [126:58] that's the basis you know, of a lot of things like the, you know, [127:02] cryptography and e-commerce and what have you. [127:06] Now, really, the starting point for getting to something like the unique games conjecture is [127:10] something called the PCP theorem. So this was a landmark [127:14] result proved in the nineties. It actually was such a big [127:17] deal that it made the front page of The New York Times. [127:21] And the crux of it is really this idea that it's even [127:25] robustly hard to fool a verifier. [127:28] So you start off with a formula, and either it's satisfiable or not [127:31] satisfiable, right? And what does that mean? If it's satisfiable, you can write [127:35] down an assignment, and I can check one by one that the entire thing evaluates to true. [127:40] But if it's not satisfiable, that means someplace you're lying. You [127:44] write down this purported satisfying assignment, and when I go [127:48] clause by clause, I'm quickly gonna find one clause [127:51] that is not true, and that invalidates the whole thing. And [127:55] I say, "Aha, I caught you. It's not actually a satisfying assignment." [128:00] So the idea behind things like probabilistically checkable proofs, without getting [128:03] into a lot of the technical definitions, the idea is really [128:07] gap amplification. So can you start off with a [128:11] formula that is hard to decide whether [128:14] it's satisfiable or not? And can you engineer a new, [128:18] maybe much bigger formula that's robustly easy to check [128:23] in the sense that if it's satisfiable, it'll still be satisfiable. You can write down an [128:27] assignment, and it'll be fine. But if it's not, I should actually be [128:31] able to just flip a few coins, query random [128:35] variables, check a random clause, and with good probability find that you're lying. [128:40] So the point is that with very sublinear inspection, with only [128:44] constant amount of inspection, I can find that you're cheating. [128:49] So this was a momentous result in theoretical computer science because [128:54] it laid the foundation really for what's called hardness of approximation. [128:58] So a lot of algorithms and complexity theory is what do you do when you're faced with [129:02] problems that are genuinely hard to solve? [129:06] And, you know, one of the things we do is we try to approximately solve them. [129:10] So maybe we can't find an assignment if there is one that satisfies all the [129:14] clauses, but can we find one that satisfies many of the [129:17] clauses? How large a fraction of the clauses? [129:22] Now, even there, we would expect problems to become computationally hard [129:26] after a certain point. It should be that it's not just that, you know, [129:30] deciding whether it's completely satisfiable or not completely satisfiable is hard, [129:35] but maybe even deciding is it 100% satisfiable or is it only 99% satisfiable. [129:41] The best case over all assignments satisfy at most [129:45] 99% of the clauses. That too might end up being hard to decide. [129:50] So these are what probabilistically checkable proofs give us. In [129:54] places where we knew that the optimization problem was hard, they tell us [129:58] that it's even hard to approximate. [130:01] And the Unique Games Conjecture is a strengthening of the [130:05] ordinary PCP theorem because it has to do with a certain type of [130:09] label cover instance. It's a bit technical to go into the details, but the [130:13] point is that if that version of a probabilistically checkable proof were [130:17] true, it would have this great organizing effect [130:21] where it would turn out that for a huge swath of [130:24] problems, we would actually know what is the best [130:28] approximation algorithm, and it would be something very simple. [130:32] So when I started off in the field, the Unique Games Conjecture had been posed, you know, a few [130:36] years before that, and there was still a lot of debate about whether it should be [130:41] true or false. [130:43] And, you know, people worked hard to try and give, you know, proofs of it. People [130:47] worked hard to try and disprove it. [130:50] There was this amazing time period, maybe in the 2010s, where there was quite a lot [130:54] of back and forth [130:55] and, you know, there would be new results came out that people's opinions would change whether they [130:59] believed unique games or not. [131:02] Uh, I like this quip. I forget who said it, so I can't properly attribute it, but [131:06] someone told me, "You know, maybe God's gonna [131:09] decide whether it's true or false based on which side is working harder." [131:14] Because it really seemed to change year to year. [131:17] And you know, one of the results, one of the headlining results in the [131:21] OpenAI drop of proofs was that the Unique Games Conjecture is true. [131:26] Now, I should mention that this is not without spectacular human progress on it. [131:32] So one of my colleagues at MIT, one of my colleagues at IES, they both [131:35] contributed, among other people, to a lot of the major strides. [131:40] So Dor Minzer and Irit Dinur, they proved a weaker [131:44] version called the Two-to-One Games Theorem, and this really [131:47] was a, you know, shot heard round the world in theoretical computer science [131:51] because at that point, I think a lot of people weren't sure what [131:55] was a path forward for Unique Games Conjecture, and I [131:59] think their work really gave a viable path forward to something that was in the [132:03] vicinity of Unique Games. So it sounded like, you know, [132:07] maybe humans could get there with another 10, 20 years. [132:12] But, you know, just when OpenAI released the results, it turns [132:15] out that the Unique Games Conjecture is actually true. [132:19] So this is quite remarkable because you know, at some points I, [132:23] I guess I did find myself on the side of the doubters [132:27] because Unique Games just sounded too nice to be true because, you [132:31] know a lot of work like Prasad Raghavendra's work, [132:36] it implies optimal hardness of approximation for so [132:40] many different types of constraint satisfaction problems that the conclusion [132:44] just looked to me too strong to be true. [132:46] - What did you feel when you found out that night? [132:49] - Okay. Well, for starters, there were already rumors, you know, in the [132:53] weeks leading up that OpenAI had solved the Unique Games Conjecture and that [132:57] it was true. [132:59] And these weren't even just completely arbitrary rumors. One of my colleagues, [133:03] Dor Minzer, he actually posted a paper where he wrote something in [133:07] the footnote at the bottom that, you know, "We would have liked to polish this paper [133:11] more, but because of the rumors that were going around, we feel the need to [133:14] put out our approach sooner rather than later." [133:18] So for me, it was in some sense a bit of a relief that [133:22] the results were finally out there. Now they're not very readable papers. [133:29] It's not like you can point to them and really quickly identify what's a [133:33] new key idea. But I'm glad that they're out there [133:37] because now we can move forward on the next phase, which [133:40] is digesting those proofs and figuring out where the field moves [133:44] after that. Now, in terms of how did I feel that night you know, as Alex [133:48] mentioned, we, you know, we all had a drink in hand. And [133:52] then I saw, you know, on my phone the news that these proofs had dropped. [133:58] And you know, people kept up coming and talking to me at the gala, [134:02] but it's already hard to socialize as a mathematician- [134:05] ... let alone, you know, when you have math you really want to read. [134:08] So it was doubly hard. [134:10] The other funny thing about it was, you know, I was in New York already, so I took it as an [134:14] opportunity before the gala to visit a lot of people I knew at NYU. [134:20] And I just happened to be meeting with Subhash Khot, the one who introduced the [134:24] unique games conjecture. And I just so happened to ask him, you know, just [134:28] a few hours before the gala, before the OpenAI paper is released, [134:31] "Subhash, what would you do, you know, if they [134:35] release a positive proof of unique games? Are you gonna read it?" [134:40] And he told me, "No." [134:44] Which, you know, that's a lot of people's reactions, unfortunately. [134:48] It's just kind of true that there are these grand challenges we work on, [134:52] and sometimes when they get solved you know, sometimes it feels like, [134:56] you know, someone has stolen your thunder. [134:59] You know, it's not like people didn't have ideas about how to solve the unique games [135:02] conjecture, and I bet you those blueprints would have panned out if [135:06] we'd given them more time. I think the other thing I worry is that [135:12] you know, that we lose out on the human effort of [135:16] trying to reach those solutions to begin with. You know, math is not just [135:20] about what is the furthest I can reach with the current set of tools [135:24] I have. Is it a convex hull? Is it a subgroup? I don't know. [135:28] But, you know, sometimes when it's a human-driven solution, we learn [135:32] a lot more in the process. So I worry about the fact that there [135:36] were these great landmark problems, field-defining problems that a lot of people [135:40] were reaching towards, [135:42] and that decisive progress was made, you know, and the results [135:46] all dropped in one night, and that people lose their enthusiasm [135:50] for going down those paths. And I wonder if we'd given those people that space [135:54] to go down those paths, what other exciting things we would have discovered. [135:59] So I think one of the things I mentioned, you know, we were all trying to process it at MoMath and, [136:03] you know, there were a lot of really nice talks. Not all of them went up on YouTube. [136:07] Some of them were, you know, about benefactors and awards named after them. [136:11] And the thing I kept coming back to is this idea that, you know, why do you do math in [136:15] the first place is to play and learn about the world. [136:19] And we saw so many manifestations of it that night. [136:23] You know, because there were awards named after people who, you know, [136:27] why were they donating to the museum was because they wanted their kids [136:31] to have a place to play with mathematical ideas and learn about the world. [136:36] It was also the opening of an Escher exhibit, and what [136:39] a great visual of, you know, what happens when we play with mathematical concepts, [136:45] right? And so it felt to me that in this moment, it's really important to remember, [136:51] as we're going through all of this change together, as we don't know what the future [136:54] holds, what AI and mathematicians' collaboration will look like in the [136:58] future, what hiring will look like, that the reason we all got into it was because [137:04] we were very curious. [137:06] You know, we wanted to understand the world, and mathematicians are very good [137:10] at abstracting things. You know, they don't just [137:13] understand one little piece, but they find some [137:17] abstraction and some set of definitions that turns out to be [137:20] unreasonably powerful in some other corner of mathematics. [137:24] So I think it's really important, whatever the future holds, to make sure that [137:29] we organize our field in a way where we still encourage people to play with [137:32] ideas. I don't know what kind of reward system will work, [137:36] but, you know, when I look out and I talk to young people in the field, they [137:40] have a lot of anxiety. You know, they hear a lot of stories even about [137:44] people they know you know, running harnesses [137:49] to, you know scrape the web for open problems, to [137:53] dump as many papers as they can to flood arXiv. [137:57] I don't know if that's a good strategy. It doesn't seem like it is, but it certainly [138:00] doesn't feel like fun to me. It doesn't feel like we're playing with [138:04] ideas and learning about the world, [138:07] and that's sad. I just wanna get back to the things I really enjoy. [138:12] The other thing maybe I just wanna add in there too is, like, you know, [138:17] it's a big identity crisis for a lot of people 'cause, you know, [138:21] some people view themselves more as problem solvers, and that can feel very comforting, [138:25] that you can draw a little circle around the types of problems you work on, the types of [138:29] techniques you use, the types of problems you care about. And that [138:33] feels very comforting because the rest of the world is messy and you've got blinders [138:37] on. This is the part that I care about. [138:40] Now, I think, is the time to remove those blinders because I just don't find that to be [138:44] a productive way to do it. [138:46] You know, we love playing and learning about the world around us, and it's [138:50] important to remember as, you know, a young mathematician, [138:53] are you smarter than you are curious, or are you more curious than you are smart? [138:58] It's very important to be more curious than anything else. [139:01] - And you mentioned fun. There could be another perspective of it, [139:05] of more about the struggle from my maybe outsider [139:08] perspective is some of the most fun in the moment sure feels like struggle. I've [139:16] talked to a lot of Olympic gold medalists. A lot of them [139:20] feel something approximating depression after they win the gold medal [139:24] because it's like you have given everything, and you [139:28] realize everything you've given is now over. [139:32] - Thirty years ago, there was a documentary of Andrew Wiles, [139:36] and there's a great scene, I think they even run it right at the very beginning, [139:41] To your gold medalist, you know, what do you do once you've won the thing? [139:46] He's sitting there at his desk, and he's describing the moment when he [139:49] realizes what the solution is, not his initial announcement, which was flawed, and [139:53] then for many months, he was trying to correct it, and he finally saw how to do it. [139:58] And he's trying to say, you know, "I will never again in my [140:04] life have a moment as powerful as the [140:08] moment that I experienced at this desk that day," and he's choking up. [140:12] - You can't get the words out. So that-- Can you get that if you [140:16] press the button and it gave you the solution? Absolutely not. So [140:22] exactly to both of your points, like [140:25] the journey, you know, if you see a giant hill, a giant mountain, [140:31] and you say, "One day I'm gonna get to the top of that," and you start training for it, and you're sort of [140:34] doing little by little, whatever, and then a helicopter lands and just drops [140:38] you at the top. Can you actually speak to that? The [140:42] interesting thing about Andrew Wiles and a little bit about Grigori [140:46] Perelman as well is working alone for a prolonged period of time. What do you [140:51] make of that? How common is that in mathematics of sitting there [140:56] and grinding a problem for, in his case, seven years? [140:59] Until the 20th century, almost every math paper that had ever been written [141:03] was written, you know, had one author. I think now the median is two [141:07] in mathematics, and so we're not a lab science. [141:12] It really is a solitary process. Now, see, the things [141:16] that I give my graduate students to do are not the things that I'm working on, sort of [141:20] by design. [141:20] I want them to have their own field. I wanna open doors for them. It's different in [141:24] different fields, right? So when they have a success, it really should be [141:28] their success, even if I've sort of-- I had an idea that this would go somewhere [141:32] and they executed on it, that's theirs. Um, but in terms of [141:40] sitting there for s-- Uh, the paper that I put out a month ago is something I've been thinking [141:44] about for 10 years. It's a five-author paper. Over those 10 [141:47] years, we made incremental progress to the point where we finally, you know, had a [141:51] solution, and that paper was an instance of there's a phenomenon [141:55] here, and I don't understand what it is. [141:58] It wasn't, "Here's a problem, solve it." Like, we could not, at the start of [142:02] the project, give you the Lean, formalized statement that needs a sorry. [142:07] Like, you discover what the statement is through the process of [142:11] hacking away at the problem little by little until you see enough of it to say, [142:15] "This is how it comes together." [142:17] Actually, that makes me think, can you actually describe how your thinking [142:20] process of how you think through really hard problems? [142:24] Is there some aspects to it that are interesting, like a lot of coffee [142:31] in the morning, at night, alone, brooding with a cigar [142:37] or on a together at a whiteboard with others? Is there some [142:42] interesting color, texture, insights you can give? [142:45] - Sure. I think a lot through world models in [142:51] some way. So, you know, I have some mental model [142:55] of how I think results in some sub-area go, [142:58] and I think that colors a lot of how I see surrounding areas too. [143:03] So for me, a lot of my work is really about, you know, building bridges [143:06] between algorithmic thinking and other problems that usually aren't studied [143:10] through that same algorithmic lens. So I think when I first started as [143:14] a advisor, I sort of made the mistake that I gave [143:18] students questions that were 100% in my area. [143:22] And the problem then was, you know, when they would come back and they'd have an idea, [143:26] I could shoot it down very quickly, and I was either right or I was wrong, but [143:30] either way, no one was having any fun. And- [143:34] You know, over time, what I started to do was I would give students questions that were [143:38] 10, 20 degrees off from my own area, things that I wanted to [143:42] understand better myself, papers that I thought were important that I just [143:46] hadn't had time to sit down and read, [143:48] and then I have the student explain the result to me. [143:52] And what's fun is we would go through that process together, not, you know, by printing out the [143:56] paper and going through it line by line, [143:59] but trying to reverse engineer why do we think this works in the first place? [144:03] And there's so many details that once you get the feel for it, you kind of have the [144:07] sense that they ought to work out. But at least in important papers, [144:11] there are one or maybe even more than that, key ideas that really change [144:15] your perspective, "Oh, this really might be doable." [144:19] So what I found was, you know, as soon as we start trying to [144:22] understand one of these results, we find our own way of looking at things, [144:27] and then it's not long after we find some other [144:30] questions that we now think are approachable because we found our own way of, [144:35] of organizing our thoughts. So that's often the way I do it is [144:39] the earlier that I can have the student in the driver's seat, not [144:42] necessarily with a well-defined question- [144:45] but with them teaching me something and them being responsible for distilling and [144:49] digesting and abstracting it down to its essence. [144:53] You know, reasoning through with them analogies that aren't meant to literally be [144:57] true, but help us figure out what to expect in neighboring areas. [145:02] You know, then it feels like they bring a unique perspective, and we're off to the [145:06] races. So for me, in a lot of ways, the technology is only gonna [145:11] enable that kind of work because it means that a much richer set of problems [145:15] are approachable. You know, for me, when I find experts in related areas that can [145:21] really explain things in my own language, I hold onto them for dear life because, [145:26] you know, they can speak the gospel. They can tell me how [145:30] this area is organized and explain it in my own language. [145:34] But now, you know, I have technology that can do that too, [145:38] and it also feels a bit sad that you lose that human element that was there [145:41] once before. I think also students have so much anxiety about, you know, [145:47] what publications are gonna look like that one of the hardest parts is, you [145:51] know, when I'm thinking about a project, I'm really trying to think about it in a sort [145:55] of multi-year time horizon because I have some [145:59] meta vision that I wanna realize, like how can we use [146:03] learning theory techniques in quantum many-body systems? And there are a lot [146:07] of stepping stones on route to that, but some of the most exciting problems are, you know, [146:11] many years out. But students are so worried about the incoming [146:15] AI tidal wave, you know, the future of work, universal basic income. [146:19] Are any of us gonna have jobs? That they're really thinking in three- and [146:23] six-month horizons, and then it's hard to find problems that we [146:27] both care about just because- [146:30] - You know, we're thinking about different worlds [146:33] - You don't think we should encourage them to continue to think long term? I mean, I understand [146:37] that- [146:37] - Oh, I do, but they don't always listen to you. [146:41] I mean, I don't know if I'm right or wrong. You know, maybe AI will solve all of our problems, [146:45] but it seems like the only ones that are actually worth us working on at this moment [146:49] in time are the ones we're still gonna care about in a bunch of [146:53] years, and you should only be working on future-proof problems. If you're working [146:57] on things that, you know, you're in a rush because you think the next generation of [147:01] models will definitely solve those problems, then maybe just go on vacation [147:05] and don't solve that problem. [147:07] - Yeah. Can you speak to how you think-- You work with a lot of natural [147:11] systems. What are the interesting insights that could be pulled out [147:15] of these natural systems? [147:17] - So I find a tension between focusing very [147:22] hard on an engineering problem I'm trying to solve, which can be very [147:26] singular. You put on blinders, you only think about this problem, [147:31] and then giving yourself the kind of mental freedom [147:36] to look around and just observe and be curious. Very often you [147:40] find the solution when you turn over a rock and when it's [147:44] not something you're thinking about, [147:46] you know, with the full front of your mind, but in the side of your mind. If you're [147:50] looking at a problem, you know, through your peripheral vision, you have a much better chance of [147:54] solving it sometimes. And so this is a constant balance for me is observing [148:02] all of the fields that I don't work in and all of the nature that I don't understand and all [148:06] of the things that I'm curious about, but then also having very focused [148:09] problems that you don't relent, you don't let go. And I-- this must be true for all of [148:13] us. We-- I've heard from both of you, you know, focus problems you've [148:17] been working on for years and decades that you haven't let go of. [148:21] But presumably that is informed by [148:26] a million other pieces of information that you happen to pick up through interesting [148:29] conversations and interesting- [148:31] - Do you have example? 'Cause it sounds like neighboring areas is a [148:35] common theme. Do you have examples of things you're like, [148:38] you're-- the curious peripheral vision of your brain went, "Oh, okay, that's an [148:42] interesting system, might be relevant," and then ended up being so or [148:46] maybe in the future will? [148:48] - So for me as a fluid dynamicist, biology and its messiness and complexity- [148:54] ... and also material science and its messiness and complexity are [148:59] neighboring in a way that's very fruitful. I learn a lot from- [149:02] ... from both of those fields, and a lot of the kind of basic math [149:06] that we develop to handle complexity does seem to transfer, and [149:10] sometimes you have an idea just because you see something is possible or, [149:16] you start to develop something and you can explain something that we didn't understand [149:19] before. But I think [149:21] for me, biology and fluids and materials have this really interesting kind of [149:24] commonality of multi-scale, non-linear, messy, [149:28] nebulous objectives, things like that. [149:30] - By the way, on that topic, do you think there's hope that AI [149:34] can have some significant breakthroughs in the messy world of biology? [149:40] - I certainly hope so, and I think so. There are [149:44] large efforts in biology to collect very large data sets [149:48] and to understand complexity in systems, so the [149:52] connectome where you have the full wiring diagram of the brains of, [149:56] of insects. It looks like we will start to have connectomes of [150:00] other animals, and eventually it'll be probably like genomes where, [150:04] you know, it is cheaper and more accessible. [150:06] - I have this, like, feeling that we'll find out, speaking to the resilience of, [150:12] you know birds being resilient, that in order to understand [150:16] how the human biology system works, you have to, like, pull that thread, [150:22] and you're gonna have to understand how the entirety of life on Earth [150:26] emerged and functioned as an entire, like, mechanism. [150:30] Like, it's all tied together. Like, you can't replicate-- [150:36] Like, it's ridiculous what happens with humans. From a single cell, this whole thing goes [150:41] It builds, and the intelligence that's built into the whole building [150:45] mechanism in the first few days, weeks, months of life, this whole thing is like [150:52] I think could be, like, many orders of magnitude more complex than some of the hard [150:56] mathematics problems that we've been talking about. Again, I suspect [151:01] if turbulence is hard, [151:03] like whatever the hell is going on in here, I don't mean the brain, I mean, like, [151:08] the fluids around the brain, the whole thing, the immune system, all of it, [151:13] the-- everything interconnected together, and [151:17] I am very deeply connected to the history of the bacteria [151:21] that was here billions of years ago. I suspect the [151:24] complexity of the entirety of the mechanism of life on [151:28] Earth is many, many, many orders of magnitude higher than any, [151:34] system that we-- that is amenable to AI today. Just a thought. Sorry. [151:40] - No, I tend to agree. [151:41] - Okay. [151:42] - I think you're right. Like, it's incredible to just imagine all of the [151:47] complexity and scales that inside one cell, it's a whole [151:51] city. It's a whole infrastructure. You know, one of your neurons [151:55] has garbage collection and power production and information pro-- like, [151:59] the whole thing is, at every single scale, is beyond our [152:02] ability to possibly fathom. You're absolutely right. And I think that's what makes it so [152:06] interesting, and it is related to math in the sense that [152:11] finding the right scale where we have any hope at all, and there [152:15] is something interesting that we can use to understand that, that [152:20] whole picture, I think that's very fascinating and that [152:24] up until now has been a human endeavor of finding [152:28] the bits that are interesting and understandable in this mess of complexity. And- [152:35] Yeah, I think you're right. [152:37] - I mean, the fact that anything is understandable about the natural [152:40] world is a miracle, right? [152:42] - Yeah, absolutely. [152:42] - There's no reason that Newton's laws can be written in physics [152:46] that Newton himself can describe the mathematics of, and, you know, [152:50] invent calculus for the purpose of being able to say what inverse square propagation [152:55] results in elliptical orbits, you know, solve the two-body problem, yada, yada. [152:59] So the fact that the universe speaks mathematics is completely [153:03] mysterious. There's no a priori reason that we... We don't have a right to know [153:07] anything that we know. [153:09] - Uh, I apologize for the question, but I'm obligated by the said universe to ask the [153:13] question, but given this topic do you think math is invented or discovered? Alex. [153:19] - Yeah, it's the age-old question. Um, I have a very sort of [153:22] pragmatic answer to this, which is the questions [153:26] that we end up asking, it seems to me, are functions of what's going [153:30] on in the world, what questions we possibly can be asking given our current state of [153:34] knowledge. The questions we're trying to answer today are not questions that Euclid could have thought [153:38] about. Math was sort of-- The things that [153:42] people got interested in were different in Greece versus India versus [153:45] China. But then the answers, once you ask a question [153:49] precisely, the answer is what God decides the answer is. So we [153:53] don't have-- So we're discovering answers to questions that we've invented. [153:57] - We're discovering answer... Okay. [154:00] Steve, you have worked with a lot of natural systems that look at equations. Equations [154:04] are, I guess, always approximations of the real world. So what's-- Is math real? [154:10] - You gotta say yes. [154:12] - Right. We got a lot of pressure here. [154:14] - Yeah. How big is the gap between the mathematical world and the real world? [154:18] - Surprisingly small. I think unexpectedly small. I [154:22] like your answer a lot, actually. I do somehow fundamentally think [154:26] that math is both created and discovered. We create mathematical formalisms to [154:34] understand the world around us, [154:36] and those approximations are unreasonably effective. It is, [154:41] it is a miracle that there are simple, compact [154:45] descriptions, F equals ma equals mc squared, that describe [154:49] everything that we observe at, you know, all of the scales that we [154:53] seem to observe. We seem to be able to distill something simple, mathematical that describes [154:57] them. That feels like we are creating those, [155:01] but that there is that simple, generalizable [155:05] description underneath everything, and that seems to be something we're also [155:08] uncovering. [155:09] - Do you think that's a good guide, you know, kind of searching [155:13] for the beautiful simplicity for us? Are we just as, like, aesthetically [155:20] pleasing things? [155:21] - I think it's a good guide for us because we are fundamentally simple. We [155:26] can only handle so much in our minds. We can only envision so much. I think simple [155:32] is essential for humans to interact with the real world. Um, [155:38] okay, if I wanna get, like, slightly deeper into the question of created versus discovered, [155:44] numbers are a good example, integers. We all fundamentally [155:48] feel like, you know, counting numbers are real in a [155:52] very essential way. I can have one rock and two rocks and [155:56] three rocks, and that is a number of rocks. It is a real number of rocks. [156:01] But those are absolutely abstractions that I have [156:04] created. A rock is not a rock, is not a rock. It's a collection of atoms. It's a different [156:08] collection of atoms. They're not, there's not one rock or two rocks. [156:13] Do numbers even exist? I wrestle with this. I don't know. Like [156:17] do we actually believe that integers are a human creation [156:23] to interact with our symbolic abstraction of what we observe, quantities and- [156:28] - I mean, it took humans thousands of years to discover the number zero. [156:31] - Right. [156:32] - And then negative numbers. Negative numbers were actually accepted by [156:36] mathematicians in the West after complex numbers- ...were accepted [156:41] as a mechanism for study, which seems insane, but there's a whole- [156:44] ...story behind that. [156:46] - What's your relationship like with infinity? Is it real to you, [156:49] or is it a nice thing to have as part of proofs? [156:53] - So much of modern mathematics is exactly dealing [156:57] seriously with infinity, right? One of the first times [157:01] people really start to hit a wall in their mathematical education is when they hit [157:04] calculus, and there's sort of several reasons for it. One is they probably have a lot [157:08] of gaps in their understanding in previous you know, [157:12] they got a, a eighty-one on geometry and a [157:16] seventy-five in trigonometry or whatever. Those are pointing to gaps in their understanding [157:20] that all of which is gonna come together in one- [157:23] ...uh, fell swoop, and the Swiss cheese is just, you know, not, they're not [157:27] able to cross that threshold. That's one issue with calculus. The other issue [157:31] with calculus is the first time you deal seriously with infinity. You have to add up infinitely [157:35] many infinitely small things and get a finite number and have that number make [157:39] sense. Um, human beings Newton and Leibniz did not understand infinity the way [157:46] we understand it today. They had an intuition for what it's supposed to represent. [157:50] But if you read Newton, I mean, Newton in his time was ridiculed for the way he wrote about, [157:55] you know, the ghosts of departed quantities. You [157:59] know, you have to divide zero by zero- [158:02] ...and, and have it make sense, and you don't do it when they're zero. [158:06] You do it after they become zero. I mean, the way that he was trying to write about these things [158:10] turned into the crisis in the nineteenth century when Cauchy and [158:14] Fourier proved conflicting theorems. Cauchy said [158:18] you have a sequence of continuous functions, and you take the limit of continuous functions, [158:22] you're gonna get a continuous function, right? Limits are a continuous process. [158:25] Continuous functions are continuous. The answer has to be continuous. And Fourier [158:29] says, "I'm gonna add up a bunch of sines and cosines. Doesn't get more continuous than sines and cosines, and [158:33] I'll get a sawtooth. I'll get whatever you want. I'll get a discontinuous function in the limit." [158:37] They broke math. [158:38] - So that's when we had to start facing-- it was no longer okay [158:43] to think about infinity the way Newton and Leibniz and the entire-- [158:46] Euler's entire, what, thousand some odd papers [158:51] lived in a world where infinity was something you play with, and in the nineteenth century, infinity had to [158:55] get professionalized. [158:56] - So it turns out math has been killed multiple times- ... before. [159:00] - That's right. And it's somehow it's only a scratch. The vast [159:05] majority of our first-year PhD courses are the result of crises that [159:10] happened in the nineteenth century. [159:13] And math-- where math was destroyed and had to be rebuilt from scratch. [159:16] - Uh, speaking of crises, let me get back to [159:20] P versus NP question Ankur was talking about. Do you think-- how hard of a problem [159:24] is that, proving-- [159:27] So this is sort of the central question of complexity theory, P equals NP. [159:31] Do you think AI will be able to prove this? In fact, we should [159:35] mention that you're kind of a part of a panel. There [159:39] was initially a debate about whether LLM-based methods, [159:44] could one day make progress on the hardest [159:48] problems in mathematics, like P and NP or the Riemann hypothesis. [159:53] That was a year ago. So what is your updated view on this? [159:59] - Well, clearly, they can make progress on- [160:02] ... central questions in the field. Uh, P versus NP, maybe it's [160:06] just because it's closest to my heart. That seems very difficult to [160:10] make progress on. You know, it's kind of remarkable the things we [160:14] don't understand, so that problem I was describing about satisfiability, [160:19] we don't actually know any super linear lower bounds. [160:23] You know, we think that it takes exponential time, [160:27] not that much faster than enumerating all possible assignments and checking them, [160:32] and yet we don't know anything that sort of is asymptotically [160:36] more running time than just reading the darn input. [160:39] - Mm-hmm. And we should actually mention, you've talked about [160:44] hard and easy. What is a more [160:48] technical way to think about what is hard and what is easy? [160:51] - Right. So this is polynomial time versus not [160:55] polynomial time, or, you know, NP, just to be clear, stands [160:59] for non-deterministic polynomial time. So the question is whether [161:02] things which you can solve with non- determinism, like guess the [161:06] assignment and then check it, can also be solved in polynomial [161:10] time. So the question there is, if I give you a description of a formula, [161:15] you know, I tell you what are the variables, what clauses they appear in, [161:19] is there an algorithm that runs in time polynomial in the size of the [161:23] input and correctly decides whether or not there is a satisfying assignment? [161:28] - We should mention that the polynomial time or exponential time [161:33] or super linear time, all this refers to worst case analysis. [161:37] - That's right. That's right. [161:38] - So can you explain what worst case analysis is, and what are its- [161:43] - Pitfalls. [161:45] - Yeah, I should say what are its pitfalls, 'cause I think it's a little bit counterintuitive from people [161:49] outside the field, like to, like, to understand exactly [161:53] when we say a certain problem is hard, you're actually talking about worst case. [161:57] It's like analyzing, I don't know, [162:00] I don't know why that popped into my head, but like marriage. [162:03] It's like analyzing whether I should marry this person or not by saying, "Well, what's [162:07] the worst thing that could possibly happen?" I think that's for [162:11] human relationships, maybe it's not such a good strategy. Maybe the [162:15] average case is a, is a better guide for making decisions. [162:19] Anyway, I don't know why I brought that up. [162:20] - Okay, I have no advice about who you should marry. [162:24] - This is com-- yeah, complexity theory does not give you. I think there is some-- [162:29] Isn't there a dating is it like an optimal stopping theory of the optimal- [162:35] - Secretary problem. [162:37] - The secretary problem. There's a lot of different names for it- [162:39] ... but there is a mathematical way. There's an optimal strategy. [162:44] - So first, so what is worst case analysis? So- [162:46] - Yes, please [162:47] - ... you know, I was describing the SAT problem earlier, and the [162:50] algorithmic problem you want to solve is, given any formula, you want to decide is [162:54] it satisfiable or not. [162:57] Now, what's crucial there is that when I'm asking to solve that algorithmic [163:00] problem, I'm not telling you anything more about the structure of the formula. [163:05] I'm not telling you, you know, these are the size of the clauses. [163:09] They look like this. If I drew a graph connecting them, it would have this topology. [163:14] So that's what worst case analysis is about, is it's about the premise that [163:18] you can treat all of these computational problems actually by [163:22] ignoring a lot of the fine-grained detail about the individual structure, [163:27] and you can try and solve them, you know, for all possible inputs. [163:31] And that's been spectacularly powerful because we've identified lots of [163:34] algorithmic problems that really can be solved in the worst case and can be [163:38] solved really efficiently. You know, like finding the max flow, as an example. [163:45] But the question is, you know, whether or not [163:49] that's a useful abstraction. So sometimes maybe you really do [163:53] need to know more about the structure of the instance you're faced with in [163:57] order to solve it better, right? And that's really an open [164:01] question. This is the area of what's called beyond worst case analysis, [164:05] and I would sort of argue that, you know, it's a very important direction for theoretical [164:09] computer science to go in. It's hard to figure out [164:13] what you should articulate about natural problems [164:17] that is really the right level of detail that your algorithm can [164:20] exploit it and can do better things than it can in the worst case. [164:25] But let's use an example. So you were talking about the secretary problem. [164:30] So the way that this works is it's an example of something called online algorithms. [164:35] So you have a sequence of options coming. [164:40] One through N. They come in each time step, and your goal [164:44] is, you know, when someone comes along, you know their [164:48] ranking relative to everyone else you've seen before, but [164:52] you don't know their ranking relative to the people who are yet to come. [164:56] And your goal is to, you know, to make a decision when [165:00] the person comes along, do you commit or do they move on, [165:04] in such a way that you have a good chance of finding the absolute best candidate. [165:09] And so the point is that you can actually get a constant competitive ratio [165:13] algorithm that what you do, the best strategy is to [165:17] wait some amount of the time, not make any decisions, don't accept anyone. [165:22] And then once that probationary window has passed, you take the first [165:25] person who's better than anyone you've seen thus far. [165:29] And that'll give you a good competitive ratio in the sense that you can prove theorems. [165:34] Assuming the inputs arrive in a uniformly random order, you're [165:38] likely to do pretty well compared to as if you knew what that [165:41] sequence of inputs was in advance. [165:45] So there's, you know, under online algorithms, there's this veil of uncertainty, [165:49] which is, I don't know what the sequence of inputs are gonna look like. I [165:53] wanna do well no matter what that sequence is because, [165:58] you know, I don't really know how to model what that sequence looks like. [166:01] Now, you can instead say, "Hey, maybe the algorithms I derive [166:06] aren't good enough for my applications. Can I do better by trying to [166:09] model more of the world around me?" And then there's kind of an art to [166:13] figuring out what you should model, what you should build into your model, [166:17] and what you shouldn't. And that seems to me like a tremendously important area, [166:23] to explore. So for example, one other variant [166:27] of, you know, online algorithms that's really important is the paging problem. [166:31] So your laptop solves this problem all the time. You know, it's getting a [166:35] sequence of requests of things that it wants to recall [166:39] from memory. And the question is, what should it actually put in its cache, [166:43] the part that's the easiest to access? [166:46] And the problem again is that you don't know what that sequence of requests looks like in [166:50] advance. So you can study the online algorithms problem of, you know, what [166:54] is a good policy for managing the cache? You know, when something gets [166:57] accessed, you know, I have to put it in, but what do I evict? [167:02] And you can design algorithms that are good in terms of their competitive [167:06] ratio. The number of times they miss in terms of those [167:10] accesses is not much worse than, [167:14] the best offline optimum if someone told me that sequence of requests in [167:18] advance and I could optimize just for that sequence of requests. [167:22] Now, this is a problem that's been studied since the '80s and '90s. The trouble is [167:26] somehow that, you know, the best algorithms you can give, we know tight upper and lower [167:30] bounds, they're just not that very good. [167:34] You know, the number of misses they make compared to the best offline optimum [167:38] is a large factor off from it, and that just [167:42] doesn't compete with what your laptop is able to do right now [167:46] because it really makes very few cache misses at the end of the day. So you're [167:50] stuck with this problem which sort of bridges mathematical thinking with trying to [167:54] model the natural world, which is, you know, what is it [167:58] about realistic sequences of requests that makes [168:02] them not the worst case? This is really an inverse problem [168:05] now because it's, you know, the other side of the coin. Instead of someone handed [168:09] you a problem that you're just trying to solve in the worst case, and we can all agree on how it's [168:13] formulated, and we know what's a better algorithm, it does better on the [168:17] competitive ratio, now you're stuck with this very fuzzy problem, which is: [168:21] how should I model the world around me? And in a lot of ways, [168:25] algorithmic thinking is necessary for figuring out what are [168:29] productive ways to model that problem. Because it's not that you're gonna [168:33] build a totally realistic model of what sequence of page requests look [168:37] like, but how am I gonna coarsen the problem in just the right way [168:41] that it leads to interesting algorithmic problems that teach me about the [168:45] world, that force me to, you know, design new approaches to [168:49] solving these very basic problems. So I'd argue that, you know, [168:53] sort of coming out of this space where we are only solving [168:57] well-defined problems, but we're actually using our mathematical and [169:01] algorithmic insights to figure out where to dig in the first place [169:05] is now more important than ever. It should have always been important, but you [169:09] know, that to me is really gonna be thrust into the spotlight. [169:14] - Like a simple example is we were talking about factoring numbers. Well, factoring a [169:17] random large number is really easy. Half of them are even. You know, [169:21] factoring means find a factor. [169:23] So it's really those very special numbers that are products of [169:27] two primes about the square root the size of the original number. That's hard factoring, sorry. [169:32] - Which makes me think one of the things you worked on is mathematics of [169:36] LLMs, AI systems, neural networks, [169:40] and those things are pretty damn good at modeling the natural world. [169:46] So what can you learn from that? First of all, why does deep learning work? [169:55] - Do we have another three hours? [169:57] - Do you have— Do we even have intuition? So at the mathematical level, at [170:01] the philosophical level, so we're in between all of that, the full range. I don't know [170:05] how we even enter that. Okay. So that's a big question. [170:10] And let me try and break off a few pieces of it just to try and [170:13] condense my own views on this. So, you know, in the OpenAI drop of problems, [170:20] there was no result in there that explained deep learning. [170:23] There was no result in there that explained LLMs. You know, there's no [170:27] bound we know that if we could improve it would suddenly explain deep learning. [170:31] The problems are just not posed that way. [170:34] And I would really argue that what's needed in a lot of deep learning theory [170:39] comes hand in hand with better ways to model the world around us. [170:43] If we look at the computational problems that deep learning is solving, that [170:47] LLMs are solving, some of these problems are very hard. You [170:51] know, trying to fit the parameters of a depth four network to data is a [170:58] cryptographically hard problem. We know that. There exist instances of [171:02] problems which, if you could really solve them in a worst-case [171:05] sense, you know, that would change our understanding of cryptography. [171:11] And yet these problems get solved all the time, every day, when we're fitting deep -- you [171:14] know, much deeper networks than that. So I would argue that, you know, [171:18] a large part of that is how nice natural data is. [171:23] It's kind of like, you know, there's the saying in physics that nature abhors a [171:26] vacuum. I guess my version of it is that in some sense, nature abhors [171:32] computationally hard problems. [171:34] And it's an art to figuring out why that is, how to draw the boundaries [171:38] around the types of problems you're studying that actually evades the hardness [171:42] of the problem in the worst case. [171:44] So there's an interesting example back from the '90s. So certainly back then, [171:48] and we still are, very interested in protein folding. [171:52] Now, you can model this in a worst-case sense. I've got a sequence of amino [171:56] acids. I've got some energy function based on different [171:59] potential 3D structures, and I want to find the 3D structure [172:03] that minimizes how much energy there is. [172:07] Now, this problem is computationally very hard, [172:11] and yet nature solves these problems all the time. So why is that? [172:15] So there's a very interesting experiment you can do, which is you start off with a bunch of [172:19] sequences of amino acids. You try out some heuristics for [172:23] folding them into 3D structures. These are things that don't solve the problem in the worst [172:27] case. They just seem to do the right thing sometimes, for whatever that means. [172:32] And then the only sequences of amino acids that live on to the next [172:36] generation are the ones that actually have very good folds. [172:41] And over time, what ends up happening is you create a body of [172:44] problems for which they're not hard in the worst case, but [172:48] eminently the heuristics you've developed are very good at solving those problems. [172:53] So in a lot of ways, you know, what are the structure of proteins that nature [172:57] seems to fold all the time? Well, if they really were hard to solve [173:01] in the worst case, then maybe the shape they take on would be [173:05] very brittle and fragile and subject to a lot of boundary [173:08] conditions that make those things behave in not entirely predictable ways. [173:14] So in some sense, you know, a lot of our data has [173:18] evolved in ways to make these computational problems easier. [173:22] So I like to think of it as you can study LLMs in terms of the optimization [173:26] problems they're solving. You can study deep learning that way. [173:30] And if you look at it from a worst-case perspective, you're gonna get very hard [173:33] problems that you just tell people, "They're hard. Give up." [173:37] But apparently, not giving up was a really good idea, and I think what [173:41] makes those problems easy you cannot extract from the structure of natural data. [173:48] The fact that our language has evolved to make these problems easier is [173:52] intertwined with why LLMs work the way that they do. So my [173:56] view is that, you know, people working in this space and theoretical machine learning, [174:00] they have to become more comfortable with the uncertainty of modeling. [174:03] You know, when we teach a lot of graduate students in math departments, we teach [174:07] them, you know, how to solve open questions that are well-defined. But I think [174:12] teaching people that skill of how to deal with messy, uncertain [174:16] things and, you know, how to model it and how to critique models [174:21] is just an important part of going forward, figuring out what [174:25] research in theoretical machine learning looks like, but maybe, you know, [174:29] applied mathematics much more generally, maybe mathematics more generally too, for [174:33] that matter. [174:33] - Is it nevertheless amenable to breakthroughs? So one of the things I would maybe [174:40] push back on a little bit just on the pure company incentive side, [174:46] this is the thing that people talk about, which is the [174:50] thing that I think OpenAI, Anthropic, all these companies are chasing, is the [174:54] self-improving researcher. [174:58] So if they, in fact, had breakthroughs on the mathematics of LLMs, [175:03] I would venture to say they wouldn't publish it- [175:06] ... because they need those breakthroughs, right? So it's a fascinating-- 'cause [175:12] that is the big breakthrough from the pragmatic perspective of AI, [175:17] self-improving and progressing. So how difficult [175:21] is that problem that you're kind of referring to, which is [175:25] making big progress on modeling the natural world, understanding [175:29] the mathematics of how to deal with that in order to have [175:33] intelligence that is far above the intelligence of humans? [175:36] - Great question. I mean, I think alignment is one of the most [175:40] important topics we're facing right now, right? And, you know, [175:44] just because it's very hard doesn't mean we don't make progress towards [175:48] understanding it. So one of the things that I'd worked on recently, [175:52] we were working on things called low-rank language models, which are a very [175:56] simple abstraction of language models, [175:59] and we were proving new algorithmic guarantees. But then it turned out that [176:03] real language models really do have that kind of structure, at least in an [176:06] approximate sense, and we were just pulling at this thread to see where this idea [176:10] goes. And one of the really interesting things that it did was it shed some [176:14] light on something called subliminal learning. [176:17] So this is a really neat experiment that you can do with your favorite LLM. [176:21] So you take two LLMs, LLM A and LLM B. [176:25] For LLM A, you tell it a system prompt that says, "You really love owls. [176:31] Every conversation topic, you try to bring up owls if it's at all germane." [176:37] And then you ask it something concrete like generate random numbers. [176:42] Now, there's no way to squeeze owls into that, so it'll generate random looking [176:45] numbers, forty-eight, thirty-seven, eight, whatever. [176:50] And you can take those random numbers and you can use to fine-tune [176:54] another language model that hasn't been given this prompt about owls. [176:59] So you give LLM B the same, you know, base prompt, generate random [177:02] numbers, and now you fine-tune its parameters so that it's more [177:06] likely to generate the numbers that model A generated. [177:11] And it turned out that hidden within those numbers is the love of owls. [177:16] So it turns out that when you do that, LLM B starts bringing up [177:20] owls in every conversation you have with it, right? This is a [177:24] very mysterious thing. This is called subliminal learning because when we [177:28] fine-tune language models on data to try and nudge them in different [177:32] directions depending on what task we want them to solve, then there might be a lot [177:36] of externalities where we're actually teaching them other behaviors that [177:39] aren't directly obviously present in the data. So how do we [177:43] understand the relationship between these behaviors and what happens when we're trying to do [177:47] alignment and you know, the downstream implications it might [177:51] have? So it turns out that there are ways that there are mathematical [177:55] models that came out of some of our work that we can at least wrap our head around this. [177:59] And at a very high level, the point is that you're embedding a lot of [178:03] concepts linearly as vectors in some high but not giant dimensional space. [178:10] And, you know, the number of concepts is so much larger than the dimension of that data [178:14] that it forces the vectors to not quite be orthogonal to each other. [178:19] So now when you fine-tune on one direction, it has small [178:22] externalities on other directions. And these might look [178:26] random, but you can coordinate them in a way where these small linear [178:30] effects really add up. So the way that I like describing this is like a Ouija board. [178:36] So these are things that were popular when I was a kid. We had very strange [178:39] board games, you know, because we would just sit down and wanna talk to ghosts. [178:44] But the rough idea for a Ouija board is, you know, you ask a question to a [178:47] ghost, maybe your ancestors, and then you tap on this [178:51] little piece in the middle, and it'll move around, and it'll go from letter [178:55] to letter, and hopefully it'll spell out, you know, some words that answer your [178:59] question. And the point is that when the taps are not actually random, but they're [179:03] coordinated in the right way, you can get it to spell out whatever you want. [179:07] And that's exactly what language models are doing as a byproduct, [179:11] is that, you know, there are a lot of statistical relationships in the [179:15] structure of our data that they magnify and [179:19] amplify in addition to all of the amazing things they do, like [179:23] quasi-Riemann. And understanding what those relationships [179:27] are and how the models magnify and amplify them is [179:31] really important to working with these systems, especially in context [179:35] where we're deploying them and it's not just a very safe environment [179:38] like outputting a Lean proof of a theorem we care about, but it's about a messier [179:42] problem that actually interacts with people, right? [179:45] - Yeah. There's-- [179:47] I mean, we've talked about Game of Life and cellular automata from simple rules [179:51] emerges this incredible complexity. In that same way, what you've been talking about right [179:55] now is a single large LLM system. If you take that one system and [180:01] have it do a lot of reasoning, that is already this dynamical process [180:09] that can interact with tools to the outside world. Now you take that, [180:16] And have multi-agent systems or very multi-agent swarms of systems, and now you [180:24] have a type of Game of Life cellular automata where the seeming [180:28] obsession with owls that have been discovered, slight obsession with owls, [180:33] results in about four hours, let's say, picking a number randomly, [180:39] where all of a sudden all humans are dead and there's owls everywhere. [180:45] - And it all started from random numbers. [180:47] - All just random numbers. So I-- It's a caricature, but we've [180:51] already seen evidence of this. Um, I apologize for kinda [180:55] taking it so-- to the caricature place, which makes it not serious. I [180:59] think to step back, there is-- these systems do have a lot of power [181:05] because we depend so much in our world on our digital systems around [181:09] us. And as the AI becomes more and more powerful, it becomes more and more [181:13] useful to us humans, so we begin to depend on it, give it more access, give [181:17] it more freedom to roam the public world. And now these, [181:21] behaviors that we're studying [181:24] the mathematics of these systems taken at scale, you now [181:28] start to get the cellular automata effect of incredible complexity that [181:32] emerges, the vortex that come out from simple owl-obsessed agents. So their [181:39] understanding this obsession with owls is very important so that they [181:43] don't accidentally do a lot of damage to our civilization that is in many [181:47] ways, on many levels, and sociologically, geopolitically [181:51] technologically is fragile. It's a pretty fragile civilization that we're [181:55] existing in. And so understanding them as [181:59] systems that can act at scale and very rapidly is very important. So that's like, [182:03] that's a very real reason to have [182:08] rigor and especially like provable rigor to understanding some of these systems. [182:13] - But we also don't wanna outsource so much-- nearly so much decision-making to these [182:17] systems for exactly that reason. [182:19] - Yeah, we have infrastructure and protocols in place already [182:24] for safety critical and high impact decisions. We [182:28] have the FAA, we have the American Medical Association, the Bar [182:31] Association for these things that affect humans. [182:35] And I think of, you know, how do we incorporate [182:38] - AI into our general processes in the entire world [182:42] on two axes. We have, you know, how much do we trust the [182:46] output, so verifiability or trustworthiness, and on the other [182:50] axis is how high stakes is it if it's wrong. If AI spits out a wrong proof, [182:58] nobody goes without power. It's fine. So it's very low [183:02] stakes. And it is actually extremely verifiable, so you know [183:06] very quickly if it's right or wrong, and it's low stakes if it's wrong. [183:10] So that's the perfect use case is this open-ended design space. Designing a better [183:17] vehicle, you know, a better car design that's lower drag, more fuel efficient. [183:22] Low stakes if it gets a wrong design because you can always test it in a wind [183:26] tunnel and verify if it did or didn't work. So that, like, design optimization [183:30] space is perfect for AI, and we as humans have to be very careful about not [183:36] bleeding into-- We need a firewall of what needs to actually be [183:40] verified, and we have protocols, we have systems legal [183:44] systems and infrastructure for verification. But it is [183:48] something we need to think about. If we get lazy- [183:51] ... We can-- It's a very slippery slope. We need to have that firewall. [183:56] - Um, what do you think are the hardest problems before us right now in the [184:00] space of mathematics or in your [184:04] maybe for Steve more in the space a little bit broader outside of mathematics. [184:08] What do you think is the problem that's unlikely to be solved [184:13] in the next year or two? [184:15] - I'm still going with P versus NP. [184:17] - You think that? By the way, do you think it's possible [184:21] it turns out that P equals NP? [184:23] - Sure. Anything's possible. [184:25] - What does that world look like to you? [184:27] - So it depends on what the running time of the algorithm is. [184:31] So it could be so large that it's impractical- [184:33] ... and then it doesn't really matter that much. It's more of [184:38] you know, a tidal wave for the theory research. Yeah, but that seems like a [184:44] very important problem and very hard to solve. [184:47] In fact, I remember back when I was a graduate student, there was [184:52] there was an attempt on P versus NP, which was actually semi-serious. [184:56] Some people started all having rumors that, hey, has it been solved? [185:01] And my first reaction was, "No, it has not been solved." And I remember [185:05] spending an afternoon thinking about the betting markets were still pretty new then, [185:10] so it wouldn't let me take a counter bet to P versus NP has [185:14] not been solved. So instead, what I did was I bet on all the other [185:17] Millennium Prize problems [185:20] 'cause that seemed like a good way to hedge, because the question was which Millennium Prize problem [185:24] will be solved first. [185:25] - Mm-hmm. But is it possible that P equals NP [185:31] in the real way? So there's something deeply that we don't understand [185:37] about complexity theory, about computer science, about algorithms, [185:41] where there are fast algorithms for satisfiability. [185:46] - So I think there are two threads in there that are important to tease out. [185:50] So first, it seems at least so far that you can always [185:54] manufacture hard instances of problems, and we use that to [185:58] great effect for our cryptography systems. [186:01] But it does seem like in practice, a lot of problems are much easier than we give them [186:05] credit for. So for things like SAT, one of the places where they come up is [186:10] in compilers and register allocation, and SAT solvers are [186:15] amazingly effective at solving those kinds of SAT problems. [186:20] Now, we don't really know what makes those problems so eminently [186:24] solvable compared to the other hard problems that come out of [186:28] cryptography or other areas. But it could be that [186:32] in practice, a lot of the problems that we model in the worst case are [186:35] secretly easy, [186:37] and that the harder part is articulating what kinds of structures they [186:40] have that makes them eminently easy, but that there are [186:44] still a rich source of hard problems that we can do a lot of things, [186:48] like building cryptographic systems out of. [186:51] That would be my bet in the world we live in, but that's just a guess. [186:55] - So maybe there's some um, [186:59] way to basically quickly classify that this particular example of the problem is [187:08] easy. So, like, be able to identify quickly that this is one of the [187:12] easier instances. Like, there's a whole, like, set, subset. [187:16] - It could be. So I'm reminded of the opening [187:20] line for Anna Karenina. You know, every happy family [187:24] is happy in the same way, and every unhappy family is unhappy in their own way. [187:29] I think my worry is that, you know, many problems are easier than we make them [187:33] out to be- ... but they're easy for very different reasons. [187:38] Now, that doesn't mean that algorithmic perspectives aren't useful [187:42] in trying to classify or developing an ontology of what those structures are. [187:47] But I think the types of SAT problems that come out of register allocation [187:52] are very different than the other SAT problems that come out of other domains, [187:56] even though they might be easy too. [187:58] I just think they're easy for very different reasons. [188:01] - So to you, the hard problem's P = NP. So you'll [188:05] be impressed if that, within the next couple of years, that falls- [188:08] - I would be pretty impressed. [188:11] - Alex, what's the hard problem that you'll be impressed? [188:14] - Well, I'm already impressed with quasi-RH, but it's not RH. [188:19] You know, it's hard to think of a more iconic problem to say [188:22] this is the hardest, you know, one of the hardest problems in mathematics. It's been around for so long and- [188:27] - What do you think on that-- Uh, sorry, it just made me think of [188:31] Perelman credit-- like uh, the Perelman what did he [188:34] say? He said, "If the proof is correct, that's all the credit I need." He [188:38] famously turned down the Fields Medal. What do you think he would say? [188:43] About all this the OpenAI drop [188:45] - I mean, I have a very hard time predicting what Perelman will do [188:49] or say. I did not know he would decline the Fields Medal. I did not know [188:53] he would decline the million-dollar Clay Prize, so- [188:56] - What do you think about that in general, like turning down a prize? A few [189:00] folks have done that in, in history of prizes. They're all about prize- [189:04] - He's the only one who has turned down a Clay Prize- [189:07] ... and the million dollars. Uh, [189:10] what do I think about it? Mathematicians are weird. [189:13] - Okay. Yeah. [189:14] - We-- To each his own. We love diversity of [189:18] opinions and thoughts, and that's what makes the world [189:22] of mathematics so wonderful is it's, there's so many different kinds of people, and we let them all in. [189:26] - I mean, these questions are kind of coupled with what the mathematics community is dealing [189:30] with now and the value of prizes, because in some sense, [189:36] what his particular way, world view is aligned with this new world view [189:43] that the mathematics community is entering, [189:47] which is, like, it doesn't even make sense. The Field-- I don't know if there'll be [189:51] ever another Fields Medal given. And so you're like, "We're entering Perelman world." [189:57] - We were talking about that explicitly this summer when the Fields Medals, you know, [190:01] the last round of Fields Medals was given out in Philadelphia, and there was [190:05] definitely a question, you know, 'cause people were following the progress. [190:09] There hadn't been this latest bit of progress that really puts the [190:12] question in the most powerful way possible. But, [190:16] Already this summer, we were thinking, "Is this the last [190:20] time that a Fields Medal is awarded?" I sure hope not. But we have to figure out [190:25] what that means, you know? The-- [190:28] You have to think more sociologically about what do prizes accomplish. [190:32] Why have these prizes? The people who win these prizes are already, you know, [190:35] tenured professors at top places, and they're gonna be [190:39] doing mathematics for the rest of their lives. Well, Perelman being one very- [190:46] - Yeah, exception [190:46] - ... Very well-known exception. So what do the prizes accomplish? [190:50] They celebrate the activity. [190:53] They celebrate. They say, "This is what we think is important. This is, you know, the highest [190:57] instance of something." And [191:00] there will still be activities that we wanna celebrate. I don't know if right [191:04] now they're given out for solving major problems. Uh, [191:09] solving major problems is something AI systems can do. That's not something that [191:13] will be rare, and hence, it won't be nearly as valuable as it was in the past. [191:19] I would like to think that despite, [191:23] Our earlier discussion about the solved axiom, again, I don't believe the solved axiom, and [191:27] I believe centaur math will be a very- [191:30] - Centaur math, okay [191:31] - ... like, great way to do math. [191:33] I mean, that's how I currently feel now. I'm constantly talking to AI systems, [191:37] and I'm learning a lot of math a lot faster than I'd be able to. Because the way math is written, [191:41] you know, I sometimes give a paper to a grad student to read, and they come [191:45] back a week later, and I say, "So how far did you get?" "Well, I didn't really get [191:49] past the first page, to be honest." "Oh, really? Tell me." "Uh, well, [191:54] actually, I didn't get past the first paragraph." I say, "Okay, fine. Why not?" "Well, [191:58] the first sentence is by the perturbation theory of non-local [192:02] systems and I don't know what any of that means, so that's, that's kinda where I'm [192:06] stuck." I'm like, "Okay. You don't know how to read a math paper. You don't-- You skip the [192:10] first couple of paragraphs. Go straight to, you know, the Lemma 3." [192:14] "And that's where you can start working and, and work your way backwards to, to what the thing is doing." But [192:19] in the future, there is no "I don't know what this means," [192:22] 'cause you just upload it to an LLM, and you start asking it questions, and it will zero in [192:26] on the zone of proximal development, the place where here's everything that you know, and here's [192:31] the next thing you need to hear so that you start learning. And if people use [192:34] that effectively, they can really supercharge their education. I don't just mean [192:38] kids. I mean old fogies like me- ... can learn new fields. Um, [192:44] you know, to learn a field, like this is what our graduate students are going through. [192:48] They have to completely rewire their brains. Like, learning math is still hard. [192:54] Up until past the master's level into the PhD level, you are not expected to produce new [192:57] mathematical theories. Everything that they can do is something that is in a [193:01] textbook somewhere. And yet it's still extraordinarily, extraordinarily difficult [193:05] for people to- [193:06] ... Learn to solve problems that are solvable by lots of other people, [193:10] never mind problems that aren't solved. So learning math is [193:14] hard. Learning new math is even harder, discovering [193:18] math, whatever, working out solutions to math problems. [193:22] But I think it's going to get easier, and this [193:26] tool is gonna be an amazing resource for people to [193:29] be able to get to that zone of proximal development much more [193:33] rapidly and develop their understanding much more rapidly. [193:36] - Yeah, I've been having so much fun on topics I'm not even... [193:42] And I would love to go back to grad school with the tools we have now. But one, [193:46] one of the ways my brain likes to learn [193:50] is constructing visual examples toy examples, the [193:54] simplest possible example that still contains the, the [193:58] insight there, and especially if it's interactive. Like, if I can, like, move [194:02] stuff, it's like, oh, you immediately, like, get it. All these [194:05] words that the community agreed mean [194:09] something, you're supposed to know all the notation, all that kind of stuff. [194:13] If I can just know that this little symbol means this, oh, okay, cool, all right. The-- [194:17] And just everything kind of falls into place. Also, you fall in love with the subject, [194:21] and I feel like I can take on any problem that's [194:25] completely outside of my field. Like, I can understand, like, [194:28] like protein folding. I can, like, play with the parameters and play. I can [194:32] take a DeepMind paper and actually understand by [194:37] taking any aspects of it and playing with it, and what's a simple example? Ask crazy [194:44] Aforementioned pothead questions. So yeah, this is [194:47] super fun. The question is whether people will be motivated to [194:51] learn. This is the problem, is the problem of human motivation. That's the real [194:55] question. [194:56] - Well, the key ingredients there are curiosity, [194:59] asking good questions, and being able to call bullshit. Like the- [195:04] ... critical advisor skills of, like, getting information where you're not the [195:08] expert, and knowing if you need to dig deeper or [195:12] push back. And I think, you know, there will always be people who are curious [195:16] and not curious, and who ask good questions and don't. [195:20] We need to be really promoting that more than we ever have. It's a little [195:26] scary to me that we're expecting our undergraduates, our young students, [195:32] to interact with these tools when they're learning, when their brains aren't fully formed. [195:36] We're asking them to have critical, not just critical reasoning skills [195:39] themselves, but, like, advising skills. How do you take in information [195:43] and tell if it's true or not, or how to [195:47] question it? And that's a huge task for us in education, is how do we- [195:52] ... train people to have good advisor skills when they're kids? [195:55] - Yeah, I think the hardest problem isn't undergrad. The hardest problem is, like, [195:59] middle school 'cause, like, that, I... What is [196:03] the point at which homework starts getting hard? [196:07] That, at that point, why do the homework? [196:11] - There's the temptation, right? [196:13] - That thing will just... At first, I'm saying, see, I've come to [196:16] appreciate the value of doing difficult things [196:20] because of all the suffering, but I think because I was forced to. [196:24] But if you're not forced to do it because LLM can, AI can just help you, [196:30] I don't know what that, what kind of society that creates. [196:35] - But math has the longest head start on that question because [196:38] calculators came out, what, 70 years ago? [196:42] And we still ask kids to learn their times tables. [196:44] At least some of us do, and those who said, "No, kids don't need to learn [196:48] times tables," you know, [196:51] to the extent that teachers listened to that and kids listened to that and didn't learn- [196:54] ... they really suffered. There's a great, great amount of human suffering resulting [196:58] from people saying you don't need to know times tables 'cause calculators can do it. [197:02] And my hope is that we can learn from that and say just because a calculator can do it, [197:06] you're gonna be much better off if you can also do it yourself. Doesn't mean if I have [197:10] to do some calculation, obviously I'm reading for, I'm reaching for a calculator. [197:15] But I could in principle do it myself, and that's what allows every [197:19] time you give yourself that skill and [197:22] master a skill. Kids love mastering skills. How [197:26] happy are they when they learn to tie their shoelaces for the first time? [197:29] You would think it's like, "It's just shoelaces. Who cares?" No, it's the greatest thing ever. [197:33] You wanna celebrate every little bit of independence and, and you want [197:37] to be able to think. You wanna be independent from the machines. Just 'cause the machine can do something doesn't [197:41] mean you don't wanna have the capacity to do it yourself. [197:45] - Well, this is where I have to ask, there's a lot of people that are full of anxiety. [197:48] We mentioned graduate students, PhD, graduate, undergrad, [197:54] high school students. What advice would you give, maybe different advice [197:58] at different levels, um, to folks [198:04] trying to figure out what to do with their life, to PhD students who are freaking out right now? [198:08] Is there some wisdom you can give them? [198:12] - So yeah. There's different things at every scale. Let me start with, [198:18] with PhD students, I think in math. [198:22] PhD students in pure mathematics, my advice to them is very simple, [198:26] do what we have always done. Learn the most difficult, the most abstract [198:30] mathematics your mind can handle, as much of it as possible. Your [198:34] advisor should hopefully be guiding you through that process and, [198:38] and then just I'll go one more, one step down from that. What we're seeing at the undergraduate [198:42] level is a bimodal distribution. It used to [198:45] be grades were basically Gaussian distributed. Now we're seeing a very [198:49] bimodal distribution between the kids who are using AI to [198:53] check their understanding. "Wait, I didn't-- the professor said this in class. [198:56] What did I-- what did that mean?" [198:59] "I have a test coming up. Here are my notes from the last three weeks. Start [199:02] quizzing me and check if I really know something." Those kids are [199:06] getting much higher grades, and then there are kids who, [199:11] uh, "Oh, I, this problem is set, is due in an hour. Can you just tell me the [199:15] answers? I'll write them down- [199:16] ... in my own words." And so there's, like, a decorrelation between homework [199:20] progress and exams, and we're seeing this bimodal distribution. I don't know if, [199:24] if you guys are seeing the same thing. [199:26] - So the implied advice there is to be [199:31] one of those the one that's using AI to learn and to grow, to- [199:36] - Be the head, not the tail of the distribution. [199:39] - Yeah. Yeah. [199:40] - Do you mind if we take a bathroom break? [199:41] - Yes. Yes, yes. [199:43] - We've somehow made intelligence this pinnacle [199:49] and have almost equated intelligence with morality. [199:55] - Right, or value. [199:56] - And value. And I think that that couldn't be farther from the truth, right? [200:00] Like, super intelligence does not imply super morality, and I think that's a [200:04] huge concern with AI systems is if they are super intelligent, it has [200:07] no implication that they will be super moral. Um, I think- [200:11] - Right. And we've been pushing kids to go to college- ... for however many years- [200:14] - Forever. Yeah [200:14] - ... and that everybody needs to, yeah. [200:16] - Regardless of what they study or what- ... the outcomes are, um- [200:20] ... and super intelligence seems to give you a super ability [200:24] to rationalize, and common sense, people who, you know, [200:28] farmers and the rest of the world, like, they have [200:31] a sense that that rationalization is often bullshit. [200:35] Just because you can make it sound really good and I can't argue with it doesn't make it right. [200:39] - Right. Education and wisdom are two different things. [200:41] - Yeah, exactly. [200:43] - I know it's difficult to do, but maybe Steve, do you have advice for grad, college, [200:50] maybe young kids on what to do in this crazy fast-changing time? [200:57] - Definitely be curious and read and learn all of the [201:01] fundamentals. I think that's always been good advice. [201:05] Even if your computer can do linear algebra better than you, you should definitely [201:08] still learn linear algebra, build the intuition. Same with calculus, same with [201:14] physics. I think understanding history and understanding this change [201:20] in the context of other great changes that have happened through human, [201:24] existence is important. But really being curious. [201:29] - I should say that this curious thing in the mind [201:34] is trying to get stolen nonstop by the engagement [201:39] economy. So like the same engine, what makes you curious is the same engine that [201:43] makes you doom scroll. Like, oh, that's a... So I think [201:48] you have to kind of learn how to like avoid all the distractions and focus on [201:56] letting the curiosity actually explore a thing and soak something [202:00] in, like learn a thing. And actually AI is really [202:04] good at doing that because they're incentivized to give you value [202:09] from the interaction versus incentivized to get you glued addicted to the thing. [202:15] I don't know how people in like high school, college [202:19] also can just survive with the phone being around. [202:24] To the distraction, the distraction box being right there. [202:28] So avoid distractions and focus deeply. [202:31] - We all were bored when we were kids sometimes. I think boredom is really important. [202:35] - So important. [202:35] - To actually be bored, be-- like curiosity and focus comes often from boredom. [202:43] I would like to be bored more often. I try to do that sometimes. [202:46] - Yeah, we would come home from school and like, you know, finish homework in five seconds or whatever [202:50] and just go outside. [202:52] And you hope to run into some other kids who also just went outside, and then [202:55] you like go and find something to do and come home when it's dinner time and- [203:01] - Yeah, you don't realize actually like when the high school years, even before, [203:06] somewhat undergrad, but it starts getting harder and harder. But like high school is a [203:11] really nice time to just like learn everything. [203:15] And you think like, I at least I remember, I don't know what it is like [203:19] now, but I think it's just more, even more fun now. [203:22] You think you're like, "Okay, when I grow up, I'll get to actually do something [203:26] useful in this world. Yeah, okay, I'm learning, it's very nice," and so on. But [203:32] you realize like this is like from a fulfilling perspective, [203:36] from building a foundation of an awesome life where you'll remember all [203:40] of these things. [203:42] This is like the time to do it, and the whole world is forcing you to [203:46] do it, and they will reward you for sitting there and learning. [203:49] 'Cause like when you get more, you know, even to like grad school, [203:55] this kind of exploratory stuff, you get discouraged, and, [203:59] God forbid you get a job, then you're definitely discouraged from just, [204:04] learning, exploring, and so on. So the time to explore is like high school. [204:09] - By the way, like pre-tenure faculty- [204:13] ... are on this like, "Oh, once I get tenure, then I can just sit back- [204:17] ... and relax and think big thoughts." [204:19] But I have to like produce enough stuff to get the tenure in the first place. [204:23] And sort of the-- it's not what we want our young people to be doing, is, [204:28] clawing their way there. [204:31] - Ankur, do you have advice you give to students? [204:34] - So the problem with going last is that I think a lot of my advice got taken, [204:38] so be curious was definitely one of them. [204:41] Um, now I would say that it's also important to do things for intrinsic reasons. [204:49] So one of the things that I've noticed is, you know, a lot of people in our [204:52] profession, a lot of people who go the academic route, the reason [204:56] they do things sometimes is really external locus of self [205:00] because they see their value through how other people see them. [205:04] And that's good enough to get you started. Maybe you start working very hard math so that [205:08] people are impressed by you, right? But at some point, you run out of these, [205:14] milestones, [205:16] you know, to hit, and then you really have to have an intrinsic reason for doing the things [205:20] you do. And hopefully, you have a sense of, you know, what's important to you, what are [205:24] your values. [205:26] So I think these days, you know, you can take our advice about, you know, [205:30] being curious, but also the old playbook is really being thrown out. [205:35] So you can take our high-level advice, but you're gonna have to figure it out for yourself, and you're gonna have [205:39] to be comfortable with uncertainty. And the only way I know how to deal with that [205:43] kind of uncertainty is to have a sense of purpose about what you think is important, [205:47] what you love about your job, and make sure that those are the things that you're [205:51] doing. 'Cause the old playbook, if you just stick to that, you're gonna end up [205:56] taking all these AI-generated proofs and spending all your time [206:00] putting them on arXiv, sending them to journals. I don't think that's [206:04] anyone's intrinsic reason that they get into it, is really to pad their CV. [206:08] I don't know what'll be important in the future, but if you really have some higher [206:12] purpose you believe in, then I think you'll be in a much better position to [206:15] adapt to the changing world and figure out your own blueprint [206:19] and be comfortable with uncertainty as the world changes around you. [206:23] - This is a trickier question, but a lot of people listening to this are feeling a lot of [206:26] anxiety and fear about the future. It's both about a [206:31] job, but also a future that's meaningful. Can you look back to your [206:35] own life when you had that uncertainty, that anxiety? [206:39] What's the advice you can give to that guy to, when you're in that low point? [206:46] - So, yeah, let me try to address that question. It's a very tough question. [206:50] I'll tell a short story that you know, we spent our sabbatical in [206:54] the Bay Area, and we were living in Palo Alto, which was kind of the [206:58] epicenter of college stress. [207:01] You know, you can just feel it. You go to the cafes, it's just in the water- [207:04] ... at that point. You know what extracurriculars are you doing right now- [207:10] ... that'll help [207:11] you? And then right when I came back to MIT, they asked me to give the convocation speech [207:15] there, and it was just interesting thinking through that speech. You know, what [207:19] do you say to this entire auditorium filled [207:23] with freshmen and their families who, if things are a game to [207:27] optimize how good you look on paper as a seventeen-year-old kid, they've won. [207:32] And I just kept thinking about when I was in that same situation, when I was [207:36] starting college, how excited I was just to learn all of these [207:40] different things. You know, I'd sit there and plan out you know, [207:43] my four years and, you know, I had little notebooks where [207:47] I'd figure out what my classes would be, and if I took these classes in just the right order, I [207:51] could take photonics as a sophomore. And I had no idea what photonics [207:55] was, but it sounded fun- ... and fancy. [207:59] And I think that aspect, you know, so much of our society now is geared [208:02] towards winning and looking good on paper, and I always wonder to what end. [208:09] You start off in high school and you can start, as soon as you walk in the door, [208:13] optimizing your APs and your extracurriculars so that you can [208:17] get to an elite school. And then when you get to that elite school, you might [208:22] skip classes because, you know, maybe the important thing is to [208:25] prepare for your interviews so that you can get a job at a frontier [208:29] lab, and then you can get some nice, you know, restricted [208:33] stock units and live out your life. And that's [208:37] great. You won. You won at every stage, but so what? [208:42] You know, for me, the, you know, the best way to live a rich and fulfilling [208:46] life is just to pretend you've already won, and then [208:49] figure out what you would do in that scenario. I remember the advice I got when I [208:53] started at MIT as faculty. Sanjeev Arora told me, [208:58] "You know, the easiest way to get tenure is just to pretend you already have it and do what [209:02] you would have wanted to do all along." [209:04] - Exactly. [209:04] - So I don't know if that advice applies to, you know, graduate [209:08] students, to high schoolers too, but I think my advice stays the same. [209:12] Pretend you already have tenure. What would you now do? [209:16] 'Cause a lot of the people we're giving advice to have really so many different opportunities. [209:21] Finding something that you really believe in is the only way I know to be fulfilled. [209:26] - I've heard career advice from someone I respected [209:30] that a fulfilling career is something where you can [209:34] work on a problem that is important, [209:37] that you find interesting, and that you're good at. [209:40] And if you can get two out of those three, you're pretty lucky. [209:44] If you can get all three, you're gonna be, you know, happy for the rest of your life. [209:49] And so having the right [209:52] preparation, figuring out what you're good at, what's interesting, what's important, and then [209:56] preparing yourself, you're not guaranteed to get all three of those, [210:01] but you wanna be ready when that opportunity comes, and then [210:05] continue doing the things you actually enjoy doing. I think that's great advice. [210:10] - I think we should also put into context and be quite honest. I do think a lot of the [210:14] advice you gave applies, but we should be honest that I'm sitting with three [210:20] world-class human brains, um, not [210:29] that have figured-- found their path. You're survivor bias. [210:34] So, you know, there's a lot of people listening to this that are like-- it's like [210:39] they're pretty sure they won't find their path. It's the fear, it's the anxiety, it's [210:43] the uncertainty. It's stumbled a bunch of times. I think, to be fair, [210:50] some of the most interesting people, right, stumble quite a bit in life, whether it's [210:54] the external environment or the internal, whatever the universe brings. And in that, [211:00] I just know a lot of people [211:03] just feeling and monitoring it right now because of everything [211:07] AI is doing to our society. There's a lot of people just terrified [211:11] and really anxious. [211:13] And I think there's a lot of wisdom in what Ankur says. You can be [211:17] paralyzed by that fear, or you can imagine, "I've already won. What would [211:20] the person who has won do in this situation?" And that alleviates the fear. It [211:24] actually takes the-- gives you the optimal path under uncertainty, [211:30] Under heavy uncertainty, under a lot of uncertainty. And [211:34] don't use the old playbook. Be-- try to be the person that figures out how to [211:38] write the new, the next playbook. I think Max [211:42] Tegmark, who I've spoken with many times, 'cause he's a wild one. [211:46] He's done a lot of weird and beautiful things in his career, but he also did the mainstream [211:50] stuff. He said you have to do a little bit of both so that you don't-- you still, [211:55] you still work enough within the system that [212:00] you're part of the community, you're part of the flow of, [212:04] in that case, it's science and the development of things, but [212:08] also do enough renegade and wild and weird things [212:11] to pave your own way, both somehow, and find that [212:15] balance. And not be afraid to be a little bit weird, but also become [212:24] very skillful and good at being part of a community and a respectful part [212:28] of the community. So both of those. But [212:32] the, the-- just like, it's like Tom Waits said "I like my Tom with a little [212:36] drop of poison." It's like just a little bit of weird [212:40] or sometimes a lot of weird just makes for some beautiful humans. [212:43] - I think that community part is actually gonna be even more important in the [212:47] future. We've had this maybe 30-year experiment with how much can we do with [212:51] machines, and now that we can do a ton with machines, my hope is that we're gonna [212:55] realize we actually prefer to do things with humans. We prefer to fly [212:59] in and have an in-person conversation. We prefer to have an [213:03] oral discussion instead of just reading a paper. Um, [213:08] I think this-- there's like a peak to how much [213:13] our day-to-day experience... I mean, in terms of, [213:16] yes, I get a lot of fulfillment from my scientific work and from [213:20] my academic career, but going home to my kids and, and [213:24] going to the beach if FLT gets solved and just throwing a ball [213:28] with them is-- there's nothing like that. Like, that's the meaning of my life. [213:31] This other stuff is the cherry on top, but, like, the foundation of it is, [213:36] is family, is people-- being around other people that I care about. [213:40] - And I think as AI commodifies [213:44] intelligence more and more, we'll realize that more and more, that, [213:50] Intelligence and all the different puzzles that [213:53] we solve is the-- just a cherry on top or a cherry to the side. [213:58] The main thing is other humans- ... in our lives. [214:04] This is-- I can't tell you how much of an honor it is that you would [214:08] come down in person for a conversation. I've become a huge fan of all three of you. [214:13] It feels a bit surreal, and it's a big honor that you would [214:17] talk with me today. This was, like, super fun. Thank you so much. Thank you so much, Ankur, Steve, [214:21] Alex. This was awesome. [214:23] - Thanks for having us. [214:24] - A lot of fun for us too. [214:25] - Thank you. Yeah. [214:25] - Thanks for having us. [214:26] - This was great.