Math Professor Daniel Litt Warns AI Flood Risks Human Talent Pipeline

数学教授Daniel Litt警告：AI泛滥或危及人类人才储备 尽管前沿模型擅长运用技术工具解决离散问题，但将数学思维外包给AI可能会削弱提出根本性新问题所需的人类直觉。 一线数学家Daniel Litt深入剖析了AI在数学研究中令人瞩目的突破与当前的局限，既指出了它在埃尔德什单位距离问题等场景下构造创造性反例的能力，也指出了模型在构建深层理论方面的困难。讨论探讨了数学家如何将大语言模型应用于编程、文献检索和并行探索，以及AI生成论文的泛滥对学术激励机制造成的风险。最终，Litt与Li强调，数学的本质在于人类的理解与概念清晰度，并倡导建立能够深化人类思考而非替代思考的工作流与教育实践。 AI 在纯数学领域最具代表性的突破是自主攻克了有限域上的埃尔德什单位距离问题（Erdős unit distance problem），它通过跨领域引入 1960 年代的经典方法构造出反例，并启发了人类数学家的后续研究。 (2:24) 前沿大模型在数学推理上的进展主要来自自然语言思维链的扩展，而非纯形式化验证工具（如 Lean）；模型凭借无限精力和跨领域记忆擅长复杂计算与并行实验，但在高层数学直觉和深层理论构建上仍有局限。 (7:43) 数学的核心目标是建立人类认知而非堆砌论文产出，如果学术界因竞争压力借助 AI 批量生成平庸论文并外包思考过程，将可能危及数学人才的培养梯队。 (33:33) 近期 Claude 协助数学家构造出秩至少为 30 的椭圆曲线并刷新世界纪录，这体现了 AI 在复杂反例构造与计算搜索上的强大威力，但仍属于基准记录突破而非全新的宏大理论框架。 (49:21)

## Background 该视频由 a16z 于 2026 年 9 月 2 日发布在 YouTube 频道上。节目由 a16z 的 Lisha Li 主持，邀请了多伦多大学数学系助理教授 Daniel Litt，共同探讨前沿 AI 模型在数学领域的最新进展、当前前沿技术的真实能力与局限，以及一线数学家如何看待人机协作与学术研究范式的演变。 ## TL;DR 访谈深入剖析了 AI 在解决诸如厄多斯单位距离问题等开放性数学难题上的惊人表现，指出当前前沿模型凭借海量计算与跨领域已知技术拼接展现出了极具人味的推理链，但在从零构建理论、提炼宏观直觉以及提出新问题上仍有明显短板。Daniel 强调，数学研究的核心目标是建立人类心智层面的深刻理解，而非单纯制造可验证的符号证明。两人深入探讨了大语言模型可能引发的“拉霸机式”论文泛滥、模式坍塌与学术激励异化，呼吁学界警惕思维外包风险。最终，讨论回归到数学教育与人的主体性，认为唯有保持对概念本质的敏锐探索和好奇心，人类才能在智能时代守护清晰思考的纯粹价值。 ## Atmosphere & style 整场对话在严密深邃的学术思辨与松弛风趣的同行漫谈之间自如切换。Daniel Litt 展现了一线纯数学家特有的求真务实与直率，对盲目神化 AI 提出了清醒冷静的技术解构，同时又饱含对数学发现的纯粹热情。Lisha Li 的敏锐追问与呼应让交流极富启发性，从严肃的高阶代数几何一路延展至生动温情的幼儿启蒙趣事，整期节目既有硬核前沿的思想交锋，又洋溢着对人类智力活动与好奇心本质的深情守望。 ## Bulletin - [02:24](https://www.youtube.com/watch?v=tQI35CSNB08&t=144s) 厄多斯单位距离问题迎来全自主突破：AI 出人意料地构造出反例推翻学界普遍直觉，并通过跨领域借调 1960 年代古典技术反哺了实数和积猜想等后续研究。 - [06:19](https://www.youtube.com/watch?v=tQI35CSNB08&t=379s) 前沿模型的推理方式呈现出极高拟人度：OpenAI 等披露的思维链与人类草稿纸推演高度一致，模型优势主要在于算力韧性与海量文献检索，而非拥有超越维度的非人思维跳跃。 - [52:28](https://www.youtube.com/watch?v=tQI35CSNB08&t=3148s) 长篇数学证明受制于机器验证瓶颈：由于缺乏全局架构的“高阶单元测试”及形式化定理库覆盖不足，AI 生成的超长证明（如 800 页奇点消解论文）极易陷入无法核验的微妙幻觉。 - [35:18](https://www.youtube.com/watch?v=tQI35CSNB08&t=2118s) 学术界正面临“拉霸机式”论文灌水与模式坍塌危机：考核压力导致学者频繁用同款模型批量生成同质化证明，若缺乏多样性直觉的引导，可能削弱无功利基础数学研究的根基。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=tQI35CSNB08&t=0s) 数学的核心追求是构建深刻的“理解”而非批量炮制论文；当这种认知仅寄宿于神经网络权重中且无法被人脑体会时，对人类而言是极度空虚的。 - [02:24](https://www.youtube.com/watch?v=tQI35CSNB08&t=144s) AI 攻克厄多斯单位距离问题的惊艳之处，在于其颠覆了猜想成立的学界共识，并成功跨领域迁移古典技术，展现出了实质性的数学创造力与后续衍生价值。 - [04:28](https://www.youtube.com/watch?v=tQI35CSNB08&t=268s) 数学不是纯粹的形式逻辑搬运，在漫长而持续的困惑中摸索、提炼直觉以及弄清楚“该问什么问题”，是当前擅长解题的 AI 远未能触及的最高壁垒。 - [06:19](https://www.youtube.com/watch?v=tQI35CSNB08&t=379s) 大模型的思维链推演风格“极具人味”，与人类草稿纸思考极为相似，其真正优势在于不厌其烦的大规模代数计算和对冷门跨领域论文的拼装整合能力。 - [08:48](https://www.youtube.com/watch?v=tQI35CSNB08&t=528s) 当前前沿突破主要依托自然语言的推理扩展而非形式化证明语言 Lean，这表明底层的非正式推理能力未来具备极强的跨学科泛化潜力。 - [10:01](https://www.youtube.com/watch?v=tQI35CSNB08&t=601s) 模型在没有人类提示时极难自主完成理论构建，但带提示交互呈现出显著的飞轮效应——模型今天需要提示才能解决的问题，约六个月后便能独立完成。 - [20:14](https://www.youtube.com/watch?v=tQI35CSNB08&t=1214s) 纯数学研究如同在概念世界中探索物理规律，过早追求证明的形式优雅反而容易诱使研究者放弃笨拙但正确的突破口，探索应当“不择手段”。 - [23:19](https://www.youtube.com/watch?v=tQI35CSNB08&t=1399s) 构造反例与构建宏大理论存在本质区别：AI 在高维组合空间搜寻反例时威力巨大，但面对深深植根于严密理论体系且大概率为真的深奥猜想时依然束手无策。 - [28:20](https://www.youtube.com/watch?v=tQI35CSNB08&t=1700s) 人类因无法无休止暴力硬算而产生的生理局限和“偷懒”本能，反而倒逼了对概念本质的抽象与压缩，从而催生出极具穿透力的数学美感与理论框架。 - [35:18](https://www.youtube.com/watch?v=tQI35CSNB08&t=2118s) 科研激励扭曲诱发了“拉霸机式”论文批量生产，arXiv 上雷同证明泛滥暴露出底层大模型在推理路径上的严重模式坍塌。 - [40:47](https://www.youtube.com/watch?v=tQI35CSNB08&t=2447s) 即使未来 AI 在智力上全面超越人类，人类依然需要保留追求宽基底基础研究的数学家群体，以确保科学探索不会因功利捷径而失去多样化的怪异直觉。 - [45:48](https://www.youtube.com/watch?v=tQI35CSNB08&t=2748s) AI 正在教育端引发学生能力的“双峰分布”：积极善用者加速迈向认知前沿，消极外包思考者则在失去工具时面临核心认知能力的崩塌。 - [49:21](https://www.youtube.com/watch?v=tQI35CSNB08&t=2961s) 由 Claude 破纪录构造出的 30 阶椭圆曲线属于典型的高水准半自主搜索成果，其科学意义需视乎其是否带来全新理论结构来客观事后审视。 - [55:08](https://www.youtube.com/watch?v=tQI35CSNB08&t=3308s) 数学家审核宏大论文依赖边界推演与反常识“单元测试”进行大局观把关，而当前的自动化框架与大模型仅擅长句法推导，缺乏宏观逻辑抗压测试能力。 - [01:01:03](https://www.youtube.com/watch?v=tQI35CSNB08&t=3663s) 数学教育的底色在于训练清晰思考与洞察世界的能力，在 AI 浪潮下应坚持向下一代传递对探索思考的纯粹热爱，将其作为人机共生下的理解加速器。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=tQI35CSNB08&t=0s) 引言：数学的目标是理解，而非单纯产出论文 对数学研究的本质而言，论文从来不是终极目的，真正的核心在于构建某种深刻的理解。当这种“理解”仅仅寄宿在 AI 模型的神经网络权重中时，对于真正的数学家来说，这不仅难以令人释怀，甚至是“相当令人不满的（pretty unsatisfying）”。在探讨 Anthropic 与 OpenAI 等前沿模型的推理能力时，人们发现模型目前仍难以完全独立自主地完成深度突破，但在人类适度的提示与引导下，它们已经能够完成许多意料之外的探索。数学史上的大部分进展，都源于“让百花齐放”式的自由探索与个人好奇心驱动，知识的边界由此相对匀称地向外拓展；而随着全自主 AI 成果的出现，数学界正在迎来范式层面的巨大震动。 ### [01:00](https://www.youtube.com/watch?v=tQI35CSNB08&t=60s) 走近 Daniel Litt：一线数学家对 AI 的敏锐洞察 主持人 Lisha Li 兴奋地引介了多伦多大学数学系助理教授 Daniel Litt。多伦多恰好是 Lisha 的故乡，而更特别的是，Daniel 不仅是一位活跃在科研一线的真正数学家，还在社交平台与学术圈对 AI 在数学中的演进发表了大量极具见地、且随模型迭代迅速更新的敏锐观察。Lisha 打趣道，只要两周不关注 Daniel 的动态，就会发现他又对最新发布的模型或学术事件形成了全新的深刻思考。面对“有很多观点（I have a lot of opinions）”的评价，Daniel 欣然承认。 两人的讨论跳过了单纯罗列模型参数与基准跑分的技术套话，将视角切入更本质的问题：在 AI 展现出惊人进展的当下，一线数学家究竟应当如何审视并回应这种冲击？数学这门学科的特殊之处，并不在于机器是否能推导公式，而在于人类数学家日常工作中那些无法被简单度量的品味、直觉与探索路径。 ### [02:24](https://www.youtube.com/watch?v=tQI35CSNB08&t=144s) 最令人瞩目的全自主突破：厄多斯单位距离问题 当被问及至今为止最令人印象深刻的 AI 数学突破时，Daniel 明确指出了 2024 年 5 月中旬公布的一项成果：AI 针对“厄多斯单位距离问题”（Erdős unit distance problem）给出的全自主反例与解答。在 Daniel 看来，AI 参与数学研究目前呈现出不同层次——既有部分辅助、人机协作，也有 AI 贡献模棱两可的工作。此前许多所谓的突破更偏向“最后一公里”（last mile）的收尾工作，即一组人类数学家已经完成了绝大部分深邃厚重的基础理论构建，AI 仅负责迈出临门一脚；又或者仅仅是将已知技术以稍微聪明一点的方式重新组合。 ```mermaid flowchart TD accTitle: AI破解厄多斯单位距离问题的影响路径 accDescr: 展示AI如何利用古典工具攻克几何问题并反哺人类数学家的全流程 subgraph Discovery ["AI 突破阶段"] A["传统直觉：数学界普遍倾向猜想成立"] --> B["AI 发现意外反例"] C["引入 1960 年代跨领域古典技术"] --> B end subgraph Impact ["学术反哺阶段"] B --> D["平面点集配置领域取得关键进展"] D --> E["人类数学家汲取 AI 解题思路"] E --> F["攻克实数和积猜想等更多开放性难题"] end ``` 厄多斯单位距离问题的特殊之处在于，它展现出了某种真正意义上的“数学创造力”： - **颠覆学界直觉**：该领域的研究者此前普遍认为原猜想应为真，但 AI 却出人意料地构造出了一个反例。 - **跨领域技术迁移**：该解法并非依赖极其艰深或崭新的概念，而是巧妙借调了 1960 年代的古典思想，将其跨界应用到研究平面点集配置（point configurations in the plane）的几何问题中。 - **理论成果的衍生丰沛度**：这一突破后劲十足，人类数学家迅速吸收了模型引入的思路，将其推广并用来解决了一系列其他重要的开放问题，例如实数上的和积猜想（sum-product conjecture over the real numbers）。Daniel 强调，评估一项数学成果是否足够惊艳，关键在于事后检验其引入的新理念是否具有普适价值，能否真正提升人类对数学世界的认知。 ### [04:28](https://www.youtube.com/watch?v=tQI35CSNB08&t=268s) 符号推演与数学创造力：为什么数学不仅是形式逻辑 对数学工作最肤浅的刻板印象，莫过于认为数学家整天只是在按照形式逻辑规则“搬运推导符号”。也正因为这种误解，强化学习（RL）才被外界视作攻克数学的万能灵药——人们理所当然地认为，形式验证成本低、游戏规则清晰，只要系统具备超越人类的庞大算力与逻辑推演速度，就能成为卓越的数学家。 然而，任何真正从事过数学研究的人都知道，这种逻辑推演能力仅仅是底层工具。数学研究最核心的魅力与挑战，在于“在漫长而持续的困惑中摸索前行（remaining confused）”，在于提炼直觉、逼近真理并建立对结构的宏观理解。因此，当讨论 AI 解决厄多斯问题具有创造力时，必须将这种真正的洞见与冰冷、不知疲倦的符号推演 Feat 明确区分开来。 ### [06:19](https://www.youtube.com/watch?v=tQI35CSNB08&t=379s) “极具人味”的思维链：模型展现出的推理质感 针对外界常用“非人”（inhuman）来形容 AI 推理能力的论调，Daniel 提出了完全相反的观察：OpenAI 披露的思维链（Chain of Thought）其实“极具人味（very human）”。除去模型可能在未公开草稿中被清洗掉的琐碎内容，其整体思考轨迹与一名人类数学家在草稿纸上试图破题时的心智流动几乎一模一样。 - **并非不可理喻的“第 37 手”**：不同于围棋界 AlphaGo 击败李世石时那种令人匪夷所思的“Move 37”，AI 的数学推理没有来自外星维度的离奇跳跃，完全就是人类数学家会做出的推导风格。 - **能力的偏科与局限**：模型所展现的“超越人类”之处，仅仅在于它渊博的知识储备和不知疲倦的计算韧性；而在解决问题所需的其他数学活动上，模型的短板依然非常明显。 - **自然语言推理 vs. 形式化语言**：当前前沿模型的核心突破在于将自然语言的大规模推理成功扩展，而非依赖大规模 Lean 形式化证明语料的硬性喂养。 - **训练数据与思维过程的脱节**：绝大多数正式发表的学术论文和教科书都经过高度抛光与精简，隐去了全部曲折试错与直觉萌芽的脉络。AI 能在缺乏人类真实“思维草稿”数据的情况下摸索出如此拟人的推理轨迹，是一件极其引人深思的现象。 ### [08:48](https://www.youtube.com/watch?v=tQI35CSNB08&t=528s) 自然语言推理与大模型横向对比：OpenAI 与 Anthropic 在探讨前沿 AI 模型时，Lisha Li 询问了 Anthropic 与 OpenAI 在数学推理上的差异，以及为何自然语言在这一领域的扩展效果如此出色。Daniel Litt 指出，当前前沿模型主要是基于“自然语言”进行推理，而非单纯依赖 Lean 这种形式化定理证明工具。许多人常说“数学是一个可验证的领域，所以才取得了如此巨大的进展”，但 Daniel 认为，正因为当前的技术进展主要源于形式化之外的非正式推理能力的扩展，这些底层技术极有可能顺利泛化到数学之外的其他领域。 针对模型之间的实际表现差异，Daniel 坦言两者在整体能力上非常接近。虽然他个人使用 ChatGPT 更多，但行业经常出现一种现象：OpenAI 刚发布了某个数学问题的解法，Anthropic 很快就会表示他们也能解出同样的问题。两者能解决的问题集高度重合，且目前覆盖的依然只是人类数学家工作版图中相对有限的一小部分。 ### [10:01](https://www.youtube.com/watch?v=tQI35CSNB08&t=601s) 模型的“算力优势”与数学直觉的匮乏：为何理论构建依然困难 从 AI 给出的解答风格中，可以清晰看出大模型的长短板。Daniel 分析道，模型目前最擅长的是那些完全契合其机械能力的任务——比如“硬啃繁琐漫长的大规模计算”，或是把散落在许多领域甚至冷门论文中的现成技术拼装融合在一起，这些文献是单一人类数学家可能根本没读过的。然而，模型在“数学直觉”和“宏观全局视角”上显得极为匮乏。 对真正的数学研究而言，数学家往往先有一种非常非形式化、近乎哲学的直觉感知（例如直觉上认为 A 结构和 B 结构存在深刻的类似），随后的核心工作则是思考如何将这种直觉精确化，并试图通过寻找未被理解的数学现象或解决具体问题来验证这种理解。然而，目前哪怕是最顶尖的 AI 模型，其产出也几乎看不到这种深层推理的影子，它们本质上依然是在极其高效地“应用所有已知技术”。尽管历史上也有不少数学家仅凭熟练运用既有技术就取得了显赫成果，但这仅仅是数学研究中很窄的一个光谱。 - **理论构建尝试**：Daniel 曾尝试引导前沿模型（如 ChatGPT 与 Claude 相关版本）进行某种程度的“理论构建”（Theory Building）。在没有人工提示（Scaffolding）的情况下，模型几乎无法自主完成这项工作。 - **提示与潜力的演进**：如果给予模型大约 100 比特的关键提示，它们能做出一些耐人寻味的推演。Daniel 总结道，在带提示交互时虽然很难界定究竟多少灵感来自人类、多少来自模型，但以往经验表明，"如果模型今天能在给足提示的情况下完成某件事，大约六个月后它就能不靠提示独立做到了"。 ```mermaid flowchart TD accTitle: Mathematical Thinking vs Current Model Strengths accDescr: 对比人类数学家的哲学直觉驱动工作流与当前大模型基于已知技术的机械解题路径 subgraph HumanApproach["人类数学家的研究范式"] A["非形式化直觉与宏观哲学"] --> B["提出好问题与寻找类比"] B --> C["从极小算例中精确化"] C --> D["构建抽象理论并产生新理解"] end subgraph AIApproach["当前 Frontier 模型的解题范式"] E["跨领域论文与已知技术检索"] --> F["强大算力进行长链代数推导"] F --> G["特定已知技巧的高强度应用"] G --> H["输出具体问题的验证解答"] end HumanApproach -.->|"当前模型极为欠缺宏观视野"| AIApproach ``` ### [12:12](https://www.youtube.com/watch?v=tQI35CSNB08&t=732s) 使用体验与模型交互：心智理论的差距与前沿的不平整性 谈及为何个人更偏向使用 ChatGPT 而不是 Claude，Daniel 解释这主要是使用惯性使然。在早期阶段，ChatGPT 在数学领域的进步显著更快，那时的 Claude 对于前沿数学研究基本派不上用场；直到后期 Claude 推出更新的 Opus 版本后，两者才基本旗鼓相当。 Lisha Li 分享了她的观察：在模型前沿能力表现参差不齐（Jagged Frontier）的现状下，不同模型在数学解释上展现出了不同的风格。在她看来，特定模型在解释结果时似乎具有更精准的“心智理论”（Theory of Mind），能更准确地预判读者掌握了哪些背景、不知道哪些概念；而另一些模型有时会喋喋不休地解释显而易见的概念，紧接着又在极难的关键步骤上一跃而过。不过，Daniel 对此幽默地表示，在他这种专业深度的视角下，其实当前所有模型在“理解人类理解状态”的深层理论构建上都相当差劲。 ### [14:00](https://www.youtube.com/watch?v=tQI35CSNB08&t=840s) 数学家的思维范式：解题者、理论构建与非形式化类比 为了厘清 AI 在数学中的真实边界，对话深入剖析了纯数学家的工作本质。Daniel 指出，数学家内部存在着截然不同的分类光谱，其中最典型的一种划分是“解题者”（Problem Solver）与“理论构建者”（Theory Builder）： - **解题者的本质**：Daniel 将自己归类为解题者。对他而言，一个公开难题（Open Problem）的根本意义在于**它客观丈量了人类认知上的失败**，就像是一个清晰的基准（Benchmark）。 - **格罗滕迪克-卡茨 p 曲率猜想（Grothendieck-Katz p-curvature conjecture）**：Daniel 举例称，这是他非常偏爱的一个难题，它衡量了人类对微分方程本质理解的不足；如果无法攻克它，就证明人类对该领域最基础的对象依然缺乏认知。 - **微观切入与理论提炼**：在实践中攻克难题的方法，通常是找到一个“人类无法理解的极小情形”反复推敲调试。一旦取得突破，数学家便会审视自己为赢下这一步所创造的方法，并将其提炼为系统性的理论。 除了以问题为导向，另一种极其重要的数学驱动力是“建立深层类比”。数学家可能凭直觉感到两套风马牛不相及的体系间存在某种镜像关系，进而建立对应表格（如将性质 $A$ 对应至 $A'$，性质 $B$ 对应至 $B'$）。 在 Daniel 的代数几何研究中，许多灵感便源自**代数簇的上同调**与**基本群表示**之间的深远类比。这一极富成果的哲学思想在过去三四十年间，经由卡洛斯·辛普森（Carlos Simpson）和望月拓郎（Takuro Mochizuki）等学者推进，孕育出了大量极其璀璨的现代数学分支。这类工作并非由某个既定的猜想驱动，其初始源头往往只是一种完全非形式化的直觉，"根本不是机械的符号推演（Symbol pushing）"。 ### [16:49](https://www.youtube.com/watch?v=tQI35CSNB08&t=1009s) 提出“正确的问题”：数学探索中最艰难的一步 不仅如此，数学研究中最具创造力也是最困难的环节，往往在于**弄清楚该问什么问题**。 数学家常常面对某个数学对象，本能地意识到自己对它一无所知，但精确定义出“我们究竟不知道什么”本身就是一项极其艰巨的智力挑战。Daniel 强调，互联网上那些公开列出的猜想清单，并不能完全概括人类数学认知的边界。这种探寻未知、构筑问题语境的能力，恰恰是目前只擅长解答既有题目的 AI 所完全无法触及的核心地带。 ### [17:48](https://www.youtube.com/watch?v=tQI35CSNB08&t=1068s) 从数据实验到数学猜想：BSD 猜想与 AI 的日常辅助角色 在纯数学研究中，提出一个深刻的猜想往往比解决它更困难。Daniel Litt 举了千禧年七大数学难题之一的“贝赫和斯维讷通-戴尔猜想”（Birch and Swinnerton-Dyer conjecture，简称 BSD 猜想）作为典型例子。该猜想揭示了椭圆曲线的 L 函数与对应方程解群的秩（rank）之间极其优美的内在联系。有趣的是，这一猜想在本质上是数学史上最早的“大数据驱动发现”：早在 20 世纪 60 年代，Bryan Birch 和 Peter Swinnerton-Dyer 便利用计算机计算了大量关于椭圆曲线的统计数据，这是最早的计算机辅助数学研究之一。当他们把统计数据绘制成图表时，意外发现某条直线的斜率竟与他们已知的另一个代数不变量密切吻合，这成了该猜想的萌芽。很多时候，数学家的工作方式与经验科学无异：通过海量实验去寻找模式，并为这些反常或规律寻求合理解释。 谈及 AI 在自身日常科研中的角色，Daniel 指出：现象越模糊、问题越缺乏精确表述，AI 的用处就越小。对于那些他已经思考了三四甚至五年的核心课题，现有模型依然无法替代深层的智力劳动，更多时候只是充当了高阶版 Google 的角色——协助了解相关未知领域、替代部分翻阅论文的时间，以对话方式加快文献检索。然而，AI 彻底改变了他的计算与编码瓶颈。Daniel 自嘲道，自己以前非常不擅长编程，遇到需要大量编程验证的问题往往会拖延半年，直到有不知疲倦的博士生来推进；而现在，AI 就像一个随时待命的编程高手，让他能够开启大批高度依赖代码的新项目，在几秒内并行跑完上千个具体案例或派遣多个子智能体进行搜索。 - **BSD 猜想的历史渊源**：20 世纪 60 年代数学家通过计算机绘制椭圆曲线的统计图表，首次在大数据趋势中发现了 L 函数与代数群性质的对应规律。 - **数学的经验科学属性**：数学并不总是由纯逻辑单向推演，往往需要先在大量具体算例中进行“科学实验”，观察数据后再反推底层机制。 - **AI 赋能的非对称性**：在已有深层长线课题上，AI 主要是“信息检索助手”；但在高并发案例筛选与脚本编写上，模型展现出了宛如“不知疲倦的博士生”般的高效。 --- ### [20:14](https://www.youtube.com/watch?v=tQI35CSNB08&t=1214s) 数学是科学而非诗歌：追求概念本质与反对“唯美主义” 当 Lisha Li 提出纯数学常被视作由社会学群体共识或审美直觉所驱动（正如 William Thurston 曾指出的那样），并询问 Daniel 个人是由什么驱动时，Daniel 坦言自己尽量避免被单纯的“数学之美”或纯粹的审美标准牵着鼻子走。他认为过度追求形式优美反而会限制数学家的视野。例如许多年轻数学家经常陷入一种失败模式：他们隐约找到了证明思路，却觉得那个证明极其笨拙、丑陋（ugly），充斥着冗长乏味的计算，于是果断放弃。而在 Daniel 看来，探索真理应当“不择手段”（"win by any means necessary"）——如果你最终证明了它，谁能断言它不是一条通往新认知的必经之路？ Daniel 将自己的研究比作“在概念世界里做物理学”（"doing kind of physics except with concepts"），而非纯粹的艺术创作。他更在意的不是证明是否看起来优雅，而是这项研究是否足够根本（fundamental），能否带来全新的思想工具去大幅拓展人们对数学对象的理解。尽管许多数学家喜欢将自己比作诗人，但他更倾向于把自己定位为探索客观规律的科学家。数学的真正飞跃往往来自于不同学者顺应个人好奇心在各自领域开拓边疆，当某人引入一个全新视角或工具时，便会在整个数学界引发连锁反应，迅速照亮大片原本难以触及的未知领域。 ```mermaid flowchart TD accTitle: Mathematical Investigation Orientations accDescr: Comparison between aesthetic-driven and physics-of-concepts orientations in pure math research. subgraph Aesthetic["审美/诗歌导向（Aesthetic Mode）"] A1["过早追求形式优雅与精简"] --> A2["排斥冗长、繁琐的强力计算"] A2 --> A3["可能错过看似丑陋但正确的突破口"] end subgraph Scientific["科学/概念物理导向（Physics of Concepts）"] B1["关注问题是否根本（Fundamental）"] --> B2["不择手段证明（Win by any means necessary）"] B2 --> B3["引入新视角与工具，引发连锁认知跃迁"] end ``` --- ### [23:19](https://www.youtube.com/watch?v=tQI35CSNB08&t=1399s) 构造反例 vs 发展新理论：为什么前沿 AI 在真正深奥的猜想前依然受限 针对 Lisha 提出的“深层思考中模型为何无法发力”的追问，Daniel 剖析了为什么某些问题适合 AI，而长期悬而未决的深水区课题却令 AI 束手无策。一个关键的分水岭在于：**猜想究竟是真还是假，解决它需要的是具体的反例构造，还是宏大的理论大厦构建？** 例如埃尔德什单位距离问题（Erdős unit distance problem），整个数学界此前普遍直觉认为它是成立的，最终却被推翻——这意味着只要能够创造性地设计出一个具体的几何构造（反例），就能彻底打破猜想。AI 在庞大的组合空间中搜寻反例、迁移不同领域的构造技巧时展现了极强的创造力与计算力。 相反，在 Daniel 已经研究了五年甚至十年的深层问题中，绝大多数猜想不仅大概率为真，而且深深植根于一个庞大而严密的理论框架中，拥有广泛的间接证据支持。要攻克这类猜想，根本没有捷径可走，既不可能通过搜寻反例来推翻它，也无法通过将现有成熟技术推向极致来解决，而是必须“发展出前所未有的全新理论与核心工具”（develop a new technique）。当前的 AI 模型虽然在跨领域调用已知工具、拼装机械结构上表现出惊人威力，但迄今为止，它们依然无法自主建立全新范式或从零构筑出深层的新理论。 - **反例型突破与结构型突破的差异**：若猜想为假，一次精巧的反例构造（Construction）即可一击致命，AI 的高维搜索在此极具威慑力；若猜想为真，通常需要证明完整的理论闭环。 - **已知技术（Known Techniques）的局限**：深层未决难题往往已经经过几代数学家的检验，证实仅凭已有工具的强力组合无法逾越屏障。 - **AI 的能力边界**：当前模型擅长在既有技术版图内进行跨领域的“技术迁移”（Porting techniques），但在从零开辟核心数学思想（Deep theory building）方面，目前尚未展现出破局迹象。 ### [26:25](https://www.youtube.com/watch?v=tQI35CSNB08&t=1585s) 理论构建与强化学习：AI 能否掌握数学直觉？ 谈到人工智能在数学领域更深层的挑战时，丽莎·李（Lisha Li）提出疑问：既然 AI 的能力正以惊人的速度提升，为什么在构建全新理论与数学工具（theory and technology building）方面它依然显得步履维艰？丹尼尔·利特（Daniel Litt）对此展现了相当开放和乐观的态度，他并不怀疑模型能力的持续增长，反而认为理论构建并非不可逾越的鸿沟，很可能完全是可行的，只是目前尚未探索出正确的强化学习（RL）训练环境。 丹尼尔分析指出，相较于纯粹的解题，发展一套数学理论或建立对一个模糊未知对象的理解，本身是一项更加难以量化的技能。在现有的强化学习框架下，设定清晰的中间奖励极其困难。例如，你可以指示模型“去深入理解黎曼 zeta 函数”，但你无法衡量中途的理解程度，总不能只有当它彻底证明了黎曼猜想（Riemann Hypothesis）才给予一次性奖励。不过，数学界本身存在着不同难度梯度的海量猜想，这些猜想自然构成了递进式的训练课程（curriculum）。丹尼尔打趣道，人类博士生在成长过程中也是类似的，导师会凭借某种学术品味（taste）赞许一句“这个想法不错”，这种来自导师的微弱正反馈引导了人类直觉的形成，未来 AI 训练或许也能引入类似的机制。 ```mermaid flowchart TD accTitle: AI数学能力与理论构建机制 accDescr: 对比传统AI目标导向解题与发展数学理论所需的渐进式训练模式 subgraph ProblemSolving ["常规解题模式（目标明确）"] A["明确的数学猜想"] --> B["大规模计算 / 策略搜索"] B --> C{"是否得出证明？"} C -- "是" --> D["二元奖励 (Reward)"] C -- "否" --> E["无反馈"] end subgraph TheoryBuilding ["理论构建模式（探索直觉）"] F["模糊的研究对象"] --> G["由浅入深的猜想体系 (Curriculum)"] G --> H["导师审美 / 概念精炼 (Taste)"] H --> I["建立结构化理解与全新理论"] end ``` ### [28:20](https://www.youtube.com/watch?v=tQI35CSNB08&t=1700s) 人类的认知限制与“偷懒”之美：为什么暴力计算阻碍了洞察？ 丽莎由此引发了一个更具哲学意味的话题：人类的大脑是否因为无法处理无限细节，才被迫对现实进行信息压缩，进而催生了物理学与数学中那些优雅的理论框架？丹尼尔坦言自己虽然对“以压缩率作为评判兴趣的标准”持保留意见，但从人类学角度来看，他完全认同这种观点。甚至可以说，“人类无法无休止暴力硬算（inability to just grind）的生理局限，反而是人类做出重大科学发现的关键动力”。 为了说明这一点，丹尼尔分享了自己近期发表论文中的亲身经历。当时他与合作者已经证明了论文的核心定理，但有几个引理的形式不够理想。丹尼尔最初尝试借助 Gemini Deep Think 等前沿模型来攻克这些引理，但当时的模型全都束手无策。面对死胡同，丹尼尔只能自己动手演算了大量特例，在这个过程中，他突然洞察到了事物背后的结构原因，从而提炼出了一个更优、更本质的引理陈述。一旦有了这个更漂亮的表述，AI 模型瞬间就完成了证明。 丹尼尔笑着自嘲自己的“虚伪”与审美偏执：如果把最初那个臃肿的引理丢给更新的前沿模型，模型完全能吐出一篇长达十页、充满残酷暴力计算的证明。“那是你能见识到的最糟糕的证明”，虽然逻辑毫无瑕疵，却没有任何数学洞察可言。丹尼尔直言，他当时其实已经意识到暴力硬算走得通，但他实在“无法忍受自己去写那种丑陋的硬算证明”，正是这种对繁琐计算的厌恶与抗拒，倒逼着他寻找更优的概念解释，从而收获了真正的数学美感。 - **非形式化数学的价值**：人类之所以不直接使用 ZFC 公理系统的形式化长符号串推导，正是因为需要提炼出人类思维能理解的非严密概念结构。 - **计算能力的双刃剑**：现在的模型已经能非常可靠地完成极长、极枯燥的技术性推导，但这往往会导致研究者跳过对概念本质的提炼。 - **发现的动力学**：很多时候，漂亮的数学洞察正是因为研究者算力有限、“懒得暴力计算”才被逼出来的。 ### [33:33](https://www.youtube.com/watch?v=tQI35CSNB08&t=2013s) 数学社区如何适应 AI：论文工厂与真正理解的冲突 面对 AI 在长篇技术推导上日益强大的现实，丽莎探讨了数学界该如何自我调适：既能让没有海量时间的业余爱好者或跨界学者参与进来，又该如何防止整个领域陷入不健康的发展轨迹？ 丹尼尔强调，前沿模型能产出高质量的数学成果确实令人心潮澎湃，他自己也热切期盼着 AI 能够解答那些长久以来让他夜不能寐的数学难题。绝大多数人最初投身数学领域，正是因为享受“学习数学”这一过程本身的纯粹乐趣——任何数学系的学生在开始独立研究之前，往往都要先花上 15 到 20 年的时间去消化前人的智慧结晶。 然而，丹尼尔对未来的发展提出了清醒的警示：数学的终极目标绝不是为了批量炮制学术论文，而是为了创造“理解”（understanding）。如果这种对数学本质的深刻洞察最终仅仅锁死在神经网络的模型权重内部，而无法被人类的心智所体会，那将是一件极其令人沮丧且空虚的事情。对他而言，做数学研究的核心动力，归根结底是为了满足人类自身对宇宙秩序的好奇心，而非仅仅追求一个打勾确认的命题结果。 ### [35:18](https://www.youtube.com/watch?v=tQI35CSNB08&t=2118s) 扭曲的学术激励与“拉霸机式”论文泛滥 前沿数学研究绝非空中楼阁，它依赖于一个庞大而深厚的人力资本支撑体系。能够站在最前沿探索根本性数学问题的学者固然只有少数顶尖人群，但这群人的存在必须仰赖数以百万计、甚至数十亿在各级教育中尝试以数学方式思考的人所构成的培养管道（pipeline）。如果这条管道断裂，顶尖研究也将不复存在。然而，Daniel Litt 尖锐地指出，当前学术界的激励体制根本不支持学者沉下心来投入大量时间与精力去积累这种真正的人力资本。对于面临就业市场巨大生存压力的博士后而言，当务之急是在短时间内炮制出尽可能多的论文，比如解决某些旧猜想，以此来换取教职。 这就导致了一种把大语言模型当成“老虎机”（slot machine）来玩的荒诞现象：研究人员甚至不需要提前挑选定理，只需坐在屏幕前反复拉下拉杆，直到模型吐出一份大概率正确的证明。Daniel Litt 分享了自己亲手做过的一个实验：他指示模型（Codex）上网抓取代数几何领域最近提出的 5 个猜想并尝试证明。在经过一些来回提示词交互后，他仅花了一个小时就得到了 3 篇虽然质量很差、但证明完全成立的完整论文。这些论文至今还静静地躺在他的硬盘里，他也懒得去联系相关学者发表，因为这纯粹是对时间的浪费。然而，现实中越来越多人正在批量化复现这一操作，导致预印本平台 arXiv 上的提交量激增，其中充斥着几乎毫无学术价值的低质灌水文章。更令人警惕的是，学术界已经出现多起在短短两三天内、来自不同团队的三四篇论文使用完全一模一样的证明路径去解决同一个定理的情况。这暴露了底层模型在推理路径上的“模式坍塌”（mode collapse），只要大家使用的提示词或方向相似，ChatGPT 就倾向于收敛在同一条极其狭隘的证明线索上。 ```mermaid flowchart TD accTitle: AI 生成论文对学术生态的冲击循环 accDescr: 展示学术考核机制如何催生拉霸机式论文生成，并最终导致模式坍塌与低质论文泛滥。 subgraph Incentive ["现实激励与研究异化"] A["学术考核与教职生存压力"] --> B["追求论文数量而非深度认知"] B --> C["'拉霸机'式交互：让模型不断生成证明"] end subgraph Outcome ["产出异化与认知局限"] C --> D["单小时批量生成可用但低质的正确论文"] D --> E["arXiv 出现大量雷同证明与灌水文章"] E --> F["模型推理模式坍塌（Mode Collapse）"] end F -.->|"挤压基础研究与真实理解"| A ``` - **代数几何速成实验**：Daniel Litt 亲自用模型测试，一小时内自动捕捉并证明了 3 个近期代数几何猜想，迅速成文，但研究体验极度缺乏实际数学意义。 - **模式坍塌的铁证**：多篇论文在相隔数日之内上线，且内部采用完全同一套推理证明框架，证明多数研究者只是在机械地“摇号”调用同款模型。 --- ### [38:05](https://www.youtube.com/watch?v=tQI35CSNB08&t=2285s) 多样化直觉的危机：百花齐放还是千人一面？ 数学史上的重大突破往往源于“百花齐放”（letting a thousand different flowers bloom）的自由探索。学者们顺应各自纯粹的好奇心，在高维且广阔的数学知识空间中向四面八方均匀推进前沿边界，随后往往在不经意间为古老难题找到答案，或诞生意想不到的实际应用。Daniel Litt 提出疑问：如果我们把数学探索的方向拱手让给模型，我们得到的究竟是“一百万个追求不同方向的数学家”，还是仅仅“将同一个数学家复制了一万次”？ Lisha Li 对此深感共鸣，并指出这是前沿 AI 实验室必须直视的潜在危机。尽管模型涌现出的推理能力令人惊艳且带来了巨大的公关声量，但模型本质上是从同一套既有的数学文献数据中汲取营养的，因此极易在相似问题上收敛。人类数学家之所以不可替代，恰恰在于他们拥有各种“千奇百怪、非标准甚至离经叛道的直觉”（weird intuitions），这种多元的思维底色才造就了能够相互交织、激荡出新成果的前沿领域。仅靠测试时计算（test-time compute）和后训练扩展（post-training scaling）等算力堆叠手段，究竟能否在模型中真正催生出全新的、具有多样性的数学直觉，目前在学术界仍是一个巨大的问号。如果当前的激励体系不能促使足够多的真人研究者带着独特直觉与模型进行深层互动，数学研究最终很可能会陷入近亲繁殖式的平庸，难以持续产出真正卓越的飞跃。 --- ### [40:47](https://www.youtube.com/watch?v=tQI35CSNB08&t=2447s) 即使 AI 超越人类，为什么我们依然需要人类数学家？ 面对“AI 终将超越人类”的假设，Daniel Litt 提出了一个更为激进的论点：退一步讲，即使大语言模型在未来稳健地达到了全方位的超人类水平，人类在认知层面上完全无法为模型增添任何增量，我们依然需要保留人类数学家。这本质上不是一个纯粹的计算效率问题，而是一个关于人类“如何设计理想社会”的根本抉择。 即便将数学研究全权托付给 AI 可能是产出应用成果或优化理解的最快路径，“最优”也绝不等于我们理应照单全收。如果任由模型主导科研，甚至为了追求功利目标将人类需求工具化（例如仅仅要求 AI “让生活变好”），模型很可能会选择一条狭隘、笔直且单调的捷径，而放弃对宽广深邃的基础理论展开漫游。Daniel Litt 坚信，无功利的宽基底基础研究（broad-based fundamental research）是人类智力活动中最有价值的瑰宝之一。要确保这种探索不受侵蚀，唯一的办法就是维持一个拥有广泛学术兴趣的人类共同体，由人类来驱动模型、引导探索方向，并构建由人类意志所塑造的未来。 Lisha Li 极力赞同这一展望，并表达了她对未来的深层隐忧：随着 AI 飞速演进，如果缺乏能够促使人类保持深度思考的人机工学交互界面（ergonomic interfaces），人类很容易因为惰性而彻底放弃思考的特权。即便模型在架构全局的顶层结构性思考上依然笨拙，人们依然极易将思维完全外包。她强调，从教育学和认知训练的角度来看，数学绝不仅仅是解决具体定理的工具，它更是一种极佳的思维框架训练，能让人在面对世间复杂问题时保持最严格的逻辑审慎。作为一名两岁孩子的母亲，她常常思考思维培育的本质：真正的智力成长从来不是为了应对指标去进行机械式的枯燥刷题（grinding），而是为了保持敏锐的思维能力和对未知探索的主体性。 ### [43:51](https://www.youtube.com/watch?v=tQI35CSNB08&t=2631s) 匈牙利数学教育与 AI 冲击下的第一线职业 Lisha Li 分享了自己过去与一些匈牙利数学家合作的经历，其中包括 László Lovász。她提到在布达佩斯，孩子们甚至在小学阶段就开始接触群论（Group Theory），这种扎实深入的数学训练让她大为赞叹。如今 AI 在解释概念方面已经相当在行且门槛极低，她认为教育界应当借此东风进一步推广深度的数学教育，把更多人带到科学探索的前沿，而不是单纯地死记硬背或机械刷题。 Daniel Litt 对此深有共鸣。他指出数学界很可能是最早直面高水准 AI 模型带来实质性冲击的职业之一。尽管在软件编程等任何需要在电脑前操作的工种里，AI 已经悄然接管了庞大的工作量，但由于数学解题能力是各家前沿实验室最乐于公开展示的“肌肉”，数学界因此成为了首个被推向聚光灯、被迫公开面对这场转折的行业。保持数学领域的人才培养与人力资本不仅关乎数学本身，更是所有专业领域的一面镜子——人类社会终究需要有专业素养、能与世界进行实质性深度互动的专家。由于数学与教育体系深度绑定，大学与中等教育目前也最先感受到了 AI 的冲击。 ```mermaid flowchart TD accTitle: AI对教育与专业能力的两极分化影响 accDescr: 展示了AI在学生和专业人士中引发的双峰分布分化过程 A["引入 AI 工具"] --> B{"学生的适应方式"} B -->|"积极善用"| C["利用 AI 答疑解惑<br/>深化理解与拔高思维"] B -->|"消极依赖"| D["外包作业与解题<br/>丧失独立思考能力"] C --> E["双峰分布高位端<br/>加速迈向认知前沿"] D --> F["双峰分布低位端<br/>考试与核心能力全面崩盘"] ``` --- ### [45:48](https://www.youtube.com/watch?v=tQI35CSNB08&t=2748s) 认知分化的“双峰分布”与保持深度思考的紧迫性 在探讨大学课堂的现状时，Daniel 提到许多教授观察到学生群体中正在出现一种明显的“双峰分布”（Bimodal Distribution）：一小部分学生迅速摸清了如何利用 AI 深化学习、辅助理解，从而学得更快更好；而另一部分学生则完全把作业外包给 AI 代劳，到了没有 AI 可用的正式考试中就一败涂地。 Lisha 与 Daniel 探讨了这种技术滥用对人类心智的潜在侵蚀： - **低成本劣质产出的泛滥**：新技术刚出现时，往往以更低的成本产出略差但勉强合格的内容，导致大量低质量输出（Low-quality outputs）迅速挤占原先高水准的工作。要想实现全维度的质量提升，必须依靠深思熟虑的人类介入和制度重新设计。 - **初级人员的技能鸿沟**：资深专家能借助 AI 实现杠杆效应，但缺乏辨别力的初学者极易陷入偷懒的人性弱点，将思考权拱手让给模型，进而无法建立底层的洞察力。 - **警惕“外包思考”的陷阱**：在 AI 飞速迭代的时代，若人类反而退化为“更差的思考者”，将是巨大的悲剧。面对“模型本身并不真正具备心智”的现实，人类必须用更具竞争性的危机感逼迫自己借助 AI 去加深理解，而非放弃认知训练。 --- ### [48:18](https://www.youtube.com/watch?v=tQI35CSNB08&t=2898s) 论文灌水现象与公众重燃的数学热情 谈及学术界的“垃圾产出”（Slop papers），Daniel 坦言这并非全是 AI 的过错。早在这一波浪潮之前，学术圈现有的评价与激励机制就已经催生了大量专业数学家制造的平庸论文。虽然 AI 降低了门槛，导致非专业人士也开始在网络上发布大量质量参差不齐的研究，使得寻找某个问题是否被真正解决的查证成本变高，但他整体上依然保持乐观：看到全世界有这么多人重新对数学燃起热情，哪怕带来了一些信息噪音，也绝对是一件利大于弊的好事。 --- ### [49:21](https://www.youtube.com/watch?v=tQI35CSNB08&t=2961s) 破纪录的 30 阶椭圆曲线与对 AI 成果的事后审视 Lisha 提到了录制前一天刚刚引发轰动的一项新进展：由 Levent Alpöge 与合作者（Ava Howell 等）利用 Anthropic 的 Claude 模型（Claude Opus）构造出的 30 阶椭圆曲线（Elliptic curve of rank 30），打破了此前由 Noam Elkies 与 Zev Klagsbrun 保持的 29 阶纪录。 Daniel Litt 对此冷静地进行了学术维度的剖析： - **历史定位与学术分量**：构造特定高阶椭圆曲线是一个极其经典且充满智力趣味的领域，像 Noam Elkies 这样顶尖的数学家都曾深耕其中。但这属于典型的“纪录型成果”（Record-setting result），在数学共同体中通常会被收录在专门的记录网站上，并不会被视为能登上顶级期刊《数学年刊》（Annals of Mathematics）级别的理论突破。 - **自主度与黑盒疑问**：目前作者团队并未公开具体的构造方法或提示词执行细节。从以往的案例来看，这类成果大多属于在人类高度引导下的“半自主”（Semi-autonomous）产物，而非 AI 完全自主的突破。在不清楚模型是暴力搜索还是发现了某种新结构之前，很难断定其真正的科学重要性。 - **事后评估原则**：无论是 AI 还是人类数学家的成果，往往只能在事后才能做出客观评价——某些当时看似必须依赖全新深层思想的问题，事后看可能只需要纯粹的算力搜索；反之亦然。 随后，Lisha 补充了她与 OpenAI 研究人员（如 Mark Chen 与 Noam Brown 等，访谈中提及 Mark 与 Hunter/Metab）交流时的体会：OpenAI 团队目前感到最为惊艳的一点，是模型生成的一些数学证明往往出奇地精炼简短，而不是动辄长达两百页的繁琐推演。这也从侧面呼应了模型是在紧凑的搜索空间内寻找优雅路径，还是仅仅在进行机械式蛮力研磨的讨论。 ### [52:28](https://www.youtube.com/watch?v=tQI35CSNB08&t=3148s) 短证明与长证明：AI 验证能力与幻觉的边界 面对模型在数学任务上的输出特征，Lisha Li 提出一个疑问：目前向模型抛出问题时，它们给出的证明往往短小精悍、易于人类理解，这究竟是因为它们本身擅长生成巧妙的短证明，还是因为长篇大论往往容易陷入失控与无意义的苦算。Daniel Litt 对此表示，大家都喜欢短小巧妙的证明，但模型目前不怎么输出冗长复杂证明的核心原因，在于“验证正确性的能力尚未达到要求”。即便是生成较短的证明，如果追问一句“这段证明是对的吗？”，模型自己往往也会坦白承认“不对”。虽然比起半年前，前沿模型的可靠性已有显著提升，但它们依旧会产出连自己都能识别出的荒谬错误。 这种验证瓶颈直接导致了模型在处理长篇证明时的不可控：一旦篇幅拉长，模型甚至无法意识到自己已经出错。Daniel 推测，OpenAI 和 Anthropic 内部实际上利用模型解决的数学问题远比对外公开的要多，但其中大量成果之所以被压下不发，正是因为研究团队无法百分之百断定其真伪。以 OpenAI 最近公布的 10 个数学问题解法为例，它们全部在形式化证明语言 Lean 中完成了形式化验证，这构成了正确性的坚实证据；但由于数学界许多先决理论概念尚未完全形式化并录入 Mathlib，再加上篇幅过长带来的验证壁垒，大量潜在解法根本无法得到机器或人工的核验。 与此同时，arXiv 等预印本平台上已经开始泛滥充斥着 AI 生成的超长证明。Daniel 举例提到，最近有人在 arXiv 上提交了一篇声称解决了“正特征代数簇的奇点消解”（resolution of singularities in positive characteristic）的论文，通篇由 AI 生成，篇幅竟长达 800 页。Daniel 直言自己甚至没去细看：“我敢肯定这绝不可能是对的。如果这个结论成立，那将是轰动整个数学界的里程碑。但根据目前模型的校准能力，这完全超出了它的能力极限。”不仅没有任何人类数学家有精力去通读这 800 页，当前的 AI 模型自身也根本缺乏审核此类长文本证明的能力。 - **模型自查的局限**：模型有时会在刚写完一段简短证明后，在用户的追问下立即推翻自己，承认刚才的推理存在漏洞。 - **Lean 形式化的双刃剑**：形式化提供了绝对的正确性证明，但数学库对复杂前沿定理的前提概念覆盖度仍然有限，严重制约了自动化验证的应用范围。 - **arXiv 上的虚假繁荣**：AI 生成的 800 页奇点消解论文成为典型案例，既没人能读，也没机器能验，完全脱离了当前前沿模型的可信能力边界。 ```mermaid flowchart TD accTitle: AI 数学证明生成的验证困境 accDescr: 对比短证明与长篇证明在生成、验证与可靠性上的巨大差异 subgraph ShortProof["短证明路径 (可信度较高)"] A["生成短小/巧妙证明"] --> B{"模型/人工快速检验"} B -->|"发现漏洞"| C["模型自我纠错/推翻"] B -->|"通过检验"| D["可读且可形式化 (如 Lean 验证)"] end subgraph LongProof["长篇证明路径 (当前能力瓶颈)"] E["生成数百页长篇证明 (如 800 页预印本)"] --> F["超出人类逐行审核极限"] E --> G["超出模型全局逻辑核查能力"] F & G --> H["无法确认真伪，充斥微妙幻觉"] end ``` --- ### [55:08](https://www.youtube.com/watch?v=tQI35CSNB08&t=3308s) 评测框架、长上下文与宏观结构检查的缺失 谈及长程任务的构建，Lisha 借用代码领域的经验作为类比：Cursor 在测试长跨度任务框架（harness）时，曾尝试让模型在 Rust 中重写 SQLite。这一实验极其直观地展示了当前技术离真正的长程自主任务还有多远——要确保整个代码库功能正确，不能仅靠裸模型（raw models），必须配备复杂的规划器（planner）、多层级脚手架以及庞大的测试用例集。Daniel 对此深有共鸣，他指出在数学中，要诱导模型输出长篇证明同样依赖严苛的外挂框架；然而矛盾的是，当框架把全部目标放在“诱导产出完整证明”上时，往往会以牺牲严谨性为代价。 很多试图利用学术界声誉机制套利的人，正借助这种机制批量催生未经审阅的论文。Daniel 认为：“任何能够凭空诱导出 250 页论文的自动化框架，本质上都极其缺乏对输出质量的审慎把控。”更深层的挑战在于人类专家与 AI 在“审稿方式”上的本质差异。人类面对一篇数百页的宏大论文，从来不是机械地逐行挑错，而是先去宏观把握论证的全局架构，并进行模糊的“单元测试”——例如思考该论据若推向极致，是否会导出某种已知为假的荒谬命题，或者在特定的极端特例下是否仍然自洽。而当前的各类前沿模型恰恰完全缺乏这种大局观维度的抗压测试能力。 Daniel 分享了自己的一套测试标准：几年前曾有一篇论文声称解决了他研究领域内的一个重大猜想。论文一上线，他和同行专家便立即下载阅读，虽然一时极难定位具体算式错在哪里，但大家从宏观逻辑架构上一眼就能断定“这个论证绝对行不通”，因为其得出的结论推演开来实在强得过于反常。经过专家们与原作者来回反复推敲探讨，大家最终才揪出了那个隐藏得极深的具体微妙错误。然而，当 Daniel 把该论文输入给当下最顶尖的模型时，它们无一例外都无法察觉这种高维度的逻辑漏洞。模型虽然擅长句法级的推导，但在宏观架构与高层思维上仍显稚嫩。正因如此，尽管 Daniel 自己写过一些基于 Codex 的简易工作流脚本，但他本人对纯粹的“自动化数学”（autonomous mathematics）毫无兴趣，更倾向于把 AI 当作辅助自己理解深奥知识的思维工具，而非用来把自己彻底移出思考回路的替代品。 - **框架与可靠性的权衡**：为了激发长程创造力而放宽约束的脚手架，极易让模型输出失控，导致长篇论文通篇充斥看似合逻辑实则站不住脚的伪证。 - **高阶“单元测试”缺失**：数学家依赖特例检验、边界推演与反常识直觉来快速判定全局可行性，而大模型目前仅局限于局部步骤推导，缺乏这种大局观视野。 - **人机协同的态度**：数学研究的核心乐趣在于理解本质，完全自动化的解题剥夺了数学家的参与感，理想的工具定位应当是“理解加速器”而非“论文生成器”。 --- ### [59:34](https://www.youtube.com/watch?v=tQI35CSNB08&t=3574s) 培养数学家：给三岁女儿的数学启蒙 在对话的尾声，话题转向了更加温馨与生活化的视角。Lisha 提到 Daniel 有一个刚满三岁的女儿，并好奇作为顶尖数学家的 Daniel 如何在日常生活中对幼儿进行数学启蒙，以及在 AI 飞速演进的当下，如何向下一代传递对数学这门学科纯粹的热爱，而非将其变成枯燥的应试刷题。 Daniel 笑着分享了女儿 Sophia 的日常趣事：她现在刚满三周岁，当然还完全没接触过任何 AI。目前她的数学进度还停留在非常基础的阶段，主要依赖数手指做单数加法；她能极其可靠地从 1 数到 30，勉强可以数到 50，此外平时父女俩也会一起辨认各种几何形状。Daniel 为此感到非常自豪与欣慰。几天前清晨他去叫女儿起床，发现小家伙正整个人缩在毯子底下，Daniel 便好奇地问：“Sophia，你躲在里面干什么呢？”三岁的小女孩认真地回答道：“噢，我正在里面做数学题呢。” 这个充满童趣的瞬间让 Daniel 忍俊不禁，此前他还把这件趣事发在了社交媒体上。他感慨道，小孩子往往能敏感地捕捉到父母的热情所在，正是因为感受到了父亲对数学的热爱，她才会觉得“做数学”是一件好玩、酷炫的事情。谈及对女儿未来的长远展望，Daniel 坦言，等到二十年过去、女儿完全成年并走上自己的人生道路时，世界必定会发生天翻地覆的剧变，但他相信，对思考本身的热忱与探索的乐趣，依然具有超越技术周期的恒久价值。 ### [01:01:03](https://www.youtube.com/watch?v=tQI35CSNB08&t=3663s) AI 浪潮下数学教育的真正底色 面对人工智能的飞速演进，Daniel Litt 坚信教育的核心目的具有极强的抗颠覆能力。他指出，人类学习数学的初衷从来都是为了“清晰地思考并更好地理解这个世界”（"to think clearly and like better understand the world"），即便未来诞生了能力极度顶尖的 AI，这种对理解力的追求也绝不会失效。这就如同人们热爱阅读文学名著、钻研人文学科一样，其根本价值在于丰盈思维本身。Daniel 强调，数学界乃至更广泛的教育界所秉持的这些核心价值，正是我们在技术变革中必须竭力守护的东西，他也渴望将这些价值观传递给下一代。至于现实中的教育机构需要做出多大程度的重塑才能守住这一内核，目前依然是一个值得深思的开放性议题。 ### [01:01:56](https://www.youtube.com/watch?v=tQI35CSNB08&t=3716s) 从二十面体到群论：数学家的硬核育儿 在个人生活层面，Daniel 正在身体力行地向自己三岁的女儿证明“数学是一门超级酷的学问”。他半开玩笑地透露，女儿最早掌握的词汇之一竟然是“二十面体”（icosahedron）——因为在他女儿一岁时，爷爷奶奶送了她一个正二十面体的小玩具，这让她在牙牙学语阶段就早早认识了柏拉图立体（Platonic solids）。两人随即打趣道，下一步的教学计划顺理成章就该轮到“群论”（group theory）了；Daniel 更是风趣地表示，未来教女儿进一步学习加减法时，自己打算直接把它们放在“一般群”（general group）的抽象框架下来讲授。 Lisha Li 笑着感叹，至少这种育儿方式讲清楚了抽象概念的底层动机，而这恰恰是许多传统教育容易忽略的一环。她总结道，这或许也是未来数学研究生乃至整个教育界应当聚焦的方向：培养更年轻的一代学会借助 AI 去深化数学理解，而不是仅仅因依赖工具而丧失思考的深度。节目在轻松愉悦的气氛中收尾，双方共同期待在 AI 与数学交叉领域取得更多突破时再次深入交流。 - **数学教育的恒常价值**：即便具备强大推理能力的 AI 普及，人类学习数学以锻炼清晰思维、洞察世界的根本动力依然成立。 - **硬核启蒙趣事**：Daniel 的女儿在一岁时便接触到了正二十面体玩具，极早就学会了辨认柏拉图立体；Daniel 甚至戏称要在一般群的体系下教她加减法。 - **人机共生的教学方向**：指导年轻一代将 AI 作为深化认知与理解的阶梯，避免因过度外包思考而导致思维能力的退化。 ## Notable quotes > [00:00](https://www.youtube.com/watch?v=tQI35CSNB08&t=0s) pretty unsatisfying — Daniel Litt > [01:00](https://www.youtube.com/watch?v=tQI35CSNB08&t=60s) I have a lot of opinions — Daniel Litt > [06:19](https://www.youtube.com/watch?v=tQI35CSNB08&t=379s) very human — Daniel Litt > [10:01](https://www.youtube.com/watch?v=tQI35CSNB08&t=601s) 如果模型今天能在给足提示的情况下完成某件事，大约六个月后它就能不靠提示独立做到了 — Daniel Litt > [20:14](https://www.youtube.com/watch?v=tQI35CSNB08&t=1214s) win by any means necessary — Daniel Litt > [20:14](https://www.youtube.com/watch?v=tQI35CSNB08&t=1214s) doing kind of physics except with concepts — Daniel Litt > [28:20](https://www.youtube.com/watch?v=tQI35CSNB08&t=1700s) inability to just grind — Daniel Litt > [28:20](https://www.youtube.com/watch?v=tQI35CSNB08&t=1700s) 那是你能见识到的最糟糕的证明 — Daniel Litt > [38:05](https://www.youtube.com/watch?v=tQI35CSNB08&t=2285s) letting a thousand different flowers bloom — Daniel Litt > [01:01:03](https://www.youtube.com/watch?v=tQI35CSNB08&t=3663s) to think clearly and like better understand the world — Daniel Litt ## Mentioned Daniel Litt, Lisha Li, a16z, Anthropic, OpenAI, ChatGPT, Claude, Claude Opus, Gemini Deep Think, Codex, Lean, Mathlib, Erdős unit distance problem, Grothendieck-Katz p-curvature conjecture, Birch and Swinnerton-Dyer conjecture, Riemann Hypothesis, AlphaGo, Noam Brown, Mark Chen, Levent Alpöge, Ava Howell, Noam Elkies, Zev Klagsbrun, Carlos Simpson, Takuro Mochizuki, William Thurston, László Lovász, Cursor, SQLite, Rust, arXiv ## Video description a16z’s Lisha Li sits down with Daniel Litt, Assistant Professor of Mathematics at the University of Toronto, to unpack AI's rapid progress in mathematics, what today's frontier models can actually do, and what they're still missing about the way mathematicians think. Daniel explains why some recent AI-generated results are genuinely impressive, including an autonomous solution to the Erdős unit distance problem, but argues that solving problems is only one part of mathematics. Today's models can grind through calculations, combine known techniques, and search enormous spaces, but still struggle with intuition, theory building, identifying the right questions, and developing the kind of big-picture understanding that drives much of mathematical progress. Lisha and Daniel also explore how AI is already changing mathematical research, why an explosion of AI-generated papers could distort academic incentives, and what happens if researchers outsource the work of thinking rather than use AI to deepen it. Ultimately, they ask a question that extends far beyond mathematics: as AI gets better at intellectual work, how do we make sure humans keep getting better at thinking too? Timestamps: 00:00 - Intro 01:00 - Meet Daniel Litt: A Practicing Mathematician's Evolving Views on AI 02:24 - The Most Impressive Result: The Erdős Unit Distance Problem 06:12 - What AI Is Actually Doing for Working Mathematicians Today 12:12 - Intuition, Taste & Why Math Isn't Just About Proofs 20:26 - Deep Thinking vs Pattern Matching: What Models Are Missing 26:17 - Why the Unit Distance Result Was Actually Creative 33:33 - How Should the Math Community Adapt to AI? 43:26 - Where AI Will Impact Applied Math First 46:24 - Taking Advantage of AI Without Losing the Craft 49:21 - Comparing Anthropic vs OpenAI in Math 51:34 - Why Some Labs Have Gone More Secretive 59:40 - Raising a Mathematician: Teaching Math to a Toddler Resources: Follow Daniel Litt on X: https://x.com/littmath Follow Lisha Li on X: https://x.com/lishali88 Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures. [Source](https://www.youtube.com/watch?v=tQI35CSNB08)

[0:00] Daniel Litt: pretty unsatisfying [1:00] Daniel Litt: I have a lot of opinions [6:19] Daniel Litt: very human [10:01] Daniel Litt: 如果模型今天能在给足提示的情况下完成某件事，大约六个月后它就能不靠提示独立做到了 [20:14] Daniel Litt: win by any means necessary [20:14] Daniel Litt: doing kind of physics except with concepts [28:20] Daniel Litt: inability to just grind [28:20] Daniel Litt: 那是你能见识到的最糟糕的证明 [38:05] Daniel Litt: letting a thousand different flowers bloom [61:03] Daniel Litt: to think clearly and like better understand the world

[5:00] Daniel Litt: They don't seem inhuman at all. They seem absolutely like something a human mathematician could produce... It's like a human mathematician doing certain types of math. [11:08] Daniel Litt: If they can do it with like a hundred bits of hints... in six months, maybe they can do it without hints. [61:02] Daniel Litt: The reason to learn math has always been like to think clearly and like better understand the world, and like presumably that's something you want to do even if there are sort of extremely capable AIs.

[0:00] The goal of mathematics is not to [0:01] produce mathematics papers. It's to [0:03] produce some kind of understanding. [0:04] Maybe some of that understanding resides [0:06] in model weights. To me, that's like [0:07] pretty unsatisfying. [0:08] >> Comparing anthropic with open AI, do you [0:10] detect any differences in how that is [0:13] similar to human reasoning? [0:14] >> They definitely are not good at it [0:16] autonomously, but with some hints, you [0:17] can kind of get them to do something [0:19] interesting. A lot of progress in [0:20] mathematics comes from like letting a [0:22] thousand different flowers bloom and [0:24] people pursue their own curiosity and [0:26] then the boundaries of knowledge expand [0:27] in some fairly uniform way. [0:29] >> What has been the most impressive result [0:32] so far? [0:33] >> My favorite fully autonomous result by [0:35] an AI so far is the solution to the [0:37] Irish unit distance problem. There was [0:39] some lema I wanted to prove none of the [0:41] frontier models could do it. So I like [0:42] worked out a ton of examples on my own [0:44] and I realized oh well maybe like here's [0:46] some reason why it could be true. Once I [0:48] had that statement, the models were able [0:49] to very quickly prove that sort of [0:51] better statement. [0:52] >> How should like the mathematics [0:53] community kind of best adapt and benefit [0:56] from this. [0:57] >> Um [1:00] >> I am so excited to have you on Daniel. [1:03] Um and so Daniel is a professor of [1:06] mathematics at the University of [1:07] Toronto. Um Toronto is my hometown. So, [1:10] uh also very exciting. Um but the thing [1:13] that is most um special here is Daniel's [1:15] an actual practicing mathematician and [1:17] in addition he's been incredibly vocal [1:18] about um his evolving views of AI in [1:22] math. And so um I feel like every if I [1:25] just don't check in with you you know [1:26] like in two weeks something you know [1:29] different has been revealed and then [1:31] you're you're um very kind of like what [1:34] do you call it? Um so you [1:36] >> I have a lot of opinions. [1:37] >> You have a lot of opinions. Exactly. So [1:38] I want to get into that. So, I mean, you [1:40] know, one of the things um that I'm most [1:43] interested in is not just like a [1:44] discussion of how the capabilities of [1:46] advance. I feel like in math it's, you [1:48] know, that that's definitely the the [1:50] headline, etc., but also, um, you've [1:53] been very thoughtful about how [1:54] practicing mathematicians should [1:56] respond. Um, and so that kind of gives [1:58] us a chance and opportunity to talk [2:00] about actually what is special about [2:01] math, you know, like it's not just like, [2:02] hey, AI has been really making progress [2:04] here, but like delve into what actually [2:06] mathemat mathematicians do. Um and so [2:09] maybe like with that um uh arc in line [2:12] we can start with um what has been the [2:15] most impressive results so far uh given [2:18] all the recent progress um uh for you [2:21] and then maybe yeah we'll kind of take [2:22] it from there. [2:24] >> Yeah. So okay so there have you know now [2:25] been a lot of results some of them [2:27] produced autonomously some produced like [2:29] semi-autonomously some who's like where [2:31] the AI contribution is just not at all [2:32] clear um they're in a lot of different [2:34] areas so anything I say is kind of you [2:37] know I can only really comment on things [2:39] that I you know have some expertise on [2:41] so it's quite possible that if you talk [2:42] to a different mathematician you'll get [2:44] like different answers here uh so my [2:46] favorite uh like fully autonomous result [2:49] by an AI so far is the solution to the [2:51] Irish unit distance problem which I [2:53] think was announced in midMay. Um, so at [2:56] least what I liked about that is it [2:58] seemed to me that like it was in some [3:00] ways a little bit creative. So uh I [3:02] think some of the results uh we've seen [3:05] have had kind of the flavor of like you [3:07] know you kind of take some known [3:08] techniques and apply them in maybe a [3:10] clever way or you uh I don't know they [3:13] have kind of been some kind of results I [3:15] would characterize as like last mile [3:16] like where some recent work was done [3:19] like quite deep work done by a group of [3:21] human mathematicians and then the AI [3:23] kind of took the final step. [3:24] >> Yeah. Um but yeah with the sort of you [3:27] know distance problem I I think it was [3:28] something where the result was like a [3:30] little unexpected. So first of all my [3:33] sense was like that people working in [3:34] the area thought it was true and then [3:36] there was a counter example found but [3:38] then also it like brought in some [3:39] techniques from from another area. I [3:42] think those techniques were like not [3:43] especially [3:45] uh you know like deep or new. they were [3:47] sort of classical ideas from the 60s but [3:49] they were new to this area of studying [3:52] um you know point configurations in the [3:55] plane and so that was pretty cool and [3:57] then afterwards we got to see like it [3:59] was kind of fruitful so [4:00] >> a bunch of mathematicians took those [4:02] ideas and used them to find counter [4:03] examples to a bunch of other interesting [4:05] open questions so for example like the [4:07] sum product conjecture over the real [4:09] numbers [4:10] >> um so that's at least what way I like to [4:11] think about how cool [4:13] >> result is like you look at it post hawk [4:14] and you see like oh well you know we're [4:16] whatever new ideas that were introduced, [4:18] if any, like kind of useful to do other [4:20] things like did they improve our [4:21] understanding of something and I think [4:22] that's maybe so far the the the main [4:25] example I know of of a of a result of [4:28] that form. Yeah, I think it's really [4:29] meaningful that you're commenting on [4:31] this because in um you know that result [4:34] came out to your point in May and [4:36] there's been so much kind of like so [4:38] many headlines uh so far and it's kind [4:40] of probably hard for somebody who's not [4:41] a practicing mathematician to appreciate [4:43] the differences in these headlines and [4:45] so you kind of already started laying [4:47] out sort of a tonomy of like what is [4:49] different in that proof and so it' be [4:52] kind of interesting maybe to use as that [4:54] as kind of both an excuse to talk about [4:56] where you sense the model differences [4:57] are and what like math mathematicians [4:59] actually do. So in this case, I mean the [5:01] most kind of maybe naive understanding [5:04] of what mathematicians do is that we're [5:05] pushing around symbols in a logical [5:07] manner. And this is why, you know, RL is [5:08] so successful at this because you can [5:10] kind of both verify it somewhat cheaply [5:13] um compared to other domains and then [5:15] also you know because like the rules are [5:17] quite um uh quite neible and so you can [5:21] kind of like you know if you're if [5:22] you're super human at that you might be [5:24] good at math. Um, but I think that of [5:26] course betrays most of actually what is [5:28] miss what is interesting about [5:30] mathematics which is perhaps I think you [5:32] said this as well but I think anybody [5:34] who's tried to do math is sort of like [5:35] it's about the understanding and sort of [5:37] getting at truth remaining confused and [5:40] developing intuitions and very little I [5:42] mean the tool to do that stuff is of [5:46] course in having really [5:48] good very uh strong abilities to push [5:52] out logical you know implic [5:54] applications. Um, but maybe if you can [5:57] kind of talk to speak to like, you know, [6:00] when you say it's most impressive and [6:02] creative, like decoupling the um just [6:05] the inhumane maybe feats of just like [6:08] logical um implication from like where [6:11] is it being creative? What is kind of um [6:14] what is it helping in gender in in terms [6:17] of like mathematical activity as well? [6:19] >> Yeah. Okay. So, first of all, I mean you [6:21] you kind of characterized it as like [6:22] inhuman in some way. I actually think [6:24] the argument was very human like you [6:25] know I'd love to judge or sorry open AI [6:27] like released some chain of thought it [6:29] was very recognizable. It was like kind [6:31] of you know if if I tried to imagine [6:33] like my chain of thought [6:35] >> uh in trying to solve a problem like it [6:36] might look kind of like that. [6:38] >> Yeah. [6:38] >> Um you know we haven't seen the raw [6:40] chain of thoughts maybe that [6:42] >> maybe they cleansed it a little bit. [6:44] Yeah. [6:44] >> Yeah. Or maybe you know maybe the model [6:46] like to swear a lot in the middle of the [6:47] chain of thought and they clean that up [6:48] or something. We don't know. But like at [6:49] least the summary seemed prettyable. And [6:52] I would say that's actually like kind of [6:53] typical of uh most of the results that [6:56] I've studied. Like they don't seem [6:57] inhuman at all. They seem absolutely [6:59] like something a human mathematician [7:01] could produce. Um and uh they're like [7:05] typically understandable if like not so [7:07] well written if you just like look at [7:08] raw model output. It's not like there's [7:11] some you know move 37 or whatever. Um [7:15] it's like a human mathematician doing [7:16] math. It's like a human mathematician [7:18] doing certain types of math. So like [7:20] they're definitely the models are still [7:21] like not very good at some mathematical [7:24] activities. And here I don't mean like [7:26] by field but just like certain things [7:27] you do when you try to solve a problem [7:29] the models don't seem to be doing but [7:30] certain things they're very good at. So [7:33] um the ways they might be a little bit [7:35] inhuman is like they don't get tired. [7:37] They know a lot. Um but it if you have [7:40] to just read the final output it doesn't [7:42] seem kind of inhuman. [7:43] >> It's interesting because I mean [7:44] obviously we can't get too much of [7:47] information from the labs who are [7:48] producing these models of like why um [7:52] you know how of what the training [7:54] recipes are or how they're kind of [7:55] advancing in reasoning. Um but at least [7:57] one of the things we do know um and I [7:58] think kind of open AI spearheaded this [8:00] is this like reasoning in natural [8:01] language is actually what they happen to [8:03] scale up and it's not actually pushing a [8:05] lot of lean verified proofs as like the [8:08] training corpus and that's kind of [8:09] amazing. Um on another side, what's kind [8:12] of interesting is it's not clear that a [8:15] lot of the mathematical um training [8:17] data, if they use that to a large extent [8:20] at all, is reflective of how [8:22] mathematicians think, if that's fair, [8:24] because a lot of it is not like legible [8:25] as traces, right? Like most of the [8:27] papers are crisp and like polished. The [8:29] textbooks certainly are just show very [8:32] little motivation of how something is [8:34] developed, which is why it's like [8:35] usually easier to kind of follow [8:37] research direction by actually talking [8:38] to the researchers and how they're [8:40] thinking about it. So kind of curious [8:42] before maybe even going to the texonomy [8:43] as an excuse like when you're examining [8:46] these uh models and their results [8:48] comparing anthropic with open AI, do you [8:50] detect any differences in in how that is [8:53] similar to human reasoning? And then [8:55] also if you have any comments on like [8:57] insights on perhaps why natural language [9:00] scales so well that way even though it's [9:03] >> okay. So so first of all I really like [9:04] your point by the way that that they're [9:06] mostly doing natural language reasoning [9:07] rather than lean. Like I think that that [9:09] suggests to me that like these you know [9:11] you hear a lot of people say like math [9:12] is a verifiable domain like that [9:14] explains these the progress whatever [9:16] like my sense is that because they're [9:18] you know primarily scaling in formal [9:20] reasoning like probably the techniques [9:22] are going to generalize to other domains [9:24] pretty well. That's just my guess. [9:26] >> Okay. Um, you asked a little bit about [9:27] like Claude versus CHBT. [9:30] >> Um, my sense is that they're pretty [9:32] similar in terms of capabilities. I' [9:34] I've played a lot around a lot more with [9:35] CHGBT than than Claude Fable, but you [9:38] know, I it seems like there's a lot of [9:40] cases where, you know, uh, Open AI will [9:42] drop a solution to some problem and [9:44] anthropics will say, "Oh, you know, [9:46] too." Exactly. [9:47] >> So, it actually seems like they're [9:48] they're solving a very similar [9:50] collection of problems and it's like [9:52] kind of a small, you know, [9:54] uh it's like a relatively small portion [9:57] of what human mathematicians do. Um so [9:59] yeah, one thing um that is interesting [10:02] is like we see the solutions have a [10:04] certain flavor, right? [10:05] There'll be things that this isn't [10:07] surprising like there'll be things that [10:09] rely on the model strengths like their [10:10] ability to like grind out along [10:12] computation or like you know pull [10:14] together kind of technical ideas for [10:15] many areas um or like many maybe many [10:19] papers that you know a human my [10:20] mathematician might not have read but [10:22] they're not like they seem like weaker [10:23] in things like intuition or like having [10:26] some big picture point of view like a [10:27] lot of what I do as a mathematician is [10:29] like I have some kind of philosophy [10:30] that's like very non-rigorous like maybe [10:33] I think this thing is kind of analogous [10:34] to thing and then like a lot of what I'm [10:36] working out is like figuring out how to [10:38] make that precise and like you know [10:40] trying to measure the extent to which [10:41] I've succeeded in understanding that by [10:43] like can I solve a problem or whatever [10:45] like can I find an interesting [10:46] phenomenon which I don't understand and [10:47] then now I understand it and like so far [10:50] um I haven't you know you can even the [10:53] best results the models are producing [10:55] you don't see um that much of this kind [10:56] of reasoning it's more like they're very [10:59] very good at applying some known [11:00] techniques [11:01] >> um which is to be clear that's like a a [11:03] very powerful thing to do to be very [11:05] good at applying like all known [11:07] techniques. [11:08] >> Uh there are mathematicians who have had [11:09] great careers uh doing very high quality [11:12] work of that flavor and I think a lot of [11:14] what the models are producing it's high [11:16] quality in that way but it's it's like [11:18] some kind of fairly narrow band of what [11:19] mathematicians care about um so far um I [11:23] do think there's like signs of both um [11:26] you know all the frontier models [11:28] starting to be able to do more fuzzy [11:30] things um so like I've I've tried to get [11:34] uh both both Fable and um Chad GPT 5.6 [11:38] six, I guess, to do some kind of theory [11:40] building. And it's like they're not [11:41] good. They definitely are not good at it [11:43] autonomously, at least with like [11:44] whatever scaffolding I've set up. But [11:47] with some hints, you can kind of get [11:48] them to do something interesting. Um, [11:50] you know, when you do when you give the [11:52] models hints, it's always a little hard [11:54] to tell like what part is from the model [11:56] and what part is from you. But my [11:58] experience is that like if they can do [12:00] it with like a hundred bits of hints or [12:02] whatever in six months, maybe they can [12:03] do it without hints. Yeah, [12:05] >> you know, I do think there's signs that [12:06] they're also kind of somehow picking up [12:08] some of this like implicit and unwritten [12:10] mathematical knowledge. Okay. I would [12:12] love to so go into the um the intuition [12:16] part and where it um sucks at um to put [12:20] it in a very basic way, but um you [12:22] actually mentioned a small detail which [12:24] is that you know from the public you've [12:27] gleaned that anthropic and open are [12:28] probably neck and neck but you [12:30] personally are using a lot more um [12:31] chatbt like why is that or 5.6? [12:35] >> Well, why is I don't know. I mean I just [12:37] I think it's just in our show like I [12:39] have [12:39] >> pretty sure of it. um it got one thing [12:43] is that chat GPD got better at math [12:45] earlier. So [12:46] >> like for a long time um the claw model [12:50] models were just like not useful for [12:51] research math and then I think maybe [12:52] around opus 4.5 or opus 4.6 they like [12:55] more or less caught up. [12:56] >> Yeah. Uh but you know some [12:58] experimentation suggests to me they're [13:00] pretty neck andneck and so for my own [13:02] work you know except when I'm just [13:03] experimenting I mostly just stick with [13:05] one and sort of random [13:06] >> as people outside of labs like us it's [13:09] really interesting just to compare how [13:12] they differ on the frontier and I mean [13:14] you know to your point it might be a [13:15] little bit of momentum I do think that [13:18] at least from my anecdotal experience [13:20] 5.6 six has been like a lot more clear [13:22] in exposition and it just like there's a [13:24] little bit and this might not be true. I [13:26] mean obviously the the models are just [13:27] incredibly jagged at the frontier. Um [13:30] but like it's in the explanations of uh [13:34] results to me. I I always find that 5.6 [13:37] is giving a more accurate theory of mind [13:40] of what it assumes I know and don't [13:41] know. whereas Fable might be explaining [13:44] something very trivial but then just [13:45] like jump at like well you know [13:47] obviously you should know these things [13:49] and [13:50] >> I'm not I find they're both pretty bad [13:51] at theory [13:52] >> okay great so then see from in your eyes [13:55] it's you're probably asking much deeper [13:57] questions um okay so um the thread that [14:00] I really wanted to pull on was when [14:02] you're talking about um the models maybe [14:03] starting to get better intuitions or [14:05] even theory building so maybe before we [14:06] even dive into that it would be useful [14:08] to kind of talk through like what is [14:10] your primary ary you know activity as a [14:13] mathematician um especially in your area [14:15] of like algebraic geometry probably has [14:18] a very different flavor than a [14:19] combinatorialist or you know um some [14:21] other areas. So if you can give a little [14:23] maybe brief lay of the land and then [14:24] kind of explain what your what your ma [14:28] ma mathematical activity was preai and [14:30] then maybe how it's kind of like [14:31] changing with AI. [14:32] >> Yeah. So so I I mean I think there are a [14:34] lot of different kinds of [14:35] mathematicians. Um there's like a lot of [14:37] different uh you know spectra on which [14:40] one can put a mathematician. Um so uh [14:44] definitely a lot of mathematicians like [14:46] solving open problems. [14:47] >> Mhm. [14:47] >> Uh and then I think I'm one of those [14:50] like I like to solve an open problem. [14:52] That's uh I think of myself as a problem [14:53] solver as opposed to like one you know [14:55] one other taxonomy you could you could [14:57] have is a problem solver versus a theory [14:59] builder. Mhm. [15:00] >> Um at least for me, uh the point of an [15:04] open problem is it's like supposed to [15:05] measure your failure to understand [15:07] something. So it's kind of like a [15:08] benchmark. [15:10] >> Um right like uh you know one problem I [15:13] really like is the gross decap curp [15:16] whatever that is. It's it measures [15:18] something about our our failure to [15:19] understand differential equations. So [15:21] it's like there's some very basic object [15:23] we would like to understand. If we can't [15:25] answer this conjecture we know we don't [15:27] understand it. [15:28] >> Mhm. Okay. So in practice like how do [15:30] you get at a problem which is supposed [15:32] to be measuring something you don't [15:33] understand? Well like of course you try [15:35] to understand the thing better. Um and [15:37] and in practice what that means is like [15:40] well you try to find the smallest [15:42] situation where you can't understand [15:43] something and and and fiddle around with [15:45] it and then you stare at like what once [15:47] you win you stare at like what you [15:49] develop to win and try to turn that into [15:50] some theory. Um so that's one thing you [15:53] might do. You might try to like solve a [15:54] problem and in so doing develop some [15:57] kind of new understanding of the [15:58] situation. Uh you might also just like [16:00] have some feeling that like this thing [16:02] is related to this other thing. Um and [16:05] you know you might start building a [16:06] table like oh this property a is related [16:08] to property a prime property b is [16:10] related to property b prime and so on so [16:12] forth. So uh for example uh in in my [16:16] work a lot of it is motivated by some [16:18] analogy between homology of algebraic [16:20] varieties and representations of [16:22] fundamental groups. Okay, so that's some [16:24] some fancy stuff. Uh, but it's that this [16:26] analogy is like very very fruitful and [16:29] like really any phenomenon that appears [16:31] on one side, you can find an analog on [16:33] the other side. And so you know trying [16:36] to uh to realize that dream has led led [16:41] to a lot of beautiful mathematics of the [16:42] last like 30 or 40 years um by people [16:45] like Carlos Simpson and and Takuro [16:47] Machisuki and others. Um, and so here [16:50] it's really just like someone noticed [16:51] like here's an analogy and then that [16:53] analogy has like led to a huge amount of [16:55] developments. There's not really like an [16:56] open problem at the end although of [16:58] course like as you develop this you come [16:59] up with lots of open problems. There's [17:01] just like a philosophy that you're [17:03] trying to uh you know you're trying to [17:04] realize and that philosophy is like [17:06] super not rigorous actually. It's not [17:08] like symbol pushing at all. [17:10] >> Um yeah so that's another kind of [17:13] activity that I like. Um yeah, beyond [17:15] that uh you know a lot of [17:21] uh what you do when you try to [17:25] you know a lot of the activities you [17:27] like you're actually trying to figure [17:28] out what the right question is even like [17:31] here you're you have some object you [17:32] feel like you don't understand it and [17:34] like figuring out what you don't know is [17:36] like actually a very challenging thing [17:38] to do. So, so um you know there's like a [17:41] list of you know you can go online and [17:43] list open conjectures or whatever and [17:45] this like doesn't really capture in a [17:46] lot of ways what we don't know like [17:48] often finding the conjecture is like [17:50] really really hard. Uh so I don't know a [17:52] good example of this is like the burton [17:53] swerend dire uh conjecture which is one [17:56] of millennium problems uh is this like [17:58] beautiful relationship between l [18:00] functions of elliptic curves and uh the [18:03] the set of solutions to the [18:04] corresponding equations of the rank of [18:06] the of the group of solutions. sort of [18:08] what that means. Um it was discovered by [18:10] like it was like the first big data [18:12] conjecture. So Burton Dyer had like [18:15] found all these statistics on elliptic [18:16] curves in the 60s like this one of the [18:18] first ever computer aated bits of [18:20] mathematics and they like graphed these [18:22] statistics and they noticed that you [18:24] know some the slope of some line on a [18:26] graph was related to some other [18:28] algebraic variant they knew and that was [18:29] the source of this conjecture. So a lot [18:31] of the time you're just like working out [18:32] examples and like trying to like you're [18:34] doing kind of science like you run an [18:36] experiment and you try to figure out an [18:37] explanation [18:38] >> for that um experiment. And so I mean I [18:41] think going back to AI I think like [18:43] these are things where AI seems so far [18:45] to you know help a lot more in some [18:47] things than other things. So like uh you [18:50] know I think the the more vague [18:53] uh uh a phenomenon is or like the the [18:56] less you have a precise question in mind [18:58] the less useful h happens to be. And so [19:00] you were asking like how I use it in my [19:02] daily life like actually what I found is [19:04] that the projects that I have that kind [19:07] of predate AI like the projects I've [19:09] been thinking about for three or four or [19:10] five years um it's just not that useful. [19:13] like it's primarily kind of a substitute [19:15] for Google or something like I might use [19:16] it to like learn about some related [19:17] topic or like something where I would [19:19] have earlier Googled something and then [19:21] like read a paper like maybe I'll [19:22] discuss it with AI instead. So it saves [19:24] some time for sure. It's like not really [19:26] doing deep intellectual work for me. But [19:28] then you know because I you know I'm now [19:30] like well I I suck at coding and so now [19:33] I have you know my good friend who's [19:35] really good at coding and now I have all [19:36] these coding projects because like [19:38] suddenly well I if I had a question [19:40] where coding would have been really [19:41] useful I would have procrastinated on it [19:43] for six months until I [19:45] >> tireless PhD student. [19:47] >> Exactly. Yeah. So, so yeah, now I picked [19:49] up all these projects where yeah, coding [19:50] is really useful. Like [19:52] >> the models are like very good for kind [19:53] of massively parallel things. Like if [19:55] you want to find an example of [19:56] something, you can just ask it to find, [19:58] you know, work through a thousand [20:00] examples in parallel or maybe 10 [20:01] examples at a time, you know, 10 [20:03] different sub aents and like that's [20:04] really useful. But these are like kind [20:06] of different activities uh which are [20:08] like I don't know on top of what I was [20:10] doing before. [20:11] >> Um [20:12] >> I'd love to dig in. [20:13] >> So yeah, it's Yeah, go. [20:14] >> Yeah. Sorry. cuz you were mentioning you [20:15] there's projects that you've been [20:16] working on for like three fi you know 3 [20:18] four five years and um I don't know if [20:20] it's correct to say like those are more [20:22] of the theory building aspect of it cuz [20:24] you did characterize yourself as like an [20:25] open problem solver or like what is the [20:28] thing that is because like the deep [20:29] thinking part maybe maybe like you know [20:31] kind of for this audience it might also [20:33] be useful to kind of say or explain you [20:36] know most of pure math is it's not like [20:39] it's being motivated by you know nothing [20:41] on applied math but applied math is like [20:43] at least there's some external [20:44] motivation for why a certain formal [20:46] structure might be interesting to study. [20:48] Whereas this one it's like purely it [20:50] seems almost sociological and to some [20:51] extent I think like thirsten made some [20:53] comment you know or point about this in [20:54] the 70s that it is a sociological [20:56] phenomenon of more and more [20:57] mathematicians start examining something [20:59] then you'll maybe you know converge on [21:02] some interesting structures but it's [21:03] just like it's not it's there's some [21:06] reason why people would prefer to study [21:09] something or they think it's beautiful [21:10] like what is it that drives maybe you in [21:12] particular and then maybe you can make a [21:14] more general comment about the [21:15] profession. Yeah. I mean, so definitely [21:18] some people are like motivated by like [21:19] beauty or or or kind of aesthetic [21:21] considerations. I try not to be [21:23] motivated by that. [21:25] >> I think that's a bad way like Well, one [21:28] thing like that sort of limits you, [21:29] right? Like one [21:31] >> thing kind of a failure mode I see among [21:33] young mathematicians sometimes is like [21:35] you you have something and you kind of [21:37] think you know how to prove it [21:39] >> and then like the proof feels really [21:41] ugly and you decide but like okay I mean [21:44] what if you're wrong and it's not ugly [21:45] like you why limit yourself you should [21:47] >> for this audience like what is ugly [21:49] because I have an intuition of what's [21:50] ugly but like what is that for kind of [21:52] spell [21:52] >> out I I don't really know I I don't [21:54] really have aesthetic feelings but [21:56] people people sometimes feel this way [21:57] like maybe it involves a lot of grinding [22:00] calculation that's not illuminating or [22:02] >> but you should like win by any means [22:04] necessary in my opinion like I I like to [22:06] think of what I'm doing like doing kind [22:08] of physics except with concepts [22:11] >> so um you know I instead of beauty I try [22:14] to think about maybe like what is kind [22:15] of fundamental what's going to open up [22:17] further understanding most [22:20] >> will you know am I introducing a new [22:22] idea that will be like broadly useful to [22:24] understand this object [22:26] >> and I guess you know in some sense [22:28] there's there's like some aesthetic [22:29] consideration there, but it's I try to I [22:31] think the the orientation of like trying [22:33] to do good science rather than trying to [22:34] do art is what I prefer. Uh but there's [22:37] a huge I mean there's a huge variety of [22:40] of opinions here and like lots of [22:41] mathematicians think of themselves as [22:43] being like closer to poets or something. [22:45] >> Um yeah, what I what I do think is [22:47] broadly true is that like progress comes [22:49] from people like it seems to come in [22:52] mathematics from people sort of pursuing [22:54] their personal curiosity. Um and that [22:57] it's sort of been crucial historically [22:59] that there's like a lot of different [23:00] people with different views of what's [23:02] interesting and then the frontier of [23:04] knowledge expands and expands and then [23:05] suddenly you know you have these [23:07] opportunistic situations where a new [23:09] idea has been introduced and you can [23:10] kind of now find you now can suddenly [23:13] like cascade through a bunch of other [23:14] things that we didn't understand before. [23:16] >> Yeah. I mean, so then going back to the [23:19] three, four, five year um problems and [23:21] where are the models sort of not useful? [23:26] kind of ask it another way like when [23:28] you're doing the deep thinking like is [23:29] it just that it's just not clear that [23:31] you formulate it as a [23:34] um problem and it's more that you're [23:36] thinking about these like what are you [23:38] know what what are the fundamental kind [23:40] of physics of you know [23:41] >> yeah so in some cases I mean the the [23:44] some cases there is like a wellstated [23:46] problem here like you know you know I [23:48] said I've been thinking about things for [23:49] se you know maybe 10 years now at this [23:51] point for certain problems sometimes [23:52] there is just like a conjecture [23:54] >> that I would like to prove that that is [23:56] is there. Um I think one reason the [24:00] models might not be useful for some of [24:01] these things is like the conjectures are [24:03] true. Um so like uh I think that certain [24:07] you know for for example um you know [24:09] this unit distance conject problem like [24:12] the the general belief uh in the [24:14] community was that it was true and then [24:15] it turned out to be false. [24:16] >> And so what that means is that there's [24:18] like a specific construction you can do [24:20] to [24:20] >> to um uh to to refute it. Um on the [24:25] other hand I think a lot of the things I [24:26] I think about like I know okay maybe I'm [24:28] about to you know that someone will come [24:30] up with a counter example to the [24:31] curvature conjecture hypothesis tomorrow [24:33] and like uh I'll look like a fool. Uh [24:36] but that in general like these [24:38] conjectures fit into some very broad [24:40] theoretical framework which means that [24:42] like we actually have a lot of evidence [24:43] that they're true. Um and so uh you know [24:49] yeah so that's part of it like there's [24:50] not like a construction you can do to [24:52] refute it. um you need to somehow kind [24:54] of uh you know there's we have this [24:56] giant framework where certain certain [24:58] pieces of it are only conjecturable and [25:00] you probably need a to resolve some of [25:01] those conjectures to win. [25:03] >> Um we also have like a pretty good sense [25:05] I think that like very serious new ideas [25:08] are needed to resolve those conjectures. [25:11] >> So like um of course you can't be sure [25:14] like maybe there's some very clever [25:15] construction that will let you I don't [25:17] know uh avoid having a big new idea. [25:20] >> Um I don't know. uh uh maybe you know [25:23] it's quite possible we'll find this out [25:24] but my sense is that for at least a lot [25:26] of the things I've been thinking about [25:28] um there are they're just not accessible [25:32] to um you know applying known techniques [25:36] very very techn in a very technically [25:38] strong way um so you need to develop a [25:40] new technique just be clear like I'm not [25:42] saying the models won't be able to do [25:44] this it's just like so far they seem not [25:46] to [25:46] >> actually that that's exactly the point [25:47] that I wanted to delve into because it's [25:49] um you know to your point to your point [25:51] is like okay we can try try try to [25:52] calibrate and forecast like why they [25:54] would get better at this but it is true [25:56] like you know that just providing a [25:58] construction for a counter example they [26:00] seem to be strong on if you have to [26:02] start developing either new theory or to [26:05] your point techniques machinery to solve [26:08] to to prove why conjecture is true it [26:11] struggles more and it's probably because [26:12] a lot of what it's drawing on is also [26:14] just like techniques that have happened [26:15] in other areas and they're porting it [26:17] over and and to your point that's why [26:18] maybe the unit distance problem was such [26:20] a more creative result because it was [26:22] maybe doing more of that on its own. [26:23] >> It was like from another at least it was [26:25] like drawing in something from [26:26] unexpected area. [26:27] >> Yeah. Exactly. Exactly. And so like [26:28] maybe then to kind of ask the question [26:30] it's not that because now we're like [26:31] okay fine AI is getting so good so fast [26:33] we can't count it out but like why what [26:35] do you think it has to um yeah I guess [26:38] it's it's you're spelling out what it [26:41] has to get better at but maybe some more [26:42] kind of feelings on like why it's kind [26:44] of particularly hard to then develop [26:46] that new theory and technology. Um, [26:48] yeah, it's a good question. I mean, I [26:50] think you just need a different like my [26:51] my guess actually is it's probably [26:53] totally doable. Um, and it just hasn't [26:55] been done yet. Like maybe you just need [26:56] a different RL environment. Um, I don't [26:59] know. Yeah. So, I at this point my [27:00] expectation is just like that the [27:02] trajectory will continue upwards. I'm [27:04] not a skeptic of continued capabilities [27:06] growth. [27:07] >> Um, yeah. You know, I think what is [27:10] definitely true is that like the skill [27:13] of like developing a theory um or like [27:16] building your understanding of some [27:17] poorly understood object is like a [27:19] fuzzier one. Mhm. [27:20] >> Uh so it might be harder you know uh I [27:23] guess you can try you can tell it you [27:25] know uh develop your understanding of [27:27] zeta functions then once it proves [27:29] through your hypothesis you give it a [27:31] reward but it's like kind of harder I [27:32] think to come up with some like [27:33] intermediate no things that you can [27:35] reward um [27:37] >> yeah that said you know I do think uh [27:42] mathematics as a whole provides a lot of [27:44] conjectures of varying levels of [27:46] difficulty so you know maybe this [27:49] explains why there seems be a little bit [27:50] of progress in these areas like [27:51] presumably they are trying to you know [27:54] get it to solve lots of problems and [27:55] some of those problems develop at least [27:57] some of the skills of theory building [27:58] and humans are able to develop these [28:00] skills um you know I guess sometimes [28:02] they get rewards from their PhD [28:04] adviserss and their advisor says oh [28:06] that's a good idea or something based on [28:07] some element of human taste or whatever [28:10] >> uh and that that might be something one [28:12] can do too but yeah and my my [28:14] expectation is that just like as part of [28:16] continued capabilities growth we'll see [28:18] growth in these areas too [28:19] >> yeah yeah And I like the framing where [28:20] you're sort of casting these [28:22] increasingly difficult conjectures as a [28:24] as a you know a form of curricula for [28:27] both humans and of course AI. And it it [28:31] you know maybe kind of getting a little [28:34] bit too philosophical for some people's [28:35] tastes. It's like it it it gets at the [28:37] question of like why are we particularly [28:39] good or maybe particularly bad at math [28:42] too because it's um you know what is it [28:45] that we're either struggling to do or or [28:49] some people are particularly good at [28:50] when you develop new theory because it's [28:53] it's not again it's like not really [28:56] or I maybe it's related to like why can [28:58] we formulate good structures for physics [29:00] as well. That's just like it's not [29:01] obvious all parts of the world are kind [29:03] of understandable um and legible that [29:05] way but some parts are and therefore we [29:06] try to do it because there's maybe [29:08] compressive pressures on our our you [29:11] know our our minds because we need to uh [29:13] we can't understand anything uh beyond [29:16] you know compressing uh stuff uh more [29:18] finely. Um, I don't know if that's also [29:21] like the correct interpretation of [29:23] >> I think that's that's I mean I I'm a [29:25] little bit skeptical as like compression [29:26] as a metric of interest, but it's [29:28] definitely like as a anthropological [29:31] reason for like why we do a certain [29:32] theory building. I just think it's true [29:34] and in fact I think it's like kind of [29:35] like our our inability to just grind is [29:38] kind of important to our ability to make [29:40] discoveries. So and just like as an [29:42] example uh so I have I have one paper [29:45] out so far where uh the models were kind [29:47] of useful. So they like like proved a [29:49] couple lemas. [29:51] >> Um so this is some some situation where [29:53] like we had kind of proven the main [29:55] result and then there were like some [29:56] lemas I was unhappy with like they [29:57] seemed not to be optimal and so I like [29:59] worked with actually this was with um [30:01] with Gemini uh deep think which at the [30:04] time was also on the frontier no longer [30:07] man. [30:08] >> Um I I where I kind of worked with it to [30:11] to prove the lemas. And so what happened [30:13] here was there was some lema um I wanted [30:15] to prove uh and the models couldn't do [30:18] it. So the no none of the frontier [30:20] models could do it. And so I like worked [30:22] out a ton of examples on my own and I [30:24] realized oh well maybe like here's some [30:25] reason why it could be true. Like so I [30:28] like I found a better statement of the [30:30] lema. And then okay once I had that [30:32] statement like okay probably I could [30:33] have done it pretty fast but also the [30:34] models were able to very quickly prove [30:36] the pro prove that sort of better [30:38] statement. Um, so like our inability to [30:41] prove it like led to an improvement in [30:42] the result. Okay, now you can take the [30:44] original lemma I had that the models [30:46] weren't able to prove, put it into chat [30:48] GPD 5.6 Pro and it will output like the [30:51] worst proof you've ever seen, like 10 [30:52] pages of like just brutal calculation [30:54] with no insight whatsoever. And so, [30:56] okay, this would have been a perfectly [30:57] fine proof, but it would not have led to [31:00] this discovery of like I think a kind of [31:02] beautiful conceptual explanation for why [31:04] this thing we discovered was true. So [31:06] like we found a better proof because we [31:07] couldn't do I mean okay I I say we [31:09] couldn't do the calculation like what [31:11] actually happened and here I I [31:12] understand that I'm being hypocritical [31:14] about my complaints about ugly proofs [31:15] before is like I realized like this [31:17] horrible grind proof would work and I [31:20] like could not bring myself to do it [31:22] >> uh and so I like for another argument uh [31:26] and so and it's not yeah but you know [31:28] now the models can do for very long [31:30] technical calculations pretty reliably. [31:33] >> Yeah. Yeah. Well, I mean to your [31:34] >> credit. I don't know. [31:34] >> I think I understand why you're saying [31:36] it's like don't, you know, just shy away [31:38] from trying to prove something just [31:39] because it seems ugly because it's like [31:41] you have to take the first step, right? [31:42] And eventually you all try to work [31:44] towards insight. And so, you know, you [31:46] might be shy about admitting it, but it [31:48] is still maybe driven by whether it's [31:50] like aesthetics or just like pure, you [31:52] know, like I want to understand. And if [31:53] understanding just means it's a little [31:55] bit simpler or, you know, more [31:57] compressed, then then I've gotten [31:58] understanding. I mean, it's just like a [31:59] deep phil philosophical question, too, [32:00] is like what it is, [32:01] >> right? Like I mean of course any you [32:03] know there's a reason we do informal [32:04] mathematics rather than like writing out [32:06] long formal strings of symbols of CFC or [32:09] whatever even though you know besides [32:11] length it's just like somehow we're [32:13] trying to put it in a way where we are [32:14] actually getting some non-rigorous [32:16] understanding out. Yeah. I mean, and [32:18] >> I think somehow [32:18] >> Yeah. And even how that even relates to [32:20] like why it helps. And again, it's like [32:23] is is it because of our inability to [32:24] grind or is it because like if we if we [32:26] are pushing towards something that's [32:27] more compressive, it also hopefully also [32:30] ascends on the does it help understand [32:32] other fields as well. And it it's just [32:35] it is somewhat magical that when we try [32:37] to optimize for two the the both that it [32:40] it tends to coincide. I don't know [32:41] actually you know if that's a fair it's [32:44] actually a quantitative statement but [32:45] like why you know that is is is kind of [32:48] magical [32:48] >> it's at least sometimes true yeah [32:50] >> yeah exactly or you know we we certainly [32:51] bias towards the cases in which it is [32:53] that's why we call that a good theory to [32:55] build um um but it's it's uh yeah it's [32:58] very much I think you know um it's like [33:02] this is what we can study and understand [33:04] and therefore it's also very convenient [33:06] that it was rich um in in kind of [33:09] mathematical uh results there. Uh yeah, [33:13] I mean okay so I think there's two [33:16] things that we can like you know pull on [33:20] which is like what um given you've [33:23] seated that or not seated but you've [33:25] believe that mathematical ability of AI [33:28] is going to continue advancing and can [33:29] do some of the things that you're you [33:31] know doing nowadays. How sh how should [33:34] like the mathematics community kind of [33:37] best um adapt and benefit from this? I [33:40] mean as a just kind of saying as [33:42] somebody who's you know doesn't have the [33:43] time to kind of practice mathematics [33:45] anymore this is kind of ve very great [33:47] because I can maybe dabble more there's [33:49] a lot of results that can come out but I [33:52] can also see where you know you've made [33:55] the more precise point of we can not [33:58] motivate the right kind of behavior of [34:00] understanding and development and so I [34:02] would love to kind of hear more about um [34:04] your views there. [34:05] >> Yeah. Yeah. So, okay. So, first of all, [34:06] like it is clearly really exciting like [34:08] that there are increasingly capable [34:10] models that are like producing high [34:12] quality results. Um, you at least, you [34:16] know, some high quality results also [34:17] also a lot of slot. [34:18] >> Uh, yeah, some some good stuff. Um, and [34:21] so, you know, as the models get really [34:22] capable, you know, my hope is that they [34:24] will answer a lot of the questions that [34:26] I've been like, you know, kept up at [34:28] night thinking about and like I I'll get [34:30] to learn the answers. I think that's [34:31] really exciting. And you know, a lot of [34:33] people got into math uh like largely [34:36] because they enjoyed learning math. [34:38] >> Yeah. [34:38] >> U like the first thing you do as a a [34:41] math student is you like learn stuff [34:42] that other people did and you do that [34:44] for like 20 years before you before you [34:47] start doing maybe not quite 20 years but [34:49] >> 15 years [34:50] >> if you're lucky 20 years. Yeah. If you [34:52] started early. Yeah. [34:53] >> Yeah. Yeah. Um yeah. So um that said [34:58] like the goal of mathematics is not to [35:00] produce mathematics papers um like it's [35:03] to produce some kind of understanding uh [35:05] so maybe okay maybe that some of that [35:07] understanding resides in model weights [35:09] or something to me that's like pretty [35:10] unsatisfying like [35:12] >> uh my own you know my motivation for [35:14] doing mathematics is like I would like [35:16] to satisfy my own personal curiosity um [35:18] I think people should be uh have the [35:21] capability to do that and like that [35:23] requires a pretty substantial apparatus [35:25] like it's like simply not the case that [35:28] you can like study the questions I think [35:30] are fundamental unless you've invested a [35:32] huge amount of time and effort um kind [35:34] of getting to the point where you can [35:35] meaningfully do so [35:36] >> and then moreover like that you know [35:38] even the people like you know you have [35:40] the small group of people doing like [35:41] fancy research math on the frontier or [35:43] whatever um that relies on like a huge [35:45] apparatus of like you know [35:49] thousands and millions or billions of [35:50] people who are trying to learn to think [35:52] mathematically like you need a an entire [35:54] mathematical community to support a [35:56] small group of people who are on the [35:58] frontier like you just without the [35:59] pipeline then the pipeline doesn't [36:01] exist. [36:02] >> Uh so if you think that's important like [36:04] development of human capitals that can [36:05] like meaningfully engage with frontier [36:08] mathematics I think that uh you know you [36:10] still have to incentivize those people [36:12] to actually invest their time and effort [36:14] getting to the point where they can [36:15] engage and then do so in a in a high [36:17] quality meaningful way. Uh so right now [36:20] I think like the existing incentive [36:21] structures for math research do not uh [36:24] do not encourage people to do that. Um [36:27] so you know right now if you're like a [36:28] posttock on the market you want to get a [36:30] job maybe for the next couple years [36:32] before the community adapts uh the best [36:36] way to do that is like you want to [36:37] produce a lot of papers which maybe [36:38] prove you know old conjectures or [36:40] whatever and you can do that by playing [36:41] the slot machine until [36:43] >> um the model produces a hopefully [36:45] correct proof of such a result. Um, so [36:48] you don't even have to pick the the the [36:50] theorem in advance. So like here's an [36:52] experiment you can do. You can take [36:54] codecs, you can say go online, find five [36:57] recent conjectures in algebraic geometry [36:59] and prove them. And uh, okay, I've run [37:01] this experiment and with some back and [37:03] forth, I was able to, you know, in an [37:04] hour get like three, you know, quite bad [37:06] papers, [37:07] >> but correct papers. Um, which, okay, are [37:10] now sitting on my hard drive waiting for [37:12] me to email the relevant people, but [37:14] this is not like a good use of my time [37:15] to invest results. But yeah, you you [37:18] definitely see people doing this. Um, so [37:20] there's, you know, been a huge uptick in [37:22] post to archive, mostly not very [37:24] interesting. Um, some of it is [37:26] interesting. [37:27] >> Um, but, you know, a lot of it is kind [37:30] of clearly low quality, even if in so [37:33] far as the like even if the result is [37:36] something that would have been like kind [37:37] of highly rewarded a year ago, just like [37:39] the there's sort of no evidence that a [37:41] human being is actually engaged with it. [37:42] Like there's no development of human [37:44] capital or understanding. Like [37:45] sometimes, you know, we've seen examples [37:47] where like three or four or five papers [37:49] with the exact same proof of the exact [37:51] same theorem have come out in within a [37:53] couple days of each other, which is [37:54] clearly, you know, some situation where [37:56] someone's playing the slot machine. [37:57] Yeah. [37:57] >> Chat GPT is kind of consistently finding [37:59] the same. [38:00] >> But that's also interesting. It's like [38:01] not it's kind of mode collapsed on like [38:03] certain paths of reasoning. [38:05] >> Yeah. [38:05] >> Yeah. And I think it's like not obvious [38:07] that this problem goes away as the [38:09] models get better. Like maybe it does, [38:10] but maybe it doesn't. Like I think a lot [38:12] of what I was saying earlier is that a [38:14] lot of progress in mathematics comes [38:16] from like letting you know a thousand [38:18] different flowers bloom and people [38:19] pursue their own curiosity and then you [38:22] know the boundaries of knowledge expand [38:23] in some kind of fairly hopefully fairly [38:25] uniform way and like this really [38:27] highdimensional space of mathematics [38:29] >> and then like opportunistically you you [38:31] suddenly get some applications or like [38:33] answers to old questions we found. Uh [38:35] and it's like not clear to me that if if [38:37] you know we kind of subordinate [38:39] mathematical exploration to what the [38:41] model want to pursue like if what you're [38:43] getting is like one mathematician [38:44] duplicated a thousand times or like [38:47] actually you know a million different [38:48] mathematicians doing a million different [38:50] things. You know, I think actually [38:52] that's pretty I mean, you know, aside [38:54] from like okay, math has to adapt um by [38:56] changing incentive structures, I think [38:58] this is actually a pretty um um maybe [39:01] dangerous or something that the labs [39:04] have to pay attention to because you [39:05] know on one side what's been successful [39:08] for them is that this emergent reasoning [39:10] capability is obviously incredibly [39:11] powerful and we've been I mean it's [39:14] creating a lot of like great PR [39:15] headlines. But [39:16] >> you know to your point and also like [39:18] where my interests tend to is that like [39:21] human mathematicians they are coming [39:23] from all sorts of weird intuitions that [39:26] like you know and that is why you end up [39:28] developing you know to your point the [39:30] frontier that is so diverse that you [39:32] actually can draw from and then be able [39:34] to connect and produce a lot more. And [39:36] so it if it's true that most of these [39:39] like proofs that are being pushed out by [39:42] the labs are converging on very similar [39:44] things because they are technically [39:46] drawing on the same body of literature. [39:47] Um and that's where they're strong at [39:49] now. It's it's not clear that I mean one [39:52] you know where are they developing their [39:53] intuitions? It's from practicing [39:55] mathematicians. It's not clear where [39:56] that other emergent you know stronger [39:59] diverse intuition might be coming from. [40:00] It might come but I don't have a good [40:02] theory for where it comes from. And it's [40:03] also just not clear that you know just [40:06] with like test time compute and various [40:09] post-training scaling that you can even [40:11] I mean it's a huge debate whether you [40:12] could introduce new capabilities and [40:14] it's like precisely I think it should be [40:16] studied in the math context because like [40:17] where it's good at and where it fails um [40:20] and where human mathematicians are good [40:22] is exactly a very kind of like precise [40:25] um question to study it on. Um, so this [40:29] is like a long way of saying that I it [40:32] is unclear to me that we'll maybe [40:34] produce um if we don't incentivize [40:37] enough people to interact with it, maybe [40:39] unlike other domains, it it might not [40:41] actually uh continue producing uh a [40:44] superior uh uh result uh without human [40:47] aid. [40:47] >> Yeah, let me actually push push a bit [40:49] further even. So like let's suppose the [40:51] models become like really robustly [40:53] superhuman like even like we're not even [40:54] adding like meaningful cognitive [40:56] capacity. Okay. [40:57] >> I claim like still actually we we still [40:59] won. [41:00] >> Okay. Great. [41:00] >> Human mathematician. So, right. So, why? [41:02] So, like it's because the you know [41:04] there's like a it's like there's a [41:06] question about how we've we're designing [41:07] society, right? Like maybe uh [41:10] >> the optimal uh situation is like you [41:13] have the models doing all sorts of math [41:14] research and like that leads to very in [41:16] you know in this like highly you know [41:19] whatever non-uniform diverse way like [41:22] maybe you don't need humans to add that [41:24] kind of diversity. It's like maybe [41:25] that's the optimal thing to do and like [41:27] in the end that leads to lots of [41:28] applications and lots of imp proofed [41:30] understanding and so on. It's like [41:32] despite like something being optimal [41:34] doesn't mean you do it [41:35] >> right like there's no reason to think [41:37] that you know if we hand over control of [41:39] whatever math research to the models and [41:41] just let them do their own thing that it [41:42] will do the optimal thing. Um and in [41:46] fact like if you you know if we are if [41:48] we kind of instrumentalize what we want [41:50] them to do like we want to say like oh [41:51] you know make our life better or [41:52] whatever [41:53] >> it might not be the case like be that [41:56] that like what they decide to do that is [41:57] like is is pursue a wide variety of [42:00] interesting research right like they [42:02] might just try to take the direct path [42:03] we don't know what's going to happen [42:05] >> um so I don't know if you believe that [42:07] there is any value in sort of this [42:09] broad-based like uh fundamental research [42:11] which I do look I think that's like one [42:13] of the most valuable as human humans or [42:15] or whatever or the models can be doing [42:17] like I think the easiest way to [42:18] guarantee it happens is like to keep a [42:20] community with like broad interests who [42:22] are like pushing the models to do and [42:23] like helping us to design a society [42:25] where like that's what we're what we're [42:27] pushing for. [42:28] >> Yeah. Like I think at least you know my [42:29] my hopeful vision for the future is that [42:32] like humans are not like totally [42:33] disempowered like we have some control [42:35] over where we're going and like if [42:36] that's the case like what we end up [42:38] doing is going to be driven by human [42:40] interests [42:40] >> and so you want to have people who have [42:42] lots of different interests and also the [42:43] capabilities to actually pursue them [42:45] like you want people who are like smart [42:46] and engaged and like you know well can [42:49] do not just like mathematical thinking [42:50] but all sorts of thinking. [42:52] >> Yeah. Yeah. I mean, yeah, a great fear [42:53] is like as AI advances, we don't develop [42:55] the right ergonomic um kind of [42:57] interfaces to actually encourage us to [42:59] also be good continue to be good [43:01] thinkers and and it's so easy to kind of [43:03] um relinquish that because you just off [43:06] and and the models aren't even good at [43:08] that level of like you know thinking [43:09] where it's like the top level structure [43:11] but despite that it's it's so easy to [43:13] and so especially for math I mean just [43:15] like you know another selfish reason is [43:17] um if you kind of math max you know I [43:20] think it's actually a great pedagogical [43:22] um uh excuse to actually get just really [43:24] rigorous at thinking about various [43:26] things. I mean this is not why [43:27] mathematicians do it. Um but as just [43:30] somebody that's part of what we do [43:31] >> Okay, great. Because it was kind of, you [43:33] know, I thought it just really helped [43:35] give me a very um [43:38] good framework to think about many [43:40] things, not just mathematics. And as [43:41] somebody who's also, you know, now a [43:43] parent of a 2-year-old, I kind of think [43:45] about this a lot, too. you know, it's [43:47] it's not about kind of grinding, even [43:49] though whatever it's still good, but [43:51] like not to not to um on grinding [43:54] too much, but you know, just hearing I I [43:57] used to collaborate a bunch with some [43:59] Hungarian mathematicians, um Bery among [44:02] them, and I I heard that um in in [44:05] Budapest, they would just teach group [44:06] theory when you're in primary school. [44:08] And I'm like, well, we should definitely [44:09] do that. We should continue doing that. [44:11] now that AI is so good at, you know, [44:12] some somewhat good at explaining, but [44:14] it's far more accessible, we should [44:16] actually, you know, probably proliferate [44:18] uh that even more. Um, and so maybe that [44:21] helps with bring more people to the [44:23] frontier rather than, you know, just [44:25] >> Yeah. I mean, this is something I'm [44:26] concerned about, right? And of course I [44:28] mostly talk about math because that's [44:29] like where I live. But like I think you [44:31] know one nice thing about thinking about [44:33] this is that we're one of the first uh [44:35] professions [44:36] uh to to kind of you know have a [44:39] significant impact of high quality [44:41] models. Although I think um maybe we're [44:44] one of the first professions for it to [44:45] happen so publicly. You know, my sense [44:47] is that there are plenty of other [44:48] professions that are [44:50] >> 100% [44:51] >> coding. But also just like I mean I [44:52] think that you know anything you do at a [44:54] computer like probably a huge amount is [44:56] being done by the models at this point [44:57] and then like there not having a public [44:59] reckoning about it but [45:01] >> um you know because uh you math [45:04] capabilities are are useful for the [45:05] company for the labs to talk about I [45:07] think probably a bit more publicly than [45:08] everyone else. Um but yeah, I mean I [45:11] think uh you know one reason to try to [45:13] maintain like human capital in this area [45:14] is just like it's a model for all [45:16] professions like presumably we still [45:17] want people who are like meaningfully [45:19] engaging with the world and like experts [45:21] and you know have like talents and [45:23] trained skills and so on and like um [45:26] yeah so at least yeah it's actually very [45:29] convenient that the math profession is [45:30] so entwined with education here because [45:32] like I think we're also seeing like you [45:34] know some amount of uh challenges um you [45:38] know among [45:39] uh college and and uh like secondary [45:43] education coming from AI too. [45:44] >> So as you said I mean it's also an [45:46] amazing tool to learn. So [45:48] >> you know people have have I I've heard [45:50] he heard people start talking about a [45:51] biodal distribution in their classes [45:53] where there's some people who are really [45:55] like figuring out how to take advantage [45:56] of new tools and other people who are [45:58] just like letting them do their homework [45:59] and then bombing everything else. [46:01] >> Unfortunately I don't think that adapts [46:02] fast enough but it's like we definitely [46:04] want to be living in a world where we're [46:05] producing better thinkers. I think it's [46:07] just, [46:08] >> you know, even if we talk about just the [46:10] pure kind of optimization game, I think [46:11] that's better for us. And so um but as [46:14] you know, just like a human being, I'm [46:15] like that would be pretty [46:18] >> um inconvenient if we became worse [46:20] thinkers just as AI ascends. And it's [46:22] too easy to let that happen. So we [46:24] should kind of be thinking hard on how [46:25] to actually take advantage of this and [46:27] harness it for our own improvement, [46:29] right? Well, [46:30] >> yeah. I think it's like it's sort of [46:32] interesting to observe like the models [46:34] um at their current level of [46:36] capabilities let you do a lot more. They [46:38] let you do a lot of things you wouldn't [46:40] have done more cheaply than uh you know [46:42] cheaply enough to do them now. But it's [46:44] not clear to me that like in many cases [46:46] they're actually improving the quality [46:48] of outputs. [46:49] >> Yeah. Um, and I think this is common [46:50] like you have a new technology that's [46:52] doing something a little bit worse than [46:54] was previously done but much cheaper and [46:56] so you get a lot of suddenly a lot of [46:58] like lowquality outputs that are [47:00] displacing previous high quality [47:01] outputs. But I think it's possible to [47:03] use the tools in a way that actually [47:04] like improves you know the quality among [47:06] along every dimension. It just refers [47:08] requires some thoughtfulness and some [47:10] some redesign of institutions to [47:11] actually incentivize that. [47:13] >> Yeah. Well hopefully capitalism works [47:15] there. I do feel like that the most high [47:17] value things do require people to use it [47:21] effectively and and right now the models [47:23] are not good enough without like the [47:25] human experts to actually participate. [47:26] Um but to your point there's a vast [47:28] majority of maybe like more junior and [47:30] entry level um kind of you know it's [47:33] it's harder for um for those uh roles to [47:37] adapt as well. And so the thing that [47:40] would be a mistake is to use the AI [47:42] models in a way that doesn't [47:45] >> basically you need to be ascending and [47:46] using the models to deepen your [47:48] understanding. And it's so easy for [47:50] human nature just to be lazy and you [47:52] have to resist that because that is the [47:54] moment that you will kind of kind of [47:55] lose basically. And so you kind of you [47:58] know kind of forgive the very [47:59] competitive language but it really is [48:01] that like it's just so easy to to kind [48:03] of relinquish the thinking to the [48:05] models. The models can't really think. [48:07] And so as things are ascending so fast, [48:09] it's like critical that um you continue [48:11] developing those facilities and actually [48:13] leverage it to to to improve those [48:15] facilities rather than relinquish. Yeah. [48:18] >> Yeah. Yeah. I mean, one one thing I I [48:21] think has been nice about, you know, [48:23] sort of this vast increase in in like [48:25] semi-expert attention or like model [48:26] attention on math problems is like now [48:28] there's been, you know, okay, I've been [48:30] complaining about slot papers or [48:31] whatever like, you know, people who are [48:32] not um you know, producing really high [48:35] quality stuff. often that's actually [48:36] coming from professionals like it's not [48:39] I'm not saying like like you know there [48:41] people who you know like within academic [48:44] mathematics there are incentives to [48:45] produce like a lot of a lot of stuff and [48:48] and that's you know that's one place the [48:50] slop come from so there's definitely [48:52] also like slop coming from non-experts [48:54] but that I I kind of actually don't see [48:56] as a net negative like okay there's a [48:58] lot of now there's a lot of like [49:00] documents on the internet one might have [49:01] to comb through to figure out if a [49:03] problem has been solved or not Um, but [49:05] to me it seems like just the fact that [49:07] there's lots of people excited about [49:08] math is like kind of a positive. So [49:10] that's like a nice like [49:11] >> I totally agree. I know I get to talk [49:12] >> and not even kind of a positive [49:14] obviously. [49:15] >> Yeah. Yeah. No, exactly. It's like [49:16] suddenly there's a spotlight on it and I [49:18] can um nerd out about math more. Um I I [49:21] was actually kind of curious um you've [49:23] got any comments on the um [49:25] >> I guess this is another constructive [49:27] result, but the elliptic curve of rank [49:30] 30 that just came out yesterday. So if [49:32] that [49:32] >> That's right. [49:32] >> Yeah. Yeah. [49:33] >> Not Well, I mean, we don't have any [49:35] details about it yet. [49:36] >> I know. There's rumors. [49:37] >> Yeah, we don't know how it how it I [49:39] mean, so it was it's due to uh I guess [49:42] Claude Fable um prompted by Levent Poge [49:44] and um right [49:46] >> a collaborator whose name I [49:47] unfortunately forget. Maybe you can [49:49] >> add it in post. Um [49:50] >> I just know the Twitter handle. [49:52] >> Yeah, we can. [49:53] >> So yeah, I mean uh so a lot of these [49:55] nice recent results have have come from [49:57] Levent with unclear amounts of autonomy. [49:59] Um so my sense is that some of them are [50:02] semi-autonomous rather than fully [50:03] autonomous. [50:05] >> Um yeah with this we don't know anything [50:07] about the methods. Uh so yeah this is a [50:09] fun construction. Um without knowing [50:12] about the methods it's very hard to say [50:13] how significant it is. [50:14] >> Yeah. [50:15] >> What I would say is that if you want to [50:17] understand um kind of historically such [50:21] results have been proved or sorry have [50:23] been have been understood by the [50:25] community they're like cool but I [50:27] wouldn't say they're like a big deal. Um [50:29] so like the typical place a result like [50:31] this might go is like someone's website [50:32] of records [50:34] >> like not [50:35] >> it's not an anals level result. [50:36] >> Yeah it's not an but that said it's cool [50:38] and it's like a you know it definitely [50:40] there were a few very very are a few [50:43] very very talented mathematicians who [50:44] like these kinds of questions and no [50:45] males being maybe the most famous [50:47] example. So Alis and Clagun are the ones [50:50] who kind of have been pushing this [50:51] record for a while and they recently [50:53] found a rank 29 [50:54] >> example which was you know the previous [50:57] record. [50:57] >> Yeah. you know, so those are [50:59] mathematicians. Um, and I think, you [51:01] know, people like it and it's cool now [51:02] that the models can do this sort of [51:03] thing. Um, but yeah, it's very, you [51:06] know, one thing I always say about a [51:08] model result is like you cannot evaluate [51:10] it except in retrospect. And like this [51:11] is also true of human mathematics. Like [51:13] >> sometimes a problem we thought was [51:14] really important or would require really [51:16] deep new ideas does not [51:18] >> and sometimes it does and you can't [51:19] really know in advance. So it's always [51:21] exciting when a problem like that gets [51:23] solved, but then like to figure out how [51:24] significant it is, you still have to [51:26] look and like well Levent and Claude uh [51:29] and I think Ava Howell maybe is the [51:30] third collaborator. [51:31] >> Okay. [51:32] >> Have not yet told us how they did it. So [51:34] uh [51:34] >> have they been mostly more secretive? I [51:36] I think they've released some traces for [51:38] stuff. But [51:39] >> yeah, so for this one, I think they [51:41] haven't yet, unless I missed it. Yeah, [51:42] they they they you know, Levent likes to [51:45] tweet out his uh his results, but yeah, [51:48] he has been, you know, slowly releasing [51:49] some kind of PDF writeups, too. I think [51:51] it's, you know, he's just having fun on [51:53] on uh fun on the internet. [51:55] >> Yeah. Yeah. [51:56] >> Um it's it reminds me of when you're [51:59] saying, you know, you have to evaluate [52:01] how the results came. Um so um I had um [52:04] Mark Zel Zelki and um uh Metab Swani on [52:09] from OpenAI recently and um they were [52:12] saying how you know what's kind of been [52:13] the most charming or delightful is just [52:16] that the proofs have been relatively [52:17] short. They're not like 200page and you [52:19] know maybe corresponding to your [52:21] grinding point as well. Um, but maybe [52:23] this is kind of optimized for in [52:26] retrospect [52:28] it was picked that it was short or maybe [52:30] do you find that on average uh stuff [52:33] that you throw at um GPT, you know, Saul [52:36] or Fable tends to be shorter and more of [52:40] legible to the human uh or versus it [52:42] might just go haywire and just grind it [52:45] out. [52:45] >> Yeah. So, so I mean I I think it is nice [52:48] that they'll sometimes produce short [52:49] clever proofs. Yeah. Um so that's of [52:51] course everyone likes a short clutter [52:53] proof. Um I think my sense is that the [52:57] uh reason they're not producing long [52:59] complicated proofs is that they cannot [53:02] >> um like that just like that the ability [53:04] to check correctness is not yet there. [53:07] Um so uh you know I mean even actually [53:11] you know if you ask the models to [53:12] produce uh [53:14] >> short proof you you can then often ask [53:16] them to you just ask is that correct and [53:18] they will often say no. [53:20] >> So like they're much more reliable than [53:22] they were six months ago for sure but [53:23] like it's still uh you know they will [53:26] still sometimes produce things that are [53:27] just wrong and they know they're wrong. [53:29] >> Yes. [53:29] >> Um I think the problem with producing a [53:31] very long thing is they might not know [53:32] they're wrong. Um, and so what I wonder [53:35] if uh, you know, presumably internally [53:38] OpenAI and and Anthropic have probably [53:40] solved a lot more problems than they've [53:42] released. [53:42] >> And I imagine quite a few of them are [53:44] they're just not sure if they're true. [53:46] >> Um, so for example, uh, you know, with [53:49] these this recent list of 10 problems [53:51] uh, released by OpenAI, those were all [53:53] formalized in lean, [53:54] >> which is of course a very good good [53:56] evidence that they're true. I have no [53:58] doubt that there were a lot more that [53:59] they could not formalize in late like [54:01] because the prerequisite options have [54:03] not been put into mass yet for example. [54:05] >> Um there were probably more that were [54:07] longer which also makes it challenging [54:09] to check. [54:09] >> Y [54:10] >> um so yeah this is my guess and we do [54:12] actually see very long generated um [54:15] proofs on the archive. Um so for example [54:18] someone recently posted a claimed proof [54:21] of resolution of singularities and [54:22] positive characteristic which was 800 AI [54:25] generated pages. Um it's like definitely [54:27] I mean I'm sorry I haven't read it I [54:28] haven't got an error but there's no way [54:29] it's correct like this will be a [54:31] >> major result. It's just not within the [54:33] capacities of the current models if [54:35] you're reasonably well calibrated. [54:36] >> Yeah. Yeah. [54:37] >> Um and it's definitely no human has read [54:39] it. Definitely the models are not able [54:41] to check this kind of thing. Yeah. [54:43] >> Um so well I mean um yeah. So this is my [54:47] my expectation is that the reason it's [54:49] producing short clever things is just [54:50] like that's what we can check. [54:52] >> Yep. and you can get it to produce long [54:53] things that are grindy or hard to check. [54:55] But then [54:56] >> yeah, yeah, we'll have to get there. [54:58] This is actually more what it says about [54:59] capabilities, frontier capabilities, [55:01] rather than in in a perhaps more [55:04] negative way rather than, hey, it's [55:05] just, you know, so good at these clever. [55:07] No, I I think that's totally fair. I [55:08] mean, especially if you look at an [55:10] adjacent domain like code, right? Um [55:12] remember reading something um that [55:14] cursor put out about testing their long [55:16] horizon um you know, harness. Um in this [55:19] case they're trying to reproduce SQL [55:21] light in Rust and it was just so telling [55:24] like how far we are from you know and [55:26] that and that seems like a very you know [55:28] comparable task of like it just it's [55:30] very long you have to make sure there [55:31] there's something to verify that it's a [55:33] correct implementation by testing a [55:35] suite of you know cases um but [55:38] you know you actually need a harness [55:40] there in this case it's not just like [55:42] the raw models and uh it takes a while [55:44] and you can actually compare with [55:46] different frontier versus um you know [55:49] non-frontier models, who's the planner, [55:51] etc. like differences in in their [55:54] capabilities as well. [55:55] >> Yeah. I also think in practice like the [55:58] uh to elicit a long proof, you kind of [56:00] need a harness. [56:01] >> Yeah. [56:01] >> And um when you make a harness whose [56:04] goal is to elicit a proof, I think it [56:06] often decreases reliability because [56:07] you're just trying to produce output. [56:09] >> Um so like TEDGPD 5.6 6 Pro is like very [56:12] it like really tries not to say wrong [56:14] stuff for example and although you know [56:17] it happens and then you'll ask it like [56:18] oh was that correct and it'll say no. Um [56:21] but uh you know when you're trying to [56:23] get it to ideulate like you try to get [56:25] it to be creative. You try to get it out [56:26] of this very like um you know uh [56:30] rigorous rut in order to like you know [56:33] actually get it somewhere. And I think [56:34] if your interest is in I mean there are [56:36] a lot of people who are uh you know [56:38] trying to arbitrage the prestige [56:40] mechanics of academic mathematics and [56:41] trying to elicit a lot of proofs [56:43] >> which are not necessarily being checked [56:44] and in order to do that I think you just [56:46] decrease the reliability in order to get [56:48] a lot of stuff. Um, so you know, I think [56:51] any harness that can elicit a 250 page [56:53] paper is probably not being very careful [56:55] about what it's producing. [56:56] >> And and you're saying that it's [56:57] decreasing or it's not reliable just [57:00] because it's the capability isn't there [57:01] yet and so it's just kind of forcing a [57:03] long longer horizon task on it. [57:05] >> Yeah. [57:06] >> Yeah. I mean in practice like how does [57:07] like a human checking it like can't [57:09] check a 250 page paper either like like [57:11] you you you know you can't reliably [57:13] check it line by line. What you try to [57:15] do to understand it is you try to like [57:17] understand the overall global structure [57:18] of the argument and you like stress test [57:20] it in various ways like like would this [57:22] argument apply something else that I [57:24] know to be false like oh like does it [57:26] work in this special case blah blah blah [57:28] and also the models seem not to be able [57:30] to do that kind of like more [57:32] >> I don't know fuzzy like unit testing a [57:33] proof very well yet. So actually like [57:36] one of my favorite tests for the models [57:37] which they haven't haven't succeeded at [57:39] yet is there's a paper um I won't name [57:41] it that came out a couple years ago uh [57:44] that was wrong um and it was like very [57:46] hard to find the like in close enough [57:49] area to myself that I like immediately [57:51] like was claiming some big result like I [57:52] immediately downloaded and like started [57:55] reading it and it was like very hard to [57:56] find the actual specific error [57:58] >> but it was also very clear from the [58:00] structure of the argument that it [58:01] couldn't work. So it was like, you know, [58:03] me and a bunch of other arguing [58:05] something that was too strong to be true [58:07] if you kind of took that a little bit [58:08] further. [58:09] >> Yeah. [58:09] >> Um so me and a bunch of other experts [58:11] like immediately like realized it was [58:13] wrong and like we you know we emailed [58:15] the the author and like we kind of went [58:17] back and forth until someone um figured [58:19] out what the precise specific error was. [58:21] >> Okay. [58:22] >> Um and so far the models seem not to [58:24] have been able to do this. like the [58:25] specific error is quite subtle and but [58:27] this they're also not able to do this [58:29] kind of overall uh kind of big picture [58:32] tax checking which is kind of like how [58:34] people in practice check papers. [58:36] >> Yeah. Yeah. Which is kind of it mirrors [58:38] you know why in code it's like it's it's [58:40] so clear it's good at the syntax but you [58:42] know higher level architectural stuff [58:45] still very weak. Maybe it'll ascend [58:46] there but probably need some harness [58:48] help. Who knows? I mean, people have, [58:50] you know, evolving opinions on how much [58:52] the harness and the model have to [58:53] co-evolve and which one is necessary, [58:56] but the next model requires less. So, I [58:57] I feel like in in math, it it'd be very [58:59] interesting to see how you um uh if you [59:02] do any experiments with the harness [59:03] there and how how that improves um [59:06] because it is a mark of general [59:07] reasoning. Yeah. Yeah. [59:08] >> Yeah. I mean I I do you know I have my [59:10] own sort of bad little harness and [59:12] codeex and codecs but yeah it's um but [59:15] you know I personally I do not enjoy uh [59:18] autonomous mathematics very much so I [59:20] mostly mostly do not use the harness I [59:22] mostly try to use it to help me [59:24] understand stuff. [59:25] >> Okay. No it's totally fair. Yeah. [59:27] Exactly. You don't want to automate your [59:28] job away because that does involve you [59:30] being in the loop to understand it which [59:32] is you know necessary to participate. Um [59:34] maybe to to finish off, I'd love to and [59:37] this this could be just something you [59:39] haven't thought about or actually [59:40] thought a lot about. Um I think you also [59:41] have a toddler, [59:43] >> right? Yeah. [59:44] >> Yeah. Yeah. And so how have you a [59:45] three-year-old? Yeah. [59:46] >> Yeah. Three-y old. Great. Right. So [59:47] you're one year more advanced and [59:49] probably you know have more thoughts on [59:51] this. Like how are you thinking about um [59:53] is it her education or his education? [59:55] >> Her. Yeah. How are you thinking about [59:56] her education in in math and [59:59] >> you know not to grind but you know [60:01] really just like how you know pass on [60:03] the love of it and and how to how to [60:04] react to AI. [60:06] >> Yeah. So she's she's three. She's she's [60:08] never used uh AI. Um she she is starting [60:11] to add that's that's about as far as we [60:13] are in math. [60:14] >> That's that's more than she can add [60:16] single single digit numbers like you [60:18] know by counting on her fingers and [60:20] >> count up to like maybe 30 reliably and [60:23] 50 semi-reliably. So I'm very proud of [60:25] that. [60:25] >> That's good. Yeah. [60:25] >> Yeah, I definitely encourage that. We [60:27] talk about shapes and stuff. Um, uh, la [60:31] actually a couple days ago I woke her up [60:34] and she was like hiding under the [60:35] blankets [60:36] >> and I was like, "Oh, what are you up to [60:37] under there, Sophia?" And she was, "Oh, [60:38] I'm doing some math." [60:39] >> Oh, I did. I think you tweeted about [60:42] that. That was horrible. [60:43] >> Yeah, it was great. So, you know, I [60:44] think she has like some sense that I [60:46] like math and she's into it because of [60:47] that. [60:48] >> Um, yeah. I mean, I I would say I uh [60:51] don't know, you know, I think the world [60:52] is probably going to look pretty [60:53] different in, you know, 20 years or [60:56] whenever she's kind of uh you know, [60:59] fully adult and and doing her own thing. [61:02] Um but you know, I think a lot of what [61:03] we educate people for is like pretty [61:05] robust changes in the nature of the [61:07] world. like uh I think the reason to [61:10] learn math has always been like to think [61:12] clearly and like better understand the [61:13] world and like presumably that's [61:15] something you want to do even if uh [61:17] there are sort of extremely capable AIs [61:19] and this is also true you know I I [61:21] personally like math a lot but also the [61:22] reason to like read a lot of books and [61:24] do the humanities and so on. So I I mean [61:26] I think like the actual values of uh the [61:29] math profession and like education more [61:32] broadly are like things we definitely [61:34] try to want to try to preserve like that [61:36] I I hope to instill in in in my [61:39] daughter. Um you know how much [61:42] institutions have to change to make sure [61:44] that happens is is maybe an open [61:45] question I think a lot. Um but yeah I [61:48] mean at least at a personal level I'm [61:50] I'm definitely trying to you know uh [61:53] convince my my three-year-old that math [61:54] is super cool. Oh yeah, 100%. [61:56] >> One of her first words was icosahedron. [61:58] So [61:58] >> was was what? [62:00] >> She she has a she my my parents gave her [62:02] a little iicosahedron toy [62:05] >> uh when she was one. She she [62:07] >> Yeah. Not literally a first word, but [62:09] she learned the platonic solids quite [62:10] early. [62:10] >> Oh, very good. Very good. Very fun. [62:12] >> Next up, group theory. I mean, it's like [62:14] very natural. That's right. Yeah. I [62:15] mean, it's [62:16] >> Yeah. I'm actually when I when I teach [62:18] her about uh uh addition and subtraction [62:20] further, we'll we'll do it in the [62:21] context of a general group. [62:23] >> Oh, very good. [62:25] Well, at least you give the motivation. [62:27] I think a lot of probably skip that [62:30] part. Um, yeah. Yeah. And you know, [62:32] maybe this is how uh math grad students [62:35] can also, you know, focus like training [62:37] the next much younger generation to use [62:39] AI in in service of actually getting [62:41] better at math rather than [62:42] >> just lacking understanding. [62:44] >> Wonderful. Okay. Well, thank you so [62:46] much, Daniel. Um, [62:47] >> thank you. It was a lot of fun. [62:48] >> Yeah. Yeah, it was a lot of fun. And [62:50] yeah, I mean, I think there's going to [62:52] be a lot more progress very soon. I'd [62:53] love to maybe catch up and chat again. [62:56] >> Sounds great.