Mathematician Daniel Litt Warns AI Paper Floods Threaten Research

数学家Daniel Litt警告AI论文泛滥或危及学术研究 尽管前沿模型在强力计算方面表现出色，但其缺乏构建概念理论或自我验证长证明的能力，可能导致重复性论文充斥学术界，从而削弱人类的理解力。 Daniel Litt与Lisha Li深入探讨了人工智能在数学领域的前沿发展，将模型的强力计算能力与人类对概念理解和理论构建的需求进行了对比。Litt重点提到了诸如自主发现埃尔德什单位距离问题变体的反例等真正的突破，但也解释了为何模型仍难以针对根深蒂固的真实猜想构想出全新的框架。他们探讨了学术激励机制如何受到低质量AI生成论文工坊的威胁，以及人类的认知局限性为何实际上构成了抽象发现的重要催化剂。最终，他们指出，即便进入超越人类水平的AI时代，保留人类人才梯队并传授严谨的数学推理，对于个体的好奇心以及社会的主观能动性而言依然至关重要。 Daniel Litt强调，针对埃尔德什单位距离问题某一变体由AI自主发现的反例是一项非凡突破，因为它是创造性地跨学科引入了20世纪60年代的技术，而非仅仅执行意料之中的渐进式步骤。 (3:00) 前沿模型主要在技术计算、文献综合以及寻找反例方面表现出色，但在理论构建、直觉形成和提出恰当问题上依然面临困难。 (10:00) 学术出版激励机制目前正受到AI的扭曲，这使研究人员能够快速炮制在数学上正确但价值较低的论文，导致预印本服务器上同时出现完全相同的证明。 (34:00) 当前AI生成的证明之所以简短，是源于其架构上的推理极限，而非出于刻意的审美克制；在缺乏外部验证的情况下，更长的推导会迅速偏离轨道。 (50:00)

Mathematician Daniel Litt argues that the true objective of mathematics is cultivating human comprehension and conceptual theory rather than maximizing paper outputs or offloading proofs to opaque automated systems. Conceptual Understanding Versus Automated Output (0:00) Litt contends that mathematical progress relies on building intuition, recognizing analogies, and formulating meaningful questions, rather than grinding through mechanical symbol shuffling. While frontier models can successfully complete complex computations or stitch together far-flung technical methods, they struggle with big-picture taste and deep theory building. When Litt initially attempted to prove a target lemma using frontier models, they stalled completely. Only after he worked through concrete manual examples to clarify why the statement held—and supplied a better-formulated statement—did the AI prove it rapidly. Litt warns that had an advanced model simply brute-forced the original lemma, it likely would have generated an unilluminating, ten-page calculation that generated zero conceptual insight. Recognizable Reasoning and the Counterexample Advantage (3:00) Examining OpenAI's public chains of thought, Litt observes that AI reasoning resembles informal, human exploratory scratchwork rather than alien maneuvers like AlphaGo's 'Move 37.' Models currently excel at counterexample discovery, as shown in mid-May when an AI solved a variant of the Erdős unit distance problem by importing 1960s techniques into planar point configurations to disprove an established conjecture. However, Litt notes that deep conjectures like the Grothendieck–Katz p-curvature conjecture or the Riemann hypothesis are deeply integrated into theoretical frameworks and are almost certainly true. Because these statements cannot be refuted by searching for isolated counterexamples, solving them demands novel theoretical machinery—an area where reinforcement learning struggles due to the lack of clear intermediate reward signals for developing fuzzy conceptual understanding. Verification Bottlenecks and Proof Lengths (50:00) Although observers have celebrated the legibility and brevity of recent AI proofs, Litt argues this brevity reflects current model limitations: architectures produce short proofs because they cannot maintain deductive stability across long derivations without derailing. While models occasionally self-correct simple arguments when prompted, extended proofs result in severe verification failures. Daniel highlights an extreme case on arXiv: an unverified 800-page AI-generated preprint purporting to resolve singularities in positive characteristic, which he considers certainly erroneous. Unlike human mathematicians, who quickly detect subtle flaws through macro-level sanity checks, edge-case testing, and inspecting global proof architecture, models entirely lack the intuition to perform these structural smell tests. Academic Distortions, Mode Collapse, and Human Agency (34:00) Current academic hiring structures reward raw paper volume over genuine insight, tempting researchers to turn language models into paper mills. Litt demonstrated this dynamic by using Codex to generate three mathematically correct but low-quality algebraic geometry papers within an hour; similar dynamics have already led to multiple preprints sharing identical proofs appearing on arXiv within days. Litt warns that this convergence indicates mode collapse, risking the replication of a single reasoning perspective rather than fostering cognitive diversity across millions of thinkers. Even in a hypothetical era of superhuman AI systems, Litt and host Lisha Li emphasize that humans must remain active directors of research and education to retain societal agency and preserve independent intellectual inquiry. While AI tools offer immediate utility as coding engines and technical sounding boards, resolving major mathematical frontiers remains constrained by verification bottlenecks and the challenge of teaching models to develop conceptual theories. Preserving broad human education and diverse research incentives remains an unresolved institutional challenge as automated output becomes increasingly cheap.

数学家 Daniel Litt 认为，数学的终极目标是构建人类对概念的深度理解而非单纯发表论文；当前 AI 擅长繁琐计算、文献拼接与反例搜索，但在概念直觉、宏观理论构建及长程验证上仍高度依赖人类的引导。 理解导向与 AI 的推理现实 (0:00) 数学的核心价值在于产生认知与理解，若理解仅停留在不透明的模型权重中，在智力层面上并不令人满意。公众常误以为数学仅是按照形式规则进行的符号推演，或者认为大模型的推理表现出类似 AlphaGo“第 37 步”的外星逻辑。然而观察 OpenAI 等前沿实验室公开的思维链（Chain of Thought）可以发现，模型的思考过程更接近人类数学家在草稿纸上探索时的非形式化自然语言思考，展现出高度可识读的人类特征。各头部实验室如 OpenAI 与 Anthropic 在该领域解决的数学问题类别高度趋同，本质上捕捉的是同一类人类数学行为片段。 在具体研究实践中，当前前沿模型自主解决深度推理仍很吃力，但若有人类提供提示与脚手架则进展显著。Litt 举例指出，前沿模型最初无法证明一个特定引理；直到他亲自手算大量具体例证、提炼出底层机制并给出表述更优的命题后，模型才迅速完成了证明。这种人机协作凸显了现状：模型具备极强的既有技术执行力与跨文献综述能力，但在提出精确命题与洞察核心机制方面，依然离不开人类的理论直觉。 反例构造突破与理论构建困境 (3:00) 在 Litt 看来，迄今最令人印象深刻的自主 AI 数学成果是 5 月中旬公布的 Erdős 单位距离问题变体。此前许多 AI 里程碑往往属于“最后一公里”的工作，即人类完成核心概念构建后由模型做渐进式收尾；而 Erdős 反例的特殊之处在于，学术界此前普遍预期该猜想为真，但 AI 跨领域引入了 1960 年代的经典技巧构造出反例，随后人类数学家迅速借鉴其核心思路攻克了包括实数上和积猜想变体在内的多个悬而未决的问题。这种成果在事后推动新理解的延展性，被 Litt 视为衡量数学工作价值的金标准。 然而，这种反例构造的成功很难直接复制到主流的宏观猜想上。诸如 Grothendieck–Katz p-曲率猜想或黎曼猜想等前沿问题，深嵌在庞大且互相关联的理论体系中，现有大量间接证据表明它们极大概率为真，因此无法通过暴力离散搜索反例来解决，必须依靠全新的理论体系与机械工具。在强化学习（RL）框架下，证明定理或找到反例能够获得明确的二元奖励信号，但对于“加深对黎曼 Zeta 函数的理解”这种模糊探索，强化学习极难设计稠密且经过校准的中间奖励，导致模型在缺乏人类渐进引导时无法自主推进长周期的理论构建。 人类认知局限与概念压缩的价值 (30:00) 人类数学的发展往往得益于认知局限。人类缺乏无限的工作记忆和暴力计算耐力，这种“无法长期生硬硬算”的弱点反而倒逼学者创造高度压缩的概念工具、抽象机制与结构定理。Litt 自述在面对引理时虽然明知可以通过长达十页的繁琐计算暴力通关，但由于对这种耗时且毫无智力启发的过程产生抗拒，从而促使自己寻找更干净的概念论证。人类之所以使用非形式化的自然语言书写数学而非展开为冗长的 ZFC 集合论符号串，本质上是为了追求认知压缩，而这种压缩产生的理论往往能够奇迹般地启发相邻领域。 相比之下，拥有无尽算力与耐力的 AI 模型若仅靠蛮力推演，不仅无法产生新的认知启发，还存在严重的模式坍塌（Mode Collapse）风险。历史上的数学突破依赖于让“千朵花齐放”的认知多样性，学者追随各自独特的哲学直觉在不同方向拓宽知识边界。如果研究过度依赖模型偏好，可能只是将同一个数学家复制了一千次，而非形成百万种不同的独特视角。如果模型不断收敛于狭隘的同质化推理轨迹，将对依赖人类文献滋养的前沿 AI 构成根基性的多样性挑战。 学术激励扭曲与长程验证瓶颈 (34:00) AI 工具的高速普及对学术界形成了双向冲击。在职业晋升和发表压力下，学术界出现了将大模型当作论文生成器的扭曲激励。Litt 曾通过实验在短时间内利用 Codex 生成三篇数学正确但价值低劣的代数几何论文；预印本网站 arXiv 上也已出现多篇由不同学者使用类似提示词生成的完全相同证明的论文。与此同时，大模型生成的长篇证明正面临严峻的验证困境。例如曾有未经检验的 800 页 AI 预印本声称解决了正特征下的奇点消解问题，Litt 结合现有模型能力断定其必然存在错误。 目前的验证瓶颈在于，大模型能够由 Lean 等形式化工具验证的领域极其有限，且模型极度缺乏人类专家的“宏观健全性检查（Macro-level sanity checks）”。人类审稿人可以通过检查全局架构、边缘特例或推演是否导出已知谬论来快速评估真伪，而当前模型在长程推演中极易迷失并产生幻觉。类似 Levent Alpoge 与 Claude 3.5 Sonnet 合作发现 30 阶椭圆曲线等记录刷新，固然展现了计算威力，但评估此类成果是否构成范式转变，仍取决于后续完全公开的算法细节与人工核验。 超级智能时代的人类能动性与教育基石 (40:00) 面对 AI 展现的算力与自动化前景，Litt 坚持认为即便未来出现强超人类 AI，社会依然绝对需要人类数学家。将研究方向全权委托给自主系统会带来方向性风险，因为模型可能会寻找最直接的优化捷径，从而忽略源于人类好奇心的探索支线。为了维持社会的能动性与掌控力，必须保留具备严密智力探究能力的人类群体。数学思维在此扮演了训练自律精神与清晰心智的通用基石，其价值在机器时代不会被抵消。 这种对底层能力的要求在教育领域尤为紧迫。当前课堂已出现显著的两极分化：一部分学生借助 AI 加深理解，另一部分学生则将其作为外包作业的工具导致基础技能萎缩。Litt 认为消除这种低质量产出需要制度上的重新设计，他在抚养三岁女儿的过程中观察到，早期数学教育的精髓在于建立对抽象规律与问题解决的自发热情，而非机械记忆与重复运算。无论技术如何演变，通过数学学习清晰思考与理解世界，始终具备不可替代的内在价值。 当前 AI 在辅助计算与局部探索上已展现出极高价值，但其无法脱离人类赋予的宏观直觉与验证把关；未来的核心未决议题在于，学术体制能否抵御低质论文膨胀的激励诱惑，以及强化学习能否在缺乏简单指标的情况下学会真正的概念构建。

## Background 本期播客由 a16z 投资人 Lisha Li 与多伦多大学数学助理教授 Daniel Litt 于 2026 年 9 月在 RSS 及各大播客平台发布。节目主要探讨了当前前沿 AI 大模型在数学研究中的最新突破与实际能力，并从一线数学家的视角剖析了 AI 在直觉构建、理论体系搭建以及人类思维方式上的核心差距与潜在冲击。 ## TL;DR 对话深入拆解了 AI 解决埃尔德什单位距离问题等前沿数学成果背后的机制，指出当前模型长于跨领域搜寻反例和海量机械计算，但在构建全局理论、提出深刻问题与宏观直觉上依然存在明显短板。Daniel 结合自身代数几何研究与日常工作流，展示了 AI 作为计算助手的高效性以及人类受限于算力反而倒逼出抽象理论的认知奇迹。针对学术界利用 AI“摇奖机式”刷低质论文导致的模式坍缩与学术评价扭曲，两人强调数学的终极目标是人类心智的理解而非论文产出，呼吁在 AI 时代更应坚守数学作为深度思考训练的教育内核。 ## Atmosphere & style 整场对话洋溢着严谨、深邃且充满理趣的智识交锋，既有一线纯数学家面对技术巨浪时的客观冷静，又穿插着对学术功利主义的辛辣讽刺与对数学本质的哲学沉思。Lisha Li 的精准提问与 Daniel Litt 坦率生动的比喻相得益彰，从硬核的代数几何猜想到女儿抓周般的几何启蒙，氛围在高度专业探讨与温暖的人文情怀之间自如切换，带给听众一种既审慎清醒又备受鼓舞的思维沉浸感。 ## Bulletin - `05:00` 前沿大语言模型在数学上的突破主要依托自然语言的非形式化推理能力扩展，而非形式化验证语言（如 Lean）的大规模训练，这表明相关推理技术有望高度泛化至非数学的现实领域。 - `25:00` 大模型在庞大离散空间中寻找反例构造（如攻破埃尔德什单位距离问题）表现强悍，但因缺乏明确的中间过程奖励机制，在凭空构建全新理论体系以证明深层真命题方面仍面临严重瓶颈。 - `35:00` 学术界出现将 AI 当作“摇奖机”刷论文的现象，不仅催生大量缺乏洞见的平庸成果，多篇预印本撞车证明路径更暴露了模型高度依赖同质文献的“模式坍缩”风险。 - `49:17` 针对近期宣称突破的代数秩为 30 的椭圆曲线等成果，顶级学者指出目前多为半自主的人机协作，尚无充分证据表明大模型能在无专家深度干预下独立完成长程严肃论证。 ## Key takeaways - `00:00` 数学的核心目标是产生深层次的理解而非单纯量产论文，如果这种理解仅停留在模型权重中，对人类数学家而言毫无意义。 - `00:00` AI 解决埃尔德什单位距离问题之所以震撼学界，是因为它不仅构造出反常识的反例，还跨学科借鉴了 1960 年代的经典思想并启发了实数和积猜想等后续突破。 - `05:00` 模型的思维链推理本质上是在模拟人类数学家的非形式化心路历程，其优势在于超常的计算耐力和知识整合能力，而非某种非人类的“第37手”。 - `10:00` 当前大模型在纯算力推演和跨文献构想整合上表现优异，但极度缺乏数学家赖以开创新方向的哲学式宏观直觉与理论构建能力。 - `13:30` 数学公开难题的本质价值在于作为标尺衡量人类现存认知的盲区，并倒逼学者将局部获胜工具系统化升华为宏观理论。 - `16:30` 大模型对探索缺乏明确表述的哲学性长期项目收效甚微，但作为不知疲倦的编程伙伴，它彻底盘活了并发运算和寻找极端反例的实验数学工作流。 - `20:00` 纯数学研究更接近“以概念为研究对象的物理学”，年轻学者若过度受限于形式优美而回避复杂硬算，反而会阻碍认知突破。 - `28:30` 由于人类大脑生理上“无法无限硬算”，这种算力瓶颈反倒迫使人类施加信息压缩压力，提炼出具有广泛洞见的结构与概念框架。 - `30:00` 面对困难引理，大模型倾向于输出缺乏直觉的十页暴力计算，而人类则被迫提炼出更精炼优美的命题表述，从而催生更深刻的解释。 - `35:00` 现有学术评价机制正诱导科研人员用 AI 批量生成正确但平庸的论文，AI 推理的模式坍缩可能使研究退化为“把同一个数学家复制一千次”。 - `40:00` 即便未来大模型在推理上达到超人水准，社会依然需要维系多元化的人类思考者社群，以确保知识探索的方向与人类核心价值对齐。 - `45:00` AI 在教育与研究中引发了双峰分化：一部分人借助工具深化理解攀登前沿，另一部分人因彻底外包思考而导致心智退化。 - `55:00` 当前模型缺乏生成和验证长篇复杂证明的能力，强行通过长程框架（Harness）催生数百页论文往往以牺牲严谨性和准确性为代价。 - `55:00` 人类数学家审阅论文依赖对全局架构的“模糊单元测试”和反常推论筛查，而当前模型在宏观逻辑把关和定位深层细微错误上依然无能为力。 - `01:00:00` 在 AI 时代，学习数学与人文学科的根本初衷并未改变，其核心依然是培养人类清晰思考、理解现实世界的能力。 ## Detailed walkthrough ### `00:00` 数学的本质与 AI 解决埃尔德什单位距离问题 对话以对数学研究终极目的的反思拉开序幕。多伦多大学数学助理教授 Daniel Litt 明确指出，数学的目的绝非仅仅是为了量产论文，而是为了获得某种深层次的理解；如果这种“理解”最终只停留在模型的权重之中，对他来说是相当令人不满的。随着前沿大语言模型在数学领域频频传出突破性进展，主持人和 a16z 投资人 Lisha Li 提出，外界关注的焦点大多停留在模型能力飞跃的头条新闻上，但更值得探讨的是在职数学家究竟该如何面对这种冲击。数学之所以特殊，不仅在于证明定理，更在于提出好的问题、建立全新的理论框架以及形成大局观与直觉。 谈及迄今为止最令人震撼的 AI 数学成果，Daniel Litt 强调，目前已有许多全自动、半自动或 AI 贡献占比模糊的成果涌现，由于数学分支众多，任何评判都受限于个人专长。但在所有全自动成果中，他最青睐的是 5 月中旬公布的关于埃尔德什单位距离问题（Erdős unit distance problem）的解答。以往许多 AI 成果更偏向于“最后一公里”（last mile）的突破，即人类数学家团队已经完成了绝大部分深层次铺垫，AI 仅仅补上临门一脚。然而，AI 在处理该问题时却展现出了非同寻常的“创造力”：这一领域内的学者普遍认为原命题成立，AI 却出人意料地构造出了一个反例；不仅如此，AI 还跨领域借用了 1960 年代的经典数学思想来解决平面点构型问题，打破了学科壁垒。更可贵的是，这一成果随后激发了人类学者的后续研究，多位数学家借鉴其思路，成功找到了实数上的和积猜想（sum-product conjecture over the real numbers）等一系列重要开放问题的反例。 ```mermaid flowchart TD accTitle: AI解决数学问题的两种模式对比 accDescr: 对比最后一公里模式与具有创造力跨领域模式的差异及其后续影响 subgraph LastMile ["模式 A：最后一公里突破"] A1["人类数学家团队深耕"] --> A2["完成绝大部分深层证明"] A2 --> A3["AI 补足最终证明步骤"] end subgraph CreativeBreakthrough ["模式 B：创造性跨领域突破（埃尔德什问题）"] B1["学界主流预期命题为真"] --> B2["AI 引入 1960 年代其他领域经典思想"] B2 --> B3["自主构造出意外反例"] B3 --> B4["激发后续研究：解决实数和积猜想等反例"] end ``` - Daniel Litt 直言数学研究的核心价值所在：“The goal of mathematics is not to produce mathematics papers, it's to produce some kind of understanding.” - 评价埃尔德什问题的解答时，Daniel Litt 认为其最具价值之处在于其延展性与启发性：“So that's at least the way I like to think about how cool Salt is. Like you look at a post-hoc and you see like, oh, well, we're whatever new ideas that we're in front of if any, kind of useful to do other things, did they improve our understanding of something?” ### `05:00` 非人类奇迹还是人类思维镜像：自然语言推理与模型表现 外界常有一种刻板印象，认为数学不过是遵循特定规则、逻辑严密地“摆弄符号”，并由此推论强化学习（RL）之所以在数学上大获成功，是因为数学形式化后易于低成本验证且规则明晰。但 Lisha Li 指出，这完全背离了数学探索的真谛——数学是一场与困惑共处、打磨直觉并逐步逼近真理的认知旅程。针对主持人将 AI 的证明称为“非人类壮举”（inhuman feats）的说法，Daniel Litt 提出了截然不同的观察。他回顾了 OpenAI 公布的模型思维链（Chain of Thought），认为其推理过程完全是在模拟人类数学家解题时的心路历程，虽然可能经过了官方的清洗和润色（去除了可能存在的冗余甚至粗口），但整体逻辑框架人类学者完全能够读懂并产生共鸣。AI 真正超越人类的地方并不在于某种难以理解的“第37手”（Move 37），而在于它不知疲倦的计算耐力以及海量知识储备。 进一步探讨模型的底层机理时，两人都注意到一个极为关键的趋势：当前顶尖前沿模型（无论是 OpenAI 的系列模型还是 Anthropic 的 Claude）的推理飞跃，主要依托于自然语言（Natural Language）的非形式化推理能力扩展，而非形式化验证语言（如 Lean）的大规模训练。由于现存的数学论文与专著普遍存在极高的压缩率，只展示最终打磨精美的逻辑结论，几乎从不记录最初的动机、直觉走过的弯路以及思维轨迹，这一事实显得尤为惊人。Daniel Litt 借此推测，正是由于前沿大模型是在自然语言层面上拓展非形式化推理，这些在数学领域跑通的推理技术很可能会高度泛化到其他非数学的复杂现实领域。就模型差异而言，尽管用户可能会察觉细微手感不同，但 Anthropic 与 OpenAI 在数学问题解决上表现出高度同质的能力水准，且目前攻克的问题往往共享着某种特定模式，所覆盖的依然只是人类数学家全部研究活动中极小的一部分。 - 针对 AI 证明方式是否脱离人类思维的讨论，Daniel Litt 给出了清晰的定性：“If you just look at raw model output, it's not like there's some move 37 or whatever. It's like a human mathematician doing math.” - 针对数学模型训练不依赖 Lean 而是依靠自然语言的事实，Daniel Litt 做出了其对泛化能力的预判：“My sense is that because they're primarily scaling informal reasoning, probably the techniques are going to generalize to other domains pretty well.” ### `10:00` 大模型在数学研究中的优势与直觉短板 Daniel Litt 指出，当前前沿大模型展现出的能力特征完全在预料之中：它们长于完成极其冗长复杂的运算推导，或者从许多人类数学家可能未曾涉猎的不同领域与海量论文中抓取技术构想并加以整合。然而，在直觉构建或形成全局性视角（big-picture point of view）方面，模型却显得相当薄弱。Daniel 解释道，作为数学家，自己的大量工作源自某种极不严格的“哲学视角”——例如隐约觉得甲事物与乙事物之间存在某种类比，接下来的大部分工作则是弄清楚如何将这种感觉精确化，并通过能否解决具体问题或能否解释某种未解现象来衡量自己是否真正理解了它。 当前模型所展现的最高水平成果中，几乎看不到这种宏观哲学式的思考与推理。Daniel 评价道：“它们非常非常擅长应用某些已知技术，坦白讲，这本身就是一件极其强大的事；历史上很多数学家凭借这类高质量工作取得了非凡的学术生涯。”然而，这仅涵盖了数学家所关切领域的狭窄一环。即便如此，包括 Claude（如 Fable）和 ChatGPT 在内的前沿模型已开始展现出涉足模糊任务的微弱迹象。Daniel 尝试让它们参与理论构建（theory building），发现如果没有人类搭建的脚手架或提示，模型完全无法独立完成；但只要给予一定提示，它们便能给出有意思的反馈。不过他坦言，提示交互中很难界定多少是来自模型本身的创见，多少是人类的引导；但从趋势来看，模型正在逐步吸收那些未见于书本的隐性数学知识。 针对日常使用偏好，Lisha Li 提及外界普遍认为 Anthropic 与 OpenAI 旗鼓相当，并询问 Daniel 个人更常使用 ChatGPT 的原因。Daniel 幽默地承认这纯属“习惯使然”的惰性： > “对于我自己的研究工作，除了纯粹做测试对比时，大部分时间我就只认准一个模型，全凭随机习惯用下去。” —— Daniel Litt 两人还讨论了模型交互体验的差异。Lisha 提到在解释数学结果时，某些模型对用户背景认知（心智理论，Theory of Mind）的把握更准，而有的模型则会在基础概念上过多赘述或突兀跳跃。Daniel 对此表示，从他的视角来看，目前各家前沿大模型在心智理论方面“都表现得相当糟糕”。 --- ### `13:30` 数学家的谱系：解题者与理论构建者 在深入探讨“直觉”与“理论构建”前，Lisha 请 Daniel 结合自身从事的代数几何（algebraic geometry）领域，拆解数学家的工作常态及其在 AI 介入前后的变迁。Daniel 解释说，数学家可以划分为不同的类型谱系，最经典的分法莫过于“解题者”（problem solver）与“理论构建者”（theory builder）。他自认倾向于解题者，但强调公开问题的本质价值不在于解题本身，而在于它是一把测量认知盲区的标尺： > “至少对我而言，一个公开问题的意义在于它衡量了你在哪方面遭遇了认知失败。” —— Daniel Litt 以著名的格罗滕迪克-卡茨 p-曲率猜想（Grothendieck–Katz p-curvature conjecture）为例，它本质上衡量了人类对微分方程理解的匮乏程度：面对极其基础的对象，若无法解答这一猜想，就证明人类对其结构尚未参透。在实际研究中，数学家寻找此类问题的方法通常是：抓住最微小、最简单的未知情境不断探索，一旦攻克难关，便凝视为了获胜而创造出的工具，尝试将其升华为一套系统性的理论。 ```mermaid flowchart TD accTitle: Mathematical Discovery Loop accDescr: A loop showing how mathematicians find edge cases, build theories, identify analogies, and formulate conjectures. subgraph ProblemSolving["路径 A：从具体瓶颈到理论构建"] A["寻找最小未知情境"] --> B["推导并攻破具体瓶颈"] B --> C["凝视获胜工具，沉淀为系统性理论"] end subgraph AnalogyDriven["路径 B：从非严格类比到宏观纲领"] D["发现非严格跨领域类比"] --> E["建立性质映射表并形式化"] E --> F["衍生大量公开命题与全新数学分支"] end C --> G["提出精准的大猜想与认知标尺"] F --> G ``` 另一种核心工作模式则是追寻深刻的跨领域类比。Daniel 透露自己的大部分研究都受到“代数簇的上同调”（cohomology of algebraic varieties）与“基本群的表示”（representations of fundamental groups）之间类比关系的启发。这种类比具有极高的产出性——一端出现的任何几何/代数现象，几乎都能在另一端找到对应物。过去三四十年间，Carlos Simpson 与 Shinichi Mochizuki（望月新一）等杰出学者在此类比推动下发展出了极度优美的数学分支。这种研究的起点并非一个现成的公开问题，而是一种完全不严谨的宏观哲学；研究者的终极目标是将这种宏观哲学具象化，它绝非单纯的代码式推演或符号运算。 --- ### `16:30` 提出好问题与 AI 在日常研究中的真实分工 数学研究的另一大核心挑战在于明确“什么是正确的问题”。Daniel 强调，在网上搜寻已有的公开猜想并不能完全反映未知的全貌，因为提出一个富有洞察力的猜想本身往往极端困难。 - **伯奇与斯温纳顿-戴尔猜想（BSD 猜想）的诞生**：作为千禧年大奖难题之一，该猜想揭示了椭圆曲线的 L 函数与方程解群的秩之间的深层联系。这一猜想是数学史上最早依赖“大数据计算”发现的猜想之一：Bryan Birch 和 Peter Swinnerton-Dyer 在 1960 年代利用早期计算机统计了大量椭圆曲线数据，敏锐察觉到图表中某条切线的斜率与已知代数不变量高度吻合，从而提出了这一伟大猜想。 - **数学研究的“实验科学”属性**：数学家经常需要像实验科学家一样，运算海量具体算例，寻找反常现象，并为此现象推导理论解释。 当谈及目前日常研究中如何具体使用 AI 时，Daniel 坦言其影响呈现出极强的分化： > “问题越具有哲学意蕴，或者脑海中越缺乏精准具体的提问，AI 能起到的作用就越小。” —— Daniel Litt 对于他在大模型爆发前就已经构思三到五年的长期攻坚项目，AI 基本派不上深层用场，充其量只是传统 Google 搜索的升级替代品——用来快速了解某篇论文或交叉领域概念，省去了一些查阅时间，但并未分担真正的核心脑力工作。相反，AI 彻底激活了他的实验性与计算性项目：由于 Daniel 自嘲极度不擅长写代码，过去若遇到需要编写脚本验证算例的课题，他往往会拖延大半年，直到有精通编程的博士生加入协助；而现在，大模型犹如一个不知疲倦的编程伙伴，他可以指示模型通过多智能体并发调度，瞬间跑完一千个复杂算例，使他能够以前所未有的速度开展计算实验、搜寻极端反例。 ### `20:00` 数学研究的驱动力：科学探索还是诗意审美？ 对话从数学家长期攻关项目的性质展开。Lisha Li 提出，许多纯数学研究并不像应用数学那样拥有明确的外部物理世界或现实问题驱动，其研究方向往往显得近乎一种“社会学现象”——正如数学家威廉·瑟斯顿（William Thurston）在 1970 年代所指出的那样，当越来越多的数学家开始关注某一个课题时，学界便会逐渐汇聚到某些有趣的结构上。她向 Daniel Litt 提问：数学家选择研究课题究竟是出于追求“美”（beauty），还是其他动力？ Daniel Litt 直言，自己刻意避免被审美或追求“优美”的考量所驱动，并认为把“美”当成目标反而会限制研究者的思维。他观察到年轻数学家常出现一种挫败模式（failure mode）：明明知道如何证明一个结论，却因为证明过程感觉“非常丑陋”而裹足不前。当 Lisha 追问什么是“丑陋”时，Daniel 解释说，人们通常认为丑陋意味着充斥着大量枯燥、繁复且无法带来认知启发的纯计算（grinding calculation）。但他坚持实用主义态度：“在我看来，你应该不择手段地去赢（You should win by any means necessary）。”他更愿意把自己所做的工作视作“针对概念进行的物理学研究（doing physics except with concepts）”，其核心目标是考察某项工作是否引入了具有广泛功用的新思想，从而最大程度地打开更深层次的理解。 ```mermaid flowchart TD accTitle: 数学研究范式的路径对比 accDescr: 对比审美驱动与科学驱动两种不同的数学研究取向及其带来的结果。 subgraph Aesthetic["审美驱动取向（类似诗歌）"] A1["追求证明的优雅与形式美"] --> A2["回避繁复、机械的推导与计算"] A2 --> A3["易受心理偏好限制，限制探索范围"] end subgraph Scientific["科学驱动取向（概念物理学）"] B1["不择手段取胜，引入新工具"] --> B2["接受硬算，以打开新理解为目标"] B2 --> B3["发现新概念，引发连锁突破"] end ``` Daniel 承认学界观点极其多元，不少数学家自视更接近“诗人”。但从数学史的宏观视角来看，真正的突破往往来自各个学者追随各自独特的好奇心。不同研究者对“何为有趣”有着迥异的品味，这种多样性不断拓宽人类知识的边界，并在新思想诞生时引发多米诺骨牌般的连锁反应，迅速攻克过去无法理解的一大批难题。 - **年轻数学家的误区**：因证明“不优雅”或包含海量无启发性的机械硬算而放弃可行的证明路线。 - **核心研究定位**：将纯数学视作“以概念为研究对象的物理学”，做“好的科学”（good science）而非“艺术创作”（art）。 - **学术进展机制**：依赖多元化学者的个体好奇心扩张认知前沿，在新概念引入后形成级联效应（cascade）。 ### `25:00` 为什么大模型善于寻找反例，却难于构建全新理论体系？ 谈到那些耗时三到十年乃至更久的深水区问题时，Lisha 与 Daniel 探讨了大语言模型目前在此类场景中难以发挥作用的技术根源。Daniel 指出，有些长期思考的难题具有非常明确的猜想陈述，但模型之所以无从下手，是因为这些猜想大概率是**真命题**。例如著名的埃尔德什单位距离问题（Erdős unit distance problem），学界最初普遍认为猜想成立，但最终被证明是错误的，这意味着只要能设计出一个精巧的具体构造（construction），就能直接将其推翻——而当前的大模型在“搜索并给出一个反例构造”方面表现得极其强悍。 然而，在面对如黎曼猜想（Riemann hypothesis）或 $p$-曲率猜想（$p$-curvature conjecture）等问题时，它们深嵌在极其庞大的现代数学理论框架之中，拥有海量的侧面证据支撑其正确性。要在这些领域取得胜利，无法靠单一构造推翻，而是必须证明猜想本身，这需要发展出目前数学界尚未掌握的“重大新思想”（very serious new ideas）与全新技术体系。现有模型所展现的所谓“创造力”，很大程度上依然是在已知领域之间迁移现有技术，一旦面临必须凭空搭建全新理论或机械体系（machinery）时便会显得力不从心。 针对 AI 发展新理论的瓶颈，Daniel 强调自己绝非模型能力持续增长的怀疑论者。他推测未来这完全可实现，目前尚未突破也许仅仅是因为缺乏适宜的强化学习（RL）环境： > “发展一套理论或建立对一个知之甚少对象的理解，这项技能本身更加模糊。所以它可能更难一些。我想你可以对它说：‘加深你对 Zeta 函数的理解’，然后一旦它证明了黎曼猜想，你就给它一个奖励。但我认为，要提出一些你能给予奖励的中间过程目标，是非常困难的。” - **反例寻找 vs. 理论构建**：模型擅长在庞大离散空间中搜索反例构造来击破假命题，但难以凭空创立新框架来证明深植于理论之中的真命题。 - **强化学习瓶颈**：缺乏明确的中间反馈机制（intermediate rewards），无法像博士生导师依据人类学术“品味”提供即时反馈那样，对抽象的“理论理解深化”进行奖励。 - **数学题库作为学习阶梯**：数学界层级分明、难度各异的猜想库，天然构成了人类与 AI 共用的“能力进阶课程”（curricula）。 ### `28:30` 人类认知的算力局限与模型的辅助实践 对话转向人类思维机制的哲学探讨。Lisha 提出，人类能够构建数学与物理理论，或许是因为人类大脑存在算力极限，被迫施加极强的信息压缩压力（compression pressures），从而只能通过高度抽象和提炼本质来理解世界。Daniel 对将“压缩”作为衡量研究趣味的指标持保留态度，但他从人类学角度认可这一观点：人类“无法无限硬算”（inability to just grind）的生理弱点，反而是人类能够做出原创性理论发现的关键催化剂——因为无法蛮力硬算，人类必须寻找更深刻的结构和概念简化路径。 Daniel 结合自身的亲身经历，说明了 AI 在辅助严密推导中的实际价值。在近期完成的一篇论文中，核心定理的大框架已经证明完毕，但他对其中的几个引理（lemmas）感到很不满意，认为它们尚未达到最优状态。为此，他借助了当时处于前沿水平的推理模型 Gemini DeepThink 进行协作攻关，模型最终成功帮助他完成了这几个关键引理的证明，在具体技术推导层面展现出了切实的辅助力量。 > “实际上，我认为正是因为我们无法一直硬算，这种无能对于我们做出发现的能力而言是非常重要的。” —— Daniel Litt ### `30:00` 暴力推导与概念理解：人类为何偏爱“更优美的证明” Daniel Litt 讲述了他与 AI 模型合作推导引理的一段亲身经历。起初，他想证明一个特定的引理，但当时所有的前沿模型都无能为力。于是他自己亲自动手演算了大量具体例子，终于悟出了这道命题之所以成立的深层原因，从而提炼出了一个更准确、更精炼的新引理表述。一旦把这个更优美的表述喂给模型，AI 很快就完成了证明。然而，Daniel 指出，如果现在把最初那个未经提炼的原始引理直接扔给最新的旗舰模型（如 ChatGPT-5.6 Pro），模型确实能够硬生生算出来，但结果将是一篇长达 10 页、极其残暴且没有任何数学洞见的纯机械计算。虽然这份证明在逻辑上完全正确，但它抹杀了发现背后优美概念解释的机会。 Daniel 坦承自己对“丑陋证明”的抱怨多少带点双标：他当时其实意识到那条令人作呕的硬算路线是走得通的，但他内心实在无法容忍这种写法，才被迫去寻找更有洞见的新论证。如今的模型在可靠执行极其冗长的技术性计算方面已经非常出色。Lisha Li 补充道，数学家追求“理解”，本质上是追求一种更简洁、更具信息压缩率的解释。这触及了一个深层的哲学命题：人类数学之所以偏爱非形式化数学（informal mathematics），而不是 ZFC 集合论公理系统下冗长无趣的符号串，正是因为人类大脑需要一种非严格却能带来直觉的理解方式。神奇的是，人类为了“易于人类理解”而对理论进行压缩和美化时，往往恰好能促成跨领域的融会贯通，并催生出极其丰硕的数学理论。 “And so our inability to prove it led to an improvement in the result.” —— Daniel Litt 说明正是因为最初没法硬算，才倒逼人类找到了更好的数学表述与更深刻的解释。 - 人类面对无法硬算的困境时，倾向于通过算例提炼出结构更清晰的猜想或引理，而 AI 更倾向于依赖算力直接暴力展开。 - 人类对数学形式的偏好带有强烈的选择偏差：那些既高度压缩、又易于理解，同时还能产生丰富数学结果的领域，才会被定义为“优秀的理论”。 ```mermaid flowchart TD accTitle: AI暴力破解与人类提炼理解的分歧 accDescr: 对比人类数学家在面对困难引理时寻找优美概念的路径，与大模型依赖暴力计算直接输出冗长证明的路径。 subgraph Problem["初始数学难题"] A["提出初始引理<br/>（早期前沿模型无法证明）"] end subgraph HumanPath["人类探索路径（追求理解）"] B["手动演算大量实例"] --> C["提炼更优美、更本质的命题表述"] C --> D["形成高度压缩的概念性理解<br/>（促进跨领域融会贯通）"] C --> E["AI 迅速完成精简证明"] end subgraph AIPath["当前强模型路径（暴力破解）"] F["输入最新超强模型"] --> G["输出 10 页高强度纯技术硬算"] G --> H["证明逻辑完全正确<br/>但缺乏任何洞见与直觉"] end A --> B A --> F ``` --- ### `35:00` 学术激励扭曲、“摇奖机式”发论文与模型思维坍缩 随着 AI 解决技术细节的能力不断飙升，数学研究共同体面临着激励机制的严峻考验。Daniel Litt 指出，绝大多数人进入数学领域是因为享受学习的过程，但数学的终极目标绝不是“生产论文”，而是“产生理解”。如果这种理解仅仅停留在模型的权重之中，对人类而言是毫无意义且令人沮丧的。 frontier 级别的数学研究依赖于庞大的人才培养金字塔，需要成千上万乃至数百万人学习数学思维来支撑塔尖的极少数人。然而，现有的学术评价体制——尤其是针对急于找教职的博士后——正在奖赏错误的行为。目前最功利的策略就是把 AI 当作“老虎机”（slot machine），让它不断生成对既有猜想的证明，直到摇出正确结果为止。Daniel 透露他做过一个实验：让 Codex 自行联网检索代数几何领域的 5 个最新猜想并进行证明，仅仅花了一个小时，经过几轮交互，他就得到了 3 篇质量极差但完全正确的论文。 这种“摇奖机”现象已经在预印本网站 arXiv 上引发了大量灌水。arXiv 上充斥着毫无数学美感、缺乏人类心智投入的平庸论文，甚至出现过几天之内有三四篇不同作者的论文给出了完全相同的证明步骤。这暴露了当前 AI 模型在推理路径上的“模式坍缩”（mode collapse）——它们高度依赖相同的预训练文献，倾向于收敛到完全一致的解题思路上。Lisha Li 与 Daniel 强调，数学进步依赖于“百花齐放”以及高维空间中极其分散且多元的人类直觉。如果未来的数学研究全面屈从于 AI 的探索偏好，其结果不是拥有成千上万个探索不同维度的数学家，而是等同于将“同一个数学家复制了一千次”。这对于前沿 AI 实验室而言同样是一个危险的信号：如果模型的直觉完全来自人类数学家，而这种多元直觉被同质化的 AI 论文反噬，AI 自身也难以在缺乏人类新颖直觉输入的情况下实现真正自主的范式突破。 “That said, like, the goal of mathematics is not to produce mathematics papers. Like it's to produce some kind of understanding.” —— Daniel Litt 尖锐地指责当下利用 AI 刷论文的功利行为背离了数学追求理解的本质。 - **激励机制错配**：当前的博士后招聘与职称晋升过于看重论文数量和解决旧猜想的结果，促使学者利用 AI 走捷径。 - **实验验证**：Daniel 仅耗时 1 小时便借助 Codex 产出了 3 篇毫无启发性但逻辑成立的代数几何论文，目前仍存放在其硬盘中。 - **模式坍缩（Mode Collapse）风险**：多篇 arXiv 论文在几天内撞车且证明路径完全一致，表明 AI 模型并非展现真正的多样化创造力，而是在收敛至特定的狭隘推理轨迹。 ### `40:00` 为什么即使 AI 超越人类，社会仍需要人类数学家 Lisha Li 与 Daniel Litt 探讨了 post-training scaling 是否真能带来全新能力，以及为什么数学是检验这一问题的最佳领域。Lisha 指出，如果没有足够多的人类与 AI 交互，模型可能无法像在其他领域那样持续产出优异结果。对此，Daniel Litt 提出了更为激进的观点：即便假设未来的模型变得稳健地超越人类（superhuman），人类在认知多样性上无法再提供增量价值，我们依然需要人类数学家。 ```mermaid flowchart TD accTitle: 人类介入数学研究的必要性逻辑 accDescr: 展示为何即使模型具备超人能力，社会仍需要多元化的人类学者引导研究方向 subgraph S1["潜在风险：完全移交控制权"] A["仅依赖 AI 自主研究"] --> B["AI 走最直接路径实现预设指标"] B --> C["丧失广泛研究兴趣与社会价值对齐"] end subgraph S2["理想路径：人类主导的未来"] D["维持广泛兴趣的人类学者社区"] --> E["引导模型探索多元化问题"] E --> F["服务于人类真实关切与社会构建"] end ``` Daniel Litt 强调，数学研究不仅仅是一个纯粹的优化问题。将所有研究控制权交给模型，并不意味着它们会自发走向对人类社会最优的道路，它们很可能只会寻找最直接的功利路径。人类需要保留决定社会走向的主动权，而实现这一目标的最可靠方式，就是维系一个具备多元兴趣、聪明且有能力进行各种深度思考的人类社群。正如 Daniel Litt 所言：“我认为对未来的充满希望的愿景是，人类没有被彻底剥夺权利；我们掌握着自己走向何方的控制权。如果是这样的话，我们最终所做的事情将是由人类的兴趣驱动的。” - Daniel Litt 认为，就算 AI 自主探索能带来大量应用与理解，也不代表人类应该放弃参与；“最优化”不等于“应该执行”。 - 推动科学发展的关键在于维系一个保有广泛好奇心的社群，他们不仅能做数学思考，还能开展全方位的智力探索。 --- ### `42:47` 培养思考者：数学作为通识思维训练的价值 Lisha Li 表达了她的担忧：如果随着 AI 的演进，人类没有开发出顺手的交互界面来鼓励人们持续深思，人类极容易因懈怠而放弃思考能力。Lisha 提到自己作为一个两岁孩子的母亲，常常思考教育的本质——数学不仅仅是刷题（grinding），更是一种极具威力的严格思考框架。她回忆起曾经与匈牙利数学家 Balázs Szegedy 等人合作时的见闻：在布达佩斯，小学阶段就会教授群论。如今 AI 解释概念的能力大幅提升且易于获取，更应该借此将高阶思维工具普及给大众，把更多人带到知识前沿。 Daniel Litt 对此深表赞同，指出数学界是首批公开直面高水平模型冲击的职业之一。虽然编程等依赖电脑的行业早已在私下被 AI 大量渗透，但由于前沿 AI 实验室热衷于公开宣传数学基准测试能力，使得数学领域的讨论尤为受人瞩目。由于数学研究与教育体系高度绑定，保持这一领域的“人力资本”（human capital）将成为所有面临 AI 冲击的行业的标杆模型。 - Lisha Li 强调，在布达佩斯的小学就能接触到群论等现代数学概念，这种扎实的数学训练是通往深度认知极佳的教学载体。 - 两位学者一致认为，AI 的易用性本应降低门槛，帮助大众更早接触深层数学思想，而非沦为代写作业的偷懒工具。 --- ### `45:00` 劣质成果膨胀与教育领域的双峰分化 AI 的普及在教育和学术产出端均引发了明显的分化现象。Lisha Li 指出，许多大学课堂已出现“双峰分布”（bimodal distribution）：一部分学生积极摸索如何利用新工具放大自身优势、加深理解；另一部分学生则完全把作业外包给 AI，结果在脱离工具的真正考核中全盘崩溃。在整个智力产出市场中，AI 极大地降低了生产成本，导致大量稍逊于以往人类水准但成本极其低廉的次品充斥市场，挤压高质量成果。 ```mermaid flowchart LR accTitle: AI 在教育与科研中的双峰分化效应 accDescr: 展现不同群体使用 AI 带来的截然相反的发展路径 subgraph G1["积极进阶路径"] A1["利用 AI 深化理解与辅助探索"] --> B1["拓宽思维深度与研究产出"] end subgraph G2["退化依赖路径"] A2["完全外包作业与论文撰写"] --> B2["思考能力退化与劣质成果膨胀"] end ``` Daniel Litt 分析，技术革命初期廉价低质内容泛滥是常见现象，但若能审慎重构机构激励机制，工具完全可以提升所有维度的产出质量。针对学术界充斥的“垃圾论文”（slop papers），Daniel 直言这不仅来自非专业人士，学术界内部的指标考核也在倒逼职业学者生产平庸成果。然而，他心态乐观地指出，业余爱好者借助 AI 涌入数学领域虽然增加了检索成本，但数学能吸引如此巨大的社会关注度本身是一件大好事。 - 课堂生态出现极端分化：一部分人借助 AI 实现认知飞跃，另一部分人因彻底放弃思考而面临能力萎缩。 - 机构若缺乏合理的评价机制，低成本的平庸内容（slop）就会驱逐高成本的深度研究。 - Lisha Li 警示，人类本性极易趋于怠惰，并严肃提醒道：“你基本上必须不断向上攀登，利用这些模型来深化你的理解。人类的天性太容易偷懒了，你必须抵制这一点，因为一旦妥协，基本上就是你输掉的时刻。” --- ### `49:17` 突破性成果与人类参与：以30阶椭圆曲线为例 在讨论的最后，Lisha Li 提到了节目播出前一天刚刚曝出的重磅构造性数学进展：一条代数秩（rank）达到 30 的椭圆曲线被发现。这一成果再次引发了两人关于 AI 自主性程度的探讨。 据传该成果是由 Claude Opus（被称为 Claude Fable 的变体或特定提示版本）在 Levent Alpoge 及其合作者的提示引导下发现的。Daniel Litt 谈到，近期 Levent Alpoge 产出了一系列令人瞩目的数学成果，但这些成果背后的模型自主程度究竟有多高仍不透明。现阶段并没有公开的技术细节与方法论披露，业界普遍推测这极可能是一种高级的“半自主”（semi-autonomous）人机协作，而非模型脱离人类专家干预的完全自主发现。这再次印证了两人的核心共识：前沿数学重大突破的背后，依然离不开人类顶级专家的深度引导与精准提问。 - 最新学术动态：数学界宣布发现了秩为 30 的椭圆曲线，该计算结果引发了广泛关注。 - 该进展归功于数学家 Levent Alpoge 及其合作者对前沿语言模型的提示词引导，凸显了专家级干预在当前 AI 辅助科研中的主导地位。 ### `50:00` 破纪录的数学构造与模型的长篇证明瓶颈 关于近期 AI 寻找高阶数学对象构造的尝试，Daniel Litt 指出，如果不了解具体方法，很难评判这些结果到底有多大的意义。从数学共同体的历史认知来看，这类寻找极值或破纪录例子的成果通常属于“很酷，但并不算什么大事件”的范畴，它们往往不会发表在《数学年刊》（Annals of Mathematics）这种顶尖期刊上，更多是收录在某位学者的个人纪录网页里。不过，依然有极其顶尖的数学家钟情于这类问题，比如 Noam Elkies 和 Zev Klagsbrun，他们长期推动椭圆曲线秩的纪录，最近就找到了秩为 29 的例子。Daniel 强调，无论是面对人类数学还是 AI 的数学产出，其重要性在事前都难以断言： > “关于模型产出的结果，我常说的一点是：除了事后回顾，你无法对它进行评估。人类数学也是如此。有时一个我们原以为非常重要、需要深刻新思想的问题，事实并非如此；而有时又确实需要，你无法预先知道。” 谈及目前大模型产出的证明往往相对短小而巧妙，Lisha 提到 OpenAI 的 Mark Chen 和 Pragnyat Patel 也曾对模型给出简短证明感到惊喜，并询问这究竟是筛选偏好还是模型天然倾向。Daniel 认为核心原因其实更务实：目前的模型根本不具备生成和检验长篇复杂证明的能力。模型之所以产出简短而聪明的解法，纯粹是因为人类和验证工具目前只能检查这种长度。事实上，如果让人类去阅读那些长篇内容，就会发现模型经常产生连它们自己再次审查时都会承认的错误。虽然 OpenAI 和 Anthropic 内部可能已经用模型解出了更多问题，但很多大概因为无法验证真实性而无法公开发布。例如 OpenAI 近期发布的 10 个数学问题之所以具有说服力，是因为它们全部在 Lean 中形式化了；而许多更长、更庞大的推导，由于 Lean 的 Mathlib 库中尚无前置定义，根本无法形式化验证。 arXiv 上已经出现了一些长篇 AI 论文的反面典型： - 曾有人在 arXiv 上上传了一篇宣称解决了“正特征下奇点消解”（resolution of singularities in positive characteristics）的长达 800 页的 AI 生成论文。 - Daniel 断言这篇论文绝不可能正确，因为奇点消解是一项极其重大的数学突破，远远超出了当今经过合理推断的前沿模型能力范畴。 - 这类 800 页的论文既没有人类读过，AI 自身也缺乏验证其正确性的能力。 --- ### `55:00` 框架（Harness）诱导长程推理的隐患与人类如何审阅论文 从数学类比到编程领域，Lisha 提到 Cursor 曾测试用长程 Harness（测试框架/脚手架）在 Rust 中重写 SQLite 的案例，说明实现长程、大跨度的复杂任务需要极其精密的 Harness 进行规划与验证。然而，Daniel 警告，在数学中单纯为了“诱导出长篇证明”而构建 Harness，往往会以牺牲可靠性为代价。当 Harness 的目标被设定为拼命生成输出时，模型为了发挥“创意”打破严谨的思维定势，胡说八道的概率会大幅上升。目前学术界甚至有些人试图利用这种套路对学术评价机制进行声誉套利（arbitrage the prestige mechanics），批量诱导生成未经验证的论文；任何能直接催生 250 页论文的 Harness，背后几乎都放弃了对严谨性的把控。 面对长达数百页的复杂证明，人类数学家并不是逐行进行机械检查的，而是先评估全局逻辑架构并进行边界压力测试（例如：该论证是否推导出了已知为假的结论？在特殊情形下是否成立？）。Daniel 指出，模型目前完全不具备这种“模糊单元测试”（fuzzy unit testing）和宏观把控能力。他分享了亲身经历的一件趣事：几年前曾有一篇论文宣称得出了重大结果，Daniel 读到后立刻察觉其论证结构存在根本问题——因为它证明出来的结论“强到了不可能为真的地步”（proving something that was too strong to be true）。然而，论文中具体的细微错误极难定位，最终是他和几位同行与作者反复沟通推敲，才终于揪出了那个隐蔽的症结。如今的模型既看不出这种大局上的逻辑破绽，也找不出细微深层谬误。 ```mermaid flowchart TD accTitle: 人类与 AI 审阅长篇数学证明的机制对比 accDescr: 对比人类数学家依赖全局架构压力测试与目前 AI 缺乏宏观逻辑检查能力的差异。 subgraph HumanApproach["人类数学家的审阅路径"] H1["宏观把控论证全局结构"] H2["寻找反常推论 / 边界压力测试"] H3["逐步定位细微隐蔽错误"] H1 --> H2 --> H3 end subgraph AIApproach["当前大模型的局限"] A1["易陷入语法与局部推导"] A2["生成超长篇幅（如数百页）"] A3["缺乏'模糊单元测试'宏观把关能力"] A1 --> A2 --> A3 end A3 -.->|"无法替代"| H1 ``` 在个人工作流上，Daniel 表示自己虽然会写一些简易的小工具，但他不会用 Harness 去追求所谓“全自动数学研究”，因为数学研究的意义在于让人类身处闭环之中去真正理解数学。在播客尾声，Lisha 聊起育儿话题，问及 Daniel 如何面对自己 3 岁女儿的数学启蒙，以及在 AI 时代如何不卷计算（grind）、而是培养对数学纯粹的热爱；Daniel 笑着提到，女儿现在才 3 岁，还从未接触过 AI。 ### `01:00:00` 女儿的数学启蒙与 AI 时代的教育本质 Daniel Litt 分享了自己三岁女儿 Sophia 令人忍俊不禁又深感自豪的日常趣事。他提到女儿最近刚刚开始接触加法，目前能够通过数手指来计算个位数的加法；在数数方面，她能够非常稳定地数到 30，甚至能较为勉强地数到 50。平时他们父女经常会一起探讨几何形状之类的话题。就在几天前的一个早晨，Daniel 去叫她起床时发现她正躲在毯子下面，便好奇地问她在底下忙什么，小家伙竟一本正经地回答说自己正在里面“做数学呢”。Lisha Li 笑着表示在社交媒体（X/Twitter）上也看到了他分享的这则推文。Daniel 坦言女儿显然感知到了父亲对数学的热爱，因而也耳濡目染地对数学产生了浓厚兴趣。 从女儿的成长出发，两人探讨了在技术剧变下教育的真正内核。Daniel 认为，当女儿在二十年后步入成年并独立生活时，世界很可能会与现在大不相同，但人类教育所培养的大多数能力，在面对外部世界翻天覆地的变化时依然具备极强的韧性。Daniel Litt 深刻地指出：“我认为学习数学的初衷一直都是为了能够清晰地思考，并更好地理解这个世界。”他强调，哪怕未来存在能力极其强大的 AI，人类自身清晰思考的意愿也绝不会消失；这一逻辑不仅适用于他个人深爱的数学，同样也适用于大量阅读书籍和学习人文学科。数学专业乃至整个教育体系所沉淀的真实价值，正是需要代代相传并努力保护的东西，尽管为了确保这一目标的实现，教育体制本身或许需要经历巨大的重塑。 - **女儿的数学进展**：目前能靠数手指做一位数加法，稳定数到 30，能半稳定数到 50。 - **教育的核心价值**：在能力强大的 AI 时代，学习数学和人文学科的根本目的依然是培养人类清晰思考、理解现实世界的能力，而非单纯作为一项工具性技能。 ### `01:02:18` 从柏拉图立体到群论：节目结语与展望 谈及对女儿潜移默化的影响，Daniel 开心地透露女儿最早学会的词汇之一竟然是“二十面体”（icosahedron），因为在她一岁的时候，祖父母曾送给她一个二十面体玩具，这让她很早就认清了各种柏拉图立体（Platonic solids）。Daniel 随即开玩笑称，女儿接下来的必修课将是“群论”（group theory），未来在教她更复杂的加法与减法时，一定要直接放在抽象的群结构语境中进行传授。Lisha 听后不禁莞尔，感叹许多人在教学时往往忽略了数学动机，而 Daniel 至少在从理解本质的角度去启蒙，而不是枯燥地背诵规则。 对谈尾声，Lisha 对 Daniel 带来的深入分享表达了感谢，认为探讨当下数学前沿与 AI 发展的交叉点非常有趣，并表示鉴于该领域日新月异的技术突破，期待不久之后能再次聚首交流。Daniel 也对此表示完全赞同。随后播客进入结语部分，Lisha 提醒听众可以在 Apple Podcasts、Spotify 上关注订阅节目，并在 X（@a16z）及官方 Substack 平台获取更多后续内容与动态，节目最终在针对 a16z 投资项目的常规合规免责声明中圆满结束。 > “在更深入地教她加法和减法时，我们会在一般群的语境下来进行。” —— Daniel Litt 幽默地谈论他对女儿下一步的“超前”数学启蒙计划。 ## Notable quotes > `00:00` The goal of mathematics is not to produce mathematics papers, it's to produce some kind of understanding. — Daniel Litt > `01:00:00` 我认为学习数学的初衷一直都是为了能够清晰地思考，并更好地理解这个世界。 — Daniel Litt > `28:30` 实际上，我认为正是因为我们无法一直硬算，这种无能对于我们做出发现的能力而言是非常重要的。 — Daniel Litt > `13:30` 至少对我而言，一个公开问题的意义在于它衡量了你在哪方面遭遇了认知失败。 — Daniel Litt > `16:30` 问题越具有哲学意蕴，或者脑海中越缺乏精准具体的提问，AI 能起到的作用就越小。 — Daniel Litt > `45:00` 你基本上必须不断向上攀登，利用这些模型来深化你的理解。人类的天性太容易偷懒了，你必须抵制这一点，因为一旦妥协，基本上就是你输掉的时刻。 — Lisha Li > `40:00` 我认为对未来的充满希望的愿景是，人类没有被彻底剥夺权利；我们掌握着自己走向何方的控制权。如果是这样的话，我们最终所做的事情将是由人类的兴趣驱动的。 — Daniel Litt > `50:00` 关于模型产出的结果，我常说的一点是：除了事后回顾，你无法对它进行评估。 — Daniel Litt ## Mentioned Daniel Litt, Lisha Li, OpenAI, Anthropic, Claude, Claude Opus, Claude Fable, ChatGPT, ChatGPT-5.6 Pro, Gemini DeepThink, Codex, Cursor, Lean, Mathlib, arXiv, Erdős unit distance problem, 格罗滕迪克-卡茨 p-曲率猜想, 伯奇与斯温纳顿-戴尔猜想, 黎曼猜想, 和积猜想, William Thurston, Carlos Simpson, 望月新一, Bryan Birch, Peter Swinnerton-Dyer, Balázs Szegedy, Levent Alpoge, Noam Elkies, Zev Klagsbrun, Mark Chen, Pragnyat Patel, Annals of Mathematics, SQLite, Rust ## Video description a16z’s Lisha Li sits down with Daniel Litt, Assistant Professor of Mathematics at the University of Toronto, to unpack AI's rapid progress in mathematics, what today's frontier models can actually do, and what they're still missing about the way mathematicians think. Daniel explains why some recent AI-generated results are genuinely impressive, including an autonomous solution to the Erdős unit distance problem, but argues that solving problems is only one part of mathematics. Today's models can grind through calculations, combine known techniques, and search enormous spaces, but still struggle with intuition, theory building, identifying the right questions, and developing the kind of big-picture understanding that drives much of mathematical progress. Lisha and Daniel also explore how AI is already changing mathematical research, why an explosion of AI-generated papers could distort academic incentives, and what happens if researchers outsource the work of thinking rather than use AI to deepen it. Ultimately, they ask a question that extends far beyond mathematics: as AI gets better at intellectual work, how do we make sure humans keep getting better at thinking too? [Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/c0772156-b98e-43c5-b01b-5cd54df1a9a5/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=c0772156-b98e-43c5-b01b-5cd54df1a9a5&feed=JGE3yC0V)

[0:00] Daniel Litt: The goal of mathematics is not to produce mathematics papers, it's to produce some kind of understanding. [60:00] Daniel Litt: 我认为学习数学的初衷一直都是为了能够清晰地思考，并更好地理解这个世界。 [28:30] Daniel Litt: 实际上，我认为正是因为我们无法一直硬算，这种无能对于我们做出发现的能力而言是非常重要的。 [13:30] Daniel Litt: 至少对我而言，一个公开问题的意义在于它衡量了你在哪方面遭遇了认知失败。 [16:30] Daniel Litt: 问题越具有哲学意蕴，或者脑海中越缺乏精准具体的提问，AI 能起到的作用就越小。 [45:00] Lisha Li: 你基本上必须不断向上攀登，利用这些模型来深化你的理解。人类的天性太容易偷懒了，你必须抵制这一点，因为一旦妥协，基本上就是你输掉的时刻。 [40:00] Daniel Litt: 我认为对未来的充满希望的愿景是，人类没有被彻底剥夺权利；我们掌握着自己走向何方的控制权。如果是这样的话，我们最终所做的事情将是由人类的兴趣驱动的。 [50:00] Daniel Litt: 关于模型产出的结果，我常说的一点是：除了事后回顾，你无法对它进行评估。

[0:00] Daniel Litt: The goal of mathematics is not to produce mathematics papers, it's to produce some kind of understanding. [60:00] Daniel Litt: I think the reason to learn math has always been to think clearly and better understand the world. [55:00] Daniel Litt: When you make a harness whose goal is to elicit a proof, I think it often decreases reliability because you're just trying to produce output.

[0:00] The goal of mathematics is not to produce mathematics papers, it's to produce some kind of understanding. Maybe some of that understanding resides in model weights. To me that's pretty unsatisfying. Comparing Anthropic with OpenAI, do you detect any differences in how that is similar to human reasoning? They definitely are not good at it autonomously, but with some hints you can kind of get them to do something interesting. expand in some fairly uniform ways. What has been the most impressive result so far? My favorite fully autonomous result by an AI so far is the solution to the Earth-ish unit distance problem. There was some lemma I wanted to prove, none of the frontier models could do it. So I worked out a ton of examples on my own and I realized, oh, well, maybe here's some reason why it could be true. Once I had that statement, the models were able to very quickly prove that sort of better statement. kind of best adapt and benefit from this. already resemble human mathematicians, and where they still fall short, particularly when it comes to intuition, developing new theories, and even figuring out which questions are worth asking. They also explore what happens to mathematics when generating a proof becomes cheap, why academic incentives may need to change, and how mathematicians can use AI without outsourcing the understanding that makes the work valuable in the first place. And more broadly, they ask as increasingly capable models move into every knowledge profession. something different has been revealed, and then you're very kind of like, what do you call it? You have a lot of opinions. You have a lot of opinions, exactly. So I want to get into that. So I mean, one of the things that I'm most interested in is not just like a discussion of how the capabilities of advanced, I feel like in math, that's definitely the headline, etc. But also, you've been very thoughtful about how practicing mathematicians should respond. And so that kind of gives us a chance and opportunity to talk about actually, what is special about math? It's not just like, to what actually mathematicians do. And so maybe like with that arc in line, we can start with what has been the most impressive results so far, given all the recent progress for you. And then maybe, yeah, we'll kind of take it from there. Yeah, so okay, so there have now been a lot of results, some of them produced autonomously, some produced semi-autonomously, some where the AI contribution is just not at all clear. There are a lot of different areas, so anything I say is kind of, you know, I can only really comment on things that I have some expertise on. So it's quite possible that if you talk to a different mathematician, you'll get different answers here. So my favorite fully autonomous result by an AI so far is the solution to the Erdős unit distance problem, which I think was announced in mid-May. What I liked about that is it seemed to me that it was in some ways a little bit creative. So I think some of the results we've seen have had kind of the flavor. If you kind of take some known techniques and apply them in maybe a clever way, or you know, they have kind of been some kind of results I would characterize as like last mile. Like where some recent work was done, quite deep work done by a group of human mathematicians. And then the AI kind of took the final step. unexpected. So first of all, my sense was that people working in the area thought it was true, and then there was a counter-exemple found. But then also it brought in some techniques from another area. I think those techniques were not especially... like deeper new. There was classical ideas from the 60s, but they were new to this area of studying, you know, point configurations in the plane. And so that was pretty cool. And then afterwards, we got to see like it was kind of fruitful. So a bunch of mathematicians took those ideas and used them to find counter examples to a bunch of other interesting open questions. So for example, like the sum product conjecture over the real numbers. So that's at least the way I like to think about how cool Salt is. Like you look at a post-hoc and you see like, oh, well, we're whatever new ideas that we're in front of [5:00] if any, kind of useful to do other things, did they improve our understanding of something? And I think that's maybe so far the main example I know of a result of that form. Yeah, I think it's really meaningful that you're commenting on this because that result came out to your point in May. And there's been so many headlines so far. And it's kind of probably hard for somebody who's not a practicing mathematician to appreciate the differences in these headlines. And so you kind of already started laying out sort of a taxonomy of like what is different in that proof. And so it'd be kind of interesting maybe to use that as kind of both an excuse to talk about where you sense the model differences are and what like mathematicians actually do. understanding of what mathematicians do is that we're pushing around symbols in a logical manner. And this is why RL is so successful at this, because you can kind of both verify it somewhat cheaply compared to other domains. And then also because the rules are quite illegible. And so you can kind of like, if you're superhuman at that, you might be good at math. But I think that, of course, betrays most of actually what is interesting about mathematics, which is perhaps I think you said this as well, but I think anybody who's tried to do math is sort of like, it's about the understanding and getting at truth. remaining confused and developing intuitions. And I mean, the tool to do that stuff is, of course, in having really good strong abilities to push out logical implications. But maybe If you can kind of speak to like, when you say it's most impressive and creative, like decoupling just the, inhumane maybe feats of just like logical implication from like where is it being creative? What is it helping engender in terms of like mathematical activity as well? Yeah, okay. So first of all, I mean, you have characterized it as like inhuman in some way. I actually think the argument was very human. Oh, I'd love to go into that. I'm sorry, OpenAI released some chain of thought. It was very recognizable. It was like if I tried to imagine like my chain of thought trying to solve a problem, like it might look kind of like that. Yeah. We haven't seen the raw chain of thought, so maybe that would... Maybe they cleanse it a little bit. Yeah. Yeah, or maybe the model like to swear a lot in the middle of the chain of thought and clean that up or something would know. kind of typical of most of the results that I've studied. They don't seem inhuman at all. They seem absolutely like something a human mathematician could produce. And they're typically understandable, if not so well-written. If you just look at raw model output, it's not like there's some move 37 or whatever. It's like a human mathematician doing math. It's like a human mathematician doing certain types of math. So they're definitely, the models are still like not very good at some mathematical activities. And here I don't mean like by field, but just like certain things you do when you try to solve a problem, the models don't seem to be doing, but certain things they're very good at. So the ways they might be a little bit inhuman is like they don't get tired, they know a lot. But if you have to just read the final output, it doesn't seem kind of inhuman. It's interesting because, I mean, obviously we can't get too much information from the labs who are producing these models. Why of what the training recipes are, how they're kind of advancing in reasoning. But at least one things we do know, and I think kind of OpenAI spearheaded this, is it's like reasoning in natural language is actually what they happen to scale up. And it's not actually pushing a lot of lean, verified proofs as like the training corpus. And that's kind of amazing. On another side, what's kind of interesting is It's not clear that a lot of the mathematical training data, if they use that, to a large extent at all, is reflective of how mathematicians think, if that's fair, because a lot of it is not legible as traces, right? Most of the papers are crisp and polished. The textbooks certainly show very little motivation of how something is developed, which is why it's usually easier to follow a research direction by actually talking to the researchers and how they're thinking about it. So I'm kind of curious, before maybe even going to the taxonomy as an excuse, when you're examining these models and their results, detect any differences in how that is similar to human reasoning. And then also if you have any comments or like insights on perhaps why... natural language scales so well that way. Okay, so first of all, I really like your point, by the way, that they're mostly doing natural language reasoning rather than lean. I think that that suggests to me that you hear a lot of people say math is a verifiable domain, that explains the progress or whatever. My sense is that because they're primarily scaling informal reasoning, probably the techniques are going to generalize to other domains pretty well. That's just my guess. Okay, you asked a little bit about Claude versus JetAPT. My sense is that they're pretty similar in terms of capabilities. with chat GBT than Claude Fable. But it seems like there's a lot of cases where OpenAI will drop a solution to some problem and anthropics will say, oh, you know, we also do that too. Exactly. It actually seems like they're solving a very similar collection of problems and it's like kind of a relatively small portion of what human mathematicians do. So, yeah, one thing that is interesting is like we see the solutions have a certain flavor, right? Like the all beefing [10:00] that this isn't surprising. There'll be things that rely on the model's strengths, like their ability to round out a long computation or pull together kind of technical ideas from many areas, or maybe many papers that a human mathematician might not have read. But they seem weaker in things like intuition or having some big-picture point of view. A lot of what I do as a mathematician is I have some kind of philosophy that's very non-rigorous. Maybe I think this thing is kind of analogous to this thing. And then a lot of what I'm working out is figuring out how to make that precise trying to measure the extent to which I've succeeded in understanding that by like, can I solve a problem or whatever? Can I find an interesting phenomenon which I don't understand and now I can't understand it. And so far... I haven't, you know, you can see even the best results the models are producing, you don't see that much of this kind of reasoning. It's more like they're very, very good at applying some known techniques, which is to be clear, that's like a very powerful thing to do, to be very good at applying like all known techniques. There are mathematicians who have had great careers doing very high quality work of that flavor. And I think a lot of what the models are producing is high quality in that way. But it's like some kind of fairly narrow band of what mathematicians care about. So far, I do think there's signs of both, you know, all the Frontier models starting to be able to do more fuzzy things. So, like, I've tried to get both Fable and... chat GPT 5.6 sol, I guess, to do some kind of theory building. And it's like, they're not, they definitely are not good at it autonomously, at least with like whatever scaffolding I've set up. But with some hints, you can kind of get them to do something interesting. You know, when you do, when you give the models hints, it's always a little hard to tell like, what part is from the model and what part is from you. But my experience is that like, if they can do it with like 100 bits of hints or whatever in six months, maybe they can do it they're also kind of somehow picking up some of this like implicit and unwritten mathematical knowledge. Okay. I would love to, so go into the, um, the intuition part and where it, sucks at. to put it in a very basic way, but you actually mentioned a small detail, which is that, you know, from the public, you've gleaned that Anthropic and OpenAR are probably neck and neck, but you personally are using a lot more ChatGPT. Like, why is that? Or 5.6. Well, I don't know. I mean, I just, I think it's just in our show. Creature of habit, yeah. It got, one thing is that ChatGPT models were just not useful for research math. And I think maybe around Opus 4.5 or Opus 4.6, they more or less caught up. But some experimentation suggests to me they're pretty neck and neck. And so for my own work, except when I'm just experimenting, I mostly just stick with one and start at random. As people outside the labs like us, it's really interesting just to compare how they differ on the frontier. And I mean, to your point, it might be a little bit of momentum. I do think that, experience 5.6 has been like a lot more clear and exposition and it just like it's there's a little bit and this might not be true I mean obviously the the models are just incredibly jagged at the frontier um but like it's In the explanations of results to me, I always find that 5.6 is giving a a more accurate theory of mind of what it assumes I know and don't know. Whereas Fable might be explaining something very trivial, but then just like jump at like, well, you know, obviously you should know these things. Yeah, I find they're both pretty bad at theory of mind. Okay, great. So then seeing it from your eyes, you're probably asking much deeper questions. Okay, so the thread that I really wanted to pull on was when you're talking about the models maybe starting to get better intuitions or even theory building. useful to kind of talk through like what is your primary you know activity as a mathematician especially in your area of like algebraic geometry. probably has a very different flavor than a combinatorialist or, you know, some other areas. So if you can give a little, maybe briefly of the land and then kind of explain... What your mathematical activity was pre-AI and then maybe how it's kind of like changing with AI. Yeah, so I think there are a lot of different kinds of mathematicians. There's like a lot of different, you know, spectra on which one can put a mathematician. So definitely a lot of mathematicians like solving open problems. And then I think I'm one of those. I like to solve an open problem. taxonomy you could have as a problem solver versus a theory builder. At least for me, the point of an open problem is it's like supposed to measure your failure to understand something. So it's kind of like a benchmark. [15:00] Right. Like, you know, one problem I really like is the gross and decaps p-curpature conjecture, whatever that is. It measures something about our failure to understand differential equations. So it's like there's some very basic object we would like to understand. If we can't answer this conjecture, we know we don't understand it. Okay. So in practice, how do you get a problem which is supposed to be measuring something you don't understand? Well, of course, you try to understand the thing better. And in practice, what that means is like... Well, you try to find the smallest situation where you can't understand something and fiddle on with it. And then you stare at what once you win, you stare at what you developed to win and try to turn that into some theory. So that's one thing you might do. You might try to solve a problem and in so doing, develop some kind of new understanding of the situation. You might also just have some feeling that this thing is related to this other thing. And you might start building a table like, oh, this property A is related to property property B prime and so on and so forth. So for example, In my work, a lot of it is motivated by some analogy between cohomology of algebraic varieties and representations of fundamental groups. So that's some fancy stuff. But it's just that this analogy is like very... very fruitful and like really any phenomenon that appears on one side, you can find an analog on the other side. And so trying to... to realize that dream has led to a lot of beautiful mathematics of the last like 30 or 40 years by people like carlo simpson and jacquero machizuki and others And so here it's really just like someone noticed, like here's an analogy, and then that analogy has led to a huge amount of developments. There's not really like an open problem at the end, although of course as you develop this you come up with lots of open problems. There's just like a philosophy that you're trying to realize. And that philosophy is like super not rigorous actually. It's not like symbol pushing at all. Yeah, so that's another kind of activity that I like. Yeah, beyond that, you know, a lot of... uh, what you do when you try to you know, a lot of the activities, you're like, you're actually trying to figure out what the right question is, even. Like here, you have some object, you feel like you don't understand it. And like, figuring out what you don't know is like actually a very challenging thing. thing to do. So, you know, there's like a list of, you know, you can go online and list open conjectures or whatever. And this doesn't really capture in a lot of ways what we don't know. Like often finding the conjecture is like really, really hard. So I don't know, a good example of this is like the Bershyn-Swinner-Gendeyer. conjecture, which is one of the millennium problems, is this beautiful relationship between L functions of elliptic curves and the set of solutions to the corresponding equations of the rank of the group of solutions, sort of what that means. It was discovered by, like, it was like the first big data conjecture. So Bersh and Swinderton Dyer had, like, found all these statistics on elliptic curves in the 60s, like it was one of the first ever computer-aided bits of mathematics. the slope of some line on the graph was related to some other algebraic invariant they knew, and that was the source of this conjecture. So a lot of the time you're just like working out examples and like trying to, like you're doing kind of science, like you run an experiment and you try to figure out an explanation for that experiment. And so I think going back to AI, I think like these are things where AI seems so far to, you know, help a lot more in some things than other things. phenomenon it is, or the less you have a precise question in mind, the less useful it happens to be. And so you were asking how I use it in my daily life. Actually, what I've found is that the projects that I have that kind of predate AI, like the projects I've been thinking about for three or four or five years, it's just not that useful. It's primarily kind of a substitute for Google or something. I might use it to learn about some related topic or something where I would have earlier Googled something and then read a paper, maybe I'll discuss it with AI instead. So it saves some time, for sure. It's like not really doing deep intellectual work for me. But then, you know, because I, you know, I'm now like, well, I suck at coding. And so now I know, you know, my good friend who's really good at coding. And now I have all these coding projects. It's like suddenly, if I had a question where coding would have been really useful, I would have procrastinated on it for six months until I. Tireless PhD student. Exactly. So, so yeah, now I've picked up all these projects where, yeah, coding is really useful. like if you want to find an example of something you can just ask it to find work through a thousand examples in parallel or maybe ten examples at a time ten different sub-agents and that's really useful but these are like kind of different activities which are like [20:00] on top of what I was doing before. I'd love to dig in. So yeah, it's... Yeah, go on. Yeah, sorry, because you were mentioning, you know, there's projects that you've been working on for like three, five, you know, three, four or five years. And I don't know if it's correct to say, like, those are more of the theory building aspect of it, because you did characterize yourself as like an open problem solver, or like, what is the thing that is, because like the deep thinking part, maybe, maybe like, you know, kind of for this audience, it might also be useful to kind of say, Or explain, you know, most of pure math is, it's not like it's being motivated by, you know, nothing on applied math, but applied math is like at least there's some external motivation for why a certain formal structure might be interesting to study. Whereas this one, it's like purely, it seems almost sociological. And to some extent, I think like Thurston made some comment or point about this in the 70s that it is a sociological phenomenon. If more and more mathematicians start examining something, then you'll maybe, you know, converge on some interesting structures. Thank you. reason why people would prefer to study something where they think it's beautiful? Like what is it that drives maybe you in particular? And then maybe you can make a more general comment about the profession. Definitely some people are motivated by beauty or aesthetic considerations. I try not to be motivated by that. I think that's a bad way. That sort of limits you. One thing, a failure mode I see among young mathematicians sometimes is you have something and you think you know how to prove it. And then the proof feels really ugly. For this audience, what is ugly? Because I have an information of what's ugly, but what is that for spelling out? I don't really know. I don't really have aesthetic feelings, but people sometimes feel this way. Maybe it involves a lot of grinding, calculation that's not illuminating. Exactly. You should win by any means necessary, in my opinion. I like to think of what I'm doing as doing physics except with concepts. open up further understanding most? Yeah. Well, you know, am I introducing a new idea that will be, like, broadly useful to understand this object? Yeah. I mean, I guess, you know, in some sense, there's, like, some aesthetic consideration there, but it's, I try to, I think the orientation of, like, trying to do good science rather than trying to do art is what I prefer. But there's a huge, I mean, there's a huge variety of opinions here, and, like, lots of mathematicians think of themselves as being, like, closer to poets or something. Yeah, what I do think is broadly true is that progress comes from people, it seems to come in mathematics, from people sort of pursuing their personal curiosity. And that it's sort of been crucial historically that there's like a lot of different people with different views of what's interesting. And then the frontier of knowledge expands and expands. And then suddenly, you know, you have these opportunistic situations where a new idea has been introduced and you can now find you now can suddenly cascade through a bunch of other things that we didn't understand before. Yeah. I mean, so then going back to the three, four, five year problems and where are the models sort of not useful. kind of ask it another way. Like when you're doing the deep thinking, like is it just that it's just not clear that you formulated as a... problem and it's more that you're thinking about these Like, what are, you know, what are the fundamental kind of physics of, you know? So in some cases, I mean, the... Some cases there is like a well-stated problem here. You know, I said I've been thinking about things for maybe 10 years now at this point for certain problems. Sometimes there is just like a conjecture that I would like to prove that that is is there. I think one reason the models might not be useful for some of these things is like the conjectures are true. So like I think that certain, you know, for example, you know, this unit distance problem, like the general belief in the community was that it was true and then it turned out to be false. And so what that means is that there's like a specific construction you can do to... to refute it. On the other hand, I think a lot of the things I think about, like, I don't know, maybe I'm about to, you know, someone will come up with a counterexact of a P curvature conjecture through my hypothesis tomorrow, and, like, I'll look like a fool. But in general, like, these conjectures fit into some very broad theoretical framework, which means that, like, we actually have a lot of evidence that they're true. Um, and so, uh, Yeah, so that's part of it. There's not a construction you can do to refute it. You need to somehow kind of, we have this giant framework where certain pieces of it are only conjectured, and you probably need to resolve some of those conjectures to win. We also have a pretty good sense, I think, that very serious new ideas are needed. [25:00] resolved those contractures. So like... Of course, you can't be sure. Like maybe there's some very clever construction that will let you, I don't know, avoid having a big new idea. I don't know. Maybe, you know, it's quite possible we'll find this out. But my sense is that for at least a lot of the things I've been thinking about, you know, they're just not accessible to applying known techniques in a very technically strong way. So you need to develop a new technique. Just to be clear, I'm not saying the models won't be able to do this. It's just like so far they seem not to... Actually, that's exactly the point that I wanted to delve into because it's... To your point, it's like, okay, we can try to calibrate and forecast why they would... get better at this, but it is true, like, you know, that Just providing construction for a counterexample, they seem to be strong on. If you have to start developing either new theory or, to your point, techniques, machinery to solve problems, to prove why conjecture is true, it struggles more. And it's probably because a lot of what it's drawing on is also just, like, techniques that have happened in other areas, and they're porting it over. And to your point, that's why maybe the unit distance problem was such a more creative result, because it was maybe doing more of that on its own. It was, like, from another. At least it was, like, drawing in something from an unexpected area. Exactly, exactly. And so, like, maybe then to kind of ask the question, it's not that, because now we're like, okay, fine, AI is getting so good so fast, we can't count it out. But, like, why, what do you think it has to, um... Yeah, I guess you're spelling out what it has to get better at, but maybe some more kind of feelings on like why it's kind of particularly hard to then develop that new theory and technique. That's a good question. I mean, I think you just need a different, like my guess actually is it's probably totally doable. It just hasn't been done yet. Like maybe you just need a different RL environment. I don't know. Yeah, so at this point, my expectation is just like that the trajectory will continue upwards. I'm not a skeptic of continued capabilities growth. But yeah, you know, I think what is definitely true is that like, The skill of developing a theory or building your understanding of some poorly understood object is a fuzzier one. So it might be harder. I guess you can tell it, develop your understanding of Zeta functions. Then once it proves the real hypothesis, you give it a reward. But it's kind of hard, I think, to come up with some intermediate things that you can reward. Yeah. Yeah, I mean, that said, you know, I do think... mathematics as a whole provides a lot of conjectures of varying levels of difficulty. So, you know, maybe this explains why there seems to be a little bit of progress in these areas. Like, presumably, they are trying to, you know, get it to solve lots of problems, and some of those problems develop at least some of the skills of theory building. Humans are able to develop these skills. You know, I guess sometimes they get rewards from their PhD advisors, and their advisor says, oh, that's a good idea or something, based on some element of human taste or whatever. that that might be something one can do too but yeah my my expectation is that just like as part of continued capabilities growth we'll see growth in these areas too yeah yeah and i like the framing where you're sort of casting these increasingly difficult conjectures as a as a you know a form of curricula for both humans and of course ai and it you know it may be kind of getting a little bit too philosophical for some people's taste it's like it it it gets at the question of like, why are we particularly good or maybe particularly bad at math too? Because it's... You know, what is it that we're talking about? either struggling to do or some people are particularly good at when you develop new theory. Because it's not, again, it's like not really... Maybe it's related to like, why can we formulate good structures for physics as well? It's just like, it's not obvious. All parts of the world are kind of understandable and legible that way. But some parts are and therefore we try to do it because there's maybe compressive pressures on our planet. our minds because we can't understand anything beyond compressing stuff more finely. I don't know if that's also the correct interpretation. I'm a little bit skeptical as compression as a metric of interest, but it's definitely as an anthropological approach. reason for like why we do a certain theory building. I think it's true. And in fact, I think it's like kind of like our inability to just grind is kind of important to our ability to make discoveries. So I'm just like as an example. So I have one hyper. So far, the models were kind of useful, so they proved a couple of lemmas. So this is some situation where we had kind of proven the main result, and then there were some lemmas I was unhappy with, like they seemed not to be optimal. And so I worked with, actually this was with Gemini DeepThink, which at the time was also on the frontier, no longer for now. Yeah. [30:00] where I kind of worked with it to improve the lemmas. And so what happened here was there was some lemma I wanted to prove, and the models couldn't do it. So none of the frontier models could do it. And so I worked out a ton of examples on my own and I realized, oh, well, maybe here's some reason... why it could be true. So I found a better statement of the lemma. And then, okay, once I had that statement, probably I could have done it pretty fast, but also the models were able to very quickly prove that sort of better statement. So our inability to prove it led to an improvement in the result. Okay, now you can take the original lemma I had that the models weren't able to prove, put it into chat GPT 5.6 Pro, and it will output the worst proof you've ever seen, like 10 pages of just brutal calculation with no insight whatsoever. this would have been a perfectly fine proof, but it would not have led to this discovery of, like, I think a kind of beautiful conceptual explanation for why this thing discovered was true. So, like, we found a better proof because we couldn't do, I mean, okay, I say we couldn't do the recognition, like, what actually happened. And here I understand that I'm being hypocritical about my complaints about ugly proofs before. It's like, I realized, like, this horrible grind proof would work. Yeah. And I, like, could not bring myself to do it. And so I look for another argument. And so I mean, it's not, yeah, but noon and out, the models can do very long technical calculations pretty reliably. Yeah. Well, I mean, to your credit, I don't know. want to understand. And if understanding just means it's a little bit simpler or more compressed than I've gotten understanding. I mean, it's just like a deep philosophical question too. There's a reason we do informal mathematics rather than writing out long informal strings and symbols of CFC or whatever. Yeah, yeah, yeah. Even though, you know, besides length, it's just like somehow we're trying to put it in a way where we are actually getting some non-rigorous understanding out. Yeah. I mean, and... I think somehow. Yeah, and even how that even relates to like why it helps... And again, it's like, is it because of our inability to grind? Or is it because like, if we are pushing towards something that's more compressive, it also, hopefully also ascends on the does it, help understand other fields as well. And it's just, it is somewhat magical that when we try to optimize for the both, that it tends to coincide. I don't know, actually, you know, if that's a fair, it's actually a quantitative statement, but like why, you know, that is, is kind of magical. It's at least sometimes true, yeah. Yeah, exactly. Or, you know, we certainly bias towards the cases in which it is. That's why we call that a good theory to build. But it's, yeah, it's very much, I think, you know, um, It's like this is what we can study and understand, and therefore it's also very convenient that it was rich in kind of mathematical results there. Yeah, I mean, okay, so... I think there's two things that we can like... you know, pull on, which is like what, given you've seated that, or not seated, but you've believe that mathematical ability of AI is going to continue advancing and can do some of the things that you're, you know, doing nowadays, how should, like, the mathematics community kind of best adapt and benefit from this? I mean, as a, just kind of saying, as somebody who's, you know, doesn't have the time to kind of practice mathematics anymore, this is kind of very great because I can maybe dabble more. There's a lot of results that can come out, but I can also see the more precise point of we can not motivate the right kind of behavior of understanding and development. And so we'd love to kind of hear more about your views there. First of all, it is clearly really exciting that there are increasingly capable models that are producing high-quality results. At least some high-quality results. Also a lot of stuff. Some good stuff. As the models get really capable, my hope is that they will answer a lot of the questions that I've kept up at night thinking about. I'll get to learn the answers. It's definitely exciting. A lot of people got into math. like largely because they enjoyed learning math. Yeah. Right. Like the first thing you do as a math student is you like learn stuff that other people do. And you do that for like 20 years before you before you start doing maybe not quite 20 years. 15 years. If you're lucky 20 years. Yeah. You started early. Yeah. Yeah. Yeah. Yeah, so... That said, like, the goal of mathematics is not to produce mathematics papers. like it's to produce some kind of understanding. So maybe, okay, maybe that some of that understanding resides in model weights or something. [35:00] thing. To me, that's pretty unsatisfying. My own, my motivation for doing mathematics is I would like to satisfy my own personal curiosity. I think people should be have the capability to do that. And that requires a pretty substantial apparatus. It's simply not the case that you can study the questions I think are fundamental unless you've invested a huge amount of time and effort. kind of getting to the point where you can meaningfully do so. And then moreover, even the people, you have the small group of people doing research math on the frontier, or whatever, that relies on a huge apparatus of thousands and millions or billions of people who are trying to learn to think mathematically. Like you need an entire mathematical community to support a small group of people who are on the frontier. Like you just, without the pipeline, the pipeline doesn't exist. So if you think that's important, like development of human capital to kind of meaningfully engage with frontier mathematics, I think that you still have to incentivize those people to actually invest their time and effort getting to the point where they can engage and then do so in a high quality, meaningful way. So right now, I think the existing incentive structures are... For math research, do not encourage people to do that. So right now, if you're a postdoc on the market, you want to get a job, maybe for the next couple years before the community adapts, the best way to do that is like, you want to produce a lot of papers, which maybe prove, you know, old conjectures or whatever. And you can do that by playing the slot machine until, you know, um, The model produces a hopefully correct proof of such a result. So you don't even have to pick the theorem in advance. So here's an experiment you can do. You can take codecs, you can say, go online, find five recent conjectures in algebraic geometry and prove them. And okay, I've run this experiment and with some back and forth, I was able to, in an hour, get three quite bad papers, but correct papers. which is okay, I'm now sitting on my hard drive waiting for me to email the relevant people, but this is not a good use of my time to invest in results. But yeah, you definitely see people doing this. So there's been a huge uptick in post archive, mostly not very interesting. Some of it is interesting. But a lot of it is kind of clearly... low quality even if insofar as the Even if the result is something that would have been highly rewarded a year ago, just like there's no evidence that a human being is actually engaged with it. There's no development of human capital or understanding. We've seen examples where three or four or five papers with the exact same proof of the exact same theorem have come out within a couple days of each other, which is clearly some situation where someone's playing the slot machine. ChatGPT is consistently finding the same. That's also interesting. It's kind of mode collapsed on certain paths of reasoning. I think it's not obvious that this problem goes away as the models get better. Maybe it does, but maybe it doesn't. What I was saying earlier is that a lot of progress in mathematics comes from like letting, you know, a thousand different flowers bloom and people pursue their own curiosity. And then, you know, the boundaries of knowledge expand in some kind of fairly, hopefully fairly uniform way. And look at this really high dimensional space of mathematics. And then like opportunistically, you suddenly get some applications or like answers to old questions we found. And it's like not clear to me that if, you know. we kind of subordinate mathematical exploration to what the model want to pursue. Like if what you're getting is like one mathematician duplicated a thousand times, or like actually, you know, a million different mathematicians doing a million different things. I think actually that's pretty, I mean, aside from like, okay, math has to adapt by changing incentive structures. I think this is actually a pretty maybe dangerous or something that the labs have to pay attention to. Because on one side, what's been successful for them is that this emergent reasoning capability is obviously incredibly powerful. And we've been, I mean, it's just created a lot of great PR headlines. like human mathematicians, they are coming from all sorts of weird intuitions that like, you know, and that is why you end up developing, you know, to your point, the frontier that is so diverse that you actually can draw from and then be able to connect and produce a lot more. And so it, If it's true that most of these proofs that are being pushed out by the labs are converging on very similar things, because they are technically drawing on the same body of literature, and that's where they're strong at now, it's not clear that, I mean, one, where are they developing their intuitions? It's from practicing mathematicians. It's not clear where that other emergent, stronger, diverse intuition might be coming from. It might come, but I don't have a good theory for where it comes from. And it's also just not clear that, you know, just with like test time compute and various [40:00] post-training scaling that you can even, I mean, it's a huge debate whether you could introduce new capabilities. And it's like precisely, I think you should be studied in the math context, because like where it's good at and where it fails, and where human mathematicians are good is exactly a very kind of like, precise question to study it on. So this is like a long way of saying that I, it isn't clear to me that we'll maybe produce, um, if we don't incentivize enough people to interact with it, maybe unlike other domains, it might not actually continue producing a superior result without human aid. Yeah. Let me actually push a bit further even. So let's suppose the models become really robustly superhuman, like we're not even adding meaningful cognitive diversity. I claim still, actually, we still want human mathematicians. how we're designing society, right? Like, you may be... The optimal situation is like you have the models doing all sorts of math research. And like that leads to very, you know, in this like highly, you know, whatever, non-uniform, diverse way. Like maybe you don't need humans to add that kind of diversity. So maybe that's the optimal thing to do. And like in the end, that leads to lots of applications and lots of improved understanding and so on. It's like despite something being optimal doesn't mean you do it. Right. Like there's no reason to think that, you know, if we hand over control of whatever math research to the models and just let them do their own thing, that it will do the optimal thing. And in fact, like if you, you know, if we are, if we kind of instrumentalize what we want them to do, like we want to say, like, oh, you know, make our life better or whatever, it might not be the case like that, like what they decide to do that is like is is pursue a wide variety of interesting research. Right. Like they might just try to take the direct path. We don't know what's going to happen. whatever or the models can be doing like i think the easiest way to guarantee it happens is like to keep a community with like broad interests we're like pushing the models to do and like helping us to design a society like that's what we're what we're pushing for Like I think at least, you know, my hopeful vision for the future is that like humans are not like totally disempowered. Like we have control over where we're going. And like, if that's the case, like what we end up doing is going to be driven by human interests. And so you want to have people who have lots of different interests and also the capabilities to actually pursue them. You want people who are smart and engaged and can do not just mathematical thinking, but all sorts of thinking. Yeah, yeah. I mean, yeah, a great fear is as AI advances, we don't develop the right ergonomic interfaces to actually encourage us to also continue to be good thinkers. And it's so easy to kind of relinquish that because you just off. structure, but despite that, it's so easy to, and so especially for math, I mean, just like, you know, another selfish reason is, um, if you kind of math max, you know, I think it's actually a great pedagogical excuse to actually get just really rigorous at thinking about various things. I mean, this is not why mathematicians do it, but as just somebody... It's part of what we do. Okay, great. Because it was kind of... I thought it just really helped give me a very... It's a good framework to think about many things, not just mathematics. And as somebody who's also now a parent of a two-year-old, I kind of think about this a lot, too. It's not about kind of grinding, even though, whatever, it's still good, but not to shit on grinding too much. I used to collaborate a bunch with some Hungarian mathematicians, Balaz Zegedi among them, and I heard that... In Budapest, they would just teach group theory when you're in primary school. And I'm like, well, we should definitely do that. We should continue doing that. Now that AI is somewhat good at explaining, but it's far more accessible, we should actually proliferate that even more. And so maybe that helps with bringing more people to the frontier rather than, you know, just. Yeah, I mean, this is something I'm concerned about, right? Of course, I mostly talk about math because that's like where I live. But like, I think, you know, one nice thing about thinking about this is that we're one of the first professions to kind of, you know, have a significant impact of high quality models. Although I think. Maybe we're one of the first professions for it to happen to publicly. But, you know, my sense is that there are plenty of other professions that are... Coding. 100%. Coding. But also just like, I mean, I think that, you know, anything you do at a computer, like probably a huge amount is being done by the models at this point. And then like, they're not having a public reckoning about it. But, you know, because math capabilities are useful for the labs to talk about, I think, a bit more publicly than everyone else. [45:00] but yeah I think one reason to try to maintain human capital in this area is just like it's a model for all professions like presumably we still want people who are like meaningfully engaging with the world and like experts and you know have like talents and trained skills and so on yeah so it's actually very convenient that the math profession is so entwined with education here because I think we're also seeing like you know some amount of challenges college and like secondary education coming from AI2. So as you said, I mean, it's also an amazing tool to learn, so. Yeah. People have, I've heard people start talking about a bimodal distribution in their classes where there's some people who are really like figuring out how to take advantage of new tools and other people who are just like letting them do their homework and then bombing everything else. Unfortunately, I don't think that adapts fast enough, but it's like we definitely want to be living in a world where we're producing better thinkers. I think it's just, you know, even if we talk about just the pure kind of optimization game, I think that's better for us. inconvenient if we became worse thinkers just as AI ascends. And it's too easy to let that happen. So we should kind of be thinking hard on how to actually take advantage of this and harness it for our own improvement. Right. Yeah, I think it's like, it's sort of interesting to observe, like, the models at their current level capabilities let you do a lot more. They let you do a lot of things you wouldn't have done more cheaply than, you know, cheaply enough to do them now. the quality of outputs. Yeah. And I think this is common, like you have a new technology that's doing something a little bit worse than was previously done, but much cheaper. And so you get a lot of suddenly a lot of like low quality outputs that are displacing previous high quality outputs. But I think it's possible to use the tools in a way that actually like improves, you know, the quality among every dimension. It just requires some thoughtfulness and some redesign of institutions to actually incentivize that. Yeah. Well, Hopefully capitalism works there. I do feel like the most high value things do require people to work. Use it. effectively. And right now the models are not good enough without the human experts to actually participate. But to your point, there's a vast majority of maybe more junior and entry level. And it's harder for those roles to adapt as well. And so the thing that would be a mistake is to use the AI models in a way that doesn't Basically, you need to be ascending and using the models to deepen your understanding. And it's so easy for human nature just to be lazy. And you have to resist that because that is the moment that you will kind of lose, basically. And so you kind of, you know, kind of forgive the very competitive language, but it really is that like it's just so easy to kind of relinquish thinking to the models. The models can't really think. those facilities rather than relinquish. Yeah. Yeah. Yeah, I mean, one thing I think has been nice about, you know, sort of this vast increase in like semi-expert attention or like model attention on math problems is like now there's been, you know, I've been complaining about slot papers or whatever. Like, you know, people who are not, you know, producing really high quality stuff. Often that's actually coming from professionals. Like it's not saying like, you know, people who, you know, like within academic mathematics, there are incentives to produce like a lot of stuff. that's one place that slop comes from. So there's definitely also slop coming from non-experts, but that I kind of actually don't see as a net negative. Like, okay, now there's a lot of documents on the internet one might have to come through to figure out if a problem has been solved or not. But to me, it seems like just the fact that there's lots of people excited about math is kind of a positive. So that's a nice thing. I totally agree. I know, I get to talk to you. Not even kind of a positive, obviously. Yeah, yeah, no, exactly. It's like suddenly there's a spotlight on it about math more. I was actually kind of curious, if you've got any comments on the... I guess this is another constructive result, but the elliptic curve of rank 30, that just came out yesterday. That's right. Yeah, yeah. That's cool. We don't have any details about it yet. I know, there's rumors that it's... Yeah, we don't know how it... I mean, so it's due to, I guess, Claude Fable, prompted by Leventhal-Poge, and a collider whose name I unfortunately forget. Maybe you can add it in post. Yeah, I just know the Twitter handle. Yeah, we can add a little bit of it. nice recent results have come from Levent with unclear amounts of autonomy. So my sense is that some of them are semi-autonomous rather than fully autonomous. Yeah, with this, we don't know anything about the methods. So yeah, this is a good example. [50:00] fun construction. Without knowing about the methods, it's very hard to say how significant it is. What I would say is that if you want to understand, um, And if historically such results have been proved, or sorry, have been understood by the community, they're like cool, but I wouldn't say they're like a big deal. So like the typical place a result like this might go is like someone's website of records. Mm-hmm. It's not an annals level result. Yeah, it's not an annals. But that said, it's cool. And it's like, you know, it definitely, there were a few... very, very, or are a few very, very talented mathematicians who like these kinds of questions. So, no, Malky's being maybe the most famous example. So, Alkys and Clagsbrun are the ones who kind of have been pushing this record for a while. And they recently found a rank 29 example. Yeah, one up. Which was, you know, the previous record. Yeah, yeah. You know, so those are mathematicians. And I think, you know, people like it. And it's cool now that the models can do this sort of thing. But yeah, it's very, you know, one thing I always say about a model result is like, you cannot evaluate it except in retrospect. This is also true of human mathematics. Sometimes a problem we thought was really important or would require really deep new ideas does not. And sometimes it does, and you can't really know in advance. So it's always exciting when a problem like that gets solved. But then to figure out how significant it is, you still have to look and Levent and Claude and I think Ava Howell, maybe is the third collaborator, have not yet told us how they did it. Have they been mostly more secretive? I think they released some traces for stuff. unless I missed it. Yeah, they, you know, Levent likes to tweet out his results. But yeah, he has been, you know, slowly releasing some kind of PDF write-ups too. I think it's, you know, he's just having fun on the internet. Yeah, yeah. Um, It reminds me of when you're saying, you know, you have to evaluate how the results came. So I had Mark Zelke and Metab Swani on from OpenAI recently. And they were saying how, you know, what's kind of been the most charming or delightful is just that the proofs have been relatively short. They're not like 200 page and, you know, maybe corresponding to your grinding point as well. But maybe this is kind of, optimized for, in retrospect, it was picked that it was short? Or maybe, do you find that on average, stuff that you throw at GPT, you know, Sol or Fable tends to be shorter and more kind of legible to the human or versus it might just go haywire and just grind it out? Yeah, so I mean, I think it is nice that they'll sometimes produce short cover proofs. Yeah. So that's, of course, everyone likes a short cover proof. I think my sense is that, The reason they're not producing long, complicated proofs is that they cannot. Just like the ability to check correctness is not yet there. Um... So, you know, I mean, even actually, you know, if you ask the models to produce short proof, you can then often ask them to just ask, is that correct? And they will often say no. So they're much more reliable than they were six months ago, for sure. But like, it's still, you know, They will still sometimes produce things that are just wrong and they know they're wrong. I think the problem with producing a very long thing is they might not know they're wrong. But I wonder if, you know, presumably internally OpenAI and Anthropic have probably solved a lot more problems than they've released. And I imagine quite a few of them are they're just not sure if they're true. Um, so for example, uh, With this recent list of 10 problems released by OpenAI, those were all formalized in Lean, which is, of course, a very good evidence that they're true. I have no doubt that there were a lot more that they could not formalize in Lean because the prerequisite options have not been put into MassLit yet, for example. There were probably more that were longer, which also makes it challenging to check. So, yeah, this is my guess. And we do actually see very long... AI-generated proofs on the archive. So, for example, someone recently posted a claimed proof of resolution of singularities and positive characteristics, which was 800 AI-generated pages. It's definitely, I mean, I'm sorry, I haven't read it. I haven't done an error, but there's no way it's correct. Like, this will be a major result. It's just not with the capacities of the current models, which are reasonably well-calculated. Yeah, yeah, yeah. And it's definitely no human has read it. Definitely the models are not able to check this kind of thing, yeah. Yeah. So, okay. Well, I mean... Yeah, so this is my expectation, is that the reason it's producing short, clever things is just like, that's what we can check. Yeah. And you can get it to produce long things that are grindy or hard to check, but. Yeah, we'll have to get there. This is actually more what it says about capabilities, frontier capabilities, in a perhaps more negative way, rather than, "Hey, it's just so good at these clever." No, I think that's totally fair. I mean, especially [55:00] if you look at an adjacent domain like code, right? I remember reading something that Cursor put out about testing their long horizon harness. In this case, they're trying to reproduce SQLite in Rust, and it was just so telling how far we are. And that seems like a very comparable task. It's very long. You have to make sure. There's something to verify that it's a correct implementation by testing a suite of cases, but You know, you actually need a harness there. In this case, it's not just like the raw models. And it takes a while and you can actually compare with different frontier versus, you know, non-frontier models. Who's the planner, et cetera. Like differences in their capabilities as well. Yeah. I was thinking in practice, like the to elicit a long proof, you kind of need a harness. Yeah. And yeah. When you make a harness whose goal is to elicit a proof, I think it often decreases reliability because you're just trying to produce output. So like 10GPT 5.6 Pro is like very, it really tries not to say wrong stuff. for example. And although it happens and then you'll ask it like, oh, was that correct? And it'll say no. But, you know, when you're trying to get it to ideate, like you try to get it to be creative, you try to get it out of this very like... rigorous rut in order to actually get it somewhere. And I think if your interest is in, I mean, there are a lot of people who are trying to arbitrage the prestige mechanics of academic mathematics and trying to elicit a lot of proofs, which are not necessarily being checked. And in order to do that, I think you just decrease the liability in order to get a lot of stuff. So, you know, I think any harness that can elicit a 250-page paper is probably not being very careful about what it's producing. where it's not reliable just because the capability isn't there yet. And so it's just kind of forcing a longer horizon task on it. Yeah. I mean, in practice, how does a human checking it, can't check a 250-page paper either. You can't reliably check it line by line. What you try to do to understand it is you try to understand the overall global structure of the argument, and you stress test it in various ways. Would this argument imply something else that I know to be false? Oh, does it work in this special case? Blah, blah, blah. Models seem not to be able to do that kind of like more... I don't know, fuzzy, like, unit testing approved very well yet. It's actually, like, one of my favorite tests for the models, which they haven't succeeded at yet, is there's a paper, I won't name it, that came out a couple years ago that was wrong, um, And it was like very hard to find like in close enough area to myself that I like immediately like it was claiming some big result. I immediately downloaded it and like started reading it. And it was like very hard to find the actual specific error. But it was also very clear from the structure of the argument that it couldn't work. So it was like, you know, me and a bunch of other, like it was proving something that was too strong to be true. Oh, I see. A little bit further. Yeah. the author and like we kind of went back and forth until someone um figured out what the precise specific error was okay um and so far the models seem not to have been able to do this like the specific error is quite subtle and but they're also not able to do this kind of overall uh kind of big picture tech checking which is kind of like how people in practice check papers yeah yeah which is kind of it mirrors you know why in code it's like it's so clear it's good at the syntax but Maybe I'll ascend there, but probably need some harness help. Who knows? I mean, people have, you know, evolving opinions on how much the harness and the model have to co-evolve and which one is necessary, but the next model requires less. So I feel like in math, it'd be very interesting to see how you, if you do any experiments with the harness there and how that improves, because it is a marker of general reasoning. Yeah, I mean, I do, you know, I have my own sort of bad little harnacks and codecs. Oh, yeah. Harnacks and codecs, but yeah, it's, but you know, autonomous mathematics very much. So I mostly do not use the harness. I mostly try to use it to help me understand stuff. Okay, no, it's totally fair. Yeah, exactly. You don't want to automate your job away because that does involve you being in the loop to understand it, which is, you know, necessary to participate. Maybe to finish off, I'd love to, this could be just something you haven't thought about or actually thought a lot about. I think you also have a toddler. Yeah. Right? Yeah. Yeah. Yeah. And so how have you. Yeah. Three year old. Great. Great. So you're one year more advanced and probably, you know, have more thoughts on this. Like, how are you thinking about, is it her education or his education? How are you thinking about her education in math and. Not to grind, but really just pass on the love of it and how to react to AI. Yeah, so she's three. She's never used [60:00] AI. She is starting to add. That's about as far as you are in math. That's more far. She can add single digit numbers by counting on her fingers. Nice. 30 reliably and 50 semi-reliably. So I'm very proud of that. Yeah, I definitely encourage that. We talk about shapes and stuff. Actually, a couple days ago, I woke her up and she was like hiding under the blankets. And I was like, oh, what are you up to under there, Sophia? And she was like, oh, I'm doing some math. Oh, I saw that. I think you tweeted about that. That was a good word. Yeah, it was great. So, you know, I think she has like some sense that I like math and she's into it because of that. Yeah, I mean, I would say I think the world is probably going to look pretty different in 20 years or whenever she's fully adult and doing her own thing. But I think a lot of what we educate people for is pretty robust changes in the nature of the world. I think the reason to learn math has always been to think clearly and better understand the world. And presumably that's something you want to do. even if they're extremely capable AIs. And this is also true, you know, I personally like math a lot, but also the reason to read a lot of books and do the humanities and so on. So I think the actual values of the math profession and education more broadly are things we definitely want to try to preserve that I hope to instill in my daughter. How much institutions have to change to make sure that happens is maybe an open question, I think, a lot. But yeah, at least at a personal level, I'm definitely trying to convince my three-year-old that math is super cool. Oh, yeah, 100%. One of her first words was icosahedron. Was it what? My parents gave her a little icosahedron toy when she was one. That's the first word. Yeah, not literally the first word, but she learned the platonic solids quite quickly. Next up, group theory. It's very natural. When I teach her about addition and subtraction further, we'll do it in the context of a general group. Oh, very good. Well, at least you give the motivation. I think a lot of folks probably skipped that part. Yeah, yeah. math rather than just understanding. Wonderful. Okay. Well, thank you so much, Daniel. Um, thank you. It was a lot of fun. Yeah. Yeah. It was a lot of fun. And yeah, I mean, I think there's going to be a lot more progress very soon. I'd love to maybe catch up and chat again. Sounds great. to Apple Podcasts and Spotify. Follow us on X and A16Z and subscribe to our substack at a16z.substack.com. Thanks again for listening and I'll see you in the next episode. in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures. you