DeepMind's Ghahramani Sees Bayesian Math Fixing Flawed AI Scale

DeepMind的Ghahramani认为贝叶斯数学可解决AI规模化缺陷 若缺乏能明确表征并更新不确定性的数学模型，仅靠算力规模扩张无法消除关键人工智能系统中危险的长尾失效。 在本次对话中，剑桥大学教授兼Google DeepMind研究副总裁Zoubin Ghahramani认为，真正的通用人工智能需要对不确定性进行数学表征和校准，而不能仅仅依赖原始算力和数据规模。他将偶然的物理随机性与认知上的知识缺陷进行了对比，阐明了现代大型语言模型如何仅通过下一个token预测来模拟置信度，而缺乏真正的自我怀疑。Ghahramani重点提到了AlphaFold和GenCast天气预报模型中成功的概率化实现，概述了贝叶斯更新、持续学习和高效近似算法如何能够克服极端情况下的失效问题。讨论最终强调，构建值得信赖的AI意味着要超越人类的认知偏差，打造出能诚实认知自身未知领域的系统。 真正的智能需要能够表征和更新不确定性的结构化架构，而不能仅仅依赖数据和算力的规模扩展。 (0:00) 当前的大型语言模型缺乏内部置信度表征，将信念分散在数十亿权重中，并通过下一个token预测伪造确定性。 (14:48) Google DeepMind的GenCast和AlphaFold等系统表明，将明确的不确定性整合到预测模型中，能产生更可靠的现实应用结果。 (26:00) 精确贝叶斯推断在计算上是难解的（NP-hard），但现代硬件的进步以及数十年来发展的高效近似算法，使得贝叶斯与深度学习的混合架构如今具有了可行性。 (40:00)

Professor Zoubin Ghahramani argues that achieving artificial general intelligence requires moving beyond brute-force compute and parameter scaling to incorporate explicit mathematical representations of uncertainty. For artificial intelligence to make reliable real-world decisions, systems must possess calibrated self-doubt, systematically distinguishing what they know from what they do not know. Calibrating Confidence Beyond Benchmark Accuracy (2:30) Ghahramani and Hannah Fry emphasize that real-world intelligence depends on decision-making under incomplete information. They divide uncertainty into aleatoric (irreducible physical randomness, such as an erratic pedestrian or a coin toss) and epistemic (a subjective lack of knowledge reducible by gathering data, measured in information theory as bits). An autonomous vehicle encountering an unusual corner case—such as a horse leaping into the road during a violent hailstorm—must register high epistemic uncertainty and slow down, rather than misclassifying the obstacle with unearned confidence. Standard neural network benchmarks often obscure dangerous overconfidence. Ghahramani notes that imperceptibly perturbing a few pixels of a school bus image can cause an image classifier to identify it as a cheetah or a monkey with 99% statistical confidence. While unconscious human perception naturally combines prior expectations with noisy sensory cues—such as inferring a predator threat when hearing rustling leaves in Costa Rica—conscious human reasoning is notoriously susceptible to cognitive fallacies. Ghahramani contends that artificial systems should not imitate human cognitive flaws, but instead achieve rigorous calibration where stated confidence matches true empirical accuracy. The Limits of Next-Token Prediction (20:00) Current large language models compute probability distributions over next tokens, but Ghahramani argues this does not amount to a careful representation of uncertainty. Because beliefs are smeared across hundreds of billions of parameters rather than tracked in an explicit module, models merely mimic confident reasoning. When challenged by users, they often apologize and flip-flop abruptly, exposing an absence of genuine conviction. Ghahramani compares training models solely on text co-occurrence to constructing a calculator by memorizing arithmetic examples: the system mimics solutions but lacks guaranteed computational mechanics when encountering novel numbers. While techniques like semantic entropy evaluate distribution sharpness across candidate tokens to surface ambiguity, principled reasoning requires updating prior beliefs using Bayes' rule as new evidence arrives. Without explicit Bayesian updating mechanisms, models trained on collective web text remain an ungrounded soup of contradictory human assertions. Validating Probabilistic Models in Real-World Domains (26:00) Probabilistic modeling already demonstrates practical advantages over deterministic systems in scientific domains. Google DeepMind’s GenCast model forecasts medium-range global weather up to 15 days ahead in roughly eight minutes by combining neural networks with diffusion models, dramatically outpacing classical physics simulations. When tracking events like Hurricane Melissa, GenCast generates an ensemble of forecast tracks and applies Bayesian updating as fresh satellite and sensor observations arrive, accounting for chaotic dynamics as well as sensor fallibility. Similarly, AlphaFold embeds uncertainty directly into its outputs by color-coding confidence scores for each amino acid in a predicted protein structure. Because physical proteins naturally fluctuate and predictive models rely on inference rather than direct observation, displaying this structural variation prevents unwarranted trust. Ghahramani emphasizes that calibration requires long-term consistency: a system asserting 70% confidence must fail exactly 30% of the time, while Bayesian probabilities provide a mathematically sound method to quantify uncertainty for unique, one-off events. Computational Intractability and Frontiers Beyond Scale (35:00) The primary historical obstacle to deploying Bayesian AI has been computational complexity: exact Bayesian inference is NP-hard or NP-complete, making full calculations practically intractable. However, Ghahramani argues that massive shifts in hardware alter this equation. A contemporary Google Pixel smartphone provides more computational capability than the 65,000-processor Connection Machine supercomputer he used in 1989, allowing modern models to pair deep learning scale with decades of mathematical approximation techniques. To move beyond the limitations of raw scaling on the path to artificial general intelligence, Ghahramani identifies four essential research priorities: continual learning that updates weights sequentially without catastrophic forgetting; radical energy efficiency to approach the human brain's 20-watt baseline; hardware-software co-design using sparse networks; and data efficiency that enables systems to build comprehensive world models from sparse observations. While mainstream machine learning has long prioritized raw accuracy benchmarks, Ghahramani and Fry conclude that true reliability depends on designing architectures with epistemic humility. Bridging the gap between theoretical Bayesian inference and tractable computational approximations will allow artificial agents to express calibrated self-doubt, making them safer partners in safety-critical domains.

剑桥大学教授兼人工智能学者邹宾·加拉马尼（Zoubin Ghahramani）指出，真正的通用人工智能无法仅靠算力和参数规模的扩张来实现，其核心缺失在于对不确定性的数学表达与量化能力。智能系统必须掌握校准自我怀疑的机制，在不完整信息下做出审慎且理性的决策，而不是在面对未知时表现出盲目的过度自信。 偶发与认知不确定性及现实决策 (2:30) 加拉马尼认为智能的本质是在感知输入不完整和无法预知未来的现实中进行决策。汉娜·弗莱（Hannah Fry）与加拉马尼将不确定性划分为两类：一是偶发不确定性（Aleatoric uncertainty），属于物理世界中不可削减的随机性（如抛硬币或行人突然变道），系统只能接受而非强行预测；二是认知不确定性（Epistemic uncertainty），源自数据或知识不足（如暴风雨中路面突然出现活马），系统应当意识到自身无知并减速或主动采集数据。在信息论与计算机科学中，获取1比特信息即意味着将观察者的不确定性减少一半。 加拉马尼以自动驾驶汽车为例说明，模型若缺乏对场景新颖性的感知机制，可能在长尾极端场景中把马误判为自行车且毫不迟疑地前行，而具备自我认知的系统则会识别出高不确定性并采取保守减速策略。人类在生物进化中形成了出色的无意识感知推理（例如在哥斯达黎加丛林中听到枝叶摩擦声便能结合先验知识推断美洲豹威胁），但丹尼尔·卡尼曼与阿莫斯·特沃斯基的研究表明，人类在意识层面的概率推理极易受偏见影响。因此，理想的人工智能在概率处理和理性决策上应当比人类更加严谨。 大模型的虚假自信与对抗样本缺陷 (7:15) 在图像分类领域，对抗样本揭示了当前神经网络的致命脆弱性：仅通过微调人类完全察觉不到的极少数像素，模型就会以高达99%的置信度将校车指认为猎豹或猴子。加拉马尼强调，基准测试中的“正确率”与系统的“置信度校准”不可混为一谈，过度自信的错误往往极度危险。现代大语言模型通过海量网络文本训练，本质上是在预测下一个词元的概率分布，并将信念隐式分散于数千亿个权重激活中，内部缺乏明确、可解释且符合数学规范的置信度记录变量。 弗莱和加拉马尼指出，当前大模型展现出的权威口吻实为一种对人类推理痕迹的语言模仿，其信心极度脆弱——一旦用户提出轻度质疑，模型往往立刻妥协道歉。这种缺乏连贯信念与因果理解的“混合浓汤”不仅滋生幻觉，也无法胜任物理世界中具身智能（如机器人和自动驾驶）的安全要求。尽管借助语义熵（Semantic Entropy）等指标可以分析候选词元分布的离散程度来衡量模型在特定问答中的确定性，但加拉马尼警告，纯粹依赖文本共现模式犹如通过背诵加乘案例来伪造计算器，真正的计算器需要具备真实的数学运算能力。 贝叶斯推理与天气预报中的成功范例 (15:47) 加拉马尼用侦探破案的经典案例解释了贝叶斯法则：侦探根据初始嫌疑设定先验概率分布，当现场发现凶器等新证据时，将先验与似然相乘并归一化为后验概率，随后后验概率成为下一轮推理的新先验。贝叶斯框架既能解释感知推理，也能解释机器学习从先验参数随数据演进的核心过程。他在2015年发表于《自然》（Nature）的论文中曾提出，智能的核心要素关键依赖于对不确定性的审慎概率表达。 谷歌DeepMind推出的GenCast中程天气预报模型展现了贝叶斯不确定性建模的实际威力。该模型结合神经网络与扩散模型操作随时间演变的概率分布，仅用约8分钟便能生成全球未来15天的气象集合预报，远快于超级计算机上运行数小时的传统物理模拟。在预测飓风路径时，GenCast不仅能呈现混沌非线性动力学（蝴蝶效应）带来的物理不确定性，还能利用贝叶斯推断融合后续卫星和地面传感器数据，校正由传感器稀疏或故障引起的认知不确定性。弗莱指出，这种在建模中主动容纳不确定性的做法，反而显著提升了最终预测的准确度与稳健性。 置信度校准、AlphaFold与算力权衡 (30:00) 在科学研究与高危决策中，准确传达怀疑至关重要。加拉马尼列举了AlphaFold的案例：蛋白质在物理现实中存在天然摆动区域，且预测模型并非直接实验观测，因此AlphaFold在输出三维分子折叠结构的同时，采用颜色编码标示每个氨基酸的置信度，呈现分子存在的“预测云”。对于可重复事件，置信度校准要求概率数值与长期频率严格对应（例如预报70%降水概率时，历史累积需有30%的时间不降水）；而对于人类首次登陆火星等单次历史事件，贝叶斯统计依然提供了在数学上赋予主观概率和更新信念的严密工具。 在临床医学等关乎生命的场景中，有效呈现模型的不确定性可以防止医生盲目依赖或彻底弃用AI。加拉马尼援引剑桥大学学者戴维·斯皮格霍尔特（David Spiegelhalter）的风险沟通研究，认为怀疑的表达必须对非专业人员直观可见。面对“规模法则能消除一切不确定性”的主张，加拉马尼列举了迈向AGI必须突破的四大研究方向：基于贝叶斯逐步更新且无灾难性遗忘的持续学习（Continual Learning）、向人脑约20瓦能耗看齐的能效改进、软硬件协同设计的稀疏激活新架构，以及摆脱对数千亿词元依赖的数据高效率机制。 贝叶斯方法虽然在理论上能赋予系统谦逊、数据效率与持续学习能力，但其严格精确计算属于NP难（NP-hard）或NP完全问题，导致过去数十年工业界不得不妥协于纯数据驱动的函数拟合。然而，加拉马尼认为现代计算硬件相较于1980年代的连接机（Connection Machine）已发生飞跃，配合数十年积累的数学近似方法，将深度学习规模与贝叶斯严谨性相结合已具备现实可行性。人工智能的终极目标不应是模仿人类容易出现偏差的认知直觉，而应超越人类弱点，构建知晓自身局限、诚实可信的决策系统。

## Background 本内容由 Hannah Fry 与剑桥大学教授、Google DeepMind 前沿 AI 联席负责人 Zoubin Ghahramani 于 2026 年 8 月 26 日通过播客平台发布。页面简介指出，当前的 AI 面对提问即便给出错误答案也往往带着绝对的权威口吻，似乎缺失了人类的核心特质——自我怀疑；节目探讨了改善机器的不确定性表征是否正是推动 AI 持续进化的关键拼图。 ## TL;DR 节目围绕“不确定性数学与机器智能”展开深入对谈。Zoubin Ghahramani 指出，真正的智能离不开在感知受限的长尾现实中做出决策，而当前主流 AI 陷入了“盲目自信的错误”与概率失准的困境。讨论梳理了从 20 世纪 80 年代神经网络反文化革命、贝叶斯定理在认知与学习中的核心地位，到现代气象大模型 GenCast 与 AlphaFold 如何借助概率校准实现精准预测的演进脉络。Zoubin 强调，面对持续学习与能效瓶颈，单纯依靠规模扩展（Scaling）存在根本天花板，唯有借助贝叶斯概率架构与高效近似算法赋予机器自我怀疑与认知谦逊，才能打造出超越人类认知偏差、在医疗等重大决策中真正值得信赖的理性系统。 ## Atmosphere & style 对谈散发着深厚而迷人的学者气息，既有前沿科学思想交锋的锐利，又不乏老友围炉畅谈般的轻松与幽默。主持人 Hannah Fry 提问机敏生动，常常以绝妙的比喻挑破技术黑盒，而深耕领域三十余年的 Zoubin Ghahramani 则以严谨从容的学者风范引经据典，从 Connection Machine 闪烁的红灯回忆到深刻的数学哲学侃侃而谈。整场对话在破除大模型“全知全能”神话的思辨中展开，兼具历史纵深感与对技术盲从的清醒反思，聆听体验如同一场启迪心智的高阶智力漫游。 ## Bulletin - `00:00` 剑桥大学教授兼 Google DeepMind 前沿 AI 联席负责人 Zoubin Ghahramani 指出，现存 AI 过度聚焦纯规模扩展（pure scale）与单一准确率，忽视了对不确定性的审慎概率表征，导致模型在面临长尾异常与对抗扰动时极易产生盲目自信的严重错误。 - `25:00` DeepMind 的气象预测模型 GenCast 通过结合扩散模型与集成预报，仅用约 8 分钟即可推演未来 15 天的天气走势，证明向模型显式引入并动态更新不确定性反而能大幅提升复杂预测系统的精确度与鲁棒性。 - `37:28` 人脑运行功率仅约 20 瓦，而大型语言模型的训练与运行能耗高出数个数量级；未来 AI 打破数据与算力瓶颈的关键不在于死板重训，而在于结合贝叶斯持续学习与软硬件协同设计的稀疏神经网络架构。 - `32:32` 在医疗诊断与自动驾驶等生死攸关的决策场景中，一个懂得概率校准、能够诚实表达自身无知与怀疑的理性机器，远比一个看似全知却自负翻供的黑盒系统更加安全且值得信赖。 ## Key takeaways - `00:00` 智能在本质上等同于在感知受限与信息不全的条件下做出决策，因此具备对不确定性的表征与动态更新能力是构建真智能的底层基石。 - `03:00` 必须区分源于物理客观随机性的偶然不确定性与源于系统知识盲区的认知不确定性，后者正是驱动智能体采集信息、压缩熵的核心动力。 - `05:00` 人类认知具有双层矛盾：显性意识层面的统计推理极易受启发式偏差干扰，而演化塑造的底层感知系统却在潜意识中精确执行贝叶斯推断。 - `07:30` 深度神经网络普遍存在正确率与置信度的脱节，轻微对抗样本扰动即可引发 99% 置信度的荒谬误判，暴露了缺乏校准能力的系统性隐患。 - `14:48` 当前大语言模型虽然依赖词元概率预测，但缺乏显式的审慎概率表征，一旦遭到质疑极易倒戈妥协，暴露出缺乏稳定的内部信念架构。 - `16:04` 贝叶斯定理提供了一套完备的机器理性更新闭环，通过先验概率乘以似然度归一化为后验概率，将信息增益量化为比特以指导后续认知。 - `20:00` 学术界提出的“语义熵”等前沿指标，旨在通过分析候选词元概率分布的尖锐或平缓状态，来量化并抽取模型内部未明言的确定性程度。 - `25:00` 单纯依赖海量数据拟合相关性犹如通过背诵算式示例制造假计算器，必须从底层理论引入运算法则推理与概率分布处理。 - `30:00` DeepMind 的 AlphaFold 利用多色置信度色标向科研人员直观传达结构预测的准确度，为复杂科学探索提供了诚实的不确定性度量示范。 - `30:00` 严格的概率校准要求长期统计下的预测概率与实际发生频率一致，而针对不可重复的单次事件，贝叶斯框架同样赋予了量化信念的严密数学有效性。 - `32:32` 高危决策场景（如医疗诊断）要求 AI 扮演与人类认知互补赋能的协作者，通过可视化不确定性来防止人类陷入盲从或全盘否定的极端态度。 - `35:00` 现行以批处理、周期性从头训练大模型的范式容易引发灾难性遗忘，而贝叶斯先验-后验更新机制在理论上天然支持终身持续学习。 - `37:28` 现代 AI 严重依赖 Transformer 与扩散模型两大工作母机，缺乏生物大脑仅用极少样本即可提取模式的高效数据利用机制。 - `40:00` 贝叶斯方法长期受困于精确计算的“NP 难”诅咒，但伴随现代硬件算力的飞跃与实用近似算法的发展，贝叶斯架构正在迎来工程破局契机。 - `42:09` 超越人类认知局限的理想 AI 不应仅追求单一确定性输出，而应将自我怀疑、谦逊与诚实作为底层设计，在关键时刻自知无知以支撑重大人类决策。 ## Detailed walkthrough ### `00:00` 不确定性是决策与智能的核心 当前的 AI 面对提问时，哪怕给出的答案完全错误，也往往带着不可动摇的权威语气。在大型语言模型（LLM）席卷全球之前，剑桥大学教授、Google DeepMind 前沿 AI 联席负责人 Zoubin Ghahramani 已经在此领域深耕了 30 年，致力于将不确定性数学融入机器体系。当前 AI 学界正处于一场思辨的核心：一方寄希望于单靠“规模化”（pure scale）解决智能进化的所有问题；而以 Zoubin 为代表的另一方则认为，真正的智能必须依靠架构创新，使机器理解“不确定性”是通向更高智能的关键拼图。 面对“为什么 AI 必须具备不确定性”的核心命题，Zoubin 强调了决策的本源。他指出，智能从本质上离不开做决策，从单细胞细菌、动物、人类到机器人莫不如此。然而在现实世界中，感知系统天然具有局限性，没有任何实体能够全知全能或准确预测未来。因此，一个真正具有智能的系统必须能够对不确定性进行表征（represent）、动态更新（update），并在这种信息不全的模糊状态下做出良好抉择。正如 Zoubin Ghahramani 所言：“To build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty, and then can use that to make good decisions under uncertainty.” ```mermaid flowchart LR accTitle: 不确定性与智能决策回路 accDescr: 展示从感知输入、概率表征、信念更新到最终决策的闭环过程。 subgraph S1["感知与建模阶段"] A["现实世界输入 (物理长尾/传感器盲区)"] --> B["不确定性表征 (映射为概率分布)"] end subgraph S2["计算与决策阶段"] B --> C["概率推断与信念更新 (收集信息/贝叶斯更新)"] C --> D["在不确定状态下做出行动/决策"] end ``` - **智能的本质要求**：无论是生命体还是人造机器，无法脱离决策而谈智能，而现实世界的决策永远建立在感知受限的基础之上。 - **长尾场景的挑战**：自动驾驶汽车即便受过海量数据训练，也难以穷尽极低概率的“长尾”偶发状况（例如在遭遇突发冰雹的同时，眼前突然跳出一匹马）。 - **机器的自我审视能力**：当自动驾驶面对未曾见过的组合场景时，应当具备对自身不确定性的认知并主动减速，而不是自欺欺人地把那匹马盲目分类为一辆自行车。 --- ### `03:00` 区分偶然不确定性与认知不确定性 针对自动驾驶与日常复杂情境，不确定性并不是单一扁平的概念。Zoubin 强调，数学的精妙之处在于能将现实中千奇百怪的不确定性全部映射并转化为概率语言，利用概率理论的完备规则去量化与调整。但对智能体而言，辨别不同类型的不确定性至关重要，因为类型直接决定了行为与决策路径。 一种是**偶然不确定性**（aleatoric uncertainty），源于物理世界固有的随机性（如抛硬币的正反面，或正常街道上普通行人在下一个路口究竟左拐还是右拐）。面对这种系统性的随机噪声，决策系统有时需要主动放弃徒劳的精确预测；另一种则是**认知不确定性**（epistemic uncertainty），指的是系统自身知识或信念状态的不完备。当面对未知事物时，系统理应通过收集更多数据来消除迷雾。Zoubin 提到了计算机科学中关于“信息”的标准定义：1 比特信息（a bit of information）在本质上就是将认知不确定性减少到原本的二分之一，信息的采集正是人类与机器压缩不确定性的核心手段。 - **随机性（偶然不确定性）**：类似硬币起落，系统应当识别出不可预测的随机性，避免做无谓的确定性押注。 - **知识盲区（认知不确定性）**：系统意识到“当前数据不足”，驱动后续的观察与数据采集行为。 - **信息论本质**：计算机科学中的 1 比特（bit）物理含义正是将自身不确定性削减一半（reduction of our uncertainty by a factor of two）。 --- ### `05:00` 认知科学中的人类直觉与潜意识概率推断 结合自己在大学本科攻读计算机科学与认知科学的双重视角，Zoubin 探讨了人类心智对不确定性的处理方式。认知科学界早已广泛采纳概率与不确定性理论来建模人类的感知与决策行为。日常生活中，人类感知物理世界本质上就是一种持续推断的过程：例如我们看不到对方的后脑勺，但脑海中自然会推断出后脑勺的存在；如果在哥斯达黎加的丛林徒步时听到树叶沙沙作响，我们的大脑会迅速结合“哥斯达黎加分布着美洲虎，而伦敦市中心绝不会有”这一先验信念（prior beliefs），感知器官的信号瞬间与环境先验融为一体。 人类在大脑演化中形成了两套处理概率的机制：在显性（explicit）或意识层面上，人类的统计推理能力出了名地糟糕，卡尼曼（Daniel Kahneman）和特沃斯基（Amos Tversky）的研究充分证明人类经常在清醒时犯下严重的概率推断认知偏差，在估算具体概率数值时动辄偏差几个数量级；然而在隐性（implicit）的潜意识与底层感知系统内部，人类却如同极为精确的概率推断机器，这种潜意识的高超推断能力是生存进化的必然产物。类似的情况也发生在婴儿和幼儿身上，幼童虽然根本不知道概率公式与数学方程，但他们通过行动去探索和检验某种结果的信念，这种学习机制与现代 AI 的概率推断框架高度吻合。 - **先验信念（Prior Beliefs）的作用**：感知并非孤立接收外界信号，而是将即时传感器数据（听觉、视觉）与背景先验环境深度结合。 - **双层认知矛盾**：人类显性思考极易出现概率认知偏差（Kahneman & Tversky 的前景理论与启发式偏差），而演化塑造的底层感知系统却在潜意识层面天然遵循贝叶斯推断。 - **幼儿探索模式**：婴儿利用行动驱动信念假说，在无意识状态下完成对物理世界概率分布的建构。 --- ### `07:30` 正确率与置信度的脱节：AI 盲目自信的危机 深入技术本身，Zoubin 辨析了两个经常被混为一谈的概念：**正确率**（correctness）与**置信度**（confidence）。在传统的图像分类任务中，系统不仅需要输出物体名称，更要评估自己对该答案的确信程度。早在十多年前，学界就发现了深度神经网络存在致命的“对抗样本”（adversarial examples）漏洞：研究人员只需对一张校车图片修改几个肉眼不可察觉的像素点，人类肉眼依然能够明确识别出那是一辆黄色校车，但神经网络却会以 99% 的绝对置信度坚称那是一只猎豹（cheetah），甚至能随意被诱导识别为猴子等任何类别。 这种现象证明，单看测试集上的“正确率”是危险的，系统绝不能表现出“盲目自信的错误”（overconfidently wrong）。在如今的大语言模型（LLM）中，过度自信的毛病依然普遍存在：模型在面对质疑时会先用无比确信的口吻给出回答，一旦用户稍作反驳，模型又立刻倒戈推翻前言（flip-flop）。Zoubin 指出，如同人们难以信任现实中盲目自大、毫无谦卑之心的人一样，无法自知知识边界的机器同样无法获得人类的信任。走向通用人工智能（AGI）的征途上，必须赋予 AI 体系关于局限性的自知之明与数学层面的“谦卑”。正如 Zoubin Ghahramani 所强调的：“We don't want systems that can be overconfidently wrong.” | 评估维度 | 核心定义 | 传统深度网络/当前 LLM 的典型表现 | 理想的 AGI 表现 | |:--- |:--- |:--- |:--- | | **正确率 (Correctness)** | 输出结果与客观事实或标签的一致性 | 在标准训练集与常规分布上表现极高，在扰动样本下骤降 | 保持高准确率，并能适应分布外场景 | | **置信度 (Confidence)** | 系统对自身推断正确性的主观概率评估 | 严重校准失衡；面对对抗样本或幻觉问题常给出接近 100% 的盲目置信度 | 良好校准（Well-calibrated）；对不确定场景能主动减速或表达怀疑 | - **对抗样本启示**：轻微的像素扰动暴露出确定性神经网络缺乏真实的置信度校准能力。 - **LLM 的翻供行为**：先是权威式确定作答，被提问后又草率反转，显示出缺乏稳定的认知自洽度。 - **早期 AI 历程**：谈及从事 AI 领域的源起，Zoubin 笑称自己比较“枯燥”，早在 14 岁上世纪 80 年代末期还是个少年时，便一头扎进了人工智能与不确定性的世界。 ### `10:00` 早期神经网络探索与“连接机”上的反文化革命 早在 15 岁时，Zoubin Ghahramani 就下定决心要从事人工智能领域的研究。进入宾夕法尼亚大学后，身为大一新生的他急需一份暑期兼职，于是直接去找了系主任——著名计算语言学家阿尔温德·乔希（Aravind Joshi）。乔希当时交给他两本刚刚出版的新书，即 1986 年问世的《并行分布式处理》（Parallel Distributed Processing，简称 PDP）。这恰好是现代反向传播算法（Backpropagation）论文发表、拉开现代神经网络革命序幕的同一年。乔希开出的工作内容非常纯粹：让邹宾通读这两本书，并把其中的核心思想讲给自己听。对一个充满好奇心的年轻人而言，这堪称世界上最完美的暑假工。 在 20 世纪 80 年代中后期，AI 领域的主流范式是“专家系统”（Expert Systems）。这类基于规则的决策系统往往非常脆弱，而模仿人类大脑机制的神经网络则展现出了极高的灵活性，最根本的突破在于它能够直接从数据中学习，弥补了传统方法的致命缺陷。那场革命不仅吸引了计算机科学家，还汇聚了认知科学、心理学、神经科学及经济学领域的学者。1989 年，Zoubin 撰写了关于利用神经网络解析人类自然语言的本科毕业论文。他笑称自己当时训练的其实是“小语言模型”（Small Language Models）。当时他们使用的硬件极具科幻感——一台标志性的“连接机”（Connection Machine）。那是一个由 65,000 个微处理器组成的黑色不透明立方体，运行时内部有 65,000 颗红色的微型指示灯闪烁不休。在那个专家系统独领风骚的年代，编写并行神经网络代码被视为不折不扣的“反文化”（counterculture）。 - 关键著作：1986 年出版的《Parallel Distributed Processing》（PDP），标志着现代神经网络的萌芽。 - 硬件设备：拥有 65,000 个处理器的 Connection Machine，机身分布着数万颗闪烁的红色 LED 灯。 - 论文年代：1989 年，Zoubin 完成了利用神经网络解析自然语言的本科毕业论文，比当下的大语言模型热潮早了近 40 年。 --- ### `12:18` 转向概率论与贝叶斯：与 Hinton 错过的深度学习浪潮 走在时代前沿的 Zoubin 随后却“走偏了”。到了 20 世纪 90 年代初，学术界普遍认为神经网络的机理已经被彻底搞清楚了：它们本质上就是优秀的函数拟合器（Function Approximators），负责将输入映射到输出（例如将图像映射为标签）。受限于当时的时代背景，万维网尚未普及，数据集极其匮乏，算力也十分薄弱。在此期间，学者们逐渐发现了神经网络与概率论、统计学以及随机过程之间深邃的数学联系。这促使 Zoubin 作出了一个重大转向：暂时搁置纯粹的神经网络，转而全力钻研概率问题。 当主持主持人 Hannah Fry 问及在神经网络最终统治世界后是否会感到懊恼时，Zoubin 坦言自己后来“完成了救赎”。他曾与因神经网络研究荣获诺贝尔物理学奖的杰弗里·辛顿（Geoffrey Hinton）共事多年，一同探索深度学习。然而当时他们走的是一条更为曲折的路——试图构建复杂的概率模型而非简单的神经网络。Zoubin 反思道：“We were overcomplicating things because we hadn't seen what happens with large-scale data.”（我们当时把事情搞得太复杂了，因为我们根本没见识过海量数据释放出的威力。）随后多年，他深耕于贝叶斯机器学习与概率图模型。他从未彻底否定神经网络，只是在当时的认知水平下，纯神经网络的数学机制看似已被研究透彻，不如概率论更具理论探索的魅力。 --- ### `14:48` 2015 年《Nature》论断与 LLM 的隐式概率短板 2015 年，《自然》（Nature）杂志刊登了一期极具里程碑意义的 AI 特刊，其中不仅收录了 Hinton 等人奠定现代深度学习与强化学习基础的综述，也刊登了 Zoubin 撰写的一篇纲领性论文。他在文中提出了一个核心论点，正如 Zoubin 所言：“Many aspects of intelligence depend crucially on the careful probabilistic representation of uncertainty.”（智能的诸多关键维度，都根本性地依赖于对不确定性的审慎概率表征。） 尽管当今的大语言模型（LLM）本质上基于概率运作——其底层任务是预测给定上下文后下一个词元（token）的概率分布，但这并不等同于真正掌握了不确定性。Zoubin 指出，当前模型普遍缺失的是对概率的“审慎且显式的表征”。业界仅仅是寄希望于投喂足够庞大的数据，让模型在黑盒中“凑合”表达概率，却缺乏系统性的严谨设计。如果你深入庞大的神经网络内部，根本找不到一个明确的单元代表“模型认为某件事有多大把握”；这种概率认知被打散并漫布在全网数百亿个神经元的激活状态中，使得模型依然无法可靠地评估自身的无知。 --- ### `16:04` 贝叶斯定理解密：从侦探探案到真正的机器理性 为了解释如何科学地度量不确定性，Zoubin 详细拆解了贝叶斯定理（Bayes' rule）。贝叶斯的核心逻辑非常符合直觉：在获取新证据或数据之前，主体持有某种“先验信念”（Prior Beliefs），并用概率分布来表示它。他用侦探小说中的破案情节打了个生动的比方：案发后有几个嫌疑人，管家看起来贼眉鼠眼，因此具有较高的先验嫌疑；随后警方在食品储藏室找到了凶器，这就是新观察到的“证据”（Evidence）。 利用贝叶斯法则，我们将先验信念乘以“似然度”（Likelihood，即不同嫌犯留下该证据的概率），然后进行归一化处理（确保所有可能性之和为 1），最终推导出“后验信念”（Posterior Beliefs），即更新后的认知状态。在这个过程中，不确定性的下降幅度可以用“比特”（bits）精确量化。一旦出现新证据，当前的后验就顺理成章转变为下一轮推演的先验，循环往复。 ```mermaid flowchart LR accTitle: Bayesian Updating Cycle accDescr: A loop showing prior beliefs updated by likelihood and normalized into posterior beliefs, which then become the new prior. subgraph Update["贝叶斯推理与学习循环"] direction LR P["先验信念<br/>(Prior)"] -->|"结合新证据/数据"| L["似然度评估<br/>(Likelihood)"] L -->|"归一化处理<br/>(总和为 1)"| Post["后验信念<br/>(Posterior)"] end Post -->|"迭代更新，成为新的先验"| P ``` Zoubin 强调，这套数学法则不仅能解释人类知觉（例如在哥斯达黎加丛林中通过连续的视觉线索确认跟踪自己的不是美洲豹），更是机器“学习”的终极模型：模型预先具备参数，面对连续输入的数据，算法借助贝叶斯定理不断迭代更新参数分布。Hannah Fry 补充道，这与人类大脑依据新线索推翻旧结论的直觉思维如出一辙。谈及对未来机器的期望，Zoubin 表示，我们希望 AI 能像《星际迷航》（Star Trek）中的仿生人 Data 一样保持极致理性，而不是毫无根据地反复横跳；人类希望计算器能分毫不差地计算超大乘法，同样地，AI 系统在处理概率和不确定性时，本应比容易产生认知偏差的人类更为理性与严谨。然而面对现实，主流 AI 在处理自身不确定性方面依然差强人意。 - 贝叶斯法则核心公式要素：后验概率（Posterior）正比于先验概率（Prior）乘以似然度（Likelihood）。 - 信息增益量化：在证据输入后，不确定性的消除可以通过信息论中的信息量（比特）进行数学测量。 - AI 理性愿景：Zoubin 总结道：“I would actually like our AI systems to be, you know, more rational, better at representing and manipulating probabilities than humans are.”（我由衷希望我们的人工智能系统能够比人类更加理性，在表征和运用概率方面做得比人类更好。） ### `20:00` 大语言模型的“确定性假象”与语义熵 大语言模型（LLM）即便能流利作答，在底层逻辑上也只是在执行下一个 Token 的概率预测（next token prediction）。当用户追问模型“你有多大把握”时，即便它给出一个看似反思的答复，其内部也并没有对自身信念显式地计算贝叶斯法则。Zoubin Ghahramani 指出，模型在海量网络文本上学习到了大量推理痕迹，因此“这有点像在装模作样”，一旦人类用荒谬的说法反驳它（例如“不，我认为你错了”），模型往往会立刻退缩附和并道歉，无法坚定立场，暴露出其内部并没有真正连贯的置信度表征。 针对“为什么模型极难表达‘我不确定’”这一疑问，Zoubin 解释道，根本原因在于当前的训练范式从未将“不确定性”列为核心优化目标；模型只是被训练去拟合多位具有不同信念的人类所产出的海量文本，其结果就像是把所有东西混在一起的“大杂烩（soup）”。然而，自动驾驶汽车或物理机器人若要在现实世界安全运作，就必须具备因果推理能力和对自身不确定性的认知。Hannah Fry 随即提到了幻觉问题，Zoubin 强调幻觉其实是一种副产物：在创意写作场景下（如编写“阿尔伯特·爱因斯坦乘火箭登月”的短篇小说），模型必须能够产生天马行空的幻觉并推断用户意图；但当面对客观事实提问时，模型则必须做到事实对齐与信息锚定（factual grounding）。当前技术真正欠缺的，是让模型针对任何陈述都能给出一个对其自身信念的概率估计。 针对如何从模型内部抽取不确定性，Zoubin 的学生等人提出了“语义熵（Semantic Entropy）”等前沿概念。在生成下一个 Token 之前，模型内部实际上存在着一个覆盖所有可能候选 Token 的概率分布： - **低熵分布（Low Entropy）**：分布极其尖锐（spiky），意味着模型对当前选项具有极高的确信度。 - **高熵分布（High Entropy）**：分布相对平缓分散（spread out），代表模型内部具有很高的不确定性。 Hannah Fry 以“埃菲尔铁塔”作为例证进行探讨：当模型被问及埃菲尔铁塔的位置时，“巴黎”这个词对应的分布会呈现显著的高概率尖峰；虽然纽约或拉斯维加斯也有复制品或同名地标，模型可能会赋予它们微小的概率，但整体尖峰依然集中在巴黎。而如果是毫无关联的词汇（如马拉喀什），概率则接近于零。 > “They struggle because the paradigm for training them hasn't prioritize that.” —— Zoubin Ghahramani ```mermaid flowchart TD accTitle: LLM 置信度推断与语义熵示意图 accDescr: 展示从输入提示词到 Token 概率分布，再根据熵的高低区分确定与不确定状态的过程。 subgraph InputStage["输入与处理"] A["用户提示词输入"] --> B["Transformer 模型运算"] end subgraph DistributionStage["Token 概率分布分析"] B --> C{"候选 Next Token<br/>概率分布"} C -->|"尖锐分布 (Low Entropy)"| D["高度确信<br/>(例如: 埃菲尔铁塔 -> 巴黎)"] C -->|"平缓分布 (High Entropy)"| E["高度不确定<br/>(需表达怀疑或降级输出)"] end ``` --- ### `25:00` 算力瓶颈与基于贝叶斯扩散的 GenCast 气象预报 仅仅依靠数据相关性来生成概率分布，在某种程度上依然是在“伪造认知”。Zoubin 打了一个生动的比方：这就像试图通过给机器展示成千上万个加法和乘法示例来制造一台计算器，一旦遇到一个从未出现过的特定数字，它就可能无法泛化；然而人类需要的不是一台“模仿计算”的伪计算器，而是一台能真正进行运算法则推理的计算器。回顾 AI 发展史，Zoubin 直言人类早在 15 到 20 年前就已经掌握了构建理性系统的核心理论构件，但由于对世间万物维护和更新概率分布在计算上是极其棘手且不可行的（computationally intractable），需要等待数百万年甚至无法实现的超级算力，学界才一度放弃并转向了单纯的数据驱动训练范式。 随着讨论从大语言模型转向能够优雅处理不确定性的尖端 AI，Google DeepMind 研发的气象预测大模型 GenCast 展现了贝叶斯方法与现代神经网络结合的巨大威力。不同于传统数值天气预报需要在超级计算机上耗费数小时运行物理方程，GenCast 仅需大约 8 分钟即可完成未来 15 天的高精度天气预测。 GenCast 的核心运作机制包含以下关键要素： - **结合扩散模型（Diffusion Models）**：借鉴图像生成领域的扩散算法，在时间维度上直接对复杂概率分布进行显式建模与操作。 - **集成预报（Ensemble Forecasts）**：面对像“梅利莎飓风（Hurricane Melissa）”这样的热带风暴路径预测，系统不会只运行单次模拟，而是通过反复运行模型生成一个覆盖所有潜在移动轨迹的预测集合（Ensemble）。 - **动态贝叶斯更新（Bayesian Updating）**：系统初始建立一套关于气象状态（如飓风风速、走向轨迹）的信念分布；随着时间推移，一旦数小时后地面和雷达传感器传回最新观测数据，系统便会动态更新这一预测集合。 Hannah Fry 认为这揭示了一个绝妙的反直觉逻辑——“向系统中引入不确定性，反而使最终的预测更加精准”。Zoubin 对此表示完全赞同。天气预测本身就交织着两重不确定性：一是天气系统固有的混沌属性与蝴蝶效应（随机性），二是在现实中部署的气象传感器数量有限且精度有偏差。AI 系统的关键在于直面真实世界的不完美，而不是盲目制造噪音；唯有承认传感器可能受损失真、模型的先验假设可能出错，系统才能产出真正稳健可靠的决策依据。 > “It's like trying to build the calculator just by showing it examples of addition and multiplication.” —— Zoubin Ghahramani > “I think there's something quite delightfully counterintuitive about that, that you add in uncertainty to the system, and it makes the predictions more accurate.” —— Hannah Fry - **计算效率跃升**：GenCast 能够在约 8 分钟内推演 15 天的天气走势，而传统超算物理模拟则需要数小时。 - **双重不确定性根源**：气象建模既要应对系统固有的混沌特性（蝴蝶效应），又要处理因传感器覆盖有限或硬件故障导致的数据缺失。 ### `30:00` AlphaFold 的不确定性表达与校准概率 当模型不仅需要给出答案，还需要对自身预测的可靠性负责时，不确定性的量化与表达就变得至关重要。Hannah Fry 提出了一个绝佳范例——DeepMind 开发的 AlphaFold。在 AlphaFold 的蛋白质结构预测中，模型会用不同的色彩直观标出其置信度。Zoubin Ghahramani 对此表示赞同，他指出从一维氨基酸序列推断三维折叠结构，本质上就是一个面对物理世界不确定性的问题：一方面，蛋白质的某些部分在物理规律下本身就存在热扰动和摆动，位置无法精确锁定；另一方面，这是模型预测而非真实的实验观测数据。因此，必须在预测出的分子空间“云图”中准确呈现这种不确定性。 ```mermaid flowchart TD accTitle: 概率校准的两类事件处理框架 accDescr: 对比可重复事件（频率校准）与不可重复单次事件（贝叶斯信念度量）的数学处理机制 subgraph Repeatable["可重复事件 (如天气预报)"] A["预测下雨概率 70%"] --> B["对所有同类预测日汇总统计"] B --> C["70% 的天数实际降雨，30% 未降雨"] C --> D["实现频率校准 (Calibrated Probability)"] end subgraph Singular["单次不可重复事件 (如人类登火星)"] E["预测事件发生日期"] --> F["赋予先验置信度 (Belief)"] F --> G["贝叶斯概率量化主观不确定性"] G --> H["事件发生或未发生时结果最终自决"] end ``` 这种追求诚实表达不确定性的做法也带来了一个滑稽的极端情况：Hannah Fry 打趣说，如果模型诚实过头，可能会变成一个永远推脱、只说“我不知道”的套话机器。Zoubin Ghahramani 解释，解决这一问题的核心在于概率校准（Calibration）。针对真实世界中的可重复事件，校准的定义是精准的——如果模型预测下雨概率是 70%，那么将所有给出该概率的日子加总，必须恰好有 70% 的日子下雨，30% 不下雨。而面对不可重复的单次事件（比如人类何时首次踏上火星），贝叶斯统计提供了严密的数学工具： > “So, what Bayesian statistics tells you is that it's perfectly valid and in fact the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once.” —— Zoubin Ghahramani - AlphaFold 利用色标直接向科研人员可视化展示结构预测的置信度。 - 概率校准的核心是预测概率与实际发生频率在长期统计下严格匹配。 - 贝叶斯框架证明了用概率度量单次独特事件的置信度不仅有效，而且是严谨正确的做法。 --- ### `32:32` 人机交互中的不确定性传达与医疗高危决策 在解决了数学表征之后，如何将这种不确定性传递给人类用户便成了关键。Hannah Fry 质疑，如何才能让不确定性的传达真正产生价值，而不是让人类要么全盘全盘否定，要么在机器宣称“高置信度”时不动脑筋地顺从。Zoubin Ghahramani 分析了公众对 AI 的两极反应：一部分人持极端怀疑态度，完全不相信 AI；另一部分人则产生过度依赖。他特别以医疗领域为例：在不久的将来，医生将在 AI 系统的辅助下查看患者症状和检测结果。由于诊疗方案直接关乎生死，AI 传达不确定性直接决定了医生的决策方向： > “Because that's literally what is going to determine your treatment plan or whether you take one decision or another, these could be life and death decisions, right? So it's absolutely essential that first of all, we don't become over-reliant on overconfident AI systems.” —— Zoubin Ghahramani 为了避免人类过度依赖自负的 AI，机器必须坦诚自身的局限性，并将这种不确定性以可视化的形态呈现给人类。Zoubin Ghahramani 提到了他在剑桥大学的同事 David Spiegelhalter 在向公众普及不确定性与概率表达方面所做的开拓性工作。他强调，AI 应当扮演与人类互补、为人类赋能的角色，而不是一个伪装成全知全能的黑盒。Hannah Fry 深受触动，感叹让机器准确表达“知与不知”是构建人类与 AI 共同未来的核心基石。 - 医疗场景是 AI 量化不确定性的试金石，关乎生死抉择。 - 剑桥大学统计学家 David Spiegelhalter 的研究为公众理解概率和风险提供了优雅的可视化手段。 - 理想的 AI 设计原则应追求与人类认知“互补且增益（complementary and additive）”。 --- ### `35:00` 扩展路线与全新架构之争：持续学习与贝叶斯更新 面对如何通往通用人工智能（AGI），学术界与工业界分化为两大阵营：一派信奉“大力出奇迹”，主张仅凭更多数据、更大算力和模型规模（Scaling）就能解决一切；另一派则坚信现有框架存在根本天花板，必须寻找全新架构。Zoubin Ghahramani 坦言自己坚定站在后者一边。单纯堆叠规模的盲区在于长尾分布（long tail）中的异常边缘情况。在日常聊天中犯错或许无关痛痒，但在自动驾驶与医疗诊断等高风险领域，概率预测失误将引发灾难。 ```mermaid flowchart LR accTitle: 现代批处理训练与贝叶斯持续学习的对比 accDescr: 展现大模型周期性重新训练的痛点，以及贝叶斯按样本连续更新避免灾难性遗忘的优势 subgraph Batch["现状：大模型批处理训练"] direction TB B1["海量静态数据集"] --> B2["训练超大模型"] B2 --> B3["部署上线"] B3 -->|"数月后丢弃旧权重"| B4["重新训练下一个巨型模型"] B4 -.->|"无法动态适应"| B5["灾难性遗忘 (Catastrophic Forgetting)"] end subgraph Bayesian["理想：贝叶斯持续学习"] direction TB L1["先验知识 Prior"] -->|"输入新数据流 Data"| L2["更新后验知识 Posterior"] L2 -->|"转化为下一轮先验"| L1 L2 --> L3["理论上无灾难性遗忘，自适应演进"] end ``` Zoubin Ghahramani 详述了现有体系难以突破的第一大瓶颈：持续学习（Continual Learning）。目前行业训练大模型的范式极其死板——训练一个庞大的模型，上线几个月，随后再从头或批量重训下一个模型。相比之下，人类和动物每时每刻都在接收数据流并实时调整神经突触连接。现代神经网络一旦试图这样边用边学，就会陷入“灾难性遗忘（catastrophic forgetting）”。Zoubin Ghahramani 从数学上指出，严格的贝叶斯先验-后验更新机制在理论上天然支持终身持续学习，且免于灾难性遗忘，目前大模型持续学习的诸多前沿探索，本质上都是对贝叶斯更新的工程逼近。 - 两大 AGI 阵营：规模扩展派（Scale/Compute/Data）与架构革新派（New Architectures）。 - 规模派在海量常见场景上行之有效，但在长尾异常分布下存在致命漏洞。 - 持续学习是突破目前大模型周期性“批处理”训练、实现生物级自适应认知的关键研究方向。 --- ### `37:28` 能效鸿沟、稀疏架构与数据效率缺陷 Zoubin Ghahramani 继而指出了当前 AI 面临的能耗危机与数据效率瓶颈。他用一个极具反差的物理指标进行了对比：人脑的运行功率仅仅约为 20 瓦——Hannah Fry 随即吐槽这相当于“一个质量很烂的传统白炽灯泡”，Zoubin 则笑着纠正为“一个极其节能的灯泡”。相比之下，在大数据中心里训练一个大型语言模型所消耗的能源与算力，在量级上高出了数个天文数字。 虽然单一的人脑承载的是个体生命经验，而大型语言模型是熔炼了人类全球知识的“巨型大杂烩”，但这种能效差距证明当前的计算范式存在巨大的浪费。不仅如此，当今驱动 AI 浪潮的几乎仅有 Transformer 和扩散模型（Diffusion Models）这两大主力架构。Zoubin Ghahramani 预测，未来的突破可能来自于软硬件的共同演进（Co-evolution），例如基于超稀疏神经网络（Sparse Neural Networks）的新型硬件。最后，他指出了另一个被主流规模论掩盖的致命短板：与人类和动物从寥寥数个样本中就能领悟规律相比，当下的深度学习系统在数据利用效率上极其低下。 > “So if you look at the power consumption of a human brain, it's about 20 watts. If you compare that to training a large language model in a big data center, it's orders of magnitude off.” —— Zoubin Ghahramani - 人脑功耗仅约 20W，而大模型训练能耗高出数个数量级。 - 现代 AI 深度依赖 Transformer 与扩散模型两大工作母机，缺乏多样性。 - 突破方向指向高能效算法、软硬件协同设计的稀疏神经网络，以及摆脱海量灌洗的数据高效学习机制。 ### `40:00` 贝叶斯的“魔法”与计算诅咒：从 NP 难题到硬件突破 贝叶斯思想在人工智能领域展现出近乎理想的特质：它不仅能将谦逊与不确定性直接融入模型，还赋予系统持续学习（Continual Learning）的能力，并显著提升数据使用效率。这一切甚至可以凝练在一行简洁的贝叶斯法则（Bayes' rule）公式之中。Hannah Fry 感叹这听起来就像一个“过于美好而不真实”的魔术。对此，Zoubin Ghahramani 坦言，学界很早就知晓这一原理，且每位机器学习的学生都应当理解这些方法的可行性，但这套“魔法”背后却伴随着巨大的代价——严苛的计算瓶颈。如果查阅经典的 AI 教科书，书中固然会详尽阐释这些机制，但往往会补充说明这些问题在计算上属于 NP 完全（NP-complete）或 NP 困难（NP-hard）问题，现实中无法实现精确求解，必须依赖近似算法。现有的现代 AI 系统，本质上都是对这些理想化数学概念的某种工程近似。 面对计算成本高昂、执行速度极其缓慢的质疑，Hannah Fry 指出这与 20 世纪 80 年代人们对神经网络的批评如出一辙，历史不应重蹈覆辙。Zoubin Ghahramani 深表赞同，并以自身经历作了对比：他在本科阶段使用过的顶尖大规模并行超级计算机（Connection Machine），其运算速度甚至还比不上如今装在他口袋里的 Google Pixel 智能手机。算力的指数级跃升，加之学术界数十年来积累的大量高效近似算法，意味着研究者已经掌握了打破计算诅咒的实用工具，目前的核心任务在于将这些理论与现代软硬件深度整合并进一步创新。 ```mermaid flowchart TD accTitle: 贝叶斯 AI 的理论优势与现实落地路径 accDescr: 展示贝叶斯法则的核心优势、面临的计算瓶颈以及算力与近似算法带来的解决方案。 subgraph Advantages["贝叶斯法则的核心优势"] A["单行优雅公式<br/>(Bayes' Rule)"] --> B["内在不确定性与谦逊"] A --> C["高效样本利用率"] A --> D["持续学习能力"] end subgraph Bottleneck["计算诅咒"] B & C & D --> E["数学精确求解极度缓慢<br/>(NP-hard / NP-complete)"] end subgraph Solutions["破局途径"] E --> F["现代近似算法体系"] E --> G["硬件算力跃升<br/>(从昔日超算到口袋手机)"] F & G --> H["实用化、高鲁棒性的新一代 AI"] end ``` - Zoubin Ghahramani 指出精确计算的严峻挑战：“The magic trick comes with a big curse. The curse here is that to do all of this is computationally very slow.”（这个魔术伴随着一个巨大的诅咒，那就是要做到所有这些，在计算上极其缓慢。） - 针对算力演进与历史对比，Zoubin 强调硬件的飞跃为传统理论带来了新生：“The giant state-of-the-art supercomputer parallel connection machine computer that I used in my undergraduate years is actually slower than the Pixel phone that I have in my pocket.”（我本科时使用的那台当时最先进的巨型并行连接机超级计算机，实际上比我口袋里的 Pixel 手机还要慢。） --- ### `42:09` 超越人类的智能：为何我们亟需懂得自我怀疑的 AI 回顾整个讨论，Hannah Fry 观察到不确定性、诚实、谦逊与自我怀疑等特质，直观上具有极其浓厚的人性色彩。这引发了一个深层追问：在 AI 研究起步之初，人们是否曾预料到系统最终欠缺的并非纯粹的计算能力或数据分析能力，而是这些微妙的人性品质？Zoubin Ghahramani 坦承，没有任何一位研究人员能完全预料到技术演进能达到如今这般惊人的速率，但他强调，这些品质不仅仅是人类的专属标签，更是任何高阶智能系统不可或缺的底层支柱。 在 Zoubin 看来，我们需要树立一种超越人类局限的“智能”概念。人类本身的思维存在大量认知偏差与非理性缺陷，在现实世界面临不确定性时，人类往往非常不擅长做出理性决策，经常会错误估计甚至彻底算错概率。因此，构建“以人为本”的 AI 系统，必须从根本需求倒推——思考人类与社会的重大现实难题到底是什么，以及我们需要什么样的智能去解决它们。在面对所有关乎重大决策的问题时，拥有一个清醒知道自身局限的模型，远比一个盲目傲慢、过度自信的系统要宝贵得多。 在节目的结语中，Hannah Fry 总结道，多年来人工智能的发展过度专注于“准确度”，倾注庞大算力吞吐海量数据以求得出唯一确定的答案。这种路径塑造出的模型虽然能力惊人，但在某些层面却异乎寻常地脆弱——一旦发生错误，它们往往表现得极端笃定且毫不自知。赋予 AI 接纳与表达不确定性的能力，并非暴露其弱点，反而注入了诚实、谦逊与怀疑的智慧，使机器能够扎根于客观现实，真正成为人类在重大决策中值得托付与信赖的坚实协作者。 - 谈及理想 AI 系统的性格特征，Zoubin Ghahramani 掷地有声地总结道：“And for, I think, all problems that matter… I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident.”（我认为，在所有至关重要的重大问题上……我宁愿要一个知道自己何时‘不知道’的 AI 系统，也不要一个傲慢且过度自信的 AI 系统。） - Hannah Fry 在总结 AI 未来的演进方向时指出：“When they fail, they fail with total confidence. But by teaching AI to embrace uncertainty, it gives it something more human, humility and honesty and the wisdom to doubt.”（当它们失败时，往往带着十足的自信去犯错。但通过教会 AI 接纳不确定性，便赋予了它更人性化的特质——谦逊、诚实，以及懂得怀疑的智慧。） ## Notable quotes > `42:09` And for, I think, all problems that matter… I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident. — Zoubin Ghahramani > `07:30` We don't want systems that can be overconfidently wrong. — Zoubin Ghahramani > `00:00` To build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty, and then can use that to make good decisions under uncertainty. — Zoubin Ghahramani > `42:09` When they fail, they fail with total confidence. But by teaching AI to embrace uncertainty, it gives it something more human, humility and honesty and the wisdom to doubt. — Hannah Fry > `25:00` I think there's something quite delightfully counterintuitive about that, that you add in uncertainty to the system, and it makes the predictions more accurate. — Hannah Fry > `30:00` So, what Bayesian statistics tells you is that it's perfectly valid and in fact the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once. — Zoubin Ghahramani > `16:04` I would actually like our AI systems to be, you know, more rational, better at representing and manipulating probabilities than humans are. — Zoubin Ghahramani ## Mentioned Zoubin Ghahramani, Hannah Fry, Google DeepMind, 剑桥大学, 宾夕法尼亚大学, Aravind Joshi, Geoffrey Hinton, Daniel Kahneman, Amos Tversky, David Spiegelhalter, Parallel Distributed Processing, Connection Machine, Nature, GenCast, AlphaFold, Google Pixel, Transformer ## Video description Now, if you ask an AI a question, it will usually give you an absolute answer with unwavering authority, even if that answer turns out to be wrong. In fact, today's AI seems to be missing a fundamental human trait: self-doubt. As a professor at Cambridge and VP of Research at Google DeepMind, Zoubin finds himself at the heart of an interesting debate: will improving machine uncertainty be one of the missing pieces to ever improving AI? [Source](https://afp-920373-injected.calisto.simplecastaudio.com/36cf10bf-f6dd-4a98-96a9-8eda9f59db4f/episodes/64f5029d-0d8f-4b02-ad63-d112c6ba9152/audio/128/default.mp3?aid=rss_feed&awCollectionId=36cf10bf-f6dd-4a98-96a9-8eda9f59db4f&awEpisodeId=64f5029d-0d8f-4b02-ad63-d112c6ba9152&feed=JT6pbPkg)

[42:09] Zoubin Ghahramani: And for, I think, all problems that matter… I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident. [7:30] Zoubin Ghahramani: We don't want systems that can be overconfidently wrong. [0:00] Zoubin Ghahramani: To build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty, and then can use that to make good decisions under uncertainty. [42:09] Hannah Fry: When they fail, they fail with total confidence. But by teaching AI to embrace uncertainty, it gives it something more human, humility and honesty and the wisdom to doubt. [25:00] Hannah Fry: I think there's something quite delightfully counterintuitive about that, that you add in uncertainty to the system, and it makes the predictions more accurate. [30:00] Zoubin Ghahramani: So, what Bayesian statistics tells you is that it's perfectly valid and in fact the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once. [16:04] Zoubin Ghahramani: I would actually like our AI systems to be, you know, more rational, better at representing and manipulating probabilities than humans are.

[42:15] Zoubin Ghahramani: I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident. [0:00] Zoubin Ghahramani: And so fundamentally, to build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty, and then can use that to make good decisions under uncertainty. [12:35] Zoubin Ghahramani: Many aspects of intelligence depend crucially on the careful probabilistic representation of uncertainty. [15:47] Zoubin Ghahramani: I would actually like our AI systems to be, you know, more rational, better at representing and manipulating probabilities than humans are. [30:00] Zoubin Ghahramani: So, what Bayesian statistics tells you is that it’s perfectly valid and in fact the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once.

[0:00] Welcome to Google DeepMind, the podcast. Now, if you ask an AI a question, it will usually give you an absolute answer with unwavering authority, even if that answer turns out to be wrong. In fact, today's AI seems to be missing a fundamental human trait, self-doubt. But long before the current wave of large language models, one academic researcher was trying to give machines a sense of their own limitations. Zubin Garamani has spent the last 30 years built on the mathematics of uncertainty. Today, as a professor at Cambridge and co-leader of Frontier AI at Google DeepMind, Zubin finds himself at the heart of another interesting debate. On the one side are those who are hoping that pure scale will be the answer to ever improving AI. And on the other are those like Zubin, who believe that true intelligence requires innovations in architecture and that improving machine uncertainty may be one of the missing pieces. Stephen, welcome to the podcast. Thank you. If you were to distill it all down, I mean, your central thesis is that we need to have uncertainty in AI. Just, I mean, give me the top line of it. So why? Yeah. Why? Well, If you think about intelligence, one of the most important parts of intelligence is decision-making. You can't have an intelligence system that doesn't make decisions. from bacteria to animals to humans to robots, decision-making is really important. And if you want to make decisions, in the real world, Our perception is limited. So, we are always uncertain about the state of the real world. and we need to make decisions under uncertainty. We can't know everything. We don't know everything from our senses. We can't predict the future. And so fundamentally, To build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty, and then can use that to make good decisions under uncertainty. Because actually, I mean, there's two different types of uncertainty. I guess, right? There's the uncertainty of just the inherent randomness of the world. There's a pedestrian in a normal, typical street, and you just don't know which way they're going to turn. But then there's the uncertainty of a scenario that you've never encountered before. Let me give an example. from something that is becoming more and more of a reality in all our lives, which is self-driving cars. So, when you're in a self-driving car, you know, the self-driving car has been trained on lots and lots of data. It's seen many, many scenarios. But you can imagine that there is what's called the long tail of things that could happen. Like, for example, the AND The car may have not been trained in many instances of hailstorms. And it may not have been trained with, you know, horses suddenly jumping in front of the car in a hailstorm. And so, essentially, what you really want from an intelligence system is a certain self-awareness, if we can use those terms, a self-awareness about its uncertainty. So it needs to be able to know the situation that it's in. is something that is unusual or it hasn't seen before. And in the case of the self-driving car, for example, if it Were to have a sense of its uncertainty, it would... basically decide to slow down because it hasn't encountered that situation before. It's not confident. that the horse isn't a bicycle or whatever it is, right? There are many different kinds of uncertainty, but the beauty of it is that you know, from a mathematical point of view, we can boil it all down to probabilities. So we can map all these different forms of uncertainty onto probabilities and then use the rules of probability theory to manipulate uncertainty, update. your state of uncertainty, etc., But how important is it that a machine can tell the different types of uncertainty apart? Yeah, I think it's important insofar that the different types of uncertainty may mean different decisions. So, for example, if you have... you know, what's called aleatoric uncertainty, which is the sort of randomness of a coin flip. Which way is the pedestrian going to turn? Or which way is the pedestrian going to turn? You might want to decide that you're going to give up on trying to predict because it is just random. Whereas in other cases, your state of belief is uncertain and you would want to collect more information to sort of, and in fact, that's the definition of information, right? So information, a bit of information that we use in computer is the reduction of our uncertainty by a factor of two. That's what a bit is. And so, collecting information is the way we reduce our uncertainty. [5:00] How did these ideas of uncertainty map onto what humans are doing? So I actually studied cognitive science. I studied computer science and cognitive science as an undergraduate. So I've always been interested in in human intelligence as well as machine intelligence. And one of the really interesting things is that the field of cognitive science has really embraced these ideas of uncertainty and probabilities and so on to try to understand both human perception and human decision-making. So, we go through our lives perceiving things, perceiving the world is fundamentally an act of sensing things, something that is uncertain, you know, I can't see the back of your head, but I can sort of infer what the back of your head might look like from... You'd hope it was there. I would hope it's there. And, you know, similarly, let's say I'm hiking in Costa Rica. and, um... Suddenly, there's a rustling of the leaves. It may be a jaguar, right? And so, my senses, through evolution, my senses have developed to take in perceptual information, take in my prior beliefs. Because, you know, there are jaguars in Costa Rica, there wouldn't be jaguars in the middle of London. And that process of combining information has been modeled by cognitive scientists and psychologists and neuroscientists, through the language of probabilistic inference, basically. And also, actually, one of the interesting things is that we as humans are actually quite bad at estimating probabilities. explicitly, like if you ask somebody what is the probability of a certain event, they might get it wrong by orders of magnitude, for example. And, you know, we have these fallacies of you know, probabilistic inference and belief, Kahneman and Tversky, showed that humans are actually quite bad at representations of uncertainty at a conscious level. But, you know, in our perceptual systems, unconsciously, we tend to be quite good about these things because our survival depends on them. I'm thinking also about babies here, you know, or toddlers. The way that they learn is a lot about their belief in what action will drive a particular outcome? Yeah, I mean, it's all implicit. Obviously, babies don't know what probabilities are, and they won't be able to write down any equations for you. But there are many cognitive scientists and psychologists who try to understand human learning, human perception, human decision making, using these same, you know, formalisms that we're using for AI systems. I think there are a few things here we should probably tease apart before we get really into it. Because there's a difference between... of correctness and confidence, which both sometimes come under the umbrella of uncertainty. Yeah, absolutely. A great example of this is we use AI systems for image classification all the time. So you give it an image, and it gives you an answer of what's in the image. And we can measure correctness. But we also wanted to tell us how confident it is. And more than a decade ago, people discovered that you can take an image, for example, of a school bus, modify just a few pixels in that image in an imperceptible way. So a human being would look at it and say, "Well, that's an image of a school bus." You give it to the neural network, and it confidently says, that's a cheetah. 99% that's a cheetah. And so, in fact, you could do it for any category. You could turn the school bus into... a monkey or whatever in the eyes of the neural network. So that sort of adversarial example shows us that, It's not correctness that we care about alone. It's actually correctness and confidence. We don't want systems that can be overconfidently wrong. Or can be fooled, I guess. Or can be fooled. And I think our systems can still be fooled in that way. These ideas of overconfidence and humility, are these going to end up being important as we start to build AGI? Yeah, absolutely. I think we've advanced a great deal now. with systems that can acquire a lot of knowledge. But when we deploy them, for example, large language models, we've all encountered this, no matter what large language model you interact with, it will confidently give you an answer. And then you question it. And maybe it'll then flip-flop and give you a different answer. And so, it's hard to trust the system necessarily. that is overconfident. It's also hard to trust people that are overconfident, right? I think we would, like all intelligence systems, to have a sense of their limits, their knowledge, and... We need to bake that into our AI systems for sure. Absolutely. I know that you've been working in AI for a really long time, right? I mean, since the late 80s. Yeah, yeah. What was the field like then? So, I'm pretty boring in that I was interested in AI as a teenager, so when I was 14, [10:00] 15, I already wanted to work in AI. When I went to university, I needed a summer job. So I went to the head of the computer science department, this very famous computational linguist, Arvind Joshi, at University of Pennsylvania, and I said, I'm here for the summer. I'm a first-year student. I need a summer job. And what he said was that these two books that have come out, these were called Parallel Distributed Processing. This was in 1986, the same year that the backpropagation paper had come out that launched the whole neural network, kind of modern neural network revolution. And he gave me a summer job reading these books and explaining them to him, which was the most wonderful summer job one could possibly have if you're interested and curious about things. So, I learned about neural networks back then. And, You asked me what was it like in the mid to late 1980s? The dominant paradigm in AI was expert systems. So, people were looking at rule-based systems that would make decisions, and they were quite brittle. And neural networks, which were sort of modeled after the human brain, were actually much more flexible. And fundamentally, they could learn from data in a way that previous methods were not very good at learning from data. So that was really a revolution back in the 1980s. We talk about the Transformer Revolution and all that. But back in the mid-1980s, that was a real revolution. And it attracted people from cognitive science, from computer science, psychology, neuroscience, economics. And then I went on to write an undergraduate thesis on science. learning, how to parse human natural language using neural networks. So it was one of the very earliest, I didn't publish it, but if I'd published it, it would have been a very early paper on small language, let's call them small language models, because my models are very small. Wait, what year was this? This is 1989. Very much ahead of time. Yeah, and we actually had a parallel computer. So now people think about data centers and all sorts of GPUs and parallel computers. And we had the most sci-fi, beautiful… iconic parallel computer of the time, which was this connection machine, which was this cube with 65,000 little red lights blinking inside this, you know, kind of opaque cube, because it had 65,000 processors. So I would sit there coding up in this parallel language, neural networks for natural language. And it was pretty revolutionary at the time because neural networks were the counterculture. People were very enamored of the old school of AI. But then, okay, despite being ahead of the game by approximately, what, 40 years? Yeah. Then I messed it up. really fascinating. So, I was working in neural networks, and at the time, this is now the early 1990s, we felt like we understood how they worked. So, neural networks are these amazing function approximators. We can feed them data, they can map from inputs to outputs, from X to Y, from, you know, images to labels of images and so on. And remember, the data sets were very small at the time. So when I was writing my undergraduate thesis, the World Wide Web didn't even exist. Compute was also very limited. So we felt like we understood neural networks, we felt like they're nice function approximators, but But... people started to uncover these beautiful relationships between neural networks and other ideas from probability theory, statistics, stochastic processes. I mean, you actually gave up working on neural networks in favour of these probability questions. Yeah, yeah, absolutely. Was it annoying, though, that in the end neural networks were the thing? And you were there so much earlier than anyone else. Yeah, well, you know, I didn't, I did... redeem myself. spent a number of years working with Jeff Hinton. Of course, he went on to win the Nobel Prize for his work on neural networks. So, we were attempting to do deep learning, but we were using probabilistic models, rather than simple neural networks. We were overcomplicating things because we hadn't seen what happens with large-scale data. And then I went off and worked on Bayesian machine learning and probabilistic models and all these things for many years. But I never... I never totally dismissed neural networks, actually, because I knew that they work. It's just that they weren't sort of from a research point of view. They weren't as... interesting to me because the mathematics was sort of, at the time, we felt well understood. In 2015, though, you wrote this seminal nature paper, which appeared in the same issue as all of the foundational deep learning and reinforcement learning papers by Hinton. And in that paper, you argue, Many aspects of intelligence depend crucially on the careful probabilistic representation of uncertainty. Yes. I mean, has everyone heard you? Like, have they needed your warning? No, no. I think there are many people who understand that and believe that. [15:00] I think that it depends on the level that you think about things. So, actually, if you look at large language models, THEY ARE probabilistic. They predict the probability of the next token or word given a sequence of previous tokens. So probabilities are at the heart of everything we do in machine learning. But what's missing is we're not really doing what I said, which is the careful representation of probabilities. We're actually sort of hoping that the models... represent probabilities okay because we've trained them on enough data. But not thinking about it in an explicit way. Yeah. If you look in a giant neural network, you can't really... find the explicit representation of say, the probability that you know, it thinks something or other, right? It's sort of spread out somehow over the you know, all the activations of the billions of of units in the neural network. And by contrast, I mean, you're more of a, I guess, a Bayesian thinker. Yeah. Just explain for anybody who hasn't come across this before, just explain to us what that actually means. Yeah. So Bayes' rule is this fascinating and very simple concept from probability theory. So before you observe something, before you get some evidence or data, you have what's called prior beliefs. you represent those with a probability distribution. So, for example, think of a detective story, like a whodunit. There's a number of suspects, and you may have some prior beliefs about, it's the butler that did it or whatever. He's looking suspicious. Yeah, somebody's looking suspicious or something. So, you have some beliefs, you represent those with a probability distribution. And And there are many, many, you know, um, reasons why probability theory is the right way of representing beliefs. whole branches of mathematics that have proven that. And now you observe some evidence, like... you know, the murder weapon is found in the pantry or something like that, right? And so, you take your prior beliefs, multiply them by what's called the likelihood, the probability under each possible... you know, culprit, uh, And Then you renormalize because probabilities have to sum to one. And from that, you get your posterior beliefs. your new state of knowledge. And through that evidence, by the way, you've gained information, literally measured in bits, how much your uncertainty has decreased. And now, if you get more evidence, you just take your current posterior probability distribution, which is now your new prior, and you repeat and rinse. You do it again. You get the new evidence. You update the probabilities and so on and so forth. And through that application of Bayes' rule, we can model both perception, like I open my eyes, I see something, then I see more things and I know it's not a jaguar that's following me in Costa Rica. but you can also model what learning is. So, learning is you have a model, The model has parameters. At the beginning, you don't know what the parameters should be. You get some data. and you update the model parameters, sequentially through that data applying Bayes' rule in theory. That's the sort of beautiful model that I had been pushing forward as a model of learning. Because I guess, I mean, on the one hand, this is a very elegant mathematical way to link together evidence and unknowns and knowns. But I guess, actually, on an intuitive level, I mean, this sort of is the way that our brains work. I mean, going back to your example of the detective, you know, it's like, oh, I did think it was that person, but now this new evidence has come in, and I've changed my mind. That's essentially a one-sentence description of what able to do it. Exactly. So we would like to build AI systems that accumulate knowledge and information over time, And we would like them to be rational, you know, sort of like data from Star Trek is a very rational being. We don't want them to flip flop around. unpredictably based on no evidence. I would argue that ideally we would like them to be even more rational than humans. I want my AI, just like I want my calculator to be really good at multiplying large numbers, I would actually like our AI systems to be you know, more rational, better at representing and manipulating probabilities than humans are. Okay, if we fast forward to today, I mean, the paper you wrote was a decade ago. Decade, yeah. How good really are the AI systems that we're all used to playing around with? How good are they at representing uncertainty? Yeah, they've been getting better, but they're not very good. Right. And you can tell that because if you interact [20:00] with a large language model and it asserts something. you can ask it, how confident are you? And it might say something back, but it's really doing next token prediction. It doesn't have a... an explicit representation of how confident it is in that statement. It's not calculating Bayes' rule. It's not calculating Bayes' rule, at least not explicitly. It may be because you've trained it on trillions of tokens. of stuff on the web. It's mimicked a lot of, you know, other kinds of reasoning traces and so on. it's sort of faking it, right? And you can tell it's faking it because then if you push back and you say something silly like, "No, I think you're wrong." then it might respond, oh, sorry, yes, I was wrong. So, it's not really coherent. It's not going to stand its ground. And our systems are getting better, better at factual grounding and things like that. But we haven't really nailed the idea of how one of these models should be able to represent probabilities over its beliefs. Because that, I mean, I feel like that would be a very useful feature for a large language model to be able to say it's not sure. Why do they struggle with that so much? They struggle because the paradigm for training them hasn't prioritize that. We train them to be just really good at modeling the data. If the data involves a lot of human reasoning by many different humans... with many different beliefs... then what you get is a soup. You get sort of a mishmash of everything. But if we want to build, like I said, self-driving cars or robots, they need to be able to reason about the real world. They need to have an understanding of cause and effect. They need to have an understanding of their own uncertainty, and they need to use that. to be able to act in the real world in a safe way. But then, I mean... I'm just thinking about hallucinations here. Because that's part of this as well, right? There are some times where you just want, there's a verifiable fact that it's getting wrong. And that all plays into this too. Yeah, I mean, hallucinations are a symptom, right? Of course, sometimes we want our models to hallucinate, so there's a tension between you know, not hallucinating at all and not being creative, right? For example, if I want to my large language model to write me a short story of that time that Albert Einstein went to the moon in a rocket. That's clearly a hallucination, but it's an act of creative writing. So, you should be able to do that. You should be able to infer that that's what you want, infer your intent. If I ask it a factual question, it should try to be grounded. And of course, at Google, we think a lot about grounding our models in what's available. And even information on the web is grounded. often contradictory, right? And so, you want to hedge your bets. So, actually, what you would like is a system that's able to tell you for any statement some estimate of its belief or probability. I think that's what we want. We don't have it yet. No. How might you build it? If you wanted large language models to have uncertainty in there, how do you do it? Because there are some quite good ideas. I'm thinking about semantic entropy here, right, which is one of your students came up with. Tell me a little bit about that. How might that work? Yeah, I mean, I think, you know, you can take the internals of a particular model. and try to infer from that its degree of belief. So before it answers, before it produces a token in a large language model, You actually have a probability distribution over all possible NEXT tokens. And that probability distribution, the entropy of that distribution, tells you something about the uncertainty. A low entropy distribution is very spiky, is very certain. A high entropy distribution is very spread out, it's very uncertain. And so there are ways of sort of teasing from the internals of a model how certain it might be. Yeah, I guess I'm thinking here about an example, let's say the Eiffel Tower in Paris, right? Like those two things would be an example of something that's quite spiky. Yeah, yeah, sort of like if you ask it. you know, where is the Eiffel Tower? It should have a high probability, over Paris, although I believe there's one in New York as well, isn't there? There's a little Eiffel Power. I think there's one in Vegas as well. I think they've got one. Maybe there's one in Vegas as well. So basically, you know, depending on the context, you might have little bits of entropy on these other Vegas and New York as options, but Paris would have a big spike on it. Is that sort of the idea then that like in the data set, Paris and the Eiffel Tower appear near each other a lot? You have a lot of like a really big signal there. Whereas, I don't know, the Eiffel Tower and sort of Marrakesh. Yeah, that might have very low probability. The That's sort of faking it. [25:00] sense that you're relying on the data. I'll give you the analogy of a calculator. It's like trying to build the calculator just by showing it examples of addition and multiplication. But imagine you never show it a particular number, then it might not generalize that particular number. And you don't want to fake a calculator. You want a calculator that actually calculates. You want it to actually reason about the world that it's in. That's what we want from our AI systems. So why hasn't this been done yet? I mean, what is it that makes it so hard to do? It is genuinely hard because it's computationally hard. So essentially, I would argue we had all the ingredients of AI maybe 15 or 20 years ago. Like we kind of know how to build rational systems. We just thought like, well, representing probability distributions over every possible thing is computationally intractable. It would take giant supercomputers that we don't have. And it would be incredibly slow. You'd be waiting for millions of years before you get the answer. So people abandoned that for let go with just training from data And I think we can revisit this idea And I have ideas for how to do this and I exploring them now but it not sort of super well formed yet. Okay. So moving away from the large language model, the transformer type system then for a moment, because I mean, there are other examples of really cutting edge artificial intelligence, which does handle uncertainty in more of this Bayesian way that you're describing. I'm thinking about weather forecasting here. Ah, yes. I mean, that's really important. Yes, absolutely. Tell me a bit about that. Yeah. So we've developed a whole series of state-of-the-art weather forecasting models at Google DeepMind. And if you look at the GenCast model, sort of very recent model, one of the key features it has, it can predict sort of weather over 15 days. And it can do it very fast, much faster. It can do it sort of like in eight minutes rather than on a giant supercomputer for hours. And it's obviously using neural networks and things like that. But a key ingredient for getting this to work is that it uses a diffusion model, so that already is like the image generation models that already is manipulating probability distributions over time But then it generates an ensemble of forecasts So if you trying to track a tropical storm like the Hurricane Melissa let say which we did with this model you have the data so far, and then you want to be able to forecast the track of this into the future because your decisions depend on that, whether you evacuate a city or whether you you know, call in emergency services and so on. And so, it represents that with an ensemble of forecasts. It has a whole probability distribution over the possible tracks. You rerun the model over and over and over again. And then as you get more data, so, you know, a few hours later, you get more observations, you update that ensemble. And that is essentially applying these basic ideas from Bayesian updating to this sort of problem. Because I mean, the original weather forecasting models that came out here did not have this. Yeah, that was sort of added on. And every time we add on, you know, these features, it makes the model better. Because you essentially saying there is an inherent uncertainty in the way that weather works Absolutely You can just sort of run the model once and be like oh well that going be the weather tomorrow You have to do it lots of times and then work out a probability Yeah yeah It a combination of both that inherent randomness the sort of you know classic butterfly effect in weather and that it's chaotic, it's very hard to predict. But also, there is an uncertainty just because we have limited numbers of sensors, right? So, you know, It's a system that represents its beliefs about the weather, whether it's the trajectory of this hurricane and its intensity. And then those beliefs get updated as you take in the sensor measurements and as time progresses. I think there's something quite delightfully counterintuitive about that, that you add in uncertainty to the system. Yeah. And it makes the predictions more accurate. Exactly. Yeah. I think it is definitely a key insight in AI. I mean, it's not that you're sort of making the system noisy in an arbitrary way. That's not going to help you. But what you're doing is you're being honest about the fact that your sensors are inaccurate. Sometimes you get faulty sensors, just like sometimes my ears are blocked and I can't hear very well. And sometimes your model assumptions are wrong. [30:00] Right? You know, that is also a form of uncertainty. And so, all of those different forms of uncertainty have to be represented somehow, approximately, we can't do it all exactly, so that we can get better calibrated forecasts. I think the other example that really manages to get this uncertainty idea right is AlphaFold. Yeah. Where the protein prediction I mean, it's color coded by how sure the model is that that's the correct folding, right? Yeah, absolutely. And essentially, you're fundamentally trying to solve an uncertain problem. You're going from a sequence. to the folded structure. And the physics itself means that some parts of the protein are going to wiggle around more, so you don't know exactly where they are. And you also have just uncertainty because you've used the model to predict that, it's not like experimental data. And so you need to be able to represent that uncertainty in the sort of cloud of forecast of where the molecules are. There is a danger here that you could go too far the other way. You could end up with a model that was so honest about its uncertainty that it just sort of didn't ever really give you an answer. It just always said, I don't know. Just hedging, hedging all the time. That would be pretty funny. But, you know, I think there's different degrees of I don't know. How do you get the balance right then? Yeah, I think... The balance is if there is a repeatable event in the real world, then you want to be calibrated in that if I say the chance of rain is 0.7 or 70%, then For all days that I've said that, if I sum up over all those days... if If I'm calibrated, then on 70% of those days, it actually rained. On 30%, it didn't rain. So that's a calibrated probability. Now, if you ask me a statement of uncertainty about something that is not a repeatable event, so for example, we may be uncertain about the first day, a human being. reaches Mars. So, that's a date. It's either going to happen or it's not going to happen, and that date, when it happens, you'll be certain of it. That's the sort of event that you can have probabilities over, you can have beliefs over. and it will resolve itself when it happens, if it happens. And Bayes also has a way of handling that, where it's not just 15%, right? It's not just like, "It happens or it doesn't." Yeah, absolutely. So, what Bayesian statistics tells you is that it's perfectly valid and in fact the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once. How do you communicate that uncertainty? in a way that actually means it adds value rather than just has a human sort of nodding along or kind of taking cognitive shortcuts when the machine says it's really confident? Yeah, I think people have different reactions to AI systems. Some people are just very skeptical and will kind of not believe anything the AI system tells it. Other people are going to end up being over-reliant, let's say. Let's imagine a future where we've got not a very distant future, We've got AI systems. in a medical domain, for example, and you have doctors aided by an AI system looking at patients and symptoms and test results and so on. It's a great example of why we need uncertainty. If the AI system says something, you really want it to convey its uncertainty. Because that's literally what is going to determine your... treatment plan or whether you take one decision or another, these could be life and death. decisions, right? So it's absolutely essential that first of all, we don't become over-reliant on overconfident AI systems. Thank you. But to be able to do that, we need our AI systems to be honest about their uncertainty and bring that uncertainty in a visible form, to the human users so we can understand it. And I mean, you know, my colleague at Cambridge, David Spiegelhalter has done a lot of amazing work on how to convey uncertainties and probabilities in elegant ways to the general public. And I think there are ways you can do that. You can visualize the answer and so on. So, I mean, I really believe that it's important to have – AI systems that are doing complementary things that are additive to humans, that are helping people solve problems that are not. we care about. And in order to do that, you want them to be honest about what they know and what they don't know. It feels like this is really very critical that we get this right. Yeah. It's like such an important part of designing our collective future with AI. Yeah, I think it is really important. I mean, I don't want to take away from... the fact that AI systems have been incredibly useful all [35:00] You know, I think they're pretty good at some of these things and we can do better. And there are open problems along the way. And, you know, that's one of the reasons we actually need more research, actually. The thing is, I mean, I'm sort of sitting here agreeing with you. You're also a Vajian thinker, so this is very much my philosophy. But not everybody does. No, no. That's right. Not everyone agrees with you. I mean, there are some people who sort of say, look, you just put in more data and then you don't need to worry about uncertainty because the model will know everything. Yeah, I think that is... a view that a lot of people have in the field. They're not completely wrong, just like I'm not completely right, in that the models are actually pretty good at a lot of useful things. The problem is... that when you stretch them in the long tail of sort of unusual things, then you can... uncover some gaps. And also, I think when the decisions If you're just interacting with a chatbot, the decisions may not be so consequential, but if we're trying to build self-driving cars that are reliable or medical AI systems that are helping us make you know, diagnosis decisions, then we really do care about getting those probabilities, right? Because I guess when it comes to building AGI, there's sort of, to oversimplify it, two camps, really. One which says you just need scale, you just need more data, more compute, off you go. And then the other that's sort of known we actually need a new architecture to be able to do things better that we can't do at the moment. It sounds like you're very much in the second camp. Yeah, I think we've made a lot of progress. in the first camp. So, of course, our systems are incredibly useful. They're used by billions of people every day. But that doesn't mean that we've… run out of interesting things to discover. So, I'll give you a few examples of things that I think are important areas of research. One example is continual learning. So, the way we currently train our models, and we means everybody in the field, we Um... We train a giant model. And then we... use it in products or release it in various forms. And then a few months later, we train another giant model. and so on. If you compare that to how humans and animals learn, we learn continuously. We're basically constantly getting a stream of data and constantly adapting our connections between our neurons and so on. And our AI systems are not really able to do that very well. They suffer from things like catastrophic forgetting and so on and so forth. Which absolutely links back to the idea of Bayesian thinking that we were talking about earlier. updating system in our minds of incorporating evidence with with with existing knowledge yeah so it turns out if you think of uh learning from a strictly Bayesian point of view. You have your prior beliefs, you get a data point you update them, you get a posterior, and you get another data point, you update them. It turns out that That sort of Bayesian updating can do continual learning and does not suffer from catastrophic forgetting and all these things, in theory. Actually, many of our attempts at doing continual learning in large neural networks our approximations of that Bayesian updating. So that's one area of research is continual learning. Another area where I think we may need breakthroughs is energy efficiency. So if you look at the power consumption of a human brain, It's about 20 watts. I don't want to make an equivalence. It's a light bulb. Yeah, it's a light bulb. It's a rubbish light bulb. Yeah, it's a nice, very energy efficient light bulb, let's say. You know, if you compare that to training a large language model in a big data center, it's orders of magnitude off. I don't want to compare a single brain to a large language model because single brains involve, you know, kind of the single lived experience of a human being. Large language models are basically giant soups of, you know, all of... world knowledge of some kind. But they're definitely less efficient. But they're way less efficient, right? So, we can certainly afford to do more research in more energy efficient, learning. I also think that there may be new architectures that we need to discover. Basically, the two workhorses of modern AI are transformers and diffusion models, which They're great. but there may be completely other... co-evolutions of software and hardware, like novel hardware architectures that may involve very sparse neural networks of various kinds. And, oh, I'll give you another one. Our learning systems are incredibly data inefficient. compared to human and animal learning. So again, [40:00] I think, Bayesian... ideas can help us there. It sort of sounds a bit like you've discovered a magic trick, you know, like this magic trick which like embeds humility and uncertainty, offers the opportunity for continual learning and makes data more efficient. Yeah, I'd like to think. And you can write in a single line of Bayes' rule. So it's like, is this a little bit too good to be true? It's too good to be true, right? Obviously, we've known this for a long time. I think it's important to understand these ideas. So, you know, I think all students of machine learning should at least understand that it's possible. to do these things. The magic trick comes with a big curse. The curse here is that to do all of this is computationally very slow. So essentially, you know, If you look at textbooks in AI, they will explain to you how to do some of these things. But they will say, we can't do these things exactly exactly. because they're computationally hard problems. They're kind of MP complete or MP hard problems. problems to solve, and so we need to approximate them somehow. And you could argue, well, we know how to build ideal AI systems using these concepts. Our modern AI systems are approximations to that. Can we have... Archaic can eat it too. Can we have the best of both worlds here? Because the thing is, what you just said there, about, well, it's very computationally expensive, it's very slow. They were saying that about neural networks in the 80s. Yeah. You're not going to make that mistake twice. Yeah, yeah. I think we can revisit some of these ideas with... The giant state-of-the-art supercomputer parallel connection machine computer that I used in my undergraduate years is actually slower than the Pixel phone that I have in my pocket. Computation is getting better. faster. And also, we have decades of ideas on how to approximate these methods, really efficiently. Right? And so we have the tools. we just need to put them together and maybe come up with a few new ideas. What really strikes me is everything we've discussed. they feel like very human-like qualities, right? Yeah. These sort of ineffable characteristics of humility and honesty. Yeah. And uncertainty and doubt. Yeah. I mean... Did you imagine when you started all these years ago that this would be the thing that our systems are lacking rather than just computational power or power for analysis? I mean, certainly, I didn't really imagine we would be where we are now, right? Because I think if you talk to an AI researcher, they will be saying that they're stunned by the rate of progress. But I also think that… although these are human qualities we want to add, they're also kind of fundamental qualities of intelligence systems. So I think we need to have a concept of intelligence that transcends humans. because, as I mentioned before, we have flaws in our reasoning, we're actually quite bad at making good rational decisions under uncertainty in the real world. we will... miscalculate probabilities or misestimate probabilities and so on. So I think if we build human-centric AI systems, we work backwards from... What do humans need? What are sort of society's biggest problems? What are humanity's biggest problems? And what are the AI systems that we need to solve those things? And for, I think, all problems that matter… I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident. What an amazing point to end on. Stephen, thank you so much. That was brilliant. Thank you, Hannah. For years, we tried to build AI that was focused on accuracy, right? Building systems that crunch through enormous amounts of data to land an answer. And these things, they're astonishingly capable, of course, but they're also quite brittle in some ways. When they fail, they fail with total confidence. But by teaching AI to embrace uncertainty, it gives it something more human, humility and honesty and the wisdom to doubt. isn't a vulnerability doesn't make ai weaker it grounds it in reality making it a collaborator we can actually rely on