Alphabet Unifies AI Units to Shield $140 Billion Search Monopoly

Alphabet整合AI部门以捍卫1400亿美元搜索垄断地位 这家科技巨头利用其自给自足的搜索利润和专有TPU芯片掌控整个AI技术栈，使其在与烧钱初创公司的竞争中跨越创新者的窘境。 本期内容回顾了Google的AI演进历程，从2001年的PageRank和概率语言模型等早期机器学习基础，到2011年YouTube“猫脸论文”以及2017年开创性的“Attention Is All You Need”Transformer研究等重大突破性里程碑。节目详细阐述了与Facebook和Tesla之间激烈的人才争夺与收购战如何催生了OpenAI等竞争对手，并最终在2022年给Google带来了“红色警报”级的市场震荡。最后，主持人分析了Google凭借定制TPU所拥有的庞大基础设施优势、Sundar Pichai领导下将DeepMind与Google Brain合并为Gemini的快速重组，以及其旗下独立自动驾驶子公司Waymo价值数十亿美元的巨大商业潜力。 Google凭借90%的市场份额和1400亿美元的年收入维系着庞大且高利润的搜索垄断地位，在其向利润率较低的AI产品转型过程中构成了教科书式的创新者窘境。 (0:00) 2010年代中期，Google几乎绝对垄断了世界顶尖的AI人才，网罗了Ilya Sutskever、Geoff Hinton、Demis Hassabis和Dario Amodei等未来的先锋人物。 (6:17) Noam Shazeer和George Harik在2001年奠定了Google早期的AI基础，他们提出数据压缩等同于理解语言，并促成了Phil等早期概率模型的诞生。 (9:50) Jeff Dean通过在标准CPU上并行化运行拥有2万亿词的大规模语言模型以实时支持Google翻译，极大地拓展了Google早期的生产级AI能力。 (18:16) Google Brain在2011年发表的“猫脸论文”通过从未标记的YouTube视频中自主合成“猫神经元”，证实了大规模无监督学习的可行性，并彻底变革了YouTube的核心推荐系统。 (41:23)

Acquired hosts Ben Gilbert and David Rosenthal argue that despite being caught off guard by consumer generative AI, Google possesses an unmatched full-stack infrastructure—spanning proprietary silicon, cloud services, and frontier models—that positions it uniquely to navigate the transition away from traditional web search. Full-Stack AI Infrastructure and Custom Silicon (3:00) The hosts assert that Google is the only enterprise operating across all four critical pillars of modern artificial intelligence: consumer applications, a frontier model family (Gemini), hyperscale infrastructure (Google Cloud generating $50 billion in revenue), and proprietary silicon. While competitors rely heavily on Nvidia GPUs, Google developed its Tensor Processing Units (TPUs) after senior fellows Jeff Dean and Urs Hölzle calculated in the 2010s that just three minutes of daily voice search per Android user would require doubling Google’s entire global data center footprint. Jonathan Ross helped architect the TPU as an application-specific integrated circuit (ASIC) leveraging reduced computational precision to drastically cut matrix-multiplication overhead. This hardware integration is further supported by an extensive network of dark fiber purchased at distressed valuations following the 2000 dot-com crash. Gilbert and Rosenthal observe that Google Cloud provides the essential commercial distribution channel that made TPUs viable beyond internal workloads, granting the company a vertically insulated compute stack as industry rivals face persistent third-party hardware constraints. Foundational Research and the Genesis of OpenAI (110:18) Google's technical heritage laid the groundwork for modern deep learning, beginning with early machine learning systems like Phil for AdSense and Franz Och's translation models. Following the 2012 ImageNet breakthrough with AlexNet, Google acquired Geoff Hinton’s DNN Research for $44 million, followed by the acquisition of DeepMind in January 2014. In 2017, Google Brain researchers published the seminal Transformer paper, whose codebase was rewritten by Noam Shazeer to demonstrate that raw scale consistently outperformed hand-tuned architectures. However, this aggressive consolidation of talent and compute sparked unintended industry counter-offensives. Alarmed by Google's emerging talent monopoly, Elon Musk and Sam Altman convened a dinner at the Rosewood Hotel in 2015 to establish OpenAI as an open counterbalance, successfully recruiting Google researcher Ilya Sutskever. OpenAI subsequently utilized Google's published Transformer architecture to build the Generative Pre-trained Transformer (GPT) lineage, turning Google's own breakthroughs into its most formidable competitive challenge. The Innovator's Dilemma and Internal Consolidation (154:05) Prior to 2022, Google management treated artificial intelligence primarily as a sustaining innovation under Clayton Christensen’s framework, assuming breakthroughs would predictably lower operational costs and improve existing products. Wary of reputational risks and recalling Microsoft’s 2016 Tay chatbot incident, Google restricted its 2022 LaMDA public deployment in the AI Test Kitchen to strict five-turn conversations to avoid erratic behavior. When OpenAI’s rapid launch of ChatGPT demonstrated unmediated consumer demand, CEO Sundar Pichai declared an internal 'code red' in December 2022, signaling that generative interfaces represented a disruptive threat capable of cannibalizing high-margin search ads. In response, Pichai executed a major structural realignment by overriding historical independence guarantees to merge Google Brain and Demis Hassabis’s DeepMind into Google DeepMind. The combined group was tasked with shipping Gemini across the company's product suite while carefully balancing AI Overviews against core ad monetization. Bundled Monetization, Moats, and Antitrust Dynamics (194:29) Evaluating Google under Hamilton Helmer’s Seven Powers framework, the hosts argue that Google benefits from massive scale economies by amortizing model training and high inference volumes across billions of daily global queries. On monetization, the hosts counter assumptions that consumer AI cannot support subscription revenue by pointing to Google One, which has surpassed 150 million subscribers and offers a path to monetize advanced AI features at $20 monthly tiers. Furthermore, because conversational AI prompts average over 20 words compared to two to three words in classic search, the hosts relay advertising analyses suggesting conversational inputs yield significantly higher consumer intent for commercial targeting. Concurrently, regulatory pressures resolved with unexpected operational leniency. Although a federal judge ruled in the US antitrust litigation that Google operated an illegal search monopoly, the court declined to impose structural remedies such as forcing the divestiture of Chrome or banning distribution payments. The presiding judge explicitly cited the high-stakes, capital-intensive race in artificial intelligence as a justification for leaving Google's corporate structure intact. The central unresolved question for Google is whether its full-stack structural advantages—spanning custom silicon, distribution, and massive balance sheet scale—can outpace the organizational drag of protecting its core search revenues against nimbler, unencumbered rivals.

播客《Acquired》主持人 Ben Gilbert 与 David Rosenthal 认为，尽管 ChatGPT 的崛起让谷歌一度陷入被动并面临创新者窘境，但凭借自研芯片、前沿模型、云计算及顶级分发渠道的完整垂直整合，谷歌仍拥有其他科技巨头无可比拟的底层壁垒；其能否赢得 AI 时代，关键取决于能否在保护核心搜索利润的同时全面释放技术储备。 技术策源：从PageRank到Transformer (8:20) David Rosenthal 指出，谷歌的 AI 基因始于创立之初。联合创始人 Larry Page 早在 2000 年便提出“人工智能是搜索的终极形态”，核心算法 PageRank 本质上即为统计机器学习。从拼写纠错、2003 年 Jeff Dean 基于 Phil 概率模型编写 AdSense，到机器翻译打破行业基准，谷歌早在 2000 年代就将概率语言模型深度商业化。2011 年 Andrew Ng 与 Jeff Dean 联合创立 Google Brain；2012 年谷歌以 4400 万美元竞购 Geoff Hinton、Alex Krizhevsky 与 Ilya Sutskever 创立的 DNN Research；2014 年又斥资收购 DeepMind。 然而技术积累并未直接转化为先发垄断。2017 年 Google Brain 团队发表的 Transformer 论文原本旨在优化机器翻译，却意外成为整个大模型时代的基石。DeepMind 的早期突破曾刺激 Elon Musk 与 Sam Altman 在 2015 年创办 OpenAI 以打破谷歌的人才垄断；此后 OpenAI 借助 Transformer 架构推出 GPT 系列，反客为主对谷歌构成了前所未有的生存挑战。 全栈垂直壁垒：四支柱模型与TPU基建 (208:49) Ben Gilbert 指出，现代 AI 竞争取决于四大核心支柱：超级应用、前沿基座模型、自研芯片与超大规模云设施。横向对比整个科技界，英伟达缺乏自有云和消费级应用，亚马逊在模型与应用端落后，微软高度依赖外部合作，而谷歌是唯一同时全面掌控这四大支柱的巨头。在芯片端，2014 年 Jeff Dean 测算发现，若全球每位安卓用户每天语音搜索 3 分钟，谷歌数据中心容量就必须翻倍；为解决算力瓶颈，谷歌于 2015 年前正式立项自研张量处理器（TPU），利用低精度量化计算大幅降低矩阵运算成本。 TPU 的成功与年收入达 500 亿美元的 Google Cloud 深度绑定，自有公有云为 TPU 提供了对外商业化与开发者生态依托。此外，谷歌早在 2000 年互联网泡沫后便以极低成本收购大量“暗光纤（Dark Fiber）”，建成了私有高速骨干网。结合 YouTube 独有的海量视频数据资产，谷歌在从文本向多模态与视频互联网演进的过程中构筑了极其坚实的底层基建优势。 “红色警戒”与重组：应对颠覆性创新 (154:05) 在 ChatGPT 出现前，谷歌管理层依据克里斯坦森的创新理论，长期将 AI 视作强化既有业务的“维持性创新”。因担忧遭遇类似微软 2016 年 Tay 聊天机器人的公关风险，谷歌推行极度审慎的产品策略，例如 2022 年 5 月推出的 AI Test Kitchen 对 LaMDA 强制设定了 5 轮对话上限。然而，ChatGPT 在 2022 年底的爆发迅速将 AI 转化为“颠覆性创新”，迫使谷歌直面自身成熟业务的创新者窘境。 面对危机，CEO Sundar Pichai 于 2022 年 12 月拉响全员“红色警戒”。他打破 DeepMind 保持十年的独立运营协议，强行将其与 Google Brain 合并为由 Demis Hassabis 掌舵的 Google DeepMind，重组研发体系并集中力量攻坚 Gemini 模型。同时，谷歌在核心搜索中谨慎测试 AI Overviews，将独立 Gemini 与核心业务区隔，在保护核心搜索百亿美元广告现金流的前提下全力追赶技术代差。 商业化与护城河：规模经济与超级分发 (227:11) 节目通过汉密尔顿·赫尔默的《7种力量》（7 Powers）框架评估了谷歌在 AI 时代的护城河。两位主持人认为，AI 时代的“规模经济”前所未有，谷歌能够将高昂的模型训练与推理成本摊销至每日数百亿次的全球搜索中。在分发渠道上，尽管消费级 AI 的切换成本较低且尚未形成强网络效应，但谷歌控制着 Chrome、Android 及搜索框等不可替代的独占资源（Cornered Resource），且享有长期的用户品牌信任。 在商业变现方面，拥有超 1.5 亿付费用户的 Google One 订阅包展示了消费级 AI 商业化的可行性；此外，相较于平均仅 2 至 3 个词的传统搜索，平均超过 20 个词的 AI 提示词展现出更高精度的商业意图，有望提升广告变现单价。外部监管层面，尽管联邦法官判定谷歌在搜索市场构成垄断，但明确以维持美国 AI 科技竞争力为由，未强制拆分 Chrome 或禁止其向苹果支付默认搜索渠道费，为谷歌的技术反击保留了关键缓冲空间。 《Acquired》总结认为，虽然谷歌在生成式 AI 浪潮早期应对迟缓，且受制于庞大搜索利润所带来的包袱，但凭借全栈垂直整合的基建资产、自研芯片、海量私有数据与数十亿级分发入口，其长期竞争壁垒极为深厚。谷歌能否在这场范式转移中最终胜出，取决于其能否在不瓦解传统商业大本营的前提下，有效将算力与数据壁垒变现为下一代超级应用。

## Background 本期节目由知名商业播客 Acquired 于 2025 年 10 月 6 日发布在 YouTube 平台。视频简介指出，Google 发明了奠定现代 AI 基石的 Transformer 架构并曾招揽全球顶尖 AI 天才与自研 TPU 算力基础设施，却在面对 ChatGPT 崛起时陷入被动；本集全面复盘 Google 长达二十余年的 AI 演进史，探讨其如何直面颠覆旧搜索与保护每年千亿美元暴利的终极“创新者的窘境”。 ## TL;DR 节目详尽拆解了 Google 自 2001 年首个微型语言模型起步，经由 Google Brain、AlexNet 算力觉醒、收购 DeepMind 到 2017 年发表《Attention Is All You Need》发轫 Transformer 的完整历程。全景式复盘了收购 DeepMind 激怒马斯克而催生 OpenAI、顶尖人才流失、ChatGPT 突袭引发全公司“红色警报”及 Bard 仓促发布的内部震荡。随后，节目剖析了皮查伊整合 Google DeepMind、推出 Gemini 统一架构、Waymo 突破全无人商业运营以及 Google Cloud 携自研 TPU 构筑单位经济学护城河的全面反击，并依据 7 Powers 框架权衡其守擂万亿美元帝国的多空逻辑。 ## Atmosphere & style 对谈氛围兼具宏大史诗感与极客视角的严密推演，Ben 与 David 两位主播配合默契，在层层推进的编年史叙事与富有张力的商业博弈复盘之间自如切换。从早期自由放养式的技术顿悟，到遭遇 ChatGPT 突袭时的紧张焦灼，再到集中全公司资源强推 Gemini 的肃穆决断，整个情绪弧线跌宕起伏。听感上既像在翻阅一部波澜壮阔的现代硅谷 AI 演变档案，又像置身于见证科技巨头面临生死战役的商业指挥所。 ## Bulletin - [01:00](https://www.youtube.com/watch?v=lCEB7xHer5U&t=60s) Google 坐拥全球搜索 90% 垄断份额及年化 1,400 亿美元净利润，却因发明的 Transformer 架构被 OpenAI 抢跑引爆而陷入商业史上最典型的“创新者的窘境”。 - [06:19](https://www.youtube.com/watch?v=lCEB7xHer5U&t=379s) 2014 年前后现代 AI 领域的顶尖领袖（包括 Hinton、Ilya、Hassabis、Amodei 等）几乎全数汇聚在 Google，其后的人才出走与并购博弈直接孕育了 OpenAI 与现代大模型竞争格局。 - [01:44:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6241s) 自研 TPU 芯片使 Google 成为全球唯一摆脱高昂英伟达硬件溢价的全栈巨头，内部部署规模达 200 万至 300 万片，建立了单位 Token 最低成本的硬件经济学壁垒。 - [02:58:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10712s) 分拆自 Google X 的 Waymo 在旧金山的订单流水已超越 Lyft，全无人行驶超 1 亿英里并将严重车祸率降低 91%，展现出潜在比肩 Google 本身体量的万亿美元级商业空间。 - [02:40:18](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9618s) 面对 ChatGPT 冲击，Google 整合 Google Brain 与 DeepMind 打造单一底座模型 Gemini，正依托 3,700 亿美元年营收、全球暗光纤骨干网与 YouTube 独家多模态数据全方位反攻。 ## Key takeaways - [08:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=500s) Larry Page 早在 2000 年就定调 AI 是 Google 的终极形态，早期通过数据压缩理论、拼写纠错和 AdSense 算法探索，确立了概率语言模型的商业底层逻辑。 - [20:24](https://www.youtube.com/watch?v=lCEB7xHer5U&t=1224s) Jeff Dean 主导的工程化改造将 Google 翻译单句处理时间从 12 小时缩短至 100 毫秒，开启了大规模分布式架构赋能工业级算法落地的先河。 - [40:44](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2444s) 2011 年 Google Brain 凭借异步分布式架构 DistBelief 发表《猫脸论文》，首次证明无监督深度学习能自主抽象高阶语义，拉开了现代互联网推荐算法革命的序幕。 - [51:35](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3095s) AlexNet 在 2012 年 ImageNet 夺冠引爆 GPU 算力革命，随后 Google 在太浩湖赌场竞拍中击败百度和微软以 4,400 万美元收购 DNN Research，将 Hinton 与 Ilya 揽入麾下。 - [01:01:02](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3662s) 2014 年 Google 以 5.5 亿美元闪电截击 Facebook 和特斯拉对 DeepMind 的竞购，该事件直接激怒马斯克，成为次年催生非营利实验室 OpenAI 成立的核心蝴蝶效应。 - [01:44:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6241s) 为应对语音识别对数据中心的算力翻倍危机，Google 仅耗时 15 个月便自研出第一代专用 ASIC 芯片 TPU，奠定了抗衡英伟达算力霸权的核心底牌。 - [02:01:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7280s) 2017 年 Google Brain 提出开创性的 Transformer 架构，但 Google 仅将其作为优化搜索的改良工具，导致全部 8 位论文核心作者流失，埋下落后于 ChatGPT 的伏笔。 - [02:06:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7561s) 马斯克撤资逼迫 OpenAI 转制为受限盈利实体，Satya Nadella 凭借 Azure 算力与微软百亿美元资金全面下注，使微软成功抢占大模型商业化先机并让 New Bing 挑战 Google 核心腹地。 - [02:20:26](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8426s) 内部安全审查、言论失控顾虑与保护千亿美元搜索广告现金流的权衡，束缚了 Google 内部聊天机器人 Meena 与 LaMDA 的发布，最终在 2022 年底被轻装上阵的 ChatGPT 迎头痛击。 - [02:40:18](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9618s) 拉响全公司红色警报后，Sundar Pichai 终结派系内耗，将 Google Brain 与 DeepMind 合并为统一实体，由 Demis Hassabis 挂帅全力打造单一大模型底座 Gemini。 - [03:03:44](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11024s) Waymo 坚持高冗余传感器与全自动驾驶 Robotaxi 路线，累计研发投入仅约 100 亿至 150 亿美元，正凭借显著降低车祸伤亡的社会经济价值开辟万亿美元级第二曲线。 - [03:22:54](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12174s) Google Cloud 在托马斯·库里安带领下年化营收突破 500 亿美元并实现扭亏为盈，成为向企业级客户分发自研 TPU 算力、打破 CUDA 生态垄断的关键技术出口。 - [03:28:49](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12529s) 自研 TPU 相比采购英伟达 GPU 能削减数倍供应链溢价，加上芯片折旧占数据中心运营成本过半，让 Google 在大模型推理时代具备全球最低的单位 Token 生产成本。 - [03:32:28](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12748s) Google 的多空博弈核心在于：看多者看好其庞大入口分发权、海量广告造血能力及 YouTube 视频训练壁垒；看空者则担忧高价值搜索查询正被 AI 对话分流，且订阅制难以复制传统搜索的高额商业回报。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=lCEB7xHer5U&t=0s) 录音室杂谈与节目开场 节目在两位主持人轻松幽默的幕后闲聊中拉开序幕。Ben Gilbert 提到自己最近在住处附近物色了一间小办公室，原本打算改造成播客录音室，结果体验极其糟糕——房间装的是矿棉板吊顶（drop ceiling），隔音极差，隔壁租客的一举一动听得一清二楚。更要命的是，隔壁似乎是一位律师，整天在电话里滔滔不绝地沟通各种棘手的烂摊子，声音清晰得甚至能直接收进麦克风里。David Rosenthal 打趣道：“那他岂不是要成我们的第三位常驻主持人了？他难道真想让几百万人听见他的保密通话？”在一阵欢笑后，两人迅速切入正题，正式开启《Acquired》2025 年秋季季度的全新篇章。 ### [01:00](https://www.youtube.com/watch?v=lCEB7xHer5U&t=60s) 终极“创新者的窘境”：Google 的 AI 战略悖论 David Rosenthal 抛出了一个极具戏剧张力的商业思想实验：想象你拥有一家极其赚钱的企业，销售的每一个单元都享有惊人的暴利；你所处的市场是全球规模最大的赛道之一，而你不仅身处其中，还被美国政府正式认定为拥有 90% 市场份额的垄断巨头，具备极高的用户锁定壁垒。然而，就在你的顶尖研发实验室里，才华横溢的科学家们联合此前一系列开创性研究，发明了一项革命性技术。这项新技术所催生的产品，在几乎所有核心体验上都彻底碾压了你现有的摇钱树产品。 出于纯粹的学术理想主义与开放开源精神，你的科学家们将这项颠覆性成果以研究论文的形式公之于众。转眼之间，外部初创公司迅速利用这项发明掀起商业化狂潮。按常理，你理应全力以赴颠覆自我，将全部身家押注在新架构上；然而致命的问题随之而来：你根本不知道如何让这款体验惊艳的新产品，赚到哪怕接近旧业务万分之一的利润。这个堪称商业史上最教科书级别的“创新者的窘境”（The Innovator's Dilemma），正是 Google 当下面临的残酷现实。 当前席卷全球的生成式 AI 浪潮——从 OpenAI、ChatGPT、Anthropic 到创下市值神话的英伟达（NVIDIA），其底层根基全部建立在 2017 年 Google Brain 团队发表的里程碑式论文《Attention Is All You Need》（Transformer 架构）之上。十年前，Google 拥有全球密度最高的 AI 顶尖大脑；即便放到今天，Google 手中握有的筹码也堪称举世无双： ```mermaid flowchart TD accTitle: Google的AI全栈资产与商业两难 accDescr: 展示Google拥有的全栈AI资源与面临的搜索现金流颠覆困境 subgraph ASSETS["Google 全栈 AI 护城河"] direction TB M["顶尖前沿模型<br/>(Gemini)"] C["超大规模云基础设施<br/>(Google Cloud 500亿美元体量)"] H["自研AI专用算力<br/>(TPU 唯一匹敌NVIDIA的规模化自研芯片)"] T["全球流量第一入口<br/>(数十亿人高频使用的搜索框)"] end subgraph DILEMMA["创新者的窘境"] direction TB D1["底层技术发明者<br/>(2017 Transformer)"] --> D2["高成本生成式回答<br/>商业模式尚未跑通"] D2 -.->|威胁侵蚀| D3["年利润超千亿美元的<br/>搜索与高毛利广告业务"] end ASSETS --> DILEMMA ``` - **全栈 AI 护城河**：除了拥有顶尖的多模态模型 Gemini，Google 还坐拥年营收达 500 亿美元的 Google Cloud，无需受制于第三方公有云；同时拥有自研 TPU（张量处理单元），是除英伟达 GPU 之外全球唯一实现超大规模部署的专用 AI 芯片集群。业内共识认为，在当今 AI 战场，若不同时具备前沿基座模型与自研芯片能力，极易沦为产业链的附庸通道，而 Google 是唯一两者兼备的巨头。 - **不可替代的入口地位**：即便 ChatGPT 已经成为生成式 AI 时代的“代名词”（如同面巾纸界的 Kleenex），Google 依然掌控着全球绝大多数网民在具备网络意图时的终极前端入口——那个无处不在的搜索框。 - **生死攸关的战略抉择**：Google 究竟应当不惜一切代价押注 AI 的“天赋王权”，还是会因为拼命保护搜索业务每年超过千亿美元的现金流帝国，最终被汹涌的 AI 浪潮吞噬？ ### [04:43](https://www.youtube.com/watch?v=lCEB7xHer5U&t=283s) 节目花絮与十周年预告 在正式深入历史长河之前，主持人向听友介绍了近期的动态与福利。Ben Gilbert 强调了节目的官方邮件通讯订阅与 Slack 社区互动渠道，并重磅宣布《Acquired》即将迎来开播 10 周年纪念。为了与长期陪伴的听众共同庆祝，节目组将于太平洋时间 2025 年 10 月 20 日下午 4 点举办一场大型公开 Zoom 连线，重温早期 LP（限量合伙人）电话会议的温馨氛围。 此外，两人推荐了衍生访谈节目《ACQ2》对 Shopify 创始人兼 CEO Tobi Lütke 的深度对话，探讨 AI 如何重构创业者的思维方式以及电商与技术的未来演进。同时，节目感谢了长期战略合作伙伴摩根大通支付（J.P. Morgan Payments）对节目的鼎力支持，并照例给出了免责声明：本期节目仅供信息参考与娱乐，不构成任何投资建议。 ### [06:19](https://www.youtube.com/watch?v=lCEB7xHer5U&t=379s) 2014 年代：全宇宙的 AI 天才都在 Google 回到整场变革的原点，David 提醒大家回望 Transformer 论文问世前数年的 2014 年代。如果将当年整个前沿人工智能领域最具代表性的大师级人物列一张名单，会发现一个令人震撼的事实——他们几乎全部受雇于 Google： - **Ilya Sutskever**（后来的 OpenAI 联合创始人兼首席科学家）、**Geoff Hinton**（深度学习教父）以及 **Alex Krizhevsky**：凭借 2012 年革命性的 AlexNet 论文彻底点燃现代深度学习革命的三位先驱，全员入职 Google。 - **Dario Amodei**：后来的 Anthropic 创始人兼 CEO。 - **Andrej Karpathy**：后来的特斯拉人工智能与 Autopilot 负责人、OpenAI 创始成员。 - **Andrew Ng（吴恩达）** 与 **Sebastian Thrun**：Google X 与 Google Brain 的早期核心奠基人。 - **Noam Shazeer**：语言模型先驱、Transformer 核心作者、Character.ai 创始人。 - **DeepMind 核心团队**：创始人 Demis Hassabis、Shane Legg 以及后来出任微软 AI 负责人的 Mustafa Suleyman。 在当时的全球顶尖科技版图里，唯一没有全职效力于 Google 的领域宗师，只有身在 Facebook 的 Yann LeCun。 > “这就像在计算机时代的黎明，全世界懂写代码的人总共就那么几十个，而像 IBM 这样的一家公司，把其中几乎所有人全部雇下了。” Ben Gilbert 用这个精妙的比喻描绘当年的行业垄断感。当时的学术生态极为小众：深度学习与现代 AI 并非普罗大众能在网上自学掌握的技术，而是蜷缩在极少数几所名校的特定博士项目里，由屈指可数的几个学术权威指导；学术圈内部更是充斥着长达数十年的门户之见，大多数主流计算机科学家甚至将神经网络视作毫无希望的“异端邪说”。 ### [08:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=500s) Larry Page 的 AI 宿命论与 Google 的早期探索 Google 能在早期集齐这批人才绝非偶然，其源头深植于联合创始人 Larry Page 的个人基因与执念。Larry Page 的父亲 Carl Page 早年就是密歇根大学的计算机科学教授，其博士方向正是机器学习与人工智能。在那个符号主义与专家系统屡屡受挫、全行业遭遇严酷“AI 寒冬”的年代，投身人工智能研究被学界普遍视为浪费生命的边缘行为，而这种特立独行的先驱背景深刻塑造了 Larry 的底层世界观。 从某种广义视角来看，Google 起家的核心基石——PageRank 算法，本质上就是一种统计学方法，天然属于广义计算机科学中的人工智能范畴。早在此后二十年席卷全球的大模型概念诞生之前，Larry Page 对 Google 的战略定位就已经极其明确。早在 2000 年（公司成立仅仅两年之时），Larry 就曾公开断言： > “人工智能将是 Google 的终极形态。如果我们打造出真正的终极搜索引擎，它将能够理解互联网上的全部信息，精准洞悉你的真实所求，并直接奉上最完美的答案。这显然就是人工智能。我们目前离这个目标还差得很远，但我们正在一步步逼近，而这正是我们在 Google 所做的一切工作的本质。” 时间推进到 2000 年底或 2001 年初的某一天，在 Google 简朴的食堂餐桌上，一场看似寻常的午餐闲聊，悄然拉开了 Google 语言模型与深度技术创新的大幕。围坐在一起的三个人，正是 Google 史上极其关键的技术骨干：前十号元老员工、同样拥有机器学习博士学位的 Georges Harik，日后长期统领 Google 搜索核心业务的传奇工程师 Ben Gomes，以及当时刚刚入职不久、日后成为 NLP 领域传奇极客的 Noam Shazeer。三个顶尖头脑的碰撞，即将点亮机器理解人类语言的最初火花。 ### [10:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=620s) 数据压缩即理解：Noam Shazeer 与乔治·哈里克在茶水间的顿悟 早期的机器学习在计算机科学界还是一个相对小众且反主流的冷门分支。当时，从密歇根大学（University of Michigan）拿到机器学习方向博士学位的乔治·哈里克（George Harik，Google 最早的员工之一），正与新入职不久的年轻工程师诺姆·沙齐尔（Noam Shazeer）在 Google 的微型茶水间（micro kitchen）里共进午餐。席间，哈里克随口抛出了自己在读博期间琢磨出的一套理论：“数据的压缩，在技术本质上等同于对数据的理解。” 这套理论的思考逻辑非常朴素却深刻：如果你能把一段庞杂的信息极度压缩并存储起来，之后又能将其还原成原本的面貌，那么执行这一过程的系统唯一能做到这一点的途径，就是它真正理解了这些数据所蕴含的意义。因为将数据降维成更小的体量必然伴随着信息损耗，若想重新复原，就像一个在学校读书的孩子——读完整本厚厚的教科书，把核心概念吸收压缩进记忆中；到了期末考试，只有真正理解了这些概念，才能在试卷上把知识重新还原输出。 节目主持人指出，这个诞生于大约 25 年前的观点，精准预示了今天现代大语言模型（LLM）的本质：将全人类浩瀚的知识库压缩进几百吉字节或若干太字节的权重向量之中。虽然模型内部是一个常人难以直观洞察的高维参数集合，但只要对其进行“解压缩”（推理生成），人类全部的文明与知识就会以连贯的形式被重新还原出来。当年轻的诺姆·沙齐尔听到哈里克这个假说时，整个人当场愣住，并立刻意识到：“如果这真是对的，那其意义将极其深远。” ```mermaid flowchart LR accTitle: 数据压缩与智能理解关系图 accDescr: 展示了海量原始信息通过语义压缩编码为低维表征，并在需要时还原重构为连贯知识的过程。 subgraph S1["输入阶段"] A["全人类海量知识与文本"] end subgraph S2["理解与压缩核心"] B["高维特征提取"] C["核心语义压缩表征<br/>(低维向量 / 模型权重)"] end subgraph S3["还原与输出"] D["解压缩与推理还原"] E["重构原始世界知识<br/>(下一词预测 / 语言生成)"] end A --> B B --> C C --> D D --> E ``` ### [12:11](https://www.youtube.com/watch?v=lCEB7xHer5U&t=731s) 自由放养时代的秘密探索与概率语言模型 这一段见载于史蒂文·利维（Steven Levy）的经典著作《In the Plex》（Google 内幕）中的细节，刚好发生在 2001 年拉里·佩奇（Larry Page）决定解雇工程部门所有管理人员的疯狂时期。在没有任何中层管理者干预的环境下，工程师们基本处于完全的“自我管理”状态。沙齐尔对这个想法着迷不已，与哈里克一拍即合。他们做出了最符合 Google 极客文化的一项决定：直接放下手头分配的所有其他工作，全职扑向数据压缩与语言模型的研究，探索能否通过数据让机器产生对语言的理解。 当时几乎全公司的人都对他们的决定冷嘲热讽，认为这两位顶级天才纯粹是在挥霍青春和才华。然而，传奇程序员杰夫·迪恩（Jeff Dean）的黄金搭档桑杰·格马瓦特（Sanjay Ghemawat）在听说后，轻描淡写地评价了一句：“我觉得挺酷的。”拿到这句话，哈里克便在内部有了无懈可击的反驳武器。每当有人跑来质疑他们的研究是在浪费时间，哈里克就会搬出这套著名的逻辑：“桑杰觉得这是个好主意，而这世界上没有人比桑杰更聪明。既然如此，诺姆和我为什么要听信你的建议，认为这是个坏主意呢？” 这趟探索将沙齐尔和哈里克直接引向了自然语言概率模型（probabilistic models for natural language）的兔子洞——给定互联网上出现的某一个特定词串序列，计算下一个特定词串跟随其后的条件概率分布。这与当今大语言模型（LLM）基于“下一个词预测（next word predictor）”或“下一个 Token 预测（next token predictor）”的核心机制完全如出一辙。 ### [14:14](https://www.youtube.com/watch?v=lCEB7xHer5U&t=854s) “您是不是要找”与工业级语言模型 PHIL 的诞生 这项概率自然语言研究交付给 Google 核心业务的第一个成果，就是搜索框里名垂青史的拼写自动纠错功能——“您是不是要找”（Did you mean...?）。 - **解决核心体验痛点**：在此之前，用户一旦手误打错搜索词，搜索结果就会一片狼藉，必须退回重输。 - **解放服务器负载**：用户每次输错词并重新查询，底层的服务器都要做一次无用的全库检索，这给 Google 宝贵的基础设施造成了沉重的算力惩罚。 - **基于统计概率的高置信度修正**：算法能够清晰洞察到，当 80% 的用户输入“god groomer”后都会立刻删改重输为“dog groomer”。一旦模型的统计置信度足够高，Google 甚至无需提示确认，直接静默更正并展现正确词条的结果。 尝到甜头的沙齐尔和哈里克继续深入，他们打造了一个在当时体量极其惊人的大型语言模型，并戏谑地给它起了一个充满学术圈文字游戏风格的名字：**PHIL**（Probabilistic Hierarchical Inferential Learner，概率分层推理学习器）。 到了 2003 年，苏珊·沃西基（Susan Wojcicki）与杰夫·迪恩正紧锣密鼓地筹备推出 AdSense 广告联盟业务。Google 需要一种技术手段来实时“读懂”第三方发行商网站网页上的具体内容，以便精准匹配并分发 Google 广告库中的广告文本。迪恩看中了 PHIL，凭借惊人的工程效率，仅用了一周时间就写完并上线了基于 PHIL 的 AdSense 最初工程实现。这一创举让现有的 AdWords 广告库存瞬间扩展到全互联网的第三方网页，在一夜之间为 Google 撬动了数十亿美元的全新营收。 ```mermaid flowchart TD accTitle: 语言模型 PHIL 在 Google 早期产品中的应用图谱 accDescr: 展示了语言模型 PHIL 如何从底层算法孵化，并成为拼写检查和 AdSense 广告变现的核心驱动力。 LM["概率语言模型研究<br/>(Noam Shazeer & George Harik)"] --> PHIL["工业级大模型 PHIL<br/>(概率分层推理学习器)"] PHIL --> DYM["搜索业务应用<br/>'您是不是要找' (Did you mean) 纠错"] PHIL --> ADS["商业变现突破<br/>AdSense 网页内容智能理解"] ADS --> REV["盘活全网广告库存<br/>创造数十亿美元新增收入"] ``` ### [16:36](https://www.youtube.com/watch?v=lCEB7xHer5U&t=996s) 调侃杰夫·迪恩的极客梗与 Google 翻译架构突破 杰夫·迪恩在解决 Google 各类基础设施难题时的传奇表现，催生了 Google 内部风靡一时的“杰夫·迪恩事实”（Jeff Dean facts，致敬查克·诺里斯笑话）： - “真空中的光速原本大约只有每小时 35 英里。后来杰夫·迪恩花了一个周末重写并优化了物理底层定律。” - “杰夫·迪恩的个人密码是圆周率 $\pi$ 的最后四位数字。” - “对于计算机科学家来说 NP 意味着多项式复杂性难题，但对杰夫·迪恩而言，NP 的意思只是‘没问题’（No Problemo）。” 到了 2000 年代中期，PHIL 带来的系统负荷已极为惊人，甚至一度霸占了 Google 全球数据中心 15% 的全部算力基础设施，足见早期自然语言系统对计算资源的极端消耗。 时间来到 2007 年，这是 Google 黄金十年的顶峰。从 2003 年到 2013 年，Google 几乎推出了日后享誉世界的所有王牌应用：Maps、Gmail、Docs，再到后来的 Chrome 和 Android；而从 2013 年直到 Gemini 推出之前的整整十年里，Google 鲜有现象级的新产品问世。在 2007 年前后，由德国自然语言处理与机器学习博士弗朗茨·奥赫（Franz Och）作为首席架构师主导的 **Google 翻译（Google Translate）** 正在崛起。 正值美国国防高级研究计划局（DARPA）举办其著名的机器翻译公开挑战赛，奥赫带领团队训练了一个在当时更为庞大的统计语言模型参赛。该模型在评估翻译质量的国际标准 BLEU 指标（Bilingual Evaluation Understudy）上拿下了前所未有的天文学级别超高分，彻底碾压了所有竞争对手。当捷报传到杰夫·迪恩耳中时，这位工程大师立即找到了奥赫，兴奋地追问这项革命性技术何时才能真正落地并规模化部署。 ### [20:24](https://www.youtube.com/watch?v=lCEB7xHer5U&t=1224s) 杰夫·迪恩降临：将机器翻译从 12 小时缩短至 100 毫秒 Google 翻译团队早期在内部测试大规模统计语言模型时，面临着一个令人哭笑不得的工程瓶颈。Jeff Dean 和 Noam Shazeer 后来在 Dwarkesh Patel 的播客中回忆过这段往事：当时 Franz Josef Och 领导的翻译团队基于 Google 搜索索引建立了一个包含 2 万亿词标记（tokens）语料库的 N-gram 语言模型。这套模型在参加 DARPA 机器翻译评测竞赛时拿到了极高分，但当被问及何时能部署到线上生产环境时，研究人员却连连摆手，直呼这根本不可能——"这不是产品，纯粹是学术研究，我们无法发布这个模型。" 原来在 DARPA 比赛中，官方规则是周一给出待翻译句子，周五前提交结果即可；团队索性压榨全部可用算力，让这台笨重的庞然大物慢慢跑，平均翻译单单一句话就要耗费整整 12 个小时。 ```mermaid flowchart TD accTitle: Google 翻译架构从串行到分布式并行的改造 accDescr: 展示 Jeff Dean 如何将原本耗时 12 小时的单句翻译重构为跨数据中心 CPU 并行处理并在 100 毫秒内完成的过程。 subgraph Before["改造前：串行计算 (耗时 12 小时)"] A["输入原始句子"] --> B["单机逐词顺序翻译"] B --> C["串行输出目标句子"] end subgraph After["改造后：分布式并行 (耗时 100 毫秒)"] D["输入原始句子"] --> E["拆解为词组与独立片段"] E --> F["切片分发至多数据中心 CPU 并行处理"] F --> G["分布式重组还原语义"] G --> H["返回即时翻译结果"] end ``` 被戏称为“工程界查克·诺里斯（Chuck Norris）”的 Jeff Dean 随即空降到翻译团队，并表示：“把代码给我看看。”在接下来的几个月里，Jeff Dean 彻底重构了算法架构。原先的程序是按部就班逐字逐句串行处理，但实际上翻译文本完全可以拆解成互相独立的片段同时进行，哪怕初期仅做粗粒度拆分，后续再将语义拼接还原，也能以极高效率保留原意。更关键的是，Jeff Dean 与 Sanjay Ghemawat 当年亲手打造的 Google 底层分布式架构天生就为高度并行化而生。Google 堪称全球在跨数据中心调度数以万计 CPU 并行分发负载领域最强悍的公司。通过将其打散为微小任务并行运算，Jeff Dean 奇迹般地将单句平均翻译时长从 12 个小时骤降到 100 毫秒，使其一举达成了直接上线生产环境的要求。 这也是 Google 历史上首个真正被部署到核心生产环境中的“大型语言模型”。这一技术突破随即引发了全公司的连锁反应：既然该模型能够精准理解上下文并预测后续词汇，团队很快意识到它可以被广泛应用于其他黄金业务。最直观的落地便是搜索框的实时输入预测（Autocomplete），而更具商业价值的则是 AdWords 的核心——广告质量得分（Ad Quality Score）。质量得分的本质就是预测用户对特定广告文案的点击率（CTR），而一个擅长消化文本、理解语义并进行预测的语言模型，直接为 Google 带来数十亿美元真金白银的利润转化。 - **训练语料规模**：源自 Google 搜索索引的 2 万亿词（trillion words）大规模语料。 - **性能飞跃**：单句推理时间从 12 小时直接压缩至 100 毫秒，实现了超过 40 万倍的速度跃升。 - **核心算力底座**：彼时驱动该系统的全部为通用 CPU 集群，而非后来的 GPU 或专用芯片。 --- ### [24:05](https://www.youtube.com/watch?v=lCEB7xHer5U&t=1445s) 斯坦福 SAIL 实验室与硅谷人才枢纽：塞巴斯蒂安·特龙入职 Google 2007 年是 Google AI 发展史上至关重要的一年。4 月，Larry Page 正式将斯坦福大学人工智能实验室（SAIL）主任塞巴斯蒂安·特龙（Sebastian Thrun）招致麾下。SAIL 在人工智能领域底蕴深厚，其历史可追溯至 20 世纪六七十年代的第一波 AI 浪潮（当时 Larry Page 的父亲也活跃于计算机学界），后来经历过关停，直到 2000 年代初才在特龙手中重启焕发生机。特龙加入 Google 的过程本质上是一场精妙的“人才收购”（Acqui-hire）：当时他和几位研究生正准备创办一家新公司，手中甚至已经拿到了来自顶级风投机构 Benchmark 和 Sequoia Capital 的投资意向书（Term Sheets）。Larry Page 听闻后直接登门截胡，开出条件：“如果我们直接把这些钱换成入职签字费，在你们还没正式创立之前就把公司收购了，怎么样？” 在特龙掌舵 SAIL 期间，该实验室不仅汇聚了顶尖学者，更宛如整个硅谷现代科技领袖的孵化温床。大量斯坦福本科生在此一边攻读计算机或符号系统（Symbolic Systems）学位，一边参与前沿课题： - **Chris Cox**：在此开启了技术与 AI 研究生涯，日后成为 Meta（Facebook）的首席产品官（CPO）。 - **Sam Altman**：大一大二期间曾在 SAIL 担任本科生研究员，随后在 2005 年夏天从斯坦福退学，带着本地移动社交网络项目 Loopt 参加了 Y Combinator 的第一期孵化班（二人打趣回忆起当年乔布斯同台发布会上 Sam Altman 穿着“双层立领 Polo 衫”登台的青涩时代）。 这证明推动现代人工智能与移动互联网浪潮的，始终是同一批极小范围的核心精英圈子。特龙的加入，正式为 Google 开启了一扇直接抽水全球顶尖学术天才的大门。 --- ### [27:17](https://www.youtube.com/watch?v=lCEB7xHer5U&t=1637s) Project Ground Truth 与杰夫·辛顿的 2007 年 Google 演讲 特龙加入 Google 后的开山之作便是主导了 Project Ground Truth，这项工程奠定了如今 Google 地图的真正基石。在此之前，Google 地图虽已上线，但底层的真实道路数据完全依赖 Tele Atlas 与 NAVTEQ 两大寡头供应商。这两家价值数十亿美元的测绘巨头提供的数据不仅成本极高，而且质量参差不齐；所有厂商使用同一份数据源，导致任何人都无法做出差异化的优质体验。特龙凭借此前在街景（Street View）项目中大规模调度车辆车队采集影像的成熟经验，在 Google 发起了一场“推倒重来”的雄心壮志，决定直接利用街景照片、美国人口普查数据等 40 多种数据源，白手起家绘制全新的全球地图。 这项工程展现了当时人机结合的真实面貌：在缺乏端到端自动化 AI 技术的年代，Google 聘请了超过 1,000 名驻印度的外包数据分析师，专门逐一核对街景图像与卫星数据的偏差，以纯手工的方式一点点绘制修正道路轨迹。Ground Truth 取得了空前成功，让特龙在公司内部赢得了极高的技术威望。 趁着 Ground Truth 的大获全胜，特龙开始向 Larry Page 和 Sergey Brin 游说一个长远的人才战略：Google 应当不惜代价大举招揽学术界顶尖的 AI 教授。他提议允许教授们保留原有的大学教职与终身职位，以兼职方式进入 Google 从事核心产品项目；学者们不仅能看到自己的理论成果被全球亿万用户日常使用，还能领到丰厚的薪酬和 Google 股票，这对学校、教授与公司而言完全是“三赢”局面。 创始人毫不犹豫地拍板批准了这一计划。2007 年 12 月，特龙邀请了一位来自多伦多大学、在当时尚属“边缘流派”的机器学习教授来到 Google 园区举办 Tech Talk。这位学者正是日后被尊称为“深度学习教父”的杰夫·辛顿（Geoffrey Hinton）。在那个全行业对人工神经网络弃之敝履的漫长 AI 寒冬里，辛顿带着他的博士与博士后团队，孤悬学界一隅坚守阵地，而这场在 Google 园区内看似不起眼的技术讲座，悄然埋下了席卷随后二十年现代 AI 革命的最初火种。 ### [30:38](https://www.youtube.com/watch?v=lCEB7xHer5U&t=1838s) 符号逻辑与神经网络的百年反讽：杰弗里·辛顿的学术血脉 在现代人工智能爆发之前，神经网络曾在长达三四十年的时间里沦为学术界的边缘异类。当时绝大多数主流计算机科学家都认为神经网络已被证伪、走入了死胡同。关于现代深度学习之父杰弗里·辛顿（Geoffrey Hinton），存在一个极富戏剧性的历史巧合：他是乔治·布尔（George Boole）与玛丽·布尔（Mary Boole）的玄孙。乔治·布尔正是布尔代数和布尔逻辑的发明者，而布尔逻辑正是确定性、符号逻辑以及传统计算机科学大厦的最核心基石。 这种血统联系在今天看来尤为滑稽。布尔逻辑代表的是符号主义 AI（Symbolic AI）——系统遵循极其严密的 `if-then` 规则树与预设指令，是完全确定性的体系；而辛顿终其一生所捍卫的神经网络，则是一种彻底相反的、非确定性的统计概率系统。这也再次印证了当年这门学科在主流计算机科学领域被视为多么离经叛道。 ### [31:39](https://www.youtube.com/watch?v=lCEB7xHer5U&t=1899s) 算力瓶颈的消解与辛顿的 2007 年谷歌演讲 神经网络的思想本身并不新颖，在理论上也极具吸引力，但在过去几十年里，最大的死穴在于多层计算所消耗的惊人算力。此前，计算机神经网络通常只能做到一层或屈指可数的个位数层级。辛顿和他的前博士后研究员杨立昆（Yann LeCun）开始在同行圈内极力倡导：如果能够构建拥有极多隐层的“深度神经网络”（即深度学习，Deep Learning），这一技术路线的巨大潜力就能彻底变现。 问题的本质从来不是构想有误，而是当时缺乏能够承受海量矩阵乘法运算并跨层反向传播的算力。到了 2007 年前后，随着摩尔定律的持续推进，计算机算力终于成长到了足以验证这些理论的临界点。就在这一年，辛顿受邀前往谷歌总部发表了一场著名的技术演讲（Tech Talk）。当时塞巴斯蒂安·特龙（Sebastian Thrun）和杰夫·迪恩（Jeff Dean）等谷歌顶尖工程师都在场。他们此前已经在谷歌翻译（Google Translate）和语言模型中尝试过很多并行统计方法，辛顿展示的深层架构让他们意识到：如果能将深层网络搬上谷歌强大的基建，机器识别复杂模式和理解海量数据的能力将发生质的飞跃。 ```mermaid flowchart TD accTitle: 深度学习在谷歌的落地脉络 accDescr: 展示从辛顿演讲到Google X及Google Brain成立的演进过程。 subgraph Phase1["理论与学术连接 (2007)"] H["Geoff Hinton 理论突破<br/>(深度学习/多层神经网络)"] --> TT["2007 Google Tech Talk<br/>震撼 Jeff Dean 与 Sebastian Thrun"] end subgraph Phase2["人才机制创新 (2008-2011)"] TT --> PartTime["学术界兼职引进机制<br/>(保留教职 + 谷歌顾问/实习生)"] PartTime --> HintonIntern["Hinton 以 60 岁高龄<br/>担任谷歌暑期实习生"] end subgraph Phase3["组织架构变革 (2009-2011)"] PartTime --> GoogleX["2009 成立 Google X<br/>(登月工厂)"] GoogleX --> Project1["Project 1: 自动驾驶项目<br/>(后来的 Waymo)"] GoogleX --> Project2["Project 2: Google Brain (2011)<br/>(Andrew Ng + Jeff Dean + Greg Corrado)"] end ``` - 演讲结束后，特龙迅速将辛顿引入谷歌阵营，起初聘其为顾问。为了绕过公司严格的全职与兼职人事聘用政策，到了 2011 至 2012 年前后，已经年过六旬的辛顿名义上甚至挂着谷歌“暑期实习生”的头衔。 - 特龙构建了一套行之有效的人才合作模式：允许顶尖计算机与机器学习学者保留大学终身教职，同时以兼职、顾问或实习生的形式深入参与谷歌内部产品的重大工程研发。 ### [34:12](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2052s) Google X 的诞生与第二大登月项目 特龙从高校引进尖端智力的试验获得了巨大成功。2009 年底，特龙与谷歌联合创始人拉里·佩奇（Larry Page）、谢尔盖·布林（Sergey Brin）决定将其制度化，正式成立一个全新的颠覆性研发部门——Google X（登月工厂）。特龙亲自挂帅掌管 Google X 的第一个项目（即后来的 Waymo 自动驾驶汽车，暂且按下不表），而该实验室内部立项的第二个项目，则成为了彻底改写全球 AI 格局与整个科技史的关键拐点：Google Brain。 ### [35:10](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2110s) 赞助支持：摩根大通支付（JP Morgan Payments） 在展开 Google Brain 的故事前，节目穿插介绍了赞助商摩根大通支付（JP Morgan Payments）的资金管理解决方案（Treasury Solutions）： - 资金管理（Treasury）已从传统的后台记账职能转变为现代企业的战略杠杆。 - 系统支持在 200 个国家/地区、以 120 种不同货币实时监控和调度企业现金流。摩根大通支付平台日均清算规模高达 10 万亿美元。 - 提供“Pay-by-Bank”等前沿服务，允许客户直接通过银行账户进行代币化、安全的即时收付款。通过单套 API 对接，即可无缝覆盖 45 个以上国家和地区。 ### [37:26](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2246s) 走廊里的会面：Google Brain 的立项与 DistBelief 架构 当特龙全职投身谷歌后，他在斯坦福大学人工智能实验室（SAIL）的主任职位交棒给了另一位机器学习泰斗——吴恩达（Andrew Ng）。特龙随后又把吴恩达争取到谷歌兼职，每周在谷歌园区工作一天。正值 Google X 刚刚组建的 2010 至 2011 年间，吴恩达在园区散步时碰到了杰夫·迪恩。迪恩向吴恩达讲述了自己与弗朗茨·奥赫在语言模型上的探索，以及辛顿在深度学习方面的突破；吴恩达则分享了他在斯坦福 SAIL 的最新进展。两人迅速达成共识：时机已经成熟，必须在谷歌高度可并行的分布式基础设施上，放手一搏搞一个规模空前的超大型深度学习模型。 ```mermaid flowchart LR accTitle: Google Brain 早期技术架构冲突 accDescr: 对比主流学术界所要求的同步单机架构与Jeff Dean采用的分布式异步CPU架构。 subgraph Academic["当时学术界共识"] direction TB A1["单机高密度计算"] A2["高度依赖 GPU 并行"] A3["强制要求全系统同步更新<br/>(Synchronous SGD)"] end subgraph GoogleBrain["Jeff Dean 的工程反击"] direction TB G1["DistBelief 基础设施"] G2["横跨数千台普通 CPU 集群"] G3["异步分布式参数传递<br/>(Asynchronous Parallel)"] end Academic -.->|"理论质疑：无法收敛"| GoogleBrain ``` 谷歌内部对此并非毫无顾虑，此前他们曾先后启动过名为“Brains on Borg”（Borg 为谷歌的核心集群管理系统）和“Cortex”的两个神经网络项目，最终都以失败告终。团队内部留有相当严重的“技术疤痕”，许多人都怀疑大型神经网络究竟能否在谷歌的架构上跑通。为了打破这一僵局，吴恩达和杰夫·迪恩又拉来了谷歌内部兼具神经科学博士背景的资深研究员格雷格·科拉多（Greg Corrado）。2011 年，这三人正式在 Google X 内部启动了第二个项目：Google Brain。 杰夫·迪恩承担了最硬核的基础系统工程，并为这套底层系统命名为 **DistBelief**。这个名字是一个绝妙的双关语：它既指代“分布式置信度”（Distributed Belief），也直指英文原意的“怀疑 / 不可置信”（Disbelief）——因为当时无论是学术界的主流学者，还是谷歌内部的工程团队，绝大多数人都确信这项试验注定会失败。 当时机器学习领域的学术权威普遍主张神经网络训练必须是严格**同步**的（Synchronous）。这意味着所有高并发计算最好堆叠在一台机器的高密度硬件上（类似于 GPU 方案），以便在参数更新前随时查询系统所有节点的最新状态。而杰夫·迪恩却要做一件颠覆学术常识的事：将模型拆散，利用数千台并不具备紧密同步机制的常规 CPU 服务器，以分布式异步的方式暴力并行推进神经网络的训练。 ### [40:44](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2444s) 分布式架构 DistBelief 与不可思议的异步参数更新 通常情况下，在跨越海量 CPU 核心、甚至跨数据中心分布的大规模系统上训练神经网络是一场工程噩梦。传统理论认为，每台机器在继续计算前，都必须等待其他所有机器同步最新参数，否则延迟会把整个集群拖垮。然而，Jeff Dean 等人设计的分布式深度学习系统 DistBelief 却走了一条离经叛道的路线：它完全以异步方式运作，计算节点根本不等其他核心的最新参数，直接基于“过期数据”（stale data）更新参数。在直觉上，这种做法几乎注定会失败，但令人难以置信的是，它不仅能跑通，而且效果极佳。 DistBelief 搭建完毕后，团队迫切想知道它能否支撑起前沿的深度神经网络研究。2011 年底，他们提交了一篇正式名称为《Building High-level Features Using Large Scale Unsupervised Learning》（利用大规模无监督学习构建高级特征）的论文。不过在 Google 内部以及整个 AI 领域，没人念这个冗长的标题，所有人都不约而同地称其为“猫脸论文”（The Cat Paper）。 - **硬件规模**：该系统动用了 1,000 台独立机器，总计 16,000 个 CPU 核心。 - **无监督测试**：从 YouTube 视频中提取了完全未标注的视频帧直接喂给模型。 - **桑达尔·皮查伊（Sundar Pichai）的印记**：主播在筹备本期节目与 Sundar 交流时，Sundar 明确提到，当年猫脸论文摆上他桌面的那一刻，是 Google 历史上最深刻印在他脑海里的关键里程碑之一。 - **划时代的 TGIF**：随后 Google 内部的全员大会（TGIF）上展示了猫脸论文的研究成果，亲历此事的 Googler 们回忆道：“那场 TGIF 彻底改变了一切。” --- ### [42:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2552s) “猫脸神经元”与万亿美元级的内容推荐革命 猫脸论文从根本上证明了一件事：在没有任何人工标签或人工监督的情况下，足够庞大的深度神经网络完全能够自主在海量数据中提取出具有实际语义的高阶特征。这项研究不仅印证了计算机科学家们长久以来的理论构想，还证明了 Google 的分布式基建能够将这一设想规模化落地。但主播们指出，该论文的技术突破固然耀眼，其背后的商业影响却更为惊人——它间接在随后的十年里为 Google、Facebook 和字节跳动催生了数千亿美元的商业价值。 当时的 YouTube 坐拥海量用户上传内容，却深陷推荐瓶颈。平台迫切希望从单纯的视频搜索工具转型为高粘性的内容消费目的地，通过信息流拉长用户的停留时长。然而，当时的推荐算法极度依赖上传者自己填写的标题和简介，而用户填写的描述往往质量极差。如果机器看不懂视频里究竟放了什么，就根本无法精准分发。猫脸论文的诞生完美解决了这一困境。 ```mermaid flowchart TD accTitle: 无监督视频特征学习到推荐系统 accDescr: 展示从YouTube海量无标注视频帧，通过DistBelief分布式神经网络自动聚类出猫脸等概念，并最终应用于推荐与内容安全系统的全流程。 subgraph InputData["原始数据输入"] A["1000万帧随机 YouTube 视频截图<br/>(无人工标注)"] end subgraph DeepLearning["DistBelief 分布式神经网络"] B["1,000 台机器 / 16,000 CPU 核心"] C["9 层深度自编码/重构网络"] B --> C end subgraph FeatureDiscovery["自主特征涌现"] D["顶层单个神经元强力激活<br/>(自主抽象出'猫脸'特征)"] E["衍生识别：iPhone、棒球、特定物品"] end subgraph BusinessValue["商业与工程落地应用"] F["个性化推荐信息流<br/>(提升 Dwell Time)"] G["低俗及色情内容过滤"] H["版权精准识别与广告分成"] end A --> B C --> D D --> E E --> F E --> G E --> H ``` Jeff Dean 曾生动地总结这一实验：“我们在 2,000 台计算机、16,000 个核心上实现了无监督学习。模型试图从高阶表征中重构视频帧，训练一段时间后，网络最高层的一个神经元会对‘猫的图像’产生强烈的放电激活反应。它从未被告知什么是猫，但因为在训练数据中见过足够多正对镜头的猫脸，这个神经元就专门针对猫点亮，其他内容则不反应。”既然能无监督识别出“猫”，系统就能同样无监督识别出“iPhone”或“旧金山巨人队”，进而引爆了包括色情内容过滤、版权识别和广告分成在内的整套工业级中台能力。 - **AI 时代的真正起点（2012 年）**：大众常将 2022 年底 ChatGPT 的问世视作 AI 元年，但 David Rosenthal 提出，对于拥有社交信息流、依赖推荐和分类算法的互联网巨头而言，AI 时代早在 2012 年就已经开启。 - **行业连锁反应**：Google 凭借此技术将 YouTube 打造成全球最大的媒体资产；Facebook 随后跟进，挖来 Yann LeCun 成立 FAIR（Facebook AI Research）并赋能 Instagram；字节跳动更是将这一算法逻辑推向极致做出了 TikTok；随后又倒逼 Meta 和 Google 推出 Reels 与 Shorts。在随后的十年中，全球人类绝大多数的闲暇时间都被这套始于“猫脸”的推荐网络所主导。 --- ### [47:12](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2832s) AlexNet 的大爆炸时刻：从 CPU 到消费级游戏显卡 在猫脸论文发布的同时，2012 年的计算机视觉领域迎来了被英伟达（NVIDIA）CEO 黄仁勋称为 AI 领域“大爆炸时刻”（Big Bang moment）的重大事件——AlexNet 的诞生。彼时还在多伦多大学的 Geoff Hinton，带着两名才华横溢的研究生 Alex Krizhevsky 与 Ilya Sutskever（后来 OpenAI 的联合创始人兼首席科学家），正准备参加由斯坦福大学李飞飞主导的 ImageNet 图像识别大赛。 当时的 ImageNet 是计算机视觉领域的殿堂级赛事，包含利用亚马逊 Mechanical Turk 众包标注的 1,400 万张图像。历年参赛算法往往由顶级学府在大型超级计算机的 CPU 上运行，即便拿下冠军，其错误率也常年徘徊在 25% 以上，意味着每四张图就识别错一张，根本无法应用于实际生产环境。而在 2012 年的比赛中，AlexNet 横空出世，将错误率一举砸到了 15% 左右。这种在一年之内将相对错误率降低 40% 的技术断层，在计算机视觉甚至整个计算机科学史上都从未发生过。 这一巨大飞跃的核心机理并不在于算法层面的彻底重构，而在于硬件层面的降维打击。以往深度学习学术界受制于摩尔定律，只能依赖 CPU 计算几层浅层网络；而多伦多团队产生了一个天才的想法：为何不抛弃 CPU，改用一种天生为了处理多边形和像素渲染、具备超高并行计算能力的芯片？ - **破局硬件**：他们没有申请动辄百万美元的超级计算机预算，而是直接去百思买（Best Buy）或零售店买了两块 NVIDIA GeForce GTX 580 消费级游戏显卡，单块售价仅需数百美元。 - **CUDA 重构**：团队使用 NVIDIA 专门为通用 GPU 计算设计的编程语言 CUDA，将深度卷积神经网络算法彻底重写，并在两块现成的游戏卡上完成了全量训练。 - **英伟达的命运转折**：AlexNet 的空前胜利向全世界证明，游戏显卡或者专用并行加速硬件才是深度学习的终极温床。这不仅彻底改变了 AI 学术界的研究范式，也单枪匹马地将 NVIDIA 从一家面临增长瓶颈的 PC 游戏显卡外设厂商，推向了未来全球市值第一的 AI 算力帝国之巅。 ### [51:35](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3095s) AlexNet 之后的轰动与 DNN Research 的诞生 在科学与技术史上，有些突破的发生方式非常奇特：前半段的飞跃几乎是在一瞬间彻底引爆，而为了搞清后半段的机理与应用，往往需要整个学术界再花上许多年的时间去摸索。当你脑海中灵光一闪并亲手将其实验出来，随后突然意识到“我的天，我刚刚跨出了该领域的下一大步”，那种感觉就像在电子游戏里瞬间解锁了下一层关卡。AlexNet 在 ImageNet 竞赛中取得惊人胜利之后，整个计算机科学界彻底沸腾了，原本对神经网络持怀疑态度的人们也开始纷纷闭嘴并转变立场。 来自多伦多大学的师徒三人——杰弗里·辛顿（Geoff Hinton）、亚历克斯·克里热夫斯基（Alex Krizhevsky）和伊利亚·苏茨克维（Ilya Sutskever）——做出了一个顺理成章的决定：他们联合创立了一家名为 DNN Research（深度神经网络研究）的公司。这家公司没有任何实际商业产品，核心资产就是这三位刚刚凭借深度卷积神经网络引爆 AI 界的顶级研究学者。很多人误以为这家公司刚成立就直接顺理成章地被 Google 迅速买下，但实际背后的内幕故事要疯狂、曲折得多。 - **核心突破**：AlexNet 凭借 GPU 加速与多层卷积神经网络，以断层优势拿下 ImageNet 冠军，终结了学术界对神经网络的长期质疑。 - **公司属性**：DNN Research 是一家纯研究型初创实体，没有既定营收模型或企业软件，其资产完全是 Hinton 三人的智力资本与前沿技术。 ### [52:41](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3161s) 太浩湖赌场的疯狂竞拍：百度抢先出价与四方角逐 DNN Research 收购战的第一枪其实是由百度（Baidu）打响的。百度率先抛出了一份价值 1200 万美元的出价。面对这笔巨款，辛顿作为一个纯粹的学者，完全不知道该如何给这样一家没有产品的公司定价，也不知道百度的报价是否合理。于是，辛顿采取了典型的学术求真思维来探索真正的“市场公允价值”：他决定组织一场严格受控的拍卖会。规则规定，每次有人给出新竞价，倒计时时钟就会重置一小时，给其他竞争者留出跟进的时间。 辛顿随后联系了学术界出身、如今分散在各大科技巨头中的熟人与好友，挑选那些他认为适合继续做深度学习研究的大平台，邀请他们加入竞拍。受邀名单包括百度、Google、微软（Microsoft），以及第四个令人意想不到的竞拍方——不是尚未建立正式 AI 实验室的 Facebook（当时 Yann LeCun 尚未加入主导 FAIR），也不是特斯拉或多年后才成立的 OpenAI，而是当时才成立大约两年的英国初创公司 DeepMind。辛顿一直非常了解并极其尊重戴密斯·哈萨比斯（Demis Hassabis），因此 DeepMind 也受邀入局。尽管 DeepMind 是一家缺钱的初创企业，刚开拍没多久就因资金不足被迫退场，但这一细节揭示了当时顶级 AI 圈子早已紧密相连。 整个拍卖过程发生在当时称为 NIPS（现为 NeurIPS）的神经信息处理系统大会期间。辛顿本人坐在太浩湖（Lake Tahoe）哈拉斯赌场酒店（Harrah's Casino）的客房里，通过笔记本电脑远程操控这场竞价。随着《Genius Makers》（由 Cade Metz 撰写）中记录的细节，微软退出了又重新杀回，DeepMind 早早退出，竞价节节攀升，最终演变成百度与 Google 之间白热化的双雄争霸。 ```mermaid flowchart TD accTitle: DNN Research 拍卖流程示意图 accDescr: 展示四方竞购 DNN Research 的过程，最终在 4400 万美元时由研究团队决定加盟 Google。 subgraph 竞拍发起 A["百度率先出价<br/>1200 万美元"] --> B["Geoff Hinton 发起拍卖机制<br/>每次加价重置 1 小时倒计时"] end subgraph 四方竞标阵容 B --> C["DeepMind<br/>因资金有限早期退出"] B --> D["微软 Microsoft<br/>中途退出后重新入局"] B --> E["百度 Baidu<br/>一路跟投紧咬 Google"] B --> F["谷歌 Google<br/>志在必得的核心买家"] end subgraph 竞拍终局 E & F --> G["金额飙升至 4400 万美元"] G --> H["三位研究者主动叫停<br/>选择心仪去向 Google"] end ``` - **拍卖地点**：内华达州太浩湖的 Harrah's 赌场酒店客房（NIPS 会议期间）。 - **四方角逐者**：百度、Google、微软以及 DeepMind。 - **终局抉择**：当竞标价一路狂飙至 4400 万美元时，三位研究人员互相商量，认为钱已经完全足够，而且他们内心真正渴望去搞科研的地方始终是 Google。于是他们主动叫停了拍卖，直接接受 Google 的 4400 万美元收购。 - **股权分配**：原本计划三人平分（各占约 33%），但 Alex 和 Ilya 主动对辛顿表示他应该拿大头，最终方案变更为辛顿独占 40%，Alex 与 Ilya 各自分得 30%。 ### [56:14](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3374s) Google Brain 实力倍增与技术红利外溢 辛顿、克里热夫斯基和苏茨克维三人加入 Google Brain 后，如同一针超级强心剂，全面加速了 Google 内部所有深度学习项目的推进。这笔仅仅 4400 万美元的“人才收购”为 Google 带来了难以估量的商业回报。在塞巴斯蒂安·特龙（Sebastian Thrun）离开后接掌 Google X 的阿斯特罗·特勒（Astro Teller）曾在《纽约时报》的专访中透露过一个惊人事实：Google Brain 技术为 Google 核心业务——包括搜索算法、广告点击率预估以及 YouTube 推荐流——所带来的微小百分比效率提升，所产生的超额利润早已远远超过了 Google X 以及全公司多年来在所有其他“登月项目”（Moonshots）上投入的总研发成本。当一项技术能够将一个数百亿、数千亿美元体量的业务提升哪怕两三个百分点，从中榨出的收益就是一笔天文数字。 - **核心影响**：三位顶尖学者的加入让 Google Brain 成为全球顶尖的深度学习圣地。 - **财务放大效应**：对搜索、广告及 YouTube 的微幅推荐与算法改良，直接带来了数十亿美元计的纯增量收入，覆盖了 Google X 各种烧钱实验的开销。 ### [57:25](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3445s) 赞助商 Sentry：为现代 AI 大模型基础设施站岗 在展开下一阶段关于 DeepMind 的宏大叙事之前，节目插播了新合作伙伴 Sentry 的介绍。作为超过 400 万开发者信赖的代码与系统监控平台，Sentry 致力于在用户察觉之前精准定位并修复软件错误、延迟与性能故障。 节目重点介绍了 Sentry 与大模型领军企业 Anthropic 的合作案例。在训练超大规模 AI 模型时，底层的细微故障会迅速引起连锁反应：如果一个计算集群节点发生硬件异常或网络挂死，可能导致数百万美元级别的计算任务中断并造成巨大损失。Anthropic 借助 Sentry 提早捕获坏硬件节点并迅速剔除，避免了集群崩溃，将故障排查时间从“天”压缩到了“小时”级。此外，Sentry 还推出了名为 Seir 的 AI 调试 Agent（利用错误日志、调用链路和源码自动分析根因并直接生成 PR），以及针对 Agent 和 MCP（模型上下文协议）服务器的全新监控方案，让模型工具调用、模型间性能对比及下游服务通信彻底告别“黑盒”。Sentry 的代表客户涵盖 Anthropic、Cursor、Vercel 和 Linear 等硅谷明星企业。 ### [59:51](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3591s) DeepMind 收购序幕：AI 领域的“YouTube 时刻” Google 在 AI 发展史上的另一个决定性拼图并非自研产生，而是一起外部并购——其战略地位堪比 Google 当年买下 YouTube。这正是收购 DeepMind 的交易。在今天看来，DeepMind 的估值可能高达数千亿美元，但在 2014 年 1 月交易敲定时，普通大众与科技媒体的反应普遍是充满困惑的。 当时在 TechCrunch 等科技媒体的头条上，人们看到的是一条略显诡异的新闻：“Google 斥巨资收购了一家位于伦敦、几乎无人知晓的神秘初创公司”。那时的通用人工智能（AGI）对于硅谷主流圈子来说完全是边缘概念。如果翻开当时的 Wayback Machine 查看 DeepMind 简陋的官方主页，会发现他们甚至没有任何一款商业软件，网页上仅写着一行令人摸不着头脑的宏大愿景：“一家前沿人工智能公司，致力于为模拟、电子商务和游戏构建通用学习算法”。在 2014 年的大环境下，这种定位在硅谷投资人与工程师看来简直是天方夜谭、不可理喻。 - **战略对标**：主持人将 Google 收购 DeepMind 评价为 AI 领域的“YouTube 收购案”或 Facebook 收购 Instagram，是科技史上最伟大的并购交易之一。 - **时代背景（2014年1月）**：AI 仍处于极客与学术圈的边缘地带，未引起主流科技巨头与风投的全面狂热。 - **DeepMind 当时画像**：总部设在英国伦敦，无商业产品，官网愿景抽象，专注研发基于模拟环境与游戏的“通用学习算法”（General-purpose learning algorithms）。 ### [01:01:29](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3689s) 现代 AI 版图的蝴蝶效应：DeepMind 的诞生 在回顾 DeepMind 早期的商业描述时，会发现外界曾用“模拟系统、电子商务与游戏”这种听起来杂乱无章的词汇来定义它。然而，这恰恰勾勒出了它早期的探索轮廓。主持人指出，Google 对 DeepMind 的收购堪称现代 AI 历史上最关键的“蝴蝶煽动翅膀”的时刻——正是这次收购引发的一连串连锁反应，直接催生了后来的 OpenAI、ChatGPT、Anthropic、Gemini，甚至由马斯克（Elon Musk）创立的 xAI，乃至安德烈·卡帕斯（Andrej Karpathy）等人推动的特斯拉自动驾驶路线。 DeepMind 于 2010 年创立，其创始团队极其独特且充满戏剧性：CEO 德米斯·哈萨比斯（Demis Hassabis）拥有神经科学博士学位，此前曾创办过电子游戏公司；联合创始人肖恩·莱格（Shane Legg）是他在伦敦大学学院（UCL）结识的博士后研究员；第三位联合创始人穆斯塔法·苏莱曼（Mustafa Suleyman）则是 Demis 从小一起长大的挚友。当时谁也无法预料，这个起步极其非主流的三人组合，在日后不仅改写了全球科技工业的格局，更诞生了英国爵士勋衔获得者与诺贝尔奖得主。 ```mermaid flowchart TD accTitle: DeepMind 与现代 AI 格局的诞生 accDescr: 展示从 DeepMind 的创立、被 Google 收购，到分化出当今各大主流 AI 组织的演进路径。 subgraph DeepMindFounders["DeepMind 创始核心 (2010)"] D["Demis Hassabis\n(国际象棋神童/游戏开发者/神经学博士)"] S["Shane Legg\n(AGI 概念推广者)"] M["Mustafa Suleyman\n(连续创业者)"] end subgraph Catalysts["关键资本与并购"] PT["Peter Thiel / Founders Fund\n(200 万美元种子轮)"] G["Google 收购 DeepMind (2014)"] end subgraph RippleEffect["现代 AI 连锁反应"] OA["OpenAI\n(ChatGPT)"] ANT["Anthropic\n(Claude)"] GEM["Google DeepMind\n(Gemini)"] X["xAI / 特斯拉 FSD"] end DeepMindFounders --> PT --> G G --> OA OA --> ANT G --> GEM G --> X ``` ### [01:02:50](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3770s) 传奇三人组：从神童程序员到 AGI 的信徒 Demis Hassabis 少年时便是国际象棋神童，17 岁（1994年）就获得了剑桥大学的录取通知书。但由于年龄实在太小，校方建议他休学一年（Gap Year）再入学。在那一年里，Demis 去了著名游戏工作室牛蛙制作（Bullfrog Productions），参与打造了名震一时的《主题公园》（Theme Park）。这款堪称《模拟城市》游乐园版的经营模拟游戏在 90 年代中期狂销 1500 万套，并直接启蒙了日后的《过山车大亨》（RollerCoaster Tycoon）系列。在剑桥修完计算机科学后，Demis 又创办了游戏公司 Elixir，在该公司折戟后，他毅然决定重回学术界攻读神经科学博士，最终来到 UCL。 在 UCL，Demis 结识了 Shane Legg。当时的 Shane 自嘲为 AI 学术圈内的“边缘极端分子”（lunatic fringe）：在 2008 至 2010 年前后，几乎没有任何主流学者认为通用人工智能（AGI）值得严肃探讨，而 Shane 却坚信 AI 的算力与模型能力每年都会成倍增长，直到彻底超越人类智慧，他也是最早在业内普及“人工通用智能”（AGI）这一术语的人之一。随后，Demis 拉来了年仅 19 岁便从牛津大学退学创业的发小 Mustafa Suleyman。 三人将新公司命名为 DeepMind，以此向杰弗里·辛顿（Geoffrey Hinton）在多伦多大学开创的深度学习（Deep Learning）致敬。他们的雄心极其激进——当时的学术界（如 AlexNet、吴恩达的“猫脸识别”论文）还在探索如何利用卷积神经网络更好地给图像分类打标签，但 DeepMind 已经直接将目标定为“创造真正的心智”，并确立了极其霸气的公司标语：“解决智能，然后用它解决其他一切”（Solve intelligence, and then use that to solve everything else）。 - **神童履历**：Demis Hassabis 在 14 岁以下组别的国际象棋排名中曾位列世界第二。 - **商业战绩**：17 岁参与制作的经典游戏《Theme Park》在 90 年代中期卖出超过 1500 万份。 - **时代反差**：2010 年前后的主流 AI 依然局限在“狭义分类任务”（如计算器或模式匹配），主流学者将追求人类级通用智能视为天方夜谭。 ### [01:06:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3992s) 奇点峰会与国际象棋局：拿下 Peter Thiel 的种子轮 宏大的雄心随即迎面撞上了最现实的阻碍：资金。深度学习神经网络想要实现突破，必须依赖极其庞大的计算资源。在 2010 年的伦敦，根本没有风险投资机构愿意给一家连商业化计划书都没有、张口闭口要“解决智能”的公司投钱。为了融到几百万美元的启动资金，他们必须找到一个同样具有极端逆向思维的“疯子”投资人。 2010 年 6 月，Demis 和 Shane 获邀参加在旧金山举办的“奇点峰会”（Singularity Summit）。这场大会由未来学家雷·库兹韦尔（Ray Kurzweil）、埃利亚泽·尤德考斯基（Eliezer Yudkowsky）以及彼得·蒂尔（Peter Thiel）等人共同发起。正如帕米·奥尔森（Parmy Olson）在《霸权》（*Supremacy*）一书中所记述，Shane 曾坦言：“我们需要一个足够疯狂、不在乎几百万美元且热衷宏大野心的人，同时这个人必须极具逆向思维，因为任何正统大学教授都会警告他千万别投我们。” 彼得·蒂尔无疑是完美人选。 为了引起 Thiel 的注意，Demis 想方设法争取到了峰会上的演讲席位，打算在台上直接隔空做一场隐秘的 Pitch。然而当他站上讲台环顾全场时，却发现 Thiel 根本没有出席。峰回路转的是，他们得知当晚 Thiel 在旧金山的豪宅举办了私人派对。Demis 成功混入派对，并且极为巧妙地没有一上来就推销创业项目，而是投其所好，和 Thiel 聊起了国际象棋。得知眼前的年轻人曾是世界顶尖的少年棋手后，Thiel 兴致大发并主动询问他的事业。Demis 顺理成章地介绍了 DeepMind 的愿景，Thiel 随即邀请他们第二天前往 Founders Fund 办公室。 这次会面非常顺利，Founders Fund 最终领投了 DeepMind 约 200 万美元的种子轮融资。在今天 AI 初创公司动辄数亿美元融资的背景下，不到 200 万美元就能领投 DeepMind 的种子轮显得不可思议。而正是通过 Peter Thiel 的引荐，Demis 认识了埃隆·马斯克，揭开了下一幕科技巨头博弈的序幕。 ### [01:11:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4292s) 埃隆·马斯克的“火星备份”与德米斯的 AI 警醒 德米斯·哈萨比斯（Demis Hassabis）通过 PayPal 黑手党成员的牵线引荐，前去拜访埃隆·马斯克（Elon Musk）。当时大家打招呼的方式非常轻松低调：“嘿，埃隆，你应该见见这个人，他非常聪明，一直在深入思考人工智能。”马斯克爽快地回应：“太棒了，来 SpaceX 参观吧，我带你看看工厂。”于是哈萨比斯前去共进午餐并参观 SpaceX 的火箭制造车间。尽管哈萨比斯觉得火箭工厂酷炫无比，但他始终在寻找机会把话题拉回到人工智能上。 《卫报》（The Guardian）的一篇报道生动还原了这段关键对话：马斯克告诉哈萨比斯，自己的头等大事是移民火星，建立一个人类文明的“备份行星”，以防地球遭遇毁灭性灾难。当时马斯克显然还没怎么深入思考过 AI 的潜在危险。哈萨比斯随即指出了这个计划的致命漏洞：“如果地球上出问题的那件事恰恰就是 AI 呢？那你跑到火星去也无济于事。因为如果我们能抵达火星，AI 顺着我们的通信网络系统或者飞行器轻而易举就能追过去。”马斯克当场愣住了，坐在那里一言不发地沉思了整整一分钟，心里琢磨着：“嗯，这很可能是真的。” 不久之后，马斯克就成为了 DeepMind 的投资人。哈萨比斯不仅成功唤醒了马斯克对 AI 安全的忧虑，也让他意识到即使逃到火星也不可能躲过失控的超级智能。自此，“为了全人类的福祉研发安全可控的 AI”这颗种子正式在马斯克脑海中生根发芽，而这也正是 DeepMind 一直以来的崇高使命——像纯粹的科学家一样通过同行评议的方式推进研究，造福人类。 - 对话发生地：加州霍桑的 SpaceX 总部火箭工厂 - 核心警示：若 AI 具有威胁，逃离到火星无法提供庇护，AI 可轻易通过通信系统蔓延至火星 - 结果：马斯克出资入股 DeepMind，并在脑海中彻底确立了 AI 安全性的核心议题 ### [01:13:05](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4385s) 认知的转变：从 AI 灭世论到特斯拉的自动驾驶野心 在德米斯·哈萨比斯与马斯克会面以及马斯克投资 DeepMind 之后的一年里，马斯克的认知开关被彻底触发（The bit flipped）。如同马斯克面对任何新技术一样，一旦他对某件事产生浓厚兴趣，他的整套世界观都会迅速颠覆，彻底抛弃旧思维，全神贯注于如何最大程度拥抱这一新视界。 马斯克不仅关注 AI 的长远生存风险，更在短期内迅速对其近期落地潜力产生巨大热情，尤其是将其应用于特斯拉（Tesla）。当时，自动驾驶汽车的研究已经在业界展开，马斯克敏锐地意识到：“用 AI 来开车，对特斯拉来说简直是绝妙的主意。”于是，马斯克开始疯狂为特斯拉招募顶级 AI、机器学习和机器视觉研究员。 不久之后，AlexNet 橫空出世，在图像识别分类、YouTube 视频猫脸识别以及算法推荐信息流上展现出惊人的效果。马斯克和团队立刻联想到：这与处理行驶车辆前方的实时视频流、理解周围路况并实时控制汽车有什么本质区别吗？如果实时处理多帧图像并比对画面差异，就能直接控制车辆转向和加减速。至此，马斯克因哈萨比斯和 DeepMind 被点燃的 AI 热情，全面与特斯拉的自动驾驶战略挂钩，并开始急剧升温。 ### [01:15:02](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4502s) 扎克伯格入局：8 亿美元收购报价与独立性拉锯 此时远在伦敦的 DeepMind 正在全力扩充研究团队、训练模型。虽然他们偶尔也会向投资人抛出一些模糊的产品构想——比如在电商购物或视频游戏中做点落地探索（正如被收购时官网上的简介所写）——但哈萨比斯心无旁骛，核心目标永远是构建顶尖大模型并攻克通用人工智能（AGI）。 2013 年末，马克·扎克伯格（Mark Zuckerberg）的一通电话打破了平静。扎克伯格提出要整体收购 DeepMind。随着 AlexNet 引爆浪潮以及 AI 彻底重塑 YouTube 推荐流，扎克伯格彻底看清了 AI 对 Facebook 和 Instagram 社交 feed 推荐算法的巨大价值。为此，扎克伯格已高薪聘请了杰夫·辛顿（Geoffrey Hinton）的前博士后、卷积神经网络（CNN）的核心奠基人杨立昆（Yann LeCun），并联手成立了 Facebook 人工智能研究院（FAIR），将其打造为抗衡 Google Brain 的核心阵地。值得玩味的是，Facebook 的早期天使投资人兼董事会成员彼得·蒂尔（Peter Thiel），恰好也是 DeepMind 的领投人，极大概率是蒂尔将 DeepMind 的消息透露给了扎克伯格。 根据帕米·奥尔森（Parmy Olson）在《霸权》（Supremacy）一书中的披露，扎克伯格向这家既无实际产品、距离 AGI 尚有遥远距离的初创公司开出了高达 8 亿美元的收购天价。凯德·梅茨（Cade Metz）在其著作中也印证了这一点：若选择 Facebook，DeepMind 创始人拿到手的钱几乎是 Google 报价的两倍。 然而这笔收购触及了 DeepMind 创立的根本红线：德米斯曾向团队郑重承诺，DeepMind 必须保持完全独立、坚持向科学界公开科研成果，绝不能沦为受资本家随意摆布的产品工具人。在细节谈判中，扎克伯格对许多条件表现得极度宽容，例如允许杨立昆留在纽约、继续在纽约大学担任教授并运营大学实验室；但在核心控制权上，扎克伯格寸步不让。哈萨比斯坚决要求成立独立监督委员会（Independent Oversight Board），赋予其在公司违背初衷时直接干预的权力。对此，扎克伯格的回答很干脆：“不行，你们进来了就是 Facebook 的一部分。” - 潜在出价：据传高达 8 亿美元（几乎是 Google 最终出价的两倍） - 关键阻碍：扎克伯格拒绝给予独立运营地位与独立伦理监督委员会干预权 - FAIR 背景：扎克伯格拉拢杨立昆（Yann LeCun）成立 FAIR，力求对抗 Google Brain ### [01:18:18](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4698s) 三方争夺：马斯克的特斯拉股票与私人飞机上的关键一瞥 ```mermaid flowchart TD accTitle: DeepMind 收购竞争与拉里·佩奇发现路径 accDescr: 展示扎克伯格、马斯克与拉里·佩奇争夺收购 DeepMind 的过程以及佩奇通过私人飞机偶然发现 DeepMind 的插曲 subgraph Bidders["三大潜在买家"] FB["Facebook / 扎克伯格<br/>开价高达 8 亿美元<br/>拒绝给予独立地位"] TSLA["特斯拉 / 马斯克<br/>愿以特斯拉股票全资收购<br/>要求全力投入自动驾驶"] GOOG["Google / 拉里·佩奇<br/>已拥有 Google Brain<br/>尊重独立科研使命与 AGI 远景"] end subgraph Discovery["佩奇发现 DeepMind 的契机"] Plane["私人飞机同行<br/>马斯克、卢克·诺塞克与拉里·佩奇"] Video["展示 DeepMind 战胜 Atari 打砖块游戏视频"] Plane --> Video Video -->|"佩奇大受震撼并追问"| GOOG end DeepMind["DeepMind (德米斯·哈萨比斯)<br/>坚守 AGI 使命与独立性"] DeepMind -.->|"拒绝吞并"| FB DeepMind -.->|"拒绝沦为单一工具"| TSLA DeepMind ==>|"一拍即合达成交易"| GOOG ``` Facebook 的收购谈判传出风声后，DeepMind 的投资人群体瞬间震动。马斯克得知后，当机立断致电哈萨比斯：“我直接用特斯拉股票把你们全买下来！”彼时正值 2013 年底至 2014 年初，特斯拉市值仅约 200 亿美元（若按当时的特斯拉股票持有至今，涨幅超过 70 倍）。面对这一极具诱惑力的提议，哈萨比斯、谢恩·莱格（Shane Legg）和穆斯塔法·苏莱曼（Mustafa Suleyman）再次陷入了与面对扎克伯格时相同的困境：马斯克买下 DeepMind 是希望他们全职为特斯拉攻克自动驾驶，而 DeepMind 绝不可能放弃 AGI 愿景，沦为一家汽车公司的专属算法车间。 就在僵持之际，哈萨比斯接到了第三个电话——来自 Google 联合创始人拉里·佩奇（Larry Page）。 佩奇得知 DeepMind 的内幕极为传奇：当时马斯克与同为 PayPal 黑手党成员兼 DeepMind 天使投资人的卢克·诺塞克（Luke Nosek）正同坐一架私人飞机，两人在飞机上阅读哈萨比斯发来的邮件进展，里面附带了一段震撼的演示视频：DeepMind 的 AI 在没有任何人类示范训练的情况下，自学玩雅达利经典的“打砖块”（Breakout）游戏。AI 甚至自己摸索出了一种极致策略——把球顺着砖块边缘直接打入顶层通道，让小球在砖块与顶部之间自动高频反弹，无需频繁在底座接球就能迅速清屏通关。 当时马斯克和佩奇还是亲密无间的好友，佩奇也在这架私人飞机上。看到两人聚精会神地盯着屏幕，佩奇凑过来问：“等等，你们看的是什么？这是哪家公司？”佩奇就此锁定了 DeepMind。事后看来，马斯克对这段经历定然懊悔不已。 佩奇与哈萨比斯迅速建立了极其深厚的灵魂共鸣，这也是收购交易最终花落 Google 的决定性因素。哈萨比斯在筹备播客时曾向主理人坦言，佩奇是真正懂他的人，完全赞同 DeepMind 的宏大愿景。不仅如此，Google 当时已经拥有了专注于具体产品与工程落地的 Google Brain，佩奇根本不需要哈萨比斯团队放弃理想去给成熟产品写业务代码，这让 DeepMind 得以保留纯粹探索 AGI 的独立净土。 - 当时特斯拉市值：约 200 亿美元（至 2024/2025 年涨幅达 70 倍） - 打砖块（Breakout）突破：AI 纯靠强化学习自学出在砖块后方反弹挖洞的高效取胜技巧 - 促成交易的战略互补：Google 内部已有 Google Brain 负责产品化落地，能为 DeepMind 提供不被商业产品干扰的独立研究空间 ### [01:21:22](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4882s) 完美的归宿：Google Brain、基础设施与伦敦的自由 Google Brain 当时已经在 Google 内部深入各个业务线。当 Larry Page 对 Demis Hassabis 许诺说“你留在伦敦就行，继续研究通用智能，做你想做的事，我不需要你跑来 Google 做具体产品”时，Demis 是完全信服的。因为 Google Brain 的立足之本就是主动对接各业务部门，琢磨着怎么把神经网络塞进现有产品中去提升体验；既然这项工作已经有人在大力推进，而且效果拔群——神经网络正被迅速整合进搜索、广告、Gmail 等核心体系——DeepMind 就可以免受产品 KPI 的束缚，继续向终极的智能目标探索。对于 DeepMind 而言，Google 简直就是一个理想的“第二故乡”。 除了研究自由度，Google 能吸引 DeepMind 的另一张王牌就是算力基础设施。DeepMind 想要多少算力，Google 的数据中心就能实时供应多少——尽管当时主要还依赖海量的 CPU 集群。组织分工的互补、无上限的算力供给加上 Larry 的放权承诺，让两者的结合显得顺理成章。 ```mermaid flowchart TD accTitle: DeepMind 与 Google 协同架构 accDescr: 展示 DeepMind、Google Brain 以及底层算力设施在 Google 体系下的分工。 subgraph Infrastructure["底座层"] Compute["Google 算力基础设施<br/>(海量 CPU 集群支持)"] end subgraph ResearchTrack["前沿研究轨 (伦敦)"] DM["DeepMind<br/>(专注通用人工智能与基础科学)"] end subgraph ProductTrack["产品落地轨 (山景城)"] GB["Google Brain<br/>(算法落地与业务赋能)"] Products["核心产品群<br/>(Search / Ads / Gmail)"] end Compute --> DM Compute --> GB GB --> Products ``` --- ### [01:22:14](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4934s) 定制跳伞吊带与私人飞机：5.5 亿美元的 DeepMind 收购案 在收购 DNNresearch 之后，时任 Google 工程高级副总裁的 Alan Eustace 下定决心要网罗全球所有顶尖的深度学习人才。此前，Larry Page 曾在南太平洋的一座秘密私人岛屿上召开高层战略会议，坚信深度学习将彻底颠覆整个科技产业，并给团队放话：“我们要玩就玩大的（Let's really go big）。”这相当于给了 Alan 一张没有上限的空白支票。2013 年 12 月临近假期，Alan 决定飞往伦敦与 DeepMind 团队展开面对面尽调。 这次尽调旅程发生了一段不可思议的轶事。当时在 Google 工作的 Geoffrey Hinton 饱受严重背疾困扰，根本无法正常坐立，只能选择站立或者平躺，长途跨洋商业飞行对他来说几乎是不可能的任务。但 Hinton 是判断深度学习技术的最权威大脑，评估 DeepMind 绝少不了他。为此，Alan Eustace 专门包下一架私人飞机，并亲手设计了一套特制安全吊带和束缚装置，好让 Hinton 在飞机起飞和降落平躺在地板上时不会滑来滑去。Alan Eustace 本人就是极限运动奇人，曾身穿特制宇航服乘平流层气球完成过人类历史上最高高度的无动力太空跳伞，打破了红牛此前的世界纪录。对他而言，给地板上的床铺装几个固定皮带简直是小菜一碟：“我穿潜水服从大气层跳下来都没事，这肯定没问题。” 团队飞抵伦敦完成尽调后，这笔收购迅速敲定，交易金额为 5.5 亿美元。Demis 与 Larry 在愿景上找到了真正的精神共鸣。为了保障 DeepMind 的初衷不被商业利益扭曲，双方还特意成立了一个独立的伦理与监督委员会；而同样来自“PayPal 黑帮”的 Reid Hoffman 也在收购后受邀加入了该委员会。主持人在节目中感叹，如果把今天的 DeepMind 独立出来拆分估值，它很可能是一家价值五千亿美元的庞然大物。 - **交易金额**：约 5.5 亿美元。 - **高管背书**：Larry Page 在南太平洋秘密岛屿会议上授权“全面加码”（Go big）。 - **特制设备**：Alan Eustace 为无法坐立的 Geoffrey Hinton 定制了机舱地板平躺固定吊带，顺利完成跨大洋尽调。 - **伦理治理**：设立独立道德监督委员会，包括 Reid Hoffman 在内的多位硅谷重量级人物加入。 --- ### [01:24:50](https://www.youtube.com/watch?v=lCEB7xHer5U&t=5090s) 数据中心节能 40% 与 AlphaGo 围棋奇迹 收购完成后，DeepMind 很快展示出其实际价值。他们派出了部分研究员作为联络大使前往 Google 业务一线，寻找神经网络的应用场景，最先大放异彩的便是数据中心散热优化。2016 年 7 月，Google 正式宣布，利用 DeepMind 的神经网络模型，其数据中心冷却系统的耗电量惊人地降低了 40%。对于运营着全球最庞大计算基础设施之一的 Google 来说，仅这一项节省的能源账单，几乎在很短时间内就覆盖了当初收购 DeepMind 的 5.5 亿美元成本。 但真正让 DeepMind 和人工智能震撼全世界的，是 2016 年的 AlphaGo 人机大战。在对阵传奇世界冠军李世乭之前，整个围棋界普遍坚信人类绝不会输，围棋的玄妙和直觉被视为机器不可逾越的鸿沟。然而在五局三胜的比赛中，AlphaGo 势如破竹连下三城，并在比赛中下出了人类千百年来从未见过的创造性妙手——最著名的莫过于第二局的“第 37 手”。那一手棋落下时，解说和棋手们一度以为是程序出现了低级 Bug，直到一百手之后，人们才愕然发现那是一记算力深远的绝妙构思。如今，全世界的顶尖职业棋手反过来通过学习 AlphaGo 的下法来探索新的围棋理论。 围棋之所以成为人工智能的终极试金石，原因在于其惊人的状态复杂度。国际象棋开局时每个回合大约有 20 种走法，中局约 30 到 40 种；而围棋棋盘在每个回合大约有 200 种可能的落子选择。从组合数学角度看，围棋可能出现的棋盘局面总数比全宇宙的基本粒子总数还要多。正如 Demis Hassabis 所言，即使集结全世界所有的电脑连续计算一百万年，也无法用传统的暴力穷举法算尽所有局面。这逼迫研究人员必须转向模拟人类直觉与归纳能力的深度神经网络，也让围棋成为了观察 AI 展现全新“创造力”的最佳舞台。在征服围棋之后，凭借 Demis 的游戏设计与电竞底色，DeepMind 很快又将目光投向了即时战略游戏《星际争霸 II》（StarCraft II）——而这一战术竞技场，不久后也将迎来来自 OpenAI 的正面对决。 - **数据中心冷却**：采用 DeepMind 算法后，机房冷却能耗直降 40%，实现商业效益自造血。 - **李世乭人机大战**：AlphaGo 连胜前三局（总比分 4:1 击败李世乭），第二局“第 37 手”（Move 37）展现出颠覆人类定式的算法创造力。 - **围棋复杂度**：合法局面数超过宇宙原子总数（约 $10^{170}$ 数量级），彻底终结了暴力穷举算法的可能。 - **下一代挑战**：在 AlphaGo 封神后，DeepMind 转向更具动态对抗与不完全信息特征的电竞项目《星际争霸 II》（AlphaStar）。 --- ### [01:28:03](https://www.youtube.com/watch?v=lCEB7xHer5U&t=5283s) 赞助商时刻：WorkOS 与 AI 时代的企业级安全门槛 在聊到 DeepMind 的最大竞争对手 OpenAI 的诞生前，节目插入了赞助商 WorkOS 的介绍。随着现代企业级软件生态的发展，单点登录（SSO）、SCIM 用户同步、细粒度权限管理以及审计日志（Audit Logs）已成为大客户采购时的硬性指标。WorkOS 将这些极其繁琐、容易阻碍大单敲定的企业级基础设施打包成简单开箱即用的 API。 在 AI 爆发的当下，WorkOS 成为了 OpenAI、Anthropic、Cursor、Perplexity、Sierra、Replit、Vercel 等绝大多数头部 AI 初创公司的标配。之所以形成这种局面，主要有两个核心驱动力： 1. **爆发式增长提速**：AI 初创企业的商业化和收入规模扩张速度远快于老一代 SaaS，迫使它们必须在极早期就迅速具备企业级准入能力； 2. **严苛的 IT 审查**：过去的轻量 SaaS 产品往往可以通过单个员工“刷信用卡”在部门内部偷偷使用，但现代 AI 工具深度穿透企业内部的核心系统与机密数据，各大公司的 IT 和合规部门对数据隐私与权限控制进行了前所未有的严格审查，促使 AI 产品必须更快完成合规准备。 ### [01:30:05](https://www.youtube.com/watch?v=lCEB7xHer5U&t=5405s) DeepMind 收购案的连锁反应与愤怒的马斯克 Google 收购 DeepMind 不仅是一笔轰动一时的交易，更在科技界引发了深远的二阶效应。这桩收购让极少数顶尖大佬极度震怒——如果算上习惯喜怒不形于色的 Mark Zuckerberg，那就是两个人；但反应最激烈、彻底暴怒（goes ballistic）的当属 Elon Musk。此前 Elon 与 Larry Page 的私交极其深厚，甚至经常在硅谷借宿 Larry 家，但彼时两人的关系已经开始出现裂痕：Elon 试图为 Tesla 的 AI 团队招揽顶尖人才，却发现 Google 的工程副总裁 Alan Eustace 正在全球各地像巨型吸尘器一样，疯狂搜罗所有的 AI 研究学者并吸纳进 Google。作为 DeepMind 的早期投资人，Elon 原本打算将 DeepMind 收归己有、并入 Tesla 的 AI 团队，结果 Google 却从他眼皮子底下横刀夺爱。 这股怒火最终促成了硅谷历史上最富传奇色彩的一场饭局。2015 年夏天，在硅谷风险投资圣地沙丘路（Sand Hill Road）的瑰丽酒店（Rosewood Hotel），当时硅谷最具号召力的两位核心人物——Elon Musk 与时任 Y Combinator 总裁的 Sam Altman——共同张罗了一场晚宴。 ```mermaid flowchart TD accTitle: OpenAI成立的背景与催化过程 accDescr: 展示Google收购DeepMind引发Elon愤怒，促成沙丘路晚宴并催生OpenAI的因果流程。 subgraph Rivalry["巨头垄断与矛盾激化"] A["Google收购DeepMind<br/>从Elon眼前抢走标的"] --> B["Elon对Larry Page心生愤懑<br/>Google与Meta垄断顶尖人才"] end subgraph TheDinner["2015年夏：沙丘路瑰丽酒店晚宴"] B --> C["Elon Musk + Sam Altman<br/>邀约顶尖研究员聚餐"] C --> D{"策反问询：<br/>要什么条件才肯离开Google？"} D -->|"大多数人"| E["不可能离开<br/>（高薪、资源、学术自由、顶尖同侪）"] D -->|"Ilya Sutskever"| F["态度动摇：<br/>'虽有风险，但值得一试'"] end subgraph Formation["OpenAI横空出世"] F --> G["Google/Jeff Dean双倍薪资强行挽留（失败）"] G --> H["Ilya携约7位研究员出走Google"] H --> I["联手Greg Brockman等创立OpenAI<br/>（非营利、开源、造福全人类）"] end ``` ### [01:32:15](https://www.youtube.com/watch?v=lCEB7xHer5U&t=5535s) 瑰丽酒店的沙丘路晚宴：挖角 Google 的世纪难题 这场晚宴的核心目的，是游说那些已被 Google 和 Facebook（Meta）招致麾下的顶尖 AI 研究员跳槽。当时，Google 和 Facebook 依靠信息流推荐与分类器算法赚取了极其夸张的广告利润，因此有充裕的资本开出数百万美元的惊人高薪，把这些顶尖学者从象牙塔拉入企业内部的“资本主义商业实验室”。 Elon 和 Sam 环顾餐桌，向研究员们抛出了那个关键问题：“到底需要什么条件，你们才愿意离开 Google？”然而环视一圈，几乎所有人的回答都是斩钉截铁的“不可能”： - **无与伦比的待遇**：他们拿到的薪资远超学术界想象，甚至还能同时保留大学教职与学术头衔。 - **神级的基础设施与顶尖同侪**：这里汇聚了像 Jeff Dean 这样的大神，拥有全球最强大的算力和硬件资源，“以铁磨铁”（Iron sharpens iron），学者们可以与全球最聪明的头脑切磋并从事最前沿的研究。 就在全场看似铁板一块时，唯独有一位关键人物被彻底打动了。正如 Cade Metz 在《连线》（Wired）报道（后扩写为《天才制造者》*Genius Makers*）中所记述的，这位学者便是因 AlexNet 和 DNNresearch 声名大噪、随后成为 Google 核心成员的 Ilya Sutskever。面对这次邀请，Ilya 给出了极具个人风格的经典回应：“我觉得这里面确实有风险，但我也觉得这是一件非常值得一试的有趣事情（I felt like there were risks involved, but I also felt like it would be a very interesting thing to try）。” ### [01:34:49](https://www.youtube.com/watch?v=lCEB7xHer5U&t=5689s) OpenAI 诞生：非营利的乌托邦理想 Elon 和 Sam 向研究员们描绘的宏伟愿景极具诱惑力：建立一个全新的、非营利性质的开源 AI 实验室。在这里，科学家们无需为商业广告或产品落地服务，一切研究成果公开发表，摆脱商业巨头的束缚，确保未来的通用数字智能真正造福全人类，而非沦为 Google 或 Facebook 的牟利工具。 最初大家觉得这太美好了，美好到令人难以置信，甚至陷入了一种“如果没有别人带头，谁也不敢轻举妄动”的启动能垒僵局。但当 Ilya 坚定地表态“我加入”之后，局势瞬间打破。得知消息的 Google 迅速反击，由 Jeff Dean 亲自出面给出了待遇翻倍的超级挽留 Offer，但被 Ilya 毅然回绝。Ilya 的坚决跳槽引发了破窗效应，最终带领约 7 位 Google 顶尖研究员出走，与 Elon Musk、Sam Altman 以及来自 Stripe 的 Greg Brockman 等人正式创立了 OpenAI。 OpenAI 当时确立的崇高宗旨是：“以最可能造福全人类的方式推进数字智能，不受产生财务回报需求的束缚（advance digital intelligence in the way that is most likely to benefit humanity as a whole unconstrained by a need to generate financial return）。”这套乌托邦式的叙事在最初几年运转良好，但它隐含了一个致命前提——只要达成使命所需的成本不会膨胀到数百亿美元。 - **承诺与实际注资**：创立之初，官方高调宣称获得了 10 亿美元的资金承诺，主要认缴人包括 Elon Musk、Sam Altman、Reid Hoffman、Peter Thiel 以及常被外界忽略的 Jessica Livingston（Founders Fund 后期也追加了巨资）。但后续事实证明，初期真正到账的资金仅约 1.3 亿美元。 - **早期资金去向**：由于当时尚未进入动辄消耗百亿美元巨额算力的阶段，这 1.3 亿美元对于早期的学术探索完全够用，其中绝大部分资金都用来支付研究员 100 万至 200 万美元的年薪（虽然低于 Google 和 Meta 的顶配包，但依然极其丰厚）。 ### [01:37:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=5852s) OpenAI 早期探索：对标 DeepMind 与游戏 AI 竞技场 OpenAI 成立之初，全员进入全面招兵买马的状态。2016 年初，实验室迎来了又一员超级大将——Dario Amodei 离开 Google Brain 加盟 OpenAI。在随后几年里，Dario 与 Ilya 共同执掌了 OpenAI 绝大部分的研究工作，直至后来离开并创办 Anthropic。 但在当时，距离 ChatGPT、Claude 或 Gemini 的诞生还有数年时间。在最初的一两年里，OpenAI 的策略非常直接：紧盯 DeepMind 的一举一动，在全世界面前证明自己这个新兴实验室同样能做出媲美 DeepMind、甚至超越后者的惊人成就。这种全面对标直接导致其早期项目高度聚焦于游戏领域： - **Dota 2（刀塔 2）**：DeepMind 拿出了强化学习玩《星际争霸》（StarCraft）的研究，OpenAI 便瞄准了战局更复杂、更依赖即时决策与团队配合的 MOBA 游戏《Dota 2》。令人惊叹的是，模型在游戏中自主摸索出了人类从未见过的独特博弈策略，呈现出强大的涌现特征（Emergent properties）。 - **Universe 平台**：一个旨在让 AI 学会玩成千上万款游戏的环境训练平台，涵盖从复古的雅达利（Atari）游戏到开放世界大作《侠盗猎车手》（Grand Theft Auto）。 - **机器人与益智挑战**：研发模型驱动机械手自主解开魔方（Rubik's cube）等。 尽管这些项目五花八门，看似散落各处、缺乏一个能最终凝聚成杀手级产品的统一主线，但它们完全契合顶级学术机构与前沿实验室的研究范式。这正是 DeepMind 在做的事，而 Elon 投入时间、名誉与资源全力背书，也在很大程度上源自被 Google 截胡 DeepMind 后想要正面对决的执念。 ### [01:40:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6001s) 离开与坚守：Google 内部的算力觉醒 当 OpenAI 在埃隆·马斯克（Elon Musk）的强力背书下拉拢顶尖人才、四处探索类似于 DeepMind 的前沿课题时，Google 内部虽然遭遇了人才流失，但其在基础设施上的底蕴依然无与伦比。回溯到 2013 年，当 Alex Krizhevsky 随导师 Geoff Hinton 以及 Ilya Sutskever 一同加入 Google 时，他震惊地发现整个公司的机器学习模型居然全都在 CPU 上运行。此前曾有人提议引入擅长并行计算的 GPU，但 Google 庞大的基础设施团队出于维护统一机群、避免复杂化运维的考量，直接驳回了需求，坚守“我们这里是 CPU 工厂”的传统理念。 面对算力瓶颈，Alex 展现了学术界的野路子风格。根据《Genius Makers》的记载，入职头几天，他就自己跑去附近的电子产品商店自费买了一台配备 GPU 的机器，悄悄塞进走廊尽头的壁橱里插上网线，开始训练神经网络模型——就像当年在多伦多大学实验室一样，只是这次电费由 Google 买单。很快，单颗 GPU 无法满足激增的实验需求，Google Brain 团队内部对算力的渴望彻底爆发。Jeff Dean 和 Alan Eustace 清楚地意识到，虽然自研的 DistBelief 架构很强大，但它必须推倒重来，改写为适配 GPU 的新系统。 ### [01:41:39](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6099s) 1.3 亿美元订单与英伟达的命运转折 到了 2014 年春季，Jeff Dean 与当时统领 Google Brain、后来成为苹果 AI 负责人的 John Giannandrea（JG）正式坐在一起，规划将 GPU 正式部署进 Google 数据中心。伴随着 AlexNet 证明了深度学习的巨大潜力，他们决定一步到位，向英伟达（NVIDIA）下单采购整整 40,000 颗 GPU，总金额高达 1.3 亿美元。 这笔巨款立刻惊动了高层。尽管财务部门极力反对，认为开销过于夸张，但 Larry Page 亲自出面拍板批准。他的理由十分明确且坚决：“深度学习就是 Google 的未来。”对当时的英伟达而言，这是一份改写命运的天文数字级订单： - **营收占比惊人**：2014 年英伟达总营收仅为 40 亿美元左右，主力仍是消费级游戏显卡，这一笔 1.3 亿美元的企业级订单占比超过 3%。 - **市值对比**：当时英伟达的整体市值仅约 100 亿美元。 - **战略泄密效应**：Google 毫无保留的千金一掷，相当于提前向黄仁勋（Jensen Huang）揭晓了商业世界的底牌——深度学习绝不仅仅是 ImageNet 比赛里的学术玩具，而是在工业界具备巨大商业价值、值得企业不计成本砸入过亿美金的绝对未来。 这笔投资迅速带来了惊人的回报。Google 全面掀起应用浪潮：Google Photos 的图像识别、Gmail 的智能打字补全，以及核心现金牛 AdWords 的广告点击率预估系统相继接入深度学习。广告业务收入立竿见影的暴增，让这 1.3 亿美元的硬件投入在极短时间内收回成本，也让管理层彻底认准了“能买多少 GPU 就买多少”的扩张道路。 ```mermaid flowchart TD accTitle: GPU adoption to TPU transition accDescr: Diagram showing the flow from GPU adoption at Google to the computational bottleneck and the birth of TPUs. subgraph GPU_Adoption ["GPU 普及与商业变现"] A["采购 40,000 颗英伟达 GPU<br/>（1.3 亿美元）"] --> B["赋能核心业务<br/>（Photos / Gmail / AdWords）"] B --> C["AdWords 点击率提升<br/>带来海量新增商业变现"] end subgraph Scaling_Crisis ["算力危机与成本瓶颈"] C --> D["语音识别等新模型上线<br/>（Nexus 手机测试）"] D --> E["推算：若 10 亿 Android 用户<br/>日均使用 3 分钟语音识别"] E --> F["Google 数据中心机群<br/>规模必须彻底翻倍"] end subgraph TPU_Solution ["自研芯片破局"] F --> G["自研 ASIC：TPU 应运而生"] G --> H["8 位低精度量化计算<br/>极致能效比支持海量推理"] end ``` ### [01:44:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6241s) “我们需要再造一个 Google”与 TPU 的极速诞生 然而，业务规模的疯狂膨胀立刻撞上了现实的物理墙。深度学习的本质是在神经网络各层之间进行海量的矩阵乘法与张量计算。当 Google 在 Nexus 系列手机上首度上线基于神经网络的语音识别功能并大受好评后，Jeff Dean 做了个速算：如果将这项特性推向全球 10 亿台 Android 手机，且每位用户每天仅使用 3 分钟语音，Google 现有的全部数据中心规模必须彻底翻倍才能抗住负载。 面对这种不可持续的算力消耗，Jeff Dean 找到负责基础设施的 Urs Hölzle 直言：“我们要么再造一个完整的 Google 数据中心机群，要么就得自研一种专属芯片。”适逢工程师 Jonathan Ross 当时正利用 20% 自由时间折腾 FPGA（现场可编程门阵列），团队迅速整合资源，将其立项为正式的自研 ASIC（专用集成电路）项目——这便是后来的 TPU（Tensor Processing Unit，张量处理单元）。 TPU 的设计完全抛弃了图形渲染等 GPU 具备的冗余能力，只专注服务于神经网络核心的矩阵乘法： - **量化与低精度运算**：TPU 舍弃了传统 32 位甚至 64 位的高精度浮点，通过模型量化技术，直接把类似 `4586.8272` 这样的长浮点数压缩四舍五入为 `4586.8` 甚至纯整数 `4586`。在软件架构能够容忍这一精度损失的前提下，芯片可以在同等晶体管、面积和功耗限制下执行数量级更高的计算，极大加速推理。 - **15 个月的极速交付**：面对刻不容缓的机房危机，TPU 从设计、验证、流片制造到最终部署进生产环境，总共仅耗时惊人的 15 个月。 - **硬核的机架工程学设计**：为了避免改建数据中心的巨大成本，团队最初用高成本的 FPGA 作为过渡测试机群；在硬件外形上，他们甚至将 TPU 精妙地做成了与标准机械硬盘完全相同的物理规格。运维人员只需拔出一块硬盘槽位，就能顺手推进去一块 TPU，无需改动现有服务器机架的供电与物理布线。 这极具 Google 早期用软木板搭建服务器机架的工程极客风采。而令人意想不到的是，整个划时代的 TPU 硬件研发核心并非发生在山景城总部，而是位于威斯康星州麦迪逊（Madison, Wisconsin）的卫星办公室，依托于当地威斯康星大学顶尖的计算机体系结构教授与其优秀的毕业生团队完成了这次挽救 Google 算力危机的奇迹。 ### [01:49:02](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6542s) 绝密的 TPU 与媲美英伟达的算力帝国 Google 在研发定制 AI 芯片 TPU（Tensor Processing Unit）时，将其视作最高机密严格保密。当时 Google 并没有打算在 Google Cloud 上公开对外出售这些芯片，因此完全没有理由向外界透露风声。整整一年多的时间里，外界对该项目一无所知，直到 Google I/O 大会上正式对外宣布。 更关键的一个里程碑是，TPU 恰好赶在举世瞩目的 AlphaGo 人机围棋大战之前完成了部署。当时击败李世石的 AlphaGo 系统，仅运行在一台搭载了四块 TPU 的 Google Cloud 机器上。当这一构架在实战中完美跑通后，Google 内部士气大振，获得了前所未有的信心，开始开足马力全面铺开生产线。 尽管第一代 TPU（TPU v1）据各方反馈并不算完善，但如今 Google 已经迭代到了 v7 乃至 v8 版本，性能取得了翻天覆地的飞跃。伴随着硬件的演进，TPU 与英伟达的 GPU 在架构和功能上互相借鉴，变得越来越相似。据估计，Google 如今内部部署的 TPU 数量已达 200 万至 300 万片。作为对比，英伟达全年在全球出货的 GPU 总量大约在 400 万片左右。许多人在讨论 AI 芯片时，总认为这是英伟达一家独大的“单人赛跑”，却忽略了 Google 在内部早已建立起一个规模几乎可与英伟达匹敌的自研芯片帝国，不仅支撑内部所有业务，还赋能 Google Cloud 外部客户。 - AlphaGo 对战李世石的核心系统仅由单台服务器搭载 4 块第一代 TPU 驱动。 - 业内估计 Google 目前拥有的 TPU 总量在 200 万至 300 万颗之间，规模逼近英伟达全年的 GPU 出货量（约 400 万颗）。 --- ### [01:50:18](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6618s) TensorFlow 登场与 Google Brain 的黄金时代 这种强大的技术护城河，让伊隆·马斯克（Elon Musk）以及 OpenAI 的初创团队感到无比挫败与具有讽刺意味。2015 年 OpenAI 成立之初，核心动机之一就是通过挖走 Google 的顶尖人才来打破 Google 的垄断、拉平行业竞技场。然而现实却是，Google 不仅没有被削弱，反而进一步加速前进。 为了统一全公司的机器学习研发基础设施，Google Brain 打造了全新的软件框架 TensorFlow，彻底取代了早期的 DistBelief 系统。TensorFlow 的设计具有极高的通用性和可移植性，研究人员编写的模型不仅能在 TPU 上高效运行，无需重写任何代码即可直接无缝迁移到 GPU 甚至普通 CPU 上。在 OpenAI 成立后的数年里，虽然确实有一批顶尖研究员相继流失，但 Google Brain 依然全速运转，不仅在业务落地上面面俱到，持续为搜索和各项商业产品输送炮火，还在学术和工程前沿全面突破。 正是在这种全盛状态下，2017 年，Google Brain 团队的八位研究人员悄然发表了一篇论文。这八位作者当时对自己提出的新架构感到极度兴奋，深信其潜力巨大；而在 Google 管理层眼中，这篇论文更像是“我们语言模型工程演进中水到渠成的下一次迭代”，重要但并不被视为需要全力押注的“下一个 Google”。然而，正是这篇被 Google 视作常规研究成果的论文，为 OpenAI 送去了逆袭的火种，让后者抓住了机会去打造真正的“下一个 Google”——这就是日后重塑整个 AI 领域的 Transformer 论文。 ```mermaid flowchart TD accTitle: Google Translate 架构演进与突破 accDescr: 展示 Google 翻译从统计学方法到 RNN/LSTM，再到 Transformer 架构的技术迭代路径。 subgraph S1["传统时代 (2000s)"] A["Franz Och 统计机器翻译"] end subgraph S2["深度学习革命 (2012-2016)"] B["Jeff Dean 主导重写代码<br/>引入卷积/循环神经网络"] C["引入 LSTM 架构<br/>错误率大幅下降 60%"] end subgraph S3["Transformer 诞生 (2017)"] D["Jakob Uszkoreit 提出全局注意力机制"] E["Noam Shazeer 重写全新代码库"] F["超越 LSTM，成为现代大模型基石"] end A --> B B --> C C -->|"遇到计算瓶颈与无法并行化"| D D --> E E --> F ``` --- ### [01:51:25](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6685s) 从循环神经网络到 LSTM：机器翻译的演化与瓶颈 要理解 Transformer 的诞生，必须回顾 Google 语言模型的演变脉络。在 2000 年代中后期，Franz Och 主导的 Google Translate 依靠统计机器翻译取得了巨大突破。随着 2012 年 AlexNet 的横空出世以及 Geoff Hinton 的加入，以 Jeff Dean 为首的研究团队“空降”翻译部门，提出用神经网络彻底重构翻译系统。他们认为，神经网络的表现完全能超越过去十年积累的经典统计方法。最终，团队推倒了积攒十余年的全部旧代码库，换上基于循环神经网络（RNN）的新模型。2016 年《纽约时报杂志》曾为此刊发长篇专题报道，整个科技界都被神经网络的爆发力震撼得目瞪口呆。 但基于 RNN 的系统很快遇到了物理瓶颈：模型在阅读长文本时“遗忘”得太快。用现代语言模型的概念来解释，就是它的上下文窗口极其短小。为了在翻译后半截句子时依然能调取前面的语义，Google 于 2016 年在翻译系统中引入了长短期记忆网络（LSTM，Long Short-Term Memory）。 LSTM 为网络赋予了持久的记忆单元，使得翻译系统的错误率断崖式下降了 60%。当时学界和业界一片沸腾，很多人甚至认定 LSTM 就是将大规模语言模型带向主流终局的终极技术方案。然而其致命软肋随之暴露：LSTM 的计算开销极其沉重，且由于存在严格的步骤依赖，它无法实现大规模并行化处理。在 AlexNet 与 TPU 相继验证了“高度并行化计算才是 AI 未来”的背景下，LSTM 成为了横亘在算力扩展道路上的巨大路障。 --- ### [01:55:52](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6952s) Transformer 的诞生与 Noam Shazeer 的魔法重写 为了寻找一种既具备 LSTM 的长程记忆能力、又能完美适配大规模并行计算的新架构，Google Brain 内部的一支小组展开了探索。研究员雅各布·乌斯科赖特（Jakob Uszkoreit）率先萌生了一个颠覆性的想法：彻底放大语言处理中的“注意力”（Attention）范围。传统的模型只能紧盯着眼前的几个单词，而乌斯科赖特设想，能否让模型在处理每一个词时，同时对整个文本语料的所有内容施加注意力，纵观全局上下文后再来预测下一个词？ 这种机制恰恰完美复刻了人类专业翻译员的工作方式：翻译绝不是机械地逐字逐句对应，而是先完整读完全文、吃透文章通篇的语境意图，再带着全盘背景展开翻译。这种全局自注意力机制计算量虽然巨大，但它没有任何前后顺序的递归依赖，因此具有极佳的可并行化特性，能将硬件集群的算力榨取到极致。 团队将这个新架构命名为“Transformer”：一方面是因为它的核心功能就是接收一整块信息、理解并“转化”（transform）它；另一方面，团队成员私下里都是《变形金刚》（Transformers）的狂热爱好者。这一理念也神奇地呼应了 2000 年前后 Noam Shazeer 在 Google 茶水间里关于将全人类语言文本压缩进模型的最初设想。 早期的 Transformer 原型虽然能够运行，但在实际效果上并没有超越既有的 LSTM 系统。关键转折点发生在 Noam Shazeer 加入项目之后。Noam 敏锐地意识到该架构的前景，随后他在团队中上演了经典的“Jeff Dean 式神迹”——凭借顶级的工程能力将整套代码库从零彻底重写。 重写后的 Transformer 展现出了碾压式的威力，一举击溃了基于 LSTM 的翻译系统；更震撼的是，团队发现模型规模造得越大，它的效果就呈现出爆发式的提升，展现出无可估量的扩展性（Scaling Law）。在科技记者 Steven Levy 撰写的报道中，团队成员对 Noam 满怀敬畏，称他为“魔术师”和“巫师”：“Noam 接过想法钻研了一阵，回过头来说：‘好了，现在它能跑了，而且无懈可击。’” ### [01:59:02](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7142s) Transformer 的极简之美与“苦涩的教训” Noam Shazeer 与 Jeff Dean 是一对配合极为默契的技术搭档，如今两人正再度携手并肩打造新一代 Gemini。回顾当年，Google Brain 的联合创始人之一 Greg Corrado 在谈及 Transformer 架构时，特别强调了这一设计的精妙与优雅。他回忆道，许多人初见该架构时的第一反应几乎都是：“这不可能奏效，它实在是太简单了，甚至勉强能算作一种神经网络架构。”这与从 AlexNet 以及 Geoffrey Hinton 一脉相承的传统复杂深度神经网络相比，走了一条截然不同的道路。 这种化繁为简的哲学深刻改变了许多人观察技术世界的方式。在自然界中，进化的法则往往倾向于最高效、最节能的运转机制——最简洁优雅的生物解决方案往往因为资源利用率最高而得以存活。Greg Corrado 将这一自然法则投射到了计算机科学与研究实验室的模式识别中：当你面对一个极其棘手的问题，大家围在白板前激烈争论、构思了各种繁冗庞杂的设计，突然有人灵光一现提出一个极度简约的方案，而最终被证实正确的，往往正是这种大道至简的架构。 这种简洁性也拉开了现代 AI 演进的大幕：核心就在于“喂给它更多的数据”。虽然强化学习先驱 Rich Sutton 那篇著名的《苦涩的教训》（The Bitter Lesson）直到 2019 年才正式成文发表，但其核心思想在此刻已展露无遗——AI 研究人员总是自视过高，认为自己的本职工作是发明精妙复杂的人类先验算法；然而从语言处理、计算机视觉到下棋博弈，历史一再证明，只要找到一种具备高度可扩展性的架构，海量算力与海量数据最终必将碾压一切手工设计的精巧算法。Transformer 正是这种人们梦寐以求的、具有无限扩展潜力的架构。 - **极简设计冲击**：Transformer 摒弃了传统的循环与卷积结构，纯粹依靠注意力机制，简单到甚至被同行质疑“根本不像神经网络”。 - **《苦涩的教训》先行印证**：架构的可扩展性远胜于复杂的手工算法，只要架构能够无上限地吞吐数据与计算资源，性能就会持续飞跃。 --- ### [02:01:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7280s) 划时代的论文与 Google 史上最沉重的人才流失 当 Transformer 的雏形出现时，Noam Shazeer 和团队敏锐地意识到，这项技术的潜力绝不仅限于改进机器翻译。然而，Google 内部其他部门对这种颠覆性潜力的觉醒却显得相当迟缓。外界常常有一种误解，认为 Google 在发表 Transformer 论文后毫无作为，但事实并非如此。Google 迅速行动并在一年内推出了大语言模型 BERT，随后又推出了 MUM 等模型，深度改进了核心搜索引擎的查询理解能力（Query Comprehension），默默为核心商业机器注入动力。然而，Google 犯下的根本性失误在于：他们仅仅将 Transformer 视为原有业务各个环节的“改良补丁”，而没有将其视作一场底层计算平台级的巨大范式转移。 2017 年秋天，Google 批准团队将这项研究公开发表，论文定名为《Attention Is All You Need》（注意力就是你所需要的一切）——巧妙致敬了披头士乐队的名曲《All You Need Is Love》。从全人类的技术福祉来看，将这项研究开源共享无疑是最伟大的善举之一；但从商业竞争角度看，这或许是 Google 历史上代价最惨重的一项决策。截至 2025 年，该论文的学术引用次数已突破 17.3 万次，成为 21 世纪引用量排名第七的科学论文（排在它前面的论文均已发表了更漫长的时间）。 更为残酷的是，在这篇划时代论文发表后的短短几年内，全部八位署名作者悉数出走 Google，他们要么创办自己的明星创业公司，要么加入了竞争对手（包括 OpenAI）。作者之一的 Noam Shazeer 后来离职创立了 Character.ai；多年之后，Google 不得不通过一种涉及技术授权与高管回聘的“反向收购”曲折路径，豪掷数十亿美元才将他重新请回 Google。从 2017 年发表论文开始，Google 陷入了长达五年的“创新者的窘境”，未能充分把握自己亲手开启的时代机遇。 ```mermaid flowchart TD accTitle: Transformer 论文发表与作者外流 accDescr: 展示 Google 发表 Transformer 论文后在业务应用与顶尖人才流失上的不同走向 subgraph GoogleBrain["Google 内部应用"] A["Transformer 架构诞生"] --> B["发表论文《Attention Is All You Need》"] B --> C["研发 BERT、MUM 模型"] C --> D["融入搜索查询理解与核心业务改进"] end subgraph Exodus["人才流失与外部开花"] B --> E["全部 8 位论文作者相继离职"] E --> F["Noam Shazeer 创立 Character.ai"] E --> G["顶尖人才加盟 OpenAI 等初创企业"] F -.->|"数年后耗资数十亿美元收购/回聘"| H["重新回归领导 Gemini"] end ``` --- ### [02:04:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7441s) OpenAI 内部震荡：马斯克最后通牒与出走 就在 Google 开启 Transformer 时代的同时，大洋另一侧的 OpenAI 正经历着一场关乎生死存亡的内部大地震。2017 年 9 月 Google 发表 Transformer 论文前后，OpenAI 的核心出资人兼董事会成员埃隆·马斯克（Elon Musk）对实验室的混乱现状感到忍无可忍。在他看来，当时的 OpenAI 缺乏清晰的主线战略——一会儿研究打电子游戏，一会儿参加竞技比赛，甚至在很大程度上只是在步人后尘、笨拙地模仿 DeepMind 的路线。 与此同时，马斯克正承受着特斯拉与 SpaceX 的巨大业务压力。随着自动驾驶技术对特斯拉未来的核心地位愈发明确，马斯克极度渴求顶尖的 AI 研究人才与突破性成果来支撑特斯拉的现实需求。眼见 OpenAI 迟迟无法交出令人满意的突破，马斯克向 Sam Altman 及董事会摊牌，下达了最后通牒：要么将 OpenAI 的控制权完全交给自己，将其整体并入特斯拉名下；要么他将撤资并彻底退出。 在马斯克的商业逻辑中，将非营利性机构与上市公司整合并不存在概念障碍，正如他掌控的其他企业之间频繁进行跨公司资源协作一样。然而，Sam Altman 和董事会断然拒绝了这一通牒。当时马斯克名义上承诺的 10 亿美元注资，实际上是按需分期调用的资金，OpenAI 实际到账的仅有 1.3 亿美元左右。谈判破裂后，马斯克于 2018 年初愤然退出，同时切断了 OpenAI 最重要的资金生命线。 - **马斯克的最后通牒**：要求取得绝对控制权并将 OpenAI 并入特斯拉，以服务自动驾驶研发，否则撤资走人。 - **资金承诺的虚实**：原定 10 亿美元的赞助款实际只注入了约 1.3 亿美元，使得 OpenAI 瞬间面临断粮绝境。 --- ### [02:06:10](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7570s) 绝境中的豪赌：向营利转型与 GPT-1 的诞生 马斯克的决裂究竟是他个人的一次重大战略误判，还是倒逼 OpenAI 绝地求生的关键催化剂？历史往往充满戏剧性。如果 OpenAI 决定全面押注基于 Transformer 的大规模语言模型，随之而来的现实是：这项路线需要难以想象的庞大数据、天量算力、巨额电能以及无底洞般的资金支持。如果保留马斯克，或许能依托其资本推动这一重大转型；而马斯克的离去，则逼迫 Sam Altman 构思一个更大胆的棋局——摆脱非营利机构的束缚，转变为营利性实体，通过引入外部风险投资和商业利润来满足这一超级吞金兽的需求。 2018 年 6 月，OpenAI 正式发布了一篇具有里程碑意义的论文，提出了全新的技术路径：首先在互联网海量的通用文本上对 Transformer 进行大规模“无监督预训练”（Pre-training），然后再针对特定的下游任务进行针对性“微调”（Fine-tuning）。伴随论文一同问世的，是该技术路线的首个概念验证模型——GPT-1（Generative Pre-trained Transformer 1）。 当时，基于 Transformer 的探索并非 OpenAI 的独家秘密。Google 内部正紧锣密鼓地推进 BERT，西雅图的艾伦人工智能研究所（Allen Institute）等实验室也在进行类似探索。然而，由于彻底押上了整个机构的命运，OpenAI 从一开始就比任何竞争对手都更孤注一掷地贯彻这一路线。随着马斯克离场带来的资金真空，Sam Altman 迅速寻找新的避风港，他将目光投向了 OpenAI 的另一位董事会核心成员——刚刚将 LinkedIn 以高价卖给微软并进入微软董事会的里德·霍夫曼（Reid Hoffman），由此拉开了改变未来 AI 格局的微软联姻序幕。 ### [02:09:04](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7744s) 太阳谷峰会与微软的10亿美元联姻 领英联合创始人兼微软董事会成员里德·霍夫曼（Reid Hoffman）成为了两家巨头搭线的关键桥梁。他直接向山姆·奥特曼（Sam Altman）提议：“你为什么不去跟萨提亚·纳德拉（Satya Nadella）聊聊这事呢？” 这个改变科技行业格局的会面发生在一个极具象征意义的场合：2018年7月，爱达荷州太阳谷举办的 Allen & Company Sun Valley 年会（即著名的“富豪夏令营”）。在阳光明媚的度假村中，奥特曼与纳德拉坐到了一起，敲定了一项价值10亿美元的重磅交易。微软承诺向 OpenAI 注入10亿美元，包含大笔现金与 Azure 云计算额度（Azure Cloud Credits）。作为回报，微软不仅获得了 OpenAI 尖端技术的独家产品授权，还把 OpenAI 牢牢绑定在了自家云服务平台之上。 为了消化这笔来自科技巨头的商业投资，非营利组织 OpenAI 实施了复杂的结构重组：非营利主体（OpenAI Inc.）成立了一家附属于其下的“受限盈利公司”（OpenAI LP，即所谓 Captive For-Profit 架构）。微软的10亿美元资金注入这家营利性实体，而控制权理论上仍留在非营利董事会手中。新的营利架构董事会成员包括里德·霍夫曼、山姆·奥特曼、伊利亚·苏茨克维（Ilya Sutskever）、格雷格·布罗克曼（Greg Brockman）、Quora 创始人亚当·德安吉洛（Adam D'Angelo）以及塔莎·麦考利（Tasha McCauley）。主播们笑谈，这个“非营利转营利”的怪物结构，直到2025年的今天都还没被彻底理顺。 但对于谷歌而言，这件事意味着最可怕的宿敌归来。回顾谷歌的历史，从早期的搜索大战到后来的全线产品布局，微软始终是那个“终极宿敌”。谷歌好不容易在各个领域将微软按在地上打，如今微软借着 OpenAI 卷土重来。正如萨提亚几年后说出的那句名言：“我们希望全世界知道，我们让谷歌跳起舞来了（We want the world to know that we made Google dance）。”当时尚处于 ChatGPT 诞生前夕，奥特曼的核心目标非常明确：他需要天文数字级别的算力与资金，来支撑团队从 GPT-2 开始的大规模模型扩展（Scaling Law）实验。 ```mermaid flowchart TD accTitle: OpenAI 与微软的投资架构 accDescr: 展示2018年微软通过现金和Azure额度投资OpenAI LP的架构关系 subgraph Board["非营利治理架构 (OpenAI Inc.)"] NonProfit["OpenAI Inc. (非营利组织)"] BoardMembers["董事会成员: Sam Altman, Ilya Sutskever, Greg Brockman, Reid Hoffman, Adam D'Angelo, Tasha McCauley"] end subgraph Investment["商业合作与投资架构"] MSFT["微软 (Microsoft) 注资10亿美元: 现金 + Azure 云额度"] ForProfit["OpenAI LP (受限盈利实体)"] License["获得 OpenAI 独家商业产品授权"] end NonProfit -->|"控股与治理管理"| ForProfit MSFT -->|"10亿美元注资"| ForProfit ForProfit -->|"提供底层模型技术"| License ``` - **太阳谷秘密协议**：2018年7月，山姆·奥特曼与萨提亚·纳德拉在爱达荷州太阳谷峰会敲定合作框架。 - **10亿美元组合包**：微软首期投资由真金白银的现金与巨额 Azure 云算力额度组合而成。 - **混合型治理架构**：OpenAI 创立 OpenAI LP 营利实体，由非营利母公司 OpenAI Inc. 绝对控制，以解决机构既要吸纳外部巨资、又要名义上坚持“造福全人类”的根本矛盾。 --- ### [02:11:27](https://www.youtube.com/watch?v=lCEB7xHer5U&t=7887s) 为什么微软是 OpenAI 最完美的归宿 从 OpenAI 的视角来看，微软绝对是独一无二的最佳合伙人。这不仅是因为西雅图巨头账上趴着堆积如山的现金，更因为微软拥有遍布全球的高性能公有云平台——Microsoft Azure。 在2018年，OpenAI 还只是一家研发实验室规模的机构，绝无可能自己斥资数亿美元去大量采购英伟达（NVIDIA）GPU、自建超大型数据中心。研究员们需要海量的高性能计算资源。如果在谷歌内部，研究员们享有全球最顶级的内部基建；但一旦脱离谷歌，能承接这种算力胃口且非谷歌系的选择，全球基本上只剩下西雅图的两家公司：微软 Azure 和亚马逊 AWS。 巧合的是，主播们挖掘到了一个坊间秘闻彩蛋：亚马逊 AWS 据传在更早之前就以赠送算力积分等形式参与过 OpenAI 最初的非营利捐赠轮。但真正拿出全部身家战略资源全力押注的，是微软。萨提亚和奥特曼同台亮相，宣布两家公司结为天作之合，这也正式拉开了 OpenAI 狂飙猛进的“GPT 时代”序幕。 --- ### [02:13:21](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8001s) 赞助商插播：Shopify 的蜕变 在进入 GPT 详细故事之前，节目切入赞助商环节。两位主播回顾了他们长期以来的合作伙伴 Shopify。在 ACQ2 播客中，他们刚与 Shopify 首席执行官托比·吕特克（Tobi Lütke）进行了深度访谈，探讨了人工智能以及自六年前节目首次报道该公司以来的巨变。 从上市之初服务个体创业者与小微商家的电商平台，Shopify 如今已跃迁为承载各种规模跨国巨头的全球领先商业基础设施。无论是 Everlane、Vuori 这样的新锐品牌，还是美泰（Mattel）这样的百年传统巨头，年营收数十亿美元的企业都在 Shopify 上平稳运转。除了独立的独立站商城，商家还可以一键分发到 Instagram、YouTube、TikTok、Roblox、Roku 甚至 ChatGPT 和 Perplexity 等各个新兴流量入口。配合覆盖超过2亿常客的快捷支付工具 Shop Pay，Shopify 成为了商界“把专业的事交给专业团队做”的典范。 --- ### [02:15:38](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8138s) GPT-2 到 GPT-3：从极客玩具到逼近图灵测试 微软首笔10亿美元投资落袋后，OpenAI 于2019年发布了 GPT-2。西雅图本地的数据科学家们开始兴奋地讨论这项新技术，但对普通大众而言，GPT-2 还只是个极客玩具。它既没有友好的聊天界面，也没有面向小白用户的产品前端（Front Door），本质上只是一个开放 API。用户必须像工程师一样，输入几大段精心准备的文字让它做续写，并且它生成的文章虽然逻辑顺畅，但字里行间依然充满浓重的“机器人机械感”。 转折发生在2020年6月，OpenAI 推出了震撼学术界与产业界的 GPT-3。尽管此时依然没有简易的终端产品前端，但 GPT-3 的生成水准出现了质的飞跃。机器写出的长文与人类手笔几乎难以区分，“GPT 能否通过图灵测试”的讨论首次登上主流科技媒体。硅谷风投们收到的创业方案中，标榜“AI 驱动”的商业计划书（Pitch Decks）呈爆发式增长，英伟达的股价也由此启动了狂暴上涨的慢牛行情。 ```mermaid flowchart LR accTitle: OpenAI 模型演进与商业化历程 accDescr: 从2019年GPT-2发布到2021年GitHub Copilot与微软追加投资的时间线 G2["2019年 GPT-2 发布 技术演示/无前端/机械感明显"] G3["2020年6月 GPT-3 发布 质的飞跃/逼近图灵测试/风投关注"] Copilot["2021年夏 GitHub Copilot 首个落地商业产品/重构编码模式"] MSFT2["2021年底 微软追加20亿美元 估值飞跃/科技市场周期剧变"] G2 --> G3 --> Copilot --> MSFT2 ``` - **GPT-2 的局限性**：2019年推出，用户需要编写几段提示文本让其预测补齐，应用范围局限于小圈子开发者。 - **GPT-3 的飞跃**：2020年6月面世，参数量骤增，文本质量极大提高，让风投圈与硬件供应链首次感受到生成式 AI 爆发的前夜。 - **代码革命：GitHub Copilot**：2021年夏季，微软将 GPT-3 首次打造成落地产品 GitHub Copilot。这是全球第一款将底层 GPT 技术规模化商业落地的产品。最初只有少数程序员在试用中惊叹它提高效率，但很快全行业便迎来质变——如今已有高达75%的代码是由 AI 辅助生成的。 - **微软追加20亿**：看到 Copilot 的突破性成果后，微软在2021年底果断向 OpenAI 再追加20亿美元投资。而与此同时，全球科技股泡沫与加密货币市场即将迎来剧烈的宏观周期下挫。 ### [02:18:55](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8335s) 宏观逆风与老态龙钟的谷歌：ChatGPT 前夜的市值腰斩 2021 年底到 2022 年，宏观环境发生剧烈逆转，市场情绪由极度激进的“Risk-on”瞬间滑落至极度避险的“Risk-off”。除了乌克兰危机等地缘政治冲突外，美联储激进加息更是重创了整个科技板块，谷歌首当其冲。2021 年 11 月 19 日，谷歌的市值触及 2 万亿美元的高光峰值；仅仅一年左右的时间，其市值便直接蒸发近半，腰斩至仅剩约 1 万亿美元。如果有人在那时坚定唱反调并看好多头，本可以在 1 万亿美元的估值低谷抄底建仓。 ```mermaid flowchart LR accTitle: 谷歌2021至2022年市值巨震 accDescr: 展现从2021年底的高峰两万亿美元滑落至2022年底一万亿美元的过程及市场情绪变化 subgraph Peak["2021年11月峰值"] A["市值达到 2 万亿美元"] end subgraph Downturn["宏观逆风冲击 (2022)"] B["加息周期开启"] C["地缘政治危机"] D["市场 Risk-off 避险"] end subgraph Trough["2022年底谷底"] E["市值滑落至约 1 万亿美元 (跌幅近50%)"] F["被市场视为动作迟缓、缺乏创新的老态公司"] end Peak --> Downturn --> Trough ``` 彼时外界对谷歌的认知开始发生微妙而负面的转变：这家公司动作迟缓，对外部变化反应钝拙，浑身上下散发着一种令人窒息的官僚与老态，仿佛重蹈了 2000 年代微软的覆辙——已经很多年没有推出过一款真正具有突破性意义的杀手级产品了。当时市场弥漫的共识是，即使有下一轮 AI 浪潮，似乎也绝不会成为谷歌的专属红利；更何况许多投资者早对 VR、加密货币等轮番炒作的“前沿概念”审美疲劳，对 AI 是否是未来同样充满怀疑。正是在这种唱衰谷歌的灰暗氛围中，ChatGPT 毫无征兆地震撼登场。 - 谷歌市值最高点：2021 年 11 月 19 日，达到约 2 万亿美元。 - 一年之内跌幅近 50%，市值直线下坠到 1 万亿美元附近。 - 市场普遍质疑谷歌丧失了敏捷性，变得如同当年守着 Windows 利润的迟钝微软。 --- ### [02:20:26](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8426s) 弃用搜索还是自缚手脚？Noam Shazeer 与先驱聊天机器人 Meena 事实上，在 ChatGPT 惊艳全球之前，谷歌早已在内部孵化出了类似的聊天机器人原型。重构了 Transformer 架构、使其真正行之有效并作为核心作者之一的传神工程师 Noam Shazeer，在公司内部享有极高威望。早在 2017 年那篇划时代的《Attention Is All You Need》论文末尾，作者们就留下了极具预见性的终章宣言：“我们对基于注意力的模型的未来感到兴奋，并计划将它们应用于其他任务……扩展到文本之外的输入和输出模态，并探索图像、音频和视频等大型输入输出。” 论文发布后，Noam 迅速向谷歌最高管理层进言，提出了一个极其激进的构想：Transformer 的颠覆性如此巨大，以至于谷歌应该果断摒弃传统的网页索引和“10 个蓝色链接”搜索模式，将整个谷歌彻底重构为一个由单一超大 Transformer 驱动的全新体系。为了验证这个设想，Noam 在 2010 年代末到 2020 年前后，亲自为大型 Transformer 模型构建了一个聊天界面，并将其命名为 Meena（即后来 LaMDA 的前身）。 然而，这个极具突破性的内部聊天机器人当时根本无法对外发布： 1. **安全对齐机制缺失**：Meena 缺乏后来 ChatGPT 所拥有的后训练（Post-training）与人类反馈强化学习（RLHF），言行极易失控脱轨；当时甚至有内部测试者透露，如果直接询问它“谁应该去死”，它真的会一本正经地列出一串应该被处决的人名名单。 2. **毁灭性的商业模式自残**：如果将 google.com 直接改造成直接给出终极答案的对话机器人，用户不再点击展示广告与外部链接，谷歌每年赖以生存的数百亿美元搜索基本盘与广告现金流将直接崩塌。在 2021 年之前，这种主动“掀翻自身摇钱树”的提议在商业上完全行不通。 3. **法律与版权红线**：谷歌多年来一直深陷于被媒体和内容发布商指控“截留流量、切断读者与原创出版商联系”的反垄断与法律官司中。就连在搜索中推出非商业查询的“信息摘要框（Info Boxes）”，内部都历经数年慎重论证；若要全盘用 AI 答案替代来源链接，面临的法律门槛不可想象。 4. **品牌信誉风险与缺乏市场诉求**：用户高度依赖并极度信任谷歌给出的事实权威性，而当时的 LLM 极易产生幻觉，根本无法保证事实准确性；加之主流用户与市场在当时压根没有主动向谷歌提出过“我们需要一个 AI 聊天机器人”的明确诉求。 --- ### [02:25:34](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8734s) 意外的消费级爆款：OpenAI 的赶工上线与 ChatGPT 破局 到了 2022 年底，OpenAI 手中握有能力强大的 GPT-3.5，但团队内部同样在苦苦思索：这项底层技术到底该以怎样的形态商业化包装？彼时外界与 OpenAI 内部的主流预测，都认为这只是一门面向 B 端企业客户的 API 授权生意。就在此时，Sam Altman 随口提议：“我们干脆直接做个聊天界面吧，这感觉是最直观的交互方式。有人能顺手搞个网页聊天框出来吗？” 仅用大约一周时间，工程师就拼装出了一个简单的 Web 页面：用户输入一条消息，前端就向后端的 GPT-3.5 API 发起一次调用并回传文本。另一项催化剂则来自竞争谍报：OpenAI 内部打听到 Dario Amodei 创办的 Anthropic 正在研发后来名为 Claude 的对话产品，为了抢在 Anthropic 之前面世，OpenAI 决定火速抢跑。2022 年 11 月 30 日（正值感恩节后不久），OpenAI 正式发布了名为 ChatGPT 的研究预览版（Research Preview），Sam Altman 当天清晨在 Twitter（X）上简短发推：“今天我们上线了 ChatGPT，可以在这里跟它聊天。”并附上了 chat.openai.com 的链接。 随后发生的一切彻底失控，连 OpenAI 自己都始料未及： - **服务器被洪峰冲垮**：访问量呈现不可思议的指数级暴增，后台算力集群瞬间不堪重负频频宕机。 - **紧急向微软求援算力**：管理层不得不在即时极度紧急的情况下与微软展开谈判，通过加速追加投资协议来借支更多 Azure 算力配额。 - **仓促上线的付费墙**：为了遏制狂飙的运营成本与算力挤兑，团队在一个周末用 Stripe 仓促上线了一个付费订阅门槛以给访问降温；然而这道防线不仅没能劝退用户，反而意外引爆了消费级软件史上罕见的疯狂吸金神话。 Ben Thompson 随后戏称 OpenAI 是一家“意外闯入成功的消费级科技公司”（The accidental consumer tech company）。OpenAI 本无颠覆消费市场的战略蓝图，却无意间撕开了人类与 AI 交互历史的全新序章。 ### [02:28:43](https://www.youtube.com/watch?v=lCEB7xHer5U&t=8923s) ChatGPT 的指数级爆发与 Google 内部的 LaMDA ChatGPT 推出后的增长曲线彻底打破了科技史上的所有纪录。上线不到一周，它的用户数便突破了 100 万；到 2022 年 12 月 31 日（仅仅一个月后），月活跃用户飙升至 3000 万；而到了 2023 年 1 月底，即发布短短两个月内，ChatGPT 的注册用户数正式跨过 1 亿大关，成为人类历史上增长最快的产品。这种病毒式蔓延让整个科技界目瞪口呆，也在 Google 内部投下了引爆“红色警报”（Code Red）的重磅炸弹。 在这场风暴之前，Google 并非没有自己的聊天机器人。早在 Noam Shazeer 开发出聊天系统 Meena 后，Google 内部就一直在持续推进该项目，并逐步将其演化为大语言模型 LaMDA（Language Model for Dialogue Applications）。当时 Google 内部仍将底层模型名称与上层应用名称明确区分开来：LaMDA 是底座模型，同时 Google 内部也搭建了基于 LaMDA 的对话界面供员工使用。Noam Shazeer 当年曾极力向管理层建言必须尽早将该产品推向公众，但由于管理层屡次犹豫不决，他最终于 2021 年离开 Google，联合创办了 Character.ai 并获得了高额融资。讽刺的是，到了 2024 年 ChatGPT 掀起巨浪之后，Google 为了赢回关键人才，不得不斥资约 27 亿美元与 Character.ai 达成技术授权交易，其核心目的正是让 Larry Page 和 Sergey Brin 视若珍宝的 Noam 能够重返 Google 领军。 在 2021 至 2022 年间，Google 仍在按部就班地迭代 LaMDA。2022 年 5 月，Google 上线了一个名为 AI Test Kitchen 的测试平台，破天荒地允许公众试用包括 LaMDA 在内的部分内部 AI 体验——这其实在时间线上早于 ChatGPT。但为了防止模型在与用户的交互中“脱轨失控”，Google 施加了极其苛刻的防御限制：AI Test Kitchen 中的对话每次最多只能进行 5 个回合，一旦达到 5 轮交互，系统就会强行中断对话并礼貌致歉。 - **ChatGPT 增长里程碑**：不到 7 天破 100 万用户，1 个月破 3000 万，2 个月（2023 年 1 月）破 1 亿注册用户。 - **Noam Shazeer 与 Character.ai**：2021 年离开 Google 创立 Character.ai，2024 年 Google 斥资约 27 亿美元达成授权交易将其重新招致麾下。 - **AI Test Kitchen 的安全限制**：对话被硬编码限制在最多 5 个回合（five turns），防止长时间上下文导致幻觉或冒犯性言论。 ### [02:31:03](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9063s) 巨头的包袱：从“维持性创新”到“颠覆性威胁” Google 当时之所以对公众对话产品如此谨慎，背后有着充分的现实考量。2016 年微软在 Twitter 上线的聊天机器人 Tay 短短数小时内就被网友诱导成为满口仇恨与种族歧视言论的灾难级产品，给大厂声誉造成了极其恶劣的影响。因此，尽管 Sundar Pichai 早在 2017 年就高调宣布 Google 转型为“AI First”公司，但出于对庞大用户基数与品牌声誉的保护，Google 在面向普通消费者的纯 AI 交互产品上表现得极端克制。在 ChatGPT 诞生之前，全行业都认为 Google 确实是当之无愧的 AI 领头羊，但 Google 的做法是将先进的 AI 能力全面下沉、赋能于原有的既有业务（如搜索算法排位、Google 翻译、推荐系统）。 ChatGPT 的横空出世击碎了这种舒适区。Sundar Pichai、Larry Page、Sergey Brin 以及全体管理层瞬间意识到：这是一场关乎生死存亡的生存危机（existential threat）。ChatGPT 展现出了一种比传统 Google 搜索在体验上更直接、更自然、更能高效完成信息获取“任务”（Job to be done）的全新交互模式。更让 Google 胆战心惊的是，老对手微软迅速展开了凌厉的攻势。2023 年 2 月，微软高调宣布将 OpenAI 的模型能力深度整合到 Bing 搜索中。Satya Nadella 在发布会上意气风发地宣称：“这是搜索领域崭新的一天，竞争从今天正式开始……我们要重新思考搜索的本质。”他甚至在采访中留下了一句火药味十足的挑衅名言：“我想让大家知道，我们让 Google 跳起舞来了（I want people to know that we made Google dance）。”这是微软多年来第一次凭借全新且差异化的产品架构，直接杀入 Google 最核心的利润腹地。 用克莱顿·克里斯坦森（Clayton Christensen）的《创新者的窘境》理论来解构这一时刻：在此之前，Google 一直把 AI 视为一种**维持性创新（Sustaining Innovation）**。它让老牌巨头的护城河更宽、规模优势更强，通过持续的资本支出降低计算成本、提高广告转化率，初创企业根本无法望其项背。然而 ChatGPT 的爆发，在瞬息之间将 AI 转变为一种**颠覆性创新（Disruptive Innovation）**。Google 过去二十年积累的庞大资产、极其赚钱的搜索商业模式与繁复的安全审查机制，瞬间变成了沉重的包袱与负债。 ```mermaid flowchart TD accTitle: 搜索模式的颠覆性变革 accDescr: 展示 Google 传统维持性创新与 OpenAI 及微软颠覆性创新的对立演进路径 subgraph Sustaining["维持性创新路径 (Google 固有壁垒)"] A["传统 Google 搜索 & 广告引擎"] --> B["AI 作为底层支撑技术"] B --> C["强化广告变现与排名 (保护既有城堡)"] end subgraph Disruptive["颠覆性创新突袭 (OpenAI + 微软)"] D["ChatGPT 对话式原生交互"] --> E["直接解答用户意图 (无需点击链接)"] E --> F["New Bing 整合发布: 挑战 Google 核心阵地"] end Disruptive -.->|"2022 年底 Code Red: 商业模式面临同室操戈"| Sustaining ``` ### [02:35:34](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9334s) 红色警报（Code Red）与仓促上阵的 Bard 面对教科书般的颠覆性创新危机，Sundar Pichai 于 2022 年 12 月在 Google 全公司范围内拉响了最高级别的“红色警报”（Code Red）。他要求各业务线不再做鸵鸟，必须以最快速度打造并上线原生级 AI 产品。在接下来的数年里，Google 上演了一场如履薄冰的“走钢丝”平衡术：他们既要在非搜索冲突领域（如视频生成工具 Veo）全力进攻，利用 YouTube 的庞大海量训练数据构筑非蚕食性优势；又要极度谨慎地处理搜索主业。例如，后续推出的“AI Overviews”（AI 概览）只针对特定适合的搜索词展示，AI 模式只开放给特定人群，独立端产品 Gemini 也没有直接替代 Google.com 的首页域名。Google 必须在保护年利润超千亿美元的搜索基本盘的同时，尽可能孕育出不产生自我反噬的新物种。 然而局势的发展在变好之前往往会变得更糟。2023 年 1 月，微软宣布向 OpenAI 追加 100 亿美元投资，并在此番交易后实质持有 OpenAI 营利性实体高达 49% 的经济利益份额。在 Google 看来，自己最大的宿敌几乎与最可怕的 AI 颠覆者结为了利益共同体。内部压力达到顶点的 Google 仓促展开反击，将原有的 LaMDA 模型及对话套壳重新包装，命名为“Bard”（即后来的 Gemini 前身），并在 2023 年 2 月仓促面向公众全面发布（General Availability）。 事实证明，仓促应战的 Bard 是一款灾难性的半成品。当时行业尚未普遍认知到基于人类反馈的强化学习（RLHF，Reinforcement Learning from Human Feedback）对于调优大语言模型有多么关键。ChatGPT 之所以令人惊艳，很大程度上得益于 OpenAI 投入了极高密度的 RLHF 对模型的语气、回答的得体程度、准确度以及逻辑严密性进行了极致的对齐调校；而初版 Bard 完全缺乏这层“魔法调优”，回答内容生硬平庸。更致命的是，在 Google 发布的官方宣传演示视频中，Bard 针对詹姆斯·韦伯太空望远镜（JWST）提出的天文常识问题就给出了肉眼可见的事实性错误，直接引发了公众对 Google AI 技术的巨大质疑与华尔街的无情抛售。 - **战略走钢丝**：既不能任由 AI 原生产品彻底蚕食传统搜索广告收入，又必须快速上线 AI 产品阻击微软与 OpenAI。 - **微软的 100 亿美元大单**：2023 年 1 月宣布追加百亿美金，获得 OpenAI 营利实体 49% 股份，加剧了 Google 的危机感。 - **初版 Bard 的溃败**：2023 年 2 月仓促发布，严重缺乏成熟的 RLHF 机制，并在官方宣传片中当场出现幻觉与事实错误。 ### [02:39:23](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9563s) Bard 发难：灾难性发布会与模型落后的窘境 在谷歌仓促上阵的演示中，官方播出了一段预先录制、精心排练的视频来展示 Bard 的对话能力。然而令人窒息的是，视频中的 Bard 面对一个基础查询竟直接给出了事实性错误。两位主持人直言，这堪称“科技史上最糟糕的发布会之一”。发布会与 Bard 亮相的当天，谷歌股价重挫 8%。更糟糕的是，当实际产品面向公众开放后，大家迅速意识到它确实不好用。 问题的根源不仅在于聊天机器人的外层体验，而在于背后的底层模型本身就不行。2023 年 5 月，谷歌试图用 Google Brain 团队推出的 PaLM 模型替换最初的 LaMDA。PaLM 的表现略有起色，但不仅依然落后于 OpenAI 早先发布的 GPT-3.5，更在 OpenAI 于 2023 年 3 月祭出震撼业界的 GPT-4 之后被彻底拉开差距。这让谷歌高层意识到，小修小补已无济于事，必须进行根本性重组。 ### [02:40:18](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9618s) 破除藩篱：DeepMind 吞并 Brain 与组织权力重组 面对落后危机，谷歌 CEO 桑达尔·皮查伊（Sundar Pichai）做出了两个极其重大的战略决定。第一项决定是打破内部长达数年的派系分立：“谷歌内部绝不能再有两个独立的 AI 团队。”他正式宣布将 Google Brain 与 DeepMind 合并为单一实体——Google DeepMind。 这一合并在本质上彻底推翻了当年收购 DeepMind 时的协议约定。起初，DeepMind 在 Alphabet 架构下保持着高度独立的子公司地位（类似于 Waymo 等 Other Bets，而像 YouTube 这样的核心资产则是紧密依附于 Google），潜心钻研以强化学习（RL）为核心的前沿技术；而 Google Brain 则深植于 Google 内部。皮查伊打破这一平衡的方式，是直接将大权赋予戴密斯·哈萨比斯（Demis Hassabis），任命他担任整个合并后 Google DeepMind 的 CEO。皮查伊的潜台词非常明确：“我知道你们两边平时不对付，我也不在乎过去各自的章程是什么。现在全员进入战备状态（all hands on deck），DeepMind 全面接管并吸收 Google Brain，必须彻底改变过去十年谷歌内部研发与交付 AI 产品的文化。” ```mermaid flowchart TD accTitle: Google AI Labs Consolidation accDescr: Diagram showing the unification of Google Brain and DeepMind into Google DeepMind under Demis Hassabis, leading to the unified Gemini model. subgraph Before["合并前（派系分立）"] Brain["Google Brain<br/>(专注大语言模型与内部工程)"] DM["DeepMind<br/>(独立子公司 / 专注强化学习)"] end Merge{"皮查伊破局决策<br/>(2023年4月合并)"} subgraph After["合并后（集中力量）"] GDM["Google DeepMind<br/>CEO: Demis Hassabis"] Gemini["统一模型: Gemini<br/>(对内对外全部统一驱动)"] end Brain --> Merge DM --> Merge Merge --> GDM GDM --> Gemini ``` 在这场震荡前后，DeepMind 的第三位联合创始人穆斯塔法·苏莱曼（Mustafa Suleyman）的人生轨迹也迎来戏剧性转折： - 苏莱曼此前已离开 DeepMind 转入 Google 内部负责 AI 政策等事务。 - 不久后，苏莱曼与曾担任 DeepMind 伦理委员会成员的里德·霍夫曼（Reid Hoffman）联手创办了 Inflection AI。 - 随后经历 2023 年感恩节期间 OpenAI 惊心动魄的“奥特曼罢黜与闪电复职”风波，微软为了构建自身 AI 独立防线，通过一笔准收购（acqui-hire）交易吸收了 Inflection AI 的核心力量，并将苏莱曼任命为微软消费者 AI（Microsoft AI）负责人。 ### [02:43:25](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9805s) 孤注一掷 Gemini：以硬核技术命名的“统一大模型”战略 皮查伊做出的第二大决断是：集中全公司所有力量只打造一款模型——**Gemini**。从今往后，谷歌内部所有业务以及外部交付的所有 AI 产品，全部归拢由 Gemini 单一底座驱动，“不再有分裂的模型，不再有各自为政的团队”。 这项战略具有双重推拉效应：一方面，任何内部业务若需要 AI 能力，必须接入 Gemini；另一方面，管理层要求每个产品团队都要主动寻找将 Gemini 深度整合进自家业务的场景。节目指出，这与当年强推 Google+ 的做法有着本质区别： - **符合缩放定律（Scaling Laws）的商业理性**：大模型遵循严格的 Scaling Law，投喂的数据和算力越多，模型越强大。如果在大模型时代分散资源维护多个庞大系统，就是重复消耗天文数字般的算力与研发成本；集中所有资源压在唯一模型上，是极其理性的商业抉择。 - **放弃多余产品修饰的技术本色**：最初谷歌在面向消费者的聊天机器人上使用 Bard 名称，内部模型叫 Gemini；但随后谷歌果断将面向用户的产品名全部统一改为 Gemini。这展现出一种回归“纯粹技术底色”的哲学——就像当年 Gmail 凭借超大存储空间和极致搜索速度直接颠覆传统邮箱一样，Gemini 的核心就是裸露在外的突破性硬核技术，无需再包裹花哨的产品概念。 ### [02:46:35](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9995s) 转场 Waymo：2004 年 DARPA 挑战赛的无人驾驶火种 在盘点完 2023 年谷歌大模型业务的自我重塑后，讨论转向了 Alphabet 旗下在现实物理世界中最亮眼的 AI 标杆——**Waymo**。要讲清 Waymo 的崛起，时间必须倒回至 2004 年的美国国防部 DARPA 大挑战赛（DARPA Grand Challenge）。 这项最初旨在为军用地面无人机器人研发铺路的赛事，最终成为了培育现代自动驾驶汽车核心人才的摇篮。首届比赛在莫哈韦沙漠（Mojave Desert）举行，要求参赛车辆在长达 132 英里的越野荒漠土路上完全自主行进，车内及周围不得有任何人类物理干预。为了打破常规政府科研拨款的限制，美国国会专门立法批准设立了 100 万美元的冠军奖金。然而在 2004 年的首届比赛中，所有参赛的 100 支注册车队无一完赛，表现最好的赛车也仅仅跑了区区几英里。 但到了 2005 年的第二届挑战赛（奖金提升至 200 万美元），整个领域在 12 个月内所展现出的技术飞跃令人震撼： - 23 支进入决赛的赛车中，有 22 支跑过了前一年最好赛车所停下的极限位置。 - 共有 5 支车队完整征服了全部 132 英里的恶劣戈壁赛道。 - 其中两台完赛车来自机器人名校卡耐基梅隆大学（CMU），而最终夺冠的正是由**塞巴斯蒂安·特伦（Sebastian Thrun）**率领的斯坦福大学车队。 在莫哈韦沙漠颠簸、高温与强光干扰的极端环境下，车载传感器（激光雷达、摄像头等）会产生巨大的噪音数据。CMU 团队秉持传统硬工程思路竭尽全力进行物理标定，而特伦带领的斯坦福团队则另辟蹊径，引入了现代机器学习与概率推理算法来处理这些不确定性，这不仅为特伦此后创立 Google X 与无人车项目（后来的 Waymo）埋下了最初的伏笔，也彻底拉开了人类自动驾驶革命的序幕。 ### [02:49:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10172s) DARPA 挑战赛：卡耐基梅隆的“硬件硬刚”与斯坦福的“算法至上” 在 DARPA 自动驾驶挑战赛中，参赛团队在技术路线上产生了极其鲜明的分歧。以卡耐基梅隆大学（CMU）为代表的硬件派，将大量精力投入到定制硬件上以对抗行驶颠簸：他们打造了极其复杂的定制支架、万向节以及用来稳定传感器的巨大减震弹簧。CMU 团队的做法简单粗暴——直接买来一辆悍马（Hummer），把它彻底拆散，从底盘车轮开始重新切割焊接，完全是在硬核造车。 与此截然相反，塞巴斯蒂安·特伦（Sebastian Thrun）领导的斯坦福团队认为，车上每多增加一个定制硬件，就意味着多了一个在比赛当天发生机械故障的风险点。为了降低硬件故障概率，斯坦福团队选择了一辆几乎未作改装的普通大众汽车（Volkswagen），并在上面直接挂载消费级的普通摄像头和传感器。他们把全部创新重心放在软件上，坚信只要算法足够巧妙，就能把那些粗糙的原始数据清洗干净——这种“以软件定义一切、数据驱动纠偏”的理念，可以说“非常 Google”（Very Googly）。 ```mermaid flowchart TD accTitle: 2005年斯坦福自动驾驶团队多模态实时感知融合流程 accDescr: 展示激光雷达探测近处安全路径与彩色摄像头远景颜色外推的融合流程。 subgraph Sensors ["车载传感器输入 (普通大众车)"] Lidar["车顶激光雷达 (固定窄视场角)<br>高精度测量地面质感与纵深"] Camera["车顶常规彩色摄像头 (宽视场角)<br>观测远景至地平线并识别颜色"] end subgraph Processing ["2005 车载实时机器学习算法"] MapSafe["激光雷达划定近距离安全行驶路面"] Overlay["将激光雷达点云实时投影叠加至彩色视频帧"] ColorMatch["识别并提取安全路面所对应的像素色彩特征"] Extrapolate["沿相机远景画面将同色区域外推为行进路径"] end Result["安全高速巡航路径规划<br>(预判前方弯道与安全行车走廊)"] Lidar --> MapSafe Camera --> Overlay MapSafe --> Overlay Overlay --> ColorMatch ColorMatch --> Extrapolate Extrapolate --> Result ``` 斯坦福团队的第二大杀手锏，是在 2005 年就前瞻性地利用机器学习算法实现了多传感器的实时数据融合。车顶安装的激光雷达虽然能极其精准地测量车前地面的材质纹理和纵深，但它的视场角固定且极其狭窄，根本看不到远处，这就导致赛车不敢开快，也无法判断前方是否有弯道。为了打破这一限制，特伦团队在车顶加装了一个与人眼视野相似、能一直看到地平线且能分辨色彩的普通视频摄像头。算法实时将激光雷达的数据投影叠加到摄像头的视频画面上：激光雷达先测算出车头正前方一小块绝对安全的行驶区域，算法随即拾取该区域在摄像头画面中的颜色，然后向远方的画面画面进行外推匹配，凡是呈现相同色彩的远端路面就被判定为安全路径。 在 2005 年，仅凭塞在车厢正中央的一台类似戴尔台式机（Dell PC）的简陋车载电脑，斯坦福团队就完成了这一系列不可思议的实时计算。赛车凭借这种算法能够极具信心地高速巡航，并提前预判远处的弯道。最终，软件路线大获全胜，斯坦福在荒漠中上演了极其戏剧性的一幕——在全无人干预的自动驾驶状态下，直接超车超越了卡耐基梅隆的悍马战车，夺得 DARPA 挑战赛冠军。 ### [02:52:12](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10332s) Chauffeur 项目与“Larry 1000”极限大考 挑战赛结束后，拉里·佩奇（Larry Page）亲自找到特伦，极力劝阻他拿 Benchmark 和红杉资本的钱创业，将其招募进 Google。但特伦最初做的是街景（Street View）和 Ground Truth 项目，并联合创立了 Google X。不久后，拉里·佩奇再次找到特伦，直接对他说：“去做自动驾驶汽车吧。”特伦起初一口回绝：“得了吧，那只是个 DARPA 挑战赛玩具。城市里到处都是行人，我不可能把几吨重的杀人机器扔到马路上随便伤人。”佩奇反问了一个直击灵魂的问题：“为什么？到底有什么技术原因导致这件事情不可能实现？”特伦带着这个问题辗转反侧睡了一宿，第二天上班时对佩奇坦白：“我想通了，并没有不可克服的技术原因，我只是在害怕。” 只要把安全门槛提到最高、做好一切预防措施，工程就可以启动。为了给团队设立清晰的成败标尺，拉里·佩奇亲自拟定了一份长达 1,000 英里的极其险峻的加州路测路线，被团队内部戏称为“Larry 1000”。这条考核路线涵盖了各种极端路况：从旧金山开往太浩湖（Lake Tahoe）的蜿蜒山路、旧金山著名的九曲花街（Lombard Street）、险峻的加州一号公路直通洛杉矶，以及交通繁忙的旧金山-奥克兰海湾大桥（Bay Bridge）。只要能无人干预开完这 1,000 英里，就证明自动驾驶系统基本具备了通向一切场景的潜力。 - **闪电战破局**：代号为“Chauffeur”（司机）的自动驾驶项目于 2009 年在 Google X 正式立项。 - **18 个月的奇迹**：这支最初仅十余人的极小团队，在启动后的短短 18 个月内就跑完了数万英里自动驾驶里程，并且一举彻底攻克了严苛的“Larry 1000”路线。 ### [02:54:17](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10457s) 商业化分歧：买下特斯拉的平行宇宙 尽管在 18 个月内跑完了前 99% 的核心里程，但自动驾驶面临着极其残酷的“第二个 99%”——随之而来的是长达十余年漫长而痛苦的工程落地。天气突变、恶劣路况、不可预测的人类驾驶员、罕见道路拓扑、夜间照明等海量 Edge Cases（极端边界场景），让产品化步履维艰。此时，团队内部对这项技术应该做成什么样的产品和商业模式展开了激烈辩论： 1. **特伦的务实路线（高速公路辅助驾驶）**：主张做风险最低、落地最现实的公路辅助驾驶（Highway Assist），本质上是一个更好用的高级自适应巡航。 2. **埃里克·施密特（Eric Schmidt）的激进设想（全资收购特斯拉）**：提议直接买下特斯拉，以此作为 Google 的硬件与造车基本盘，再把 Google 的自动驾驶软硬件系统全面植入进去。当时特斯拉正处于 Model S 产能地狱的深渊中，双方甚至已经就收购展开过谈判。据透露，当时 Google 仅仅需要出资大约 30 亿美元即可将特斯拉收入囊中。如果这笔交易成真，后来的商业历史将彻底改写：DeepMind 可能不会以原有方式被收购，而埃隆·马斯克（Elon Musk）大概率也不会因为与佩奇在 AI 理念上的反目而转头创立 OpenAI。 3. **拉里·佩奇的终极野心（Robotaxi 无人出租车）**：直接一步到位打造全无人驾驶出租车网络。这也是 Google 最终坚定践行的路线。 ### [02:56:25](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10585s) Waymo 独立与世界首个商业化无人车队 自动驾驶行业的发展极度依赖一系列非线性的技术突破，而任何人都无法准确预判下一个突破究竟会在 2 年还是 10 年后到来。一个极其惊人的事实是：**在 Chauffeur 项目开展的前 5 年里，他们根本没有使用任何深度学习技术**。无论是攻克“Larry 1000”还是后续三年多的开发，全靠传统算法支撑。 - **2013-2014 年（引入 CNN）**：团队开始引入卷积神经网络（CNN）进行物体识别，系统的感知能力大幅跃升。这一时期恰逢 Google 全面部署 40,000 张 GPU，全公司内部对深度学习建立起了强烈的信仰。 - **2016 年（Waymo 独立）**：技术可行性得到充分验证，Alphabet 决定将其从 Google X 毕业拆分，正式成立独立子公司 Waymo。 - **2017 年（引入 Transformer）**：随着《Attention Is All You Need》论文发表，Waymo 迅速将 Transformer 架构的理解与建模能力引入到车辆的轨迹预测与路径规划系统之中。 - **2020 年 3 月（首轮外部巨额融资）**：Waymo 完成了 32 亿美元的大规模融资，投资方包括银湖资本（Silver Lake）、加拿大养老基金投资公司（CPPIB）、穆巴达拉（Mubadala）、Andreessen Horowitz（a16z），而母公司 Alphabet 开出了最大的支票，至今依然持有 Waymo 的绝对控股权。 - **2020 年 10 月（全球首发商用全无人运营）**：在 Chauffeur 项目立项整整 11 年之后，Waymo 在亚利桑那州凤凰城正式上线了全球首个主驾驶位完全没有人类安全员的商业化 Robotaxi 运营服务，成为自动驾驶商用化历史上的里程碑。 ### [02:58:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10712s) Waymo 商业化突破与旧金山的日常化 在经历了长期的研发试炼并达成“Larry 1000”（连续行驶 1,000 英里无需人工干预）等关键里程碑后，Waymo 迎来转折。主播感叹自己曾一度对自动驾驶彻底放弃希望，认为全自动驾驶不过是行业画饼，永远无法落地。然而，Waymo 却在菲尼克斯（Phoenix）不仅做到了大规模安全承载普通乘客，还真正实现了收费运营。随后，Waymo 进军旧金山，并在 2024 年 6 月正式全面开放服务。对于旧金山市民而言，Waymo 迅速从新鲜科技变成了日常生活密不可分的一部分。旧金山居民叫车的首选往往是“先看看有没有 Waymo，如果没有再考虑其他传统网约车”，且即将拓展至西雅图。 硬件层面，Waymo 自身并不制造整车底盘，而是采用捷豹（Jaguar）纯电车型定制改装，随后还将推出全新形态的厢式车（Van）。在此期间，Waymo 独立完成了两轮巨额融资：25 亿美元的 B 轮和 56 亿美元的 C 轮。到 2024 年 1 月，据报道 Waymo 在旧金山的订单总额（Gross Bookings）已经超越了 Lyft。人们身处未来，却往往极快地习惯并对身边的科技奇迹习以为常。 - **车队与车型**：基于捷豹（Jaguar I-PACE）电动车深度定制改装，下一代硬件平台正转向专属厢式车（极氪代工结构等）。 - **融资里程碑**：外部分别完成了 25 亿美元的 Series B 以及 5.6 亿美元的 Series C 轮融资。 - **旧金山市场份额**：总预订流水（Gross Bookings）已超过老牌网约车巨头 Lyft，成为当地首选出行工具。 ### [03:00:02](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10802s) 无人驾驶的全新体验与技术路径之争（Waymo vs. 特斯拉） 随着无人驾驶融入生活，乘客体会到的已不仅是“驾驶座没人”的初始猎奇感，而是一种彻底重构的产品体验。带着孩子出行时，乘客可以毫无顾虑地花时间在后排安装儿童安全座椅；带着宠物狗乘车时，完全免去了面对人类司机时的尴尬和被拒风险；商务出行时，乘客能在完全私密的车厢内畅谈敏感话题或拨打工作电话。 在运营规模上，Waymo 已拓展至菲尼克斯、旧金山、洛杉矶、奥斯汀和亚特兰大 5 座城市，并在筹备进军海外首站东京。整个车队拥有 2,000 多辆车，每周提供数十万次付费行程，累计全无人驾驶里程已突破 1 亿英里，且每周以 200 万英里的速度递增，累计付费行程突破 1,000 万次。 在感知路线上，Waymo 始终坚持始于 DARPA 挑战赛的多传感器融合架构，配备了 13 个摄像头、4 个激光雷达（LiDAR）、6 个毫米波雷达以及外部麦克风阵列。与特斯拉依靠纯视觉（Vision-only）的廉价方案相比，Waymo 的硬件成本极高，但 Waymo 团队坚信这是达到极高安全标准与通过监管审查的唯一路径。 ```mermaid flowchart TD accTitle: Waymo 多模态传感器融合方案 accDescr: 展示 Waymo 采集的四种物理信号如何融合汇入感知规划系统以达到极高安全指标 subgraph Sensors ["Waymo 外部传感器阵列"] A["13 颗高精度摄像头\n(视觉感知)"] B["4 颗激光雷达 LiDAR\n(厘米级 3D 几何建模)"] C["6 颗毫米波雷达\n(全天候速度与测距)"] D["外置麦克风阵列\n(紧急警报与环境音捕捉)"] end subgraph Perception ["自动驾驶感知与决策系统"] E["高冗余多模态数据融合"] F["严苛的安全冗余与监管合规标准"] end A --> E B --> E C --> E D --> E E --> F ``` - **运营数据规模**：覆盖 5 城，全无人累计行驶超 1 亿英里，每周新增 200 万英里，累计付费订单超 1,000 万次。 - **安全性实证**：Waymo 发布的对标研究显示，在城市地面道路严格同口径对比下，Waymo 相比人类驾驶员将造成重伤及以上的车祸率降低了 91%。 - **产品体验差异**：特斯拉 FSD 属于需要人类时刻监管的高级辅助驾驶，而 Waymo 提供了类似“Google 搜索”一般的确定性信任感，乘客可以完全放空。 ### [03:03:44](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11024s) 商业模式测算：事故消除价值能否撑起另一个 Google？ 无人驾驶出租车（Robotaxi）在早期面临极其沉重的成本结构：巨额的模型训练算力、实时推理开销、昂贵的传感器硬件，以及高昂的线下重资产运营成本（包括车辆充电、清洁、调度、出入库检修及传感器校准）。为了控制运营支出，Waymo 正在各类城市测试轻资产合作模式，例如与租车公司合作维护车队，或将运力直接接入 Uber 等网约车平台。 在估算其商业天花板时，主播提出了几种不同的估值拆解维度： 1. **主机厂总市值**：全球汽车制造商总市值约 1.3 万亿美元（包含特斯拉则为 2.5 万亿美元），但 Waymo 本质不是造车硬件厂商。 2. **现有网约车市场**：全球网约车上市公司市值约 3,000 亿美元（Uber 占绝大部分），但 Waymo 的愿景远超网约车，还涵盖干线物流卡车、私家车赋能以及为老人、视障人群提供出行自由。 3. **消除事故带来的全社会经济价值**：根据 CDC（美国疾病控制与预防中心）数据，2022 年全美交通事故致死造成的直接医疗与生命价值损失达 4,700 亿美元。如果 Waymo 将重特大事故率削减 90%（即 10 倍降低），每年可为美国社会节约超过 4,200 亿美元的综合成本。 这一潜在节约规模甚至超越了 Google 当下的年度总营收。只要未来能将单车制造成本与运营费用持续打磨下行，Waymo 具备在 Google 内部孵化出一个体量堪比“第二个 Google”的巨大商业前景。历经二十余年跋涉，这项投资从早期看似疯狂的登月计划，最终沉淀为了极具战略价值的颠覆性资产。 ### [03:07:31](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11251s) Waymo 的账面成本与谷歌全面转向 AI 主持人回顾了 Waymo 发展到如今规模的资本消耗：Waymo 迄今为止大约烧掉了 100 亿至 150 亿美元。虽然绝对数字听起来很惊人，但横向对比之下却显得格外划算——在当前前沿大模型动辄数百亿美元的基础训练与算力竞赛面前，这笔钱不过是“零花钱”（jump change）；甚至放到打车出行行业内部对比，100 到 150 亿美元仅仅相当于 Uber 一年的利润。更夸张的是，这笔累积多年的全部研发投入，不过是谷歌集团单单上个月一个月的净利润而已。曾经看似天方夜谭的无人驾驶“疯狂追逐”，在如今的技术落地与商业化节点下，被证明是一笔极其精明的战略投资。 随后讨论重心转回谷歌的核心 AI 进展。在 2022 年底 ChatGPT 问世引发震动后，谷歌首席执行官桑达尔·皮查伊（Sundar Pichai）于 2023 年中颁布了两项重大行政命令：第一，打破部门壁垒，将 Google Brain 与 DeepMind 正式合并为统一的 Google DeepMind 团队；第二，全公司统一技术底座，集中所有资源押注在下一代旗舰模型 Gemini 上，由合并后的团队主导开发，并强制要求谷歌内部所有产品未来必须全面接入此模型。 与此同时，谷歌联合创始人谢尔盖·布林（Sergey Brin）也重新戴上工牌，回到一线亲自编写代码、参与 Gemini 研发。来自 Google Brain 的顶尖科学家杰夫·迪安（Jeff Dean）与奥利奥尔·维尼亚尔斯（Oriol Vinyals）与 DeepMind 团队展开深度协同。更具标志性的是，谷歌通过对 Character.AI 的交易成功将核心大牛诺姆·沙齐尔（Noam Shazeer）迎回公司，由杰夫·迪安与诺姆·沙齐尔共同担任 Gemini 的联席技术负责人（Co-technical Leads）。顶级技术统帅的回归与重组，让外界重新对谷歌的 AI 战斗力产生坚定的信心。 ```mermaid flowchart TD accTitle: Google AI 战略重组与 Gemini 推进流程 accDescr: 展示谷歌在 2023 年整合 Brain 与 DeepMind 研发力量并全力押注统一模型 Gemini 的架构路径 subgraph Consolidation["组织与人才整合"] A["Google Brain"] & B["DeepMind"] --> C["合并为 Google DeepMind"] D["Sergey Brin 一线回归"] -.-> C E["Jeff Dean & Noam Shazeer<br/>出任 Gemini 联席技术负责人"] --> F["全公司标准模型: Gemini"] end subgraph Delivery["产品落地与技术迭代"] F --> G["多模态架构 (原生支持图文音视频)"] G --> H["Gemini 1.5 Pro<br/>(1M Token 超长上下文)"] H --> I["Gemini 2.0 / 2.5 Pro<br/>与 AI Mode / AI Overviews"] end ``` --- ### [03:09:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11372s) 全面以 AI 速度前进：Gemini 迭代与搜索核心变革 在 2023 年 5 月的 Google I/O 大会上，谷歌迅速正式对外公布了 Gemini 路线图，并同步发布了基于生成式 AI 的搜索试验功能——AI Overviews（初版名为 Search Generative Experience, SGE）。AI Overviews 先作为 Google Labs 项目试水，随后直接推广至普通用户的全量搜索场景。考虑到谷歌每天承受着不可思议的海量搜索查询，要在如此高吞吐的请求中嵌入大语言模型（LLM）推理，展现了谷歌底层 TPU 与数据中心基础设施极其恐怖的工程吞吐实力。从 2022 年 11 月 30 日 ChatGPT 发布，到 2023 年 5 月完成战略大转向并快速推向生产环境，谷歌在不到半年的时间里彻底切换到了“AI 运行速度”。 谷歌在产品与模型架构上的交付节奏随后进入爆发期： - **原生多模态架构**：2023 年 5 月立项的 Gemini 定位为原生多模态模型，单模型统一处理文本、图像、音频与视频，仅用 6 个月时间训练并于 2023 年 12 月向公众开放。 - **超长上下文突破**：2024 年 2 月发布 Gemini 1.5，带来了 100 万 Token 的上下文窗口，远超市场上任何同类竞品，直接解锁了一大批此前受限于上下文长度的企业与分析级长文本场景。 - **高频版本迭代**：2025 年 2 月推出 Gemini 2.0；紧接着 3 月发布实验版 Gemini 2.5 Pro，并于 6 月正式全面商用（GA），这一高密度迭代节奏直逼英伟达的硬件研发周期。 - **搜索产品变革**：2025 年 3 月，google.com 正式加入 AI 模式（AI Mode），用户可以直接切换至交互式聊天机器人形态，谷歌甚至开始进行 A/B 测试自动将部分用户导流至 AI 模式。这种在核心利润支柱上做结构性试验的动作，被主持人形象地比喻为“让大象跳踢踏舞”。 - **生态与创意应用群**：同期孵化了备受瞩目的 NotebookLM（其生成的双人播客音色逼真、节奏自然，甚至让主持人调侃“感觉像是照着 Acquired 训练的”）、视频生成工具 Veo 3、Nano-Banana，以及根据文字和视频提示实时构建 3D 虚拟世界的 Genie 3。 关于用户规模，谷歌官方披露 Gemini 拥有 4.5 亿月活跃用户（MAU）。主持人对这一数据进行了辩证拆解：尽管 Gemini 登顶了应用商店榜单，但 4.5 亿这一惊人数字很可能将接入 Nano-Banana、搜索里的 AI Overviews 或其他偶发触发场景全盘计入。这类似于 Meta 曾宣称其 Meta AI 用户量突破数亿，但其中大部分仅是在 Instagram 搜索栏误触 Llama 模型且迅速退出的被动用户。不过不可否认的是，在保持公司总收入创历史新高、核心搜索护城河未受毁灭性冲击的前提下，谷歌在极短时间内将 AI 业务从零拉升至当前规模，展现了极其强大的平台韧性与组织动员力。 --- ### [03:15:04](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11704s) 反垄断裁决反转与 AI 烧钱大战的真实图景 在审视谷歌当前业务大盘时，主持人们提到了美国司法部对谷歌反垄断案的最新戏剧性转折。主审法官虽然在裁定中明确认定谷歌在互联网搜索领域构成了非法垄断，但随后出台的救济措施却极为疲软：法院既没有强制拆分 Chrome 浏览器，也没有禁止谷歌每年向苹果等分销渠道支付数百亿美元的默认搜索引擎费用。 法官选择“高高举起、轻轻放下”的核心抗辩理由之一正是**生成式 AI 带来的新兴竞争局面**。法院认为，随着 OpenAI、Anthropic、Perplexity 等新兴创业公司相继获得数百亿美元的巨额融资，市场已经出现了能够挑战谷歌搜索霸权的全新力量，因此应当由自由市场去自我演化和竞争，而非通过司法行政手段直接重创谷歌的业务生态。 然而主持人对这一裁决逻辑提出了尖锐质疑： - **自我造血能力缺失**：目前无论是 OpenAI、Anthropic 还是 Perplexity，没有一家具备正向净利润或自我造血能力，完全依靠外部源源不断的风险投资来支撑极其昂贵的算力与推演成本。 - **资金供给风险**：一旦宏观流动性收缩或风险资本的注资龙头关闭，这些创业公司手头的现金储备将在短时间内被极速烧光。 - **垄断本质并未动摇**：如果没有这些外部资本的无限输血，谷歌在传统搜索与前沿 AI 领域依旧没有任何真正意义上自给自足的独立竞争对手。法院以“依靠风投输血的初创企业存在”为由判定谷歌面临严峻市场竞争，逻辑根基实际上非常脆弱。 ### [03:17:11](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11831s) 创新的多米诺骨牌与反垄断反转 谈到反垄断与司法部的压力，主持人指出一个极其讽刺的现实：如果这一切最终的结局是让 Google 避免了像当年微软反垄断案那样巨大的分心和毁灭性打击，甚至免于被美国联邦法院拆分，那么 Google 在生成式 AI 竞赛中遭遇的挫折可以说是“完全值得的”。 这里有一个充满黑色幽默的“多米诺骨牌”网络迷因：第一块倒下的小骨牌是 Ilya Sutskever 离开 Google 加入并创立 OpenAI；而一路推倒巨大墙壁的终极下游效应，竟然是“Google 没有被拆分反垄断肢解”。这个戏剧性的连锁反应让 Google 不再是一家独大、毫无制约的绝对垄断者，客观上“甚至拯救了 Google”。 ```mermaid flowchart LR accTitle: Ilya离职引发的多米诺效应 accDescr: 展示Ilya离开Google促成OpenAI崛起，最终反向减轻Google反垄断压力的逻辑图 subgraph S1["事件起因"] A["Ilya Sutskever<br/>离开 Google"] end subgraph S2["连锁反应"] B["联合创立 OpenAI<br/>触发 ChatGPT 浪潮"] C["AI 市场竞争急剧加剧<br/>打破垄断定式"] end subgraph S3["最终结局"] D["Google 摆脱<br/>'绝对垄断'拆分危机"] end A --> B B --> C C --> D ``` --- ### [03:17:50](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11870s) 史上最强印钞机：3700 亿美元营收与 1400 亿利润 回归到 Google 如今的底盘业务实力，其财务数字展现了令人窒息的赚钱机器属性： - 在过去 12 个月中，Google 创造了 **3700 亿美元** 的营收。 - 在净利润层面，过去 12 个月录得 **1400 亿美元** 利润，高于全球任何一家科技公司。放眼全世界，唯一利润超过 Google 的仅有石油巨头沙特阿美（Saudi Aramco）。 - 自 2015/2016 年节目制作《Alphabet 篇》以来的近十年中，即便经历了整个 AI 时代与激烈的产业变迁，Google 核心业务的规模依然暴增了 5 倍。 - 市值方面，Google 突破了此前 2 万亿美元的前高，在近期刚刚冲上 **3 万亿美元** 大关，成为仅次于英伟达（NVIDIA）、微软（Microsoft）和苹果（Apple）的全球第四大最具价值公司。 --- ### [03:18:54](https://www.youtube.com/watch?v=lCEB7xHer5U&t=11934s) 现金分配的谜团：激进资本开支、巨额回购与分红 分析资产负债表时出现了一个耐人寻味的现象。Google 目前账面上持有 **950 亿美元** 的现金及有价证券。主持人本以为会有更多，因为在 2021 年时这个数字曾高达 1400 亿美元。过去四年里，Google 已经从单纯的“现金积累模式”彻底切换到“疯狂配置/部署现金模式”： 1. **进攻型 Capex（资本支出）**：像 Meta、微软和亚马逊一样，将巨额现金倾注于 AI 数据中心的基础设施竞赛。 2. **股票回购与首次派息**：消耗现金最大宗的实际是庞大的股票回购计划以及开始向股东派发股息。 从公司金融的角度解读这一策略：Google 核心搜索广告业务的现金造血能力实在太夸张了。哪怕面对人类商业史上资本最密集的一场军备竞赛，Google 在豪掷千金确保“我们一定要赢”并且留足了安全缓冲垫之后，手里竟然还有花不完的过剩现金，不得不通过分红和回购返还给股东。 --- ### [03:20:25](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12025s) Gemini 业务迷局与 1.5 亿订阅的“全家桶”逻辑 在具体评估 Google 的 AI 与企业端战略时，焦点落在两大业务上：Gemini 和 Google Cloud。 Google 一向是财报颗粒度极其模糊的公司，外界无法从财报中单独拆出 Gemini 带来的独立收入。但公开数据透露了关键线索： - **Google One 订阅人数破 1.5 亿**：Google One 云端存储等服务的付费订阅用户已超过 1.5 亿，且每年维持近 50% 的惊人增速。 - **层级分化**：绝大部分订阅者集中在 5 美元或 10 美元/月的低阶存储套餐；包含高级 AI 功能的套餐需要 20 美元/月，目前只占这 1.5 亿用户中的一小部分。 主持人在与前 YouTube 产品高管 Shishir Mehrotra 深度交流后改变了看法：以往外界普遍认为像搜索一样，AI 只能走广告变现，因为一旦收费，潜在市场规模（TAM）就会缩水 90% 到 99%。但 Shishir 提出了强有力的反驳——**“杀手级捆绑包”（Bundle）逻辑**。Google 坐拥无与伦比的数字资产体系： - YouTube Premium - NFL Sunday Ticket（周日门票） - Google Play 商店特权 - YouTube Music - Google One 云存储 如果将 Gemini 等先进 AI 工具巧妙打包进这样的超级消费级订阅包中，依托精妙的捆绑包经济学，Google 有望打造出全球规模最庞大、覆盖数亿级付费用户的 AI 订阅产品，彻底颠覆传统收费逻辑。 --- ### [03:22:54](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12174s) Google Cloud 的曲折蜕变：从极客 PaaS 到开源对决 Google Cloud 如今已成为 AI 战略的核心支撑，但其诞生之路充满坎坷： - **2008 年的 Google App Engine（PaaS 路线）**：Google 早期推出的是高度死板、自上而下的平台即服务（Platform as a Service）。它极其带有“Google 工程师的傲慢偏好”，开发者只能用特定 SDK、限定在 Python 或 Java 编写，并强制按照 Google 规定的方式部署，无法像 AWS 那样提供自由的基础设施。 - **AWS 与 IaaS 的胜利**：亚马逊 AWS（以及随后的微软 Azure）证明了“基础设施即服务”（IaaS，提供最底层的虚拟机、存储、计算与数据库积木）才是开发者真正需要的解法。 - **2012 年亡羊补牢**：历时四年，Google 才推出 Google Compute Engine，也就是后来重命名的 Google Cloud Platform（GCP）。 ```mermaid flowchart TD accTitle: Google Cloud发展演变图 accDescr: 展示Google Cloud从App Engine到IaaS再到Kubernetes的演化路线 subgraph P1["2008: 局限的PaaS时代"] A["Google App Engine<br/>限定语言/SDK/强管控架构"] end subgraph P2["2012: 转向底层IaaS"] B["Google Compute Engine (GCP)<br/>提供基础存储、虚机与计算资源"] end subgraph P3["2018: 容器生态反攻"] C["开源 Kubernetes (K8s)<br/>确立容器编排标准，推行多云战略"] end A -->|对标AWS落败后转型| B B -->|对抗AWS/Azure两强| C ``` 早期的 Google 基因与企业端（To B）完全格格不入： - **缺乏企业服务文化**：Google 习惯于打磨极致的消费级产品和完全自助式（Self-serve）的广告平台。广告商不用 Google 搜索就无处可去，因此 Google 过去根本不需要伺候企业大客户。但云计算是一场血腥残酷的“白刃战”，比拼的是极低的价格、复杂的解决方案打包和极深的企业关系。 - **吝啬核心资产**：早期 Google 视底层基础设施为“皇冠上的明珠”，将 Bigtable、Borg、DistBelief 等内部最优秀的分布式与 AI 框架严密保护起来，拒绝开放给外部客户。 - **转折点（2018-2019）**：直到 2017 年（业务开展 10 年后），其云营收仅有约 40 亿美元。随着前甲骨文（Oracle）高管托马斯·库里安（Thomas Kurian）的加盟，局面彻底改变。 - **Kubernetes 破局**：2018 年 Google 做出关键战略举措——开源并主推 Kubernetes。作为云计算的“老三”，Google 没有任何存量包袱，通过将容器编排技术确立为业界通用范式，支持应用在多个云之间自由迁移，成功对 AWS 和 Azure 的封闭生态发起了漂亮的“反定位”（Counter-positioning）进攻。 ### [03:26:35](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12395s) Thomas Kurian 入主与 Google Cloud 的企业级蜕变 在确立多云（Multicloud）策略后，Google 向企业界传达的信息变得极其务实：“如果你想要多云架构，我们会帮你轻松管理，你当然也可以选用 AWS 或 Azure。”这构成了其云业务复兴的核心支柱之一。随后在 2018 年底，前 Oracle 总裁托马斯·库里安（Thomas Kurian）正式掌舵 Google Cloud。对于一家长期深谙底层架构却极度不懂企业客户需求的工程师文化公司来说，找来一位在传统企业级市场摸爬滚打数十年的高管，简直是最完美的人选。 ```mermaid flowchart TD accTitle: Google Cloud Transformation Under Thomas Kurian accDescr: 展现托马斯·库里安接手 Google Cloud 后在组织架构、产品战略和业绩上的爆发性转变。 subgraph OrgTransition["GTM 团队跨越式扩张"] A["2018 年初基准<br/>约 150 人销售团队<br/>（多局限于加州本部）"] --> B["激进扩张阶段<br/>招募超 10,000 名 GTM 人才<br/>实现全球本地化覆盖"] end subgraph StrategicPillars["战略与产品重塑"] C["拥抱多云架构 Anthos"] D["全面迎合企业级开发者习惯"] E["自研 TPU 算力集群独家赋能"] end OrgTransition --> F["业绩成果爆发"] StrategicPillars --> F subgraph FinancialMilestones["财务里程碑与规模爆发"] F --> G["2020: 营收突破 130 亿美元"] G --> H["2022: 营收达 260 亿美元"] H --> I["2023: 首次实现扭亏为盈"] I --> J["当前: 年化营收超 500 亿美元<br/>（YoY 增长 30%，5年翻5倍）"] end ``` 这种组织层面的根本性补课立刻在财务数字上体现出来。Google 过去其实一直是一家“云公司”，坐拥全世界最顶尖的基础设施工程师，但他们的产品化思路极其“Google 范儿”——即只为自己或超级极客写工具，而不是帮传统企业用他们习惯的方式建构软件。Kurian 彻底颠覆了这一点： - **GTM（进入市场）军团暴增**：接管初期 Google Cloud 的销售团队仅有 150 人左右且多偏安于加州本土；Kurian 大刀阔斧在全球招募了约 10,000 名销售与商务拓展人员，搭建起遍布全球的企业服务网络。 - **财务暴涨与盈利突破**：2020 年云业务收入突破 130 亿美元，三年时间翻了近两倍；2022 年翻番至 260 亿美元；到了 2023 年，Google Cloud 不仅站稳了全球真正可靠的“第三大公共云”，更历史上首次扭亏为盈。 - **当下的行业增速领跑者**：目前其年化营收运行率（Annual Run Rate）已突破 500 亿美元，同比增速高达 30%，成为全球头部超大规模云厂商（Hyperscalers）中增长最快的玩家，在过去五年内实现了 5 倍的规模扩张。 ### [03:28:49](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12529s) AI 时代的垂直全栈与自研 TPU 的生态底牌 之所以将 Google Cloud 放到本期节目的核心位置，是因为它在 AI 时代具备了当年推出 App Engine 时始料未及的两大战略意义。第一，云是当代 AI 最根本的分发引擎。放眼全球科技巨头，参与当下的 AI 竞赛至少需要手握四大支柱之一：超级应用（Application）、顶尖前沿模型（Model）、定制专用芯片（Chip）、或者超大规模云（Cloud）。Google 是全球唯一一家同时将这四大要素牢牢握在手中的全栈巨头： - **英伟达（Nvidia）**：本质上是一家芯片公司（虽有云端服务但并非核心）； - **Meta**：目前全力聚焦在应用层，在前沿闭源模型的直接商业竞争中暂时缺席； - **亚马逊（AWS）与微软（Azure）**：核心依托庞大的云基础设施与生态，缺乏自研达到顶尖水平的核心前沿模型； - **OpenAI 与 Anthropic**：拥有全球最好的模型，却没有自己的数据中心与芯片，极度依赖外部资本与算力代工。 这也引出了 Google Cloud 独一无二的第二大战略价值——TPU 芯片的商业化出海口。如果 Google 没有自己的公共云，其历时十余年研发的 TPU 就只能局限在内部业务使用，因为竞争对手 AWS 或微软绝无可能主动在自己的数据中心部署 Google 的硬件。Google Cloud 让外部初创团队、研究机构和开发者得以直接调用海量 TPU。在全行业向英伟达极度饥渴地“乞讨”GPU 配额、受制于 CUDA 软硬件锁定的背景下，只要开发者愿意拥抱 Google 的开发技术栈，Google Cloud 就能提供取之不尽的 TPU 算力支持。 不仅如此，坊间已有传闻称部分新兴的“新云厂商（Neoclouds）”可能在未来数月内提供 Google TPU。Google 推动 TPU 外流的目标并非转型为英伟达那样的纯硬件销售商，而是希望打破 CUDA 的垄断，效仿其建立一个跨越公有云与异构环境的 TPU 软件生态圈。 ### [03:32:28](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12748s) 多头论点（Bull Case）：无可比拟的分发、造血能力与 YouTube 终极护城河 随后节目重新带回经典环节——“多头与空头（Bull and Bear）”推演，首先展开对 Google 未来的极度看多视角。David 与 Ben 一致认为，从产品与商业基本面看，AI 依然是“Google 占尽优势、输赢全看自身执行（Google's to lose）”的战局： 1. **人类互联网“总前门”的绝对分发权**：Google 掌握着通往全球数十亿网民的默认入口。从传统的十条蓝色链接平滑过渡到 AI Overviews，即使 ChatGPT 席卷全球，Google 的搜索总体流量依然维持在历史最高水平附近，用户几十年的默认习惯具有巨大的惯性。在模型能力上，Gemini 并不逊色于 OpenAI 或 Anthropic 的主力模型，其商业化的“价值创造”能力毋庸置疑，只是在等待新的“价值捕获（商业变现）”形态彻底定型。 2. **唯一具备自我造血闭环的 AI 巨头**：放眼全球的大模型军备竞赛，所有顶尖模型初创公司（如 OpenAI、Anthropic）本质上都极度失血并依赖外部风险投资；而 Google 是全球唯一一家自研顶尖前沿模型、同时拥有无底洞般广告造血现金流的公司。其搜索与广告业务不仅能全额资助数百亿美元的算力开销，甚至还能盈余大量资金向股东派息和回购股票。 3. **未被充分定价的超级底层网络**：2000 年互联网泡沫破裂时，Google 曾以“一折甩卖”的地摊价大举收购了全美铺设的海量暗光纤（Dark Fiber）。在过去十余年中，Google 陆续激活了这些资产，搭建起当今世界上规模最庞大、带宽极宽的数据中心私有骨干网。没有任何一家竞争对手拥有如此低延迟且吞吐量惊人的跨机房直连管网。 4. **视频互联网的终极壁垒——YouTube**：科技分析师本·汤普森（Ben Thompson）指出，以纯文字为核心的互联网只是因早期带宽受限而出现的过渡形态，互联网的终极体验必然是无处不在的高清视频，而当今人类实质上已经生活在“YouTube 互联网”之中。YouTube 不仅是全球第二大搜索引擎，更拥有独一无二、规模最大且持续更新的长短视频 UGC 媒体库，为 Google 多模态模型的训练提供了任何对手都无法企及的独家真实世界数据燃料。 ### [03:35:53](https://www.youtube.com/watch?v=lCEB7xHer5U&t=12953s) YouTube 的多模态变现潜力与顶级人才储备 在看多 Google 的逻辑中，视频与多模态是一个巨大的增长引擎。例如，Google 可以在 YouTube 视频中利用 AI 自动标记或识别出现的每一件商品，让每一个视频瞬间变成可直接点击购买的货架（shoppable video）。这种操作完全无需人工参与，机器自动标注后，直接套用 Google 成熟的广告投放与分成系统即可运作。同时，Google 正在构建的视频生成与编辑模型（如 Flow 和 Veo）将极大降低创作门槛、丰富平台内容，直接拉动用户参与度并为 YouTube 带来成倍的广告收入。 此外，尽管近年来面临激烈的外部竞争甚至人才流失，Google 依然拥有全行业最深厚的顶尖人才梯队。更重要的是，他们已经用行动证明，为了留住或重新召回关键顶尖人才，公司毫不吝啬斥资数十亿美元。 - **视频电商化**：通过 AI 自动解析并标记视频内商品，无缝接入广告竞价变现体系。 - **创意工具加持**：Flow 与 Veo 等生成式视频工具不仅提升创作者生产力，还直接提高用户停留时间与广告展示量。 - **人才护城河**：底蕴深厚的研究团队与雄厚的资本后盾，使其在核心人才竞争中依旧保持绝对竞争力。 --- ### [03:36:48](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13008s) 芯片单位经济学：TPU 与“黄仁勋税”的巨大差距 从底层硬件的单位经济学（Unit Economics）来看，Google 拥有其他竞争对手难以企及的成本优势。目前全行业都在向 NVIDIA 采购芯片，NVIDIA 的毛利率高达 75% 至 80%，这意味着芯片硬件成本上叠加了 4 到 5 倍的溢价，行业常称之为“黄仁勋税”（Jensen tax）或 NVIDIA 税。不管将其归结为定价权还是供应短缺，只要无法自研芯片，就必须向 NVIDIA 支付巨额溢价。 反观 Google，其 TPU 依赖硬件合作伙伴博通（Broadcom）协助处理芯片与台积电（TSMC）的对接制造。据业内测算，博通在 TPU 业务上的毛利率大约在 50% 左右。供应商 50% 的毛利率对应约 2 倍溢价，而 80% 的毛利率则对应 5 倍溢价，这两者在规模化部署中有着天壤之别。 ```mermaid flowchart TD accTitle: AI芯片采购成本结构对比 accDescr: 对比行业主流采购NVIDIA芯片与Google自研TPU的供应商毛利及加价倍数差异 subgraph Industry["行业普遍现状（依赖 NVIDIA）"] N1["台积电代工制造"] --> N2["NVIDIA 硬件系统"] N2 -->|"75%~80% 供应商毛利<br/>(4~5倍加价)"| N3["高昂的 AI 集群部署成本"] end subgraph GoogleTPU["Google 路径（自研 TPU）"] G1["台积电代工制造"] --> G2["博通 (Broadcom) 协助工程"] G2 -->|"约 50% 供应商毛利<br/>(约 2倍加价)"| G3["低成本自研 TPU 集群"] end ``` 在 AI 数据中心的总体拥有成本（TCO）或模型训练开销中，芯片是绝对的大头： - **芯片与折旧**：占 AI 数据中心总运营成本的 50% 以上。由于算法需求与制造工艺迭代极其迅猛，AI 芯片的折旧周期最多只有 5 年，甚至在当前摩尔定律般的军备竞赛下，5 年折旧都显得过于乐观（正如黄仁勋在 GTC 大会上调侃 Hopper 时所言，刚火爆的 H100 转眼就面临被 Blackwell 淘汰）。 - **研发与人力成本**：招聘顶尖 AI 研究员与软件工程师等 R&D 支出占比约为 25% 至 33%。 - **电力成本**：虽然外界广泛关注能耗，但实际仅占运营总成本的 2% 至 6%。 因此，数据中心总成本对芯片供应商毛利率极度敏感。正如 Atreides Management 投资人 Gavin Baker 所分析：传统科技浪潮中，最低成本制造者往往不是赢家（Google 并非靠低搜索成本打败对手，苹果也非靠低价取胜）；但生成式 AI 时代大不相同——AI 公司目前毛利率普遍只有 50% 左右，不再享有传统软件 80% 的高毛利。在这种重算力模型下，Google 凭借自主基础设施与低溢价硬件，成为了全球“最低成本 Token 生产商”，这为其构筑了极具确定性的长期护城河。 --- ### [03:40:39](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13239s) 多模态数据生态与 AI 搜索的超高广告意图 看多逻辑的另一根支柱，是 Google 庞大消费级产品矩阵所沉淀的专有数据。从 Gmail、Google Maps、Google Docs 到 Chrome 和 Android 系统，这些涵盖用户日常各维度的全方位个性化数据属于 Google 私有，没有任何一家第三方模型拥有如此丰富的上下文来打造个性化 AI 体验。 关于 AI 商业模式能否超越传统搜索，看多派认为它蕴含着更大的商业价值： - **查询长度与意图精准度**：Bill Gross 指出，传统网页搜索平均输入仅 2 到 3 个词，而 AI 对话往往超过 20 个词。用户在对话中吐露了极深的需求细节，这种极高的“商业意图”（Intent）让广告靶向匹配精度达到前所未有的水准，理论上应支撑远高于传统搜索的广告单价（Ad rates）。 - **交互大盘的扩张**：AI 承接了大量过去在线下发生或原本不会被搜索解答的复杂任务，数字交互的总时长与场景显著增加，整个流量与变现的基本盘变得更大。 - **超越搜索的衍生巨头**：Waymo 已经展现出成为另一个“Google 级”体量业务的势头，再加上机器人等衍生赛道，AI 开辟的全新市场体量远远超越了传统搜索的边界。至于终极的 AGI 畅想，如果 Google 率先达成，甚至将颠覆现有的商业分析框架。 --- ### [03:43:09](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13389s) 看空逻辑：价值捕获缺位与搜索腹地受袭 转向看空视角（Bear Case），现实的商业转化并没有理论推导那么乐观： - **商业变现与价值捕获（Value Capture）严重断层**：在粗略测算下，Google 依靠免费搜索在全美平均每年能从每位用户身上榨取约 400 美元的广告收益。但面对纯 AI 界面，全美能有多少比例的用户愿意自掏腰包每年支付 400 美元订阅费？目前 AI 产品形态极其不利于传统广告展示，造就了“创造了海量价值，却难以有效捕获价值”的尴尬处境。回看 1998 年 Google 创立，仅用 2 年便推出了 AdWords 这一现金奶牛；而现在的 AI 商业模式依然迷雾重重。 - **产品不再具备绝对领先优势**：1998 年的 Google 搜索一经推出就展现出压倒性的产品优势；然而在 AI 聊天机器人领域，Google 起步阶段的产品甚至是肉眼可见的落后，如今经过追赶也仅仅是与竞争对手打成平手。 - **市场份额的稀释与高价值场景流失**：Google 牢牢掌控着 90% 的传统搜索市场，但在 AI 市场根本不可能维持 90% 的统治力。稳态下其市场份额可能被稀释至 25% 到 50%，这意味着即便单位变现能力追平搜索，整体营收基数也会被多寡头瓜分。更致命的是，AI 正在直接分流传统搜索的核心用例——尤其是像“旅行规划”这类商业价值极高、原本能带来大量高客单价广告点击的黄金场景，正被用户加速转移到 AI 对话界面中完成。 ### [03:45:17](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13517s) 搜索垂直领域的侵蚀与在位者的舆论劣势 高价值垂直搜索正在被生成式 AI 迅速分流。以往在传统搜索中，像 Expedia 这样的旅游预订广告，或是针对“间皮瘤（Mesothelioma）”这类极高商业价值的医疗健康与诉讼法律广告，都是 Google 利润最丰厚的现金奶牛。但如今，当用户直接向 AI 询问复杂的病症可能或制定旅游计划时，传统的广告位究竟该安插在哪里成了难题。即便这最终可能被证明只是一个广告产品形态设计的过渡问题，不可否认的是，这些原本单次点击价值极高的搜索查询，正在成为第一批被分流到 AI 对话界面中的阵地。 更深层的看空逻辑在于 Google 如今所处的舆论与生态位困局。创业时期乃至整个移动互联网转型期，Google 都是受人爱戴的挑战者，公众和整个科技生态都在为他们喝彩；然而在这次 AI 变革中，Google 成为了庞大而保守的“在位者（incumbent）”。科技巨头如今在公众心目中已退化为冷冰冰的“大型基础设施”，不再享有当年的公关光环与群众好感，甚至整个科技行业在全球范围内的风评都大不如前。相反，现在的初创企业（OpenAI、Anthropic 等）牢牢占据着公众的情感与关注度。 - 相比 Google 当年敢在 IPO 静默期接受《花花公子》（Playboy）杂志专访的无所畏惧与随性，现代初创企业过早地背负了科技巨头的包袱。 - 尽管 OpenAI 内部人事与管理闹剧频出，称不上是成熟的企业，但大众和生态系统仍然本能地更偏向初创企业，使得 Google 这次转型所面对的社会心理阻力远大于以往。 ### [03:47:15](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13635s) 7 Powers 框架解剖：Google 在 AI 时代的护城河 通过汉密尔顿·赫尔默（Hamilton Helmer）的“七大权力（7 Powers）”模型，可以严格评估 Google 在面对 OpenAI、Anthropic、Perplexity、Grok 及 Meta AI 等竞争对手时，其 AI 产品（如 Gemini、AI Overviews 等）能否获得可持续的超额利润： ```mermaid flowchart TD accTitle: Google 在 AI 时代的核心权力评估 accDescr: 评估 Google 在 AI 领域具备、潜在具备或缺失的竞争优势（7 Powers） subgraph CorePowers["核心具备的核心壁垒"] A["规模经济 (Scale Economies)<br/>天量推理摊销固定训练成本"] B["独占资源 (Cornered Resource)<br/>不可撼动的 Google Search 分发入口"] C["品牌认知 (Branding)<br/>大众对传统巨头的信任（双刃剑但整体向好）"] end subgraph PotentialPowers["潜在可能具备"] D["转换成本 (Switching Costs)<br/>深度整合日历、邮件等个人 AI 后形成"] end subgraph LackingPowers["目前欠缺或被针对"] E["逆向定位 (Counterpositioning)<br/>正处于被挑战者逆向定位的被动局面"] F["网络效应 (Network Economies)<br/>模型单方面抓取全网数据，无显著用户飞轮"] G["流程权力 (Process Power)<br/>前沿突破分散在各家实验室，无垄断性工艺"] end ``` **规模经济（Scale Economies）**是 Google 最无可撼动的超级护城河。AI 模型高昂的固定训练成本需要巨大的推理吞吐量来摊销。Google 拥有横跨全球产品的海量推理需求：2024 年 4 月，Google 旗下所有平台处理的 Token 总量为 10 万亿；到 2025 年 4 月，这一数字暴涨至近 500 万亿，一年内实现了惊人的 50 倍增长；而在 2025 年 4 月至 6 月间，月处理量更是从略低于 500 万亿飙升至 980 万亿（接近 1000 万亿 / 1 Quadrillion），并在夏末突破多千兆量级。这种恐怖的规模使得 Google 能够以极低的边际成本将经过蒸馏的轻量模型直接嵌入每一次搜索中。 - **独占资源（Cornered Resource）**：Google 掌握着全网最无敌的分发渠道——Google Search。即便 ChatGPT 已经像“舒洁（Kleenex）”一样成为了对话 AI 的代名词，Google 拥有的分发能力依旧无人能及。 - **转换成本（Switching Costs）**：目前消费级独立 AI 工具的转换成本极低，用户在 Gemini 与其他工具间横跳毫无负担；但一旦进入企业服务，或未来个人 AI 深度整合了个人的 Gmail、日历与工作流，转换成本将大幅攀升。 - **网络效应与流程权力（Network Economies & Process Power）**：目前基础模型行业尚未显现出显著的网络效应，用户使用 Gemini 并不会直接让其他用户的体验变好；同时，突破性论文和顶尖模型在各家实验室交替领先，Google 无法宣称拥有垄断的流程能力。 - **逆向定位（Counterpositioning）**：Google 毫无此项优势，自身正处于被所有 AI 初创企业逆向定位攻击的防守境地。 ### [03:51:43](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13903s) 创新的窘境之巅：使命与利润的终极走钢丝 Google 正在上演商业史上最扣人心弦的“创新者的窘境（Innovator's Dilemma）”。创始人拉里·佩奇与谢尔盖·布林牢牢掌控着公司投票权，两人曾多次表态“宁可破产，也绝不在 AI 竞争中落败”。然而，摆在他们面前的是一个极其残酷的抉择：AI 究竟能否成为一个像搜索那样暴利的商业模式？如果 AI 的盈利能力注定逊色于搜索，管理层究竟会优先选择践行“整合全球信息，供大众使用并使人人受益”的宏大初衷，还是不惜代价维持全球最赚钱科技帝国的利润机器？ 如果纯粹以使命为导向，Google 应当更加激进地将全站流量直接一刀切切换为完整的 Gemini 模式。但作为一家需要对公开市场股东负责的上市公司，桑达尔·皮查伊（Sundar Pichai）带领的领导团队正在极其狭窄的刀刃上小心平衡。 令人佩服的是，Google 在过去一段时间展现出了极高的战略执行力：果断合并 Google Brain 与 DeepMind、终结内部算力与人才内耗、标准化统一到单一模型家族快速迭代，同时在产品端谨慎防护核心搜索的基本盘，做到了“敏捷而不轻率（rapid but not rash）”。虽然这种温和的过渡可能隐藏着底层护城河正在被侵蚀的隐患，但与五年前的混乱相比，Google 当前所交出的答卷已经展现了顶尖科技巨头惊人的自我调适能力。 ### [03:54:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=14072s) 重磅预告：Acquired 即将奔赴超级碗创新峰会 在结束长达数月、跨越多个篇章的 Google 巨幅商业史梳理之际，两位主持人兴奋地宣布了一则特别消息：NFL（美国国家橄榄球联盟）官方正式向 Acquired 发出邀请。播客团队将前往超级碗（Super Bowl）现场，参与由 NFL 主办的创新峰会，开启一段全新的内容探索之旅。 ### [03:54:59](https://www.youtube.com/watch?v=lCEB7xHer5U&t=14099s) 超级碗创新峰会预告与 Ben 的 Carve Out：电影《F1》与 Travelpro 行李箱 节目进入尾声，主持人 Ben Gilbert 和 David Rosenthal 宣布了一项重磅活动：今年 2 月的超级碗（Super Bowl）即将在旧金山举行，NFL 顺理成章地决定在超级碗周五（即超级碗周日的两天前）在旧金山举办一场“创新峰会（Innovation Summit）”，而 Acquired 节目组受邀担任这场峰会的主持人。Ben 和 David 届时将在台上主持一系列重磅采访和现场环节。考虑到超级碗周旧金山全城的场地异常紧俏，小礼堂容纳不下几百万听众，现场活动将向公众提供线上直播，后续会公布收看方式。 随后进入例行的 Carve Out（好物/内容推荐）环节。Ben 首先分享了他最近终于去影院看的电影《F1》（由布拉德·皮特主演）。他盛赞无论你是不是 F1 车迷，这部电影都是极致震撼的“优美电影美学”，环绕音效体验无与伦比。David 感叹自己有了两个年幼的孩子后，除了泰勒·斯威夫特的《时代巡回演唱会》（The Eras Tour）大电影外，就再也没进过电影院。 接着 Ben 推荐了第二样好物——Travelpro 大号行李箱。David 调侃这难道不是飞行员和空乘人员专用的品牌吗？Ben 解释道，空乘人员通常用 Briggs & Riley 或 Tumi 等高端品牌，而 Travelpro 则是极致性价比的选择。他最近为了带着 2 岁的幼童进行国际长途旅行，在亚马逊上买了两只 Travelpro 最大的托运行李箱（约 416 美元一只，打折时更便宜）。虽然是软壳设计，但保护层极厚、万向轮滑行极其丝滑顺畅，既经摔又能狂塞行李。Ben 笑称自己原本以为自己是只买日默瓦（Rimowa）的高端用户，没想到彻底被这套实惠耐造的箱子圈粉，David 调侃 Acquired 正在变成数码生活评测媒体《The Wirecutter》。 --- ### [03:58:09](https://www.youtube.com/watch?v=lCEB7xHer5U&t=14289s) David 的 Carve Out：《The Glue Guys》播客与掌机“育儿奇迹” David 分享了他的推荐。第一个是播客《The Glue Guys》，由红杉资本（Sequoia）合伙人 Ravi Gupta 与前 NBA 球员 Shane Battier（肖恩·巴蒂尔）、前 NFL 知名四分卫 Alex Smith 共同主持。该播客一半是三人闲聊，一半带嘉宾，探讨竞技体育与商业。David 特别提到他们在节目中采访体育作家 Wright Thompson 的那一期：那期节目仅有约 5000 次播放量，但三位主持人的心态极具启发性——他们丝毫不觉得没人听是丢脸的事，反而“为那些还没听过这期神仙内容的人感到遗憾”。Wright Thompson 说话带有一口类似知名风投家 Bill Gurley 的独特口音，全身心致力于研究顶尖运动员与教练的心态与心理学，内容极其精彩。 紧接着，David 带来了在《Google Part I》中挖下的掌机坑后续。当时他在任天堂 Switch 2 与 Valve 的 Steam Deck 之间纠结，最后出于“女儿还太小无法一起玩，买来纯粹是自己玩”的考量入手了 Steam Deck。然而最近发生了一件不可思议的亲子奇迹：在度假期间，他在 Steam Deck 上玩复古像素风 JRPG 独立神作《星之海》（Sea of Stars，致敬《超时空之轮》风格），快满 4 岁的女儿凑过来好奇围观，并要求尝试操控。 David 称这是他人生中最美妙的育儿时刻之一：在几天的“监督学习（Supervised Learning）”式教学下，甚至还不识字的女儿迅速掌握了摇杆和按键的用法，从看着爸爸玩演变到了女儿主玩、爸爸在旁边引导提示剧情目标的阶段。David 兴奋地表示，借着玩游戏还能名正言顺地算作“高质量带娃陪护”，Steam Deck 意外成为了带女儿入坑游戏的最佳启蒙工具，他正打算在女儿接下来的生日给她买一台 Switch 一起联机。 --- ### [04:02:34](https://www.youtube.com/watch?v=lCEB7xHer5U&t=14554s) 赞助商致谢与核心参考书目 在节目的最后致谢阶段，两位主持人首先感谢了本季度的核心商业合作伙伴： - **J.P. Morgan Payments**：提供值得信赖、高可靠性的全球企业支付基础设施。 - **Sentry**：软件应用监控与错误排查平台，助力团队在用户发火前迅速修复 Bug。 - **WorkOS**：助力 SaaS 产品快速迈向企业级功能，只需几行代码即可集成 SSO 单点登录。 - **Shopify**：全球顶尖的电商平台，赋能独立创业者与跨国巨头在线销售。 随后，他们向为本期长达 4 小时的 Google AI 史诗节目提供关键史料的作家与书籍表达了深切感谢： - **Steven Levy**（《连线》杂志特约编辑）及其经典巨著《In the Plex》（中文译名《这就是谷歌》），该书是 Acquired 制作整个 Google 三部曲的基石。 - **Parmy Olson**（彭博社专栏作家）的新书《Supremacy: AI, ChatGPT, and the Race that Will Change the World》，详实记录了 DeepMind 与 OpenAI 的世纪竞逐，是本期节目的核心一手来源。 - **Cade Metz**（《纽约时报》记者）的《Genius Makers》（中文译名《深度学习浅史》）。 同时，他们特别向提供深度访谈与背景支持的 Google 内部及前员工团队致谢，包括 Max Ross、Liz Reid、Josh Woodward、Greg 等核心关键研发与产品负责人。 ### [04:03:51](https://www.youtube.com/watch?v=lCEB7xHer5U&t=14631s) 致谢：访谈对象、研究伙伴与技术先驱 在本期播客的尾声，两位主持人向为这期内容乃至整个 Google 三部曲提供无私帮助的众多科技领袖、研究员及行业专家表达了由衷的敬意与感谢。名单中汇聚了塑造现代人工智能与科技版图的核心人物，包括 Greg Corrado、Sebastian Thrun、Anna Patterson、Bret Taylor、Clay Bavor、Demis Hassabis、Thomas Kurian 以及 Sundar Pichai。主持人特别向 Nick Fox 致谢——他是唯一一位在整个 Google 系列全部三期节目的调研中都接受了访谈的嘉宾，成功达成了贯穿始终的“帽子戏法”（hat trick）。 此外，主持团队还致谢了在产业洞察与深度调研中提供巨大支持的思想伙伴。其中包括撰写了 Alphabet 深度研究报告的 Worldly Partners 的 Arvin Navaratnam；TPU 早期核心成员、现任推理芯片公司 Groq 创始人兼 CEO 的 Jonathan Ross；来自 Waymo 的 Dmitri Dolgov 与 Suzanne Philion；Atreides Management 的 Gavin Baker；科技媒体界元老、《Spyglass》作者 M.G. Siegler；以及 DeepMind 负责打造核心 Gemini 模型的 Koray Kavukcuoglu。同时，主持人们还感谢了曾掌舵 YouTube 产品、现任科技高管的 Shishir Mehrotra；Phaidra 创始人兼前 DeepMind 团队成员 Jim Gao；Benchmark 合伙人 Chetan Puttagunta；知名对谈播客主持人 Dwarkesh Patel；以及在 AI 数据中心经济模型分析上给予关键启发的老朋友——Oakcliff Capital 的 Brian Lawrence。 谈到好友 Ben Eidelson 时，主持人不吝赞美之词，热情推荐了 Ben 共同主持的新播客《Step Change》，尤其是其关于“数据中心历史”的最新单集。主持人打趣道，《Step Change》刚播到第三期，播放量就已经突破了三四万次，其上升势头甚至远胜当年的早期 Acquired；如果你热爱 Acquired 这种抽丝剥茧的商业与技术剖析，就绝不该错过这档节目。 - **核心受访与致谢名单**： - 科技领袖与学者：Greg Corrado、Sebastian Thrun、Anna Patterson、Bret Taylor、Clay Bavor、Demis Hassabis、Thomas Kurian、Sundar Pichai - 全勤受访嘉宾：Nick Fox（唯一一位贯穿 Google 三部曲全系列调研的嘉宾） - 硬件与前沿技术先锋：Jonathan Ross（Groq 创始人兼 CEO、TPU 初创团队成员）、Jim Gao（Phaidra CEO、前 DeepMind 成员）、Koray Kavukcuoglu（DeepMind Gemini 核心团队） - 自动驾驶领导者：Dmitri Dolgov 与 Suzanne Philion（Waymo） - 投资与行业分析专家：Arvin Navaratnam（Worldly Partners）、Gavin Baker（Atreides Management）、M.G. Siegler（Spyglass / 原 TechCrunch）、Chetan Puttagunta（Benchmark）、Dwarkesh Patel、Brian Lawrence（Oakcliff Capital） - 播客与思想伙伴：Ben Eidelson（《Step Change》播客主理人，近期详解了数据中心演变史） ### [04:05:36](https://www.youtube.com/watch?v=lCEB7xHer5U&t=14736s) 往期回顾与十周年特别活动预告 在节目收束之际，主持人为意犹未尽的听众梳理了扩展收听路径：如果想建立对科技巨头竞争壁垒的完整认知，可以回顾 Acquired 早先推出的 Google 创业早期史、聚焦 2010 年代业务转型的 Alphabet 单集，以及经典的微软（Microsoft）与英伟达（NVIDIA）全套商业编年史。此外，旗下深度专访栏目 ACQ2 刚刚上线了与 Shopify 创始人兼 CEO Tobi Lütke 的精彩对谈，十分值得同步收听。 最后，主持人们向全球社区发出了重磅预告：为庆祝 Acquired 创办十周年，节目组将重现早期的经典传统，于太平洋时间 10 月 20 日下午 4:00 举办面向所有听众与 LP（Limited Partners）的公开 Zoom 在线交流会。听众也可以随时加入 `acquired.fm/slack` 参与更热烈的实时讨论。伴随着熟悉的标志性片尾曲与经典追问——"Who got the truth? Is it you? Now, huh?"，这期长达四个多小时的 Google AI 史诗级复盘正式画上句号。 - **推荐收听图谱**： - Google 系列前作：早期发展史专集与 Alphabet 2010 年代专题 - 巨头算力与平台图景：微软（Microsoft）系列、英伟达（NVIDIA）三部曲 - ACQ2 深度访谈：对话 Shopify 创始人兼 CEO Tobi Lütke - **十周年庆典活动**： - 时间：太平洋时间 10 月 20 日下午 4:00 - 形式：面向全体听众与社群的公开 Zoom LP 在线电话会（详情见 Show Notes） - 交流社区：`acquired.fm/slack` ## Notable quotes > [06:19](https://www.youtube.com/watch?v=lCEB7xHer5U&t=379s) 这就像在计算机时代的黎明，全世界懂写代码的人总共就那么几十个，而像 IBM 这样的一家公司，把其中几乎所有人全部雇下了。 — Ben Gilbert > [08:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=500s) 人工智能将是 Google 的终极形态。如果我们打造出真正的终极搜索引擎，它将能够理解互联网上的全部信息，精准洞悉你的真实所求，并直接奉上最完美的答案。这显然就是人工智能。我们目前离这个目标还差得很远，但我们正在一步步逼近，而这正是我们在 Google 所做的一切工作的本质。 — Larry Page > [10:20](https://www.youtube.com/watch?v=lCEB7xHer5U&t=620s) 数据的压缩，在技术本质上等同于对数据的理解。 — George Harik > [12:11](https://www.youtube.com/watch?v=lCEB7xHer5U&t=731s) 桑杰觉得这是个好主意，而这世界上没有人比桑杰更聪明。既然如此，诺姆和我为什么要听信你的建议，认为这是个坏主意呢？ — George Harik > [42:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=2552s) 我们在 2,000 台计算机、16,000 个核心上实现了无监督学习。模型试图从高阶表征中重构视频帧，训练一段时间后，网络最高层的一个神经元会对‘猫的图像’产生强烈的放电激活反应。它从未被告知什么是猫，但因为在训练数据中见过足够多正对镜头的猫脸，这个神经元就专门针对猫点亮，其他内容则不反应。 — Jeff Dean > [01:02:50](https://www.youtube.com/watch?v=lCEB7xHer5U&t=3770s) 解决智能，然后用它解决其他一切 — DeepMind > [01:11:32](https://www.youtube.com/watch?v=lCEB7xHer5U&t=4292s) 如果地球上出问题的那件事恰恰就是 AI 呢？那你跑到火星去也无济于事。因为如果我们能抵达火星，AI 顺着我们的通信网络系统或者飞行器轻而易举就能追过去。 — Demis Hassabis > [01:44:01](https://www.youtube.com/watch?v=lCEB7xHer5U&t=6241s) 我们要么再造一个完整的 Google 数据中心机群，要么就得自研一种专属芯片。 — Jeff Dean > [02:31:03](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9063s) 我想让大家知道，我们让 Google 跳起舞来了（I want people to know that we made Google dance） — Satya Nadella > [02:40:18](https://www.youtube.com/watch?v=lCEB7xHer5U&t=9618s) 谷歌内部绝不能再有两个独立的 AI 团队。 — Sundar Pichai > [02:52:12](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10332s) 为什么？到底有什么技术原因导致这件事情不可能实现？ — Larry Page > [02:52:12](https://www.youtube.com/watch?v=lCEB7xHer5U&t=10332s) 我想通了，并没有不可克服的技术原因，我只是在害怕。 — Sebastian Thrun > [03:51:43](https://www.youtube.com/watch?v=lCEB7xHer5U&t=13903s) 宁可破产，也绝不在 AI 竞争中落败 — Larry Page / Sergey Brin ## Mentioned 7 Powers, AdSense, AdWords, Alan Eustace, Alex Krizhevsky, AlexNet, AlphaGo, Alphabet, Andrew Ng, Andrej Karpathy, Anthropic, Apple, Astro Teller, Attention Is All You Need, AWS, Azure, Baidu, Bard, Ben Gilbert, Ben Gomes, Ben Thompson, Broadcom, Bullfrog Productions, Cade Metz, Carnegie Mellon University, Character.ai, Chris Cox, Clayton Christensen, Claude, CUDA, DARPA Grand Challenge, Dario Amodei, David Rosenthal, DeepMind, Demis Hassabis, DistBelief, DNN Research, Elon Musk, Eric Schmidt, FAIR, Founders Fund, Franz Josef Och, Gavin Baker, Gemini, Geoffrey Hinton, George Harik, GitHub Copilot, Google, Google Brain, Google Cloud Platform (GCP), Google DeepMind, Google One, Google X, GPT-1, GPT-2, GPT-3, GPT-4, ChatGPT, Greg Brockman, Greg Corrado, Groq, Hamilton Helmer, Ilya Sutskever, ImageNet, Inflection AI, Instagram, Jakob Uszkoreit, Jeff Dean, Jensen Huang, John Giannandrea, Jonathan Ross, J.P. Morgan Payments, Kubernetes, LaMDA, Larry Page, Lyft, Mark Zuckerberg, Meena, Meta, Microsoft, Mustafa Suleyman, Noam Shazeer, NotebookLM, NVIDIA, OpenAI, Oriol Vinyals, PaLM, Parmy Olson, Perplexity, Peter Thiel, PHIL, Reid Hoffman, Rich Sutton, Sam Altman, Sanjay Ghemawat, Satya Nadella, Saudi Aramco, Sebastian Thrun, Sentry, Sergey Brin, Shane Legg, Shopify, SpaceX, StarCraft II, Steven Levy, Sundar Pichai, Tesla, TensorFlow, Thomas Kurian, TikTok, Tobi Lütke, TPU, TSMC, Uber, Veo, Waymo, WorkOS, xAI, Yann LeCun, YouTube ## Video description Google faces the greatest innovator's dilemma in history. They invented the Transformer — the breakthrough technology powering every modern AI system from ChatGPT to Claude (and, of course, Gemini). They employed nearly all the top AI talent: Ilya Sutskever, Geoff Hinton, Demis Hassabis, Dario Amodei — more or less everyone who leads modern AI worked at Google circa 2014. They built the best dedicated AI infrastructure (TPUs!) and deployed AI at massive scale years before anyone else. And yet... the launch of ChatGPT in November 2022 caught them completely flat-footed. How on earth did the greatest business in history wind up playing catch-up to a nonprofit-turned-startup? Today we tell the complete story of Google's 20+ year AI journey: from their first tiny language model in 2001 through the creation Google Brain, the birth of the transformer, the talent exodus to OpenAI (sparked by Elon Musk's fury over Google’s DeepMind acquisition), and their current all-hands-on-deck response with Gemini. And oh yeah — a little business called Waymo that went from crazy moonshot idea to doing more rides than Lyft in San Francisco, potentially building another Google-sized business within Google. This is the story of how the world's greatest business faces its greatest test: can they disrupt themselves without losing their $140B annual profit-generating machine in Search? *Links:* - Geoff Hinton’s 2007 Tech Talk at Google: https://youtu.be/AyzOUbkUf3M?si=sLQJsEwq-JEOQmIN - Our recent ACQ2 episode with Tobi Lutke: https://www.acquired.fm/episodes/how-to-live-in-everyone-elses-future-with-shopify-ceo-tobi-lutke - Worldly Partners’ Multi-Decade Alphabet Study: https://worldlypartners.com/businesshistory - In the Plex: https://www.amazon.com/Plex-Google-Thinks-Works-Shapes/dp/1416596585 - Supremecy: https://www.amazon.com/Supremacy-ChatGPT-Race-Change-World/dp/1250337747 - Genius Makers: https://www.amazon.com/Genius-Makers-Mavericks-Brought-Facebook/dp/1524742678 - All episode sources: https://www.acquired.fm/episodes/google-the-ai-company#sources *Carve Outs:* - F1: The Movie: https://www.imdb.com/title/tt16311594/ - Travelpro suitcases: https://travelpro.com - Glue Guys Podcast: https://open.spotify.com/show/18n6FcVMR5J0hSeboeETPc?si=b876358f752a474d - Sea of Stars: https://seaofstarsgame.co - Stepchange Podcast: https://open.spotify.com/show/3RHJ129caYkxh2FliqtaBP?si=2bef3a854e084a39 *More Acquired:* - Get email updates https://www.acquired.fm/email and vote on future episodes! - Join the Slack http://acquired.fm/slack - Check out the latest swag in the ACQ Merch Store https://www.acquired.fm/store! 0:00 The Innovator's Dilemma: Google's AI Challenge 4:36 From PageRank to language models: Google's Early AI Foundations 17:21 Parallelizing Language: Jeff Dean's Google Translate Revolution 24:05 Sebastian Thrun and Geoff Hinton: Laying Google X Foundations 35:10 Unsupervised Learning: Google Brain's Cat Paper Breakthrough 47:05 From AlexNet to DeepMind: The GPU AI Revolution 1:00:02 The DeepMind Acquisition: AlphaGo and AI's New Frontier 1:40:22 Building TPUs: Google's Hardware Bet on AI 1:51:25 "Attention Is All You Need": The Transformer's Impact 2:04:01 Elon's Exit, Microsoft's Entry: OpenAI's New Path 2:15:43 The Chat GPT Shockwave: Google's Code Red Response 2:37:32 Unifying AI: Gemini and the Google DeepMind Merger 2:46:58 From DARPA Challenge to Robotaxis: Waymo's Long Road 3:03:44 Waymo's Billion-Dollar Potential and Google's Financial Power 3:20:26 Google Cloud's Strategic Role in the AI Ecosystem 3:32:31 Bull and Bear Cases on the Future of Google 3:53:10 Navigating the Innovator's Dilemma: Google's AI Strategy 3:54:34 NFL Summit Carveouts, and Outro _Note: Acquired hosts and guests may hold assets discussed in this episode. This podcast is not investment advice, and is intended for informational and entertainment purposes only. You should do your own research and make your own independent decisions when considering any financial transactions._ [Source](https://www.youtube.com/watch?v=lCEB7xHer5U)

[6:19] Ben Gilbert: 这就像在计算机时代的黎明，全世界懂写代码的人总共就那么几十个，而像 IBM 这样的一家公司，把其中几乎所有人全部雇下了。 [8:20] Larry Page: 人工智能将是 Google 的终极形态。如果我们打造出真正的终极搜索引擎，它将能够理解互联网上的全部信息，精准洞悉你的真实所求，并直接奉上最完美的答案。这显然就是人工智能。我们目前离这个目标还差得很远，但我们正在一步步逼近，而这正是我们在 Google 所做的一切工作的本质。 [10:20] George Harik: 数据的压缩，在技术本质上等同于对数据的理解。 [12:11] George Harik: 桑杰觉得这是个好主意，而这世界上没有人比桑杰更聪明。既然如此，诺姆和我为什么要听信你的建议，认为这是个坏主意呢？ [42:32] Jeff Dean: 我们在 2,000 台计算机、16,000 个核心上实现了无监督学习。模型试图从高阶表征中重构视频帧，训练一段时间后，网络最高层的一个神经元会对‘猫的图像’产生强烈的放电激活反应。它从未被告知什么是猫，但因为在训练数据中见过足够多正对镜头的猫脸，这个神经元就专门针对猫点亮，其他内容则不反应。 [62:50] DeepMind: 解决智能，然后用它解决其他一切 [71:32] Demis Hassabis: 如果地球上出问题的那件事恰恰就是 AI 呢？那你跑到火星去也无济于事。因为如果我们能抵达火星，AI 顺着我们的通信网络系统或者飞行器轻而易举就能追过去。 [104:01] Jeff Dean: 我们要么再造一个完整的 Google 数据中心机群，要么就得自研一种专属芯片。 [151:03] Satya Nadella: 我想让大家知道，我们让 Google 跳起舞来了（I want people to know that we made Google dance） [160:18] Sundar Pichai: 谷歌内部绝不能再有两个独立的 AI 团队。 [172:12] Larry Page: 为什么？到底有什么技术原因导致这件事情不可能实现？ [172:12] Sebastian Thrun: 我想通了，并没有不可克服的技术原因，我只是在害怕。 [231:43] Larry Page / Sergey Brin: 宁可破产，也绝不在 AI 竞争中落败

[8:20] Larry Page: Artificial intelligence would be the ultimate version of Google. If we had the ultimate search engine, it would understand everything on the web. It would understand exactly what you wanted and it would give you the right thing... We're nowhere near doing that now. However, we can get incrementally closer and that is basically what we work on here. [62:11] Demis Hassabis: solve intelligence and use it to solve everything else. [71:27] Demis Hassabis: What if AI was the thing that went wrong here? [71:27] Elon Musk: that's probably true. [75:02] Mark Zuckerberg: No, you'll be a part of Facebook. [79:12] Larry Page: Wait, what are you watching? What company is this? [82:12] Larry Page: really go big [82:12] Alan Eustace: I jumped out of the atmosphere in a scuba suit. I think we'll be fine. [93:20] advance digital intelligence in the way that is most likely to benefit humanity as a whole unconstrained by a need to generate financial return. [100:02] Jeff Dean: We need another Google. [145:33] Sam Altman: We should make a chatbot. That seems like a natural interface for this. Can someone just make a chat? [152:19] Satya Nadella: I want people to know that we made Google dance. [152:19] Sundar Pichai: code red [260:13] Sundar Pichai: I don't care that you had different charters before [231:43] Larry Page and Sergey Brin: rather go bankrupt than lose at AI.

[0:00] I went and looked at a studio. Well, a [0:02] little office that I was going to turn [0:03] into a studio nearby, but it was not [0:06] good at all. It had drop ceilings, so I [0:08] could hear the guy in the office next to [0:09] me. You would be able to hear him [0:10] talking on episodes. [0:13] >> Third co-host. [0:14] >> Third co-host. [0:14] >> Is it Howard? [0:16] >> No, it was like a lawyer. It seemed to [0:17] be like talking through some horrible [0:19] problem that I didn't want to listen to, [0:20] but I could hear every word. [0:22] >> Does he want millions of people [0:24] listening to this conversation? [0:26] >> Right. [0:28] >> All right. [0:28] >> All right. Let's do a podcast. Let's do [0:30] a podcast. [0:34] [Music] [0:37] >> Who got the truth? Now, [0:40] is it you? Is it you? Is it you? Sit me [0:43] down. Say it straight. Another story on [0:46] the way. Got the truth. [0:49] >> Welcome to the fall 2025 season of [0:51] Acquired, the podcast about great [0:53] companies and the stories and playbooks [0:55] behind them. I'm Ben Gilbert. [0:57] >> I'm David Rosenthal. and we are your [0:59] hosts. Here's a dilemma. Imagine you [1:02] have a profitable business. You make [1:04] giant margins on every single unit you [1:07] sell and the market you compete in is [1:10] also giant. One of the largest in the [1:12] world, you might say. But then on top of [1:15] that, lucky for you, you also are a [1:18] monopoly in that giant market with 90% [1:20] share and a lot of lock in. [1:23] >> And when you say monopoly, monopoly as [1:25] defined by the US government. That is [1:27] correct. But then imagine this. In your [1:31] research lab, your brilliant scientists [1:33] come up with an invention. This [1:35] particular invention when combined with [1:37] a whole bunch of your old inventions by [1:39] all your other brilliant scientists [1:42] turns out to create the product that is [1:43] much better for most purposes than your [1:46] current product. So you launched the new [1:49] product based on this new invention. [1:50] Right. [1:51] >> Right. I mean, especially because out of [1:53] pure benevolence, your scientists had [1:56] published research papers about how [1:58] awesome the new invention is and lots of [2:00] the inventions before also. So, now [2:03] there's new startup competitors quickly [2:05] commercializing that invention. So, of [2:07] course, David, you change your whole [2:09] product to be based on the new thing, [2:10] right? [2:11] >> Uh, this sounds like a movie. [2:13] >> Yes. But here is the problem. You [2:16] haven't figured out how to make this new [2:19] incredible product anywhere near as [2:21] profitable as your old giant cash [2:24] printing business. So maybe you [2:27] shouldn't launch that new product. [2:29] David, this sounds like quite the uh [2:30] dilemma to me. Of course listeners this [2:34] is Google today and in perhaps the most [2:36] classic textbook case of the innovators [2:38] dilemma ever the entire AI revolution [2:42] that we are in right now is predicated [2:44] by the invention of the transformer out [2:47] of the Google brain team in 2017. So [2:50] think open AI and chat GBT anthropic [2:53] NVIDIA hitting all-time highs all the [2:55] craziness right now depends on that one [2:58] research paper published by Google in [3:00] 2017. And consider this. Not only did [3:03] Google have the densest concentration of [3:05] AI talent in the world 10 years ago that [3:07] led to this breakthrough, but today they [3:10] have just about the best collection of [3:12] assets that you could possibly ask for. [3:14] They've got a top tier AI model with [3:16] Gemini. They don't rely on some public [3:19] cloud to host their model. They have [3:21] their own in Google Cloud that now does [3:23] $50 billion in revenue. That is real [3:26] scale. They're a chip company with their [3:29] tensor processing units or TPUs, which [3:31] is the only real scale deployment of AI [3:34] chips in the world besides Nvidia GPUs. [3:37] Maybe AMD maybe, but these are [3:39] definitely the top two. Somebody put it [3:41] to me in research that if you don't have [3:45] a foundational frontier model or you [3:50] don't have an AI chip, you might just be [3:54] a commodity in the AI market. And Google [3:56] is the only company that has both. [3:58] >> Google still has a crazy bench of [3:59] talent. And despite ChatGpt becoming [4:01] kind of the Kleenex of the era, Google [4:03] does still own the textbox, the single [4:06] one that is the front door to the [4:09] internet for the vast majority of people [4:11] anytime anyone has intent to do anything [4:13] online. But the question remains, what [4:16] should Google do strategically? Should [4:19] they risk it all and lean into their [4:20] birthright to win in artificial [4:22] intelligence? Or will protecting their [4:25] gobs of profits from search hamstring [4:28] them as the AI wave passes them by? But [4:31] perhaps first we must answer the [4:33] question, how did Google get here? David [4:36] Rosenthal. So listeners, today we tell [4:39] the story of Google, the AI company. [4:42] >> Woo. [4:43] >> You like that, David? Was that good? [4:44] >> I love it. Did you hire like a Hollywood [4:47] script writing consultant without [4:48] telling me? [4:49] >> I wrote that 100% myself with no AI. [4:51] Thank you very much. [4:52] >> No AI. [4:54] >> Well, listeners, if you want to know [4:55] every time an episode drops, vote on [4:57] future episode topics or get access to [5:00] corrections from past episodes, check [5:01] out our email list. That's [5:03] acquired.fm/e. [5:05] Come talk about this episode with the [5:07] entire acquired community in Slack after [5:09] you listen. That's acquired.fm/slack. [5:12] Speaking of the acquired community, we [5:14] have an anniversary celebration coming [5:16] up. We do 10 years of the show. We're [5:20] going to do an open Zoom call with [5:22] everyone to celebrate. Kind of like how [5:24] we used to do our LP calls back in the [5:27] day with LPS. And we are going to do [5:29] that on October 20th, 2025 at 4:00 p.m. [5:34] Pacific time. Check out the show notes [5:36] for more details. [5:37] >> If you want more acquired, check out our [5:39] interview show, ACQ2. Our last interview [5:41] was super fun. We uh sat down with Toby [5:43] Lutka, the founder and CEO of Shopify, [5:46] about how AI has changed his life and [5:49] where he thinks it will go from here. [5:50] So, search ACQ2 in any podcast player. [5:53] And before we dive in, we want to [5:54] briefly thank our presenting partner JP [5:56] Morgan Payments. [5:57] >> Yes, just like how we say every company [5:59] has a story, every company's story is [6:01] powered by payments. And JP Morgan [6:03] Payments is a part of so many of their [6:04] journeys from seed to IPO and beyond. [6:07] >> So, with that, this show is not [6:09] investment advice. David and I may have [6:10] investments in the companies we discuss [6:11] and this show is forformational and [6:13] entertainment purposes only. David, [6:17] Google, the AI company. [6:19] >> So Ben, as you were alluding to in that [6:21] fantastic intro, really, you're really [6:22] up in the game here. [6:25] If we rewind 10 years ago from today, [6:29] before the Transformer paper comes out, [6:32] all of the following people, as we've [6:36] talked about before, were Google [6:38] employees. Ilia Sidskever, founding [6:41] chief scientist of OpenAI, who along [6:43] with Jeff Hinton and Alex Koschesky had [6:46] done the seinal AI work on Alexnet and [6:50] just published that a few years before. [6:52] All three of them were Google employees, [6:54] as was Daario Amade, the founder of [6:59] Anthropic, [7:00] Andre Karpathy, chief scientist at Tesla [7:03] until recently, Andrew Ing, Sebastian [7:05] Thrron, Nam Shazir, all the deep mind [7:08] folks. Demisabis, Shane Le, Mustafa [7:10] Sullean, Mustafa now, in addition to in [7:13] the past having been a founder of [7:14] DeepMind runs AI at Microsoft. [7:16] Basically, [7:18] every single person of note in AI worked [7:22] at Google with the one exception of Yan [7:24] Lun who worked at Facebook. [7:26] >> Yeah, it's pretty difficult to trace a [7:29] big AI lab now back and not find Google [7:33] in its origin story. Yeah, I mean the [7:35] analogy here is it's almost as if at the [7:38] dawn of the computer era itself, a [7:41] single company like say IBM had hired [7:44] every single person who knows how to [7:45] code. So it' be like you know if anybody [7:47] else wants to write a computer program [7:49] oh sorry you can't do that. Anybody who [7:50] knows how to program works at IBM. This [7:52] is how it was with AI and Google in the [7:54] mid2010s. But learning how to program a [7:57] computer wasn't so hard that people out [7:59] there couldn't learn how to do it. [8:01] learning how to be an AI researcher [8:03] significantly more difficult, [8:04] >> right? It was the stuff of very specific [8:06] PhD programs with a very limited set of [8:10] advisers and a lot of infighting in the [8:13] field of where the direction of the [8:14] field was going, what was legitimate [8:16] versus what was crazy heretical [8:18] religious stuff. [8:20] >> Yeah. So then yes, the question is how [8:22] do we get to this point? Well, it goes [8:24] back to the start of the company. I [8:25] mean, Larry Page always thought of [8:27] Google as an artificial intelligence [8:29] company. And in fact, Larry Page's dad [8:32] was a computer science professor and had [8:34] done his PhD at the University of [8:36] Michigan in machine learning and [8:37] artificial intelligence, which was not a [8:39] popular field in computer science back [8:41] then. [8:42] >> Yeah. In fact, a lot of people thought [8:43] specializing in AI was a waste of time [8:48] because so many of the big theories from [8:50] 30 years prior to that had been kind of [8:53] disproven at that point, or at least [8:55] people thought they were disproven. And [8:57] so it was frankly contrarian for Larry's [8:59] dad to spend his life and career and [9:03] research work in AI. [9:04] >> And that rubbed off on Larry. I mean, if [9:06] you squint, page rank, the page rank [9:08] algorithm that Google was founded upon [9:11] is a statistical method. You could [9:14] classify it as part of AI within [9:17] computer science. And Larry, of course, [9:19] was always dreaming much much bigger. [9:20] here. I mean, there's the quote that [9:21] we've said before on this show in the [9:23] year 2000, 2 years after Google's [9:26] founding when Larry says artificial [9:28] intelligence would be the ultimate [9:29] version of Google. If we had the [9:31] ultimate search engine, it would [9:32] understand everything on the web. It [9:34] would understand exactly what you wanted [9:35] and it would give you the right thing. [9:37] That's obviously artificial [9:39] intelligence. We're nowhere near doing [9:41] that now. However, we can get [9:43] incrementally closer and that is [9:45] basically what we work on here. It's [9:47] always been an AI company. [9:48] >> Yep. And that was in 2000. [9:50] Well, one day in either late 2000 or [9:54] early 2001, the timelines are a bit hazy [9:56] here, a Google engineer named Gor Heric [10:00] is talking over lunch with Ben Gomes, [10:03] famous Google engineer who I think would [10:05] go on to lead search and a relatively [10:08] new engineering hire named Gnome Shazir. [10:12] Now, Gor was one of Google's first 10 [10:15] employees, incredible engineer. And just [10:17] like Larry Paige's dad, he had a PhD in [10:20] machine learning from the University of [10:22] Michigan. And even when George went [10:24] there, it was still a relatively rare [10:26] contrarian subfield within computer [10:28] science. So, the three of them are [10:30] having lunch and George says [10:32] off-handedly to the group that he has a [10:34] theory from his time as a PhD student [10:37] that compressing data is actually [10:41] technically equivalent to understanding [10:45] it. And the thought process is if you [10:47] can take a given piece of information [10:49] and make it smaller, store it away and [10:52] then later reinstantiate it in its [10:55] original form. The only way that you [10:57] could possibly do that is if whatever [11:00] force is acting on the data actually [11:02] understands what it means because you're [11:04] losing information going down to [11:06] something smaller and then recreating [11:08] the original thing. It's like you're a [11:10] kid in school. You learn something in [11:12] school. You read a long textbook. You [11:13] store the information in your memory. [11:15] Then you take a test to see if you [11:16] really understood the material. And if [11:18] you can recreate the concepts, then you [11:20] really understand it. [11:21] >> Which kind of foreshadows big LLMs today [11:25] are like compressing the entire world's [11:27] knowledge into some number of terabytes [11:29] that's just like this smash down little [11:31] vector set. Little at least compared to [11:34] all the information in the world. But [11:36] it's kind of that idea, right? You can [11:38] store all the world's information in an [11:40] AI model in something that is like kind [11:42] of incomprehensible and hard to [11:43] understand. But then if you uncompress [11:46] it, you can kind of bring knowledge back [11:47] to its original form. [11:49] >> Yep. And these models demonstrate [11:52] understanding, right? [11:54] >> Do they? That's the question. That's the [11:56] question. They certainly mimic [11:57] understanding. [11:59] >> So this conversation is happening. You [12:00] know, this is 25 years ago. And Gnome, [12:03] the new hire, the, you know, young buck, [12:06] he sort of stops in his tracks and he's [12:07] like, "Wow, if that's true, that's [12:10] really profound." [12:11] >> Is this in one of Google's micro [12:12] kitchens? [12:13] >> This is in one of Google's micro [12:14] kitchens. They're having lunch. [12:16] >> Where did you find this, by the way? A [12:17] 25-year-old [12:18] >> uh this is in in the Plex. This is like [12:20] a small little passage in Steven Levy's [12:22] great book that's been a source for all [12:23] of our Google episodes, In the Plex. [12:25] There's a small little throwaway passage [12:27] in here about this because this book [12:29] came out before ChatgBT and AI and all [12:31] that. So Gnome kind of latches on to [12:34] George and keeps vibing over this idea [12:37] and over the next couple months the two [12:39] of them decide in the most googly [12:41] fashion possible that they are just [12:44] going to stop working on everything else [12:46] and they're going to go work on this [12:47] idea on language models and compressing [12:50] data and can they generate machine [12:52] understanding with data and if they can [12:54] do that that that would be good for [12:56] Google. I think this coincides with that [12:58] period in 2001 when Larry Pageige fired [13:01] all the managers in the engineering [13:02] organization and so everybody was just [13:05] doing whatever they wanted to do. [13:07] >> Funny. [13:08] >> So there's this great quote from Gor in [13:11] the book. A large number of people [13:14] thought it was a really bad thing for [13:17] Nome and I to spend our talents on, but [13:19] Sanjay Gamat Sanjay of course being Jeff [13:21] Dean's famous prolific coding partner [13:24] thought it was cool. So George would [13:28] posit the following argument to any [13:30] doubters that they came across. Sanjay [13:32] thinks it's a good idea and no one in [13:34] the world is as smart as Sanjay. So why [13:36] should Nome and I accept your view that [13:38] it's a bad idea? [13:41] >> It's like if you beat the best team in [13:43] football, are you the new best team in [13:45] football no matter what? [13:46] >> Yeah. So all of this ends up taking [13:48] Noman George deep down the rabbit hole [13:51] of probabilistic models for natural [13:54] language. Meaning for any given sequence [13:57] of words that appears on the internet, [14:00] what is the probability for another [14:02] specific sequence of words to follow? [14:05] This should sound pretty familiar for [14:06] anybody who knows about LLM's work [14:08] today. [14:08] >> Oh, kind of like a next word predictor. [14:10] >> Yeah. Or next token predictor if you [14:13] generalized it. [14:14] >> Yep. [14:14] >> So, the first thing that they do with [14:16] this work is they create the did you [14:19] mean spelling correction in Google [14:22] search. [14:22] >> Oh, that came out of this. that came out [14:24] of this. Gnome created this. [14:27] >> So this is huge for Google because [14:29] obviously it's a bad user experience [14:31] when you mistype a query and then need [14:33] to type another one. But it's attacks to [14:36] Google's infrastructure because every [14:38] time these mistyped queries are going [14:40] well, Google's infrastructure goes and [14:42] serves the results to that query that [14:44] are useless and immediately overwritten [14:46] with the new one, [14:47] >> right? And it's a really tightly scoped [14:49] problem where you can see like, oh wow, [14:51] 80% of the time that someone types in [14:53] god groomer. Oh, they actually mean dog [14:55] groomer and they retype it. And if it's [14:58] really high confidence, then you [15:00] actually just correct it without even [15:01] asking them and then ask them if they [15:03] want to opt out instead of opting in. [15:04] It's a great feature and it's sort of a [15:06] great first use case for this in a very [15:09] narrowly scoped domain. [15:11] >> Totally. So they get this win and they [15:13] keep working on it nomen and they end up [15:16] creating a fairly large I'm using large [15:20] in quotes here you know for the time [15:22] language model that they call [15:25] affectionately Phil the probabilistic [15:28] hierarchical inferential learner. [15:31] >> These AI researchers love creating their [15:34] uh backronyms. [15:35] >> They love their word puns. [15:37] >> Yeah. [15:37] >> Yep. [15:39] So fast forward to 2003 and Susan [15:42] Majiski and Jeff Dean are getting ready [15:44] to launch AdSense. They need a way to [15:48] understand the content of these third [15:50] party web pages, the publishers, in [15:53] order to run the Google ad corpus [15:55] against them. Well, Phil is the tool [15:58] that they use to do it. [16:00] >> Huh. I had no idea that language models [16:02] were involved in this. [16:03] >> Yeah. So Jeff Dean borrows Phil and [16:08] famously uses it to code up his [16:10] implementation of AdSense in a week [16:12] because he's Jeff Dean. And boom, [16:15] AdSense. This is billions of dollars of [16:18] new revenue to Google overnight because [16:20] it's the same corpus of ads that are [16:23] adwords that are search ads that they're [16:25] now serving on third party pages. They [16:27] just massively expanded the inventory [16:29] for the ads that they already have in [16:31] the system. Thanks to Phil. Thanks to [16:33] Phil. All right, this is a moment where [16:36] we got to stop and just give some Jeff [16:38] Dean facts. Jeff Dean is going to be the [16:40] throughine of this episode of Wait, how [16:42] did Google pull that off? How did Jeff [16:45] Dean just go home and over the weekend [16:47] rewrite some entire giant distributed [16:49] system and figure out all of Google's [16:50] problems? Back when Chuck Norris facts [16:53] were big, Jeff Dean facts became a thing [16:55] internally at Google. I just want to [16:56] give you some of my favorites. The speed [16:58] of light in a vacuum used to be about 35 [17:00] mph. Then Jeff Dean spent a weekend [17:03] optimizing physics. [17:06] >> So good. [17:07] >> Jeff Dean's pin is the last four digits [17:09] of pi. [17:13] >> Only Googlers would come up with these. [17:16] >> Yes. To Jeff Dean, NP means no [17:18] problemmo. [17:19] >> Oh yeah, I've seen that one before. I [17:21] think that one's my favorite. [17:22] >> Yes. [17:23] >> Oh, man. So, so good. Also a wonderful [17:26] human being who we spoke to in research [17:27] and was very, very helpful. Thank you, [17:29] Jeff. [17:30] >> Yes. So, language models definitely [17:33] work, definitely going to drive a lot of [17:36] value for Google, and they also fit [17:39] pretty beautifully into Google's mission [17:41] to organize the world's information and [17:43] make it universally accessible and [17:45] useful if you can understand the world's [17:47] information and compress it and then [17:49] recreate it. Yeah, that fits the [17:51] mission. I think I think that checks the [17:52] box. [17:53] >> Absolutely. So Phil gets so big that [17:56] apparently by the mid 2000s Phil is [17:59] using 15% of Google's entire data center [18:02] infrastructure and I assume a lot of [18:04] that is AdSense ad serving but also did [18:07] you mean and all the other stuff that [18:09] they start using it for within Google. [18:11] >> So uh early natural language systems [18:14] computationally expensive. [18:16] >> Yes. So okay now mid 2000s fast forward [18:18] to 2007 which is a very very big year [18:23] for the purposes of our story. Google [18:26] had just recently launched the Google [18:28] translate product. This is the era of [18:31] all the great great products coming out [18:32] of Google that we've talked about. You [18:34] know maps and Gmail and docs and all the [18:37] wonderful things that Chrome and Android [18:38] are going to come later. They had like a [18:40] 10-year run where they basically [18:42] launched everything you know of at [18:44] Google except for search truly in a [18:47] 10-year run. And then there were about [18:49] 10 years after that from 2013 on where [18:52] they basically didn't launch any new [18:54] products that you've heard about until [18:55] we get to Gemini, which is this [18:57] fascinating thing. But this 03 to 2013 [19:00] era was just so rich with hit after hit [19:04] after hit, [19:04] >> magical. And so one of those products [19:06] was Google Translate. you know, not the [19:08] same level of user base or perhaps [19:11] impact on the world as Gmail or maps or [19:13] whatnot, but still a magical magical [19:16] product. And the chief architect for [19:18] Google Translate was another incredible [19:21] machine learning PhD named Fron O. So [19:25] Fran had a background in natural [19:27] language processing and machine learning [19:29] and that was his PhD. He was German. He [19:31] got his PhD in Germany at the time. [19:34] DARPA, [19:35] >> the Defense Advanced Research Projects [19:37] Agency, division of the government, [19:39] >> had one of their famous challenges going [19:43] for machine translation. So Google and [19:46] France of course enters this and France [19:49] builds an even larger language model [19:54] that blows away the competition in this [19:56] year's version of the DARPA challenge. [19:58] This is either 2006 or 2007. gets a [20:02] astronomically high blue score for the [20:05] time. It's called the bilingual [20:06] evaluation understudy is the sort of [20:09] algorithmic benchmark for judging the [20:10] quality of translations at the time, [20:13] higher than anything else possible. So [20:16] Jeff Dean hears about this and the work [20:19] that France and the translate team have [20:20] done and it's like this is great. This [20:22] is amazing. Uh when are you guys going [20:24] to ship this in production? [20:25] >> Oh, I heard this story. [20:27] >> So Jeff and Nome talk about this on the [20:28] Door Cash podcast. Yes, [20:30] >> that episode is so so good. And Fron is [20:33] like, "No, no, no, no, Jeffy, you don't [20:35] understand. This is research. This isn't [20:38] for the product. We can't ship this [20:40] model that we built. This is a n g [20:44] language model." Grams are like number [20:46] of words in a cluster. And we've trained [20:48] it on a corpus of two trillion words [20:53] from the Google search index. This thing [20:55] is so large it takes it 12 hours to [20:58] translate a sentence. So the way the [21:00] DARPA challenge worked in this case was [21:03] you got a set of sentences on Monday and [21:06] then you had to submit your machine [21:08] translation of those set of sentences by [21:10] Friday. [21:11] >> Plenty of time for the servers to run. [21:13] >> Yeah. They were like, "Okay, so we have [21:14] whatever number of hours it is from [21:16] Monday to Friday. Let's use as much [21:18] compute as we can to translate these [21:20] couple sentences. [21:23] Hey, learn the rules of the game and use [21:24] them to your advantage. [21:26] >> Exactly. So Jeff Dean being the [21:28] engineering equivalent of Chuck Norris, [21:30] he's like, let me see your code. So Jeff [21:34] goes and parachutes in and works with [21:35] the translate team for a few months. And [21:38] he rearchitects the algorithm to run on [21:42] the words and the sentences in parallel [21:45] instead of sequentially. Because when [21:47] you're translating a set of sentences or [21:49] a set of words in a sentence, you don't [21:51] necessarily need to do it in order. You [21:54] can break up the problem into different [21:56] pieces, work on it independently. You [21:58] can parallelize it [21:59] >> and you won't get a perfect translation, [22:00] but you know, imagine you just translate [22:02] every single word. You can at least go [22:05] translate those all at the same time in [22:06] parallel, reassemble the sentence and [22:08] like mostly understand what the initial [22:10] meaning was. [22:11] >> Yeah. And as Jeff knows very well [22:13] because he and Sanjay basically built it [22:15] with Zhoza, Google's infrastructure is [22:18] extremely parallelizable, distributed. [22:21] You can break up workloads into little [22:23] chunks, send them all over the various [22:25] data centers that Google has, reassemble [22:27] the projects, return that to the user. [22:29] >> They are the single best company in the [22:31] world at parallelizing workloads across [22:33] CPUs across multiple data centers. [22:36] >> CPUs. We're still talking CPUs here. [22:38] >> Yep. And Jeff's work with the team gets [22:43] that average sentence translation time [22:45] down from 12 hours to 100 milliseconds. [22:50] And so then they ship it in Google [22:52] Translate. And it's amazing. [22:53] >> This sounds like a Jeff Dean fact. Well, [22:55] you know, it used to take 12 hours and [22:57] then Jeff Dean took a few months with [22:58] it. Now it's a 100 milliseconds. [23:00] >> Right. Right. Right. Right. Right. [23:01] Right. So this is the first large I'm [23:05] using large in quotes here language [23:07] model used in production in a product at [23:10] Google. They see how well this works [23:13] like hm maybe we could use this for [23:15] other things like predicting search [23:18] queries as you type. [23:21] That might be interesting, you know, and [23:23] of course the crown jewel of Google's [23:26] business. That also might be interesting [23:28] application for this. The ad quality [23:30] score for Adwords is literally the [23:34] predicted click-through rate on a given [23:38] set of ad copy. You can see how an LLM [23:41] that is really good at ingesting [23:43] information, understanding it, and [23:46] predicting things based on that might be [23:48] really useful for calculating ad quality [23:51] for Google. [23:52] >> Yep. Which is the direct translation to [23:54] Google's bottom line. [23:56] >> Indeed. Okay. So, obviously all of that [23:58] is great on the language model front. I [24:02] said 2007 was a big year. Also in 2007 [24:08] begins [24:09] the sort of momentous intersection of [24:12] several computer science professors [24:16] on the Google campus. So in April of [24:19] 2007, Larry Page hires Sebastian Thrun [24:24] from Stamford to come to Google and work [24:27] first part-time and then full-time on [24:31] machine learning applications. Sebastian [24:34] was the head of sale at Stanford, the [24:36] Stanford artificial intelligence [24:38] laboratory. Legendary AI laboratory that [24:42] was big in the sort of first wave of AI [24:45] back in the ' 60s7s when Larry's dad was [24:48] active in the field then actually shut [24:50] down for a while and then had been [24:52] restarted and re-energized here in the [24:54] early 2000s. And Sebastian was the [24:57] leader, the head of sale. [24:59] >> Funny story about Sebastian, the way [25:00] that he actually comes to Google. [25:02] Sebastian was kind enough to speak with [25:03] us to prep for this episode. I didn't [25:05] realize it was basically an aqua hire. [25:08] He and some I think it was grad students [25:10] were in the process of starting a [25:12] company had term sheets from Benchmark [25:14] and Sequoia. [25:15] >> Yes. [25:16] >> And Larry came over and said, "What if [25:18] we just acquire your company before it's [25:19] even started in the form of signing [25:21] bonuses?" [25:22] >> Yes. Probably a very good decision on [25:24] their part. So sale, this group within [25:28] the CS department at Stanford, not only [25:31] had some of the most incredible, most [25:34] accomplished professors and PhD AI [25:37] researchers in the world, they also had [25:40] this stream of Stanford undergrads that [25:42] would come through and work there as [25:44] researchers while they were working on [25:46] their CS degrees or symbolic system [25:48] degrees or, you know, whatever it was [25:50] that they were doing as Stanford [25:51] undergrads. One of those people was [25:54] Chris Cox who's the chief product [25:56] officer at Meta. Yeah, that was kind of [25:58] how he got his start in [26:01] >> all of this and AI and obviously [26:03] Facebook and Meta are going to come back [26:04] into the story here in a little bit. [26:06] >> Wow. [26:06] >> You really can't make this up. Another [26:08] undergrad who passed through sale while [26:11] Sebastian was there was a young freshman [26:14] and sophomore who would later drop out [26:17] of Stanford to start a company that went [26:20] through Y Combinator's very first batch [26:24] in summer 2005. [26:26] >> I'm on the edge of my seat. Who is this? [26:28] >> Any guesses? [26:29] >> Uh Dropbox, Reddit. I'm trying to think [26:32] who else was in the first batch. [26:34] >> Oh, no. No. But way more on the nose for [26:36] this episode. [26:37] The company was a failed local mobile [26:41] social network. [26:42] >> Oh, Sam Alman looped. [26:45] >> Sam Alman. [26:48] >> That's amazing. He was at sale at the [26:51] same time. [26:51] >> He was at sale. Yep. As an undergrad [26:54] researcher. [26:54] >> Wow. [26:56] >> Wild, right? We told you that it's a [26:58] very small set of people that are all [27:00] doing all of this. [27:01] >> Man, I miss those days. Sam presenting [27:03] at the WWDC with Steve Jobs on stage [27:06] with the double popped collar, right? [27:08] >> Different time in tech. [27:09] >> Yeah, the double popped collar. That was [27:11] amazing. That was a vibe. That was a [27:12] moment. Oh, man. All right. So, April [27:17] 2007, Sebastian comes over from sale [27:19] into Google, Sebastian Thread. And one [27:21] of the first things he does over the [27:23] next set of months is a project called [27:25] Ground Truth for Google Maps, [27:28] >> which is essentially Google Maps. [27:30] >> It is essentially Google Maps. Before [27:32] ground truth, Google Maps existed as a [27:34] product, but they had to get all the [27:36] mapping data from a company called Tel [27:38] Atlas. [27:39] >> And I think there were two. They were [27:40] sort of a duopoly. Navtech was the other [27:42] one. [27:42] >> Yeah. Navtech and Tel Atlas. [27:44] >> But it was this like kind of crappy [27:47] source of truth map data that everyone [27:49] used and you really couldn't do any [27:50] better than anyone else because you all [27:51] just use the same data. [27:52] >> Yep. It was not that good and it cost a [27:56] lot of money. Tell Atlas and Navtech [27:58] were multi-billion dollar companies. I [28:00] think maybe one or both of them were [28:02] public at some point then got acquired [28:04] but a lot of money lot of revenue. [28:06] >> Yep. And Sebastian's first thing was [28:08] street view, right? So he already had [28:10] the experience of orchestrating this [28:12] fleet of all these cars to drive around [28:13] and take pictures. [28:14] >> Yes. So then coming into Google, ground [28:17] truth is this sort of moonshot type [28:20] project to recreate all the tea atlas [28:24] data [28:24] >> mostly from their own photographs of [28:27] streets from street view. And they [28:29] incorporated some other data. There was [28:30] like census data they used. I think it [28:32] was 40 something data sources to bring [28:34] it all together. But ground truth was [28:35] this very ambitious effort to create new [28:38] maps from whole cloth. [28:40] >> Yep. And just like all of the AI and AI [28:43] enabled projects within Google that [28:45] we're talking about here works very very [28:47] well. [28:49] Huge win. [28:50] >> Well, especially when you hire a [28:51] thousand people in India to help you uh [28:54] sift through all the discrepancies in [28:55] the data and actually handdraw all the [28:57] maps. Yes, we are not yet in an era of a [29:00] whole lot of AI automation. So on the [29:02] back of this win with ground truth, [29:04] Sebastian starts lobbying to Larry and [29:06] Sergey. Hey, we should do this a lot. We [29:08] should bring in AI professors, [29:11] academics, I know all these people into [29:13] Google part-time. They don't have to be [29:15] full-time employees. Let them keep their [29:17] posts in academia, but come here and [29:19] work with us on projects for our [29:22] products. They'll love it. They get to [29:24] see their work used by millions and [29:25] millions of people. We'll pay them. [29:26] They'll make a lot of money. They'll get [29:28] Google stock and they get to stay [29:30] professors at their academic [29:32] institutions. [29:33] >> Win-winwin. [29:34] >> Win-winwin. So, as you would expect, [29:36] Larry and Sergey are like, "Yeah, yeah, [29:37] yeah, that's a good idea. Let's do that. [29:39] More of that." So, in December of 2007, [29:42] Sebastian brings in a relatively little [29:46] known machine learning professor from [29:50] the University of Toronto named Jeff [29:53] Hinton to the Google campus to come and [29:55] give a tech talk, not yet hiring him, [29:57] but come give a tech talk to, you know, [29:59] all the folks at Google and talk about [30:02] some of the new work, Jeff, that you and [30:05] your PhD and postoc students there at [30:07] the University of Toronto are doing on [30:10] blazing new paths with neural networks [30:13] >> and Jeff Hinton for anybody who doesn't [30:15] know the name now very much known as the [30:18] godfather of neural networks and really [30:20] the godfather of kind of the whole [30:22] direction that AI went in [30:24] >> modern AI [30:25] >> he was kind of a fringe academic [30:28] >> at this point in history I mean neural [30:30] networks were not a respected subtree of [30:34] AI [30:34] >> no totally not [30:36] >> and part of the reason is there had been [30:38] a lot of hype 30 40 years before around [30:42] neural networks that just didn't pan [30:44] out. So it was effectively everyone [30:47] thought disproven and certainly [30:49] backwater. [30:50] >> Yep. Then do you remember from our [30:52] Nvidia episodes my favorite piece of [30:55] trivia about Jeff Hinton. [30:58] >> Oh yes. That his grandfather [30:59] great-grandfather was George Bool. [31:01] >> Yep. He is the great great grandson of [31:04] George and Mary Bool who invented [31:06] Boolean algebra and Boolean logic [31:09] >> which is hilarious now that I know more [31:11] about this because that's the basic [31:13] building block of symbolic logic of [31:15] defined deterministic computer science [31:18] logic. And the hilarious thing about [31:21] neural nets is it's not it's not [31:23] symbolic AI. It's not I feed you these [31:26] specific instructions and you follow a [31:27] big if then tree. It is [31:31] non-deterministic. It is the opposite of [31:33] that field. [31:34] >> Which actually just underscores again [31:35] how sort of heretical this branch of [31:37] machine learning and computer science [31:38] was. [31:39] >> Right. [31:39] >> So Ben, as you were saying earlier, [31:42] neural networks not a new idea and had [31:46] all of this great promise in theory, but [31:49] in practice just took too much [31:52] computation to do multiple layers. You [31:55] could really only have a single or maybe [31:57] small singledigit number of layers in a [32:00] computer neural network up until this [32:02] time. But Jeff and his former postto a [32:07] guy named Yan Lun start evangelizing [32:09] within the community, hey, if we can [32:12] find a way to have multi-layered, [32:16] deep layered neural networks, something [32:17] we call deep learning, we could actually [32:20] realize the promise here. It's not that [32:22] the idea is bad. It's that the [32:25] implementation which would take a ton of [32:28] compute to actually do all the math to [32:30] do all the multiplication required to [32:32] propagate through layer after layer [32:34] after layer of neural networks to sort [32:36] of detect and understand and store [32:38] patterns. If we could actually do that, [32:42] a big multi-layered neural network would [32:45] be very valuable and possibly could [32:47] work. [32:48] >> Yes. Here we are now in 2007, mid200s. [32:51] Moore's law has increased enough that [32:53] you could actually start to try to test [32:55] some of these theories. Yep. So Jeff [32:58] comes and he gives this talk at Google. [33:00] It's on YouTube. You can go watch it. [33:02] We'll link to it in the show notes. This [33:04] is incredible. This is an artifact of [33:05] history sitting there on YouTube. And [33:09] people at Google, Sebastian, Jeff Dean, [33:12] and all the other folks who are talking [33:13] about, they get very, very, very excited [33:16] because they've already been doing stuff [33:18] like this with translate and the [33:20] language models that they're working [33:21] with. That's not using deep neural [33:24] networks that Jeff's working on. So [33:26] here's this whole new architectural [33:28] approach that if they could get it to [33:30] work would enable these models that [33:32] they're building to work way better, [33:35] recognize more sophisticated patterns, [33:38] understand the data better. Very, very [33:40] promising. [33:40] >> Again, kind of all in theory at this [33:42] point. [33:42] >> Yep. So Sebastian Throne brings Jeff [33:46] Hinton into the Google fold after this [33:48] tech talk. I think first as a consultant [33:51] over the next couple years and then this [33:53] is amazing. Later, Jeff Hinton [33:55] technically becomes an intern at Google. [33:58] Like that's how they get around the [33:59] >> That's correct. [34:00] >> part-time, full-time policies here. [34:02] >> Yep. He was a summer intern in somewhere [34:05] around 2011, 2012. And mind you, at this [34:08] point, he's like 60 years old. [34:09] >> Yes. So in the next couple years after [34:12] 2007 here, Sebastian's concept of [34:15] bringing these computer science machine [34:18] learning academics into Google as [34:20] contractors or part-time or interns, [34:23] basically letting them keep their [34:24] academic posts and work on big projects [34:27] for Google's products internally goes so [34:30] well that by late 2009, Sebastian and [34:34] Larry and Sergey decided, hey, we should [34:36] just start a whole new division within [34:39] Google and it becomes Google X the [34:43] moonshot factory the first project [34:46] within Google X Sebastian leads himself [34:49] >> David don't say it don't say it [34:51] >> I won't say the name of it we will come [34:52] back to it later but for our purposes [34:54] for now the second project would be [34:57] critically important not only for our [34:59] story but to the whole world everything [35:02] in AI changing the entire world and that [35:06] second project is called Google Brain. [35:10] But before we tell the Google Brain [35:12] story, now is a great time to thank our [35:14] friends at JP Morgan Payments. [35:16] >> Yes. So today we are going to talk about [35:18] one of the core components of JP Morgan [35:20] Payments, their Treasury solutions. Now [35:23] treasury is something that most [35:24] listeners probably do not spend a lot of [35:26] time thinking about, but it's [35:28] fundamental to every company. [35:29] >> Yep. Treasury used to be just a back [35:32] office function, but now great companies [35:34] are using it as a strategic lever. With [35:36] JP Morgan Payments Treasury Solutions, [35:38] you can view and manage all your cash [35:40] positions in real time and all of your [35:43] financial activities across 120 [35:45] currencies in 200 countries. And the [35:48] other thing that they acknowledge really [35:49] in their whole strategy is that every [35:51] business has its own quirk. So, it's not [35:53] a cookie cutter approach. They work with [35:55] you to figure out what matters most for [35:57] you and your business and then help you [35:59] gain clarity, control, and confidence. [36:01] So whether you need advanced automation [36:03] or just want to cut down on manual [36:05] processes and approvals, their real-time [36:07] treasury solutions are designed to keep [36:09] things running smoothly. Whether your [36:11] treasury is in the millions or billions, [36:14] or perhaps like the company we're [36:16] talking about this episode, in the [36:17] hundreds of billions of dollars. [36:19] >> And they have some great strategic [36:20] offerings like Payby Bank, which lets [36:22] customers pay you directly from their [36:24] bank account. It's simple, secure, [36:26] tokenized, and you get faster access to [36:28] funds and enhance data to optimize [36:30] revenue and reduce fees. This lets you [36:32] send and receive real-time payments [36:34] instantly just with a single API [36:36] connection to JP Morgan. And because JP [36:38] Morgan's platform is global, that one [36:40] integration lets you access 45 countries [36:43] and counting and lets you scale [36:45] basically infinitely as you expand. As [36:47] we've said before, JP Morgan Payments [36:49] moves $10 trillion a day. So scale is [36:53] not an issue for your business. [36:55] >> Not at all. If you're wondering how to [36:57] actually manage all that global cash, JP [37:00] Morgan again has you covered with their [37:01] liquidity and account solutions that [37:03] make sure you have the right amount of [37:04] cash in the right currencies in the [37:06] right places for what you need. So [37:08] whether you're expanding into new [37:09] markets or just want more control over [37:11] your funds, JP Morgan Payments is the [37:13] partner you want to optimize liquidity, [37:15] streamline operations, and transform [37:17] your treasury. To learn more about how [37:19] JP Morgan can help you and your company, [37:21] just go to jporggan.com/acquired [37:24] and tell them that Ben and David sent [37:25] you. [37:26] >> All right, David. So, Google Brain. [37:29] >> So, when Sebastian left Stanford [37:32] full-time and joined Google full-time, [37:34] of course, somebody else had to take [37:36] over sales. And the person who did is a [37:39] another computer science professor, [37:41] brilliant guy named Andrew Ing. [37:43] >> This is like all the hits. [37:44] >> All the hits. This is all the AI hits on [37:46] this episode. [37:47] So, what does Sebastian do? He recruits [37:50] Andrew to come part-time, start spending [37:53] a day a week on the Google campus. And [37:55] this coincides right with the start of X [37:58] and Sebastian formalizing this division. [38:02] So, one day in 2010, 2011 time frame, [38:05] Andrew's spending his day a week on the [38:07] Google campus and he bumps into who [38:09] else? Jeff Dean. And Jeff Dean is [38:13] telling Andrew about what he and Fron [38:15] have done with language models and what [38:17] Jeff Hinton is doing in deep learning. [38:19] Of course, Andrew knows all this. And [38:20] Andrew is talking about what he and Sale [38:21] are doing at Stanford. and they decide, [38:24] you know, the time might finally be [38:27] right to try and take a real big swing [38:30] on this within Google and build a [38:33] massive really large deep learning model [38:38] in the vein of what Jeff Hinn has been [38:40] talking about on highly paralyzable [38:43] Google infrastructure. [38:45] >> And when you say the time might be [38:46] right, Google had tried twice before and [38:48] neither project really worked. They [38:50] tried this thing called brains on Borg. [38:52] Borg is sort of an internal system that [38:54] they use to run all of their [38:56] infrastructure. They tried the Cortex [38:58] project and neither of these really [39:00] worked. So there's a little bit of scar [39:02] tissue in the sort of research group at [39:04] Google of are large-scale neural [39:07] networks actually going to work for us [39:08] on Google infrastructure. So the two of [39:11] them, Andrew Ang and Jeff Dean, pull in [39:14] Greg Curado, who is a neuroscience PhD [39:16] and amazing researcher who was already [39:18] working at Google. And in 2011, the [39:22] three of them launch the second official [39:25] project within X, appropriately enough, [39:27] called Google Brain. And the three of [39:30] them get to work building a really, [39:32] really big, deep neural network model. [39:37] >> And if they're going to do this, they [39:38] need a system to run it on. You know, [39:40] Google is all about taking this sort of [39:43] frontier research and then doing the [39:45] architectural and engineering system to [39:47] make it actually run. [39:49] >> Yes. So Jeff Dean is working on this [39:53] system on the infrastructure and he [39:54] decides to name the infrastructure [39:58] disbelief which of course is a pun both [40:02] on the distributed nature of the system [40:05] and also on of course the word disbelief [40:08] because [40:08] >> no one thought it was going to work. [40:10] >> Most people in the field thought this [40:11] was not going to work and most people in [40:12] Google thought this was not going to [40:14] work. [40:14] >> And here's a little bit on why and it's [40:17] a little technical but follow me for a [40:18] second. All the research from that [40:20] period of time pointed to the idea that [40:22] you needed to be synchronous. So all the [40:24] compute needed to be sort of really [40:26] dense happening on a single machine with [40:28] really high parallelism kind of like [40:30] what GPUs do that you really would want [40:33] it all sort of happening in one place so [40:34] it's really easy to kind of go look up [40:37] and see hey what are the computed values [40:39] for everything else in the system before [40:41] I take my next move. What Jeff Dean [40:44] wrote with disbelief was the opposite. [40:46] it was distributed across a whole bunch [40:49] of CPU cores and potentially all over a [40:52] data center or maybe even in different [40:53] data centers. So in theory, this is [40:55] really bad because it means you would [40:57] need to be constantly waiting around on [40:59] any given machine for the other machines [41:01] to sync their updated parameters before [41:04] you could proceed. But instead, the [41:06] system actually worked asynchronously [41:08] without bothering to go and get the [41:10] latest parameters from other cores. So [41:13] you were sort of updating parameters on [41:15] stale data. You would think that [41:17] wouldn't work. The crazy thing is it [41:20] did. Yes. Okay. So you've got disbelief. [41:23] What do they do with it now? They want [41:25] to do some research. So they try out can [41:28] we do cool neural network stuff? And [41:30] what they do in a paper that they [41:33] submitted in 2011 right at the end of [41:36] the year is I'll give you the name of [41:38] the paper first. building high-level [41:41] features using largecale unsupervised [41:44] learning. But everyone just calls it the [41:46] cat paper. [41:47] >> The cat paper. [41:48] >> You talk to anyone at Google, you talk [41:50] to anyone in AI, they're like, "Oh yeah, [41:51] the cat paper." What they did was they [41:55] trained a large nine layer neural [41:58] network to recognize cats from unlabeled [42:01] frames of YouTube videos using 16,000 [42:05] CPU cores on a thousand different [42:08] machines. And listeners, just to like [42:10] underscore how seminal this is, we [42:12] actually talked with Sundar in prep for [42:14] the episode. And he cited seeing the cat [42:17] paper come across his desk as one of the [42:19] key moments that sticks in his brain in [42:22] Google's story. [42:23] >> Yeah. A little later on, they would do a [42:25] TGIF where they would present the [42:27] results of the CAT paper and you talk to [42:29] people at Google, they're like, "That [42:30] TGIF, oh my god, that's when it all [42:32] changed." [42:32] >> Yeah. It proved that large neural [42:35] networks could actually learn meaningful [42:37] patterns without supervision and without [42:41] labeled data. And not only that, it [42:43] could run on a distributed system that [42:46] Google built to actually make it work on [42:48] their infrastructure. And that is a huge [42:50] unlock of the whole thing. Google's got [42:53] this big infrastructure asset. Can we [42:55] take this theoretical computer science [42:57] idea that the researchers have come up [42:58] with and use disbelief to actually run [43:01] it on our system? Yep, that is the [43:03] amazing technical achievement here. That [43:06] is almost secondary to the business [43:09] impact of the CAT paper. I think it's [43:12] not that much of a leap to say that the [43:15] cat paper led to probably hundreds of [43:20] billions of dollars of revenue generated [43:23] by Google and Facebook and by dance over [43:27] the next decade. [43:28] >> Definitely pattern recognizers in data. [43:31] So, YouTube [43:33] had a big problem at this time, which [43:36] was that people would upload these [43:38] videos, and there's tons of videos being [43:39] uploaded to YouTube, but people are [43:41] really bad at describing what is in the [43:44] videos that they're uploaded. And [43:46] YouTube is trying to become more of a [43:47] destination site, trying to get people [43:50] to watch more videos, trying to build a [43:53] feed, increase dwell time, etc., etc. [43:57] And the problem is the recommener is [43:59] trying to figure out what to feed and [44:00] it's only just working off titles and [44:02] descriptions that people were writing [44:04] about their own videos, [44:05] >> right? And whether you're searching for [44:06] a video or they're trying to figure out [44:08] what video to recommend next, they need [44:10] to know what the video is about. [44:12] >> Yep. So the CAT paper proves that you [44:15] can use this technology, a deep neural [44:18] network running on disbelief [44:21] to go inside of the videos in the [44:24] YouTube library and understand what they [44:27] were about and use that data to then [44:30] figure out what videos to serve to [44:32] people. [44:32] >> If you can answer the question, cat or [44:34] not a cat, you can answer a whole lot [44:35] more questions, too. [44:36] >> Here's a quote from Jeff Dean about [44:38] this. We built a system that enabled us [44:40] to train pretty large neural nets [44:42] through both model and data parallelism. [44:44] We had a system for unsupervised [44:46] learning on 10 million randomly selected [44:48] YouTube frames. As you were saying, Ben, [44:50] it would build up unsupervised [44:52] representations based on trying to [44:54] reconstruct the frame from the highle [44:57] representations. We got that working and [44:59] training on 2,000 computers using 16,000 [45:02] cores. After a little while, that model [45:05] was actually able to build a [45:06] representation at the highest neural net [45:08] level where one neuron would get excited [45:11] by images of cats. It had never been [45:13] told what a cat was, but it had seen [45:15] enough examples of them in the training [45:17] data of head-on facial views of cats [45:19] that that neuron would then turn on for [45:21] cats and not much else. It's so crazy. I [45:25] mean, this is the craziest thing about [45:26] unlabelled data, unsupervised learning, [45:28] that a system can learn what a cat is [45:31] without ever being explicitly told what [45:33] a cat is and that there's a cat neuron. [45:36] >> Yeah. And so then there's a iPhone [45:39] neuron and a San Francisco Giants neuron [45:41] and all the things that YouTube [45:43] recommends, [45:44] >> not to mention porn filtering, explicit [45:46] content filtering, [45:48] >> not to mention copyright identification [45:51] and enabling revenue share with [45:52] copyright holders. Yeah, this leads to [45:55] everything in YouTube. Basically puts [45:57] YouTube on the path to today becoming [45:59] the single biggest property on the [46:01] internet and the single biggest media [46:03] company in the planet. This kicks off a [46:06] 10-year period from 2012 when this [46:09] happens until Chat GPT on November 30th, [46:12] 2022 [46:14] when AI is already shaping the human [46:17] existence for all of us and driving [46:19] hundreds of billions of dollars of [46:20] revenue. It's just in the YouTube feed [46:23] and then Facebook borrows it and they [46:25] hire Yan Lun and they start Facebook AI [46:27] research and then they bring it into [46:29] Instagram and then Tik Tok and Bite [46:31] Dance take it and then it goes back to [46:33] Facebook and YouTube with reals and [46:34] shorts. This is the primary way that [46:37] humans on the planet spend their leisure [46:39] time for the next 10 years. [46:41] >> This is my favorite David Rosenthalism. [46:43] Everyone talks about 2022 onward as the [46:45] AI era. And I love this point from you [46:47] that actually for anyone that could make [46:50] good use of a recommener system and a [46:52] classifier system, basically any company [46:54] with a social feed, the AI era started [46:56] in 2012. [46:57] >> Yes, the AI era started in 2012 and part [47:00] of it was the cat paper. The other part [47:03] of it was what Jensen at NVIDIA always [47:07] calls the big bang moment for AI, which [47:10] was AlexNet. [47:12] >> Yes. So, we talked about Jeff Hinton [47:15] back at the University of Toronto. He's [47:18] got two grad students who he's working [47:21] with in this era. Alex Kreseky and Ilia [47:26] Sutskyver, [47:27] >> of course, [47:28] >> future co-founder and chief scientist of [47:30] OpenAI. And the three of them are [47:33] working with Jeff's deep neural network [47:37] ideas and algorithms to create an entry [47:42] for the famous imageet competition in [47:46] computer science. [47:47] >> This is Fe Lee's thing from Stanford. [47:50] >> It is a annual machine vision algorithm [47:54] competition. And what it was was FFE had [47:58] assembled a database of 14 million [48:02] images that were handlabeled. Famously, [48:05] she used Mechanical Turk on Amazon, I [48:07] think, to get them all handlabeled. [48:09] >> Yes. And I think that's right. And so [48:11] then the competition was what team can [48:13] write the algorithm that without looking [48:15] at the labels, so just seeing the images [48:17] could correctly identify the largest [48:20] percentage, the best algorithms that [48:23] would win the competitions [48:25] year-over-year. We're still getting more [48:27] than a quarter of the images wrong. So [48:29] like 75% success rate, great. Way worse [48:32] than a human. [48:33] >> Can't use it for much in a production [48:35] setting when quarter the time you're [48:36] wrong. So then the 2012 competition [48:40] along comes Alex Net its error rate was [48:44] 15%. Still high but a 10% leap from the [48:48] previous best being a 25% error rate all [48:51] the way down to 15 in one year. A leap [48:54] like that had never happened before. [48:57] >> It's 40% better than the next best. [48:59] >> Yes. [49:00] >> On a relative basis. [49:01] >> Yes. [49:02] >> And why is it so much better, David? [49:04] What did they figure out that would [49:06] create a $4 trillion company in the [49:07] future? [49:10] >> So, what Jeff and Alex and Ilia did is [49:15] they knew like we've been talking about [49:17] all episode that deep neural networks [49:19] had all this potential and Moors law [49:21] advanced enough that you could use CPUs [49:23] to create a few layers. They had the aha [49:27] moment of what if we rearchitected [49:32] this stuff not to run on CPUs but to run [49:37] on a whole different class of computer [49:40] chips that were by their very nature [49:43] highly highly highly parallelizable [49:47] video game graphics cards made by the [49:52] leading company in the space at the time [49:54] Nvidia. [49:56] not obvious at the time and especially [49:58] not obvious that this highly advanced [50:01] cutting edge academic computer science [50:04] research [50:04] >> that was being done on supercomputers [50:06] usually [50:06] >> that was being done on supercomputers [50:08] with incredible CPUs would use these toy [50:12] video game cards [50:13] >> that retail for $1,000. [50:15] >> Yeah. Less at that point in time. A [50:16] couple hundred bucks. So the team in [50:19] Toronto, they go out to like the local [50:21] Best Buy or something. They buy two [50:25] Nvidia GeForce GTX 580s, which were [50:29] Nvidia's top-of-the-line gaming cards at [50:31] the time. The Toronto team rewrites [50:34] their neural network algorithms in CUDA, [50:37] Nvidia's programming language. They [50:39] train it on these two off-the-shelf GTX [50:42] 580s and this is how they achieve their [50:47] deep neural network and do 40% better [50:51] than any other entry in the imageet [50:53] competition. So when Jetson says that [50:55] this was the big bang moment of [50:57] artificial intelligence, a he's right. [51:00] This shows everybody that holy crap, if [51:02] you can do this with two off-the-shelf [51:04] GTX 580s, imagine what you could do with [51:07] more of them or with specialized chips. [51:09] And B, this event is what sets Nvidia on [51:13] the path from a somewhat struggling PC [51:16] gaming accessory maker to the leader of [51:19] the AI wave and the most valuable [51:21] company in the world today. And this is [51:23] how AI research tends to work is there's [51:25] some breakthrough that gets you this big [51:27] step change function and then there's [51:29] actually a multi-year process of [51:31] optimizing from there where you get [51:33] these kind of diminishing returns curves [51:35] on breakthroughs where the first half of [51:38] the advancement happens all at once and [51:41] then the second half takes many years [51:42] after that to figure out. It's rare and [51:45] amazing and it must be so cool when you [51:48] have an idea, you do it, and then you [51:50] realize, "Oh my god, I just found the [51:52] next giant leap in the field." [51:54] >> It's like I unlocked the next level to [51:55] use the video game analogy. [51:57] >> Yes, [51:58] >> I leveled up. So after Alexet, the whole [52:02] computer science world is a buzz. [52:04] >> People are starting to stop doubting [52:06] neural networks at this point. [52:08] >> Yes. So after Alexnap, the three of them [52:11] from Toronto, Jeff Hinton, Alex [52:14] Kashevky, and Ilaskever do the natural [52:16] thing, they start a company called DNN [52:19] Research, deep neural network research. [52:21] This company does not have any products. [52:23] This company has AI researchers [52:26] >> who just won a big competition. [52:28] >> And predictably, as you might imagine, [52:31] it gets acquired by Google almost [52:33] immediately. [52:34] >> Oh, are you intentionally shortening [52:36] this? [52:37] >> That's what I thought the story was. Oh, [52:38] it is not immediately. [52:40] >> Oh, okay. [52:41] >> There's a whole crazy thing that happens [52:43] where the first bid is actually from BU. [52:48] Oh, [52:49] >> I did not know that. [52:50] >> So, BU offers $12 million. [52:53] Jeff Henton doesn't really know how to [52:55] value the company and doesn't know if [52:56] that's fair. And so, he does what any [52:59] academic would do to best determine the [53:02] market value of the company. He says, [53:04] "Thank you so much. I'm gonna run an [53:06] auction now and I'm going to run it in a [53:08] highly structured manner where every [53:11] time anybody wants to bid the clock [53:14] resets and there's another hour where [53:16] anybody else can submit another bid. [53:19] >> No way. [53:20] >> So, [53:21] >> I didn't know this. This is crazy. [53:22] >> He gets in touch with everyone that he [53:24] knows from the research community who is [53:26] now working at a big company who he [53:28] thinks, hey, this would be a good place [53:30] for us to do our research. That includes [53:33] BYU, that includes Google, that includes [53:36] Microsoft, and there's one other [53:39] >> Facebook. Of course, [53:40] >> it's a two-year-old startup. [53:42] >> Oh, wait. So, it does not include [53:44] Facebook. [53:44] >> It does not include Facebook. Think [53:47] about the year. This is 2012. [53:50] So, Facebook's not really in the AI game [53:54] yet. They're still trying to build their [53:56] own AI lab. [53:58] >> Yeah. Yeah. Because Yan Lun and Fairwood [53:59] start in 2013. Is it Instagram? [54:02] >> Nope. It is the most important part of [54:05] the end of this episode. [54:06] >> Wait. Well, it can't be Tesla because [54:08] Tesla is older than that. [54:09] >> Nope. [54:10] >> Well, OpenAI wouldn't get founded for [54:11] years. [54:13] >> Wow. Okay, you really got me here. [54:15] >> What company slightly predated OpenAI [54:18] doing effectively the same mission? [54:21] >> Oh, [54:23] of course. Of course. Hiding in plain [54:26] sight. [54:27] Deep Mind. Wow. Deep Mind, baby. They [54:31] are the fourth bidder in a four-way [54:33] auction for DNN Research. Now, of [54:35] course, right after the bidding starts, [54:37] DeepMind has to drop out. They're a [54:39] startup. They don't actually have the [54:40] cash to be able to buy. [54:41] >> Yeah. Didn't even cross my mind cuz my [54:43] first question was like, where the hell [54:44] would they get the money because they [54:46] had no money. [54:46] >> But Jeff Hinton already knows and [54:48] respects Demis. [54:50] >> Ah, [54:50] >> even though he's just doing this at the [54:52] time startup called DeepMind. [54:54] >> That's amazing. Wait, how is Deep Mind [54:57] in the auction, but Facebook is not? [54:58] Isn't that wild? [55:00] >> That's wild. [55:00] >> So, the timing of this is concurrent [55:04] with the it was then called NIPS, now [55:07] it's called Nurips Conference. So, Jeff [55:10] Hinton actually runs the auction from [55:12] his hotel room at the Hera's Casino in [55:15] Lake Tahoe. [55:16] >> Oh my god, amazing. [55:18] >> So, the bids all come in and we got to [55:20] thank Cade Mets, the author of Genius [55:22] Makers, great book on the whole history [55:24] of AI that we're actually going to [55:25] reference a lot in this episode. The [55:27] bidding goes up and up and up. At some [55:29] point, Microsoft drops out. They come [55:30] back in. Told you DeepMind drops out. [55:32] So, it's BU and Google really going at [55:35] the end. And finally, at some point, the [55:38] researchers look at each other and they [55:39] say, "Where do we actually want to land? [55:40] We want to land at Google." And so, they [55:42] stop the bidding at $44 million and just [55:45] say, "Google, this is more than enough [55:47] money. We're going with you." [55:49] >> Wow. I knew it was about $40 million. I [55:52] did not know that whole story. It's [55:54] almost like Google itself and you know [55:55] the Dutch auction IPO process, right? [55:58] How fitting. [55:59] >> That's kind of a perfect DNA. Yes. [56:01] >> Wow. [56:02] >> And the three of them were supposed to [56:03] split it 33 each and Alex and Ilia go to [56:07] Jeff and say, "I really think you should [56:08] have a bigger percent. I think you [56:10] should have 40% and we should each have [56:11] 30." And that's how it ends up breaking [56:13] down. [56:14] >> Ah, wow. What a team. Well, that leads [56:18] to the three of them joining Google [56:20] Brain directly. And turbocharging [56:23] everything going on there. Spoiler [56:25] alert, a couple years later, Astro [56:28] Teller, who would take over running [56:30] Google X after Sebastian Threaten left, [56:32] he would get quoted in the New York [56:34] Times in a profile of Google X, that the [56:38] gains to Google's core businesses and [56:40] search and ads and YouTube from Google [56:43] Brain have way more than funded all of [56:48] the other bets that they have made [56:50] within Google X and throughout the [56:52] company over the years. It's one of [56:53] these things that if you make something [56:54] a few% better that happens to do tens of [56:57] billions of dollars or hundreds of [56:58] billions of dollars in revenue, you find [57:00] quite a bit of loose change in those [57:02] couch cushions. [57:03] >> Yes, quite quite a bit of loose change. [57:05] But that's not where the AI history ends [57:08] within Google. There is another very [57:12] important piece of the Google AI story [57:15] that is an acquisition from outside of [57:17] Google. The AI equivalent of Google's [57:20] acquisition of YouTube. It's what we [57:22] talked about a minute ago, Deep Mind. [57:25] But before we tell the Deep Mind story, [57:27] now is a great time to thank a new [57:30] partner of ours, Sentry. [57:32] >> Yes, listeners, that is S N T R Y, like [57:36] someone's standing guard. [57:38] >> Yes, Sentry helps developers debug [57:41] everything from errors to latency and [57:43] performance issues, pretty much any [57:45] software problem, and fix them before [57:48] users get mad. As their homepage puts [57:50] it, they are considered quote unquote [57:52] not bad by over four million software [57:55] developers. [57:56] >> And today we're talking about the way [57:57] that Sentry works with another company [57:59] in the acquired universe, Anthropic. [58:01] Anthropic used to have some older [58:03] monitoring systems in place, but as they [58:05] scaled and became more complex, they [58:07] adopted Sentry to find and fix issues [58:09] faster. [58:10] >> So when you're building AI models, like [58:12] we're talking about all episode here, [58:15] small issues can ripple out into big [58:17] ones fast. Let's say you're running a [58:19] huge compute job like training a model. [58:21] If one node fails, it can have massive [58:24] downstream impacts, costing huge amounts [58:26] of time and money. Sentry helped [58:28] Anthropic detect bad hardware early so [58:31] they could reject it before causing a [58:33] cascading problem and taking debugging [58:35] down to hours instead of days for them. [58:37] And one other fun update from Sentry, [58:39] they now have an AI debugging agent [58:41] called Seir. Seir uses all the context [58:44] that Sentry has about your app usage to [58:46] run root cause analysis as issues are [58:48] detected. It uses errors, span data, [58:51] logs, and tracing and your code to [58:53] understand the root cause, fix it, and [58:54] get you back to shipping. It even [58:56] creates pull requests to merge code [58:58] fixes in. And on top of that, they also [59:01] recently launched agent and MCP server [59:03] monitoring. AI tooling tends to offer um [59:06] limited visibility into what's going on [59:08] under the hood, shall we say. Century's [59:10] new tools make it easy to understand [59:12] exactly what's going on. This is [59:15] everything from actual AI tool calls to [59:18] performance across different models and [59:19] interactions between AI and the [59:21] downstream services. We're pumped to be [59:23] working with Sentry. We're big fans of [59:25] the company and of all the great folks [59:26] we're working with there. They have an [59:28] incredible customer list including not [59:30] only Anthropic, but Cursor, Verscell, [59:32] Linear, and more. And actually, if [59:34] you're in San Francisco or the Bay Area, [59:36] Sentry is hosting a small invite only [59:38] event with Dave and I in San Francisco [59:40] for product builders on October 23rd. [59:43] You can register your interest at [59:45] century.io/acquired. [59:47] That's century sy.io/acquired. [59:51] And just tell them that Ben and David [59:53] sent you. All right, David. Deep mind. I [59:57] kind of like your framing. The YouTube [59:59] of AI. [60:00] >> The YouTube of AI for Google. They [60:03] bought this thing for, we'll talk about [60:04] the purchase price, but it's worth what [60:07] $500 billion today. I mean, this is as [60:10] good as Instagram or YouTube in terms of [60:13] greatest acquisitions of all time. [60:14] >> 100%. So, I remember when this deal [60:18] happened, just like I remember when the [60:20] Instagram deal happened cuz the number [60:21] was big at the time. [60:22] >> It was big, but I remember it for a [60:24] different reason. It was like when [60:25] Facebook bought Instagram, like, "Oh my [60:27] god, this is wow, what a tectonic shift [60:31] in the landscape of tech." In January [60:35] 2014, I remember reading on TechCrunch [60:38] this random news, [60:39] >> right? You're like, "Deep what?" [60:41] >> That Google is spending a lot of money [60:43] to buy something in London that I've [60:45] never heard of that's working on [60:49] artificial intelligence. [60:52] >> Right. This really illustrates how [60:54] outside of mainstream tech AI was at the [60:57] time. [60:58] >> Yeah. And then you dig in a little [60:59] further and you're like, this company [61:01] doesn't seem to have any products. And [61:04] it also doesn't even really say anything [61:06] on its website about what Deep Mind is. [61:09] It says it is a quote unquote [61:11] cuttingedge artificial intelligence [61:13] company. [61:14] >> Wait, did you look this up on the way [61:16] back machine? [61:16] >> Yeah, I did. I did. [61:17] >> Oh, nice. to build generalpurpose [61:20] learning algorithms for simulations, [61:23] e-commerce, and games. This is 2014. [61:27] This does not compute, does not [61:28] register. [61:29] >> Simulations, e-commerce, and games. It's [61:32] kind of a random spattering of [61:34] >> Exactly. It turns out though, not only [61:37] was that description of what DeepMind [61:39] was fairly accurate, this company and [61:43] this purchase of it by Google was the [61:47] butterfly flapping its wings equivalent [61:49] moment that directly leads to OpenAI, [61:53] Chat, GPT, Anthropic, and basically [61:57] everything. [61:58] >> Certainly Gemini [61:58] >> that we know. Yeah, Gemini directly in [62:01] the world of AI today [62:03] >> and probably XAI given Elon's [62:05] involvement. [62:05] >> Yeah, of course XAI. [62:07] >> In a weird way, it sort of leads to [62:08] Tesla self-driving too. Carpathy. [62:11] >> Yeah, definitely. Okay, so what is the [62:14] story here? Deep Mind was founded in [62:17] 2010 by a neuroscience PhD named Demis [62:23] Hassabis who previously started a video [62:26] game company. [62:26] >> Oh yeah. and a posttock named Shane Le [62:31] at University College London and a third [62:35] co-founder who was one of Demis' friends [62:39] from growing up, Mustafa Sullean. This [62:42] was unlikely to say the least. [62:45] >> This would go on to produce a knight and [62:49] Nobel Prize winner. [62:50] >> Yes. So Demis, the CEO, was a childhood [62:56] chess prodigy turned video game [62:58] developer who when he was aged 17 in [63:03] 1994, [63:05] he had gotten accepted to the University [63:08] of Cambridge, but he was too young and [63:11] the university told him, "Hey, take a, [63:13] you know, gap year, come back." He [63:15] decided that he was going to go work at [63:17] a video game developer at a video game [63:19] studio called Bullfrog Productions for [63:20] the year. And while he's there, he [63:22] created the game Theme Park, if you [63:25] remember that. It was like a theme park [63:27] version of Sim City. This was a big [63:30] game. This was very commercially [63:32] successful. Roller Coaster Tycoon would [63:34] be sort of a clone of this that would [63:36] have many, many sequels over the years. [63:38] >> Oh, I played a ton of that. Yeah, it [63:40] sells 15 million copies in the mid 90s. [63:45] Wow, wild. Then after this, he goes to [63:48] Cambridge, studies computer science [63:50] there. After Cambridge, he gets back [63:52] into gaming, founds another game studio [63:54] called Elixir that would ultimately [63:56] fail. And then he decides, you know [63:59] what, I'm going to go get my PhD in [64:00] neuroscience. And that is how Demis ends [64:02] up at University College London. There [64:05] he meets Shane leg who's there as a [64:08] postoc. Shane is a self-described at the [64:12] time member of the lunatic fringe in the [64:16] AI community in that he believes [64:22] this is 2008 9 10 he believes that AI is [64:27] going to get more and more and more [64:30] powerful every year and that it will [64:33] become so powerful that it will become [64:36] more intelligent than humans and Shane [64:38] is one of the people who actually [64:41] popularizes the term artificial general [64:43] intelligence, AGI. [64:45] >> Oh, interesting. Which of course lots of [64:48] people talk about now and approximately [64:49] zero people were afraid of that. I mean, [64:52] you had like the Nick Bostonramm type [64:54] folks, but very few people were thinking [64:57] about super intelligence or the [64:59] singularity or anything like that. For [65:01] what it's worth, not Elon Musk. He's not [65:03] included in that list because Demis [65:05] would be the one who tells Elon about [65:08] this. [65:10] Yes, we'll get to it. So, Demis and [65:12] Shane hit it off. They pull in Mustafa, [65:16] Demis' childhood friend, who is himself [65:19] extremely intelligent. He had gone to [65:20] the University of Oxford and then [65:22] dropped out, I think, at age 19 to do [65:24] other startupy type stuff. So, the three [65:26] of them decided to start a company, [65:28] DeepMind. The name of course being a [65:30] reference to deep learning, Jeff [65:32] Hinton's work and everything coming out [65:33] of the University of Toronto. and the [65:35] goal that the three of these guys have [65:36] of actually creating an intelligent mind [65:40] with deep learning. Like Jeff and Ellie [65:42] and Alex aren't really thinking about [65:43] this yet. As we said, this is lunatic [65:45] fringe type stuff. [65:47] >> Yes, AlexNet, the cat paper, that whole [65:50] world is about better classifying data. [65:53] Can we better sort into patterns? It's a [65:56] giant leap from there to say, "Oh, we're [65:57] going to create intelligence." [65:59] >> Yes. I think probably some people [66:02] almost almost certainly at Google were [66:03] thinking, "Oh, we can create narrow [66:06] intelligence that'll be better than [66:07] humans at certain tasks." [66:09] >> I mean, a calculator is better than [66:10] humans at certain tasks, [66:12] >> right? But I don't think too many people [66:14] were thinking, "Oh, this is going to be [66:16] general intelligence, smarter than [66:18] humans, [66:19] >> right?" [66:19] >> So, they decide on the tagline for the [66:22] company is going to be solve [66:23] intelligence and use it to solve [66:26] everything else. [66:27] >> Ooh, I like it. I like it. Yeah. Yeah. I [66:30] mean, they're they're they're good [66:30] marketers, too, these guys. [66:32] >> So, there's just one problem to do what [66:36] they want to do. [66:38] >> Money. Just saying. Money is the [66:40] problem. [66:40] >> Right. Right. Right. Money is the [66:42] problem for lots of reasons. But even [66:45] more so than any other given startup in [66:48] the 2010 era, it's not like they can [66:51] just go spin up an AWS instance and like [66:54] build an app and deploy it to the app [66:56] store. They want to build really, [67:00] really, really, really, really big deep [67:03] learning neural networks that requires [67:05] Googleiz levels of compute. Well, it's [67:08] interesting. It actually they don't [67:10] require that much funding yet. The AI of [67:13] the time was go grab a few GPUs. We're [67:15] not training giant LLMs. That's the [67:18] ambition eventually, but right now, what [67:20] they just need to do is raise a few [67:21] million bucks. But who's going to give [67:23] you a few million bucks when there's no [67:25] business plan? When you're just trying [67:26] to solve intelligence, you need to find [67:29] some lunatics. [67:30] >> It's a tough cell to VCs, [67:32] >> except for the exact right, [67:34] >> as you say, they need to find some [67:35] lunatics. [67:38] >> Oh, I chose my words carefully, didn't [67:39] you? [67:39] >> Yeah, we use the term lunatic in uh [67:42] >> it's endearing is [67:43] >> most endearing possible way here given [67:45] that they were all basically right. So, [67:48] in June 2010, Demis and Shane managed to [67:51] get invited to the Singularity Summit in [67:55] San Francisco, California, [67:56] >> cuz they're not raising money for this [67:58] in London. [67:58] >> Yeah, definitely not. I think they tried [68:00] for a couple months and learned that [68:02] that was not going to be a viable path. [68:04] >> Yes. This summit, the Singularity [68:06] Summit, organized by Ray Kerszswhile, [68:10] uh, future Google employee, I think, [68:12] chief futurist, noted futurist, [68:15] >> Elzar Yudkowski, [68:18] and [68:19] Peter Teal. [68:21] >> Yes. So, Demis and Shane are uh excited [68:25] about getting this invite. like this is [68:28] probably our one chance to get funded. [68:31] >> But we probably shouldn't just walk in [68:33] guns blazing and say, "Peter, can we [68:35] pitch you?" [68:35] >> Yeah. So, they finagle their way into [68:39] Demis getting to give a talk on stage at [68:43] the summit. [68:44] >> Always the hack. [68:45] >> They're like, "This is great. This is [68:46] going to be the hack. The talk is going [68:48] to be our pitch to Peter and Founders [68:51] Fund." Peter has just started Founders [68:52] Fund at this point. you know, obviously [68:54] member of the PayPal mafia, very [68:56] wealthy. [68:56] >> I think he had a big Roth IRA at this [68:58] point is the right way to frame it. [69:00] >> Big Roth IRA that he had invested in [69:02] Facebook, first investor in Facebook. He [69:05] is the perfect target. They architect [69:07] the presentation at the summit to be a [69:10] pitch directly to Peter essentially a [69:12] thinly veiled pitch. Shane has a quote [69:16] in Parm Olsen's great book Supremacy [69:18] that we used as a source for a lot of [69:19] this deep mind story. And Shane says, [69:22] "We needed someone crazy enough to fund [69:24] an AGI company. Somebody who had the [69:27] resources not to sweat a few million and [69:29] liked super ambitious stuff." They also [69:31] had to be massively contrarian because [69:34] every professor that he would go talk to [69:36] would certainly tell him absolutely do [69:38] not even think about funding this. [69:42] That ven diagram sure sounds a lot like [69:44] Peter Teal. So they show up at the [69:46] conference. Demis is going to give the [69:47] talk. Goes out on stage. He looks out [69:50] into the audience. Peter is not there. [69:53] Turns out Peter wasn't actually that [69:55] involved in the conference. [69:57] >> He's a busy guy. He's a co-founder or [69:59] co-organizer, but is a busy guy. [70:01] >> Yes. Guy's like, shoot. Oh, we missed [70:04] our chance. [70:05] What are we gonna do? And then fortune [70:08] turns in their favor. They find out that [70:11] Peter is hosting an afterparty that [70:14] night at his house in San Francisco. [70:17] They get into the party. Deis seeks out [70:20] Peter and he's Deis is very very very [70:22] smart as anybody who's ever listened to [70:24] him talk would immediately know. He's [70:26] like rather than just pitching Peter [70:28] headon. I'm going to come about this [70:31] obliquely. He starts talking to Peter [70:34] about chess because he knows as [70:35] everybody does that Peter Teal loves [70:37] chess. And Demis had been the second [70:40] highest ranked player in the world as a [70:43] teenager in the under 14 category. [70:46] >> Good strategy. [70:47] >> Great strategy. The man knows his chess [70:49] moves. So Peter's like, "hm, I like you. [70:52] You seem smart. What do you do?" And [70:54] Deis explains, he's got this AGI [70:56] startup. They were actually here. He [70:58] gave a talk on stage as part of the [71:01] conference. People are excited about [71:02] this. And Peter says, "Okay, all right. [71:04] Come back to Founders Fund tomorrow and [71:06] give me the pitch." So they do. They [71:08] make the pitch. It goes well. Founders [71:10] Fund leads Deep Minds seed round of [71:13] about $2 million. My how times have [71:16] changed for AI company seed rounds these [71:18] days. [71:19] >> Oh yes. [71:20] >> Imagine leading Deep Mind seed round [71:24] with less than $2 million check. And [71:27] through Peter and Founders Fund, they [71:29] get introduced, [71:30] >> hey Elon, you should meet this guy. [71:32] >> To another member of the PayPal mafia, [71:34] Elon Musk. [71:36] >> Yes. [71:37] So, it's teed up in a pretty low-key [71:41] way. Hey, Elon, you should meet this [71:42] guy. He's smart. He's thinking about [71:44] artificial intelligence. So, Elon says, [71:47] "Great. Come over to SpaceX. I'll give [71:48] you the tour of the place." So, Deus [71:50] comes over for lunch and a tour of the [71:52] factory. Of course, Deus thinks it's [71:54] very cool, but really, he's trying to [71:56] reorient the conversation over to [71:58] artificial intelligence. And I'll read [72:00] this great excerpt from an article in [72:03] the Guardian. Musk told Hassabis his [72:06] priority was getting to Mars as a backup [72:08] planet in case something went wrong [72:09] here. I don't think he'd thought much [72:11] about AI at this point. Hassabis pointed [72:13] out a flaw in his plan. I said, "What if [72:16] AI was the thing that went wrong here? [72:18] Then being on Mars wouldn't help you [72:20] because if we got there, then it would [72:21] obviously be easy for an AI to get there [72:23] through our communication systems or [72:25] whatever it was." He hadn't thought [72:27] about that. So he sat there for a minute [72:29] without saying anything, just sort of [72:31] thinking, hm, that's probably true. [72:34] Shortly after, Musk too became an [72:37] investor in DeepMind. [72:38] >> Yes. [72:40] >> Yes. Yes. [72:41] >> I think it's crazy that Demis is sort of [72:43] the one that woke Elon up to this idea [72:45] of we might not be safe from the AI on [72:48] Mars either. [72:49] >> Right. Right. I hadn't considered that. [72:54] So, uh, this is the first time the bit [72:55] flips for Elon of we really need to [72:58] figure out a safe, secure AI for the [73:01] good of the people. That sort of seed [73:03] being planted in his head. [73:04] >> Yep. [73:05] >> Which of course is what Deep Mind's [73:08] ambition is. We are here doing research [73:11] for the good of humanity like scientists [73:12] in a peer-reviewed way. [73:14] >> Yep. I think all that is true. Also [73:19] in the intervening months to year after [73:24] this meeting between Demis and Elon and [73:26] Elon investing in DeepMind, Elon also [73:29] starts to get really really excited and [73:32] convinced about the capabilities of AI [73:35] in the near term and specifically the [73:38] capabilities of AI for Tesla. [73:41] >> Yes. Like with everything else in Elon's [73:44] world, once the bit flips and he becomes [73:47] interested, he completely changes the [73:49] way he views the world. Completely sheds [73:51] all the old ways and actions that he was [73:53] taking. And it's all about what do I [73:56] most do to embrace this new worldview [73:58] that I have? [73:59] >> And other people have been working on [74:02] for a while already by this point. AI [74:05] driving cars. [74:07] >> Yep. [74:08] >> That sounds like it would be a pretty [74:09] good idea for Tesla. does. [74:11] >> So Elon [74:14] starts trying to recruit as many AI [74:16] researchers as he possibly can and [74:19] machine vision and machine learning [74:21] experts into Tesla. And then Alexet [74:24] happens and man, Alex Net's really, [74:27] really, really good at identifying and [74:30] classifying images and cat videos on [74:32] YouTube and the YouTube recommener feed. [74:34] Well, is that really that different from [74:37] a live feed of video from a car that's [74:40] being driven and understanding what's [74:41] going on there? [74:42] >> Can we process it in real time and look [74:45] at differences between frames? [74:46] >> Perhaps controlling the car not all that [74:50] different. So Elon's excitement [74:53] channeled initially through Deep Mind [74:55] and Demis about AI and AI for Tesla [74:59] starts ratcheting up big time. [75:02] >> Yep. Meanwhile, back in London, DeepMind [75:04] is getting to work. They're hiring [75:06] researchers. They're getting to work on [75:09] models. They're making some vague noises [75:12] about products to their investors. Maybe [75:15] we could do something in shopping. Maybe [75:17] something in gaming like the description [75:19] on the website at the time of [75:21] acquisition said. But mostly what they [75:24] really really want to do is just build [75:25] these models and work on intelligence. [75:27] And then one day in late 2013, [75:31] they get a call from Mark Zuckerberg. He [75:35] wants to buy the company. Mark has woken [75:40] up to everything that's going on at [75:41] Google after Alexet and what AI is doing [75:46] for social media feed recommendations at [75:49] YouTube, the possibility of what it can [75:52] do at Facebook and for Instagram. He's [75:54] gone out and recruited Yan Lun Jeff [75:58] Hinton's old postoc who's together with [76:00] Jeff one of the sort of godfathers of AI [76:02] and deep learning [76:03] >> and really popularized the idea of [76:05] convolutional neural networks the next [76:07] hot thing in the field of AI at this [76:09] point in time [76:10] >> and so with Yan they have created fair [76:12] Facebook AI research which is a Google [76:16] brain rival within Facebook and remember [76:21] who the first investor in Facebook was [76:22] who's still on the board [76:23] >> Peter and is also the lead investor in [76:25] DeepMind. Where do you think Mark [76:27] learned about DeepMind? Peter Teal, [76:29] >> was it? Do you know for sure that it was [76:31] from Peter? [76:31] >> No, I don't know for sure, but like how [76:32] else could Mark have learned about this [76:34] startup in London? [76:35] >> I've got a great story of how Larry [76:36] Paige found out about it. [76:38] >> Oh, okay. Well, we'll get to that in one [76:39] sec. [76:41] >> So, Mark calls and offers to buy the [76:42] company. And there are various rumors of [76:46] how much Mark offered, but according to [76:49] Parmmy Olsson in her book, Supremacy, [76:51] the reports are that it was up to $800 [76:53] million. Company with no products and a [76:55] long way from AGI. [76:57] >> That squares with what Cade Mets has in [76:59] his book that the founders would have [77:00] made about twice as much money from [77:02] taking Facebook's offer versus taking [77:04] Google's offer. [77:05] >> Yep. So Demis of course takes this news [77:09] to the investor group which by the way [77:12] is kind of against everything the [77:14] company was founded on. The whole aim of [77:16] the company and what he's promised the [77:18] team is that DeepMind is going to stay [77:21] independent, do research, publish in the [77:23] scientific community. We're not going to [77:25] be sort of captured and told what to do [77:27] by the whims of a capitalist [77:28] institution. [77:29] >> Yep. So definitely some deal point [77:32] negotiating that has to happen with Mark [77:34] and Facebook if this offer is going to [77:36] come through. [77:37] >> But Mark is so desperate at this point. [77:39] He is open to these very large dealpoint [77:41] negotiations such as Yan Lun gets to [77:44] stay in New York. Yan Lun gets to stay [77:46] operating his lab at NYU. Yan Lun is a [77:48] professor. He's flexible on some things. [77:50] Turns out Mark is not flexible on [77:53] letting Demis keep control of Deep Mind [77:56] if he buys it. Demis sort of argued for [77:58] we need to stay separate and carved out [78:00] and we need this independent oversight [78:02] board with his ability to intervene if [78:05] the mission of Deep Mind is no longer [78:07] being followed. And Mark's like, "No, [78:10] you'll be a part of Facebook." [78:11] >> Yeah. And you'll make a lot of money. [78:15] So, as this negotiation is going on, of [78:18] course, the investors in Deep Mind get [78:20] wind of this. Elon [78:23] finds out about what's going on. He [78:26] immediately calls up Demis and says, "I [78:28] will buy the company right now with [78:30] Tesla stock." [78:32] This is late 2013, like early 2014. [78:35] Tesla's market cap is about $20 billion. [78:38] So Tesla stock from then to today is [78:42] about a 70x runup. [78:46] Deis and Shane and Mustafa are like, [78:49] "Wow." Okay, there's a lot going on [78:50] right now. But to your point, they have [78:53] the same issues with Elon and Tesla that [78:56] they had with Mark. Elon wants them to [78:59] come in and work on autonomous driving [79:03] for Tesla. They don't want to work on [79:05] autonomous driving, [79:06] >> right? Or at least exclusively. [79:08] >> At least exclusively. Yep. So then [79:12] Dennis gets a third call from Larry [79:16] Page. [79:16] >> Do you want my story of how Larry knows [79:18] about the company? I absolutely want [79:20] your story of how Larry knows about the [79:21] company. [79:22] >> All right, so this is still early in [79:24] Deep Mind's life. We haven't progressed [79:26] all the way to this acquisition point [79:27] yet. Apparently, Elon Musk is on a [79:30] private jet with Luke Nosk, who's [79:32] another member of the PayPal mafia and [79:34] an angel investor in DeepMind, and [79:36] they're reading an email from Demis with [79:39] an update about a breakthrough that they [79:41] had where DeepMind AI figured out a [79:43] clever way to win at the Atari game [79:45] breakout. [79:46] >> Yes. And the strategy it figured out [79:48] with no human training was that you [79:51] could bounce the ball up around the [79:53] edges of the bricks and then without [79:56] needing to intervene, it could bounce [79:58] around along the top and win the game [80:00] faster without you needing to have a [80:02] whole bunch of interactions with the [80:03] paddle down at the bottom. They're [80:05] watching this video of how clever it is. [80:07] And flying with them on the same private [80:10] plane is Larry Page. Of course, because [80:14] Elon and Larry used to be very good [80:16] friends. [80:17] >> Yes. And Larry is like, "Wait, what are [80:20] you watching? What company is this?" And [80:22] that's how he finds out. [80:24] >> Wow. [80:25] >> Yes. [80:26] >> Elon must have been so angry about all [80:30] this. [80:31] >> And the crazy thing is this kinship [80:33] between Larry and Demis is I think the [80:36] reason why the deal gets done at Google. [80:38] Once the two of them get together, they [80:41] are like peas in a pod. Larry has always [80:44] viewed Google as an AI company. [80:47] >> Yeah. [80:47] >> Demis of course views DeepMind so much [80:50] as an AI company that he doesn't even [80:51] want to make any products until they can [80:53] get to AGI. [80:55] >> And Demis, in fact, we should share with [80:57] listeners. Demus told us this when we [80:58] were talking to him to prep for this [81:00] episode, just felt like Larry got it. [81:02] Larry was completely on board with the [81:03] mission of everything that DeepMind was [81:05] doing. And there's something else very [81:09] convenient about Google. They already [81:12] have Brain. So Larry doesn't need Demis [81:16] and Shane and Mustafa and DeepMind to [81:18] come work on products within Google. [81:21] >> Right? [81:22] >> Brain is already working on products [81:23] within Google. Demis can really believe [81:26] Larry when Larry says, "Nah, stay in [81:28] London. Keep working on intelligence. Do [81:30] what you're doing. I don't need you to [81:31] come work on products within Google." [81:33] brain is like actively going and [81:35] engaging with the product groups trying [81:36] to figure out, hey, how can we deploy [81:38] neural nets into your product to make it [81:40] better? That's like their reason for [81:41] being. So, they're happy to agree to [81:44] this [81:44] >> and it's working. Brain and neural nets [81:47] are getting integrated into search, into [81:49] ads, into Gmail, into everything. It is [81:52] the perfect home for Deep Mind. Home [81:56] away from home, shall we say? [81:57] >> Yes. And and there's a third reason why [82:00] Google's the perfect fit for Deep Mind. [82:02] infrastructure. Google has all the [82:04] compute infrastructure you could ever [82:06] want right there on tap. [82:08] >> Yes. At least with CPUs so far. [82:11] >> Yes. [82:12] >> So, how's the deal actually happen? [82:14] Well, after buying DNN research, Alan [82:16] Eustace, who David you spoke with, [82:17] right? [82:18] >> Yep. [82:18] >> Was Google's head of engineering at the [82:20] time, he makes up his mind that he [82:22] wanted to hire all the best deep [82:25] learning research talent that he [82:26] possibly could and he had a clear path [82:28] to do so. A few months earlier, Larry [82:30] Pageige held a strategy meeting on an [82:33] island in the South Pacific in Cade [82:35] Mets's book, It's an Undisclosed Island. [82:37] >> Of course, he did. [82:38] >> Larry thought that deep learning was [82:40] going to completely change the whole [82:41] industry. And so, he tells his team, [82:43] this is a quote, "Let's really go big." [82:45] Which effectively gave Allen a blank [82:47] check to go secure all the best [82:48] researchers that he possibly could. So, [82:50] in 2013, he decides, I'm going to get on [82:52] a plane in December before the holidays [82:54] and go meet DeepMind. Crazy story about [82:57] this. Jeff Hinton, who's at Google at [82:59] the time, had a thing with his back [83:01] where he couldn't sit down. He either [83:04] has to stand or lay. And so a long [83:06] flight across the ocean is not doable. [83:09] But he needs to be there as a part of [83:10] the diligence process. You have Jeff [83:12] Hinton. You need to use him to figure [83:13] out if you're going to buy a deep [83:14] learning company. And so Alan Eustace [83:17] decides he's going to charter a private [83:20] jet. And he's going to build this crazy [83:23] custom harness rig so that [83:26] Jeff Hinton won't be sliding around when [83:29] he's laying on the floor during takeoff [83:31] and landing. [83:32] >> Wow. I was thinking the first part of [83:34] this I'm pretty sure Google has planes. [83:36] They could just get into Google Play. [83:37] >> For whatever reason, this was a separate [83:38] charter. [83:39] >> But it's not solvable just with a [83:40] private plane. You need also a harness, [83:43] >> right? And Allan is the guy who set the [83:47] record for jumping out of the world's [83:50] highest Was it a balloon? I actually [83:52] don't know. The highest freef fall jump [83:54] that anyone has ever done, even higher [83:56] than that Red Bull stunt a few years [83:58] before. So, he's like very used to [84:00] designing these custom rigs for [84:02] airplanes. He's like, "Oh, no problem. [84:04] You just need a bed and some straps. I [84:06] jumped out of the atmosphere in a scuba [84:08] suit. I think we'll be fine." [84:09] >> That is amazing. [84:10] >> So, they fly to London. They do the [84:12] diligence. They make the deal. Deis has [84:14] true kinship with Larry and it's done. [84:18] $550 million US. There's an independent [84:21] oversight board that is set up to make [84:24] sure that the mission and goals of [84:26] DeepMind are actually being followed and [84:29] this is an asset that Google owns today [84:32] that again I think is worth half a [84:34] trillion dollars if it's independent. [84:36] >> Do you know what other member of the [84:38] PayPal mafia gets put on the ethics [84:41] board after the acquisition? [84:43] >> Reed Hoffman. [84:44] >> Reed Hoffman [84:44] >> has to be given the open AI tie later. [84:47] We are gonna come back to Reed in just a [84:49] little bit here. [84:50] >> Yes. So after the acquisition, it goes [84:52] very well very quickly. Famously the [84:55] data center cooling thing happens where [84:58] DeepMind carved off some part of the [85:00] team to go and be an emissary to Google [85:03] and look for ways to use DeepMind. And [85:06] one of them is around data center [85:07] cooling. Very quickly, July of 2016, [85:10] Google announces a 40% reduction in the [85:13] energy required to cool data centers. I [85:16] mean, Google's got a lot of data [85:18] centers, a 40% energy reduction. I [85:20] actually talked with Jim Gao, who's a [85:21] friend of the show and actually led a [85:23] big part of this project. And I mean, it [85:26] was just the most obvious application of [85:29] neural networks inside of Google right [85:31] away. Pays for itself. [85:33] >> Yeah. Imagine that paid for the [85:34] acquisition pretty quickly there. [85:36] >> Yes. David, should we talk about AlphaGo [85:38] on this episode? [85:39] >> Yeah. Yeah. Yeah. [85:40] >> I watched the whole documentary that [85:42] Google produced about it. It's awesome. [85:44] This is actually something that you [85:45] would enjoy watching. Even if you're not [85:46] researching a podcast episode and you're [85:48] just looking to pull something up and [85:49] spend an hour or two, I highly recommend [85:51] it. It's on YouTube. It's the story of [85:54] how Deep Mind postacquisition from [85:56] Google trained a model to beat the world [85:59] Go champion at Go. And I mean, everyone [86:03] in the whole Go community coming in [86:05] thought there's no chance. This guy Lee [86:06] Seedall is so good that there's no way [86:09] that an AI could possibly beat him. It's [86:11] a fivegame thing. It just won the first [86:13] three games straight. I mean, completely [86:15] cleaned up and with inventive new [86:18] creative moves that no human has played [86:21] before. That's sort of the big crazy [86:23] takeaway. [86:23] >> There's a moment in one of the games, [86:25] right, where it makes a move of people [86:26] like, is that a mistake? Like that must [86:27] have just been an error. Yeah. Move 37. [86:29] Yeah. Yeah. And then a 100 moves later [86:32] it plays out and [86:33] >> that it was like completely genius. And [86:35] humans are now learning from Deep Mind's [86:38] strategy of playing the game and [86:40] discovering new strategies. A fun thing [86:43] for acquired listeners who are like, why [86:44] is it go? Go is so complicated compared [86:48] to chess. Chess has 20 moves that you [86:50] can make at the beginning of the game in [86:52] any given turn and then midame there's [86:53] like 30 to 40 moves that you could make. [86:56] Go on any given turn has about 200. And [86:59] so if you think cominatorilally, the [87:02] number of possible configurations of the [87:04] board is more than the number of atoms [87:06] in the universe. [87:09] >> That's a great Demis quote by the way. [87:10] >> Yeah. [87:11] >> And so he says, even if you took all the [87:12] computers in the world and ran them for [87:14] a million years as of 2017, that [87:16] wouldn't be enough compute power to [87:18] calculate all the possible variations. [87:21] So it's cool because it's a problem that [87:23] you can't brute force. You have to do [87:26] something like neural networks. And [87:27] there is this white space to be creative [87:30] and explore. And so it served as this [87:31] amazing breeding ground for watching a [87:35] neural network be creative against a [87:37] human. [87:38] >> Yeah. And of course it's totally in with [87:40] Demis' background and the DNA of the [87:42] company of playing games. You know Deus [87:44] was chess champion. And then after go [87:48] then they play Starcraft, right? [87:50] >> Oh really? I actually didn't know that. [87:51] >> Yeah. That was the next game that they [87:53] tackle was Starcraft, a real-time [87:55] strategy game against an opponent. And [87:57] that'll um come back up in a sec with [87:59] another opponent here in OpenAI. [88:03] >> Yes, David. But before we talk about the [88:05] creation of the other opponent, should [88:07] we thank another one of our friends here [88:08] at Acquired? [88:09] >> Yes, we should. [88:10] >> All right, listeners, we are here today [88:12] to tell you about a new friend of the [88:13] show we are very excited about. Work OS. [88:16] >> Yes. If you're building software that is [88:18] used in enterprises, you've probably [88:20] felt the pain of integrating things like [88:21] SSO, SCIM or SKIM, permissions, audit [88:26] logs, and all the other features that [88:28] are required by big customers. And if [88:30] you haven't felt this pain yet, just [88:32] wait until you get your first big [88:34] enterprise customer. And trust us, you [88:36] will. [88:36] >> Yes, Work OS turns these potential deal [88:39] blockers into simple drop-in APIs. And [88:42] while work OS had great product market [88:44] fit a few years ago with developers who [88:46] just want to save on some headache, they [88:48] really have become essential in the AI [88:50] era. [88:51] >> Yeah, I was shocked when they sent over [88:54] their latest customer list. Almost all [88:56] the big AI companies use work OS today [88:59] as the way that they've been able to [89:01] rapidly scale revenue so fast. Companies [89:03] like OpenAI, Anthropic, Cursor, [89:05] Perplexity, Sierra, Replet, Verscell, [89:08] hundreds of other AI startups all rely [89:11] on work OS as their O solution. So I [89:14] called the founder to ask why, and he [89:16] said it's basically two things. One, in [89:18] the AI era, these companies scale so [89:20] much faster that they need things like [89:23] authorization, authentication, and SSO [89:25] quickly to become enterprise ready and [89:28] keep up with customer demand even early [89:30] in life. Unlike older SAS companies of [89:33] yestery year, and two, unlike that world [89:36] where you could bring your own little [89:37] SAS product just for you and your little [89:39] team, these AI products reach deep into [89:41] your company's systems and data to [89:43] become the most effective. So, IT [89:44] departments are scrutinizing heavier [89:46] than ever to make sure that new products [89:48] are compliant before they can adopt [89:49] them. [89:50] >> Yeah, it's this kind of like second [89:51] order effect of the AI era that the days [89:54] of, oh, just swipe a credit card, bring [89:57] your own SAS solution for your product [89:58] team. You actually need to be enterprise [90:01] ready a lot sooner than you did before. [90:03] >> Yeah, it's not just about picking up [90:05] that big potential customer for the [90:06] revenue itself either. It's about doing [90:08] it so your competitors don't. Enterprise [90:10] readiness has become so table stakes for [90:12] companies no matter their stage. And [90:14] work OS is basically the weapon of [90:16] choice for the best software companies [90:18] to shortcut this process and get back to [90:20] focusing on what makes their beer taste [90:21] better, building the product itself. [90:23] >> Amen. Amen. So if you're ready to get [90:26] started with just a few lines of code [90:28] for SAML, SKIM, Arbok, SSO, [90:31] authorization, authentication, and [90:32] everything else to please IT admins and [90:34] their checklists, check out work OS. [90:37] It's the modern software platform to [90:39] make all this happen. That's works.com [90:42] and just tell them that Ben and David [90:44] sent you. [90:45] >> All right, David. So what are the second [90:48] order effects of Google buying DMIN? [90:51] Well, there's one person who is really, [90:55] really, really upset about this and [90:57] maybe two people if you include Mark [90:59] Zuckerberg, but Mark tends to play his [91:01] cards a little closer to the vest. Of [91:03] course, Elon Musk is very upset about [91:07] this acquisition. When Google buys [91:10] DeepMind out from under him, Elon goes [91:12] ballistic. As we said, Elon and Larry [91:15] had always been very close. And now [91:18] here's Google who Elon has already [91:22] started to sour on a little bit as he's [91:24] now trying to hire AI researchers. And [91:27] you've got Alan Eustace flying around [91:28] the world sucking up all of the AI [91:30] researchers into Google [91:33] and Elon's invested in DeepMind wanted [91:35] to bring Deep Mind into his own AI team [91:37] at Tesla and gone out from under him. [91:41] So, this leads to one of the most [91:44] fateful dinners in Silicon Valley's [91:47] history, organized in the summer of 2015 [91:52] at the Rosewood Hotel on Sand Hill Road. [91:55] Of course, where else would you do a [91:57] dinner in Silicon Valley, but the [92:00] Rosewood, by two of the most leading [92:03] figures in the valley at the time, Elon [92:06] Musk and Sam Alman. Sam of course being [92:11] president of Y Combinator at the time. [92:15] So what is the purpose of this dinner? [92:18] They are there to make a pitch to all of [92:22] the AI researchers that Google and to a [92:26] certain extent Facebook have sucked up [92:28] and basically created this duopoly [92:31] status on. [92:32] >> Again, Google's business model and [92:34] Facebook's business model. feed [92:36] recommenders or these classifiers turn [92:39] out to be unbelievably valuable. So they [92:41] can, it's funny in hindsight saying [92:43] this, pay tons of money to these people, [92:46] >> tons of money, like millions of dollars, [92:49] >> take them out of academia and put them [92:52] into their dirty capitalist research [92:54] labs inside the companies [92:56] >> selling advertising. [92:58] >> Yes. [92:59] >> How dirty could you be? And the question [93:01] and the pitch that Elon and Sam have for [93:05] these researchers gathered at this [93:07] dinner is what would it take to get you [93:12] out of Google for you to leave? And the [93:15] answer they go around the table from [93:17] almost everybody is nothing. You can't. [93:21] Why would we leave? We're getting paid [93:22] way more money than we ever imagined. [93:24] Many of us get to keep our academic [93:27] positions and affiliations [93:30] and we get to hang out here at Google [93:33] >> with each other [93:34] >> with each other. [93:35] >> Iron sharpens iron. These are some of [93:37] the best minds in the world getting to [93:38] do cutting edge research with enormous [93:40] amount of resources and hardware at [93:42] their disposal. It's amazing. [93:44] >> It's the best infrastructure in the [93:45] world. We've got Jeff Dean here. There [93:48] is nothing you could tell us that would [93:52] cause us to leave Google. [93:54] Except there's one person who is [93:57] intrigued. And to quote from an amazing [94:01] Wired article at the time by Cade Mets, [94:03] who would later write Genius Makers, [94:05] right? [94:05] >> Yep. Exactly. Quote is the trouble was [94:08] so many of the people most qualified to [94:10] solve these problems were already [94:12] working for Google. And no one at the [94:13] dinner was quite sure that these [94:14] thinkers could be lured into a new [94:15] startup even if Musk and Maltman were [94:17] behind it. But one key player was at [94:21] least open to the idea of jumping ship. [94:24] And then there's a quote from that key [94:25] player. I felt like there were risks [94:28] involved, but I also felt like it would [94:30] be a very interesting thing to try. [94:32] >> It's the most Ilia quote of all time. [94:34] the most Ilia quote of all time because [94:36] that person was Ilia Sutskkever of [94:39] course of AlexNet and DNN research and [94:42] Google and about to become founding [94:46] chief scientist of Open AI. So the pitch [94:49] that Elon and Sam are making to these [94:51] researchers is let's start a new [94:55] nonprofit AI research lab where we can [94:58] do all this work out in the open. You [95:00] can publish free of the forces of [95:03] Facebook and Google and independent of [95:05] their control. [95:06] >> Yes, you don't have to work on products. [95:08] You can only work on research. You can [95:11] publish your work. It will be open. It [95:13] will be for the good of humanity. All of [95:15] these incredible advances, this [95:18] intelligence that we believe is to come [95:21] will be for the good of everyone, not [95:24] just for Google and Facebook. [95:25] >> And for one of the researchers, it [95:27] seemed too good to be true. So, they [95:28] basically weren't doing it cuz they [95:30] didn't think anyone else would do it. [95:31] It's sort of an activation energy [95:32] problem where once Ilia said, "Okay, I'm [95:35] in." And once he said, "I'm in, by the [95:37] way," Google came back with a big [95:39] counter, something like double the [95:41] offer. And I think it was delivered from [95:42] Jeff Dean personally, and Ilia said, [95:44] "Nope, I'm doing this." That was massive [95:47] for getting the rest of the top [95:49] researchers to go with him. [95:51] >> And it was nowhere near all of the top [95:52] researchers who left Google to do this, [95:54] but it was enough. It was a group of [95:57] seven or so researchers who left Google [96:00] and joined Elon and Sam and Greg [96:02] Brockman from Stripe who came over to [96:05] create open AI because that was the [96:09] pitch. We're all going to do this in the [96:11] open. [96:11] >> And that's totally what it was. [96:12] >> It totally is what it was. And the [96:14] stated mission of OpenAI was to quote [96:18] advance digital intelligence in the way [96:21] that is most likely to benefit humanity [96:23] as a whole unconstrained by a need to [96:26] generate financial return which is fine [96:30] as long as the thing that you need to [96:33] fulfill your mission doesn't take tens [96:36] of billions of dollars. [96:38] >> Yes. [96:38] >> So here's how they would fund it. [96:40] Originally there was a billion dollars [96:43] pledged. [96:44] >> Yes. [96:44] >> And that came from famously Elon Musk, [96:47] Sam Alman, Reed Hoffman, Jessica [96:51] Livingston, who I think most people [96:52] don't realize was part of that initial [96:54] trunch, and Peter Teal. [96:57] >> Yep. [96:58] >> Founders Fund of course would go on to [96:59] put massive amounts of money into OpenAI [97:01] itself later as well. The funny thing is [97:05] it was later reported that a billion [97:07] dollars was not actually collected. Only [97:09] about 130 million of it was actually [97:11] collected to fund this nonprofit. And [97:14] for the first few years that was plenty [97:16] for the type of research they were [97:17] doing, the type of compute they needed. [97:19] >> Most of that money was going to paying [97:21] salaries to the researchers. Not as much [97:24] as they could make at Google and [97:25] Facebook, but still million or$2 million [97:27] for these folks, [97:29] >> right? And Yeah. So that really worked [97:31] until it really didn't. [97:32] >> Yeah. So David, what were they doing in [97:34] the early days? [97:36] >> Well, in the first days, it was all [97:39] hands-on deck recruiting and hiring [97:40] researchers. And there was the initial [97:42] crew that came over and then pretty [97:45] quickly after that in early 2016, they [97:48] get a big big win when Dario Amade [97:52] leaves Google, comes over, joins Ilia [97:55] and crew at OpenAI [97:58] dream team, you know, assembling here. [98:00] And was he on Google Brain before this? [98:03] >> He was on Google Brain. Yep. And he [98:05] along with Ilia would run large parts of [98:08] OpenAI for the next couple years before [98:10] of course leaving to start Anthropic. [98:13] But we're still a couple years away from [98:17] Anthropic, Clawud, Chat GPT, Gemini, [98:21] everything today. for at least the first [98:24] year or two. Basically, the plan at [98:27] OpenAI is let's look at what's happening [98:29] at DeepMind and show the research [98:31] community that we can do as a new lab do [98:35] the same incredible things that they're [98:37] doing and maybe even do them better. [98:38] >> Is that why it looks so game like and [98:40] game focused? [98:41] >> Yes. Yes. So, they started building [98:44] models to play games. Famously, the big [98:46] one that they do is Dota 2, Defense of [98:48] the Ancients 2, the uh massively online [98:51] battle arena video game. They're like, [98:53] "All right, well, Deep Mind, you're [98:54] playing Starcraft. Well, we'll go play [98:56] Dota 2. That's even more complex, more [98:59] real time." [99:00] >> And similar to the emergent properties [99:02] of Go, the game would devise unique [99:05] strategies that you wouldn't see humans [99:07] trying. So, it clearly wasn't humans [99:09] coded their favorite strategies and [99:11] rules in, it was emergent. [99:13] >> Yeah, [99:14] >> they did other things. They had a [99:15] product called Universe which was around [99:17] training computers to play thousands of [99:19] games from Atari games to open world [99:21] games like Grand Theft Auto. They had [99:23] something where they were teaching a [99:26] model how to do a Rubik's cube. And so [99:29] it was a diverse set of projects that [99:32] didn't seem to coales around one of [99:35] these is going to be the big thing. [99:37] >> Yeah. It was research stuff. It was what [99:40] Deepmind was doing. [99:41] >> Yeah. It was like a university research. [99:43] It was like Deep Mind. And if you think [99:46] back to Elon being an investor in Deep [99:48] Mind, being really upset about Google [99:50] acquiring it out from under him makes [99:53] sense. [99:54] >> And I think Elon deserves a lot of [99:55] credit for having his name and his time [99:58] attached to OpenA at the beginning. A [100:01] lot of the big heavy hitter recruiting [100:04] was Elon throwing his weight behind [100:06] this. I'm willing to take a chance. [100:08] >> Absolutely. [100:09] >> Okay. So that's what's going on over at [100:10] OpenAI doing a lot of Deep Mind like [100:13] stuff. Bunch of projects, not one single [100:17] obvious big thing they're coalesing [100:19] around. It's not chat GBT time. Let's [100:21] put it that way. Let's go back to Google [100:24] cuz last we sort of checked in on them. [100:25] Yeah, they bought Deep Mind, but they [100:26] had their talent rated. And I don't want [100:29] you to get the wrong impression about [100:30] where Google is sitting just because [100:31] some people left to go to OpenAI. So [100:34] back in 2013 when Alex Kashesky arrives [100:37] at Google with Jeff Hinton and [100:39] Ilaskever, [100:40] he was shocked to discover that all [100:42] their existing machine learning models [100:44] were running on CPUs. People had asked [100:47] in the past for GPUs since machine [100:48] learning workloads were well suited to [100:50] run in parallel, but Google's [100:51] infrastructure team had pushed back and [100:53] said the added complexity and expanding [100:55] and diversifying the fleet. Let's keep [100:57] things simple. That doesn't seem [100:58] important for us. [100:59] >> We're a CPU shop here. [101:01] >> Yes. And so to quote from Genius Makers, [101:04] in his first days at the company, he [101:05] went out and bought a GPU machine, this [101:07] is Alex, from a local electronic store, [101:10] stuck it in the closet down the hall [101:11] from his desk, plugged it into the [101:13] network, and started training his neural [101:14] networks on this lone piece of hardware [101:17] just like he did in academia, except [101:20] this time Google's paying for the [101:21] electricity. Obviously, one GPU was not [101:24] sufficient, especially as more Googlers [101:26] wanted to start using it, too. And Jeff [101:28] Dean and Alan Eustace had also come to [101:30] the conclusion that disbelief while [101:32] amazing had to be rearchitected to run [101:35] on GPUs and not CPUs. So spring of 2014 [101:39] rolls around. Jeff Dean and John Gandra [101:43] >> who we haven't talked about this [101:44] episode. [101:44] >> Yeah, JG. [101:46] >> Yes, you might be wondering, wait, isn't [101:47] that the Apple guy? Yes, he went on to [101:49] be Apple's head of AI who at this point [101:52] in time was at Google and oversaw Google [101:54] Brain 2014. They sit down to make a plan [101:57] for how to actually formally put GPUs [102:00] into the fleet of Google's data centers, [102:02] which is a big deal. It's a big change, [102:04] but they're seeing enough reactions to [102:07] neural networks that they know to do [102:08] this. [102:08] >> Yeah. After Alex, it's just a matter of [102:10] time. [102:11] >> Yeah. So, they settle on a plan to order [102:13] 40,000 GPUs [102:16] from Nvidia. [102:17] >> Yeah, of course. Who else are you going [102:19] to order them from? [102:20] >> For a cost of $130 million. [102:23] That's a big enough price tag that the [102:26] request gets elevated to Larry Page who [102:28] personally approves it even though [102:30] finance wanted to kill it because he [102:32] goes look the future of Google is deep [102:34] learning. As an aside, let's look at [102:36] Nvidia at the time. This is a giant [102:40] giant order. Their total revenue was $4 [102:43] billion. This is one order for 130 [102:47] million. [102:47] >> I mean Nvidia is primarily consumer [102:49] graphics card company at this point. [102:51] >> Yes. and their market cap is $10 [102:52] billion. [102:54] It's almost like Google gave Nvidia a [102:58] secret that hey, not only does this work [103:00] in research like the imageet [103:02] competition, but neural networks are [103:03] valuable enough to us as a business to [103:06] make a hundred plus million dollar [103:08] investment in right now, no questions [103:11] asked. We got to ask Jensen about this [103:13] at some point. This had to be a tell. [103:16] >> This had to really give Nvidia the [103:19] confidence. Oh, we should way forward [103:21] invest on this being a giant thing in [103:23] the future. So, all of Google wakes up [103:26] to this idea. They start really putting [103:28] it into their products. Google Photos [103:30] happened. Gmail starts offering typing [103:32] suggestions. David, as you pointed out [103:35] earlier, Google's giant Adwords business [103:37] started finding more ways to make more [103:39] money with deep learning. In particular, [103:41] when they integrated it, they could [103:42] start predicting what ads people would [103:44] click in the future. And so Google [103:46] started spending hundreds of millions [103:48] more on GPUs on top of that 130 million, [103:51] but very quickly paying it back from [103:54] their ad system. So it became more and [103:56] more of a no-brainer to just buy as many [103:57] GPUs as they possibly could. But once [104:01] neural nets started to work, anyone [104:04] using them, especially at Google scale, [104:06] kind of had this problem. Well, now we [104:08] need to do giant amounts of matrix [104:09] multiplications anytime anybody wants to [104:12] use one. The matrix multiplications are [104:14] effectively how you do that propagation [104:16] through the layers of the neural [104:18] network. So you sort of have this [104:20] problem. [104:20] >> Yes, totally. There's the inefficiency [104:23] of it, but then there's also the [104:24] business problem of wait a minute, it [104:27] looks like we're just going to be [104:28] shipping hundreds of millions, soon to [104:30] be billions of dollars over to Nvidia [104:32] every year for the foreseeable future. [104:35] >> Right? So there's this amazing moment [104:37] right after Google rolls out speech [104:39] recognition, their latest use case for [104:41] neural nets just on Nexus phones because [104:45] again they don't have the infrastructure [104:46] to support it on all Android phones. it [104:49] becomes a super popular feature and Jeff [104:51] Dean does the math and figures out if [104:54] people use this for I don't know call it [104:55] three minutes a day and we roll it out [104:57] to all billion Android phones we're [105:00] going to need twice the number of data [105:02] centers that we currently have across [105:05] all of Google just to handle it [105:07] >> just for this feature yeah [105:09] >> there's a great quote where Jeff goes to [105:11] Holtzel and goes we need another Google [105:17] or David, as you were hinting at, the [105:20] other option is we build a new type of [105:23] chip customized for just our particular [105:27] use case. [105:28] >> Yep. Matrix multiplication, tensor [105:31] multiplication, a tensor processing [105:33] unit, you might say. [105:34] >> Ah, yes. Wouldn't that be nice? So, [105:37] conveniently, Jonathan Ross, who's an [105:39] engineer at Google, has been spending [105:41] his 20% time at this point in history [105:43] working on an effort involving FPGAAS. [105:45] These are essentially expensive but [105:47] programmable chips that yield really [105:49] fantastic results. So they decide to [105:51] create a formal project to take that [105:54] work combine it with some other existing [105:55] work and build a custom ASIC or an [105:57] application specific integrated circuit. [106:00] So enter David as you said the tensor [106:02] processing unit made just for neural [106:06] networks that is far more efficient from [106:08] GPUs at the time with the trade-off that [106:11] you can't really use it for anything [106:13] else. It's not good for graphics [106:14] processing. It's not good for lots of [106:16] other GPU workloads, just matrix [106:19] multiplication and just neural networks, [106:22] but it would enable Google to scale [106:24] their data centers without having to [106:26] double their entire footprint. So the [106:28] big idea behind the TPU, if you're [106:30] trying to figure out like what was the [106:31] core insight, they use reduced [106:33] computational precision. So it would [106:35] take numbers like 4586.8272 [106:39] and round it just to 4586.8 [106:43] or maybe even just 4586 with nothing [106:46] after the decimal point. And this sounds [106:48] kind of counterintuitive at first. Why [106:50] would you want less precise rounded [106:53] numbers for this complicated math? The [106:55] answer is efficiency. If you can do the [106:57] heavy lifting in your software [106:59] architecture or what's called [107:00] quantization to account for it, you can [107:03] store information as less precise [107:06] numbers, then you can use the same [107:08] amount of power and the same amount of [107:09] memory and the same amount of [107:10] transistors on a chip to do far more [107:12] calculations per second. So you can [107:14] either spit out answers faster or use [107:16] bigger models. The whole thing is quite [107:18] clever behind the TPU. M [107:21] >> the other thing that has to happen with [107:22] the TPU is it needs to happen now cuz [107:25] it's very clear speech to text is a [107:27] thing. It's very clear some of these [107:29] other use cases at Google. [107:30] >> Yeah. Demand for all of this stuff [107:32] that's coming out of Google Brain is [107:34] through the roof immediately. [107:35] >> Right. And we're not even two LLMs yet. [107:37] It's just like everyone sort of expects [107:39] some of this whether it's computer [107:41] vision in photos or speech recognition [107:43] like it's just becoming a thing that we [107:45] expect and it's going to flip Google's [107:46] economics upside down if they don't have [107:48] it. So the TPU was designed, verified, [107:51] built, and deployed into data centers in [107:54] 15 months. [107:55] >> Wow. [107:56] >> It was not like a research project that [107:58] could just happen over several years. [107:59] This was like a hair on fire problem [108:00] that they launched immediately. One very [108:03] clever thing that they did was a they [108:05] used the FPGAAS as a stop gap. So even [108:08] though they were like too expensive on a [108:09] unit basis, they could get them out as a [108:11] test fleet and just make sure all the [108:12] math worked before they actually had the [108:14] AS6 printed at I don't know if it was a [108:16] TSMC, but you know, fabbed and ready. [108:18] The other thing they did is they fit the [108:20] TPU into the form factor of a hard [108:22] drive, so it could actually slot into [108:25] the existing server racks. You just pop [108:28] out a hard drive and you pop in a TPU [108:30] without needing to do any physical [108:32] rearchitecture. [108:33] >> Wow, that's amazing. That's the most [108:36] googly infrastructure story [108:39] >> since the corkboards. [108:40] >> Exactly. Also, all of this didn't happen [108:43] in Mountain View. It was at a Google [108:44] satellite office in Madison, Wisconsin. [108:47] >> Whoa. [108:48] >> Yes. [108:49] >> Why Madison, Wisconsin? [108:51] >> There was a particular professor out of [108:52] the university and there was a lot of [108:54] students that they could recruit from [108:56] and [108:56] >> Wow. [108:57] >> Yeah. I mean, it was probably them or [108:58] Epic. Where are you going to go work? [109:01] >> Yeah. [109:02] >> Wow. They also then just kept this a [109:04] secret, [109:05] >> right? Why would you tell anybody about [109:06] this? [109:07] >> Because it's not like they're offering [109:08] these in Google Cloud, at least at [109:10] first, and why would you want to tell [109:12] the rest of the world what you're doing? [109:13] So, the whole thing was a complete [109:15] secret for at least a year before they [109:17] announced it at Google IO. So, really [109:19] crazy. The other thing to know about the [109:21] TPUs is they were done in time for the [109:23] AlphaGo match. So, that match ran on a [109:26] single machine with four TPUs in Google [109:29] Cloud. And once that worked, obviously [109:30] that gave Google a little bit of extra [109:32] confidence to go really, really rip [109:34] production. So that's the TPU. V1 by all [109:38] accounts was not great. They're on V7 or [109:41] V8 now. It's gotten much better. TPUs [109:43] and GPUs look a lot more similar than [109:45] they used to than they've sort of [109:47] adopted features from each other. But [109:49] today, Google, it's estimated, has 2 to [109:52] 3 million TPUs. For reference, Nvidia [109:55] shipped, people don't know for sure, [109:56] somewhere around 4 million GPUs last [109:59] year. So people talk about AI chips like [110:01] it's this just oh one horse race with [110:03] Nvidia. Google has like an almost Nvidia [110:06] scale internal thing making their own [110:09] chips at this point for their own and [110:11] for Google Cloud customers. The TPU is a [110:14] giant deal in AI in a way that I think a [110:17] lot of people don't realize. [110:18] >> Yep. This is one of the great ironies [110:20] and maddening things to OpenAI and Elon [110:24] Musk is that OpenAI gets founded in 2015 [110:27] with the goal of, hey, let's shake all [110:29] this talent out of Google and level the [110:32] playing field and Google just [110:34] accelerates, [110:35] >> right? They also build TensorFlow. [110:37] That's the framework that Google Brain [110:39] built to enable researchers to build and [110:41] train and deploy machine learning [110:43] models. And they built it in such a way [110:45] that it doesn't just have to run on [110:46] TPUs. super portable without any [110:49] rewrites to run on GPUs or even CPUs [110:51] too. So this would replace the old disc [110:54] belief system and kind of be their [110:56] internal and external framework for [110:58] enabling ML researchers going forward. [111:01] So somewhat paradoxically during these [111:04] years after the founding of Open AI, [111:08] yes, some amazing researchers are [111:10] getting siphoned off from Google and [111:12] Google Brain, but Google Brain is also [111:15] firing on all cylinders during this time [111:17] frame, [111:18] >> delivering on the business purposes for [111:20] Google left and right. [111:21] >> Yes. And pushing the state-of-the-art [111:24] forward in so many areas. And then in [111:27] 2017, a paper gets published from eight [111:30] researchers on the Google brain team [111:32] kind of quietly. These eight folks were [111:36] obviously very excited about the paper [111:39] and what it described and the [111:41] implications of it and they thought it [111:43] would be very big. Google itself, uh, [111:47] cool, this is like the next iteration of [111:49] our language model work. Great. [111:51] >> Which is important to us. But are we [111:54] sure this is the next Google? No. [111:57] >> No. There are a whole bunch of other [111:58] things we're working on that seem more [112:00] likely to be the next Google. But this [112:04] paper and its publication would actually [112:06] be what gave OpenAI the opportunity [112:09] >> to build the next Google [112:11] >> to grab the ball and run with it and [112:13] build the next Google because this is [112:15] the transformer paper. [112:16] >> Okay. So where did the transformer come [112:18] from? like what was the latest thing [112:20] that language models had been doing at [112:22] Google? So coming out of the success of [112:25] Fran Ox's work on Google Translate and [112:28] the improvements that happened there [112:30] >> in like the late 2000sish 2007 [112:32] >> yeah mid to late 2000s they keep [112:35] iterating on translate and then once [112:38] Jeff Hitten comes on board and AlexNet [112:41] happens they switch over to a neural [112:43] networkbased language model for [112:45] translate which was dramatically better [112:48] and like a big crazy cultural thing [112:51] because you've got these researchers [112:53] parachuting in again led by Jeff Dean [112:55] saying I'm pretty sure our neural [112:59] networks can do this way better than the [113:02] classic methods that we've been using [113:04] for the last 10 years. What if we take [113:06] the next several months and do a proof [113:08] of concept? They end up throwing away [113:10] the entire old codebase and just [113:12] completely wholesale switching to this [113:14] neural network. There's actually this [113:15] great New York Times magazine story that [113:18] ran in 2016 about it. And I remember [113:20] reading the whole thing with my jaw on [113:22] the floor. Like, wow. Neural networks [113:24] are a big effing deal. And this was the [113:26] year before the Transformer paper would [113:28] come out. [113:29] >> Before the Transformer paper. Yes. So, [113:32] they do the rewrite of Google Translate, [113:34] make it based on recurrent neural [113:35] networks, which were state-of-the-art at [113:37] that point in time. And it's a big [113:39] improvement. But as teams within Google [113:42] Brain and Google Translate keep working [113:44] on it, there's some limitations. And in [113:47] particular, a big problem was that they [113:48] quote unquote forgot things too quickly. [113:52] I don't know if it's exactly the right [113:54] analogy, but you might say in sort of [113:56] like today's transformer world speak, [113:58] you might say that their context window [113:59] was pretty short. As these language [114:02] models progressed through text, they [114:04] needed to sort of remember everything [114:07] they had read so that when they need to [114:09] change a word later or come up with the [114:11] next word, they could have a whole [114:14] memory of the body of text to do that. [114:17] >> So, one of the ways that Google tries to [114:20] improve this is to use something called [114:23] long short-term memory networks or LSTMs [114:28] as the acronym that people use for this. [114:30] And basically what LSTMs do is they [114:33] create a persistent or long [114:36] shortterm memory. [114:39] You got to use your brain a little bit [114:40] here for the model so that it can keep [114:43] context as it's going through a whole [114:45] bunch of steps. [114:46] >> And people were pretty excited about [114:47] LSTMs at first. [114:48] >> People are thinking like, oh, LSTMs are [114:52] what are going to take language models [114:54] and large language models mainstream, [114:57] >> right? And indeed in 2016 they [115:00] incorporated into Google Translate these [115:02] LSTMs. It reduces the error rate by 60%. [115:06] Huge jump. Yep. [115:08] >> The problem with LSTMs though, they were [115:10] effective but they were very [115:12] computationally intensive and they [115:14] didn't parallelize that great. All the [115:18] efforts that are coming out of Alex Net [115:20] and then the TPU project of [115:22] parallelization. This is the future. [115:24] this is how we're going to make AI [115:26] really work. LSTMs are a bit of a [115:30] roadblock here. Yes. So, a team within [115:34] Google Brain starts searching for a [115:37] better architecture that also has the [115:40] attractive properties of LSTMs that it [115:43] doesn't forget context too quickly, but [115:46] can parallelize and scale better [115:49] >> to take advantage of all these new [115:50] architectures. [115:52] >> Yes. And a researcher named Jakob [115:55] Oscarite had been toying around with the [115:58] idea of broadening the scope of quote [116:01] unquote attention in language [116:04] processing. What if rather than focusing [116:07] on the immediate words, instead what if [116:11] you told the model, hey, pay attention [116:14] to the entire corpus of text, not just [116:17] the next few words. Look at the whole [116:18] thing. And then based on that entire [116:21] context and giving your attention to the [116:23] entire context, give me a prediction of [116:26] what the next translated word should be. [116:30] Now, by the way, this is actually how [116:32] professional human translators translate [116:34] text. You don't just go word by word. I [116:37] actually took a translation class in [116:39] college, which was really fun. You read [116:40] the whole thing of the original in the [116:42] original language. you get and [116:44] understand the context of what the [116:47] original work is and then you go back [116:50] and you start to translate it with the [116:52] entire context of the passage in mind. [116:55] >> So it would take a lot of computing [116:57] power for the model to do this but it is [117:01] extremely parallelizable. So Yakob [117:04] starts collaborating with a few other [117:06] people on the brain team. They get [117:07] excited about this. They decide that [117:09] they're going to call this new technique [117:12] the transformer because one, that is [117:15] literally what it's doing. It's taking [117:16] in a whole chunk of information, [117:19] processing, understanding it, and then [117:21] transforming it. And B, they also love [117:24] transformers as kids. That's not not why [117:27] they named it the transformer. [117:29] >> And it's taking in the giant corpus of [117:32] text and storing it in a compressed [117:33] format. Right. [117:34] >> Yeah. I bring this up because that is [117:37] exactly how you pitched the micro [117:39] kitchen conversation with Nom Shazir in [117:41] 2000 2001 17 years earlier who is a [117:45] co-author on this paper. [117:46] >> Yes. Well, so speaking of Nam Shazir, he [117:50] learns about this project and he [117:52] decides, hey, I've got some experience [117:54] with this. This sounds pretty cool. [117:57] LSTMs definitely have problems. This [117:58] could be promising. I'm going to jump in [118:01] and work on it with these guys. [118:05] And it's a good thing he did because [118:06] before Gnome joined the project, they [118:08] had a working implementation of the [118:10] transformer, but it wasn't actually [118:12] producing any better results than LSTMs. [118:16] Gnome joins the team, basically pulls a [118:19] Jeff Dean, rewrites the entire codebase [118:22] from scratch, and when he's done, the [118:25] transformer now crushes [118:29] the LSTMbased [118:31] Google Translate solution. And it turns [118:34] out that the bigger they make the model, [118:37] the better the results get. It seems to [118:39] scale really, really, really well. [118:41] Steven Levy wrote a piece in Wired about [118:44] the history of this. And there are all [118:45] sorts of quotes from the other members [118:47] of the team just littered all over this [118:49] piece with things like Gnome is a [118:51] magician. Gnome is a wizard. Gnome took [118:54] the idea and came back and said, "It [118:56] works now." [119:00] Yeah. And you wonder why Noom and Jeff [119:02] Dean are the ones together working on [119:04] the next version of Gemini now. [119:06] >> Yes. Noom and Jeff Dean are definitely [119:08] two peas in a pod here. [119:10] >> Yes. So we talked to Greg Curado from [119:14] Google Brain, one of the founders of [119:15] Google Brain, and it was a really [119:17] interesting conversation because he [119:19] underscored how elegant the transformer [119:21] was. And he said it was so elegant that [119:23] people's response was often, "This can't [119:26] work. It's too simple. Transformers are [119:28] barely a neural network architecture, [119:30] >> right? It was another big change from [119:34] the AlexNet Jeff Hinton lineage neural [119:37] networks. [119:38] >> Yeah, it actually has changed the way [119:40] that I look at the world cuz he pointed [119:42] out that in nature, this is Greg, the [119:44] way things usually work is the most [119:47] energyefficient way they could work. [119:49] almost from an evolution perspective [119:51] that the most simple, elegant solutions [119:54] are the ones that survive because they [119:57] are the most efficient with their [119:58] resources. And you can kind of port this [120:00] idea over to computer science, too, that [120:03] he said he's developed a pattern [120:05] recognition inside of the research lab [120:07] to realize that you're probably on to [120:09] the right solution when it's really [120:11] simple and really efficient versus a [120:14] complex idea. [120:15] >> Mhm. [120:16] >> It's very clever. It's I think it's very [120:18] true. You know how when you sit around [120:19] and you have a thorny problem and you [120:21] debate and you whiteboard and you come [120:22] up with all and then you're like, "Oh my [120:23] god, oh my god, it's so simple." And [120:25] that ends up being the right answer. [120:26] >> Yeah. There's an elegance to the [120:29] transformer. [120:30] >> Yes. And that other thing that you [120:32] touched on there, this is the beginning [120:35] of the modern AI, just feed it more [120:39] data. The famous piece, the bitter [120:41] lesson by Rich Sutton, wouldn't be [120:42] published until 2019. For anyone who [120:45] hasn't read it, it's basically we always [120:47] think as AI researchers, we're we're so [120:48] smart and our job is to come up with [120:50] another great algorithm, but effectively [120:52] in every field from language to computer [120:54] vision to chess, you just figure out a [120:58] scalable architecture and then the more [121:00] data wins. Just these infinitely [121:03] scaling, [121:04] >> more data, more compute, better results. [121:06] >> Yes. And this is really the start of [121:08] when that starts to be like oh we have [121:10] found the scalable architecture that [121:12] will go at so far for I don't know close [121:15] to a decade of just more data in more [121:18] energy more compute better results. [121:20] >> So the team and noom like yo this thing [121:24] has a lot a lot of potential. [121:27] >> This is more than better translate. We [121:28] can really apply this. [121:30] >> Yeah this is going to be more than [121:31] better Google translate. The rest of [121:34] Google though definitely slower to wake [121:37] up to the potential. [121:38] >> They build some stuff within a year. [121:40] They build BERT, the large language [121:43] model. [121:43] >> Yes, absolutely true. It is a false [121:46] narrative out there that Google did [121:47] nothing with the transformer after the [121:49] paper was published. They actually did a [121:51] lot. [121:52] >> In fact, BERT was one of the first LLMs. [121:54] >> Yes, they did a lot with [121:56] transformer-based large language models [121:58] after the paper came out. What they [122:00] didn't do was treat it as a wholesale [122:04] technology platform change, [122:05] >> right? They were doing things like BERT [122:07] and uh MUM, this other model, you know, [122:09] they could work it into search results [122:10] quality. And I think that did [122:11] meaningfully move the needle even though [122:13] Google wasn't bragging about it and [122:14] talking about it. They got better at [122:16] query comprehension. They were working [122:17] it into the core business just like [122:19] every other time Google Brain came up [122:22] with something great. [122:22] >> Yep. So, in perhaps one of the greatest [122:26] decisions ever for value to humanity and [122:30] maybe one of the worst corporate [122:31] decisions ever for Google, Google allows [122:34] this group of eight researchers to [122:36] publish the paper under the title [122:38] attention is all you need. Obviously, a [122:40] nod to the classic Beatles song about [122:42] love. As of today in 2025, [122:47] this paper has been cited over [122:51] 173,000 times in other academic papers, [122:55] making it currently the seventh most [122:58] cited paper of the 21st century. And I [123:02] think all of the other papers above it [123:04] on the list have been out much longer. [123:07] Wow. And also of course within a couple [123:10] years all eight authors of the [123:13] transformer paper had left Google to [123:16] either start or join AI startups [123:19] including open AAI. Brutal. And of [123:23] course Noom starting Character AI which [123:26] what are we calling it? Aquisition. He [123:29] would end up back at Google via some [123:31] strange licensing and IP and hiring [123:33] agreement on the few billion dollars [123:36] order. very very expensive mistake on [123:39] Google's part. It [123:40] >> is fair to say that 2017 begins the [123:42] 5-year period of Google not sufficiently [123:45] seizing the opportunity that they had [123:47] created [123:47] >> with the transformer. Yes. So speaking [123:50] of seizing opportunities, what is going [123:51] on at OpenAI during this time? [123:53] >> And does anyone think the transformer is [123:55] a big deal over there? [123:57] >> Yes. Yes, they did. But here's where [123:59] history gets really, really crazy. Right [124:02] after Google publishes the Transformer [124:05] paper in September of 2017, [124:10] Elon gets really, really fed up with [124:14] what's going on at OpenAI. [124:16] >> There's like seven different strategies, [124:18] are we doing video games? Are we doing [124:20] competitions? What's the plan? [124:23] >> What is happening here? As best as I can [124:25] tell, all you're doing is just trying to [124:27] copy Deep Mind. Meanwhile, I'm here [124:31] building SpaceX and Tesla. Self-driving [124:33] is becoming more and more clear as [124:35] critical to the future of Tesla. I need [124:38] AI researchers here, and I need great AI [124:42] advancements to come out to help what [124:44] we're doing at Tesla. Open AAI isn't [124:47] cutting it. So, he makes an ultimatum to [124:51] Sam and the rest of the OpenAI board. He [124:54] says, "I'm happy to take full control of [124:58] OpenAI and we can merge this into Tesla. [125:02] I don't even know how that would be [125:03] possible to merge a nonprofit into [125:06] Tesla." [125:06] >> But in Elon Land, if he takes over as [125:08] CEO of Open AI, it almost doesn't [125:11] matter. We're just treating it as if [125:12] it's the same company anyway, just like [125:14] we do with the deals with all of my [125:15] companies, [125:16] >> right? or he's out completely along with [125:20] all of his funding. And Sam and the rest [125:23] of the board are like, "No." [125:25] >> And as we know now, they're sort of [125:26] calling capital into the business. It's [125:28] not like they actually got all the cash [125:29] up front, [125:30] >> right? So they're only 130 millionish [125:32] into the billion dollars of commitment. [125:36] They don't reach a resolution and by [125:38] early 2018, Elon is out along with him [125:43] the main source of OpenAI's funding. So [125:47] either this is just a really really [125:49] really bad misjudgment by Elon [125:54] or the sort of panic that this throws [125:57] Open AI into is the catalyst that makes [126:02] them reach for the transformer and say, [126:06] "All right, we got to figure things out. [126:08] Necessity is the mutter of invention. [126:10] Let's go for it." [126:10] >> It's true. I don't know if during this [126:13] personal tension between Elon and Sam if [126:17] they had already decided to go all in on [126:19] Transformers or not because the thing [126:21] you very quickly get to if you decide [126:25] transformers language models were going [126:27] all in on that. You do quickly realize [126:30] you need a bunch of data, you need a [126:31] bunch of compute, you need a bunch of [126:33] energy, and you need a bunch of capital. [126:34] And so if your biggest backer is walking [126:37] away, the 3D chess move is, "Oh, we got [126:40] to keep him because we're about to pivot [126:42] the company and we need his capital for [126:45] this big pivot we're doing." The 4D [126:47] chess is if he walks away, maybe I can [126:50] turn it into a for-profit company and [126:54] then raise money into it and eventually [126:58] generate enough profits to fund this [127:00] extremely expensive new direction we're [127:02] going in. I don't know which of those it [127:05] was. [127:05] >> Yeah, I don't know either. I suspect the [127:08] truth is it's sum of both. [127:10] >> Yes. But either way, how nuts is it that [127:12] a these things happened at the same time [127:14] and b the company wasn't burning that [127:17] much cash and then they decided to go [127:19] allin on we need to do something so [127:22] expensive that we need to be a [127:23] for-profit company in order to actually [127:26] achieve this mission cuz it's just going [127:28] to require hundreds of billions of [127:29] dollars for the far foreseeable future. [127:31] >> Yep. So in June of 2018, OpenAI releases [127:35] a paper describing how they have taken [127:40] the transformer and developed a new [127:42] approach of pre-training them on very [127:47] large amounts of general text on the [127:50] internet and then fine-tuning that [127:53] general pre-training to specific use [127:55] cases. And they also announced that they [127:57] have trained and run the first proofof [128:01] concept model of this approach which [128:04] they are calling GPT1 [128:09] generatively pre-trained transformer [128:12] version one [128:14] >> which we should say is right around the [128:16] same time as BERT and right around the [128:17] same time as another large language [128:19] model based on the transformer out of [128:21] here in Seattle the Allen Institute. [128:24] >> Yes indeed. So it's not as if this is [128:28] heretical and a secret. Other AI labs [128:30] including Google's own is doing it. But [128:32] from the very beginning, OpenAI seem to [128:35] be taking this more seriously given the [128:38] cost of it would require betting the [128:40] company if they continued down this [128:42] path. [128:42] >> Yeah. Or betting the nonprofit, betting [128:44] the entity. [128:45] >> Yes. [128:46] >> We're going to need some new terminology [128:47] here. [128:48] >> Yes. [128:48] >> So Elon's just walked out the door. [128:50] Where are they going to get the money [128:50] for this? Sam turns to one of the other [128:53] board members of OpenAI, Reed Hoffman. [128:57] Reed just a year or so earlier had sold [129:01] LinkedIn to Microsoft and Reed is now on [129:04] the board of Microsoft. So Reed says, [129:07] "Hey, why don't you come talk to Satia [129:09] about this?" [129:10] >> Do you know where he actually talks to [129:11] Satia? [129:11] >> Oh, I do. Oh, I do. In July of 2018, [129:15] they set a meeting for Sam Alman and [129:18] Satia Nadella to sit down while they're [129:21] both at the Allen and Company Sun Valley [129:24] Conference in Sun Valley, Idaho. [129:26] >> It's perfect. [129:27] >> And while they're there, they hash out a [129:29] deal for Microsoft to invest $1 billion [129:33] into OpenAI in a combination of both [129:36] cash and Azure cloud credits. And in [129:40] return, Microsoft will get access to [129:43] OpenAI's technology, get an exclusive [129:45] license to OpenAI's technology for use [129:48] in Microsoft's products. And the way [129:50] that they will do this is OpenAI the [129:52] nonprofit will create a captive [129:55] for-profit entity called OpenAI LP [129:58] controlled by the nonprofit OpenAI Inc., [130:01] and Microsoft will invest into the [130:03] captive for-profit entity. Reed Hoffman [130:06] joins the board of this new structure [130:08] along with Sam, Ilia, Greg Brockman, [130:11] Adam D'Angelo, and Tasha Macaulay. And [130:14] thus, the modern OpenAI forprofit [130:18] nonprofit question mark is created. [130:21] >> The thing that's still being figured out [130:23] even today here in 2025 is created. This [130:27] is like the complete history of AI. This [130:29] is not just the Google AI episode. [130:31] >> Well, these things are totally [130:32] inextricable. And I was just going to [130:34] say this is the Google part three [130:36] episode. Microsoft, they're back. [130:39] Microsoft is Google's mortal enemy. Yes. [130:43] That in our first episode on the [130:45] founding of Google and search and then [130:46] in the second episode on Alphabet and [130:48] all the products that they made, the [130:50] whole strategy at Google was always [130:52] about Microsoft. They finally beat them [130:55] on every single front and here they are [130:58] >> showing up again saying, "What was [131:00] Sati's line? We just want to see them [131:01] dance." I think the line that would come [131:04] a couple years later is we want the [131:06] world to know that we made Google dance. [131:13] Oh man. But this is all still pre-Chat [131:16] GPT. This is just Sam lining up the [131:19] financing he needs for what appears to [131:21] be a very expensive scaling exercise [131:24] they're about to embark on with GPT2 and [131:26] onward. [131:27] >> Yep. And this is the right time to talk [131:29] about why from OpenAI's perspective [131:33] Microsoft is the absolute perfect [131:35] partner. It's not just that they have a [131:38] lot of money, [131:39] >> although that helps. [131:40] >> I mean, that helps. That helps a lot. [131:42] But more important than money, they have [131:45] a really, really great public cloud. [131:47] Azure. [131:48] >> Yes. OpenAI is not going to go buy a [131:51] bunch of NVIDIA GPUs and then build [131:52] their own data center here at this point [131:54] in 2018. That's not the scale of company [131:57] that they are. They need a cloud [131:59] provider in order to actually do all the [132:02] compute that they want to do. If they [132:04] were back at Google and these [132:05] researchers are doing it, great. Then [132:06] they have all the infrastructure. But [132:08] OpenAI needs to tie themselves to [132:11] someone with the infrastructure. [132:13] >> And there's basically only two non- [132:16] Googlele options. They're both in [132:18] Seattle. [132:20] And hey, one of them in Microsoft is [132:22] really interested, also has a lot of [132:25] cash. It seems like a great partnership. [132:28] >> That's true. I wonder if they did talk [132:30] to AWS at all about it cuz I think this [132:32] is a crazy Easter egg. I hesitate to say [132:34] it out loud, but I think AWS was [132:37] actually in the very first investment [132:40] with Elon in Open AI. [132:43] >> Oh wow. And I don't know if it was in [132:45] the form of credits or what the deal [132:46] was, but I'd seen it reported a couple [132:48] places that AWS actually was in that [132:51] nonprofit round. [132:52] >> Yeah, in the uh nonprofit funding, the [132:55] donations to [132:57] >> Yes. [132:57] >> the early OpenAI. [132:59] >> Anyway, Microsoft Open AI, they end up [133:02] tying up [133:02] >> a match made in heaven. Satya and Sam [133:05] are on stage together talking about how [133:07] this amazing partnership and marriage [133:09] has come together and they're off to [133:12] model training. [133:14] >> Yeah. And this paves the way for the GPT [133:17] era of OpenAI. But before we tell that [133:20] story, [133:21] >> yes, now is a great time to thank one of [133:24] our favorite companies, Shopify. [133:26] >> Yes. And this is really fun because we [133:28] have been friends and fans of Shopify [133:30] for years. We just had Toby on ACQ2 to [133:34] talk about everything going on in AI and [133:37] everything that has happened at Shopify [133:39] in the six years now since we covered [133:41] the company on acquired. [133:42] >> It's been a pretty insane transformation [133:44] for them. [133:45] >> Yeah. So, back at their IPO, Shopify was [133:49] the go-to platform for entrepreneurs and [133:51] small businesses to get online. What's [133:53] happened since is that is still true. [133:56] And Shopify has also become the world's [133:59] leading commerce platform for [134:00] enterprises of any size, period. [134:03] >> Yeah. So, what's so cool about the [134:04] company is how they've managed to scale [134:06] without losing their soul. Even though [134:08] companies like Everlane and Vori and [134:11] even older established companies like [134:13] Mattel are doing billions of revenue on [134:15] Shopify, the company's mission is still [134:17] the same as the day Toby founded it to [134:19] create a world where more entrepreneurs [134:22] exist. [134:22] >> Oh, yeah. Ben, you got to tell everyone [134:24] your favorite enterprise brand that is [134:26] on Shopify. [134:27] >> Oh, I'm saving that for next episode. I [134:30] have a whole thing planned for episode [134:31] two of this season. [134:33] >> Okay. Okay, great. Anyway, the reason [134:35] enterprises are now also using Shopify [134:38] is simple. Because businesses of all [134:39] sizes just sell more with Shopify. They [134:42] built this incredible ecosystem where [134:44] you can sell everywhere. Obviously, your [134:47] own site. That's always been true. But [134:49] now with Shopify, you can easily sell on [134:52] Instagram, YouTube, Tik Tok, Roblox, [134:55] Roku, ChatgPT, Perplexity, anywhere. [134:59] Plus, with Shop Pay, their accelerated [135:01] checkout, you get amazing conversion, [135:04] and it has a built-in user base of 200 [135:07] million people who have their payment [135:09] information already stored with it. [135:10] Shopify is the ultimate example of not [135:13] doing what doesn't make your beer taste [135:15] better. Even if you're a huge brand, [135:17] you're not going to build a better [135:18] e-commerce platform for your product. [135:20] But that is what Toby and Shopify's [135:23] entire purpose is. So, you should use [135:25] them. [135:25] >> Yes. So, whether you're just getting [135:27] started or already at huge scale, head [135:29] on over to shopify.com/acquired. [135:32] That's hop fy.com/acquired. [135:36] And just tell them that Ben and David [135:37] sent you. [135:38] >> All right. So, what are we in GPT2? Is [135:42] that what's being trained right here? [135:43] >> Yes, GPT2. This was the first time I [135:45] heard about it. Data scientists around [135:48] Seattle were talking about this cool, [135:50] >> right? So, after the first Microsoft [135:53] partnership, the first billion dollar [135:54] investment in 2019, OpenAI releases [135:57] GPT2, which is still early but very [136:00] promising that can do a lot of things, [136:04] >> a lot of things, but it required an [136:06] enormous amount of creativity on your [136:08] part. You kind of had to be a developer [136:10] to use it. And if you were a consumer, [136:14] there was a very heavy load put on you. [136:16] You had to go write a few paragraphs and [136:19] then paste those few paragraphs into the [136:22] language model and then it would suggest [136:25] a way to finish what you were writing [136:27] based on the source paragraphs. But it [136:29] wasn't interactive. [136:30] >> Yes, it was not a chat interface. [136:32] >> Yes, [136:33] >> there was no interface essentially for [136:35] it. [136:36] >> It was an API, but it can do things like [136:38] obviously translate text. I mean, [136:39] Google's been doing that for a long [136:40] time, but GPT2, you could do stuff like [136:43] make up a fake news headline and give it [136:46] to GPT2 and it would write a whole [136:48] article. You would read it and you'd be [136:50] like, "Uh, sounds like it was written by [136:52] a bot." [136:53] >> Yeah. [136:53] >> But again, there was no front door to it [136:56] for normal people. You had to really be [136:58] willing to wait in the muck to use this [137:00] thing. So then the next year in June of [137:03] 2020, GPT3 [137:06] comes out. Still no front door, you [137:09] know, user interface to the model, but [137:12] it's very good. GPT2 showed the promise [137:16] of what was possible. GPT3, [137:19] it's starting to be in the conversation [137:22] of can this thing pass the Turing test. [137:24] >> Oh, yeah. [137:24] >> You have a hard time distinguishing [137:26] between articles that GPT wrote and [137:30] articles that humans wrote. It's very [137:33] good. And there starts to be a lot of [137:35] hype around this thing. And so even [137:37] though consumers aren't really using it, [137:38] the broader awareness is that there's [137:41] something interesting on the horizon. I [137:43] think the number of AI pitch decks that [137:45] VCs are seeing is starting to tick up [137:48] around this time as is the Nvidia stock [137:50] price. [137:51] >> Yes. [137:52] >> So then in the next year in the summer [137:54] of 2021, [137:57] Microsoft [137:59] releases [138:01] GitHub Copilot using GPT3. This is the [138:04] first not just Microsoft product that [138:07] comes out with GPT baked into it. But [138:10] first [138:10] >> productization [138:12] >> product anywhere. Yeah. First [138:13] productization of GPT. [138:14] >> Yes. Of any open AI technology. [138:16] >> Yeah. It's big. This starts a massive [138:19] change in how software gets written in [138:22] the world. [138:22] >> Slowly then all at once. It's one of [138:24] these things where at first just a few [138:26] software engineers and there was a lot [138:27] of whispers of how cool is this? It [138:29] makes me a little bit more efficient. [138:30] And now you get all these comments like [138:31] 75% of all companies code is written [138:33] with AI. [138:34] >> Yep. So after that, Microsoft invests [138:36] another $2 billion in open AI, which [138:39] seemed like a lot of money at the time. [138:42] So that takes us to the end of 2021. [138:45] There's an interesting kind of context [138:47] shift that happens around here. [138:50] >> Yeah. The bottom falls out on tech [138:52] stocks, crypto, the broader markets [138:55] really, everyone suddenly goes from risk [138:57] on to risk off. And part of it was war [139:00] in Ukraine, but a lot of it was interest [139:02] rates going up. And Google gets hit [139:06] really hard. The high water mark was [139:09] November 19th of 2021. Google was right [139:13] at $2 trillion of market cap. About a [139:16] year after that slide began, they were [139:18] worth a trillion dollars. Nearly a 50% [139:21] draw down. [139:22] >> Wow. So towards the end of 2022 leading [139:25] up to the launch of Chat GPT, [139:28] >> people I think are starting to realize [139:31] Google's slow. They're slow to react to [139:34] things. It feels like they're a old [139:37] crusty company. Are they like the [139:39] Microsoft [139:40] 2000s where they haven't had a [139:43] breakthrough product in a while? [139:45] People are not bright on the future of [139:47] Google and then chat GPT comes out. [139:50] >> Yeah. Wow. Which means if you were [139:53] bullish on Google back then and [139:54] contrarian, you could have invested at a [139:57] trillion dollar market cap. [139:58] >> Which is interesting. Like in October of [140:00] 21, the market was saying that the [140:03] fourthcoming AI wave will not be a [140:05] strength for Google. Or maybe what it [140:07] was saying is we don't even know [140:09] anything about a forthcoming AI wave cuz [140:11] people are talking about AI, but they've [140:12] been talking about VR and they've been [140:13] talking about crypto and they've been [140:14] talking about all this frontier tech and [140:16] like that's not the future at all. This [140:18] company just feels slow and unadaptive. [140:20] and slow and unadaptive at that point in [140:21] history I think would have been a fair [140:23] characterization. They had an internal [140:25] chatbot, right? [140:26] >> Yes, they did. All right. So, before we [140:29] talk about chat GPT, Google had a [140:32] chatbot. So, Nom Shazir, incredible [140:36] engineer, rearchitected the transformer, [140:39] made it work, one of the lead authors of [140:41] the paper, storyried career within [140:43] Google, has all of this sway, should [140:45] have all of this sway within the [140:47] company. After the transformer paper [140:49] comes out, he and the rest of the team [140:51] are like, "Guys, we can use this for a [140:54] lot more than Google Translate." And in [140:56] fact, the last paragraph of the paper. [140:58] >> Are you about to read the transformer [141:00] paper? [141:00] >> Yes, I am. We are excited about the [141:03] future of attention-based models and [141:05] plan to apply them to other tasks. We [141:07] plan to extend the transformer to [141:10] problems involving input and output [141:12] modalities other than text and to [141:16] investigate large inputs and outputs [141:18] such as images, audio, and video. This [141:22] is in the paper. [141:23] >> Wow. [141:24] >> Google obviously does not do any of that [141:27] for quite a while. Gnome though [141:29] immediately starts advocating to Google [141:31] leadership, hey, I think this is going [141:34] to be so big. the transformer that we [141:38] should actually consider just throwing [141:40] out the search index and the 10 blue [141:42] links model and go all in on [141:45] transforming all of Google into one [141:48] giant transformer model. And then Gnome [141:52] actually goes ahead and builds a chatbot [141:56] interface to a large transformer model. [142:00] >> Is this Lambda? [142:01] >> This is before Lambda. Mina is what he [142:05] calls it. [142:06] >> And there is a chatbot in the like late [142:09] teens 2020 time frame that Gnome has [142:11] built within Google that arguably is [142:16] pretty close to chat GPT. Now, it [142:19] doesn't have any of the post-trading [142:20] safety that chat GPT does. So, it would [142:24] go off the rails. [142:25] >> Yeah. Someone told us that you could [142:26] just ask it who should die and it would [142:28] come up with names for you of people [142:30] that should die. It was not a shippable [142:32] product. It was a very raw, not safe, [142:37] not post-trained chatbot and model, [142:40] >> right? But it existed within Google and [142:44] they didn't ship it. [142:46] >> And technically, not only did it not [142:48] have post- training, it didn't have RLHF [142:50] either. This very core component of the [142:51] models today, the reinforcement learning [142:53] with human feedback that chat GPT, I [142:57] don't know if it had it in three, but it [142:58] did in 3.5 and it did for the launch of [143:00] ChatGpt. realistically it wasn't [143:03] launchable even if it was an open AI [143:05] thing cuz it was so bad. But a company [143:06] of Google stature certainly could not [143:09] take the risk. So strategically they [143:11] have this working against them. But [143:13] aside from the strategy thing, there's [143:15] two business model problems here. One, [143:18] if you're proposing drop the 10 blue [143:20] links and just turn google.com into a [143:23] giant AI chatbot, revenue drops when you [143:26] provide direct answers to questions [143:28] versus showing advertisers and letting [143:31] people click through to websites. That [143:33] upsets the whole Apple cart. Obviously, [143:35] they're thinking about it now, but until [143:37] 2021, that was an absolute non-starter [143:41] to suggest something like that. Two, [143:44] there were legal risks of sitting in [143:46] between publishers and users. I mean, [143:48] Google at this point had spent decades [143:49] fighting the public perception and court [143:52] rulings that they were disintermediating [143:54] publishers from readers. So, there was [143:56] like a very high bar internally, [143:59] culturally to clear if you were going to [144:01] do something like this. Even those info [144:03] boxes that popped up that took until the [144:06] 201s to make it happen, those really [144:09] were mostly on non-monetizable queries [144:11] anyway. So anytime that you were going [144:14] to say, "Hey, Google's going to provide [144:15] you an answer instead of 10 blue links," [144:18] you had to have a bulletproof case for [144:20] it. [144:21] >> Yeah. And there was also a brand promise [144:25] and trust issue, too. Consumers [144:29] trusted Google so much for us even [144:31] today. You know, when I'm doing research [144:33] for acquired, we need to make sure we [144:35] get something right. I'm going to [144:36] Google. [144:36] >> I look something up in Claude. Yeah. [144:38] >> It gives me an answer. I'm like, that's [144:39] a really good answer. And then I verify [144:41] by searching Google that I can find [144:43] those facts too if I can't click through [144:45] the sources on Claude. That's my [144:46] workflow. [144:47] >> Which sort of sounds funny today, but [144:49] it's important. If you're going to [144:50] propose replacing the 10 blue links with [144:53] a chatbot, you need to be really damn [144:56] sure that it's going to be accurate. [144:59] >> Yes. [144:59] >> And in 2020 2021, that was definitely [145:03] not the case. Arguably still isn't the [145:05] case today. And there also wasn't a [145:09] compelling reason to do it because [145:13] nobody was really asking for this [145:15] product, [145:17] >> right? [145:17] >> Gnome knew and people in Google knew [145:20] that you could make a chatbot interface [145:22] to a transformer-based LLM and that was [145:26] a really compelling product. The general [145:28] public didn't know. Open AAI didn't even [145:31] really know. I mean GPT was out there. [145:33] >> Do you know the story of the launch of [145:34] Chat GPT? Well, I think I do. I have it [145:36] in my notes here. [145:37] >> All right. So, they've got GPT 3.5. It's [145:40] becoming very, very useful. [145:42] >> Yeah, this is late 2022. They've got [145:44] 3.5, [145:45] >> but there's still this problem of how am [145:47] I supposed to actually use it? How is it [145:49] productized? And Sam just kind of says, [145:54] "We should make a chatbot. That seems [145:56] like a natural interface for this. Can [145:57] someone just make a chat?" And within [145:59] like a week internally, [146:01] >> someone makes a chat. They just turn [146:04] calls to the chat GBT 3.5 API into a [146:08] product where you're just chatting with [146:09] it. And every time you kick off a chat [146:11] message, it just calls GBT3.5 on the API [146:14] and that turns out to be this magic [146:17] product. I don't think they expected it. [146:20] I mean, servers are tipping over. [146:21] They're working with Microsoft to try to [146:23] get more compute. They're cutting deals [146:25] with Microsoft in real time to try to [146:27] get more investment to get more Azure [146:29] credits or get advances on their Azure [146:30] credits in order to handle the [146:33] incredible load in November of 2022 [146:36] that's coming in of people wanting to [146:37] use this thing. They also just throw up [146:40] a payw wall randomly because they [146:42] thought that the business was going to [146:43] be an API business. They thought that [146:45] the projections were all about how much [146:47] revenue they were going to do through [146:49] B2B licensing deals and then they just [146:52] realized, oh, there's all these [146:53] consumers trying to use this. Put up a [146:54] payw wall to at least dampen the most [146:57] expensive use of this thing so we can [146:59] kind of offset the cost or slow the roll [147:01] out, [147:02] >> right? This isn't uh Google search, you [147:04] know, 89% gross margin stuff here, [147:07] >> right? So they end up having incredibly [147:10] fast revenue take off just from the [147:12] quick stripe payw wall that they threw [147:14] up over a weekend to handle all the [147:17] demand. So to say that OpenAI had any [147:20] idea what was coming would also be [147:22] completely false. They did not get that [147:24] this would be the next big consumer [147:26] product when they launched it. [147:27] >> Ben Thompson loves to call Open AAI the [147:30] accidental consumer tech company, right? [147:32] >> Yes, [147:33] >> it was definitely accidental. Now there [147:36] is actually another slightly different [147:40] version of the motivation for launching [147:42] the chat. [147:43] >> Is this the Daario [147:44] >> interface? Yeah, the Daario and [147:45] Anthropic version. So Anthropic was [147:48] working on what would become Claude and [147:52] rumors were out there and people at [147:54] OpenAI got wind of like, oh hey, [147:56] Anthropic and Daario are working on a [147:58] chat interface. [148:00] We should probably do one, too. and if [148:03] we're going to do one, we should [148:04] probably launch it before they launch [148:06] theirs. So, I think that had something [148:08] to do with the timing, but again, I [148:10] don't think anybody including OpenAI [148:12] realized what was going to happen, which [148:14] is Ben, you alluded to it, but to give [148:17] the actual numbers, on November 30th, [148:20] 2022, [148:21] >> basically Thanksgiving, [148:22] >> OpenAI launches a research preview of an [148:28] interface to the new GPT 3.5 called [148:32] ChatGpt. [148:33] That morning on the 30th, Sam Alman [148:36] tweets, "Today we launched ChatGpt. Try [148:38] talking with it here." And then a link [148:40] to chat. [148:43] Within a week, less than a week [148:45] actually, it gets 1 million users. By [148:48] the end of the year, so you know, one [148:51] month later, December 31st, 2022, it has [148:55] 30 million users. By the end of the next [148:58] month, by the end of January 23, so two [149:01] months after launch, it crosses 100 [149:05] million registered users. The fastest [149:08] product in history to hit that [149:11] milestone. Completely insane. Completely [149:15] insane. Before we talk about what that [149:18] unleashes within Google, which is the [149:21] famous code red, to rewind a little bit [149:23] back to Gnome and the chatbot within [149:25] Google, Mina, Google does keep working [149:28] on Mina. They develop it into something [149:32] called Lambda, which is also a chatbot, [149:36] also internal. [149:37] >> I think it was a language model. At this [149:40] point in time, they still differentiated [149:42] between the underlying model brand name [149:45] and the application name. [149:47] >> Yes, Lambda was the model and then there [149:49] also was a chat interface to Lambda that [149:52] was internal for Google use only. Gnome [149:55] is still advocating to leadership, we [149:58] got to release this thing. He leaves in [150:00] 2021 and founds a chatbot company, [150:04] Character AI, that still exists to this [150:06] day. And they raise a lot of money, as [150:10] you would expect. And then Google [150:12] ultimately in 2024 after ChatGpt [150:15] launches, pays $2.7 billion, I think, to [150:20] do a licensing deal with Character AI, [150:23] the net of which Gnome comes back to [150:25] Google. Yeah, I think Larry and Sergey [150:27] were like, if we're going to compete [150:30] seriously, we kind of need Gnome back [150:33] and blank check to go get him. [150:35] >> Yeah. So, throughout 2021 2022, Google's [150:39] working on the Lambda model and then the [150:42] chat interface to it. In May of 2022, [150:45] they do release something that is [150:47] available to the public called AI test [150:49] kitchen, which is a AI product test area [150:53] where people can play around with [150:55] Google's internal AI products, including [150:58] the Lambda chat interface. [151:00] >> Yep. And all fairness, predates Chat [151:02] GPT. [151:03] >> Do you know what they do to nerf chat so [151:06] that it doesn't go too far off the [151:07] rails? This is amazing. [151:08] >> No. For the version of Lambda chat that [151:11] is in AI test kitchen, they stop all [151:14] conversations after five turns. So you [151:17] can only have five turns of conversation [151:19] with the chatbot and then it's just and [151:21] we're done for today. Thank you. [151:22] Goodbye. [151:23] >> Oh wow. [151:24] >> And the reason they did that was for [151:25] safety of like, you know, if the more [151:27] turns you had with it, the more likely [151:29] it would start to go off the rails. [151:31] >> And honestly, it was a fair concern. I [151:33] mean, this thing was not for public [151:36] consumption. And if you remember back a [151:38] few years before, Microsoft released [151:40] Tay, which was this crazy racist [151:43] chatbot. [151:44] >> Yeah. They launched it as a Twitter bot, [151:46] right? And it was going off the rails on [151:48] Twitter. This was in 2016, I think. [151:51] >> Right. Maximal impact of badness. [151:53] >> Yeah. And so despite Google all the way [151:56] back in 2017 Sundar declared we are an [151:59] AI first company is being understandably [152:02] very cautious in real public AI launches [152:05] especially on consumerf facing things. [152:07] >> Yep. And as far as anyone else is [152:09] concerned before chat GPT they are an AI [152:12] first company and they're launching all [152:14] this amazing AI stuff. It's just within [152:17] the vector of their existing products. [152:19] Right? So chat GPD comes out becomes the [152:21] fastest product in history to 100 [152:23] million users. It is immediately obvious [152:26] to Sundar, Larry, Sergey, all of Google [152:29] leadership that this is an existential [152:30] threat to Google. Chat GPT is a better [152:34] user experience to do the same job [152:36] function that Google search does. And to [152:39] underscore this, so if you didn't know [152:41] it in November of 22, you sure knew it [152:43] by February of 23 because good old [152:46] Microsoft, our biggest scariest enemy. [152:49] Oh yeah. [152:50] >> announces a new Bing powered by OpenAI. [152:53] And Satia has a quote. It's a new day [152:55] for search. The race starts today. [152:59] There's an announcement of a new AI [153:01] powered search page. He says, "We want [153:04] to rethink what search was meant to be [153:06] in the first place. In fact, Google's [153:08] success in the initial days came by [153:10] reimagining what could be done in [153:12] search. And I think the AI era we're [153:14] entering gets us to think about it. This [153:18] is the worst possible thing that could [153:21] happen to Google. That now Microsoft can [153:25] actually challenge Google on their own [153:27] turf intent on the internet with a [153:30] legitimately [153:32] different, better, differentiated [153:34] product vector. Not what Bing was trying [153:36] to do, copycat. This is the full leaprog [153:39] and they have the technology partnership [153:41] to do it. [153:42] >> Or so everybody thinks at the moment. [153:44] >> Oh my god, terrifying. This is when [153:46] Satia says the quote in an interview [153:49] around this launch with Bing. I want [153:51] people to know that we made Google [153:53] dance. [153:56] Oh boy. Well, hey, if you come at the [153:58] king, you'd best not miss, [154:00] >> right? [154:03] >> And this big launch kind of misses. [154:05] >> Yes. So what happens in Google December [154:08] 2022 even before the big launch but [154:11] after the chat GPT moment Sundar issues [154:14] a code red within the company [154:16] >> and what does that mean? [154:17] >> Up until this point Google and Sundar [154:21] and Larry and everyone had been thinking [154:24] about AI as a sustaining innovation in [154:28] Klay Christensen's terms. This is great [154:30] for Google. This is great for our [154:32] products. Look at all these amazing [154:33] things that we're doing. It further [154:35] entrenches incumbents. [154:37] >> It further is entrenching our lead in [154:39] all of our already leading products. [154:43] >> We can deploy more capital in a [154:45] predictable way to either drive down [154:47] costs or make our product experiences [154:49] that much better than any startup could [154:52] make. [154:52] >> Got more monetized that much better. All [154:55] the things. Once chat GPT comes out on a [154:59] dime overnight, AI shifts from being a [155:03] sustaining innovation to a disruptive [155:06] innovation. It is now an existential [155:08] threat. And many of Google's strengths [155:11] from the last 10, 15, 20 years of all [155:15] the AI work that's happened in the [155:16] company are now liabilities. They have a [155:20] lot of existing castles to protect. [155:22] >> That's right. They have to run [155:24] everything through a lot of filters [155:26] before they can decide if it's a good [155:27] idea to go try to out open AAI open AAI. [155:31] >> Yep. So this code red that Sundar issues [155:34] to the company is actually a huge moment [155:38] because what it means and what he says [155:40] is we need to build and ship real native [155:47] AI products ASAP. This is actually what [155:51] you need to do in the textbook response [155:54] to a disruptive innovation as the [155:56] incumbent. You need to not bury your [155:58] head in the sand and you need to say, [156:00] "Okay, we need to like actually go build [156:03] and ship products that are comparable to [156:06] these disruptive innovators." And you [156:08] need to be laser operationally [156:11] in all the details to try and figure out [156:14] where is it that the new product is [156:18] actually cannibalizing our old product [156:20] and where is it that the new product can [156:22] be complimentary and just lean into all [156:25] the ways in which you can be [156:26] complimentary in all the different [156:28] little scenarios. And really what [156:30] they've been trying to do, this ballet [156:32] from 2022 onward, is protect the growth [156:35] of search while also creating the best [156:39] AI experiences they can. And so it's [156:42] very clever the way that they do AI [156:44] overviews for some but not all queries. [156:46] And they have AI mode for some but not [156:49] all users. And then they have Gemini, [156:51] the full AI app, but they're not [156:54] redirecting Google.com to Gemini. It's [156:57] this like very delicate dance of [156:59] protecting the existing franchise while [157:01] also building a hopefully [157:03] non-cannibalizing as much as we can new [157:05] franchise. [157:06] >> Yep. And you see them really going hard [157:09] and I think building leading products in [157:13] nonarch cannibalizing categories like [157:16] video, [157:16] >> right? V3 or nano banana. These are [157:19] things that don't in any way cannibalize [157:22] the existing franchise. They in fact use [157:24] some of Google's strength, all the [157:25] YouTube training data and stuff like [157:26] that. [157:27] >> Yeah. So, what happens next? As you [157:30] might expect, it gets worse before it [157:32] gets better. [157:34] Code Red goes out December 2022. [157:36] >> Bard baby launch Bard. [157:38] >> Oh boy. Well, even before that, January [157:41] 23, when OpenAI hits 100 million [157:45] registered users for ChatGpt, Microsoft [157:48] announces they are investing another $10 [157:51] billion in OpenAI and says that they now [157:55] own 49% of the for-profit entity. [157:59] Incredible in and of itself. But then [158:01] now think about this from the Google [158:02] lens of Microsoft, our enemy. They now [158:06] arguably own obviously in retrospect [158:09] here they don't own open AAI but it [158:12] seems at the time like oh my god [158:14] Microsoft might now own open AI which is [158:17] our first true existential threat in our [158:19] history as a company. [158:20] >> Not great Bob. [158:21] >> So then February 2023 the Bing [158:23] integration launches. Satia has the [158:26] quote about wanting to make Google [158:27] dance. Meanwhile Google is scrambling [158:29] internally to launch AI products as fast [158:33] as possible. So the first thing they do [158:35] is they take the Lambda model and the [158:37] chatbot interface to it. They rebrand it [158:40] as Bard. [158:41] >> They ship that publicly [158:42] >> and they release it immediately. [158:44] February 2023, ship it publicly. [158:48] Available GA to anyone, [158:50] >> which maybe was the right move, but god [158:52] it was a bad product. [158:54] >> It was really bad. [158:55] >> I didn't know the term at the time, [158:56] RLHF, but it was clear it was missing a [158:59] component of some magic that ChatGpt [159:02] had. this reinforcement learning with [159:04] human feedback where you could really [159:06] tune the appropriateness, the tone, the [159:09] voice, the sort of correctness of the [159:12] responses, it just wasn't there. [159:14] >> Yep. So, to make matters worse, in the [159:18] launch video for Bard, a video, this is [159:23] a choreographed pre-recorded video where [159:25] they're showing conversations with Bard. [159:29] Bard gives an inaccurate factual [159:31] response to one of the queries that they [159:33] include in the video. [159:35] >> This is one of the worst keynotes in [159:36] history. [159:37] >> After the Bard launch and this keynote, [159:40] Google's stock drops 8% on that day. And [159:44] then like we were saying, once the [159:45] actual product comes out, it becomes [159:46] clear it's just not good. [159:48] >> Yep. [159:48] >> And it pretty quickly becomes clear, [159:50] it's not just that the chatbot isn't [159:51] good, it's the model isn't good. So in [159:56] May they replace Lambda with a new model [159:58] from the Brain team called Palm. It's a [160:01] little bit better, but it's still [160:03] clearly behind not only GPT3.5, but in [160:07] March of 2023, OpenAI comes out with [160:10] GPT4, which is even better. [160:13] >> You can access that now through chatgpt. [160:16] And here is where Sundar makes two [160:18] really, really big decisions. Number [160:20] one, he says, "We cannot have two AI [160:24] teams within Google anymore. We're [160:27] merging Brain and Deep Mind into one [160:31] entity called Google Deep Mind, [160:33] >> which is a giant deal. This is in full [160:36] violation of the original deal terms of [160:40] bringing Deep Mind in." [160:43] >> Yep. And the way he makes it work is he [160:46] says, "Demis, you are now CEO of the AI [160:50] division of Google, Google DeepMind. [160:52] This is all hands on deck and you and [160:56] Deep Mind are going to lead the charge. [160:58] You're going to integrate with Google [160:59] Brain and we need to change all of the [161:03] past 10 years of culture around building [161:07] and shipping AI products within Google." [161:10] To further illustrate this, when [161:12] Alphabet became Alphabet, they had all [161:14] these separate companies, but things [161:16] that were really core to Google, like [161:18] YouTube actually stayed a part of [161:19] Google. DeepMind was its own company. [161:22] That's how separate this was. They're [161:25] working on their own models. In fact, [161:27] those models are predicated on [161:28] reinforcement learning. That was the big [161:30] thing that DeepMind had been working on [161:32] the whole time. And so reading in [161:35] between the lines, it's Sundar looking [161:37] at his two AI labs and going, "Look, I [161:40] know you two don't actually get along [161:42] that well, but look, I don't care that [161:44] you had different charters before. I am [161:46] taking the responsibility of Google [161:49] Brain and giving it to DeepMind and [161:51] DeepMind is absorbing the Google Brain [161:53] team." I think that's what you should [161:54] sort of read into it because as you look [161:56] at where the models went from here, they [161:59] kind of came from DeepMind. [162:01] >> Yep. There's a little bit of interesting [162:03] backstory to this too. So Mustafa [162:06] Sullean, the third co-founder of [162:08] DeepMind, [162:09] at some point before this, [162:11] >> he became like the head of Google AI [162:13] policy or something. [162:15] >> He had already shifted over to Brain and [162:17] to Google. [162:19] >> He stayed there for a little while and [162:21] then he ended up getting close with who [162:24] else? Reed Hoffman. Remember Reed is on [162:27] the ethics board for DeepMind and [162:30] Mustafa and Reed leave and go found [162:35] Inflection AI which fast forward now [162:38] into 2024 after the absolute insanity [162:43] that goes down at OpenAI in Thanksgiving [162:46] 2023 when Sam Alman gets fired over the [162:48] weekend during Thanksgiving and then [162:51] brought back by Monday when all the team [162:53] threatened to quit and go to Microsoft. [162:55] Open eye loves Thanksgiving. Can't wait [162:57] for this year. [162:58] >> They love Thanksgiving. Yeah. Gosh. [163:01] After all that, which certainly strains [163:03] the Microsoft relationship, remember [163:06] again, Reed is on the board of [163:07] Microsoft. Microsoft does one of these [163:10] acquisition type deals with Inflection [163:13] AI and brings Mustafa in as the head of [163:17] AI for Microsoft. [163:20] >> Crazy. [163:21] >> Wild, right? Just wild. [163:22] >> Crazy turn of events. Okay, so that [163:25] first big decision that Sundar makes is [163:28] unifying deep mind and brain. That was [163:31] huge. Equally big, he says, I want you [163:36] guys to go make a new model and we're [163:39] just going to have one model that is [163:41] going to be the model for all of Google [163:45] internally for all of our AI products [163:47] externally. It's going to be called [163:49] Gemini. No more different models, no [163:52] more different teams. just one model for [163:55] everything. This is also a huge deal. [163:58] >> It's a giant deal and it's twofold. It's [164:01] push and it's pull. It's saying, "Hey, [164:03] if anyone's got a need for an AI model, [164:05] you got to start using Gemini." But two, [164:08] it's actually kind of the plus thing [164:09] where they go to every team and they [164:11] start saying, "Gemini is our future. You [164:13] need to start looking for ways to [164:14] integrate Gemini into your product." [164:16] >> Yes, I'm so glad you brought up Plus. [164:19] This came up with a few folks I spoke to [164:20] in the research. Obviously, this is all [164:23] playing out real time, but the point a [164:26] lot of people at Google made is the [164:28] Gemini situation is very different than [164:30] the Google+ situation. This is a [164:33] technical thing, A, which has always [164:35] been Google's wheelhouse, but B, even [164:38] more importantly, this is the rational [164:41] business thing to do in the age of these [164:44] huge models. Even for a company like [164:47] Google, there are massive scaling laws [164:50] to models. [164:51] >> The more data you put in, the better [164:52] it's going to get, the better all the [164:54] outputs are going to be. [164:55] >> And because of scaling laws, you need [164:59] your models to be as big as possible in [165:02] order to have the best performance [165:04] possible. If you're trying to maintain [165:06] multiple models within a company, you're [165:09] repeating multiple huge costs to [165:12] maintain huge models. You definitely [165:15] don't want to do that. You need to [165:16] centralize on just one model. [165:18] >> Yeah, it's interesting. There's also [165:20] something to read into where at first it [165:23] was the Gemini model underneath the Bard [165:26] product. Bard was still the consumer [165:28] name. Then at some point they said, "No, [165:29] we're just calling it all Gemini and [165:31] Gemini became the userfacing name." [165:32] Also, this pulls in my quintessence from [165:38] the Alphabet episode. I know it's a [165:40] little bit woowoo, but with Google [165:44] saying, "We're actually going to name [165:45] the consumer service the name of the AI [165:47] model." They're sort of admitting to [165:49] themselves, this product is nothing but [165:52] technology. There isn't productiness to [165:56] do on top of it. It's just like Gmail. [165:59] Gmail was technology. It was fast [166:01] search. It was lots of storage. It was [166:03] use it in the web. The productiness [166:05] wasn't particular the way that like [166:07] Instagram was all about the product. [166:09] Gemini the model, Gemini, the chatbot [166:11] says, "We're just exposing our amazing [166:14] breakthrough technology to you all and [166:17] you get to interface directly with it." [166:19] Anthropologically looking from afar, it [166:21] kind of feels like it's that principle [166:22] at work. I totally agree. I think it's [166:25] actually a really important branding [166:28] point and sort of rallying point to [166:31] Google and Google culture to do this, [166:33] >> right? All right, so this is all the [166:35] stuff going on in Google 2023ish [166:39] in AI. Before we catch up to the [166:41] present, I have a whole other branch of [166:44] Alphabet that has been a real bright [166:46] spot for AI. Can I go there? Can I take [166:50] this offramp, if you will? [166:52] >> Can you uh take the wheel, so to speak? [166:54] >> May I take the wheel? May I investigate [166:56] another bet? [166:57] >> Yeah, please tell us the Whimo story. [166:59] >> Awesome. So, we got to rewind back all [167:02] the way to 2004, the DARPA Grand [167:05] Challenge, which was created as a way to [167:07] spur research into autonomous ground [167:10] robots for military use. And actually, [167:12] what it did for our purposes here today [167:14] is create the seed talent for the entire [167:17] self-driving car revolution 20 years [167:19] later. So, the competition itself is [167:21] really cool. There is a 132m raceourse. [167:25] Now, mind you, this is 2004 in the [167:28] Mojave Desert that the cars have to race [167:31] on. It is a dirt road. No humans are [167:34] allowed to be in or interact with the [167:36] cars. They are monitored 100% remotely. [167:39] And the winner gets $1 million. [167:43] >> $1 million, [167:44] >> which was a break from policy. Normally, [167:47] these are grants, not prize money. So, [167:50] this needs to be authorized by an act of [167:52] Congress. The $1 million eventually felt [167:55] comical. So the second year they raised [167:57] the pot to $2 million. It's crazy [167:59] thinking about what these researchers [168:01] are worth today. That that was the prize [168:02] for the whole thing. So the first year [168:04] in 2004 went fine. There were some [168:06] amazing tech demonstrations on these [168:08] really tight budgets, but ultimately [168:10] zero of the 100 registered teams [168:13] finished the race. But the next year in [168:16] 2005 was the real special year. The [168:18] progress that the entire industry made [168:21] in those first 12 months from what they [168:23] learned is totally insane. Of the 23 [168:26] finalists that were entering the [168:28] competition, 22 of them made it past the [168:31] spot where the furthest team the year [168:34] before had made it. The amount that the [168:36] field advanced in that one year is [168:38] insane. Not only that, five of those [168:41] teams actually finished all 132 miles. [168:45] Two of them were from Carnegie Melon and [168:47] one was from Stanford, led by a name [168:50] that all of you will now recognize, [168:52] Sebastian Thrun. [168:54] >> Indeed, [168:55] >> this is Sebastian's origin story before [168:57] Google. Now, as we said, Sebastian was [169:00] kind enough to help us with prep for [169:02] this episode, but I actually learned [169:03] most of this from watching a 20-year-old [169:05] NOVA documentary that is available on [169:07] Amazon Prime Video. Thanks to Brett [169:09] Taylor for giving us the tip on where to [169:11] find this documentary. Yes, the hot [169:14] research tip. [169:15] >> So, what was special about this Stanford [169:17] team? Well, one, there's a huge problem [169:20] with noisy data that comes out of all of [169:23] these sensors. You know, it's in a car [169:24] in the desert getting rocked around. [169:26] It's in the heat. It's in the sun. So, [169:28] common wisdom and what Carnegie Melon [169:30] did was to do as much as you possibly [169:32] can on the hardware to mitigate that. So [169:34] things like custom rigging and gimbals [169:36] and giant springs to stabilize the [169:38] sensors. Carnegie Melon would [169:40] essentially buy a Hummer and rip it [169:42] apart and rebuild it from the wheels up. [169:44] We're talking like welding and real [169:47] construction on a car. The Stanford team [169:49] did the exact opposite. They viewed any [169:51] new piece of hardware as something that [169:52] could fail. And so in order to mitigate [169:54] risks on race day, they used all [169:56] commodity cameras and sensors that they [169:58] just mounted on a nearly unmodified [170:01] Volkswagen. So they only innovated in [170:03] software and they figured they would [170:05] just kind of come up with clever [170:07] algorithms to help them clean up the [170:09] messy data later. Very googly, right? [170:11] >> Very googly. [170:12] >> The second thing they did was an early [170:15] use of machine learning to combine [170:17] multiple sensors. They mounted laser [170:19] hardware on the roof just like what [170:21] other teams were doing. And this is the [170:22] way that you can measure texture and [170:24] depth of what is right in front of you. [170:27] And the data, it's super precise, but [170:29] you can't drive very fast because you [170:31] don't really know much about what's far [170:33] away since it's this fixed field of [170:35] view. It's very narrow. Essentially, you [170:38] can't answer that question of how fast [170:40] can I drive or is there a turn coming [170:42] up. So, on top of that, the way they [170:45] solved it was they also mounted a [170:47] regular video camera. That camera can [170:49] see a pretty wide field of view just [170:51] like the human eye, and it can see all [170:53] the way to the horizon just like the [170:54] human eye. And crucially, it could see [170:57] color. So what it would do, this is like [171:00] really clever. They would use a machine [171:03] learning algorithm in real time in 2005. [171:07] This computer is like sitting in the [171:08] middle of the car. They would overlay [171:10] the data from the lasers on top onto the [171:13] camera feed. And from the lasers, you [171:15] would know if the area right in front of [171:17] the car was okay to drive or not. Then [171:19] the algorithm would look up in the [171:21] frames coming off the camera overlaid [171:23] what color that safe area was and then [171:26] extrapolate by looking further ahead at [171:28] other parts of the video frame to see [171:30] where that safe area extended to [171:33] >> so you could figure out your safe path [171:34] through the desert. [171:36] >> That's awesome. [171:37] >> It's so awesome. [171:38] >> I'm imagining like a Dell PC sitting in [171:41] the middle of this car in 2005. [171:43] >> It's not far off. In the email that we [171:45] send out, we'll share some photos of it. [171:47] It could then drive faster with more [171:49] confidence and it knew when turns were [171:51] coming up. Again, this is real time on [171:54] board the camera. 2005 is wild on that [171:57] tech. So ultimately, both of these bets [172:00] worked and the Stanford team won in [172:02] super dramatic fashion. They actually [172:04] passed one of the Carnegie Melon teams [172:06] autonomously through the desert. It's [172:08] like this big dramatic moment in the [172:10] documentary. So you would kind of think, [172:12] so then Sebastian goes to Google and [172:14] builds Whimo. No. As we talked about [172:17] earlier, he does join Google through [172:19] that crazy, please don't raise money [172:21] from Benchmark and Sequoia and we'll [172:23] just hire you instead. But he goes and [172:25] works on Street View and Project Ground [172:27] Truth and co-founds Google X. David, as [172:30] you were alluding to earlier, this [172:33] project chauffeur that would become [172:35] Whimo is the first project inside Google [172:39] X. And I think the story, right, is that [172:41] Larry came to Sebastian and was like, [172:44] "Yes, yo, that self-driving car stuff, [172:46] like, do it." And Sebastian was like, [172:48] "No, come on. That was a DARPA [172:49] challenge." And Larry's like, "No, no, [172:51] you should do it." He's like, "No, no, [172:52] that won't be safe. There's people [172:53] running around cities. I'm not just [172:55] going to put multi-tonon killer robots [172:57] on roads and go and potentially harm [173:00] people." And Larry finally comes to him [173:03] and says, "Why? What is the technical [173:05] reason that this is impossible?" And [173:07] Sebastian goes home, has sleep on it, [173:10] and he comes in the next morning and he [173:12] goes, "I realized what it was. I'm just [173:14] afraid." [173:18] >> Such a good moment. [173:20] >> So they start, he's like, "There's not a [173:22] technical reason. As long as we can take [173:24] all the right precautions and hold a [173:26] very high bar on safety, let's get to [173:28] work." So Larry then goes, "Great. I'll [173:31] give you a benchmark so that way you [173:32] know if you're succeeding." He comes up [173:34] with these 10 stretches of road in [173:36] California that he thinks will be very [173:38] difficult to drive. It's about a [173:39] thousand miles and the team starts [173:41] calling it the Larry 1000 and it [173:43] includes driving to Tahoe, Lumbard [173:46] Street in San Francisco, Highway 1 to [173:48] Los Angeles, the Bay Bridge. This is the [173:51] bogey. [173:52] >> Yep. If you can autonomously drive these [173:55] stretches of road, pretty good [173:57] indication that you can probably do [173:58] anything. [173:59] >> Yep. So they start the project in 2009. [174:02] Within 18 months, this tiny team, I [174:05] think they hired, I don't know, it's [174:06] like a dozen people or something, [174:08] they've driven thousands of miles [174:10] autonomously, and they managed to [174:12] succeed in the full Larry 1000 within 18 [174:16] months. [174:17] >> Totally unreal how fast they did it. And [174:21] then also totally unreal how long it [174:24] takes after that to productize and [174:27] create the Whimo that we know today. [174:29] >> Right. It's like the first 99% and then [174:31] the second 99% that takes 10 years. [174:34] >> Yeah. Self-driving is one of these [174:36] really tricky types of problems where [174:38] it's surprisingly easy to get started [174:39] even though it seems like it would be an [174:41] impossible thing. But then there's edge [174:42] cases everywhere. Weather, road [174:44] conditions, other drivers, novel road [174:46] layouts, night driving. So it takes this [174:49] massive amount of work for a production [174:51] system to actually happen. So then the [174:54] question is what business do we build? [174:55] What is the product here? And there was [174:57] what Sebastian wanted which was highway [174:59] assist. Sort of the lowest stakes, most [175:02] realistic. Let's make a better cruise [175:04] control. There's what Eric Schmidt [175:06] wanted, which is crazy. He proposed, oh, [175:10] let's just go buy Tesla and that'll be [175:11] our starting place and then we'll just [175:13] put all of our self-driving equipment on [175:14] all the cars. David, do you know what it [175:16] would have cost to buy Tesla at the [175:18] time? [175:19] >> I think at the time that negotiations [175:22] were taking place between Elon and Larry [175:25] and Google, this was in the depths of [175:28] the Model S production scaling wos. I [175:32] think Google could have bought the [175:33] company for $5 billion. That's what I [175:35] remember. [175:35] >> It was three billion. [175:37] >> $3 billion. Oh my goodness. [175:39] >> Obviously, that didn't happen, but what [175:41] a crazy alternative history that could [175:42] have been, [175:43] >> right? I mean, I think if that had [175:45] happened, DeepMind would not have gone [175:47] down in the same way and probably OpenAI [175:50] would not have gotten founded. [175:52] >> That's probably right. [175:54] >> I think that is obviously unprovable, [175:57] >> right? The counterfactuals that we [175:59] always come up with on this show, you [176:00] can't know. [176:01] >> Yeah. Seems more likely than not to me [176:04] that at a minimum, Open AAI would not [176:07] exist, [176:07] >> right? So, then there was what Larry [176:09] wanted to do. Option three, build robo [176:12] taxis. Yeah. [176:14] >> And ultimately that is at least right [176:16] now what they would end up doing. So we [176:20] could do a whole episode about this [176:22] journey, but we will just hit some of [176:23] the major points for the sake of time. [176:25] The big thing to keep in mind here, [176:27] neither Google nor the public really [176:29] knew if self-driving was something that [176:31] could happen in the next 2 years from [176:33] any given point or take another 10. And [176:36] just to illustrate it, for the first 5 [176:38] years of project chauffeur, it did not [176:40] use deep learning at all. They did the [176:43] Larry 1000 without any deep learning and [176:46] then went another three and a half [176:48] years. [176:49] >> Wow, that's crazy. [176:50] >> Yeah. And yet totally illustrates you [176:53] never know how far away the end goal is. [176:58] >> And this is a field that comes from the [177:00] only way progress happens is through [177:02] these series of breakthroughs. and you [177:04] don't know a how far the next [177:06] breakthrough is because at any given [177:07] time there's lots of promising things in [177:09] the field most of which don't work out [177:10] and then b when there is a breakthrough [177:12] actually how much lift that will give [177:14] you over existing methods so anytime [177:16] people are forecasting oh in AI we're [177:18] going to be able to do xyz in x years [177:20] it's a complete fool's errand even the [177:21] experts don't know here are the big [177:23] milestones 2013 they started using [177:25] convolutional neural nets they could [177:27] identify objects they got much better [177:29] perception capabilities this 2013 2014 [177:32] period is when Google found religion [177:34] around deep learning. So this is like [177:36] right after the 40,000 GPUs rolled out. [177:39] So they've actually got some hardware to [177:41] start doing this on now. 2016 they've [177:43] seen enough technology proof that they [177:45] think let's commercialize this. We can [177:47] actually spin this out into a company. [177:48] So Whimo becomes its own subsidiary [177:51] inside of Alphabet. It's no longer a [177:53] part of Google X anymore. 2017 obviously [177:56] the transformer comes out. They [177:58] incorporate some learnings from the [177:59] transformer especially around prediction [178:02] and planning. March of 2020, they raised [178:04] $3.2 billion from folks like Silverlake [178:08] Canada Pension and Investment Board, [178:09] Mubatala, Andrea Horowitz, and of [178:12] course, the biggest check, I think, [178:14] Alphabet. And I think they're always the [178:16] biggest check because Alphabet is still [178:17] the majority owner, even after a bunch [178:19] more fundraises. In October of 2020, [178:23] they launched the first public [178:25] commercial, no human behind the driver's [178:27] seat thing in Phoenix. It's the first in [178:30] the world. This is 11 years after [178:32] succeeding in the Larry 1000. And this [178:35] is nuts. I had given up at this point. I [178:37] was like, that's cute that Whimo and all [178:39] these other companies are trying to do [178:40] self-driving. Seems like it's never [178:42] going to happen. And then they actually [178:44] were doing a large volume of rides [178:46] safely with consumers and charging money [178:49] for it in Phoenix. [178:50] >> Then they bring it to San Francisco [178:52] where for me and lots of people in San [178:54] Francisco, it is a huge part of life in [178:57] the city here now. It's amazing. Yeah, [179:00] every time I'm down, I love taking them. [179:02] They're launching in Seattle soon. I'm [179:03] pumped. Interestingly, they don't make [179:05] the hardware. So, they use a Jaguar [179:07] vehicle. Yep. That from what I can tell [179:10] is only in Whimos. Like, I don't know if [179:12] anybody else drives that Jaguar or if [179:14] you can buy it, but they're working on a [179:17] sort of van next. They have some next [179:19] generation hardware. For anyone who [179:21] hasn't taken it, it's an Uber, but with [179:23] no driver. And that launched in June of [179:26] 24. Along the way there, they raised [179:28] their quote unquote series B, another [179:31] 2.5 billion. Then after the San [179:33] Francisco roll out, they raised their [179:35] quote unquote series C, 5.6 billion. [179:39] This year in January, they were [179:41] reportedly doing more in gross bookings [179:44] than Lyft in San Francisco. Wow. I [179:46] totally believe it. I mean, it is the [179:48] number one option in San Francisco that [179:51] I and everybody I know to always goes to [179:53] for ride hailing. It's like try to get a [179:55] Whimo. if there's not a Whimo available [179:57] anytime soon, you know, then go down the [179:59] stack. [179:59] >> Like we're living in the future and how [180:01] quickly we fail to appreciate it. [180:02] >> Yeah. And what's cool, I think, for [180:04] people who it hasn't come to their city [180:06] and is not part of their lives yet, it's [180:09] not just that it's a cool experience to [180:12] not have a driver behind the like pretty [180:14] quickly that just fades. It's actually a [180:17] different experience. M [180:19] >> so if I need to go somewhere with my [180:22] older daughter, I don't mind hailing a [180:24] Whimo, bringing the car seat, installing [180:26] the car seat in the Whimo and driving [180:28] with my daughter and she loves it. We [180:30] call it a robot car and she's like, "A [180:31] robot car? I'm so excited." [180:33] >> Huh. [180:34] >> I would never do that with an Uber. [180:36] >> That's interesting. [180:37] >> To my dog, whenever I need to go with my [180:39] dog, like it's super awkward to hail an [180:41] Uber and be like, "Hey, I got my dog. [180:43] You know, can the dog come in it?" Not a [180:44] big deal with a Whimo. And then when [180:47] you're in town, [180:48] >> Yeah. we can actually have sensitive [180:49] conversations in the car. [180:51] >> You can have phone calls. It really is a [180:53] different experience. [180:54] >> Yeah, that's so true. Yeah. So, may as [180:57] well catch up to today. They're [180:59] operating in five cities, Phoenix, San [181:00] Francisco, LA, Austin, and Atlanta. They [181:03] have hundreds of thousands of paid rides [181:05] every week. They've now driven over a [181:07] 100 million miles with no human behind [181:10] the wheel, growing at 2 million every [181:12] week. There's over 10 million paid rides [181:14] across 2,000 vehicles in the fleet. [181:17] They're going to be opening a bunch more [181:18] cities in the US next year. They're [181:20] launching in Tokyo, their first [181:21] international city, slowly and then all [181:24] at once. I mean, that's kind of the [181:26] lesson here. The technology, they really [181:28] continued with that multi-ensor approach [181:30] all the way from the DARPA Grand [181:32] Challenge. Camera, LAR, they added radar [181:35] and actually they use audio sensing as [181:37] well. And their approach is basically [181:39] any data that we can gather is better [181:41] because that makes it safer. So they [181:43] have 13 cameras, four LAR, six radar, [181:45] and the array of external microphones. [181:48] This is obviously way more expensive of [181:50] a solution than what Tesla is just doing [181:52] with cameras. But Whimo's party line is [181:56] they believe it is the only path to full [181:58] autonomy to hit the safety bar and [181:59] regulatory bar that they're aiming for. [182:01] >> Yeah. [182:02] >> It seems like a really big line in the [182:03] sand for them anytime you talk to [182:05] somebody in that organization. [182:06] >> Yeah. And look, as a regular user of [182:09] both products, you know, happy owner and [182:11] driver of a Model Y in addition to [182:13] regular Waybo user, at least with the [182:16] current instantiation of full [182:17] self-driving on my Tesla, vastly [182:20] different products. Full self-driving on [182:23] my Model Y is great. I use it all the [182:25] time on the freeway, but I would never [182:28] not pay attention. Whereas, every time I [182:30] get in a Whimo, it's almost like Google [182:32] search, right? It's like I just trust [182:33] that, oh, this is going to be completely [182:36] and totally safe and I'm sitting in the [182:38] back seat and I can totally tune out. [182:40] >> I think I trust my Model Y FSD more than [182:42] you do. But I get what you're saying and [182:44] frankly regulatory you are required to [182:46] still pay attention in Tesla and not in [182:48] the Whimo. The safety thing is super [182:50] real though. I mean, if you look at the [182:52] numbers, over a million motor vehicle [182:56] crashes cause fatalities every year or [182:59] there's over a million fatalities in the [183:01] US alone. Over 40,000 deaths occur per [183:04] year. So if you break that down, that's [183:06] 120 every day. That's like a giant cause [183:10] of death. [183:11] >> Yes. [183:12] >> The study that Whimo just released last [183:14] month showed that they have 91% fewer [183:17] crashes with serious injuries or worse [183:20] compared to the average human driver, [183:22] even controlled for the fact that Whimos [183:25] right now are only driving on city [183:26] surface streets. So they controlled it [183:28] apples to apples with human driving [183:31] data. And it's a 91% reduction in those [183:34] serious either fatality or a serious [183:36] injury things. Why aren't we all talking [183:38] about this all the time every day? This [183:40] is going to completely change the world [183:41] and a giant cause of death. [183:43] >> Yeah. [183:44] >> So, while we're in Whimo land, what do [183:46] you think about doing some quick [183:47] analysis? [183:48] >> Great. [183:49] >> Cuz I've been scratching my head here of [183:51] what is this business? Then I promise [183:53] we'll go back to the rest of Google AI [183:55] and catch up to today. It is super [183:58] expensive to operate especially at early [184:01] scale. The training is high, the [184:03] inference is high, the hardware is high [184:06] etc etc etc. [184:07] >> Also the operations are expensive. [184:09] >> Yes. And in fact they're experimenting. [184:11] Some cities they actually outsource the [184:13] operations. So the fleet is managed by [184:17] there's a rental car company in Texas [184:19] that manages it or they've partnered I [184:21] believe with Lyft and with Uber and [184:23] different. So they're trying all sorts [184:24] of O and O versus uh partnership models [184:27] to operate it. [184:28] >> Yeah. And the operations are like these [184:30] are electric cars. They need to be [184:31] charged. They need to be cleaned. They [184:33] need to be returned to depots. They need [184:35] to be checked out. They need to have [184:37] sensors replaced. [184:39] >> So the question is what is the potential [184:42] market opportunity? How big could this [184:44] business be? And there's a few different [184:45] ways you could try to quantify it. One [184:48] total market size thing you could do is [184:49] try to sum the entire automaker [184:52] market cap today and that would be 2.5 [184:56] trillion globally if you include Tesla [184:59] or 1.3 trillion without but Whimo is not [185:02] really making cars so that's probably [185:04] the wrong way to slice it. You could [185:06] look at all the ride sharing companies [185:07] today which might be a better comp [185:09] because that's the business that Whimo [185:10] is actually in today. That's on the [185:12] order of 300 billion most of which is [185:15] Uber. [185:16] >> Yep. So that's addressable market cap [185:18] today with ride sharing. Whimo's [185:20] ambitions though are bigger than that. [185:22] They want to be in the cars that you [185:24] own. They want to be in long haul [185:26] trucking. So they believe they can grow [185:29] the share of transportation because [185:31] there's blind people that could own a [185:33] car. There's elderly people who could [185:35] get where they need to go on their own [185:37] without having a driver. That sort of [185:38] thing. So the most squishy but I think [185:41] the most interesting way to look at it [185:43] is what is the value from all of the [185:47] reduction in accidents because that's [185:50] really what they're doing. It's a [185:51] product to replace accidents with [185:54] non-acs. [185:55] >> I think that's viable but again I would [185:57] say as a regular user of the product it [185:59] is a different and expanding product to [186:03] human ride share. So your argument is [186:05] whatever number I come up with for [186:07] reducing accidents, it's still a bigger [186:09] market than that because there's [186:11] additional value created in the product [186:12] experience itself. [186:13] >> Yeah. Scoping just to ride share now [186:16] that we have Whimo in San Francisco. I [186:19] use Whimo in scenarios where I would [186:22] never use an Uber or a Lyft. [186:24] >> Yeah, makes sense. So here's the data we [186:26] have. The CDC released a report saying [186:28] deaths from crashes in 2022 in the US [186:31] resulted in $470 billion in total costs, [186:35] including medical costs and the cost [186:37] estimates for lives lost, which is crazy [186:40] that the CDC has some way of putting the [186:42] costs on human life, but they do. So, if [186:44] you reduce crashes 10x, which is what [186:46] Whimo seems to be saying in their data, [186:48] at least for the serious crashes, that's [186:50] over $420 billion a year in total costs [186:53] that we would save as a nation. Now, [186:56] it's not totally apples to apples. I [186:58] recognize this, but that cost savings is [187:00] more than Google does today in revenue [187:04] in their entire business. You could see [187:06] a path to a Google sized opportunity for [187:08] Whimo as a standalone company just [187:11] through this analysis as long as they [187:13] figure out a way to get cost down to the [187:15] point where they can run this as a large [187:17] and profitable business. Yeah, it is a [187:21] incredible [187:22] 20 plus year success story within [187:25] Google. [187:25] >> The way I want to close it is the [187:28] investment so far actually hasn't been [187:29] that large. When you consider this [187:31] opportunity, they have burned somewhere [187:34] in the neighborhood of 10 to 15 billion. [187:36] That's sort of why I was listing all the [187:38] investments to get to this point. [187:40] >> Jump change compared to foundational [187:42] models. [187:43] >> Dude, also let's just keep it scoped in [187:45] this sector. That's one year of Uber's [187:49] profits. [187:50] >> Wow. Seems like a good bet. [187:52] >> I used to think this was like some wild [187:54] goose chase. It now looks really, really [187:57] smart. [187:58] >> Yep. Totally agree. [188:00] >> Also, that cost 10 to 15 billion is the [188:02] profits that Google made last month. [188:10] >> Google. Well, speaking of Google, should [188:13] we catch us up to today with Google AI? [188:16] >> Yes. So, I think where you were is the [188:19] Gemini launch. [188:20] >> So, Sundar makes these two decrees mid [188:25] 2023. One, we're merging Brain and Deep [188:29] Mind into one team for AI within Google. [188:33] And two, we're gonna standardize on one [188:35] model, the future Gemini and Deep [188:39] Mindbrain [188:41] team. You go build it and then everybody [188:43] in Google, you're going to use it. [188:44] >> Not to mention, apparently Sergey Bran [188:46] is like now back as an employee working [188:48] on Gemini. [188:49] >> Yes. [188:51] Employee number [188:52] >> got his badge back. [188:54] >> Yeah. Got his badge back. [188:57] So once Sundar makes these decisions, [189:01] Jeff Dean and Oriel Vignalis from Brain [189:04] go over and team up with the Deep Mind [189:06] team and they start working on Gemini. [189:09] >> I'm a believer now. By the way, you got [189:10] Jeff Dean working on it, I'm in. [189:12] >> If you got Jeff Dean on it, it's [189:13] probably going to work. If you weren't a [189:15] believer yet, wait till I'm going to [189:16] tell you next. Once they get Noom back [189:19] when they do the deal with Character AI, [189:21] bring him back into the fold. [189:23] Gnome joins the Gemini team and Jeff and [189:26] Gnome are the two co-technical leads for [189:30] Gemini now. So, [189:31] >> let's go. [189:32] >> Let's go. So, they actually announced [189:35] this very quickly at the Google IO [189:39] keynote in May 2023. They announced [189:43] Gemini. They announced the plans. They [189:46] also launch AI overviews in search first [189:49] as a labs product and then later that [189:52] becomes just standard for everybody [189:54] using Google search which is crazy by [189:56] the way the number of Google searches [189:58] that happen is unfathomably large I'm [190:01] sure there's a number for it but just [190:02] think about that's about the highest [190:05] level of computing scale that exists [190:07] other than like high bandwidth things [190:08] like streaming but just think about the [190:10] instances of Google searches that happen [190:13] they are running an LLM inference [190:15] on all of those or at least as many as [190:18] they're willing to show AI overviews on [190:20] which I'm sure is not every query but [190:22] many [190:22] >> a subset. [190:23] >> Yeah. [190:23] >> But still a large large number of Google [190:25] I mean I see them all the time. [190:27] >> Yep. [190:27] >> This is really Google immediately [190:30] deciding to operate at AI speed. I mean [190:32] chat GPT happened in November 30th 2022. [190:36] We're now in May 2023. [190:38] All of these decisions have been made. [190:40] all of these changes have happened and [190:41] they're announcing things at IO [190:43] >> and they're really flexing the [190:45] infrastructure that they've got. I mean [190:46] the fact that they can go like oh yeah [190:48] sure let's do inference on every query [190:49] we're Google we can handle it. [190:51] >> So a key part of this new Gemini model [190:54] that they announced in May 2023 is it's [190:58] going to be multimodal. Again this is [191:00] one model for everything text images [191:02] video audio one model. They release it [191:06] for early public access in December [191:10] 2023. So also crazy 6 months. They build [191:14] it, they trade it, they release it. [191:16] >> That is amazing. [191:17] >> While February 2024, they launched [191:20] Gemini 1.5 with a 1 million token [191:24] context window. Much much larger context [191:27] window than any other model on the [191:29] market, [191:30] >> which enables all sorts of new use [191:31] cases. There's all these people who were [191:33] like, "Oh, I tried to use AI before, but [191:35] it couldn't handle my XYZ use case." Now [191:37] they can. [191:38] >> Yep. The next year, February 2025, they [191:41] release Gemini 2.0. March of 2025, one [191:45] month later, they launch Gemini 2.5 Pro [191:49] in experimental mode. And then that goes [191:52] G in June. [191:53] >> This is like Nvidia pace, how often [191:55] they're shipping. [191:56] >> Yeah, seriously. And also in March of [191:59] 2025, they launch AI mode. So you can [192:02] now switch over on google.com to chatbot [192:06] mode. [192:06] >> And they're split testing auto opting [192:09] some people into AI mode to see what the [192:11] response is. This is the golden goose. [192:14] >> Yeah, the elephant is tap dancing here. [192:17] >> Yep. [192:17] >> Then there's all the other AI products [192:20] that they launch. So Notebook LM comes [192:22] out during this period. AI generated [192:25] podcasts [192:26] >> which does that sound like us to you? It [192:29] feels a little trained. [192:31] >> The number of texts that we got when [192:32] that came out of this must be trained on [192:35] acquired. [192:37] >> I do know that a bunch of folks on the [192:39] notebook LM team are acquired fans. So I [192:41] don't know if they trained on us. And [192:43] then there's the video the image stuff [192:45] VO3 Nano Banana Genie 3 that just came [192:48] out recently. Genie, this is insane. And [192:50] this is a world builder based on prompts [192:53] and videos. [192:54] >> Yeah. You haven't actually used it yet, [192:55] right? You watch that hype video. [192:57] >> Yeah, I watched the video. I haven't [192:58] actually used it. [192:59] >> Yeah. I mean, if it does that, that's [193:01] unbelievable. It's a real time [193:03] generative [193:05] >> world builder. [193:06] >> World builder. Yeah. You look right and [193:08] it invents stuff to your right. I mean, [193:10] you combine that with like a vision pro [193:12] hardware, you're just living in a [193:13] fantasy land. So, they announced there [193:16] are now 450 million monthly users of [193:21] Gemini. Now, that includes everybody [193:23] who's accessing Nano Banana. [193:25] >> Yeah, I can't believe this stat. This is [193:27] insane. Even with recently being number [193:29] one in the app store, it still feels [193:31] hard to believe. Google's saying it, so [193:33] it must be true. But I just wonder what [193:35] are they counting as use cases of the [193:38] Gemini app, [193:39] >> right? Certainly everybody who's using [193:41] Nano Banana is using Gemini. [193:43] >> But is it counting AI overviews or is it [193:45] counting AI mode or is it counting [193:47] something where I'm like accidentally [193:48] like Meta said that crazy high number of [193:51] people using Meta AI and [193:52] >> Right. Right. Right. [193:53] >> That was complete garbage. That was [193:54] people searching Instagram who [193:56] accidentally hit a llama model that made [193:58] some things happen and they were like, [194:00] "Uh, go away. I actually am just looking [194:01] for a user." Is it really 450 million or [194:04] is it 450 million? [194:06] >> Yeah, good question. Either way, going [194:09] from zero is crazy impressive in the [194:12] amount of time that they have done, [194:14] >> especially given revenue is at an [194:15] all-time high. They seem to so far be at [194:19] least in this squishy early phase able [194:21] to figure out how to keep the core [194:23] business going while doing well as a [194:27] competitor in the cutting edge of AI. [194:29] >> Yeah. And to foreshadow a little bit to [194:31] we're going to do a bull and bear here [194:33] in a minute. As we talked about in our [194:35] Alphabet episode, Google does have a [194:37] history of navigating platform shifts [194:40] incredibly well in the transition to [194:42] mobile. [194:43] >> It's true. [194:44] >> Definitely a rockier start here in the [194:46] AI platform shift. [194:48] Much rockier. But hey, look, I mean, if [194:51] you were to lay out a recipe for how to [194:55] respond given the rocky start, be hard [194:57] to come up with a much better slate of [194:59] things than what they've done over the [195:00] last two years. [195:02] >> Yeah. [195:04] All right. Should I give us the snapshot [195:05] of the business today? [195:07] >> Give us the snapshot of the business [195:08] today. Oh, yeah. Also, by the way, the [195:10] federal government decided they were a [195:12] monopoly and then decided not to do [195:14] anything about it because of AI. [195:16] >> Yeah. So, between the time when we [195:18] shipped our Alphabet episode and here [195:20] with our Google AI episode or our uh [195:23] part two and part three for those who [195:25] prefer simpler naming schemes. Yeah, [195:29] there was a US versus Google antitrust [195:32] case. The judge first ruled that Google [195:37] was a monopoly in internet search and [195:40] then did not come up with any material [195:42] remedies. I mean there are some, but I [195:45] would call them immaterial. They did not [195:47] need to spin off Chrome and they did not [195:49] need to stop sending tens of billions of [195:51] dollars to Apple and others. In other [195:53] words, yes, Google's a monopoly and the [195:56] cost of doing anything about that would [195:58] have too many downstream consequences on [196:00] the ecosystem. So, we're just going to [196:01] let them keep doing what they're doing. [196:04] And one of the reasons that the judge [196:06] cited of why they weren't going to [196:09] really take these actions is because of [196:11] the race in AI. That because tens of [196:15] billions of dollars of funding have gone [196:17] into companies like OpenAI and Anthropic [196:20] and Perplexity, Google essentially has [196:22] this new war to fight and we're going to [196:24] leave it to the free market to do its [196:26] thing where it creates viable [196:28] competition on its own and we're not [196:30] going to hamstring Google. Personally, I [196:33] think this argument is a little bit [196:34] silly. I mean, none of these AI [196:36] companies are generating net income, and [196:38] just because they've raised a huge [196:39] amount of money, it doesn't mean that [196:41] will last forever. They'll all burn [196:43] through their existing cash in a pretty [196:44] short period of time. And if the [196:46] spigotss ever dry up, Google doesn't [196:48] have any self-sustaining competition [196:50] right now, whether in their old search [196:51] business or in AI. It is all dependent [196:55] on people believing that the opportunity [196:57] is so large that they keep pouring tens [196:59] of billions of dollars into these [197:00] competitors. Yeah, plenty of other folks [197:03] have made the sort of glib comment, but [197:05] there's merit to it of, hey, as [197:07] flat-footed as Google was when Chat GPT [197:11] happened, if the outcome of this is they [197:15] avoid a Microsoft level distraction and [197:17] damage to their business from a US [197:21] federal court monopoly judgment. Worth [197:24] it. [197:25] >> Well, there's a funny meme here that you [197:27] could draw. You know that meme of [197:28] someone pushing the domino and it [197:30] knocking over some big wall later. [197:32] >> Yeah. [197:32] >> There's the domino of Ilia leaving [197:35] Google to start OpenAI and the [197:39] downstream effect is Google is not [197:41] broken up. [197:42] >> Yeah. Right. Exactly. [197:43] >> It actually saves Google. [197:45] >> It actually saves Google. [197:48] >> It's totally wild. [197:49] >> Totally wild. [197:50] >> All right. So, here's the business [197:51] today. Okay, over the last 12 months, [197:52] Google has generated $370 [197:55] billion [197:57] in revenue. On the earnings side, [197:59] they've generated [198:01] 140 billion over the last 12 months, [198:04] which is more profit than any other tech [198:07] company. And the only company in the [198:09] world with more earnings is Saudi [198:11] Aramco. Let's not forget Google is the [198:15] best business ever. And we also made the [198:17] point at the end of the Alphabet [198:20] episode, even in the midst of all of [198:23] this AI era and everything that's [198:25] happened over the last 10 years, the [198:27] last 5 years, Google's core business has [198:31] continued to grow 5x since the end of [198:36] our alphabet episode in 2015 2016. [198:39] >> Yeah. Market cap. Google surged past [198:42] their old peak of two trillion and just [198:44] hit that three trillion mark earlier [198:46] this month. They're the fourth most [198:48] valuable company in the world behind [198:50] Nvidia, Microsoft, and Apple. It's just [198:54] crazy. On their balance sheet, I [198:56] actually think this is pretty [198:56] interesting. I normally don't look at [198:57] balance sheet as a part of this [198:58] exercise, but it's useful. And here's [199:00] why. In this case, they have 95 billion [199:03] in cash and marketable securities. And I [199:05] was about to stop there and make the [199:06] point, wow, look how much cash and [199:08] resources they have. [199:09] >> I'm actually surprised it's not more. So [199:11] it used to be 140 billion in 2021 and [199:15] over the last four years they've [199:17] massively shift from this mode of [199:19] accumulating cash to deploying cash and [199:22] a huge part of that has been the capex [199:24] of the AI data center buildout. So [199:26] they're very much playing offense in the [199:29] way that Meta, Microsoft and Amazon are [199:31] in deploying that capex. But the thing [199:34] that I can't quite figure out is the [199:35] largest part of that was actually [199:39] buybacks and they started paying a [199:41] dividend. So if you're not a finance [199:44] person, the way to read into that is [199:46] yes, we still need a lot of cash for [199:48] investing in the future of AI and data [199:50] centers, but we still actually had way [199:53] more cash than we needed and we decided [199:54] to distribute that to shareholders. [199:56] >> Yeah, [199:57] >> that's crazy. [199:58] >> Best business of all time, right? That [200:01] illustrates what a crazy business their [200:04] core search ads business is. If they're [200:06] saying, "The most capital intense race [200:09] in business history is happening right [200:11] now. We intend to win it." [200:13] >> Yeah. [200:14] >> And we have tons of extra cash lying [200:17] around on top of what we think plus a [200:19] safety cushion for investing in that [200:21] capex race. [200:23] >> Yeah. [200:23] >> Yes. [200:25] >> Wow. So there are two businesses that [200:29] are worth looking at here. One is Gemini [200:32] to try to figure out what's happening [200:34] there and two is a brief history of [200:37] Google cloud. I want to tell you the [200:38] cloud numbers today but it's probably [200:39] worth actually understanding how did we [200:41] get here on cloud. [200:42] >> Yep. [200:43] >> First on Gemini because this is Google [200:45] and they have I think the most [200:47] obfuscated financials of any of the [200:49] companies we've studied. They anger me [200:50] the most in being able to hide the ball [200:53] in their financial statements. Of [200:54] course, we don't know Gemini specific [200:56] revenue. What we do know is there are [200:58] over 150 million paying subscribers to [201:02] the Google 1 bundle. Most of that is on [201:04] a very low tier. It's on like the $5 a [201:07] month, $10 a month. The AI stuff kicks [201:09] in on the $20 a month tier where you get [201:12] the premium AI features, but I think [201:14] that's a very small fraction of the 150 [201:16] million today. [201:16] >> Yeah, I think that's what I'm on. [201:18] >> But two things to note. One, it's [201:19] growing quickly. that 150 million is [201:22] growing almost 50% year-over-year. But [201:24] two is Google has a subscription bundle [201:28] that 150 million people are subscribed [201:30] to. And so I've kind of had it in my [201:33] head that AI doesn't have a future as a [201:36] business model that people pay money for [201:38] that it has to be ad supported like [201:39] search. [201:40] >> But hey, that's not nothing. That's like [201:43] a [201:44] >> that's almost half of America. [201:45] >> I mean, how many subscribers does [201:47] Netflix have? [201:48] >> Netflix is in the hundreds of millions. [201:50] Yeah, [201:57] >> there are realcaled [202:00] consumer subscription services. I owe [202:03] this insight to Shashir Moroto. We [202:05] chatted actually last night cuz I name [202:07] dropped him on the last episode and then [202:08] he heard it and so we reached out, we [202:10] talked and that's made me do a 180. I [202:12] used to think if you're going to charge [202:13] for something your total addressable [202:15] market shrunk by 90 to 99%. But he kind [202:18] of has this point that if you build a [202:21] really compelling bundle and Google has [202:23] the digital assets to build a compelling [202:25] bundle. [202:26] >> Oh my goodness. YouTube Premium, NFL [202:29] Sunday Ticket. [202:30] >> Yes. Stuff in the Play Store, YouTube [202:33] Music, all the Google One storage stuff. [202:36] They could put AI in that bundle and [202:38] figure out through clever bundle [202:39] economics a way to make a paid AI [202:43] product that actually reaches a huge [202:45] number of paying subscribers. Totally. [202:48] >> So, we really can't figure out how much [202:49] money Gemini makes right now. Probably [202:51] not profitable anyway. So, what's the [202:53] point of even analyzing it? [202:54] >> Yeah. But, okay, tell us the cloud [202:57] story. So, we intentionally did not [203:00] include cloud in our Alphabet episode. [203:03] >> Google part two effectively. [203:04] >> Google part two. Yes. because it is a [203:08] new product and now very successful one [203:11] within Google that was started during [203:13] the same time period as all the other [203:15] ones that we talked about during Google [203:17] part two. But it's so strategic for AI. [203:22] Yes, it is a lot more strategic now in [203:26] hindsight than it looked when they [203:28] launched it. So just quick background on [203:30] it, it started as Google App Engine. It [203:33] was a way in 2008 for people to quickly [203:36] spin up a backend for a web or soon [203:39] after a mobile app. It was a platform as [203:42] a service. So you had to do things in [203:44] this very narrow googly way. It was very [203:47] opinionated. You had to use this SDK. [203:49] You had to write it in Python or Java. [203:52] You had to deploy exactly the way they [203:53] wanted you to deploy. It was not a thing [203:55] where they would say, "Hey developer, [203:58] you can do anything you want. Just use [203:59] our infrastructure." It was opinionated. [204:02] super different than what AWS was doing [204:05] at the time and what they're still doing [204:06] today, which the whole world eventually [204:08] realized was right, which is cloud [204:11] should be infrastructure as a service. [204:12] Even Microsoft pivoted Azure to this [204:14] reasonably quickly where it was like, [204:16] you want some storage, we got storage [204:17] for you. You want a VM, we got a VM for [204:19] you. You want some compute, you want a [204:20] database, [204:21] >> we got you. [204:21] >> Fundamental building blocks. So [204:24] eventually, Google launches their own [204:25] infrastructure as a service in 2012. [204:27] Took four years. They launched Google [204:29] Compute Engine that they would later [204:31] rebrand Google Cloud Platform. That's [204:33] the name of the business today. The [204:35] knock on Google is that they could never [204:37] figure out how to possibly interface [204:39] with the enterprise. Their core [204:41] business, they made really great [204:43] products for people to use, that they [204:45] loved polishing, they made them all as [204:47] self-s serve as possible, and then the [204:49] way they made money was from [204:50] advertisers. And let's be honest, [204:53] there's no other choice but to use [204:55] Google search, [204:56] >> right? it didn't necessarily need to [204:58] have a great enterprise experience for [205:00] their advertising customers because they [205:02] were going to come anyway, [205:03] >> right? And so they've got this self-s [205:04] serve experience. Meanwhile, the cloud [205:07] is a knife fight. These are commodities [205:10] >> all about the enterprise. [205:11] >> It's the lowest possible price and it's [205:13] all about enterprise relationships and [205:15] clever ways to bundle and being able to [205:18] deliver a full solution. [205:19] >> You say solution, I hear gross margin. [205:21] >> Yes. But yes, so Google out of their [205:24] natural habitat in this domain [205:27] >> and early on they didn't want to give [205:28] away any crown jewels. They viewed their [205:30] infrastructure as this is our secret [205:32] thing. We don't want to let anybody else [205:34] use it. And the best software tools that [205:36] we have on it that we've written for [205:38] ourselves like big table or borg how we [205:40] run Google or disbelief. These are not [205:43] services that we're making available on [205:44] Google cloud. [205:45] >> Yeah. These are competitive advantages. [205:47] >> Yes. And then they hired the former [205:50] president of Oracle, Thomas Kurrion. [205:53] >> Yes. And everything kind of changed. So [205:57] 2017, 2 years before he comes in, they [205:59] had $4 billion in revenue 10 years into [206:02] running this business. 2018 is their [206:05] first very clever strategic decision. [206:07] They launched Kubernetes. The big [206:09] insight here is if we make it more [206:12] portable for developers to move their [206:15] applications to other clouds, the world [206:18] is kind of wanting multicloud here, [206:20] >> right? We're the third place player. We [206:22] don't have anything to lose. [206:24] >> Yes. [206:25] >> So we can offer this tool a kind of [206:27] counterposition against AWS and Azure. [206:29] >> We shift the developer paradigm to use [206:31] these containers. They orchestrate on [206:33] our platform and then you know we have a [206:35] great service to manage it for you. It [206:37] was very smart. So this kind of becomes [206:39] one of the pillars of their strategy is [206:40] you want multicloud, we're going to make [206:42] that easy and you can sure choose AWS or [206:45] Azure 2. It's going to be great. So [206:46] David, as you said, the former president [206:48] of Oracle, Thomas Currion, is hired in [206:51] late 2018. You couldn't ask for a better [206:55] person who understands the needs of the [206:57] enterprise than the former president of [206:59] Oracle. This shows up in revenue growth [207:02] right away. In 2020, they crossed 13 [207:05] billion in revenue, which was nearly [207:07] tripling in three years. They hired like [207:09] 10,000 people into the go to market [207:11] organization. I'm not exaggerating that. [207:14] And that's on a base of 150 people when [207:17] he came in, most of which were seated in [207:20] California, not regionally distributed [207:22] throughout the world. The funniest thing [207:24] is Google kind of was a cloud company [207:27] all along. They had the best engineers [207:29] building this amazing infrastructure, [207:31] >> right? They had the products, they had [207:33] the infrastructure, they just didn't [207:34] have the go to market organization, [207:36] >> right? And the productization was all [207:38] like googly. It was like for us, for [207:40] engineers. They didn't really build [207:42] things that let enterprises build the [207:44] way that they wanted to build. This all [207:45] changes. 2022, they hit 26 billion in [207:49] revenue. 2023, they're like a real [207:51] viable third cloud. They also flipped to [207:54] profitability in 2023. And today, [207:57] they're over $50 billion in annual [208:00] revenue run rate. It's growing 30% [208:02] year-over-year. They're the fastest [208:04] growing of the major cloud providers, 5x [208:07] in five years. And it's really three [208:09] things. It's finding religion on how to [208:12] actually serve the enterprise. It's [208:13] leaning into this multi cloud strategy [208:15] and actually giving enterprise [208:16] developers what they want. And three, AI [208:19] has been such a good tailwind for all [208:23] hyperscalers because these workloads all [208:25] need to run in the cloud because it's [208:26] giant amounts of data and giant amount [208:28] of compute and energy. But in Google [208:30] Cloud, you can use TPUs, which they make [208:33] a ton of, and everyone else is [208:36] desperately begging Nvidia for [208:38] allocations to GPUs. So, if you're [208:41] willing to not use CUDA and build on [208:44] Google Stack, they have an abundant [208:47] amount of TPUs for you. [208:49] >> This is why we saved cloud for this [208:52] episode. There are two aspects of Google [208:56] cloud that I don't think they forsaw [208:58] back when they started the business with [209:01] App Engine but are hugely strategically [209:03] important to Google today. One is just [209:07] simply that cloud is the distribution [209:10] mechanism for AI. So if you want to play [209:14] an AI today, you either need to have a [209:16] great application, a great model, a [209:19] great ship or a great cloud. Google is [209:24] trying to have all four of those. [209:26] >> Yes, [209:27] >> there is no other company that has I [209:31] think more than one. [209:33] >> I think that's the right call. Think [209:35] about the big AI players. Nvidia [209:37] >> chips [209:38] >> kind of has a cloud but not really. They [209:39] just have chips and they the best chips [209:41] and the chips everyone wants but chips. [209:42] And then you just look around the rest [209:44] of the big tech companies. Meta right [209:46] now only an application. They're [209:48] completely out of the race for the [209:51] frontier models at the moment. We'll see [209:52] what they're hiring spree yields. You [209:54] look at Amazon infrastructure, they have [209:57] application maybe. I don't actually know [209:59] if Amazon.com I'm sure it benefits from [210:02] LLMs in a bunch of ways. [210:03] >> Mainly it's cloud. [210:04] >> Yes, cloud and cloud leader. Microsoft [210:07] >> cloud. [210:08] >> It's just cloud, right? They make some [210:10] models, but [210:11] >> I mean they've got applications, but [210:12] yeah, cloud [210:13] >> cloud. Apple [210:15] >> nothing. Nothing. [210:18] >> AMD just chips. [210:20] >> Yep. Open AAI model. [210:24] >> Anthropic model. [210:25] >> Yep. [210:26] >> Yep. [210:26] >> These companies don't have their own [210:27] data centers. They are like making noise [210:29] about making their own chips, but not [210:31] really and certainly not at scale. [210:33] Google has scale data center, scale [210:36] chips, scale usage of model. I mean, [210:39] even just from google.com queries now on [210:41] AI overviews [210:42] >> and scale applications. [210:44] >> Yes. Yeah, they have all of the pillars [210:49] of AI and I don't think any other [210:50] company has more than one [210:52] >> and they have the very most net income [210:54] dollars to lose. [210:56] >> Right? So then there's the chip side [210:57] specifically of this. If Google didn't [210:59] have a cloud, it wouldn't have a chip [211:01] business. It would only have an internal [211:03] chip business. The only way that [211:06] external companies, users, developers, [211:09] model researchers could use TPUs would [211:12] be if Google had a cloud to deliver them [211:14] because there's no way in hell that [211:16] Amazon or Microsoft are going to put [211:17] TPUs from Google in their clouds. [211:19] >> We'll see. [211:20] >> We'll see. I guess [211:21] >> I think within a year it might happen. [211:23] There are rumors already that some [211:25] NeoClouds in the coming months are going [211:27] to have TPUs. [211:28] >> M interesting. Nothing announced, but [211:32] TPUs are likely going to be available in [211:34] Neocloud soon, which is an interesting [211:36] thing. Why would Google do that? Are [211:37] they trying to build an NVIDIA type [211:39] business where they make money selling [211:41] chips? I don't think so. I think it's [211:44] more that they're trying to build an [211:47] ecosystem around their chips the way [211:48] that CUDA does. And you're only going to [211:50] credibly be able to do that if your [211:52] chips are accessible in anywhere that [211:55] someone's running their existing [211:56] workloads. [211:57] >> Yep. be very interesting if it happens. [212:00] And you know, look, you may be right. [212:02] Maybe there will be TPUs in AWS or Azure [212:05] someday, [212:06] but I don't think they would have been [212:08] able to start there. If Google didn't [212:12] have a cloud and there weren't any way [212:13] for developers to use TPUs and start [212:16] wanting TPUs, [212:18] would Amazon or Microsoft be like, "Ah, [212:21] you know, all right, Google, we'll take [212:23] some of your TPUs even though no [212:25] developer out there uses them." Right. [212:28] >> All right. Well, with that, let's move [212:30] into analysis. I think we need to do [212:32] Bull and Bear on this one. [212:34] >> You have to this time. [212:35] >> Got to bring that back. [212:36] >> For these episodes in the present, it [212:38] seems like we need to paint the possible [212:39] futures. [212:40] >> Yes. Bringing back bull and bear. I love [212:42] it. Then we'll do playbook powers [212:45] quintessence. Bring it home. [212:47] >> Perfect. All right. So, here's my set of [212:51] bull cases. Google has distribution to [212:54] basically all humans as the front door [212:56] to the internet. They can funnel that [212:58] however they want. You've seen it with [213:00] AI overviews. You've seen it with AI [213:02] mode. Even though lots of people use [213:04] chat GBD for lots of things, Google's [213:07] traffic, I assume, is still essentially [213:09] an all-time high and it's a default [213:10] behavior. [213:11] >> Yep. Powerful. So that is a bet on [213:15] implementation that Google figures out [213:17] how to execute and build a great [213:19] business out of AI, but it is still [213:21] theirs to lose. [213:22] >> Yeah. And they've got a viable product. [213:24] It's not clear to me that Gemini is any [213:28] worse than OpenAI or Anthropics [213:31] products. [213:31] >> No, I completely agree. This is a value [213:33] creation, value capture thing. The value [213:34] creation is there in spades. The value [213:36] capture mechanism is still TBD. [213:38] >> Yeah. Google's old value capture [213:40] mechanism is one of the best in history. [213:43] So that's the issue at hand. Let's not [213:45] get confused that it's not like a good [213:46] exper it's a great experience. [213:48] >> Yeah. Yeah. Yeah. Okay. So we've talked [213:50] about the fact that Google has all the [213:52] capabilities to win an AI and it's not [213:54] even close. Foundational model chips [213:58] hyperscaler all this with self- [214:00] sustaining funding. I mean that's the [214:02] other crazy thing is you look at the [214:04] clouds have self-sustaining funding. [214:06] Nvidia has self-sustaining funding. None [214:08] of the model makers have self-sustaining [214:10] funding, so they're all dependent on [214:12] external capital. [214:13] >> Yeah. Google is the only model maker who [214:15] has self-sustaining funding. [214:17] >> Yes. Isn't that crazy? [214:18] >> Yeah. [214:19] >> Basically, all the other large scale [214:22] usage foundational model companies are [214:24] effectively startups. [214:26] >> Yes. [214:26] >> And Google's is funded by a money funnel [214:29] so large that they're giving extra [214:31] dollars back to shareholders for fun. [214:33] >> Yeah. [214:35] >> Again, we're in the bullc case. [214:36] >> Well, when you put it that way. Yeah, a [214:38] thing we didn't mention, Google has [214:40] incredibly fat pipes connecting all of [214:42] their data centers. After the dot crash [214:45] in 2000, Google bought all that dark [214:47] fiber for pennies on the dollar, and [214:49] they've been activating it over the last [214:51] decade. They now have their own private [214:54] backhole network between data centers. [214:56] No one has infrastructure like this. [214:58] >> Yep. [214:59] >> Not to mention that serves YouTube. [215:01] They're fat pipes, [215:04] >> which in and of itself is its own [215:06] bullcase for Google in the future. [215:09] >> That's a great point. [215:10] >> Yeah, Ben Thompson had a big article [215:11] about this yesterday at the time of [215:13] recording. [215:14] >> Yeah, that was like a mega bullc case [215:16] that Ben Thompson published this week [215:18] that it was an interesting point. A [215:20] textbased internet is kind of the old [215:22] internet. It's the first instantiation [215:24] of the internet because we didn't have [215:26] much bandwidth. The user experience that [215:28] is actually compelling is [215:31] >> video, [215:31] >> high resolution video everywhere all the [215:34] time. [215:35] >> We already live in the YouTube internet, [215:38] >> right? And not only can they train [215:39] models on really the only scale source [215:43] of UGC media across long form and short [215:46] form, but they also have that as the [215:50] number two search engine, this massive [215:51] destination site. So they previewed [215:53] things like you'll be able to buy AI [215:56] labeled or AI determined things that [215:58] show up in videos. And if they wanted [216:00] to, they could just go label every [216:02] single product in every single video and [216:04] make it all instantly shoppable. Doesn't [216:06] require any human work to do it. They [216:07] could just do it and then run their [216:09] standard ads model on it. That was a [216:12] mind expanding piece that Ben published [216:13] yesterday or I guess if you're listening [216:15] to this a few weeks ago about that. And [216:17] then there's also all the video AI [216:20] applications that they've been building [216:23] like Flow and VO. What is that going to [216:26] do for generating videos for YouTube [216:30] that will increase engagement and add [216:32] dollars for YouTube? [216:34] >> Yep. [216:34] >> Going to work real well. [216:36] >> Yep. They still have an insane talent [216:38] bench. Even though, you know, they've [216:40] bled talent here and there and lost [216:41] people. They have also shown they're [216:43] willing to spend billions for the right [216:44] people and retain them. unit economics. [216:48] Let's talk about unit economics of [216:49] chips. Everyone is paying Nvidia 75 80% [216:54] gross margins implying something like a [216:56] four or 5x markup on what it costs to [216:59] make the chips. A lot of people refer to [217:01] this as the Jensen tax or the Nvidia [217:03] tax. Uh you can call it that, you can [217:05] call it good business, you can call it [217:06] pricing power, you could call it [217:07] scarcity of supply, whatever you want. [217:10] But that is true. Anyone who doesn't [217:13] make their own chips is paying a giant [217:15] giant premium to Nvidia. Google has to [217:18] still pay some margin to their chip [217:20] hardware partner Broadcom that handles a [217:22] lot of the work to actually make the [217:23] chip interface with TSMC. I have heard [217:27] that Broadcom has something like a 50% [217:29] margin when working with Google on the [217:31] TPU versus Nvidia's 80%. But that's [217:35] still a huge difference to play with. A [217:38] 50% gross margin from your supplier or [217:40] an 80% gross margin from your supplier [217:42] is the difference between a 2x markup [217:46] and a 5x markup. [217:47] >> Yeah, I guess that's right. [217:48] >> When you frame it that way, it's [217:50] actually a giant difference of the [217:52] impact to your cost. So you might wonder [217:54] appropriately, well, are chips actually [217:57] the big part of the cost of like the [217:59] total cost of ownership of running one [218:01] of these data centers or training one of [218:02] these models? Chips are the main driver [218:05] of the cost. They depreciate very [218:07] quickly. I mean, this is at best a [218:09] five-year depreciation because of how [218:11] fast we are pushing the limits of what [218:14] we can do with chips, the needs of next [218:17] generation models, how fast TSMC is able [218:20] to produce. [218:21] >> Yeah. I mean, even that is ambitious, [218:22] right? If you think you're going to get [218:24] 5 years of depreciation on AI chips, [218:27] five years ago, we were still two years [218:30] away from chat GPT, [218:31] >> right? Or think about what Jensen said [218:33] at um we were at GTC this year. He was [218:36] talking about Blackwell and he said [218:37] something about Hopper and he was like, [218:39] "Eh, you don't want Hopper." My sales [218:41] guys are going to hate me, but like you [218:42] really don't want Hopper at this point. [218:44] I mean, these were the H100s. This was [218:46] the hot chip just when we were doing our [218:48] most recent NVIDIA episode. [218:49] >> Yes. Things move quickly. [218:51] >> Yes. So I've seen estimates that over [218:54] half the cost of running an AI data [218:56] center is the chips and the associated [219:00] depreciation. The human cost that R&D is [219:03] actually a pretty high amount because [219:04] hiring these AI researchers and all the [219:06] software engineering is meaningful. Call [219:08] it 25 to 33%. [219:11] The power is actually a very small part. [219:12] It's like 2 to 6%. So when you're [219:15] thinking about the economics of doing [219:16] what Google's doing, it's actually [219:17] incredibly sensitive to how much margin [219:20] are you paying your supplier in the [219:22] chips because it's the biggest cost [219:24] driver of the whole thing. [219:25] >> Mhm. [219:26] >> So I was sanity checking some of this [219:28] with Gavin Baker who's the partner at a [219:31] trades management to prep for this [219:32] episode. He's like a great public [219:34] equities investor who's studied the [219:36] space for a long time. We actually [219:37] interviewed him at the Nvidia GTC [219:39] pregame show and he pointed out normally [219:43] like in historical technology eras it [219:46] hasn't been that important to be the [219:48] lowcost producer. Google didn't win [219:51] because they were the lowest cost search [219:52] engine. Apple didn't win because they [219:53] were the lowest cost. You know, that's [219:54] not what makes people win. But this era [219:58] might actually be different because [219:59] these AI companies don't have 80% [220:02] margins the way that we're used to in [220:04] the technology business or at least in [220:06] the software business at best these AI [220:08] companies look like 50% gross margins. [220:11] So Google being definitively the lowcost [220:14] provider of tokens because they operate [220:17] all their own infrastructure and because [220:19] they have access to low markup hardware. [220:24] It actually makes a giant difference and [220:26] might mean that they are the winner in [220:28] producing tokens for the world. [220:30] >> Very compelling bill case there. [220:32] >> That's a weirdly winding analytical [220:34] bullcase, but it's kind of the if you [220:36] want to really get down to it, they [220:38] produce tokens. [220:39] >> Yep. I've got one more bullet point to [220:42] add to the bulcase for Google here. [220:46] Everything that we talked about in part [220:48] two, the Alphabet episode, all of the [220:52] other products within Google, Gmail, [220:54] Maps, Docs, Chrome, Android, that is all [220:58] personalized data about you that Google [221:01] owns that they can use to create [221:04] personalized AI products for you that [221:07] nobody else has. [221:08] >> Another great point. So really the [221:10] question to close out the bullc case is [221:13] is AI a good business to be in compared [221:15] to search. Search is a great business to [221:18] be in. So far AI is not. But in the [221:21] abstract again we're in the bullcase. So [221:23] I'll give you this. It should be. With [221:25] traditional web search you type in two [221:28] to three words. That's the average query [221:29] length. And I was talking to Bill Gross [221:31] and he pointed out that in AI chat [221:33] you're often typing 20 plus words. So [221:37] there should be an ad model that emerges [221:40] and ad rates should actually be [221:42] dramatically higher cuz you have perfect [221:45] precision, [221:45] >> right? You have even more intent. [221:47] >> Yes, you know the crap out of what that [221:50] user wants. So you can really decide to [221:52] target them with the ad or not. And AI [221:55] should be very good at targeting with [221:56] the ad. So it's all about figuring out [221:58] the user interface, the mix of paid [222:01] versus not, exactly what this ad model [222:03] is. But in theory, even though we don't [222:05] really know what the product looks like [222:06] now, it should actually lend itself very [222:09] well to monetization. [222:10] >> Yep. [222:10] >> And since AI is such a amazing [222:13] transformative experience, all these [222:15] interactions that were happening in the [222:17] real world or weren't happening at all [222:19] like answers to questions and being on a [222:22] time spent is now happening in these AI [222:24] chats. So, it seems like the pie is [222:27] actually bigger for digital interactions [222:29] than it was in the search era. So again, [222:32] monetization should kind of increase [222:34] because the pi increases there. [222:36] >> Yep. [222:37] >> And then you've got the bullcase of [222:38] Whimo could be its own Googleiz [222:40] business. [222:41] >> I was just thinking that yeah, that's [222:43] scoping all of this to a replacement to [222:46] the search market. Whimo and potentially [222:49] other applications of AI beyond the [222:53] traditional search market could add to [222:55] that, [222:55] >> right? And then there's the like galaxy [222:57] brain bullcase, which is if Google [222:59] actually creates AGI, none of this even [223:01] matters anymore. And like of course it's [223:03] the most valuable thing. [223:04] >> That feels out of the scope for an [223:06] acquired episode. [223:07] >> It's disconnected. [223:09] Yes, agree. Barecase. So far, this is [223:13] all fun to talk about, but then the [223:14] product shape of AI has not lent itself [223:16] well to ads. So despite more value [223:19] creation, there's way less value [223:21] capture. Google makes something like [223:24] $400ish dollars per user per year just [223:27] based on some napkin math in the US. [223:30] That's a free service that everyone uses [223:34] and they make $400ish dollars a year. [223:37] Who's going to pay $400 a year for [223:40] access to AI? It's a very thin slice of [223:42] the population. [223:43] >> Some people certainly will, but not [223:45] every person in America. [223:47] >> Some people will pay 10 million, but [223:49] right. So if you're only looking at the [223:52] game on the field today, I don't see the [223:56] immediate path to value capture. And [223:59] think about when Google launched in [224:01] 1998, it was only 2 years before they [224:04] had AdWords. They figured out an amazing [224:06] value capture mechanism instantly, very [224:09] quickly. Yep. Another bare case. Think [224:12] back to Google launch in 1998. It was [224:16] immediately obviously the superior [224:18] product. Yes, [224:19] >> definitely not the case today. [224:21] >> No, there's four, five great products. [224:24] >> Google's dedicated AI offerings in [224:26] chatbot was initially the immediately [224:29] obviously inferior product and now it's [224:32] arguably on par with several others, [224:35] right? They own 90% of the search [224:37] market. I don't know what they own of [224:38] the AI market, but it ain't 90%. Is it [224:41] 25%? I don't know. But at steady state, [224:44] it probably will be something like 25, [224:46] maybe up to 50%. But this is going to be [224:48] a market with several big players in it. [224:50] So even if they monetized each user, as [224:53] great as they monetize it in search, [224:55] they're just going to own way less of [224:57] them. [224:57] >> Yep. Or at least it certainly seems that [224:59] way right now. [225:00] >> Yes. AI might take away the majority of [225:03] the use cases of search. And even if it [225:06] doesn't, I bet it takes away a lot of [225:08] the highest value ones. [225:10] >> Mhm. [225:10] >> If I'm planning a trip, I'm planning [225:13] that in AI. I'm no longer searching on [225:15] Google for things that are going to land [225:17] Expedia ads in my face. [225:18] >> Or health, another huge vertical. [225:21] >> Hey, I think I might have something that [225:23] reminds me of misotheloma. Is it that or [225:26] not, [225:27] >> right? [225:27] >> Oh, where are you going to put the [225:28] lawyer ads? Maybe you put them there. [225:30] Maybe it's just an ad product thing, but [225:32] these are very high value [225:34] >> queries, [225:35] >> former searches that those feel like [225:37] some of the first things that are [225:38] getting siphoned off to AI. [225:39] >> Yep. [225:40] >> Any other bare cases? I think the only [225:42] other bare case I would add is that they [225:44] have the added challenge now of being [225:46] the incumbent this time around and [225:49] people and the ecosystem isn't [225:52] necessarily rooting for them in the way [225:54] that people were rooting for Google when [225:57] they were a startup and in the way that [225:59] people were still rooting for Google in [226:01] the mobile transition. I think the [226:04] startups have more of the hearts and [226:06] minds these days, [226:08] >> right? So, I don't think that's [226:10] quantifiable, but is just going to make [226:13] it all a little harder path to row this [226:16] time around. [226:17] >> Yep. You're right. They had this [226:18] incredible PR and public love tailwind [226:22] the first time around. [226:23] >> Yep. And part of that's systemic, too. [226:25] Like all of tech and all of big tech is [226:27] just generally more out of favor with [226:30] the country and the world now than it [226:32] was 10 or 15 years ago. [226:33] >> There's more important. It's just big [226:35] infrastructure. It's not underdogs [226:36] anymore. [226:37] >> Yep. And that affects the open AIS and [226:40] the anthropics and the startups too, but [226:43] I think to a lesser degree. [226:44] >> Yeah, they had to start behaving like [226:46] big tech companies really early in their [226:48] life compared to Google. I mean, Google [226:50] gave a Playboy interview during their [226:52] quiet period of their IPO. Times have [226:55] changed. [226:56] >> Well, I mean, given all the drama at [226:57] OpenAI, I I don't know that I [226:59] characterize them as acting like a [227:02] mature company. [227:03] >> Fair. Fair [227:06] >> company, entity, whatever they are. [227:08] >> Yes. [227:09] >> Yeah. But point taken. [227:11] >> Well, I worked most of my playbook into [227:13] the story itself. So, you want to do [227:14] power? [227:15] >> Yeah. Great. Let's move on and do power. [227:17] Hamilton Helmer's seven powers analysis [227:20] of Google here in the AI era. And the [227:23] seven powers are scale economies, [227:24] network economies, counterpositioning, [227:26] switching costs, branding, quartered [227:28] resource, and process power. And the [227:30] question is which of these enables a [227:33] business to achieve persistent [227:35] differential returns? What entitles them [227:38] to make greater profits than their [227:39] nearest competitor sustainably? Normally [227:42] we would do this on the business all up. [227:44] I think for this episode we should try [227:46] to scope it to AI products. [227:48] >> Yes, agreed. usage of Gemini AI mode and [227:52] AI overviews versus the competitive set [227:55] of anthropic open AI, perplexity, Grock, [227:59] meta AI, [228:01] >> etc. Scale economies for sure. Even more [228:05] so in AI than traditionally in tech. [228:08] >> Yeah, they're just way better. I mean, [228:10] look, they're amoritizing the cost of [228:12] model training across every Google [228:14] search. I'm sure it's some super [228:16] distilled down model that's actually [228:18] happening for AI overviews, but think [228:20] about how many inference tokens are [228:24] generated for the other model companies [228:27] and how many inference tokens are [228:28] generated by Gemini. They just are [228:31] amortizing that fixed training cost over [228:33] a giant giant amount of inference that I [228:36] saw some crazy chart. We'll send it out [228:38] to email subscribers. In April of 24, [228:42] Google was processing 10 trillion tokens [228:46] across all their surfaces. In April of [228:49] 25, that was almost 500 trillion. Wow. [228:54] >> That's a 50x increase in one year of the [228:58] number of tokens that they're vending [229:00] out across Google services through [229:02] inference. And between April of 25 and [229:05] June 25, it went from a little under 500 [229:08] trillion to a little under one [229:10] quadrillion tokens. Technically 980 [229:13] trillion, but they are now, cuz it's [229:15] later in the summer, definitely sending [229:17] out maybe even multiple quadrillion [229:20] tokens. [229:21] >> Wow. [229:24] >> Wow. So among all the other obvious [229:26] scale economies things of amortizing all [229:28] the costs of their hardware, they are [229:30] amortizing the cost of training runs [229:32] over a massive amount of value creation. [229:34] >> Yeah, scale economies must be the [229:38] biggest one. [229:39] >> I find switching costs to be relatively [229:41] low. I use Gemini for some stuff then [229:43] it's really easy to switch away. That [229:45] probably stops being the case when it's [229:47] personal AI to the point that you're [229:48] talking about integrating with your [229:50] calendar and your mail and all that [229:52] stuff. Yeah, the switching costs have [229:54] not really come out yet in AI products, [229:59] although I expect they will. [230:01] >> Yes, they have within the enterprise for [230:03] sure. [230:04] >> Yep. [230:04] >> Network economies. I don't think if [230:07] anyone else is a Gemini user, it makes [230:09] it better for me because they are [230:11] sucking up the whole internet whether [230:12] anyone's participating or not. [230:14] >> Yep, agree. I'm sure AI companies will [230:18] develop network economies over time. I [230:19] can think of ways it could work, but [230:21] yeah, right now, no. And arguably for [230:23] the foundational model companies, can't [230:26] think of obvious reasons right now. [230:28] Where does Hamilton put distribution? [230:31] Because that's a thing that they have [230:32] right now that no one else has despite [230:34] ChatGBT having the Kleenex brand. Google [230:37] distribution is still unbelievable. I [230:39] don't Is that a cornered resource? [230:40] >> Cornered resource, I guess. Yeah, [230:42] >> definitely have that. [230:43] >> Yeah, Google search is a cornered [230:45] resource for sure. [230:46] >> Certainly don't have counterpositioning. [230:48] They're getting counterpositioned. [230:49] >> Yeah. [230:50] >> I don't think they have process power [230:52] unless they were like coming up with the [230:54] next transformer reliably, but I don't [230:57] think we're necessarily seeing that. [230:58] There's great research being done at a [231:00] bunch of different labs. Branding they [231:02] have [231:03] >> Yeah, branding is a funny one, right? [231:05] Well, I was going to say it's a little [231:07] bit to my barecase point about they're [231:09] the incumbent. [231:10] >> It cuts both ways, but I think it's net [231:12] positive. [231:13] >> Yeah, probably. For most people, they [231:15] trust Google. Yeah, they probably don't [231:17] trust these who knows AI companies, but [231:19] I trust Google. I bet that's actually [231:21] stronger than any downsides as long as [231:23] they're willing to still release stuff [231:25] on the cutting edge. [231:26] >> Yep. [231:27] >> So, to sum it up, it's scale economies [231:29] is the biggest one. It's branding and [231:32] it's a cornered resource [231:33] >> and potential for switching costs in the [231:35] future. Yep. Sounds right to me. [231:37] >> But it's telling that it's not all of [231:39] them. You know, in search it was like [231:40] very obviously all of them or most of [231:42] them. [231:43] >> Yep. Quite telling. Well, I'll tell you, [231:45] after hours and hours spending multiple [231:48] months learning about this company, my [231:50] quintessence when I boil it all down is [231:53] just that this is the most fascinating [231:55] example of the innovators dilemma ever. [231:57] I mean, Larry and Sergey control the [231:59] company. They have been quoted [232:01] repeatedly saying that they would rather [232:04] go bankrupt than lose at AI. Will they [232:07] really? If AI isn't as good a business [232:11] as search, and it kind of feels like of [232:13] course it will be. Of course, it has to [232:15] be. It's just because of the sheer [232:16] amount of value creation. But if it's [232:17] not, and they're choosing between two [232:21] outcomes, one is fulfilling our mission [232:23] of organizing the world's information [232:25] and making it universally accessible and [232:28] useful and having the most profitable [232:32] tech company in the world. Which one [232:34] wins? [232:36] Cuz if it's just the mission, they [232:38] should be way more aggressive on AI mode [232:40] than they are right now. And full flip [232:42] over to Gemini. It's a really hard [232:44] needle to thread. I'm actually very [232:46] impressed at how they're managing to [232:48] currently protect the core franchise, [232:50] but it might be one of these things [232:52] where it's being eroded away at the [232:55] foundation in a way that just somehow [232:57] isn't showing up in the financials yet. [232:59] I don't know. [233:00] >> Yep. I totally agree. And in fact, [233:03] perhaps influenced by you, I think my [233:06] quintessence is a version of that, too. [233:10] I think if you look at all the big tech [233:14] companies, Google, as unlikely as it [233:17] seems, given how things started, is [233:20] probably doing the best job of trying to [233:24] thread the needle with AI right now. And [233:28] that is incredibly commendable to Sundar [233:32] and their leadership. They are making [233:34] hard decisions like we're unifying deep [233:38] mind and brain. We're consolidating and [233:41] standardizing on one model and we're [233:43] going to ship this stuff real fast while [233:47] at the same time not making rash [233:50] decisions. [233:50] >> It's hard. Rapid but not rash, you know. [233:54] >> Yes. And obviously we're still in early [233:57] innings of all this going on and we'll [234:00] see in 10 years where it all ends up. [234:03] Yeah. Being tasked with being the [234:06] steward of a mission and the steward of [234:08] a franchise with public company [234:10] shareholders is a hard dual mission and [234:14] Sundar and the company is handling it [234:16] remarkably well especially given where [234:19] they were 5 years ago. [234:20] >> Yep. And I think this will be one of the [234:23] most fascinating examples in history to [234:24] watch it play out. [234:26] >> Totally agree. Well, thus concludes our [234:30] Google series for now. [234:32] >> Yes. All right, let's do some carveouts. [234:35] >> All right, let's do some carveouts. [234:38] Well, first off, we have a uh very, very [234:42] fun announcement to share with you all. [234:45] The NFL called us. [234:46] >> We're going to the Super Bowl, baby. [234:48] >> Acquired is going to the Super Bowl. [234:50] This is so cool. [234:52] >> It's the craziest thing ever. [234:54] >> The NFL is hosting a innovation summit [234:59] the week of the Super Bowl, the Friday [235:01] before Super Bowl Sunday. The Super Bowl [235:03] is going to be in San Francisco this [235:05] year in February. And so it's only [235:07] natural coming back to San Francisco [235:09] with the Super Bowl that the NFL should [235:11] do an innovation summit. [235:13] >> Yep. [235:14] >> And we're going to host it. [235:15] >> That's right. So, the Friday before [235:18] there's going to be some great onstage [235:20] interviews and programming. Most of you, [235:23] you know, we can't fit millions of [235:25] people in a tiny auditorium in San [235:27] Francisco the week of the Super Bowl [235:29] when every other venue has tons of [235:31] stuff, too. So, there will be an [235:33] opportunity to watch that streaming [235:34] online. And as we get closer to that [235:36] date in February, we will make sure that [235:39] you all know a way that you can tune in [235:41] and watch the uh MCing, interviewing, [235:43] and festivities at hand. Super Bowl [235:46] week. [235:46] >> It's going to be an incredible, [235:48] incredible day leading up to an [235:51] incredible Sunday. [235:52] >> Yes. Well, speaking of sport, my carve [235:56] out is I finally went and saw F1. It is [235:59] great. I highly recommend anyone go see [236:02] it, whether you're an F1 fan or not. It [236:04] is just beautiful cinema. [236:06] >> Amazing. Did you see it in the theater [236:08] or [236:08] >> I did see in the theater. Yeah. [236:09] >> Wow. [236:09] >> I unfortunately missed the IMAX window, [236:11] but it was great. It was my first time [236:12] being in a movie theater in a while. And [236:14] whether you watch it at home or whether [236:15] you watch it in the theater, I recommend [236:16] the theater. But it's going to be a [236:18] great surround sound experience wherever [236:20] you are. [236:20] >> I haven't been to the movie theater [236:22] since the era tour. [236:24] >> Ah, [236:25] >> which I think is just more about the [236:27] current state of my family life with two [236:29] young children. [236:30] >> Yes. My second one, some of you are [236:33] going to laugh, is the Travel Pro [236:35] suitcase. [236:37] >> Ah, this is the brand that pilots and [236:39] flight attendants use, right? [236:41] >> Maybe. I think I've seen some of them [236:42] use it. Usually they use something [236:43] higherend like a Briggs and Riley or a [236:46] Tumi or, you know, Travel Pro is not the [236:49] most high-end suitcase, but I bought two [236:51] really big ones for some international [236:54] travel that we were doing with my [236:55] 2-year-old toddler. And I must say, [236:58] they're robust. The wheels glide really [237:01] well. They're really smooth. They have [237:03] all the features you would want. They're [237:05] soft shell, so you can like really jam [237:06] it full of stuff, but it's also a thick [237:09] amount of protection. So, even if you do [237:11] jam it full of stuff, it's probably not [237:12] going to break. This is approximately [237:15] the most budget suitcase you could buy. [237:16] I mean, I'm looking at the big honken [237:19] international check bag version. It's [237:22] $416 on Amazon right now. I've seen it [237:26] cheaper. They have great sales pretty [237:28] often. Everything about this suitcase [237:30] checked lots of boxes for me and I [237:33] completely thought I would be the person [237:34] buying the Ramoa suitcase or the [237:37] something very high-end and this is just [237:40] perfect. So, I think I may be investing [237:42] in more Travel Pro suitcases. [237:45] >> More Travel Pro. Nice. Nice. Well, I [237:47] mean, hey, look, for family travel, you [237:50] don't want nice stuff. [237:51] >> Yeah. I mean, I bought it thinking like [237:52] I'll just get something crappy for this [237:54] trip, but it's been great. I don't [237:55] understand why I wouldn't have a full [237:56] lineup of Travel Pro gear. So [237:58] >> amazing. [237:59] >> This is my like budget pick gone right [238:02] that I highly recommend for all of you. [238:04] >> I love how uh Acquired is turning into [238:06] the wire cutter here. [238:07] >> That's it for me today. [238:09] >> Great. All right. I have two carveouts. [238:13] I have one carve out and then I have a [238:15] update in my ongoing Google carveout [238:18] saga. But first, my actual carveout. [238:22] It is the Glue Guys podcast. [238:25] >> Oh, it's great. Those guys are awesome. [238:27] So great. Our buddy Robbie Gupta, [238:29] partner at Sequoia, and his buddies [238:32] Shane Badier, the former basketball [238:35] player, and Alex Smith, the former [238:37] quarterback for the 49ers and the Kansas [238:39] City Chiefs and the Redskins. Their [238:41] dynamic is so great. They have so much [238:43] fun. Half of their episodes, like us, [238:45] are just them, and then half of their [238:46] episodes are with guests. Ben and I, we [238:48] went on it a couple weeks ago. That was [238:50] really fun. When we were on it, we were [238:53] talking about this dynamic of some [238:55] episodes do better than others and [238:57] pressure for episodes and whatnot. And [238:59] the guys brought up this interview they [239:02] did with a guy named Wright Thompson. [239:04] And they said like, "Look, this is an [239:05] episode. It's got like 5,000 listens. [239:07] Nobody's listened to it. It's so good." [239:10] And the mentality that we have about it [239:14] is not that we're embarrassed that [239:15] nobody listened to it. It's that we feel [239:18] sorry for the people who have not yet [239:19] listened to it because it's so good. I [239:22] was like that is the way to think about [239:23] >> that's great [239:24] >> your episodes. [239:25] >> So here you are. You're giving everyone [239:26] the gift of [239:27] >> I'm giving everyone the gift because I [239:28] then I was like all right well I got to [239:29] go listen to this episode. Ray Thompson [239:33] I didn't know anything about him before [239:35] I probably read his work in magazines [239:37] over the years without realizing it. [239:39] >> He's the coolest dude. [239:40] >> He has the same accent as Bill Gurley. [239:43] So listening to him sounds like [239:45] listening to If Bill Gurley instead of [239:47] being a VC only wrote about sports and [239:51] basically dedicated his whole life to [239:53] understanding the mentality and [239:54] psychology of athletes and coaches. It's [239:57] so cool. It's so cool. It's a great [239:59] episode. Highly, highly, highly [240:01] recommend. [240:02] >> All right. Legitimately, I'm queuing [240:03] that up right now. [240:04] >> Great. That's my carve out. And then my [240:07] ongoing family video gaming saga in [240:12] Google part one. I said I was debating [240:15] between the Switch 2 and the Steam Deck. [240:17] >> That's right. First, you got the Steam [240:18] Deck because you decided your daughter [240:21] actually wasn't old enough to play video [240:22] games with you, so you just got the [240:23] thing for you. [240:24] >> The update was I went with the Steam [240:26] Deck for that reason. I thought if it's [240:28] just for me, it would be more ideal. I [240:31] have an update. [240:32] >> You also got a Switch. [240:34] >> Uh, no, not yet. [240:35] >> Okay. [240:35] >> But the most incredible thing happened. [240:39] My daughter noticed this device that [240:41] appeared in our house that dad plays [240:43] every now and then. And we were on [240:46] vacation and I was playing the Steam [240:48] Deck and she was like, "What's that?" [240:49] Well, let me tell you. [240:51] >> And I was playing I've been playing this [240:54] really cool indie old school style RPG [240:58] called Sea of Stars. It's like a chrono [241:00] trigger style Super Nintendo style RPG. [241:03] I'm playing it and my daughter comes up. [241:04] She's like, "Can I watch you play?" And [241:07] I'm like, "Hell yeah, you can watch me [241:09] play. I get to play video games and you [241:11] sit here and snuggle with me and like, [241:13] you know, amazing. [241:15] >> I get to play video games and call it [241:16] parenting." [241:17] >> Then it gets even better. Probably like [241:20] two weeks ago, we're playing. And she's [241:24] like, "Hey, Dad, can I try?" I'm like, [241:26] "Absolutely, you can try." I hand her [241:29] the Steam Deck and it was the most [241:32] incredible experience, one of the most [241:34] incredible experiences I've had as a [241:36] parent because she doesn't know how to [241:37] play video games and I'm watching her [241:40] learn how to like use a joystick and hit [241:43] the button. [241:43] >> Supervised learning. Yeah. Yeah. Yeah. [241:44] Supervised learning. I'm telling her [241:46] what to do and then within two or three [241:48] nights she got it. She doesn't even know [241:51] how to read yet, but she figured it out [241:52] and like I'm watching in real time. And [241:54] so now the last week it's turned to [241:57] mostly she's playing and I'm like [242:00] helping her asking questions of like [242:01] well what do you think you should do [242:02] here? Like you know should you go here? [242:04] I think this is the goal. I think this [242:06] is where it's so so fun. So I think I [242:09] might actually pretty soon her [242:11] birthday's coming up end up getting a [242:13] Switch so that we can play, you know, [242:16] together on the Switch, right? [242:18] >> But unintentionally the Steam Deck was [242:20] the gateway drug for my soon tobe [242:22] four-year-old daughter. That's awesome. [242:24] There you go. Parent of the year right [242:26] there. Getting to play video games and [242:28] Oh, honey. I got it. I'll I'll take it. [242:31] >> Oh, yeah. I got it. I got it. [242:34] >> All right. Well, listeners, we have lots [242:35] of thank yous to make for this episode. [242:38] We talked to so many folks who were [242:40] instrumental in helping put it together. [242:42] First, a thank you to our partners this [242:43] season. JP Morgan Payments, trusted, [242:45] reliable payments infrastructure for [242:47] your business, no matter the scale. [242:48] That's JPorggan.com/acquired. [242:51] Sentry, the best way to monitor for [242:53] issues in your software and fix them [242:54] before users get mad. That's [242:56] centry.io/acquired. [242:58] Workos, the best way to make your app [243:00] enterprise ready, starting with single [243:02] sign on in just a few lines of code. [243:04] Workos.com. And Shopify, the best place [243:07] to sell online, whether you're a large [243:09] enterprise or just a founder with a big [243:11] idea. Shopify.com/acquired. [243:14] The links are all in the show notes. As [243:16] always, all of our sources for this [243:17] episode are linked in the show notes. [243:20] Yes. First, Steven Levy at Wired and his [243:25] great classic book on Google in the [243:27] Plex, which has been an amazing source [243:29] for all three of our Google episodes. [243:31] Definitely go buy the book and read [243:33] that. Also to Parm Olsen at Bloomberg [243:35] for her book Supremacy about Deep Mind [243:38] and Open AI, which was a main source for [243:40] this episode. And I guess also to Kade [243:42] Mets, right, [243:42] >> for Genius Makers. Yeah. [243:44] >> Yeah. [243:44] >> Great book. Our research thank yous. Max [243:48] Ross, Liz Reed, Josh Woodward, Greg [243:51] Curado, Sebastian Thrun, Anna Patterson, [243:54] Brett Taylor, Clay Bavor, Dennis Asabis, [243:57] Thomas Kurrion, Sundar Pachai. A special [244:01] thank you to Nick Fox, who is the only [244:04] person we spoke to for all three Google [244:07] episodes for research. We got the hat [244:09] trick. [244:09] >> Yeah. to Arvin Navaratnam at Worldly [244:12] Partners for his great write up on [244:13] Alphabet linked in the show notes to [244:16] Jonathan Ross original team member on [244:19] the TPU and today the founder and CEO of [244:21] Grock that's Grock with a Q making chips [244:24] for inference to the Whimo folks [244:26] Dimmitri Doglov and Suzanne Fyion to [244:30] Gavin Baker from Atrades management to [244:32] MG Seagler writer at spy glassass MG is [244:35] just one of my favorite technology [244:37] writers and pundits [244:38] >> OG techrunch writer That's right to Ben [244:41] Idolen for being a great thought partner [244:43] on this episode and his excellent recent [244:45] episode on the Step Change podcast on [244:47] the history of data centers. I highly [244:49] recommend it if you haven't listened [244:51] already. It's only episode three for [244:53] them of the entire podcast and they're [244:54] already getting I don't know 30 40,000 [244:57] listens on it. I mean, this thing is [244:58] taking off. [244:59] >> Amazing, dude. That's way better than we [245:01] were doing on episode three. [245:03] >> It's way better than we were doing. And [245:04] if you like Acquired, you will love the [245:06] Step Change podcast. And Ben is a dear [245:08] friend. So, highly recommend checking it [245:10] out. To Cororai Kovaktalu from the [245:13] DeepMind team building the core Gemini [245:15] models to Shashir Maroda, the CEO of [245:18] Grammarly, formerly ran product at [245:20] YouTube. To Jim Gao, the CEO of Fedra [245:22] and former DeepMind team member, Chathan [245:24] Pudigonta, partner at Benchmark. Dwarash [245:27] Patel for helping me think through some [245:28] of my conclusions to draw. And to Brian [245:30] Lawrence from Oakcliffe Capital for [245:33] helping me think about the economics of [245:35] AI data centers. If you like this [245:37] episode, go check out our episode on the [245:39] early history of Google and the 2010s [245:41] with our Alphabet episode and of course [245:43] our series on Microsoft and Nvidia. [245:45] After this episode, go check out ACQ2 [245:47] with Toby Lutka, the founder and CEO of [245:50] Shopify. And come talk about it with us [245:52] in the Slack at acquire.fm/Slack. [245:56] And don't forget our 10th anniversary [245:58] celebration of acquired. We are going to [246:00] do a open Zoom call, an LP call just [246:04] like the days of your with anyone. [246:06] Listeners, come join us on Zoom. It's [246:08] going to be on October 20th at 400 p.m. [246:12] Pacific time. Details are in the show [246:14] notes. [246:15] >> And with that, listeners, we'll see you [246:16] next time. [246:17] >> We'll see you next time. [246:19] >> Who got the truth? [246:22] Is it you? Is it you? Is it you? Who got [246:24] the truth now? Huh? [246:28] [Music]