OpenAI, Broadcom Partner to Deploy 10-Gigawatt Custom AI Systems

OpenAI联手Broadcom部署10吉瓦定制AI系统 全球必须迈向绝对的算力充沛，因为若要为每个人提供全天候运行的个性化AI智能体，所需的基础设施规模将是天文数字，当前平台根本无法支撑。 本期内容公布了OpenAI与Broadcom已持续18个月的深度合作，双方正从单纯的芯片定制，转向从明年底开始构建完全集成的10吉瓦定制AI计算系统。双方高管探讨了利用OpenAI自身模型优化芯片布局的做法，并阐释了从晶体管级别到用户Token的垂直整合对于降低成本和实现效率最大化为何至关重要。在向全天候后台运行AI智能体转变的驱动下，OpenAI概述了将其算力规模从2吉瓦扩展至近30吉瓦的历史性使命。Broadcom则重点介绍了3D芯片堆叠和集成光学等前沿进展，以推动世界从算力匮乏迈向真正的算力充沛。 OpenAI 与博通（Broadcom）宣布已秘密合作 18 个月，双方合作范围已从定制推理芯片扩展为包含芯片、网络与机架的完整自研系统，并计划从明年底开始部署 10 吉瓦（GW）规模的计算基础设施。 (0:00) 双方通过从晶体管刻蚀到输出 token 的全栈垂直优化，力求实现每瓦智能产出的最大化；同时 AI 领域存在“需求悖论”，算力效率提升 10 倍反而会激发出 20 倍以上的新增需求。 (2:00) OpenAI 在芯片设计阶段直接调用了自家 AI 模型进行布局优化，大幅缩减了芯片面积并加速了交付流程；驱动这一庞大算力需求的正是 ChatGPT 正在向 24/7 全天候运行的后台自主 Agent 转型。 (5:16) 双方正在推进 3D 芯片堆叠以及单芯片 100 Terabits 硅光集成等尖端硬件技术，预计成果将于明年底初步落地并在随后三年内全面铺开，推动整个行业从算力稀缺迈向算力丰沛。 (23:50)

OpenAI and Broadcom have announced an 18-month-old partnership to co-design and deploy 10 gigawatts of custom silicon, racks, and networking systems starting late next year. Company leadership frames the vertically integrated build-out as necessary to support the immense computational demands of autonomous, 24/7 AI agents. Custom Silicon and Full-Stack Vertical Integration (0:00) OpenAI's collaboration with Broadcom began as an effort to design an application-specific chip for inference workloads, but expanded into co-designing an end-to-end system including server racks and networking architecture. Sam Altman explained that optimizing across the entire computing stack—from transistor layout to final output tokens—allows OpenAI to maximize intelligence produced per watt of power while lowering per-query operating costs. Hock Tan and Charlie Kawwas noted that the custom silicon effort involves manufacturing below the two-nanometer node, moving beyond standard 800-square-millimeter two-dimensional limits via 3D die stacking, and integrating optical switching directly into silicon packages (including a recently announced 100-terabit milestone). Tan distinguished workload requirements, observing that while model training prioritizes raw compute (TFLOPs) and massive scale-out networking, inference workloads depend primarily on high memory capacity and memory bandwidth relative to compute. AI-Assisted Engineering and Deployment Schedules (5:16) The collaboration leverages machine learning within the semiconductor design cycle itself. Greg Brockman reported that OpenAI applied its own models to optimize custom chip layouts, achieving significant floorplan area reductions and discovering routing optimizations that would have otherwise taken human engineering teams an additional month. Kawwas added that Broadcom software engineers are utilizing GPT tooling, delivering productivity gains comparable to dozens of engineers. Altman projected that the first silicon and system deployments will arrive in late next year, with the full 10-gigawatt capacity deployed over the following three years. Broadcom and OpenAI technical teams conduct weekly reviews to coordinate progress, with prototype silicon expected back for evaluation in the near term. Agentic Workloads and the Demand Paradox (20:20) The push for custom infrastructure is driven by a structural shift from interactive chat models to continuous, background-operating agents, such as OpenAI's "Pulse" feature. Brockman explained that providing personalized agents operating 24/7 for billions of people requires computing power orders of magnitude beyond current global hardware capacity, noting that agentic features are currently restricted to pro-tier subscriptions due to compute constraints. Altman argued that hardware efficiency improvements do not resolve capacity shortages due to a historical demand paradox: ten-fold efficiency gains consistently unlock twenty-fold increases in demand. As models progress from handling isolated code snippets to autonomously completing days or weeks of complex engineering tasks, OpenAI anticipates that commercial appetite for intelligence will rapidly saturate incoming capacity. Infrastructure Scaling From Scarcity to Abundance (15:30) OpenAI's focus on hardware stems from a 2017 empirical discovery in Dota 2, where scaling compute directly doubled reinforcement learning performance when other methods stalled. Altman traced the company's power footprint from an initial 2-megawatt cluster to 20 megawatts, 200 megawatts, and slightly over 2 gigawatts by the end of this year, with broader partnerships projected to expand OpenAI's total footprint toward roughly 30 gigawatts. The 10 gigawatts developed with Broadcom represents entirely incremental capacity alongside existing cloud and GPU supplier relationships. Kawwas and Tan compared the multi-decade build-out to the deployment of the railroad network and the internet, arguing that open, transparent networking standards across the industry are required to transform compute from an internally rationed resource into an abundant global utility. OpenAI and Broadcom project that custom silicon and 10 gigawatts of dedicated infrastructure will alleviate severe compute bottlenecks, though the sheer scale of the multi-decade deployment and global energy limitations leave open how quickly hardware capacity can catch up with the exponential demand of persistent AI agents.

OpenAI 与博通（Broadcom）宣布达成一项已秘密开展 18 个月的深度合作，计划从明年底开始部署总计 10 吉瓦（GW）的定制芯片、机架及网络系统。双方试图通过从晶体管制造到模型推理的全栈垂直整合，以应对从交互式聊天向全天候个人智能体演进带来的计算需求，并将能耗转化为智能的效率最大化。 全栈垂直整合与专用芯片定制路径 (2:00) 该合作始于约 18 个月前，最初聚焦于为 OpenAI 特定的推理负载设计专用 AI 加速芯片（XPU），但随着系统复杂度上升，合作范围已扩展至涵盖芯片、机架架构和网络互联的完整定制系统。Sam Altman 表示，这种“从蚀刻晶体管到产出 Token”的垂直整合优化，能够大幅降低推理成本并提高模型响应速度。他指出，随着计算规模扩大，能源终将成为主要限制瓶颈，垂直整合的核心目标是在每一个瓦特的能耗中压榨出最大化的智能产出。Hock Tan 指出，定制芯片使企业能够掌控自身技术命运；同时在硬件优化上，训练芯片更依赖原始算力（TFLOPs）与海量横向扩展网络，而推理芯片则对高内存容量和高访存带宽有更强依赖。Altman 补充说明，定制系统所提供的 10 吉瓦算力完全属于增量资源，OpenAI 仍将继续维持与英伟达、AMD 等现有 GPU 供应商的合作，以兼顾前沿研究所需的架构灵活性。 AI辅助芯片设计与底层硬件技术突破 (5:16) 在研发流程中，OpenAI 引入自身的 AI 模型直接参与芯片设计，在物理版图布局中实现了显著的面积缩减。Greg Brockman 透露，AI 模型发现了人类工程师列入长期规划但需数月才能完成的优化方案，在面临严苛工期时，团队选择信任 AI 的设计成果并推进研发。此外，博通自身软件工程师在日常工作中也深度使用 GPT 工具，Charlie Kawwas 称其产出的效率提升相当于增加了数十名工程师的工作量。 在底层半导体架构方面，面对传统芯片单颗约 800 平方毫米的 2D 物理面积极限，博通与 OpenAI 正在推动 3D 芯片堆叠（沿 Y 轴与 Z 轴垂直集成）以及 2 纳米以下先进制造工艺的落地。同时，双方在硅片级集成光学元件，博通近期公布了单芯片整合光互连实现 100 Tbps 交换能力的技术成果。Kawwas 预测，借助此类先进封装与光电集成技术，未来集群计算性能有望实现每 6 至 12 个月翻倍的演进速度。 算力指数级跃迁与智能体需求驱动 (20:20) 回顾算力演进历程，Altman 阐述了 OpenAI 能源消耗的增长轨迹：从最初启动训练集群时的 2 兆瓦（MW），逐步提升至 20 兆瓦、200 兆瓦，并预计在今年年底达到略高于 2 吉瓦（GW）。仅凭这 2 吉瓦算力，OpenAI 目前支撑了覆盖全球 10% 人口的 ChatGPT 服务、Sora 视频生成、API 接口调用以及前沿科研探索。结合最新合作规划，OpenAI 的总体规划算力储备预计将推升至近 30 吉瓦。 驱动如此庞大硬件需求的核心在于模型交互形态的本质转变。Brockman 解释称，AI 正在从传统的交互式单次聊天工具，演变成为用户在后台 24 小时全天候运行的个性化自主智能体（例如 Pulse 功能）。若要让全球百亿级人口均拥有专属智能体，当前算力仍是沧海一粟。Altman 指出，历史上算力能效提升 10 倍往往不会导致产能闲置，反而因成本降低和能力跃升（如编程模型从单次粘贴代码进化到可自主完成数周高级工程任务）而激发 20 倍以上的全新经济需求，算力需求在可预见的未来仍将维持极度饥渴状态。 工业级基建规模、标准与部署时间表 (9:28) 双方将本次 AI 基础设施建设定义为长达数十年的工业演进过程。Kawwas 与 Tan 将其与需要百年普及的铁路网、耗时约 30 年的互联网以及工业革命相提并论，指出单个 1 吉瓦规模的数据中心其复杂程度与体量已相当于一座小型城市，Altman 亦视其为人类历史上最大规模的跨行业、跨国界协同工业项目。为避免专有封闭体系造成的生态割裂，Tan 呼吁在系统架构与网络互联层面推行开放、透明的行业标准，以促进全球软硬件生态共同扩容。 在具体执行推进上，Altman 预计该合作的首批硬件成果将在明年底交付，并在随后的三年内实现大规模部署上线。Brockman 表示双方管理团队每周保持高频同步推进。通过自研系统的规模化交付，OpenAI 期望彻底逆转目前因资源受限导致的内部与外部算力争夺，促成计算资源由严重稀缺走向全面充裕，使智能算力成为普惠的基础设施。 尽管 OpenAI 与博通描绘了从定制芯片到 10 吉瓦系统集群的宏大蓝图，但将 2 纳米以下制程、3D 芯片堆叠及共封装光学技术投入超大规模生产仍伴随极高的工程交付难度。此外，面对能源供给这一物理制约，数十吉瓦数据中心的电力落地周期能否追赶上智能体负载的爆发式增长，以及开放互联标准能否在巨头竞争中确立，仍是长期演进中悬而未决的关键问题。

[2:00] Sam Altman: from like etching the transistors all the way up to the token that comes out [5:16] Greg Brockman: We are nowhere near being able to build 10 billion chips [9:28] Charlie Kawwas: We're defining civilization's next-generation operating system. [9:28] Sam Altman: the biggest joint industrial project in human history [15:30] Greg Brockman: everything else didn't work nearly as well. [15:30] Greg Brockman: 10 gigawatts is not enough. That is a drop in the bucket compared to where we need to go. [17:03] Sam Altman: melt sand, run energy through it and get intelligence out the other end. [17:03] Sam Altman: wring out so much more intelligence per watt. [17:03] Hock Tan: control your destiny. [17:03] Sam Altman: incredible, incredible things [18:49] Charlie Kawwas: is not going to take five years. It's going to take a long time. [23:50] Sam Altman: miracle of technology chip quickly and at scale [23:50] Charlie Kawwas: dozens of engineers. [27:04] Greg Brockman: astronomical amount of work

[0:00] Andrew Mayne: Hello, I'm Andrew Mayne, and welcome to the OpenAI podcast. [0:03] Andrew Mayne: Today, we're excited to be breaking some news involving Broadcom and OpenAI. [0:08] Andrew Mayne: Joining me from OpenAI is Sam Altman and Greg Brockman, [0:11] Andrew Mayne: and from Broadcom: Hock Tan and Charlie Kawwas. [0:14] Sam Altman: A lot of ways that you would look at the AI infrastructure build-out right now, [0:17] Sam Altman: you would say it's the biggest joint industrial project in human history. [0:23] Charlie Kawwas: We're defining civilization's next-generation operating system. [0:27] Greg Brockman: That is a drop in the bucket compared to where we need to go. [0:30] Sam Altman: It's a big drop... [0:34] Andrew Mayne: So what are we talking about today? [0:35] Andrew Mayne: What brought you all together? [0:37] Sam Altman: So today we're announcing a partnership between Broadcom and OpenAI. [0:40] Sam Altman: We've been working together for about the last 18 months designing a new custom chip. [0:44] Sam Altman: More recently, we've also started working on a whole custom system. [0:47] Sam Altman: These things have gotten so complex, you need the whole thing. [0:50] Sam Altman: And we will be starting in late next year, [0:52] Sam Altman: deploying 10 gigawatts of these racks of these systems and our chip, [0:56] Sam Altman: which is a gigantic amount of computing infrastructure to serve the needs of the world to use advanced intelligence. [1:02] Andrew Mayne: So this is going to entail both compute and chip design and scaling out? [1:08] Sam Altman: This is a full system. [1:10] Sam Altman: So we worked, we closely collaborated for a while on designing a chip that is specific for our workloads. [1:16] Sam Altman: When it became clear to us just how much capacity, inference capacity, the world was going to need, [1:21] Sam Altman: we began to think about, could we do a chip that was meant just for that kind of a very specific workload? [1:29] Sam Altman: Broadcom is the best partner in the world for that, obviously. [1:32] Sam Altman: And then to our great surprise, this was not the way we started. [1:34] Sam Altman: But as we realized that we were going to really need the whole system together to support this, [1:40] Sam Altman: as this got more and more complex, it turns out Broadcom is also incredible at helping design systems. [1:47] Sam Altman: So we are working together on that entire package, and this will help us even further increase the amount of capacity we can offer for our services. [2:00] Andrew Mayne: So, Hock, how did this come about? [2:01] Andrew Mayne: When did this start? [2:03] Andrew Mayne: When did you guys first talk about working together on this? [2:05] Hock Tan: Well, other than the fact that Sam and Greg are great people to work with, it's a natural fit because OpenAI has been doing, continues to do the most advanced models, frontier models in generative AI out there. [2:23] Hock Tan: But as part of it, you continue to need compute capacity, the best, latest compute capacity as you progress in a roadmap towards a better and better frontier model and towards super intelligence. [2:39] Hock Tan: And compute is a key part, and that comes with semiconductors, and as Sam indicated, more than semiconductors. [2:48] Hock Tan: And we are, even though I say it myself, probably the best semiconductor company out there. [2:54] Hock Tan: And more than that, AI is a very, very exciting opportunity for us in terms of we are, my engineers are pushing the innovation envelope and newer and newer generations of semiconductor technology. [3:12] Hock Tan: So for us, collaborating with the best generative AI company out there is a natural fit. [3:20] Andrew Mayne: And this isn't just chips, it's going out to scale like 10 gigawatts. [3:23] Andrew Mayne: And I have trouble kind of even understanding that. [3:25] Andrew Mayne: What does that even mean when you're talking about 10 gigawatts? [3:28] Sam Altman: First of all, you said it's not just chips that Hock touched on this too, [3:30] Sam Altman: but the vertical integration point is really important. [3:33] Sam Altman: We are able to think from like etching the transistors all the way up to the token that comes out [3:41] Sam Altman: ask ChatGPT a question and design the whole system, all of the stuff about the chip, the way we design [3:48] Sam Altman: these racks, the networking between them, how the algorithms that we're using fit the inference chip [3:53] Sam Altman: itself, a lot of other stuff all the way to the end product. And one of the many reasons I'm so [3:59] Sam Altman: excited about it is by being able to optimize across that entire stack, we can get huge efficiency gains [4:05] Sam Altman: and that will lead to much better performance, faster models, cheaper models, all of that. [4:11] Sam Altman: As you get that better performance and cheaper and smarter models, one thing that we have consistently seen is people just want to use way more. [4:19] Sam Altman: So we used to think like, oh, we'll optimize things by 10x and we'll solve all of our problems. [4:24] Sam Altman: But you optimize by 10x and there's 20x more demand. [4:28] Sam Altman: So 10 gigawatts, 10 incremental gigawatts, this is all on top of what we're already doing with other partners and all the other data centers and silicon partnerships we've done. [4:38] Sam Altman: 10 gigawatts is a gigantic amount of capacity. [4:41] Sam Altman: And yet, if we do as good of a job as we hope, even though it's vastly more than the world has today, [4:47] Sam Altman: we expect that very high-quality intelligence delivered very fast and at very low price, [4:53] Sam Altman: the world will absorb it super fast and just find incredible new things to use it for. [4:58] Sam Altman: So what is the hope with this? [4:59] Sam Altman: The hope is that the kinds of things people are doing now with this compute, [5:06] Sam Altman: writing code, doing more and automating more and more of enterprises, generating videos in Sora, [5:11] Sam Altman: whatever it is, they will be able to do that much more of it and with much smarter models. [5:16] Andrew Mayne: It's amazing. So Greg and Charlie, when you think about historically, when people have tried to [5:22] Andrew Mayne: develop chips or hardware to suit whatever was the current modem for using computing at that point, [5:28] Andrew Mayne: what examples have you looked upon historically to figure out how to plan forward? What's been inspiring you when you think about this? [5:32] Greg Brockman: Well, I'd say the number one thing, honestly, is working [5:36] Greg Brockman: with good partners. I think it's very clear that we, as a company, are not able to do everything [5:42] Greg Brockman: ourselves. And getting into actually building our own chips for our own specific workloads [5:47] Greg Brockman: was not something we could do from a total standstill without working with Hock and Charlie [5:51] Greg Brockman: and Broadcom. So, it's just been really incredible to lean on their expertise, together with our [5:56] Greg Brockman: understanding of the workload. And it's been actually very interesting to see the places [6:00] Greg Brockman: where OpenAI is able to do things very differently [6:03] Greg Brockman: from the rest of the industry [6:04] Greg Brockman: or the way that things would historically be done. [6:06] Greg Brockman: For example, we've been able to apply our own models [6:09] Greg Brockman: to designing this chip, which has been really cool. [6:12] Greg Brockman: We've been able to pull on the schedule. [6:13] Greg Brockman: We've been able to get massive area reductions, right? [6:15] Greg Brockman: You take components that humans have already optimized [6:18] Greg Brockman: and just pour compute into it, [6:21] Greg Brockman: and the model comes up with its own optimizations. [6:24] Greg Brockman: And it's very interesting. [6:25] Greg Brockman: We're at the point now where I don't think [6:27] Greg Brockman: any of the optimizations we have [6:29] Greg Brockman: are ones that human designers couldn't have come up with. [6:31] Greg Brockman: Like usually our experts take a look at it later [6:34] Greg Brockman: and say, yeah, like this was on my list, [6:36] Greg Brockman: but it was like 20 things [6:37] Greg Brockman: that would have taken them another month to get to. [6:40] Greg Brockman: And that's actually really, really interesting [6:41] Greg Brockman: that we were coming up on a deadline [6:44] Greg Brockman: working with Charlie's team [6:45] Greg Brockman: and we were running optimizations. [6:47] Greg Brockman: We had a choice of, [6:48] Greg Brockman: do we actually take a look [6:49] Greg Brockman: at what those optimizations were [6:51] Greg Brockman: or do we just keep going until the deadline [6:53] Greg Brockman: and then take a look after? [6:54] Greg Brockman: And we decided, of course, [6:55] Greg Brockman: you got to just keep going. [6:56] Greg Brockman: And so we've really been building up this expertise [6:59] Greg Brockman: in-house to understand this domain. [7:01] Greg Brockman: And that's something we actually think [7:02] Greg Brockman: can help lift up the whole industry. [7:04] Greg Brockman: But I think that we are heading to a world [7:05] Greg Brockman: where AI intelligence is able to help humanity [7:08] Greg Brockman: make new breakthroughs [7:09] Greg Brockman: that just would not be possible otherwise. [7:11] Greg Brockman: And we're going to need just as much compute [7:13] Greg Brockman: as possible to power that. [7:15] Greg Brockman: Like one example of something very concrete [7:17] Greg Brockman: is that we are in a world now [7:20] Greg Brockman: where ChatGPT is changing from something [7:22] Greg Brockman: that you talk to interactively [7:23] Greg Brockman: to something that can go do work for you behind the scenes. [7:26] Greg Brockman: If you've used features like Pulse, [7:28] Greg Brockman: You wake up every morning. [7:29] Greg Brockman: It has some really interesting things [7:32] Greg Brockman: that are related to what you're interested in. [7:34] Greg Brockman: It's very personalized. [7:36] Greg Brockman: And our intent is to turn ChatGPT [7:38] Greg Brockman: into something that helps you achieve your goals. [7:40] Greg Brockman: The thing is, we can only release this to the pro tier [7:43] Greg Brockman: because that's the amount of compute that we have available. [7:45] Greg Brockman: And ideally, everyone would have an agent [7:47] Greg Brockman: that's running for them 24-7 behind the scenes, [7:50] Greg Brockman: helping them achieve their goals. [7:52] Greg Brockman: And so ideally, everyone has their own accelerator, [7:54] Greg Brockman: has their own compute power that's just running constantly. [7:58] Greg Brockman: And that means there's 10 billion humans. [8:00] Greg Brockman: We are nowhere near being able to build 10 billion chips. [8:03] Greg Brockman: And so there's a long way to go before we are able to saturate not just the demand, but what humanity really deserves. [8:10] Andrew Mayne: So, Charlie, being very deeply technical and being with a company that's been at a number of forefronts of some of these revolutions, [8:16] Andrew Mayne: what's it been like working with a company like OpenAI and working with Greg on this? [8:20] Charlie Kawwas: So for us, it's been absolutely exciting and refreshing because the beauty of the work we do together is focus on a certain workload. [8:27] Charlie Kawwas: We started actually first looking at the IP and AI accelerator, which is what we call the XPU. [8:35] Charlie Kawwas: And then we realized very quickly that we now can actually go to the workload all the way down to the transistor. [8:42] Charlie Kawwas: And as Greg was just explaining, how we can both work together to go customize that platform for your workload, resulting in the best platform in the world. [8:52] Charlie Kawwas: Then we realized, as Sam was saying earlier on, it's not just that XPU or accelerator. [8:58] Charlie Kawwas: Actually, it's the networking that needs to go to scale it up, scale it out, and scale it across. [9:03] Charlie Kawwas: And so suddenly we started seeing that we actually can drive next level of standardization and openness that not just only benefits us. [9:12] Charlie Kawwas: I think it actually will benefit the entire ecosystem and it gets Gen AI to an AGI much faster. [9:17] Charlie Kawwas: So very excited about the technical capabilities of the teams we have. [9:22] Charlie Kawwas: but also the vision and I think the speed at which we've been moving. [9:28] Andrew Mayne: I'm still kind of wrapping my head around the scale of it [9:29] Andrew Mayne: because it's just from both trying to design something like a chip [9:33] Andrew Mayne: and to help figure out how you're going to get the maximum efficiency on this [9:36] Andrew Mayne: to just the size of it, the infrastructure, what's involved in this. [9:39] Andrew Mayne: This is a global effort. [9:41] Andrew Mayne: And what comparisons have you been able to draw for this to other examples in history? [9:45] Sam Altman: I always think the historical analogies are tough, [9:50] Sam Altman: But this is, as far as I know, I don't know what fraction building the Great Wall was of global GDP at the time. [10:00] Sam Altman: But a lot of ways that you would look at the AI infrastructure build out right now, you would say it's the biggest joint industrial project in human history. [10:09] Sam Altman: And this is like, this requires a lot of companies, a lot of countries, a lot of industries to come together. [10:16] Sam Altman: And a lot of stuff has to happen at the same time, and we've all got to kind of like invest together. [10:20] Sam Altman: But at this point, given everything we see coming on the research front, given all of the value we see being created on the business front, I think the whole industry has decided this is like a very good bet to take. [10:36] Sam Altman: But it is huge. [10:37] Sam Altman: You go to one of these even one gigawatt data centers, and you look at the scale of what's happening there. [10:42] Sam Altman: It's like a tiny city. [10:44] Sam Altman: it's a big complex thing. So it is just like incredible skill. [10:53] Greg Brockman: To the point of this being a massive collaborative project, I feel like whenever I call Charlie, he's in a different part of the [10:57] Greg Brockman: world trying to secure capacity, trying to find a way to help us build what we're trying to do together. [11:01] Charlie Kawwas: Exactly. Actually, one of the coolest thing actually I was thinking about is what we're [11:06] Charlie Kawwas: doing together in this wonderful partnership. We're defining civilization's next generation [11:12] Charlie Kawwas: operating system. And we're doing it, as you're saying, at the transistor level, building new [11:17] Charlie Kawwas: fabs, building new manufacturing sites, all the way to building these racks, and ultimately the [11:23] Charlie Kawwas: data centers you're talking about, 10 gigawatts of data centers. Yeah, I think it's an important [11:26] Andrew Mayne: thing to keep track of, is often people get fixated just on the chips themselves. And it's [11:31] Andrew Mayne: kind of like thinking the National Highway Project was about selling asphalt, or railroads are about [11:35] Andrew Mayne: steel. In reality, it's the things become possible on top of that. And you've probably thought a lot about that? Like what happens? [11:39] Hock Tan: Well, I think this is like railroad, internet. That's what I think [11:46] Hock Tan: this is becoming over time, critical infrastructure or critical utility and more than just critical [11:54] Hock Tan: utility for say 10,000 enterprises. This is critical utility over time, right Sam, for 8 billion people [12:03] Hock Tan: globally. That's, I think, it's like the industrial revolution of a different sort coming [12:10] Hock Tan: But it cannot be done with just one party or we like to think it can be done with two. [12:16] Hock Tan: But more than that, it needs a lot of partnerships. [12:19] Hock Tan: It needs a collaboration across an ecosystem. [12:24] Hock Tan: And also because of that, it's important to create, much as we say about developing chips for specific workloads, applications and LLM. [12:34] Hock Tan: It also requires somewhat standards that are open, more transparent for all to use because you need to build up a whole infrastructure at the end of the day to become a critical utility for 6 billion people in the world. [12:50] Hock Tan: And we're very excited, frankly, which is why we think we make great partners because I think we share the same conviction. [13:00] Hock Tan: And more than that, it is about scaling computing to create breakthroughs in super intelligence and models. [13:10] Hock Tan: It's building the foundation of that. [13:13] Andrew Mayne: You guys have a lot on your plate. [13:15] Andrew Mayne: Why design chips now? [13:17] Greg Brockman: Well, you know, this project, we've probably been working on it for 18 months now, and it's moved incredibly quickly. [13:24] Greg Brockman: We've hired some really amazing people. [13:27] Greg Brockman: And I think what we found is that we have a deep understanding of the workload. [13:33] Greg Brockman: And we work with a number of parties across the ecosystem. [13:36] Greg Brockman: And there's a number of chips out there that I think are really incredible. [13:41] Greg Brockman: And there's a niche for each one. [13:43] Greg Brockman: And so we've really been looking for specific workloads that we feel are underserved. [13:48] Greg Brockman: How can we build something that will be able to accelerate what's possible? [13:52] Greg Brockman: And so I think that that ability to say [13:56] Greg Brockman: that we are able to do the full vertical integration [13:59] Greg Brockman: for something we see coming, [14:00] Greg Brockman: but it's hard for us to work through other partners, [14:02] Greg Brockman: like that's a very clear use case for this kind of project. [14:04] Hock Tan: Yeah, actually more than that, [14:06] Hock Tan: and Greg, you put it very well. [14:08] Hock Tan: Really why you want to do your chip is computing [14:12] Hock Tan: is a big part of what's gating this journey [14:17] Hock Tan: towards super intelligence, [14:18] Hock Tan: towards creating better and better frontier model. [14:21] Hock Tan: It really, a lot of it down to computing, and not just any computing, [14:25] Hock Tan: computing that is effective, high performance, and efficient, [14:30] Hock Tan: given especially on power. [14:32] Hock Tan: And what Greg is saying is exactly what we learned and saw here. [14:36] Hock Tan: For instance, if you want to train, you design chips that are much stronger [14:43] Hock Tan: in computing capacity measured TFLOPs, as well as network, [14:49] Hock Tan: because it's not just one chip that makes it happen. [14:52] Hock Tan: It's a cluster, as Charlie put it. [14:54] Hock Tan: But if you want to do inference, you put in more memory [14:58] Hock Tan: and memory access relative to compute. [15:02] Hock Tan: So you are actually, over time, creating chips, [15:06] Hock Tan: optimised for particular workloads, applications, as we go along. [15:13] Hock Tan: And that, at the end of the day, is what will create the most effective models [15:19] Hock Tan: is a platform that you want to create end-to-end. [15:22] Greg Brockman: And also one piece of historical context [15:25] Greg Brockman: is that when we started OpenAI, [15:27] Greg Brockman: we didn't really have that much of a focus on compute. [15:30] Greg Brockman: We felt that the path to AGI is really about ideas. [15:33] Greg Brockman: It's really about tryouts and stuff. [15:35] Greg Brockman: Eventually, we'll put the right conceptual pieces in place [15:38] Greg Brockman: and then AGI. [15:40] Greg Brockman: And about two years in, in 2017, [15:43] Greg Brockman: the thing that we found [15:44] Greg Brockman: was that we were getting the best results out of scale. [15:47] Greg Brockman: It wasn't something we set out to prove. [15:49] Greg Brockman: It was something we really discovered empirically [15:51] Greg Brockman: because of everything else that didn't work nearly as well. [15:53] Greg Brockman: And the first results were scaling up our reinforcement learning [15:57] Greg Brockman: in the context of the video game Dota 2. [15:59] Greg Brockman: Did you guys pay attention to the Dota 2 project back in the day? [16:02] Greg Brockman: Yes. [16:03] Greg Brockman: It was a super cool project. [16:05] Greg Brockman: And we really saw you scale it by 2x, [16:07] Greg Brockman: and suddenly your agent is 2x better. [16:08] Greg Brockman: It's like, okay, we have to push this to the limit. [16:11] Greg Brockman: And at that point, we started paying attention to the whole ecosystem. [16:14] Greg Brockman: There were all sorts of chip startups with novel approaches [16:17] Greg Brockman: that were very different from GPUs. [16:18] Greg Brockman: And we started giving them a ton of feedback saying, [16:20] Greg Brockman: here's where we think things are going. [16:21] Greg Brockman: It needs to be models of this shape. [16:23] Greg Brockman: And honestly, a lot of them just didn't listen to us, right? [16:26] Greg Brockman: And so it's like very frustrating to be in this position [16:28] Greg Brockman: where you say we see the direction the future should be going. [16:31] Greg Brockman: We have no ability to really influence it [16:33] Greg Brockman: besides sort of, you know, [16:34] Greg Brockman: just like sort of trying to influence other people's roadmaps. [16:37] Greg Brockman: And so by being able to take some of this in-house, [16:39] Greg Brockman: we feel like we are able to actually realize that vision. [16:42] Greg Brockman: And again, in a way that like we hope [16:44] Greg Brockman: that we can show a direction [16:46] Greg Brockman: and other people will fill in because the amount of compute required to bring our vision of AGI to the world, [16:53] Greg Brockman: 10 gigawatts is not enough. [16:54] Greg Brockman: That is a drop in the bucket compared to where we need to go. [16:57] Sam Altman: It's a big drop... [16:59] Andrew Mayne: The bucket's really big. [17:03] Andrew Mayne: What becomes possible with this when you're building your own chips for inference and for training? [17:07] Andrew Mayne: Where can you take this? [17:08] Sam Altman: To zoom out a little bit, if you simplify what we do in this whole process [17:14] Sam Altman: to, you know, melt sand, run energy through it [17:17] Sam Altman: and get intelligence out the other end. [17:19] Sam Altman: You're not literally melting sand. [17:20] Sam Altman: Like it's a nice visual. [17:21] Hock Tan: That's a good one. [17:22] Charlie Kawwas: That's all we have to do. [17:23] Hock Tan: I like that. [17:25] Sam Altman: What we want is the most intelligence we can get [17:31] Sam Altman: out of each unit of energy. [17:33] Sam Altman: And because that will become the gate at some point. [17:40] Sam Altman: And I hope what this whole process will show us, which is, you know, from the model we design to the chip to the rack, [17:50] Sam Altman: we will be able to wring out so much more intelligence per watt. [17:54] Sam Altman: And then everybody that's using these models in all of these incredible ways will do so much with it. [18:01] Sam Altman: That's what I hope for. [18:02] Hock Tan: And you control your own destiny. [18:07] Hock Tan: If you do your own chips, you control your destiny. [18:11] Andrew Mayne: Yeah, it's interesting to think about how the things that we're doing today are pretty amazing, remarkable, [18:16] Andrew Mayne: but we're using stuff that wasn't actually designed specifically for the way we're doing it. [18:20] Sam Altman: Oh, I mean, the GPUs of today are incredible, incredible things. [18:27] Sam Altman: I'm very grateful, and we will continue to really need a lot of those. [18:31] Sam Altman: The flexibility and the ability to let us do fast research is amazing. [18:36] Sam Altman: But you are right that as we get more and more confident in what the shape of the future is going to look like, a very optimized system to the workload will let us ring more out per watt. [18:48] Sam Altman: That's great. [18:49] Charlie Kawwas: And it's a long journey that takes decades. [18:52] Charlie Kawwas: So if you go back to Hock's example, take railroads, it took about a century to roll it out as a critical infrastructure. [18:59] Charlie Kawwas: If you take the Internet, it took about 30 years. [19:02] Charlie Kawwas: This is not going to take five years. [19:04] Charlie Kawwas: It's going to take a long time. [19:06] Charlie Kawwas: So I think as we collectively, especially with this partnership, continue to figure out ways to wring out more tokens out of it, we'll discover that, oh, for this training or research, maybe a GPU is great. [19:19] Charlie Kawwas: Or maybe, you know what, we can take whatever we're doing with Greg. [19:23] Charlie Kawwas: It's actually a platform that allows you, like a Lego block, to take in things and out. [19:29] Charlie Kawwas: And now suddenly we can get another XPU or an accelerator for next-gen that's targeted at a training or an inference or a research. [19:38] Greg Brockman: Yeah, and to the point that Sam said of GPUs have really come an incredible way, [19:43] Greg Brockman: in 2017 when we started looking at all these other accelerators, [19:46] Greg Brockman: it was actually very non-obvious about what the landscape would look like in 5, 10 years. [19:52] Greg Brockman: And I think it's really a testament to companies like NVIDIA AMD for how much the GPU has just moved forward and continued to be the dominant accelerator. [20:06] Greg Brockman: But at the same time, there's a massive design space out there, right? [20:09] Greg Brockman: And I think that what we see is workloads that are not served through existing platforms. [20:14] Greg Brockman: And that's where that full vertical integration is something unique. [20:18] Andrew Mayne: It's interesting to you because the idea that you'd want to put inferences close to the user is something kind of relatively new. [20:24] Andrew Mayne: You know, we understood training, but then you think about like the number of people every day using these products and how much they need compute to do fun things or serious things. [20:32] Andrew Mayne: And when you start thinking about kind of like the scale of it, like we talked before, I keep coming back to it's a very big thing. [20:40] Andrew Mayne: Where, you know, where does it keep going? [20:43] Andrew Mayne: Is it just a thing that we're going to continuously find new things to use compute for? [20:47] Sam Altman: The first cluster OpenAI had, the first one that I can remember the energy size for, was 2 megawatts. [20:57] Sam Altman: Adorable. [20:58] Greg Brockman: Yeah. [20:58] Greg Brockman: We got things done with those two. [21:00] Sam Altman: I don't remember when we got to 20. [21:01] Sam Altman: I remember when we got to 200. [21:04] Sam Altman: You know, and we will finish this year a little bit over 2 gigawatts. [21:10] Sam Altman: And these recent partnerships will take us close to 30. [21:17] Sam Altman: the world has done far more than I thought they were going to do. [21:22] Sam Altman: Turns out you can serve 10% of the world's population with ChatGPT [21:27] Sam Altman: and do the research and do Sora and do our API and a few other things on 2 gigawatts. [21:32] Sam Altman: But think about how much more the world would like to do than they get to do right now. [21:37] Sam Altman: If we had 30 gigawatts today with today's quality of models, [21:42] Sam Altman: I think you would still saturate that relatively quickly in terms of what people would do, [21:46] Sam Altman: especially with the lower cost we'll be able to do with this. [21:49] Sam Altman: But the thing we have learned again and again is, [21:53] Sam Altman: let's say we can push GPT-6 to feel like, you know, 30 IQ points past GPT-5, something big. [22:03] Sam Altman: And that it can work on problems not for a few hours, but for a few days, weeks, months, whatever. [22:08] Sam Altman: The amount, and while we do that, we bring the cost per token down. [22:11] Sam Altman: The amount of economic value and sort of surplus demand that happens each time we've been able to do that, [22:17] Sam Altman: goes up a crazy amount. [22:18] Sam Altman: So you can see, to pick a, I think, [22:21] Sam Altman: well-known example at this point, [22:24] Sam Altman: when ChatGPT could write a little bit of code, [22:26] Sam Altman: people actually used it for that. [22:27] Sam Altman: They would very painfully paste in their code [22:29] Sam Altman: and wait and they would say, [22:30] Sam Altman: do this for me and paste it back in and whatever. [22:32] Sam Altman: And models couldn't do much, [22:34] Sam Altman: but they could do a few things. [22:36] Sam Altman: The models got better, the UX got better, [22:38] Sam Altman: and now we have Codex. [22:40] Sam Altman: Codex is growing unbelievably fast [22:42] Sam Altman: and can now do a few hours of work [22:45] Sam Altman: at a higher level of kind of capability. [22:47] Sam Altman: And when that's possible, [22:48] Sam Altman: the demand increase is crazy. [22:50] Sam Altman: Maybe the next version of Codex [22:52] Sam Altman: can do like a few days of work [22:54] Sam Altman: at kind of one of the best engineer you know level, [22:56] Sam Altman: or maybe that takes a few more versions, [22:58] Sam Altman: whatever, it'll get there. [22:59] Sam Altman: Think how much demand there will just be for that [23:01] Sam Altman: and then do it for every knowledge work industry. [23:03] Greg Brockman: And one way I like to think of it [23:05] Greg Brockman: is that intelligence is the fundamental driver [23:09] Greg Brockman: of economic growth, [23:10] Greg Brockman: of increasing the standard of living for everyone. [23:12] Greg Brockman: And what we're doing with AI [23:14] Greg Brockman: is actually bringing more intelligence and amplifying the intelligence of everyone. [23:19] Greg Brockman: And so as these models get better, I think everyone's going to become more productive. [23:23] Greg Brockman: The output of what is possible is going to just be totally different from what exists today. [23:29] Andrew Mayne: It's interesting, too, that going from a point with GPT-3, which was pretty cost, [23:35] Andrew Mayne: you know, it was expensive comparatively to where you're at a level of a GPT-5 [23:38] Andrew Mayne: and the fact that you can provide that freely to people. [23:41] Andrew Mayne: And is that a motivating factor for you, the fact that every time you create these new efficiencies, that it just benefits so many more people? [23:48] Andrew Mayne: Yes. [23:48] Andrew Mayne: Absolutely. [23:50] Hock Tan: Absolutely. [23:51] Hock Tan: And from our side on hardware, compute capacity, where to some extent, the rubber hits the road on this, it's really incumbent on us to keep optimizing, pushing the envelope on leading-edge technology. [24:10] Hock Tan: And there's still room to go. [24:12] Hock Tan: And there's room to go even on where we are [24:16] Hock Tan: as we go from two nanometers going forward, [24:19] Hock Tan: less smaller than two nanometers [24:21] Hock Tan: as we start doing all kinds of different technology. [24:24] Hock Tan: It is really great, exciting times, [24:27] Hock Tan: especially for the hardware and the semiconductor industry. [24:31] Sam Altman: What Broadcon has done here is really quite incredible. [24:37] Sam Altman: It used to be extremely difficult for a company like ours [24:40] Sam Altman: about making a competitive chip. [24:42] Sam Altman: In fact, so hard we just wouldn't have done it. [24:45] Sam Altman: And I think a lot of other companies [24:46] Sam Altman: wouldn't have done it as well. [24:47] Sam Altman: And all of these sort of, [24:48] Sam Altman: this customized chip and system to a workload [24:51] Sam Altman: just wouldn't be a thing in the world. [24:53] Sam Altman: But the fact that they have pushed so hard and so well [24:56] Sam Altman: on making it so that they can, [24:58] Sam Altman: a company can partner with them [24:59] Sam Altman: and they can do a miracle of technology chip quickly [25:04] Sam Altman: and at scale, unfortunately do it [25:06] Sam Altman: for all of our competitors too, [25:07] Sam Altman: but hopefully our chip will be the best. [25:09] Hock Tan: - Yes, of course. [25:11] Sam Altman: It's really quite incredible. [25:13] Greg Brockman: And I think also not just what they can do for us today, [25:16] Greg Brockman: but looking at the upcoming roadmap, [25:17] Greg Brockman: it's just so exciting the kinds of technologies [25:19] Greg Brockman: that they're going to be able to bring to bear [25:21] Greg Brockman: for us to be able to utilize. [25:22] Hock Tan: Well, it's just the excitement of, I mean, [25:26] Hock Tan: enabling joint and collaboratively models, [25:30] Hock Tan: chat GPT-5, 6, 7, on and on. [25:33] Hock Tan: And each of them will require a different chip, [25:37] Hock Tan: a better chip, a more developed chip, advanced chip [25:42] Hock Tan: that we haven't even begun to figure out how to get there. [25:46] Hock Tan: But we will. [25:47] Greg Brockman: And actually, the GPTs are definitely going to be an increasing part of that. [25:51] Greg Brockman: Yes. [25:51] Greg Brockman: It'll be very interesting. [25:53] Charlie Kawwas: We're actually looking forward to that because my software engineers now [25:56] Charlie Kawwas: already use that from a software point of view, [25:58] Charlie Kawwas: and it's delivering efficiencies of dozens of engineers. [26:01] Sam Altman: Really? [26:02] Charlie Kawwas: Yes. [26:02] Sam Altman: Great. [26:02] Charlie Kawwas: On the hardware side, we're not there yet. [26:05] Charlie Kawwas: But, you know, the good news. [26:06] Charlie Kawwas: We'll get down very little. [26:07] Sam Altman: We should talk. [26:08] Charlie Kawwas: Yes, we should absolutely leverage this. [26:10] Charlie Kawwas: But I was going to say with respect to compute. [26:13] Charlie Kawwas: So when we started building these XPUs, [26:16] Charlie Kawwas: you can maximum build a certain number of compute in 800 square millimeter. [26:20] Charlie Kawwas: That's it. [26:21] Charlie Kawwas: Now, today, we're actually working together to ship multiple of these [26:25] Charlie Kawwas: in a two-dimensional space. [26:27] Charlie Kawwas: The next thing we're talking about is stacking these into the same chip. [26:30] Charlie Kawwas: So now we're actually going in the Y dimension or Z dimension, [26:35] Charlie Kawwas: if you want to think three-dimensional. [26:37] Charlie Kawwas: And then the last step we're actually also talking about [26:40] Charlie Kawwas: is now we're going to bring optics into this, [26:42] Charlie Kawwas: which is actually what we just announced, [26:44] Charlie Kawwas: which is 100 terabits of switching [26:47] Charlie Kawwas: with optics integrated all into the same chip. [26:50] Charlie Kawwas: So these are sort of the technologies [26:52] Charlie Kawwas: that will take compute, the size of the cluster, [26:55] Charlie Kawwas: the total performance and wattage of the cluster [26:58] Charlie Kawwas: to a whole new level. [26:59] Charlie Kawwas: I think it will keep doubling at least every six to 12 months. [27:03] Andrew Mayne: What kind of timeframe are we talking about? [27:04] Andrew Mayne: When are we going to first start to see what's coming out of the relationship? [27:07] Sam Altman: End of next year, and then we'll deploy very rapidly over the next three years. [27:12] Sam Altman: Absolutely. [27:13] Charlie Kawwas: Greg and I are talking about this at least once a week. [27:17] Charlie Kawwas: We just had a chat earlier today on this. [27:19] Charlie Kawwas: Yes, good progress today. [27:20] Charlie Kawwas: Yes, exactly. [27:21] Greg Brockman: But yeah, we're really excited to get Silicon back starting soon, actually. [27:25] Greg Brockman: Yes, very soon. [27:26] Greg Brockman: Yeah, I think that my view of this whole project is it's not easy, right? [27:30] Greg Brockman: It's easy to just say, oh, yeah, 10 gigawatts. [27:32] Greg Brockman: But like when you look at what is required to actually design a whole new chip and to actually deliver this at scale, get the whole thing working end to end, it's just an astronomical amount of work. [27:40] Greg Brockman: And I would say that we're very serious. [27:42] Greg Brockman: You know, our mission is to ensure that AGI benefits all of humanity. [27:45] Greg Brockman: We're very serious about benefits everyone. [27:49] Greg Brockman: Like we really want this to be a technology that is accessible to the whole world, that lifts up everyone. [27:53] Greg Brockman: And you can really see that in trying to make the world be one of compute abundance. [27:58] Greg Brockman: Because I think by default, we're heading towards one that is like quite compute scarce. [28:02] Andrew Mayne: You ask my wife when she's trying to get more Sora credits, it feels very scarce. [28:06] Greg Brockman: Yeah, no, no. We feel it so concretely. Teams within OpenAI, their output is just like a direct [28:13] Greg Brockman: function of how much compute they get. And so that the amount of intensity on who gets the compute [28:20] Greg Brockman: allocation is so extreme. And so I think that what we really want is to be a world where just if you [28:27] Greg Brockman: You have an idea, you want to create, you want to go build something that you have the compute [28:31] Greg Brockman: power behind you to make it happen. [28:35] Andrew Mayne: Gentlemen, thank you very much for sharing this with us. [28:36] Andrew Mayne: It's going to be very exciting to see where this goes, and I hope we can keep talking [28:40] Andrew Mayne: about this as it continues to develop. [28:42] Andrew Mayne: Thank you. [28:42] Sam Altman: Thank you guys for the partnership. [28:43] Hock Tan: Thank you. [28:44] Hock Tan: Thank you for the partnership. [28:45] Hock Tan: We're really enjoying it. [28:46] Greg Brockman: We are too. [28:47] Sam Altman: Yeah. [28:48] Hock Tan: Thank you.