Gelsinger Warns AI Buildout Faces Defaults as Power Grid Stalls
Raghu Raghuram, Guido Appenzeller, Pat Gelsinger
Multibillion-dollar computing deployments will trigger widespread project defaults because stagnant electrical grids and eight-year equipment lead times prevent data centers from energizing planned GPU clusters.
Energy availability has become the ultimate ceiling on artificial intelligence buildouts, with Pat Gelsinger predicting a wave of data center project defaults as capital commitments outpace electrical grid capacity and equipment lead times for gas turbines hit eight years. (43:08)
The market of roughly 100 specialized AI inference chip startups is poised for drastic consolidation down to a few architectures due to shifting reasoning workloads, massive packaging and fabrication capital requirements, and hyperscaler software co-design overhead. (15:00)
Packaging and thermal constraints limit high-performance 3D chip stacks to a practical sweet spot of two to four memory dies rather than extreme 16-high stacks, driven by compound defect rates and unmanageable thermal dissipation. (27:18)
Interconnect bottlenecks are forcing the transition from copper to near-packaged and co-packaged optics by roughly 2028–2029, with predictable AI network flows naturally fitting Optical Circuit Switching (OCS) over traditional packet switching. (35:00)
Detailed notes
## Background
Published on October 9, 2026, by hosts Raghu Raghuram and Guido Appenzeller alongside guest Pat Gelsinger via an a16z podcast feed, this episode explores the physical limits shaping the AI infrastructure boom. Drawing on Gelsinger's extensive leadership at Intel and VMware, the discussion examines how AI is transforming chip design while running up against hard bottlenecks in manufacturing, memory architectures, advanced packaging, optical interconnects, and energy availability.
## TL;DR
Former Intel and VMware CEO Pat Gelsinger argues that while generative AI is accelerating front-end silicon design, physical constraints—specifically memory bandwidth, advanced packaging, and grid-scale power capacity—now dictate the pace of AI progress. He predicts that the current explosion of roughly 100 specialized inference chip startups will inevitably consolidate due to hyperscaler co-design demands and rapidly shifting reasoning workloads. Gelsinger highlights the arrival of genuine material innovations in memory after a 30-year lull, alongside an imperative shift from copper to optical interconnects and Optical Circuit Switching by 2028–2029. Finally, drawing on his virtualization heritage, he outlines how enterprise computing must recreate its foundational software hierarchy to build an 'agent-native VMware' tailored for autonomous software rather than human administrators.
## Atmosphere & style
The conversation carries the warm, collegial energy of an insider reunion among longtime Silicon Valley veterans who share deep technical roots and mutual history at VMware. Gelsinger speaks with brisk, pragmatic confidence, punctuating sober engineering assessments with wry humor about recruiter notes, failed memory architectures, and venture capital over-funding. The tone strikes a sharp contrast between Silicon Valley's euphoric software narratives and unyielding physical realities, blending hard-won microprocessor history with thermodynamic limits, mechanical fluidics, and electrical grid mathematics.
## Bulletin
- `43:08` Energy availability has become the ultimate ceiling on artificial intelligence buildouts, with Pat Gelsinger predicting a wave of data center project defaults as capital commitments outpace electrical grid capacity and equipment lead times for gas turbines hit eight years.
- `15:00` The market of roughly 100 specialized AI inference chip startups is poised for drastic consolidation down to a few architectures due to shifting reasoning workloads, massive packaging and fabrication capital requirements, and hyperscaler software co-design overhead.
- `27:18` Packaging and thermal constraints limit high-performance 3D chip stacks to a practical sweet spot of two to four memory dies rather than extreme 16-high stacks, driven by compound defect rates and unmanageable thermal dissipation.
- `35:00` Interconnect bottlenecks are forcing the transition from copper to near-packaged and co-packaged optics by roughly 2028–2029, with predictable AI network flows naturally fitting Optical Circuit Switching (OCS) over traditional packet switching.
## Key takeaways
- `10:00` AI-assisted design tools can compress the digital front-end chip design phase to three months, but the physical back-end pipeline—fabrication, 3D packaging, and rack integration—still demands at least nine to 18 months.
- `15:00` Extreme hyper-specialization in AI inference silicon faces severe obsolescence risks as reasoning models require general-purpose CPU-like flexibility and flat LLMs merge into 64-bit scientific high-performance computing.
- `20:00` Incumbent platform vendors like NVIDIA will absorb heterogeneous computing features inside proprietary software abstractions rather than exposing raw architectural diversity to application developers.
- `25:02` After a 30-year stagnation driven by commoditization and boom-and-bust cycles, the memory sector is seeing renewed fundamental materials research in FeRAM, non-capacitive cells, and 3D-stackable memory layers.
- `27:18` Extreme 3D vertical stacking encounters punishing yield mathematics: multiplying compounding defect rates across many stacked logic layers causes composite die failure regardless of built-in redundancy.
- `32:00` Moving memory completely off-package over optical links is fundamentally penalized by high optical-electrical-optical (OEO) conversion energy, which is roughly 1,000 times worse in femtojoules per bit than on-package communication.
- `32:00` Processing-in-Memory (PIM) continues to struggle with mainstream adoption because force-partitioning software algorithms across in-memory compute elements severely restricts general-purpose workload compatibility.
- `35:00` NVIDIA's NVL72 rack system demonstrated the physical limits of high-speed copper cabling, serving as an engineering marvel but a manufacturing challenge that required an extra 18 months for the data center supply chain to digest.
- `37:30` Unlike traditional internet traffic characterized by arbitrary packets between unknown endpoints, AI collective communications produce large, predictable flows that naturally suit Optical Circuit Switching.
- `40:00` The strict architectural separation between training and inference will increasingly blur as systems shift toward continuous, real-time learning workflows.
- `46:04` Because CMOS scaling has flatlined in power efficiency per teraflop across recent chip generations, break-even efficiency gains will depend on post-CMOS alternatives such as superconducting substrates.
- `46:04` Data center infrastructure is pivoting to 800V direct current (DC) architectures powered by vertical Gallium Nitride (GaN) converters to eliminate wasteful stepping stages inherited from alternating current standards.
- `50:00` Managing autonomous agent swarms requires rebuilding the entire computing and virtualization hierarchy from scratch, shifting the customer from patient human administrators to low-latency programmatic consumers.
- `51:17` A modern 'VMware for Agents' requires both a human governance control plane with constitutional guardrails and an execution plane offering dynamic capabilities like live agent-state migration.
## Detailed walkthrough
### `00:00` Energy Constraints, the Memory Renaissance, and Silicon Valley Roots
The discussion opens with a rapid sequence of observations framing the physical and economic boundaries of the current AI boom. Pat Gelsinger highlights that computing can no longer outpace the electrical grid, observing that modern compute deployments face immediate physical walls: “In an AI digital age, energy capacity equals economic capacity.” He questions the wisdom of pouring capital into massive hardware deployments without secured power, predicting that project defaults will surge across the industry when power commitments fail to materialize. At the same time, the hardware landscape is experiencing a dramatic awakening after decades of incremental progress. Gelsinger notes that while true memory innovation has sat at virtually zero for the past 30 years, a fundamental breakthrough is finally arriving. Similarly, long-anticipated architectural shifts—such as replacing copper interconnects with optical networking—are reaching fruition. Amid an unprecedented explosion of venture capital pouring into dozens of specialized AI inference accelerator startups, the hosts question whether the market can realistically sustain so many competing silicon architectures or whether industry consolidation is inevitable.
Raghu Raghuram introduces Pat Gelsinger, noting his roles as General Partner at Playground Global, former CEO and CTO of Intel, and former CEO of VMware. Joining Raghuram and co-host Guido Appenzeller, Gelsinger jokes that being reunited with his longtime colleagues feels like coming home. Raghuram points to the newly founded Andreessen Horowitz (a16z) Academy, which identifies talented individuals between 16 and 22 to accelerate their careers, and compares it to Gelsinger's non-traditional path into tech. Gelsinger recalls securing a scholarship at age 16, skipping the last year and a half of high school to attend a technical trade school, and falling in love with computers. When Intel arrived on campus to recruit, an 18-year-old Gelsinger became the twelfth interview candidate of the day for recruiter Ron Smith.
- Recruiter Ron Smith scribbled an assessment of the teenage Gelsinger that sealed his hiring: “Smart, aggressive, arrogant. He'll fit right in.”
- Gelsinger joined Intel as an entry-level technician, rapidly advancing to become design engineer number four on the Intel 386 microprocessor.
- By his mid-twenties, he served as the lead architect and design manager for the Intel 486, all while concurrently completing his bachelor’s, master’s, and doctoral studies.
`04:10` Stanford Clashes, Early BIST, and Testing Academic Theory on Real Silicon
Gelsinger’s dual life—studying university theory by day and implementing microcode and layout at Intel by night—led to clashes between academic concepts and real-world silicon engineering. While attending graduate courses at Stanford University, a professor presented a theoretical carry-lookahead adder design. Gelsinger, who was actively building the adder for the Intel 386, immediately tested the professor's equations against his production work and found that the math failed in silicon implementation. Returning to class two days later, he openly challenged the instructor. The dispute consumed two full lecture periods, made particularly urgent and awkward because the professor was in the final stages of publishing a textbook centered on that exact circuit design.
A similar dynamic unfolded with legendary Stanford professor Edward J. McCluskey, widely recognized as a father of digital testing and built-in self-test (BIST). While McCluskey pioneered fundamental fault models and algorithmic testing, Gelsinger pushed back against their practical viability inside commercial microprocessor floorplans, arguing that pure academic BIST logic consumed excessive die area without providing usable yields. Nevertheless, Gelsinger adapted and embedded practical BIST techniques directly onto the 386. His thesis advisor, future Stanford University president and Turing Award laureate John Hennessy, also guided him through these intense debates. Appenzeller notes that this bidirectional friction between academia and commercial industry is the defining engine of Silicon Valley, where students routinely challenge ivory-tower assumptions with operational telemetry and silicon errata.
```mermaid
flowchart LR
accTitle: Academic Theory versus Production Silicon Validation
accDescr: Flowchart showing how Stanford academic designs were validated against production chips at Intel to create viable EDA standards.
subgraph Academic["Stanford University"]
A["Hennessy / McCluskey / Faculty"] -->|"Theories: BIST & Carry-Lookahead"| B["Academic Hypotheses"]
end
subgraph Industry["Intel 386 / 486 Design"]
B -->|"Day: Classwork\nNight: Production Design"| C["Silicon Implementation"]
C -->|"Empirical Errata & Failures"| D["Debate & Silicon Validation"]
end
D -->|"Practical Feedback"| A
```
`06:12` The 486 Era and the Genesis of Modern EDA
The development of the Intel 486 marked a turning point in semiconductor history because it required creating the software design automation tools alongside the microprocessor itself. In the mid-to-late 1980s, the electronic design automation (EDA) industry as it exists today was largely nonexistent. Complex chips were still heavily hand-crafted at the transistor and gate levels. To manage the immense transistor density and integrated floating-point execution of the 486, Gelsinger’s team had to invent internal abstraction layers, including early iterations of HDL (Intel Hardware Description Language), and write custom compilers to translate that descriptive code into hardware logic.
To solve the escalating physical complexity of automated place and route, Intel partnered with academic researchers, working closely with Alberto Sangiovanni-Vincentelli and his students at UC Berkeley. That collaboration yielded the industry's earliest algorithms for automated cell placement, signal routing, and timing management. Looking back on those breakthroughs, Pat Gelsinger reflects: “And that was really one of the hallmarks that not only was the 486 this compatible pipeline microprocessor, but we ushered in many of the foundations of what became the modern EDA industry and was really a pretty magical period of the industry.”
- The Intel 486 was the first mainstream microprocessor to incorporate complete pipelining and integrated cache while pioneering modern automated EDA methodologies.
- Prior to the 486 project, synthesis, standard cell routing, and automated timing checks did not exist as off-the-shelf commercial software.
- The university-industry partnerships forged during the 486 era directly seeded commercial EDA giants like Synopsys and Cadence.
`07:44` The AI Design Revolution and Future Semiconductor Bottlenecks
Drawing a direct parallel between the EDA transformation of the 486 era and today's AI-driven workflows, Appenzeller and Gelsinger explore whether current flagship architectures—such as NVIDIA’s Blackwell—represent the final generation of frontier chips designed primarily by human engineers. Emerging tools, such as the Jalapeno framework, represent a first-principles rethink of chip development, using machine intelligence to generate hardware description code, optimize datapath layouts, and automate verification. However, Gelsinger cautions that automation faces severe physical limits in non-digital domains. Analog circuits, precision high-speed SerDes (serializer/deserializer) interfaces, and physical I/O cannot be easily synthesized by an LLM or neural model without massive libraries of empirical silicon measurements; teams must either over-engineer their analog margins conservatively or hand-tune them the traditional way.
The conversation turns to where the primary industry bottlenecks will migrate once front-end chip specification and Verilog generation become trivial. Appenzeller observes that AI is exceptionally capable of generating low-level software layers, writing optimized compute kernels, and translating high-level workload intent into functional hardware descriptions. When designing an accelerator for next-generation frontier models, knowing the mathematical operations—such as matrix multiply-accumulate structures and register hierarchies—is no longer the gating factor. When automated toolchains allow any competent team of engineers to synthesize an optimized accelerator, the limiting constraint shifts decisively from design logic to raw physical realization: memory bandwidth, advanced packaging capacity, thermal dissipation, and electrical substation power.
`10:00` The Post-Design Bottlenecks: Fabrication, Packaging, Memory, and Power
Pat Gelsinger notes that while AI-driven EDA tools can now compress the digital front-end design cycle down to three months—even while remaining conservative on analog components—the industry runs straight into severe post-design physical bottlenecks. Even with supercharged design flows, getting a design through a semiconductor fab takes at least three months, followed by another three months for complex 3D advanced packaging, and then another three months to integrate everything into a rack-scale system. Because computing is no longer built as standalone chips but as integrated racks, the total time from finished design to usable silicon at scale stretches to nine months, or up to a year and a half before running optimized software.
```mermaid
flowchart LR
accTitle: Chip Design to Deployment Timeline
accDescr: A comparison showing a three-month AI-assisted design phase versus a nine-month physical manufacturing and rack deployment pipeline.
subgraph Design["Design Phase (3 Months)"]
D1["AI-Assisted Digital Design"]
end
subgraph Hardware["Physical Pipeline (9 Months)"]
H1["Silicon Fab<br/>(3 Months)"]
H2["Advanced 3D Packaging<br/>(3 Months)"]
H3["Rack-Scale Integration<br/>(3 Months)"]
end
D1 --> H1
H1 --> H2
H2 --> H3
```
This delay creates an acute strategic vulnerability because AI model architectures evolve much faster than the silicon lifecycle. Gelsinger highlights the example of Graphcore: it was not an inherently bad architecture, but the AI workload landscape shifted completely before the hardware could reach scale. To fix this, Gelsinger argues the industry requires fundamentally new lithography techniques and cheaper prototyping flows that do not demand a $50 million mask set before testing silicon. Beyond fabrication speed, memory bandwidth is the next critical choke point:
> “And, you know, for that, as I've described, HBM is a hideous memory. It's just the best one that we've got. It's a terrible memory. I get terrible bit density. I'm limited to shoreline bandwidth, power issues, thermal issues. DRAM doesn't like to be hot.”
Thermal and power delivery challenges compound these limitations. Current 3D modeling tools for power dissipation, thermal hotspots, and voltage drops remain inadequate. Because designers cannot accurately simulate dynamic power, they are forced to guard-band almost their entire operating envelope, losing roughly 40% of their power efficiency simply to ensure safe margins.
- Fabrication time cannot realistically fall below three months today, even under expedited runs.
- Prototyping costs are inflated by mask set expenses that can reach $50 million per run.
- Guard-banding against voltage drop and thermal uncertainty wastes an estimated 40% of delivered electrical power.
`13:30` The Explosion of AI Accelerators: Bubble or Architectural Shift?
The massive demand for AI infrastructure and lower barriers to front-end design have unleashed an unprecedented wave of silicon startups. Guido Appenzeller points out that there are currently around 100 specialized AI inference accelerator chips on the market, targeting distinct trade-offs on the Pareto efficiency curve. Pat Gelsinger jokingly questions the venture capitalists in the room:
> “Why are you guys funding so many of those anyway? You funded your fair share, too.”
Appenzeller and Raghu Raghuram press whether this explosion marks a permanent shift toward heterogeneous computing or a temporary bubble destined to collapse back into an oligopoly. Appenzeller observes that historically, no computing market has ever sustained 100 competing processor architectures. Raghuram notes that while vast design options are being explored, history suggests high capital requirements and platform complexity inevitably trigger consolidation.
`15:00` Why the 100 AI Chip Startups Will Consolidate
Gelsinger predicts that the current wave of specialized AI chip startups will prove temporary and consolidate down to a small handful of winners. He lays out three distinct structural reasons for this inevitable contraction:
```mermaid
flowchart TD
accTitle: Drivers of AI Chip Consolidation
accDescr: Three main market pressures driving chip consolidation: evolving workloads, market and capital dynamics, and hyperscaler software co-design.
subgraph Workloads["1. Rapid Workload Evolution"]
W1["Hyper-specialization<br/>(pre-fill vs decode vs mid-fill)"]
W2["Reasoning models demand<br/>CPU-like flexibility"]
W3["HPC and 64-bit precision merge<br/>for 3D molecular/scientific AI"]
end
subgraph Economics["2. Scale and Capital Realities"]
E1["Manufacturing scale barriers"]
E2["Venture capital limits across 100 startups"]
end
subgraph Ecosystem["3. Platform Co-Design"]
S1["Hyperscalers pick single architectures"]
S2["Software stack co-evolution<br/>(CUDA, PyTorch kernels)"]
end
W1 & W2 & W3 --> C["Industry Consolidation<br/>to Few Winning Architectures"]
E1 & E2 --> C
S1 & S2 --> C
```
First, workloads are shifting too quickly for brittle, hyper-specialized chips to remain viable across multi-year silicon cycles. Today's hardware tries to subdivide inference into narrow tiers—pre-fill, decode, mid-fill, or speculative verification. Yet the rapid emergence of reasoning models suddenly requires compute that looks far more like a traditional CPU than a specialized systolic array. Furthermore, flat large language models are reaching structural limits; moving into 3D scientific domains like chemistry, biology, and materials modeling requires integrating AI with traditional High-Performance Computing (HPC) workflows where 64-bit floating-point precision matters again:
> “So all of that said is this extreme specialization to me in the compute architect. I sort of don't like it because I don't think it's going to be what the workloads will look like two, three, four years from now.”
Second, manufacturing economics and scale requirements make a fragmented market impossible to sustain. Silicon economics reward massive unit volumes; 100 disparate architectures—including specialized optical and probabilistic processors—cannot all capture sufficient capital, wafer allocation, or developer adoption.
Third, the major AI model builders and hyperscalers (such as OpenAI, NVIDIA, or Anthropic) must co-design their software stacks with the underlying silicon. Because optimizing software libraries, compilers, and model execution graphs for a specific hardware architecture requires immense engineering overhead, these large players will choose one or two partners rather than spread engineering resources across ten different accelerators.
Guido Appenzeller agrees with Gelsinger's core thesis, adding that in the startup landscape, tier-one status is fundamentally determined by whether a chip vendor has secured deep technical traction and deployment commitments with the small handful of hyperscale consumers.
`20:00` Software-Driven Heterogeneity and the Physical Bottlenecks of Scale
The discussion opens on whether the explosion of specialized AI hardware architectures will persist. Historically, computing standardized on just one or two architectures because supporting diverse instruction sets required writing and maintaining complex compilers and toolchains. Raghu Raghuram and Guido Appenzeller suggest that generative AI and agent swarms might alter this dynamic overnight by automating the creation of software layers, making it trivial to program heterogeneous chips. Pat Gelsinger acknowledges that while agentic software shifts the software barrier, it immediately shifts the bottleneck right back onto stubborn physical and financial realities.
Fabricating any chip still takes at least a year and a half, but the steeper barrier is scaling it inside data centers. Building clusters of novel chips requires tens of billions of dollars in capital expenditure, grid-scale power commitments, and massive operational infrastructure. Because of those staggering deployment costs, market forces resist extreme fragmentation. Instead, large incumbents absorb and obscure heterogeneity rather than exposing raw architectural diversity to developers. Gelsinger points to how NVIDIA integrates novel approaches like Groq-style streaming architectures: within a couple of years, specialized capabilities simply become proprietary internal features tucked behind standard software interfaces. As Pat Gelsinger explains: “So you'll see the architectures, you know, sort of hide the heterogeneity, right, inside of it, which to me is just fine, right? You know, I mean, that's what layering, that's what abstraction has always done associated with it.”
```mermaid
flowchart TD
accTitle: Hardware Architecture Constraints Flow
accDescr: Flowchart illustrating how AI agent software reduces software complexity but runs directly into physical manufacturing and capital expenditure bottlenecks.
subgraph PhysicalConstraints ["Physical & Economic Bottlenecks"]
B --> C["18-Month Fab Lead Times"]
B --> D["$50B+ CapEx & Power Scale"]
end
subgraph Resolution ["Market Consolidation"]
C & D --> E["Incumbent Abstraction (e.g. NVIDIA)"]
E --> F["Unified Programming Interfaces"]
end
```
- Software generation reduces the friction of supporting non-standard instruction sets, but hardware deployment remains bound by 18-month tape-out-to-production cycles.
- Hyperscale AI deployment demands commitments up to $50 billion in data center buildouts and electrical grid allocations, heavily favoring consolidated, standardized platforms.
---
`22:35` The 30-Year Memory Drought and the AI-Driven Boom
Turning from compute to memory, the panel examines High Bandwidth Memory (HBM). While Gelsinger calls HBM hideous from an architectural standpoint, he concedes it remains by far the best option available today, comparing it to democracy as the least bad alternative. The hosts note that true breakthrough innovations in memory have been virtually nonexistent in recent memory, recalling Intel Optane (3D XPoint) as one of the last major commercial attempts. Gelsinger jokes about Optane’s fate: “That one died. We killed it.” Over his career, Gelsinger has been directly involved with at least five novel memory architectures that failed to survive commercialization and has tracked nearly one hundred industry attempts across three decades, none of which broke into the mainstream. Computing remains tethered to the same triad it has used for decades: DRAM, SRAM, and NAND Flash.
This total lack of physics breakthroughs stems from the punishing economics that defined memory for forty years. It was historically a brutal commodity business where manufacturers would make money for a single year only to bleed cash through the next four downturns, leaving little margin for sustained fundamental research and development. However, the rise of modern AI workloads has fundamentally upended the memory sector's fortunes. As Pat Gelsinger observes: “And anybody who looks at the AI workloads, and it's always, you know, it's much harder to, you know, it's easy to predict the past, but it's a little bit harder to predict the future, right? And, you know, in that, you look at it and say, AI is a memory workload.” Over the past four years alone, the memory industry's market capitalization has surged by approximately $2.5 trillion, placing all three major memory manufacturers firmly within the world's twenty most valuable corporations.
---
`25:02` Next-Generation Memory Architectures and Materials
With trillions of dollars flowing into AI infrastructure, capital is finally available to fund fundamental memory physics. Gelsinger highlights two primary vectors of excitement. The first is overcoming the severe shoreline bandwidth limitations imposed by placing HBM alongside processors on an interposer. Moving data horizontally across the die edge consumes excessive energy and creates physical routing bottlenecks, necessitating novel 3D architectures where memory and compute are vertically fused together. He points to startups like d-Matrix as well as wafer-scale approaches like Cerebras as early indicators of this shift toward tightly integrated compute-in-memory structures.
Beyond packaging existing silicon in new arrangements, Gelsinger believes we are on the verge of the first true material-level memory breakthroughs in thirty years. While expressing skepticism toward using dense Flash or MRAM for high-performance AI due to underlying cell latency and performance limits, he sees immense promise in alternative material systems:
- **Ferroelectric Memory (FeRAM):** Leveraging ferroelectric materials to achieve high-speed, non-volatile state switching without traditional charge-storage limitations.
- **Non-Capacitive Cells:** Eliminating the bulky charge capacitors required in traditional DRAM cells to dramatically shrink cell area and enable monolithic stacking.
- **High-Density 3D-Stackable Layers:** Memory materials designed to be grown directly on top of logic wafers without requiring high-temperature processing steps that damage underlying CMOS transistors.
To put skin in the game, Gelsinger shares that he recently funded a stealth-stage memory startup aimed squarely at these material and architectural breakthroughs. As Pat Gelsinger states: “I think speed and performance of the underlying cell are somewhat problematic with a flash or an MRAM structure. But I do think memory innovations are coming for the first time in 30 years.”
---
`27:18` The Reality of 3D Stacking: Yields, Reticle Limits, and the Layer Sweet Spot
Guido Appenzeller questions how tall chips will realistically become—asking whether processors will settle at two bonded dies or quickly scale into 16-layer vertical sandwiches intermixing logic, DRAM, and SRAM. Gelsinger is openly skeptical of extreme vertical stacks for logic and compute, warning that the mathematics of semiconductor manufacturing impose harsh limits on multi-layer bonding. While traditional DRAM handles defects through built-in redundancy, spare rows, and bad block replacement, stacking high-value logic dies introduces compounding exponential yield loss:
If each layer carries even a slight defect rate, or if thermal stress causes microcracks during packaging, the entire composite die becomes scrap metal. As Pat Gelsinger puts it: “You know, if I have a cracked die, I don't care how much resilience you've built into it. It's a bad die. So as a result of that, I'm not a crazy 16, 32 stack kind of guy. I think there's going to be nice three, four, five stacks that just end up sort of being the sweet spot.”
Furthermore, advanced compute tiles have hit physical reticle limits, forcing designers to break chips apart into modular chiplets to maintain acceptable yields and dissipate extreme heat. Adding multiple stacked memory layers on top of those compute tiles creates thermal and power delivery nightmares. Instead of an eight- or sixteen-layer monolith, Gelsinger projects that standard commercial packages will settle around a balanced configuration: a logic tile base paired with two to four stacked memory dies, beneath which backside power delivery networks must be integrated directly into the silicon stack to supply hundreds of amperes at sub-one-volt tolerances.
`30:00` Advanced 3D Packaging, Thermal Dissipation, and the "Plumber" Era of Chip Design
Pat Gelsinger describes the physical architecture required to build modern high-performance silicon complexes, emphasizing that vertical integration in the Z-dimension has become mandatory. Modern architectures require intelligent redistribution layers (RDL) to route signals across the package, alongside dedicated power delivery rails. In Gelsinger's view, a canonical advanced packaging assembly is already roughly eight layers thick once all these structural components are combined. Furthermore, silicon-centric structures must be integrated with III-V semiconductor materials to enable native optical connectivity directly at the package boundary. Balancing these layers, optical interconnects, and power rails simultaneously represents an extraordinary feat of physical engineering. Because of these geometric and thermal realities, Gelsinger dismisses extreme vertical memory structures: “So because of that, throwing a 16 high stack, it's like, that's crazy. A two or four high memory stack, I think that's probably where the sweet spot will be.”
```mermaid
flowchart TD
accTitle: Canonical Multi-Layer 3D Package Architecture
accDescr: Diagram illustrating the structural layers of an advanced 3D packaging complex including optics, power delivery, compute, and cooling.
subgraph ComputeMem["Silicon & Memory Stack"]
M1["2-High to 4-High Memory Stack"]
K1["Core Compute Complex & RDL Signal Routing"]
P1["Integrated Power Rails & Voltage Regulation"]
end
subgraph IOOptics["Optical & External I/O"]
O1["III-V Material Structures (Integrated Optical I/O)"]
end
When Guido Appenzeller points out that this architectural stack does not yet account for thermal dissipation or novel thermal materials, Gelsinger agrees that heat removal is becoming the defining operational bottleneck. Designers can no longer tolerate concentrated physical hot spots; they must intelligently spread heat while managing both power delivery and thermal extraction. By optimizing voltage regulation, designers avoid creating severe "thermal chimneys," but extreme cooling demands are pushing the industry toward novel materials like synthetic diamonds and direct-to-package liquid cooling solutions. Appenzeller quips, “We engineers are becoming plumbers,” to which Gelsinger replies that engineering today fundamentally means dealing with mechanical engineering, fluidics, and thermodynamics at chip scale.
- **Stacking constraints:** A realistic multi-layer canonical packaging stack is already about eight layers thick; adding a 16-high memory stack creates prohibitive manufacturing and thermal complexity, making 2-high or 4-high stacks the practical sweet spot.
- **Material integration:** Silicon dies must be complemented with III-V compound semiconductor structures to bring optical input/output directly into the compute packaging complex.
- **Thermal engineering:** Advanced heat spreaders using diamond substrates and direct chip-scale liquid cooling are increasingly required to manage high heat flux and eliminate localized thermal hot spots.
---
`32:00` The Limits of Off-Package Optical Memory and Processing-in-Memory (PIM)
Guido Appenzeller raises an alternative architectural paradigm proposed by some in the industry: moving memory completely off-package and linking compute to memory pools entirely via optical channels, thereby gaining architectural degrees of freedom. Gelsinger strongly pushes back against this approach due to the severe power and efficiency penalties of optical-electrical-optical (OEO) conversions. He notes that while digital-to-analog converters (DACs) and conversion interfaces are improving, repeatedly transforming signals between electrical and optical domains wastes unacceptable amounts of energy. The physics of computation versus communication create an extreme disparity: “The femtojoules per bit of communication is comparatively a thousand times worse, right? And if I'm spending that much to go get to large pools of... So I don't like those approaches. You know, myself, I think I somewhat defy the underlying physics of the solution.”
The conversation then turns to the opposite end of the spectrum: Processing-in-Memory (PIM), where small compute engines are placed directly inside memory dies to reduce data movement. Gelsinger is equally unenthusiastic about PIM architectures. He explains that PIM forces software developers to partition workloads into tiny algorithmic fragments pushed onto specialized in-memory compute elements, which introduces severe constraints on software and workload generality. While acknowledging that advanced design tools and compiler intelligence might eventually mitigate some of these issues, Gelsinger notes that PIM proposals have recurred cyclically for decades without winning mainstream adoption: “They've been around for 25, 30 years. And I think 25, 30 years from now, they'll still be hanging around.” For Gelsinger, providing massive memory bandwidth situated physically close to general-purpose compute engines remains the only durable, generalizable approach across evolving AI workloads.
---
`35:00` Optical I/O, the Limits of Copper, and the NVL72 Precedent
While rejecting optical links inside the core compute-memory complex, Gelsinger enthusiastically advocates for optics everywhere else across system I/O. He jokes about his historical track record on interconnect transitions: “I declared the death of copper about 25 years ago. Eventually, I'll be right.” Appenzeller and Raghu Raghuram agree that the physical limits of copper are finally arriving. At current multi-gigabit signaling rates, copper traces effectively behave like lossy waveguides whose reach is collapsing: running copper across five meters is becoming more expensive and complex than running optical fiber over one hundred meters.
```mermaid
flowchart LR
accTitle: Transition from Copper to Optical Scale-Up
accDescr: Compares the physical limitations of high-speed copper interconnects with the emerging adoption of optical interconnects for rack-scale systems.
subgraph CopperLimits["Copper Interconnect Bottleneck"]
direction TB
Cu1["5-Meter Copper Cabling"] --> Cu2["High Attenuation / Waveguide Limits"]
Cu2 --> Cu3["Extreme Physical Mass & Cost"]
end
subgraph OpticalEvolution["Optical Scale-Up (2028-2029)"]
direction TB
Opt1["Near-Packaged Optics (NPO)"] --> Opt2["Co-Packaged Optics (CPO)"]
Opt2 --> Opt3["Optical Circuit Switching (OCS)"]
end
When Raghuram asks why data centers have not already made the leap to full optical scale-up fabrics, Gelsinger points directly to supply chain immaturity. The industry must manufacture these optical components at a volume of tens of millions of GPUs, yet the optical manufacturing base has historically lacked adequate laser fabrication capacity, robust thermal resilience, and automated packaging lines. Gelsinger points to NVIDIA's NVLink-based NVL72 rack system as concrete proof of this dynamic:
> “In reality, I should have never built an NVL72. You know, it's an engineering marvel and it's a manufacturing nightmare, right? You know, it took the industry, you know, one extra year and a half to digest that beast, right? And it really is amazing that, you know, we're able to get it to scale and so on. But, you know, that really proved that had we had a more mature optical supply chain, we'd be much more advanced today.”
Gelsinger expects that near-packaged optics (NPO) and co-packaged optics (CPO) will become the standard conversion point across the industry around the 2028–2029 timeframe, driven by the impossibility of building higher-radix scale-up clusters on pure copper.
- **Copper reach degradation:** At contemporary data rates, copper cables act as high-frequency waveguides whose useful transmission length has contracted to short distances, making short-run copper more costly than 100-meter optical cabling.
- **The NVL72 lesson:** The extreme mechanical complexity and heavy copper cabling of systems like the NVL72 required an extra 18 months for the broader supply chain and data center ecosystem to operationalize.
- **Adoption timeline:** The widespread transition to near-packaged optics (NPO) and co-packaged optics (CPO) for GPU clusters is projected for 2028 to 2029 once laser fabrication volumes and thermal integration issues are solved.
---
`37:30` Optical Circuit Switching (OCS) and Predictable AI Flow Architectures
Appenzeller asks whether scaling optical networks will require transitioning away from packet switching toward circuit-switched optical networks, given the difficulty of scaling ultra-fast packet-based optical switches indefinitely. Gelsinger agrees and references Stanford professor Nick McKeown, who happened to be Appenzeller's PhD thesis advisor. Gelsinger contrasts the historical assumptions of networking with the realities of modern AI infrastructure: traditional packet-switched networks were engineered to handle arbitrary packets traveling between unpredictable endpoints with zero prior knowledge of traffic patterns.
AI training and inference workloads exhibit precisely the inverted behavior: they produce massive, long-lived, highly deterministic data flows between known nodes. Because traffic flows are stable and coordinated during collective communications, circuit switching represents a far more natural architectural fit. Gelsinger predicts that the industry will converge on optical interfaces brought directly into the compute packaging complex coupled to Optical Circuit Switches (OCS). This transition will blur the classical distinction between scale-up and scale-out networking, allowing large-radix compute clusters to interlink through unified, predictable optical switching topologies.
> “When you think about AI, you know, it's almost exactly the opposite, right? You know, we're doing flows that are large and predictable. So in that sense, just the very description says, hmm, that looks more like a switch than like a packet. So I do think this confluence of moving to optical and having a workload that is far more predictable, manageable, large in flow, to me, I do see that, yeah, we will go all the way to bringing optical into the compute complex. And we will go all the way to OCS or some form of optical switching with more predictable flows.”
`40:00` Scale-Up vs. Scale-Out and the Convergence of Training and Inference
Guido Appenzeller and Pat Gelsinger examine the architectural boundary between scale-up and scale-out systems. Gelsinger views this boundary as largely artificial, noting that while front-end and back-end networks currently employ different protocols, they ultimately aim to accomplish the same architectural goals and will likely converge over time. A major factor challenging increasingly specialized, heterogeneous hardware architectures is the future blurring between training and inference workloads. Rather than treating training as a static, isolated phase, Gelsinger suggests that algorithms will inevitably drift toward continuous learning, much like human cognition:
> “Now I'll go on to, you know, right, does your brain stop learning? Right, you know, by the way, as another characteristic of workload evolution, I do think the separation of training and inferencing is going to become less so going forward, not more so.”
While Raghu Raghuram and Guido Appenzeller counter that inference today relies on smaller datasets and smaller node diameters, Appenzeller notes that diversity across AI workloads could still drive hardware heterogeneity. A lightweight diffusion model, for example, demands memory bandwidth and compute characteristics radically different from an ultra-dense, 15-trillion-weight large language model. Gelsinger concedes that theoretical niche optimizations exist, but emphasizes the operational reality of hyper-scale deployments. If an operator deploying a million GPUs attempts to maintain boutique clusters of 1,000 cards optimized for diffusion alongside distinct configurations for frontier LLMs, they destroy scale efficiencies and create severe manageability, connection, and fault-domain overhead. Appenzeller jokes that “the agents will do all the upgrades,” prompting Gelsinger to laugh that fleet management is apparently solved.
```mermaid
flowchart TD
accTitle: Training and Inference Workload Convergence
accDescr: Diagram illustrating how continuous learning bridges the historical divide between static training clusters and low-diameter inference deployments.
subgraph Hist["Historical Separation"]
Train["Massive Training Clusters<br/>(Compute & Interconnect Intensive)"]
Infer["Inference Deployments<br/>(Smaller Node Diameters)"]
end
subgraph Convergence["Continuous Learning Paradigm"]
ContLearn["Continuous Learning Engine<br/>(Online Weight Updates)"]
Workload["Unified System Architecture<br/>(Standardized Protocols & Fleet Management)"]
end
`43:08` The Energy Bottleneck: National Grid Stagnation and Power Limits on AI
The conversation shifts from compute and networking to the overriding physical constraint: power delivery, generation, and capacity. Gelsinger delivers a stern diagnosis of the United States energy infrastructure, explaining that the country spent 10 to 15 years replacing coal with renewables at roughly a 1:1 replacement rate, which effectively flatlined total national generating capacity. In an AI-driven digital economy, Gelsinger argues that energy capacity is directly tied to gross domestic product:
> “Because in an AI digital age, energy capacity equals economic capacity.”
Recent efforts to expand energy capacity have only managed growth of roughly 4% per year, which remains insufficient. Severe geopolitical and supply-chain vulnerabilities further compound the issue: primary renewable supply chains remain heavily dependent on China, gas turbine lead times have stretched to eight years, and the domestic nuclear industry sat virtually dormant for decades. Gelsinger warns that this energy ceiling will dampen market euphoria and trigger cancellations:
> “Because why build a new data center and buy the million GPUs if I can't power them? And I think you're going to see more and more defaults happening on many of those data center projects because the energy will be there.”
- **National capacity stagnation:** 10–15 years of coal retirement matched renewable additions, yielding net-zero national capacity growth until the recent rise to ~4% annually.
- **Gas turbine delivery lag:** New gas turbine equipment faces lead times stretching out to roughly eight years.
- **Capital mismatch:** Project commitments to mega-scale GPU clusters are colliding directly with missing grid interconnects and unbuilt baseload power.
---
`46:04` 800V DC Data Centers, Thermodynamic Realities, and Post-CMOS Physics
To bypass grid and efficiency bottlenecks, data center power delivery must be re-engineered from the utility interconnect down to the silicon die. Moving facilities to 800-volt direct current (DC) avoids multiple inefficient stepping and conversion stages inherited from legacy alternating current standards. The hosts joke that this transition marks “the revenge of Edison” over Westinghouse. Gelsinger highlights vertical Gallium Nitride (vertical GaN) as a critical technology that can step 800V down directly to 48V, 12V, or even 5V in a single conversion step, alongside solid-state transformers and redesigned high-efficiency cooling loops.
For hardware engineers, this shift has renewed interest in classic thermodynamics: Gelsinger jokes that after 25 years he had to revisit Carnot cycle efficiency limits because thermal dissipation, refrigerants, and turbines have re-emerged as premier technical battlegrounds. Yet cooling and architectural tricks only go so far when underlying chip efficiency hits a wall. Gelsinger stresses that CMOS scaling has stalled, requiring a pivot to radically different physical substrates:
> “Because CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline.”
To break this plateau, Gelsinger points to investments in novel physics, such as superconducting startup Snowcat, which targets a 1,000× improvement in power-performance over standard CMOS silicon.
---
`48:38` Virtualization in the Agentic Age: Revisiting the VMware Abstraction
As time runs short, the hosts turn to their shared heritage at VMware—both Pat Gelsinger and Raghu Raghuram having served as CEO of the virtualization pioneer. Looking at the rise of agentic infrastructure, Appenzeller brings up the resurgence of virtual machines (VMs) and asks whether the virtualization abstractions they championed remain relevant. Gelsinger affirms that throughout computing history, technical breakthroughs have consistently produced new abstraction layers rather than eliminating them:
> “And hey, I think the virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy.”
The fundamental triad of computer science—data structures, algorithms, and abstractions—remains intact. Raghuram and Gelsinger banter about whether the industry's full circle means it is time for everyone to simply go back and run VMware again.
`50:00` Recreating the Computing Hierarchy for Agent Swarms
Reflecting on their shared history at VMware, Pat Gelsinger and the hosts observe that the advent of AI necessitates a complete overhaul of the classic enterprise computing stack. At VMware, the challenge centered on managing and abstracting every layer of the compute hierarchy—workloads, networking, and storage systems. Looking ahead to modern AI deployments dominated by agent swarms, Gelsinger notes that an entire foundational layer of tooling must be invented from scratch to handle agent governance, operational security, and runtime performance. As Gelsinger puts it: “Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy.” This vacuum represents a massive greenfield opportunity for novel enterprise software companies to emerge and define the operational control plane for autonomous agents.
Guido Appenzeller steers the conversation deeper into virtualization, arguing that infrastructure requirements undergo a radical shift when virtual machines are built for autonomous agents rather than human operators. For human end users, infrastructure can tolerate slow spin-up times, administrative complexity, and conventional enterprise procurement. In contrast, when an autonomous software agent is the consumer selecting and running on a VM, everything from product packaging to boot speed must transform. Agents require ultra-fast, near-instantaneous startup times, simpler operational interfaces, and frictionless programmatic selection, underscoring that human administrators possess far more patience than algorithmic agents.
- **Foundational virtualization primitives:** Managing agent swarms will require modern equivalents of workload scheduling, network isolation, performance monitoring, and security boundaries.
- **Architectural divergence for agent-native VMs:** Unlike human-facing virtual infrastructure, agent-facing VMs demand instant instantiation, lower interface friction, and automated programmatic discovery.
---
`51:17` Designing "VMware for Agents": From vMotion to Constitutional Guardrails
When asked how one would rebuild VMware from first principles today, Gelsinger explains that the entire system architecture must be anchored around servicing agents rather than abstracting underlying hardware for traditional human users and operating systems. Referencing Dario Amodei's concept of constitutional AI and guardrails, Gelsinger argues that the core design mandate divides into two interrelated planes: an agent execution plane that provides speed, security, and abstraction, and a human control plane that defines high-level constraints. Just as VMware pioneered vMotion to seamlessly migrate running virtual machines across physical servers without downtime, agent infrastructure requires an equivalent capability to migrate, orchestrate, and isolate running agent states on the fly. As Gelsinger summarizes the shift: “It's abstracting the hardware and the operations for agents. It's a fundamental thing that you're serving.”
```mermaid
flowchart TD
accTitle: Architecture of Agent-Native Virtualization
accDescr: Diagram illustrating the two-sided design paradigm for agent-native virtualization, connecting human governance to runtime agent operations.
Gelsinger stresses that developers cannot ignore the human side of the equation; administrators still require governance frameworks, policy definition tools, and dashboards to supervise what swarms are executing. However, the true technical heavy lifting mirrors the engineering hurdles VMware solved decades ago: creating high-performance abstractions so that agents can interact with distributed compute, memory, and operational tools reliably. Closing out the conversation, Gelsinger exchanges warm remarks with his former colleagues, expressing his enthusiasm to return for future discussions on the evolving semiconductor and infrastructure landscape.
- **Dual-plane management paradigm:** The control plane provides human operators with policy engines, dashboards, and constitutional guardrails, while the data plane focuses strictly on agent performance, abstraction, and security.
- **vMotion for agents:** Dynamic workload migration must evolve from moving raw memory pages of a VM to migrating live execution contexts, memories, and tool access states of autonomous agent swarms.
- **Direct quote from Pat Gelsinger on core design constraints:** “Every aspect of the virtual machine, as we know, we'll love it today. It is servicing agents right as opposed to servicing hardware and humans So I think that becomes your fundamental design constraint on this side do I make agents great Right How can I make them secure How can I make them performant You know you know you know how do I abstract them How can I migrate them All of those things you know have the new embodiments you know V motion for agents It's like every one of these will have the new embodiment here for them.”
## Notable quotes
> `00:00` In an AI digital age, energy capacity equals economic capacity. — Pat Gelsinger
> `10:00` HBM is a hideous memory. It's just the best one that we've got. — Pat Gelsinger
> `46:04` CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline. — Pat Gelsinger
> `50:00` Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy. — Pat Gelsinger
> `40:00` I do think the separation of training and inferencing is going to become less so going forward, not more so. — Pat Gelsinger
> `48:38` The virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy. — Pat Gelsinger
## Mentioned
Pat Gelsinger, Raghu Raghuram, Guido Appenzeller, Ron Smith, Edward J. McCluskey, John Hennessy, Alberto Sangiovanni-Vincentelli, Nick McKeown, Dario Amodei, Intel, Intel 386, Intel 486, Intel Optane, VMware, vMotion, Playground Global, a16z, Synopsys, Cadence, NVIDIA, NVIDIA Blackwell, NVL72, Graphcore, Groq, OpenAI, Anthropic, Cerebras, d-Matrix, Snowcat, Stanford University, UC Berkeley
## Video description
a16z's Raghu Raghuram and Guido Appenzeller sit down with Playground Global General Partner and former Intel CEO Pat Gelsinger to discuss the next wave of semiconductor innovation and the physical constraints shaping the AI buildout.
Drawing on his experience designing Intel's 386 and 486 processors, Pat explains how AI could transform chip design, but also why faster design alone won't solve the industry's biggest problems. They examine the bottlenecks in manufacturing, memory bandwidth, advanced packaging, and power, and why today's explosion of specialized AI chips may eventually consolidate around a smaller number of architectures.
They also discuss the potential for new memory technologies, the shift from copper to optical networking, and why energy capacity could become a major constraint on AI growth. Finally, they revisit Pat's VMware years to ask what virtualization might look like when infrastructure is built for agents rather than humans.
Pat Gelsinger argues that the primary constraints on artificial intelligence have shifted from front-end silicon logic to physical realization, memory bandwidth, packaging limits, and electric grid capacity. Consequently, the proliferation of specialized accelerator startups will inevitably consolidate into standardized platforms as deployment realities dominate software flexibility.
Post-Design Bottlenecks and Physical Manufacturing Limits (10:00)
While modern AI-driven design automation can compress digital front-end chip design to roughly three months, physical realization remains tethered to rigid manufacturing timelines. A completed design requires at least three months in a semiconductor fab, three months for complex 3D packaging, and another three months for rack-scale system integration, yielding a nine-month delivery cycle or up to eighteen months before optimized software executes. Gelsinger notes that this delay creates severe strategic exposure because AI model architectures evolve far faster than the physical silicon lifecycle, referencing how Graphcore struggled when AI workloads shifted before its hardware reached scale.
Physical realization faces compounding economic and thermal barriers. Semiconductor prototyping is burdened by mask sets that can reach $50 million per run. Furthermore, because existing modeling tools fail to accurately simulate dynamic 3D power dissipation, thermal hotspots, and voltage drops, chip designers routinely guard-band roughly 40% of their operational envelopes, wasting significant power simply to maintain safe functional margins.
Market Consolidation Facing AI Hardware Startups (15:00)
Despite the presence of around 100 specialized inference accelerator startups exploring points on the Pareto efficiency curve, Gelsinger predicts the field will contract down to a handful of winners. bristle, hyper-specialized architectures cannot easily keep pace with rapid algorithmic shifts. Recent reasoning models increasingly require CPU-like versatility rather than rigid systolic arrays, and frontier scientific applications require 64-bit floating-point precision alongside traditional high-performance computing.
Silicon manufacturing economics also fundamentally demand massive volume that cannot sustain dozens of competing architectures. Furthermore, hyperscalers and frontier model developers face high engineering overhead when co-designing software compilers, runtime libraries, and execution graphs. These players will focus on one or two hardware partners, prompting dominant incumbents like NVIDIA to absorb architectural innovations—such as streaming cores—directly into unified software stacks.
Packaging Realities, Memory Physics, and Optical Interconnects (22:35)
Although Gelsinger characterizes High Bandwidth Memory (HBM) as architecturally flawed due to poor bit density, thermal vulnerabilities, and shoreline bandwidth bottlenecks, he concedes it remains the best existing commercial option. He remains skeptical of processing-in-memory (PIM) and extreme 16-die vertical stacks, noting that stacking logic and memory induces compounding exponential yield loss and thermal concentration. Instead, he projects standard packages will settle on two-to-four stacked memory dies paired with backside power delivery networks. Long-term relief will rely on non-capacitive cells, ferroelectric memory (FeRAM), or post-CMOS materials, as standard CMOS power per teraflop has largely flatlined over five chip generations.
At the interconnect level, copper signaling faces physical limits where short 5-meter cables exhibit severe waveguide degradation and cost more than 100-meter fiber lines. While NVIDIA's NVL72 rack demonstrated the extreme mechanical overhead of scaling massive copper assemblies, immature laser manufacturing and thermal challenges delayed optical scaling. Gelsinger expects near-packaged optics (NPO) and co-packaged optics (CPO) to reach mainstream adoption around 2028–2029, paired with optical circuit switching (OCS) optimized for predictable, high-volume AI data flows.
Grid Stagnation and Re-Engineering Power Delivery (43:08)
Computing deployment is ultimately bounded by the electric grid, prompting Gelsinger to argue that energy capacity directly dictates economic capacity in an AI era. A decade of retiring coal generation in favor of renewables resulted in roughly net-zero total generation growth across the United States before recent increases to roughly 4% annually. Compounded by eight-year lead times for gas turbines and nuclear delays, unpowered data center projects face imminent default risks.
Within data centers, physical efficiency is driving a shift toward 800-volt direct current (DC) architectures, eliminating redundant AC-DC conversion stages. Technologies such as vertical Gallium Nitride (GaN) enable direct step-downs from 800V to operating voltages like 48V or 12V in a single conversion stage. As thermal extraction at the package level increasingly relies on liquid cooling and synthetic diamond spreaders, basic thermodynamics and Carnot efficiency limits have re-emerged as decisive factors in data center engineering.
Rebuilding Virtualization for Autonomous Agent Swarms (50:00)
Reflecting on VMware's architectural model, Gelsinger and the hosts argue that the enterprise software stack must be redesigned around autonomous software agents rather than human operators. Unlike human users who tolerate provisioning delays and interface complexity, programmatic agent swarms require near-instantaneous virtual machine startup and minimal operational friction.
This emerging control plane mirrors classic virtualization challenges across two distinct layers: an operational plane providing human governance, policy enforcement, and constitutional guardrails, alongside an execution plane delivering sandboxing, security, and scheduling. Gelsinger notes that managing state across agent swarms will require modern equivalents of core virtualization primitives, such as creating a runtime 'vMotion for agents' to seamlessly migrate execution contexts, memories, and tool access states across infrastructure without interruption.
While automated software and generative design tools lower the barrier to creating chip logic, hardware progress remains constrained by physical thermodynamics, long manufacturing lead times, and national electrical power generation. Whether alternative material physics and agent-native virtualization can outpace these real-world grid and fab limitations remains an unresolved test for the broader computing industry.
## Background
本期播客由 a16z 的 Raghu Raghuram 与 Guido Appenzeller 主持发布于 2026 年 10 月 9 日，邀请到了 Playground Global 通用合伙人兼英特尔前首席执行官 Pat Gelsinger。节目深入探讨了下一代半导体创新的浪潮，以及制约当前 AI 基础设施规模化扩张的制造、存储与电力等底层物理约束。
## TL;DR
Pat Gelsinger 结合自身设计 386 与 486 芯片及掌舵 VMware 的传奇职业生涯，剖析了 AI 时代底层硬件的范式转移。他指出 AI 虽能极大地加速芯片设计，却无法消除长达九个月的制造与先进封装周期，且以 HBM 为代表的存储架构已逼近物理瓶颈，亟需材料与三维堆叠层面的代际突破。在网络与互连方面，铜线已达物理极限，行业将于 2028 至 2029 年全面向共封装光学（CPO）与光线路交换（OCS）演进。最终，他警告能源容量将直接锁死 AI 的经济上限，未来数据中心不仅面临严峻的电力短缺，更需要在 800V 直流、垂直氮化镓乃至超导等物理底层重塑能效体系，同时虚拟化抽象也将为适应 Agent 蜂群而经历底层重构。
## Atmosphere & style
整场对话洋溢着硅谷工业界老将重聚的默契、幽默与深刻的历史纵深感。Pat Gelsinger 表现得极具工程师锋芒与洞察力，既能绘声绘色地回忆早年顶撞斯坦福教授的硬核往事，又能从第一性原理对当下的 AI 泡沫与物理瓶颈给出辛辣研判。三人围绕芯片、热力学与 VMware 时代的虚拟化往事谈笑风生，在欢快的专业互捧中将冷硬的底层物理学与高阶算力架构娓娓道来，呈现出一种兼具思想厚重感与极客热情的独特魅力。
## Bulletin
- `00:00` Pat Gelsinger 警告称，在 AI 数字化时代“能源容量等同于经济容量”，美国过去 10 至 15 年电力总容量几乎没有净增长，燃气轮机交付周期已长达八年，因缺电导致的数据中心违约潮即将来临。
- `10:00` 尽管 AI 工具能将芯片设计周期缩短至三个月，但受限于晶圆代工、3D 高级封装和机架级集成，芯片从设计到部署的物理交付周期仍长达九个月，且业界目前约有 40% 的电能白白浪费在保守的安全冗余（guard banding）上。
- `13:30` 市场上上百家专用 AI 推理芯片必将走向收敛，因为软件工作负载迁移过快、科学计算需要向 64 位双精度回归，且资本开支与头部生态只会重度下注少数主流架构。
- `35:00` 5 米高频铜缆的实现成本已反超 100 米光纤链路，NVL72 机架的高密度铜线虽是工程奇迹但更是制造噩梦，半导体生态预计将在 2028 至 2029 年全面向近封装光学（NPO）与共封装光学（CPO）切换。
- `47:18` NVIDIA 芯片过去五代产品的 CMOS 本质能效比几乎处于完全停滞（flatline）状态，未来必须依赖 800V 直流架构、垂直氮化镓以及千倍级能效提升的超导计算来打破物理瓶颈。
## Key takeaways
- `00:00` 能源是 AI 基础设施最根本的硬物理约束，能源容量停滞实质上意味着国家潜在经济增长容量被冻结，盲目采购 GPU 最终会遭遇无电可供的窘境。
- `05:05` 英特尔 486 处理器是历史上第一颗广泛引入现代 EDA 理念的芯片，其自研硬件描述语言与自动化布局布线奠定了现代芯片工具链的工业基石。
- `06:58` AI 重构芯片设计呈现出结构性分化：数字逻辑与高阶抽象能够高效自动化，但以 SerDes 为代表的高速模拟模块极度依赖硅片实测数据，依然无法被 AI 完全取代。
- `10:00` 当前大热的 HBM 本质上是一种存在位密度差、周长带宽受限及发热严重的“糟糕内存”，但它是目前解决 AI 内存计算瓶颈所能拿出的最好折中方案。
- `13:30` 硬件过度专业化存在巨大陷阱，随着大模型向 Reasoning 模型演进以及向 3D 科学计算拓展，算力需求正重新回归通用 CPU 特性与 64 位高精度浮点计算。
- `20:00` 通过 AI 生成软件栈虽然降低了专用芯片适配门槛，但数十亿美元级别的机架与电力 CAPEX 等物理现实壁垒，最终会迫使巨头将异构模块收拢进统一架构中。
- `22:15` 存储行业三十年来主流形态几乎零创新，背后原因是传统存储周期五年亏四年的恶劣商业回报，而 AI 带来的万亿美元级资本繁荣正驱动新型存储材料（如铁电存储器）的历史性突破。
- `25:00` 芯片 3D 垂直堆叠存在指数级良率惩罚与物理破碎风险，盲目堆叠 16 层在工程上不可行，计算核心配合 2 至 4 层垂直内存堆叠是制造与散热的黄金平衡点。
- `30:00` 存算分离通过全光链路组建大内存池在物理上不可行，因为计算单比特能耗已降至飞焦级，而跨封装通信传输一个比特消耗的能量高出上千倍，不能盲目浪费功耗在链路互连上。
- `38:00` AI 智算网络的流量特征是可预测的大带宽洪流，天然不同于传统互联网不可预测的小数据包，光线路交换（OCS）将取代传统电分组交换成为千万卡集群的骨干网络。
- `40:00` 随着模型持续学习（continuous learning）在推理端实时更新权重，训练与推理之间的人为界限将逐步消失，Scale-up 与 Scale-out 的网络架构与协议亦将走向融合。
- `47:18` 传统 CMOS 能效比已遭遇瓶颈，数据中心供电正向 800V 直流（800V DC）演进，借助垂直氮化镓减少变压层级，前沿甚至需要押注超导计算以突破千倍能效比提升。
- `50:00` 在 AI 智能体时代，虚拟化技术的核心服务对象从人类变成了 Agent，基础设施必须围绕零容忍冷启动延迟、Agent 动态迁移（vMotion for Agents）与蜂群安全边界从零重构。
## Detailed walkthrough
### `00:00` 芯片创新的新周期与能源物理瓶颈
对话开篇直指当前 AI 浪潮最根本的物质约束：电力。Pat Gelsinger 强调，在 AI 数字化时代，能源容量直接等同于经济容量；如果缺乏足够的电力支持，兴建数据中心并采购上百万张 GPU 毫无意义。他警告称，未来人们将会看到越来越多数据中心项目违约，其核心诱因就是“电力跟不上”。与电力瓶颈并存的，是底层硬件三十年来未曾有过的技术复兴：内存架构即将迎来三十年来首次重大创新突破；铜介质互连的物理局限正在逼近，光网络互连的落地势在必行；市场上还涌现出海量面向 AI 推理加速的专用芯片架构。
随后，主持人 Raghu Raghuram 与 Guido Appenzeller 引导话题，将 Pat Gelsinger 从他在 VMware 和英特尔担任掌门人的辉煌生涯，拉回到他最初踏入半导体产业的起点。针对 a16z 成立 Horowitz Andreessen Academy 旨在选拔 16 至 22 岁年轻人的举措，Raghu 提到 Pat 同样是在 18、19 岁未走传统学院路线就早早加入了英特尔。
* “在 AI 数字化时代，能源容量等于经济容量。如果我连供电都做不到，为什么还要建新的数据中心并买上一百万张 GPU 呢？你将会看到越来越多的数据中心项目发生违约，因为那里的能源根本无法到位。”（Pat Gelsinger 谈数据中心建设面临的电力红线：“In an AI digital age, energy capacity equals economic capacity. Why build a new data center and buy a million GPUs if I can't power that? You're going to see more and more defaults happening on many of those data center projects because the energy won't be there.”）
* “过去 30 年里的内存创新有多少？零。30 年来内存创新的突破第一次真正临近了。”（Pat Gelsinger 预测底层存储架构的代际转变：“Memory innovation for the first time in 30 years is nigh upon us.”）
### `01:50` 18岁进入英特尔与“白天求学、晚上造芯”的传奇开局
Pat Gelsinger 动情地回顾了自己 16 岁跳过最后一年半高中、进入技术学校并阴差阳错获得奖学金的经历。18 岁那年，他在技校深深迷上计算机，恰逢英特尔前来招聘。面试官 Ron Smith 面试了他整整一天中的第 12 个人——在连续面试 12 个人后，考官甚至连男女都很难分清了。但面试官在评估表上写下了简短而辛辣的评语：“聪明、有攻击性、傲慢自大，他完全适合这里。”
于是，年仅 18 岁的 Pat 被招募进英特尔。他从底层的技术员（Technician）起步，一路杀入设计核心团队，成为英特尔 386 处理器的 4 号工程师（engineer number four），紧接着担任 486 处理器的首席架构师兼设计经理。在这一阶段，他边全职工作边攻读学士、硕士乃至博士课程。
这种一边吸收前沿理论、一边在工业一线打破常规的生活，让他在斯坦福的课堂上频频与教授发生尖锐碰撞。一次，斯坦福某教授在课堂上推导了一种新型的超前进位加法器（carry-lookahead adder）设计；由于 Pat 正在 386 项目上实操设计加法器，他下课就回实验室动手验证，两天后便直接回教室当面反驳教授：“这根本行不通。”接下来的整整两节课，变成了 Pat 与教授在讲台上争论其设计为何失败，而此时教授包含了该设计的教科书已经送厂排印。在内建自测（Built-In Self-Test, BIST）领域，师从“自测之父”Edward McCluskey 的 Pat 同样因学术构想不切实际而与其激烈争吵，但他最终把经过工程改良的 BIST 成功集成到了 386 芯片中。
* “面试官在纸上写道——我是他那天面试的第 12 个人，如果你连续面试 12 个人，到最后你连男女都分不清了——他写道：‘聪明、有攻击性、傲慢自大。他完全适合这里。’”（Pat Gelsinger 回忆自己被英特尔录取的奇特评语：“He writes on his page... smart, aggressive, arrogant. He'll fit right in.”）
* “接下来整整两堂课，就只有我和教授在争论为什么他的设计行不通。当时他一本收录了该设计的书正要出版，所以这事情搞得非常尴尬和紧迫。”（Pat Gelsinger 讲述推翻斯坦福教授论文与教材算法的往事：“The next two class periods were just me and the professor arguing over why his design didn't work. He had a book that was being published with that design in.”）
### `05:05` 486 与现代 EDA 萌芽：从手工布局到自动化工具链
谈及 486 处理器的开发，Pat 强调那段经历之所以神奇，是因为当时半导体行业甚至还不存在现代意义上的 EDA（电子设计自动化）产业。早期的芯片设计严重依赖手工与经验，而 486 则是历史上第一颗广泛引入现代 EDA 理念和技术的大规模商用微处理器。
由于当时缺少现成的描述语言，英特尔团队必须自研 HDL（英特尔硬件描述语言）并从零编写对应的编译器；因为当时不存在成熟的自动布局布线（Automatic Place and Route）工具，Pat 团队与加州大学伯克利分校的 Alberto Sangiovanni-Vincentelli 教授及其门下学生深度合作，联合攻关并实现了行业首批自动化布局、自动布线以及自动化时序管理（automated timing management）算法。486 不仅是一款划时代的微处理器架构，更直接催生并奠定了现代 EDA 工业的底层基石。
```mermaid
flowchart TD
accTitle: 486微处理器研发与现代EDA萌芽
accDescr: 展示英特尔486芯片在无现成设计工具环境下如何自研语言与算法推动EDA行业诞生
subgraph Lang["硬件描述与编译"]
A["自研硬件描述语言<br>(Intel HDL)"] --> B["自研专用编译器<br>(HDL Compiler)"]
end
subgraph Tool["物理设计与时序验证"]
C["伯克利联合研发<br>(Alberto Sangiovanni-Vincentelli)"]
C --> D["自动布局布线<br>(Place & Route)"]
C --> E["自动化时序管理<br>(Timing Management)"]
end
subgraph Prod["硬件落地"]
B --> F["Intel 486 微处理器<br>(开发现代EDA先河)"]
D --> F
E --> F
end
```
* “那真是一个不可思议的时期，因为它发生在现代 EDA 产业诞生之前。486 真正算得上是第一颗全面采用你所认为的现代 EDA 技术的芯片。”（Pat Gelsinger 解释 486 在半导体工具链发展史上的里程碑意义：“It was a pretty magic period as well because it was before what you would think of as the EDA industry... the 486 was really the first chip to implement what you would think of as modern EDA techniques.”）
### `06:58` AI 重塑芯片设计：AI-able 的领域与模拟电路的硬骨头
Guido 与 Raghu 顺势提问：当前半导体行业是否正处在类似当年 486 诞生时的新旧交替节点？后人回头看英伟达 Blackwell 架构时，是否会认为它是人类在全面转向 AI 自主设计芯片之前、由传统人工方式研发的“最后一代旗舰芯片”？
Pat 认为行业确实正在进入一个全新的转折期。像 Jalapeno 这样的团队正在从“第一性原理”出发，重新构建利用 AI 彻底重写芯片设计的全流程。然而，设计全盘 AI 化并非毫无阻碍，数字逻辑与模拟电路之间存在着巨大的鸿沟：
1. **数字与高阶抽象领域（AI-able）**：在高级系统目标向 Verilog 硬件语言的自动编译、软件层构建、算子核（Kernel）自动编写以及乘累加寄存器结构的逻辑编排上，AI 表现出极高的效率；少数资深工程师通过引导 AI 工具，就能完成以往大型团队的架构设计。
2. **高速模拟领域（Non-AI-able）**：以 SerDes（串行器/解串器）为代表的模拟信号模块极其依赖深层物理特性与庞大的实际硅片测试数据（silicon data）。针对这些模拟部分，工程师要么采取极其保守的设计余量，以便让 AI 能够勉强应对；要么就必须老老实实地依靠极少数顶尖模拟专家的经验与硬功夫去手动雕刻。
这印证了半导体物理的铁律：当 AI 技术让上层的逻辑综合与工具链设计变得轻而易举时，芯片研发的系统瓶颈并不会消失，而是会直接转移到更底层、更难以被算法吞噬的物理世界——制造工艺壁垒、内存带宽墙以及最核心的能量供给极限上。
* “模拟设计的很多方面——其中 SerDes 大概是最好的例子——依然是无法被 AI 化的。你必须拿到大量的真实硅片数据才能搞定这些。”（Pat Gelsinger 剖析 AI 无法轻易替代的高难度物理模块：“Now, there are pieces of design that are still really hard. In that sense, a lot of the analog aspects, SerDes is probably the best example of that, are still not AI-able. You just need lots of silicon data to get those.”）
* “设计工具正在变得足够聪明，只需要少数专家引导工具该如何应用。我真的认为这在某种程度上会像当年的 486 一样，我们日后回望时会说：是的，那就是芯片设计新时代的开端。”（Pat Gelsinger 展望 AI 时代芯片设计范式的巨变：“Design tools are getting smart enough. A few experts guiding the tools and how to apply it. And I really think it will be somewhat like the 486 that way... yep, that was the beginning of a new era of chip design.”）
### `10:00` 芯片制造周期的九个月瓶颈、内存之痛与功耗仿真挑战
随着 AI 工具被全面引入芯片设计流程，团队完全有能力在短短三个月内交出一份优秀的数字芯片设计，哪怕在模拟电路组件上稍微保守一点，设计效率也已经发生了质的飞跃。然而，Pat Gelsinger 明确指出，当下芯片产业依然面临几大极为致命的实体瓶颈。首先就是长达九个月的物理加工周期：“我可以在三个月内设计出这颗芯片，但要等到九个月后才能真正拿到规模化的实体硅片，这实在太糟糕了。”
```mermaid
flowchart LR
accTitle: 芯片从设计到部署的九个月交付周期瓶颈
accDescr: 展示芯片在三周完成设计后，需经历晶圆代工、3D先进封装以及机架级集成的漫长流程。
subgraph S1["快速设计阶段"]
D["AI辅助芯片设计<br/>（约 3 个月）"]
end
subgraph S2["物理落地瓶颈（约 9 个月）"]
F["晶圆代工制造<br/>（最快约 3 个月）"]
P["3D 先进封装<br/>（约 3 个月）"]
R["机架级集成与部署<br/>（约 3 个月）"]
end
D --> F
F --> P
P --> R
```
当被问及这九个月是否包含验证与测试（validation and verification）阶段时，Pat 拆解了整个物理落地的真实流程：
- 晶圆代工（Fab）：即便是配备了极高加速的设计工作流，物理上晶圆下线最快也需要大约三个月。
- 3D 高级封装（Advanced Packaging）：如今的高端芯片离不开先进封装，其复杂程度与日俱增，这又要耗费大约三个月。
- 机架级方案（Rack Scale Solution）：在如今的算力架构中，“没有任何东西再只是单纯的一颗芯片，它是一个整机架（Rack）”，机架的物理集成与构建同样需要差不多三个月。
这一连串流程导致从设计完成到真正可用耗时长达九个月。Pat 强调，半导体行业必须压缩这一周期，探索全新的光刻技术（lithography），并开发出原型制作阶段不需要预付 5000 万美元掩模（mask）成本的设计流程。如果设计只需三个月，而流片与量产要等上一年半才能部署软件，那么彼时的 AI 工作负载可能早已彻底迭代，原有的设计假设将完全失效——历史上的 Graphcore 并非设计拙劣，只是行业演进的脚步把它甩在了身后。
除了周期过长，另一个致命瓶颈是内存与内存带宽。Pat 直言不讳地将当前大红大紫的高带宽内存（HBM）批为一种“糟糕的内存”：“HBM 是一种可怕的内存。它只是我们目前能拿出来的最好选择。”它存在位密度极差、受制于芯片边缘周长带宽（shoreline bandwidth）、功耗高以及热管理极其棘手等问题。DRAM 本身极不耐热，而 HBM 却偏偏把一切高功耗组件全堆叠挤压在同一个物理热点上。偏偏 AI 本质上就是“内存计算（memory compute）”型工作负载，这一软硬件错配成为不可忽视的死穴。
第三大瓶颈则是供电与散热。如今业界的功率仿真分析（power simulation）工具相当原始，缺乏优秀的 3D 功耗建模工具来预测散热与热点；为了确保电压稳定，工程师不得不留出极大的设计余量（guard banding），这意味着“实际上有大约 40% 的电能白白浪费在了安全冗余（guard banding）上”。供电与散热架构必须迎来彻底的革命。
### `13:30` 上百家 AI 芯片争鸣：过度专业化的陷阱与传统算力的回潮
面对 AI 计算工作负载的爆发，硬件领域迎来了一场架构大繁荣，市场上涌现出上百家专注于推理加速的芯片初创企业，涵盖光子计算（optical）、概率计算（probabilistic）等五花八门的架构路线。主持人笑称：“既然如此，你们这些风投为什么还要给这么多芯片公司投钱？”Pat 则打趣回应称对方也投了不少。Guido Appenzeller 指出，从计算历史来看，任何一个细分硬件行业都不可能长期容纳上百家并存的处理器供应商，这究竟是一场终将向少数赢家收敛的短暂狂热，还是未来算力真的会走向极度异构？
Pat 坚信这只是一种暂时的过渡现象，市场最终必将收敛，其核心驱动因素主要有三点：
1. **工作负载自身的快速演进与极度专业化（Extreme Specialization）的不可持续**：
如今的计算架构正在走向过度拆分：从预填充（pre-fill）与解码（decode）的拆分，演进到还要专门设立“中填充（mid-fill）”或推测与验证（speculative and verify）层级。历史经验表明，对某个细小阶段进行极度定制化的硬件方案永远无法持久，因为下游的软件工作负载迁移速度实在太快。例如，当行业重心向推理模型（Reasoning Models）倾斜时，算力需求突然又开始回归到类似通用 CPU 的架构特性。
2. **从扁平 LLM 迈向 3D 科学建模与 HPC 精度回归**：
大语言模型（LLM）正逐渐逼近其能力边界。当 AI 转向化学分子模拟、材料分析或生物成像时，扁平化的 LLM 难以胜任复杂的物理空间建模。更重要的是，工作负载开始将传统高性能计算（HPC）与 AI 融合，64 位高精度浮点计算（64-bit precision）重新变得举足轻重。Pat 举例说明道：假设 AI 负责从海量空间中归纳推导出最具潜力的五种化学分析算法，那么接下来的核心算力消耗并不是让 AI 瞎猜，而是运行这五种化学高精度算法本身。过度针对低精度矩阵乘法进行特化的芯片架构，在两到三年后很可能完全跟不上这种算法形态的转移。
3. **规模效应、资本筛选与头部生态“钦点”**：
实体芯片是一场极度依赖规模效应的游戏，无法获取足够资本和实际负载的初创公司根本无法生存。与此同时，OpenAI、NVIDIA、Anthropic 等超级巨头最终会亲自“挑选赢家（pick winners）”。由于硬件与软件必须深度协同进化（co-evolve），巨头们最多只会挑出一两家具有代表性的架构路线重度投资其软件生态，绝不可能为一百家芯片各自适配底层软件库。面对如此庞大的部署复杂性，整个市场必然会加速向少数主流架构收拢。
### `20:00` 异构计算的软件门槛与规模化硬件落地的现实壁垒
对话首先聚焦于计算架构的多元化演进。历史上，行业之所以收敛并标准化在一两种主流硬件架构上，主要原因是不同的指令集开发极具挑战：工程师必须为每种架构编写多套编译器，巨大的软件复杂度倒逼硬件平台必须保持少数与集中。然而在当前 AI 时代，这种逻辑似乎正在被改写——如果一个智能体集群（agent swarm）在一夜之间就能生成软件支持，那么任何具备特定能力的专用芯片都可以迅速拥有匹配的上层软件栈。
面对“软件变简单是否会导致硬件呈现更高异构性”的疑问，Pat Gelsinger 明确指出，软件门槛的降低只是转移了行业瓶颈，而非消除了瓶颈。他强调，现实中制造专用芯片依然面临长达一年半的物理周期，而且将其集成到机架、机房以形成实用规模，需要极其庞大的资本投入：
> “好吧，我能借 50 亿美元吗？我是说，你讨论的不是小数目的资本支出。你讨论的是在数据中心、电力承诺等方面极其巨大的资本支出。我必须把这套东西按规模建立起来。因此，这会在结果上直接否定掉很多异构性。” —— Pat Gelsinger
Pat 进一步举例说明行业走向：巨头们会逐步“收口”这些异构能力。以 NVIDIA 对 Groq 架构的潜在应对方式为例，过不了两三年，外界根本无法在 NVIDIA 内部区分出 Groq 的痕迹，它只会变成架构中又一个特化的加速模块。硬件的分层与抽象（layering and abstraction）始终在幕后消化并隐藏异构性，加之规模化制造部署的极端困难，市面上并不会真正出现无休止分散的异构硬件生态。
```mermaid
flowchart LR
accTitle: Hardware Heterogeneity Bottlenecks
accDescr: 流程图展示了软件生成门槛降低后，硬件异构化依然遭遇的生产与资本瓶颈。
subgraph SW["软件层演进"]
A["智能体生成编译器/软件栈"] --> B["开发特定专用芯片门槛降低"]
end
subgraph HW["物理与资本瓶颈"]
B --> C["制造部署周期需 1.5 年"]
C --> D["数百亿美元级数据中心与电力 CAPEX"]
end
subgraph Result["行业收敛结果"]
D --> E["巨头通过底层抽象隐藏异构模块"]
end
```
---
### `22:15` 存储产业三十年沉寂与 AI 带来的资本繁荣
话题随后转向存储器创新。当谈到高带宽内存（HBM）时，大家调侃它就像邱吉尔眼中的民主制度——虽然毛病众多（被 Pat 称为“可怕的内存”），却是当下能找到的最好选择。回顾整个半导体历史，上一轮引人瞩目的存储技术创新还要数 Intel 的 Optane（傲腾，基于 3D XPoint 技术），而 Pat 直言：“那个技术死了。是我们亲手杀了它。”
Pat 回顾道，他在职业生涯中至少直接参与过 5 种未能存活的新型存储架构，熟悉行业在过去 30 年中尝试过的近 100 种方案。然而惨烈的事实是：过去 30 年中，全行业诞生的主流全新存储形态数量为零，整个世界依旧只能依赖 DRAM、SRAM 和 Flash（NAND 闪存堆叠仅算微小改进）。之所以未能突破出低成本、高性能、高韧性且可量产的新物理层存储，根本原因在于过去几十年来存储器是一个糟糕透顶的行业：
> “因为你可能一年赚了钱，接下来的四年都在亏钱。那么，谁愿意去一个五年只赚一年钱的行业里做研发呢？这一直以来都极其艰难，因为它始终被剧烈的商品化周期所驱动。” —— Pat Gelsinger
但这一格局在过去四五年间发生了翻天覆地的剧变。AI 本质上是一个极端依赖存储吞吐的工作负载（“AI is a memory workload”）。全球前三大存储巨头（三星、SK海力士、美光）一跃全部跻身全球市值前二十强公司之列，其总体市值在近四年暴涨了约 2.5 万亿美元。正是这种天量资本的涌入，为新一轮存储物理特性的技术突破提供了历史性的回报支撑。
---
### `25:00` 突破海岸线瓶颈与芯片垂直堆叠的物理现实
在 AI 驱动下，Pat 看到两大最具前景的存储发展方向：
1. **打破 HBM 的“海岸线带宽限制”（Shoreline Bandwidth Limitation）：** 传统的芯片周边引脚与物理互连受限，计算与存储必须以革命性的新结构融合，正如 d-Matrix 或 Cerebras 所探索的技术路径。
2. **新型存储材料的量产落地：** 包括铁电存储器（Ferroelectrics）、非电容型高密度存储等兼具高性能、高密度与可垂直堆叠特性的材料。不过，Pat 个人对利用低成本 Flash 或 MRAM 充当计算内存的路线并不看好，因为底层存储单元的速度和延迟性能存在难以克服的物理缺陷。Pat 透露自己近期刚投资了一家处于隐形模式（stealth mode）的新存储初创公司，坚信“三十年来未曾有过的存储创新已经近在眼前”。
针对未来芯片是否会像千层饼一样迅速发展到 16 层逻辑与存储混合堆叠的问题，Pat 给出了基于制造良率的冷静研判：
> “堆叠的问题在于，随着堆叠维度的升高，整个堆叠结构的价值也呈指数级放大，这意味着堆叠工艺中任何单一单层的良率都必须持续提升。它不能是线性提升，而必须是指数级提升。你的制造流程必须接近绝对完美。” —— Pat Gelsinger
尽管业界熟悉 DRAM 的坏块替换（bad block replacement）与备用冗余线路机制，但在三维空间内，如果底层硅片物理碎裂（cracked die），任何容错冗余设计都会彻底失效。因此，Pat 认为业界不会盲目走向 16 或 32 层的极限堆叠：
- **甜蜜点预测：** 3 到 5 层的堆叠是工程制造与良率的最佳平衡点（sweet spot）。
- **计算与存储配置：** 计算核心芯片受光刻掩模版尺寸（reticle limit）限制，会向高良率、利于散热的小芯片（Chiplet）拆分演进；而上方搭载的垂直存储堆叠，2 层或 4 层堆叠是切实可行的合理配置，不太可能走向 8 层甚至更高的非理性堆栈。
- **协同设计挑战：** 在这一垂直高度下，供电网络（Power Delivery）的纵向贯穿集成将成为下一个必须攻克的核心工程难题。
### `30:00` 3D 封装极限、热管理困境与存算分离之争
在先进封装的演进路径上，Pat Gelsinger 详细拆解了垂直堆叠（Z 轴）面临的物理极限。在他构想的标准计算复合体中，不仅需要供电轨（power rails）直接引入，还需要高智能度的再分布层（RDL, Redistribution Layer）来分发信号，再加上底层的硅基计算单元，整个结构在物理上已经达到了约 8 层的厚度。与此同时，为了将 I/O 深度集成，还必须引入光学互连，这意味着硅基结构需要与 III-V 族化合物半导体材料紧密结合。Pat 坦言：“这既是一项工程壮举，也是一场制造噩梦（an engineering feat and a manufacturing nightmare）。”在制造和物理的双重约束下，盲目堆叠 16 层的内存无异于天方夜谭，他判断堆叠 2 到 4 层的内存结构才是兼顾性能与制造可行性的黄金平衡点。
随后，Guido Appenzeller 提醒了热耗散（thermal dissipation）与新材料的挑战。Pat 指出，散热的第一法则是消除物理热点（thermal hotspots），通过更智能的电路设计来平衡热量；如果能在稳压设计上做出改进，就能避免产生破坏性的“热烟囱”（thermal chimneys）。业界目前正在探索包括人造金刚石在内的高导热新材料，以及直接贴近芯片的先进液冷方案。面对这些机械与热力学挑战，Pat 调侃道：“我们工程师现在都变成水管工了（We engineers are becoming plumbers）。”
针对部分架构师提出的“将内存完全移出封装、依靠全光链路连接计算与大内存池”的激进设想，Pat 持坚决反对态度。光电转换（OEO, Optical-to-Electrical-to-Optical）不仅复杂度极高，还会伴随严重的能耗损失。Pat 强调：“如今单次计算消耗的飞焦数（femtojoules per compute）已经低得惊人，相比之下，每传输一个比特所消耗的能量要糟糕上千倍（femtojoules per bit of communication is comparatively a thousand times worse）。”为了换取庞大的内存池而在通信链路中白白浪费大量功耗，在他看来违背了底层物理规律。
```mermaid
flowchart TD
accTitle: 异构集成封装架构与物理挑战
accDescr: 展示自上而下的垂直供电、内存堆叠、III-V 族光学 I/O 与底部液冷散热复合体。
subgraph Thermal["散热与结构层"]
LQC["微流道液冷 / 金刚石导热界面"]
end
subgraph ComputeComplex["3D 混合集成复合体 (~8 层)"]
PWR["背部供电轨 (Power Rails)"]
RDL["高密度重分布层 (RDL) 信号路由"]
MEM["2-4 层 HBM 内存堆叠 (平衡良率与高度)"]
CORE["核心逻辑计算芯片 (Silicon Core)"]
OPT["III-V 族光电集成模块 (Optical I/O)"]
end
PWR --> RDL
RDL --> CORE
MEM <--> CORE
OPT <--> CORE
CORE --> LQC
```
关于另一种技术路线——存内计算（PIM, Processing-In-Memory），Pat 同样抱有怀疑态度。虽然存内计算试图将细碎的计算逻辑下推到内存中以削减带宽需求，但这种做法对上层软件和计算工作负载施加了过于严苛的限制。Pat 总结道：“这项技术断断续续发展了 25 到 30 年，我相信 25 到 30 年后它依然会是这副不死不活的样子。”他坚信，能够提供高密度、高性能内存带宽，并使其尽可能贴近核心计算引擎的传统方案，才是能够适应不断演变的工作负载的通用正道。
- 堆叠高度瓶颈：8 层结构的 Z 轴复合体已逼近极限，16 层堆叠在工程上极不可行，2 至 4 层堆叠是当前工艺的实际平衡点。
- 能耗断层对比：计算单比特的能耗极低，而跨封装通信传输一个比特消耗的飞焦（femtojoules）高出数千倍。
- 存算一体（PIM）评价：局限性在于对特定工作负载的强绑定，缺乏通用计算的普适扩展能力。
---
### `35:00` 铜线物理极限、CPO 光互连与 2028 年供应链转折点
尽管在核心计算复合体内部坚决抵制光电转换的损耗，Pat 却全力支持将所有 I/O 功能全面转向光学技术。他笑着回忆：“我在 25 年前就宣告了铜线的死期，最终我总会对一次的（I declared the death of copper about 25 years ago. Eventually, I'll be right）。”如今，物理学的极限已经将这个预言逼到了眼前：在 Scale-Up 算力集群中，高频铜缆正在变成传播距离越来越短的波导；让铜缆在 5 米距离下稳定传输的成本，已经反超了光纤在 100 米距离下的综合成本。
当被问及为何全光架构至今仍未全面铺开时，Pat 指出瓶颈不在于理论，而在于整个供应链缺乏在极端规模下的验证。以英伟达的 NVLink Rack（NVL72）为例，Pat 直言不讳地指出：“说实话，我们本来根本不该去造 NVL72 这种怪物。它是一个工程奇迹，却是一场制造噩梦（In reality, I should have never built an NVL72. You know, it's an engineering marvel and it's a manufacturing nightmare）。”业界整整耗费了一年半的时间才勉强消化并量产了这款充满密集铜缆的机柜方案。这充分暴露出成熟光互连供应链的缺位，如果早有成熟的方案，集群将具备更出色的能效与成本表现。
Pat 预测，2028 到 2029 年将成为全行业向近封装光学（NPO）与共封装光学（CPO）切换的关键窗口期。要支撑 1000 万颗 GPU 规模的超大型集群，工业界必须正面解决一系列棘手工程问题：光器件对芯片高温环境的极度敏感、高密度封装集成工艺，以及全球激光器（laser）制造产能的严重不足。虽然挑战严峻，但这只是繁琐工程细节的落地与攻坚，并无无法逾越的物理死结。
- 传输成本反转：5 米高速铜缆的实现成本与工程复杂度已高于 100 米光纤链路。
- 架构替代周期：整个半导体生态系统预计将在 2028 至 2029 年全面向 NPO 和 CPO 方案转移。
- 制造瓶颈清单：工业界面临激光器晶圆制造产能不足、光电共封装热应力失配以及千万级 GPU 部署规模下的长期可靠性考验。
---
### `38:00` AI 流量特征与光线路交换（OCS）架构的未来
话题转向网络拓扑与架构时，Guido 询问全光互连是否会导致网络转向电路交换（circuit-switched networks），因为超高带宽的光网络很难无限期扩展传统的分组交换节点。Pat 借用斯坦福大学教授 Nick McKeown（恰好是 Guido 的博士论文导师）的经典洞见剖析了底层逻辑：传统互联网交换架构建立在“网络必须在毫无先验知识的情况下，处理去往任意目的地的不可预测数据包”这一前提之上；而 AI 集群中的算力通信流则完全相反——它们是高度可预测的、巨大的吞吐洪流（flows that are large and predictable）。
面对这种大带宽且极具规律性的流量特征，底层的网络拓扑天然更契合交换电路而非传统的分组交换（packet switching）。Pat 预测，引入光线路交换（OCS, Optical Circuit Switching）或某种形式的光电混合交换机制，将光技术直接穿透到计算核心，是未来超大规模算力集群的必经之路。
```mermaid
flowchart LR
accTitle: 传统网络与 AI 专用光网络特性对比
accDescr: 对比通用互联网的分组交换与 AI 集群的大容量可预测光线路交换架构。
subgraph Legacy["传统数据中心网络"]
direction TB
L1["小包混合流量"] --> L2["路由不可知"]
L2 --> L3["传统电分组交换 (Packet Switching)"]
end
subgraph AINetwork["AI 智算集群网络趋势"]
direction TB
A1["大带宽矩阵数据流"] --> A2["周期与模式可预测"]
A2 --> A3["光线路交换 (OCS / Optical Switching)"]
end
```
这种架构的重构最终模糊了 Scale-Up（纵向扩展）与 Scale-Out（横向扩展）的边界。Pat 强调：“如果你的纵向扩展集群基数（radix）足够庞大，那么如何定义它何时连接到其他超级集群，将变成一个极具优雅美感的工程答案。”将光交换与高预测性负载结合，不仅打破了现有网络交换芯片的端口数瓶颈，更为未来处理数以千万计 GPU 的互连拓扑指明了方向。
- 流量模式对比：传统互联网属于无序小数据包交换，AI 智算工作负载则是高预测性的大规模张量流动。
- 交换技术变革：光线路交换（OCS）能够跨过传统电交换芯片的高功耗与端口数（radix）限制，成为超大规模算力集群的主干。
- 系统边界融合：光互连的高密度扩展能力正在打破 Scale-Up 与 Scale-Out 的物理界限，推动超大单体拓扑的诞生。
### `40:00` Scale-up 与 Scale-out 的融合，以及训练与推理界限的消失
讨论首先围绕系统架构中的 Scale-up（纵向扩展）与 Scale-out（横向扩展）展开。Pat Gelsinger 指出，这两种扩展方式之间的界限在某种程度上是一种人为的人工划分，之所以在工程实践中感觉差异巨大，主要是因为人们在前端和后端网络中采用了不同的通信协议。但如果跳出微观实现、从纯粹的系统架构视角俯瞰，两者的根本目的是一致的，随着时间推移，协议和架构终将走向融合。
紧接着，对话转向了 AI 训练与推理工作负载演进的趋势。Pat Gelsinger 提出了一个核心观点：未来训练与推理之间的区隔会越来越小，而非越来越大。人的大脑从来不会停止学习，AI 也是如此——持续学习（continuous learning）将成为许多算法的固有属性。在推理环境中实时更新模型权重、借助推理对各类工作负载进行持续微调与演进，会使传统的“离线集中式大规模训练”与“静态在线推理”之间的壁垒彻底打破。虽然 Guido Appenzeller 提出，推理所需的一致性节点直径较小，且不同类型的 AI 工作负载在底层计算需求上差异巨大（例如计算受限、内存带宽受限、小规模 Diffusion 模型与具有数万亿参数的超大规模 LLM 对应的硬件卡完全不同，这种差异可能会助长芯片架构的异构化），但 Pat Gelsinger 警示了大规模运维的复杂性。如果数据中心要部署一百万颗 GPU，却针对不同任务碎片化地采购一千颗这类专用卡、两千颗那类专用卡，那么管理成本、升级维护、网络互联以及故障域（fault domain）划分等问题将彻底击垮规模优势。Raghu Raghuram 随即开玩笑打趣道：“智能体（Agent）会搞定所有升级，所以这不再是问题了”，引来全场会心一笑。
Pat Gelsinger 在阐述模型演进时断言：
> “Right, you know, by the way, as another characteristic of workload evolution, I do think the separation of training and inferencing is going to become less so going forward, not more so.”（顺便说一句，作为工作负载演进的另一个特征，我确实认为展望未来，训练与推理之间的界限会越来越小，而不是越来越大。）
- **Scale-up 与 Scale-out 现状**：目前依赖不同的前端网络与后端互联协议，但底层系统目标一致，未来架构会逐步收敛。
- **异构架构的利弊权衡**：尽管 Diffusion 等不同模型在算力与内存带宽上的配比不同，但定制硬件会严重破坏数据中心的统一可伸缩性与运维效率。
```mermaid
flowchart LR
accTitle: AI工作负载与架构收敛趋势
accDescr: 展示从当前的离线训练和静态推理向融合的持续学习架构演进的过程。
subgraph Current["当前分割架构"]
direction TB
T1["大规模集中式训练<br/>(Scale-out / 昂贵算力)"]
I1["静态离线推理<br/>(较小节点集群 / 只读模型)"]
end
subgraph Convergence["未来收敛趋势"]
direction TB
CL["持续学习架构 (Continuous Learning)<br/>推理同时更新权重"]
Unified["网络协议与系统架构融合<br/>Scale-up 与 Scale-out 界限模糊"]
end
Current -->|"算法演进与系统标准化"| Convergence
```
---
### `43:24` 能源瓶颈：AI 时代的算力扩张与国家电力困境
话题从计算、内存和网络转向了最为致命的现实物理瓶颈——电力供应。Guido Appenzeller 指出，从芯片内部的垂直供电，到数据中心内 800V 直流电（800V DC），再到整个电网发电与复杂的社会政治环境，电力链条极其庞大。Pat Gelsinger 从第一性原理剖析了当前的危机：在过去 10 到 15 年间，美国在能源容量建设上的表现极其糟糕，基本上是“以新增可再生能源的速度同步关停煤电”，导致全国总电力容量长期处于停滞状态。
Pat Gelsinger 强调了一个核心判断：在 AI 与数字时代，能源容量直接等同于经济容量。国家能源容量停滞 15 年，实质上意味着国家潜在的经济增长容量被冻结了 15 年。尽管近五年各方资本开始疯狂涌入能源领域，但国家年均能源容量增长率也仅仅爬升到 4% 左右。更严峻的是，可再生能源产业链极度依赖中国，而燃气轮机（gas turbine）的交付周期目前长达整整八年。与此同时，作为最佳基荷电力（baseload）的核电在美国几乎荒废了整整一代人，上一座核反应堆并网已经是数十年前的事情了。这种电力基建的迟滞将对 AI 的狂热形成实质性的压制——如果买了 100 万颗 GPU 却根本无法通电，建数据中心便毫无意义，未来甚至可能目睹大量数据中心项目因缺电而发生违约。
Pat Gelsinger 直言不讳地指出能源对经济和 AI 狂潮的根本制约：
> “Because in an AI digital age, energy capacity equals economic capacity.”（因为在 AI 数字时代，能源容量就等于经济容量。）
关于数据中心缺电引发的连锁反应，他进一步警告：
> “And fundamentally, energy capacity somewhat dampens how euphoric we can get on AI. Because why build a new data center and buy the million GPUs if I can't power them?”（从根本上说，能源容量在某种程度上遏制了我们在 AI 上的狂热情绪。因为如果我根本无法给它们供电，为什么还要建新的数据中心并购买一百万颗 GPU 呢？）
- **电网产能停滞**：过去 10 至 15 年美国因以新能源平替煤电，整体电力容量几乎没有净增长；近来年均增速也仅勉强达到约 4%。
- **供应链瓶颈**：可再生能源深度依赖外部供应链；燃气轮机采购排期长达 8 年；本土核能产业链遭遇长期断档。
- **商业风险**：Oracle 等企业面临的电力挑战只是冰山一角，缺乏稳定电网承载力的资本开支将直接遭遇违约与交付阻碍。
---
### `47:18` 硬件复兴与颠覆物理学：800V 直流、垂直氮化镓与超导技术
面对巨大的电力缺口，数据中心内部的电能输送与转换效率成为创新的焦点。Guido Appenzeller 调侃转向 800V DC 是“爱迪生的复仇”（The revenge of Edison is upon us）。Pat Gelsinger 赞同并解释道，过去由于历史沿革和标准化原因，电力在进入芯片前要经历繁琐的多级降压与交直流转换，损耗惊人。当下必须大刀阔斧削减转换步骤，例如通过 Vertical GaN（垂直氮化镓）技术与固态变压器，实现从 800V 一级直接降压到 48V、12V 甚至 5V，从而大幅提升能效。
这一系列的现实挑战让传统硬件技术迎来了文艺复兴。Pat Gelsinger 幽默地感叹，时隔 25 年，自己居然必须重新去读卡诺循环（Carnot cycle）定律和卡诺效率（Carnot efficiencies），研究涡轮机和新型制冷剂（refrigerants），“热力学终于再次发挥作用了”。然而，传统的 CMOS 工艺能效在过去五代 NVIDIA 芯片中每 Teraflop 的功耗几乎没有实质性改善，处于完全停滞（flatline）状态。单靠优化工程已经不够，必须从底层打破现有的物理极限。他以 Playground Global 投资的超导初创公司 Snowcat 为例，指出超导技术有望将能效比直接提升 1,000 倍，从底层物理学维度重塑计算体系。
Pat Gelsinger 批评当前芯片架构能效停滞并呼吁物理级创新：
> “Because CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline. Not okay.”（因为在 NVIDIA 芯片过去的五代产品中，CMOS 并没有改善每 Teraflop 的本质功耗，完全停滞了。这不可接受。）
- **供电架构演进**：向 800V DC 演进，采用垂直氮化镓（Vertical GaN）将多级降压压减为单级转换（如 800V 直转 48V/12V/5V）。
- **硬件工程复兴**：散热与制冷技术成为前沿热点，热力学卡诺效率研究重返工程师视线。
- **打破物理瓶颈**：CMOS 能效比遭遇平台期，前沿押注聚焦于超导计算（如初创公司 Snowcat），寻求千倍级的能效跨越。
---
### `51:40` 经典的抽象永存：虚拟化与虚拟机的永恒价值
节目的尾声，Raghu Raghuram 提起时间管理失控的话题，但执意要在结束前抛出最后的重磅问题。鉴于 Pat Gelsinger 和 Raghu Raghuram 分别是 VMware 的前任 CEO，Raghu 调侃道：“虚拟机（VM）又回来了，作为 VMware 的前 CEO，你怎么看？” Guido Appenzeller 立即在旁假装委屈地打趣：“我觉得自己被这场对话排挤在外了。”
Pat Gelsinger 随后回归到计算机科学的核心本质，给出了富有哲理的回应。他认为，技术历史上的每一次重大创新，最终都会引申出更高维度的抽象。计算机科学的根基永远是“数据结构、算法和抽象”，无论虚拟机模型是在底层基础设施层面实现，还是在顶层应用层面构建，这种抽象形态都历久弥新，在计算体系结构中理应永远占据不可磨灭的核心席位。在欢快的笑声中，他们互相调侃着现在是不是“该回去重新运行 VMware 了”。
Pat Gelsinger 总结虚拟机的长久生命力时表示：
> “And hey, I think the virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy.”（我认为，虚拟机抽象无论是在基础设施层还是在应用层实现，它仍然是一个基础性的抽象模型，理应在计算层级中始终占有一席之地。）
- **计算机科学三要素**：所有前沿创新的基石依然是数据结构（Data structures）、算法（Algorithms）与抽象（Abstraction）。
- **虚拟机的角色定位**：即便在面对 AI 和智能体驱动的新时代，虚拟化抽象依然是不可或缺的系统隔离、调度与资源解耦范式。
### `50:00` 为 Agent 重塑虚拟化：下一代计算架构的基础设施
对话切入了一个引人深思的虚拟化历史命题。Pat Gelsinger 结合早年在 VMware 的经历指出，当年他们几乎管理了计算层级的方方面面——从工作负载、网络到存储系统，核心都在于如何对这些资源进行抽象。而在当下的 AI 背景中，面对即将到来的“Agent 蜂群”（Agent swarms），整个管理架构必须从零重构：究竟由谁来管理这些 Agent？谁来为它们构建安全边界？谁来负责监控和优化其性能？Pat 强调，虚拟化与系统管理层面的每一个核心要素，都必须在这一全新的计算层级中重新发明，这为初创企业提供了颠覆性突破的广阔空间。
Guido Appenzeller 进一步将讨论推向高潮，他指出一个本质变化：过去打造虚拟机（VM）是为了供人类使用，而现在的虚拟机则是为了供 Agent 使用。这种服务对象的转变彻底颠覆了产品的形态——它改变了基础设施的获客和营销逻辑（系统必须让 Agent 主动挑选你的 VM 方案），也极大地压缩了启动延迟的容忍度，因为“人类比 Agent 有耐心得多”。当 Guido 追问如果今天重新创办一家“面向 Agent 的 VMware”会是怎样时，Pat 认同这一视角的革命性，但也强调必须兼顾人类治理：
> “Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy.” —— Pat Gelsinger
```mermaid
flowchart TD
accTitle: 面向 Agent 的虚拟化双层架构
accDescr: 展示人类策略层与 Agent 基础设施层之间的交互与映射关系
subgraph Governance ["人类治理层 (Human Governance)"]
A["设定宪法与安全护栏<br>(Constitutions & Guardrails)"]
B["策略配置与仪表盘监控<br>(Policies & Dashboards)"]
end
subgraph Infrastructure ["Agent 基础设施层 (Agent Infrastructure)"]
C["Agent 专用虚拟机<br>(VM for Agents)"]
D["瞬时启动与性能调度<br>(Performance & Cold Start)"]
E["Agent 动态迁移<br>(vMotion for Agents)"]
F["蜂群安全与隔离边界<br>(Agent Swarm Security)"]
end
Governance -->|"下发安全与业务约束"| Infrastructure
Infrastructure -->|"上报状态与遥测数据"| Governance
```
- **从面向硬件转向面向 Agent**：过去的虚拟化难题在于如何高效抽象底层物理硬件以服务操作系统和应用；而现在系统的核心约束变成了“如何让 Agent 变得高效、安全且便于抽象”。
- **经典机制的重构**：如 VMware 时代标志性的实时迁移技术（vMotion），在未来演变为针对 Agent 状态、上下文及运行环境的无缝迁移（“vMotion for Agents”）。
- **人机双轨控制**：Pat 借用 Anthropic 联合创始人 Dario Amodei 提出的“宪法/护栏”（Constitutional AI / Guardrails）概念指出，系统不仅要在底层承载 Agent，还要为人类留出顶层的策略定义接口与监控仪表盘。
### `52:45` 对话尾声与免责声明
在探讨完 Agent 时代的虚拟化演进后，Raghu Raghuram 与 Guido Appenzeller 表达了对 Pat Gelsinger 的由衷感谢，Pat 也热情回应，称两位主持是他非常喜爱的人，并表示非常乐意未来再次参与讨论。节目随后进入标准的主持人致谢、各收听渠道（YouTube、Apple Podcasts、Spotify、X、Substack）推荐，以及 a16z 针对播客内容的常规法律与投资免责声明，正式为本期关于芯片、内存、能源与 AI 基础设施底层演进的深度对谈画上句号。
> “Well, two of my favorite people, anytime.” —— Pat Gelsinger
## Notable quotes
> `00:00` 在 AI 数字化时代，能源容量等于经济容量。如果我连供电都做不到，为什么还要建新的数据中心并买上一百万张 GPU 呢？你将会看到越来越多的数据中心项目发生违约，因为那里的能源根本无法到位。 — Pat Gelsinger
> `43:24` Because in an AI digital age, energy capacity equals economic capacity. — Pat Gelsinger
> `25:00` 堆叠的问题在于，随着堆叠维度的升高，整个堆叠结构的价值也呈指数级放大，这意味着堆叠工艺中任何单一单层的良率都必须持续提升。它不能是线性提升，而必须是指数级提升。你的制造流程必须接近绝对完美。 — Pat Gelsinger
> `47:18` Because CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline. Not okay. — Pat Gelsinger
> `43:24` And fundamentally, energy capacity somewhat dampens how euphoric we can get on AI. Because why build a new data center and buy the million GPUs if I can't power them? — Pat Gelsinger
> `50:00` Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy. — Pat Gelsinger
> `51:40` And hey, I think the virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy. — Pat Gelsinger
> `40:00` Right, you know, by the way, as another characteristic of workload evolution, I do think the separation of training and inferencing is going to become less so going forward, not more so. — Pat Gelsinger
## Mentioned
Pat Gelsinger, Raghu Raghuram, Guido Appenzeller, Intel, VMware, Playground Global, a16z, Horowitz Andreessen Academy, Ron Smith, Edward McCluskey, Alberto Sangiovanni-Vincentelli, Jalapeno, Graphcore, NVIDIA, OpenAI, Anthropic, Groq, Samsung, SK Hynix, Micron, Optane, d-Matrix, Cerebras, Nick McKeown, Snowcat, Oracle, Dario Amodei
## Video description
a16z's Raghu Raghuram and Guido Appenzeller sit down with Playground Global General Partner and former Intel CEO Pat Gelsinger to discuss the next wave of semiconductor innovation and the physical constraints shaping the AI buildout.
Drawing on his experience designing Intel's 386 and 486 processors, Pat explains how AI could transform chip design, but also why faster design alone won't solve the industry's biggest problems. They examine the bottlenecks in manufacturing, memory bandwidth, advanced packaging, and power, and why today's explosion of specialized AI chips may eventually consolidate around a smaller number of architectures.
They also discuss the potential for new memory technologies, the shift from copper to optical networking, and why energy capacity could become a major constraint on AI growth. Finally, they revisit Pat's VMware years to ask what virtualization might look like when infrastructure is built for agents rather than humans.
[Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/2a98eb9e-80e8-4172-958c-8f92f8976e78/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=2a98eb9e-80e8-4172-958c-8f92f8976e78&feed=JGE3yC0V)
Source quotes
[0:00] Pat Gelsinger: 在 AI 数字化时代，能源容量等于经济容量。如果我连供电都做不到，为什么还要建新的数据中心并买上一百万张 GPU 呢？你将会看到越来越多的数据中心项目发生违约，因为那里的能源根本无法到位。
[43:24] Pat Gelsinger: Because in an AI digital age, energy capacity equals economic capacity.
[25:00] Pat Gelsinger: 堆叠的问题在于，随着堆叠维度的升高，整个堆叠结构的价值也呈指数级放大，这意味着堆叠工艺中任何单一单层的良率都必须持续提升。它不能是线性提升，而必须是指数级提升。你的制造流程必须接近绝对完美。
[47:18] Pat Gelsinger: Because CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline. Not okay.
[43:24] Pat Gelsinger: And fundamentally, energy capacity somewhat dampens how euphoric we can get on AI. Because why build a new data center and buy the million GPUs if I can't power them?
[50:00] Pat Gelsinger: Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy.
[51:40] Pat Gelsinger: And hey, I think the virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy.
[40:00] Pat Gelsinger: Right, you know, by the way, as another characteristic of workload evolution, I do think the separation of training and inferencing is going to become less so going forward, not more so.
Source quotes
[0:00] Pat Gelsinger: In an AI digital age, energy capacity equals economic capacity.
[10:00] Pat Gelsinger: HBM is a hideous memory. It's just the best one that we've got.
[46:04] Pat Gelsinger: CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline.
[50:00] Pat Gelsinger: Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy.
[40:00] Pat Gelsinger: I do think the separation of training and inferencing is going to become less so going forward, not more so.
[48:38] Pat Gelsinger: The virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy.
Original transcript
[0:00] In an AI digital age, energy capacity equals economic capacity. Why build a new data center and buy a million GPUs if I can't power that? You're going to see more and more defaults happening on many of those data center projects because the energy won't be there. over the last 30 years? Zero. Memory innovation for the first time in 30 years is nigh upon us. I declared the death of copper about 25 years ago. Eventually, I'll be right for all of us hardware guys. It's like renaissance in front of AI inference accelerator chips, and I'm sure I don't even know them all. Why are you guys funding so many of those anyways? You funded your first share, too. Historically, there have not been 100 competing processor vendors in any industry ever. Is this a temporary thing? I see it as... the tools they needed to build the chips themselves. He explains why AI could bring a similar shift today, even as manufacturing timelines, memory bandwidth, and power become harder constraints. They also debate whether the explosion of specialized AI chips will last, why Pat believes memory innovation is finally approaching a breakthrough, and how optical networking could change the architecture of AI clusters. In drawing on their shared history at VMware, virtualization for a world where AI agents rather than humans are the primary users. I'm here with our esteemed guest and dear friend and former boss, Mr. Pat Gelsinger. Welcome, Pat. Pat is currently the general partner at Playground Global, but as you're very, very well known in the industry for leading Intel, for being the CTO of Intel, of course, leading VMware and many other things. So, welcome. Hey, thank you, Raygu. Great to be with you and Guido. And to me, this feels just a little bit like old home, right? Yeah. We superimposed a year of change, right? But you both look the same. I feel energetic. So let's dive in. There has never been a day when you stop being energetic. So there's no news there. But yeah, let's start actually from your Intel career. Recently, Andreessen Horowitz created this Horowitz Andreessen Academy, which was to find talented people between 16 and 22, and then put them into a modern world. educational setting, so they'd be well prepared for either doing things of their own or joining the companies of today. You had a similar experience, like you went to a regular trade school and then jumped into Intel when you were, what, 18 or 19, I think? Yeah, it was really a sort of magical period. I'm 16 years old. I accidentally went to a scholarship. I go to tech school. I skipped my last year and a half of high school, and Intel comes recruiting when I'm 18 had already fallen in love with computers at tech school. And the interviewer, right, Ron Smith was his name. He writes on his page. And I was number 12 that he interviewed that day. And if you've interviewed 12 people in a row, you can't tell male from female by the end of it, right? I'm number 12. He says, smart, aggressive, arrogant. He'll fit right in. So I got invited to come to Intel, and it just became really glorious. Starting as a technician, moved into the design team engineer number four on the 386, architect and design manager for the 486. All of that while doing my bachelor's, my master's, and PhD work. So it was like the best career that you could possibly have. You're learning by day, and you're putting it to practice at night. That's right. That's right. You're creating a completely new era of chips. I bet most of what you learned and what you built, there was a big gap because you're breaking new ground. Yeah, in a lot of ways. and it really was a pretty incredible time in the industry. And I remember one of the professor classes, he postulated a new Cary look-ahead adder design. Right. And one of my classes at Stanford. So I said, well, I'm working on the 386 carry look ahead at her. So let me go try it. So I come back two days later to the professor and I said, it doesn't work. Right. And I started arguing with my professor at Stanford. You know, it was like, what?
[5:00] The next two class periods were just me and the professor arguing over why his design didn't work. He had a book that was being published with that design in. So it was very annoying. Oh, it was very time-sensitive. And we started built-in self-test, right? And I remember Ed McCluskey at Stanford was my professor, and he was sort of the father of that. And we had wonderful arguments around why most of his ideas were bad. They weren't practical for real chips, but I was putting built-in self-test to work on the three and six. my thesis advisor, and obviously he did okay. I helped his career. He became president, and he's done okay since then. Later on, we'll talk about the CIS courses, this kind of debate. I had the same experience when I was teaching at Stanford. You come with something that you think is the truth, and then the student comes that works in industries. Well, actually, not quite. So the next lecture, you come back with, okay, I looked this up. You're right. This is outdated. That interaction, and I think that's part of what makes Silicon Valley so unique. You have so many companies and entrepreneurship. It's like in the water here, right? Starting things up and new ideas and challenging the professors. And you can't be a professor at a place like Stanford and not expect to get challenged and enjoy it. There's a good foot traffic in and out of that ivory tower. Yeah. Now, 486 was your big Yeah. accomplishment, if you will. Obviously, lots of people worked on it. And you guys broke some new ground in ship design there, right? Mm-hmm. and increase what functions the chip was supposed to do and so on and so forth. Yeah, it was a pretty magic period as well because it was before what you would think of as the EDA industry. Oh, wow. Yes, that's right. And at the end of the, a little bit of the 386, but the 486 was really the first chip to implement what you would think of as modern EDA techniques. So we invented HDL, the Intel hardware description language. So I wrote my own language. Well, you had to build a compiler for that language, right? So we created a compiler for the language. There was no automatic place and route. So we had to invent that. And we worked with Alberto Sanjivangi, Vincentelli at Berkeley, and some of his students for the first place, the first routing, the first automated timing management. So they created EDATA to some degree. And that was really one of the hallmarks that not only was the 486 this compatible pipeline microprocessor, but we ushered in many of the foundations of what became the modern EDA industry and was really a pretty magical period of the industry. Yeah, yeah. If we look at today, will people look back as Blackwell and saying this was of the last generation that was done before AI took over the design process for Microsoft? Are we in a similar transition right now? And I think when you look at things like Jalapeno today, you sort of say, that's sort of a first principles use of AI in the chip design process, where you throw out a lot of those things. Now, there are pieces of design that are still really hard. In that sense, a lot of the analog aspects, CERTES is probably the best example of that, are still not AI-able. You just need lots of silicon data to get those. So you either get so conservative in your analog design that you're able to, I'll say, AI it, or you got to do those the hard way. Design tools are getting smart enough. A few experts guiding the tools and how to apply it. And I really think it will be somewhat like the 46 that way, right? We just sort of look back and say, yep, that was the beginning of a new era of chip design. Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else. Can we guess at this point where the bottleneck will be in the future? There's some things which it seems like AI is incredibly good at, right? Like creating the software layers, you know, writing kernels, a lot of the... I think a lot of this sort of being able to specify the objectives at a higher level, which then gets sort of translated into a very long, right? And so what is the new frontier? What is the thing that, that will be difficult going forward. Well, when you look at today, so let's say Guido and Pat, we're off to do a great new chip together. We're going to build an accident. Right, right. And it's the one that's going to leap ahead because we have understanding of the AI. We have understanding of the AI workloads. We have understanding of how to compose that into the right multiply, accumulate, register structure. Perfect visibility of the model workloads of tomorrow as well. Yeah, and all of that kind of stuff. And let's say we turn our AI.
[10:00] tools loose, we are great. Within three months, We have an awesome design. for that. We've been a little bit conservative on all the analog components of it. Now, we still have a couple of really major bottlenecks today. One is, we still have nine months of silicon processing time. Yeah. Right. Something like that. So I can design the thing in three months, but I can't actually get it into real silicon at scale for nine months. Okay. That sucks. Right. So we have to really see that bottleneck improve if we're going to have this innovation because we know. Are you talking about the validation and verification stages or even beyond that? Well, you know, first I can't get anything out of fab, you know, in less than three months. And that's if I have, you know, like supercharged design flows. Right. So I said, you know, for it. And then I got to get into advanced packages, 3D packages are getting complex. So that's like another three months, right? And then I have to put it into a rack scale solution because nothing's a chip anymore. It's a rack. So we're nine months. It took me three months to design it, but it's nine months until I can actually start to use it. So how do we start to compress those aspects of design? And I think we need new forms of lithography to enable that. We need design flows that don't require $50 million a mass cost until I can get things into prototyping. So, you know, to me, how do you compress that to a month or two? Because I can now do the design in three months. How can I have that done in a month or two? Because if it takes me a year and a half until I actually get scale and software on it, okay, my understanding of the AI workloads is no longer applicable to the chip that I design. We see that play out live right now, I think, with the AI chips. Right. We can look at it historically. It was like Graphcore wasn't a bad design, but the world moved on in that. all of those aspects somewhat obviate the fact that AI has made the design piece easy, all of those manufacturing scale issues to me become a huge bottleneck, right? Another bottleneck, of course, is memory. Yep. You know, memory bandwidth. We noticed, yeah. Right. And, you know, for that, as I've described, HBM is a hideous memory. It's just the best one that we've got. I was going to ask you about that. It's a terrible memory. I get terrible bit density. I'm limited to shoreline bandwidth, power issues, thermal issues. DRAM doesn't like to be hot. Concentrating everything in one spot. It's terrible to give. And AI is a memory compute workload. Yes. So I need to compress that development and embodiment. into a real scale. I need to fix memory. associated with this. So to me, that's another critical bottleneck for us to attack. And power. Power in, thermal, and most of the power simulation aspects today are bad. You know, you don't have good 3D modeling tools for power, power dissipation, hotspots, etc. You know, my voltage in, right, you know, I'm essentially guard banding almost my full power envelope. Correct. Right. So essentially I'm losing, you know. 40% of my power in guard banding, so I got to get much better at power modeling, you know, and then my power dissipation techniques are bad, right? So, you know, the delivery model has to change too. Yeah. So all of these things, you know, to me, those are the, you know, the next big bottlenecks, right? You know, how do I, you know, improve each one of those? And now, right, you know, if I've done that, okay, now we're on to an exciting era for what's next. So, you know, thank you for those great AI tools, but, you know, I got big challenges. even when those are fully deployed. And so... we've seen sort of a rebirth in the number of chips and chip architectures and so on and so forth. Yeah, yeah. Thanks to these workloads. Yeah, it's like there's 100 AI inference accelerator chips, and I'm sure I don't even know them all. Yeah, exactly. Yeah. So, so... Why are you guys funding so many of those anyway? You funded your fair share, too. Yeah, so... Sorry, they're all... to that credit of making different design assumptions, either in the components that you talked about or in the – in the part of the Pareto curve that they're targeting. So do you think there's room for all of these? Or are we in a world where... Partly because of what Guido talked about, AI tooling, we now have the ability to build more different kind of chips. Yeah. And the time is enormous, whichever way you want it. To answer this even more provocatively. Historically, there have not been 100 competing processor vendors in any industry ever, right? Yeah, yeah. Is this a more heterogeneous future? Will we see, is this a temporary thing? It'll convert back to a view? Yeah.
[15:00] I see it as a more temporary thing and it will convert is what I expect. And there's probably three different reasons that I see that to be the case. You know, one of them is, you know, this... that emerging heterogeneity that you see in the compute environment where, okay, now I have specialized pre-fill versus decode. Well, you know, and now people are saying, well, we really need a specialized tier for mid-fill, not just pre-fill, right? So now I have early pre-fill and now I have mid-fill, which is, you know, different. Well, maybe speculate and verify if we listen to OpenAI here, right? the workload. And I think any time that you've seen that in history is not sustainable. And then all of a sudden, right, when, you know, as you look at that, and now people are saying, well, as I go to reasoning models, I want something that looks more like a CPU again. So, you know, all of a sudden, I didn't want that level, you know, that didn't become the predominant portion of my compute capacity. I needed more of it over here, right, and model and model workflows and, at-scale specialization, right? Because I think the workloads are going to continue to moderate, you know, and migrate so significantly. First, you know, second is I think today's models are about to go through some evolutionary breakthroughs as well, right? You know, I think the LLM is sort of reaching the limits. And now as people look to how do I really model 3D? Right, you know, where, you know, a flattish LLM doesn't work well when you go to molecules and chemicals and, you know, imaging. So I think we'll see limits of that that will cause different shifts in the algorithmic domain. I also think that some of the most interesting workloads become where you start, you know, I'll say bringing HPC-like things back into AI-like things. Where all of a sudden, you know, 64-bit precision counts again. Right, so I see... What would be an example of that? well, imagine that my AI model is now inducing the five most interesting domains for chemical analysis. Okay, those chemical algorithms are going to run with high precision, which is going to be the dominant piece of the workload, getting to the five algorithms I need to run or running the five algorithms. on different chemical or biological systems. So I do think that as we optimize our AI systems, they will lead us back to some of the things that have been more traditional high-performance computing. So I see that aspect of workload as well. So all of that said is this extreme specialization to me in the compute architect. I sort of don't like it because I don't think it's going to be what the workloads will look like two, three, four years from now. reason I don't see all of these AI chips as they get more and more specialized for portions of the compute workload to necessarily be right. Second is, there's like a hundred of them now. Right. You know, there's multiple optical ones, there's probabilistic ones and so on. And, you know, they're not all going to win and they're not all going to win because at the end of the day, you have to get scale. Yeah. These things and scale requires, you know, you have to win, you have to get capital, you have to get workloads onto it. So I think it sort of defies logic that you're going to have a hundred of these things. So I see them, you know, coming back as a result of, you know, simple capital market share, et cetera. So I see the workload driving that, you know, I see the natural, you know, industry consolidations, you know, that will occur. And then I also expect that the winners will pick some winners. where an open AI, an NVIDIA, an Anthropic will say, I like this one. because it isn't just hardware, is how do the hardware and software code evolve? And I do expect that there's going to be, hmm, here's this 10 I could pick for this. I like that one. And it does require investment in the software and the workload evolution to take advantage of those platforms. So for those three reasons, yeah, we're going to see a narrowing of the field. And the deployment complexity as well. Sorry, go ahead. Makes total sense. And I think I agree in most part. here just to make sense. You, Guido, storing controversy? No. You guys have changed it, right? So, So obviously, 100 companies won't access or consolidate, right? And I completely agree with your argument that at the end of the day, the large consumers will pick winners to some degree, right? Today, I think to be a first-tier consumer, chip startup to some degree, whether you're that or not, is defined by how much traction you have with the really big guys. That said, um, And there's also an argument to say, look,
[20:00] transition period, you probably have a bit more of an explosion of different cultures. No time that'll shake out. Now, All that said, I think there's an argument you can make that basically says, look, historically, we standardized on, you know, one or two architectures because, you know, different instruction sets were hard. You had to build multiple compilers, you know, like the, basically the software complexity dictated that you can only have a certain small number of hardware platforms. That part seems to be changing with AI a little bit. more, but an agent swarm overnight can do it. That's all we need, right? So basically you can say, if you can build a chip that has a certain capability, it's now much, much easier to build the software layers on top that fully leverages that. Could a result of that be that we have a little bit more heterogeneity just because it's easy to program that heterogeneity? Yeah. And I do think it changes some of the, you know, where the bottleneck lands, right? Changes in that regard. You know, the first bottleneck is the one I point to that we've already talked about. Still takes a year and a half to build these things. Yeah. Right. You know, and then the second bottleneck is not only does it take a year and a half to get these things into racks and scales and so on like that. Now, to make them useful... Okay, well, could I borrow $50 billion? I mean, you're not talking about small capital expenditures. You're talking about very large capital expenditures and data centers, power commitments and so on like that. I've got to build this thing at scale associated with it. So that's going to defy a lot of heterogeneity as a result. Now, clearly big players, and we saw this with what NVIDIA did with Grok. okay, you know, we're going to sort of own some of that heterogeneity, right? And, you know, two years from now, you're not going to tell what Grok is inside of NVIDIA, right? It's just going to be like another special, right? You know, someone like that. So you'll see the architectures, you know, sort of hide the heterogeneity, right, inside of it, which to me is just fine, right? You know, I mean, that's what layering, that's what extraction has always done associated with it. You know, but I think those forces are powerful ones to, I'll say, expose the, A lot of the heterogeneity, no. I think it will get abstracted underneath some of these. But at the end of the day, it's still hard to build these things on scale. Earlier, you talked about HPM being hideous memory. By far the best one we have today. Exactly. It's like the thing about democracy. We can't recall when there was the last memory innovation. It might have been obtained. I think you probably had something to do with it. Yeah, that one died. That one died. We killed it. Yeah, we killed it. So... Do you think memory innovation... is around the corner or what needs to happen for Obviously, we understand the scale problem because it's even more... Yeah. Larger there. Yeah. But if you keep the scale aside, even from a technology point of view. Yeah. And let's, you know, look at that history. I've probably personally been associated with at least five different new memory architectures, Optane just being one of them, right? You know, that did not see the light of day. right, as well. I'm probably familiar with close to 100 that have happened in the industry over the last 30 years. And exactly how many major new memories have we had over the last 30 years? Zero. Right, which, you know, okay, DRAM, SRAM, Flash. Okay. What else is there? DRAM, SRAM, Flash. So I do think... Flash stacking comes for a little bit. Barely. But anyway, so in that, it has been a disappointing that we haven't created the next physics that enable low-cost, high-performance, resilient, manufacturable memory. So I do think, and now, over the last 30, 40 years, memory was a terrible industry. Because you would make money one year and then you'd lose money for the next four. Right. So let's go do R&D in an industry that makes money one out of five years. It's just always been so terribly difficult because it's been driven by such violent commoditization cycles. That has changed a little bit, though, recently. Over the last five years? Happy days, man. So all three big members of Anderson on the planet. in the top 20 most valuable companies on the planet. Yeah. That's a crazy change. It's just so, right, you know, who would have thought, right, associated with it. And anybody who looks at the AI workloads, and it's always, you know, it's much harder to, you know, it's easy to predict the past, but it's a little bit harder to predict the future, right? And, you know, in that, you look at it and say, AI is a memory workload. Exactly. Right, you know, inside of it. Oh,
[25:00] needs that will justify memory innovation. Now, there's two domains that I'm excited about for it. One is this idea, I do think that the shoreline bandwidth limitations of HBM is a fundamental issue. So with that, the idea, and obviously we have a company, D-Matrix, that's doing this, but I think others will be, whether it's Cerebris and their techniques. The future is stacked, yeah. Somehow you've got to bring memory. In more ways than one. You know, you got to bring memory and compute together. Yeah. in fundamental new structures. I do think that given the workload now has such capital... being deployed against it in the AI space, that I do think we'll actually be able to break through some of those physics challenges as well. So I do believe, and I've just funded a new memory company as well. That's still a stealth, but I'm quite excited about some of the innovations that will occur in new memories. And you're going to see new materials for memory. A lot of people looking at things like ferroelectrics, non-capacitive, you know, high-density memories, finding memories that can, you know, be high-performance, high-density, and stackable. But people are looking at low-cost flash. For that, I'm not particularly enthusiastic about that idea myself. I think speed and performance of the underlying cell are somewhat problematic with a flash or an MRAM structure. But I do think memory innovations are coming for the first time in 30 years. And I do think that's going to be an exciting space. And we'll have a couple of companies there, I'm sure. Hopefully, we'll do a few of them together. Yeah, hopefully. should we do? Okay, you know, but we'll find some things to do here. But I think memory innovation for the first time in 30 years is nigh upon us. Yeah. Yeah, it's amazing what a couple of $10 billion of market cap is doing 100 billion. A trillion? A trillion, okay, sorry. You know, I think the memory industry has now increased by market cap by $2.5 trillion over the last four years. I think that's about right. We could have just invested there and not done any of our financial stuff. Yeah, exactly. How long? What's your view? Sorry, go ahead, Mia. To expand a little bit on the stacking part, If you have to predict, how quickly will our chips get taller? Is this going to be like, you know, we're going to go to two and then sit there for a while? Or will we have, you know, the 16-layer sandwich, you know, happily mixing logic, memory, and SRAM? HPM pulled out this. HPM is going down at Moffat. But how do you see this evolve? Yeah, you know, I think there's... You know, the problem was stacking. is now essentially the value of the stack becomes greater as you go to higher dimensionality of the stack, which means the yield of any individual layer in the stacking process has to keep coming. Essentially, it can't go up linearly. It has to go up exponentially. And improving the manufacturing and defect characteristics in an exponential way, on that associated with it. You know, your manufacturing flows have to be so close to perfect. Although you could see designs that take into account a certain model class, like we've seen it for a long time for two-dimensional. Yeah, of course. But remember, that was the nature of DRAM. You always have bad block replacement and spare lines and so on like that. So not in any way, right, you know, minimizing that. But assuming you're doing that, you still need the yields to be so good. You know, if I have a cracked die, I don't care how much resilience you've built into it. It's a bad die. So as a result of that, I'm not a crazy 16, 32 stack kind of guy. I think there's going to be nice three, four, five stacks that just end up sort of being the sweet spot. most things land. But even there, right, you know, so let's say I now have, you know, I now have a great compute tile. How big does that tile need to be? I mean, everything's become reticle limited now. Right, you know, can I go to more chiplets that better yield characteristics, better thermal? You know, and then you're going to say, I want a memory stack on that. You know, is it a two stack memory, a four stack memory? You know, I don't see it becoming an eight stack memory. You know, so to me, two or four sort of right. You know, but then power delivery needs to get integrated in.
[30:00] to that. So that becomes sort of another element. And I want that in the Z dimension, right, associated with it because, you know, I want my power rails coming into this as well, right? And then I need, you know, fairly intelligent RDL layers to distribute signals across that platform. So to some degree, you know, my canonical thing is already about eight layers thick when you add up all of those pieces associated with it. And I think that's about all you can do, right, at least Of course, we expect optics to come in, so you're going to have more 3-5 material structures that are going to be closer to this complex because I want my I.O. getting essentially deeply integrated into that construct as well. So my silicon-centric pieces need to get complemented by some of these other 3-5 materials. How do I get optical connectivity built into this? How do I get power rails built into that? engineering feet and a manufacturing nightmare. to bring all those pieces. But that's where the physics is taking us over time. So because of that, throwing a 16 high stack, it's like, that's crazy. A two or four high memory stack, I think that's probably where the sweet spot will be. You didn't talk about the thermal dissipation and new materials needed for that. Yeah, yeah. And obviously, there's going to be a lot of, you know, first thing, right, you just can't have some of the, you know, the physics hotspots that you have today. So, you have to be more intelligent about how you use the heat and spread the heat. But, you know, then you're going to be saying, okay, how do I both get power in and dissipate power out of those environments? my voltage regulation, then I don't create as much thermal chimneys. But I do have to think, okay, how do I cool these things? And what role do new materials, diamonds, and other things have been postulated as ways to do a better job of the thermal cooling? How do I then have better liquid cooling techniques and others sitting right on it with good thermal characteristics? We engineers are becoming plumbers. Yeah, you got to worry about MEC, the chip scale. Let me ask you about an operative phenomenon, which is some people have said, hey, let's get the memory off of the package and just go all optical between the compute and the memory, and that'll give you more degrees of freedom and more ways of building things. What's your view on that as opposed to stacking? That's other extreme. Yeah, you know, I think the more OEO, interfaces, optical to electrical to optical. The more of those that you demand in your core compute complex, it is just hard. Right. In that sense. And you're always going to have losses associated with it. You know, people are, you know, working hard to come up with more clever techniques to decrease, you know, both the cost and the power. Right. You know, how much of the power am I wasting as I go through my, you know, my DAX and, you know, conversions. So I'm not a fan of those because I think it gives you access to large pools of memory. Right. But I burn a lot of power getting there. essentially, you know, the femtojoules per compute It's now actually... pretty amazingly good, right? The femtojoules per bit of communication is comparatively a thousand times worse, right? And if I'm spending that much to go get to large pools of... So I don't like those approaches. You know, myself, I think I somewhat defy the underlying physics of the solution. And then there's sort of the... Other approach to say, like, look, can we actually integrate compute and memory directly, right? Shift some of the compute into the memory. Some of the PIM approaches. Yeah, exactly. Right. So what... What are your thoughts about that? I'm not a fan of those either, right? Because I think workloads, you know, because now I have to take my workload and I have to, like, push these little pieces of compute into the memory, right? You know, so that, you know, I don't need as much bandwidth of memory coming into my real compute, right? And that puts constraints on workloads. And again, you know, I could be wrong on this, particularly some of the design flexibility and the intelligence that we can now bring, you know, through the design aspects. around for 25, 30 years. And I think 25, 30 years from now, they'll still be hanging around. Right. In that sense, you know, give me great memory bandwidth, you know, that I can have, you know, high density, high performance and get it close to the real compute engines. I expect that's going to be a more generalizable solution across many and evolving workloads over time. So we're still stuck with denser and denser chips, more compute, more log.
[35:00] More power. But much better IO. Much better IO. Now, I am a huge fan, right, of moving to optics for all IO function. Yeah. Right? You know, I declared the death of copper about 25 years ago. Eventually, I'll be right. You're still right. One day. Yeah. One day soon, I'll be right. But I think is that, you know, to me, all IO should go to optical, right? You know, but that core compute memory complex, no. Right? So, Wendy, the first step for going on optical is to scale up, even more so than scale out. Yeah, yeah. What's your views on that? Yeah, you know, and I think we're now seeing... What would be the application of the workload driver, if you will? Well, I think the workload driver is the one in front of us. Okay. Right, I mean, you know, today, you know, our scale-up environments, we're putting tons of copper, right? You know, and essentially, copper is becoming a wave guide that's getting shorter and shorter, right? And all of a sudden, to make my copper work at five meters is more expensive than making optical work at 100 meters. Yeah. So I think the physics is driving us there today And so if that the case Pat why haven we moved there already It a complex supply chain that has never been tested at scale And I think now, I think everybody is looking at some form of NPO, CPO, et cetera, in the 28, 29 timeframe because I can't scale up to large radix, compute clusters from a scale-up environment without making that move. So, you know, clearly NVIDIA has indicated that. I think everybody else in the industry expects, you know, that's the conversion point. You know, in reality, I should have never built an NBL 72. You know, it's an engineering marvel and it's a manufacturing nightmare, right? You know, it took the industry, you know, one extra year and a half to digest that beast, right? And it really is amazing that, you know, we're able to get it to scale and so on. But, you know, that really proved that had we had a more mature optical supply chain, we'd be much more advanced today. We'd be able to build better scale up environments today, you know, better power, better cost, better energy. But we didn't have the maturity of the optical supply chains to allow that. And those things are not trivial to get resolved There a lot of work to happen but I think 28 29 is the year Fab capacity We probably gaining a lot of experience how in optics work in networking right And I think many of that will translate nicely to GPUs. But all of a sudden, you know, it's like, okay, these things have to work, you know, at 10 million GPU scale. Wow, right? You know, and it has to fit in packages. And, you know, there's just a lot of things there, you know, and many forms of optical don't like bad thermals. Well, guess what? Bad thermals is the nature of the game. So we have thermal problems to deal with, package integration problems. We didn't have enough laser capacity in the industry. So a bunch of things. But I don't see anything there fundamental. It's just a lot of hard work at this point to pull all of those pieces together. In terms of the networked architecture, will this also mean a switch? What's more circuit switch networks? Just because you can't scale these very fast optical networks indefinitely. Well, you know, I like Nick McCune, another Stanford professor here, a good friend of mine as well. I mean, you know him a little bit also, Guido. Was he your thesis advisor? He was my thesis advisor. Yes But you know to me he describes it well You know we built you know networks to be able to handle any packet going anywhere with no knowledge of where it might go When you think about AI, you know, it's almost exactly the opposite, right? You know, we're doing flows that are large and predictable. So in that sense, just the very description says, hmm, that looks more like a switch than like a packet. So I do think this confluence of moving to optical and having a workload that is far more predictable, manageable, large in flow, to me, I do see that, yeah, we will go all the way to bringing optical into the compute complex. And we will go all the way to OCS or some form of optical switching with more predictable flows. You know, I mean, that is the right, you know, architecture. And then you sort of say, is that scale up or scale out? And to some degree, if my scale up is big enough, then yeah, you know, just saying, okay, what's the radix of my scale up environment? And when do I want to hook it up to other clusters that are really big? And to me, that ends up being a pretty beautiful answer, you know, to the future. But I'm looking at somebody who's way more expert on this than myself, so.
[40:00] No, look, I always found scale-up was a scale-up to some degree in artificial distinction. You understand where it comes from, right? We're kind of using different protocols. One is front-end, one is back-end network, and so on. So right now, practically speaking, it is a big difference. If you take a step back and look at it from a system architecture perspective, it achieves the same goal with slightly different flavors, slightly different protocols that over time I think will converge. Yeah. I mean, especially for the AI training workload, Yeah, although, I mean, training and inference this time should look fairly similar. The inference corpus lesson. Yeah, now I'll go on to, you know, right, does your brain stop learning? Right, you know, by the way, as another characteristic of workload evolution, I do think the separation of training and inferencing is going to become less so going forward, not more so. That's a little bit of the argument against more and more heterogeneous architectures is, I think, continuous learning. I think there'll be algorithmic domains that will say, yes, continuous learning becomes the nature of the algorithm where, yeah, I do want my inference environment to be updating my model weights because that giving me more and more refinement evolution across workloads moving into environments that are less formalized training as well Well I have smaller data sets in those environments So, I'm not sure that's a... But the number of nodes you need in order to be useful is a smaller diameter for inference. That's the training. That's what it means. Today, that's true. I still think there might be a role for heterogeneity coming from a different angle, which is that different types of AI workloads require very different underlying code. Like some are more compute limited, some are more memory bandwidth. They have a small diffusion workload or so, right? The optimal card for that looks very different from, you know, top of the line, you know, $15 trillion, sorry, $15 trillion weight LLM, right? That means a great massive scale. So I think that might drive heterogeneity, if anything. Yeah, I think there's some room for that. But I'd also say, okay, now I'm talking about putting a million GPUs to work in my data center. And you say, well, for that particular diffusion workload, I should really put something optimized this way. So I'm now going to start putting 1,000 of those in. And for that workload I need 2 of these in And all of a sudden you don get some of the flexibility of scale that you would say and how much does it cost me to manage and upgrade and connect and fault domain all those types of things The agents will do all the upgrades, so that's no longer a problem. Yeah, thank you. Everything gets easy, so good, Alan. Yeah, so fleet management is done. so we talked about what compute, memory, networking power is the next one and you touched upon it with vertical delivery inside the chip on the chip but I mean there's 800 volt DC there's power delivery to the data center there's an entire ecosystem all the way to power generation and the sociopolitical dimensions of that and so on and so forth. Yeah, yeah. So where does innovation come in and where does it's pure execution? Well, if we start at first principles here, our nation has done a terrible job with its energy capacity, right? Essentially we went through 10 to 15 years where essentially I was taking coal offline at the rate I was adding renewables And essentially the nation was flatlined in terms of overall energy capacity Not good. Because in an AI digital age, energy capacity equals economic capacity. So essentially my economic capacity as a nation, flat for 15 years if you buy that thesis, which I think is very provable. Over the last five years, we've seen this enormous influx into just giving more capacity. But I think even with that, we've gone to maybe increasing our national energy capacity 4% per year. Something like that. Going from essentially zero to one. Wow, 4x. Or 4%. So to me, it's a bad situation. So we just need more energy capacity for it. So, number one, and obviously we have some companies in that space. We talked about nuclear operating as one of those with Alva. And I think nuclear, right, as a baseload is a fabulous thing that we want to go build more of. Unfortunately, all of our renewables have deep dependencies in China, which is very unfortunate.
[45:00] You know, for us, you know, we're, you know, you know, gas turbine lead times are only eight years at this point. Right. So, you know, you know, I mean, we really have a difficult environment to scale up rapidly. But one, we just need more capacity. And fundamentally, energy capacity somewhat dampens how euphoric we can get on AI. Because why build a new data center and buy the million GPUs if I can't power them? And I think you're going to see more and more defaults happening on many of those data center projects because the energy will be there. So you think it'll be a significant headwind? I think it will be a headwind. And I think you're already starting to see some of the first indicators of that, the Oracle, you know, I think that's the first of what you're going to see many. Now, it's not like we're going to slow down as a result, but I think people will start saying, do I really have the energy, you know, to underwrite the capital commitments that I'm making, you know, GPU purchases. So do you think it's going to be an upper bound? But we need more energy capacity and we need to get more innovation into that. And we need more of those supply chains in the U.S. Nuclear, one of my favorites. When was the last nuclear reactor that came online in the U.S.? Right. You know, it's 20 years ago. Yeah. We just stopped a very capable industry. So that's when we need more. We need to get much more efficient at delivering. the power networks. As you said, moving to 800 volt DC data centers, absolutely. And being able to eliminate so many conversion steps along the way because of historical reasons and standardization reasons. Okay, we got to fix this. In 800 volt DC, the revenge of Edison is upon us. Yes, I was just going to say. We have a couple of companies, one of those vertical GAN, I think will be a killer technology to be able to do 800 to 48 or 800 to 12 or even 800 to 5 in one conversion step, which just gives you a lot more energy. Solid state transformers? again, right? And then you're going to need all of those in the cooling systems as well, right? And so there'll be a bunch of innovations in that. So turbines and refrigerants and all of a sudden they get to be exciting new technologies again. So for all of us hardware guys, this is like renaissance. After 25 years, I had to go back and read about Cardinal's Law. Cardinal efficiencies. Oh, yeah, this is like good stuff. First of all, thermodynamics work again. It's something after all. So it's going to be an exciting time that way. But we also say we need to change the physics. Because CMOS hasn't improved the essential power per tera flop for the last five generations of NVIDIA chips, flatline. Not okay. And that's where some of the things, we have a superconducting company, Snowcat, which affords the opportunity to fundamentally change the physics, a thousand times better power performance. So I do think some of those innovations are upon us. in the near future as well. I've been a terrible time manager here for the past hour. So, so many topics yet to cover, we didn't cover, but we've got to cover at least one before we leave. Okay. Because now... You're in charge. Yes. So I thought... Now VMs are back. How do you think about that? The former CEO of VMware? Well, you know, I... I feel excluded from this conversation. No, no, no. Former CEO of VMware. We'll go on. One of it was I was going to talk about networking as well. But I don't think we want to tell you. But, you know, hey, I think the, you know, every innovation... has always led to the next abstraction. And I think many of these ideas, I would like to say, what's new in our world today? Data structures, algorithms, and abstraction. We bring it back to those foundationals. And hey, I think the virtual machine abstraction, whether it's done from the infrastructure level or from the application level, I think it's still a foundational abstraction model that deserves to always have a place in the compute hierarchy. us go back and run VMware? Yeah, that's what I tell. I think I told Guto's how it's time to go back and run VMware again. Yeah, yeah. And, you know,
[50:00] do you think some of these abstractions, you know, but you also think about it and like one of the things we did at VMware, we, you know, essentially, you know, managed every aspect of the computing hierarchy, right? You know, and you manage the workload, you manage the network, you manage the storage, you know, system and how do you abstract those? And I think in the AI context, right, we think about, you know, these agent swarms, well, who's going to manage the agents? Who's going to create the security profiles around all of the agents? Who's going to manage the performance of the agents. Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy. And there's going to be lots of wonderful companies that you'll sort of break through for how those things will be done. Okay. Do you want to end it on networking? Actually, I'd love to stay with the VMs for a second. The most interesting thing for me is that we're now building VMs so that they're usable for agents as opposed to usable for humans And what we seen in some of other companies is that changes a lot the form factor that changes a lot how you market them right You want the agent to make the pick for your VM offering It changes a lot how much complexity you can have It changes how quickly the startup should be. Humans are a lot more patient than agents. So it's, if you have to rebuild VMware today, how would it look different? Yeah, and I do think. Like VMware for agents, right? Yeah, and I do think there's, does there need to be a VMware for agents for humans? I see what you mean, yes. In the sense that I need humans. Let's stick to the agents. They're way more interesting than they are right now. But I think it's an important point because I do think that there needs to be, and I'll use Dario's constitution or guardrails, or being able to set policies. And I think then you really say, okay, Every aspect of the virtual machine, as we know, we'll love it today. It is servicing agents right as opposed to servicing hardware and humans So I think that becomes your fundamental design constraint on this side do I make agents great Right How can I make them secure How can I make them performant You know you know you know how do I abstract them How can I migrate them All of those things you know have the new embodiments you know V motion for agents It's like every one of these will have the new embodiment here for them. But then I also need to think about how can humans be setting the policies, describing the constitutions, getting the dashboards, et cetera. And I think that's sort of where the interesting thing is because you need to do both of those, right? Not just one of those, but the fundamental, I'll say the hard design stuff like we did for virtual machines was associate abstracting the hardware in a performant way for applications or operating systems. here. It's abstracting the hardware and the operations for agents. It's a fundamental thing that you're serving. It was great discussing this. Thank you so much, Pat, for coming in, and we'll do it again. Well, two of my favorite people, anytime. All right That was awesome Thank you Thank you Thanks for listening to this episode of the A16Z podcast If you liked this episode be sure to like comment subscribe leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures. Thank you.
