Gelsinger Warns Power Deficits Will Trigger AI Data Center Defaults
a16z
Former Intel Chief Executive Officer Pat Gelsinger argues that physical electricity shortages will force debt defaults on speculative data center projects while nine-month manufacturing cycles crush overfunded chip startups into a handful of winners.
Energy capacity has become the hard ceiling on AI economic expansion, with eight-year lead times on gas turbines and stagnant grid capacity threatening accelerated data center project defaults. (42:47)
AI-accelerated chip design creates an operational mismatch: while digital logic can be designed in three months, physical fabrication, 3D packaging, and rack integration still require a nine-month turnaround. (9:28)
The market of roughly 100 specialized AI inference chip startups faces inevitable consolidation down to a few winners due to rapid model evolution, massive CapEx barriers, and software ecosystem lock-in. (13:39)
External scale-up cluster fabrics are projected to transition from constrained copper waveguides to co-packaged optics (CPO) and near-package optics (NPO) around the 2028–2029 window. (34:46)
Detailed notes
## Background
This video was published on YouTube by the a16z channel on October 9, 2026. According to the show description, hosts Raghu Raghuram and Guido Appenzeller sit down with former Intel CEO and current Playground Global General Partner Pat Gelsinger to explore the physical bottlenecks facing the artificial intelligence buildout, spanning chip design, manufacturing lead times, memory architectures, optical networking, power constraints, and agentic infrastructure.
## TL;DR
Pat Gelsinger unpacks why software-level acceleration and AI-assisted electronic design automation cannot bypass hard physical constraints in semiconductor manufacturing, where fabrication and packaging still impose nine-month delivery cycles. He explains why the current wave of over 100 specialized AI chip startups will inevitably consolidate into a handful of architectures, critiques the severe compromises of High Bandwidth Memory, and details the manufacturing ceiling of 3D vertical stacking. The conversation also analyzes the urgent transition from copper interconnects to co-packaged optics, frames energy availability as the ultimate economic ceiling on data center expansion, and outlines the need for a reimagined virtualization hypervisor built explicitly for autonomous agent workloads.
## Atmosphere & style
The discussion carries the relaxed, collegial warmth of seasoned Silicon Valley veterans who share deep historical ties across Intel and VMware, trading inside jokes alongside rigorous engineering debates. Gelsinger speaks with frank, punchy conviction—pulling no punches when calling out 'hideous' memory compromises or the unworkable economics of over-stacked silicon—while the hosts actively push back with counter-theses around automated software generation. The tone balances technical depth with grounded realism, cutting through venture capital euphoria by repeatedly anchoring the grand ambitions of artificial intelligence to the uncompromising laws of thermodynamics, fabrication timelines, and electrical grids.
## Bulletin
- [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) Energy capacity has become the hard ceiling on AI economic expansion, with eight-year lead times on gas turbines and stagnant grid capacity threatening accelerated data center project defaults.
- [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) AI-accelerated chip design creates an operational mismatch: while digital logic can be designed in three months, physical fabrication, 3D packaging, and rack integration still require a nine-month turnaround.
- [13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) The market of roughly 100 specialized AI inference chip startups faces inevitable consolidation down to a few winners due to rapid model evolution, massive CapEx barriers, and software ecosystem lock-in.
- [34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) External scale-up cluster fabrics are projected to transition from constrained copper waveguides to co-packaged optics (CPO) and near-package optics (NPO) around the 2028–2029 window.
## Key takeaways
- [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) In the AI era, national energy capacity equates directly to economic capacity, turning the 10-to-15-year stagnation of the US electrical grid into a primary economic bottleneck.
- [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) Accelerated AI front-end chip design shifts the core bottleneck downstream to physical fab processing, complex 3D packaging, and rack integration, which still take approximately nine months.
- [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) The rapid cadence of model architecture shifts risks obsoleting hyper-specialized AI silicon before physical manufacturing and deployment cycles can reach scale.
- [11:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=716s) High Bandwidth Memory (HBM) remains a deeply flawed compromise due to shoreline bandwidth limits, thermal vulnerability near compute logic, and poor bit density.
- [11:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=716s) Chip designers routinely lose roughly 40% of their power envelope to excessive guard-banding due to primitive 3D power dissipation and thermal hotspot modeling.
- [13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) The proliferation of around 100 AI inference chip designs will inevitably consolidate to a few platforms because hyperscalers and frontier model builders demand unified software ecosystems and massive capital scale.
- [18:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1136s) Automated AI code generation lowers software porting friction, but multi-billion-dollar data center commitments and deployment lead times prevent long-term hardware fragmentation.
- [22:08](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1328s) Novel memory architectures experienced a complete 30-year commercial drought because boom-and-bust commodity cycles disincentivized long-term physics R&D, a dynamic now reversed by generative AI.
- [28:06](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1686s) Vertical 3D chip stacking will likely settle on modest 'mid-rises' of three to five layers due to compounding assembly yield losses, silicon cracking risks, and thermal dissipation limits.
- [32:14](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1934s) Disaggregated all-optical memory pools and processing-in-memory (PIM) are unviable as general architectures because data movement costs roughly 1,000 times more energy per bit than local computation.
- [34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) Scale-up interconnect fabrics will inevitably abandon copper for co-packaged optics (CPO) by 2028–2029 as rising signal frequencies turn copper lines into impractical, short-range waveguides.
- [37:26](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2246s) AI networking traffic consists of massive, deterministic, scheduled flows that naturally favor optical circuit switching (OCS) over traditional packet inspection and switching.
- [45:27](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2727s) Data center efficiency will shift to 800V DC power architectures and single-step Vertical GaN down-conversion to bypass redundant alternating current transformation stages.
- [46:50](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2810s) Because GPU power efficiency per teraflop has plateaued across five generations, foundational computing leaps require non-CMOS physics such as superconducting logic.
- [48:42](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2922s) The rise of autonomous agent swarms necessitates a modern 'VMware for agents' that provides rapid sub-second provisioning, state migration, and constitutional governance frameworks.
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=0s) Teaser: The Physical Bottlenecks of the AI Buildout
The conversation opens with a series of provocations regarding the hard physical limits confronting modern artificial intelligence infrastructure. In an AI digital era, "energy capacity equals economic capacity." The premise of spending billions on data centers and acquiring a million GPUs collapses entirely if there is insufficient power to run them, signaling that defaults on massive data center projects will soon accelerate simply because the electricity is not available.
Technological progress historically relocates constraints: making one layer easy pushes the bottleneck somewhere else. Modern computing has fundamentally moved past single dies—"nothing's a chip anymore, it's a rack." Even if an architecture can be designed in three months, physical fabrication still locks teams out for nine months before silicon is operational. Meanwhile, core memory technology has essentially seen zero foundational breakthroughs over the last three decades, though massive shifts in advanced packaging, optical interconnects (finally realizing the decades-overdue "death of copper"), and specialized AI inference accelerators are sparking a hardware renaissance. Yet despite hundreds of competing AI chip startups being funded, history dictates that the semiconductor industry cannot sustain a hundred processor vendors; the landscape will inevitably consolidate back to a small handful.
[01:02](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=62s) Joining Intel at 18: From Trade School to Stanford
Raghu Raghuram introduces guest Pat Gelsinger—former Intel CEO and CTO, former VMware CEO, and currently General Partner at Playground Global—noting how familiar it feels to sit down together alongside Guido Appenzeller. The discussion starts with Gelsinger's unorthodox entry into the semiconductor industry, contrasting it with modern talent pipelines like the Andreessen Horowitz (a16z) fellowship programs that scout talent between ages 16 and 22.
```mermaid
flowchart LR
accTitle: Pat Gelsinger Early Career Progression
accDescr: Pipeline showing Pat Gelsinger's journey from technical school to Intel design manager.
subgraph Early["Early Education"]
A["Lincoln Tech<br/>(Associate Degree, age 16-18)"]
end
subgraph Intel["Intel Engineering Ladder"]
B["Technician (age 18)"] --> C["80286 Test & Fixes"]
C --> D["80386 Engineer #4"]
D --> E["80486 Architect & Design Manager"]
end
subgraph Academia["Concurrent Studies"]
F["Santa Clara Univ. (BS)"] --> G["Stanford Univ. (MS)"]
G --> H["Doctoral Work with John Hennessy"]
end
A -->|"Recruited by Ron Smith"| B
B -.-> F
D -.-> G
```
Gelsinger recounts his background as a 16-year-old in rural Pennsylvania who won a scholarship to Lincoln Tech, skipped the final year and a half of high school, and was scouted by Intel at age 18. Having never flown on an airplane before, he was the twelfth candidate interviewed that day by Intel hiring manager Ron Smith. As Gelsinger recalls, after interviewing a dozen candidates in a single day, an interviewer can barely distinguish people, but Smith wrote down: *"Smart, aggressive, arrogant. He'll fit right in."*
Gelsinger joined Intel as a technician, transitioned to the tail end of the 80286 processor team, became engineer number four on the 80386 (386), and went on to serve as architect and design manager for the 80486 (486)—all while simultaneously completing his bachelor’s degree, master’s degree, and PhD research:
- **Hands-on friction with theory:** Taking a Stanford course where the professor proposed a new carry-lookahead adder design, Gelsinger implemented it directly on the 386 team and returned two days later to announce, "It doesn't work," sparking two entire class sessions of debate with a professor rushing to publish a book containing the flawed circuit.
- **Built-in self-test (BIST):** Collaborating and arguing vigorously with Stanford professor Edward J. McCluskey (the pioneer of digital logic testing) over which BIST concepts were completely impractical for commercial silicon versus those Gelsinger could actually deploy on the 386.
- **Academic mentorship:** Working on doctoral research under future Stanford president John Hennessy, with Gelsinger joking that he helped Hennessy's career along before Hennessy went on to do "okay since then."
Guido Appenzeller mirrors the observation from his own time lecturing at Stanford: practicing engineers often shatter academic theories in the classroom, forcing professors to verify and update their syllabi. Silicon Valley's unique advantage lies in this porous boundary between academia and industry.
[05:44](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=344s) The 80486 and the Birth of Modern EDA
The development of the Intel 80486 represented a watershed moment because it predated the commercial Electronic Design Automation (EDA) ecosystem. To deliver a pipelined, backward-compatible microprocessor with roughly 1.2 million transistors, the engineering team had to build the software infrastructure from scratch:
- **Hardware description languages:** Because Verilog had not yet emerged as a universal standard, Intel created its own proprietary HDL (Intel Hardware Description Language) alongside its internal compiler.
- **Automated place and route:** Without commercial automated layout tooling, Gelsinger and Intel partnered with Professor Alberto Sangiovanni-Vincentelli and his graduate students at UC Berkeley to invent early automated placement, routing, and static timing analysis tools.
- **Foundational legacy:** These proprietary internal frameworks established the core principles that subsequently birthed the commercial EDA industry.
[07:23](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=443s) AI in Chip Design: Promises, Analog Limits, and New Bottlenecks
Transitioning to current architectures, the hosts ask whether Nvidia's Blackwell will be remembered as the last major processor family designed before AI took over chip creation. Gelsinger points to modern initiatives—such as Jalapeno—as examples of using AI from first principles to rewrite traditional design workflows, although clear physical boundaries remain:
- **The logic versus analog divide:** Pure digital logic synthesis, kernel generation, and RTL generation from high-level specifications map exceptionally well to generative models and automated optimization tools.
- **The analog hurdle:** High-speed analog components—most notably SerDes (serializer/deserializer) interfaces—remain fundamentally resistant to pure AI automation. They demand massive amounts of empirical silicon measurement and physical validation. To apply AI to analog circuits today, designers must adopt hyper-conservative design rules, or otherwise execute the layout the hard, traditional way.
- **Shifting the bottleneck:** When generative AI abstracts away software layers, low-level RTL generation, and standard logic synthesis, the overall design cycle does not magically shrink to zero. With the digital frontend accelerated, the bottleneck immediately moves outward into downstream physical implementation and manufacturing timelines.
[09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) The Physical Bottleneck: Why Faster Design Can't Fix a Nine-Month Silicon Cycle
Even if AI-assisted electronic design automation (EDA) tools allow engineers to complete a sophisticated chip architecture in just three months—optimizing multiply-accumulate units, register structures, and conservative analog components—the physical reality of semiconductor manufacturing remains stubbornly slow. Pat Gelsinger points out that after design tape-out, a chip still faces roughly nine months of silicon processing time before working hardware arrives at scale. Even under the best conditions with supercharged design flows, getting a raw wafer out of the fabrication plant takes at least three months.
```mermaid
flowchart LR
accTitle: Semiconductor Turnaround Timeline
accDescr: Progression from chip design to deployed rack showing a nine-month physical manufacturing gap.
subgraph Design["Design Phase"]
A["AI-Assisted EDA<br>(~3 Months)"]
end
subgraph Bottleneck["Physical Manufacturing Pipeline (~9 Months)"]
B["Fab Processing<br>(~3 Months)"]
C["Advanced 3D Packaging<br>(~3 Months)"]
D["Rack-Scale Integration<br>(~3 Months)"]
end
A -->|"Tape-out"| B
B --> C
C --> D
```
From there, advanced packaging adds another major delay. Modern designs rely heavily on complex 3D packaging, which easily absorbs another three months. Finally, because single monolithic chips are virtually obsolete in high-end compute, the device must be integrated into a full rack-scale solution—adding another three months before anyone can write software against it. This creates a severe mismatch: if a team spends three months designing a chip but requires nine months to manufacture and deploy it, the total turnaround spans a year or more. By the time production chips reach customer data centers, the fast-moving AI workloads the silicon was customized for have often shifted entirely. Gelsinger cites Graphcore as a cautionary historical example: it was not a poor hardware design, but the algorithmic state of the art simply moved on before the architecture could establish itself at scale.
To unlock innovation, Gelsinger argues that the industry must compress physical prototyping down to one or two months. Achieving that speed will demand radical breakthroughs:
- Novel lithography techniques capable of rapid turnaround.
- Prototyping design flows that eliminate upfront mask costs running as high as $50 million.
- Better alignment between rapid software evolution and slow hardware realization cycles.
---
[11:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=716s) The Core Physical Hurdles: "Hideous" HBM and Primitive Power Modeling
Beyond the manufacturing calendar, Gelsinger identifies memory bandwidth and thermal power delivery as the two critical physical walls confronting modern AI compute. Addressing memory, Gelsinger pulls no punches: "HBM is a hideous memory. It's just the best one that we [have]." Because AI workloads are fundamentally memory-bandwidth bound, systems are forced to rely on High Bandwidth Memory despite its crippling engineering compromises:
- Poor bit density relative to the physical silicon area it consumes.
- Severe shoreline bandwidth constraints around the chip perimeter.
- Vulnerability to thermal degradation, as DRAM inherently degrades when exposed to heat, yet advanced packaging concentrates extreme heat right alongside the compute logic.
Power simulation and delivery represent an equally severe bottleneck. Current 3D modeling tools for power dissipation, voltage drop, and localized thermal hotspots remain rudimentary. Because designers lack precision simulation tools, they are forced to apply massive safety margins across the chip's operational envelope:
- Designers routinely lose approximately 40% of their total power budget to excessive guard-banding just to guarantee system stability under unknown localized thermal spikes.
- Power delivery mechanisms and cooling architectures must fundamentally change to handle rack-scale power densities.
- Gelsinger underscores that until these thermal, power, and memory bottlenecks are tackled, AI-driven EDA improvements will hit diminishing physical returns.
---
[13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) Will a Hundred AI Accelerators Converge to a Few?
Raghu Raghuram and Guido Appenzeller turn the discussion to the explosion of custom silicon startups, noting that the market has seen a resurgence with roughly 100 different AI inference accelerator designs funded across venture capital and big tech. When Gelsinger jokingly asks the hosts why venture firms keep funding so many of them, Raghuram counters that Gelsinger funded his fair share as well. Appenzeller points out that historically, no computing market has ever sustained 100 competing processor architectures.
Gelsinger agrees that the current proliferation of heterogeneous AI accelerators is a temporary bubble that will inevitably consolidate down to a small handful of dominant platforms. He outlines three core structural reasons why massive fragmentation cannot last:
1. **Extreme specialization is inherently fragile.** Data center compute fleets are fracturing into specialized silicon tiers for prefill, midfill (differential updates), decode, and speculative verification. However, hyper-specialized hardware rarely survives over the long term because model architectures evolve faster than silicon lead times. For example, as the industry pivots toward complex reasoning models, workloads demand capabilities that look much more like traditional general-purpose CPUs rather than narrow matrix multipliers.
2. **Algorithmic shifts toward scientific modeling and high precision.** Current flat Large Language Models are reaching scaling limits when applied to complex 3D physical structures like molecular biology, chemical synthesis, and multi-dimensional imaging. Gelsinger expects these domains to blend AI with traditional High-Performance Computing (HPC). In these hybrid workflows, an AI system might sift through candidate spaces to isolate the five most promising chemical structures, but the subsequent validation requires traditional, high-precision 64-bit floating-point math running on classic scientific architectures.
3. **The unforgiving economics of software ecosystems and scale.** Hardware startups cannot survive on micro-architectural cleverness alone—whether using digital, analog, optical, or probabilistic techniques. To sustain continuous silicon generations, a platform must capture immense customer scale, secure billions in capital, and build deep software co-design partnerships. Gelsinger emphasizes that influential foundation model builders and tech giants—such as OpenAI, NVIDIA, and Anthropic—will ultimately pick a very small set of hardware winners whose co-evolved software stacks best support their operational roadmaps.
[18:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1136s) Software Abstraction, Capital Scale, and the Limits of Hardware Heterogeneity
The discussion continues on whether the explosion of specialized AI chip startups will inevitably consolidate into just a few winning architectures. Large hyperscalers and cloud providers ultimately pick winners; achieving first-tier status as a chip startup is almost entirely defined by whether a company wins traction with the tier-one tech giants. Guido Appenzeller pushes back slightly to stir controversy, offering a counter-thesis: historically, the industry standardized on only one or two instruction set architectures (such as x86 and Arm) because software complexity was prohibitive. Compilers were brutally hard to write, and supporting multiple platforms imposed massive engineering friction. Today, however, AI agents and automated swarms can generate highly optimized hardware kernels overnight, eliminating the human programming bottleneck. If software layers can now adapt automatically to unconventional architectures, Guido asks, couldn't the industry sustain far more hardware heterogeneity than in the past?
```mermaid
flowchart TD
accTitle: The Limits of AI Hardware Heterogeneity
accDescr: Diagram showing how agentic code generation lowers software friction while massive CapEx and deployment lead times enforce hardware consolidation.
subgraph SoftwareEase ["Software Layer (Friction Decreasing)"]
A["Agent Swarms & AI Tooling"] -->|"Generate Kernels Overnight"| B["Novel Chip Architectures"]
end
subgraph PhysicalBottlenecks ["Physical & Economic Bottlenecks"]
B --> C["18-Month Silicon & Rack Fabrication Cycle"]
C --> D["$50B+ Capital Expenditures"]
D --> E["Power & Gigawatt Data Center Commitments"]
E --> F["Consolidation & Monolithic Abstraction"]
end
```
Pat Gelsinger acknowledges that automated kernel generation shifts where the bottleneck lands, but insists that the ultimate constraints remain uncompromisingly physical and financial. First, fabricating custom silicon, packaging it, and qualifying it into production racks still takes at least a year and a half. Second, deploying hardware at meaningful modern scale requires staggering capital commitments:
> "Okay. Well, could I borrow 50 billion, right? I mean, you're not talking about small capital expenditures. You're talking about very large capital expenditures and data centers, power commitments and so on like that. Well, I got to build this thing at scale... and that's going to defy a lot of heterogeneity."
Rather than hundreds of niche chips thriving in the data center, market incumbents will simply swallow and abstract whatever specialized advantages emerge. Pat points out that whether through acquisitions or architectural convergence—pointing to how the industry and players like Nvidia absorb specialized approaches like Groq's—the dominant platforms will hide internal architectural heterogeneity underneath standardized software abstractions. Layering will do what it has always done in computing history, preserving the illusion of uniformity while the massive capital barrier prevents fringe architectures from surviving independently.
[22:08](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1328s) Three Decades of Stagnation: Why HBM is "Hideous Memory" and Optane Died
The conversation pivots to High Bandwidth Memory (HBM), which Pat had earlier dubbed a "hideous memory." While acknowledging it as a deeply compromised engineering compromise, Pat concedes that it remains "by far the best one we have today"—drawing the analogy to Winston Churchill's quip about democracy being the worst form of government except for all the others. Raghu Raghuram observes that genuine breakthroughs in memory technology have been virtually nonexistent in recent memory, asking about Intel Optane (3D XPoint), in which Pat had a personal hand. Pat bluntly responds: "Yeah, that one died... We killed it."
Reflecting on the broader history of the semiconductor industry, Pat recounts being personally involved with at least five novel memory architectures that never succeeded in production, and observing nearly a hundred attempted memory innovations across the industry over the last 30 years. When asked how many major new commercial memory technologies have actually stuck over those three decades, the answer is stark: zero. The industry remains locked into the exact same trio that has dominated for generations:
- **DRAM** (Dynamic Random-Access Memory)
- **SRAM** (Static Random-Access Memory)
- **NAND Flash** (with vertical 3D NAND stacking providing only marginal architectural novelty)
The root cause of this 30-year innovation drought was economic rather than purely technical. For decades, memory was a notoriously brutal, commoditized business plagued by vicious boom-and-bust cycles. Memory makers would generate substantial profits for one year, followed by four consecutive years of bleeding cash. Nobody wanted to fund ambitious, capital-intensive physics R&D inside an industry that only made money twenty percent of the time.
[24:44](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1484s) The $2.5 Trillion Wake-Up Call and the Next Generation of Stacked Memory
That dismal economic reality has abruptly inverted. Over the last four to five years, the explosive rise of generative AI turned memory into the primary bottleneck of modern computing. Today, all three major memory manufacturers (Samsung, SK Hynix, and Micron) sit comfortably among the most valuable companies on Earth, collectively adding an estimated $2.5 trillion in market capitalization. With the industry recognizing that "AI is a memory workload," the massive influx of capital finally creates the economic justification required to crack long-standing physics challenges.
Pat outlines two major fronts where genuine memory innovation is poised to break through:
1. **Solving the Shoreline Bandwidth Bottleneck:** HBM's primary physical limitation is the 2D perimeter ("shoreline") of the silicon die, which severely caps how many parallel interconnect traces can link compute to memory. Companies like d-Matrix and Cerebras are attacking this constraint by bringing compute and memory physically together through tight vertical and architectural coupling, establishing that "the future is stacked."
2. **Novel Materials and Non-Capacitive Cells:** Researchers and startups are investigating fundamentally new memory physics, such as ferroelectric materials (FeRAM), to build high-density, high-performance, non-capacitive cells that can be stacked directly on logic without the volatility, refresh overhead, or latency penalties of legacy cells.
Pat expresses deep skepticism about using cheap Flash or MRAM for core AI compute, noting that the intrinsic speed and latency profiles of their underlying cells remain deeply problematic. However, he reveals that he has personally seeded a new stealth memory company at Playground Global, predicting that after 30 years of total stagnation, revolutionary memory innovations are finally within reach. Raghu and Pat joke that instead of venture investing, they should have simply parked their capital into pure-play public memory stocks four years ago and watched it multiply.
[27:19](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1639s) Going Vertical: The Compounding Yield Challenge of 3D Chip Stacking
Guido asks Pat to forecast the practical timeline and physical limits of vertical 3D stacking. Will the industry transition to a modest two-layer configuration and plateau there for years, or will packaging advance rapidly toward dense 16-layer sandwiches that seamlessly interleave compute logic and memory?
Pat explains that vertical stacking introduces a brutal economic penalty driven by compounding yields. In a high-dimensional 3D stack, the cumulative economic value of the assembly escalates with every added layer, because losing a completed stack at final assembly destroys all the underlying silicon. Consequently, stacking cannot scale linearly unless the defect density and individual yield of every single layer in the stack approach near-perfection:
- Every added layer exponentially increases the risk of discarding fully functional adjacent dies.
- Thermal dissipation and power delivery worsen dramatically as active logic layers are sandwiched beneath one another.
- The manufacturing tolerances required for wafer-to-wafer and die-to-wafer hybrid bonding become unforgiving when stacking heterogeneous silicon across dozens of layers.
[28:06](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1686s) The Sweet Spot for 3D Chip Stacking
Building tall vertical semiconductor structures requires defect characteristics and manufacturing yields to improve exponentially with each layer added. While conventional two-dimensional DRAM architectures have long incorporated fault-tolerance mechanisms such as bad block replacement and spare lines, physical stack assembly introduces failure modes that redundancy cannot cure: "If I have a cracked die, I don't care how much resilience you've built into it. You can't, you know, it's a bad die." Because yields must be near-perfect across an entire process flow, stacking eight or sixteen dies high compounds yield losses to unworkable levels. Rather than massive 16-high or 32-high stacks, realistic production will settle on "mid-rises"—three-, four-, or five-layer configurations that hit the economic and physical sweet spot.
```mermaid
flowchart TD
accTitle: Canonical 3D Chip Architecture Layers
accDescr: Diagram illustrating the layer breakdown of Pat Gelsinger's canonical eight-layer stacked semiconductor package.
subgraph Stack["Integrated 3D Tile Stack (~8 Layers Total)"]
direction TB
Optics["Optical IO & III-V Materials"]
Power["Power Rails & Voltage Delivery (Z-Dimension)"]
RDL["Intelligent Redistribution Layers (RDL)"]
Mem["2- to 4-Die High Memory Stack"]
Compute["Compute Tile / Chiplets (Reticle-Limited)"]
Cooling["Thermal Interface (Diamond / Liquid Cooling)"]
end
Power --> Compute
Mem --> Compute
RDL --> Compute
Optics --> RDL
Compute --> Cooling
```
As individual silicon dies hit reticle limits, systems are disaggregated into chiplets that yield better and manage thermals more effectively. On top of those compute tiles, memory will likely top out at two or four vertical layers rather than eight. Crucially, power delivery rails must also be integrated in the Z dimension alongside intelligent redistribution layers (RDL) to route signals. Adding up the compute tile, two to four memory dies, vertical power delivery, RDL routing, and integrated III-V optical materials creates a canonical assembly that is already around eight layers thick. Coordinating these dissimilar materials and tight physical tolerances represents an extraordinary engineering feat and an undeniable manufacturing challenge.
- **Stacking limits:** Sweet spot lands at 3 to 5 total layers; memory stacks will realistically be 2 to 4 dies high rather than 8, 16, or 32.
- **Canonical assembly:** Compute tile + 2–4 memory dies + Z-axis power delivery + RDL routing + integrated optical layers totals ~8 physical layers.
- **Yield constraints:** Silicon cracking and mechanical assembly defects cannot be bypassed with DRAM spare lines or logic redundancy.
---
[31:09](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1869s) Thermal Dissipation, Power Delivery, and "Chip Plumbers"
Stacking components vertically shifts major constraints toward thermodynamics and power delivery. Chip designers must eliminate physical hot spots by spreading thermal load more intelligently. A major driver of excess heat is inefficient power delivery; delivering power cleanly at desired voltages with fast transient response and tighter voltage regulation prevents the formation of destructive "thermal chimneys."
Dissipating that heat requires exotic materials and advanced mechanical designs. Materials like diamond substrates have been proposed to conduct heat away rapidly, paired with direct-to-chip liquid cooling loops. Hardware engineers are finding that advanced semiconductor architecture is no longer just electrical logic design: "We engineers are becoming plumbers," managing fluid dynamics, mechanical stresses, and heat transfer directly at chip scale.
- **Voltage regulation:** Tighter power responsiveness and lower delivery losses directly prevent local thermal chimneys from forming.
- **Thermal materials:** Exploration of synthetic diamond heat spreaders alongside direct-on-die liquid cooling solutions.
---
[32:14](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1934s) The Rejection of All-Optical Memory Disaggregation and PIM
Proposals to move memory entirely off-package using optical interconnects to create massive shared memory pools run into steep thermodynamic penalties. Relying on continuous optical-to-electrical-to-optical (OEO) conversions incurs severe energy losses across DACs and transceivers. Computing energy has dropped significantly—to remarkable femtojoule-per-operation levels—whereas data communication costs roughly a thousand times more femtojoules per bit. Paying that exorbitant energy tax simply to access remote memory pools violates the underlying physics of efficient computation.
Similarly, processing-in-memory (PIM) architectures—which attempt to solve bandwidth bottlenecks by pushing compute operations directly into the memory dies—introduce severe software and architectural friction. Offloading micro-tasks to memory forces awkward workload partitioning and limits versatility. PIM concepts have circulated for 25 to 30 years without dominating mainstream architecture, and they will likely remain niche 25 to 30 years from now. The enduring, generalizable architecture across evolving AI workloads remains high-density, high-performance, wide-bandwidth memory sitting physically adjacent to high-powered compute engines.
| Architecture Paradigm | Proposed Advantage | Core Physical / Practical Limitation |
|:--- |:--- |:--- |
| **All-Optical Disaggregated Memory** | Unconstrained memory pool size decoupled from package physical area | Heavy OEO conversion overhead; communication femtojoules/bit are ~1,000× costlier than compute |
| **Processing-in-Memory (PIM)** | Reduces required memory bus bandwidth by moving execution into DRAM | Workload fragmentation, rigid scheduling constraints, and poor generality across evolving models |
| **Co-Packaged Compute + Memory** | Maximum localized bandwidth with minimal energy per bit transferred | Complex 3D packaging assembly, thermal dissipation limits, and yield stacking math |
---
[34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) The Death of Copper and the Impending Shift to Co-Packaged Optics
While optical links make little sense between tightly coupled local compute and memory, optics must inevitably replace copper for external I/O and cluster-level scale-up fabrics. Copper interconnects are effectively becoming shorter and shorter waveguides as data rates rise; driving signals over a five-meter copper link is already becoming more expensive and complex than driving optical fiber across 100 meters.
```mermaid
flowchart LR
accTitle: Migration from Scale-Up Copper to Optical Fabrics
accDescr: Sequence showing the technological shift from legacy rack-scale copper to near-package and co-packaged optics driven by radix scaling.
subgraph Legacy["Current Era: Copper Waveguides"]
Cu["Extreme Copper Harnesses\ne.g., NVL72"] --> Limit["Physics Limit:\nRange shrinks to <5m\nExtreme weight & rigidity"]
end
The sole reason the industry has not transitioned entirely to optical scale-up fabrics is supply chain immaturity. Systems like NVIDIA's NVL72 rack represent an "engineering marvel and a manufacturing nightmare," using dense copper spines that required the broader supply chain nearly an extra year and a half to digest and manufacture reliably at scale. Had high-density near-package optics (NPO) or co-packaged optics (CPO) been mature enough, that architecture would look entirely different, providing superior scale-up radix, better energy efficiency, and lower overall cost. The industry-wide pivot toward optical engines for scale-up interconnects is expected to reach production maturity around the 2028–2029 timeframe.
- **Copper reach limitations:** Physics is turning copper cabling into highly constrained waveguides where transmission beyond 5 meters is no longer cost- or power-competitive with optical interconnects.
- **NVL72 manufacturing toll:** Exemplified the extreme mechanical and logistical ceiling of dense copper backplanes, absorbing 18 months of supply chain stabilization.
- **Target adoption window:** Broad industry transition to NPO/CPO for large-radix scale-up clusters is pegged for 2028–2029.
[37:26](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2246s) Optical Networking and the Shift to Circuit Switching
Pat Gelsinger addresses the enormous physical hurdles of bringing optical interconnects directly to GPUs at a scale of 10 million units. Optics naturally resist harsh thermal environments, yet severe heat is unavoidable inside high-density accelerator packages. Integrating optics directly into chip packaging also strains industry laser manufacturing capacity, though Gelsinger frames these not as fundamental physics blockers, but as an engineering grind: "It's just a lot of hard work at this point to pull all of those pieces together."
When Guido Appenzeller asks whether scaling limitations will force a shift from traditional packet switching toward optical circuit switching (OCS), Gelsinger references Stanford professor and networking pioneer Nick McKeown. Traditional networking was engineered to transport any packet anywhere with zero prior knowledge of its destination. AI workloads behave in the exact opposite manner: they consist of massive, predictable, scheduled flows. Because these flows are so structured, the network inherently resembles a circuit switch far more than a packet switch.
```mermaid
flowchart LR
accTitle: Evolution from Traditional Packet to AI Optical Circuit Switching
accDescr: Diagram comparing unpredictable traditional packet networks with predictable AI optical circuit networks.
subgraph Traditional["Traditional Packet Networks"]
direction TB
T1["Any packet to anywhere"] --> T2["Zero knowledge of destination"]
T2 --> T3["Heavy packet inspection & routing"]
end
subgraph AIWorkloads["AI Optical Fabric (OCS)"]
direction TB
A1["Massive, predictable flows"] --> A2["Deterministic endpoints"]
A2 --> A3["Direct Optical Circuit Switching (OCS)"]
end
```
This structural shift brings optics straight into the compute complex and blurs the historical line between scale-up and scale-out architectures. If the radix of a scale-up domain becomes vast enough, the distinction between internal fabrics and inter-cluster networks largely dissolves. Appenzeller agrees that the historical division between scale-up and scale-out stems mostly from different protocol stacks across front-end and back-end networks, predicting that system architectures will eventually converge around AI training fabrics.
- Optical integration bottlenecks include thermal sensitivity, packaging complexity, and insufficient global laser production capacity.
- Workload profile inversion: traditional networks manage unpredictable packets, whereas AI clusters execute large, deterministic, scheduled data flows.
- Optical Circuit Switching (OCS) combined with in-package optics effectively dissolves the architectural boundary between scale-up fabrics and scale-out clustering.
---
[40:21](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2421s) Continuous Learning and the Heterogeneity Debate
The discussion turns to whether training and inference will remain separate regimes. Gelsinger challenges the strict division by asking a fundamental biological question: "Does your brain stop learning?" He argues that algorithms will inevitably evolve toward continuous learning, where models continuously update their weights directly from inference environments to achieve ongoing refinement. As smaller, specialized datasets enter enterprise workflows, the wall between training and inferencing will erode, undermining the justification for overly fragmented chip architectures.
Appenzeller counters that inference currently operates over a much smaller node diameter than foundation model training. He also makes the case for hardware heterogeneity based on workload demands: a lightweight image diffusion model has radically different compute and memory bandwidth requirements than a 15-trillion-parameter frontier large language model.
Gelsinger acknowledges the theoretical appeal of specialization, but points out the brutal operational cost in mega-datacenters. Deploying bespoke clusters of 1,000 or 2,000 specialized chips for discrete workloads forfeits the management simplicity and fungibility of running a million uniform GPUs. Operating disparate chip fleets creates operational nightmares around fault domains, upgrades, interconnects, and maintenance. Appenzeller deadpans with tech humor: "The agents will do all the upgrades, so there's no longer a problem," prompting Gelsinger's amused reply that fleet management is officially solved.
- Continuous learning algorithms may merge inference and training by dynamically updating model weights in production.
- Workload heterogeneity pits tailored hardware (optimized for compute-heavy vs. memory-bandwidth-heavy tasks like diffusion) against the operational efficiency of homogenous datacenter fleets.
- Datacenter-level fleet management favors uniform GPU architectures to avoid fragmented fault domains, connection fabrics, and upgrade cycles.
---
[42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) Energy Capacity as the Ultimate Constraint on AI Growth
Moving past compute, memory, and networking, the hosts identify power as the overarching limit on the AI buildout. Gelsinger grounds the crisis in national policy failures, pointing out that the United States spent 10 to 15 years essentially flatlining its total electric capacity by decommissioning coal power plants at the exact rate it introduced renewable sources.
In the AI era, Gelsinger argues, this stagnation represents a grave economic threat: "In an AI digital age, energy capacity equals economic capacity." Under that thesis, national economic ceiling remained stagnant for a decade and a half. While the past five years have triggered frantic investment to secure megawatts, national capacity has only crept up by roughly 4% annually—a marginal improvement when viewed against the exponential demands of AI infrastructure.
```mermaid
flowchart TD
accTitle: The Macro Energy Bottleneck for AI Datacenters
accDescr: Flowchart illustrating the chain from energy generation deficits to potential datacenter capital defaults.
subgraph GridStagnation["Grid Stagnation & Generation Lead Times"]
G1["Flatlined Grid: Coal retired at rate of renewable adds"]
G2["Gas turbine order backlogs: 8-year lead times"]
G3["Nuclear additions stalled for decades"]
end
subgraph DataCenterCap["AI Datacenter Constraints"]
D1["Multi-gigawatt campus demands"]
D2["Energy availability as hard upper bound"]
end
subgraph EconomicImpact["Economic Fallout"]
E1["Power shortfall triggers datacenter project defaults"]
E2["Capital commitments outpace real grid interconnection"]
end
GridStagnation --> DataCenterCap --> EconomicImpact
```
The physical realities of power generation make rapid scaling extraordinarily difficult. Lead times for commercial natural gas turbines currently stand at eight years, and the renewable supply chain remains heavily reliant on China. Because compute cannot run without electricity, Gelsinger warns that energy deficits will act as a major headwind dampening AI market euphoria. Without power to energize planned facilities, the industry will inevitably witness project defaults where capital was committed for land, concrete, and GPUs before grid interconnections were secured.
- National US electrical generating capacity remained virtually flat for 10 to 15 years due to offsetting coal retirements with renewables.
- Recent generation capacity growth has expanded by only ~4% annually, lagging far behind hyperscaler demand.
- Heavy equipment lead times present severe bottlenecks, including eight-year backlogs for natural gas turbines.
- Speculative datacenter builds risk defaulting on capital commitments due to a lack of power interconnection.
---
[45:27](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2727s) Grid Realities, Nuclear Base Load, and 800V DC Architecture
To break the power bottleneck, Gelsinger calls for deep technological innovation across domestic supply chains, championing nuclear fission as essential baseload energy. He points to next-generation nuclear ventures—such as Playground Global's portfolio company Aalo Atomics—while lamenting that the broader US nuclear buildout went dormant for two decades. Revitalizing domestic nuclear capability is critical to securing clean, uninterruptible power that does not rely on foreign supply chains.
Beyond raw generation, major efficiency gains must occur within datacenter power distribution. Facilities are burdened by cumbersome power conversion cascades that step alternating current down through multiple intermediate stages simply due to historical utility standards.
Transitioning modern facilities to an 800-volt direct current (DC) distribution architecture cuts out conversion losses between the grid intake, rack rectifiers, and chip voltage regulators. Celebrating this shift away from legacy alternating current standards, Gelsinger quips: "The revenge of Edison is upon us."
- Nuclear energy provides essential zero-carbon baseload power, but commercial US reactor deployment remained largely stalled for over twenty years.
- Modernizing infrastructure requires reshoring foundational energy manufacturing and deploying compact, advanced fission systems.
- Adopting 800V DC power architectures within datacenters eliminates redundant AC-to-DC power conversion steps, directly reducing energy losses.
[46:50](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2810s) The Hardware Renaissance: Power Delivery and New Physics
Pat Gelsinger dives into the massive wave of hardware innovation required across power conversion and delivery to sustain the AI compute buildout. At Playground Global, they back startups tackling this head-on, such as Vertical GaN (gallium nitride), which Gelsinger views as a potential "killer technology." Vertical GaN enables direct, single-step down-conversion from high voltages straight to chip-level requirements—stepping from 800V directly down to 48V, 12V, or even 5V in one jump, dramatically improving power efficiency. The group points out that rebuilding the entire power delivery network through solid-state transformers, advanced switches, and reimagined cooling infrastructures has suddenly made "power cool again." Cooling systems are experiencing a parallel technological resurgence, forcing hardware engineers to reconsider turbines, novel refrigerants, and fundamental thermodynamic limits. Guido Appenzeller jokes that after 25 years he had to re-read Carnot efficiency principles, while Gelsinger laughs, asking how the first law of thermodynamics works again.
Beyond optimizing current silicon, Gelsinger argues that the industry will eventually hit a wall unless it alters the underlying physics. He observes that across the last five generations of Nvidia GPUs, the fundamental power per teraflop metric has essentially flatlined. To break through this thermodynamic wall, Playground Global invested in Snowcap Computing, a startup working on superconducting logic that could deliver up to a 1,000x improvement in energy-efficiency per compute operation.
```mermaid
flowchart LR
accTitle: AI Power Delivery Hierarchy
accDescr: Diagram illustrating high-voltage single-step down-conversion and alternative physics.
subgraph GridDelivery["Power Delivery Innovations"]
HV["800V High Voltage Feed"]
SST["Solid-State Transformers"]
VGaN["Vertical GaN Converter"]
HV --> SST --> VGaN
end
subgraph ChipLevel["Chip-Level Operation"]
Steps["Direct Single-Step Step-Down<br/>(800V to 48V / 12V / 5V)"]
CMOS["Conventional CMOS<br/>(Flatlining Power/TFLOP)"]
Super["Superconducting Logic (Snowcap)<br/>(1,000x Energy Efficiency)"]
VGaN --> Steps
Steps --> CMOS
Steps -.->|"Alternative Physics"| Super
end
```
- **Vertical GaN conversion**: Compresses multi-stage power steps into a single conversion jump (800V directly to 48V, 12V, or 5V).
- **Physical limitations**: GPU compute efficiency (power per teraflop) has largely plateaued over the past five silicon generations.
- **Superconducting compute**: Startups like Snowcap target roughly a 1,000x leap in power efficiency by moving away from traditional CMOS physics.
---
[48:42](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2922s) Rebuilding Abstractions: A "VMware for Agents"
Turning to Pat Gelsinger's and Raghu Raghuram's shared history leading VMware, the hosts shift to infrastructure abstractions in the AI era. Although topics like optical networking were on the agenda, Appenzeller observes that virtual machines are making a dramatic resurgence, prompting the question of how virtualization principles apply to autonomous agentic workloads. Gelsinger notes that all breakthroughs ultimately boil down to fundamental computer science primitives—data structures, algorithms, and abstraction layers. Just as VMware abstracted processors, storage, and networking away from bare-metal hardware, the next computing wave requires an analogous management layer built specifically for swarms of AI agents.
Appenzeller observes that VMs today are increasingly spun up and consumed by autonomous agents rather than human operators. This shift fundamentally alters the design constraints: agents demand near-instant startup latencies and cannot tolerate the delays humans accept, while the selection, orchestration, and complexity profiles of VMs become automated. In response, Gelsinger sketches out how a modern "VMware for agents" would function:
- **Target consumer shift**: Where classic hypervisors abstracted physical silicon to serve operating systems and human-built enterprise applications, the new abstraction serves dynamic agent swarms.
- **Core virtualization primitives reinvented**: Every core feature of enterprise infrastructure must be re-architected for agents—including life-cycle provisioning, security boundaries, performance orchestration, and an equivalent of "vMotion for agents" to migrate executing state across machines.
- **Dual-layer architecture**: While runtime operations cater entirely to low-latency agents, a parallel supervisory plane must give humans the tools to enforce governance—establishing policies, defining constitutional AI guardrails (referencing Dario Amodei's concepts at Anthropic), and observing systemic behavior through dashboards.
## Notable quotes
> [00:00](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=0s) energy capacity equals economic capacity — Pat Gelsinger
> [00:00](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=0s) nothing's a chip anymore, it's a rack — Pat Gelsinger
> [01:02](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=62s) Smart, aggressive, arrogant. He'll fit right in. — Ron Smith (quoted by Pat Gelsinger)
> [11:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=716s) HBM is a hideous memory. It's just the best one that we [have]. — Pat Gelsinger
> [18:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1136s) Okay. Well, could I borrow 50 billion, right? I mean, you're not talking about small capital expenditures. You're talking about very large capital expenditures and data centers, power commitments and so on like that. Well, I got to build this thing at scale... and that's going to defy a lot of heterogeneity. — Pat Gelsinger
> [22:08](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1328s) Yeah, that one died... We killed it. — Pat Gelsinger
> [24:44](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1484s) AI is a memory workload — Pat Gelsinger
> [28:06](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1686s) If I have a cracked die, I don't care how much resilience you've built into it. You can't, you know, it's a bad die. — Pat Gelsinger
> [31:09](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1869s) We engineers are becoming plumbers — Pat Gelsinger
> [34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) engineering marvel and a manufacturing nightmare — Pat Gelsinger
> [40:21](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2421s) Does your brain stop learning? — Pat Gelsinger
> [45:27](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2727s) The revenge of Edison is upon us. — Pat Gelsinger
## Mentioned
Pat Gelsinger, Raghu Raghuram, Guido Appenzeller, Ron Smith, Edward J. McCluskey, John Hennessy, Alberto Sangiovanni-Vincentelli, Nick McKeown, Dario Amodei, Intel, a16z, Playground Global, Lincoln Tech, Santa Clara University, Stanford University, UC Berkeley, VMware, Graphcore, OpenAI, NVIDIA, Anthropic, Groq, Samsung, SK Hynix, Micron, d-Matrix, Cerebras, Aalo Atomics, Snowcap Computing, Intel 80286, Intel 80386, Intel 80486, Intel Optane, 3D XPoint, Blackwell, NVL72, Jalapeno, High Bandwidth Memory (HBM), Vertical GaN
## Video description
a16z's Raghu Raghuram and Guido Appenzeller sit down with Playground Global General Partner and former Intel CEO Pat Gelsinger to discuss the next wave of semiconductor innovation and the physical constraints shaping the AI buildout.
Drawing on his experience designing Intel's 386 and 486 processors, Pat explains how AI could transform chip design, but also why faster design alone won't solve the industry's biggest problems. They examine the bottlenecks in manufacturing, memory bandwidth, advanced packaging, and power, and why today's explosion of specialized AI chips may eventually consolidate around a smaller number of architectures.
They also discuss the potential for new memory technologies, the shift from copper to optical networking, and why energy capacity could become a major constraint on AI growth. Finally, they revisit Pat's VMware years to ask what virtualization might look like when infrastructure is built for agents rather than humans.
Timestamps:
00:00 - Intro
01:00 - From tech school to Intel at 18
05:44 - The 486 and the birth of modern EDA
07:23 - How AI changes chip design
09:18 - Why silicon still takes nine months
14:49 - Will 100 AI chips converge to a few?
22:08 - Why HBM is a hideous memory
25:15 - How tall can chips get?
42:49 - Energy capacity equals economic capacity
48:42 - A VMware for agents
Resources:
Follow Pat Gelsinger on X: https://x.com/PGelsinger
Follow Raghu Raghuram on X: https://x.com/RaghuRaghuram
Follow Guido Appenzeller: https://x.com/appenz
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://twitter.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.
## Background
该内容由知名风险投资机构 a16z 于 2026 年 10 月 9 日在 YouTube 频道发布。视频中 a16z 的 Raghu Raghuram 和 Guido Appenzeller 邀请了 Playground Global 通用合伙人兼英特尔前首席执行官 Pat Gelsinger，共同探讨下一波半导体创新浪潮以及正在塑造 AI 基础设施建设的物理现实与瓶颈约束。
## TL;DR
对谈中 Pat Gelsinger 结合自身从 80386 到 80486 的传奇芯片研发经历，指出尽管 AI 原生工具已将前端芯片设计大幅压缩至数月，但晶圆制造、先进封装和机架集成的 9 个月物理周期依然是无法绕过的硬约束。他深入剖析了 HBM 的物理缺陷、3D 垂直堆叠的良率天花板，以及 NVL72 机架背后铜缆互连面临的功耗极限，断言未来芯片架构与全光 I/O 必将走向收敛。同时，Pat 警示能源容量正是当下 AI 经济的终极天花板，而面对爆发的智能体蜂群，计算世界正迫切呼唤一套面向 Agent 的全新虚拟化基础设施。
## Atmosphere & style
整场对话充满着老友重聚般的轻松幽默与高密度的硬核工程洞察，主持人和嘉宾之间既有相互打趣的默契，也有针对底层架构哲学的犀利交锋。Pat Gelsinger 展现出深厚且笃定的工业界老兵气场，用直白辛辣的语言打破当下的风投狂热与技术神话。整场讨论犹如置身于顶尖工程大师的工作室，在热力学定律、硅片物理极限与前沿系统架构的碰撞中，让人真切感受到硬件文艺复兴带来的澎湃脉动。
## Bulletin
- [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) Pat Gelsinger 强调 AI 芯片设计前端虽可压缩至 3 个月，但晶圆制造、3D 封装与机架集成仍需 9 个月，长达一年半的软硬件落地周期极易导致硬件架构面世即落后于最新算法。
- [13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) 市场上上百种专用 AI 芯片的异构繁荣只是暂时的，受制于 1.5 年的流片周期、数百亿美元规模的机架部署资本壁垒及云厂商软件运维复杂度，底层架构终将向少数赢家收敛。
- [28:06](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1686s) 高密度垂直堆叠面临良率指数级归零的残酷惩罚，2 到 4 层的内存“中层建筑”才是物理极限下的甜点区；而通信比特能耗比计算高出千倍，使得全光解耦内存池与存内计算缺乏通用可行性。
- [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) 在 AI 数字时代，能源容量直接等同于经济容量；面对美国重型燃气轮机 8 年交付排期与发电容量停滞，万卡集群正倒逼数据中心全面推进 800V DC 直流供电与垂直氮化镓革新。
- [50:40](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=3040s) 随着 AI 基础设施的调用主体由人类转向软件智能体，系统层迫切需要围绕亚秒级启动、上下文状态迁移（Agent VMotion）及安全护栏重构面向 Agent 的全新虚拟化技术栈。
## Key takeaways
- [05:44](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=344s) 80486 微处理器是人类芯片史上首颗系统性采用现代 EDA 概念打造的产品，奠定了硬件描述语言与自动布局布线等现代工业基石。
- [07:23](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=443s) AI 原生工具极大降低了纯数字逻辑设计的门槛，英伟达 Blackwell 架构可能是纯手工主导微处理器设计的最后一代，但 SerDes 等高速模拟电路依然高度依赖硬核工程经验。
- [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) 从晶圆流片、3D 先进封装到机架级集成部署耗时达 9 个月，若从设计到规模化上机耗时一年半，芯片原有的架构假设将面临被算法演进淘汰的巨大风险。
- [11:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=716s) HBM 内存受制于边缘带宽物理瓶颈且发热严重，加之落后的 3D 功耗与热点仿真迫使芯片加入极度保守的保护带，白白损失了高达 40% 的电力效率。
- [13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) 百家 AI 专用加速芯片竞争无法长期维持，过度细分切分工作负载违背运维规律，且大模型正向依赖传统 HPC 双精度计算与深度推理演进，促使架构重新收敛。
- [18:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1136s) 虽然 AI Agent 降低了为异构硬件编写算子内核的门槛，但百亿美元级别的资本开支与巨头构建的系统抽象层，仍将把异构硬件形态在顶层吞没。
- [22:08](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1328s) 过去三十年全行业商业化的全新物理内存形态为零，直到 AI 内存墙爆发推动内存板块市值暴增 2.5 万亿美元，才为铁电材料等颠覆性创新注入资本土壤。
- [28:06](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1686s) 3D 堆叠存在严苛的良率惩罚，追求 16 层或 32 层的高密度堆叠在制造上不可行，包含 2 到 4 层内存的小芯片复合体“中层建筑”才是当前工艺与散热的平衡点。
- [31:09](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1869s) 通信每比特能耗比单次计算高出整整上千倍，将内存完全移出封装的光学解耦池以及缺乏通用性的存内计算（PIM）均违背了底层物理规律。
- [34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) 铜缆在 5 米内的功耗与成本代价正急剧恶化，基于光电共封装（CPO）或近封装光学（NPO）的大基数集群互连将在 2028 至 2029 年迎来规模化商用拐点。
- [37:26](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2246s) AI 工作负载呈现规模巨大且路径高度可预测的数据流特征，与传统不可预测的离散数据包网络截然相反，天然契合将光互连深度引入计算核心的光线路交换（OCS）架构。
- [39:31](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2371s) 持续学习将促使模型在推理过程中实时动态微调权重，逐渐抹平专门训练芯片与专门推理芯片之间的二元对立。
- [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) 能源容量等同于经济容量，美国过去十五年发电装机容量几乎零增长，重型燃气轮机交付周期长达 8 年，电力短缺将直接成为 AI 数据中心违约的硬约束。
- [46:50](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2810s) 英伟达近五代架构的单 Teraflop 基础功耗已近乎走平，必须依赖垂直氮化镓单级变压、固态变压器甚至超导计算等底层物理革新打破热力学瓶颈。
- [48:42](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2922s) 面向 Agent 蜂群爆发的全新时代，系统层必须重新发明包括调度、安全隔离画像及上下文迁移（Agent VMotion）在内的全套虚拟化技术底座。
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=0s) 引言与硬件创新的新黄金时代
在 AI 数字时代，能源容量直接等同于经济容量。如果不解决供电问题，建设新数据中心并采购上百万张 GPU 毫无意义；由于电力缺口，未来许多数据中心项目将面临违约风险。与此同时，芯片设计领域的技术突破正在重塑行业——"当一项技术让某件事变容易时，瓶颈就会随之转移"。芯片不再仅仅是单块硅片，而是演变成整个机架（Rack）。尽管设计周期可能缩短至 3 个月，但芯片物理制造仍需要 9 个月的时间。
与此同时，过去 30 年里基础存储技术的重大创新几乎为零，如今这一领域终于迎来了突破拐点。伴随铜互连向光互连的演进，硬件工程正在迎来一次名副其实的文艺复兴。尽管目前市场上涌现出上百家 AI 推理加速芯片公司，但历史上任何处理器行业都从未长期容纳过 100 家竞争对手，这场百舸争流的狂欢最终必将向少数几家核心架构收敛。
```mermaid
flowchart LR
accTitle: AI硬件瓶颈转移链条
accDescr: 展示从AI设计加速到物理制造与能源约束的瓶颈转移过程
subgraph Design["设计层加速"]
A["高层次目标规范"] --> B["AI驱动EDA与逻辑生成"]
end
subgraph Physical["物理制造约束"]
B --> C["9个月晶圆流片周期"]
C --> D["模拟电路/SerDes实测验证"]
end
subgraph Infrastructure["基础设施瓶颈"]
D --> E["存储带宽与光互连"]
E --> F["机架级供电与能源容量上限"]
end
```
### [01:00](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=60s) 从中专到英特尔：18岁加入芯片先锋阵营
Raghu Raghuram 与 Guido Appenzeller 邀请到了老朋友兼前上司、Playground Global 通用合伙人及英特尔前首席执行官 Pat Gelsinger。对话回顾了 Pat 的早年经历：他并未走传统的名校本科路线，而是 16 岁时意外赢得奖学金进入技术学校（Trade/Tech school），提前跳过了高中最后一年半的课程。18 岁那年，英特尔来到他的学校招聘。
在此之前，Pat 从未坐过飞机，但在中专期间已经彻底迷上了计算机。面试官 Ron Smith 当天连续面试了 12 个人，Pat 恰好是第 12 位。Ron 在评估单上留下了极具戏剧性的评语："聪明、好斗、自傲，他会非常适应这里"（Smart, aggressive, arrogant. He'll fit right in）。正是凭借这份评价，18 岁的 Pat 正式加入英特尔，开启了他的半导体生涯。
- 职业晋升路径：最初担任技术员（Technician），在 80286 尾声阶段转入设计团队；成为 80386 的第 4 号工程师；随后担任 80486 的架构师兼设计经理。
- 工读结合：在英特尔参与划时代芯片设计的同时，Pat 完成了学士学位、硕士学位以及博士课题的研究。
### [03:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=227s) 斯坦福讲堂上的交锋与产学互促
Pat 回顾了在斯坦福大学半工半读的独特时光。由于白天在英特尔最前沿的一线芯片项目实操，晚上再去斯坦福上课，课堂理论与产业实践之间产生了极其生动的碰撞。有一次，一位教授在课堂上提出了一种新型先行进位加法器（Carry-lookahead adder）的设计。Pat 恰好正在 80386 上开发超前进位加法器，便把教授的理论带回英特尔测试，两天后回到课堂直接对教授说："这根本行不通。"
- 加法器争论：由于那位教授即将出版一本包含该设计的著作，出版时间非常紧迫，师生二人连续两堂课都在全班面前激烈争辩该设计为何在实际工程中失效。
- 内置自测（BIST）：Pat 师从被称为"内置自测之父"的斯坦福教授 Ed McCluskey。Pat 直言不讳地指出教授的大多数想法缺乏实际可操作性，同时成功将实用的内置自测技术引入了 80386。
- 导师 John Hennessy：图灵奖得主、后来的斯坦福校长 John Hennessy 担任 Pat 的论文导师，Pat 开玩笑说自己当年也算"拉了他一把"，助力了他的学术与职业生涯。
这种高校与产业界紧密交织、学生敢于直接挑战权威象牙塔的氛围，正是硅谷创新生态得以生生不息的核心特质。
### [05:44](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=344s) 80486 与现代 EDA 工业的奠基
在 80486 研发之前，现代意义上的电子设计自动化（EDA）行业尚不存在。虽然 80386 尾期做过少量高层次探索，但 80486 是人类芯片历史上第一颗系统性采用现代 EDA 概念打造的微处理器。
- 硬件描述语言：当时 Verilog 尚未普及，设计团队必须从零定义高层次 RTL 规范，Pat 亲自编写了英特尔专有的硬件描述语言（HDL），并为此开发了配套的编译器。
- 自动布局布线：当时没有自动化的布局布线工具（Place and Route），Pat 团队与加州大学伯克利分校的 Alberto Sangiovanni-Vincentelli 教授及其学生展开合作，首创了第一批自动布局、自动布线以及自动时序管理工具。
- 行业里程碑：80486 不仅是一款兼容 x86 的高性能流水线微处理器，其研发过程中沉淀的工具和方法论，直接奠定了整个现代 EDA 工业的基础。
### [07:23](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=443s) AI 驱动芯片设计：新时代的转折点
对比 80486 时代的工具演进，当前半导体行业正处于由 AI 驱动的新一轮设计变革之中。正如当年 80486 开拓了自动化设计一样，英伟达的 Blackwell 架构很可能会被后人视作由纯人工主导微处理器设计的"最后一代产品"。
- 逻辑与高层设计的颠覆：通过类似于 Jalapeno 等 AI 原生工具（从第一性原理出发利用 AI 设计芯片），设计人员只需在高层次上指定目标规范，AI 便能极其高效地自动生成并优化逻辑网表、编写内核及底层驱动软件，极大降低了对大规模纯手工逻辑设计的依赖。
- 难以被 AI 取代的物理瓶颈：AI 在处理纯数字逻辑时表现惊艳，但在模拟电路领域（尤其是 SerDes 等高速串行接口）依然束手无策。这类模拟模块高度依赖真实的硅片物理反馈和海量实测数据。工程师要么选择极其保守、牺牲性能的设计以便 AI 介入，要么只能依赖资深专家的硬核工程经验人工攻关。
当 AI 将逻辑设计和代码生成的门槛大幅拉低时，芯片创新的核心瓶颈必然向物理制造与基础设施转移。设计一款全新领先芯片可能只需数月，但从硅片流片、晶圆制造到实际上机部署，依然横亘着无法被纯软件轻易压缩的漫长时间窗口。
### [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) 芯片设计的真正瓶颈：九个月的物理制造周期与封装挑战
Pat Gelsinger 深入剖析了当前芯片行业面临的严峻现实：虽然凭借对 x86 架构以及当前 AI 工作负载的深刻理解，团队可以将乘法累加（MAC）单元和寄存器结构优化得极其出色，甚至借助先进的 AI 辅助设计工具，在短短三个月内就能交付一套令人惊叹的芯片设计，但后续环节却撞上了坚硬的物理制造之墙。从设计完成到拿到真正可规模化量产的硅片，依然需要长达九个月的时间。这种制造周期的滞后严重制约了创新的节奏——“你可以花三个月完成设计，但九个月内你根本拿不到规模化芯片，这太糟糕了。”
即便采用了极度优化的设计流程，芯片从晶圆厂（Fab）制造出来也至少需要三个月；紧接着是复杂的 3D 先进封装（Advanced Packaging），这又要耗费大约三个月；最后，由于当今的算力单元早已不再是单颗芯片，而是演变成了机架级（Rack-scale）系统，组装与部署进机架又要花上三个月。Pat 尖锐地指出，如果从设计到软件跑上规模化硬件需要耗费整整一年半，那么当初针对 AI 负载所做的架构假设和理解，在芯片真正面世时早已过时。正如历史上的 Graphcore 芯片，其本身设计并不差，但等它推向市场时，整个世界的技术路线早就向前演进了。为了打破这一僵局，行业亟需变革：必须出现新型光刻技术（Lithography），并将动辄 5000 万美元的光罩（Mask）成本彻底降下来，唯有将原型试制周期压缩至一到两个月，硬件才能真正跟上算法迭代的步伐。
```mermaid
flowchart LR
accTitle: 现代芯片开发与交付流程瓶颈
accDescr: 展示从前端架构设计到机架部署的时间周期与潜在风险
subgraph S1["前端设计阶段 (3 个月)"]
D["AI 加速芯片设计<br/>(算法/架构匹配)"]
end
subgraph S2["物理实现与部署阶段 (9 个月瓶颈)"]
F["晶圆厂制造<br/>(~3 个月)"]
P["3D 先进封装<br/>(~3 个月)"]
R["机架级集成部署<br/>(~3 个月)"]
end
subgraph S3["落地后果"]
W["算法与工作负载脱节<br/>(原设计在一年半后过时)"]
end
D --> F
F --> P
P --> R
R --> W
```
### [11:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=716s) 内存带宽与电力建模：隐藏在背后的致命硬伤
除了漫长的制造周期，Pat 认为阻碍 AI 计算跃升的另外两大核心瓶颈是内存与供电。他毫不客气地将当前大红大紫的 HBM（高带宽内存）斥为一种“极其丑陋的内存技术”（hideous memory），直言 HBM 只是在没有更好选择时的无奈之选。HBM 的位密度（Bit Density）很差，带宽受到芯片边缘海岸线（Shoreline bandwidth）的物理限制，且发热问题极其严重；而 DRAM 本身极度不耐热，将庞大的发热元件全部集中压在一个微小空间内，对于 AI 这种高度依赖内存计算（Memory-compute workload）的任务而言，几乎是一种灾难性的妥协。
另一个常被低估的危机在于供电与热仿真技术。当前的 3D 功耗仿真与热点建模工具十分原始，无法精确模拟微架构内的热点耗散与动态电压跌落。为了防止芯片崩溃，工程师被迫在整个功耗区间内加入极其保守的保护带（Guard-banding），这直接导致芯片白白损失了高达 40% 的潜在电力效率。伴随着电力传输模型面临的颠覆性重构，如果无法在内存架构、高精度功耗仿真以及供电交付方式上实现系统级突破，AI 硬件即便拥有再先进的 EDA 工具，也终将撞上物理散热与能耗的铜墙铁壁。
### [13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) 百舸争流的 AI 芯片：大爆发后的必经收敛
针对当前风投疯狂押注 AI 芯片的现象，Pat 与主持人们展开了风趣的交锋。Raghu Raghuram 提到，市面上涌现了上百种针对特定工作负载的专用 AI 推理加速芯片，Pat 随即打趣反问：“既然如此，你们这些风投为什么还要投这么多家？”主持人则笑着回击：“你也投了不少啊！”Guido Appenzeller 随后抛出一个严肃的历史命题：纵观计算发展史，处理器行业从未存在过能同时容纳上百家供应商的先例。随着 AI 设计工具的普及，未来究竟会走向一个长期异构共存的世界，还是像过往一样最终收敛至少数几家巨头？
Pat 明确断言，当前的异构繁荣只是“暂时的表象”，市场最终必然会发生大规模整合并收敛到少数几家，并给出了三点核心论据：
- **过度细分（Granularity）无法持续**：目前行业开始对工作负载进行病态的精细切分，不仅在 Prefill（提示词预填充）和 Decode（自回归解码）阶段使用不同芯片，甚至开始区分 Midfill 和推测验证（Speculate and verify）层级。但历史证明，底层硬件过度颗粒化绝不可能长久支撑大规模机房运维。随着以 OpenAI 为代表的模型向深度推理（Reasoning models）演进，计算特征反而重新开始偏向传统 CPU 架构，过度针对狭窄阶段特化的 ASIC 会迅速失去算力弹性。
- **算法范式转移与 HPC 的回归**：现有平面化 LLM 正逐渐触及能力边界，在面对分子结构、生物化学及高维 3D 视觉建模等场景时，计算需求将从低精度推演回归到依赖传统高性能计算（HPC）的 64 位双精度浮点计算。例如由 AI 筛选出最具潜力的五个化学分析域，后续仍需极高精度的物理仿真算法，这种工作负载的剧烈迁移将使极端特化的芯片被迅速淘汰。
- **规模效应与软硬件协同的客观规律**：光子计算、概率计算等上百种芯片争奇斗艳，但半导体归根结底依赖规模经济、资本获取与生态占有率。更重要的是，顶级模型企业（如 OpenAI、NVIDIA、Anthropic）会亲自下场挑选能够与其核心算法实现“软硬件协同演进”（Software-hardware co-design）的最终赢家。无法跨越规模壁垒并实现软硬件深度绑定的边缘芯片架构，终将在行业整合的大潮中被无情清洗。
### [18:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1136s) AI 时代的异构算力与巨额资本壁垒
Pat Gelsinger 解释了为什么芯片市场最终会收敛到少数几家胜出者：大型云厂商面对市面上十几种方案，最终只会挑中最中意的一两种，因为支持这些平台需要对软件生态和工作负载演进投入巨额资源；再加上部署复杂度极高，收窄是必然的。
Guido Appenzeller 提出了一个反向观点来“挑起争议”：过去硬件之所以高度标准化为一两种架构，核心原因是软件复杂度太高——不同指令集需要维护专属编译器，迫使产业收敛。但 AI 正在重塑这一逻辑。例如像 Groq 这类芯片，尽管人类很难为其手工编写底层优化算子，但如今让一个“Agent 蜂群”（agent swarm）跑上一整晚就能把内核调优搞定。既然软件适配门槛大幅降低，硬件市场是否反而会容纳更多的异构性？
Pat 承认软件生成的瓶颈确实转移了，但异构芯片仍面临两道极其残酷的现实壁垒：
```mermaid
flowchart TD
accTitle: 异构芯片落地的双重现实壁垒
accDescr: 展示芯片从研发到大规模部署面临的时间和资本开支瓶颈
subgraph Timeline ["周期壁垒"]
A["芯片流片与制造<br/>(耗时约 1.5 年)"] --> B["上架机架并完成规模化验证"]
end
subgraph Capital ["资本与基建壁垒"]
B --> C["巨额资本开支<br/>(借款 500 亿美元级别)"]
C --> D["数据中心建设与电力容量锁定"]
D --> E["超大规模部署并跑通商业价值"]
end
```
Pat 指出，芯片制造依旧需要一年半，而把它们大规模部署到机架中更是天文数字级别的资金游戏——“你得先去借 500 亿美元”，还要抢占稀缺的数据中心电力指标。这种规模效应天然抗拒无休止的异构化。此外，巨头们往往会通过架构抽象直接吸纳异构性，比如英伟达整合类似 Groq 的技术后，两年内用户根本分不清内部用的是什么，“一切都只是底层的另一层抽象”，异构特性在系统级被吞没，最终难以在市场上以独立形态大量存活。
- **核心矛盾**：虽然 AI Agent 降低了为异构硬件编写算子和软件层的门槛，但硬件物理制造周期（1.5 年）与数十亿至数百亿美元的部署资本开支，依然是无法绕过的硬约束。
- **抽象层吃掉异构**：平台巨头通过分层与抽象（layering and abstraction）吸收专用计算架构，开发者面对的仍是统一接口，异构硬件难以直接在顶层显露。
---
### [22:08](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1328s) 内存产业的三十年荒漠与 2.5 万亿美元爆发
话题转向了被 Pat 戏称为“糟糕内存”（hideous memory）的 HBM。Pat 笑称它就像丘吉尔口中的民主制度——虽然缺点一堆，却是“目前我们能找到的最好选择”。
Raghu Raghuram 提到，行业上一次重大内存创新似乎还是英特尔的傲腾（Optane），并打趣问 Pat 是不是参与其中。Pat 坦率回应：“那项技术已经死了，被我们亲手杀死了。”回顾过去三十年，Pat 亲自经手过至少 5 种未能存活的全新内存架构，见证了全行业近百种尝试，但真正实现商业化的大规模全新内存类别是：**零**。整个产业依然被 DRAM、SRAM 和 Flash 牢牢统治，连 3D 闪存堆叠也算不上真正的底层物理革新。
其根源在于过去三四十年间，内存是一门极端残酷的行业：
> “你赚一年钱，紧接着就要赔上四年。去一个五年里只有一年盈利的行业里搞研发？这种由剧烈同质化周期（violent commoditization cycles）驱动的市场，实在是太艰难了。”
但过去四年间，格局发生了翻天覆地的逆转。随着 AI 被广泛认知为“内存瓶颈型工作负载”（AI is a memory wall），全球三大内存巨头全部跻身全球市值前 20 强，整个内存板块市值暴增约 2.5 万亿美元。资本的大量涌入与技术需求的紧迫性，终于为打破底层物理极限的内存创新创造了土壤。
- **三十年技术停滞**：主流内存形态始终停留在 DRAM、SRAM 与 Flash，缺乏具备低成本、高性能、高韧性且易制造的新物理形态。
- **资本格局巨变**：内存行业在过去四年中市值激增 2.5 万亿美元，彻底改变了研发投入的财务逻辑，使得面向 AI 内存墙的激进创新重新具备商业可行性。
---
### [24:55](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1495s) 突破内存墙：堆叠架构与新材料探索
Pat 透露，自己刚刚隐秘支持了一家全新的内存初创公司（处于 Stealth 模式），并对未来几年即将涌现的内存突破充满期待。当前内存创新的发力点主要集中在两大领域：
1. **破除边缘带宽限制**：HBM 面临严峻的芯片边缘（shoreline bandwidth）互连物理瓶颈，迫使计算与内存必须走向三维堆叠整合。无论是 d-Matrix 还是 Cerebras，都在探索将逻辑算力与内存深度结合的全新物理拓扑结构。
2. **材料级物理突破**：铁电材料（Ferroelectrics）成为热门研究方向，旨在寻找非电容性（non-capacitive）、高密度且天生可堆叠的新介质。
对于目前部分团队尝试利用低成本闪存（Flash）或 MRAM 来应对 AI 负载的路线，Pat 明确表达了怀疑态度，认为 Flash 和 MRAM 底层单元的速度与延迟表现过于棘手，很难胜任严苛的 AI 运算需求。但他坚信，沉寂 30 年后，真正的内存创新终于近在眼前。
---
### [27:19](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1639s) 3D 堆叠的物理与良率天花板
Guido 追问计算芯片向上堆叠的演进速度：“我们的芯片会多快长高？是堆到两层停滞一阵，还是很快就会出现把逻辑与内存随意混合的 16 层三明治架构？”
Pat 指出，向垂直维度拓展面临着极其残酷的“良率惩罚”（yield penalty）。随着堆叠维度的提升，整个芯片组装体的价值呈指数级上升；这意味着堆叠中每一层单独裸片（die）的制造良率必须极高，否则底层良率相乘将导致整颗多层芯片的综合良率迅速归零。堆叠层数并非简单的机械叠加，而是直接受制于制造良率与物理特性的非线性递减规律。
### [28:06](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1686s) 芯片堆叠的极限与“中层建筑”甜点区
Pat Gelsinger 深入剖析了芯片 3D 垂直堆叠（Z 轴）的工艺现实。随着堆叠层数增加，对制造流程和缺陷控制的要求呈指数级提升。当堆叠达到 8 层甚至 16 层时，从良率数学模型来看几乎是不可逾越的鸿沟，整个制造链条必须无限接近“零缺陷”。针对 Guido Appenzeller 提出业界长期在 2D 维度通过冗余设计容忍缺陷（例如 DRAM 的坏块替换和备用行）的观点，Pat 反驳指出，这无法解决根本问题：无论设计有多大冗余度，一旦晶圆出现物理破裂，这颗芯片就彻底报废。
因此，Pat 坚决不看好疯狂追求 16 层或 32 层的高密度堆叠构想。相反，他认为由 3 层、4 层或 5 层组成的“中层建筑”（Mid-rises）才是兼顾制造可行性与复杂度的平衡甜点区。在如今计算晶片面积受限于光刻掩模版尺寸（Reticle Limited）的背景下，业界必须依靠良率与散热更好的小芯片（Chiplet）。若要在其上堆叠内存，2 层到 4 层的内存堆叠最为合理，而绝非 8 层。
```mermaid
flowchart TD
accTitle: 典型 3D 异构堆叠芯片架构
accDescr: 展示自上而下的微流道冷却、内存层、计算芯片、供电轨及光电 I/O 的八层结构。
subgraph Cooling["散热与流体层"]
L1["直接液体微流道 / 新型散热材料（金刚石等）"]
end
subgraph Stacking["核心 3D 堆叠结构（约 8 层）"]
L2["2~4 层高带宽内存堆叠 (Memory Stack)"]
L3["计算核心小芯片 (Compute Chiplets)"]
L4["高密度再布线层 (RDL) 信号分发"]
L5["垂直 Z 轴供电轨 (Power Rails / Backside Power)"]
end
subgraph Interconnect["混合信号与 I/O"]
L6["III-V 族光电集成层 (Optical I/O Engine)"]
end
Cooling --> Stacking
Stacking --> Interconnect
```
此外，垂直整合不仅包含计算与存储，还必须在 Z 轴引入供电轨（Power Rails）以及高度复杂的智能再布线层（RDL）来分发信号，再加上用于光学 I/O 的 III-V 族半导体材料集成。将所有要素叠加后，一个标准的芯片复合体厚度已达到约 8 层。Pat 感叹，将不同材料与功能强行融合成一体既是一项工程壮举，也是一场“制造噩梦”，而 2 到 4 层的内存堆叠正是当下物理现实能承受的上限。
---
### [31:09](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1869s) 散热困境、全光解耦的缺陷与存内计算的局限
Raghu Raghuram 提醒道，堆叠架构绕不开严峻的热耗散（Thermal Dissipation）挑战。Pat 直言，今天的半导体工程师实际上“已经变成了管道工”（Engineers are becoming plumbers），必须在芯片尺度上处理机械工程问题。解决发热的第一步是避免局部物理热点并优化热扩散；其次是提高电压调节响应速度，高能效地将所需电压直接送入芯片，减少“热烟囱”效应。此外，业界正在探索金刚石等新型高导热材料，并将直接液体冷却方案紧贴芯片表面。
针对部分学界与业界提出的极端方案——将内存完全移出封装，全靠光学链路连接计算与解耦内存池——Pat 持强烈的否定态度。他指出，频繁的“光-电-光”（OEO）转换不仅成本高，而且会白白消耗大量功耗。在物理能耗尺度上，当下每计算一次消耗的飞焦（Femtojoules per compute）已经极度高效，但每传输一个比特所消耗的能量比计算高出整整上千倍。这种通过光互连换取巨大内存池的做法完全“违背了底层的物理规律”。
同样，对于将计算逻辑推入内存的“存内计算”（PIM, Processing-In-Memory）理念，Pat 同样不看好。他解释称，PIM 迫使软件将计算细碎地拆解并强塞进内存层，会对复杂的工作负载施加极大的约束。尽管 PIM 的概念已经流传了二三十年，但它大概率在未来的二三十年里依然只是纸上谈兵。能够覆盖广泛演进工作负载的最通用、最可持续的方案，仍然是打造具备超高带宽、高密度的内存，并让它在物理距离上尽可能紧贴核心计算引擎。
- **能耗鸿沟**：每比特通信消耗的能量比每单位计算消耗的能量高出约 1000 倍。
- **PIM 的核心缺陷**：割裂工作负载，缺乏通用性，限制了复杂的计算流演进。
- **工程师的职能转变**：必须在芯片尺度（Chip Scale）全面统筹热力学与微流体设计。
---
### [34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) 全光 I/O 破局 Scale-Up，NVL72 是“工程奇迹与制造噩梦”
尽管 Pat 反对在核心计算与存储复合体之间滥用光学，但他对在芯片的全部外部 I/O 领域转向光学抱有坚定信念：“大约 25 年前我就宣称铜线已死，总有一天我会是对的。”如今，高带宽短距互连正在一步步验证他的预言。在当前的纵向扩展（Scale-Up）集群环境中，机柜内部塞满了密密麻麻的铜缆。然而，铜缆正沦为物理性能极易见底的波导；为了让铜线在短短 5 米内维持稳定信号所付出的功耗与成本代价，甚至已经超过了让光纤传输 100 米。
```mermaid
flowchart LR
accTitle: 机柜互连介质的技术转型趋势
accDescr: 对比传统铜缆在短距下的瓶颈与未来光学 I/O 的推进时间线。
subgraph Current["当前现状：铜缆局限"]
A["高密铜缆连接"] -->|"极限传输距离缩小（< 5米）"| B["成本/功耗急剧恶化"]
B --> C["NVL72：工程奇迹与制造噩梦"]
end
subgraph Future["行业演进：全光化"]
D["光电共封装 (CPO) / 近封装光学 (NPO)"] -->|"突破大基数 (Radix) 扩展瓶颈"| E["2028-2029 规模商用周期"]
end
```
谈及行业现状，Pat 尖锐地评述了 NVIDIA 的 NVL72 机架系统。他指出，从工程角度看它确实是一项不可思议的奇迹，但从制造业角度看却是一场“噩梦”。整个半导体和系统供应链花了整整一年半的额外时间，才勉强消化和量产这头庞然大物。NVL72 的诞生本身就证明了一个痛点：如果行业此前拥有成熟的光学供应链，根本就不应该去造这样一台靠海量铜缆硬撑的怪物。
推动全光网络落地的主要阻力在于光学供应链的复杂性，它此前从未经历过现代数据中心这种超大规模制造的压力测试。但在物理规律的强力倒逼下，为了构建具备超大基数（Large Radix）的 Scale-Up 算力集群，转向近封装光学（NPO）或光电共封装（CPO）已成为唯一的解法。Pat 明确预判，在 NVIDIA 及全行业的共同推动下，光学 I/O 的全面转型节点将在 **2028 至 2029 年** 真正到来。
### [37:26](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2246s) 光互连的规模化挑战与光线路交换（OCS）的崛起
随着计算集群向数百万甚至千万级 GPU 的规模狂奔，传统基于铜线的互连方案遇到了物理瓶颈，光电共封装与光学互连（Optical Networking）被视为必然路径。Pat Gelsinger 指出，光学器件天生对恶劣的热环境极其敏感，而高功耗 AI 芯片内部恶劣的散热工况恰恰是行业常态，这不仅带来了严峻的热管理难题与先进封装集成挑战，还暴露出全球激光器制造产能的严重不足。不过他强调，这些并非物理层面的根本障碍，纯粹需要全行业付出大量扎实且艰难的工程努力去逐一攻克。
Guido Appenzeller 顺势提出，面对光学网络无法无休止扩展的现实，网络架构是否会转向电路交换网络（Circuit Switched Networks）。Pat 引用了斯坦福大学教授 Nick McKeown 的深刻洞察：人类过去几十年构建的传统网络（如以太网、TCP/IP）核心逻辑是处理“任何可能前往任何目的地、但网络对此毫无先验知识的离散数据包”，而 AI 训练与推理的工作负载特征恰好完全相反——它们表现为规模巨大、流向高度可预测的巨型数据流（Large and Predictable Flows）。因此，整个底层网络在物理表征上更贴近交换机内部的电路直连，而非不可预测的数据包路由。Pat 坚信，将光学互连直接引入计算核心，并配合光线路交换（OCS, Optical Circuit Switching），正是适应这种高预测性大流量的终极正确架构。
```mermaid
flowchart TD
accTitle: AI工作负载驱动网络架构向光学电路交换转型
accDescr: 展示传统网络与AI网络负载特性的对比，以及AI负载如何推动计算复合体采纳光线路交换架构。
subgraph WORKLOAD["工作负载特征对比"]
TRAD["传统网络工作负载<br/>• 离散小数据包<br/>• 目的地址随机未知"]
AI["AI 深度学习工作负载<br/>• 规模巨大的数据流<br/>• 路径高度确定与可预测"]
end
subgraph ARCH["硬件与网络架构演进"]
OCS["光线路交换 (OCS) 与电路交换"]
OPTIC["光学互连深度集成入计算核心"]
LIMIT["工程瓶颈：封装散热、激光器产能"]
end
TRAD -->|"传统方案"| PKT["分组交换网络 (Packet Switching)"]
AI -->|"架构转型"| OCS
OCS --> OPTIC
OPTIC -.->|"工程攻坚"| LIMIT
```
- 光学器件面临的两大核心瓶颈：恶劣热工况下的稳定性差、先进封装集成极度复杂，以及全行业激光器制造产能紧缺。
- 传统网络为不可预测的离散数据包而生；AI 计算网络则由确定性的大数据流主导，天然匹配光线路交换（OCS）架构。
---
### [39:31](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2371s) Scale-up 与 Scale-out 的界限消融及异构计算之辩
当光学交换将超大规模集群的连接性能推向新高度时，计算架构中长期存在的 Scale-up（纵向垂直扩展）与 Scale-out（横向水平扩展）的界线正变得模糊。Guido 认为这两者的划分在某种程度上原本就是人为的：在现实工程中，人们使用不同的协议去区分前端网络与后端算力互连网络，但跳出具体协议、从整体系统架构的视角来看，两者的底层目标完全一致，未来必然在 AI 训练等关键负载驱动下走向收敛。Pat 也认同这一点：只要单个计算单元的 Scale-up 规模足够庞大、拓扑基数（Radix）足够高，何时将其连接到其他超大集群就变成了统一的架构设计问题，从而消解了人为分割的界线。
针对训练与推理分立的传统观念，Pat 提出了颠覆性思考：“你的大脑会停止学习吗？”他认为在未来的算法演进中，训练与推理的二元对立将逐步消退，持续学习（Continuous Learning）将成为算法的核心形态。在边缘或实际运行环境中，推理系统需要实时根据数据流动态更新模型权重以实现演化，这意味着不需要完全割裂的离散芯片架构。Guido 虽承认推理所需的通信节点半径目前小于训练，但他从另一维度指出异构化的可能：极小型的 Diffusion 模型与 15 万亿参数的巨型大语言模型（LLM）对计算和内存带宽的瓶颈截然不同。Pat 则提醒，在数据中心管理百万级 GPU 时，部署针对特定负载过度碎片化、定制化的异构芯片，会导致管理、升级、故障域（Fault Domain）隔离等运维复杂度和整体拥有成本（TCO）急剧飙升，最终丧失规模化灵活性。
- 持续学习（Continuous Learning）将促使模型在推理过程中实时微调权重，逐渐抹平专门训练芯片与专门推理芯片之间的鸿沟。
- 追求极致定制异构芯片（针对特定算力或内存敏感负载）与维持百万卡超大数据中心统一运维灵活性之间存在巨大的工程博弈。
---
### [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) 能源即经济上限：数据中心电网危机与直流供电革命
在梳理完算力、内存与网络之后，对话聚焦到制约 AI 爆发的终极物理天花板——电力能源。Pat 从宏观的第一性原理直击痛点：过去十至十五年间，美国在能源基础设施建设上表现糟糕，基本是在以增加可再生能源的同等速率淘汰煤电，导致国家整体发电装机容量几乎处于完全停滞（Flatlined）的状态。Pat 斩钉截铁地指出："在 AI 与数字时代，能源容量就等于经济容量（Energy capacity equals economic capacity）。"过去十五年国家能源容量零增长，本质上意味着国家经济潜力的停滞；尽管近五年全社会开始拼命呼吁能源扩张，但目前全国年发电容量的增速也仅有勉强 4% 左右，完全无法承载算力指数级爆发的诉求。
面对现实的能源供应链瓶颈，Pat 直言可再生能源过度依赖外部供应链，重型燃气轮机的订货交付周期（Lead Time）如今长达 8 年之久，而美国上一座商业核反应堆并网投运已是 20 年前的事。这种残酷的现实必然会成为阻碍 AI 狂热泡沫的硬着陆阻力，未来许多画大饼的数据中心项目会因为根本无法通电而陷入违约。为了突破电力瓶颈，除了需要大力重启核能等基荷电力外，数据中心内部也必须掀起供电效率革命——全面转向 800V 直流（800V DC）架构。通过砍掉无数因历史标准包袱而保留的交直流转换环节，数据中心能大幅降低线损、提升能效。正如 Pat 带着历史讽刺感所调侃的："800V 直流的普及，标志着爱迪生的复仇终于降临了。"
- 宏观断言："在 AI 数字时代，能源容量就等于经济容量。"美国过去 15 年的发电总量几乎零增长，目前增速仅约 4%/年。
- 电力设备供应链出现严重枯竭：大型燃气轮机的采购排期长达 8 年，核能产业在过去 20 年间基本陷于停滞。
- 架构变革：数据中心正加速推进 800V DC 直流供电，摒弃冗余的交直流多次变换级数，解决万卡集群在机架侧的输电瓶颈。
### [46:50](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2810s) 硬件的文艺复兴：垂直 GaN、固态变压器与热力学物理极限
Pat Gelsinger 探讨了电力转换领域的底层突破。他指出目前投资的初创公司中，垂直氮化镓（Vertical GaN）技术堪称杀手级创新，它能实现从 800V 一步降压至 48V、12V 甚至 5V 的单级电能转换。这种单步直降技术大幅削减了多级变换中的能量损耗。与此同时，固态变压器（Solid-State Transformers, SST）与全新电子开关正在彻底重构数据中心的配电网络架构（PDN）。
Pat 感叹道：“电力突然间又变酷了（power's cool again）。”伴随着配电革新的，是冷却系统的全面演进——涡轮机、新型环保制冷剂等传统机械和化工领域再次成为技术前沿。对所有做硬件的人来说，这无疑是一场崭新的“硬件文艺复兴”。主持人和现场嘉宾开玩笑说，时隔 25 年，自己居然不得不重新翻出课本去温习卡诺循环效率（Carnot efficiency）和热力学第一定律。
在芯片能效方面，Pat 尖锐指出，单纯依靠传统工艺已经无法满足算力爆发的需求。在英伟达（Nvidia）过去五代芯片架构的演进中，其单 Teraflop 算力所消耗的基础功耗几乎已经“停滞走平（flatlined）”，这在物理上是不可持续的。为了打破这一瓶颈，必须从物理底层寻找解法。例如 Playground 投资的超导计算初创公司 Snowcap，正在探索利用超导技术将能效比提升 1000 倍，从根本上改变计算的物理定律。
- **垂直氮化镓（Vertical GaN）**：支持 800V 直降至 48V / 12V / 5V，实现高倍率单步功率转换。
- **固态变压器与新型开关**：重塑数据中心底层电力输送网络。
- **能耗瓶颈**：英伟达近 5 代芯片的每 Teraflop 基础功耗未见明显改善，亟需超导等新型物理突破。
```mermaid
flowchart LR
accTitle: Power Conversion and Physical Limits Flow
accDescr: Diagram showing vertical GaN stepping down voltage directly, alongside challenges with GPUs and superconductivity solutions.
subgraph PowerConversion["电能转换革新"]
A["800V 高压输入"] -->|"垂直 GaN 单级转换"| B["48V / 12V / 5V 低压直达"]
C["固态变压器 (SST)"] -->|"重塑配电网络"| B
end
subgraph PhysicsLimits["物理与能效挑战"]
D["传统 GPU 架构"] -->|"5 代芯片能效走平"| E["热力学与功耗瓶颈"]
F["超导技术 (Snowcap)"] -->|"物理定律突围"| G["1000x 能耗性能提升"]
end
```
---
### [48:42](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2922s) 虚拟机强势归来与面向 Agent 的全新虚拟化栈
随着讨论进入尾声，主持人将话题引向 Pat Gelsinger 曾执掌八年之久的 VMware 以及如今再度火热的虚拟化技术（VMs）。Pat 认为，技术史上的每一次重大创新最终都会走向下一层抽象。无论技术如何更迭，计算机科学最底层的核心始终是数据结构、算法与抽象机制。虚拟机抽象无论位于基础设施层还是应用层，在算力层级中始终占据着不可磨灭的基石地位。现场嘉宾甚至打趣，两位前 VMware 高管（Pat 与 Raghu）是不是该回去重新掌舵 VMware。
Pat 进一步引申，当年 VMware 解决的核心问题是对计算层级的方方面面进行全栈管理——管理计算负载、调度底层网络、虚拟化存储池。而在生成式 AI 时代，系统正面临“智能体蜂群（Agent swarms）”的爆发。这就抛出了一连串崭新的系统级问题：谁来统一调度和管理这些成千上万的 Agent？谁来为它们构建安全隔离环境与安全画像（Security Profiles）？谁来实时监控并保障其运行性能？传统虚拟化与系统管理中的每一个基本单元，都必须在面向 Agent 的全新计算层级中重新被发明一遍。
- **抽象的周期性**：计算架构从底层硬件到应用，始终依赖“数据结构、算法与抽象”这三大基石。
- **Agent 管理诉求**：AI 时代面临多 Agent 并发协同，迫切需要对标传统虚拟化的全套管理、调度与安全隔离工具。
---
### [50:40](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=3040s) 从服务人类到服务智能体：重构面向 Agent 的“VMotion”
Guido Appenzeller 提出了一个深层转变：过去的虚拟机是为“人类运维与开发者”设计的，而如今虚拟机正在变成直接为“软件智能体（Agents）”服务的基础设施。这一受众的转变彻底颠覆了软件的设计与商业模式：
1. **采购与选型主体变化**：未来的基础设施选型决策可能直接由 Agent 做出，而非人类工程师；
2. **极低的延迟与启动耐受度**：人类对服务器启动等待数十秒尚有耐心，但 Agent 毫秒级的调用链条要求瞬时启动和极简复杂度。
面对“如果今天重新做一家面向 Agent 的 VMware 会如何设计”的追问，Pat 分析认为，新一代虚拟化架构必须双轨并行：一方面底层要“专为 Agent 服务”，像 Dario Amodei 提出的 Constitutional AI 那样，系统需内建安全护栏（Guardrails）与策略约束；另一方面顶层仍需为人类提供设定全局策略、规则宪法以及观测指标的 Dashboard 界面。
Pat 强调，如同当年 VMware 打造出颠覆性的动态迁移技术 VMotion 一样，AI 基础设施领域也即将诞生面向 Agent 的全新核心能力——例如针对 Agent 状态、上下文及运行环境的无缝迁移与抽象（“Agent VMotion”）。过去虚拟机攻克的技术硬核在于以极高的性能向操作系统和应用程序屏蔽硬件细节；而今天，新一代基础设施的使命则是面向 Agent 屏蔽底层异构硬件与复杂运维操作。在高度评价了这场跨越硬件与系统的深度探讨后，对谈在热烈的致谢中正式落幕。
- **极速启动要求**：面向 Agent 的轻量级虚拟化容器或运行环境需要亚秒级冷启动。
- **面向 Agent 的核心机制**：涵盖安全隔离、上下文状态迁移（Agent 版本的 VMotion）、权限护栏与底层算力抽象。
- **双层交互设计**：底层对 Agent 屏蔽异构硬件并提供运行沙箱，上层为人类管理员提供安全规范定义与策略监控能力。
## Notable quotes
> [01:00](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=60s) Smart, aggressive, arrogant. He'll fit right in. — Ron Smith (quoted by Pat Gelsinger)
> [09:28](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=568s) 你可以花三个月完成设计，但九个月内你根本拿不到规模化芯片，这太糟糕了。 — Pat Gelsinger
> [13:39](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=819s) 既然如此，你们这些风投为什么还要投这么多家？ — Pat Gelsinger
> [18:56](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1136s) 你得先去借 500 亿美元 — Pat Gelsinger
> [22:08](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=1328s) 你赚一年钱，紧接着就要赔上四年。去一个五年里只有一年盈利的行业里搞研发？这种由剧烈同质化周期驱动的市场，实在是太艰难了。 — Pat Gelsinger
> [34:46](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2086s) 大约 25 年前我就宣称铜线已死，总有一天我会是对的。 — Pat Gelsinger
> [39:31](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2371s) 你的大脑会停止学习吗？ — Pat Gelsinger
> [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) 在 AI 与数字时代，能源容量就等于经济容量。 — Pat Gelsinger
> [42:47](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2567s) 800V 直流的普及，标志着爱迪生的复仇终于降临了。 — Pat Gelsinger
> [46:50](https://www.youtube.com/watch?v=1Q_7yU7FZ1k&t=2810s) power's cool again — Pat Gelsinger
## Mentioned
Pat Gelsinger, Raghu Raghuram, Guido Appenzeller, Ron Smith, Ed McCluskey, John Hennessy, Alberto Sangiovanni-Vincentelli, Nick McKeown, Dario Amodei, Intel, Playground Global, a16z, VMware, Nvidia, OpenAI, Anthropic, Graphcore, Groq, Cerebras, d-Matrix, Snowcap, Intel 80286, Intel 80386, Intel 80486, Blackwell, NVL72, Optane, Jalapeno, VMotion, HBM, DRAM, SRAM, Flash, MRAM, OCS, CPO, NPO, EDA, Vertical GaN, Solid-State Transformers (SST)
## Video description
a16z's Raghu Raghuram and Guido Appenzeller sit down with Playground Global General Partner and former Intel CEO Pat Gelsinger to discuss the next wave of semiconductor innovation and the physical constraints shaping the AI buildout.
Drawing on his experience designing Intel's 386 and 486 processors, Pat explains how AI could transform chip design, but also why faster design alone won't solve the industry's biggest problems. They examine the bottlenecks in manufacturing, memory bandwidth, advanced packaging, and power, and why today's explosion of specialized AI chips may eventually consolidate around a smaller number of architectures.
They also discuss the potential for new memory technologies, the shift from copper to optical networking, and why energy capacity could become a major constraint on AI growth. Finally, they revisit Pat's VMware years to ask what virtualization might look like when infrastructure is built for agents rather than humans.
Timestamps:
00:00 - Intro
01:00 - From tech school to Intel at 18
05:44 - The 486 and the birth of modern EDA
07:23 - How AI changes chip design
09:18 - Why silicon still takes nine months
14:49 - Will 100 AI chips converge to a few?
22:08 - Why HBM is a hideous memory
25:15 - How tall can chips get?
42:49 - Energy capacity equals economic capacity
48:42 - A VMware for agents
Resources:
Follow Pat Gelsinger on X: https://x.com/PGelsinger
Follow Raghu Raghuram on X: https://x.com/RaghuRaghuram
Follow Guido Appenzeller: https://x.com/appenz
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://twitter.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.
[Source](https://www.youtube.com/watch?v=1Q_7yU7FZ1k)
Source quotes
[1:00] Ron Smith (quoted by Pat Gelsinger): Smart, aggressive, arrogant. He'll fit right in.
[9:28] Pat Gelsinger: 你可以花三个月完成设计，但九个月内你根本拿不到规模化芯片，这太糟糕了。
[13:39] Pat Gelsinger: 既然如此，你们这些风投为什么还要投这么多家？
[18:56] Pat Gelsinger: 你得先去借 500 亿美元
[22:08] Pat Gelsinger: 你赚一年钱，紧接着就要赔上四年。去一个五年里只有一年盈利的行业里搞研发？这种由剧烈同质化周期驱动的市场，实在是太艰难了。
[34:46] Pat Gelsinger: 大约 25 年前我就宣称铜线已死，总有一天我会是对的。
[39:31] Pat Gelsinger: 你的大脑会停止学习吗？
[42:47] Pat Gelsinger: 在 AI 与数字时代，能源容量就等于经济容量。
[42:47] Pat Gelsinger: 800V 直流的普及，标志着爱迪生的复仇终于降临了。
[46:50] Pat Gelsinger: power's cool again
Source quotes
[0:00] Pat Gelsinger: energy capacity equals economic capacity
[0:00] Pat Gelsinger: nothing's a chip anymore, it's a rack
[1:02] Ron Smith (quoted by Pat Gelsinger): Smart, aggressive, arrogant. He'll fit right in.
[11:56] Pat Gelsinger: HBM is a hideous memory. It's just the best one that we [have].
[18:56] Pat Gelsinger: Okay. Well, could I borrow 50 billion, right? I mean, you're not talking about small capital expenditures. You're talking about very large capital expenditures and data centers, power commitments and so on like that. Well, I got to build this thing at scale... and that's going to defy a lot of heterogeneity.
[22:08] Pat Gelsinger: Yeah, that one died... We killed it.
[24:44] Pat Gelsinger: AI is a memory workload
[28:06] Pat Gelsinger: If I have a cracked die, I don't care how much resilience you've built into it. You can't, you know, it's a bad die.
[31:09] Pat Gelsinger: We engineers are becoming plumbers
[34:46] Pat Gelsinger: engineering marvel and a manufacturing nightmare
[40:21] Pat Gelsinger: Does your brain stop learning?
[45:27] Pat Gelsinger: The revenge of Edison is upon us.
Original transcript
[0:00] In a AI digital age, energy capacity
[0:03] equals economic capacity. Why build a
[0:05] new data center and buy the million GPUs
[0:08] if I can't power them? You're going to
[0:10] see more and more defaults happening on
[0:13] many of those data center projects cuz
[0:15] the energy won't be there.
[0:16] >> Whenever you have the technology to make
[0:18] something easy, that means the
[0:19] bottleneck moves somewhere else.
[0:21] >> Nothing's a chip anymore. It's a rack.
[0:23] It took me 3 months to design it, but
[0:24] it's 9 months until I can actually start
[0:26] to use it. Exactly. How many major new
[0:28] memories have we had over the last 30
[0:30] years?
[0:31] >> Zero.
[0:31] >> Zero.
[0:32] >> Memory innovation for the first time in
[0:34] 30 years is nigh upon us. I declared the
[0:37] death of copper about 25 years ago.
[0:39] Eventually, I'll be right for all of us
[0:40] hardware guys. This is like renaissance
[0:43] in front of us. AI inference accelerator
[0:45] chips. And I'm sure I don't even know
[0:47] them all. Why are you guys funding so
[0:48] many of those?
[0:49] >> You funded your fair share, too.
[0:52] >> Historically, there have not been a 100
[0:53] competing processor vendors in any
[0:55] industry ever. This is a temporary
[0:57] thing. It'll converge back to a view.
[0:59] >> I see it as a
[1:02] >> I'm here with uh our esteemed guest and
[1:05] dear friend and former boss, Mr. Pat
[1:08] Gellinger. Welcome Pat. Pat is currently
[1:12] the general partner at uh at Playground
[1:16] Global, but uh as uh you're very very
[1:19] well known in the industry for leading
[1:22] Intel, for being the CTO of Intel, of
[1:25] course, leading VMware and many other
[1:26] things.
[1:27] >> So, welcome.
[1:28] >> Hey, thank you, Rayu. Great to be with
[1:30] you and Guido, right? And to me, this
[1:32] feels just a little bit like old home,
[1:33] right? You know, it's like, you know,
[1:35] you know, it was like uh we superimposed
[1:37] a year of change, right? But uh you both
[1:39] look the same. I feel uh energetic. So
[1:42] let's dive in.
[1:43] >> There has never been a day when you
[1:44] stopped being energetic. So there no
[1:46] news there. Uh but yeah. No, let's start
[1:49] actually from uh from your Intel career,
[1:53] right? I mean recently Andre Horovitz
[1:57] created this Horowitz and recreent
[1:59] academy
[2:00] >> which was to find talented people
[2:02] between 16 and 22 and then put them into
[2:06] a a modern educational setting so they'd
[2:10] be well prepared for either doing things
[2:13] on their own or joining the companies of
[2:14] today. You had a similar experience like
[2:18] you went to a regular trade school and
[2:19] then jumped into Intel.
[2:21] >> Yeah. Yeah. When we were what 18 or 19
[2:23] is
[2:24] >> Yeah. Uh, you know, it was really a sort
[2:26] of magical period. You know, I'm 16
[2:28] years old. I accidentally win a
[2:30] scholarship. I go to tech school, right?
[2:32] Skipped my last year and a half of high
[2:34] school and Intel comes recruiting when
[2:36] I'm 18 years old. And, uh, you know, I
[2:39] never been on an airplane. Uh, had
[2:41] already fallen in love with computers at
[2:43] tech school. And, uh, the interviewer,
[2:45] right, Ron Smith was his name. He writes
[2:47] on his page. And I was number 12 that he
[2:49] interviewed that day. And if you've
[2:51] interviewed 12 people in a row, you
[2:53] can't tell male from female, you know,
[2:54] by the end of it, right? You know, I'm
[2:56] number 12. He says, "Smart, aggressive,
[2:58] arrogant. He'll fit right in." So, I got
[3:01] invited to go, you know, come to uh
[3:04] Intel and, you know, just became uh, you
[3:06] know, really uh glorious. uh you know
[3:09] starting as a technician moved into the
[3:11] design team at the end of the 286 you
[3:14] know engineer number four on the 386 you
[3:18] know uh architect and design manager for
[3:21] the 486 all of that while doing my
[3:23] master my bachelor's my masters and PhD
[3:27] work so it was like the the the best
[3:29] career that you could possibly have
[3:31] you're learning by you know day and
[3:33] you're putting it to practice at night
[3:35] and
[3:35] >> that's right that's right you're
[3:36] creating new completely new era of chips
[3:39] for I bet most of what you learned
[3:43] and what you built there was a big gap
[3:45] because you're breaking new ground.
[3:47] >> Yeah. In a lot of ways and uh you know
[3:49] really was a pretty incredible time in
[3:51] the industry and I remember you know one
[3:53] of the professor classes uh you know he
[3:56] uh postulated a new carry look ahead
[3:59] adder design right in one of my classes
[4:01] at Stanford. So I said well I'm working
[4:03] on the 386 carry look ahead adder so let
[4:05] me go try it. So I come back two days
[4:08] later to the professor and I said it
[4:09] doesn't work right and I started
[4:11] argument with my you know professor at
[4:13] Stanford and it was like you know it was
[4:15] like what you know the next two class
[4:17] periods were just me and the professor
[4:18] arguing over why his design didn't work.
[4:21] >> He had a book that was being published
[4:22] what that design is. So he was he was
[4:24] very annoying. It was very time
[4:26] sensitive for him
[4:27] >> and you know we started builtin self
[4:29] test right you know and I remember Ed
[4:32] McCcluskey at Stanford was my professor
[4:34] and he was sort of the father of that
[4:36] and we you know had wonderful arguments
[4:38] around why most of his ideas were bad
[4:40] they weren't practical for real chips
[4:42] but I was putting you know builtin self
[4:43] test to work on the 36 you know John
[4:46] Hennessy uh you know was my thesis
[4:48] advisor and obviously he did okay I say
[4:51] you know I helped his career he became
[4:53] president and he's done okay since then
[4:55] Yeah. Yeah. Yeah.
[4:56] >> Later on we'll talk about the CIS versus
[4:57] risk debate, but I want to talk about
[4:59] the 486. Sorry. Go ahead.
[5:02] >> I had the same experience when I was
[5:03] teaching at Stanford. You know, you you
[5:05] come with something that you think is
[5:06] the truth and then the student comes
[5:08] that works in industry is like, well,
[5:10] actually not quite. So, you know, the
[5:13] next lecture you come back with like,
[5:14] okay, uh, I looked this up. You're
[5:17] right. You know, this is this is
[5:18] outdated.
[5:19] >> Yeah. And I tell you, you know, that's
[5:20] part of the set of students. Yeah. that
[5:21] interaction and I think that's part of
[5:23] what makes Silicon Valley so unique.
[5:25] Yeah.
[5:25] >> Right. You know is you have you know
[5:26] these so many companies and you know
[5:29] entrepreneurship it's like in the water
[5:31] here right you know starting things up
[5:33] and new ideas and challenging the
[5:35] professors and you can't be a professor
[5:37] at a place like Stanford and not expect
[5:39] to get challenged and enjoy it.
[5:41] >> There's a good foot traffic in and out
[5:42] of that ivory tower.
[5:44] >> Yeah I think it's good.
[5:46] >> Yeah. Now 486 was your big
[5:50] >> Yeah.
[5:51] >> accomplishment if you will. Obviously
[5:52] lots of people worked on it. Um
[5:56] and you guys broke some new ground in
[5:58] chip design there, right?
[5:59] >> Mhm.
[6:00] >> And um increased what functions the chip
[6:03] was supposed to do and so on and so
[6:04] forth.
[6:05] >> Yeah. It was a pretty magic period uh as
[6:07] well because it was before what you
[6:09] would think of as the EDA industry.
[6:12] >> Oh wow. Yes, that's right. and uh you
[6:14] know the the end of the a little bit of
[6:16] the 386 but the 486 was really the first
[6:21] chip to implement what what you would
[6:23] think of as modern EDA techniques. Yeah,
[6:27] we did a highle design, right? Uh
[6:30] description and RTL description, but
[6:32] there was no vera log.
[6:33] >> So, we invented HDL, right? The Intel
[6:36] hardware description language. So, you
[6:38] know, I wrote my own language. Well, you
[6:40] had to build a compiler for that
[6:41] language, right? You know, so we created
[6:43] the compiler for the language. There was
[6:45] no automatic place in route, you know.
[6:47] So, we had to invent that. And we worked
[6:49] with Alberto Sanji Yavanji Vincentelli
[6:52] at Berkeley and some of his students for
[6:54] the first you know placement the first
[6:56] routing the first automated timing you
[6:59] know management and
[7:00] >> so they created EDA to some degree. to a
[7:03] great degree that uh you know and that
[7:05] was really one of the hallmarks that not
[7:07] only was the 486 you know this
[7:09] compatible you know pipeline
[7:11] microprocessor but we ushered in you
[7:14] know many of the foundations of what
[7:15] became the modern EDA uh industry and
[7:18] you know it was really you know pretty
[7:20] magical period of the industry.
[7:22] >> Yeah. Yeah. So if you look at today
[7:25] >> Yeah. Will people look back as Blackwell
[7:28] and saying you know this was of the last
[7:29] generation that was was done before AI
[7:32] took over the design process for for
[7:34] micros are we in a similar transition
[7:37] right now
[7:37] >> I think there are certainly aspects of
[7:39] that and uh you know I think you know
[7:42] when you look at you know things like
[7:43] jalapeno today right you sort of say you
[7:46] know that you know that's that's sort of
[7:47] a first principles use of AI in the chip
[7:50] design process right where you throw out
[7:52] a lot of those things now you know un
[7:55] You know there are pieces of design that
[7:57] are still really hard right uh in that
[8:00] sense and a lot of the analog right
[8:02] aspect certis is probably the best
[8:04] example of that you know are still not
[8:06] aiable right you just need lots of
[8:09] silicon data right to get those so you
[8:11] either get so conservative in your
[8:13] analog design you know that you're able
[8:15] to I'll say AI it uh or right you got to
[8:19] do those the hard way uh you know for
[8:22] but to now many of the logic functions
[8:24] can really be done with uh AI tools and
[8:27] techniques in pretty incredible ways,
[8:30] right? Your transistor budgets are big
[8:31] enough, the design tools are getting
[8:33] smart enough, you know, a few experts
[8:35] guiding the tools and how to apply it.
[8:37] And I really think it will be somewhat
[8:39] like the 46 that way, right? Where you
[8:41] sort of look back and say, "Yep, that
[8:42] was the beginning of a new era of chip
[8:45] design."
[8:46] >> Whenever you have the technology to make
[8:48] something easy, that means the
[8:49] bottleneck moves somewhere else.
[8:52] >> Yeah. Can we guess at this point where
[8:54] the bottleneck will be in the future?
[8:56] >> There's some things which it seems like
[8:57] AI is incredibly good at, right? Like
[8:59] creating the software layers, you know,
[9:01] writing kernels, um, you know, a lot of
[9:03] the
[9:05] >> I think a lot of the sort of being able
[9:07] to specify the the the objectives at a
[9:10] higher level which then gets sort of
[9:11] translated into very log right and so
[9:14] what is the new frontier? What is the
[9:15] thing that that will be difficult going
[9:17] forward? Well, you know, when you look
[9:19] at today, you know, so let's say, you
[9:21] know, Guido and Pat, we're off to do a
[9:23] great new chip together. We're building,
[9:25] >> right? You know, right? You know, and
[9:26] it's the one that's going to leap ahead
[9:28] because we have understanding of 8646,
[9:32] >> you know, we have understanding of the
[9:34] AI workloads. We have understanding then
[9:36] of how to compose that into the right,
[9:38] multiply, accumulate, you know, register
[9:40] structures,
[9:41] >> the perfect visibility model workloads
[9:42] of tomorrow as well,
[9:44] >> you know, and all of that kind of stuff.
[9:46] And you know let's say you know we turn
[9:47] our AI you know tools loose we are great
[9:50] within three months we have an awesome
[9:53] design right you know you know for that
[9:56] we've been a little bit conservative on
[9:57] all the analog components of it
[10:00] >> you know now we still have like a couple
[10:02] of really major bottlenecks today you
[10:05] know one is right we still have nine
[10:07] months of silicon processing time
[10:10] >> right you know someone like that right
[10:11] so I can design the thing in three
[10:13] months but I can't actually get it into
[10:15] real silicon at scale for 9 months.
[10:17] Okay, that sucks, right? So, we have to
[10:19] really see that bottleneck improve if
[10:21] we're going to have this innovation
[10:23] because we know that
[10:23] >> about the validation verification stages
[10:26] or even beyond that. Well, you know,
[10:27] first I can't get anything out of fab.
[10:29] Yeah. You know, in less than three
[10:31] months and that's if I have, you know,
[10:32] like supercharged design flows, right?
[10:35] So, I said, you know, you know, how do I
[10:36] compress that, right? Uh, you know, for
[10:39] it and then I got to get into advanced
[10:41] packages. 3D packages are getting
[10:43] complex. So, that's like another three
[10:44] months, right? You know, and then I have
[10:46] to put it into a rack scale solution
[10:48] because nothing's a chip anymore. It's a
[10:50] rack, you know. So we're 9 months, you
[10:52] know, it took me 3 months to design it,
[10:53] but it's nine months until I can
[10:55] actually start to use it, you know. So
[10:57] how do we start to compress, you know,
[10:59] those aspects of design? And I think we
[11:01] need new forms of lithography, you know,
[11:03] to enable that. We need, you know,
[11:06] design flows that don't require, you
[11:08] know, $50 million of mass cost until I
[11:10] can get things into prototyping. So you
[11:12] know, to me, how do you compress that to
[11:14] a month or two, right? Because I can now
[11:16] do the design in three months. How can I
[11:19] have that done in a month or two?
[11:20] Because if it takes me a year and a half
[11:22] until I actually get scale and software
[11:24] on it,
[11:25] >> okay, my understanding of the AI
[11:27] workloads is no longer applicable to the
[11:30] chip that I designed.
[11:31] >> Right. You know, so you know, all of
[11:33] those
[11:33] >> you see that play out live right now
[11:35] with AI. Yeah.
[11:36] >> Right. You said, you know, it's almost,
[11:37] you know, we can look at it
[11:38] historically. It was like, you know,
[11:39] Graph Core wasn't a bad design, but the
[11:41] world moved on,
[11:42] >> right? and you know uh in that. So all
[11:46] of those aspects somewhat obviate the
[11:48] fact that AI has made the design piece
[11:52] easy. All of those manufacturing scale
[11:54] issues to me become a huge bottleneck.
[11:56] Right. Another bottleneck of course is
[11:58] memory.
[11:59] >> Yep.
[12:00] >> Right. You know memory bandwidth
[12:01] >> we noticed. Yeah.
[12:02] >> Right. And you know for that as I've
[12:04] described HBM is a hideous memory. It's
[12:07] just the best one that we
[12:08] >> ask you about that. Yes.
[12:09] >> You know it's a terrible memory. You
[12:10] know I get terrible bit density. I'm
[12:12] limited to shoreline bandwidth the power
[12:15] right uh issues thermal issues DRAM
[12:18] doesn't like to be hot
[12:20] >> concentrating everything in one spot
[12:22] >> you know it's terrible to give
[12:25] >> that AI is a memory compute workload yes
[12:29] >> right so right you know for that so I
[12:32] you know I need to you know compress
[12:34] that development uh and embodiment into
[12:37] real scale you know I need to fix memory
[12:40] >> right uh associated with So to me that's
[12:42] another critical bottleneck for us to
[12:44] attack. Uh and uh power right you know
[12:48] power in you know thermal uh and most of
[12:51] the power simulation aspects today are
[12:53] bad
[12:54] >> right you know you don't have good 3D
[12:56] modeling tools for power power
[12:58] dissipation hotspots uh etc. you know my
[13:01] voltage in right you know I'm
[13:03] essentially guard banding almost my full
[13:05] power envelope correct
[13:06] >> right so essentially I'm losing you know
[13:09] 40% of my power and guard banding so I
[13:12] got to get much better at power modeling
[13:14] you know and then my power dissipation
[13:16] techniques are bad right so you know
[13:18] >> delivery model has to change too
[13:20] >> yeah so all of these things you know to
[13:22] me those are the you know the next big
[13:24] bottlenecks right you know how do I you
[13:26] know improve each one of those and now
[13:28] right you know if I've done that Okay,
[13:30] now we're on to an exciting era for
[13:32] what's next. So, you know, thank you for
[13:34] those great AI tools, but you know, I
[13:36] got big challenges even when those are
[13:38] fully deployed.
[13:39] >> And so,
[13:42] we've seen sort of a rebirth in the
[13:44] number of chips and chip architectures
[13:46] and so on and so forth.
[13:47] >> Yeah.
[13:48] >> Thanks to these workloads.
[13:49] >> Yeah. It's like there's a 100 AI
[13:51] inference accelerator chips and I'm sure
[13:53] I don't even know them all.
[13:54] >> Yeah, exactly. Yeah. So, so
[13:58] >> why are you guys funding so many of
[13:59] those anyway?
[14:02] >> You funded your fair share too.
[14:06] >> Yeah. So, um, so are we, you know,
[14:10] they're all to their credit making
[14:11] different design assumptions either in
[14:13] the components that you talked about or
[14:15] >> in the
[14:16] >> in the part of the para curve that
[14:18] they're targeting. So do you think
[14:20] there's room for all of these or are we
[14:22] in a world where partly because of what
[14:25] Guido talked about AI tooling
[14:28] um we now have the ability to build more
[14:32] different kind of chips.
[14:33] >> Yeah.
[14:34] >> And the time is enormous whichever way
[14:36] you want to even more provocatively
[14:38] historically there have not been a 100
[14:40] competing processor vendors in any
[14:41] industry ever right. So
[14:44] >> is this a more heterogeneous future?
[14:45] Will we see is this a temporary thing?
[14:48] It'll converge back to a view.
[14:50] >> Yeah. And you know, I see it as a more
[14:52] temporary thing and it will converge is
[14:54] is what I expect. And there's probably
[14:56] three different reasons that I see that
[14:58] to be the case. You know, one of them
[15:01] is, you know, this emerging
[15:04] heterogeneity that you see in the
[15:06] compute uh environment where okay, now I
[15:09] have specialized prefill versus decode.
[15:12] Well, you know, and now people are
[15:14] saying, well, we really need a
[15:15] specialized tier for midfill, not just
[15:17] prefill, right? You know, so now I have
[15:19] early prefill and now I have midfill,
[15:21] which is, you know, differential.
[15:22] >> Maybe speculate and verify if we listen
[15:25] to OpenAI here, right?
[15:26] >> Yeah. You know, and you know, all of a
[15:28] sudden my compute fleet becomes more and
[15:31] more granular across the workload. And I
[15:34] think anytime that you've seen that in
[15:36] history is not sustainable. And then all
[15:38] of a sudden right when you know as you
[15:41] look at that and now people are saying
[15:43] well as I go to reasoning models I want
[15:45] something that looks more like a CPU
[15:46] again. So you know all of a sudden I
[15:49] didn't want that level you know that
[15:50] didn't become the predominant portion of
[15:52] my compute capacity. I needed more of it
[15:55] over here, right? and model and model
[15:57] workflows and you know so on you know so
[16:00] I really don't like I'll say at scale
[16:04] specialization right because I think the
[16:06] workloads are going to continue to
[16:08] moderate you know and migrate so
[16:10] significantly first you know second is I
[16:13] think today's models are about to go
[16:15] through some evolutionary breakthroughs
[16:16] as well right you know I think the LLM
[16:19] is sort of reaching the limits and now
[16:21] as people look to how do I really model
[16:23] 3D right you know where you a flattish
[16:26] LLM doesn't work well when you go to
[16:29] molecules and chemicals and you know
[16:32] imaging you know so I think we'll see
[16:33] limits of that that'll cause different
[16:35] shifts in the algorithmic domain I also
[16:38] think that uh some of the most
[16:40] interesting workloads become where you
[16:41] start you know I'll say bringing HPC
[16:44] like things back into AI like things
[16:47] where all of a sudden you know 64-bit
[16:49] provision uh precision counts again
[16:52] right so I see
[16:52] >> what would be an example of that
[16:54] >> well you know imagine that my AI model
[16:56] is now inducing the five most
[16:59] interesting domains for a chemical
[17:01] analysis,
[17:02] right? Okay, those chemical algorithms
[17:04] are going to run with high precision,
[17:06] right? You know, when you know which is
[17:08] going to be the dominant piece of the
[17:09] workload, right? Getting to the five
[17:12] algorithms I need to run or running the
[17:14] five algorithms, right, on different
[17:16] chemical or biological systems. So you
[17:19] know I do think that uh you know or as
[17:21] we optimize our AI systems they will
[17:23] lead us back to some of the things that
[17:25] have been more traditional high
[17:27] performance computing. So I see that
[17:29] aspect of workload uh as well. So you
[17:32] know all of that said is you know this
[17:34] extreme specialization to me in the
[17:36] compute architect I I sort of don't like
[17:38] it because I don't think it's going to
[17:40] be what the workloads will look like 2 3
[17:43] four years from now. So that's one
[17:45] reason I don't see all of these AI chips
[17:47] as they get more and more specialized
[17:49] for portions of the compute workload to
[17:52] necessarily be right you know second is
[17:54] right there's like a hundred of them now
[17:56] >> right you know there's multiple optical
[17:58] ones there's probabilistic ones and so
[18:00] on and you know they're not all going to
[18:01] win and they're not all going to win
[18:03] because at the end of the day you have
[18:05] to get scale
[18:06] >> on these things and scale requires you
[18:08] know you have to win you have to get
[18:09] capital you have to get workloads onto
[18:11] it so I think it sort of defies logic
[18:13] that you're going to have a hundred of
[18:15] these things. So I see them you know
[18:17] coming back uh as a results of you know
[18:20] simple you know capital market share
[18:22] etc. you know which are the winning
[18:23] teams you know winning designs winning
[18:26] architecture so I see the workload
[18:28] driving that you know I see the natural
[18:31] you know industry consolidations you
[18:33] know that will occur and then I also
[18:35] expect that the winners will pick some
[18:37] winners
[18:39] >> right you know where you know an open AI
[18:42] an Nvidia anthropic will say I like this
[18:45] one because it isn't just hardware it's
[18:49] how do the hardware and software code
[18:51] evolve Yeah,
[18:52] >> right. You know, for it and I do expect
[18:54] that there's going to be h, you know,
[18:56] here's this 10 I could pick for this. I
[18:59] like that one. And it does require
[19:01] investment in the software and the
[19:03] workload evolution to take advantage of
[19:05] those platforms. So for those three
[19:07] reasons, yeah, we're going to see, you
[19:09] know, a narrowing of the field uh in
[19:11] that sense
[19:12] >> and the deployment complexity as well.
[19:14] Sorry, go ahead.
[19:14] >> I mean, makes total sense and I think I
[19:16] agree in most point. Let me still try to
[19:18] stir a little bit controversy here just
[19:19] to to stirring controversy to you guys
[19:24] have changed it right
[19:26] >> so so obviously 100 companies won't
[19:29] access consolidate right and and
[19:31] completely agree with your argument that
[19:33] at the end of the day the large
[19:34] consumers will pick winners to some some
[19:36] degree right today I think
[19:38] >> to be a first tier chip startup to some
[19:40] degree whether you you're that or not is
[19:43] defined by how much fraction you have
[19:45] with with the with the really big Nice.
[19:47] That said, um, and there's of also an
[19:50] argument to say like, look, in any kind
[19:51] of transition period, you probably have
[19:52] a bit more of an explosion of different
[19:53] approaches over time that that'll shake
[19:55] out.
[19:55] >> Now, all that said, I think there's an
[19:58] argument you can make that basically
[19:59] says, look,
[20:00] >> historically, we standardized on, you
[20:03] know, one or two architectures because,
[20:05] you know, different instruction sets
[20:06] were hard. we had to build multiple
[20:07] compilers you know like the this
[20:09] basically um this the the software
[20:11] complexity dictated that you can only
[20:13] have a certain small number of hardware
[20:15] platforms
[20:16] >> that part seems to be changing with AI
[20:18] right today you know writing an
[20:20] optimized kernel for I mean I think was
[20:23] a great example where they basically
[20:24] said look humans can't really program
[20:25] this thing anymore but an agent swarm
[20:27] overnight can do it that's all we need
[20:29] right so where basically you can say
[20:30] like if you can build a chip that has a
[20:32] certain capability it's now much much
[20:34] easier to build the software layers on
[20:36] top that fully leverages that could a
[20:38] result of that be that we have a little
[20:40] bit more hetrogenity just because the
[20:42] you know it's it's easier to program
[20:43] that hetrogenity going forward
[20:44] >> and and I I do think it changes some of
[20:47] you know you you know the you know where
[20:49] the bottleneck lands right changes in
[20:51] that regard you know the first
[20:52] bottleneck is the one I'd point to that
[20:54] we've already talked about still takes a
[20:56] year and a half to build these things
[20:58] >> right you know and then the second
[20:59] bottleneck is not only does it take a
[21:01] year and a half to get these things into
[21:02] racks the scales and so on like that now
[21:04] to make them useful full. Okay. Well,
[21:07] could I borrow 50 billion, right? I
[21:09] mean, you're not talking about small
[21:11] capital expenditures. You're talking
[21:13] about very large capital expenditures
[21:15] and data centers, you know, power, you
[21:18] know, commitments and so on like that.
[21:19] Well, I got to build this thing at
[21:21] scale, right, associated with it. So,
[21:23] and that's going to defy a lot of
[21:24] heterogeneity uh as a result. Now, you
[21:27] know, clearly big players and we saw
[21:28] this with what, you know, Nvidia did
[21:30] with Grock, okay? you know, we're gonna
[21:32] sort of own some of that heterogeneity,
[21:35] right? And you know, you know, two years
[21:37] from now, you're not going to tell what
[21:38] Grock is inside of Nvidia, right? It's
[21:40] just going to be like another, right?
[21:43] You know, someone like that. So, you'll
[21:44] see the architectures, you know, sort of
[21:46] hide the heterogeneity, right? Uh inside
[21:49] of it, which to me is just fine, right?
[21:50] You know I mean that's what layering
[21:52] that's what abstraction has always done
[21:53] associated with it you know but I think
[21:55] those forces are powerful ones to I'll
[21:58] say expose
[22:00] a lot of the heterogeneity no right I
[22:03] think it will get abstracted underneath
[22:05] you know some of these but at the end of
[22:06] the day it's still hard to build these
[22:07] things at scale
[22:08] >> earlier you talked about HPM being
[22:10] hideous memory
[22:12] >> by far the best one we have today.
[22:14] >> Exactly. It's like the thing about
[22:15] democracy.
[22:17] >> Um,
[22:18] we can't recall when there was the last
[22:20] memory innovation. It might have been
[22:22] obtained. I think you probably had
[22:24] something to do with it.
[22:24] >> Yeah, that one died. So,
[22:26] >> that one died.
[22:27] >> We killed it.
[22:28] >> Yeah, we killed it. We killed it.
[22:30] >> So,
[22:31] do you think memory innovation
[22:34] is around the corner or what needs to
[22:36] happen for obviously we understand the
[22:39] scale problem because it's even more
[22:41] >> Yeah.
[22:41] >> larger there.
[22:42] >> Yeah. But if you keep the scale aside
[22:44] even from a technology point of view.
[22:46] >> Yeah. And let's you know uh you know
[22:48] look at that history. I've probably
[22:50] personally been associated with at least
[22:53] five different new memory architectures.
[22:56] Optane just being one of them right you
[22:58] know that did not see the light of day
[23:00] right uh as well. Uh I'm probably
[23:03] familiar with close to a hundred that
[23:05] have happened in the industry over the
[23:07] last 30 years. And exactly how many
[23:10] major new memories have you had over the
[23:11] last 30 years?
[23:13] >> Zero.
[23:13] >> Zero.
[23:14] >> Right. You know, which you know, okay.
[23:15] DRAM, SR, RAM, flash. Okay. What else is
[23:18] there? D RAM, SRAM, flash. Yeah. You
[23:20] know, so I I do think
[23:21] >> flash stacking console a little bit.
[23:23] >> Barely. Yeah. Right. You know, but
[23:25] anyway. Right.
[23:26] >> So, uh, you know, in that it it you
[23:29] know, it has been right you know, a a
[23:32] disappointing
[23:33] >> that we haven't created, you know, the
[23:35] next physics. Yeah. that enable you know
[23:38] lowcost high performance resilient
[23:42] manufacturable memory right you know for
[23:45] it so I do think and and now and and
[23:48] over the last 30 40 years you know
[23:51] memory was a terrible industry
[23:53] >> right because you would make money one
[23:55] year and then you lose money for the
[23:56] next four right so let's go do R&D in an
[23:59] industry that makes money one out of
[24:01] five years right you know it's just
[24:03] always you know it's just been so
[24:05] terribly difficult because it's been
[24:07] such driven by such violent
[24:09] commoditization cycles.
[24:11] >> That has changed a little bit though
[24:12] recently
[24:12] >> over the last 5 years.
[24:16] >> Happy days, man.
[24:18] >> So So all three big memory vendors are
[24:20] on the in the top 20 most valuable
[24:22] companies on the planet.
[24:23] >> Yeah, that's a crazy change, isn't it?
[24:24] >> Oh, it's just so right. you know who who
[24:27] would have thought right uh associated
[24:29] with it and anybody who looks at the AI
[24:32] workloads and it's always you know it's
[24:33] much harder to you know you know it's
[24:35] easy to predict the past but it's a
[24:36] little bit harder to predict the future
[24:38] right and you know in that you look at
[24:40] it and say AI is a memory role
[24:43] >> exactly
[24:44] >> right you know inside of it so I think
[24:46] there is the combination of capital and
[24:50] technology needs that will justify
[24:53] memory innovation now there's two
[24:55] domains things that I'm excited about uh
[24:58] you know for it you know one is you know
[25:00] this idea you know I do think that the
[25:01] shoreline bandwidth limitations of HBM
[25:04] is a fundamental issue
[25:06] >> right so with that the idea you know and
[25:08] obviously you know we have a company
[25:09] Dmatrix that's doing this but I think
[25:11] others will be you know whether it's you
[25:13] know Cabbrris you know and their
[25:15] techniques
[25:15] >> the future is stacked yeah
[25:17] >> somehow you got to bring memory and
[25:19] >> in more ways than one
[25:21] >> you know you got to bring memory and
[25:22] compute together
[25:23] >> right you know in a fundamental until uh
[25:26] new structures. Um I do think that uh
[25:30] given the workload now has such capital
[25:33] being deployed against it you know in
[25:35] the AI space that I do think we'll
[25:37] actually be able to break through some
[25:39] of those physics challenges as well uh
[25:41] as well. So I do believe and you know
[25:44] I've just substited
[25:47] a new memory company as well that's
[25:48] still a stealth but uh you know you know
[25:51] I'm quite excited about some of the
[25:53] innovations that will occur in new
[25:55] memories and you're going to see new
[25:57] materials uh you know for memory people
[25:59] you know a lot of people looking at
[26:00] things like ferro electrics uh you know
[26:03] finding non-capacitive you know uh
[26:05] highdensity memories finding memories
[26:08] that can you know be high performance
[26:10] high density and stackable right but you
[26:13] know thing you know people are looking
[26:15] at uh lowcost uh flash you know for that
[26:18] I'm not particularly enthusiastic about
[26:20] that idea myself I think you know speed
[26:22] and performance of the underlying cell
[26:24] are somewhat problematic with a flash or
[26:26] an MRAM structure you know but I do
[26:29] think there's memory innovations are
[26:31] coming for the first time in 30 years
[26:33] and I do think that's going to be an
[26:35] exciting space and you know we'll have a
[26:36] couple of companies there I'm sure
[26:39] hopefully we'll do a few of them
[26:40] together
[26:42] What do you think? Should we do Okay.
[26:43] You know, but we'll we'll find some
[26:45] things to do here. But I think memory
[26:47] innovation for the first time in 30
[26:49] years is nigh upon us.
[26:51] >> Yeah. Yeah.
[26:52] >> It's it's amazing what a couple of$10
[26:53] billion of market cap. I don't even
[26:56] know. You say a trillion. Okay. Sorry.
[26:59] >> You know, I think the memory industry
[27:01] has now increased by market cap by $2
[27:03] and a half trillion dollars. That's
[27:04] great. You know, over the last four
[27:05] years,
[27:06] >> I think that's about right.
[27:08] That is Yeah,
[27:09] >> we could have just invested there and
[27:10] not done any forential stuff.
[27:12] >> Yeah, exactly. Use your money. Um,
[27:16] how long what's your view? Sorry, go
[27:18] ahead.
[27:19] >> Yes. To to expand a little bit on the
[27:21] stacking part if you have to predict h
[27:25] how quickly will our chips get get
[27:27] taller? Is this going to be like, you
[27:29] know, we're going to go to two and then
[27:30] sit there for a while or will we have,
[27:32] you know, the 16 layer sandwich, you
[27:34] know, happily mixing logic, memory,
[27:37] >> pulled down this
[27:40] HPM is going down at Nvidia, but how how
[27:42] do you see this evolve? Yeah, you know,
[27:44] I think there's um um you know, the
[27:47] problem with stacking, right, is now
[27:50] essentially the value of the stack,
[27:53] right, becomes greater as you go to
[27:55] higher dimensionality of the stack,
[27:56] which means the yield of any individual
[27:59] layer in the stacking process has to
[28:02] keep coming, you know, essentially it
[28:04] can't go up linearly. Yep. Right. you
[28:06] know, it has to go up exponentially and
[28:08] improving the manufacturing and defect
[28:10] characteristics in an exponential way.
[28:13] You know, when you get to eight or 16
[28:15] high stacks gets to be so so hard when
[28:18] you run the math on that uh associated
[28:20] with it. You know, your manufacturer
[28:22] flows have to be so close to perfect.
[28:24] >> Although you could see designs that take
[28:26] into account a certain model cost like
[28:28] we've seen it for like a long time for
[28:30] two dimensions.
[28:30] >> Yeah, of course. But remember that was
[28:32] the nature of DRAM. they always have bad
[28:33] block replacement and you know spare
[28:36] lines and so on like that. So I'm not in
[28:38] any way, right, you know, minimizing
[28:40] that, but assuming you're doing that,
[28:42] you still need the yields to be so good.
[28:44] You know, if I have a cracked die, I
[28:45] don't care how much resilience you've
[28:47] built into it. You can't, you know, it's
[28:49] a bad die, right? So, you know, those
[28:50] kind, you know, uh, so as a result of
[28:53] that, I'm not, you know, a crazy 16 32
[28:58] stack, you know, kind of guy. I think
[28:59] there's going to be, you know, nice
[29:01] three, four, five stacks, you know, that
[29:03] just end up sort of being the sweet
[29:05] spot.
[29:05] >> The mid-rises. Yeah. Yeah. Right. Uh you
[29:08] know for and I and I expect for you know
[29:10] part of its manufacturing part of its
[29:11] complexity uh associated with it you
[29:14] know where's those yield sweet spots
[29:16] associated with. So that's sort of where
[29:18] I think most things land. But even there
[29:21] right you know so let's say I now have
[29:23] you know I now have a great compute tile
[29:27] right you know how big does that tile
[29:28] need to be? I mean everything's become
[29:30] reticle limited now right? you know hm
[29:32] can I go to more chiplets that have
[29:34] better yield characteristics you better
[29:36] thermal you know and then you're going
[29:38] to say I want a memory stack on that you
[29:40] know is it a two stack memory a four
[29:42] stack memory you know I don't see it
[29:44] becoming an eight stack memory you know
[29:46] so to me two or four sort of right you
[29:48] know but then power delivery needs to
[29:50] get integrated into that so that becomes
[29:52] sort of another element and I want that
[29:54] in the Z dimension right associated with
[29:57] it because you know I want my power
[29:59] rails coming into this as well
[30:01] >> right and Then I need, you know, fairly
[30:03] intelligent RDL layers to distribute
[30:05] signals across that platform. So to some
[30:08] degree, you know, my canonical thing is
[30:10] already about eight layer thick when you
[30:12] add up all of those pieces associated
[30:14] with it. And I think that's about all
[30:16] you can do, right? At least as we
[30:18] understand manufacturing today. And then
[30:21] of course you we expect optics to come
[30:22] in. So you're going to have more 35
[30:25] material structures, you know, that are
[30:26] going to be closer to this complex
[30:28] because I want, you know, my IO getting,
[30:30] you know, essentially, you know, deeply
[30:33] integrated into that construct as well,
[30:35] you know. So my siliconentric pieces
[30:38] need to get complemented by some of
[30:39] these other 35 materials and how do I
[30:42] get optical connectivity built into
[30:44] this? How do I get power rails, you
[30:46] know, built into that? You know, that is
[30:48] a, you know, I'll say that is an
[30:51] engineering feat. and a manufacturing
[30:54] nightmare, right, to bring all of those
[30:56] pieces. But that's where the physics is
[30:57] taking us uh over time. So because of
[31:00] that, you know, throwing a 16 high stack
[31:02] and to me, you know, it's like that's
[31:03] crazy. You know, a two or four high
[31:05] memory stack, you know, I think that's
[31:07] probably where the speed sweet spot will
[31:08] be.
[31:09] >> And you didn't talk about the thermal
[31:12] dissipation
[31:13] >> and new materials needed for that.
[31:16] >> Yeah. And obviously there's going to be
[31:17] a lot of, you know, you know, first
[31:18] thing, right? You just can't have some
[31:20] of the, you know, the physics hotspots
[31:21] that you have today. So you have to be
[31:22] more intelligent about how you use the
[31:24] the heat and spread the heat. But you
[31:26] know then you're going to be saying okay
[31:28] how do I both get power in and dissipate
[31:31] power out uh of those environments. And
[31:33] by the way if I can you know de you know
[31:35] get much more efficient power in at the
[31:37] voltages I want with much better you
[31:39] know power responsiveness and my voltage
[31:41] regulation you know then I don't create
[31:44] as much thermal chimneys.
[31:45] >> Yeah. But I do have to think okay how do
[31:47] I cool these things and you know what
[31:49] role do new materials diamonds and other
[31:51] things have been postulated as ways to
[31:53] you know do a better job of the thermal
[31:55] cooling how do I then have better you
[31:58] know liquid cooling techniques and
[32:00] others you know sitting right on it you
[32:02] know with good thermal characteristics
[32:04] so yeah you know uh you know we we
[32:07] engineers are becoming plumbers right
[32:09] >> exactly
[32:11] yeah we got to worry about MEC the chip
[32:13] scale
[32:14] >> let me ask about an operate phenomenon
[32:16] which is some people have said hey let's
[32:18] get the memory off of the package and
[32:20] just go all optical between the compute
[32:23] and the memory and that'll give you more
[32:26] degrees of freedom and more ways of
[32:28] building things what's your view on that
[32:30] as opposed to stacking that's other
[32:32] extreme
[32:33] >> yeah you know I think the more OEO right
[32:37] uh interfaces that you have you know
[32:39] it's you know you know optical to
[32:41] electrical to optical to you know the
[32:43] more of those that you demand in your
[32:45] core compute complex you enmy is just
[32:48] hard right uh in that sense and you're
[32:50] always going to have losses associated
[32:52] with it you know people are you know
[32:54] working hard to come up with more clever
[32:56] techniques to decrease you know both the
[32:59] cost and the power right you know how
[33:01] how much of the power am I wasting as I
[33:03] go through my you know my DAX and uh you
[33:06] know conversions so I'm not a fan of
[33:09] those you know because I think it gives
[33:10] you access to large pools of memory
[33:13] right but I I burn a lot of power
[33:15] getting there.
[33:16] >> Yeah.
[33:16] >> Right. And remember my you know the
[33:19] essentially you know the phentojoules
[33:22] per compute is now actually pretty
[33:25] amazingly good. Right. The phentojlesw
[33:28] per bit of communication is
[33:29] comparatively a thousand times worse
[33:32] right and if I'm spending that much to
[33:34] go get to large pools of so I don't like
[33:36] those approaches you know myself. I
[33:38] think they somewhat defy the underlying
[33:40] physics of the solution. I mean and then
[33:43] there's sort of the other approach to
[33:46] say like look can we actually integrate
[33:47] compute and memory directly right shift
[33:49] some of the compute into the memory it's
[33:52] >> some of the pim approaches
[33:53] >> yeah exactly right so what
[33:55] >> what are your thoughts around that
[33:57] >> I'm not a fan of those either right um
[33:59] because I think workloads you know
[34:01] because now I have to take my workload
[34:02] and I have to like push these little
[34:04] pieces of compute into the memory right
[34:07] you know so that you know I don't need
[34:09] as much bandwidth of memory coming into
[34:11] my real compute Right? And that puts
[34:13] constraints on workloads. And again, you
[34:15] know, I could be wrong on this,
[34:16] particularly some of the design
[34:17] flexibility and the intelligence that we
[34:19] can now bring, you know, through the the
[34:22] design aspects. But most PIM ideas, you
[34:25] know, have been around for 2530 years.
[34:27] And I think 2530 years from now, they'll
[34:29] still be hanging around, right? In that
[34:31] sense, you know, give me great memory
[34:32] bandwidth, you know, that I can have,
[34:34] you know, high density, uh, high
[34:36] performance, and get it close to the
[34:39] real compute engines. I expect that's
[34:40] going to be a more generalizable
[34:42] solution across many and evolving
[34:45] workloads over time.
[34:46] >> Well, so we're still stuck with denser
[34:49] and denser chips, more compute, more
[34:51] logic, more power,
[34:53] >> and much better IO.
[34:54] >> Now, I am a I am a huge fan, right, of
[34:57] opt, you know, moving to optics for all
[35:00] IO function. Yeah. Right. You know, I
[35:02] declared the death of copper about 25
[35:04] years ago. Eventually, I'll be right.
[35:06] >> You're still right. one day one day soon
[35:09] uh I'll be right but I think is that a
[35:11] you know to me all IO should go to
[35:13] optical right you know but that core
[35:15] compute memory complex no right
[35:18] >> so when do you the first step for it
[35:22] going um all optical is to scale up even
[35:25] more so than scale out
[35:28] >> what's your views on that
[35:30] >> yeah you know and I think we're now
[35:32] >> what would be the like the application
[35:34] or the workload driver if you will Well,
[35:36] I think the workload driver is the one
[35:37] in front of us.
[35:38] >> Okay.
[35:39] >> Right. I mean, you know, today, you
[35:40] know, our scaleup environments, we're
[35:42] putting tons of copper, right? You know,
[35:44] and essentially copper is becoming a
[35:46] wave guide that's getting shorter and
[35:48] shorter, right? And all of a sudden, you
[35:50] know, to make my copper work at uh, you
[35:53] know, 5 m is more expensive than making
[35:56] optical work, right, at 100 meters,
[35:58] right? You know, so you know, I think
[36:00] the physics is driving us there today,
[36:03] >> you know. And you know so if that's the
[36:06] case Pat why haven't we moved there
[36:07] already? Yeah it's a complex supply
[36:09] chain that you know has never been
[36:11] tested at scale and I think now you know
[36:14] I think everybody is looking at you know
[36:16] some form of NPO CPO etc you know in the
[36:21] 28 29 time frame because you know I
[36:24] can't scale up to large radics you know
[36:26] compute uh clusters for my scale up
[36:28] environment without making that move.
[36:31] So, you know, clearly Nvidia has uh
[36:33] indicated that. I think everybody else
[36:35] in the industry expects, you know,
[36:36] that's the conversion point. You know,
[36:38] in reality, I should have never built an
[36:40] NVL72. You know, it's an engineering
[36:43] marvel and it's a manufacturing
[36:44] nightmare, right? You know, it took the
[36:46] industry, you know, one extra year and a
[36:48] half to digest that beast, right? And it
[36:51] really is amazing that, you know, we're
[36:52] able to get it to scale, you know, so
[36:54] on. But you know that really proved that
[36:57] had we had a more mature optical supply
[37:00] chain, we'd be much more advanced today.
[37:02] We'd be able to build better scale up
[37:04] environments today, you know, better,
[37:06] you know, power, better cost, better
[37:08] energy. But we didn't have the maturity
[37:11] of the optical supply chains to allow
[37:14] that and those things are not trivial to
[37:16] get resolved. There's a lot of work to
[37:17] happen, but I think 28 29 is the year
[37:20] >> capacity. Good. for CSS.
[37:22] >> We we're probably gaining a lot of
[37:24] experience how in package optics work in
[37:26] networking, right? And and I think many
[37:29] of that will translate nicely to to
[37:31] >> GPU. I do think but all of a sudden, you
[37:33] know, it's like, okay, these things have
[37:35] to work, you know, at 10 million GPU
[37:37] scale. Wow. Right. You know, it has to
[37:39] fit in packages and, you know, there's
[37:41] just a lot of things there, you know,
[37:43] and many forms of optical don't like bad
[37:45] thermals. Well, guess what? Bad thermals
[37:47] is the nature of the game for it, you
[37:49] know. So we have thermal problems to
[37:50] deal with, you know, package integration
[37:52] problems. We didn't have enough, you
[37:54] know, laser capacity in the industry. So
[37:57] a bunch of things, but I don't see
[37:58] anything there fundamentals. It's just a
[38:00] lot of hard work at this point to pull
[38:02] all of those pieces together.
[38:03] >> In terms of the network architecture,
[38:05] will this also mean a switch towards
[38:08] more circuit switch networks just
[38:10] because, you know, you you can't scale
[38:12] these these very fast optical networks
[38:14] indefinitely? Well, you know, uh, you
[38:16] know, I like, uh, you know, uh, Nick
[38:18] Mune, you know, another Stanford
[38:20] professor here, good friend of mine, you
[38:22] know, as well, you know, I mean, you
[38:23] know him a little bit also, uh, Guido,
[38:27] you know,
[38:27] >> was he adviser?
[38:29] >> He was not the advisor.
[38:31] >> Yeah. Yeah. So, but, you know, to me, he
[38:33] describes it well, you know, we built,
[38:36] you know, uh, networks for to be able to
[38:39] handle any packet going anywhere with no
[38:41] knowledge of where it might go. When you
[38:43] think about AI, you know, it's almost
[38:45] exactly the opposite. Yes. Right. You
[38:47] know, we're doing flows that are large
[38:49] and predictable.
[38:51] >> So, in that sense, just the very
[38:53] description says, hm, that looks more
[38:55] like a switch than like a packet.
[38:58] >> So, I do think this confluence of moving
[39:00] to optical, right? And, you know, having
[39:04] a workload that is far more predictable,
[39:07] manageable, you know, large and flow,
[39:10] you know, to me, you know, I do see
[39:12] that. Yeah, we will go all the way to
[39:14] bringing optical into the compute
[39:16] complex and we will go all the way to
[39:19] OCS or some form of optical uh switching
[39:22] you know with more predictable flows you
[39:24] know I mean that is the right you know
[39:26] architecture and then you sort of say is
[39:29] that scale up or scale out and to some
[39:31] degree
[39:32] >> say it sort of dissolves the boundary
[39:34] there
[39:34] >> yeah right if my scale up is big enough
[39:36] then yeah you know just saying okay
[39:38] what's the radics of my scale up
[39:39] environment and when do I want to hook
[39:40] it up to other clusters that are really
[39:43] big and to me that ends up being a
[39:44] pretty beautiful answer uh you know to
[39:47] the future but I'm looking at somebody
[39:48] who's way more expert on this than
[39:50] myself. So
[39:51] >> no I I I look I always found scale up
[39:54] versus scale up to some degree an
[39:55] artificial distinction. You understand
[39:57] where it comes from right is we're kind
[39:58] of using different protocols one is
[40:00] front end one is backend network and so
[40:01] on. So right now practically speaking it
[40:04] is a it is a big difference.
[40:05] >> Yeah.
[40:06] >> If you take a step back and look at it
[40:07] from a system architecture perspective
[40:10] it achieves the same goal you know with
[40:12] slightly different flavors slightly
[40:14] different protocols that over time I
[40:15] think will converge.
[40:16] >> Yeah.
[40:16] >> I mean especially for the AI training
[40:18] workload.
[40:21] >> Yeah. Although I mean training and
[40:22] inferences actually look fairly similar.
[40:25] >> The inference corp is less of
[40:26] >> Yeah. Now I'll go on a you know right.
[40:28] Does your brain stop learning?
[40:30] >> Yeah.
[40:30] >> Right. you know you know when you know
[40:32] by the way as another characteristic of
[40:34] workload evolution I do think the you
[40:36] know the separation of training and
[40:37] inferencing is going to become you know
[40:39] less so going forward not more so right
[40:42] that's a little bit of the argument
[40:43] against you know you know more and more
[40:44] heterogeneous architectures is I think
[40:46] continuous learning you know I think
[40:48] there'll be algorithmic domains that
[40:50] will say yes continuous learning you
[40:53] know becomes the nature of the algorithm
[40:55] where yeah I do want my inference
[40:57] environment to be updating my model
[40:59] weights because you know that's giving
[41:01] me more and more refinement you know
[41:03] evolution across you know uh workloads
[41:06] moving into environments that are less
[41:08] formalized training uh as well we'll
[41:11] have smaller data sets in those
[41:12] environments so you know I'm not sure
[41:14] that's a
[41:15] >> but the number of nodes you need in
[41:17] order to be useful is a smaller diameter
[41:20] for inference than than for training
[41:22] that's what remains
[41:23] >> today that's true
[41:24] >> yeah yeah
[41:24] >> I mean I I still think there might be a
[41:27] role for hetrogenity coming from a
[41:29] different angle which is Different types
[41:32] of AI workloads require very different
[41:35] underlying like some are more comput
[41:36] limited some are more more memory
[41:38] bandwidth if you have a small diffusion
[41:39] workload or so right the optimal card
[41:41] for that looks very different from you
[41:43] know a top-of-the-line you know 15
[41:46] trillion 15 sorry 15 trillion weight LLM
[41:49] right that needs to operate at massive
[41:50] scale right so it's I I think that that
[41:52] might driveity if anything
[41:54] >> yeah I I think there's some room for
[41:56] that you know but I'd also say okay now
[41:58] I'm talking about putting a million GPUs
[42:00] to work in my data center, right? And
[42:02] you say, well, you know, for that
[42:04] particular diffusion workload, you know,
[42:06] I should really put something optimized
[42:08] this way, right? So, I'm now going to
[42:10] start, you know, putting, you know, a
[42:11] thousand of those in, right? And for
[42:13] that workload, I need, you know, 2,000
[42:15] of these in and, you know, for some, you
[42:18] know, and all of a sudden, you know, you
[42:19] don't get some of the, you know, the
[42:21] flexibility of scale, right? that you
[42:24] would say and you know how much does it
[42:25] cost me to manage and upgrade and you
[42:28] know connect and you know fault domain
[42:30] you know all of those types of things
[42:32] >> the agents will do up all the upgrades
[42:33] so there's no longer a problem
[42:35] >> yeah yeah yeah thank you everything gets
[42:37] easy so good out there
[42:38] >> yeah so fleet management is done um so
[42:42] we talked about what compute memory
[42:45] networking
[42:47] power is the next one
[42:49] >> yeah yeah yeah
[42:50] >> and you touched upon it with vertical
[42:52] delivery inside the chip
[42:53] on the chip.
[42:55] >> But
[42:57] I mean there's 800 volt DC, there's
[42:59] power delivery to the data center.
[43:01] >> Um there's an entire ecosystem all the
[43:04] way to
[43:06] power generation and and
[43:10] the sociopolitical dimensions of that
[43:12] and so on and so forth.
[43:14] >> Yeah. Yeah.
[43:15] >> So where does innovation come in and
[43:17] where does like its pure execution?
[43:19] Well, you the you know and if we start
[43:22] at you know uh first principles here you
[43:24] know our our nation has done a terrible
[43:26] job with this energy capacity right you
[43:29] know and essentially we went for 10 to
[43:32] 15 years where essentially I was taking
[43:34] coal offline at the rate I was adding
[43:37] renewables and essentially the nation
[43:39] was flatlined in terms of overall energy
[43:42] capacity. Not good because in a AI
[43:46] digital age energy capacity equals
[43:49] economic capacity right so essentially
[43:52] my economic capacity as a nation flat
[43:55] for 15 years if you buy that thesis
[43:57] right which I think is very you know
[44:00] provable over the last 5 years I you
[44:02] know we've seen this enormous influx
[44:04] into just give me more capacity uh but
[44:07] you know I think uh you know even with
[44:08] that we've gone to maybe increasing our
[44:12] national energy capac capacity 4% per
[44:14] year
[44:15] >> something like that
[44:15] >> going from you know essentially 0 to one
[44:18] right wow 4x
[44:20] or hm 4% right so to me you know that
[44:25] you know it's a bad situation so we just
[44:27] need more energy capacity
[44:29] >> uh for it so number one and obviously we
[44:31] have some companies in that space you
[44:33] know we talked about nuclear operating
[44:35] as one of those with Alva um and uh you
[44:38] know I think nuclear right you know as a
[44:41] base load is you know fabulous thing
[44:43] that we want to go build more of. You
[44:45] know, unfortunately, all of our
[44:47] renewables have, you know, deep
[44:48] dependencies in China, which is very
[44:50] unfortunate uh you know, for us. You
[44:53] know, we're, you know, uh, you know, gas
[44:55] turbine lead times are only eight years
[44:58] at this point, right? So, you know, you
[45:01] know, I mean, we really have a difficult
[45:03] environment to scale up rapidly, but
[45:05] one, we just need more capacity. Yeah.
[45:07] And fundamentally energy capacity
[45:09] somewhat dampens how euph fork we can
[45:11] get on AI, right? Because you know why
[45:13] build a new data center and buy, you
[45:15] know, the million GPUs if I can't power
[45:18] them, right? And I think you're going to
[45:20] see more and more defaults happening on
[45:23] many of those data center projects
[45:25] because the energy won't be there.
[45:27] >> So you think it'll be a significant
[45:28] headwind?
[45:28] >> I think it will be a headwind and I
[45:30] think you're already starting to see
[45:31] some of the first indicators of that.
[45:33] the Oracle, you know, uh, comet, you
[45:35] know, was just I and I think that's the
[45:37] first of what you're going to see many.
[45:38] Now, it's not like we're going to slow
[45:40] down as a result, but I think people
[45:42] will start saying, do I really have the
[45:43] energy, you know, to underwrite the
[45:46] capital commitments that I'm making, you
[45:48] know, for, you know, cement for data
[45:50] center racks and, uh, GPU purchases. So
[45:54] I do think it's you know going to be a
[45:57] you know an upper bound
[45:58] >> but we need more energy capacity and we
[46:00] need to get more innovation into that
[46:02] and we need more of those supply chains
[46:03] in the US you know nuclear one of my
[46:06] favorites you know when was the last
[46:08] nuclear reactor that came online in the
[46:09] US right you know it's 20 years ago
[46:12] >> right you know we just stopped you know
[46:14] a very capable uh industry so that's one
[46:17] we need more uh you know we need to get
[46:20] much more efficient at delivering you
[46:21] know the power networks and as you you
[46:23] know, moving 800 volt DC data centers.
[46:26] Absolutely. You know, right, you know,
[46:27] and being able to eliminate so many
[46:29] conversion steps along the way because
[46:31] of uh of uh uh historical uh you know,
[46:36] reasons and you know, standardization
[46:38] reasons. Okay, we got to fix this. You
[46:40] know, an 800 volt DC, the revenge of
[46:42] Edison is upon us.
[46:43] >> Yes, I was just going to say,
[46:46] >> yeah, I like to say so, you know, I do
[46:48] think that ends up being we need better
[46:50] power conversion. You know, we have a
[46:52] couple of companies, one of those
[46:53] vertical, you know, vertical GAN, you
[46:55] know, I think will be, you know, a
[46:57] killer technology, you know, to be able
[46:59] to do 800 to 48 or 800 to 12 or even 800
[47:02] to 5 in one conversion step, right?
[47:05] Which just gives you a lot more uh
[47:07] energy, you know, we touched on
[47:08] >> solid state transformers, you know.
[47:10] >> Yeah. You know, all of those are, you
[47:11] know, solid state transformers,
[47:13] switches, etc., you know, rebuilding,
[47:15] you know, the power delivery network.
[47:16] And I think that will, you know, have a
[47:18] lot of, you know, innovation, you know,
[47:20] it's sort of like, okay, power's cool
[47:21] again, right? You know, you know, for
[47:23] it. And then you're going to need all of
[47:24] those in the cooling systems as well,
[47:27] right? You know, and so there'll be a
[47:29] bunch of innovations and that, you know,
[47:31] so you know, you know, turbines and
[47:33] refrigerants and, you know, all of a
[47:34] sudden they get to be, you know,
[47:36] exciting new technologies again. So, you
[47:38] know, for all of us hardware guys, this,
[47:40] you know, this, you know, is like
[47:42] renaissance in front of us. It's just
[47:44] going to be
[47:44] >> after 25 years. I had to go back and
[47:46] read about car's car
[47:48] >> carinal efficiencies. Oh yeah. Is it
[47:50] like good stuff? How does that first law
[47:52] of thermodynamics work again?
[47:55] >> Something after all here,
[47:57] >> you know. So, so it's going to be you
[47:59] know exciting time that way you know but
[48:02] you know you know we also say you know
[48:03] we need to change the physics right
[48:05] because cos hasn't improved you know the
[48:09] essential power per terror you know
[48:10] power per teraflop for the last five uh
[48:13] generations of Nvidia chips you know
[48:15] flatline not okay right and that's where
[48:18] some of the things you know we have a
[48:19] superconducting company snowcap right
[48:22] which affords the opportunity to
[48:24] fundamentally change the physics you
[48:25] know a thousand times better power
[48:27] performance. So, I do think some of
[48:29] those innovations are upon us in the
[48:31] near future as well.
[48:31] >> I've been a terrible time manager here
[48:33] for the past hour. So, so many topics
[48:37] yet to cover. We didn't cover, but we
[48:39] got to cover at least one before we
[48:40] leave.
[48:41] >> Okay.
[48:42] >> Because now
[48:43] >> you're in charge.
[48:44] >> Yes. So, I thought
[48:47] um
[48:48] now VMs are back.
[48:50] >> How do you think about that as a former
[48:52] CEO of VMware?
[48:54] >> Well, you know, I I uh I feel excluded
[48:57] from this conversation obviously if we
[49:00] go on
[49:02] >> part of it was I was going to talk about
[49:03] networking as well so but I don't think
[49:05] we have the time
[49:06] >> you know hey you know I I think the you
[49:08] know every innovation has always led to
[49:11] the next abstraction right and I think
[49:13] many of these ideas and you right you
[49:15] know I always like to say you know you
[49:16] know what's what's new in our world
[49:18] today you know data structures
[49:19] algorithms you know an abstraction you
[49:22] know you know we bring it back to those
[49:24] you know foundationals and hey you know
[49:26] I you know the virtual machine
[49:27] abstraction you know whether it's done
[49:29] from the infrastructure level or from
[49:31] the application level you know I think
[49:33] it's still a foundational abstraction
[49:35] model you know that deserves to always
[49:37] have a place uh in the compute hierarchy
[49:40] so you know so what do you think one of
[49:42] us go back and run VMware again
[49:44] >> yeah that's what I tell I think I told
[49:47] it's time to go back and run VMware
[49:48] again
[49:49] >> yeah yeah and uh you know and I do think
[49:51] some of these abstractions you know but
[49:53] you also think about it and like one of
[49:54] the things we did at VMware where we you
[49:57] know essentially you know managed every
[49:59] aspect of the computing hierarchy.
[50:01] >> Yeah.
[50:02] >> Right. You know and you manage the
[50:03] workload, you manage the network, you
[50:05] manage the storage you know system and
[50:07] how do you abstract those? And I think
[50:09] in the AI context, right? We think about
[50:12] you know these agent swarms. Well, who's
[50:13] going to manage the agents? Who's going
[50:15] to create the security profiles around
[50:18] all of the agents? Who's going to be
[50:19] manage the performance of the agents?
[50:21] You know, essentially every fundamental
[50:24] element of virtualization and management
[50:26] needs to get recreated in this next
[50:28] computing hierarchy and there's going to
[50:30] be lots of wonderful companies that
[50:32] you'll sort of break through for how
[50:33] those things will be, you know, done.
[50:36] >> Okay. Do you want to end it on
[50:38] networking?
[50:40] >> Actually, I'd love to stay with the the
[50:42] VMs for a second. And the most
[50:44] interesting thing for me is that we're
[50:45] now building VMs so that they're usable
[50:48] for agents as opposed to usable for
[50:49] humans. What we've seen in sort of other
[50:50] companies is that that changes a lot the
[50:52] form factor that changes a lot how you
[50:55] market them, right? You want you want
[50:56] the agent to make the pick for for for
[50:58] your VM offering.
[50:59] >> Um it it changes a lot how much
[51:01] complexity you can have. It changes how
[51:02] how quickly the startup should be.
[51:04] Humans are a lot more patient than than
[51:05] agents.
[51:06] >> Um
[51:07] >> so it's
[51:09] if you have to rebuild VMware today, how
[51:12] would it look different?
[51:13] >> Yeah. And and I do think
[51:14] >> like the VMware for agents.
[51:17] >> Yeah. And I and I do think there's, you
[51:19] know, does there need to be a VMware for
[51:21] agents for humans, right? You know,
[51:24] >> I see what you mean. Yes.
[51:25] >> Right. You know, in the sense that, you
[51:26] know, I need humans.
[51:27] >> Let's stick to the agents. They're way
[51:29] more interesting than right now.
[51:30] >> Yeah. But, but, you know, I think it's
[51:32] an important point because I do think
[51:34] that there needs to be, you know, and
[51:36] I'll use, you know, Dario's
[51:37] constitution, right, or guard rails,
[51:40] right? Or being able to set policies.
[51:43] And I think then you you really say okay
[51:46] every aspect of the virtual machine as
[51:48] we know we'll love it today it is
[51:49] servicing agents
[51:51] >> right as opposed to servicing hardware
[51:53] and humans right so I think you know
[51:55] that becomes your fundamental design
[51:57] constraint on this side you know how do
[51:59] I make agents great
[52:01] >> right how can I make them secure how can
[52:03] I make them performant you know you know
[52:06] uh you know how do I abstract them how
[52:08] can I migrate them all of those things
[52:10] you know have the new embodiment you V
[52:13] motion for agents right it's like every
[52:16] one of these will have the new
[52:17] embodiment you know for them but then I
[52:19] also need to think about how can humans
[52:21] be setting the policies describing the
[52:24] constitutions you know getting the you
[52:26] know dashboards uh etc and I think
[52:28] that's sort of you know where the
[52:30] interesting thing is because you need to
[52:31] do both of those right not just one of
[52:33] those but the fundamental you know I'll
[52:35] say the hard design stuff like we did
[52:37] for you know virtual machines was
[52:39] associate abstracting the hardware in a
[52:41] performant way for applications or
[52:43] operating systems here. It's, you know,
[52:45] abstracting the hardware and the
[52:47] operations for agents. It's a
[52:49] fundamental thing that you're serving.
[52:50] It
[52:50] >> was great discussing this and thank you
[52:53] so much, Pat, for coming in and we'll do
[52:55] it again.
[52:55] >> Well, two of my favorite people.
[52:57] Anytime.
[52:57] >> All right. Thanks.
[52:58] >> That was awesome. Thank you.
[52:59] >> Thank you.
