OpenAI and Intel Veteran Seek Three-Quarter AI Chip Deployment
Ai4
Current chip lead times of three years risk architectural obsolescence and face a hard power ceiling without a 10,000-fold increase in token efficiency.
## Background
Published on YouTube by the Ai4 channel on August 22, 2026, this video features an 18-minute fireside conversation on the foundational compute infrastructure powering the AI economy. The page description outlines a discussion between former Intel CEO Pat Gelsinger and OpenAI's Sachin Katti, moderated by technology writer Gideon Lewis-Kraus, examining semiconductor constraints, hardware developments, and the strategic bottlenecks limiting frontier AI.
## TL;DR
Panelists Pat Gelsinger and Sachin Katti explore how severe physical and infrastructural constraints—spanning 2-nanometer foundry limits, power grid delays, and hardware inefficiency—are creating major bottlenecks for expanding AI compute. Grounded in Rich Sutton's 'Bitter Lesson,' they analyze the shift of economic value from SaaS down to silicon while highlighting a striking nuclear energy disparity between China and the United States. To solve these limits, the discussion emphasizes compressing hardware deployment cycles from three years down to three quarters, revolutionizing memory-on-chip architectures, and drastically lowering energy consumption per token.
## Atmosphere & style
The conversation carries a candid, authoritative, and urgency-driven tone, blending high-level strategic analysis with deep engineering realism. Gelsinger brings a blunt, veteran semiconductor perspective—openly critiquing current GPU power inefficiencies—while Katti offers a focused, ambitious outlook from the frontier of model deployment. The rapport between the speakers feels collaborative and collegial, capturing the intense gravity and rapid pace of the global AI hardware race.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=4s) Every major AI breakthrough directly relies on compute, with the challenge evolving from pure chip manufacturing to global-scale efficiency and rapid deployment.
- [00:04](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=4s) Semiconductor manufacturing faces brutal physical constraints at 2 nanometers, making chips the essential physical fuel of a token-driven economy.
- [00:04](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=4s) Rich Sutton's 'The Bitter Lesson' shows that scaling compute consistently yields superior results compared to human-engineered algorithmic optimizations.
- [05:37](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=337s) Economic value across the technology stack has heavily shifted away from traditional SaaS to hardware, foundries, equipment makers, and chip designers.
- [05:37](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=337s) The cost of AI intelligence is dropping by 10x to 100x annually, far outpacing historical Moore's Law.
- [05:37](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=337s) Automating lower-level code generation with AI will eventually make hardware interchangeable and turn compute into a true commodity.
- [08:50](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=530s) Gelsinger's 'IEEE' engineering framework outlines four key pillars for future progress: Infrastructure, Efficiency, Energy, and Economics.
- [08:50](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=530s) Token delivery economics require a 10,000x efficiency improvement to sustain the massive global CapEx investments in AI.
- [08:50](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=530s) There is a significant geopolitical disparity in energy infrastructure development, highlighted by China having 39 nuclear reactors under construction versus zero in the US.
- [13:18](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=798s) Integrating memory directly onto compute dies offers a potential 50x to 100x reduction in inference costs.
- [13:18](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=798s) The semiconductor and data center industry must compress the hardware concept-to-deployment timeline from 3 years down to 3 quarters.
- [13:18](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=798s) Designing overly specialized hardware carries significant risk due to the rapid pace of algorithmic shifts during a 3-year chip development cycle.
- [18:03](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=1083s) Grid power availability remains the ultimate constraint on AI expansion, with US electrical grid capacity growing at only 4% to 5% per year.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=4s) The Foundation of AI: Compute, Supply Chains, and Perceptions
The session opens with a brief introduction highlighting that every AI breakthrough relies directly on compute. As demand shifts from training frontier models to enterprise deployment, the challenge expands beyond simply manufacturing more chips to doing so faster, more efficiently, and at global scale. Moderator Gideon Lewis-Kraus (technology writer for *The New Yorker*) introduces the panelists: **Pat Gelsinger** (General Partner at Playground Global and former CEO of Intel) and **Sachin Katti** (Vice President of Compute Strategy and GPT Infrastructure at OpenAI).
Lewis-Kraus frames the core issue based on his recent reporting on the global semiconductor supply chain. He notes two frequent observations: even tech-literate people often understand almost nothing about chips, and chips act as a mirror reflecting an observer's narrow interests. A corporate executive views chips through inference economics, a data center landlord focuses entirely on power, AI critics view data centers as physical symbols of unwanted AI encroachment, and frustrated observers wonder why chips cannot simply be mass-produced like standard widgets. When asked what three fundamental points they would teach a non-specialist, Gelsinger emphasizes that chips are the physical "fuel" or "oil" of a token-driven economy. While humanity's demand for intelligence is virtually unlimited, semiconductor manufacturing faces brutal physical constraints. Building modern capital-intensive foundries requires engineering transistor features down to 2 nanometers—where a human hair looks like a boulder compared to the microscopic structures inside. Gelsinger notes that AI's explosive growth is possible because it builds directly on top of pre-existing internet, cloud, and semiconductor infrastructure rather than starting from scratch.
Adding to this perspective, Sachin Katti highlights **"The Bitter Lesson,"** a famous observation penned around 15 years ago by AI researcher Rich Sutton. The principle states that regardless of algorithmic cleverness or human domain heuristics, throwing more compute at a problem consistently yields superior solutions as scale increases. Katti traces OpenAI's view of model evolution—progressing from basic chatbots to reasoners, agents, AI interns, AI engineers, and ultimately autonomous AI researchers. He explains that OpenAI's future model capabilities and revenue projections are directly mapped to the total volume of compute they can secure over time.
- **Panelists:** Pat Gelsinger (Playground Global / ex-Intel) and Sachin Katti (OpenAI).
- **Manufacturing Scale:** Advanced foundries produce 2-nanometer transistors requiring some of the most capital-intensive facilities in human history.
- **The Bitter Lesson:** Rich Sutton's paper demonstrates that compute scaling consistently outperforms handcrafted algorithmic optimizations.
- **AI Capability Progression:** Chatbots $\rightarrow$ Reasoners $\rightarrow$ Agents $\rightarrow$ AI Interns $\rightarrow$ AI Engineers $\rightarrow$ AI Researchers.
---
[05:37](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=337s) Is Compute a Commodity? Value Capture Across the Supply Chain
Lewis-Kraus picks up on the analogies to oil and "big blobs of compute," asking whether compute and intelligence should be viewed fundamentally as commodities, and if so, who will capture economic value across the supply chain. Gelsinger points out that value capture has heavily sedimented down to the silicon and infrastructure layer. While model developers hope to capture vast downstream value in the future, the traditional software-as-a-service (SaaS) sector has suffered a massive disruption—what he calls the "SaaS apocalypse"—as software shifts toward chat and agentic interfaces operating over unified algorithmic and data layers. Consequently, semiconductor foundries, toolmakers, and memory manufacturers have seen unprecedented economic windfalls, yielding multi-billion and trillion-dollar valuations for memory suppliers, equipment makers like ASML, and foundries like TSMC. In Gelsinger's words: *"It has never been this good to be a silicon guy."*
Katti responds by outlining OpenAI's mission to lower the cost of intelligence, making it universally accessible. He highlights that the cost of intelligence has been dropping by **10x to 100x every year**—a pace far exceeding historical Moore's Law. Because the global demand for intelligence is effectively infinite, cheaper intelligence continuously unlocks new use cases. While chips and compute are not yet true commodities—evidenced by the vast margins premium chips command today—Katti envisions a future where AI automates the software layer required to translate raw hardware into tokens across any architecture. Once software adapts seamlessly to any underlying silicon, compute will behave far more like a fungible commodity.
- **Economic Shift:** Value has moved away from traditional SaaS software layers down to silicon foundries, equipment makers (ASML), and chip designers.
- **Cost Trends:** The cost of delivering AI intelligence has been falling by 10x to 100x per year.
- **Future Fungibility:** As AI automates low-level code generation to run models on arbitrary hardware, compute will increasingly behave like a true commodity.
---
[08:50](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=530s) The "IEEE" Framework and Infrastructure Bottlenecks
Moving into their ongoing work and multi-year outlooks, Gelsinger outlines a four-part engineering framework he summarizes as **"IEEE"**:
1. **Infrastructure:** Massively expanding semiconductors, data centers, and communication fabrics.
2. **Efficiency:** Overcoming today's highly inefficient hardware architectures. Gelsinger bluntly states that *"today's GPUs suck,"* citing severe power limits and the fact that High Bandwidth Memory (HBM) is energy-inefficient despite being the best currently available option.
3. **Energy:** Recognizing that in a digital AI economy, *economic capacity equals energy capacity*. Gelsinger points out a stark geopolitical disparity: China currently has 39 nuclear reactors under active development, whereas the United States has zero.
4. **Economics:** Transforming the unit economics of tokens. Gelsinger argues that token efficiency cannot just improve by 10x—it must improve **10,000x** to sustain the trillions of dollars in global capital expenditure, which will soon be measured as a non-trivial percentage of world GDP.
Katti reflects on his previous tenure at Intel working directly under Gelsinger, joking that he spent years working for Pat. Over the past year at OpenAI, Katti's primary focus has been convincing global leaders to invest heavily across every layer of the physical and virtual infrastructure stack. In the Western world, nearly every layer is severely bottlenecked: data center construction is slowed by permitting and grid interconnects, energy is scarce, fab and memory capacity remain constrained, and integrating components into usable compute clusters takes far too long.
- **Gelsinger's IEEE Model:** Infrastructure, Efficiency, Energy, and Economics.
- **Nuclear Disparity:** China has 39 nuclear reactors under construction versus zero in the US.
- **Economic Requirement:** Token delivery economics need a 10,000x efficiency improvement to justify global infrastructure CapEx.
---
[13:18](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=798s) Breaking Bottlenecks: From 3 Years down to 3 Quarters
Lewis-Kraus pushes the panelists on how to accelerate a semiconductor ecosystem that historically took decades to build, noting that titan companies like TSMC and ASML required 40 years to achieve their dominance. Gelsinger outlines specific technical breakthroughs on the horizon: resurrecting progress in lithography beyond current 13.5nm extreme ultraviolet (EUV) systems (where lithography equipment now accounts for half of a fab's total capital cost), integrating memory directly onto compute dies to boost bandwidth by orders of magnitude and reduce inference costs by 50x to 100x, transitioning interconnects from copper to optical networking, and deploying cryogenic cooling for superconducting logic to achieve 1,000x efficiency gains.
Katti focuses on the data center buildout timeline. Today, building a custom data center facility often takes up to 24 months or longer, while TSMC can ramp an entire new chip manufacturing process in just 18 months. Currently, the cycle from initial chip concept to a fully ramped production cluster takes roughly 3 years (9–12 months for design, 3–6 months in the fab, followed by bring-up and datacenter deployment). Katti argues that the industry must compress this entire timeline from **3 years down to 3 quarters**. Achieving this requires standardizing modular data center construction—moving away from bespoke, "stick-built" on-site construction toward robotic prefabrication—and securing direct, on-site energy generation to bypass lengthy power grid interconnection queues.
Gelsinger adds a warning regarding algorithmic shifts: because hardware development takes 3 years, specialized chips optimized for today's separate pre-fill and decode steps may become obsolete by the time they ship. If frontier models begin folding training and inference back together, hyper-specialized accelerator chips risks being algorithmically mismatched for next-generation architectures.
- **Current Hardware Lead Times:** Concept-to-production for new silicon takes ~3 years (9–12 months design, 3–6 months fab, plus bring-up).
- **Target Compression:** OpenAI aims to compress chip design-to-datacenter deployment down to 3 quarters.
- **Inference Innovation:** Direct compute-on-memory integration could reduce inference costs by 50x to 100x; cryogenic optics offer potential 1,000x efficiency gains.
- **Architectural Risk:** Designing overly specialized chips risks mismatch with rapidly evolving algorithmic paradigms.
[18:03](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=1083s) The Ultimate Constraint: Energy Capacity and Token Efficiency
Addressing the overarching limits facing artificial intelligence investments, the conversation zeroes in on energy as the primary upper bound on growth. Expanding AI capabilities relies directly on available power infrastructure, which in the United States is currently growing at a modest rate of 4% to 5% annually—a slight improvement after a dismal decade where growth hovered around just 1% per year. This structural bottleneck means that simply throwing more capital at AI infrastructure will inevitably hit a hard ceiling defined by the electrical grid's total capacity.
To sustain the trajectory of frontier AI models, the industry must tackle the energy challenge from two directions: expanding total supply and drastically reducing consumption. There is an urgent imperative not only to generate significantly more energy overall, but also to deliver that power far more efficiently directly to computing hardware. Ultimately, the central goal for next-generation computing infrastructure must be to dramatically improve the energy required per token produced, ensuring that intelligence can scale within real-world physical constraints.
- **US Grid Growth:** Current energy capacity expansion in the United States is limited to 4%–5% annually, following a decade-long plateau of roughly 1% per year.
- **Strategic Imperative:** Future chip and system design must focus on maximizing token output per watt to bypass energy bottlenecks.
## Notable quotes
> [05:37](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=337s) It has never been this good to be a silicon guy. — Pat Gelsinger
> [08:50](https://www.youtube.com/watch?v=ZCgKHvGpxd0&t=530s) today's GPUs suck — Pat Gelsinger
## Mentioned
Pat Gelsinger, Sachin Katti, Gideon Lewis-Kraus, Intel, OpenAI, Playground Global, The New Yorker, ASML, TSMC, Rich Sutton, The Bitter Lesson
## Video description
Pat Gelsinger - General Partner; Former CEO of Intel, Playground Global
Sachin Katti - Head of Compute, OpenAI
Gideon Lewis-Kraus - Technology Writer, The New Yorker
AI’s ambitions have never been bigger, and neither have the demands on the infrastructure beneath it. As organizations race to deploy AI at scale, a critical question is being overlooked: is the compute foundation of the AI economy capable of delivering on what the field now promises?
In this candid fireside conversation, semiconductor veteran and former Intel CEO Pat Gelsinger joins Sachin Katti, Head of Compute at OpenAI, to examine the defining constraints of the AI age from both sides of the equation: the evolution of chips and systems, and the real-world compute demands of frontier AI. Together, they will explore what is truly limiting AI’s next phase, how the next generation of computing infrastructure may unlock capabilities today’s systems cannot support, and what business and technology leaders should understand now as compute becomes one of the central strategic bottlenecks of the AI era.
OpenAI's Sachin Katti and Playground Global's Pat Gelsinger argue that AI advancement is fundamentally tethered to compute scaling, which has shifted massive economic value into physical silicon and foundational infrastructure. However, sustaining this trajectory requires overcoming severe physical bottlenecks in chip fabrication lead times, power grid capacity, and computing efficiency.
Compute Scaling and Shifting Supply Chain Economics (5:37)
Invoking Rich Sutton's 'Bitter Lesson,' Sachin Katti emphasizes that scaling compute consistently outperforms handcrafted algorithmic optimizations. OpenAI directly ties its capability roadmap—advancing from basic chatbots to autonomous AI researchers—and projected revenues to the physical volume of compute it can secure. Simultaneously, Katti reports that the cost of delivering intelligence is dropping by 10x to 100x annually, outpacing historical Moore's Law.
Pat Gelsinger notes that this dynamic has triggered a 'SaaS apocalypse,' shifting economic value downstream away from traditional software layers and concentrating it in equipment makers like ASML, foundries like TSMC, and memory manufacturers. While premium chips command high margins today, Katti forecasts that compute will gradually become a fungible commodity once AI automates the low-level software layers needed to map workloads across heterogeneous silicon architectures.
Overcoming Silicon Architecture and Memory Bottlenecks (13:18)
Gelsinger contends that current hardware architectures remain deeply inefficient, arguing that today's GPUs and High Bandwidth Memory (HBM) consume excessive power. Manufacturing advanced 2-nanometer transistors requires extreme capital expenditure, with lithography tools alone accounting for approximately half of a modern foundry's total capital cost.
To overcome these physical limitations, Gelsinger outlines emerging hardware interventions: integrating compute logic directly onto memory dies to lower inference costs by 50x to 100x, transitioning chip interconnects from copper to optics, and exploring cryogenic cooling for superconducting logic to target potential 1,000x efficiency improvements.
Compressing the Three-Year Infrastructure Deployment Cycle (13:18)
Bringing new compute clusters online currently takes roughly three years—spanning 9 to 12 months for chip design, 3 to 6 months in the foundry, and up to 24 months or more for data center construction and bring-up. Katti argues that the industry must compress this entire timeline down to three quarters by replacing bespoke on-site construction with modular, robotically prefabricated data centers and securing on-site power generation to bypass utility connection queues.
Gelsinger cautions that long hardware lead times introduce significant algorithmic risk. If frontier AI architectures evolve—such as by recombining pre-fill and decode stages that are currently separated—hyper-specialized accelerators designed during a multi-year cycle risk becoming obsolete by the time they reach production.
Energy Capacity and the Token Economics Ceiling (8:50)
Framing the challenge through his 'IEEE' model (Infrastructure, Efficiency, Energy, Economics), Gelsinger asserts that in an AI economy, economic capacity directly equals energy capacity. He highlights a severe infrastructure disparity, noting that China has 39 nuclear reactors under active construction compared to zero in the United States, while the US electrical grid expands at just 4% to 5% annually following a decade of 1% growth.
Because power grids impose a hard ceiling on compute expansion, both panelists argue that capital expenditures cannot scale indefinitely without radical efficiency gains. Gelsinger estimates that token delivery economics must improve by 10,000x to economically justify global infrastructure investments that are beginning to register as a non-trivial share of world GDP.
The central tension facing artificial intelligence lies between the demand for compute scaling and the physical realities of power generation, fab economics, and construction timelines. Whether the industry can compress deployment cycles to three quarters and deliver a 10,000x leap in token efficiency remains an open question that will dictate the economic viability of global AI infrastructure.
[5:37] Pat Gelsinger: It has never been this good to be a silicon guy.
[8:50] Pat Gelsinger: today's GPUs suck
Original transcript
[0:04] Every AI breakthrough has something in common.
Starts with compute. From training frontier
[0:09] models to scaling AI across the enterprise,
compute has become the foundation of the AI era.
[0:15] As demand continues to grow, the challenges
are no longer just about building more compute,
[0:20] but building it faster, more
efficiently, and a global scale.
[0:24] Today, we'll hear from two of the leaders helping
shape the future of compute. Pat Gelsinger, former
[0:29] CEO of Intel and now general partner at Playground
Global, has had a front-row seat to major
[0:34] technology transitions. Few people understand the
infrastructure powering AI better than he does.
[0:39] Sultan Katti is vice president of compute strategy
and GPT infrastructure at OpenAI, where he helps
[0:44] build the compute infrastructure that powers some
of the world's most advanced models. Please join
[0:49] me in welcoming Pat Gelsinger and Sultan Katti.
So, it's really an honor for me to be here with
[0:54] both of you guys today, especially because over
the last couple of months, I've been working on
[0:58] a long story about the global semiconductor supply
chain. And there are two related things that I've
[1:04] noticed. The first is that even relatively
sophisticated people who really understand AI
[1:10] tend to know absolutely nothing about chips.
And the second, relatedly, is that chips are a
[1:16] prism through which people really just see their
own interests. And that people at the There are
[1:22] depending on where you are in this environment,
you see chips extremely differently. So,
[1:27] you know, if you're running a company, you're
you're impressions of chips are largely about
[1:31] the economics of inference. If you are involved
in data center leasing, you're thinking about
[1:37] energy. If you are You know, this morning I
was listening to Jasmine Sun on Ezra Klein,
[1:43] and she's talking about how for like so much of
this AI populist backlash, of course, just sees
[1:48] data centers as a symbol of AI's encroachment in
ways that they might not appreciate. And then,
[1:55] you know, for a certain kind of person, compute
just feels like this incredibly frustrating
[2:00] bottleneck. Like, these are just widgets.
Why can't we make more of these widgets? Um
[2:06] so, the first question that I want to open up with
this morning is, if you were talking to, you know,
[2:11] a relatively sophisticated interlocutor who
doesn't really know all that much about compute
[2:17] or just understands compute in their very limited
sense, what do you think are, say, the three most
[2:23] important things that you would want them to come
away with better understanding chips and compute?
[2:30] Right. Well, you know, when you think about
compute, it all runs on chips, right? So,
[2:34] there ain't no compute without chips sitting under
underneath it. And obviously, you know, the chips
[2:40] that sit underneath it, you know, they are the
underlying fuel. They're the oil of a token-driven
[2:46] economy, right? That you have. You know, for AI,
I think the demand is unlimited for intelligence.
[2:53] So, we see this insatiable demand, right?
Underneath that, you have then almost insatiable
[2:59] demand for compute for infrastructure, for memory,
you know, that goes against it. We have severe
[3:05] limitations of how we build those chips today. You
know, the capacity of manufacturing is limited,
[3:10] right? And it takes a long time to build these
extraordinary capital-intensive factories,
[3:16] you know, that go under it with some of the most
sophisticated manufacturing that's ever been
[3:22] contemplated in humanity. You know, 2-nanometer
transistors, right? You know, your hair is like
[3:28] a boulder compared to, right, you know, a
flea inside of these. It's just incredible,
[3:33] you know, what's being done. And all of that, you
know, it just says, "Give me a chat interface."
[3:38] I mean, it's just so spectacular. And what's going
on today with AI, you know, it's building on,
[3:43] you know, it didn't have to build the internet.
It didn't have to build semiconductors. It didn't
[3:47] have to, you know, build clouds. It's building on
all of those, and that's why it is so explosive
[3:53] today. Yeah, I I think that summarized what goes
into compute really well and kind of how we hard,
[4:01] but at the same time how important the whole
thing is. I think the one maybe thing that
[4:06] I'll highlight for the audience, there's this
concept in AI called the bitter lesson. So,
[4:11] this was a famous essay written around 15 years
ago by an AI researcher who basically Rich Sutton
[4:20] Sutton wrote this, and he basically observed that
it doesn't really matter how smart your algorithms
[4:27] are or how creative uh whatever research you're
doing is. The bitter lesson was that as you throw
[4:33] more compute at a problem, you just are able to
solve more and more complex problems. And that's
[4:40] really been the lesson of this decade. Uh I think
when OpenAI talks about this and consistently,
[4:45] we've kind of always talked about AI
evolving through chatbots, reasoners, agents.
[4:53] We are now believe at the point where there are
AI interns, and soon uh AI engineers and then
[5:00] AI researchers itself. So, the AI itself going
through this progression as if it's growing up
[5:06] from just chatting with us to reasoning to
this. And all of that is just scaling laws,
[5:12] uh the bitter lesson and showing up. Uh
it's just being more compute thrown at it.
[5:16] So, the compute essentially is and this
sounds simplistic, but how we think about
[5:22] what the future revenue of OpenAI will be and
what the capability of our models will be,
[5:27] we can effectively trace it from how much compute
we will have uh in whatever time frame. So,
[5:33] it's that fundamental. So, I want to get to what
you guys are working on in a second, but I want
[5:37] to follow up because there's one thread common to
what you guys both said. First, in comparison to
[5:42] oil, and then talking about Rich Sutton's bitter
lesson, which also, you know, famous OpenAI paper
[5:48] from almost 10 years ago called it a big blob
of compute, right? So, is you know, there's
[5:54] been a lot of discussion, especially as far as
inference goes, about like whether we should
[6:00] be thinking of intelligence as a commodity. The
question is on the other side of the supply chain,
[6:07] is oil the right metaphor? Is it a big blob? Are
we talking about something that is fundamentally
[6:12] a commodity or not a commodity? And if so,
if the chips could be considered a commodity,
[6:17] if at a certain point, uh intelligence could be
considered a commodity, who then captures the
[6:23] value of this supply chain if we're talking about
these as commodities? Well, I mean, clearly you've
[6:30] seen the chip guys have gained extraordinary,
you know, value capture in this cycle as well,
[6:35] right? You know, the model guys are looking to
capture value in the future. I believe they will,
[6:42] you know, but you've hollowed out, right? You
know, value has sedimented down to the, you know,
[6:47] silicon and infrastructure layer, right? You
know, software layer as we've known and loved it,
[6:51] the SaaS apocalypse has, you know, destroyed that
sector, right? And essentially it becomes a chat
[6:56] interface with increasingly agentic level, right?
You know, against models underneath it, right? You
[7:01] have a data layer, right? You have an algorithmic
layer and a chat interface, right? And then,
[7:06] you know, the belief is is that will fundamentally
change business, business process, and business
[7:11] efficiencies at the top. But so far, the silicon
guys are the big winners. I mean, we have,
[7:16] you know, a trillion-dollar memory companies. How
that happened, right? You know, just, you know,
[7:21] commodities that are extraordinary, you know,
fab companies, ASML, you know, TSMC, you know,
[7:26] the silicon guys, you know, it has never been this
good to be a silicon guy. And that's what silicon
[7:30] guys are like that. [snorts] Yeah, I I think we
we fundamentally believe intelligence should be
[7:37] accessible to everyone, and we need to figure out
how to to driving the cost of intelligence down.
[7:42] It has I mean consistently I mean this is a faster
pace than Moore's law ever was, right? So, we've
[7:48] been driving cost of intelligence down by 10 to
100 X every year and we don't see that stopping.
[7:55] Uh and that's just because the demand for
intelligence is effectively infinite, right? So,
[7:59] the more you cheaper intelligence you can deliver
there's more use cases, more demand for it.
[8:04] And so, just if I draw the analogy to its
logical conclusion, that will mean that we
[8:10] want to make compute very fungible and something
that is able to deliver all kinds of intelligence
[8:18] at different price points uh at at the very
affordable price points to the whole world.
[8:23] Now, compute today is not yet a commodity, right?
There is different kinds of chips with which are
[8:31] obviously the market is paying different margins
for different chips today. But, as AI becomes
[8:36] capable of writing the software that is needed to
turn chips into intelligence, chips into tokens
[8:43] for any chip, we do think compute becomes uh
more and more a commodity in the long term.
[8:50] So, tell us then more specifically, what have both
of you guys been working on in the last few years
[8:56] and what do you what's your plan for the next few
years? Yeah, and you know, I I'll simply describe
[9:02] it as I E E E, IEEE and being an engineer that
sort of works for me. But, you know, first is lots
[9:09] of infrastructure, right? You know, we need more
semiconductors, we need more data centers, right?
[9:14] You know, more communications infrastructure,
build out infrastructure. You know, second is
[9:19] efficiency. You know, today's GPUs suck, right?
You know, they're very power inefficient, you
[9:25] know, they're you know, computationally limited.
Memory today, HBM is a very inefficient memory.
[9:31] It's just the best one we have, you know, for
AI modeling today. So, efficiency, we need much
[9:38] greater, you know, efficiency in AI processing,
you know, if it's going to reach its potential.
[9:43] You know, third, energy, right? In a digital AI
economy, you know, essentially, economic capacity
[9:50] equals energy capacity. You know, today, China,
39 nuclear reactors under development. In the US,
[9:58] zero, right? You know, we have an energy issue.
We need more energy efficiency and getting power
[10:03] delivered to the AI computation, and I just need
more energy, you know, to power it. And finally,
[10:09] economics, right? You know, the economics today
suck, right? You know, for uh you know, tokens.
[10:15] And you know, as I like to say, it doesn't need
to get 10x better, it needs to get 10,000x better,
[10:21] right? Today, you know, we're driving,
you know, trillions of infrastructure
[10:25] spend. Imagine that. You know, we're going to
start measuring the AI infrastructure spend
[10:30] spend as a percent of world GDP, you know,
going forward. And the economics for that
[10:35] are terrible. We have to improve the economics
of AI dramatically from where we are. So, IEEE.
[10:43] You asked what we've been working on for
the last 4 years. I've been working for Pat.
[10:47] So, [laughter] I was at Intel. I was
literally working for Pat. graduates, so you
[10:53] you guys have any stories to tell about working
together? I would Everyone would love to hear some
[10:57] tales. I'm sure we will. Uh but uh kidding
aside, I think the last uh year especially
[11:02] uh I don't really I mean, the main real
focus has been kind of what Pat alluded to,
[11:08] which is convincing the world that we need to
invest in every layer of the infrastructure stack
[11:16] because of the potential this technology has
in delivering intelligence to the whole world.
[11:20] Uh we are definitely very much in a
world, especially in the Western world,
[11:25] where every layer of the infrastructure stack
is bottlenecked. It's more It is bottlenecked.
[11:30] Uh it's not easy to build data centers quickly
in the US. It's not easy to find energy in the US
[11:38] uh, that we need. Uh, there's not enough fab
and memory capacity to supply the chips we need
[11:44] and uh, it's not easy to put it all together
into usable compute. It takes too long. Uh,
[11:50] I think all of those layers of the stack
and we have basically spent the time kind
[11:54] of convincing that we need to go faster,
make it cheaper, the IEEE concept here. Uh,
[12:00] but that that's really important if we are going
to unlock the ability to scale intelligence
[12:05] to all the things that we want to do. So, I
think that's how do we get the narrative uh,
[12:12] get across the narrative to the world
about what's coming with this technology,
[12:16] how capable these things are going to be,
how impactful these things are going to be
[12:20] and deduce from there how we need to invest in
every layer of the stack so that we can actually
[12:25] get the benefits of this to everyone and not
just the select few. Well, I mean, I'd like to
[12:31] push you to be maybe a little bit more specific
about what that looks like because, you know,
[12:35] obviously we've had tremendous investments, we've
had CapEx expenditures like nobody's ever seen
[12:39] before. But, if you look, you know, Pat, you were
saying that um, right now like the biggest winners
[12:45] have been the like the the people at the foundry
level, a place like TSMC or like the tool makers
[12:52] like an ASML. But, both TSMC and ASML are they're
not exactly overnight success stories. I mean,
[12:59] like these are it took them four decades to get to
where they were. This was a process of building up
[13:05] an enormous ecosystem around them. How does one
like so, what are the things that one can do now
[13:12] to try to accelerate that given that it's almost
definitionally a kind of generational project?
[13:18] Yeah. You know, and there's so many opportunities
here, right? You know, some of those, hey,
[13:22] we got to make, you know, semiconductors,
you know, we have to wake up Moore's Law.
[13:27] Right? You know, we statically, you know, sort
of landed at 13.5 nanometer lithography. Hey,
[13:32] we got to blow the doors open on that. They got to
get more economic. You know, it's now half of the
[13:37] capital of a semiconductor fab is lithography.
Right, it's just risen. So, we got to attack that.
[13:43] You know, if you look at today's chips, you
know, they're these memory bound, right,
[13:48] uh limitations. Okay, you know, almost all of
the new chips will put memory directly onto the,
[13:55] you know, uh compute device, right, increasing
by or, you know, exponentially the memory
[14:01] bandwidth available. And that will change
the cost dynamics of inferencing, you know,
[14:05] by a factor of 50 [snorts] to 100. You know, we
need to move from copper to optical networking,
[14:11] you know, way overdue to make that transition.
You know, we're going to start seeing cryogenics,
[14:16] right, you know, for superconducting devices
that afford, you know, a thousand X improvement
[14:22] in efficiency. You know, these are just some of
the things that will start to crack open the door
[14:27] that then fundamentally change the economics,
which I think start to change dramatically,
[14:31] you know, late in the decade. And if I can
make my chips do inferencing a thousand times,
[14:36] you know, more effectively than yours, you
know, I have changed the economic dramatically,
[14:41] you know, forward. And I think some of the capital
risk that people are playing today will start to
[14:45] come under pressure late this decade. Not
that we don't need more data center space,
[14:49] but what we put into those data centers will
try start to change in a pretty dramatic way.
[14:56] Yeah, uh a part covered the chip and the fab
space quite well. All the things that you could be
[15:02] doing to kind of change the curve here
effectively. I'll cover the other part of compute.
[15:07] Uh I think we need to figure out how to
innovate on just building these things,
[15:13] right? So, building a data center today is
a 24-year project in the in some of the best
[15:18] case scenarios. Often it can take longer. And so,
how do we actually TSMC can do it in 18 months.
[15:24] Yeah. TSMC can ramp a process in 18 months, right?
So, it takes longer to get the data center up and
[15:31] running. I think the joke is from concept to a
chip in production is 3 years today. It's not a
[15:38] joke, it's the reality, right? So, I think about
a chip, it's usually 9 to 12 months to design it,
[15:44] then there's a 3 to 6 month fab cycle, comes
back, you bring it up, and then you ramp it. So,
[15:48] it's usually 3 years. And that kind of goes along
with the data center. Our vision is can we cut
[15:53] that down to three quarters instead of 3 years?
Can we go from a chip design to a chip ramped up
[16:00] in production in a data center in three quarters?
I think many things will have to change to go
[16:05] from 3 years to three quarters. So, obviously the
chips and the fab cycles have to get compressed.
[16:10] But at the same time, we need to have the ability
to build data centers quickly and repeatedly,
[16:16] most likely with robotics, right? The way
we do it stick built today, and every site,
[16:21] every site is different, is just not a scalable
approach. How do we get energy quickly on site
[16:28] instead of going through the grid, which takes a
long time? And then, how do we make sure that we
[16:32] can actually put this back into the grid and make
it a very good citizen of the grid that actually
[16:38] helps lower electricity prices for everyone? So,
those are all the other things that have to be
[16:44] solved as you solve the chip bottlenecks because
that is actually the big bottleneck. You know,
[16:50] it's 3 years to get just go from start to finish
on building a data center. Well, what that tells
[16:56] you today, you know, we think about some of the
chip evolution and you know, chips becoming more
[17:00] optimized, but in many ways the algorithms may
become less optimized. You know, where you start
[17:06] seeing inference and training folding back on
top of each other, and all of a sudden these
[17:10] more specialized pre-fill and decode chips, you
know, won't be algorithmically suitable you know,
[17:15] for some of the next generation models as well.
So, some of our natural progressions here,
[17:20] you know, that people are looking at for
specialization in the data center and the
[17:23] chip designs could be exactly the wrong things,
you know, by the time they start bringing those
[17:27] products into the marketplace. Well, then do
you worry about I mean like like one obvious
[17:32] objection to all of this is that they're going to
be economic externalities in the form of crowding
[17:38] out effects both at like when if we're going to
make these kind of investments at every step from
[17:43] energy, you know, through that HBM. Like are
you concerned about that kind of concentration?
[17:50] Well, you know, one of the good things is, you
know, that fundamentally you're energy limited.
[17:54] Right? Nobody's going to build a data center for
chips that you can't power up. Nobody's going to,
[17:59] you know, put billions of dollars into chips
that you don't have power for. You know, and I've
[18:03] called it, you know, there's an upper bound on how
you fork we get with our AI investments. And that
[18:09] upper bound is energy capacity, you know, which
in the US is 4 to 5% per year, right? You know,
[18:14] after a dismal decade of about 1% per year. You
know, that is the upper bound on the constraint.
[18:21] What goes into that energy, that we need to
make a lot more efficient. But fundamentally,
[18:25] we need a whole lot more energy, and
we need the energy to more efficiently
[18:29] get to the devices that we're building
so that we improve, right, the, you know,
[18:33] energy per token being produced dramatically.
Well, on that note, I think we're about out of
[18:38] time. Thank you guys both so much, Pat and Sachin,
for joining us this morning. It's been a pleasure.
