CoreWeave Signs 5-Year GPU Deals as Power Bottlenecks Limit AI Growth
Bloomberg
AI labs are locking in non-cancelable five-year compute contracts because scaling laws hold and powered data center capacity, rather than chip availability, has become the primary constraint on growth.
## Background
Published by Bloomberg on June 8, 2026, as an episode of the Odd Lots podcast hosted by Joe Weisenthal and Tracy Alloway, this episode features Brannin McBee, co-founder and Chief Development Officer of CoreWeave. According to the show description, the discussion explores how the market for AI compute has evolved three years after CoreWeave's IPO, focusing on persistent bottlenecks like electricity and transformers, NVIDIA's Vera Rubin systems, enterprise lease durations, and training versus inference demand.
## TL;DR
CoreWeave CDO Brannin McBee details how the corporate landscape for AI compute is shifting as enterprise customers face bill shock and transition to multi-year infrastructure commitments. The conversation highlights that while raw GPU supply constraints have eased, the primary operational bottleneck is now securing powered data center shells equipped with transformers and high-voltage grid connections. McBee explains why customer demand remains overwhelmingly centered on NVIDIA hardware, how inference workloads are dominating compute usage, and how CoreWeave uses structured SPVs to secure long-term debt financing while remaining skeptical of compute commoditization.
## Atmosphere & style
The episode carries an analytical, fast-paced, and insider-driven tone characteristic of financial journalism deep-dives. Weisenthal and Alloway establish a conversational, witty dynamic—replete with humor about corporate AI spending sprees—while McBee provides grounded, pragmatic execution perspectives from the frontlines of AI infrastructure development. The narrative shifts smoothly from high-level corporate market trends to technical mechanics, creating an engaging and intellectually sharp conversation about financial structures and physical data center constraints.
## Key takeaways
- `00:00` Enterprise clients are experiencing bill shock from unchecked AI usage, driving a shift toward model routing to send simpler queries to cheaper, smaller models.
- `05:00` CoreWeave sees robust demand, expanding its footprint across financial services and supporting 9 of the top 10 global AI labs outside China.
- `05:00` AI hardware maintains value longer than previously expected, with older GPUs like Nvidia A100s and H100s remaining active for 6 to 8 years.
- `15:00` Contract lengths for cloud compute have extended from 3 years to 5-year non-cancelable agreements as labs seek long-term capacity guarantees.
- `20:00` Nvidia’s technological roadmap is moving from air-cooled Hopper racks to liquid-cooled Blackwell NVL72 chassis and upcoming Vera Rubin systems.
- `22:30` Nvidia retains its dominant market position across both training and inference due to its mature 15-year CUDA software ecosystem.
- `28:00` The primary operational bottleneck for AI scale is securing powered shells equipped with high-voltage transformers, battery systems, and specialized labor.
- `32:00` CoreWeave intentionally rejects single-client capacity concentration to build a diversified, multi-tenant cloud provider.
- `35:45` Special Purpose Vehicle (SPV) financing structures allow AI cloud providers to pair 5-year take-or-pay customer contracts with fixed debt amortization schedules.
- `40:00` GPU compute cannot be easily traded as a standardized commodity because performance metrics like Goodput and Model FLOPs Utilization (MFU) vary across hardware environments.
- `45:00` Proprietary orchestration software that manages node drops and prevents downtime provides a key competitive advantage over raw hardware access.
- `45:00` A major market divide exists between individual users renting single GPUs and enterprise customers needing massive clusters of hundreds of thousands of GPUs.
## Detailed walkthrough
### `00:00` Introduction and the Rapid Evolution of the Compute Market
The episode kicks off with host Joe Weisenthal musing about the breakneck pace of change in the artificial intelligence landscape. He jokes with co-host Tracy Alloway that they should "bite the bullet" and turn *Odd Lots* into a weekly market update specifically focused on AI compute and inference, as the market moves too fast for occasional check-ins. Moving past early-stage consumer experiments like writing silly poems, the current AI wave is driven by massive corporate spending on raw compute. However, this explosive growth has triggered severe sticker shock among corporate CFOs attempting to manage skyrocketing token costs.
To illustrate this "corporate reckoning" with AI expenses, Tracy highlights recent news stories detailing corporate missteps:
* **Uber's Budget Burn:** Uber reportedly burned through its entire 2026 AI budget in just four months, prompting its COO to explicitly question whether the company was seeing tangible productivity gains to justify the cost.
* **The $500 Million Runaway Tab:** A notable headline from *Axios* reported an AI consultant whose corporate client accidentally spent half a billion dollars in a single month after failing to put strict usage limits on their API calls.
> "An AI consultant told *Axios* that one of their clients recently spent half a billion dollars in a single month after failing to put usage limits on..." — Tracy Alloway
The hosts discuss how employee behavior drives these spikes—such as using the most compute-heavy, frontier-level AI model just to look up a simple guest title. Joe predicts a surge in corporate investment toward **optimal model routing**, where simple queries are automatically diverted to models that cost a hundredth of a frontier model's price. They also recall a live episode with Hudson River Trading's Ian Dunning, who noted that finding physical data center real estate with available power to plug in GPUs has become a bigger bottleneck than sourcing the chips themselves.
---
`05:00` CoreWeave’s Perspective on Enterprise Demand and Token Costs
The hosts welcome back Brannin McBee, co-founder and Chief Development Officer of CoreWeave, a specialized "neo-cloud" provider offering AI training and inference services. Recalling that CoreWeave’s origins were tied to using Nvidia GPUs for crypto mining, Joe asks whether enterprises—reeling from high compute bills in Q2—are beginning to pull back or slow down their rate of AI adoption.
McBee pushes back against the idea of a demand slowdown, clarifying that headline-grabbing bill shocks are simply proof of massive, authentic consumer and enterprise consumption:
* **Product Breakthroughs:** The release of advanced models around January served as a major product breakthrough, particularly for coding and finance professionals who require massive infrastructure scale.
* **Broadening Enterprise Footprint:** CoreWeave now boasts a financial services backlog approaching tens of billions of dollars, alongside supporting 9 of the top 10 AI labs globally (excluding China).
* **Model Routing and Asset Longevity:** McBee agrees with Joe's thesis on model routing, adding that using smaller GPUs for simpler tasks dramatically extends hardware lifespans. Countering 2025 Wall Street fears that GPUs would depreciate to zero after two years, he notes that legacy hardware like Nvidia A100s and H100s are retaining value and remaining active for 6 to 8 years.
> "Users are going to need to find the way to use the appropriate model for their prompts... But that just further enables this concept that infrastructure is going to be used longer." — Brannin McBee
* **Client Base Diversification:** Once criticized during its IPO for customer concentration, CoreWeave doubled its new client "logos" in Q4 alone across enterprise sectors.
* **Direct Institutional Deals:** Financial giant Jane Street recently partnered directly with CoreWeave to build and run proprietary models, bypassing intermediate AI labs entirely.
---
`15:00` The Shift to Longer-Term Infrastructure Commitments
Addressing the shift in how major AI labs approach compute investments, Joe asks whether labs are pivoting funding away from pushing raw model frontiers to building better software "harnesses" (such as automated coding environments). McBee explains that while CoreWeave isn't directly involved in internal lab software decisions, they observe a clear, dramatic change in client behavior regarding infrastructure procurement: AI labs are locking in hardware commitments for longer durations than ever before.
The duration of standard take-or-pay cloud compute contracts has steadily lengthened over recent years:
* **Two Years Ago:** Standard client contracts locked in 3-year commitments.
* **Last Year:** Duration extended to 4-year contracts for Nvidia Hopper (H100/H200) and Blackwell infrastructure.
* **Current Trend:** Labs are now demanding **5-year non-cancelable contracts** to guarantee uninterrupted access to cutting-edge compute, accepting fixed economics without requiring hardware upgrades over the term.
`20:00` NVIDIA's Roadmap from Blackwell to Vera Rubin and the Demand for GPU Generations
Brannin McBee outlines how AI labs are approaching their infrastructure decisions. Rather than short-term experiments, these organizations are committing to larger, multi-year deployments—typically five-year agreements—because scaling laws continue to hold. Increasing raw compute directly correlates with advancing frontier models, and this dynamic is expected to persist through NVIDIA's next-generation architectures.
McBee explains the technological progression of NVIDIA's chips:
* **Hopper:** Deployed in traditional 42U rack chassis, housing around eight GPUs per server case, relying primarily on air cooling.
* **Blackwell:** Transitioned to large liquid-cooled NVL72 chassis—tall, integrated towers housing 72 GPUs per rack.
* **Vera Rubin:** The upcoming architecture currently undergoing initial rack testing, engineered to deliver significantly higher token output per unit of energy consumed.
Despite the hype around new hardware, demand remains strong across all GPU generations. Enterprise clients often prioritize previous or current generations like Ampere, Hopper, and Blackwell due to cost considerations and existing workflows, while cutting-edge AI labs compete to secure the earliest access to Vera Rubin.
* **Deployment Duration:** AI labs are seeking minimum 5-year contracts to lock in compute scale.
* **Cooling Evolution:** CoreWeave ran early liquid cooling tests during the Hopper era to prepare for Blackwell, which is overwhelmingly liquid-cooled in production.
* **Market Segmentation:** AI labs line up first for next-gen silicon (Vera Rubin), while enterprise clients focus heavily on Hopper and Blackwell.
---
`22:30` Proprietary Silicon, Custom ASICs, and the Persistence of the CUDA Moat
Joe Weisenthal notes the complexity of keeping up with GPU names and highlights recent moves by tech giants into custom hardware—such as Microsoft's MAI Thinking One model running on its proprietary Maya 200 chip, or trading firms like Hudson River Trading exploring custom silicon. Addressing whether custom chips threaten NVIDIA's market dominance, McBee clarifies that CoreWeave operates on a demand-led, "success-based CapEx" model. They do not speculatively buy hardware; they only procure and deploy what customers explicitly request and commit to funding.
Currently, customer demand remains almost exclusively focused on NVIDIA infrastructure. McBee attributes this to NVIDIA's 15-year head start building the CUDA software ecosystem, which renders their platform the most efficient, reliable, and scalable hardware stack available. While commentators often argue that non-NVIDIA chips or custom ASICs are viable for inference workloads, McBee emphasizes that CoreWeave's real-world customer traffic tells a different story.
> "The client isn't asking for anything but NVIDIA infrastructure... people are still using NVIDIA infrastructure. They're committing to NVIDIA infrastructure for five plus year contracts in these billion, multi-billion dollar commitments because they know that that is going to be a critical part of how they scale their business."
* **Inference Share:** Inference accounts for well over 50% of active infrastructure utilization on CoreWeave's platform.
* **Fungibility:** CoreWeave uses the exact same physical NVIDIA GPU clusters for both training and inference workloads interchangeably.
* **Alternative Hardware:** While clients occasionally run small-scale trials on alternative silicon, NVIDIA remains the de facto choice across both training and inference.
---
`28:00` Physical Bottlenecks: Powered Shells, Electrical Supply Chains, and Financing
Addressing operational constraints, McBee clarifies that securing physical land or raw GPU chips is no longer the primary bottleneck for expanding compute capacity. Instead, the critical industry-wide constraint is the availability of **powered shells**—fully constructed, energized data center spaces equipped with high-voltage electrical delivery, integrated liquid cooling, and power infrastructure ready to feed electrons into racks.
Building a powered shell requires navigating complex, slow-moving supply chains for critical electrical components:
* High-voltage transformers and backup battery systems.
* Specialized labor, particularly certified high-voltage electricians who require five-plus years of apprenticeship training and cannot be rapidly scaled up.
Financing constraints have eased substantially for CoreWeave due to their proven execution track record. The market has begun differentiating between companies holding hypothetical power allocations for 2030 and operators capable of converting energized sites into active, billable GPU hours today.
* **Capital Raised:** CoreWeave has raised over $21 billion in financing year-to-date.
* **Active Scale:** The company operates over 1 gigawatt of active, GPU-populated data center capacity delivering live compute to clients.
* **Execution Premium:** Converting raw data center space into billable GPU hours is the key operational hurdle driving down CoreWeave's cost of capital.
---
`32:00` Credit Markets, Risk Management, and Customer Diversification
Weisenthal observes that financial stress metrics—such as CoreWeave's Credit Default Swap (CDS) spreads—have tightened significantly compared to late 2025, reflecting diminished market skepticism over GPU depreciation and debt viability. Drawing a parallel to the banking industry, Weisenthal raises the issue of tenant concentration risk, referencing how concentrated depositor bases led to the failure of Silicon Valley Bank (SVB).
McBee agrees that customer diversification is a central focus. CoreWeave faced criticism leading up to its IPO over revenue concentration tied to a few large clients. Today, despite overwhelming demand where single counter-parties offer to take down all unallocated capacity, CoreWeave deliberately rejects single-client concentration in favor of building a broad, multi-tenant cloud serving diverse enterprise AI producers and consumers.
* **Capacity Demand:** Unallocated capacity represents strategic supply being curated for diversification, not unsold excess inventory.
* **Market Perception:** Tighter CDS spreads reflect increased bond market confidence in long-term AI compute demand and CoreWeave's operational stability.
* **Strategic Direction:** CoreWeave is intentionally building a multi-tenant cloud rather than acting as a dedicated host for one or two mega-cap clients.
---
`35:45` Corporate Bond Markets and Structured SPV Financing Mechanics
Tracy Alloway and McBee explore how the institutional debt market warmed up to funding large-scale AI infrastructure. Investor confidence shifted markedly in late Q4 and early Q1 as inference-driven demand surged, demonstrating that GPUs generate recurring operational cash flows beyond experimental training budgets.
McBee breaks down CoreWeave's capital architecture into two distinct layers:
* **ParentCo Financing:** Handles corporate-level equity and balance sheet activities.
* **AssetCo Financing:** Houses hardware assets inside Special Purpose Vehicles (SPVs), colloquially referred to as "boxes."
Within each SPV, CoreWeave packages physical GPU racks, data center leases, and debt obligations against a non-cancellable, 5-year take-or-pay customer contract. The long-term revenues flow directly into the SPV, systematically servicing data center operating costs and paying down debt amortization schedules over the 5-year term.
> "You're able to pair this like five year take or pay contract to an amortization schedule on the debt. And you have the revenue come into the box, pay down the amortization schedule, pay down the operating costs of the data center..."
`40:00` SPV Financing Structure and the Non-Fungibility of GPU Compute
Brannin McBee outlines CoreWeave’s sophisticated financing strategy, explaining how highly profitable contracts are housed within Special Purpose Vehicles (SPVs) to access credit markets directly. He highlights a recent milestone: a first-of-its-kind, investment-grade-rated, non-recourse HPC infrastructure financing deal (DDTL4) priced at SOFR plus 225 basis points. By unlocking insurance charge-of-capital alongside convertible notes, unsecured debt, and strategic equity, CoreWeave is actively establishing the blueprint for financing AI hyperscalers.
When asked whether a liquid, tradable futures market for GPU capacity will emerge—such as the contract being developed by the Compute Exchange for CME listing—McBee expresses near-term skepticism. Drawing on his past experience as a commodity trader in natural gas, electricity, and agriculture, he explains that true commoditization requires **fungibility**. Unlike gold, GPU compute is currently non-fungible because hardware performance varies drastically between cloud environments.
* **DDTL4 Financing Details**: Term loan structured at SOFR + 225 bps as an investment-grade, non-recourse HPC financing vehicle.
* **Performance Variance Metrics**:
* **Goodput**: Measures the fraction of peak hardware performance that a training job actually extracts.
* **MFU (Model FLOPs Utilization)**: A key hardware metric for evaluating real-world floating-point efficiency in LLM training.
* **Configuration Standard**: CoreWeave builds to Nvidia’s **DGX Reference Spec**, which is considered the baseline requirement for optimal GPU deployment.
---
`45:00` Software Differentiation, Scale Bottlenecks, and the Geopolitics of Compute
McBee emphasizes that hardware configuration to DGX reference spec is merely a starting point. CoreWeave's primary competitive moat lies in its proprietary software stack, which manages infrastructure failures, predicts node drops, and hot-swaps hardware seamlessly to prevent downtime. As the industry advances from air-cooled 42U Hopper racks to hyper-dense liquid-cooled Blackwell and Vera Rubin architectures, operating complexity increases—further delaying any immediate commoditization of compute.
```
OPERATIONAL COMPLEXITY ESCALATION OVER TIME
[ Hopper Era ] [ Blackwell Era ] [ Vera Rubin Era ]
• Air-cooled 42U racks • Ultra-dense liquid • Advanced custom
• Simpler provisioning cooling systems topologies
• Lower fault risk • Complex fault-spread • Specialized software
management requirements stack needed
───────────────────────────────────────────────────────────────────►
(Commoditization threshold moves further away as complexity rises)
```
Addressing why small-scale inference users (such as host Joe Weisenthal, who trained a fine-tuned BERT model via Modal for $5) can easily find cheap capacity while enterprises struggle, McBee points to a massive **scale disconnect**. Securing single GPUs or small instances is easy; securing dedicated clusters of hundreds of thousands of interconnected GPUs is a hyper-scarce capability restricted to a few top operators globally.
* **Target Deployment Scale**: CoreWeave’s primary clients negotiate deployments involving **hundreds of thousands of GPUs**, where market liquidity completely disappears.
* **Geographic Strategy**: CoreWeave operates in the US, Canada, and Europe, with plans to expand further east, though it explicitly avoids the mainland Chinese market.
* **Market Demand Outlook**: Demand for AI compute remains unyielding, with supply chain constraints preventing any near-term or medium-term saturation.
---
`50:00` Data Center Permitting Obstacles and the Commodity Market Debate
Addressing political pushback and anti-data center sentiment—such as high-profile project cancellations in Northern Virginia—McBee states that CoreWeave handles community relations and permitting proactively. While public debates often center around water consumption and local grid strain, he notes that operational delays are primarily driven by hard supply chain lead times: constructability, transformer delivery, and physical equipment installations.
Following McBee's exit, co-hosts Joe Weisenthal and Tracy Alloway debate the broader market structure implications. They reflect on the tension between standardized hardware specs and the custom software needed to run them efficiently. If cloud providers can run workloads efficiently only through custom orchestration, compute cannot be traded as a simple standardized commodity.
> "Even if you're not that interested in AI, this is a really interesting market structure story... It's basically the creation of a brand new market and poses all these interesting philosophical questions about how you do that." — Joe Weisenthal
* **Northern Virginia Context**: Referenced as a prime example of a data center hotspot experiencing heightened local government scrutiny and project delays.
* **Core Operational Constraint**: Transformer lead times, grid interconnect delays, and physical construction remain larger bottlenecks than local policy hurdles.
---
`55:00` Custom Silicon vs. Standardized GPUs and Podcast Outro
The hosts explore the rise of proprietary in-house AI chips—such as Google's TPUs, Amazon's Trainium, Microsoft's custom silicon, and potential proprietary designs from quantitative trading powerhouses like Jane Street and Hudson River Trading. They note that the industry is split between general-purpose Nvidia platforms and custom hardware tailored for specific model architectures.
Alloway and Weisenthal wrap up the episode, teasing a future deep-dive episode into custom silicon, model-chip alignment, and non-Nvidia hardware efficiency. They sign off with standard show closing information, discord references, and sponsor announcements.
* **Hosted By**: Tracy Alloway (`@TracyAlloway`) and Joe Weisenthal (`@TheStalwart`).
* **Featured Guest**: Brannin McBee (`@BranninMcBee`), Co-Founder & Chief Development Officer at CoreWeave.
* **Production Team**: Carmen Rodriguez, Dashiell Bennett, Kale Brooks, and Kevin Lozano.
* **Community Platforms**: Odd Lots Discord (`discord.gg/oddlots`) and the Bloomberg Odd Lots newsletter.
<FollowUp label="Explore how GPU architecture choices impact Goodput and MFU metrics" query="Explain how hardware design differences between Nvidia Hopper, Blackwell, and custom chips affect Goodput and MFU in AI model training."/>
## Notable quotes
> `00:00` An AI consultant told Axios that one of their clients recently spent half a billion dollars in a single month after failing to put usage limits on... — Tracy Alloway
> `05:00` Users are going to need to find the way to use the appropriate model for their prompts... But that just further enables this concept that infrastructure is going to be used longer. — Brannin McBee
> `22:30` The client isn't asking for anything but NVIDIA infrastructure... people are still using NVIDIA infrastructure. They're committing to NVIDIA infrastructure for five plus year contracts in these billion, multi-billion dollar commitments because they know that that is going to be a critical part of how they scale their business. — Brannin McBee
> `35:45` You're able to pair this like five year take or pay contract to an amortization schedule on the debt. And you have the revenue come into the box, pay down the amortization schedule, pay down the operating costs of the data center... — Brannin McBee
> `50:00` Even if you're not that interested in AI, this is a really interesting market structure story... It's basically the creation of a brand new market and poses all these interesting philosophical questions about how you do that. — Joe Weisenthal
## Mentioned
CoreWeave, Brannin McBee, Joe Weisenthal, Tracy Alloway, Nvidia, Bloomberg, Uber, Axios, Hudson River Trading, Ian Dunning, Jane Street, Microsoft, Compute Exchange, CME, Modal, Google, Amazon, Carmen Rodriguez, Dashiell Bennett, Kale Brooks, Kevin Lozano, Nvidia Hopper, Nvidia Blackwell, Nvidia Vera Rubin, Nvidia A100, Nvidia H100, Nvidia H200, Nvidia NVL72, Maya 200, BERT, Google TPU, Amazon Trainium, Amazon Inferentia, Microsoft Maia
CoreWeave Chief Development Officer Brannin McBee asserts that enterprise AI compute demand is accelerating rather than slowing, leading AI labs and enterprises to commit to unprecedented five-year infrastructure contracts. Rather than chip availability or raw real estate, physical power delivery and operational complexity now dictate industry bottlenecks, keeping high-performance compute resistant to financial commoditization.
Enterprise Demand and Multi-Year Infrastructure Commitments (5:00)
Despite corporate alarm over surging AI expenses—exemplified by Uber exhausting its 2026 AI budget in four months and an enterprise client racking up a $500 million monthly bill—McBee contends these events reflect authentic, expanding consumption rather than budgetary retrenchment. CoreWeave reports a financial services backlog approaching tens of billions of dollars, alongside serving 9 of the top 10 global AI labs outside China.
To manage costs and preserve access, client behavior has shifted from short-term experimentation to locking in long-term capacity. Standard take-or-pay cloud contract lengths have extended from three years to four, with frontier AI labs now committing to five-year non-cancelable agreements without hardware refresh requirements. Furthermore, optimal model routing—diverting routine queries away from frontier models to smaller instances—has countered early market fears of rapid GPU obsolescence, extending the operational lifespan of hardware like Nvidia A100s and H100s to an estimated six to eight years.
Persistent Nvidia Dominance Across Diverse Workloads (22:30)
Despite growing industry interest in custom ASICs and hyperscaler silicon, CoreWeave operates under a demand-driven capital expenditure model where procurement is dictated entirely by client requests. McBee states that customer demand remains almost exclusively concentrated on Nvidia hardware, driven by the maturity and efficiency of its 15-year-old CUDA software ecosystem.
While industry observers often argue that alternative chips can adequately service inference workloads, CoreWeave's operational data shows that Nvidia chips dominate both training and inference. Inference workloads now account for well over 50% of CoreWeave's active capacity, yet customers deploy the same fungible Nvidia GPU clusters interchangeably across both training and inference tasks rather than migrating to specialized non-Nvidia hardware.
Electrical Supply Chains as the Primary Bottleneck (28:00)
Securing physical land and sourcing individual GPUs are no longer the primary operational hurdles for AI expansion. Instead, the critical constraint is the availability of "powered shells"—data centers fully provisioned with high-voltage electricity, utility interconnections, and integrated cooling systems ready to energize server racks.
Expansion timelines are dictated by severe supply chain delays for electrical equipment, notably high-voltage transformers and backup battery arrays, as well as a severe shortage of certified high-voltage electricians, who require more than five years of apprenticeship training. Capital markets have increasingly favored operators who can translate power into active compute; CoreWeave has raised over $21 billion to support more than one gigawatt of energized, operational capacity.
Structured SPV Financing and Compute Non-Fungibility (40:00)
To fund massive capital expenditures without overloading the corporate balance sheet, CoreWeave structures its hardware deployments into bankruptcy-remote Special Purpose Vehicles (SPVs). These entities package GPU hardware, data center leases, and non-cancelable five-year take-or-pay customer contracts together, matching predictable revenues directly against debt amortization schedules. This structure recently enabled an investment-grade-rated, non-recourse debt facility (DDTL4) priced at SOFR plus 225 basis points.
McBee expresses skepticism toward near-term attempts to create standardized financial futures or liquid commodity markets for compute capacity. Because hardware performance metrics like Goodput and Model FLOPs Utilization (MFU) vary wildly depending on network topology, liquid cooling setups, and proprietary orchestration software, large-scale compute clusters running hundreds of thousands of GPUs remain fundamentally non-fungible.
While structured debt markets have unlocked the capital necessary to fund large-scale AI infrastructure, physical electrical constraints and proprietary orchestration software continue to separate high-end performance from commoditized compute. Whether emerging custom hyperscaler silicon and standardized software frameworks will eventually break Nvidia's ecosystem advantage remains the central unresolved tension in compute market structure.
[0:00] Tracy Alloway: An AI consultant told Axios that one of their clients recently spent half a billion dollars in a single month after failing to put usage limits on...
[5:00] Brannin McBee: Users are going to need to find the way to use the appropriate model for their prompts... But that just further enables this concept that infrastructure is going to be used longer.
[22:30] Brannin McBee: The client isn't asking for anything but NVIDIA infrastructure... people are still using NVIDIA infrastructure. They're committing to NVIDIA infrastructure for five plus year contracts in these billion, multi-billion dollar commitments because they know that that is going to be a critical part of how they scale their business.
[35:45] Brannin McBee: You're able to pair this like five year take or pay contract to an amortization schedule on the debt. And you have the revenue come into the box, pay down the amortization schedule, pay down the operating costs of the data center...
[50:00] Joe Weisenthal: Even if you're not that interested in AI, this is a really interesting market structure story... It's basically the creation of a brand new market and poses all these interesting philosophical questions about how you do that.
Original transcript
[0:00] Whatever your goal, trade show giveaways, client gifts, or team gear, 4imprint has the promo products to match. With thousands of options, from apparel and drinkware to tech and totes, it's easy to find the right fit for your brand and budget, with standout choices at every price point. And with their 360-degree guarantee, you can be 4imprint certain your order will show up just right, right on time. Explore more at 4imprint.com. 4imprint. 4certain. So there's a lot of noise about AI. But time's too tight for more promises. So let's talk about results. At IBM, we work with our employees to integrate technology right into the systems they need. Now, a global workforce of 300,000 can use AI to fill their HR questions, resolving 94% of common questions. Not noise. Proof of how we can help companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business. IBM. Wise is the smart way to manage the currencies you need around the globe. When you send money abroad using your bank, you could get hit with hidden fees and exchange rate markups. There's a better way. Try Wise. Wise uses the exchange rate you usually find on Google with no unwelcome surprises Plus most transfers happen in under 20 seconds which means your money arrives in less time than you been listening to me It simple and free to sign up when you download the WISE app. Be smart, get wise. T's and C's apply. Bloomberg Audio Studios. Podcasts, radio, news. Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal. And I'm Tracy Alloway. Tracy, I'm envisioning this future where like we have to do a state of the sort of AI inference market episode like once a month, you know, Like where it's like things are moving so rapidly and there's so much change either in terms of what models are using or what they're being used for, et cetera, that in the same way we would do like, you know, the occasional regular stock market episode or whatever, we would just do, OK, what are we seeing right now in inference trends? Because it just feels like the moment we do an episode a few weeks later it may be out of date We should just bite the bullet and do a weekly episode Transform lots more into a market update on compute We could do inference I don know We have to workshop Odd inference? No. No. We'd have to. But anyway. Lots of inference. Lots of inference. This is like the story of the moment. And we know that, you know, a couple years ago, everyone was sort of dabbling around with various things and experimenting and using AI. I had like, oh, like write a poem for me about this, et cetera. That phase of AI is long over. And we know that companies specifically are spending a ton on compute, so much so that CFOs around the world are getting sticker shock about their compute budgets. And there was even a headline of like Uber saying like, OK, like fifteen hundred dollars of max per employee, like don't spend more than that in a month on tokens. So like this is a very fast moving area. Yeah, you're starting to get headlines about, I guess, a corporate reckoning with AI as more people experiment and spend money on it. The Uber headline that you mentioned, apparently Uber burned through its entire 2026 AI budget in four months, basically. And like, what's more important is the COO was actually asking whether or not that was worth it like whether they saw productivity gains or whatever as a result of that the other very amusing headline that i saw and it was citing an unnamed source it's from axios so you know oh yeah not entirely sure it's true but reportedly a great headline it was a great headline an ai consultant told axios that one of their clients recently spent half a billion dollars in a single month after failing to put usage limits on yeah it's Because everyone is like, oh, I just have a simple question. I want to look up our guest's title. I'm going to use the most advanced model to do that, et cetera. I have a theory, and we will get into this with our guests, that one of the things that will – and we've talked about this with Goldman's Marco Argenti. But one of the things I predict is that companies are like clearly – they're going to keep using it more and more would be my guess. but there were probably a lot of investment made in sort of like optimal model routing because some models are like a hundredth per query of what a frontier model is probably a lot of people don't know like what is the sort of like efficient frontier model usage and so actually routing the query to the sort of most uh efficient model i have a feeling we're going to see a lot of investment in that area specifically well there's also just the question of whether or not the models get cheaper overall as they advance right and we have a lot of investment in that area
[5:00] I've seen some, I think NVIDIA has a new system or chip out or something that is supposed to reduce token usage. We can get into that as well. And, you know, we did that live episode recently with Ian Dunning of Hudson River Trading, and he said a lot of interesting things in that. But one of the things he said is that the scarcity is increasingly like just the real estate component. finding a suitable place to plug in your GPUs, at least from his perspective right now, is as much, if not more so of a challenge than securing GPUs themselves. Which is different to what it was like three years ago. Yeah. Yeah. So just like where you plug it in, we know there's all the like the anti-data center politics out there. So it's like, yeah, we got to take the pulse of this market. All right. Consider this our inference update. Yeah. Well, I'm really excited to say we really do have the perfect guest. Someone we spoke to truly feels like eons ago. I think the first thing we ever connected with this company, they've always had a lot of chips. But I think the first time we ever linked up with this company was still in the era where people were excited about NVIDIA chips being used for crypto mining and stuff like that. But we are now in this very different era. And this is truly one of the companies of the moment And that is of course CoreWeave one of the so NeoClouds offering both training and inference services for all sorts of different AI workloads I'm very excited to say back on the show, we have Brandon McBee, CoreWeave's co-founder and chief development officer. So, Brandon, thank you so much for coming on OddLots. Appreciate being invited back, guys. And that was a fantastic intro. We look forward to hitting these topics today. All right. Here's my question. So, we know that at the tail end of last year. And then in the first quarter of this year, everyone started using cloud code and just this clearly a key inflection moment for sort of like overall AI demand. And then we get into Q2 and suddenly the CFO is like, oh my gosh, we're spending this much on inference. We got to like figure things out just straight up. Like in the last month, whatever, do you see any signs of that happening yet of these companies, which are all like still AI, eager AI adopters trying to get a little bit of a handle and maybe slowing the rate of the rate of growth? Is that happening yet Yeah I think you see headlines there that there are surprises of spend etc I say our interpretation of it is entirely look at the authentic and foundational demand that is out there All we're really doing is talking about how much consumption there is of AI and use for it. And I think that that was a real question in the market 12, 18, 24 months ago is, will there be demand for AI? Where is this inference demand that everyone's been talking about? And I think you're absolutely correct. January or so with this kind of like next group of models that were coming out everyone all of a sudden and all at once said this is what we've needed. Like this is the real product breakthrough. But I think it's worth keeping in mind that product breakthrough was like for a limited set of people at the end of the day, right? We're talking like coding professionals, some finance professionals, but it's a relatively small group of people that are using infrastructure at this enormous scale. And so where we see this moving towards next is broader enterprise use, like likely not seeing this whole token mapping approach And I think that that is unsustainable But do we see adoption in other sectors and how this can continue to spread out Absolutely I mean you know on our end I think we have 10 over billion clients at this point And our financial services client backlog is in the tens of billions of dollars at this point And so we're now talking about things outside of AI labs, outside of hyperscalers. And look, as you guys know, we support nine of the top 10 AI labs on the planet. If you exclude China and everything that's going on over there, like we have a lot of visibility into what people are doing. And we're not seeing any pullback on what they're doing on inference today. If anything, it just remains this unrelenting demand for access to the best technology solution in the market for running artificial intelligence. And that's core weak solution in the market. Wait, say more about the customer mix now versus, say, three years ago. So you have hyperscalers, you've got startups, you've got various businesses. How has that, I guess, composition shifted over time? Yeah, it's.
[10:00] It's shifted enormously towards a more diverse customer base, right? We got a lot of flack for this in our IPO, right? People were noting that we only had a handful of large clients, that our clients were just the hyperscalers and AI Lab or two. And I think that we have made tremendous progress in driving diversification. So I'd say it's probably across three buckets today, right? We have hyperscaler clients who continue to grow with us. We have AI lab clients. As I said, nine of the top 10 AI labs on the planet choose CoreWeave. And then we have this enterprise base. And the enterprise base just doesn't grab as many headlines as you would expect. This is not these massive multibillion dollar contracts that are being signed. But I think in Q4 alone, we added twice as many logos to our client base as we had ever done versus any previous core. And that enterprise base is the one that's growing so much. And there was a point you guys hit on in the intro that I think is really worth acknowledging And it was this concept of model routing and the idea that like not everyone needs just the latest model that it's different types of models. I can hit different use cases. And this is something we've been talking about for a while, right? As it relates to the infrastructure side of things as well, right? Because you don't need that latest model for everything. And accordingly, you don't need the latest piece of infrastructure to support every single inference or training query that's out there. You can kind of conceptualize this matrix of different sizes of workloads relative to different sizes of GPUs. And all of a sudden that tells you, my God, like H100s could last six, seven, eight years. A100s are going to last longer. And it totally changes the entire conversation around depreciable life of infrastructure as that was a really popular topic during 2025. People were saying like, oh, this stuff will last two years. It's worth zero afterwards. And like, we've never seen any semblance of that because of the point you guys are accurately making, which is users are going to need to find the way to use the appropriate model for their prompts And that be solved by ModelRabbit to your point But that just further enables this concept that infrastructure is gonna be used longer. And we see that every day in our portfolio, extending all the way back to A100s. I just wanna ask a specific question about the broadening out of the customer base. And you mentioned, for example, financial services clients. When you talk about, say, a financial services client as being distinct client from one of the major AI labs, does that mean what you're saying? So it's like I'm just making it up. Let's just say I don't know if these relationships exist. Let's say a city group has an enterprise license with an anthropic. Does that count as anthropic as a customer or city as a customer? And when you talk about this broadening out, are there essentially more types of entities who are building some type of model, not necessarily an LLM per se, but some type of internal house specific model from which they want to run inference? It's a great question. The scenario you presented anthropically be our client. OK, got it. So what I highlighting I want to correct a number I said earlier are financial service clients And this is direct to those financial services they approaching billion in backlog So this would be a good example of this and that something we made recently is with Jane Street That's not Jane Street coming through OpenAI or Anthropik to get to us. That is Jane Street coming directly to us and using our platform. For a model that they're building. So it's a Jane Street. Or inference, right? It's training. No, no, no. I'm not setting aside training, but it would be inference of a model that is Jane Street's model of something rather than Jane Street's contract and enterprise relationship with one of the major labs. At the end of the day, we don't know what exact workloads these entities are running, especially for entities like Jane Street. I would imagine that's highly secretive. But the point I would say is more that this is not them coming through an AI lab to us. they are interfacing with and managing the infrastructure directly on our platform. And that's a really important distinction as we grow this diversified client base. And again, I think that we've just done a wonderful job of executing on that over the past year.
[15:00] Whether you're planning a big tech event, launching a new campaign, or just stocking up on team gear, finding the right promotional products makes all the difference. 4imprint offers thousands of options. From on-trend apparel and premium drinkware to tech, totes, and giveaways, so you can find the right fit for any audience, purpose, or budget. You can customize it all. Your logo, your message, your look, and many items come with no setup charge to help you save. And if you're really watching the bottom line, you'll find standout choices at every price point so you can make a real impact while staying on budget. Plus, you'll get expert help, fast turnaround times, and their 360-degree guarantee. So you can be 4imprint certain your order will arrive on time and look exactly right. Whatever your goal, 4imprint makes it easy to find your perfect promo match. Explore the possibilities today at 4imprint.com. 4imprint. 4certain. buying the dip, manually sweeping idle cash, putting on a hedge. On public, you can now create AI agents that handle all these tasks on your behalf. Just describe what you want to do in plain English. Like, if the VIX hits 25, buy a put option on the S&P 500. Or, if my cash balance goes above $20,000, move the excess into my direct index. You approve of the workflow, and your agent handles the rest. Monitoring the market, watching for your conditions, and executing your strategies exactly as defined. An investing platform driven by your intent, not just your clicks. You can also get full read and write access to your account via the public API. Go to public.com slash market and fund your account in five minutes or less. That's public.com slash market. Paid for by Public Investing. Brokered services by Open to the Public Investing, Inc., Member FINRA, and SIPC. Advisory services by Public Advisors, LLC, SEC Registered Advisor. Complete disclosures available at public.com slash disclosures. The thing about AI for business, it may not automatically fit the way your business works. At IBM, we've seen this firsthand. But by embedding AI across HR IT and procurement processes we reduced costs by millions slash repetitive tasks and freed thousands of hours for strategic work Now we helping companies get smarter by putting AI where it actually pays off deep in the work that moves the business. Let's create smarter business, IBM. As you've talked about including in earnings releases, and as you can just tell from these huge token budgets, inference demand is booming, but model training is still important. But in addition to model training, so you say, okay, if you have a pie chart, the part that's inference is getting bigger, but I assume the training is also growing as well. But I'm curious from the perspective of like, say, the AI labs, when they think about growth, has there been a subtle shift from investing to push the pure model frontier, having the absolute best state-of-the-art model versus investing in, say, better harnesses? Because a big reason we're excited and all talking about AI right now is really the excitement that happened over with Cloud Code in the final quarter of 2025 And it like oh this harness has really unlocked a bunch of capabilities Has there been a shift in investment from rather than just the purest most advanced model to let invest more in tooling capacity and other things that allow companies and clients to get more juice from an advanced model I don't think that we're exposed to that decision making with the AI labs as counterparties to us. The observation I would make in a behavior change for the AI labs is they want access to more infrastructure for longer duration. And I'll qualify it a little bit, which is a year or two years ago, we were signing three-year committed contracts. The type of contracts we sign are basically like take-or-pay contracts, which is the best way to finance the infrastructure that we are building for our clients. last year, it was four year contracts, right? They were saying we want explicit access to Hopper for four years or Blackwell for four years. Now they're coming and saying, well, actually we want it for five years. We don't want any interruption of use. We'll commit to the exact same economics throughout the full duration of the contract. You can't upgrade or change the infrastructure within it. You cannot cancel the contract. We
[20:00] want it for five years and they want it at more scale, right? The deployments are getting larger and larger. So that's probably the best characterization we can offer on decision-making that AI labs are going through right now as they look from an infrastructure perspective. It absolutely seems like tooling is important, but scaling laws are still holding, right? Like your ability to advance your frontier model through accessing more infrastructure at scale holds. and that will hold through Vera Rubin, we expect. And seemingly it's not stopping anytime soon. Oh yeah, what's the deal with Vera Rubin? Can you explain that to us? Which aspect of it? What is it? Oh, yeah, yeah, yeah, yeah. Basically. Yeah, so it's just NVIDIA's next architecture that's coming out, right? Like the current architecture that we're deploying today is Blackwell. Blackwell comes, we deploy predominantly in a NBL 72 configuration, which was an entire architecture change. from deployment, right? If you recall, Hopper came before Blackwell. Hopper, you could deploy these 42U racks, which was typically like eight GPUs in a server case. You would take it plug it in largely air as well right We ran some liquid cooling just so we understood the requirements of liquid cooling because Blackwell for our deployments is overwhelmingly liquid in its deployment configuration. And instead of eight GPUs in a 42U configuration, it's in this larger 72 GPU rack. It's like an entire chassis that's being brought in, And it just looks entirely different in the data center. It's like this giant tower thing that you've seen in pictures floating around on X. So Verirubin will be the next architecture that comes out. And we've started receiving testing racks for Verirubin. But the basic idea is like the new configuration makes the whole system more efficient, like more tokens per energy use and that sort of thing. Yes. Yeah. I think that's kind of where you're getting to with it. But that doesn't necessarily mean, going back to the point earlier, that everyone only wants the latest generation of GPU. Right. We have massive demand for Ampere, Hopper, Blackwell, etc. And it just varies by use case model and type of client as well Like I would qualify that AI labs are probably the ones who are lining up first to secure access to the latest generation GPUs whereas enterprise clients might be very focused on current generation like Hopper and Blackwell right now I'm going to be honest for a second. You know, I try to keep up on a lot of things AI related. I really do. And every single day, the one thing I do not keep like in my mind, if you asked me, like I liked it in the old days when it was like 186, 286, 386, 486, Pentium, and then like Pentium 2, et cetera. There was just this numerical sequence that I could keep track of in my head. And so if someone asked me like Joe, like Vera Rubin, Hopper, Blackwell, what was the sequence? I'd be like, I got to be honest with you. I don't exactly remember. And I will prioritize that at some point. But speaking of silicon, so yesterday Microsoft came out with a big, they're really, they wanna be in the game too. They don't wanna just be connected to the labs. They wanna have advanced models too. And apparently it's a good model. And they announced the MAI Thinking One model but they said it optimized on the Maya 200 chip which is their own chip And this is a thing which is even again going back to our recent conversation we had, even a place like Hudson River Trading is thinking about getting into the customized hardware game. How much juice for the squeeze is there of aligning the model with custom silicon from your vantage point what what we could offer is what we hear from our clients yeah on that and it's important to keep in mind we can run any type of silicon on our platform okay right we are entirely customer-led in what we build like we don't go commit to capex and speculatively hope people come and use infrastructure right like we wait until a client says we want you to go do this specific build, here's what we want it to look like. And then we go commit to that CapEx, right? It's more like a success-based CapEx approach. And the client isn't asking for anything but NVIDIA infrastructure. And I think a large contributor to that is, I mean, they built this incredible ecosystem around their chipset. They have been dedicated to that for, I think, over 15 years at this point through the CUDA architecture.
[25:00] And NVIDIA, from what we hear from our clients, that platform just remains the most efficient, the most scalable, the most reliable set of infrastructure that is in the market. So I think others, there's always been, I mean, if you think over the past few years, there's always been talk like, what is it? But yeah, this other silicon and these other chips. And at the end of the day, like people are still using NVIDIA infrastructure. They're committing to NVIDIA infrastructure for five plus year contracts in these billion, multi-billion dollar commitments because they know that that is going to be a critical part of how they scale their business. We really don't see demands on a material basis for anything but that NVIDIA compute. And that's what we are building today. Obviously, just to push back on this a little bit, and I'm not really in any position to push back. I can only relay what past guests have said in my own reading. So what one of our guests said is that absolutely NVIDIA has the lock on model training that if you want to train a model that yes NVIDIA chips are the only game in town But then for inference there really his view this is Ian Dunning again his view is there really were options. And then, of course, we had someone who was much more biased. We interviewed the CEO of Cerebros, the company that makes the gigantic plate and, or sorry, the gigantic chip. The gigantic plate. And of course, he did. But I mean, of course, he was going to say, yeah, the CUDA moat is vastly overrated for inference. It barely exists. Now, of course, of course, he's going to say that. So like, you know, he's in a competitor. But we've also heard it from a user of inference. And intuitively, it makes sense. Like training is very complicated and all that stuff. But what you're saying is that from the customer standpoint, you see the demand for NVIDIA on both the training and the inference as being steady and that you perceive that advantage to be consistent through both aspects? So I believe in our last quarterly report, our CEO might qualify that inference workloads represent well in excess of 50% of infrastructure utilization on our platform. Okay. It's the exact same infrastructure that we use for training as well right Going back to my comment of like it very fungible between those different types of workloads Those customers are choosing NVIDIA to work with on inference I think what you going to see is people will want to try at small scale other types of silicon, but the reliable, proven and remains from our perspective, most efficient infrastructure to use is NVIDIA today. Does that change over time? Who really knows? But I think we've seen NVIDIA battling this concept for years and every year they show up and like they remain the de facto choice for AI infrastructure. I think we're going to be one of the first people in the market to see it because that will be a tone shift change from our clients asking us to run something else. That hasn't happened. Okay. So have the constraints on your business changed at all? So three years ago, we were talking about GPUs and how hard they were to actually get. I imagine securing GPUs is still competitive, to say the least. But are you seeing other constraints emerge, like Joe mentioned in the intro, just land usage just places to actually build data centers Land usage specifically I wouldn say is as much of a concern Having a powered shell is the bottleneck today. And let me qualify a powered shell. Powered shell is effectively an empty data center that is energized. It has all the power and associated components. I can come into it and deliver electrons into a rack. has the cooling system built within it. Like it has a whole thing, right? A powered shell is the industry term for it. That is the bottleneck because of all of the supply chains that come into that, right? Like not only do you have electricity, do you have the land, et cetera, but you have the backup battery supplies, you have the transformers, you have personnel, right? Let's just think about the electricians for these sites and getting the accreditation on the electrician side to be able to participate in these bills. I mean, I think it's a five-year plus apprenticeship to be able to go through that program, right? We can't just make new electricians leveraging a supply chain, right? Like that's a trade that you can't really scale efficiently. So that is absolutely the bottleneck for us. And I think our peer set,
[30:00] that's out there right now. Access to chips. I think we have a phenomenal relationship with NVIDIA where we've just proven to be the best operator of this infrastructure on the planet. You know, a bottleneck that existed for us previously, I think, was access to financing. Yeah. Doesn't seem to be an issue anymore. I would agree with that broadly, but that's years of work and execution that has delivered that ability for us. I mean, year to date, we've raised over $21 billion of financing for our business. Like you don't get to do that and just go from zero to 20, right? Out of nowhere. And I think that's largely driven by our track record of execution, right? Our investors, our creditors can see this deep set of experience over the years of consistently delivering on these builds. I mean, we have over a gigawatt in active power at this point, right? Like a gigawatt like at the data center level with GPUs you know delivering into clients And I think that there has been kind of a misunderstanding in the market where people are conflating the concept that like you know something on paper is the equivalent to being physically done and delivered. And all I can say is there's an enormous gap between signing for power delivery in 2030 versus actually delivering that into billable GPU hours. And that gap of execution is what has driven down our cost of capital so aggressively. That gap is where our business sits and why it's been so successful. I mean, that's the secret sauce is our ability to take these data center deployments and these customer relationships and deliver billable GPU hours into them. You know, speaking of financing, I just want to say, you know, during last year, like maybe six months ago, that might have been the sort of near peak of the Michael Burry inspired. These chips are like in the last two years stuff. And one of the viral charts that you would see on Twitter was the core we've CDS chart. Those have come way in So it is it is you know I haven seen those charts in a while yeah that right that the thing about cds no one ever posts charts of credit default swaps when they coming in they people love to post them when they're blowing out they have um come in so you know that does speak to some of this point about these anxieties uh having been alleviated at least uh somewhat since the start of the year you know it occurred to me like we're talking about credit default swaps So we're talking about financing. I'm sort of gearing up to write a big thing maybe, but I'm writing it in my head currently, that there really are a lot of analogies between the business of data centers and the business of banking. And one of the things in banking, as we all learned from SVB, was the risk of industry and depositor concentration. That if you have all your depositors are either in like one depositor gets too big or all your depositors are in the same industry, then you have this risk of like correlated withdrawals. And that's what obviously did in SVB. When you think about planning and you think about, OK, here's an investment, et cetera. How much does this come up sort of like thinking about I guess tenant diversification as something that you think about in your multi planning It a critical aspect of it right as i said earlier too like this was a key criticism of us coming into our idea last year right where we had that customer concentration in our revenue and we have made enormous progress there and i think the best way to think about it is we could take all of our unallocated capacity and i say that very specifically it's not unsold capacity implying that there's no demand for it It's unallocated. There's intense demand for it. We're figuring out where it should go. And that customer piece of it, I think, honestly, we could allocate all that capacity to single-name clients. There is a pretty significant number of single-name clients we can go allocated out into. But I don't think that is the business we are supposed to be building here. I think the business we are supposed to be building is a diversified cloud that is supporting the leading AI consumers and producers on the planet. I don't think we're supposed to be supporting just one or two companies.
[35:00] Thank you. watching the bottom line, you'll find standout choices at every price point so you can make a real impact while staying on budget. Plus, you'll get expert help, fast turnaround times, and their 360-degree guarantee so you can be 4imprint certain your order will arrive on time and look exactly right. Whatever your goal, 4imprint makes it easy to find your perfect promo match. Explore the possibilities today at 4imprint.com. 4imprint. 4certain. Support for this show comes from public.com. If you're actively involved in your portfolio, you probably catch yourself repeating the same actions, buying the dip, manually sweeping idle cash, putting on a hedge. On public, you can now create AI agents that handle all these tasks on your behalf Just describe what you want to do in plain English Like if the VIX hits 25 buy a put option on the S 500 Or if my cash balance goes above $20,000, move the excess into my direct index. You approve the workflow and your agent handles the rest. Monitoring the market, watching for your conditions, and executing your strategies exactly as defined. An investing platform driven by your intent, not just your clicks. You can also get full read and write access to your account via the public API. Go to public.com slash market and fund your account in five minutes or less. That's public.com slash market. Paid for by Public Investing. Brokered services by Open to the Public Investing, Inc., member FINRA, and SIPC. Advisory services by Public Advisors, LLC, SEC Registered Advisor. Complete disclosures available at public.com slash disclosures. The thing about AI for business, it may not automatically fit the way your business works. At IBM, we've seen this firsthand. But by embedding AI across HR, IT, and procurement processes, we've reduced costs by millions, slash repetitive tasks, and freed thousands of hours for strategic work Now we helping companies get smarter by putting AI where it actually pays off deep in the work that moves the business Let create smarter business IBM When it comes to financing, can you say a little bit more about what changed to make the market more comfortable with this? Because this is the big story in markets, just how much AI is now being issued through the corporate bond market. The equity market, as we know, is basically all big tech at the moment. What changed on the part of investors? Was it just pure return and performance or were there, I guess, efforts to make the contracts more robust or increase visibility into demand and that sort of thing? Seeing the inference aspect of it really emboldened investors. But that was really just January or maybe late Q4, where you started seeing there's just massive inflection of demand driven by inference. For us, right, it's tough for me to speak about other companies, but for us, why have we been underwritten at such scale and at a decreasing cost of capital I think it goes back to that track record of execution right It just the market has watched us execute and watch us deliver on these contracts And the way tell me if I going into too much detail here but the way that we finance our business, you kind of break it into two broad buckets, right? You have parent co-financing and asset co-financing. And asset co-financing is where all of the GPUs get financed, right? It's where all of our client contracts sit. And we can take these financings and put them into SPDs, or we'll just call it a box, so to say. And you... Sorry, I'm sorry. Keep going. Mentioning boxes is dangerous. Boxes on lots of connotations, but keep going. We put them into SPDs. And these these STVs, they have the infrastructure, they have the data center costs and they have the the debt agreements within them. And so you're able to pair this like five year take or pay contract to an amortization schedule on the debt. And you have the revenue come into the box, pay down the amortization schedule, pay down the operating costs of the data center. and it still contributes a, it has a 25% cost.
[40:00] contribution margin of profit up to the parent code, right? Like these are highly profitable agreements down in the SPV stack. And so you take that SPV out to the credit market and say, look at this instrument. It's a discrete set of contracts with counterparties, like entities who want to consume GPU compute. You have the data centers within it, et cetera. and you know one of the latest ones we did was uh as we call ddtl4 this was a investment grade rated first of its class no one had done this before for gpu financing non-recourse hpc infrastructure financing and got done it so for plus 225 like that is a phenomenal cost of capital for us and importantly we were able to bring in uh the insurance charge of capital which is a massive Chantra Capital out there that is looking to do allocations into the space. So we're kind of continuously making progress through these different stacks of capital, unlocking access to more and more types of investors. It's why you've seen us move into the convertible note market, into the unsecured market as well, along with taking direct strategic equity investments But for us it really important for the entire investor space to understand this business because this business largely didn't exist before, right? Like people weren't making loans into the hyperscalers to go credit these build-outs, right? It's on core weave, honestly, to be building this path into how do you finance the AI hyperscaler effectively. And I think we've just done a terrific job of it over the past few years. You used to be in a prior lifetime, a trader, right? Yes, I was a commodity trader. So I'm curious, like, you know, there's a lot of interest in and I don't know if it's going to materialize in GPU capacity trading and there's going to be a new contract. We recently interviewed the CEO of Compute Exchange and they're very close to having something listed on the CME. from your perspective, because I don't have a view on this yet. On the one, like you see like, okay, a big AI company does a five-year contract. As you say, the duration is lengthening. We're going to lock this in. I don know like what the need is for tradable compute in that environment et cetera What your guess Like do you anticipate that there will be a sufficient ecology of hedgers and speculators such that there will be a liquid market for tradable compute i think it a it a very much a timeline question that's out there short term no let me offer why no short term and then I'd say maybe in the long term. And it all comes back to fungibility, right? If you think about gold, gold is defined by its chemical composition, right? And there's no question of what is gold and not gold, et cetera. Compute really isn't, right? And especially GPU compute. GPU compute today is not fungible. And I think that this is well understood by our client base, by our suppliers, by third-party consultants like Sini Analysis. And it's this idea that an H100 deployed in one cloud doesn't have the same performance of an H100 deployed in another cloud. And the metrics that people use are things like good put or model flop utilization, MFUs, and there are these measurements of how much more performant is the exact same GPU by the versus another gpu deployed in another facility and so in order for something to be commoditized it has to be fungible right otherwise there just too much you know murkiness and there isn't like uh an exact data plan in there can i push on that a little bit further so yeah i mean i think that seems like a reasonable view is the non-fungibility related to configuration of like how they literally like the configuration of the GPUs within physically? Like, what is it? Is it about power? I mean, I think they all have like, you know, there are plenty of places that will say, you know, we have nine nines or however many nines you need in your industry or whatever. What is it in your view that would cause significant changes in the performance of an H100 in one cloud versus another? It could be in some part configuration, We build everything to DGX reference spec, which is the most, outlined by NVIDIA, it's the most performant way to build, operate, and deliver GPUs. But the rest of it, honestly, is just how you operate the GPUs. And that is the core weave software stack that is...
[45:00] keep these GPUs online, right? Like what happens if a GPU fails? Can you predict if a GPU is about to fail and swap in other infrastructure so that the client doesn't have downtime on that component? And there's an immense suite of software solutions and infrastructure management solutions that we have built to have the best good put, to have the best MFUs in the industry. And none of that is off the shelf, right? And so I wouldn't say it comes down to the strict components. That's kind of like a bare minimum starting point, right? Like you have to start in DJX reference spec, but where's differentiation come from there? I mean, that's the core product you're describing right there. By the way, Tracy, I'm just looking up terms of art. Good put measures the fraction of peak hardware performance that the training job can extract. This is according to Google in MFUs, model flops utilization, hardware metric for evaluating real world efficiency of LLM training. So two new terms. I actually hadn't heard of MFUs or Goodput before this. So I just learned two new terms today. We got to create a glossary. A glossary. Yeah, we do. Brandon when Joe asked you that question about compute markets earlier you said it was a timeline question which in my mind implies that it inevitable Like it just a question of how long it takes But then when you describe the fungibility problem it seems like this is an actual issue that will be very difficult to solve Yes, I think that characterization is absolutely correct, right? Like if you just take general commodity theory, and I traded natural gas, electricity, agriculture products for over a decade, like it suggests that it should become that at some point. But what is the reality today? The reality is this stuff isn't getting easier to operate, right? We've moved from these kind of relatively simple 42U air-cold racks of Hopper to these immensely complex Blackwell deployments moving into Vera Rubin following that. It's not getting easier to build, operate, provision, deliver these GPUs. It's getting more difficult. And I think until it starts becoming easier, you don't really have a path to commoditization. You will have to continue to prioritize working with the world and world operators of infrastructure That where we sit First of all this is helpful And I like that we getting multiple perspectives because I do think this is gonna be like one of the big questions for financial markets because let's say if they took off, then you could imagine that might even improve financing conditions because then the lender can hedge against. Yeah. So like there would probably be some good things for the industry if this took off. So I appreciate it's good to have your perspective on this. Why is it, you know, I'm an inference user, by the way. So I made a little machine learning model in one of my hobby projects. And I provide inference over at havelock.ai, or I'm a user of inference or whatever. I have a model, whatever. Why is it that I... It'd be impressive if you were providing inference. I'm trying to, I guess I'm a consumer of inference. I use a, anyway, why is it that I'm actually very easily able to get, now not a huge allocation of like GPU access? So I was like how do I train this model It a model called BERT that Google released in 2018 or 2019 I fine tuned it for my purposes And then literally using Cloud Code I was able to in 10 minutes sign up I started using this company called Modal and I was able to start training a model I was surprised that there was like and it didn cost me very much and I have like no volume. But nonetheless, evidently, there was a little GPU capacity out there that I could get, and it cost me like $5 or something for the whole thing. Given what you always hear about like a utilization is slammed, why is it actually not that hard to find GPU capacity for someone like myself? You know, I think it's the scale difference right there. Finding ones or tens of GPUs, I think that's way more accessible out there. Okay. Our clients are focused on the hundreds of thousands of GPUs. I'm not there yet, but I'm not there yet. Not yet. I'm sure you'll get there. Yes. And that's where it kind of decommoditizes itself with scale as well, right? Like as you're in the hundreds of thousands component, there's just not that many deployments. It's handfuls of deployments at that size. But getting access to ones of GPUs, I think that there is a lot more ability to go secure that sizing in the market. So Joe and I are heading to Hong Kong very soon, and I expect that AI in China is going to be a big topic of conversation.
[50:00] How would you characterize, I guess, the difference between the U.S. and the Chinese market at the moment? I'm sure this is something you think about, even though you don't participate in the Chinese market directly. Yeah, that's like... Tracy's asking for questions. Yeah, that's basically it. It's like questions that we can ask people when we're over there. Yeah, that's likely going to be my response, Tracy, is like we just do not participate in that market. I think that there's opportunity for us to be expanding. As you guys know, we operate in Canada, Europe. I think moving further east makes a lot of sense for us, but we're trying to be very methodical in the way that we expand. So unfortunately, I'm not going to be able to help you with specific questions in that market. But I would imagine you're going to encounter a lot of the same things that you're seeing in the U.S., which is just insatiable, unrelenting demand for AI. And like, we just kind of keep coming back to this. It's like, there is no solution in sight for being able to satiate demand, right? There's just too many supply chain. There's no path to solving demand in the near term or even the medium term frankly You mentioned so Tracy asked you about land use You said that really was an issue But like the first time we talked to you in 2023 or whenever that was there was not a major growing movement of people who are just like anti centers in America Maybe there were a few fringe people, but it was not something that was on the minds of politicians and activists and so forth. And you do see these headlines, you know, about some projects really having been shelved. It was like a big one. Northern Virginia is a huge hotspot for it. And there was a big project that was they pulled the plug on due to some they couldn't get an agreement with the local government. That must affect you. What are you seeing in terms of like your capacity to build? How has it changed specifically in light of or have you seen a change? Would you be able to build faster in a world where this had never become a political hot button issue? I believe it has become that hot button issue. It's something that we're quite proactive about in market. And I think you just kind of go through the checks on the diligence process to make sure you're going through it correctly. I think that there's misconceptions out there. like water usage. Yeah, setting aside the misconception, like setting aside I know setting aside the whole debate about but just in terms of like operationally what it changed for you in terms of your plan No I would say our greatest challenge is still just getting that delivery of our like the construction and all the components and getting everything in there Like that is truly more of the bottleneck that's in the market today. Brandon, thank you so much for coming back on OddLots. We'll have you back next month for another market. No, or at least, or maybe in three years. Not three years. Yeah, not three years, but really. That's an eternity. Yeah, I know. Thank you so much. Thanks, guys. Appreciate it. I'm very excited about whether compute features will take off. I think that's an exciting story. You know, it's not the biggest story in the world, but it is actually a very exciting story. I've said this before. Even if you're not that interested in AI, this is a really interesting market structure story, right? It's basically the creation of a brand new market and poses all these interesting philosophical questions about how you do that. And I thought Brandon point about fungibility I mean that is a real issue And it does seem like it a challenging one to fix at the moment I don't know if it's inevitable in the future, but who knows? No, no. I mean, it makes a lot of sense. This was also Lewis Hart's point that it's like, you know, it's in the word commodity, right? If it's not a commodity, you're not going to get a commodity market for it. And of course, a number of entities are betting that it will be commoditized. But if the if it's true that, like, you know, they're getting more difficult to work, that the technical demands on the inference provider on the data center company are getting greater in order to get the maximum juice. Yeah. Then maybe it doesn't become commoditized. But I think that's like a fascinating question. like if you do see those efficiency improvements and new designs and things like that, you could imagine that like the demand is there for a standardized GPU as well. So I don't know, like I'm really torn. It feels like it could go either way. Well, and even in his answer, he talked about how they configure their own GPUs to a spec largely that NVIDIA itself has come up with. So in theory, like there is aspect
[55:00] that everyone can match to. So that's like a really interesting, that's a really interesting question. I also really want to do more on all of these. So Google has TPUs, Amazon has Tranium, Microsoft has its own hardware, maybe even Jane Streets and the Hudson River tradings will have their own hardware. If they're not like, I want to understand better why, right? Because like, they presumably have some reason and they at least like the Microsoft will say, well, this will run better on our customized hardware. I want to understand why that would be, how much difference in performance is there? And then the degree to which demand materializes from users for non-NVIDIA silicon is like a really big question. Yeah. Why custom chips? Yeah. And what can you get out of that if you align model and chip to optimally work together I have no idea but I feel like it an episode I would like to do Yeah we should All right Shall we leave it there in the meantime Let leave it there All right This has been another episode of the All Thoughts Podcast I Tracy Allaway You can follow me at Tracy Allaway And I'm Joe Weisenthal. You can follow me at The Stalwart. Follow our guest, Brandon McBee at Brandon McBee. Follow our producers, Carmen Rodriguez at Carmen Armand, Dashiell Bennett at Dashbot, Kale Brooks at Kale Brooks, and Kevin Lozano at Kevin Lloyd Lozano. And for more Odd Lots content, go to Bloomberg.com slash Odd Lots, or the daily newsletter and all of our episodes. And you can chat about all these topics 24-7 in our Discord, discord.gg slash oddlots. And if you enjoy Oddlots, if you want us to do an episode on custom chips, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad-free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening. Thank you Music Music Thank you. and sixty degree guarantee you can be for imprint certain your order will show up just right right on time explore more at for imprint dot com for imprint for certain when you're running a business the best days are the ones where priorities stay on track for midsize and large companies risk can affect multiple parts of the organization at once from property and liability to cyber and regulatory challenges at that level managing risk becomes an ongoing discipline We see you next time insurance company and its property and casualty affiliates, Hartford, Connecticut. These days, it seems like AI agents are just about everywhere you turn, every field and every function. But without identity, you can't trust they'll serve your business instead of jeopardizing it. Fortunately, Okta helps you get identity right by securing your AI agent's identities, giving you a single layer of control, a single standard of trust. So whether an AI agent supports a single user or your entire enterprise, with Okta, you'll turn risk into opportunity. Secure every agent. Secure any agent. Okta secures AI.